Continuous deployment with Jenkins and Salt

LinuxCon CloudOpen ContainerCon

North America 2015, Seattle
Anirban Saha
About me
Techie…
Traveller…
Thinker…

Author of ‘Salt Cookbook’

Deployment
Should be…

• Simple
• Stable
• Fast (good to have…)
• Reliable
• One click
Problems faced
• Code distribution
• Additional tasks (commands, migrations)
• Latency
• Parallel execution
• Batch deployments
• Ensuring uptime
Methods available
Deployment

Configuration management tools :

• Puppet, Chef, Ansible, Salt

Remote execution tools:

• SSH, Parallel SSH, Rundeck

Packages:

• Code packaged as RPM or DEB files

Code storage
• GIT server (Github, Gitlab, Bitbucket, etc.)

• Object storage (Amazon S3, Swift, etc.)

• Package repositories (YUM/Debian)

• Configuration management repositories

• File servers managed manually

Problems
Deployment:
• Remote execution tools :
• Time consuming (over SSH)
• Need to maintain inventory of hosts

Code storage:
• GIT repositories:
• Usually centrally located, latency when deployed from GIT
• Package repositories (RPM/DEB):
• Problems in simultaneously updating repo metadata

Rollback:
• Version manipulation is a problem with most configuration
management and remote execution tools
Salt
Provides…

• Agent based communication

• Fast execution
• Orchestration
• Batch execution
• Strong Integration with providers and
services
Salt features
Targeting deployment nodes :

• By hostname
• ‘myappweb*’

• By specific grains
• -G ‘node_type:webserver’

• By nodegroups
• -N webgroup
Salt features
Task dependencies :

• Salt requisites to make tasks dependent on each other

get_archive:
module.run:
.
.
deploy_app:
module.run:
.
.
- require:
- module: get_archive
Salt features
Orchestration :

webgrp2_deploy:
salt.state:
- tgt: ' webgrp2'
- tgt_type: nodegroup
- sls:
- webserver.deploy
- require:
- salt: webgrp1_deploy

webgrp1_deploy:
salt.state:
- tgt: 'webgrp1'
- tgt_type: nodegroup
- sls:
- webserver.deploy
Salt features
Batch Execution :

• By number of hosts
• --batch-size 5
• Executes on 5 hosts at a time

• By percentage of hosts
• --batch-size 25%
• Executes on 25% of total target
hosts at a time
Salt features
Salt API:
# curl -H "Accept: application/json"
-d secretkey= "mysupersecretkey"
-k https://10.0.0.2:8080/hook/deploy

{"success": true}
Tools
To be used…

• GIT for code repository

• Jenkins for CI
• Ant for build and tasks
• Amazon S3 for code archive storage
• Salt for deployment
• Shell scripts (good old bash) for
additional tasks
Salt deployment methods

• Salt hosted on the Jenkins server.

Jenkins calls Salt binary

• Salt hosted independently.

Jenkins calls Salt via SSH

• Jenkins calls Salt via Salt API

Objective
 Steps
• salt-cloud used to launch instances

• Post install actions used to synchronize EC2 grains

• Salt reactor used to run states on new instances ,

perform deployment and register with load balancer

• Jenkins build job used to build new code, create tags

and push new deployment ready archive to Amazon
S3

• Jenkins deploy job used to fetch new code version

and deploy code on target servers
 Steps explained
• Salt cloud is used to spawn instances
• Following parameter is provided in the profile to push custom EC2 grains,
• sync_after_install: grains

• The node is registered and deregistered from the load balancer with the
following module definition,

register:
module.run:
- name: boto_elb.register_instances
- m_name: mywebapp
- instances:
- {{ grains['ec2']['instance_id'] }}

• Here ['ec2']['instance_id'] is one of the custom grains pushed to the node

Steps explained
Fetching archive:

Salt s3 execution module used to get code archive from

Amazon S3

fetch_app_archive:
module.run:
- name: s3.get
- bucket: mywebapp-us
- path: mywebapp-{{ app_version }}.tar
- local_file: /tmp/mywebapp-{{ app_version }}.tar
Steps explained
Setting App version grain:

After every deploy, a grain is set to record the

app version deployed for tracking,

app_version:
grains.present:
- value: {{ app_version }}
- require:
- cmd: deploy_app
Steps explained
Deployment based on App version:

At every deploy, it is checked if the version to

be deployed is already on the node using the
app_version grain,

{% if grains['app_version'] != app_version %}
deregister:
fetch_app_archive:
deploy_app:
register:
{% endif %}
Lets do it then !!!
Demo repository
Get the demo repository at the
following location,

https://github.com/rosesnthornz/
cloudopen-na-2015
Questions ?
Contact
Email :
sahaanirban1988@gmail.com

Twitter : @rosesnthornz
Thank You !!!