Updated: Zero-touch deployment with Aegir and Jenkins

This is an update to a previous post

Just a quick recap for those unfamiliar: I've been looking for a "zero-touch" development-staging-live workflow for Drupal, that would allow me to quickly roll out new Drupal platforms, and even migrate existing sites to those new platforms. I store all of my platform make files in a single Git repo, with branches like "live_platform1" and "dev_platform1". Ideally, I could commit a change to a makefile, and immediately my Aegir server would build a new platform based on that changed makefile and even migrate sites on the existing platform to the new platform.

mig5 has an excellent post describing a zero-touch workflow for Drupal- however, I needed to modify it a little to suit my needs. Note that this currently can build platforms, but not migrate sites.

First, I created a Jenkins project called "aegir_platforms", and added a string parameter "BRANCH". I entered the URL of my Git repo, and used "$BRANCH" as the branch identifier. Finally, I entered the following shell command:

/usr/local/bin/deployment.sh example.com ${BRANCH} dummyprofile dummywebserver dummydbserver ${WORKSPACE}/build.make ${BRANCH}_${BUILD_NUMBER} ${BUILD_NUMBER}

Of course, I needed to modify the deploy.sh and fabfile.py to take into account the new parameter, since I like to name my jobs something nicer than just a date string: deployment.sh #!/bin/bash # # Wrapper script for our fabfile, to be called from Jenkins # # Where our fabfile is FABFILE=/usr/local/bin/fabfile.py HOST=$1 SITE=$2 PROFILE=$3 WEBSERVER=$4 DBSERVER=$5 MAKEFILE=$6 NAME=$7 BUILD_NUMBER=$8 DATE=`date +%Y%m%d%H%M%S` if [[ -z $HOST ]] || [[ -z $SITE ]] || [[ -z $PROFILE ]] || [[ -z $WEBSERVER ]] || [[ -z $DBSERVER ]] || [[ -z $MAKEFILE ]] || [[ -z $NAME ]] || [[ -z $BUILD_N$ then echo "Missing args! Exiting" exit 1 fi # Array of tasks - these are actually functions in the fabfile, as an array here for the sake of abstraction if [ $BUILD_NUMBER -eq "1" ]; then # This is a first-time ever build. Let's install a site instead of migrate it TASKS=( build_platform save_alias install_site ) else TASKS=( build_platform migrate_site save_alias import_site ) fi # Loop over each 'task' and call it as a function via the fabfile, # with some extra arguments which are sent to this shell script by Jenkins for task in ${TASKS[@]}; do fab -f $FABFILE -H $HOST $task:site=$SITE,profile=$PROFILE,webserver=$WEBSERVER,dbserver=$DBSERVER,makefile=$MAKEFILE,build=$NAME || exit 1 done fabfile.py from fabric.api import * import time env.user = 'aegir' env.shell = '/bin/bash -c' env.key_filename = '/path/to/aegir/key' # Download and import a platform using Drush Make def build_platform(site, profile, webserver, dbserver, makefile, build): print "===> Building the platform..." run("drush make %s /var/aegir/platforms/%s" % (makefile, build)) run("drush --root='/var/aegir/platforms/%s' provision-save '@platform_%s' --context_type='platform'" % (build, build)) run("drush @hostmaster hosting-import '@platform_%s'" % build) run("drush @hostmaster hosting-dispatch") # Install a site on a platform, and kick off an import of the site def install_site(site, profile, webserver, dbserver, makefile, build): print "===> Installing the site for the first time..." # run("drush @%s provision-install" % site) # run("drush @hostmaster hosting-task @platform_%s verify" % build) # time.sleep(5) # run("drush @hostmaster hosting-dispatch") # time.sleep(5) # run("drush @hostmaster hosting-task @%s verify" % site) # Migrate a site to a new platform def migrate_site(site, profile, webserver, dbserver, makefile, build): print "===> Migrating the site to the new platform" # run("drush @%s provision-migrate '@platform_%s'" % (site, build)) # Save the Drush alias to reflect the new platform def save_alias(site, profile, webserver, dbserver, makefile, build): print "===> Updating the Drush alias for this site" # run("drush provision-save @%s --context_type=site --uri=%s --platform=@platform_%s --server=@server_%s --db_server=@server_%s --profile=%s --client_name=ad$ # Import a site into the frontend, so that Aegir learns the site is now on the new platform def import_site(site, profile, webserver, dbserver, makefile, build): print "===> Refreshing the frontend to reflect the site under the new platform" # run("drush @hostmaster hosting-import @%s" % site) # run("drush @hostmaster hosting-task @platform_%s verify" % build) # run("drush @hostmaster hosting-import @%s" % site) # run("drush @hostmaster hosting-task @%s verify" % site)

Note that I also had to create an SSH key to allow the Jenkins user to log in as the Aegir user, and specify this key in fabfile.py using the env.key_filename parameter.

Then, I just needed to set up a Git post-receive hook to notify Jenkins of changes to a branch (platform):

while read oldrev newrev ref do branch=${ref##*/} job="aegir_platforms" echo "COMMAND: wget -q --delete-after http://example.com/jenkins/job/$job/buildWithParameters?BRANCH=$branch" `wget -q --delete-after http://example.com/jenkins/job/$job/buildWithParameters?BRANCH=$branch` done

So the overall workflow now is that I make a change to a platform make file (build.make), and when I push the new/updated make file to my Git repo, it tells Jenkins to build a new platform based on the branch that was just updated.