Updated: Zero-touch deployment with Aegir and Jenkins

This is an update to a previous post

Just a quick recap for those unfamiliar: I've been looking for a "zero-touch" development-staging-live workflow for Drupal, that would allow me to quickly roll out new Drupal platforms, and even migrate existing sites to those new platforms. I store all of my platform make files in a single Git repo, with branches like "live_platform1" and "dev_platform1". Ideally, I could commit a change to a makefile, and immediately my Aegir server would build a new platform based on that changed makefile and even migrate sites on the existing platform to the new platform.

mig5 has an excellent post describing a zero-touch workflow for Drupal- however, I needed to modify it a little to suit my needs. Note that this currently can build platforms, but not migrate sites.

First, I created a Jenkins project called "aegir_platforms", and added a string parameter "BRANCH". I entered the URL of my Git repo, and used "$BRANCH" as the branch identifier. Finally, I entered the following shell command:

/usr/local/bin/deployment.sh example.com ${BRANCH} dummyprofile dummywebserver dummydbserver ${WORKSPACE}/build.make ${BRANCH}_${BUILD_NUMBER} ${BUILD_NUMBER}

Of course, I needed to modify the deploy.sh and fabfile.py to take into account the new parameter, since I like to name my jobs something nicer than just a date string: deployment.sh


#!/bin/bash
#
# Wrapper script for our fabfile, to be called from Jenkins
#

# Where our fabfile is
FABFILE=/usr/local/bin/fabfile.py

HOST=$1
SITE=$2
PROFILE=$3
WEBSERVER=$4
DBSERVER=$5
MAKEFILE=$6
NAME=$7
BUILD_NUMBER=$8
DATE=`date +%Y%m%d%H%M%S`

if [[ -z $HOST ]] || [[ -z $SITE ]] || [[ -z $PROFILE ]] || [[ -z $WEBSERVER ]] || [[ -z $DBSERVER ]] || [[ -z $MAKEFILE ]] || [[ -z $NAME ]] || [[ -z $BUILD_N$
then
  echo "Missing args! Exiting"
  exit 1
fi


# Array of tasks - these are actually functions in the fabfile, as an array here for the sake of abstraction
if [ $BUILD_NUMBER -eq "1" ]; then
  # This is a first-time ever build. Let's install a site instead of migrate it
  TASKS=(
   build_platform
   save_alias
   install_site
  )
else
  TASKS=(
    build_platform
    migrate_site
    save_alias
    import_site
  )
fi
# Loop over each 'task' and call it as a function via the fabfile,
# with some extra arguments which are sent to this shell script by Jenkins
for task in ${TASKS[@]}; do
  fab -f $FABFILE -H $HOST $task:site=$SITE,profile=$PROFILE,webserver=$WEBSERVER,dbserver=$DBSERVER,makefile=$MAKEFILE,build=$NAME || exit 1
done

fabfile.py


from fabric.api import *
import time

env.user = 'aegir'
env.shell = '/bin/bash -c'
env.key_filename = '/path/to/aegir/key'

# Download and import a platform using Drush Make
def build_platform(site, profile, webserver, dbserver, makefile, build):
  print "===> Building the platform..."
  run("drush make %s /var/aegir/platforms/%s" % (makefile, build))
  run("drush --root='/var/aegir/platforms/%s' provision-save '@platform_%s' --context_type='platform'" % (build, build))
  run("drush @hostmaster hosting-import '@platform_%s'" % build)
  run("drush @hostmaster hosting-dispatch")

# Install a site on a platform, and kick off an import of the site
def install_site(site, profile, webserver, dbserver, makefile, build):
  print "===> Installing the site for the first time..."
  # run("drush @%s provision-install" % site)
  # run("drush @hostmaster hosting-task @platform_%s verify" % build)
  # time.sleep(5)
  # run("drush @hostmaster hosting-dispatch")
  # time.sleep(5)
  # run("drush @hostmaster hosting-task @%s verify" % site)

# Migrate a site to a new platform
def migrate_site(site, profile, webserver, dbserver, makefile, build):
  print "===> Migrating the site to the new platform"
  # run("drush @%s provision-migrate '@platform_%s'" % (site, build))

# Save the Drush alias to reflect the new platform
def save_alias(site, profile, webserver, dbserver, makefile, build):
  print "===> Updating the Drush alias for this site"
  # run("drush provision-save @%s --context_type=site --uri=%s --platform=@platform_%s --server=@server_%s --db_server=@server_%s --profile=%s --client_name=ad$

# Import a site into the frontend, so that Aegir learns the site is now on the new platform
def import_site(site, profile, webserver, dbserver, makefile, build):
  print "===> Refreshing the frontend to reflect the site under the new platform"
  # run("drush @hostmaster hosting-import @%s" % site)
  # run("drush @hostmaster hosting-task @platform_%s verify" % build)
  # run("drush @hostmaster hosting-import @%s" % site)
  # run("drush @hostmaster hosting-task @%s verify" % site)

Note that I also had to create an SSH key to allow the Jenkins user to log in as the Aegir user, and specify this key in fabfile.py using the env.key_filename parameter.

Then, I just needed to set up a Git post-receive hook to notify Jenkins of changes to a branch (platform):


while read oldrev newrev ref
do
        branch=${ref##*/}
        job="aegir_platforms"
        echo "COMMAND: wget -q --delete-after http://example.com/jenkins/job/$job/buildWithParameters?BRANCH=$branch"
        `wget -q --delete-after http://example.com/jenkins/job/$job/buildWithParameters?BRANCH=$branch`
done

So the overall workflow now is that I make a change to a platform make file (build.make), and when I push the new/updated make file to my Git repo, it tells Jenkins to build a new platform based on the branch that was just updated.

Comments