Using Jenkins to run remote deployment scripts over SSH

We use Jenkins to deploy code to multiple servers, so that we can manage builds and deployments from the same (even better if you’re using the Jenkins IRC plugin).

The deployment is done by a parameterized build job, where the parameter is the version of the project that we want to deploy. The job will run remote commands over ssh on servers that you’ve defined in the Jenkins configuration. Those commands will pull down a version of our code, unpack it, and run the rest of the install steps.


First you’ll need to install the Publish over SSH Plugin, which will allow files to be transferred to your servers and remote commands to be run.

Set up the SSH key for remote access of your target servers, in the Manage Jenkins page:

and setup the definitions for each of the servers that you want to deploy to:

Then in the configuration for the new deployment job you’ve set up, you’ll use the “Send files or execute commands over SSH before the build starts” settings in the “Build Environment” section to remotely execute a script to carry out the install steps on each remote server:

Notice that the build parameter “$version” is available to the Exec command that gets remotely executed – other Jenkins environment variables will also be available (e.g. $BUILD_NUMBER, $JOB_NAME etc).

Use the “Add Server” button to add more target servers, with the same Exec command.

Now you can deploy your project (or run any other remote scripts) by running the build job and specifying a version number.


Building github branches with Jenkins

We usually work on several parallel branches of a repo on github, and we wanted to be able to build and test any branch on demand.

So we set up a parameterised job in Jenkins that will take the name of a branch and run the build process.

As for all github builds, you need to have installed the git plugin first ( and set up your github globals in the Jenkins settings:




Then set up a parameterized build job with the repo as the GitHub project and with “branch” as the parameter to be specified:

and in the Source Code Management section, add the parameter to the “Branches to build”:

Don’t specify any build triggers – you’ll probably just want to run this on-demand against specific branches, rather than every time there’s a push to the repo (which is what happens by default).

Now you can build any branch just by giving the branch name as the required parameter when the job is started.




Importing a project into github

We’re moving our development projects into github, so we wanted to get a copy of the code (checked out of Subversion) into a new project in github.

Here are the steps (assumes that you’ve set up your ssh keys correctly, according to e.g.

Setup the git globals:

$ git config --global "Julian Higman"
$ git config --global "jh @"

Create the new project, add a README file, and connect it to the github remote project:

$ mkdir my-new-project
$ cd my-new-project/
$ git init
$ touch README
$ git add README
$ git commit -m "First commit"
$ git remote add origin
$ git push -u origin master

Now copy in the project source files, excluding svn files and other project config files:

$ cd ..
$ rsync -r --exclude=.svn --exclude=.buildpath --exclude=.project --exclude=.git my-old-project/ my-new-project/

Then add the new files, commit them, and push to the remote repo:

$ cd my-new-project/
$ git add -A
$ git status
$ git commit -m "added project files"
$ git push



Using Drush Make while is down

The Drush Make utility downloads modules from during the build – but the site is currently down, which means Drush Make won’t work (and so our continuous integration has ground to a halt).

There’s a fix for general Drush usage here:

That fix changes the download url to use (which is working) instead of (which isn’t).

Unfortunately Drush Make uses it’s own download functions, so this doesn’t help.. Here’s my (very similar) hack to get the Drush Make downloads working..

Change line 367 in to the following:

'url'  => str_replace('ftp.', '', $release['file']),

(The file should be somewhere like /usr/share/drush/commands/drush_make)

And don’t forget to remove the hack when comes back..

Enable trackpad coasting in Ubuntu 10.04

I like the way trackpad edge-scrolling allows you to “coast” (start scrolling with the edge of the trackpad, then release it, and the scrolling continues until you tap the trackpad again).

It’s not enabled by default on Ubuntu 10.04, but here’s how to turn it on (put it in a startup script):

xinput set-prop –type=float “AlpsPS/2 ALPS DualPoint TouchPad” “Synaptics Coasting Speed” 1

PHP 5.3.0 and the “global” keyword

This had me stumped for a while – using some 3rd party code with PHP 5.3.0, the “global” keyword didn’t seem to work any more (references to the supposedly “global” variable from within functions always gave NULL)..

You have to declare the variable as “global” before setting it in the outer scope – then it works again.

So, this used to work (but doesn’t under PHP 5.3.0) :

$util = new Utility();
global $util;
function show() {
    global $util;
    echo "$util->version";
but if you swap the lines of the declaration of $utils, then it does work under PHP 5.3.0 :
global $util;
$util = new Utility();
function show() {
    global $util;
    echo "$util->version";

Installing Ruby Active Record on Ubuntu 10.04

I had to jump through a few hoops..

I tried the obvious “sudo gem install activerecord”, but it gave an error – it needs to install the i18n gem, but that needs rubygems version >= 1.3.6, and I had rubygems 1.3.5.

So I had to upgrade  rubygems first, which would normally be :

sudo gem update --system

but that reports that it’s been disabled on Debian, and directs you to use apt-get instead (which doesn’t have a better version). So I had to use the gem-updater gem:

sudo gem install rubygems-update
sudo update_rubygems
after which I had rubygems 1.3.7. Then I could get active record:
sudo gem install activerecord

I still needed to get the mysql gem installed, which in turn needed the libmysql-dev stuff installed

sudo apt-get install libmysqlclient15-dev
sudo gem install mysql

Ubuntu and CloudInit on Amazon EC2

Alestic and Canonical have released new Ubuntu AMIs for EC2 – the EBS version in the eu-west-1 region has AMI ID “ami-38bf954c”, and the source is “099720109477/ebs/ubuntu-images/ubuntu-lucid-10.04-i386-server-20100827”.

Ubuntu images support CloudInit, which runs scripts on startup to allow you to configure the server (set up ssh keys, update the repos etc). There’s a load of different ways to specify what should get run (see for full details).

One of the simplest is just to give it a script in the User Data, like this:

echo "Hello World.  The time is now $(date -R)!" | tee /root/output.txt

It runs as the root user, so you can do pretty much anything you want to configure the box.

Google Maps on Android downloads 60MB in 10 minutes

I was just having a look at my Vodafone account to see how I’ve managed to use 2GB of data this month (oops), and noticed these series of entries:

Tue Jun 08 17:39 INTERNET 2931.108 KB
Tue Jun 08 17:38 INTERNET 2930.525 KB
Tue Jun 08 17:38 INTERNET 2930.143 KB
Tue Jun 08 17:37 INTERNET 2930.299 KB
Tue Jun 08 17:36 INTERNET 2930.785 KB
Tue Jun 08 17:36 INTERNET 2930.895 KB
Tue Jun 08 17:35 INTERNET 2929.817 KB
Tue Jun 08 17:35 INTERNET 2930.283 KB
Tue Jun 08 17:35 INTERNET 2931.05 KB
Tue Jun 08 17:34 INTERNET 2930.991 KB
Tue Jun 08 17:34 INTERNET 2930.765 KB
Tue Jun 08 17:33 INTERNET 2930.441 KB
Tue Jun 08 17:33 INTERNET 2929.775 KB
Tue Jun 08 17:33 INTERNET 2929.608 KB
Tue Jun 08 17:32 INTERNET 5637.054 KB
Tue Jun 08 17:31 INTERNET 2930.137 KB
Tue Jun 08 17:31 INTERNET 2931.101 KB
Tue Jun 08 17:30 INTERNET 2930.31 KB
Tue Jun 08 17:30 INTERNET 2930.408 KB
Tue Jun 08 17:29 INTERNET 2930.107 KB

That’s about 60M in the space of 10 minutes. Which seems like a lot. Similar patterns appear at regular intervals..

Looking at the times of the entries, the data usage is due to Google Maps running on my HTC Hero, which I’m using for sat nav.

If that happens on, say, 10 days in the month, then that’s already 600M of data used. Hm.  (Add to that the Spotify usage, and 2GB starts to make sense..)

Anyone know if Google Maps can cache its data on Android?

Agile Coaching – new book from Rachel Davies

I’ve just been reading the excellent new book called Agile Coaching by Rachel Davies and Liz Sedley.

It’s the result of many years spent coaching teams to be agile, and the cumulative experience really shows.

Writing about agile processes often seems to get lost in an abstract discussion of the nature of processes – analysing the team as a Complex Adaptive System and so on. Rachel and Liz comprehensively avoid such mistakes, and keep their advice firmly rooted in practical examples from their own real-world knowledge.

So, at the start of the book, they describe a generic agile process (usually a blend of XP, Scrum, Lean etc), and then use that as the basis for examining in detail how the process really works and how to coach people in using it.

The emphasis is always on the pragmatic application of the process, illustrated with plenty of examples and stories, which makes it very easy to relate to the daily experience  of working with agile teams.  Each chapter ends with a checklist of bullet points to summarise and reinforce the key messages.

The first section of the book describes the basics of the coach’s role – how to get started, how to work with people and lead change, how to build a team, and when to move on.  Some of this would probably be just as relevant  to coaching any activity, but much of it is specifically about the issues in applying agile principles to software development.  In the chapters about working with other people there’s plenty of good advice – how to give feedback, resolve conflict and so on – that would make great reading for anyone who has to work with other people in ANY environment. I particularly like the bit about “Emotional Outbursts in Meetings”..!

The next section walks the reader through the iteration processes in more detail. There are chapters covering all the main elements of an iteration, including standups, stories and story cards, acceptance tests, estimating and planning,  and keeping progress visible. As usual, there are plenty of hints and tips for coping with problems – ideas for what to do when the team is dispersed, or work at different hours, or just hate planning.

The third section focuses on issues of quality. There’s lots of techie detail here that’ll be familiar to anyone who’s worked in an agile development team – discussion of unit testing, continuous integration, pair programming and other practices. But the key focus is on how all these practices work to the end goal of running, tested, maintainable software. And again, some pointers for techniques to try when things are difficult.

Finally, there are some chapters on listening to feedback – in terms of process, that includes demos for external feedback, and retrospectives for internal feedback. Retrospectives in particular can be quite difficult to get right, and there’s a range of techniques to try as well as a checklist of “Retrospective Smells” (like “History Lesson” and “Hot Air”) that can indicate that the retrospective isn’t working as well as it should.   The “Growing You” chapter gives some ideas for planning your own personal development.

Throughout the book, the tone that comes across is one of calm, intelligent guidance, and a very humane approach to dealing with everyone involved in an agile project (including yourself).

It’s also a great grab-bag of hints and tips, and reminders of key principles.

There are plenty of pointers scattered throughout the book to further reading, from personality types to Kanban to pair programming. And as for the stories and examples that illustrate the chapters, having worked with Rachel at Connextra, I recognise some of them, and can vouch for the fact that they’re genuine..

This is not just a book for coaches though – anyone who works in an agile
team will find lots here to help with understanding the process more deeply. For those just starting out with agile, it’s a great way to learn what it feels like to be in a properly functioning agile team. And for those who have been doing it for years, there are fresh ways to look at the process, and some inspirational advice for when times are rough.