Serving a website from Amazon S3

Amazon S3 can be used to serve static content, with all the advantages of scaling and redundancy that you get from Amazon.

There are comprehensive guides to setting up a site in S3 – this is a shortened version to highlight a couple of things.

In my case, I wanted to set up a website to serve some static content at a subdomain of an existing domain which was being managed in Route53.

First, set up a bucket which will hold the website. Note, if you’re planning to use an alias for the domain, then the bucket needs to be named as a subdomain of one that you’re managing in Route53 – eg. static.julianhigman.com.

Next, set up the bucket to be a website, using the properties pane to select “Enable website hosting”. Give it an index filename of index.html (remember, S3 isn’t really serving content the same way as say Apache, which has directory listing built in, so you need to give it a default file to serve at the root URL).

bucket-properties

Now, you need to allow anonymous visitors to read the contents of the bucket. The simplest way is to add a bucket policy. In the Permissions section of the properties, click the “Edit bucket policy” button, and add this (but replacing “static.julianhigman.com” with the name of your bucket):

{
    "Version": "2008-10-17",
    "Statement": [
        {
            "Sid": "PublicReadGetObject",
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::static.julianhigman.com/*"
        }
    ]
}

You should now be able to access your bucket in a web browser, at a URL like http://static.julianhigman.com.s3-website-eu-west-1.amazonaws.com (again, replace static.julianhigman.com with your bucket).

You can create folders in the bucket, and upload content. Folders can have their own index.html file, which will be served by default if the URL path doesn’t include a filename. (And remember, objects in S3 actually all live in the bucket, the hierarchy of folders is being faked by S3 based on the path that you specify for an object when storing or retrieving it).

Finally, set up the alias for your subdomain in Route53 – create a new Record Set for the subdomain, specifying an “A” record, but selecting the “Alias” radio button, and choosing your new S3 website from the Alias Target dropdown (the available S3 websites should appear at the top of the list).

Save the records, and you’re done.

You should now be able serve static content from your S3 bucket.

XDebug, PHP and Sublime Text 2 on Ubuntu 12.04

Getting Xdebug for PHP working with Sublime Text 2 is slightly tricky.

Most of the instructions are at https://github.com/Kindari/SublimeXdebug - these notes are just a bit extra as a reminder to myself.

After installing Xdebug with “sudo apt-get install php5-xdebug”, you have to add some xdebug  settings in /etc/php5/conf.d/xdebug.ini:

zend_extension = /usr/lib/php5/20090626+lfs/xdebug.so
xdebug.remote_enable=On
xdebug.remote_host="localhost"
xdebug.remote_port=9000
xdebug.remote_handler="dbgp"
xdebug.remote_autostart=1

In ST2, you use SHIFT+F8 to get the Xdebug menu, and select “Start debugging”. In your browser session, add a url parameter like this:

?XDEBUG_SESSION_START=sublime.xdebug

The page request should hang while the debugger has control – if nothing’s happening in ST2, you’ll need to check the console window by using ”CTRL`”.

You may get this output:

Traceback (most recent call last):
File "./sublime_plugin.py", line 362, in run_
File "./Xdebug.py", line 553, in run
File "./Xdebug.py", line 97, in read
File ".\xml\dom\minidom.py", line 1927, in parseString
File ".\xml\dom\expatbuilder.py", line 32, in <module>
File ".\xml\parsers\expat.py", line 4, in <module>
ImportError: No module named pyexpat

In which case you’ve got the problem with Python versions – 2.7 is the default on Ubuntu 12.04, and ST2 and the Xdebug plugin are using Python 2.6.

So follow the instructions at the bottom of the README at https://github.com/Kindari/SublimeXdebug

On Ubuntu 12.04, Python 2.6 isn't available, so here's what worked for me:

    * Download python2.6 files from <a href="http://packages.ubuntu.com/lucid/python2.6">Ubuntu Archives</a>
    * Extract the files: dpkg-deb -x python2.6_2.6.5-1ubuntu6_i386.deb python2.6
    * Copy the extracted usr/lib/python2.6 folder to {Sublime Text directory}/lib

(I actually copied the usr/lib/python2.6 directory to /usr/lib/python2.6, and symlinked to there from {Sublime Text directory}/lib)

Then restart ST2 and try again. Once you hit a breakpoint, it should look like this, with the Xdebug Context and Xdebug Stack windows showing content:

Why two machines are still essential for web development (hint: stupid IE6)

Here’s a modern web development setup:

The machine on the left runs:

a web server, multiple databases, Eclipse IDE, Sublime Text 2, test suites, source code control, Chrome, Firefox, terminal sessions, Skype, IRC, Spotify..

The machine on the right runs:

IE6

 

..because IE6 just WILL NOT DIE, EVER.

And every time you release some web software (assuming you want it to be used within one of the  benighted organisations that still use IE6 – local government, big-but-slow companies who are paralysed by fear and cost) you’ll have to test every bit of every page all over again JUST ON IE6.

AND – the rest of us are all paying for it, in wasted development effort, longer development timescales, increased bug-fix cycles, and lost hair.

Rant ends.

 

Running SauceLabs Selenium test suite locally with PHPUnit

SauceLabs provide a great hosted Selenium service for cross-browser testing in the cloud, which allows you to run a Selenium test suite against multiple browsers using the SauceLabs API and SauceConnect.

They also provide integration with PHPUnit, so that you can plug your SauceLabs Selenium tests directly into your PHPUnit test suite.

Sometimes, though, you want to run the same Selenium test suite against a single browser on your local machine, before you run the full thing against all the browsers on SauceLabs.

However, the SauceLabs PHPUnit integration requires you to extend their special base class, which they’ve added to their custom PHPUnit library (which you need to have installed instead of the standard PHPUnit). That base class assumes that the test suite is always going to run against the remote SauceLabs API.

To run a Selenium test suite locally, you need to be extending the standard PHPUnit Selenium base class instead of the custom SauceLabs one – but you don’t want to duplicate all your tests in two class hierarchies.

How can you have a class where the parent class can be one of two different things, depending on some config setting?

The answer is by using  PHPs “class_alias” and a custom autoloader (note, “class_alias” is only available in PHP from 5.3.0 onwards). It works something like this..

Instead of extending either the special SauceLabs base class or the PHPUnit selenium base class directly, you define two different intermediate base classes – called RemoteSeleniumTestCase and LocalSeleniumTestCase – which each extend one of the base classes:

// RemoteSeleniumTestCase.php
class RemoteSeleniumTestCase extends PHPUnit_Extensions_SeleniumTestCase_SauceOnDemandTestCase 
{
    function setUp() {..}    // special setup for SauceLabs 
}
// LocalSeleniumTestCase.php
class LocalSeleniumTestCase extends PHPUnit_Extensions_SeleniumTestCase
{
    function setUp() {..}    // standard setup for local Selenium
}

The your actual test suite extends a (virtual) class – called DynamicSeleniumTestCase:

// your selenium test cases
class HomeTest extends DynamicSeleniumTestCase
{
    function testHomePage()
    {
        // check home page is there
        $this->open('/');
        $this->assertTextPresent('Welcome to BigCo');
   } 
}

So, where does DynamicSeleniumTestCase come from?

That’s where class_alias and the autoloader come in – you define a custom autoloader, which will alias DynamicSeleniumTestCase to one or other of your intermediate base classes, based on the value of an environment variable (I’ve used RUN_SELENIUM_REMOTE):

spl_autoload_register(function($class) {
    if (strcasecmp($class, 'DynamicSeleniumTestCase') === 0) {
        $remote = getenv('RUN_SELENIUM_REMOTE');
        if ($remote) {
            require_once 'RemoteSeleniumTestCase.php';
            class_alias('RemoteSeleniumTestCase', 'DynamicSeleniumTestCase');
        } else {
            require_once 'LocalSeleniumTestCase.php';
            class_alias('LocalSeleniumTestCase', 'DynamicSeleniumTestCase');
        }
    }
}, true, true);

Then, to make your test suite run against your local Selenium setup and local browser, just do

export RUN_SELENIUM_REMOTE=0

before firing off the build, and to run the same thing against the SauceLabs server, do

export RUN_SELENIUM_REMOTE=1

Genius or insanity?! You decide.

 

Upgrading PEAR if Guzzle install fails

I had a problem with the PEAR installer with the Guzzle package (a generally excellent PHP HTTP framework).

A channel discover would give the following error:

root@ip-10-56-47-23:~# pear channel-discover guzzlephp.org/pear
 Discovering channel guzzlephp.org/pear over http:// failed with message: channel-add: Cannot open "http://guzzlephp.org/pear/channel.xml" (File http://guzzlephp.org:80/pear/channel.xml not valid (received: HTTP/1.1 404 Not Found
 ))
 Trying to discover channel guzzlephp.org/pear over https:// instead
 Discovery of channel "guzzlephp.org/pear" failed (channel-add: Cannot open "https://guzzlephp.org/pear/channel.xml" (Connection to `guzzlephp.org:443' failed: Connection refused))

Strangely, I could fetch the channel.xml file manually and add the channel. But even then, the install would fail (with similar messages about 404 errors).

The solution seemed to be upgrading PEAR – I was running version 1.9.0 (check it with “pear -V”), so I upgraded it:

root@ip-10-56-47-23:~# pear upgrade pear

after which I had version 1.9.4. Then I tried the channel discover again, with much better results:

root@ip-10-56-47-23:~# pear channel-discover guzzlephp.org/pearAdding Channel "guzzlephp.org/pear" succeeded
 Discovery of channel "guzzlephp.org/pear" succeeded

 

Using Jenkins to run remote deployment scripts over SSH

We use Jenkins to deploy code to multiple servers, so that we can manage builds and deployments from the same (even better if you’re using the Jenkins IRC plugin).

The deployment is done by a parameterized build job, where the parameter is the version of the project that we want to deploy. The job will run remote commands over ssh on servers that you’ve defined in the Jenkins configuration. Those commands will pull down a version of our code, unpack it, and run the rest of the install steps.

 

First you’ll need to install the Publish over SSH Plugin, which will allow files to be transferred to your servers and remote commands to be run.

Set up the SSH key for remote access of your target servers, in the Manage Jenkins page:

and setup the definitions for each of the servers that you want to deploy to:

Then in the configuration for the new deployment job you’ve set up, you’ll use the “Send files or execute commands over SSH before the build starts” settings in the “Build Environment” section to remotely execute a script to carry out the install steps on each remote server:

Notice that the build parameter “$version” is available to the Exec command that gets remotely executed – other Jenkins environment variables will also be available (e.g. $BUILD_NUMBER, $JOB_NAME etc).

Use the “Add Server” button to add more target servers, with the same Exec command.

Now you can deploy your project (or run any other remote scripts) by running the build job and specifying a version number.

 

Building github branches with Jenkins

We usually work on several parallel branches of a repo on github, and we wanted to be able to build and test any branch on demand.

So we set up a parameterised job in Jenkins that will take the name of a branch and run the build process.

As for all github builds, you need to have installed the git plugin first (https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin) and set up your github globals in the Jenkins settings:

 

 

 

Then set up a parameterized build job with the repo as the GitHub project and with “branch” as the parameter to be specified:

and in the Source Code Management section, add the parameter to the “Branches to build”:

Don’t specify any build triggers – you’ll probably just want to run this on-demand against specific branches, rather than every time there’s a push to the repo (which is what happens by default).

Now you can build any branch just by giving the branch name as the required parameter when the job is started.

 

 

 

Importing a project into github

We’re moving our development projects into github, so we wanted to get a copy of the code (checked out of Subversion) into a new project in github.

Here are the steps (assumes that you’ve set up your ssh keys correctly, according to e.g. http://help.github.com/linux-set-up-git/)

Setup the git globals:

$ git config --global user.name "Julian Higman"
$ git config --global user.email "jh @ kasabi.com"

Create the new project, add a README file, and connect it to the github remote project:

$ mkdir my-new-project
$ cd my-new-project/
$ git init
$ touch README
$ git add README
$ git commit -m "First commit"
$ git remote add origin git@github.com:kasabi/my-new-project.git
$ git push -u origin master

Now copy in the project source files, excluding svn files and other project config files:

$ cd ..
$ rsync -r --exclude=.svn --exclude=.buildpath --exclude=.project --exclude=.git my-old-project/ my-new-project/

Then add the new files, commit them, and push to the remote repo:

$ cd my-new-project/
$ git add -A
$ git status
$ git commit -m "added project files"
$ git push

 

 

Using Drush Make while ftp.drupal.org is down

The Drush Make utility downloads modules from ftp.drupal.org during the build – but the ftp.drupal.org site is currently down, which means Drush Make won’t work (and so our continuous integration has ground to a halt).

There’s a fix for general Drush usage here: http://contrib.linalis.com/content/continuer-dutiliser-drush-quand-ftpdrupalorg-est-down

That fix changes the download url to use drupal.org (which is working) instead of ftp.drupal.org (which isn’t).

Unfortunately Drush Make uses it’s own download functions, so this doesn’t help.. Here’s my (very similar) hack to get the Drush Make downloads working..

Change line 367 in drush_make.drush.inc to the following:

'url'  => str_replace('ftp.', '', $release['file']),

(The file should be somewhere like /usr/share/drush/commands/drush_make)

And don’t forget to remove the hack when ftp.drupal.org comes back..