Installing Ruby Active Record on Ubuntu 10.04

I had to jump through a few hoops..

I tried the obvious “sudo gem install activerecord”, but it gave an error – it needs to install the i18n gem, but that needs rubygems version >= 1.3.6, and I had rubygems 1.3.5.

So I had to upgrade  rubygems first, which would normally be :

sudo gem update --system

but that reports that it’s been disabled on Debian, and directs you to use apt-get instead (which doesn’t have a better version). So I had to use the gem-updater gem:

sudo gem install rubygems-update
sudo update_rubygems
after which I had rubygems 1.3.7. Then I could get active record:
sudo gem install activerecord

I still needed to get the mysql gem installed, which in turn needed the libmysql-dev stuff installed

sudo apt-get install libmysqlclient15-dev
sudo gem install mysql

Ubuntu and CloudInit on Amazon EC2

Alestic and Canonical have released new Ubuntu AMIs for EC2 – the EBS version in the eu-west-1 region has AMI ID “ami-38bf954c”, and the source is “099720109477/ebs/ubuntu-images/ubuntu-lucid-10.04-i386-server-20100827″.

Ubuntu images support CloudInit, which runs scripts on startup to allow you to configure the server (set up ssh keys, update the repos etc). There’s a load of different ways to specify what should get run (see https://help.ubuntu.com/community/CloudInit for full details).

One of the simplest is just to give it a script in the User Data, like this:

#!/bin/sh
echo "Hello World.  The time is now $(date -R)!" | tee /root/output.txt
EOF

It runs as the root user, so you can do pretty much anything you want to configure the box.

Google Maps on Android downloads 60MB in 10 minutes

I was just having a look at my Vodafone account to see how I’ve managed to use 2GB of data this month (oops), and noticed these series of entries:

Tue Jun 08 17:39 INTERNET 2931.108 KB
Tue Jun 08 17:38 INTERNET 2930.525 KB
Tue Jun 08 17:38 INTERNET 2930.143 KB
Tue Jun 08 17:37 INTERNET 2930.299 KB
Tue Jun 08 17:36 INTERNET 2930.785 KB
Tue Jun 08 17:36 INTERNET 2930.895 KB
Tue Jun 08 17:35 INTERNET 2929.817 KB
Tue Jun 08 17:35 INTERNET 2930.283 KB
Tue Jun 08 17:35 INTERNET 2931.05 KB
Tue Jun 08 17:34 INTERNET 2930.991 KB
Tue Jun 08 17:34 INTERNET 2930.765 KB
Tue Jun 08 17:33 INTERNET 2930.441 KB
Tue Jun 08 17:33 INTERNET 2929.775 KB
Tue Jun 08 17:33 INTERNET 2929.608 KB
Tue Jun 08 17:32 INTERNET 5637.054 KB
Tue Jun 08 17:31 INTERNET 2930.137 KB
Tue Jun 08 17:31 INTERNET 2931.101 KB
Tue Jun 08 17:30 INTERNET 2930.31 KB
Tue Jun 08 17:30 INTERNET 2930.408 KB
Tue Jun 08 17:29 INTERNET 2930.107 KB
...etc

That’s about 60M in the space of 10 minutes. Which seems like a lot. Similar patterns appear at regular intervals..

Looking at the times of the entries, the data usage is due to Google Maps running on my HTC Hero, which I’m using for sat nav.

If that happens on, say, 10 days in the month, then that’s already 600M of data used. Hm.  (Add to that the Spotify usage, and 2GB starts to make sense..)

Anyone know if Google Maps can cache its data on Android?

Agile Coaching – new book from Rachel Davies

I’ve just been reading the excellent new book called Agile Coaching by Rachel Davies and Liz Sedley.

It’s the result of many years spent coaching teams to be agile, and the cumulative experience really shows.

Writing about agile processes often seems to get lost in an abstract discussion of the nature of processes – analysing the team as a Complex Adaptive System and so on. Rachel and Liz comprehensively avoid such mistakes, and keep their advice firmly rooted in practical examples from their own real-world knowledge.

So, at the start of the book, they describe a generic agile process (usually a blend of XP, Scrum, Lean etc), and then use that as the basis for examining in detail how the process really works and how to coach people in using it.

The emphasis is always on the pragmatic application of the process, illustrated with plenty of examples and stories, which makes it very easy to relate to the daily experience  of working with agile teams.  Each chapter ends with a checklist of bullet points to summarise and reinforce the key messages.

The first section of the book describes the basics of the coach’s role – how to get started, how to work with people and lead change, how to build a team, and when to move on.  Some of this would probably be just as relevant  to coaching any activity, but much of it is specifically about the issues in applying agile principles to software development.  In the chapters about working with other people there’s plenty of good advice – how to give feedback, resolve conflict and so on – that would make great reading for anyone who has to work with other people in ANY environment. I particularly like the bit about “Emotional Outbursts in Meetings”..!

The next section walks the reader through the iteration processes in more detail. There are chapters covering all the main elements of an iteration, including standups, stories and story cards, acceptance tests, estimating and planning,  and keeping progress visible. As usual, there are plenty of hints and tips for coping with problems – ideas for what to do when the team is dispersed, or work at different hours, or just hate planning.

The third section focuses on issues of quality. There’s lots of techie detail here that’ll be familiar to anyone who’s worked in an agile development team – discussion of unit testing, continuous integration, pair programming and other practices. But the key focus is on how all these practices work to the end goal of running, tested, maintainable software. And again, some pointers for techniques to try when things are difficult.

Finally, there are some chapters on listening to feedback – in terms of process, that includes demos for external feedback, and retrospectives for internal feedback. Retrospectives in particular can be quite difficult to get right, and there’s a range of techniques to try as well as a checklist of “Retrospective Smells” (like “History Lesson” and “Hot Air”) that can indicate that the retrospective isn’t working as well as it should.   The ”Growing You” chapter gives some ideas for planning your own personal development.

Throughout the book, the tone that comes across is one of calm, intelligent guidance, and a very humane approach to dealing with everyone involved in an agile project (including yourself).

It’s also a great grab-bag of hints and tips, and reminders of key principles.

There are plenty of pointers scattered throughout the book to further reading, from personality types to Kanban to pair programming. And as for the stories and examples that illustrate the chapters, having worked with Rachel at Connextra, I recognise some of them, and can vouch for the fact that they’re genuine..

This is not just a book for coaches though – anyone who works in an agile
team will find lots here to help with understanding the process more deeply. For those just starting out with agile, it’s a great way to learn what it feels like to be in a properly functioning agile team. And for those who have been doing it for years, there are fresh ways to look at the process, and some inspirational advice for when times are rough.

Stefan Tilkov talks about REST

Old but good, this podcast  on se-radio.net with Stefan Tilkov talks around the ideas of RESTful applications – why building applications this way makes them part of the web, rather than just “on” the web, and why the REST style exploits the existing architecture of the web.

There’s also a great introduction to REST ideas at http://www.infoq.com/articles/rest-introduction.

I like the way Stefan characterises RESTful applications as being a specialisation of the REST principles – meaning that an application can provide the basic operations (allowing data to be read, caching observed, mime types honoured etc) but still having a level of functionality that can only be used by a client which understands the api more fully (so, POST operations that will create new domain objects, that require specific inputs, for example).

And there’s a nice write up of some of the frequently-heard objections to REST at http://www.infoq.com/articles/tilkov-rest-doubts, describing how you can achieve things like asynchronous operations. Many of these techniques are things that we’re currently using on projects at Talis.

Cultural Agoraphobia

Interesting to see that John Naugton was talking about open data and “cultural agoraphobia” in The Observer this weekend..

Talis, where I work, has just announced its Talis Connected Commons, which offers free storage of semantic data sets to anyone – as long as the data is open. Peter Murray-Rust, who is quoted in the Observer article,  is one of the people who will hopefully be making use of the Talis triple store.

And I’ve just been listening to Paul Miller’s recent podcast with Reuven Cohen about the Open Cloud Manifesto, which tries to create a coherent idea of what an “open cloud” might be (although that seems to be more concerned with interoperability and portability, rather than openness of data). That seems to have caused some controversy along the way, even making it into The Economist.  

It certainly feels like there’s a tipping point approaching for the next version of data on the web, but (as usual) the barriers are more cultural than technical.

Validate your inputs

Todays podcast listening for the commute was Bruce Sams talking about web app security from Software Engineering Radio.

Starting with a live demo of some hacking techniques (surprisingly effective even with just the audio), it covers some of the popular attacks – SQL injection, javascript in input fields, cookie stealing, guessing adjacent ID numbers and so on.

Apparently about 70% of web app vulnerabilities come from the inputs to the system – we spend a lot of time worrying about things like SSL and encrypted logins, but actually the vast majority of attacks use the applications themselves.

Sams says that when he’s asked for his top 10 tips for making your web app secure, he says:

  1. Validate your inputs properly
  2. See (1).

An interesting aspect of validation, though, is that it applies not just the obvious things like form fields and text strings, but to all the HTTP header elements as well.

For example, WordPress MU (in versions prior to 2.7) had a function that would echo the HTTP Host header without having sanitised it. The attacker can craft a request that contains some Javascript in the Host header which, when echoed, can grab cookies (or other evil cross-site scripting stuff)..

Resetting the oil service light on a 2006 BMW 320i

The oil service warning light on the BMW was still on after the last oil service, so I looked around for how to reset it. Lots of advice on the internet, but mainly involving special tools (aka paperclips) and dodgy shorting-out of pins in the diagnostic port.

There’s also talk of holding down the trip reset button whilst turning the ignition key to the first position, then releasing and pressing and holding the button again. But there IS no ignition key in the recent BMWs..

The answer turned out to be a bit more easily guessed (after a bit of trial and error). When you start the engine, the oil service warning shows on the dashboard, at which point you just press the BC button on the end of the indicator stalk (which is how you access most of the diagnostics anyway). The word “reset” appears, and another click and hold on the BC button makes it show “resetting” (and a little clock icon so you know it’s busy). And that’s it.

Put away the paperclip.

PhpUnit Mocks that suddenly stopped working

We came across a strange thing with the mock framework in PhpUnit .

We had some test code that created a mock for a Store class, and gave it a method to mock out:

        $mockStore = $this->getMock(‘Store’, array(‘save’));

That seemed to work fine in the tests, we could setup expectations, the correct return values got returned, and so on.

Then, in some unrelated code, we changed some require_once statements, and suddenly the test with the mocks stopped working – it threw exceptions trying to create the Store mock. 

It turns out that the constructor for the Store class needed some parameters, and the mock framework needs you to supply those parameters when you create the mock, because it will call the real constructor behind the scenes.

The mock had been working OK previously because the real Store class hadn’t been loaded anywhere by the test – but then the changes to some require_once statements elsewhere in the code meant that the Store class WAS now loaded, and instantiated by the call to create a mock.

 

So the mock framework in PhpUnit will create a mock for you even if it has no idea what class it is that you’re trying to mock out – you could do

        $mockStore = $this->getMock(‘foo’, array(‘save’));

and it would still give you a mock that worked fine. If it DOES find a class of that name, though, it’ll instantiate it.

In fact, this behaviour is probably a good thing for TDD – it means you can create the mocks you need before you’ve ever created the real classes. You just have to be aware that when you DO create the real class, the mocks will start being created based on the real thing.

 

The solution in our test was to use the flag that tells the mock framework not to call the original constructor – it’s a bit more clumsy, because you also have to supply some other additional parameters:

        $mockStore = $this->getMock(‘Store’, array(‘save’), array(), ”, FALSE);

JISC dev8D conference 2009

I spent a couple of days last week at the JISC dev8D “un-conference” (see http://code.google.com/p/developerhappinessdays/) and gave a talk on building semantic web applications. The slides are at http://www.slideshare.net/jhigman/semantic-web-applications.

Some of the highlights for me from the lightning talks were Georgi Kobilarov of the Freie Universität Berlin, and George Kroner from Blackboard.

Georgi Kobilarov gave a great description of Linked Open Data, why it matters, and how dbpedia make use of it – specifically, for powering the tagging vocabulary of BBC data so that cross-site connections can be automatically made. Nice to have real-world examples of the strength of LOD, and it points the way for more uses of the dbpedia data set.

George Kroner  talked about the IMS Learning Tools Interoperability APIs in the latest version of Blackboard. The APIs are also implemented by Sakai, so (in theory, at least) plugins for one will work in the other. I haven’t seen any examples of it in practice, but hopefully will soon. (More in-depth discussion at http://mfeldstein.com/connecting-blackboard-to-sakai-and-moodle/).

Thursday 12th included some discussion sessions about VLEs (Blackboard, Moodle, Sakai) and about using collaborative tools (Google Docs, VOIP, MediaWiki). Both were enlightening, though the fact that SOAS have moved their entire email and collaboration infrastructure to Google Docs caused some heated(-ish) debate about the dangers of concentrating even more data in the hands of Google. No firm conclusions were reached..!