Testing Javascript with QUnit, PhantomJS and JSCover

Javascript is often the part of an application that's most difficult to test.

There are several pieces that fit together to give a framework for running the tests in an automated build system - QUnit, PhantomJS and JSCover.

QUnit

QUnit works at the unit test level, allowing tests to invoke methods and make assertions about the results. The methods under test can also manipulate DOM fragments, and test the results.

QUnit is packaged as a set of Javascript files that you include in your test page.

A very simple test looks something like this:

<script>
  test('datahub.generateLabel()', function() {
    label = datahub.generateLabel('http://example.com/TheBigTown');
    equal(label, 'The Big Town', "should get expected label from slug - got " + label);
  });
</script>

And the output looks like this:

The QUnit tests run in the browser, as shown - so you normally have to load the test page manually.

PhantomJS

Loading a webpage in a browser is no good for automated testing, which is where
PhantomJS comes in. It's a headless webkit, which allows the QUnit tests to be run as a command line task rather than in the browser.

To run the QUnit tests, you invoke the PhantomJS with a Javascript utility called run-qunit.js that evaluates the page as if it were in a browser:

phantomjs run-qunit.js file://`pwd`/test.html

If any tests fail, you'll get an error exit code, which you'll need for automated builds.

There are some prerequisites for running PhantomJS on some platforms (looking at you, Centos 5) - it needs the extra package fontconfig installed:

yum install fontconfig

Download the latest version of PhantomJS from the PhantomJS downloads page, unpack it, and make sure it's command-line accessible by symlinking from the /usr/bin directory to the exe, e.g.:

ln -s /opt/phantomjs/bin/phantomjs /usr/bin/phantomjs

(this is for Ubuntu, and assumes that you've unpacked the downloaded archive into /opt/phantomjs)

JSCover

The final piece is JSCover, a tool that show you how much of your Javascript code is actually being exercised when your test suite runs.

JSCover works by instrumenting the Javascript and recording which lines are called during the tests. To do this, it needs to run a minimal server that serves the QUnit page that contains the tests.

Download the JSCover files and unpack them to somewhere like /opt/JSCover.

Start the server process with:

java -jar /opt/JSCover/target/dist/JSCover-all.jar -ws --document-root=. --port=8081

Then you can load the QUnit test page (e.g. "test.html"):

http://localhost:8081/test.html

JSCover will load the page, instrument the Javascript, excecute the tests, and write out a coverage report.

The web report looks like this:

Putting it all together

To run the whole thing as part of a build, you need a build target that will start the JSCover server, execute the QUnit test suite using PhantomJS, write the coverage report to a specified directory, and shutdown the JSCover server.

Note, running with PhantomJS AND JSCover needs a slightly modified Javascript "glue" utility called run-jscover-qunit instead of run-qunit.js

In Ant, the targets might look something like this:

<target name="jstest-start">
     <java jar="/opt/JSCover/target/dist/JSCover-all.jar" fork="true" spawn="true">
        <arg value="-ws"/>
        <arg value="--report-dir=coverage"/>
        <arg value="--document-root=."/>
        <arg value="--no-instrument=src"/>
        <arg value="--no-instrument=test/javascript/qunit/"/>
        <arg value="--no-instrument=wwwroot/js/jquery/"/>
        <arg value="--port=8081"/>
    </java>
    <waitfor maxwait="5" maxwaitunit="second" checkevery="250" checkeveryunit="millisecond" timeoutproperty="failed">
        <http url="http://localhost:8081/jscoverage.html"/>
    </waitfor>
    <fail if="failed"/>
</target>

<target name="jstest-run">
    <exec dir="." executable="phantomjs" failonerror="true">
        <arg line="run-jscover-qunit.js http://localhost:8081/test.html"/>
    </exec>
</target>

<target name="jstest-stop">
    <get src="http://localhost:8081/stop" dest="stop.txt" />
</target>

<target name="jstest" description="Run javascript tests">
    <antcall target="jstest-start"/>
    <antcall target="jstest-run"/>
    <antcall target="jstest-stop"/>
</target>

The coverage report should appear in the "coverage" directory (unless errors have occurred, in which case the build exits with an error, as expected).