All APIs are Web APIs Everything Else is Bullshit

17 Jan 2013

The world is waking up to the idea of APIs. This is a good thing.

Scraping Planning Applications

15 Jan 2013

In response to a question posted on the ScraperWiki mailing list, I have written a scraper to extract planning applications for the Hammersmith & Fulham planning application site. This is very much a non-scraper friendly site, I had to make use of quite a few little tricks to extract the data.

Argue for SEO if You Can't Argue for Open Data

14 Jan 2013

This post is for any one who cares about open data and works in a company web team.

Three Essentials for a Modern Web Publishing Architecture

20 Dec 2012

This past year I’ve been thinking an awful lot about web publishing platforms. The landscape of the web is changing rapidly. We are going through an enormous shift in the way people access online information, the devices they use and how they consume content.

How To Create an Interactive With Google Docs Data

28 Sep 2012

We all know Google Spreadsheets. They’re easy to use, easy to share, easy to work with. They’re also great to manage the data to power an interactive visualisation.

Clearing 2012 Interactive Visualisation

18 Sep 2012

Over the past few weeks I’ve been building an interactive to explore clearing this year.

UCAS Clearing Data 2008 to 2012

17 Sep 2012

I’ve recently built an interactive visualisation comparing clearing 2012 to 2011.

UK Uni Twitter Data API

31 Jul 2012

At the start of the month I began collecting data about UK university twitter accounts. I wanted to see how the friends and followers of these accounts changes over time. Not just the absolute quantities, but the make up of the demographics themselves.

Join Data Files on the Command Line

13 Jul 2012

In my previous post, I drew up a list of my top 5 shell commands for working with data files. One of the (many) commands that did not make it on to the list was join. This command is only ever useful when working across multiple data files, which is why I didn’t include it before. However, when you do need it, it can be very powerful.

Top 5 Shell Commands for Working with Data Files

09 Jul 2012

There is a glut of tools out there for cleaning up and exploring data sets. But time and time again I find myself coming back to the good old command line. Here are my top 5 shell commands for working with data files. If you have your own favourites, please do drop a comment below.