MongoDB shows the way

Specialmoves Labs

Written by Gavin Clark,

MongoDB is an open-source document-oriented datastore. It’s one of the darlings of the NOSQL world, and with good reason, using it on our 24 Days project really helped getting started very quickly - run mongod.exe and you are good to go. There’s no schema, set up is effortless and there are drivers for every major language and platform. As you probably can tell I quite like it.

Download Mongo and follow along on in the shell if you have the time - the data I used for this write-up is in a script here. Some instructions for running it. If you don’t know Mongo there is a great tutorial shell - http://try.mongodb.org/ to get started.

Geo-spatial indexing

There are plenty of great features in MongoDB; auto-sharding, map/reduce, GridFS support...but one that really caught my eye is geo-spatial indexing. A lot of the applications we create have location data, and it’s provided with nearly every API; tweeting, checking-in, reviewing - web applications love to know where you are.

MongoDB has an index that will allow you to make blazingly fast location based queries - that are easier to maintain and hack with.

In comparison to the standard techniques for location based queries in MySQL (and WKT), geo-spatial indexing is a breeze:

Get some JSON data with a latitude and longitude - loc: [-0.11006, 51.52603]
Create a spatial index - ensureIndex({ loc: "2d"})
Run a beautifully simple query - find({ loc: { $near: [-0.11023, 51.52551], $maxDistance: 0.0145 } })

No “sine” of the Haversine formula there. A few points to bear in mind:

Notice we’ve got lng, lat here. The order is important as Mongo follows the GeoJSON spec for spherical queries - so get used to ordering all your data this way
$maxDistance here is in degrees as we’re dealing in lng and lat. A mile in degrees is very roughly 1/69 = 0.0145
In case you’re wondering, -0.11023, 51.52551 is our office in clerkenwell and -0.11006, 51.52603 is our local the Wilmington Arms

Now to the real world

It’s very simple to extend this query as well, to include a tag like ‘real ale’ within 1 mile - geo-spatial indexes can be combined with secondary indexes, for example a tags MultiKey (another great MongoDB feature is MultiKeys - indexing on values in in array).


        office = [-0.11023, 51.52551];
max = 1/69;
db.things.ensureIndex({ loc: "2d", tags: 1 });
r = db.things.find({ loc: { $near: office, $maxDistance: max }, tags: "real ale" })

Perhaps you need to find a drinking hole in within the boundaries of Exmouth Market. That’s easy too with bounds queries. You can find in a $box, $sphere or a super-flexibile $polygon - the polygon will close join the last point to the first.


        exm = [[-0.11034, 51.52555], [-0.10851, 51.52647], [-0.10802, 51.52625], [-0.10986, 51.52535]]
db.things.find({ loc: { "$within": { "$polygon": exm } }, tags: "real ale" })

There’s also the geoNear command (though the others are preferred) which will handily return the distance from the position given and also sort the results for you. “things” is the collection here, the command will know to look through the geo spatial index you’ve created on its own.


        db.runCommand({ geoNear: "things", near: office, num: 10, maxDistance:max/2 });

The Earth isn’t flat

The examples above assume a flat earth - there’s a good explanation here. This is fine for smaller distances and the most common use cases (the closest beer gardens to the office). If you’re measuring over greater distances (> 30mi, or close to the poles), you’re probably going to want to take the spherical nature of our wonderful planet into account - unless you’re reading this from Flatland or building an application like Word².

Fortunately this is really quite simple with Mongo: adding “Sphere” to your query name (e.g. nearSphere) and if you’re using the geoNear command add “spherical:true” to the options. Be aware that:

wrapping across the poles or lng flipping from 180deg to -180deg isn’t supported - an error will be thrown to warn you
distances (when using the geoNear command or maxDistance) are in radians - seems inconvenient but it allows any size sphere (like an mmporg world perhaps?)
you must query in [lng, lat] order

Editing the query above to look over 200km for a beer garden:


        // max distance in km / earth's radius in km gives us radians
max = 200 / 6378;
db.things.find({ loc: { $nearSphere: office, $maxDistance: max }, tags: "beer garden" })

What’s next?

We’ll be using this in some R&D projects soon - and can see lots of applications in the future. In the meantime; have a look at the official documentation, use MongoDB for everything and share your results.

---

Gavin is Head of Flash at Specialmoves, but he’s not an idiot* @villaaston

^{*this is a Specialmoves in-joke which Gavin will surely be happy to explain if you ask him}