Alex Little

I updated the map grids so they are divided equally, and all appears to be working in that the markers divided and combined in an expected way. However I did find there were quite a few little bugs etc with the reclustering algorithm which were all related to floating points and the number of decimal places.

The problems appear to be due to rounding errors when I’m storing the coordinates in the database. For example, when I think I’m storing say “51.2″, actually “51.2000000000001″ gets stored in the database, so then when I look up all the points matching 51.2 the database doesn’t return anything!

I have found a way around this, though I’m not sure how much of a hack it is! The solution seemed to be to treat the coordinates as strings when adding them to the prepared statement for updating and querying on. So instead of using:

String sql ="SELECT * FROM mytable WHERE lat = ?";
Connection conn = conn.getConnection();
PreparedStatement pstmt = conn.prepareStatement(sql);
pstmt.setDouble(1,51.2);

I’m using:

String sql ="SELECT * FROM mytable WHERE lat = ?";
Connection conn = conn.getConnection();
PreparedStatement pstmt = conn.prepareStatement(sql);
DecimalFormat f = new DecimalFormat("#.000000000000");
pstmt.setString(1,f.format(51.2));

This all appears to be working for storing/retrieving the correct values/records – note that the values are still stored as datatype decimal(18,15). My only concern is that I’ve only tested it against a MySQL database and I’m not sure whether other rdbmss would accept decimals/floats passed in string format.

Another possible solution that was suggested to me was to multiply up the coordinates so they are stored as integers in the database – but as this would be a fairly significant change at this stage (and may introduce more bugs!!), I’ll stick with the solution I’ve implemented for now

The maps seem to be running much more smoothly now I have tidied up the JavaScript quite a lot and tried to make it more efficient, it seems to run much better now (with around 400 users) than it was previously, when it could fell very sluggish. With only 10s of users it is far quicker than before. Using the firebug plugin to narrow down the bottlenecks really helped. I’ve also added the number of user who are on/off line for each group – as you get with the normal MSG client.

Have now run into another problem with the clustering though. We noticed that sometimes you could get (on a wide view) two different markers displayed, then when you zoom in one level these combine into one, zoom in yet further and they go back to being 2 different points. The reason (I think) is to do with the size of the grids not being exact multiples of each other, so on a wide view the 2 points could be in different grids (hence the 2 markers), then on the next zoom in they’re actually in the same grid square – hence just the one marker. This looks a bit odd as you would imagine the the makers to only divide out as you zoom in (only combining when you zoom out). So the solution will be to have the grids for each zoom levels exact multiples of each other. So I’m looking into this now

I’ve spent most of this week virtually rewriting the javascript for MSG on Google maps and also spending time improving the clustering algorithm.

The reason for updating the javascript was to ensure that only markers were loaded as and when they were actually needed for display – my previous version was retrieving all the data for the markers whether or not they were currently visible on the map. I’ve been using chapter 7 of a Google Maps Applications with PHP and Ajax bookto help me get this sorted out and it’s turned out to be very helpful. Having implemented this it REALLY speeds up the application and makes if run far more smoothly – even for a relatively low number (approx 70) markers.

I slightly changed the exact functions in the book so that when the user moved around the map any existing markers were left in place – only the new ones were added in – rather than clearing all and redrawing. The reason for doing this was that if a marker appeared close to the top of the map and then you clicked to open the HTML info window, the map would auotmatically scroll down to be able to fit the info window within the map area – using the functions from the book this would lead to the marker being removed and rewritten (hence also closing the info window), but keeping the marker in place and just adding any new markers seems to get around this. However I’m not sure how my maps would cope if a user spent a long time moving around a single zoom level of the map as the number of markers stored would gradually build up. Fortunately the markers are completely refreshed when the user changes zoom level, so hopefully this build up will not be a big issue.

I’m also in the process of updating the clustering algorithm that I was using for the various zoom levels. The main problem I was envisaging with the algorithm I’d implemented was simply the execution time in updating the clusters – for only 100 points it could take a loooonnnng time to run in PHP. So I started to look at using a grid system instead – in which the earth is divided into grid squares and whichever markers were in a particular square these would be clustered at the squares’ centre. This runs much faster than my original algorithm since it doesn’t need to iterate to calculate distances between points. However, I did find several drawbacks using this grid system, firstly, that when displayed on the map the points looked “too regular” and some appeared in the sea etc, secondly a grid square could cut a ‘natural cluster’ into 2 (or more) markers with half the points being allocated to the marker in the centre of one square and half going to another.

I’ve not been too bothered about this second drawback – as the markers we’re using are only meant to be approximations and I think they are good enough for what we need. But, I really didn’t like markers appearing in the sea and being too regimented on managscreen – so I’ve ed to fake a way around this…. rather than putting the markers at a point in the centre of the square, I’ve just selected (randomly) one of the actual marker points in the grid square and given all other points in this grid square the same coordinates. This has several advantages… 1) the map ‘looks’ better (!), 2) all markers will be at a ‘real’ position – so no-one will appear to be located in the middle of the North Sea, 3) for grid squares which only contain a single point, the marker will always be at exactly the correct position no matter what the zoom level is.

Still got a bit of finishing off to do, but very nearly there

Have progressed well over the last few days (and the few days before Christmas) getting the Google Maps fixed up. I now have a clustering algorithm implemented (admittedly not a very sophisticated one, but it does the job) so we don’t end up with loads of overlapping markers when the user has zoomed out (see below):

Without clustering:

With clustering:

This appears to work quite well with around 100 or so different markers on a map (not yet tested for much higher numbers of markers). The ‘cluster’ that each marker belongs to for a given zoom level is cached in a database, this is to save having to recluster the markers on the fly – which will get really resource intensive for large numbers of markers. We should also be able to use more efficient clustering algorithms without having to change any of the map interface code.

Few more things that I’m working on now…

• Using different marker icons depending on the number of users in a cluster (larger icon = more users)
• Moving the maps so they become part of the MSG client rather than part of Moodle (as they currently are) – this shouldn’t be a massive job, but will involve moving services currently provided by MSG block in Moodle to be from MSG & Wildfire servers
• Looking at how to implement custom maps within Google Maps API

That’s probably enough to be getting on with for now