Archive for January 2007

CVS for MSG nearly there!

We’ve now got a CVS server to put all the various bits of code for MSG (including the MSG-Moodle integration & work we’re doing on maps) and make available for people to download and use for themselves. We’re still in the process of putting the code up and getting it structured in the best way, and we’ve still got a fair bit of documentation to get written up before we provide the details.

I’ve created a developer section on the MSG website (see:, which is where we’ll put up the documentation and details for how to access the CVS server.

As well as getting this arranged, I’ve been working on moving the MSG maps, API and clustering algorithm over to run as a servlet. Currently it’s just built on top of Moodle, but it makes more sense for all the maps and API to sit within the main MSG (BuddyXML) server for those users who aren’t necessarily using Moodle.

Developing an API for MSG

Had some discussions yesterday with the FlashMeeting guys as to how we could integrate MSG and FlashMeeting in some way, the upshot was that we’re going to look at developing an API for MSG so that presence info can be embedded in other web applications – simply stick a few references to JavaScript files in your application, add the tags for who and where you’d like presence info displayed and then the page will automatically display presences for those users (and keep them up to date).

In fact this morning I’ve been working on a little demo of how this could work, so I’ve got a plain html page running on my desktop PC which is calling the relevant JavaScript files on my development Moodle server – and all seems to be working well, it automatically logs you in to MSG (if you aren’t already). Next step is some more generalisation of the code and getting it moved over to the actual MSG server.

The main issue that we’ll have is with authentication, but I think we’ve got a method for coping with this. Essentially the MSG server needs to know how to authenticate the user, before the user can be logged in (and their contacts presences displayed). So, suppose I have Website X where I would like to display presences and I needed to have logged in to this website (so I’ve got a cookie). When I call the JavaScript initialisation function for MSG, I pass it the authentication type (in this case ‘WebsiteX’) and an array of authentication parameters (which would include my cookie). JavaScript calls the connect service in BuddyXML (on MSG server) providing these authentication details. BuddyXML then knows how to authenticate the cookie that’s passed (i.e. in this case to call a validate service running on Website X) and can automatically log the user in to MSG, on the basis that we trust Website X.

Improving the clustering

I’ve spent most of this week virtually rewriting the javascript for MSG on Google maps and also spending time improving the clustering algorithm.

The reason for updating the javascript was to ensure that only markers were loaded as and when they were actually needed for display – my previous version was retrieving all the data for the markers whether or not they were currently visible on the map. I’ve been using chapter 7 of a Google Maps Applications with PHP and Ajax bookto help me get this sorted out and it’s turned out to be very helpful. Having implemented this it REALLY speeds up the application and makes if run far more smoothly – even for a relatively low number (approx 70) markers.

I slightly changed the exact functions in the book so that when the user moved around the map any existing markers were left in place – only the new ones were added in – rather than clearing all and redrawing. The reason for doing this was that if a marker appeared close to the top of the map and then you clicked to open the HTML info window, the map would auotmatically scroll down to be able to fit the info window within the map area – using the functions from the book this would lead to the marker being removed and rewritten (hence also closing the info window), but keeping the marker in place and just adding any new markers seems to get around this. However I’m not sure how my maps would cope if a user spent a long time moving around a single zoom level of the map as the number of markers stored would gradually build up. Fortunately the markers are completely refreshed when the user changes zoom level, so hopefully this build up will not be a big issue.

I’m also in the process of updating the clustering algorithm that I was using for the various zoom levels. The main problem I was envisaging with the algorithm I’d implemented was simply the execution time in updating the clusters – for only 100 points it could take a loooonnnng time to run in PHP. So I started to look at using a grid system instead – in which the earth is divided into grid squares and whichever markers were in a particular square these would be clustered at the squares’ centre. This runs much faster than my original algorithm since it doesn’t need to iterate to calculate distances between points. However, I did find several drawbacks using this grid system, firstly, that when displayed on the map the points looked “too regular” and some appeared in the sea etc, secondly a grid square could cut a ‘natural cluster’ into 2 (or more) markers with half the points being allocated to the marker in the centre of one square and half going to another.

I’ve not been too bothered about this second drawback – as the markers we’re using are only meant to be approximations and I think they are good enough for what we need. But, I really didn’t like markers appearing in the sea and being too regimented on managscreen – so I’ve ed to fake a way around this…. rather than putting the markers at a point in the centre of the square, I’ve just selected (randomly) one of the actual marker points in the grid square and given all other points in this grid square the same coordinates. This has several advantages… 1) the map ‘looks’ better (!), 2) all markers will be at a ‘real’ position – so no-one will appear to be located in the middle of the North Sea, 3) for grid squares which only contain a single point, the marker will always be at exactly the correct position no matter what the zoom level is.

Still got a bit of finishing off to do, but very nearly there 😉

New Year New Maps!

Have progressed well over the last few days (and the few days before Christmas) getting the Google Maps fixed up. I now have a clustering algorithm implemented (admittedly not a very sophisticated one, but it does the job) so we don’t end up with loads of overlapping markers when the user has zoomed out (see below):

Without clustering:

With clustering:

This appears to work quite well with around 100 or so different markers on a map (not yet tested for much higher numbers of markers). The ‘cluster’ that each marker belongs to for a given zoom level is cached in a database, this is to save having to recluster the markers on the fly – which will get really resource intensive for large numbers of markers. We should also be able to use more efficient clustering algorithms without having to change any of the map interface code.

Few more things that I’m working on now…

  • Using different marker icons depending on the number of users in a cluster (larger icon = more users)
  • Moving the maps so they become part of the MSG client rather than part of Moodle (as they currently are) – this shouldn’t be a massive job, but will involve moving services currently provided by MSG block in Moodle to be from MSG & Wildfire servers
  • Looking at how to implement custom maps within Google Maps API

That’s probably enough to be getting on with for now 😉