Alex Little

My 'hello world' heatmap

Over that past few days I’ve been playing around with creating heatmaps to overlay onto OpenStreetMap – the image on the left is where I’ve got to so far. My interest in this is for creating heatmaps from data collected during my colleagues health sciences research in Ethiopia.

There are already several programs and services available to create heatmaps, for example gheat and OpenHeatMap. But none of these quite suited me, gheat because I didn’t want to create a full tile server – just an image to overlay on a particular area of the map and OpenHeatMap because I wanted to have access to the code to tweak how I wanted.

I then found heatmap.py which (using a similar algorithm to gheat) did almost exactly what I wanted. There were only a couple of changes that I needed to make:

1) Update to account for the Mercator projection – I wanted my overlay to be on a view of the whole world (zoom level 2 in OpenStreetMap). When I first ran the program the areas over northern Europe (and others) were almost but not quite inline with the marker overlays, but this was due to the projection. I just edited the input script to convert my lat/lng coordinates into Mercator coordinates. I also hooked up the heatmap.py script to read the lat/lng coordinated from a MySQL database.

2) Allow the script to have weights against each point. The current script looks at the number of points in a particular area (or on top of each other) to generate the ‘heat’. I also wanted to allow for points to have a weight – as mentioned here. I’ve not yet implemented the weighting, but I’m not anticipating this to be too tricky. GHeat (as far as I can tell) doesn’t allow for weights on points, but OpenHeatMap does (please let me know if I’m wrong about this).

All has been much easier than I’d expected, I’ve learnt a little more about python and once I’ve got the weighting working how I’d like, then I’ll share the code back.

Visiting a school near Wukro

Yesterday, I had a visit out to Wukro, to meet some the heads from the local health bureaus and visit some schools in the area. Two of the PhD students from Mekelle have projects in this area, looking at prevalence of parasites in children and how this may be reduced. We’re investigating using phones and GPS to collect the survey data, and have it automatically sent to a server (via GPRS) when the data collector comes into a mobile reception area. We’re also looking at how learning materials and/or diagnostic tools could be made available electronically (again using phones) to the Health Extension Workers.

We visited two schools, both a fairly short distance (10-15km) from the main Mekelle to Adigrat road, but both feeling quite remote. Both were primary schools with around 1000 pupils each – but neither had access to clean water and one had no electricity – although both did have mobile phone reception.

Getting connected to GPRS ought to be relatively straightforward, however the SIM card needs to be GPRS enabled first. It’s free to enable, but isn’t automatic. We went to the ETC office in Mekelle to ask them to do this, but for some reason currently it can’t be enabled for Tigray mobile numbers (0914xxxxxx), I then asked them to enable my Addis mobile number (0910xxxxxx), but was then told it can only be enabled in Addis. So we’re a little stuck, being unable to test the GPRS availability on any of the phones that we have between us. The good news is that SIM cards are now much cheaper – 85 birr each (with 15 birr credit included), compared to the 360+ birr I paid less than a couple of years ago.

The past month has flown by, feels as if I’ve only be back for 5 minutes. I’m generally very happy with the progress we’ve made on the project over the past year. Looking forward to coming back again in September again to see how everything has moved on again.

UK map generated on my OSM server

After using OpenStreetMap in a fairly limited way for the past year or so, essentially just to upload data/tracks from wandering around Ethiopia, I thought I’d have a closer look, especially as a possible replacement for Google Maps which I was using a lot whilst at the OU.

Firstly I looked at how I could use OSM and the OpenLayers API as an alternative for the map in my Online Users Map – which certainly looks feasible. So I’m now thinking about making this an option in the block settings, as shouldn’t be too much work to provide the option.

I also set up my own open street map server (instructions for installing on Ubuntu Lucid Lynx). Although quite a long process everything worked well first time with no errors. I didn’t download the entire 8Gb+ OSM dataset to generate my tiles, rather I just selected a much smaller area of detailed data (using extracts from GeoFabrik and CloudMade). This avoided the potential ’30 hours or longer’ process of importing the world into my database – the whole dataset for Ethiopia took less than 30 seconds to import.

The only part where I ran into a problem was at the very end when I wanted to generate all the tiles. When I ran ./generate_tiles.py I kept getting an error telling me that osm-local.xml didn’t exist. Fortunately this was quite an easy fix, I just needed to edit ~/bin/mapnik/generate_tiles.py to point to ~/bin/mapnik/osm.xml instead, then all ran fine (generating 55Mb+ of png images).

I could then create an alias in my Apache to point to my generated tiles and create a slippy map using OpenLayer and the tiles generated on my desktop.

I still have a lot to learn about all this, especially with what can be done with Mapnik for generating tiles (and the python script necessary to achieve it) and using OpenLayers as a replacement for Google Maps API – but all going well so far

After a long, long delay, I finally had chance to look at the ‘Bad Gateway’ problem that seemed to be stopping the Moodle Online Users Map block working. Whilst in Ethiopia my internet connection there wasn’t good enough to be able to spend time debugging this, but now I’m back on a more reliable connection I’ve been able to have a look.

It turned out to be a relatively simple issue to resolve, basically I wasn’t correctly constructing the call to the geonames website, so it gave the ‘bad gateway’ response.

If you are using (or trying to use) this block then please update to the latest version as soon as possible (go to: Moodle Modules and Plugins).

I’ve not yet had chance to test the changes when the Moodle server is behind a proxy server, I’ll try to get this tested soon. If you are using a proxy server I’d be very interested to know if it’s now working (or not).

[Update 20 Apr 12:30]: have just tested by going through a proxy server and appears to be working fine, but please let me know if it doesn’t work for you.

Turns out that the signing and dancing I mentioned yesterday is the Ashenda festival. It also goes on for several days, so I’m still getting stopped in the street and asked for money!

In an attempt to avoid getting stopped quite so often I thought it’d be better to go out on my bike. This mainly worked, though there we a few times when I was almost knocked off by groups of kids trying to stop me to give them money! Fortunately I was going quite slowly and didn’t actually get knocked off. Out on the bike I was able to do some more mapping for OpenStreetMap, adding some roads that I don’t usually walk along. After uploading these updates for OpenStreetMap I took a look at Mekelle on Google Maps. Last time I looked, several months ago now, there was virtually nothing marked, but now there’s almost a complete map. I’m interested to know where they got this data from, given that most of the printed maps I’ve seen of this region aren’t great and certainly don’t include many of the new residential areas on the outskirts of town.

Yesterday I also went to the cinema for the first time I’ve been back, to watch Teza, an Ethiopian film about a doctor returning to his home village near Lake Tana during the 70′s and 80′s. Although depressing in parts, it was well worth watching – the film won best film at an African film festival earlier this year. I had expected it to mainly be in Amharic, hopefully with English subtitles, but it was actually almost all in English. For info, the word ‘teza’ roughly translates as drop or drip.

If anyone feels like posting something over to me then anything, postcards, packet cheese sauce, etc, it would be much appreciated… the address is:

PO Box 3060
Mekelle University
Mekelle
Tigray
Ethiopia

Cheers!

I’ve now been back in Ethiopia only 2 weeks and I’ve got back into the way of life here very quickly. Work has been very busy making arrangements for the networking and furnishing of the two new computer labs, ensuring all the necessary equipment has been ordered. Most of the orders have now gone off, the final part is the order for the tables for the labs. This morning I went up to the agricultural college in Wukro to ask them to build the tables for us – the photo above is one of their main workshops, now just waiting for them to return with a quote.

I’ve been spending far too much time in line taxis (local minibuses) traveling between the three university campuses, my office is at one campus and the labs we’re building are at each of the others. So I’ve had little time in front of the computer. In a way this is fortunate as the power supply has been very poor, so I wouldn’t have got much done in my office anyway. We’re also due to be moving the Computer Science dept to the main campus later this week. The new building still has no network connection, so I’m not sure how that will work when the new students arrive in less than a month.

All the traveling around town has given me plenty of opportunity to use my GPS and contribute towards the OpenStreetMap of Mekelle (there was none before). Here is the map as it currently stands (I’ve yet to upload the changes following my trip up to Wukro):

View the ‘live’ version of the map with my most recent edits.

Last weekend I had a few people over to visit and I finally went to visit the second of the only two tourist attractions in Mekelle, Emperor Yohannes palace. Good to finally go, though not entirely sure it was worth the 10 month wait before visiting!

After a little time away from programming and active web development, I thought I’d get back into things by giving myself a little project to create a Google maps and Upcoming mashup.

At the moment it’s fairly basic as I’ve only spent a few hours working on it, but seems to be working well. I’ve deliberately tried to keep the number of options and fields down to a minimum (I know there are far more options and functions I could add).

Enter some search text in the ‘What?’ field, then a location in the ‘Where?’, finally select ‘When?’. All the events matching your search (within a 50mile radius of the location) will be shown on the map. Click on the marker to get more info about the event.

Neither the ‘what?’ or ‘where?’ field are required, either or both may be left blank. If the ‘what?’ is omitted then all events in the area are shown and if the ‘where?’ is omitted then the centre of the current view of the map is used.

All the events are from Upcoming and you can use their interface to add your own event, to then appear on the map.

For the technically minded, the site uses the Google Maps API (including for geocoding the location) and the Upcoming API.

If you know of any other feeds I could use for adding events to the map, then please let me know so I can try adding them.

I’d also be grateful for any feedback, problems or suggestions – just leave a comment below.

After a long time of not doing much (ok, any) development on my Moodle online users map block, I’ve finally had chance to return to the code to make the necessary changes for the block to run on Moodle 2.

It was always going to be tricky trying to maintain this block whilst in Ethiopia, not only with the slow internet connection, but the university firewall blocks many ports and this is my normal internet access point. So now that I’m temporarily back in the UK, I thought I’ve got chance to get back up to speed.

In the end only minor changes were needed to get the block running on Moodle 2. If you’re downloading the block from the Moodle modules & plugins pages, then please check you get the right one (v1.9 and earlier or v2) for your Moodle installation.

If you’re getting the code via CVS then HEAD currently contains the version for v2 and the MOODLE_19_STABLE branch has the version for 1.9 and earlier.

I’ve also set up a demonstration Moodle site so you can see the block in action. Just sign up, and you should see yourself marked on the Google map. You may have to wait until the location lookup script has run before you appear on the map, but this should only be a few minutes.

Please let me know if you have any problems with either getting the block to run on Moodle 2 or with the demonstration site.

Maybe I didn’t need to buy myself GPS after all… (from tecznotes):

Walking Papers is a website and a service designed to close this final loop by providing OpenStreetMap print maps that can be marked up with a pen, scanned back into the computer, and traced using OSM’s regular web-based editor, Potlatch. It’s designed for the casual mapper who doesn’t want to fill their pockets with gadgets to record what’s around them, the social mapper who might be out and about taking notes and comparing them with friends, and the opportunistic mapper who might make notes during a commute or a walk if they had a notebook-sized slip of paper to write on. Finally, it’s designed for the luddite mapper who would like to help the OpenStreetMap project but needs help from a distributed community to convert their handwritten annotations into OpenStreetMap’s tagged data and local conventions.

Since I’ve been back in the UK, although I’ve been pretty much tied to the house, with Amazon now back available to me, I’m able to spend money again. One of the items I bought was a GPS. Originally I was going to be borrowing one from someone in Addis to be able to do some mapping of Mekelle, but since I’m in the UK it seemed to make sense to get hold of my own whilst I was here. I eventually opted for the very basic Garmin eTrex H, thinking that it does everything I would want and I’m unlikely to spend money buying maps to download onto it – the main reason for buying it was to contribute to the OpenStreetMap project.

Once it had arrived my first challenge was getting it hooked up to my Asus EeePC (running Ubuntu) and installing the right bits of software.

For up & downloading to the GPS I installed QLandkarte, which only started recognising my device once I’d also installed the gpsbabel package.

Next I needed a desktop program for editing OpenStreetMap – using the online Potlatch application wouldn’t be a great option for me once back in Ethiopia with no decent internet connection.

I started off by installing Merkaartor but quickly ran into problems. When I tried connecting to the OpenStreetMap (OSM) server to download a map to edit, I kept getting “403: Forbidden” messages. The problem was that the particular version of Merkaartor the Ubuntu package manager installs doesn’t work with OSM Protocol v0.6. I think there may be a version of Merkaartor which works with v0.6, but I’m not generally very keen on installing software outside the Ubuntu Add/Remove Applications

I then tried Java OpenStreetMap (JOSM), again installing from the Add/Remove Apps and I was getting similar problems in being unable to connect to the OSM server. Again the problem was the protocol version.

Merkaartor and JOSM appeared to be the only programs available for editing OSM on Ubuntu (please let me know if there are others), so I had to resign myself to manually installing one of them. I opted for JOSM as it was quite easy to run manually and once installed all seems to be working relatively well.

I do however had a few gripes about the user interface, especially on the small screen of the Asus EeePC. For example some of the dialog boxes are fiddly to expand to get the OK/Cancel buttons to appear. Also, the drop down list of “presets” (the different map features, places of interest, amenities etc which can be added) is too long to appear on the screen, but there’s no way of navigating down to the bottom of the list to see which options are available.

Apart from that, I feel like I’ve got the hang of OSM editing (a little at least) and I’ve managed to add a few new roads. Hopefully once I get back to Mekelle, I’ll be much more productive, especially since I’ll be starting on a blank canvas.