Alex Little

Have just come back from the JISC conference in Edinburgh where I attended the Learning Design strand. We’re still now sorting out what we can bid for in the latest JISC toolkit and demonstrator project calls – we have a few ideas as to what we could do, and just need to get these refined into an actual bid in time

At the end of last week I finally got the ePET ePortfolio system up and running on my laptop, all seems to be running fine, though not sure at the moment whether or not I’ll be able to get time to link this up to the CCSI module and run with the Sled player. I’ve just come back from the JISC conference in Edinburgh, and Hubert and Harrie at OUNL have sent me the updates to the CCSI module so that it can run with a webservices connection, so I need to spend some time getting the Sled player running using this instead of the java api. The JISC meeting was useful and seems that there are a few people interested in taking the work we’ve done on the CCSI intergration layer forward, and also developing the Sled player further.

We are in the process of figuring out what we do next, especially since the latest JISC calls seem to preclude us from getting funding form them to develop Sled further – since the model appears to be that other organisations bid for funding to take up our work and we simply provide some support for them to do this.

Yesterday I was looking at switching the database that was used for CopperCore so that instead of using the default Hypersonic database it used an MSSQL database instead, I got most of the way in doing this, but then got a bit stuck when needing to add the JDBC jar path to the classpath for coppercore startup file, I tried a few different ways of doing this but didn’t actually get it working- will try again later.

I’ve also updated some of the CCSI adapters, as the ones that I originally created (to get it working to start with), had URLs and the Google search API key coded into the java code -which is obviously not the best thing to do. What I’ve done instead is move these parameters out to a properties file. I’ve put these changes back into Souceforge CVS and when I put out the next version of the Sled player, this will include these changes too.

The other thing I’m going to work on is creating a page or two on the Sled website to explain what the CCSI stuff is all about and how people could add new services or switch service provider.

I’ve now think I’ve fixed the problem when a blank page was returned when the user double clicked on a link. The error that was being generated was ‘javax.ejb.EJBException – Application error – tried to enter Stateful bean with different tx context’ and I think again that the root cause of this is java threading. All I’ve done is temporarily fix the Sled player so that it won’t display blank page to user -it will generate the correct content, but it doesn’t stop the errors appearing in the JBoss console – they are just being handled better!

I’ve updated the Sled demo site with this new version, and I’ll include this fix in the next release I put up on Sourceforge.

More results from the load testing. Previously I had just been testing with a server which only had 5 users registered on the LD system and 2 Units of Learning, and I was only load testing with 3 concurrent users. This morning I added more users and UoLs, so we had 20 users registered and 10 UoLs. I then tried the load testing but still with only 3 concurrent users.

For the first set of testing (with 5 users registered) we had average response times of 11-12 secs for Sled player and 2-3 secs for CopperCore player. Though as I mentioned earlier – it;s difficult to compare these two directly as they are measuring different things – I’m really looking at the relative differences, not the absolute figures. Then when I tested with 20 users registered on the server (ecah assigned to 10 UoLs) the speed decreased quite dramatically to 30-35 secs average for Sled and 9-10 secs for CopperCore – so both players were about 3 times slower with a very small increase in the number of users registered.

For info, I was doing this testing on an old laptop (about 4 years old – PII, 200MB ram) and with a default installation of CopperCore and Sled. I also restarted the CopperCore service before the start of each test.

We’ve also found another little problem that is affecting the player, and that is that if someone double clicks a link in the activity tree menu (so clicking again before the server has had chance to return anything after the first click) this is generating a blank page (just the Sled header and footer) – so I’ll need to see what is causing this problem!

I have uploaded the latest version of Sled (2.0.2) to Sourceforge – which includes the fixes described in my last couple of entries in my LD diary/blog

I have uploaded the latest version of Sled (2.0.2) to Sourceforge – which includes the fixes described in my last couple of entries below

I have been trying again with load testing Sled and CopperCore. I’ve managed to update the Sled player so that it doesn’t generate errors, but it is still being quite slow at times, especially when there are multiple users (I’ve tested with up to concurrent 15 users)

The main bottleneck on the Sled player appeared to be the server side transformation of the XML (I’d put some counters in the code to test where the code seemed to be slow), so I tried caching some of the transformed content (parts of the pages which didn’t tend to change from hit to hit), but the slow transformation was usually the one that generates the activity menu tree – which I can’t really cache very easily because it changes so much depending on which user is logged in and which page they are visiting. But the XML transformation isn’t the only speed problem, it’s just part of it.

I also went back to have a look at testing the CopperCore player again and this time I did manage to generate the same error I was getting with Sled (but Sled now retries if an error is generated) – the reason I found that didn’t come up before was that I was only hitting the frame page and not the individual frames. So I amended the load testing script so that it also hit the individual frames. It’s difficult to compare the response times between Sled and the CC player, because Sled generates the whole page all at once, and CC needs a number of hits to get the equivalent content displayed (and also the XSL transformation isn’t being done in CC). But from the results it looks like the generation of the activity tree is also the slowest part on the CC player.

Overall, the speed problem seems to be a combination of a number of factors. So far I’ve still only tested when the server has a couple of UoLs & a couple or users, so I’ll run the same tests again once I’ve added more users & UoLs to see if that slows it down significantly further – we were finding that the speed on a standalone installation with just a few users and UoLs was fine, but on our Sled demo site where we have around 10 UoLs and over 50 users (registered, not concurrent) the speed slowed right down – even thought the users aren’t accessing the site simultaneously.

The fix I have put in Sled is really just a sticking plaster, not a full & proper solution and that there could also be other reason for the player being slow under load that I haven’t spotted! Please feel free to let me know if you have any suggestions as to how it could be made faster

I’ve spent a bit of time trying to sort out the performance problems with Sled – and the error messages that get generated. I’ve been trying to recreate the problems by using a load testing tool (JMeter) on the Sled/CC installation on my laptop. By forcing about 5 or 6 virtual users to access the site at the same time (and go through a sequence of hitting several pages on the Sled site) I could get errors to be generated, or get pages returned which were blank. I also tried to hit the CopperCore player in the same way but couldn’t get the same problems created.

I think that what was happening was that the error messages that appeared in the browser were actually due to an error that had previously occurred on another page request which meant that a java object wasn’t created properly, this then meant that on another page which needed to refer to the object it didn’t exist – so created the error.

I’m not 100% sure but I think the root cause of this is java threading with the CCSI module – as the errors that were generated were when the system tried to create objects defined in the CCSI module. If this is the case, I think I can explain why this doesn’t affect the CopperCore in the same way. The CC player is essentially a single piece of code, which is written so that this single bit of code can only process one request at time (subsequent requests have to wait until the first request has completed), which means that it will only ever make contact with the CCSI module with one request at a time.  However the Sled player is coded differently – so although the Struts framework is thread safe and will handle multiple concurrent users, this doesn’t stop it making multiple requests to the CCSI module at the same time, which is where the problem occurs.

I’ve slightly altered the Sled code so that when it tries to create one of the CCSI objects, it checks that it’s a valid (ie not null & hasn’t generated an error) and if it’s not the thread tries again a few milliseconds later, and keeps trying until a valid object has been created. Testing this again with the load tester seems to have stopped the error messages appearing to the user. However I am still sometimes getting deadlock exceptions which I think are arising in the CC code, but these don’t; seem to be affecting the actual operation and output back to the user – so it seems that java is just reporting this as an error but it coping ok with it.

I’m sure that I’ve not resolved this in the ‘proper’ way – I’m sure there must be better ways of doing it, but I need to be a better java programmer to do that!

However, I’m not sure that this will fix the speed of the Sled application, the machine I’m testing it on only has a couple of UoLs and a couple of users on it – and the speed problems seemed to mainly occur once there were more users & UoLs than this. I’ll add some more users an UoLs and keep testing to see what happens. The speed that’s being reported by the load tester is an average of about 7 seconds per request – which is really quite bad! – but that’s with 10 users hitting the site making about 250 pages requests in total over a period of about 1 minute. The CopperCore player is much quicker than this (around 1 sec per request for same no of page hits over same length of time) – and I’m sure much of this is to do with the fact that the Sled player is doing transformations server side (so doing file reads to get the XSL) so this could be speeded up by maybe caching the XSL files or something similar.

Once I’ve tried load testing with more users and UoLs I’ll be able to see if it slows down significantly more or not – if it does then I’ll try and find out what is really slowing the processing down – but if not, it might not be a good use of my time to spend ages just trying to cut a second or so off the processing time – especially as it hasn’t really been built with production or heavy use in mind.

Had some of the results back from the evaluation at the Sled workshop and a number of people have mentioned about the left hand menu being too ‘close’ to looking like learning design and so being a bit confusing. I can see exactly where they are coming from as I agree, but the problem I have is what to do about it.

The current men system is set up as it is because the LD spec allows titles to be places at virtually all levels, eg activity structure, activities, items etc (made more complicated again by the ability to nest all these items), so different LD authors may decide to place the titles at different places. For example author A might decide to use the title tag within the item, author B might use the title tag within the activity and author C might decide to use a mixture.

This then makes it very difficult for the player to know which ones should be displayed (any blank titles or missing title elements are just ignored by the player anyway) – especially if the author has been ‘thorough’ and completed all the title tags for each element. The only way around that I can see is just letting the player display say the activity structure and item titles and them everything else is ignored, but then authors who have completed more might get confused as to why it’s not getting displayed.

Anyway, if you have any bright ideas as to how this could be resolved then please let me know!