Last week I spent the day in Birmingham at a UK Turnitin Users’ Group meeting. At these events the company likes to share its road map and talk about the new features they hope to release that year, but most of the delegates had something else on their mind – the poor performance of the system in December and January, with one of the outages lasting 8 hours. Just as bad was the poor communications whilst the service was unavailable.
Chris Caren (CEO) had flown in from the US. He was obviously well-briefed and began the session in a contrite manner. He offered his sincere apologies, recognising that Turnitin had clearly let us down. Much of the frustration felt by the audience regarding the poor performance at peak times was that we feel Turnitin hold all the information necessary to manage this. They know the submission deadlines and the size of the classes and have a wealth of historical data that should allow them to estimate likely and peak loads. Chris acknowledged this and said that the recent failures were because of people, not technology. Turnitin (iParadigms) run two installations using the same core software, one in the US, the other in the UK. The US service has remained up and coped with loads more than ten times anything experienced in the UK. So the UK system should have coped. The problem was that someone (no longer working for the company) specified a database for the UK system that was too small and it ground to a halt under load. He also recognised that their system monitoring and root cause analysis methods are not as good as they should be. Delays in identifying the problem and several false starts contributed to the eight hour outage. He was very clear, large outages like this must not be allowed to happen again. He did point out though that outages are a fact of life – even giants like Google do suffer the odd outage.
He talked a lot about changes they are planning to the infrastructure. Currently everything is too inter-connected, meaning that changes may require alterations to the database, the servers, the software code and the user interface. This makes them difficult to test, risky and harder to roll back later. The situation is further complicated by the fact that they have to do all this whilst supporting a large number of custom integrations with other systems – Blackboard, Moodle, Sakai, PebblePad, iPad apps, etc. adding a series of legacy constraints. They are planning to address this by re-engineering the system, reducing the number of ways systems can talk to it (the new default will be the standard LTI 2 protocol when it is finally agreed by the IMS) and using a wrapper around the system to try and abstract many of the functions. An immediate focus is to improve the way the data queues requests, so users are still able to submit work when the system is busy, even if it takes a while for the file to be processed at the back end. It all sounds sensible but it needs to be carefully managed both whilst it is being developed and then during the migration process. One change is the location of Turnitin’s product development team – it is now led by a UK-based member of staff (Will Murray) and they plan to increase communication with UK clients. That said, it was good to see that many of the changes planned for release on their road map came from suggestions raised in past UK Users Meetings. They are also running a beta pilot for the new Blackboard integration (which Durham have already signed up for and are testing ).
The poor communications is also something they are working on, with plans to centralise details of issues and upcoming changes via their http://submit.ac.uk/en_gb/support/system-status site and the uservoice site they use to help users prioritise new features and fixes. They will also increase the number of people who can receive emails from Turnitin about service issues – currently this is limited to one per institution.
Much of the road map presentation was speculative and some slides were marked as ‘confidential’ – I will share what I can later when I am sent the slides.
Over lunch there was a special listening session with Chris Caren, Will Murray and eight individuals (including me) representing the Heads of e-Learning Forum (HeLF). This came about after a letter we sent to Turnitin in January expressing our dissatisfaction with the current service. This is not the first time we have had to take this approach. HeLF members made it clear to Turnitin that as well as the issues just experienced, we feel that particularly because of its near monoply position, Turnitin has a moral, ethical and quality related obligation to work in a way that recognises the national responsibility and regulatory framework that UK universities and colleges have to work within. We feel strongly that the current performance and way changes have been pushed out with insufficient testing and communication is highly damaging to the reputation of both Turnitin and UK HEI and FEIs, particularly when some of these changes seem to contradict or require significant reworking of existing policies and practice. We also emphasized the negative emotional impact that any problems with online submission can have upon our students and the staff who manage and mark the work. We reminded Turnitin that in some institutions failure to submit work on time can result in a mark of zero – this point hit home. Later Chris pledged to appoint UK account managers to come and listen to individual clients, understand they way we work and the impact of any potential changes upon our staff and students. This dialogue is ongoing and a formal response from Turnitin to HeLF is expected in the next few days. UCISA is also pursuing this matter, but chose to do so separately from HeLF.
All in all it was an interesting day, with plenty of food for thought on both sides. A good summary is available on Tim Smale’s blog – see http://blog.kelf.co.uk/2014/02/turnitin-uk-user-group-aston-university.html