Funded by the: Jisc e-Learning programme.

Lead Institution: University of Kent.

Learner Provider Type: Higher Education

Project Duration: January 2012 - March 2013

Key Words: Course Data

Case study tags: course data, process improvement, kis, course information, stakeholder engagement, course information, change management, University of Kent

Note: This is an abridged version of this project's final report.  The full version is available here.

 

Project Summary

In an increasingly competitive environment the way that the University of Kent presents information to prospective and current students and other interested parties is paramount to maintaining and improving the high level of customer satisfaction the University currently enjoys. Our primary vehicle for presenting course data is now our on-line course directory but printed copies of the prospectus, subject leaflets and information published by third party websites also have an important role to play. Additionally there is a requirement for the provision of this information and sub-sets of this information for a range of other purposes such as Key Information Sets (KIS) and the Higher Education Achievement Report. These requirements will be augmented in the near future by the provision of an XML feed of course data, compliant to the eXchanging Course Related Information – Course Advertising Profile (XCRI-CAP) standard. It is anticipated that this open source XML feed will be provided to organisations such as UCAS, HEFCE and UNISTATS, by participating HEIs and FEIs to facilitate the provision of up-to-date course information. These organisations may then repurpose  the data in various ways eg Course comparison websites.

 

The Kent XCRI project examined the ways in which course data is currently compiled, and administrated, and investigated how we provide and present course data in order to seek efficiencies, reduce duplication and improve accuracy. This examination informed our decision to develop a new course data administration system from the ground up which would assist staff to compile and maintain data on course programmes and modules and feed the University’s online course pages and the XCRI-CAP compliant open feed. The project helped us to ensure that our XCRI-CAP feed  provides up-to-date and accurate data in a structured, machine readable form which can be consumed internally and externally, facilitating re-purposing of the data in the most efficient way possible.

 

What did we learn?

Systems for administrating course data at the University of Kent are complex and sometimes convoluted and have developed ‘organically’ over a number of years. Unravelling the workflows and identifying requirements is therefore an equally complex task, Development of new systems has to take place in parallel to existing services to ensure there is no gap in provision.

 

We learnt very early in project discussions that the procedures, workflows and processes in place for the administration of course data at the University of Kent were far from being optimal. This was not controversial. That is not to say that we were failing in our duties to provide information relating to the courses we run, either to bodies such as HEFCE or to prospective and current students via our website and printed prospectuses and leaflets. However the credit for this was in the main due to the dedication and tenacity of a small number of staff in EMS and in schools and departments who worked long hours to make sure that courses were advertised and that we carried out our responsibilities. The University of Kent like most universities of a similar age or older has been forced to maintain many IT systems and data silos which have grown up in an organic manner over several decades. Universities were amongst the first organisations to employ information technology routinely but usually not as a result of an organisation-wide strategy or implementation plan, rather as a result of pioneering and innovative thinking in individual schools and departments. Of course we now have an Information Services department and run a modern, secure infrastructure which supports most of the functions of the university and its administration. However the legacy of the way IT developed is that there still exist various methods for the collection and maintenance of programme and module data (with associated data silos) which do not all successfully interface with one another. Hence in order to carry out the task of compiling all this data on an annual basis into the on-line and printed prospectuses, staff in various departments have had to employ many different methods resulting in inaccuracies, inefficiencies and duplication.

 

Because the work of course administration is on-going with few periods of inactivity it has been difficult to analyse the problems, to map current procedures and to begin the process of designing a better method.  The Jisc Course Data 2 Programme gave us the opportunity to do this.

 

The proposed development for a new course data administration system need to be cognisant of, and built in such a way that it could interface with, other parts of the course data lifecycle which could not be incorporated into the project deliverables.

 

At the University of Kent the responsibility for publishing course data lies with Enrolment Management Services. The Kent XCRI project sponsor was the Director of Enrolment Management Services. Staff in this department were key to the success of the project as they already had a great deal of information about where data was stored and who was responsible for updating that data and getting it to EMS for publication. It was a given that we would not, even with the resources afforded by a 15 month project, be able to design and implement methods and applications that would solve all problems associated with the entire life cycle of a programme or module. However we also felt it important that we understood the entire life cycle. This influenced who we would ask to be part of the project team. We had to walk a difficult line as we needed the input of some stakeholders in the whole process of course administration even though we knew it was unlikely we would be able to offer a solution to their problems within the life of the project. The process of investigating all aspects of course administration did help many stakeholders to have a better understanding of their own processes. It was important that stakeholders were aware of the plans of the Kent XCRI project so that any projects that they had planned for  the future  could be designed in a way that they would facilitate interfacing as seamlessly as possible with the proposed new Course Data administration system. The process helped to define the paths between different departments and roles and provided valuable insights even though it was accepted that these insights were sometimes in areas that would not be directly impacted upon by the work of the project.

 

We needed to work towards a ‘single source of authority’ and to avoid creating secondary data stores and problems with synchronisation

 

In effect we had to define where in the Course Data lifecycle the Programmes Plant (the name chosen for the application) would consume course data and take responsibility for its validation, modification and publication. We wanted to avoid duplication partly because, obviously, it was much more efficient to re-use data rather than to re-enter it but also because of the dangers of creating a secondary data store with the risk of non-synchronisation. In time, we hope that the Programmes plant will become the single source of authority for all programme and module data but that does not mean that all data must originate in the Programmes Plant. Currently the Student Data Systems (SDS) is the most important database of information on our students and the courses and modules they are enrolled on. Much of the information we needed to get into the Programmes Plant could be obtained from SDS if we could design an efficient and expedient way of accessing that data. The SDS was also key to our goal to display, a list of core and optional modules associated with a programme of study as part of the online course pages and links to descriptions of those modules held in the Module Catalogue.

 

Proposed improvements to the quality and range of data in the on-line course pages would also give additional impetus to the task of updating, cleaning and validating the sources of that data

 

Currently information on Kent modules is held in the Modules Catalogue. This is a .NET application written in-house by a member of the Planning and Student Information department. Prior to this project the only way that prospective and current students could obtain detailed information on modules was by accessing this web-based application armed with the module codes or exact titles of the modules they were interested in. More importantly, information on core and optional modules belonging to a programme of study was only presented on Kent’s Online Course pages as static lists which are compiled manually. There were no direct links to the detailed descriptions of Modules held in the Modules Catalogue and users had to enter the Module title into the search bar of the catalogue to obtain further information. The project team felt that the benefits of obtaining the links to a programme’s module requirements would not only allow us to present this data as part of the online course pages but would also provide an impetus to ensure the module information was up-to-date and accurate. The necessity to ensure that all the information we publish online on programmes and modules is inarguable. 

 

The project should guard against ‘scope creep’ and determine to identify core functionality and workflows relating to course data administration and to do that well

During the early discussions within the Implementation Group there was a great deal of pressure to incorporate the module data into the proposed Programmes Plant. However detailed investigation of what was needed led us to conclude that this would be a work package too far. Instead we decided to confine ourselves to minor improvements and to creating an interface between the online course data and the Module Catalogue so that users could jump straight to module information from the course data pages. As part of this work package we transferred ownership of the module catalogue to the Project Sponsor.

 

We committed to looking at some minor improvements to the Module Catalogue with a view to revisiting how we handle module data at a later date. The Module Catalogue as it stands does a good job and it was therefore not seen as a priority to replace it.

 

Regular honest and detailed communication between the development team and other stakeholders was key to the success of the project

 

The constant thread that has run through this project has been the importance of regular and honest communication with stakeholders and all members of the project team. This could be seen to be a given but it is often underestimated and insufficient allowance made for it. The appointment of a member of the EMS team as the Product Owner (probably the person whose role would most benefit from the successful implementation of the course administration system) made an enormous difference to the team’s ability to investigate requirements, to revise work packages and to prioritise and re-prioritise throughout the project. We stuck almost unfailingly to our schedule of fortnightly implementation meetings with the Project Sponsor in attendance. By this mechanism the Project Sponsor was always aware of the issues that were likely to block or endanger the project almost as soon as they arose and was able to take action to protect the work of the project. It is worth pointing out that although our Product Owner was able to contribute a huge number of hours to the project that was still only achievable by working extra hours and the ideal situation would have been to have had the Product Owner seconded to the project for at least some of its duration and relieved of all other duties.

 

Internal dissemination of the aims and work of the project is important throughout the whole project not just at the end

 

A related area which we did not execute as well as we might have done is that of internal dissemination. We did use presentations, blogs and other announcements to inform those not on the project team what we were doing but in retrospect we should have done more. This pays dividends as it gives the end users advance notice of changes and gives them the chance to ask questions and voice concerns before roll-out. A project of this magnitude is certain to create disruption, and for many colleagues extra work in the initial stages implementation. This extra work can be viewed as the price one has to pay for the benefits that the development will bring in due course. However if you happen to be the individual who is working long extra hours to deal with the changes caused by the implementation phase that sort of logic is of little comfort and can even add to resentment. We thought we had made sufficient effort to bring everyone on board but even towards the end of the project we had complaints that some people had no idea this was happening. In conclusion, we should have done more to ensure this did not happen. The danger of getting too much involvement from too many people – and thus creating ‘noise’ and potentially distracting project workers from the key tasks, however has to be a factor taken into account in deciding how much dissemination to do.

 

Understanding of the workflows and requirements is greatly increased by the release of a Minimum Viable Product

 

Despite all of these efforts there were still points at which it was sometimes difficult for the development team to understand precisely what the requirements of the customer were. It was not until the minimum viable product was delivered that the team began to fully understand some of the requirements. The understanding of some terminology across teams was not always the same which added to the difficulty but an unavoidable conclusion is that is difficult to understand all the nuances of procedures and workflows in a discipline you do not work in on a day to day basis.  Having a Product Owner ‘embedded’ in the development team was an enormous help. This took some investment in explaining the role and ensuring sufficient time would be available to that person to carry out the necessary tasks. We also provided formal Product Owner training for that person and perhaps we should have organised this earlier in the project but we did also feel that the training would have more value if the person in the role had some hands on experience under the guidance of our Agile teams before she got classroom training.

 

The lesson learned here or perhaps more accurately the lesson re-verified as we knew this already, is that interpreting requirements is not easy and requires lots of time. In retrospect we should have done more, sooner. Early wireframes created by the customer are useful tools for explanation. We did also conclude that no amount of planning, discussion and modelling will trap all requirements. It is not until a working model is delivered to the customer and used for the intended purposes that the team can be sure they really have met the requirements of the customer. It is also difficult, and this is definitely not a criticism, for the customer to tell developers exactly what they want until they start working with an early version of the product.

 

Much value was gained from observing the experiences of other Course Data project teams

We learnt, by attendance at face-to-face and on-line Programme Meetings and by reading project blogs and the course data mailing list, that the problems we were experiencing were not dissimilar to the problems that many other institutions were also experiencing. The benefits of this are not easy to either itemise or quantify but there is little doubt that we gained a better understanding and took strength from the knowledge we gained from other projects.

 

Immediate Impact

Early feedback from the EMS team and the temporary staff they have engaged to help with initial data loading has been very positive. The Programmes Plant application is responsive and intuitive. To date a maximum of ten users have been entering data and the application has worked well under this load with only minimum delays observed. The design is good and restful on the eye and workflow follows a logical path through the page. As we have had to load the database with a large amount of data in this implementation phase it is not possible to assess how the application would perform in a ‘normal’ year – which is of course when we would expect to see the advantages of such features as version control, automated roll over, drop down menus etc to be a real advantage. In this initial period, all data for all courses is being loaded by EMS staff but in future years most of the initial data entry and editing will be done by teams in the schools and departments running the courses .The task is also reduced for this group of staff by the roll over capability built into the Programmes Plant which allows them to use the previous year’s data as a basis for the new year and by the cloning function which aids in the creation of new programmes similar to or based on existing programmes. Obviously this will cut down the work load for the EMS team by an enormous amount though they will still have a role to review text before it goes live to the website. The ability to push data direct from the Programmes Plant to the test server and then to the live server, although present to a degree in the previous system, is much improved and faster. Previewing runs on the production server so will be an accurate representation of what we will publish and indexing of new pages will also happen immediately. With the previous system indexing took place over night.

 

In summary the work of the Kent XCRI project has had the following impacts:

 

• We have made available to Jisc, and in future to anyone interested a comprehensive XML feed of all our course data.  Hence University of Kent course data is more accessible and the preparation of stats and data for organisations such as UCAS and HESA is now much less onerous,
• the Programmes Plant is available on the desktop to all University of Kent staff with responsibility for course data with real time changes visible to all, reducing enormously the amount of time required to initiate, moderate and publish changes to course data,
• preparation and checking of course data to inform the online prospectus is now a much less onerous task,
• built in validation and approval/moderation workflow ensures a high degree of accuracy for the XCRI feed,
• relationships forged during planning and implementation of the Programmes Plant have engendered better mutual understanding of the tasks and roles of staff involved in course data gathering and preparation across schools and departments,
• Code snippets from the online course pages have been made available for use of the websites of individual schools thus reducing the need for re-coding and increasing parity between school websites and the University’s central course pages,
• Course pages are much easier to read and have a better, leaner design with links to module data. We have achieved the goal of having all information a prospective student is likely to want on programmes of study at Kent available from the links within online prospectus. This improves a user’s prospect of identifying the correct course of study,
• Stakeholders have increased faith in the department’s ability to tackle large scale IT projects and to understand their needs (with an understanding that the success of a project is dependent on the ‘customer’s’ commitment to reviewing developments and continued requirements gathering/prioritisation etc)
• All the Programme Plant code is available on Github for the use of the wider community and the public. This will give an insight into the use of PHP and Laravel in projects of this nature and could even allow an organisation to implement a version of the Programmes Plant in their own organisation due to the built in feature allowing customisation of data fields.
• Within the university, communication between stakeholders from many departments and with many different roles has not only helped us to succeed in the aims of the project but also fostered a better understanding of the how the support services of the university work and are inter-dependent. Relationships have developed which will continue and no doubt improve other areas of co-operation between departments. Without doubt these developments have changed attitudes of stakeholders as empathy has increased with understanding leading to positive benefits in workflows and information channels. This is evidenced by the fact that the director of EMS has already initiated early discussion on a further project to develop similar solutions for other parts of the course data lifecycle.
• Version history within the Programmes Plant will allow EMS and Schools staff to view the details of programmes and modules in the exact form in which they were advertised at any point in the past back, to the initial implementation of the application

 

To summarise the work of the Kent XCRI project has produced a standards compliant open XML feed of the University’s course data accessible via Cool URIs. In addition, the Programmes Plant application has provided staff in EMS and staff within schools and departments who have a responsibility for compiling and maintaining course data with a robust, well-designed, multi-user, web-based application to help them manage these tasks. The application features automated year-on-year roll over, version control and well thought out workflows for the moderation of changes to the information held within it. The application facilitates rapid or scheduled updates to the published online information to expedite corrections, changes and new information which can be initiated by staff within the schools. Schools and departments will have permanent access so can push changes to course data at any point and after moderation by EMS it will go live to the web.

 

Future Impact

Perhaps the most important anticipated benefit, though one which we cannot yet demonstrate, is the effect of improved information being available to students and prospective students. We are confident that our course pages present more comprehensive accurate information which is also better formatted and presented than previously. We hope this will lead to:

 

• improved satisfaction levels for those using our webpages
• increased and better targeted recruitment and
• improved retention rates and
• reduced demand for mid-course subject changes.

 

The provision of an open XML feed of our course data increases the visibility of the University and its offer and facilitates Kent’s inclusion in course comparison sites which are likely to be an important  resource for prospective students at an early stage of their exploration of paths through Higher Education.

 

The redesign of our course webpages has been based on good design principles and feedback from users. We have also considered the importance of Search Engine Optimisation and implemented the Google Search Appliance – both important elements in the way we have chosen to present information. We already employ Google Analytics and will use this as a tool to track the effect of the changes we have made. Of course the Student Survey will also give us feedback on the usability and usefulness of our course data pages. Through these methods we expect to be able to improve the University’s ranking in UK HEIs.

 

We will also run User Acceptance tests for the Programme Plant. Informally this process has already occurred with permanent and temporary EMS staff. The team work closely with the Product Owner who incorporated feedback in our regular sprint planning and sprint demo meetings. When development reaches the stage at which we can implement role based versions of the software (planned for end of March 2013) we will run User Acceptance workshops with Faculty and School Administration staff.

 

The version control built in to the Programmes Plant will allow EMS staff to roll back to previous versions of information or to promote later versions to live. It will also provide evidence when necessary should disputes arise over advertised content of programmes of study at given times in the past.

 

Annually a great deal of time will be save by the automated roll-over facility which will reduce the need for inputting course data

 

Staff in the schools will have the responsibility for entering details for  new courses and modules owned by that school and updating existing ones where necessary thus helping to ensure accuracy and reducing duplication and re-keying. This will also reduce demands on the resources of EMS

 

The proven ability of the Web development team to deliver large scale projects such as the Kent XCRI project will attract the commission of more internal development work for the teams and help to maintain the value of the department to the work of the University.

 

Conclusions

The provision of an open XML feed of the University’s course data which is XCRI compliant places the University in a stronger position within the UK Higher Education sector by exposing course data and allowing its repurposing by third party organisations. The work required to ensure confidence in the accuracy and completeness of this data provided the impetus to examine workflows and processes within the University and to develop and implement a much improved coursed data administration system.

 

We expect to be able to demonstrate through the use of Google Analytics and other tools, that the improved course pages created as a result of the Kent XCRI project,  will lead to increased usage by prospective and current students and hence to improved and better targeted recruitment and retention rates.

 

The increased visibility of the University’s course data has given additional impetus to the ongoing requirement to ensure our data is accurate and up-to-date and gives consistent information across all forms of course data publication.

 

The availability of all our code through Creative Commons licensing via Github and the publication of this report and the project and webdev blogs allows other organisations and Jisc to benefit from the work we have put into this project and the knowledge we have gained.

 

Recommendations

The project team recommends that all HEIs and FEIs embark on an audit of their current methods for the administration of course data as a precursor to the publication of an open XCRI-CAP compliant feed of course data. It seems highly likely that this method of dissemination of an organisation’s offer will become increasingly important and through the expected development of third party tertiary education comparison sites, a popular method for exploration of the sector.

 

The project team recommends the adoption of open source principles in approaches to development of software applications and the dissemination of developers’ code via repositories such as GitHub.

 

The project team recommends that Jisc continues to encourage Higher and Further Education institutions to produce XCRI-CAP compliant open feeds and promotes the benefits of this method of data dissemination to its partners in the education sector and beyond. In particular the commitment to the consumption of XCRI-CAP data by UCAS would be welcomed.

 

Further details: email and contact names etc

Project Director: Mary Hughes

Project Manager:  Leo Lyons

Contact email: l.lyons@kent.ac.uk