Funded by the: Jisc e-Learning programme.

Lead Institution: University of Sheffield.

Learner Provider Type: Higher Education

Project Duration: January 2012 - March 2013

Key Words: Course Data

Case study tags: course data, process improvement, enterprise architecture (ea), course information, university of sheffield,

Note: This is an abridged version of this project's final report.  The full version is available here.

 

 

 

 

Project Summary

The University of Sheffield participated in Stages 1 and 2 of the Jisc Course Data programme in the period September 2011-March 2013.  We were keen to implement changes to improve the quality of the course information that is available externally, and to explore making our data more accessible via XCRI-CAP, as we recognised that this could aid decision-making by students and applicants as well as helping us to meet increasing demands for transparent data. Stage 1 funding allowed us to investigate current systems and processes for the management of course data at this institution.  With a number of course data initiatives already underway here and beginning to  encounter common issues, we realised that we needed to support cross-institutional collaboration in order to address these.  We also identified that significant improvements to the way that course data is gathered, updated and shared at the University could be enabled by the creation of a new system to manage programme information electronically: an Online Programme Management System (OPMS).  These became two of our key objectives during the longer Stage 2 of the project.

 

We can now report significant achievements due to our ‘joined-up’ approach.  We have identified and addressed complex course data issues to enable successful local implementation of the Key Information Set (KIS) and the Higher Education Achievement Report (HEAR).  We are building an online system that will provide a common, intuitive user interface for the management of all types of course data, which is being developed via our OPMS project. We hope that this will support better course information management and ownership of information by academic departments, strengthen consistency across information sources for internal and external use, and reinforce the principle of “enter once, re-use many times”.  We produced an XCRI-CAP feed and successfully proved the concept by submitting this to the test validator, but further improvements to our course data are essential if we are to make a feed available to any confirmed aggregators in the future.

 

These efforts have generated a great deal of interest across the institution, and work initiated by the Course Data project is expected to continue well beyond the formal end of the programme.  This report will evaluate our achievements so far and explain how we hope to build on this to support effective course data management in the future.

 

What did we learn?

Our first discoveries during Stage 1 contributed to a better understanding of existing processes for the management of course data...one early project output was a set of business process maps, depicting ‘as-is’ states, that have been agreed by key stakeholders.  The aim was to map these procedures as comprehensively as possible given the time and resources available, and in as much detail as it was felt would be useful to inform a review of these processes.  In some cases, however, it was found that processes were more fluid in their implementation than these maps have the capacity to show.

 

We would recommend  this approach as a first step to any institution concerned with the need to improve their course data, as this area commonly involves users from a wider cross-section of the University, which can lead to contested accounts of current practice.

 

We have come to realise that course data is a much bigger area than it first appears.  In common with other projects involved in the Course Data programme, we came to realise that the bulk of the necessary work did not revolve around the technical procedures for generating an XCRI-CAP feed, but was instead focussed on implementing organisational change needed in order to be sure of the integrity of our data.

 

Paper-based course information processes and systems, always cumbersome, now seem inadequate to fulfil an increasing range of different purposes and to meet the expectations of external stakeholders.  Reviewing these expectations and consulting with a wide range of internal and external users led us to develop a proposal for the OPMS, and the project is striving to produce a system that will meet these user requirements.  We hope to have agreed ‘to-be’ versions of many streamlined processes the middle of 2013.

 

As stakeholder awareness of our efforts continues to grow, there have been calls to address areas that were not initially considered, for example helping departments to complete reviews for accrediting bodies (PRSBs) by providing a system to map unit (module) outcomes to PRSB criteria, introducing new processes around ATAS certification, or improving the existing system for managing unit (module) information.  ‘Course data’ includes both programme and unit data, linked together by the regulations.  The overlaps between programme and units data have often made it difficult to focus on programme-level information alone, but addressing unit information is beyond the scope of current initiatives given constraints on staff resources.  The project also brought to light the lack of central data on accreditation bodies, and data surrounding collaboration and ATAS arrangements.  This gives an idea of the scale of the task of improving course data, even in an institution that was relatively prepared for such changes as most core data about programmes and modules was already held in a central Corporate Information System (CIS).  It is therefore recommended that institutions considering undertaking similar work consider and carefully delimit the scope of the work that they undertake depending on an honest assessment of the resources available now and in the future, in order to manage stakeholder expectations.

 

Evaluation of our project’s achievements is ongoing, and will continue into the future as we evaluate the impact of changes to processes, data quality and new systems.

 

Immediate Impact

One of the most noticeable immediate consequences of the Course Data project has been the wide use of a number of business process maps and other diagrams that were created during stage 1, following consultations with relevant operational personnel across the University.  As a consequence of the availability of these maps, and of a member of staff with information analysis skills dedicated to exploring course data relationships between departments and between projects, a shared understanding of the current situation has been formed among stakeholders.  For this it was crucial to make use of the Course Data Forum to share outputs with key stakeholders.  By clarifying and documenting the ways in which various aspects of course data management inter-relate, conversations have been started and projects have progressed around a shared frame of reference.  The diagrams have helped improvement initiatives to move forward, ensuring that they are not held up by a lack of information about relevant processes.  They have helped to re-focus projects where necessary, affording some balance to these by ensuring that project teams take course data dependencies and relationships into account rather than pressing ahead with solutions that could cause problems elsewhere.  Because they were not created from the perspective of any one department, they are not “a view from one silo” but an attempt at an objective institutional view.  In the creation of these documents we have discovered good practice, and we hope to use this approach elsewhere in the future.

 

The second major and immediate impact has been the presence of a developer to begin work on the OPMS project, which arose directly from the Course Data work.  Given our in-house development approach, the project would not have been able to make progress without a development resource, and this has proved to be a cost-effective way of implementing an “iterative development” approach whereby user feedback is consistently gathered to improve the final outputs.  We are already seeing the benefits of this with a constantly improving prototype system for the better management of course data.

 

Future Impact

The Course Data project helped us to develop the case for improvements to course information management to the extent that the delivery of an online system to improve this area is seen as an urgent and essential priority.  The fact that work in this area will continue well beyond the formal end of this project is a positive indicator of the significant impact of the work done so far.

 

We expect significant changes to business processes to be enacted as a result of work undertaken during the Course Data project, but these will be implemented after the submission of this final report, which limits the evidence we are currently able to provide about the impact of the system on internal and external users of the University’s course information...

 

Conclusions

Conclusions with relevance for the HE/FE community

• Paper-based processes for course information management no longer meet external requirements for data, and are unlikely to meet the expectations of internal users.  Improving processes and systems for the management of course information is just as beneficial to a higher education institution as it is to their external stakeholders.
• For this project the most significant changes in terms of both impact and workload were those concerned with processes for gathering, updating and validating course information, rather than those that focussed on generating an XCRI-CAP feed.
• In order to improve our processes and data outputs, we found it very useful to first document current processes and gain agreement of this version of events.  Gaining wide agreement amongst stakeholders about less structured processes was sometimes difficult, but it was also essential and rewarding.  As well as helping to inform some new ‘to-be’ process maps, these are a valuable reference resource to be used and developed by the institution.
• The KIS and the HEAR are both initiatives that require accessible, accurate and well-structured course data.  There are many overlaps between these in terms of data and key processes, and these in turn often overlap with internal requirements.
• Our joined-up approach was key to making progress on this project.  Our Course Data Forums provided a framework for discussions which led to better understanding and co-ordinated action on common issues, and shared awareness of ‘the bigger picture’.  With this framework, we could be more confident in our understanding of how our work would affect processes across the University.  It allowed us to engage key stakeholders and helped to increase the transparency and visibility of the project.  Users also value co-ordinated communication efforts.  With so many related initiatives happening at one time, this approach proved invaluable to us.  The Course Data Forum will continue to meet beyond the end of this project.

 

Conclusions relevant to Jisc

• Opportunities to network with institutions in a similar position (programme meetings) and keep up to date with their progress (through the collection of project blogs) proved very useful to us.
• Also useful were the opportunities to learn new and relevant skills, for example the sessions on Enterprise Architecture methodology.

 

Recommendations

Recommendations for the HE/FE community

• Because the scope of this project is so much broader than it first appears, ensure that your institution has planned for the long-term provision of resources to see through any changes that you decide to make as a result of course data analysis work.  It may also be important to ensure that permanent funding is in place to maintain any new systems, processes or models that you create.  If there are limited resources available, it may be necessary to manage stakeholders’ expectations.
• XCRI-CAP’s usefulness depends on the context in your institution. We recommend that you consider:
• which external parties do you send data to?
• what are the specific fields of information they require and do these match the XCRI-CAP standard (which is quite flexible)?
• do you have any internal uses for XCRI-CAP, e.g. the prospectus? This may require cross-institutional commitment, and in our institution it was decided that using XCRI-CAP would not be an improvement on current processes
• how close is your data to being something you would be happy to share in this way? For us, getting to this point is expected to take longer than the duration of our formal participation in the programme, but the improvements we have made will have many, many more benefits than we first anticipated
• It also depends on aggregators making a firm and continued commitment to accepting the XCRI-CAP standard, as without this agreement across the sector, multiple data submissions and duplicated work will continue.
• Enterprise Architecture is a useful project tool, but institutions should employ caution before adopting any aspects of it that will require maintenance in perpetuity.  We found it most appropriate to use EA techniques as a project tool, using Archi to model business areas of interest (particularly relationships between systems) and then using these models in combination with other tools to start necessary conversations with stakeholders.

 

Recommendations for Jisc

• If XCRI-CAP is to become a de facto standard, used widely across the sector, as well as a formal standard, then there must be a critical mass of support behind it among the various bodies that make up the wider sector.  This is particularly of relevance to UCAS, because although we welcome the support of other aggregators, it does not provide quite enough incentive to put scarce resources into management of an XCRI-CAP feed, considering that the same work will still need to be done in a different way for UCAS.
• Our project team felt that at the outset we would have benefitted from a clearer idea of the extent to which this project was likely to continue beyond the formal end of Jisc funding, and a warning that we should be prepared to secure resources for this.  We have found ourselves in a situation where we cannot wind the course data initiatives down at this point because work on these is still at a peak, and yet the funding is ending.

 

 

Further details: email and contact names etc

Project Director Helen Grindley

Project Manager Anne Rodgers a.rodgers@sheffield.ac.uk

Project Web URL http://www.shef.ac.uk/cics/coursedata