Funded by the: Jisc e-Learning programme.
Lead Institution: University of East London.
Learner Provider Type: Higher Education
Project Duration: January 2012 - March 2013
Key Words: Course Data
Case study tags: course data, process improvement, course information, university of east london, costing, business intelligence (bi),
Note: This is an abridged version of this project's final report. The full version is available here.
Project Summary
The overall aim of the project has been to produce high quality, accurate information about the course and programmes delivered by the University of East London. In order to do this, we needed to look in detail at our internal processes for producing new programmes and the course information required for marketing these. During stage one of the project, we took the opportunity to examine our current procedures using the XCRI-CAP self assessment framework and identified some weaknesses in our procedures.
As a consequence, the project team agreed to contract an independent business analyst and the results of the process review clearly indicated that the existing methods for producing course specification and marketing material was cumbersome, inefficient and difficult to repurpose for other uses. To assist the project in streamlining our processes, the business analyst produced a requirements specification that has been used to inform the procurement of the SITS Curriculum and Process Manager modules. Once fully implemented, this supplier based solution should allow us to significantly reduce the time taken for developing new programmes and will enable us to introduce tighter version control, log changes, store the data in our student record system, automatically publish course information as web pages and more importantly, generate the XCRI-CAP feed. Although we have procured the supplier based solution, the project team had concerns over whether these could be delivered in time and as an interim solution, we have developed a web-based new programme application which was rolled-out for the 2012-13 academic year. Although this is not as sophisticated as the SITS-based solution, it is proving to be a useful intermediate step before we introduce a slightly more complex process going forward.
One additional issue that emerged from the previous experiences of the Project manager and colleagues in Strategic Planning and Finance was that, the existing processes paid less attention to the financial viability of a proposed course. As a consequence and with significant experience in developing business intelligence applications, the Project Manager with colleagues developed an interactive model that allows the user to test assumptions about the financial viability of a programme.
What did we learn?
We have gained some valuable lessons from every part of this project and we will continue to do this once this project has formally ended. Perhaps the most valuable lesson we have learnt is that no one person has a comprehensive understanding of the processes used to develop new courses or the production of documentation in terms of course and module specifications. Key stakeholders understand the components of the process they are responsible for but have a more limited grasp of the processes that follow or the level of resource required to complete the task. This became particularly evident when conducting the business process review and the project manager was struck by the number of staff involved in the process of validating a new programme and producing course information. The outcome of the business process review indicated to the project team that was in fact, a bigger project than we originally envisaged and that the full implementation of a technology based solution will need to continue after the formal completion of the Jisc project.
Given what we came to realise was the size of the project, implementing an interim solution in terms of our new programme application has been particularly useful. Not only does it represent an intermediate step from purely a paper based process, we learnt some valuable lessons in designing the template and attempting to develop a workflow. Although it’s not as technically advanced as the proposed SITS solution and doesn’t allow us to repurpose the data for other uses, it has nevertheless enabled colleagues in Quality Assurance to ascertain the progress of new or re-validation of existing programmes and to make comments relatively promptly. At the time of writing, there are 61 new or revalidated programmes that have used this application and this appears to be growing by 4-5 per week. Feedback from colleagues that have used this application have suggested that although this is relatively easy to use, some supplementary documentation or exemplars would have been useful in supporting the proposer.
One of the main objectives of this project was to streamline our processes for validating new or revalidating existing programmes thereby reducing the level of resource and associated costs. Calculating the financial viability of new programmes was previously an iterative process between the course proposer and colleagues in Finance but often undertaken after much of the documentation had been produced. Given our expertise in developing business intelligence applications, we decided to build the model in QlikView which would allow course proposers to interactively model their estimates of student recruitment and the costs in developing and delivering the programme. In addition, it would provide us with a mechanism for evaluating whether we had reduced the overall costs, particularly in terms of course design and preparation.
Not surprisingly, one of the issues that have emerged from evaluating the new programme application and the programme costing model is the requirement for improvements in training and support. When sampling a range of new course proposals, it became apparent that there were widespread differences in how the template was completed. Some documents were relatively sparse, particularly in terms of consultation with end users and marketing. Similarly, when analysing the costs of proposed programmes, there is generally an under-estimate of the time required to develop and deliver a course and often an over-estimate of student recruitment, particularly during the first year of the programme. As a consequence, these issues will need to be addressed as part of the training programme and we will incorporate these when we roll-out the SITS Curriculum Manager model supplemented by additional support in how to accurately cost the development of courses.
Finally, we have had to develop the virtues of patience and perseverance. This has been particularly important during the delays in procurement and response of SITS in delivering the Curriculum Manager solution. Instead, we have used the time to develop and refine the programme costing model and our new programme application.
Immediate Impact
Although the project is not completed, the greatest impact has been in a shared understanding of the immediate need to transform our processes for course approval from a largely paper-based exercise to one that makes more effective use of technology. In some ways, this has been helped by the realisation that HEIs now operate in a more market-driven environment and institutions that can respond quickly by bringing new and innovative courses are more likely to prosper. The process review highlighted some of the inefficiencies in the way we validate programmes and the generation of course information. The opportunity to embrace an IT based solution has already evidenced by the number of new programmes developed using our new programme application. This web-based application replaces the creation of numerous word documents that are circulated amongst peers by email for comments and amendments with a limited concept of version control or logging of changes. Because the production of the documentation is so visible, this has meant that comments can be made more quickly and scheduling of priorities easier for colleagues in Quality Assurance.
The development of an interactive and visible programme costing model has served to highlight the financial outlay in developing and delivering courses. Previously, this was calculated by Finance and there was some uncertainty in how this worked. In most cases, the financial viability analysis was carried out at the end of the process. Deploying the programme costing model has demystified the process, although there is a requirement for an enhanced staff training session in how to accurately calculate programme development and running costs.
Future Impact
In many ways, the future impact of the project will be even more beneficial to the University. The capacity to incorporate automated workflows into our course approval processes, the enabling of version control, logging of changes and storage of data meaning that we have a single source of programme information will represent a significant step forward. These benefits will largely contribute to meeting our objective of process improvements. However, it is feasible that the largest impact on the institution may result from being able to repurpose the data for other uses. The production of course information webpages can be largely automated, key information required as part of the course record can be generated and we can integrate contextual data about a course into our Business Intelligence applications. This will allow us to portray course performance data alongside details of how the programme is taught, assessment methods, etc.
Conclusions
Before the project began, there was an understanding and desire to improve our processes for producing course information. Typically, the majority of programme information was produced as part of the course approval process and was difficult to repurpose for other uses. This was recognised and articulated in our original project plan and process improvement constituted an important element of our objectives. However, it was only after contracting an independent business analyst who mapped our course approval processes that we began to fully understand what was required. We could have easily carried on our existing processes and produced our XCRI-CAP feed. However, there has been a recognition that we need to employ a leaner, smarter solution and that we could maximise the value of technology in producing our course information. Although we procured a supplier based solution, the full benefits of this will not be fully realised until later in the year. However, we have made significant strides in transitioning to a technology based solution by developing and implementing our own new programme application and the uptake of this and feedback has generally been positive. This will make it easier to deploy when we fully implement our SITS based solution which features workflows and will be more complex to use.
In our project plan, we expressed the desire to integrate course information with our business intelligence functionality. Originally, we envisaged this to be adding contextual information about courses to our performance data held in our student lifecycle application. However, it became apparent that calculating the financial viability of our programme was almost a supplementary exercise to be carried out at the end of the process. The project team decided that this needed if anything to be located earlier in the course approval process and the way to achieve this was, to develop a Business Intelligence application in QlikView. Although more training is needed in costing, this has already provided colleagues with a more informed view of the costs of developing programmes.
One of the main issues in completing this project has been the capacity for a single supplier to satisfy the requirements of a large number of institutions. Whilst we recognise that institutions have different processes for generating course information and therefore require different solutions, we had legitimate concerns over whether our supplier of our student record system could deliver what was required given the large number of HEIs they support. We have been encouraged by recent progress in developing a template model and the architecture behind that will enable us to deliver what we set out to achieve. However, it’s unlikely that this solution will be fully implemented before the end of the project in March.
Recommendations
General Recommendations
a) In a project of this type, conduct a detailed process review as soon as possible into the project. This will enable you to scope the size of the process improvements and to commence work on developing a solution.
b) At an early stage of the project, determine whether other HEIs have already implemented a similar institution to ascertain how they accomplished it, the obstacles to be overcome and the time it took. Learn from their experiences.
Recommendations for the wider community
a) Where possible, look to team up with other institutions undertaking a similar solution for potential economies of scale.
b) Examine all parts of a process and assess whether there is a more efficient way of undertaking this. In particular, look at a technology based solution to streamline processes.
c) Consult as widely as possible. There are often improvements that can be made about the use of information that are not immediately apparent.
Recommendations for Jisc
a) The length of project was probably too short for what turned out to be the improvement of some complex processes. Although stage 1 of this project enabled institutions to review their processes for generating course information, it was only when undertaking a more detailed process review during stage 2 that the size of the project truly emerged.
b) It might be worth revisiting the projects in 6 and 12 months for a fuller evaluation of outcomes. To fully implement a technological solution would most likely take 9-12 months, with a further 9-12 months for a full evaluation.
c) Although the project plans were produced as part of stage 1, it might have been worth projects re-evaluating the scope of their plans after 6 months of stage 2.
Further details: email and contact names etc
Project Manager Gary Tindell
Contact email g.c.tindell@uel.ac.uk
Project Web URL http://blogs.uel.ac.uk/course-data/
Costing Model Video: http://www.youtube.com/watch?v=KnWK7kAuVf0
Note: further information on the costing model in the context of Business Intelligence at UEL is available in a case study on Bringing Corporate Data to Life at http://bit.ly/uel-bi
