Funded by the: Jisc e-Learning programme.

Lead Institution: Falmouth University.

Learner Provider Type: Higher Education

Project Duration: January 2012 - March 2013

Key Words: Course Data

 

Case study tags: course data, process improvement course information, stakeholder engagement, falmouth university

Note: This is an abridged version of this project's final report.  The full version is available here.

 

Course Marketing Data Collector

Project Summary

Within the HE sector it has been shown that there is a shortage of accessible information relating to specialist courses like part- time, distance learning, CPD, postgraduate and writing opportunities. As a small specialist institution within the areas of Art, Media, Design, Performance, this further restricts our audience and presents challenges to our potential student base and to our marketing team in terms of advertising and navigating to course information.

 

Falmouth is committed to developing new types of learning experiences for an expanded student base, this includes work based flexible learning opportunities and non-full time campus based courses at home and internationally.

 

The work completed under stage one of this funding stream enabled us to complete a high level review of the effectiveness of our current course marketing data creation and reporting processes. This highlighted issues in relation to data inconsistencies and process inefficiencies. In addition the current processes and structures do not readily support inclusion in an automated information feed.

 

Our project included the development and trial of a new internal online course data management tool, which enables us to collate all marketing-relevant course data from initial proposal through the full lifecycle of delivery.  Our vision was for this to be a dynamic data collection process and external marketing feed (XCRI-CAP) ensuring our visibility to prospective students and other stakeholders as well as providing the most up to date information possible. We hope that this facility will raise our profile and enable choice and comparison with bigger institutions that may have more marketing resources at their disposal.

 

What did we learn?

Workshops are a good way to engage stakeholders in the project and aid communication between “silo’d” departments. Bringing in and allowing others to hear the issues that exist in a process can aid in solution design.

 

The ROI from automating the course creation process is not as high as anticipated and the benefits of the work done to date are more qualitative than quantitative, we anticipate data quality and communication improvements to be the biggest improvement.

 

As we were unable to fully automate the XCRI CAP feed from our course creation process the maintenance actually creates an additional overhead for the marketing department.

 

We have found a staged approach to end user testing where technical and project office staff complete initial cycles before allowing business owners to ‘get their hands on the solution’ to be the most effective way at managing user expectations of new systems. If end users are exposed to new systems too early in testing (for example where major bugs still exist) they can find it hard to buy into its effectiveness as a solution.

 

With hindsight it would have been wiser to get more client engagement with the beta system at an earlier stage of the project, by building in more demonstrations of development in progress, which would have helped us to the problem with the user interface design at earlier on in the development cycle. We utilise a waterfall development technology at Falmouth, but we are exploring utilising a more agile approach to future developments, especially those developments with web user interface design requirements.

 

Open source development using Drupal where you are building a solution based on utilising existing Drupal modules can restrict your ability to deliver bespoke functionality, which can lead to compromises in design.  Obviously this can be overcome by creating original Drupal modules/developments, but original developments requires a much higher technical skill set and would require a much more rigorous and detailed design phase leading to a longer and more costly development cycle.  

 

The new requests for external reporting including the KIS, HEAR mean that universities are under real pressure to provide more and more core data electronically.  Where key and complex business processes are only supported by paper based systems this can mean that providing this data is a time consuming and complex process.  Falmouth need to be reviewing where core data that is required for external reporting purposes resides so it can manage and plan the automation of any data collection processes effectively. 

 

Engagement of key stakeholder teams can be difficult especially given other project commitments even if resources are available to buy-in additional resources it is not always that easy to achieve the commitment.

With hindsight it would have been wiser to get more client engagement with the system at an earlier stage of the project, and capture the problem with the user interface design at earlier on in the development cycle.

 

Immediate Impact

On reflection we feel the Course Marketing Data Collector project has had an immediate impact on;

 

Quality Office

• Defined the Course creation process and delivered a clear process model for training and communication purposes.
• Created a centralised repository which can be used for storing definitive course documentation.
• Raised awareness of the course planning cycle and issues with the process.

 

Academics

• As we are still in the pilot phase of the project the effect on the Academics to date is limited other than.
• Provided clear guidance on course creation process.

 

Marketing

• Captured and publicised marketing department issues with current course creation process.
• Developed a ‘To Be’ process which addresses the lack of communication between academic planning and marketing through automation of communication tasks.
• Developed new marketing central repository of up to date/approved non UG course information.
• Delivered new non UG course marketing data collection process.

 

IT services

• Created database (Drupal XCRI and CCMS) of up to date course marketing data.
• Developed process and technical solution for publishing xml files out to third party securely.
• Increased knowledge about EA application and business process modelling change management techniques.

 

Planning

• Captured and validated planning process for courses.
• Developed CCMS solution to improve communication of planning cross department

 

Data Quality

• Created process for collating course marketing data
• Created central repository of course marketing data (XCRI feed) for non UG courses.
• Hope to see an increased quality of document content through provision of clear process and system to support the process…

 

…We feel to date the actual external benefits that have materialised are;

• Collaboration with other Arts institutes – the Arts Assembly has been extremely useful in terms of sharing learning and experiences on the project, and also sharing knowledge. Our XCRI feed data is being used by the contractor on Plymouth’s aggregator project as a test feed and we have utilised Courtaulds XCRI module. Our experiences to date from this group represent a strong voice on the experience of specialist institutions and our collective ‘voice’ makes  a strong argument that more though needs to go into XCRI data  definitions if they are to be truly ‘useful’ moving forwards.
• We have delivered a case study and data feed to be used to build the case for XCRI adoption with the HEI community. Our involvement with the Art Assembly Aggregator tool produces a real example of how the feeds can be used in practice.
• We have and continue to share our experiences on XCRI CAP and our approach to change with the HEI community.

 

Future Impact

Quality Office

• Once the CCMS system is rolled out past the pilot phase the Quality office should see a reduction in administration effort through the course creation and validation process which we estimate at 1.5 days per course created.
• We believe the quality of course data will improve, by creating a user friendly interface with clear process and guidance notes academic teams will be able to complete course documentation in a more controlled and managed environment.

 

Academics

• Once the CCMS system is rolled out Academics will have online access to an online data repository of quality documents.
• Reduction of effort, we have calculated that an average of 6 days effort per full course created should be saved in Academic time, by introducing a central data repository and by managing document versioning we will enable collaborative working on course documentation.

 

ADPC

• Deliver one central repository of quality documents.
• Speed up the Course Validation process.
• Provide clarity on the status of courses moving through validation.

 

Students

• If successful aggregator sites are in place existing students will enjoy the opportunity to ‘compare the market’

 

Potential Students

• If aggregator sites are in place – provide clear and comparable data on our non UG courses for students to consider.

 

IT Services

• Opportunity to standardise one data feed of course related information – feeding all other MIS systems from one data input.

 

XCRI CAP Standards

• Opportunity to feed into the evolving design of the feed and add additional fields and images, also to standardise some fields so that data fed out can truly be utilised by aggregators to produce comparable searches.

 

Aggregator sites

• Opportunity to work with specialist aggregator sites and the Arts Assembly to further develop XCRI data standards and definitions to improve searchability and usability of data.

 

 

Conclusions

General conclusions

Process benefits – Falmouth is a reasonably small institution with a reasonably small change year-on-year in course provision. When we started the project we anticipated that automating the course management systems would deliver a large quantitative time saving for the quality office and Academic staff, but on further analysis of the ‘As Is’ and ‘To Be’ process model the actual estimated annual admin effort saved is relatively small. The real process improvement and efficiencies will be qualitative and hard to measure, improving data quality, clarity of process, communication through the course planning process, and through reduction in process lead time. Initiating a similar project on a calculated ROI will be hard to calculate financially.

 

The XCRI Cap feed is a good concept, allowing students to ‘compare the market’ at a glance. In reality such comparisons across all HEI institutions is difficult, being a specialist institution we not only need to stand out, but sometime the standard measurements don’t and should not apply to us. Our specialist nature led to the formation of the Arts Assembly and the findings of the group have all been similar in terms of the challenges we face to get our voices heard. XCRI and  the work we have undertaken in this project gives Falmouth a head start – if and when aggregators do start-up we have done the extensive ground work and process development to be represented with the ‘big boys’ a position that without the Jisc funding we would not have been in a position to achieve.   

 

Conclusions relevant to the wider community

Very hard to get buy in from Marketing with no working or signed up aggregators. The XCRI feed is additional work and process which means there needs to be a driver to continue to maintain the feed and develop the process. For us the driver has been our commitment to Jisc outputs – but for others it will be very hard to gain any commitment for change without proven usage.

 

XCRI – CAP data still needs some work – if we are to adopt we need to formalise more of the fields so that the end data is usable by aggregators - more work is required between the community to define data.

 

Much of the process work undertaken has focused on quality of data and clarity of process rather than efficiency savings. This has helped to engage stakeholders who might otherwise have been reluctant to engage.

 

Academic restructuring – Universities continue to reorganise and restructure in light of the current environment, any restructure obviously has consequences on change initiatives like our CCMS/XCRI CAP project and as such to continue to impact on the value of such projects. At the moment Falmouth are going through an Academic restructure which may well alter the course creation quality process, which could then require  a substantial reworking of the ‘To Be’ process and CCMS system.

 

We had hoped that the CCMS system could eventually feed into our student records system, we now understand that the volume of work required to integrate the two systems would  not be worthwhile, significant investment would be required in an integration software tool, training and development and for the volume of courses created each year it would not be financially viable. We will continue to create validated courses manually, however the new CCMS system will aid this by automatically prompting the Student Records Team when courses reach validation.

 

Recommendations

 

General recommendations

Falmouth should look to utilise more programme management tools and techniques to ensure that resources spread across multiple change initiatives are planned and monitored effectively.

 

Recommendations for the wider community

Drive any XCRI CAP project from the Marketing department.

 

Recommendations for Jisc

• Continue funding the XCRI CAP community and development of the XCRI CAP standard.
• Provide and monitor who is utilising the XCRI CAP feed and report regularly on its uptake to encourage other institutions.
• Publish a list of aggregator sites that will be live from 1 April 2013.

 

Further details: email and contact names etc

Project Director: Dr Andrew Upton, Pro-Vice Chancellor (Learning & Teaching)

Project Manager: Johanna Smith, Business Information System Programme Manager

Contact email: Jo.smith@falmouth.ac.uk