Jisc case studies wiki Case studies / Course Data - Middlesex University
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

View
 

Course Data - Middlesex University

Funded by the: Jisc e-Learning programme.

Lead Institution: Middlesex University..

Learner Provider Type: Higher Education

Project Duration: January 2012 - March 2013

Key Words: Course Data

Case study tags: course data, kis, hearcourse information, stakeholder engagement, middlesex university

Note: This is an abridged version of this project's final report.  The full version is available here.

 

 

Middlesex University Skills and Education Planning Tools (MUSKET) – Improving Course Information Flow (ICIF)

 

Middlesex University

 

 

Project Summary

The MUSKET-ICIF project provides the means for enhancing the creation and manipulation of course data. More specifically, the project aimed at (i) producing tools transforming information for academic courses that may be in the form of structured documents into the XCRI-CAP information model, (ii) generating XCRI-CAP feeds from course data that reside in the institution’s existing systems, and (iii) providing training courses and supporting resources for the dissemination of the role of XCRI-CAP in the management of course data. A range of stakeholders, including marketing of new and existing courses, registrar functions, career service and open day events as well as curriculum design, development and delivery procedures, could use the project’s outputs.

 

A common problem for all academic institutions is having a consistent approach in the way course data are being used. There is no harmonisation when it comes to creating course descriptions as different departments and services may have different templates that they are using. There is also a range of programmes that may not fit the typical three-year undergraduate programme format, such as CPD courses, work based learning modules, APE modules and individual training sessions. There are several factors why institutions fail to align course data being created at different sources, including the use of several systems, the fact that different departments may focus on certain data elements, the format being used for the creation of data and the type of data needed for each operation.

 

The project has produced a solution for the effective and efficient use of course data cross the institution. The project’s outputs included (i) a set of tools that can be used to transform any structured-document that contains programme or module specification into the XCRI-CAP model that defines each element of course data, (ii) an XCRI-CAP feed for a range of courses offered by Middlesex University based on course data residing in the existing database (PIP) used by the University’s marketing unit, and (iii) a range of resources including accredited course at postgraduate level, with supporting documentation for sustainable training in the relevant technologies (i.e. XCRI-CAP).

 

What did we learn?

The MUSKET-ICIF project was a rewarding learning experience as it allowed the project team to assess the opportunities and the constraints for the project’s activities and intended objectives. On one hand, the project received the necessary support and endorsement from senior management while on the other hand technical and system issues affected the project. The main findings could be summarised as follows:

 

  • The self-assessment exercise helped the project team and the stakeholders to evaluate the institution’s readiness for XCRI-CAP and a course data project. The Green/Amber/Red mapping of each issue ensured that the team had a clear starting point for the project. The project team produced to self-assessment maps, as part of a Jisc funded study and the first stage of the programme. The team has early evidence of improved results with respect to the self-assessment questions following the creation of the XCRI-CAP feed and MUSKET-ICIF tools.
  • The main source of course data is the PIP database used by the marketing unit. The customised database is based on the institution’s MISIS system that contains all necessary data for each credit bearing course. Still there were courses that were not included in the main system, mainly because they were not accredited or did not fit in the typical course provision. The project team had identified a list of course data elements that were missing. Some of these data such as postcode and contact details were consistent across the entire course provision. Eventually certain fields were identified as common between courses, meaning that there was no need to change the original database to provide course data that did not dynamic updates.
  • The Tridion system that is used by marketing for CRM purposes is customised to meet certain needs and could not provide the necessary data for the course data generated. This meant that the generation of the XCRI-CAP feed could become possible only by following one of the two possible options. First, the Tridion developers would have to redevelop the system to produce the missing data, which has proven an expensive option for the project’s budget. Second, the PIP database should be used and the missing fields identified, should be filled by data retrieved from alternatives sources. The second option was successfully followed. The produced feed does not have any missing elements and is validated.
  • The initial discussions with stakeholders were time consuming and challenging as it was quite difficult to align schedules and workloads but eventually it was possible to complete the survey with input from all stakeholder groups as identified in the original proposal. It was necessary to provide additional demonstrations and presentations to key stakeholders to explain how XCRI-CAP might affect the generation of course data.
  • The MUSKET web portal took very long to set up due to security concerns and even domain name issues. The main problem was allowing access to the tools outside the University security systems. This delayed testing as an additional server was set up to test the earlier versions before uploading the finalised versions. The website problems affected the blogging progress of project activities. Another issue was initial delays in accessing MUSKET-ICIF tools for demonstrations and testing.
  • The tools had to be reviewed and APS.Ltd participated in the testing of the results generated by the algorithm used for semantic analysis. The thorough testing reports showed initial inefficiencies and inaccuracies and also provided useful insights on interface and usability issues. The tools were revised and currently usability heuristics are fully adhered to and the comparison results between courses are fully tested. 
  • The XCRI-CAP feed once all fields were filled with actual data was validated with the validation service with no problems. The feed is also uploaded on the aggregator service. There were additional feeds generated for different course types and each one of them was validated with no errors. It is necessary to ensure that the project officers were capable to produce the required code.
  • One of the most challenging aspects was to test the MUSKET-ICIF algorithms and ensure that they were described in a way that can help less-technical staff to become aware of how the semantic analysis takes place. The team still experiences difficulty in disseminating the algorithm details due to the technical nature of the work.
  • Finally the creation of a CoolURI proved a challenging task as it required the collaboration of various stakeholders. It was decided to maintain a domain within the MUSKET tools for two reasons. The MUSKET portal will be maintained for at least five years and also the work in this sector as part of Middlesex projects that have been funded by Jisc and other bodies attracts more interest and views. The project team will also create further CoolURIs as required by future aggregator services.

 

Immediate Impact

From its early stages, the project has identified a number of areas where the project outputs would have an immediate impact. The obvious area for improvement is the institution’s marketing as it can be based on more effective use of course data. The focus of the XCRI-CAP information model is to improve the way course-advertising profiles are created and accessed by applicants.

 

However the MUSKET-ICIF project has produced outputs that can have a clear impact on the following areas:

 

Admissions –

quite often departments have combined honours or major-minor programmes that are very similar to each other. Quite a few programmes may also share the same core modules, making it quite difficult for applicants to become aware of the differences between programmes. To add to the confusion, cross-discipline curriculum design may lead to programmes that are very similar and difficult to distinguish. A common pattern has emerged with applicants asking the same questions during open day events, applicant day events or even via email to programme leaders. Such questions aim at identifying the differences between similar programmes with respect to the topics covered, the learning outcomes, skills gained, technologies or specific aspects taught and even job prospects after graduation. The mapping of course descriptors against certain XCRI-CAP fields allowed the alignment of programme aspects such as modules taught, learning outcomes and career prospects. It means that students are able to understand differences between similar courses and perhaps the impact of their decisions when they wish to change modules or transfer to another programme.

 

Curriculum design –

the final few months of the project coincided with the validation of all the undergraduate provision in the Department of Computer Science. Typically, the design of a new programme or the review of an existing one, takes under consideration a number of factors that may affect the programme structure and content. Some of the recent consideration of the validation team included (i) establishing a clear programme scope, (ii) attempting to target a specific segment of the student market interested in the area of each programme, (iii) avoiding significant overlaps with existing programmes, and (iv) providing competitive programmes compared to existing programmes from HEI competitors. The use of XCRI-CAP allowed creating a consistent representation of programme structures regardless of discipline. The creation of XCRI-CAP feeds from a critical mass of institutions ensured that the Middlesex feed could be used by aggregator services in the future. The project’s online tools allowed the like-for-like comparison of entire programmes or a selection of their headings that correspond to specific XCRI-CAP fields.  The main benefit is that the project team has collected programmes from other institutions that are included in the project’s repository that continuously grows. The repository is used for comparing programmes that are transformed to XCRI-CAP. One of the future steps is to integrate the online tools with aggregator services that utilise the XCRI-CAP feeds produced by projects participating in the course data programme. Another benefit for curriculum designers is that it is possible to use the visualisation functions to assess the extent to which certain programmes are similar, making better-informed decisions when it comes to programme content.


Programme delivery –

the use of XCRI-CAP allows the creation of programme pathways and facilitates mapping similar programme components such as modules and units. This ensures that when significant similarities are identified between two modules this can be used when delivering a programme with optional modules. The use of XCRI-CAP can be also used to determine pathways including pre-requisites, possible module options and alternative programme structures. This is achieved by transforming the programme content into XCRI-CAP and searching for certain criteria in order to identify potential programme paths. The MUSKET-ICIF tools are currently being extended to map course information to career prospects and possible study pathways.

 

Marketing –

the MUSKET-ICIF tools are currently being demonstrated to the project stakeholders. Their role can be critical to the institution’s efforts to pull course data and better manage necessary course advertising information. The use of the XCRI-CAP feed allows a consistent view of all programmes offered by the institution, while the online tools support the identification of marketing aspects of courses that do not fall under any of the traditional undergraduate programmes offered by Middlesex. Currently the use of the developed tools allows the classification of different programme details according to how suitable they are for different marketing objectives. Furthermore, the use of XCRI-CAP allows the generation of information that could be used for both KIS and HEAR initiatives….

 

The project has made a significant contribution to the institution’s marketing resources as well as several operations at School level. The project’s relevant outputs include:

  • The XCRI-CAP feed that enhances the use of course data.
  • The MUSKET-ICIF tools that support tasks such as curriculum design, career service, registry and marketing.
  • The training resources that facilitate the adaptation of the course data programme resources, meaning that it is much easier for Middlesex staff to become familiar and start using XCRI-CAP.

 

The project has helped the wider community to become aware that XCRI-CAP has many more potential uses and that the creation of a course data feed is only the foundation for further developments. The project team as with previous projects and toolkits has tried to reach as wide an audience as possible to ensure that institutions participate in discussions of how to utilise the XCRI-CAP feed in the future.

 

There is a clear impact on the wider community and evidence includes the reception the project outputs have received so far. The project tools are currently reviewed for potential use from both marketing and career services. The participants in the training events, XCRI-CAP courses and presentations both internally and externally have followed up with possible collaborations and future extension of the tool functionalities. The project team has identified a number of institutions interested in investigating possible applications of the XCRI-CAP transformation and course data generation tools.

 

Further evidence of the project’s success include the validated XCRI-CAP feed and the exhaustive evaluation performed by APS Ltd on the accuracy of the XCRI-CAP tools in producing comparable course data. The project has also triggered discussions with CLTE in terms of further marketing and employability activities that can be reviewed in the light of the developed tools and the project’s XCRI-CAP feed output. The project also feeds in the MUSAPI project on embedding benefits for linking job profiles and specific job opportunities to course data.

 

The above observations have affected how the project is now perceived at institutional level. Initially stakeholders expressed concerns regarding the resources needed for the project’s activities and challenged the usefulness of the project’s deliverables. There was some initial resistance whether the project would antagonise the KIS initiative. The project has now been used in (i) preparation for the validation of the Computer Science Department programmes, (ii) comparative analysis of course data across the University, (iii) semantic analysis of course data from other institutions and (iv) support the curriculum design of a Joint Masters is Software Engineering as part of a Tempus project funded by the EU.

 

Future Impact

The project has produced outputs that are well documented and could be used in a sustainable way in the future. The project contributions can be classified under the following streams:

 

XCRI-CAP feeds –

the main feed provides n XML based view of course data for any course that exists in the Middlesex University course repository. The impact on marketing operations and administration activities is significant as it can completely transform the way course data is being used. We expect the marketing unit at institutional level as well as other School level operations to utilise the feed for producing customised, applicant focused, real time course data. The creation of additional feeds for specific types of courses aims at providing the means to summarise and manipulate data from courses that may not adhere to the structures and do not use templates of the institution.


Course data tools –

these tools are focused on educational planning and more specifically using the MUSKET-ICIF tools to transform course documentation into XCRI-CAP format. The tools also support the semantic analysis of course data and comparison between courses. This stream will have a dramatic impact on the decision making capabilities of staff, students and applicants in a number of ways including (i) curriculum design, (ii) course review, (iii) open day events with respect to presenting course pathways, (iv) direct entry decisions, (v) marketing research for new courses, and (vi) student transfers between programmes. The project team will be monitoring and analysing the use of the course data tools and will be performing further evaluations during dissemination stages. Emphasis will be on identifying patterns of course data analysis and the usefulness of the tool functionality.  


Training resources –

the project has produced a series of short courses with supporting documentation. These courses are accredited for six years and a virtual learning environment is created to support participants to these courses. Three instructors (Alan Paull, Kirstie Coolin and George Dafoulas) have been identified for providing future support for the training courses. Participants in these courses follow a work based learning approach, which offers self paced study that matches individual needs.  Currently the project team investigates the possibility to publish a course textbook for XCRI-CAP. The participants to these courses will be participating in an XCRI-CAP community of practice to ensure that a significant number of institutions that participated in the Jisc programme maintain the same momentum.

 

 

Conclusions

The project team had the opportunity to reflect quite often during the project’s lifecycle. The project development team assessed progress on a weekly basis while the stakeholder group had periodic meetings every six to eight weeks. The emphasis of these meetings was on assessing whether the project was able to implement its deliverables and also on evaluating whether the project’s aims were being achieved.

 

We could identify the following project-specific conclusions:

  • The project was initially based on performing the XCRI self-assessment, which meant that the project team required access to stakeholders who were kind enough to provide their input. Such an exercise may prove futile if a project does not have the full support from senior management. The project had an impressive level of contributions and participation in the initial workshops but it was difficult to align schedules to ensure that every stakeholder could attend a single meeting. Therefore, an acceptable compromise was to hold a number of meetings until an accurate XCRI self-assessment was reached.
  • The project had to overcome initial scepticism in terms of how the XCRI-CAP technology can transform the way course data are being created. The main difficulty was due to the fact that the XCRI-CAP technology is not widely known and that there is a steep learning curve required to understand how the information model can be used. This is the reason why the project produced the short courses and associated materials to support further dissemination of the project’s deliverables and establish a critical mass of staff that is trained in XCRI-CAP.
  • The project has produced the required XCRI-CAP feed, a valuable resource for the institution’s marketing operations. The project also identified other areas across the University where the generated feed could be used. It is necessary for the institution to identify how to better use the generated XCRI-CAP feed in a formal way for the production of its marketing materials based on course data.
  • The project’s course data tools have provided an integrated solution that can be used by non-technical staff that need to generate XCRI-CAP feeds from individual course description documents and use the generated feeds to compare different courses with the aid of semantic analysis and visualisation of analysis results.

 

The wider community can also benefit from the project’s outputs that were at first created with institutional benefits in mind. Early evidence from the participation of other institutions reassured that there is a need, which can be fulfilled by the project’s outputs. More specifically:

  • Institutions that have not yet received support from Jisc via the course data project can use the MUSKET-ICIF tools to achieve the generation of an XCRI-CAP feed.
  • Institutions that are able to produce their own XCRI-CAP feed can utilise the MUSKET-ICIF tools for comparing data from different courses.
  • The entire sector can benefit from the existence of a repository of XCRI-CAP feeds in the form of an aggregator service layer to existing higher education sector data.
  • The course data generated in the form of XCRI-CAP feeds could help with the mapping of UK higher education provision to international programmes and initiatives such as the Bologna process.

 

The project received invaluable support from Jisc and its resources and has attempted to add value to the course data programme in a number of ways:

  • Jisc may choose to include the courses developed and the associated materials as a toolkit for institutions that require the introduction of XCRI-CAP in their course data management operations.
  • Jisc may be interested in supporting the dissemination of the MUSKET-ICIF publications (e.g. project output posters, tool demonstrations) to explicitly show the possible applications of the XCRI-CAP feeds.
  • Jisc may use the XCRI-CAP feed along with the ones from the other institutions participating in the course data programme in a single aggregator service maintained as a national repository for course data.
  • Jisc may use the integrated MUSKET-ICIF tools to provide a demonstration for the possible applications of XCRI-CAP with emphasis on course data management, registrar functions and employability services.

 

Recommendations

Following the completion of the project’s activities there are several recommendations that can be made. These can be classified according to their focus on:

 

Institutional level

  • The institution should review its marketing operations and assess the most feasible way for integrating the generated XCRI-CAP to the existing strategy and methods.
  • The institution should consider whether further development is required to the existing functionality of the course data tools for applying XCRI-CAP to employability and career services.
  • The institution should investigate further the deployment of the XCRI-CAP feed applications across schools.
  • The institution should consider whether the MUSKET-ICIF tools could further support its KIS and HEAR initiatives in generating rich course data.

 

Wider community

  • The sector should take the opportunity and use the MUSKET-ICIF resources that are freely accessible and available through Jisc to examine the possible application of XCRI-CAP.
  • Institutions that are in need of further support in developing interfaces for utilising the XCRI-CAP feed produced by the course data programme could integrate them to the MUSKET-ICIF tools.
  • Individuals or institutions that require further training in XCRI-CAP could receive support from the MUSKET-ICIF PGCert programme.

 

Jisc specific

  • It is very important for the course data programme to use the momentum gathered by building a critical mass of institutions with XCRI-CAP feeds and press forward with the utilisation of aggregator services.
  • The produced training materials on XCRI-CAP should be also used to disseminate XCRI-CAP further within institutions but also across the sector.
  • The MUSKET-ICIF tools should be exploited in a series of demonstrations and dissemination activities to kick-start an investigation on extending the role of XCRI-CAP feed in further operations.

 

Further details: email and contact names etc

 

Project Director - Balbir Barn/Martin Loomes/Nicky Torrance

Project Manager - George Dafoulas, g.dafoulas@mdx.ac.uk

Project website - http://www.musket.mdx.ac.uk/

MUSKET Tools videohttp://www.musketwebtools.mdx.ac.uk:8080/musket/web3.0/index.html