Case study written October 2012.
Contents
Background
Business Case
Following on from the RAE2008 there was a high level of support at Principal Office level that the effort expended during the data gathering process was not dissipated and the experience gained was built upon. In practice this meant that there was an overwhelming need to redevelop the RIM systems in order to effectively manage research. Because the benefits were so dramatically more than the costs, a cost/benefit analysis was unnecessary.
Aims and Objectives
The whole project was intended to ensure that the University was able to run its research administration in a much better and integrated manner, building on the work already done during the REF to improve data management using the key principle of reuse of golden source data. A primary aim was to reduce the burden on academics and administrators during the REF2014 preparation and submission. Additional objectives included:
- presenting a dynamic and up-to-date public portal to the research taking place at St Andrews,
- reducing the burden on academics by reusing data in, for example, web pages and CVs
- improving data quality in other corporate systems, such as Finance (research projects) and Student Records (postgraduate students) by making the data accessible to the key stakeholders (the academics and administrators).
Context
The University had lost most of the expertise gained in creating the RAE 2001 submission which meant that the RAE 2008 was more testing that it might otherwise have been.
Following RAE 2008 a paper was created on the data used in the submission. The paper addressed the areas that went well and badly. The paper highlighted that:
- Some data were not fit for purpose
- Some business processes were not fit for purpose
- The systems holding research information needed further integration
- There were gaps in information, such as Postgraduate Research (PGR) students award dates
- Management Information (MI) was ad-hoc and for the most part held in spreadsheets
As well as highlighting the problems the paper also proposed a solution which involved:
- Addressing data quality
- Addressing business processes
- Joining up the systems
Additionally it addressed the need to be able to assure the quality of research data when the RAE was not an active driver i.e. eliminate periodic upheavals in favour of a continuous high level of accuracy.
Project Approach
It was realised that the introduction of a new Current Research Information System (CRIS) would not solve all of the issues that the University was facing. As well as a new system, in order to provide a full set of data, the existing data warehouse would need to be enhanced in order to provide better management information reporting across the University, not just for research. In addition to the reporting within the purchased system, reporting would be via a Business Intelligence tool which would use the data warehouse to supply outputs tailored to each set of stakeholders.
The University adopted a very simple selection process and also entered into a partnership with the University of Aberdeen, which was involved in a similar process.
Due to limited resource, an in-house development was not considered viable and, additionally, was not believed to be cost effective given the availability of fully functional commercial CRIS options (which was not the case in 2003 when the in-house system was developed.)
The project team that was created had already carried out some initial market research.
As a result the budget that was allocated to the project was based on the expected spend on the systems that the team had identified.
One of the requirements of the project team was that the solution was based on the Common European Research Information Format (CERIF) standard. This would support exchange of research information with other sector bodies e.g. HEFCE for REF, RCUK for outputs and outcomes.
In addition the Scottish Funding Council (SFC) has a strategy of research pooling across Scottish Higher Education Institutions (HEIs). The use of a standard data format would also support this strategy.
Two suppliers of CERIF systems, Atira and Avedas, were invited to demonstrate their product at the University and subsequently Atira and their product Pure were chosen to be the University's solution provider.
The system feeds the University's new research portal: 'Research@StAndrews' and is fully integrated with the University’s Institutional Repository.
Governance
The day to day project team was very small consisting of:
- A data architect from the Business Improvement department
- A member of the Research Office
- A member of the Library
Additionally there was input from HR. Registry and Finance in their separate roles as data stewards of Staff, Student and Project data respectively.
The project reported to a Steering Group which was chaired by the Vice-Principal (VP) for Research. As well as the members of the project team and the Vice-Principal the Steering Group consisted of:
- The Directors of Research from three of the schools covering a range of humanities and science
- The Director of Business Improvement
- The Director of Corporate Information Technology
- The Deputy Director of the Library
The project officially was part of the University's 'Programme to Improve Data Quality'. The VP Research controlled the budget though initially it had to be approved by the Quaestor and Factor (a post unique to St Andrews which equates to a combined director of Finance and Estates).
Technologies and Standards Used
In addition to CERIF compliance, the university also looking for a solution that matched its IT infrastructure, in particular ORACLE RDMS and Apache Tomcat application server and their LDAP (and later Active Directory) authentication systems so that users could use their existing University usernames and passwords.
The ability to integrate with the university’s established DSpace Institutional Repository to allow reasonably straightforward (and therefore cheap) extensions or changes to integrations with other corporate data sources, such as HR, Student Records and Finance were also very important.
Finally the solution had to provide a documented data model and secure web services to allow reuse of the data via SOAP and REST, and via standard SQL queries to extract information into the university’s reporting warehouse for internal and external reporting.
Summarising : the solution could not be a silo – the users needed to be able to get data in and out.
Establishing and Maintaining Senior Management Buy-in
The report on the RAE submitted to the Principal was enough to alert the senior management of the urgent need for action. By sanctioning the project and providing funding for it, the University’s senior management was implicitly expressing support. In addition the project steering group was chaired by the VP Research.
Outcomes
Usage Model
The entry of research information is at individual level by the researcher themselves or an assistant, and the University library has a quality assurance role for bibliographic data. Once entered into the system, by default, all published research outputs appear in the University's new 'Research@StAndrews' portal. The individual researcher can opt to remove publications from the portal and also make other publications, such as those ‘In Press’ visible.
Individual researchers have access to corporate project, personnel and postgraduate student data via the CRIS. Any queries concerning these data can be reported to a central helpdesk which directs each query to the appropriate data steward (Finance, HR or Registry).
The Library also provides resource to support and encourage researchers to deposit full-text versions of publications into the institutional repository, which is fully integrated with the Pure CRIS.
The Research Policy Office (RPO) will have the responsibility for generating management information from the system and generating performance indicators from it. All research-related statutory information will be generated by the RPO.
Achievements
The principal achievement is that the PURE system is now in use across the whole University and is a key enabler for the University’s preparations for REF2014. A dynamic institution-wide research portal has also been delivered linked directly to our open access Institutional Repository.
In addition the project has continued to deliver improved data quality, management information and reduction in the burden on departments and individuals by reusing single golden source data. Examples include the reuse of data via web services for school, department and research group web sites.
The university has also benefited by working closely with the other Pure Users in the UK (21 HEIs as of September 2013) and Atira to extend and improve the product including a fully functional REF module. In addition the CERIF-compliance of the Pure model has been beneficial in working with others in the UK research community, e.g. RCUK, on work to improve data exchange across institutional boundaries; again reducing burden on academics and administrators by reusing single golden source data and ensuring better data quality for all.
Benefits (tangible)
The PURE system has been generally well received by academics, but it took time before the impact it would have on the preparation of the REF was well understood.
The REF is a key factor in the lives of a growing number of researchers, and PURE is supporting the schools and the research policy office that are making preparations. For example, for the most part the need to email spreadsheets has been eliminated although there are still issues with the flexibility of the Reporting Module. The schools are particularly pleased with the ease of access to all the data, (including from other central systems) that they need to help them prepare for REF. Pure is also providing additional help in preparing joint submissions where both Institution’s are Pure users.
Pure has also facilitated a fairly substantial increase in full text deposits into the Institutional Repository – from 50 prior to the launch of Pure in mid-2010 to nearly 800 now – 600 of which are open access; the remainder are subject to copyright restrictions or time embargoes.
This has been possible both because of the integration between Pure and the IR and the cross-unit team (RPO, Library and ITS) who support and promote Pure and OA as part of one service for Academics. Engagement with Pure means that authors are already familiar with the system and depositing an output is a very easy additional step.
Improvements such as integrated Sherpa/Romeo data and our ability now to tailor this information in the Pure interface helps simplify even further – along with the project team’s message that the Library can manage the policy checking etc. Having publication metadata in Pure also helps significantly with being able to proactively add full text, and there are additional opportunities e.g. during the Research Output System (ROS) work on linking projects to identify outputs for green open access. The proposed ‘repository’ developments in Pure e.g. OA flag will help drive visibility of Open Access.
The university was able to upload data to the new Research Councils Research Outcomes System (ROS) albeit via querying data directly from the Pure backend. This did reduce burden for academics since they could make the links between outputs and grants in PURE and this information could be reused. Atira are adding functionality to improve future information exchanged with ROS, as part of the Cerif in Action project.
Data is being reused on more than half of the School and Department websites and many of those remaining have expressed a desire to move this way also. Thus academics and support staff are no longer having to maintain separate static web pages of content.
Data is being reused by Research Pools in Scotland. For example, the Marine Alliance for Science and Technology Scotland (MASTS) is developing a website to include names of researchers and their publications details from member partners. The five members who are Pure users are collaborating to provide the information in the same format.
The project has also improved several data maintenance processes within the University as a result of the visibility of the central data, e.g. the process to change supervisor for postgraduate students.
Underpinning the successful realisation of these benefits has been the excellent communication and cross team working within the Institution; in particular the core team from RPO, Library and ITS. This itself has delivered efficiencies in the project and is an exemplar for any future projects.
Benefits (intangible)
The CERIF-compliant nature of Pure means that the university is well placed to provide data in the recognised standard which has been adopted by Research Councils and others in the UK to improve data exchange and quality throughout the community.
Working as part of the Pure UK User Group has led to sharing of burden across the Institutions in, for example, developing the REF Module, improving Reporting and improving the Activity module to support the ROS requirements. It has also led to considerable exchange of best practice in areas related to research information management.
Research and Senior Managers are able to access rich data about research from across the entire Institution from a single portal; with these data being updated dynamically as academics update their profiles.
Key Lessons Learned
Project management Issues/Opportunities
Working on a joint project with Aberdeen was beneficial in that the institutions could share ideas and issues and split tasks between them. This included all stages of the project including the procurement, data modelling, implementation and testing phases.
Being the first to implement Pure in the UK had the advantage that the project team could work closely with the supplier, Atira, on the data model and functionality to suit their needs; although, equally, there was a downside as they spent considerably more time on this phase of the project compared with Institutions who followed. The project has, though, experienced benefits in that there is now a very active Pure UK User Group that works closely with Atira to continually improve and extend the data model for the benefit of all Institutions.
Organisational Issues/Opportunities
Implementing the CRIS which brings together corporate data from several central units – HR, Finance and Student Records has continued to improve data quality across the Institution and embed the principles of good information management. Although the CRIS is managed by the central Research Policy Office, there is a sense of shared Institutional ownership and responsibility, with the central units and the Library. The latter work closely with RPO and the Schools to promote the deposit of open access articles and carries out bibliographic metadata checking.
The senior management at the organisation has also recognised that a CRIS is a key corporate system along with the traditional systems, such as Student Records, Finance and HR.
Hard/Technical Issues/Opportunities
Having a central data hub (or warehouse) has been critical to the implementation of the CRIS as it has allowed us to reuse data from the university’s other corporate systems.
There are issues around managing deletion of data within the source systems and ensuring these flow through correctly to the CRIS. The project team has been able to promote the reuse of data across Schools and Departments via the web services provided by Pure which has in turn improved our data quality by highlighting errors in central data and allowing RPO to work with colleagues to improve data maintenance procedures. It should be noted that these errors are sometimes down to data entry error but also can be due to processes and definitions not being clearly understood by Schools and Departments.
Soft/Personnel/Process Issues/Opportunities
Schools and departments have been involved in using the CRIS from the launch with the goal to make the Portal publicly available. However there has been the need to provide considerable training and promotion to groups and individuals to continue to encourage the use of the system. REF2014 and the OA funder mandates provide the biggest sticks at the moment but RPO encourages the ‘carrot’ element by promoting reuse of the data e.g. for ROS, for Research Pools and for use on staff and departmental web pages. There is a CV module in the CRIS which the project hopes will provide further functionality specifically to support the individual researcher but this has not been promoted due to REF2014 taking priority, and it needs further development to make it truly useful.
Having one system/interface to provide data on all research activity and outputs makes it very easy to encourage academics to also deposit full-text items.
Looking Ahead
Sustainability
The CRIS is accepted as a critical corporate system for the Institution and provides the framework for the project to continue to develop support to other areas of research management, particularly in the areas where the Institution is working with funders to comply with policies and directives relating to research outcome and impact (e.g. via ROS), research data management and the use of equipment and facilities.
Future Plans
In addition to the above they are also reviewing the area of research grants management and, in particular, whether and where the CRIS could help improve the university’s processes by capturing data once for multiple reuses and increasing the internal visibility of data.
