Jisc case studies wiki Case studies / University of Newcastle - Research Information Management
  • If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

University of Newcastle - Research Information Management

Case study written October 2012.

 

Contents

 


Background

 

Unlike the majority of universities, the University of Newcastle has chosen to build large parts of its Research Information Management infrastructure in-house. The in-house system, known as MyImpact, was seen as a first step to a larger development of a comprehensive support structure for the research domain.

 

Context

 

In the wake of the 2001 Research Assessment Exercise (RAE) the Faculty of Medical Sciences built an in-house Institutional Repository, called MyProfiles.

Subsequently the repository was rolled out across the entire University. The system acted as a hub for other research systems such as: 

 

  • Research Awards
  • Postgraduate Research (PGR) student registrations
  • Human Resources (HR) information


This, in turn, fed researcher’s Personal Plans with an initial set of data to which individuals could add reflective information on research, teaching and other activities.

Personal plans  are used for annual appraisals and, as well as historic data, contain a section on future plans which can be completed by the researcher. Additionally, Personal Plans feed web profiles which, amongst other things, show each individual's top six publications. Interested readers can, if they so wish, then access other publications by following a 'More Publications' button at the bottom of the list.

The system provided good Management Information and has functions to group staff together and report on their outputs, PGR students and research projects.

For RAE 2008 a Research Information Management system was built that took information from MyProfiles and other corporate (SAP) systems. The key publication data was provided by MyProfiles. This allowed individuals to be formed into Unit of Assessment (UoA) groupings and profiles of the groups built. The system allowed individuals to be flagged as eligible/ineligible, submitted/not submitted, etc, with in-built functionality to scenario plan for all aspects of the RAE. This in turn, allowed the University to carry out sophisticated modelling and anticipate the RAE results before they were submitted. There was a significant amount of planning prior to submission of the data to the Higher Education Funding Council for England (HEFCE). The system generated Extensible Markup Language (XML) to feed the HEFCE RAE data collection system. Most importantly it brought all the information on an individual into a single 'location'.

For the RAE, reports were sent to individuals showing all their data that was to be submitted (papers, income, PGR). This received a large, and mostly positive, reaction because it was the first time that all the information had been presented in a single (unified) format. However publication data in the system was frequently incomplete as it relied upon individuals accessing MyProfiles and manually adding in the bibliographic details of research outputs.

MyProfiles is now regarded as a mature system and any development has now been frozen. Only essential work is permitted. It is, effectively, an historic data warehouse.

 

The business case

 

MyImpact is the successor system to MyProfiles and builds on the foundation of its predecessor. MyProfiles had been built as a system in one of the faculties to meet the needs of the RAE. Subsequently the MyProfiles system had been made the basis of an institutional system. While MyProfiles had been very valuable there were several issues with this approach:
 

  • The system had started its life in a single faculty and scalability was proving difficult
  • It utilised technology that was not widely understood or supported within the university


As a result developing the system was problematic and a business case was developed for its replacement.

In late 2007 the University Research Office submitted a bid to the University Executive via the Finance Committee to replace MyProfiles with a more robust, maintainable and scalable system to be known as MyImpact.

The submitted proposal considered and rejected the feasibility of a proprietary solution on the grounds that the functionality that was available at that time was insufficient for the University's needs. Any bought-in solution would require so much customisation and development that it would be very difficult to implement supplier upgrades etc, and consequently an in-house solution was chosen. It was also acknowledged that MyProfiles already provided much of the functionality required and in some aspects was better than the commercially available alternatives.

The proposal contained a cost benefit analysis that was very favourable and the project was granted £348k for an in-house development (the £348k was not a Full Economic Cost (FEC), as project management and other 'fixed costs' were not included).

Aims and objectives

 

The University is engaged in a large-scale rolling development of its Research Information Management (RIM) systems with a view to providing a comprehensive infrastructure that supports:
 

  • Improving its ability to meet its statutory reporting obligations
  • Providing a better service for its research community
  • Providing comprehensive management information for research managers
  • Providing comprehensive information of the University's research landscape to the wider community

 

Key drivers

 

Principal drivers for change underpinning this development are (in order):

 

  • Research Excellence Framework (REF)
  • Improving management information
  • Improving research performance
  • Streamlining of systems and processes
  • Improving data accuracy

 

Project approach

 

The development resource allocated from the budget for MyImpact was an information analyst with two developers.  In developing the system the team (led from within the University Research Office) were mindful of the key deliverables but also the existing processes of the university.

Common European Research Information Format (CERIF) was not a consideration when designing and developing the system, but in light of recent developments, is now being considered. However the ability to interact with Research Council and HEFCE systems is seen as key drivers rather than the broader CERIF landscape.

Integration with other corporate systems was difficult and a significant portion of the budget was spent developing an 'individual linker' to uniquely identify an individual to the various corporate systems that interface with the research domain. By resolving the issues around identifying an individual the developers  allowed MyImpact to leverage information that was until then not easily available.

 

Scope

 

The project has been designed to be phased with the first phase concentrating on Research Outputs and then moving on to Project Proposals. Eventually the system will expand and link to other systems to cover the entire research system domain.


The MyImpact system will eventually be the single place of truth for all research information. It already has this status for publication and output data. The system takes information feeds from a variety of sources both internal and external. For example the University has bought upload licences from both Web of Science (WoS) and Scopus to ensure maximum coverage of publications with the minimum of staff effort.

The system is predicated on the individual researcher. For reporting and analysis  larger groupings, such as departments, are then aggregated from the individual data. By adopting this approach in addition to internal structures, external structures such as UoA can be imposed on the data.

 

Governance

 

The project is run from the University Research Office (URO) with the Head of URO as the project manager. The project reports to the Research Committee. The project champion is the Pro Vice-Chancellor (PVC) Research and Innovation and principal stakeholders have been identified as:

 

  • The University Research Office
  • The Library
  • Researchers
  • Research Managers

 

Outcomes

 

Usage Model

 

In the past the responsibility for quality assurance (QA) of outputs entered into the system rested with the individual researcher. However this caused a number of difficulties and the responsibility now rests within the library where the staff have  a clearer idea of the issues involved and are able to give the task greater attention. This change of emphasis has resulted in a significant improvement in data quality.

An output is entered into the system by either an academic or a secretary within the academic department. There is then a grace period of two weeks during which the academic can amend the item before it is 'passed' to the library for QA. The library also checks the system for duplicates entries (eg joint authors both entering the output).

Once everything is correct, the status is changed to 'approved' and finally to 'published'. There is an option which allows the researcher to inhibit the visibility on the web in which case the output remains 'approved' but not 'published'.

Every two weeks there is a web crawl of Web of Science (WoS) looking for articles with a 'Newcastle' author. If previously unknown articles are found they are sent to the academic asking them to confirm that they are the author and, if so, whether they want to add the article to their publications. If no reply has been received after fourteen days they are reminded and if, after a further seven days, there is still no reply the article is 'locked' by the Library, checked and then added to the individual's list. It is then down to the individual to make it visible on the web. There have been problems identifying an individual as they can use variations on their name (e.g. Anthony, Tony, A, T; may all refer to the same individual) but over time these are becoming better understood and consequently better managed.

If the researcher wishes the full text of the output to be made publicly accessible, they can upload it easily into MyImpact and it is then managed by the Library's dedicated e-Prints officer (copyright checked, etc) prior to being placed in the e-Prints repository.

There is an existing institutional directive that this should be the default option but there is limited ability to enforce compliance and it is currently at about 15%.

The system will produce much richer and more relevant Management Information. Much of the information in MyProfiles was textual and subject to various different interpretations but MyImpact is much more structured and consequently the quality of the data and therefore the accuracy of the reports is higher.

 

Research Excellence Framework (REF)

 

As a major business driver for the development of the system was the REF, unsurprisingly, MyImpact contains an array of dedicated  REF functionality. The evolving process model for this functionality is complex with input from researchers and various levels of academic management to collect the base data; this is then collated, modelled and will be submitted by the Research and Enterprise Services.

For all REF eligible staff the system identifies all their outputs within the REF period. The researchers then go into the system and use it to identify their top four outputs and score them using the REF classifications. There is a facility to attach notes to explain why the score was given e.g. prestigious publication, large number of citations etc. This is then peer reviewed within the faculty by an internal panel and the scores can be changed if they disagree with the original assessment. If this happens the notes are updated to explain why the change took place. Finally the dean reviews the scores and again can change them if he feels it is necessary. The whole process is designed to be very transparent, the researcher can see any changes made and the explanation of why. From this it is possible for the University management to decide who and what  will be submitted to the REF.

There is a REF administration and modelling module in development.  As well as showing the likely REF submission it allows additional data to be appended to the individual such as ‘Early Career Researcher’, supervised students etc and allows staff to be grouped into Units of Assessment (UoAs). Depending on the HEFCE submission system (currently not yet released) the system will generate XML for the actual submission itself.

 

Project Management Function

 

The University has an existing research grant management system (MyProjects) for the detailed management and tracking of research grants and projects;  data is pulled from this system into MyImpact. By populating MyImpacts with this data researchers are able to keep track of their applications, and awarded projects from a single point of reference. The system displays a summary of the information which is held in MyProjects and  tracks an application’s status, the percentage of the funds that the researcher will receive, the finance system reference etc.  If the researcher wishes to drill down into the detail of the project and track expenditure etc they can do so.

The system also provides a detailed audit trail of the activity of the project; for example  all email exchanges are logged and the administrators can log completed actions and tasks relating to the project that they have carried out. All externally funded research projects are held in the system.

In addition to project management, the system drives external reporting such as the HEBCIS return  as well as patent management.

 

Post Graduate Research (PGR) Students

 

For each researcher, MyImpacts draws information on the research students they supervise from the SAP student record system. For an occasional, none-expert user, SAP is non-intuitive, and the data is very complex and difficult to manipulate. MyImpact simplifies the structure and renders it in terms relevant to research therefore making the data much more comprehensible. For each PGR students that an individual researcher supervises, the system shows detail such as the percentage split of the supervision, name and stage of the student, award date (if relevant) etc.

 

Achievements

 

The project has led to a significant improvement in the University’s ability to manage its research activity.

  • MyImpacts is now embedded as the primary and definitive source of data for much of the University’s research output information.
  • The process of gathering research outputs has been greatly simplified and by passing the responsibility for the outputs to the Library made more rigorous and less error prone. Previous problems such as duplicate entries and mis-assignment of outputs have been greatly reduced.
  • It is the definitive tool for collecting data for the REF and will provide modelling and reporting tools to allow the University to accurately predict their REF submission and review amend it as necessary before submitting it. Depending on the submission mechanisms provided by HEFCE it is anticipated that the system will also create that actual submission.

 

Tangible Benefits

 

The development of a holistic approach to research management in general and MyImpacts in particular has produced significant benefits to both individual researchers and the university:

Research staff

 

  • Less data entry
  • less confusion
  • an easier to manage personal profile

 

University

 

  • More efficient processes
  • greater consistency in research data
  • higher quality of data
  • much better REF processing (data already all in one place and one format and much of the selection done on a continuous basis)
  • greater visibility of research outputs
  • vastly increased amount of accurate MI

 

The use of the system is understood by researchers and there is a high level of acceptance of it.

The system produces  a number of reports at a variety of different organisational levels from individual to faculty. These show published research outputs, the number of research students being supervised, the number and value of research grants awarded etc.

MyImpact now feeds into a detailed report known as the ‘Personal Development Report’ and the personal development process is largely informed by the report.


Key lessons learned

 

Project management Issues/Opportunities

 

A project of this size and complexity needs to be overseen by a competent, experienced project/programme manager.

In addition a system or data architect would be helpful in order to ensure consistent, coordinated development.

 

Organisational Issues/Opportunities

 

Though the system is now recognised as the single point of truth this was not achieved without a struggle. In all large organisations there is inertia and opposition to what are perceived to be new and intrusive methods no matter what the motivation.

MyImpact is part of a larger rolling programme and the larger system is still incomplete.

Ultimately  it needs to be expanded to include all data about all research staff and research activity.

The system has, rightly given its importance, been moved onto corporate servers and the development and maintenance resource absorbed into the central IT department (ISS). This has been good for the security and resilience of the system but has impacted on the system development.  This is caused by several factors:
 

  • Historically ISS have predominantly dealt with student , Finance and HR systems and it took them some  time to understand the research business processes and its importance to the institution.  Neither did they initially grasp the high degree of reliance there is on the availability and usability of the MyImpact system and its related, research domain systems.
  • There are now 3 FTEs to manage all maintenance and development but the demands for resource outstrip availability.
  • There is resistance to bringing in outside resource on a fixed term contract on the grounds that it would not be effective due to the steep learning curve that an incoming developer would encounter.
  • The chosen development methodology is agile. While this is very good for delivering small sets of functionality that accurately reflect the business need, at times the methodology conflicts with the need to deliver a set amount of functionality by a given and unmovable deadline.


As a result there are continuous discussions on prioritisation and the importance of maintenance against new development.


Looking ahead

 

Current Developments

 

Currently the MyImpact system deals with many aspects of research outputs and outcomes, and the MyProjects system covers the current applications and awards/projects. A new piece of functionality to be known as MyProjectProposal is being developed.

MyProjectProposal  is intended to bring major efficiency gains by replacing the existing business process and the Pfact system with a more streamlined and efficient process.

The system will build a workflow that will minimise duplication and manual processes and wherever possible draw data from other corporate systems such as HR and finance (both SAP). As part of the workflow, the system will replace ‘pen and ink’ sign off with a digital signature process which will further reduce the need for hard copy.  

The purpose of the system is to support the development of a proposal to a funder, such as a research council. The user selects which funding body is being approached. The system then allows the value of the proposal to be built up by selecting costs initially at staff/non-staff and then at an individual level using actual payroll data. It also supports the selection of additional items such as:
 

  • Whether ethics committee approval will be needed and if so the current status
  • Insurance details
  • Estate requirements and costs

 

Also, in the same way as for MyImpact, an audit trail/activity log is maintained showing all actions and who carried them out. This includes a log of email activity.

The ultimate end point of the system will be to deliver automatic data entry into the Research Council’s J-ES system. To date only one other university (Bristol) has achieved this.