Project Case Study: Mentoring Enhancement Project
“Making more effective use of ICT to support mentoring schemes in higher education”
See the full Transformations programme playlist
1. The business problem
Mentoring and coaching schemes are an increasingly important component of professional and career development, particularly in higher education institutions, where there are a number of drivers pushing mentoring and coaching up the agenda.
At the University of St Andrews we have been running centrally coordinated mentoring schemes since 2005. We have found mentoring to be a potentially transformational process and a high impact solution for learning and development practitioners.
However, mentoring schemes generate large amounts of data and require significant administrative resource to maintain.
Each new scheme cycle increases the amount of data and as membership builds the volume of data increases further. Data capture, processing and recording, and the management of administrative processes therefore become increasingly demanding.
Our problem was summed up by this question:
How can we make more effective use of information and communications technology (ICT) to support more mentoring, more effectively, with the same or less resource?
2. The project proposal
The purpose of this project was to understand the work flows and processes within our existing mentoring schemes and investigate the options for making more effective use of ICT to support these schemes. Based on this investigation the project would seek to identify solutions with the potential to:
- reduce the administrative burden of coordinating mentoring schemes
- increase the accuracy of data processing
- reduce the opportunity for errors through multiple handling of data
- increase the capacity to expand mentoring schemes (and thus the benefits of mentoring to individuals and Institutions)
- be easy to use, easy to implement and seamless for applicants and scheme coordinators
- be adaptable enough to accommodate a wide range of ‘what if’ scenarios
- have the potential for wider application to other HE and research institutions
The final outcomes of the project would be the development and testing of a ‘proof of concept’ version of the identified solution/s, which would demonstrate that these objectives were achievable.
3. The approach adopted
The approach adopted was to:
- map out the process flows of our mentoring schemes and to try and understand the range of variables and exceptions that the existing management systems accommodated
- translate this into a clear specification which would be used to define and evaluate potential solutions
- investigate existing proprietary solutions and assess them against the specification
- if no satisfactory proprietary solution was found, design a ‘fit for purpose’ data management solution, tailored to our specific requirements, but with the potential for wider application to the higher education sector
- build a working model as ‘proof of concept’ that better use could be made of ICT to support more mentoring, more effectively
As a learning and development organisation we were also anxious to make the project itself a learning opportunity, and designed the project around a small team including two interns. The interns would benefit from the experience, gaining insight into team working, problem solving, and communication in a business environment.
4. The outcomes
The key project deliverables were:
- clear conclusions about the opportunities to generate concrete benefits from the more effective application of ICT to the coordination of mentoring schemes
- a proof of concept 'working model', to demonstrate that we could build a functioning data management system to meet the stated objectives
In practical terms the project managed to achieve and exceed these planned deliverables.
The conclusions about the likely benefits of moving from the existing paper- and spread sheet-based systems for data management/administration were very apparent from the outset. These confirmed and were reflected by the opportunities set out in section 2, to improve accuracy, reduce errors, create resource efficiency and to build institutional capacity. In effect to do more mentoring better. Further discussion of the economic benefits is included in Section 5, below.
It was also established that there was no satisfactory proprietary solution available that would do what we wanted at a reasonable cost. The commercial solutions are over-specified, complex to use and contain redundant or irrelevant features. They are attempting to be all things to all potential customers, and thus fail to deliver precisely what each specific customer requires in an economical way. Some of the available solutions are also ‘invasive’ in IT terms, requiring installation and integration into existing IT systems, which creates additional complexity. Finally, they were also generally considered to be expensive.
The project team moved to planning and designing a ‘working model’ data management and administration system that would meet the specification and deliver the objectives outlined.
The solution was gradually developed and resulted in a data management system capable of running as a ‘black box’ version for testing on one of our own mentoring schemes.
The ‘black box’ testing proved the assumptions made during the design and development process, generated a range of technical problems and fixes and provided a great deal of insight into the potential development of this system.
The project team were able to integrate many of the outcomes from the ‘black box’ testing into a ‘development version’ of the system. This was widely demonstrated within the sector, resulting in a stream of suggestions and constructive feedback from practitioners in other Institutions. These were all added to a development list and prioritised for on-going system development and enhancement.
In spring 2013 this resulted in the launch of a Beta Testing version of the system, now named ‘SUMAC’. This system is currently in use with the University of Strathclyde, the University of Edinburgh, Glasgow Caledonian University and SRUC, with the Royal Conservatoire of Scotland, the University of the Highlands and Islands, the University of Durham and Bournemouth University all in the process of joining the programme.
5. Benefits – the economic case
To learning and development practitioners, the case for increased mentoring provision is clear. There is much research and experience that supports the professional and career development benefits offered by mentoring partnerships to mentees and mentors and to their Institutions. The Equality Challenge Unit paper Mentoring: progressing women's careers in higher education [1] and The benefits associated with workplace mentoring relationships from the Blackwell handbook of mentoring [2] both provide an indication of the potential benefits that can be derived from mentoring partnerships. As do the many other examples[3, 4, 5] of scholarly articles and other published papers on the subject.
However, for mentoring scheme coordinators and the teams they work with, the technology-based solution for managing mentoring schemes has a strong economic case.
Here are a few examples:
A) For most centrally coordinated schemes, when a new applicant joins the scheme the details of the applicant must be added to records held in a hard copy file or spread sheet. The applicant will also usually receive an acknowledgement thanking them for their application, welcoming them to the scheme and advising them about the process.
Where manual systems are in use, this process may realistically take 10 minutes (and conceivably double this). Multiply this by say 70 scheme members (a realistic number during the annual cycle for one of our own schemes) and the total time required is 700 minutes – or 11 hours and 40 minutes, equivalent to 1.5 person days. If this task is being carried out by a member of staff on an annual salary of £24,000 (it could well be someone on a higher scale), when on-costs are included this equates to £160.00. If Full Economic Costing (FEC) is factored-in this figure could be as high as £240.00 (assuming FEC adds 50% to full costs).
Assuming the task takes 20 minutes and the employee coordinating the scheme is on a salary of £32,000, this figure could rise to anywhere between £428.00 and £642.00
If there are more than 70 applicants, the figure goes up again.
Using the SUMAC system, this task requires a one-off ten minute job to configure the default ‘acknowledgement’ email. Everything else during this stage of the workflow is then handled automatically by the system at a total staff cost of anywhere between £2.50-£5.00 depending on the various assumptions indicated above.
B) In most centrally coordinated mentoring schemes, for each mentee applicant, there is a process to define potential mentors. When a suitable mentor is identified, a match is offered to the mentee. The mentee will then either accept or decline the match. Assuming the match is accepted first time, using a manual system this whole process - looking through the available mentors, selecting a suitable mentor and emailing the details to the mentee (carefully transcribing the relevant details, or cutting and pasting from one document to another, and composing or editing the accompanying message) - may realistically take one hour.
Using the costing assumptions from Example a) during one annual cycle the total cost could be between £525.00 and £1050.00 in staff time (with a salary range of £24-32k).
Using the SUMAC system the time to process each mentee applicant, using the built-in matching process and automated (but editable) email functionality, can be reduced to 20 minutes. This means a reduction of staff costs to £175.00 to £350.00, or a potential saving of up to £700.00.
C) Similar savings can be made at other stages in the administration of mentoring schemes, such as sending formal notification emails to both parties in a mentoring partnership, sending group emails to different categories of scheme members, adding notes to records, producing reports and so on.
D) These examples do not even consider the costs savings resulting from increased accuracy that result from eliminating multiple data handling and data entry.
Producing dozens of formal matching notification emails, transcribing and cutting-and-pasting details from different sources requires a huge amount of concentration and mistakes are easy to make.
If sending out one correct formal matching letter using a manual system takes 30 minutes, this could cost £7.50 - £15.00. If a mistake is made, the cost of rectifying this – not counting the intangible cost of loss of reputation and credibility – could be the same again, leading to a total staff cost of up to £30.00.
Using the SUMAC system errors of this sort would not occur, and the cost of getting the process right first time every time would be around £2.50-£5.00 per partnership.
6. Dissemination
During the project the team members engaged in a series of dissemination activities including:
- presentations to practitioner groups and networks such as ScotHERD and Organisation and Staff Development Scotland
- presentations to specific Institutions and groups of Institutions, including University of Durham and University of Edinburgh
- contribution to the Vitae Researcher Development International Conference 2012
- contribution to the Mentoring Scotland Conference 2012
Further activities are planned including a presentation to the Staff Development Forum Executive board and contributions to the Vitae Researcher Development International Conference 2013 and Staff Development Forum National Conference in November 2013.
7. Project learning
As the project progressed through the stages outlined in Section 3 the project team learned some valuable lessons about the process of moving from manual paper-based or spread sheet-based systems to Information Technology-based systems.
7.1 Low tech is highly adaptable but inefficient – high tech is highly efficient but inflexible
The foremost lesson, perhaps, is that simple, low technology systems, while inefficient and fraught with opportunities for errors, are almost infinitely adaptable, while technology-based systems offer hugely improved levels of efficiency and accuracy, but have to balance the level of adaptability with ease of use.
Think of a few ‘what ifs’ - sets of unanticipated and unusual circumstances. For example:
- a mentee rejects a potential match with a mentor, but then changes his or her mind
- a mentor is willing to take on 4 mentees
- a scheme member agrees a match but will be on study leave for 6 months before starting the partnership
- one individual is both a mentee and a mentor and participates in more than one scheme…
With a paper-based system, all that is required is a note added to a pro-forma in a file, or even just a handwritten note on a piece of paper.
7.2 Timing is... everything
We have also learned, through our dissemination and engagement activities, that our objectives of reducing errors, increasing efficiency and building institutional capacity are shared very widely in the sector. We have repeatedly heard that Institutions are planning to launch new mentoring schemes, re-launch existing schemes or are being required to expand mentoring provision. Almost universally they say that they are concerned about their capability to achieve these goals without additional administrative resources, which will not be available.
This experience has very much confirmed the validity of the original project concept at a time where interest in mentoring in the sector is apparently growing month by month and is supported by the huge interest in this project from other institutions.
8. Jisc resources
In the project proposal a number of potentially useful JISC resources had been identified. In the early stages of the project, these resources were reviewed and learning from them applied to the project. In practice, the most useful and instructive resources were:
Records management InfoKit (and managing the Information Lifecycle InfoKit)
These resources contained lessons that helped us to design a slick, workable database conforming to industry standards. It also flagged up a range of issues we would face, including technological, financial and legal and how to overcome them. An illustration of the useful content in these resources is the following quote:
“when designing a new record-creating system, define exactly what information it is appropriate to capture ….. and where possible use system design to capture this information automatically as part of carrying out a transaction…..when designing document and form templates consider their design and specify which elements must or should be completed. Use document properties to enforce completion of all mandatory elements ….. consider the appropriate ‘unit of management’ for a record. For example, when managing web resources, does each webpage stand alone as a compete record …?”
Process Improvement InfoKit (and Process Review Data collection Template, process Review InfoKit and HESA benchmarking Report)
These resources contained techniques that were helpful during the process of reviewing existing practices for efficiency, cost effectiveness and quality of user experience. There were also techniques that helped to identify particular strengths and weaknesses in existing processes and quantify improvements.
Instructive quotes include:
“Problem name: haven’t I seen you somewhere before?
Symptoms: Excessive transfer and rekeying of data.
Causes: lack of information management. Lack of system integration. Reliance on paper processing.”
Business Intelligence InfoKit (and Efficency and flexibility: Getting fit for a changing funding environment)
These resources provided guidance on organising a database and the user interface, so that information is organised intuitively and is therefore easy and quick to access. Also included is a staged procedure for ensuring that data is centralised, coherent and managed automatically, as well as important standards for exporting reports.
Instructive quotes include:
“(Departments have) disparate unconnected information sources….since these sources are not always accessible to all staff or are not trusted (or both) staff (in different departments) often keep local data on spread sheets or local databases. Planning or reporting meetings may spend up to 2/3 of their time wrangling about data quality and disputing specific data items.”
9. The future
The SUMAC system is currently in use at 6 Universities’, and in all likelihood another 4-6 will join the Beta Testing Programme. The project will be sustainable during the beta testing phase, but it will be important to establish how the system will be sustainable in the longer term.
It is likely that the Beta testing partners and others will want to continue to use the system and the project team is currently investigating how this might be achieved in a way that will be cost-effective to the University of St Andrews, but which will ensure that the system can continue to be developed and accessible across the HE Sector. A sustainability plan will be finalised by the end of summer 2013, with the intention of achieving these goals.
Conclusions
This has been a productive and successful project that taps into a very hot subject in the sector, and particularly within the staff development community.
The project has produced some valuable lessons which have been widely shared within the sector via conferences, presentations and practitioner forums. This has generated a great deal of interest in the project and has been helpful to other institutions setting up or expanding their own mentoring provision.
The project outputs have not only been instructive to ourselves and other institutions but have also formed the basis for a sustainable resource that has the potential to add genuine value to mentoring within the sector.
10. Want to know more?
Read the project blog at: http://makingmentoringbetter.blogspot.co.uk/
See the conference slideshows at:
Get in touch!
Jos Finer
Staff Developer
CAPOD (Centre for Academic, Professional and Organisational Development)
University of St Andrews
75 North Street
St Andrews, Fife
KY16 9AJ