Strategic ICT for Managing Student Engagement (SIMSE) 

 

University of Bedfordshire (UoB) has committed to enhance student experience with an improved and wider deployed Student Engagement System (SES). This transformation project aims to explore and utilise JISC’s Strategic ICT Toolkit and other relevant resources to facilitate and support UoB in developing towards more strategically directed Student Engagement System. In particular, the project will apply JISC resources to help the university in its effort to transform the present fragmented student information systems and processes to a more systematic and integrated system environment.

See the full Transformations programme playlist

 

Contact persons:

Professor Yanqing Duan

Project Manager

Yanqing.duan@beds.ac.uk

 

Mr. Marcus Woolley

Project Co-ordinator

Marcus.woolley@beds.ac.uk

 

Dr. Guangming Cao

Project Investigator

Guangming.cao@beds.ac.uk

 

Background

 

Aims and objectives

 

The overall aim of the project was to transform the university’s Student Engagement tracking System (SES) by exploring and utilise JISC’s Strategic ICT Toolkit. In particular, the project attempted to use the toolkit to facilitate and support UoB in developing more strategically directed student engagement management systems, to evaluate the extent to which the improved Student Engagement System (SES) was aligned with the enterprise wide institutional strategy,  thereby identifying potential business changes that would enable the SES to deliver maximum value and agility. Specifically, the project objectives included:

 

1. Exploring and utilising JISC’s SICT Toolkit and other relevant resources to facilitate and support UoB in developing a more strategically directed SES;
2. Providing feedback and recommendations on the possible improvement of SICT Toolkit;
3. Helping raise awareness of the usefulness and impact of SICT Toolkit; and
4. Offering lessons learned and practical experience for other institutions.

 

Context

 

Information and Communication Technologies (ICT) form an essential and integral part of UoB to support the University’s capability to achieve its strategic goals and deliver its core services. In particular, UoB values the role of ICT in enhancing student experience and managing student engagement in their academic study. In 2009, UoB decided to identify better ways of measuring student engagement using ICT. It was hoped that the use of ICT would support the University’s key initiative of identifying students at risk of disengagement and/or those needing more help to achieve their academic goals. The University commissioned a project to develop a system which would allow for a holistic approach to data collection, integration and analysis.  It would enable effective student engagement reporting and alerting. Working with Solstone Plus (SSP), who are a leading consulting company in the fields of Business Intelligence (BI) and Data Warehousing, the University came up with a student engagement tracking system (SES) with data input collected from a number of the University’s online applications including:

 

• student location data recorded by fixed digital devices positioned in lecture theatres and seminar rooms across University campuses
• student use of University systems such as BREO (Blackboard VLE), university email, student record system
• student library use
• student use of the Student Information Desk
• assignment submission data.

 

This information was then communicated back to a central database and analysed using Oracle BI to provide an engagement score for every university student.

 

The concept of “Big Data” is emerged to describe the volume, variety, and velocity of the data generated with ICTs. SES is an example of the University’s attempt to utilise the value of Big Data in the Higher Education. Our effort and experience help to raise awareness of Big Data in the HEI community.

 

Solstone Plus (SSP) were aqured by Red Stack Technology Group in February 2013 and have now been rebranded as 3sixty-analytics Ltd

 

The business case

 

The SES went live in December 2009; however based on our investigations by using surveys, focus groups and interviews it appeared that it was still in its infancy in providing reliable data to support decisions and strategic planning. The emerging consensus from these investigations was that the SES served merely as an information source for a limited number of users, rather than a decision support environment for achieving transformational changes and enhancing student experience. It had a number of problems and limitations in many critical areas, such as limited value for higher level support, poor usability, lack of flexibility and integration with other student data sources. Consequently, it was clear that the SES needed significant improvement to be able to provide important information in support of the University’s strategy and policies.

 

The UoB senior management team had realised the potential strategic value of the SES and the importance of aligning the deployment of the SES with the University’s student engagement policies and objectives. After a careful review of the current SES applications, the University decided to provide appropriate human and financial resources for a SES improvement project to leverage its full strategic, operational and transformational value. However, we were unable to identify an analytical framework and support toolkit that could be directly applied to facilitating the transformation of the present fragmented systems to a more systematic and integrated system environment.

 

JISC’s SICT Toolkit, which has been developed over the years for UK HEIs to support education and research through the innovative use of ICT, was seen to provide useful tools to help UoB to analyse, assess and develop its strategic use of the SES in the area of student data and management.

  

Key drivers

 

It was hoped that an integrated SES should support UoB to:

 

• improve the ability to realise institutional retention goals
• improve the ability to make early successful interventions with students who required additional support
• improve student and staff experiences with greater enterprise capability
• enable greater sharing of knowledge and resources across the university
• improve communications with students
• increase the value of ICT investments
• improve organisational efficiency

 

It was envisaged that this project could benefit the wider HEI community with its outputs in the following two important areas:

 

• evaluation of the SICT toolkit and other relevant resources (if applied), which would provide evidence based assessment and feedback to improve the usability and usefulness of these resources, so they could be widely adopted by UK HEIs
• a case study report, which would offer lessons learned and critical success factors in the context of UoB SIMSE project.

 

 

JISC resources/technology used

 

The SICT toolkit was used as the main resource which provided a self-analysis toolkit to help the SIMES project to analyse, assess and develop the strategic use of ICT within UoB.

(http://www.nottingham.ac.uk/gradschool/sict/).

 

This project explored the potential and usefulness of Enterprise Architecture (EA) (http://www.jisc.ac.uk/whatwedo/programmes/elearningcapital/enterprisearchitectures.aspx)

 

Outcomes

 

The project achieved two key results that are summarised below.

 

1.  Examining and selecting relevant JISC resources

The project team selected several tools from the SICT Knowledge Base and Self Analysis, based on the relevance of these tools to SIMSE and considering the project timeframe and resource available to SIMSE.

 

Three strategic ICT enablers in the Knowledge Base were selected by the project team to help challenge and improve the existing SES, including governance, Shared Services, and EA. The project team believed that ICT governance would help clarify the institutional SES governance structure, drivers in formulating SES strategy, the alignment of the SES strategy and budgets with institutional strategy and budgets, and the assignment of the SES responsibility.

 

Shared Services were seen to help the institution to become more agile. The key focus of the project team was to ascertain how UoB would address Shared Services across the institution in response to developing more institutionally aligned ICT.

 

EA was seen to be extremely useful to create new and future SES applications (the ‘to be’) based on clearly identifying the existing SES, relevant business processes, organisational factors, and information and application technologies (the ‘as is’), and to implement the steps that would form the transition between the ‘as is’ state and the ‘to be’ state.

 

EA was developed primarily for improving institutional level strategic alignment. The SIMSE project team felt that there was no doubt that the overall EA framework could be used to guide the implementation of SES; however, it needed to be modified to improve the SES for the following reasons. The SIMSE was a small-scale project focussing on improving a specific project-SES. The resource and time available to the project were restricted. It was difficult to identify a suitable and structured tool to operationalise the EA framework. Another main reason was the need to integrate EA with IT governance, which was seen to enhance the SES alignment with University strategy, structural requirements and technology, budgetary control, and selection and use of new technology. However, the overall IT governance was not particularly designed to improve and implement a specific project like the SES; rather, it focused on high level strategic alignment. Due to the nature and context of the SIMSE project discussed previously, the project team felt that EA and IT governance approaches selected for this project needed to be combined together to be more structured for improving the SES.

 

Taking these into consideration and after investigating various relevant tools, the project team found that Critical Systems Heuristics (CSH) could be used to integrate and operationalise EA and IT governance frameworks for this SIMSE project. CSH is a systems approach to encouraging critical thinking about the value judgements that underlie planning decisions through a set of 12 questions. The questions examine the boundaries of decisions to find a way of counteracting the monopoly of knowledge and power given to experts to the exclusion of ordinary people. Although challenging dominant ideas was not an issue at all in the case of the SES implementation, the project team considered CSH a valuable holistic approach that was compatible with the EA/IT governance frameworks. As a result, the project team had designed a questionnaire survey by combining the key features of CSH, the EA and IT governance frameworks to facilitate the SES implementation, which is enclosed in Appendix I.

 

In addition to the selected key enablers from the SICT Knowledge Base, the Maturity model of Self-Analysis was selected to obtain a preliminary assessment of the institutional strategic ICT maturity level, which would inform further improving of the SES and its strategic use.

 

2.  Identify the ‘as is’ state and the ‘to be’ state of Student Engagement System

The project was to identify the SES current situation and its future state that UoB wished to reach. This involved two forms of assessment: conducting a questionnaire survey designed by the project team and evaluating the UoB ICT maturity level using the Maturity model of Self-Analysis.

 

The questionnaire survey was distributed to the 9 members of the Business Action Group (BAG) team who were responsible for determining exactly how the SES should be improved to best support student experience. The members included key representatives from Learning Resources, Information Systems Department, Student Services, Faculty Admin, and International Office.

 

Six responses were received and the consensus was that the current state of the SES was by and large in stark contrast to the future state of the SES with regard to sources of motivation, decision-making and governance, sources of knowledge and trust and legitimacy of the data sources (See Appendix II for details). In general, the current state was much narrower in its scope than the perceived future state. When the clients considered the current state of the SES they had concerns regarding the purposefulness and value of the application, the limited scope of systems integration and limited scope of measures of systems success than they considered should be in the future state. Regarding decision-making and governance, namely who is in control of what is going on and is needed for success, the current state, compared with the future state, had limited control of resources, an incomplete understanding of the decision environment, and no involvement of the senior decision-makers. 

 

The UoB ICT maturity level was assessed by the Information Services Directorate (ISD) Head of Corporate Information Systems using the Maturity model of Self-Analysis, which is presented below. The result of the analysis suggested that the UoB ICT maturity level was operational.

 

 

Uob Pre-change Self-Analysis

 

Achievements 

Based on the findings from the surveys and analysis, the gap between the current state and the future state had been identified, which provided valuable information for UoB to identify business changes that could enable the SES to be improved to deliver maximum strategic value and enhance student experience. So far, the SES had achieved the following:

 

• the number of the SES direct users has increased from 6 users at the start of the project to 100 active users now
• the diversity of the types of the users is now increased, so is the number of applications. Whilst the main users are admin and sample academics, there is an increasing indirect use by personal tutors. The University has recruited eight additional administrative staff to utilise the information provided by the SES. These members of staff are faculty student attendance trackers and engagement officers who are relying on the SES to support their work. They use the SES to provide information to managers and tutors, thus enhance the benefit and impact of the SES
• the SES mechanism to capture individual students’ movement on the campus has been developed and the data is now stored in a central database
• student engagement behaviour overall has improved over the period. However as the UK Higher Education sector has experienced much change during the period, not least the move to £9,000 fees, there are a number of external drivers that may have influenced this.  No direct causal link can be claimed
• other positive changes experienced in this timeframe include an increased level of class attendance and student use of the library. Library utilisation rate at the peak time (e.g. 11am in the morning) increased from 80% to 95% in the main campus library in Luton
• the Student Engagement Team can now identify students at risk and set meetings with Personal Tutors, which was not previously possible
• shared ICT services are much improved; ICT, Learning Resources, Registry and academics are now working in partnership. However, student attendance monitoring data is still not linked to the timetabling system.
• raised awareness on the needs of student involvement in future SES improvement. Students should not be seen as passive participants. Their voice should be heard. A focus group was carried out with students. Value feedback was collect and discussed with SES managers. The feedback provided useful suggestions and directions for exploring how students can benefit from using SES to benchmark their own engagement level with their peer group.
• raised awareness on the value of Big Data using our project experience through publications and seminars. Two conference papers will be presented and published in two international conferences in September and November 2013. A research seminar on “Big Data in High Education” was hold in the university discussing and debating on the benefits, challenges and opportunities of Big Data in Higher Education Sector. Project team members used the project as an example to demonstrate the value of Big Data in student engagement management.

 

As a result, The UoB post-change ICT maturity level has been analysed as strategic by the ISD director, as presented below.

Uob Post-change Self-Analysis

Benefits

 

The key benefits derived from the SIMSE project include:

 

• raised awareness on the value of Big Data in student management among senior managers resulted in their buy-in of new big data projects.
• raised awareness of the strategic usefulness and impact of the SICT Toolkit in helping achieve institutional goals through strategic use of ICT within UoB
• increased institutional awareness of ICT governance in providing the management framework for harmonising institutional strategy and structures with performance goals, measures and accountabilities
• improved knowledge and understanding on the importance and necessity of strategic alignment between ICT and institutional strategy
• improved knowledge and understanding of systems integration enabling greater sharing of knowledge and resources across the enterprise
• improved effectiveness in identifying students at risk at an early stage, thereby taking more proactive approaches to improving student retention and positive changing student engagement behaviour
• improved communications, increased SES value, and improved organisational efficiency. 

 

Drawbacks

 

Retrospectively, whilst the Strategic ICT enablers such as governance and EA were seen to be valuable tools to guide the SIMSE project, it was felt that they were developed mainly to help senior managers achieve institutional level business and organisational change though the strategic use of ICT. A more structured approach would be required if developing a specific ICT project.

 

Key lessons

 

• It is essential that all key stakeholders including students are involved in organisation wide development and deployment of ICT transformation project. A good partnership among ICT, Register and Academic staff can ensure the delivery of a successful system.
• the SICT toolkit is a useful means for HEIs to deploy ICT to achieve institutional goals
• in order for ICT to play a strategic role in achieving institutional objectives, ICT needs to be aligned with institutional strategies
• ICT governance is essential for an institution to harmonise institutional strategy, structures, performance measures, and accountabilities when developing ICT applications
• systems/data integration and more shared ICT services would allow an institution to create more value from ICT applications.

 

Looking ahead

 

Future plans will involve the following:

 

• promoting the wider use of the SES in the University. For instance, there is an indication that course leaders wish to identify students at risk of disengagement themselves and take appropriate proactive action
• the University is currently investigating the market place to identify the appropriate BI tool to support future “Big Data” projects.  Following the introduction of a new BI tool the current SES system software will need to be ported to the new application tool.  Although too early to make a firm decision it is likely that the system is to be subsumed into a larger scale in-house development
• use of the “Big Data” concept to develop a data warehouse and analysis system to link the current mifare and UHF based attendance tracking hardware with timetable data to provide smart tracking information to student trackers and personal tutors.  This would make a significant impact on the University’s strategic objectives related to retention and attainment
• the SES has demonstrated the strategic importance of enterprise level ICT and the Deputy VC Academic has recently issued a mandate for a system to link, search and report on all documents and information concerning students’ help on any University system.  The University is currently engaged in detailed discussions with HEFCE concerning participation in a major HE big data project.

 

Sustainability

 

Although the SIMSE project has finished and the SES is functioning and used on a daily basis, the transformational momentum continues. The SES will be linked to other new ICT developments as listed above, to help better achieve institutional strategic priorities such as enhancing student experience.

 

The student trackers and engagement advisers continue to use the SES system as part of their day to day activities to identify students who may benefit from academic or support staff interventions, for example:

 

Anna Swales, Senior Engagement Adviser

 

“The SES identified a student as having a particularly low engagement score in relation to the rest of the cohort which prompted further investigation of the student's assignment submissions. This allowed a holistic picture of the student's engagement to be passed to the Personal Academic Tutor. The Tutor was then able to contact the student to offer focussed additional support.”

 

While the attendance data feed and collection equipment installed as part of the SES initiative will continue to be used by the SES it will also input into the timetable/attendance system project which is currently at feasibility stage and possibly the retention system that is currently at the project start-up phase.

 

Immediate related developments include the implementation of a system of data collection and its dissemination of student use of study skills services. This small scale project is due for implementation in August 2013 and makes use of the existing reporting and collection capabilities of the existing student record system.  It aims to significantly improve academics’ ability to understand individual students’ academic skills requirements and it will facilitate appropriate interventions to ensure high levels of student retention. Once the system is proven it may be added to the SES as an additional positive data source indicating that the student is actively engaged in improving their levels of attainment.

 

Project Blog

 

http://simsebeds.blogspot.co.uk/

 

Project Story Telling Video

 

http://youtu.be/z1Ac8BKjyjU 

 

Appendix

 

Appendix I Questionnaire Survey – Strategic ICT for Managing Student Engagement 

 

Appendix I Questionnaire Survey.docx

 

Appendix II The AS IS State and the TO BE state

 

Appendix II The AS IS State and the TO BE state.docx