Supporting institutional decision making with an intelligent student engagement tracking system

 

Case study written October 2012.

 

Contents

 

Background Outcomes Key lessons learned Looking ahead  Summary and reflection

Background

 

Business Case and Context

 

This project aims to demonstrate how the adoption and enhancement of a student engagement system (SES) can support and enhance institutional decision making with evidence in three business intelligence (BI) data subject categories: student data and information, performance measurement and management, and strategic planning.

 

Aims and Objectives

 

The project aims to:

 

• Assess the current and expected institutional BI maturity level in the three chosen BI data subject categories in general, and the intelligent engagement system in particular
• Work closely with the University's chosen BI vendor (SolstonePlus) to make better use of student engagement data in support of institutional decision making through developing a BI toolkit using data collected by the student engagement system (SES) and other internal and external data sources
• Promote the use of SES and determine how the SES can be used to support and enhance institutional decision making
• Showcase examples in relation to the realisation of a successful BI implementation (intelligent engagement) in the UoB and promote wider adoption of BI in the UK Higher Education sector

 

Rationale 

 

In 2009, in order to manage student engagement in their academic study more effectively, the Registry of University of Bedfordshire (UoB) jointly with BI vendor came up with an effective student engagement tracking system – called Student Engagement System (SES). Initially, the system served as merely an information source, rather than a decision support environment. On this backdrop, the UoB are interested in developing a BI solution or toolkit – software designed to enhance the functionality of the existing system. It is hoped that the project will help managers and academic staff take a more proactive approach in student management and strategic planning.

 

Historical Context  

 

Engagement data input comes from a number of online and offline sources. Fixed digital devices have been positioned in lecture theatres and seminar rooms across University campuses. Before entering lectures and seminars, students are required to scan their student card, which identifies the time and the place or event attended. Events may also include: examination and library attendance, open day registration and assignment submission. Student online login information, such as BREO (virtual learning environment and support), University email and digital library are also collected by the tracking system. This information is then communicated back to a central database within the University. SES also interacts with other Information Services Team systems including the SITS student database which consists of personal and academic information held by the University for each student.

 

Although the system went live in December 2009, it served as merely an information source, rather than a decision support environment. Furthermore, users of the system have highlighted that the current systems’ interface is fairly basic and have expressed a desire for enhanced functionalities. The University felt that the system has great potential and would like to build on the system further. Consequently, the University had expressed an interest in developing a BI solution or toolkit in order to improve the functionality of the existing SES, thus the University is able to monitor and understand student engagement behaviours at different levels (individual, group and cohort, etc).

 

Key Drivers  

 

With higher education institutions facing increasing pressures from social and economic change, student acquisition, engagement and retention becomes more critical than ever.Student engagement, in particular, is viewed as an important antecedent to student learning and achievement, as well as to institutional success. Therefore, key drivers for this project are:

 

• Monitoring and improving student engagement and retention
• Improving risk management related to student performance management and planning, i.e. students at risks can be identified at early stage
• Supporting institutional decision making with more informed and evidence based information in all levels, i.e. academic staff (module leaders, course managers, personal tutors and lecturers) and administrative staff at the operational level, faculty managers at the tactical level and registrar at the strategic level.

 

Project Approach

 

The broad project approach is to: 

 

• Benchmark BI maturity level using the JISC infoNet BI Maturity conceptual framework levels. This will involve an examination of the current and expected BI maturity level in the areas of student data and information; performance measurement and management and strategic planning as outlined in the JISC infoNET BI Resource data subject categories. 
• Identify BI toolkit requirement specifications. This will involve undertaking surveys, observations and interviews with end-users of the system, i.e. institutional decision makers. The outcomes of user requirements identification will inform the selection of appropriate decision modelling techniques to be used in the BI toolkit. 
• Develop and test BI toolkit. Technical expertise in designing, coding and testing of the BI toolkit will be facilitated by the BI software vendor. The toolkit will be installed and tested with key stakeholders in UoB. If necessary, adjustments will be made at this stage. A showcase of key BI toolkits will be developed in parallel using open source software. 
• Identify BI implementation issues. This work will examine the key implementation issues faced when deploying the BI toolkit. 
• Showcase and promote BI toolkit to the HE community. This will include the production of a demo showcase and guides for other institutions and a video testimonial from the University's executive management team demonstrating the benefits of using the system. 

 

Scope 

 

The scope of project comprises the above project approach, but the focus is on the development and enhancement of SES with BI toolkit or solution for the Registry in UoB. Although the project initially aims to support and enhance institutional decision making with evidence in three business intelligence (BI) data subject categories - student data and information, performance measurement and management, and strategic planning; priority is given to the student engagement measurement and reporting dashboard for the purpose of supporting performance measurement and management, and strategic planning. The scope of student engagement measurement is based on live data events collected from both offline and online sources. 

 

Governance

 

The project champion is the Registry of UoB. The project is a joint effort with the University's Information Systems Department (ISD) and Learning Resources and Student Services Department (LRSSD). Hence, the project board consists of a JISC BI project director, a JISC BI project manager, a SES project director from LRSSD, a SES project manager from ISD and a project manager from BI vendor. The project board meets monthly and receives support from key academic and professional services stakeholders for the development and implementation of BI toolkit in the student engagement system. During the course of this project, a Business Action Group (BAG) has been set up, comprising members from the registry, the international office, the student services office, the learning resource centre, faculty managers, course directors and personal tutors. BAG members meet regularly (4-6 weeks) to ensure that the improved SES is implemented smoothly and successfully. The project board also seeks consultation from a critical friend provided by JISC. 

 

Technologies and Standards Used

 

SES development and BI toolkit - Oracle BI technologies

Showcase development - open source software, MySQL

 

Establishing & maintaining senior management buy-in

 

Senior management buy-in has been established right from the beginning of this project, whereby the University's Registry has initiated this project and further demonstrated their strong support by providing both human and financial resources for the improvement of the current SES. This has been established and maintained as follow:

 

• The support and approval of initial project plan, followed by the establishment of a governance structure for this JISC project that includes senior team member from the Registry, academics, and Information Systems Department (ISD)
• The Think Tank meeting chaired by the university's Registrar to collect feedback from senior managers, senior academic, and professional services stakeholders for system's requirements and specifications.
  
• The creation and regular meetings of a SES Business Action Group (BAG) which comprises key stakeholders and managers of academic and professional services, such as faculty managers, International Office manager, Student Union manager, Learning Resources and Student Services Manager, ISD manager and personal tutors.The purpose of the Business Action Group is to determine how the Student Engagement System can be best used to support the student experience and engagement.

 

Outcomes

 

The project has achieved a number of important interim and final deliverables. These are summarised in the following sections.

 

Understanding UoB’s BI Maturity Level 

 

The first task was to outline the BI maturity level of the Registry in University of Bedfordshire (UoB) based on the examination of the current and expected BI maturity level in the areas of student engagement data as outlined in the JISC infoNet BI Resource data subject categories. 

 

Based on the evidence of our various investigations and assessments, the initial assessment was that UoB's Registry had moved forward from BI maturity stage 1, 2 and 3 to Stage 4 which was that an initial BI system (SES) was put in place in 2009 allowing managers at each level to access data when they need to. Prior to the beginning of this JISC BI project, the SES project was in its infancy in terms of realising its full potential as a data source to provide evidence based decisions and planning. The UoB senior management team demonstrated their strong support and provided appropriate financial resources for the improvement of the current SES to leverage its value. It is believed that the current improved SES has ensured that the university’s BI maturity level is working at the maturity level 5, and will be moving towards 6 in the near future. The UoB's JISC BI project has been an important part of the university’s SES improvement project and played an essential role in helping transforming UoB's BI maturity level. The university is now working on a number of new student experience related ICT projects (e.g. integrated intelligent system for student attendance monitoring system) which will be an integral part of the university's unified BI system.

 

Initial assessment of of SES

 

At the start of the project, the project team did an initial assessment on the old SES to identify problems, needs and establish the the technical specifications for BI solutions. Across all the investigations by using surveys, focus groups and interviews, there was a general consensus that the current SES is potentially very valuable in supporting institutional decision-making on student engagement in UoB. The SES project has had profound impact on student behaviour especially in attending classroom based learning activities, such as lecture, seminar, and workshop. However, most users agreed that the SES requires further enhancement and development to be considered as usable. Summary of problems and limitations of old SES are outlined below.

 

Problems and Limitations of old SES

 

• System speed - The sheer amount of data collected makes the current system unable to provide acceptable speed for user to use the system. This has been identified as the bottleneck for making further use of the current SES. 
• User interface – the current user interface is not user friendly and flexible. No online and offline user guide is available.
• Data output – this is regarded as not operationally useful and action-oriented at the moment.
• Data sources and modes of engagement – nine engagement areas are used to collect student engagement data, but it may not provide a complete and true picture of the student engagement. Therefore, other important sources should also be included. 
• Lack of the integration with other student data systems, such as the student record system, e-learning platform, VLE (BREO) and e-portfolio system (PebblePad), etc. 

 

Expected Impact of BI Toolkit in SES

 

Before introducing BI solutions to the existing SES in UoB, existing or potential users of SES were asked to provide their expected impact of BI toolkit in SES. From the findings, the expected impact of BI toolkit in SES is summed up below.

 

Purpose of Using BI tools        

    

• Checking and monitoring student engagement in specific areas and activities at respective units or courses
• Early intervention of disengaged students
• Automatic risk identification with alert function
• Correlation analytics between engagement and performance
• Student performance prediction
• More accurate reports as true reflection of student engagement

 

Perceived Benefits         

 

• Positive impact on student behaviour
• More effective and efficient student management
• Evidence based feedback to students
• Early Interventions on student at risks
• Real time monitoring
• Meet UK Border Agency (UKBA) & other legal requirements
• Inclusive, reliable data that can be used for comparisons across course/ departments/ faculties

 

Desired Decisions         

  

• Student progression and retention
• Level of support for disengaged students
• Provide tailored guidance and advice to disengaged students
• Course design
• Support recruitment, planning intake

 

Desired Data Analysis Models and Tools       

        

• Statistical analysis, such as mean, mode, variance
• Analytic hierarchy process, regression analysis, predictive analysis, trend analysis, what-if analysis
• Graphical representation of student engagement activities
• Desired Data Sources    
• Class attendance
• Online engagement activities, ie. VLE (Blackboard)
• Student profile and record, ie. SITS
• Timetabling data
• Assessment data

 

Other External Database   

          

• UK border database
• Admission data, ie. UCAS

 

Identification and Development BI Toolkit Requirements Specification

 

The second task of the project approach was to identify the BI toolkit requirements specification and the appropriate and relevant decision support model(s) and analytical techniques. Based on the evidence of our various empirical research methods, users and potential users of SES have highlighted three essential requirements for improving current SES with BI solutions.

 

a. Student Engagement Measurement Index 

 

The student engagement measurement index provides the ability for users to customise engagement measurement index.  Three functions can be provided for users to:

 

• Use default engagement measuring index – The index should be developed based on the common perception and definition which constitutes student engagement weighting - impact and decay factors. This default measurement index can be regarded as a standard engagement KPI which is suitable for most users.
• Customise and prioritise the engagement weighting - impact and decay factors, e.g. class attendance and VLE are more important than other engagement events. This can be provided by giving users the ability to define their own engagement weighting parameters to reflect their priorities.
• Customise the engagement measuring system with their selected engagement events. Users should be able to pick and mix different engagement data events captured by the system.

 

b. Interactive Dashboard for Reporting

 

The improved SES is expected to provide the ability to create recurring standard report and ad hoc (demand) reports through an interactive dashboard. These reports can be generated either by request of an end-user or refreshed periodically through an automatic scheduler (ie. weekly, fortnightly, monthly, quarterly). Users are allowed to modify and save their report parameters, such as frequency, date ranges, course and/or unit cohort. Key functionalities of reporting can be summed up as follow:

 

Standard View

 

• Standard report aims to provide an overview of student engagement activities based on a set of default date ranges. 
• It is an automatic generation of standard reports for board meetings and decision-makings.
• Standard report is generated through a set of pre-defined parameters related to student engagement activities, such as student profile, class attendance, library attendance, and VLE engagement.
• It extracts all the pre-defined parameters and presents its output in a standard report format.
• Graphical representations of student engagement activities through graphs and matrix bar charts are expected in the dashboard reporting.  

 

Selective View (Pick n Mix)

 

• Selective or ad hoc report is generated by request based on an end-user’s information requirements.
• User can select or modify parameters or engagement events and save the setting for future use.
• User can view different types of engagement data according to their interests. The dashboard reporting also provides customised graphical representations of selected student engagement activities. 

 

c. Risk Alert System

 

The risk alert function provides the ability to alert users of exception activity of student engagement, for example, disengaged students who are at risk can be identified early in a course. Users can be notified automatically through push mechanism, such as email, pop-up window etc. Key functionalities of alert function can be summed up as follow:

 

• User can define the thresholds and levels of risk for individuals or groups.
• Ability to flag out individual students or groups at risk automatically.
• Exception data or push report can be pushed to the users’ email or through the pop-up window of SES.
• User can specify the email address and select different frequencies (weekly, monthly etc.) to run the push report.
•  User or Faculty defines parameters and outputs for the automatic generation of push notification. 

 

Improvement on Student Engagement System (SES) with additional BI tools

 

BI vendor has developed and completed a number of enhancements based on the BI toolkit requirements specification. Dashboard 1 depicts the standard view of default engagement measurement index, whereby  the scoring calculates all the engagement events based on predefined weighting parameters. The weighting parameters are based on the respective impact and decay factors of each engagement event of each cohort. The impact factor determines the importance of each engagement event. The decay factor refers to the value that does not affect scoring anymore after a certain period of time. The actual scoring is then presented in percentage and with colour indicators as depicted in Dashboard 1. The colour indicator helps the users to identify students who are disengaged, ie. in red colour. From the scoring percentage and colour indicator, users are able to drill into data progressively, such as the daily engagement data and cohort average data. 

 

Dashboard 1: Standard view of student engagement scoring

 

 

 

Users can either use the default engagement measurement index or have the capability to modify the engagement weighting parameters. The engagement weighting parameters vary between different engagement event types and cohorts because students are not expected to engage in all events at all time, such as assignment submission event; and some cohorts place more emphasis on different engagement events, for example, nursing students are expected to engage more in work placement rather than in-class attendance. In this case, users can define their own impact and decay factors based on their individual perceptions and definitions which constitute student engagement weighting in their respective cohorts (see Dashboard 2). 

 

Dashboard 2: Engagement weighting parameters modifier

Users can also customise the engagement measuring system with their selected engagement events in mind (see Dashboard 3). This can be done in the selective view, whereby users can select the engagement events and save the setting for future use. 

 

Dashboard 3: Customise student engagement events

 

 

Various visualisations of reporting are provided in the improved SES. Student engagement activities are presented in graphs and matrix bar charts as depicted in Dashboard 4 and 5. 

 

Dashboard 4: Student engagement matrix bar chart

Dashboard 5: Student engagement line chart

 

A benchmarking bar chart is also presented to compare individual student with the respective cohorts (see Dashboard 6). The average score of engagement in the selected cohort is presented along with the engagement activities of the specific student. This allows the users to benchmark individual student engagement against fellow peers. 

 

Dashboard 6: Benchmark Bar Chart 

 

Another BI tool is the risk alert function. Users can define the thresholds of risk for specific cohort. The system will then present students who are at risk due to low level of engagement. At the moment, the exception report is presented through a separate screen of SES (see Dashboard 7). The risk alert function is in pilot mode as further testing is required before it can be rolled out to all users. 

 

Dashboard 7: Risk alert system

 

Key SES Implementation Issues

 

Business Action Group members meets regularly to ensure that the improved SES is implemented for maximum benefits and impact in the university. Key issues in relation to the effective and wide SES implementation can be outlined in the following:

 

Technology and Data - Although technology itself can not warrant the successful implementation, it  is, however, an essential enabler. At the early stage of the project, technology was the bottleneck of the progress. Numerous problems with data collection devices, data feed integration, and database bugs have prevented the expected level of BI applications. One notable technical challenge is managing and monitoring the engagement data collecting devices to ensure the data reliability and accuracy. Another is how to manage the daily growth of overwhelming volume of data which affect the system speed.

 

Organisation – From the fusion and transformation point of view, the project recognised the importance of raising awareness among senior managers on the importance of seeing BI as an embedded component of student engagement management system. A clear system governance structure that involves senior management team should be put in place at the very early stage in order to allow systematic interventions from top management team. Organisation implementation strategy, guidance, and system ownership are also important issues to address throughout the project. 

 

People – The system’s benefits can only be realised through the users, therefore, people is the central concern of the implementation success. Most of challenges and inhibitors for the successful implementation appear to be centred around human issues. Motivating people to use and maintain their use for long term benefits is critical.  Misunderstanding and mismatch of expectations among IT providers, managers, academic staff and faculty administrators can cause delays and affect the success significantly.

 

Showcasing SES for HE Community

 

A demo showcase has been developed using open source solutions to promote the use of BI in student engagement to Higher Education community -http://www.solstoneplus.com/showcase/campus. The SES showcase helps users to rapidly identify students who are at risk of dropping out, so that an early and positive intervention can be taken.  SES showcase produces an engagement score based on students' interactions with the institution's systems, or from data within those systems.  These scores are then used as a means of comparison within a cohort.  This comparative approach helps prevent "false alarms" when activity falls through a natural lull in the academic cycle.

 

 

The cohort page below enables the comparison of engagement scores across a cohort of students.  In this context, a 'cohort' is a set of students related to the structure of the institution and its courses; so in this Showcase example, it is possible to compare students by module, course year, course, department etc. Please note however, that the higher up the hierarchy you go, the longer it will take to compile the data.  The Red - Amber - Green (RAG) display is set by thresholds at the institution level, and is based on standard deviation.  The alerts page offers individual users an alternative facility to set a  "warning" threshold of their own.   

 

 

 

The individual pages below enable a fast lookup of an individual student from student id or last name.  A complete history of the student's engagement is presented in chart form (for the Showcase this is just 3 months).  Although it is likely that the most recent scores will be of greatest interest, there are sometimes patterns of activity which provide further insight.

 

 

 

The alerts page below gives a concise view of the students within a cohort whose engagement score does not meet a user defined threshold, calculated as a percentage variation from the cohort average. Alert thresholds on this page are set by an individual user, as opposed to the Red - Amber - Green (RAG) display of the cohort page, which is set at the institution level.  Normally, user settings would be stored in the database, but for the Showcase, settings revert to defaults once you navigate away from the page.

 

 

 

An engagement score is based on a positive event occurring - an assignment submitted, a class attended, use of the VLE etc. There are two basic parameters used in the calculation - the relative base score, and a decay value.  These are used to represent the relative importance of an event, and the period of time over which the effect of that event diminishes to 0. (The reason for the decay parameter is that attending a lecture yesterday still has "engagement value", but attendance 6 months ago is no longer relevant in considering engagement today.)

On the customise page below, these parameters can be set up differently for different courses, module, departments etc, to provide full flexibility in reflecting the many different study patterns.

 

 

Scoring is based on event types - these are defined on the page below.  Types can also be "tagged" with multiple categories to assist with analysis.

 

 

Tangible Benefits 

 

This project concerns with the organisation wide BI development and deployment for intelligent student engagement management which has potentially profound impact on student behavior, student management, and evidenced-based decision making cross the university’s operational, managerial and strategic levels. This project has generated the following tangible benefits:

 

• Increased the number and diversity of SES users. For example, we had a 20-30 active users of the old SES, but now the active users have increased to 60-80 active users. The university is therefore considering buying more licenses, e.g. to make the total licenses to over 150.
• Useful reporting and auditing tool for student management. For example, SES has been an essential resource for benchmarking and reporting individual student engagement for the recent UKBA auditing visit. The university is able to provide the evidence of individual students' engagement report over different periods of time (e.g. from recent 30 days up to one year).  The system also benefits students, especially those who are at risk of dropping out as they can receive support at early stage. These students are able to see the benchmark of their engagement pattens against their peers which is evidence based and meaningful to them.
• Significantly improved working SES with additional tools for supporting reporting and decision making.
• Flexibility to customise various users information requirements and engagement measures.
• Various types of visualisation of student engagement activities through graphs and matrix bar charts.
• A student engagement team has been set up in the University's registry office to focus on supporting student engagement performance and student retention. 
• Management buy-in for new SES improvement project to install better attendance monitoring equipment for extended data feed locations and sources and to use web-based emerging software for monitoring and quality control of data feeds. 

 

Intangible Benefits 

 

In additional to the tangible benefits of the improved SES with BI toolkit, intangible benefits so far include:  

 

• Improved knowledge and understanding on BI benefits and impact among stakeholders.  
• Raised awareness on the strategic usefulness of the information provided by the improved SES.   
• Positive change among students engagement behaviour, especially class attendance.
• Demonstrated the case for making informed and evidence based decisions on student engagement activities.
• Improved acceptance by managers and tutors on better risk management by identifying students at risk at early stage and thus taking more proactive approaches for improving student retention.
• Support the internal and external audit purpose for different bodies or agencies, such as Research Graduate School, UKBA, and sponsoring institutions.  

 

Key lessons learned

 

This project has been concerned with the organisation wide BI development and deployment for intelligent student engagement management which has potentially profound impact on student behavior, student management, and evidenced-based decision making cross the university’s operational, managerial and strategic levels. The project has mainly achieved its objectives and key deliverables. There are a number of important lessons learned and can be shared with the HE community from our experiences.

 

Technology

 

Technology can still be the bottleneck of the project success. In our experience, system problems were the major reason to constrain the further BI enhancement and deployment because if the database was not reliable and working, it was not possible to develop higher level BI solutions which would require a workable and reliable database system. For example, at the early stage of the project, numerous problems with data collection devices and database system bugs have prevented the expected level of BI applications. 

 

Data

 

One notable challenge is managing and monitoring the engagement data collecting devices to ensure the data reliability and accuracy.  Another is how to manage the daily growth of overwhelming volume of data which affect the system speed. Measuring data has also been challenging as decay and impact factors vary among the users, cohorts and courses. BI technology must consider the element of flexibility and customisability. 

 

Organisation

 

Important lessons in relation to the organisational issues are concerned with the identification of SES priority and strategy in implementing BI solutions within the current resource (human and financial) constrains. It is also necessary to raise awareness among senior managers on the importance of seeing BI as an embedded component of student engagement system, thus needs systematic interventions from top management team. Organisational implementation strategy, guidance, and system ownership should be address and communicated to all stakeholders. A clear system governance structure should be considered at the very early stage to bring key stakeholders working together.  The creation of the SES Business Action Group has been an example of good practice by bringing key stakeholders together regularly to address organisation wide implementation issues.

 

Stakeholders

 

Although senior management buy-in was established right from the beginning, we faced unclear human resources support until a governance structure was firmly put in place. There were misunderstanding and mismatches of expectations among BI vendors and providers, managers, academic staff, and administrators which caused conflicts of interest and priorities for system development. Effective and regular communications have helped to minimise the problems, although it may not be possible to eliminate them completely. Influencing and convincing key stakeholders, especially the SES Project Director and ISD Manager to buy in additional BI tools requires compelling justification and persistence. Therefore, it is vital to maintain regular and continuous communication among key stakeholders for the success of change management.

 

Supplier

 

The development and enhancement of SES and BI showcase has been outsourced to external BI vendor. This requires good negotiation and management skills to ensure the requirements are met and the budget is adhered to. In our case, maintaining a good relationship with IT providers has been a challenge. Our experience stresses the necessity that all legal documents and contract must be carefully checked and approved by legal experts and professional staff to avoid any dispute late on. Budget and contributions in kind by the BI provider must be written as explicit as possible in the formal document. All email correspondents should be kept as evidence.

 

Looking ahead 

 

Sustainability 

 

This BI project is highly sustainable due to the following reasons:

• The project has been and will continue to be an important part of the university’s SES system which is a university’s long term strategic ICT investment. The university's registrar stressed that SES is critical to the university's future development. Deploying BI solutions to enhance university’s student management systems is part of the university’s ICT strategy, therefore, the university will support further development and deployment of BI initiatives.
• The improved SES will be promoted and used more widely across the University. The feedback can be used to explore future applications of BI solutions. More BI toolkits can be developed and added to the SES for other tactical and strategic planning purposes, such as benchmarking student engagement with achievement, risk management, key engagement indicators etc.
• The Business Action Group can continue to serve as a steering committee for promoting and enhancing the use and development of SES.

 

Future Plans

 

Future plans will involve the following:

• Promote the wider use of SES in the University and continuously seek feedback from various users for further improvement. 
• Expand the current placement of attendance monitoring equipment and web based analysis software in all facilities and campuses.
• Explore further opportunities to develop more advanced BI toolkits for student engagement analytics and more strategic view of BI dashboard visualisation for senior management team.
• Incorporate learning analytics and predictive tools, for example to benchmark and predict student performance against their engagement pattens, to analyse and predict pattens of engagement of different groups (e.g. culture, ethnicity, age, etc.)
  

 

Summary and reflection 

 

This project has significantly transformed the Registry of UoB's BI maturity to a higher level, whereby an improved student engagement system with BI solutions have been developed and implemented. Relevant student engagement data sources are now automatically and efficiently collected and processed for reporting through a number of interactive dashboards. These dashboards present various types of visualisation to users and allow users to make better informed decisions related to student engagement management and retention. The improved system also gives users the flexibility to customise the system and reporting based on their respective requirements. As a result, the number of active users have increased which leads to senior management buy-in for more user licenses and for installing more engagement monitoring equipments across the University. In the future, the BI maturity level can be improved by incorporating advanced BI and evidence-based decision making toolkits, such as learning analytics and predictive analytics. 

 

The JISC BI maturity model is a useful framework for benchmarking an institution’s maturity level of a particular BI system, but it has certain limitations. First, it seems not applicable to assess the overall BI maturity of the entire institution, especially if the institution have a number of BI initiatives and systems that may be at different levels of BI maturity. Second, once an institution has reached a more advanced stage from stage 4, indicators for benchmarking the diversity, extent, scope and impact of the BI development and deployment should be explored. A series of comprehensive indicators can help to further differentiate the level BI system applications. For example, quantitative measures can be introduced to indicate the extent and scope of the BI systems adoption. Users’ assessment on systems effectiveness and impact should also be an important part of the maturity indicators.

 

The experience and insights gained from this BI project will provide greater understanding for other higher education institutions that are interested in BI initiatives. A showcase has been developed for sharing and promoting BI initiatives between higher education institutions and external data asset owners and BI suppliers. 

 

Action learning with BI cluster group members plus support from JISC and JISC's critical friend proved to be helpful and useful for knowledge sharing. It is found that some projects in the BI programme have similar area of BI initiatives, hence, the cluster groups can be formed on this basis.

 

Video case study for this project