Funded by the: Jisc e-Learning programme.
Lead Institution: Birkbeck, University of London.
Learner Provider Type: Higher Education
Project Duration: January 2012 - March 2013
Key Words: Course Data
Case study tags: course data, process improvement, course information, blogging, stakeholder engagement, birkbeck - university of london
Note: This is an abridged version of this project's final report. The full version is available here.
BBK-XCRI: Making the Most of Course information
Project Summary
Birkbeck is unique. Part of the University of London, we are a world-class research and teaching institution that specialises in evening higher education for non-traditional students, offering courses at all levels – from certificates of higher education up to MPhil/PhD research opportunities. All of the taught courses in the Birkbeck portfolio are offered by part-time, evening study – although we also offer an increasing number of full-time undergraduate courses and courses by distance learning.
Making our course information easily accessible is an important aim for us, as our non-traditional, mature students (average age 34 years) do not have the benefit of school or college support structures to help them make their university choices and decisions. Our ongoing analysis of user visits on the web shows us that our online recruitment information receives the highest number of visitors every year and also that many of our visitors go no further on our site. Our published course information is key to our ability to recruit students.
Since 2005, our course catalogue (both online and print) has been published using a combination of SITS, a bespoke Oracle database, and a proprietarial CMS (Serena Collage). This system has served its purpose, but Serena no longer actively develops or supports its Collage product. Therefore, in order to make changes, we rely almost entirely on the expertise and technical ability of one member of Birkbeck staff (a Business Systems Analyst), who both created and developed the system, but without being given any additional time to develop documentation to explain how it works.
Considering how critical these pages are to our recruitment, this has been a matter of considerable concern to us. And, with the increasing demands placed on our course catalogue – not just by staff at Birkbeck, but also prospective students, government policy-makers and other stakeholders – it became apparent that our systems were no longer fit for purpose and needed to be changed.
The XCRI-CAP project offered us an ideal opportunity to review and improve our course information systems, at a critical time in Birkbeck’s recruitment.
What could have been improved? What lessons have been learned?
As we have yet to roll out our new infrastructure to the live site, much of what we’ve learned so far stems directly from the experience of running this project rather than evaluation of what we’ve done:
The value of use cases: until we launched this project, a vulnerability in our course publishing system was the lack of documentation relating to how we published our course catalogue online (most of the information being held in the heads of critical course staff). The use cases developed by our web developer provide a detailed specification of our Plone infrastructure, in plain English, which means they can be understood by non-technical staff. They are also designed to specify the success criteria for our Plone development work, which means they can be used as the basis of evaluation activities.... In combination with the documentation developed by our database developer, this will enable us to future proof the systems driving our course catalogue and enable us to ensure that we can effectively devolve technical support requirements for it.
The challenges of blogging: although we launched the Birkbeck-XCRI blog in good faith, in practice, we have found that blogging didn’t prove particularly helpful to us, as much of the development and thinking around this project needed to be done by and within the project team and with other specific staff at Birkbeck, rather than in the public arena. Also, as we took decisions throughout the project rather than at predetermined times (which was our original, rather naïve plan), we didn’t feel that we could comfortably share our progress with Birkbeck and the wider education community until we were at a more advanced, ‘show and tell’ stage.
The importance of a good project team: Our project team consists of highly specialised technical support staff and more content-oriented/student-facing staff. This has proven a very effective combination both in terms of identifying the objectives we needed to achieve (e.g. to make our content more robust) and to identify the technical infrastructure that would enable us to deliver on them. Moreover, for example, when we realised we weren’t able to meet our original deadlines, our project team decided that, although we were still committed to delivering the whole project, we needed to convert it into more realistic stages, with more than one roll-out stage, and back-up plans should other unforeseen developments delay our project work. This means we are still going to deliver on the original project, just not by the original deadline.
The importance of project meetings: starting in June 2012, we held fully minuted project meetings every three weeks. These meetings were critical to the development of the project, not just for sharing information, but also for feeding in expertise and improvements to our planning. We were able to take decisions that moved on our planning significantly, purely as a result of these discussions – and also able to track these project shifts through the minutes.
The value of good risk assessment and planning: we spent a lot of time planning and preparing for this project, organising it into nice clean stages and considering how we wanted to achieve our objectives. Then we spent more time carefully crafting our project bid and getting sign-off and buy-in from our senior management. Then, in the first month of launching the project, two members of our small project team handed in their notice, and we were suddenly faced with an ambitious, mission-critical project, a fixed deadline and an inability to resource it, owing to the fact that our remaining project team staff had already been allocated to other projects. This is when the risk planning we did during our project specification was helpful (although we hadn’t really expected to call on it quite that soon!), and this has emphasised for us the importance of considering project risk as early in the process as possible.
The power of bringing all process stakeholders together: it was only when we held our process review meeting and brought together representatives from professional service departments across Birkbeck that we understood the true complexity of our current systems to publish course information and how many anomalies had crept in over the years. Bringing all these people together and having an open discussion enabled us to identify inefficiencies in our process, and it was only at that point that we were able to consider improvements.
The importance of effective communication: We took an early decision to use Yammer, an internal communications tool, to host all of our discussions related to this project, upload files for sharing (e.g. all project meeting agendas and minutes) and develop project solutions via a wiki sharing page tool. This has proven to be very productive for us – as in using it, we can ensure that all group members had access to our ongoing discussions, as well as providing us with an archive of decisions, files and developments. This not only made it easier to arrive at consensus within the project team, but it will make it relatively straightforward for us to develop dissemination activities and case studies, which we are aiming to do once we have finalised this project.
Immediate Impact
We believe this project is going to have a critical impact on Birkbeck, particularly in relation to access to and promotion of our course data (although, as we haven’t yet rolled out the new system, we don’t have the evidence to support this). We expect the following:
- Improved quality assurance: In the past, we have relied a great deal on free-text fields in Collage for inputting course data. This means a possibility exists whereby, without rigorous checking, we could publish information that might not be compliant with quality-assured programme information. Although we have a disclaimer on our website, this is obviously a matter of concern. We have therefore implemented improvements to our structures and workflow, to reduce the chances of this happening when we go live with our new infrastructure.
- Improved course quality: our school colleagues sometimes feel a ‘disconnect’ with our published course information, and believe that it is ‘owned’ by External Relations, rather than them. This has proven unhelpful in the past and has led to the situation where our Schools have wanted to publish duplicate course information on their local School/department microsites, simply so they get immediate access to it. This has also led to the situation whereby Schools didn’t always check their course data throughout the year, but rather only when prompted.
- With our new workflow, we are planning to devolve content maintenance responsibilities, so that our academic Schools will be able to update their own content throughout the year (in partnership with External Relations, who will provide editing/proofing expertise). We expect that this will address the concerns related to content ownership.
- Improved CMS experience: Collage isn’t just difficult to support technically (owing to the fact that it is not supported by Serena), but from a user perspective, it is clumsy to use. We use Plone for the rest of the Birkbeck website, and have a devolved content maintenance strategy in place, which has proven to work effectively. We are confident that Birkbeck staff, already familiar with Plone from their work elsewhere on the site, will find working on the new system an improvement over their experience of Collage.
Future Impact
We have several plans for further improving access to our course information, among them:
More applications of the XCRI-CAP feed to enable efficient sharing of our course information:
- Birkbeck is in partnership with UEL to provide a shared teaching space in Stratford, and we are aiming to use the XCRI-CAP feed to source Birkbeck content for a shared web space to promote our courses.
- We are very interested in the Facebook XCRI-CAP Course Search App that has been developed to facilitate course search in this environment: https://www.facebook.com/XcriCourseSearchApp – and we will be exploring this once we have developed the Birkbeck XCRI-CAP feed.
Populating course finders across the Birkbeck website: we are already planning to make it easier to find our courses wherever you are on the Birkbeck website, by enabling content sharing between school and department microsites and our central course catalogue. This means that, where visitors use these microsites as their entry point onto the Birkbeck web site, they will get easy access to this critical student recruitment information.
Curriculum database
There is a plan to develop and launch a new database that will convert the current paper-based curriculum approval process into an online process. We are expecting that the work we’ve done on our systems will enable us to link our systems, so that we pull approved data into our published information.
Conclusions
General conclusions
‘Sometimes serendipity is just intention unmasked’[1]: the Jisc call for participants in its XCRI-CAP project went out at a time when we were considering that something needed to be done about the systems driving our online course catalogue, but before we had actually identified what that something needed to be and probably at least a year before we would have been able to undertake a project on this scale. What Jisc did was to provide the impetus and, thankfully, the resources that enabled us to bring this important redevelopment project forward and also help us set a timescale to implement it. This project is now a mission-critical part of Birkbeck’s continuing efforts to support recruitment, and we are grateful that we were able to take part.
Conclusions relevant to the wider community
The importance of ring-fenced funding: Without the Jisc funding and the allocated time span, we would not have been in a position to undertake this work – and, even if we had, we certainly wouldn’t have been able to carry on once we lost key project staff. Because the funding for this project was ring-fenced, we were able to outsource aspects of our project to trusted suppliers, which meant that we made up for a lot of lost time once we commissioned them.
Having a specified project lifespan focuses the mind: at the beginning of this project, the March 2013 deadline seemed immeasurably distant, and we were convinced we would finish the work well in advance of the deadline. Yet the first year of this project has flown by, and that deadline – while still achievable for Stage 1 and for our undergraduate XCRI-CAP feed – is now looming large. Nonetheless, having this deadline has proven very helpful, because it has meant that we’ve had to be absolutely clear about what we could achieve and plan our project accordingly.
Conclusions relevant to Jisc
The value of formal reports: we know that the model of providing interim and final reports to Jisc is a tried-and-tested model, so we’re not saying anything new or original. But we have found the requirement to provide progress reports – albeit a somewhat pressured task – a great way to recognise how much we’ve accomplished on this project, and how much we’ve learned – two things that might otherwise have gone unrecognised. Looking back on our first interim report, we can hardly believe how very much we’ve done since then and how much clarity we’ve gained over the course production process and what we want to achieve.
The value of face-to-face project meetings: it was really great to meet other participants in this programme – and reassuring to learn that we weren’t the only ones to have resource issues. It was also really interesting to see how many different ways this particular cake could be sliced, with the same outcomes. We simply would not have realised this without this face-to-face interactions – and by the time we got to the Show and Tell in Birmingham, we felt a real sense of kinship with other project members.
Recommendations
General recommendations
The importance of good communication: our business planning meeting, our minuted project meetings and our use of Yammer enabled us to engage a wide range of colleagues in our project development work and also helped us move this project on significantly, at key points in our process.
Look after your project team: each member of our project team played a critical and complementary role in developing and delivering this project. However, it is important to recognise that they are probably juggling their specific project work with other work responsibilities, and that they have a life beyond the institution. Therefore, it is important to ensure they are well-supported and that you listen to their project concerns and, if necessary, adjust your project deadlines.
Recommendations for the wider community
Always have a Plan B: the loss of key staff early in our process meant that we were unable to achieve our project objectives in the timescale we had originally set, in the way we had originally intended. It took some time to put in place, but our Plan B – where we have outsourced aspects of our project and have also converted the project into more manageable stages – means we will still be able to deliver our project and manage the Birkbeck community’s expectations at the same time.
Take the opportunity to learn from others: the information we gained from other institutions in the Stage 2 programme, as well as those who kindly shared their learning experience, helped us immeasurably. The problems we were facing were the same problems faced by any number of higher and further education institutions, and it was helpful to explore the different solutions colleagues adopted to address them. We are very fortunate indeed that members of the higher education and further education community are willing to share expertise and lessons, and we recommend that anyone planning to undertake a project like this build this into their planning. Once we finish our project, we will happily share our own experience.
Recommendations for Jisc
Managing expectations: a recurring theme at our various project meetings was the frustration felt by some institutions at the fact that UCAS and other third-party course aggregators, all of whom are critical to course information promotion and dissemination, hadn’t yet signed up to the XCRI-CAP standard and therefore were not in a position to pick up the course information that was going to be provided at the time our XCRI-CAP feeds were generated. On Birkbeck’s part, as our project roll-out has been delayed, this hasn’t in any way undermined our work or made us feel that what we were doing wasn’t valuable. However, it might be helpful to make it clear from the start whether suppliers have signed up, if their involvement is a necessary part of a process.
Contact details:
Project Director: Monica Law, Assistant Director, External Relations
Project Manager: Jane Van de Ban, Web and Social Media Manager
Contact email: j.vandeban@bbk.ac.uk
Project Web URL: http://blogs.bbk.ac.uk/xcri/
[1] Elizabeth Berg, The Year of Pleasures. (Ballantine, 2006).
