Lead Contact: George Roberts (groberts@brookes.ac.uk)
JISC Programme: DeL Regional Pilots
Lead Institution and Partners: Oxford Brookes University (L), University of Brighton, Association for Learning Technology (ALT), Knowledge Integration Ltd. (K-Int), Abingdon and Witney College, Plumpton College, Sussex Downs College, Thames Valley Professional Instutes Partnership (TVPIP), Oxfordshire Community Learning Support Unit (CLSU), Brighton Community University Partnership Project (CUPP)
Project Dates: April 2005 - March 2006
Case study tags: online learning, e-portfolios,oxford brookes university, e-portfolios for application, e-portfolios considerations - identity management, e-portfolios considerations - customisation, e-portfolios considerations - defining requirements,employability drivers for e-portfolios, e-portfolios for evidence and presentation, e-portfolios considerations - it skills, learner perspectives on e-portfolios, e-portfolios considerations - legal, e-portfolios considerations - open source, e-portfolios for cpd and pdp, e-portfolios for reflection,widening participation drivers for e-portfolios, e-portfolios for supporting learning processes, professional body perspectives on e-portfolios
Background & Context
What is the background to the e-portfolio initiative?
There is growing interest in the implementation of e-portfolios at all levels of education. The Government's e-strategy anticipates that schools, colleges and universities "will want to develop eventually an e-portfolio where learners can store their own work, record their achievements" (Dfes, 2005). The e-learning strategy for higher education also has as an objective "encouraging e-based systems of describing learning achievement and personal development planning" (HEFCE, 2005).
If e-portfolios are to become so widespread, it is important to understand the impact of using such tools for learners, practitioners and institutions.
The PETAL 2 and myWORLD projects set out to build on the results of the Personal e-portfolios for Teaching and Learning (PETAL 1) which aimed to support learner-centred, reflective and dialogic learning practice. The project was based on existing partnerships for lifelong learning and widening participation. The project extended and built on learning domain services and common services tools and systems for personal development planning (PDP). Like the first PETAL project, myWORLD and PETAL 2 set out to adapt, implement and trial the Open Source Portfolio (OSP) in a range of post compulsory educational contexts: HE, FE, adult community education and professional institutes.
What were the aims and objectives of the initiative?
The main aim of the PETAL 2 and myWORLD projects was to develop, test and evaluate the use of an Open Source OSPI e-Portfolio in a range of post compulsory education settings.
The projects combined technical and pedagogic aims. Technically it was intended to adapt Open Source OSPI for use in the different post compulsory education settings and to develop a stable installation. From a pedagogic viewpoint the intention was to pilot and evaluate the use of e-portfolios with up to 100 end users. This involved conducting 10 case studies across the South East of England working with up to 100 end users and evaluating these use cases and their impact on learners and institutions.
How was the initiative implemented?
There were five main stages to the project:
- Software development which followed an iterative development cycle
- Test installations
- Reiterations
- Test users
- Evaluation using a case approach with data collection through a survey, focus groups and interviews
The project had two centres: Oxford Brookes University and the University of Brighton. Each centre worked with a group of established partners: FE colleges and community learning projects. Oxford Brookes also worked with representatives of local branches of professional institutes.
The two projects were consolidated under a single steering group. All technical development work was undertaken by Knowledge Integration.
After brief initial familiarisation, case study leaders were asked to write a scenario of use using a template. Then they were asked to develop a hierarchy for the local implementation of the software.
The collection of detailed user data became focused on five case studies representing further education (Abingdon & Witney College), higher education (Visual Arts at University of Brighton, and Viticulture at Plumpton College), adult and community learning (Access to Arts at University of Brighton) and professional (Chartered Management Institute). Visits were made to five case study sites to conduct interviews with the project leaders. Visits at four of these sites included a focus group with learners.
The survey was developed in order to gather information about the learners using the PETAL tools and to elicit their expectations and early experiences of using the e-portfolio tool.
While the survey was intended to gather initial experiences, the focus groups were planned to be conducted after the learners had had more time working the e-portfolio tool.
Interviews with the project leaders were conducted during the site visits. A semi-structured interview was prepared in advance which was intended to elicit the project leaders' views on: the intended pedagogical rationale for using e-portfolios on their course, and their perspective on the extent to which this had been achieved; their use of the tool including issues around usability and customisability; their experiences of being a partner site including the impact of their involvement in the project within their institution.
Technology Used
What technologies and/or e-tools were available to you or did you seek to develop?
The main aim of the PETAL 2 and myWORLD project was to develop, test and evaluate the use of the Open Source OSPI e-portfolio in range of post compulsory education settings by building on the work of PETAL 1.
In terms of the methodology, the software development followed an iterative developmental cycle:
- take the current OSP tools, make a local instance
- trial with users/requirements gathering
- update and refine based on initial user experience
- install updated software
- re-trial and re-gather new requirements
PETAL 1 went through three iterations of the cycle with OSPv1.5, adapting the tool to serve the need of ALT's Certified Member of ALT scheme (CMALT).
myWORLD went through two iterations with a larger trial base.
- Case Study leaders were provided with a "vanilla" OSPv1.5 based on the PETAL 1 project software
- Case study leaders were to evaluate and gather localisation requirements to adapt the database to the needs of their particular learning situation
- Localisation of the User Interface (UI) and hierarchy
- Use with trial groups and gathering further requirements
- Revision of UI and development of output templates
- The continuation project developed and tested PETAL 2 based on the OSP 2.x version
Case study leaders were asked to develop a hierarchy for the local implementation of the software. This revealed a number of difficulties which underlay the OSP tools including issues with changing the data structure. Another problem was that, although the OSP e-portfolio tool was supposedly generic, it became clear that the taxonomy had a very US, undergraduate focus. In addition, the data hierarchy could only be edited by an administrative user giving it an institutional rather than learner focus. Hosting multiple instances proved difficult and substantial input from developers was required to resolve this. Generally the set up was not as easy as hoped and required advanced system administration and web deployment skills.
There were also version application issues based on functionality (e.g. customizing the hierarchy) and pedagogical assumptions (lack of adaptability to apply to different educational contexts). Although some of these were resolved in later releases of the software, significant issues remained.
Success Factors
What are the key outcomes of the initiative?
Outputs - Case Studies
Access to HE, Abingdon and Witney College
The e-portfolio was introduced to learners taking the Access to HE course. The e-portfolio was intended to build up a more complete picture of their accomplishments to date and encourage them to see these events as a reflection of their ability to pass the Access course and complete a university education.
Visual and Performing Arts, University of Brighton
The Visual and Performing Arts programme at the University of Brighton took the opportunity of myWORLD to see if the PETAL tool could help the university respond to its commitment to providing PDP tools for its learners. The e-portfolio was intended to support both standard CV building and the process of documenting a piece of work that learners are creating.
Viticulture, Plumpton College
As part of the Viticulture BSc, students in their final year can take a 10 point Career Development module. Rather than producing a standard CV, the e-portfolio was introduced to encourage learners to reflect on their personal skills and establish the habit of collecting and selecting evidence to support their job applications.
Access to Art at Brighton
The Access to Art course has enabled artists with learning disabilities to develop their arts practice. By using PETAL, learners could create an e-portfolio of their work which could be promoted to galleries and be shared with case workers and advocates. As well as recording their progress, the e-portfolio could act as a valuable prompt for their future goals and ambitions.
Chartered Management Institute (CMI)
The Thames Valley Professional Institutes Partnership (TVPIP) and the Chartered Management Institute (CMI) saw the potential of e-portfolios to support professionals' continuing professional development. Maintaining an e-portfolio could allow members to record their CPD, set targets, and demonstrate how they've applied their skills. The resulting e-portfolio could be used to organise evidence to encourage applications for Chartered Management accreditation, as well for other CVs, job applications or own marketing.
Observations and Issues
Some of the common themes which arose from the case studies included:
Despite the logistical difficulties caused by the size and complexity of the project, the project was able to describe a scenario for e-portfolio use from each of the case studies and uncover many learning points about the introduction of e-portfolios into learning and teaching contexts.
In terms of learner experience, the main finding was that the software was difficult to use and in some aspects did not work sufficiently to complete the intended tasks. This was made worse because learners had high expectations of the technology in terms of functionality and usability. The ability to customise and personalise presentational work in particular was of particular importance.
There were clear issues with the general IT literacy/capabilities of the participants with learning disabilities. However, the participants generally enjoyed the sessions and were clearly excited about the prospect of gathering together examples of their work for display to others. As with the visual arts students, the shortfalls in flexibility with regard to presentation output were the most critical issues. The use of video as a means of capturing participant input was also trialled and proved successful, both in terms of eliminating some of the accessibility issues and in capturing a more 'realistic' reflection of the thoughts and experiences of the participants.
The extent to which the case studies had been able to achieve their aims was restricted by the technical difficulties. Some tutors had specified real, practical outputs in terms of employment or accreditation. However none of the case studies progressed to the stage where learners used the tool to achieve their goal. In part this was due to technical difficulties which resulted in a late start so application deadlines were missed. It was also due to technical difficulties in creating the final presentations: the instability of the system and its lack of flexibility to create different presentation templates or allow for learners to customise the look of their own presentations.
Some of the project leaders reported that their involvement in the project had had a small impact on their role within their institution. Those at Abingdon and Witney (Access to HE) and University of Brighton (Visual and Performing Arts) were able to engage in institution-wide discussions about the possible role of learner personal development planning as a result of their involvement in myWORLD.
Lessons Learned
What are the lessons learned from the project?
There are three different areas that get blurred in e-portfolio systems design and implementation - pedagogical design; domain knowledge representation; system interoperability. If these three areas are given equal attention then interoperability and usability will be facilitated.
The lack of customisability with the system was a disappointment and highlighted that it is an important consideration for practitioners and users.
Although learners saw the potential offered by e-portfolios in terms of employment, it was highlighted that acceptance of e-portfolios has to come from the industry and employers. Some learners expressed concerns as to whether potential employers would bother to logon to their site.
The teacher is crucial to achieving success with "learner" uptake of the system. Where the teachers had been most active in conceiving and designing the pedagogical intervention that the portfolio was supposed to serve, the best results were achieved. Where the portfolio was a late, optional, difficult bolt-on to an already full curriculum and the teachers had not had much input to the pedagogical design, there was less chance of user uptake.
Recommendations made on the basis of the user experiences are:
Local customisation
The e-portfolio system needs to provide the ability for project leaders to customise their hierarchy and interface locally to suit their learners and their courses. Staff need technical support with this customisation.
Reliability
The system needs to work reliably to save students' work and allow them to produce their final presentations and share them with others.
Usability
The system would benefit from an improved user interface with larger fonts, more readable text colours, larger buttons and easily recognisable and colour-coded symbols.
Functionality
Staff and learners made many suggestions to improve the functionality of the system. The dominant request was for more flexibility in personalising both data input and the output presentation.
Support
Learners requested more support with using the system including guidance and support which is embedded into the system.
Data protection
Learners are aware of data protection and privacy issues around working online and need clear answers and reassurance on what happens to their data.
Timing
It is important to have any new software available at the start of each course and for learners to know for how long their data will be stored and they will have access to it.
Task design
The learners who were most able to express the benefits from working with the system were those who had worked on authentic, relevant and timely tasks. Devising such tasks is likely to be important to the uptake and value of e-portfolios by learners.
Employers and accreditors
Learners are keen that those receiving the outputs from their e-portfolios are welcoming of their work. Future e-portfolio developments would be enhanced by the involvement of employers and/or accreditors.
Networking
It would be helpful to promote staff at project sites working in pairs or teams and greater networking between case studies in order to motivate and support each other throughout the implementation and evaluation process.
Further Resources
myWORLD Final Report
myWORLD Abingdon and Whitney case study
myWORLD Plumpton case study
