Authors: S.J. Cotterill (s.j.cotterill@ncl.ac.uk), P.R. Kyle (p.r.kyle@newcastle.ac.uk)

JISC e-Learning Activity Area: e-portfolios

Higher Education Academy Subject Centre: medicine dentistry and veterinary medicine

 

Case study tags: online learning, an effect on learning, an effect on exam results, an effect on student personal development,student satisfaction with e-learning,innovation in learning and teaching, an influence on educational research, staff personal development, stakeholder engagement, a positive effect on recruitment, an influence on policy, use of resources, modifications to learning spaces, management of learning assets,an effect on social equality, tangible benefits of e-learning, newcastle university,e-portfolios, medicine dentistry and veterinary medicine

 

Background & Context

 

Why did you use this e-learning approach?

 

Modern professions require employees to independently advance themselves, often by means of personal reflection and appraisal. The skills required are not necessarily innate. e-Portfolios were developed as a method to help foster a reflective approach to evidencing the achievement of both module-specific and programme learning outcomes. This was in-line with GMC (General Medical Council) guidelines, QAA requirements and the University's e-learning strategy. The system had to be easily deliverable but sufficiently flexible for use in different contexts. The e-portfolio was implemented in the undergraduate medical programme at the University of Newcastle from September 2003. The e-portfolio can be used on a 'stand-alone' basis but is best integrated with virtual learning environments (VLEs).

 

What was the context in which you used this e-learning approach?

 

The medical degree programme has, at any one time, over 1700 students and 1600 contributing staff. The first two years (Phase I) are based at the medical school. There is a joint programme with Durham University, where a smaller group of students complete Phase I at Queen's Campus, Stockton; joining Newcastle University for the final three years (Phase II). During Phase II, students are dispersed over a wide area in the North East of England where their training and administration are provided by four regional clinical centres called 'base units'. This means that throughout almost all of Phase II, students are not physically located on campus.

 

In undergraduate Medicine portfolios/log books were designed to support recording the achievement of learning outcomes and reflection in Phase 1 (Years 1 & 2) and Years 3 & 5. The log books were mandatory but not formally graded for assessment.

 

Before e-portfolios were introduced in 2003, the process was entirely paper based. Due to the nature of the portfolio the document was often very complex in print and inconvenient to carry around hospitals in A4 format. In addition to building on established logbooks, new pedagogy supported by e-portfolios, have been introduced around summative assessment in Year 4 student selected modules (SSMs) and annual appraisal.

 

Anticipated Problems and Challenges:

• One of the challenges for Medicine is that students are dispersed over a wide geographical area, spending 3 years located at various hospitals administered by one of five regional base units. [ref 1] As such reliable access to web-based resources is essential.
• Students with a high workload and tight deadlines require an efficient and reliable system. Thus a large challenge was to design a system that was easy to use and intuitive.
• People wishing to continue using a paper copy
• Curriculum leaders were rightly cautious on introducing new technologies, particularly around 'high stakes' assessment.

 

[ref 1: Cotterill SJ, Bradley PM, Hammond GR. ePortfolios: Supporting assessment in complex educational environments. In Bryan C and Clegg K (eds) (2006) Innovative Assessment in Higher Education, Taylor and Francis Group Ltd, London.]

 

What was the design?

 

The e-portfolio was designed to be highly flexible and configurable on a programme and year-group basis. The design needed to support a constantly changing programme with multiple entry points. This was a lesson learned from the TLTP project (see case study 2) where there was a need to support multiple versions of the curriculum for a 5 year programme - for example a 5th year student could still see the 1st year curriculum as they studied it, not just the latest version. The e-portfolio was developed soon after a major change to outcome-based curricula and the creation of new points of entry (an accelerated programme and joint programme with Durham University).

 

The e-portfolio implemented for Medicine includes support for a range of different pedagogy for different purposes within the curriculum including:

 

• providing a framework for recording and evidencing 'high-level' programme outcomes during the year to support an end of year appraisal
• providing a framework for recording and evidencing student derived objectives during Student Selected Modules (summatively assessed)
• supporting personal tutoring (e.g. recording meetings and actions)
• support for recording, reflecting and sharing information including structured (e.g. CV and clinical log books) and unstructured (e.g. folders for file uploads) information

 

Most of the pedagogy follows a 'blended learning' approach (especially that relating to appraisal and assessment).

 

Further details of the design are provided in Reference 2: Cotterill SJ, McDonald [SH -link wiki page does not exist] AM, Drummond P, Hammond GR. Design, implementation and evaluation of a 'generic' ePortfolio: the Newcastle experience. Proc. ePortfolios 2004, La Rochelle, 2004 ISBN 2-9524576-0-3.. http://www.eportfolios.ac.uk/docs

 

How did you implement and embed this e-learning approach?

 

The e-portfolio was embedded in the VLE used by medical students.

 

The initial implementation of the e-portfolio was done on a trial basis with 1st year students, who were given the choice of using either paper or online versions of the portfolio/logbook. In addition year 4 students were required to complete an e-portfolio relating to one of their 3 SSMs. Initially this was a mandatory requirement for progression but not formerly graded.

 

Detailed evaluation studies (with ethical approval) were undertaken. These used online questionnaires and focus groups. A 4th year student facilitated focus groups for 1st year students as part of his SSM in Medical Education.

 

As anticipated about a third of students elected to use paper-based log books in the initial trial with year 1 students. A third used the e-portfolio, but what was not anticipated was that a third used neither! The evaluation indicated that many students did not give the logbook (regardless of technology) much priority as it was not regularly monitored or summatively assessed (strategic learning).

 

The initial evaluation demonstrated feasibility and efficacy of the e-portfolio approach and as such the following year the Phase 1 log book became electronic only and subsequently the Year 4 e-portfolio is now formerly graded and counts towards final marks for the SSMs.

 

A Teaching Fellowship was obtained in 2005 to extend the e-portfolio to evidence the achievement of high-level learning outcomes to support end-of-year appraisal. This was piloted with year 1 students and following a successful evaluation is being rolled out across all 5 years of the curriculum.

 

Technology Used

 

What technologies and/or e-tools were available to you?

 

We used the 'ePET' portfolio which was developed in-house as part of a collaborative FDTL-4 project (http://www.eportfolios.ac.uk). This was developed on Open Source platforms (Zope, python & MySQL). The portfolio was designed to be used either as a stand-alone system or (as with the case of Medicine) integrated into the VLE.

 

The technology was developed in response to sector-wide requirements for Personal Development Planning (PDP) and for new policy requirements in Medicine.

 

Tangible Benefits

 

What tangible benefits did this e-learning approach produce?

 

Within Medicine the research studies and programme evaluations indicate there have been real benefits to student learning. For example in the initial evaluation of the e-portfolio process for SSMs 80% of students found it to be a useful learning experience and 72% said it influenced their approach to learning. 93% said it led them to reflect following the end of the placement. Likewise, in the evaluation of the annual appraisal supported by e-portfolio 70% of students found it useful and 58% said it would influence their learning in the subsequent year.

 

Where the e-portfolio replaced existing paper-based systems there are reductions in printing, copying and other administrative costs. Where the e-learning supports new pedagogy it is possible that these costs may have been prohibitive if it had been paper-based.

 

There have been significant benefits to the wider institution where the flexible nature of the e-portfolio has meant that it could be applied to support other curricula and a broad range of educational requirements. (Ref 3: Cotterill SJ, Horner P, Hammond GR, McDonald AM, Drummond P, Teasdale D, Aiton J, Orr G, Bradley PM, Jowett T, Heseltine L, Ingraham B, Scougall K. Implementing ePortfolios: adapting technology to suit pedagogy and not vice versa! Proc. ePortfolio 2005, Oxford, 2005). This includes rollout to Bioscience (to support employability and 3rd year projects), Speech and Language Sciences, Dentistry and an institution-wide e-portfolio for postgraduate research students (to promote research and transferable skills).

 

In addition, the e-portfolio is used at 11 other institutions and has been adapted in a number of JISC projects to develop support for interoperability and the transfer of life-long learning records. For further information see: http://www.eportfolios.ac.uk/ePET and http://www.epics.ac.uk/EPICS/

 

Newcastle changed to an outcomes based curricula related to the needs of the NHS and professional bodies as defined by the General Medical Council (GMC). The e-portfolio supports these learning outcomes and the growing demand for reflective practitioners who can manage their own continuing professional development as defined by the GMC 'Tomorrows Doctors' report (2002). See: http://www.eportfolios.ac.uk/FDTL4?pid=47

 

Did implementation of this e-learning approach have any disadvantages or drawbacks?

 

There are few pedagogical drawbacks because the software has been developed in-house and has therefore been carefully tailored for the various curricula requirements.

 

The e-portfolio has been gradually embedded in the curriculum over a 4 year period with careful piloting and fine-tuning. It has been designed not to be too onerous to students and the pedagogy was initially developed to have minimal staff impact (e.g. students could print out SSM portfolios for sign-off, rather than all supervisors having to log on). However, the new approaches involving face-to-face appraisal supported by a summary of evidence from the e-portfolio and the grading of the SSM portfolio by supervisors does represent an increase in staff time (though, possibly less than having introduced this new pedagogy supported by paper).

 

The software has been developed using Open Source products used by the team at the Faculty of Medical Sciences Computing. However, the business services of the central university primarily support systems on Microsoft platforms so collaboration has been problematic, though web service approaches will overcome this.

 

How did this e-learning approach accord with or differ from any relevant departmental and/or institutional strategies?

 

This e-learning approach around e-portfolios has accorded well with faculty and institutional teaching and learning strategies. It has helped meet the requirements for PDP within the institution, professional policy drivers in Medicine and Dentistry, employability in Bioscience and the 'Robert's Agenda' for skills training for postgraduate research students.

 

In the context of Medicine the e-portfolio is fully embedded and used across all year groups. It includes:

 

• PDP
• Recording meetings with personal tutors
• Clinical logbooks
• CV
• SSC
• Evidencing outcomes for end of year appraisal.

 

After careful piloting and evaluation it now has a greater role in formative and summative assessment.

 

Lessons Learned

 

Summary and Reflection

 

The e-portfolio has become successfully embedded in the Medical Curricula at Newcastle University, where it plays an important role in both formative and summative assessment, as well as supporting annual appraisal. The e-portfolio was developed around pedagogic and learning requirements, with a high level of stakeholder buy-in and ownership, rather than being 'driven' by technology. This has been vital to its success, as has the integration with the curriculum (e.g. includes learning outcomes and programme-specific terminology and content). It has taken over ten years, with iterative refinements of both pedagogy and technology, to implement and embed the e-portfolio from concept to mainstream use within Medicine.

 

For others thinking about implementing e-portfolios it is important to start with a clarity of purpose and it needs to be recognised that finding a 'good-fit' with the curriculum will take lots of time and usage, and this will require continuous review (with input from students) as requirements change.

 

Further Evidence

 

"The e-portfolio implemented for Medicine includes support for a range of different pedagogy for different purposes within the curriculum..."

 

"...the initial evaluation of the e-portfolio process for SSMs 80% of students found it to be a useful learning experience and 72% said it influenced their approach to learning; 93% said it led them to reflect following the end of the placement."

 

"There have been significant benefits to the wider institution where the flexible nature of the e-portfolio has meant that it could be applied to support other curricula and a broad range of educational requirements [such as] postgraduate research students."

 

"...the e-portfolio is used at 11 other institutions and has been adapted in a number of JISC projects to develop support for interoperability and the transfer of life-long learning records."