Using an ePortfolio system as an electronic laboratory notebook in undergraduate biochemistry and molecular biology practical classes

Authors


Abstract

Despite many apparent advantages, including security, back-up, remote access, workflow, and data management, the use of electronic laboratory notebooks (ELNs) in the modern research laboratory is still developing. This presents a challenge to instructors who want to give undergraduate students an introduction to the kinds of data curation and notebook keeping skills that will inevitably be required as ELNs penetrate normal laboratory practice. An additional problem for the teacher is that ELNs do not generally have student-administrative functions and are prohibitively expensive. In this report, we describe the use and impact of an ePortfolio system as a surrogate ELN. Introduction of the system led to several pedagogic outcomes, namely: increased preparedness of students for class, encouragement of creativity and reflection with respect to experimental methods, greatly enhanced engagement between students and tutors, and it gave instructors the ability to scrutinize original data files and monitor student-tutor feedback cycles. However, implementation led to a disruption of tutor workloads and incurred new levels of accountability that threatened to undermine the initiative. Through course evaluations and other reflective processes, we reached an appreciation of how an ELN should be introduced into practical class teaching so that it not only becomes an appropriate aid for teaching the laboratory experience, but also becomes a life-long resource for students. © 2013 by The International Union of Biochemistry and Molecular Biology, 42(1):50–57, 2014

Paper-Based and Electronic Notebooks

For as long as Life Science research has been practiced, hard-bound paper laboratory notebooks have been the gold-standard for data collection and experimental observations. Indeed, most of our intellectual property and patent laws are based around accurate, timely upkeep of laboratory journals. Accordingly, instructing students in notebook keeping forms a central part of our practical class culture. However, nearly all the data collected in a modern research lab is electronic and the paper notebook cannot accommodate the large data-sets and varied digital media that comprise contemporary experimental results. For this reason, the last few years have seen an increasing interest in Electronic Laboratory Notebooks.

Surveys of users allowed Machina and Wild [1] to collate an extensive list of benefits conferred by ELNs, and these can be summarized into three main categories: consistency and efficiencies in storing data; the improved ability to search and filter data; improvements in sharing data. Earlier articles, for example Kihlen [2], also cites user-controlled templates as a major draw-card, while advantages such as setting up differential access via security protocols, backup of data, and remote access have always been self-evident. Integrating the ELN with other applications (for example, spreadsheet, graph-drawing, and statistical analysis packages) is important for successful implementation, while streamlining the acquisition of data from instruments to the ELN is a further benefit.

The Use of eNotebooks in Modern Research Labs

Curiously, despite the fact that eNotebooks carry the promise of so many extra advantages, they are not commonly used in modern day research labs. So, despite regular calls for universal adoption of electronic notebooks from prestigious journals like Nature [3], most life-science researchers still record all their experimental plans and observations in a paper journal, separate from the original electronic data which may be stored in disparate locations with variable degrees of backup, security, and cataloguing. The situation is further complicated by the fact that scientists can be very selective about what actually makes it into their paper notebook. For example, although several hundred microscopy images might be captured during a particular experiment, only one “representative” picture may actually be archived in the journal, thus increasing the chances of misrepresentation or bias.

Surely This Is the 21st Century—Why Aren't eNotebooks Used More?

The reasons for the reluctance of researchers to adopt eNotebooks are probably multifactorial; conventions have not been established (intellectual property laws, as mentioned above, are entrenched in a paper philosophy), the options are exceptionally expensive (often thousands of dollars per person per year) and there is no clear market leader or consensus as to how the software should work. In this environment it is not surprising that most University administrators vacillate, paralyzed by the fear of choosing a version that will not stand the test of time and/or suit the needs of the disparate disciplines in each institution.

Indeed the choice is vast, with a recent analysis [4] revealing some 30 separate vendors in this emerging market. This tally did not even include “home grown” products which are developed largely because the commercial software is too expensive and/or does not meet the needs of the specific researchers. Rudolpi and Goosen [5] describe one such experience of developing their own web-based ELN, so obtaining a customized product to suit their needs, whilst others (for example, Walsh and Cho [6]) describe the adaptation of existing cloud-based applications such as Evernote as surrogate ELNs. And here is where distinctions and definitions become blurred, for if one argues that an ELN is just a collection of electronic files related to a particular experiment or lab, then the computer and local network of any laboratory worker is effectively an ELN.

The business case for a well-defined, self-contained ELN often focuses on efficiencies, but these are not always clear cut. There are hidden implementation costs involving, amongst other things, the reconfiguration of laboratory systems and instruments, training staff and maintaining the ELN over its life cycle. A recent commentary [7] neatly sums up the feeling of many bench scientists when it comes to using ELNs through the observation that “forcing a change in work habits to meet the needs of the software is a sure way to prevent adoption”.

It must seem surprising to those from outside life science research that so many bench scientists still use paper based lab notebooks. Although dissecting the reasons for this disconnect with the original, electronic data could warrant a study all of its own, it is reasonable to suggest that when the right product comes along, this situation will not persist. Indeed, it is more likely that eNotebooks will show all the features of a classic Disruptive Technology: an innovation which initially struggles to attract a significant number of new adoptees but, when critical mass is achieved, rapidly displaces an existing paradigm and which, furthermore, creates new ways of thinking and doing things far beyond what we can currently imagine. It is probable that, as eNotebooks become increasingly familiar, researchers will embrace new ways of reflecting on data and interacting with geographically dispersed colleagues—perhaps the results of experiments will be released live as part of grant awarding body requirements or even as part of a new metric in judging research output!

In the meantime, the gulf between what happens at the bench and what is actually recorded widens and it is not clear how we, as educators, should best prepare our students for life in the laboratory of the 21st century.

What Does This Mean for Undergraduate Practical Classes?

We felt that there was an opportunity for the teachers to take the lead in this area and, for once, to introduce an innovation into our practical classes before it becomes established in research culture. Of course, we had to be sure that there were genuine advantages to adopting an eNotebook in an undergraduate practical class environment. Simply reproducing what is currently done on paper with an electronic interface would not have been worth the effort. Similarly, we had to be careful that we would not lose our strong culture in proper record collection, and we had to take into account how the change would affect our tutors' workload.

Which eNotebook to Choose?

Choosing a product that would suit the several hundred students spread across our various Biochemistry and Molecular Biology units presented a problem.

When we started our work in early 2011, commercial ELNs were prohibitively expensive and, just as importantly, did not incorporate student administration features. Therefore, we had to find a product that was able to capture and organize student-created content in a way that could be appraised and certified by teaching staff and class coordinators. The ePortfolio system, PebblePad, subscribed to by our institution, seemed to fulfill those criteria. It had the extra benefit that it was part of our enterprise architecture meaning that students could log in using their University IDs, and that the product was backed up and supported via a professional Help Desk.

What We Asked the Students to Do

Using the PebblePad ePortfolio system, students wrote up every practical session by creating a collection of web pages called a Webfolio. An example of this is shown in Figure 1. Students submitted their webfolios to particular ‘Gateways’ which could be accessed by tutors. Students could continue working on their work after submission, and updates would be reflected in what the tutor saw until such time as a hard deadline was implemented. After providing comments, the tutors could make the webfolios available again for student editing, thus allowing closure of the feedback loop.

Figure 1.

The structure and processing of submissions within PebblePad (a) A Typical Webfolio. A webfolio is essentially a collection of web pages. Links to the pages are on the left (1) and the each page (2) can include all the elements possible in a normal web page. Tutors can leave comments (3) and track back through versions (4) to see when and how the submission was modified. Students were encouraged to create customized page banners (5). (b) The submission system Webfolios (1) are published to gateways (2). The submission and modification dates are recorded (3) and the tutor monitor student interaction with their feedback (4). Submission lists can be filtered so that each tutor can only see their own students (5). Staff with appropriate permissions can put up templates or other materials in the resources section (6). Note: names of students and gateways obscured for privacy. privacy. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Training Was Minimal

Except for a brief introduction to the features of PebblePad, we specifically did not provide extensive training. We assume that, in this day and age, individuals are used to learning and exploring new software and that the latter is written to generally accepted conventions. In addition, we deliberately did not want to make the ePortfolio system appear onerous by running dedicated training sessions. Of course, help was available to the very small number of students who found certain behaviors in the system non-intuitive but, by and large, we wanted to leverage off the inherent curiosity that most science students should have.

Models of Implementation

Across our courses we used a range of different models, with both variation in the expected content of the eNotebook and variation in the materials and guidance provided by instructors. Our content models ranged from a minimal “archiving” approach to one where the eNotebook replaced all the reporting and assessment of practical material. In general, each unit of study used a mix of these models and, although in hindsight it might have been best for student involvement and effort to have gradually increased as a course progressed, each coordinator was keen to establish the blend that they felt most appropriate for their course.

In the Minimalist content model, students were required to simply use the eNotebook as an archive to link and curate electronic files related to the practical class (Fig. 2). The aim was to leave the students with easily navigable structure containing well contextualized links to all prework, results, supporting files, and assessment tasks. This model emphasized the strength of the eNotebook as a searchable and time-robust repository of electronic data. It also provided an accessible entry to the world of eNotebooks where the logistical challenges of a new platform might interfere with content learning outcomes.

Figure 2.

Minimalist Model. The webfolio is simply used to act as a repository for electronic files (data, figures, and reports, etc.) associated with the practical class. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

In the Intermediate content model, in addition to archiving data, students did a fuller write-up of some or all practical sessions. This model had a more traditional format with a formal introduction, methods, results, and discussion. Students were encouraged to also include a relatively informal methods section which we called the “bench log”. This allowed them to reflect on understanding without being too concerned about strict scientific style. The bench log section particularly gave scope for students to include video, animation, and images and questions/reflections. In this model, if students also had to produce materials in hard copy due to the nature of the assessment or the data (e.g., calculations, drawings, ease of providing written feedback), these hard copies were scanned and linked in electronic form for archiving.

In the Full content model, the eNotebook became the sole assessment and reporting instrument, incorporating all prework, lab write-up, reports, and even postwork. In addition, the instructors provided all materials that would otherwise be available in a lab manual (including instructions and background material) within the eNotebook platform (Fig. 3). Some instructors included sections for students to list and provide evidence for the accumulation of skills and graduate attributes. For example, for each practical, two or three graduate attributes were stated by the instructor, and students were asked to illustrate how they had accomplished these. Using a both text and rich media, it was hoped that students would generate a list of practical skills and techniques including a self-assessment of their confidence in those achievements.

Figure 3.

Full Model. The webfolio contains everything required to prepare for the practical (Background Information, 1) and perform the experimental work (Protocol at a Glance, 2). The student records what they actually did (3) with the help of rich media (photos, movies, etc.). All results are written up to presentation standard and figures and tables are supported with appropriate legends (4). Discussion sections and post-practical question pages are followed by a space for students to record their reflections and competencies (providing evidence as appropriate). Any hard copy material (e.g., spectrophotometer readings, gel loading patterns) are scanned and placed on and additional page (5) [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

The Issue of Templates

Another potential variation is the amount of structure provided by the instructor. Initially, students can be provided with eNotebook templates, or can be given autonomy to generate eNotebooks with varying degrees of prescription. A template can provide just an empty framework of folders and pages or can also include materials usually provided in a lab manual. For example background information including links to websites, animations, images, and video as well as instructions for practicals and for assessments. Templates can be increasingly prescriptive, even providing forms with spaces to fill in answers or detailed checklist of thus becoming more like a “workbook”.

eNotebooks generated autonomously (without templates) obviously show a wider range of proficiency and creativity, with stronger students producing more organized and thorough eNotebooks. It could be argued that the use of templates may assist weaker or less creative or technically proficient students to begin on a level playing field, and to concentrate more on content than on design. However, this comes at the expense of students being able to produce their own structure and engage in organizing their work. An advantage of a template structure is that a consistent structure increases ease of marking.

The Impact That the eNotebooks Had

Prework—Finally Students Come Prepared for Class

The first and most dramatic impact was that, for the first time in decades, it was practical for us to set and mark prework. Through PebblePad, tutors and students discussed prework 2 or 3 days before a practical class. This meant that students were familiar with the content of the laboratory sessions, and tutors were made aware of general misconceptions well before the actual class. Indeed, for many classes in our second semester of implementation, it became obvious that it was no longer necessary for us to have in-class prepractical tutorials. This represented a huge pedagogic improvement since these sessions are normally conducted in a sub-optimal environment (a noisy laboratory with poor visuals and seating arrangements). It also meant that more time was released for hands-on practical work and this increased the opportunity for repetition and redesign of experiments—thus mimicking the genuine research laboratory experience. Of course, these benefits would be returned from the implementation of any system that facilitated prework submitted in a timely fashion and are not a specific feature of eNotebooks. However, being able to administer the prework through the same system as all the other tutor/student dialogue clearly facilitated this change.

Bench Log—Students Actually Began to Show Some Interest in the Methods

Since we provide students with an extensive laboratory manual, it had become common for students, in their paper-based write-ups, to simply describe the methods used as “as in the lab manual” However, the ELN encouraged the students to describe the methods in their own terms, with reference to pictures and movies taken on their smart-phones. As a result, they responded with stunning creativity, producing expansive narratives rich in various kinds of media, even creating animated “time-lapse” compilations of some experimental procedures and time-courses.

Results—Data Now Kept Together With the Report and Accessible to the Tutors

As is the case with the traditional paper notebook, students copied/pasted in graphs, tables, and figures into the data section of their eNotebook. However, as in real on-line electronic journals, the ELN system allowed figures to be presented as thumbnails and then expanded if desired. Most importantly, students attached the original data files to their write-ups. So, in contrast to a paper-based notebook which can only present a static snap-shot of the data, we can check calculations and refer to the original data in its entirety if we wish.

Reflections—Students Feel More Free to Express Themselves in PebblePad

By encouraging a free-flowing narrative, we have found that the students are more likely to reflect on the techniques used and results obtained. Clearly, since training in scientific writing is one of our key aims, caution should be exercised in allowing a too casual expression style to develop, but there are obvious benefits in encouraging an increased willingness to interpret data, suggest trouble-shooting strategies, design further experiments, and put forward hypotheses.

Competencies—Our Great Vision Has Not Been Realized

One area in which we have had mixed success is in the documentation of skills. It is regrettable that recently graduated students are often unable to identify the specific proficiencies that we hope we have taught them. This is especially true for the less easy-to-define attributes such as critical thinking, problem solving and communication. We were hoping that, by identifying particular skills during practical classes, we could convince the students to document how they had acquired or nurtured those attributes. Sadly, this task did not resonate with the students and we will be grateful for advice from colleagues on how to make this work in future.

It Wasn't All Plain Sailing

Of course an operation of this magnitude and originality was not without its problems. As noted above, the implementation model differed somewhat between courses and this caused some confusion when students were enrolled in several units. In particular, when some retention of traditional notebooks or hard-copy report was involved, this caused confusion about what should go in the ePortfolio and what should be archived on paper. Sometimes we were over-ambitious in using too many of the ePortfolio features. PebblePad allows the creation of several different types of submission (achievements, thoughts, plans, etc). This diversity created confusion and, indeed, student creativity was actually at its best when they were given the minimum level of guidance. Disappointingly, however, the students found the software frustrating to use. The upload of rich media was particularly problematic and, the construction of product by the students was cumbersome and time-consuming.

Probably the biggest problem of all was a huge escalation in the expectations and work-load of the teaching staff. The on-line interaction with the students was time-consuming, continual and, often, exigent in nature. One other interesting cause of tension was the loss of the prepractical tutorial. This came as an affront to many tutors because of its high remuneration to effort ratio combined with a perceived high professional status. Most tutors would prefer to be “the sage on the stage” rather than having to respond to random (and often difficult) questions. Even allocating more resources (and a higher rate of pay!) to the on-line engagement did not fully restore tutor empathy. Another contributing factor was the fact that tutors felt overly scrutinized. The ELN allows course organizers to monitor all activity (both student submissions and tutor feedback), including the timing and richness of teacher–student interactions. Tutors could no longer get away with providing cursory feedback, and if conflicts with students arose over marks or comments, then the course coordinator could arbitrate remotely without even having to consult either party. Of course, the inevitable consequence of a loss of tutor confidence in the system is that they communicate this to the students. Any teaching innovation is doomed if the coal-face teaching staff are not fully enthusiastic. Therefore in current implementations we have tried very hard to get buy-in from the tutors.

Another issue relates to the relevance of the ELN to some specific sessions. The eNotebook is particularly suitable for labs where the bulk of the data is electronic. Students can appreciate the value more when they do not feel as if they are double handling data (for example, by having to produce a hard copy report as well as printing out for pasting into a traditional notebook). However, when data were directly transcribed from a machine on to paper (for example with older spectrophotometer or balance readings), the students perceived the eNotebook as unnecessarily increasing their workload.

The Student Voice

The ELN was introduced into our practical classes in the various modes described above over 2012–2013 (>800 student enrolments). As shown in Table 1, the student voice was garnered using anonymous, voluntary online surveys (using directed questions and freeform comments), compulsory paper-based surveys, and in informal discussion with students.

Table 1. The student voice
ClassSecond year Biochemistry and Molecular Biology (two units), Sem 1Third year Biochemistry, Sem 1Second year Biochemistry, Sem 2Third year Biochemistry, Sem 2
  1. The eNotebook was implemented in various models (Minimalist, Intermediate and Full) as described in the text. Students opinion was obtained via either voluntary, online surveys or in-class, paper based forms.

Survey modeCompulsory in-class, PaperVoluntary, OnlineVoluntary, OnlineVoluntary, Online
Response rate (number/total class, %)>300, >320, >95%31/124, 25%65/150, 40%49/161, 30%
Results>80% negative∼70% positive∼90% positive∼60% positive
Implementation ModelMixture of intermediate and full, multiple submissionsMixture of intermediate and minimalist, multiple submissionsIntermediate, single notebookMinimalist, multiple submissions

Three major conclusions emerged: (i) that the platform needs to be easy to use (and the students were highly critical of the performance of the current ePortfolio platform); (ii) that the students could appreciate that secure storage and curation of laboratory data is important and is the way of the future; (iii) that successful implementation depends on a clear and strong purpose and direction being conveyed to the students by coordinators and tutors.

As a result of overwhelming negative feedback in our initial implementation in both Biochemistry and Molecular Biology classes at the 2nd year level, substantial changes were made in the following semester, most notably setting up a single ELN with multiple sections (requiring only one submission) rather than a system of separate submissions for each practical session. In addition, more was done to foster tutor support, mainly through strict attention to appropriate work-load/remuneration analysis but also through selling the benefits of being involved in a teaching innovation and giving them confidence that they could perform away from the protection of the lectern. Since then we have even had tutors showing the initiative to use particular features of the ePortolio system (such as group blogs for the class exchange and discussion of experimental results).

Conclusions—The Worth of a Notebook as a Professional Resource

In pondering upon whether there has been (or will ever be) any benefit to using an ELN in practical classes, it is perhaps relevant to reflect on a small, but noticeable, cultural change that becomes apparent at the end of the semester. In the old paper-based system, the students submitted their notebooks for a general inspection at the end of the course. After final checking, grading and feedback, the notebooks were stored in boxes awaiting student collection (Fig. 4). A significant percentage of students never come to pick up their work and, eventually, the notebooks are discarded. Not only does this mean that all the students' efforts and all the feedback from the tutors is wasted, but it also shows us that those students do not actually value their notebooks as a lifelong resource. While it is naïve of us to imagine that the electronic system will suddenly change student attitude, it does tell us that in order for any notebook to be effective, the students must trust it as their resource and not just something that they complete to satisfy us. They need to depend on what they put in their ELN in the same way as a post-graduate or real laboratory researcher worker relies on their notebook. Ultimately, their ELN needs to represent a portfolio of their professional thoughts and abilities for life. Accurate and timely recording of workplace activities is a valuable and transferrable skill across the professional spectrum and these activities are increasingly likely to happen with electronic tools.

Figure 4.

Unwanted Laboratory Notebooks At the end of each semester several students neglect to collect their laboratory notebooks, despite each containing a full record of their experimental activities, reflections, tutor feedback, and other resources. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

As we move forward with our experimentation with ELNs, we see the ideal situation as one where cross-faculty disciplines are all using the same system, allowing protocols and procedures introduced in one course or semester to be used again elsewhere, allowing assessments to be set in which students are required to revisit data collected over a period of time and where cross-disciplinary connections can be nurtured.

Ancillary