P. J. Ford The University of Queensland School of Dentistry Brisbane Qld 4000 Australia Tel: 61 7 33658085 Fax: 61 7 33592173 e-mail: firstname.lastname@example.org
Context: In this time of rapid expansion of the scientific knowledge base, subject matter runs the risk of becoming outdated within a relatively short time. Instead of adding more content to already crowded curricula, the focus should be on equipping students to adapt to their changing world. The ability to access, evaluate and apply new knowledge for the benefit of patients has been acknowledged as an important goal for dental education. Information literacy is key to achieving this.
Methods: An information literacy programme for first year oral health students was instituted. This was integrated within a biosciences course and linked with its assessment. Small group instruction reinforced by the use of a tailored online Assignment Guide was used in the context of a specific task. Effectiveness was measured in terms of assessment outcome, processes used and student experience.
Results: Twenty-seven students participated in the intervention which was effective in enhancing foundation literacy skills and confidence of students in accessing and evaluating information sources in the context of a clinical problem. Improvement in higher level literacy skills required to articulate this information in the synthesis of a scientific review was not demonstrated.
Conclusions: Integration of this information literacy programme within the learning activities and assessment of a basic sciences course resulted in significantly enhanced information literacy skills. As this is highly relevant for higher education students in general, the wider promotion of information literacy should be encouraged.
The explosion in information made available by recent advances in biomedicine and biotechnology has made it impossible for individuals to keep pace with all aspects. ‘This is a world which is radically unknowable: even though we may make modest gains here and there, our ignorance expands in all kinds of directions’ (1). For oral health students, these rapid advances in biotechnologies mean that in their future practice, they will be required to have awareness of, evaluate and implement appropriately techniques, materials and therapies as they emerge. Recognition that students must be equipped by their undergraduate degree to cope with these challenges is enshrined in the Graduate Attribute statements published by most Australian universities; for example, graduates of the School of Dentistry at the University of Queensland will acquire ‘the ability to engage effectively and appropriately with information and communication technologies’ and ‘the ability to collect, analyse and organise information and ideas’ (2). The critical learning outcome becomes the ability to learn when a need is recognised (3). This ability to learn is critically dependent upon the attainment of information literacy. At the recent Dental Education Global Congress, information literacy emerged as one of the key recommendations for the continued relevance of the profession.
Information literacy should be taught so as to ensure students know how to access, evaluate and apply new knowledge in the molecular biosciences and technologies for the benefit of patients (4).
What is information literacy?
The Association of College and Research Libraries (ACRL) has developed competency standards for information literacy which have been widely accepted in higher education (5). Information technology skills, such as the use of software and databases, are required; however, information literacy is much more than simply computer skills. These US standards were reviewed for use in the Australian higher education context by the Australian and New Zealand Institute for Information Literacy (ANZIIL) with information literate people defined as ‘those who know when they need information, and are then able to identify, locate, evaluate, organise, and effectively use the information to address and help resolve personal, job related, or broader social issues and problems’ (6).
The broadly understood but often ill-defined concept of academic or discipline-based literacy encompasses a range of skills which include information literacy, critical thinking and problem solving as well as written and oral communication (7). As such literacy may be defined as ‘being able to participate in appropriate ways in the discourse of one’s chosen discipline, to enquire, interpret, hypothesise and challenge – in short to negotiate meaning’ (8). This definition describes a professional and it is important to realise that it involves a broader and more complex set of attributes than the ANZIIL definition of information literacy. The intervention reported here was designed for first year students and addressed the foundation skills of information literacy. A recent report (9) describes a similar programme of library instruction and practical testing embedded within a first year university science course. These authors found that this approach was successful in its aims of enhancing students’ information literacy skills and suggested that longitudinal studies are required. Our subsequent research will involve a curriculum approach to build on these learning outcomes and to develop progressively higher level literacy skills throughout the programme.
Strategies for promoting information literacy
Library skills classes are a commonly used method for the development of information literacy and Owusu-Ansah (10) argues for a separate course that ensures every undergraduate is provided information literacy instruction. He sees librarians as best suited to teach information literacy and predicts that the academic library will evolve into a distinct and formal teaching department. Many such as Elmborg (11) and Maybee (12) however support an integrated approach. Galvin (13) points out that what librarians sometimes overlook is that information literacy encompasses skills and concepts which are ‘learned over time, both in and outside the library’. She maintains that active learning experiences are needed in addition to any classroom instruction.
The first year of university study is the critical time to engage students with their learning community and to provide the framework and resources for the acquisition of academic skills, including information literacy, necessary for their success as undergraduates and beyond (14, 15). In the first year of the Oral Health Program, students are introduced to student-centred learning environments, where learning activities develop problem solving skills, facilitate deep learning providing the basis for clinical reasoning in dental practice (16). First year students are also required to produce written reports which critically analyse currently available information. Information literacy learning activities should therefore be initiated in the first university year.
The aim of the educational intervention reported here was to foster information literacy for students in a first year course in the Bachelor of Oral Health Program at the University of Queensland. The intervention consisted of the following learning activities and assessment tasks embedded within a first year biosciences course for oral health students.
1Baseline formative information literacy practical test (immediately prior to InfoSkills Day, week 1)
2Information literacy workshop including a questionnaire and followed by a second formative practical test (InfoSkills Day, week 1)
3Assignment (literature review on an aspect of the course) with information use record and reflective essay (week 6)
4Summative practical test (week 6)
This embedded approach was used as the literature supports the integration of information literacy within the curriculum and the use of relevant and active learning experiences. The effectiveness of this intervention was measured in terms of assessment outcomes and student experience.
The guiding question was – does the intervention enhance information literacy? Specifically, does the intervention improve the process of finding suitable information sources; does the intervention improve the end product, which is the written report; and does the intervention improve the student experience and confidence in accessing information and producing a report?
All 27 students enrolled in a compulsory basic sciences course in the first year of the Oral Health Program at The University of Queensland experienced the intervention. The students were mostly young female school leavers with English as their first language. All students regularly used computers and the Internet with a large proportion reporting very frequent use. Data were also collected from 22 second year students and 19 third year students in this program. All student year groups had received annual library skills workshops previously, although these were not embedded within courses and were therefore decontextualised experiences. This project was approved by the institutional human ethics review committee and informed consent was received from all participants.
The intervention consisted of a number of learning activities and assessment tasks which took place during the first 6 weeks of semester 2 in a first year biosciences course.
Students first completed a practical test which functioned as a baseline measure of their information literacy skills. A clinical problem (Appendix S1) was given and students were asked to find three appropriate information sources which would be of likely benefit in solving the problem. No guidance or assistance was offered. Student responses were assessed using predetermined criteria (Appendix S1); however, these marks did not count towards their final grades. A questionnaire was then administered which provided additional data on students’ information literacy and demographics prior to the intervention (Appendix S2). An InfoSkills workshop followed. This workshop was conducted by the librarian in small groups with each student having access to a computer. The students were shown the following and then given the opportunity to practise for themselves.
1How to search the library catalogue
2How to search a database (Ovid Medline)
3How to find and evaluate Internet resources
Immediately following the workshop, a web-based Assignment Guide (Appendix S3) was explained. This was tailored to a specific assessment task and provided some initial resources and hints to get the students started with their information search. Responses from a practical test given immediately after the workshop allowed direct comparison with the pre-test and therefore gave a measure of the effect of the InfoSkills session in enhancing information literacy skills. In order to obtain data on the students’ experience of InfoSkills Day, they were given the opportunity to provide anonymous written feedback.
Assignment and information use record
Students were required to write a scientific review of a given topic. Using a template located on the Assignment Guide, students were also required to record strategies and resources used. The template included a space for the students to write a short reflective essay on their use of information resources. As reflection is a ‘process by which learners connect theory and practice’ (17), this activity was designed to ask the students what information literacy meant for them and what issues remained unresolved. This was intended to guide the students to conceptualise information use in more complex ways as encouraged by Maybee (12). These reflections were examined as a measure of the impact of the intervention on the student experience.
Information literacy practical test
Following assignment submission, students were given a final information literacy practical test. This time, the task was assessed and the marks contributed to the final grade for the course. The questionnaire was administered again at this session. In order to compare information literacy skills of this student cohort with other students in the same programme, current second and third year students were invited to participate in this practical test.
Practical test results for the first year students were compared across different time points – before the workshop, after the workshop and after completion of the assignment. The results for first year students at this last time point were then compared with the results achieved by second and third year students who had not taken part in the information literacy intervention. The use of data from both the second and third year cohorts enabled a cross-sectional evaluation of current information literacy of the entire Oral Health Program. Mean scores for the practical tests were calculated and pairs of groups were tested for significance using the Student’s t-test. Data were paired for analysis of results for the first year students at different time points. The proportion of students successfully completing the practical test was compared across time points for the first year students using McNemar’s test for matched samples, and across year groups using the chi-squared test.
The students experiencing the intervention showed a significant increase (from 1.07 to 2.12) (P < 0.0005) in mean practical test scores immediately after the InfoSkills session. Scores increased somewhat after the assignment had been completed (from 2.12 to 2.28), however this was not significant (Fig. 1). Students who had not experienced the intervention (second year students 1.45; third year students 1.95) scored lower than those students who had (first year students 2.28) and for second year students this difference reached statistical significance (P < 0.007) (Fig. 2).
The data were also analysed to examine the percentage of students who had successfully completed the task, that is had found three relevant information sources. The percentage of students achieving this outcome dramatically increased after the InfoSkills session, from 4% (1/27) to 37% (10/27) (P < 0.01). There was a further increase to 48% (13/27) after assignment completion (Fig. 3). Surprisingly, given the additional experience of the third year students, the percentage of students successfully completing the task (32%; 6/19) was lower than for the first years. The second year student group had the lowest number of students who were able to successfully complete the task (18%; 4/22) and this was significantly lower than for the first years (P < 0.05) (Fig. 4).
Comments by first year students after their InfoSkills workshop were overwhelmingly positive and demonstrated a high level of student satisfaction and a student perception of learning; for example ‘…actually found 3 articles! Better use of keywords, now know how to search…’.
The initial questions provided demographic and computer use information about the students. The remaining questions allowed a score each for ‘Catalogue Searching Skills’, ‘Database Search Strategies’ and ‘Plagiarism Awareness’ before and after the intervention. Scores in each of the categories of information skills increased after the intervention. The per cent of correct responses increased from 77% to 83% for catalogue searching skills, from 56% to 69% for database search strategies and from 81% to 89% for plagiarism awareness.
Assessment task outcomes were analysed using comparison of semester 1 assignments with those submitted by the same students in semester 2. The use of criterion referenced assessment was used to facilitate standardisation of the assessment. Despite significant enhancement of the performance of the students in the practical tests following the intervention, this was not mirrored in the mean assignment marks, which showed only a small increase in semester 2 (from 74% to 77%).
The reflective essays however indicated marked changes in the students’ practice and understanding of information literacy.
I tackled the assignment firstly by writing my research questions. This is not how I have started my assignments in the past, so it was a new experience…I must admit, had I not had to write down my research strategies into a template, I probably would not have used these techniques. However, now that I have, I will definitely be using the planning strategy for all future assignments.
Throughout the intervention, the authors recorded their reflections on the progress of the intervention. The positive and productive experience of the academic/librarian collaboration was of particular note. The librarian felt that the insight gained into the effectiveness of the library classes would be valuable in informing her future teaching. The dramatic increase in confidence and skill of students in accessing and evaluating information sources in the context of a particular clinical problem was felt to be an extremely useful learning outcome for clinical practice. The need to specifically address academic writing skills was acknowledged and this will be incorporated into the intervention for the next year. A satisfying culmination of the intervention for teachers and students alike was the acceptance of a student manuscript for publication by the regional professional newsletter.
The impact of the intervention was evaluated on both the ‘process’, that is the demonstration of information skills and the experience of the students and also the ‘product’, that is the submitted work. The use of the practical test for current second and third year students has provided data to allow comparison across three cohorts. The value of the data is that it allowed comparison of the cohort experiencing the intervention with other cohorts who had not. Obvious limitations are the intrinsic differences between the groups, different levels of experience – both academic and clinical, and the recency of exposure to information skills-learning activities.
The marked differences in performance of the practical test across year cohorts are particularly surprising as the content of the InfoSkills session had been delivered to all year groups on several occasions throughout the Program. These workshops commenced with database searching in year 1 and progressed onto reference managing software in year 2 and evidence-based oral health in year 3. This content had always been tailored around a specific assignment task, however never assessed as such. The second and third year students would be expected to have had more opportunity to develop their information skills and to understand the clinical basis of the given problems more fully, however the differences may be at least partially explained by the time elapsed since their last formal InfoSkills session. This period was up to 1 year for the third year students. Lack of retention of these skills however signals a surface approach to learning and/or limited opportunities to put the skills into practice (18). Embedding literacy instruction within a course and its assessment provides an immediate reason for using the newly acquired skills and this approach has been used successfully in other disciplines (19) as well as in the current intervention.
It would appear that a related reason for the success of this intervention was its close alignment to the assessment tasks. Following Biggs’ model of constructive alignment, all components of the curriculum – learning objectives, learning context, assessment tasks and marking criteria – were aligned, so that each student was ‘entrapped’ in a ‘web of consistency’ that optimised his/her learning (18). This made the information literacy programme highly relevant and immediately useful for the students, guiding them towards deeper learning approaches. The nature of the online template for the assignment was critical as it led the students through the required steps from formulating the research question through to search strategies and reflection on this process. These results support the assertions of Galvin that a combination of approaches to the learning of information literacy, including active learning experiences, is beneficial to students (13). Despite the familiarity of today’s students with information technologies, they seem ill prepared to make use of the vast amount of information available and are attracted to search engines such as Google because they believe that will save time (20). Student comments show that although library instruction can make students aware of scientific databases and searching techniques, unless they are specifically required to use them, they remain unconvinced of their usefulness.
Whilst the majority of students learnt to successfully find information during the course of this intervention, few were able to use this information appropriately in their written report. There may be several explanations for this. First, the intervention may have improved the ability of the students to access relevant literature, but they lacked the skills to use these information sources effectively in the production of their report. Therefore information literacy was demonstrated according to the narrower ANZIIL definition, but academic writing skills, as part of a broader concept of literacy as outlined by Kirkpatrick and Mulligan (8), had not been addressed. Rather predictably then these skills did not improve. Critical thinking, use of language, structuring and argument have been put forward as four core criteria for academic writing (21). The higher level skills of critical thinking and argument in terms of a scientific review are highly dependent upon information literacy and are often the most difficult learning outcomes to achieve, particularly for students inexperienced in writing (21). Secondly, although formal criteria and standards were used to assess these reports, they may have failed to pick up on those particular aspects of the work which were improved. These criteria will now be reviewed and refined using the core criteria described by Elander et al. (21). The provision of well written criteria is an important means of communicating the desired learning outcomes to students, who are often confused about what constitutes a good assignment. Finally, there may have been some bias in marking the work as the expectations of the teacher may have been higher in semester 2 compared with semester 1 with most students new to tertiary study.
Defining the attributes and skills of an information literate person is a difficult task. Rangachari and Rangachari (9) utilised an easily understood and therefore helpful interpretation of the ACRL information literacy standards. This consisted of the ability to: frame a clear question to be answered; list sources needed to obtain the relevant information; find sources if possible; evaluate the credibility of the source; and synthesise the information available. The use of standards in the absence of an overarching understanding of information literacy has been cautioned. Elmborg is critical of the attempt to define information literacy through standards and calls on the academic library community to engage in the literacy literature (11). Maybee also addresses the shortfalls in formulaic and teacher-centred approaches to information literacy (12). This author argues that the ‘list of skills’ approach actually limits the potential for student learning and suggests that a relational approach is needed to integrate information literacy into the curriculum and that this should focus on guiding learners to conceptualise information use in more complex ways. Maybee describes a desired process of change in the student conception of information – from one where information is seen as something separate from the user, to one where information ‘is an integral part of the individual’. Student reflections provide valuable information about such changes. These ideas are supported by Elmborg who argues that learning should be redefined as the ‘humanistic process of engaging and solving significant problems in the world’ and information as ‘the raw material students use to solve these problems and to create their own understandings and identities’ (11). Although standards could be seen negatively as a ‘recipe’ for literacy, their articulation provides a useful framework for implementing and evaluating embedded approaches to information literacy skills.
In conclusion, this information literacy intervention has demonstrated significant enhancement of the process students used to access and to some extent evaluate information sources. It has also shown that the quality of the experience was greatly improved for these students. It did not show a real improvement in the quality of the product; however, it is arguable whether the written report was the best product to evaluate to demonstrate these particular foundation level learning outcomes. This is likely due to the failure to address academic writing skills as part of the intervention.
In this study, the teacher and librarian worked collaboratively to produce a programme which integrated the library with the course being taught. Whilst such collaborations are dependent upon the views and attitudes of the academic, there is a requirement to provide students with the skills to be competitive in the global workforce. As such, there will be increasing pressure from universities to ensure that this occurs. Evans (22) asserts that in higher education ‘integrating teaching, learning and technology is a mandate, not an option, and doing any less would border on professional irresponsibility’. Although this intervention was carried out in the context of an Oral Health Program, it has addressed an issue highly relevant for all students in the health sciences and perhaps for higher education students generally.