A full report of this review is being published as part of the HCPRDU reports series. Further details are available from the HCPRDU website: http://www.fhsc.salford.ac.uk/hcprdu
Information skills training: a systematic review of the literature*
Article first published online: 18 JUL 2008
Health Information & Libraries Journal
Volume 20, Issue Supplement s1, pages 3–9, June 2003
How to Cite
Brettle, A. (2003), Information skills training: a systematic review of the literature. Health Information & Libraries Journal, 20: 3–9. doi: 10.1046/j.1365-2532.20.s1.3.x
- Issue published online: 18 JUL 2008
- Article first published online: 18 JUL 2008
- Top of page
The objectives of this study were to undertake a systematic review to determine the effectiveness of information skills training, to identify effective methods of training and to determine whether information skills training affects patient care. A systematic review, using an iterative approach to searching, was employed. Studies selected for inclusion in the review were critically appraised using a tool used in previous reviews. A tabular approach was used to provide a summary of each paper allowing synthesis of results. One thousand, three hundred and fifty-seven potentially relevant papers were located. On the basis of titles and abstracts, 41 potentially relevant studies were identified for potential inclusion. Further reading and application of the inclusion criteria left 24 studies for critical appraisal and inclusion in the review. Study designs included randomised controlled trials, cohort designs and qualitative studies. The majority of studies took place in US medical schools. Wide variations were found in course content and training methods. Eight studies used objective methods to test skills, two compared training methods and two examined the effects on patient care. There was limited evidence to show that training improves skills, insufficient evidence to determine the most effective methods of training and limited evidence to show that training improves patient care. Further research is needed in a number of areas.
- Top of page
The ability to search the literature effectively is essential if practising and future clinicians are to implement evidence-based practice. Librarians and information professionals are well placed to teach these skills and spend large amounts of time teaching searching skills to students and qualified clinicians. Training often focuses on searching electronic resources, in particular medline. Librarians are rarely qualified teachers and there are many different methods of training. However, is there any evidence to show that training in literature searching skills is effective? What aspects of literature searching should be taught and what are the best methods of doing this? Finally, do information skills make a difference to patient care?
The author began to consider these questions after having provided a large number of training sessions, using a variety of methods in a range of settings to both student and practising clinicians. Whilst subjective evaluations of training sessions showed that users valued and benefited from training, informal comments showed that they continued to experience problems undertaking literature searches. With the evidence-based librarianship movement beginning to gain momentum, as signalled by a conference in Sheffield in September 2001, it seemed timely to undertake a systematic review to answer these questions.
The aims of the review were to:
- • examine whether training improves searching skills;
- • identify effective methods of training;
- • examine whether training effects patient care.
- Top of page
An iterative approach was adopted to ensure that all relevant papers examining information skills training and its effects were identified. Three databases were selected for searching to represent different areas of interest: eric (education), lisa (information) and medline (clinical). Following the search spiral approach1 (an iterative approach to searching and refining based on search results), the investigator undertook a scoping search using subject headings and free text searching on these databases covering the period 1995 to September 2002. This time frame was selected as this was the period when evidence-based practice was developing within the UK and end-user searching of electronic databases was becoming more widespread. The scoping search identified a number of relevant studies available on this topic area, highlighted further relevant search terms and helped to clarify the inclusion and exclusion criteria (Appendix 1).
The search strategies were refined and a more comprehensive search was developed and rerun for each database (Appendix 2). This included a wide range of terms (subject headings and free text) to describe training, education and literature searching. A pragmatic approach to the review meant that no restrictions on types of study (qualitative and quantitative) were made during the search. However, only papers in English were included. A total of 1345 potentially relevant papers were located from these searches (545 from eric, 288 from lisa and 512 from medline). Titles and abstracts of these papers were scanned and full papers were obtained for those meeting the inclusion criteria (n = 29). To supplement the database searches, hand searches of two key journals were undertaken covering the period of the review: the Journal of the Medical Library Association (formerly Bulletin of the Medical Library Association) and Health Information and Libraries Journal (formerly Health Libraries Review). These journals were selected as they offered US and UK perspectives, respectively, of the health information and library field. Hand searches identified a further 12 potentially relevant studies. Their citations were checked for further studies, but none were identified.
Sixteen studies were excluded from the review after further reading as they did not meet the inclusion criteria. One paper could not be obtained and was also excluded. A total of 24 studies were selected for review and were critically appraised using a tool developed by the Health Care Practice R&D Unit (HCPRDU).2 This tool covers quantitative, qualitative and multi-method studies. It was developed for, and extensively used in, systematic review work undertaken by the HCPRDU. As the studies were significantly heterogeneous, the investigator used a pragmatic approach to synthesis in extracting similar themes from which useful conclusions could be drawn.
- Top of page
Quantitative and qualitative designs were included in the review. A wide range of study designs was employed. These are summarized in Table 1.
|Study design||Paper||Relevance to review|
|Randomised||Rosenberg et al., 1998||Effectiveness of information skills course6|
|controlled trial (2)||Erickson and Warner, 1998||Impact of a medline course3|
|Experimental (1)||Verhoeven et al., 2000||Evaluation of different methods of literature retrieval10|
|Quasi-experimental (1)||Ghali et al., 2000||Evaluate mini EBM course15|
|Cohort (10)||Dorsch et al., 1990||Integrate EBM course into curriculum21|
|Verhey, 1999||Evaluate information literacy programme11|
|Fox, N. J. et al., 1999||Evaluate information programme delivered via virtual classroom16|
|Grant, 1996||Evaluate impact of literature searching course7|
|Sowinski et al., 2000||Evaluate literature searching module delivered via Internet12|
|Burrows et al., 1999||Evaluate adequacy of medline instruction22|
|Fox, L. M. et al., 1996||Effectiveness of information literacy programme8|
|Gibson et al., 2000||Evaluate computer literacy course17|
|Vogel et al., 2002||Evaluate effectiveness of medline workshop9|
|Poyner et al., 2002||Evaluate intensive information skills training5|
|Observational (1)||Brandt and Lehman, 1995||Evaluate traditional versus web-based literature searching session4|
|Qualitative (4)||Martin, 1998||Evaluate new style user education13|
|Martinale, 1995||Investigate teaching search process versus practical skills23|
|Lambert-Lanning, A., 2000||Evaluation of literature searching workshop14|
|Richwine and McGowan, 2001||Impact of access and training in virtual health library18|
|Questionnaire only (5)||Earl and Neutens, 1999||To determine whether EBM course improves patient care19|
|Schwartz and Schwartz, 1995||Impact of medline training24|
|Hicks, A., 1998||Evaluation of information skills programme25|
|Hook, S. A., 1996||Evaluate Internet searching workshop26|
|Butros, A., 1997||Evaluate e-mail delivery medline course27|
Studies took place in many different settings. The majority took place in North America (16 in the United States and one in Canada). Six of the remaining studies were undertaken in the UK and one in the Netherlands. Twenty of the studies took place in an academic setting. The majority were in medical schools (13), nursing schools (4), pharmaceutical courses (2) and dental schools (1). Four studies involved practising clinicians. Three of these took place in the UK involving general practice staff and one took place with clinicians operating in rural areas of the United States.
Methods of training
Numerous different training methods were utilized. These included didactic sessions, demonstration of techniques, hands-on sessions, one-to-one sessions, small groups, large groups, interactive web packages, sessions delivered via e-mail and various combinations of the above methods. The number of different methods used makes comparisons of effectiveness across studies highly problematic. Two studies involved a direct comparison of different types of training methods.3,4 Erickson and Warner compared a one-hour user-driven tutorial with either a one-hour instructor-led tutorial or no intervention, but found little differences between groups who had received training. Brandt and Lehman compared a ‘traditional’ practical with a facilitated web-based session. Based on student perceptions the authors conclude that both methods were equally effective.
The content of the training programmes varied widely, again compromising any comparison across studies. The majority of sessions included instruction on question formulation, controlled vocabulary (thesaurus searching/MESH) and Boolean operators. Length of sessions also varied greatly with a range of 1–3 h. One study provided intensive one-to-one training including four individual 2–3 h sessions.5
Effectiveness of training
Outcome measures. Most studies used user questionnaires to assess the value and nature of the training given. Eight of these also included questions regarding users’ perceived improvement in skills following training. Objective tests of skills were used as outcome measures in a number of studies. Seven of these undertook post-skills testing. Only five,5,6,7−9 carried out pre- and post-skills testing. Two studies3,10 used recall and precision as outcome measures and four studies linked assessment to student assignments as part of their curriculum.11−14
Training versus no training. A number of studies compared training with no provision of training. One of these,6 showed that training was effective when compared with no training, whereas others3,15 also indicated that training improved medical students’ skills, based on perceived improvements rated by the students. It is difficult to make more direct comparisons between these studies as both their approach to teaching and their focus was quite different. Within a nursing setting, one study16 demonstrated that training was effective within a broader information literacy programme whilst another11 concluded the opposite.
Improvement in skills. Fifteen studies considered improvement in skills. Five of these used an objective measure and assessed skills pre- and post-training.6−9 These provided limited evidence of the effectiveness of training. Three studies indicate that training can improve skills.6−8 One showed a non-significant improvement in skills3 and another demonstrated that training can provide a basic level of competence.17
Improvements in patient care. Two studies considered the effect of providing literature search skills on patient care, using user perceptions as an outcome measure.18,19 Earl and Neutens, in a small exploratory study, reported that participants believed training would improve patient care. Richwine and McGowan, in a study of the effectiveness of providing health information services and training on patient care in a virtual library setting, found that the majority of users reported that use of the library made them handle a clinical situation differently.
- Top of page
The quality of studies included in this review has been problematic. Many studies were flawed, either in their design, execution or reporting. Common flaws included use of insufficient or inappropriate outcome measures and small sample sizes. Inadequate reporting presented difficulties in understanding the nature of the interventions, how results had been obtained and how conclusions had been drawn.
Furthermore, the studies were highly heterogeneous, including differences in settings, training methods and outcome measures. This is to be expected, as such a wide range of health professionals and trainee health professionals require information skills and relationships between library and information professionals and those responsible for course development are likely to differ at every institution. However, such factors will likely impact on the results of training and need to be taken into account when assimilating the results. The content, timing and method of delivery may be more appropriate in some situations rather than others. For example, students may benefit from receiving basic instruction at the beginning of a course and then more advanced instruction at a later stage. This remains to be investigated.
Although user questionnaires are appropriate for measuring the outcome of training—particularly in ascertaining the value of training or the nature and format of the session—more objective methods are likely to be appropriate when determining the effectiveness of training in terms of improvement in skills. Most studies failed to include user questionnaires in their report, with each study seemingly having developed its own questionnaire rather than using a pre-validated tool or validating the questionnaire. Asking users to rate their perceived change in skills following training may be useful, but not as accurate as testing the skills of participants using a more objective method. A small-scale evaluation20 has indicated that users under-report their skills post training in comparison to the results obtained from an objective test taken at the same time.
Although the findings of this review are limited by the poor quality of included studies, the following factors should also be taken into consideration when interpreting its results. The review covered a relatively short time period, 1995–2002. Some studies undertaken before this time may have an impact on the conclusions drawn. Sources used to locate studies for inclusion in the review were selected purposively to reflect subject areas within which studies on health information skills training might be reported. No attempt was made to locate grey literature or unpublished studies. A more comprehensive search of the literature would likely affect the numbers of studies retrieved and the results obtained. Furthermore, use of more free text terms and similar strategies across databases may improve the number of studies retrieved. An alternative or complementary approach would be to widen the search to non-health related studies which might provide more evidence in relation to methods of training.
- Top of page
This paper has described a systematic review undertaken to ascertain whether providing information skills training is effective, whether some training methods are more effective than others and, finally, whether providing information skills training improves patient care. Taking a pragmatic approach to analysing the studies and interpreting the results revealed very limited evidence to show that training does improve search skills. If more studies had used objective, rather than subjective, outcome measures this evidence would have been much stronger.
Comparatively few studies compared different training methods and the heterogeneity of studies meant that it was not possible to compare methods across studies. Therefore no conclusions can be drawn regarding effective training methods. In relation to the effect of information skills training on patient care, two studies (using subjective methods) indicated a positive effect on patient care.
Implications for research
Further research is therefore needed in a number of areas. This includes testing skills pre- and post-training, comparing training methods and conducting research to determine whether there is an actual, rather than perceived, effect on patient care. It would be particularly beneficial to conduct larger studies with longer follow-up using objective outcome measures, where this is appropriate. There is a shortage of validated measures available. Research could be undertaken to develop and validate measures to enable library and information professionals to evaluate the effects of their training more easily.
Better reporting of studies would significantly improve the quality of the systematic review. Although there appear to be sufficient numbers of studies being undertaken to conduct a review, their quality and reporting could be improved. This issue could be addressed by those doing the research, by editors and peer reviewers and by those who provide training in research methods.
Implications for practice
One conclusion that can be drawn from all the studies is that users value training and, on the whole, believe that their skills have improved following training. This is encouraging for those library and information professionals who provide training. It should also encourage others to undertake more research to provide more objective evidence regarding what works, in what situations, and why. However, there appears to be substantial duplication of effort with many information professionals choosing to develop their own courses and methods of evaluation. Better access to research evidence and mechanisms to reduce duplication, or to encourage sharing of resources and ideas, could prove to be invaluable.
Finally, the results of this review are also encouraging for evidence-based library practice. Teaching of library skills is obviously valued and there are some indications that training is effective. It is believed that over the next few years the number and quality of studies undertaken will continue to improve to allow the evidence base to yield a more definitive answer.
- Top of page
The author would like to thank Andrew Long for his encouragement and support in undertaking the study and for the constructive feedback on the draft article.
- Top of page
- 1Developing a review question: a spiral approach to literature searching. Poster presentation. Beyond the basics of systematic reviews, Oxford, July, 2000., &
- 2Health Care Practice R&D Unit. Assessment tools. 2001. http://www.fhsc.salford.ac.uk/hcprdu/assessment.htm (accessed 1 November 2002).
- 4Teaching literature searching in the context of the WWW. Proceedings of the Annual Symposium on Computer Applications in Medical Care 1995, 888–92., &
- 7Teaching a systematic search strategy improves literature retrieval skills of pharmacy students, American Journal of Pharmaceutical Education 1996, 60, 281–6., &
- 9Finding the evidence: teaching medical residents to search medline. Journal of the Medical Library Association 2002, 90, 327–30., &
- 12An internet training module for traditional and non traditional doctor of pharmacy students: content and evaluation. American Journal of Pharmaceutical Education 2000, 64, 15–9., &
- 14Integrating medical informatics into the undergraduate curriculum. Bibliotheca Medica Canadiana 2000, 21(3), 16–9., &
- 20Development of Information Skills within the Masters in Research [MRes (Health and Social Care)], University of Salford, Final Report to TLQIS Programme. Salford: HCPRDU, University of Salford, 2002., &
- 23Teaching information skills on CD_ROM: a conceptual approach. Learning Resources Journal 1995, 11(2), 37–40.
- 26Learning to use the Internet: an interactive team taught workshop for dental faculty and staff. Bulletin of the Medical Library Association 1996, 83, 409–11.
- 27Using electronic mail to teach melvyl medline. Medical Reference Services Quarterly 1997, 16(1), 69–75.
- Top of page
- • Studies undertaken between 1995 and 2001.
- • Studies that took place in a health related setting or studies that took place with student clinicians (any profession), in an academic or clinical setting.
- • Studies that involved any information skills related training (whether part of a course, library induction, etc.) or training that involved at least one of the following: literature searching related skills, health related sources, or studies examining methods of training in information skills or studies examining the effect of information skills training on patient care.
- • Studies that involved at least one evaluation component (user evaluation or other kind of evaluation or testing) of course or skills obtained.
- • Information skills training that was not related to health.
- • Basic IT training (e.g. use of windows and e-mail) (unless part of a course that included literature searching skills).
- • Papers that described training or the development of courses without an evaluation component.
- • Papers evaluating training methods that were not related to health and information skills.
- • Papers evaluating whether any training (other than information skills related) affects patient care.
Appendix 2—search strategies
- 1su = ‘information retrieval’
- 2su = ‘online searching’
- 3su = databases
- 4su = ‘information literacy’
- 5su = ‘information seeking’
- 6su = ‘computer literacy’
- 7su = ‘search strategies’
- 8su = ‘library skills’
- 9su = ‘bibliographic databases’
- 10su = ‘information skills’
- 111 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10
- 12su = ‘library instruction’
- 13su = ‘skill development’
- 14su = ‘library education’
- 15su = ‘instructional effectiveness’
- 16su = ‘computer assisted instruction’
- 17su = ‘training’
- 1812 or 13 or 14 or 15 or 16 or 17
- 1911 and 18
- 20year = 1995–2002
- 21In English
Key OCLC interface
Su = search in subject field
‘xx’ = searches as phrase
- 1su = online information retrieval
- 2su = searching
- 3su = CD ROMS
- 4su = information literacy
- 5su = MEDLINE
- 61 or 2 or 3 or 4 or 5
- 7su = user training
- 8training program*
- 10‘virtual classroom’
- 127 or 8 or 9 or 10 or 11
- 136 and 12
- 14Year of publication = 1995–2002
Su = search subject field
* = truncation symbol
- 1exp information storage and retrieval/
- 2computer literacy/
- 3medical informatics/
- 6Literature search$.tw.
- 8Computer user training/
- 10Computer-assisted instruction/
- 137 and 12
- 14limit 13 to English and abstracts and 1995–2002
Key Ovid Biomed software interface
/ = MESH term
$ = truncation symbol
tw—searches title, abstract and subject heading fields