SEARCH

SEARCH BY CITATION

Keywords:

  • evidence-based practice;
  • instrument development;
  • nursing;
  • research implementation;
  • survey design

Abstract

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References

Title. Factors influencing the development of evidence-based practice: a research tool

Aim.  The paper reports a study to develop and test a tool for assessing a range of factors influencing the development of evidence-based practice among clinical nurses.

Background.  Achieving evidence-based practice is a goal in nursing frequently cited by the profession and in government health policy directives. Assessing factors influencing the achievement of this goal, however, is complex. Consideration needs to be given to a range of factors, including different types of evidence used to inform practice, barriers to achieving evidence-based practice, and the skills required by nurses to implement evidence-based care.

Methods.  Measurement scales currently available to investigate the use of evidence in nursing practice focus on nurses’ sources of knowledge and on barriers to the use of research evidence. A new, wider ranging Developing Evidence-Based Practice questionnaire was developed and tested for its measurement properties in two studies. In study 1, a sample of 598 nurses working at two hospitals in one strategic health authority in northern England was surveyed. In study 2, a slightly expanded version of the questionnaire was employed in a survey of 689 community nurses in 12 primary care organizations in two strategic health authorities, one in northern England and the other in southern England.

Findings.  The measurement characteristics of the new questionnaire were shown to be acceptable. Ten significant, and readily interpretable, factors were seen to underlie nurses’ relation to evidence-based practice.

Conclusion.  Strategies to promote evidence-based practice need to take account of the differing needs of nurses and focus on a range of sources of evidence. The Developing Evidence-Based Practice questionnaire can assist in assessing the specific ‘evidencing’ tendencies of any given group of nurses.


What is already known about this topic

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References
  • Existing instruments for assessing evidence-based practice have focused on examining research utilization rather than taking a broader view of different sources of evidence that can inform practice.
  • Nurses experience major barriers to implementing research findings due to insufficient time to access and review research reports, a shortfall in critical appraisal skills, together with lack of authority and support to implement findings.
  • Whereas evidence-based guidelines and protocols are increasingly available to support evidence-based practice, the extent to which nurses use these various forms of evidence is not clear, nor is how skilled they are in accessing them.

What this paper adds

  • The development and initial validation data for a new measure of a range of factors involved in evidence-based practice for use with hospital and community nurses in England.
  • The questionnaire could be used as an outcome measure in ‘before and after’ intervention studies that aim to assess the impact of service development, training or other innovations on the extent of evidence-based practice.
  • Further research is needed to test the validity of the instrument in other countries, and it would also be interesting to test its relevance to other professions such as allied healthcare professions and social work.

Introduction

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References

Over the past 15 years, evidence-based practice has emerged as a major policy theme in Western healthcare systems. The increased emphasis internationally on clinical and cost-effectiveness in health policy has highlighted the need for quality health services to be built upon the use of best evidence (McKenna et al. 2004). Various governments have introduced initiatives to support the development of evidence-based healthcare systems in which decisions made by healthcare practitioners, managers, policy makers and patients are based on high quality evidence. Activity has focused on developing evidence-based guidelines for clinical interventions. For example, in the United States of America (USA) the Agency for Healthcare Research and Quality (http://www.ahrq.gov) leads national efforts in the use of evidence to guide healthcare decisions. The establishment of the National Institute for Health and Clinical Excellence (http://www.nice.org.uk) in England, the Scottish Intercollegiate Guidelines Network (http://www.sign.ac.uk), and the National Institute for Clinical Studies (http://www.nicsl.com.au) in Australia have similar responsibilities for developing evidence-based guidelines and providing information on the clinical and cost-effectiveness of interventions.

Developing evidence-based guidelines is just one step in a complex process of ensuring that nurses actually base their practice on evidence. Achieving evidence-based practice requires skill on the part of nurses to appraise research evidence in order to decide whether it is appropriate to use. The evidence then needs to be translated into a form that can be implemented in practice and following implementation, the change needs to be evaluated (Gerrish 2006). Whereas the publication of systematic reviews of research and national clinical guidelines makes some aspects of the process easier, implementing change can still be challenging (Collett & Elliot 2000). In recognizing the importance of evidence-based practice to contemporary health care this paper reports on the development and testing of a questionnaire designed to identify factors which influence the development of evidence-based practice in nursing.

Background

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References

Despite widespread calls for nursing to be evidence-based, there is a lack of clarity regarding the concept of evidence-based practice. Sackett et al.’s (1996) definition of evidence-based medicine is one of the most widely cited:

Evidence-based medicine is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external evidence from systematic research. (p. 71)

More recently, Sackett et al. (2000) acknowledged the need also to take account of patient values.

Whereas Sackett's definition of evidence-based medicine has been applied to evidence-based practice in nursing (for example, Ingersoll 2000, DiCenso et al. 2004), there is some concern that the definition is too restrictive. Debates focus on the perceived over-emphasis on research evidence, especially that derived from randomized controlled trials, to the neglect of other sources of evidence, the devaluing of patient experiences and values, and the largely atheoretical medically dominated model of evidence which is contrary to nursing's disciplinary focus on theory-guided practice (DiCenso et al. 2004).

There is general consensus that a broader definition of evidence should be considered which takes account of other ways of knowing that inform nursing practice (Lomas et al. 2005). For example, although Rycroft-Malone et al. (2004) acknowledge the relationship of research, clinical experience and patient experience as the core of evidence-based practice, they argue that the evidence-base for nursing should also include information derived from the local context. Clinical experience as a source of evidence is elaborated by Gerrish (2003) who, in drawing upon the work of Liaschenko and Fisher (1999), differentiates between scientific, empirically based knowledge, patient knowledge developed through an understanding of how patients are located within the healthcare system and knowledge derived from the personal biography of individual patients.

Nolan (2005) draws attention to the international growth of policies promoting user participation which are underpinned by a belief that users should be active shapers of knowledge and subsequent action. He argues that evidence-based practice should encompass this tacit expertise of patients in addition to that of professionals and research – this moves beyond taking account of patient preferences to valuing the knowledge that patients bring to the nurse–patient interaction.

Fawcett et al. (2001) argue for a more theory-guided approach to evidence-based practice in which multiple patterns of knowing in nursing are acknowledged. Drawing upon Carper's typology of ways of knowing (empirical, ethical, personal and aesthetic) they caution against the virtually exclusive emphasis on empirical theories in evidence-based practice and argue for a more holistic approach in which different ways of knowing provide different lenses for critiquing and interpreting different kinds of evidence.

Research examining the implementation of evidence-based practice in nursing has focused primarily on research evidence, in particular on the barriers nurses encounter in using research. These studies have consistently identified that the major obstacles that nurses experience in seeking to implement research findings relate to insufficient time to access and review research reports, a shortfall in critical appraisal skills together with lack of authority and support to implement findings (Funk et al. 1991a, Bryar et al. 2003, McKenna et al. 2004). Researchers have also come under criticism for not presenting their research to clinical audiences in a way that is easy to understand and in which the implications for practice are made clear (Nolan et al. 1998).

Much of the responsibility for evidence-based practice has been placed on individual practising nurses. However, although it is recognized that all nurses have a professional responsibility to base their care on the best available evidence, implementing evidence-based practice in healthcare settings is a complex undertaking (Royle & Blythe 1998). It has been argued that healthcare organizations should support the development of a culture of evidence-based practice and provide resources for its implementation (DiCenso & Cullum 1998, Gerrish & Clayton 2004). Consideration of this broader context has highlighted the importance of the leadership styles of senior clinical nurses in promoting a ward/team culture that is patient-centred, values members and promotes a learning environment to support evidence-based practice (McCormack et al. 2002). Some models for promoting evidence-based practice also emphasize the need for facilitation by external and internal change agents to support the process of change and identify the importance of the personal characteristics of the facilitator, the style of facilitation and the role of the facilitator in terms of authority (Harvey et al. 2002).

Existing questionnaires used to examine evidence-based practice have focused on research utilization, in particular nurses’ ability to access and appraise research reports and implement research findings in practice. The Barriers to Research Utilization Questionnaire developed in the USA by Funk et al. (1991a) has been used extensively over the past 15 years in a number of countries including Australia (Retsas & Nolan 1999, Hutchinson & Johnson 2004), Finland (Oranta et al. 2002, Kuuppelomäki & Tuomi 2005), Ireland (Glacken & Chaney 2004), Sweden (Kajermo et al. 1998), and the United Kingdom (UK) (Dunn et al. 1998, Nolan et al. 1998, Closs & Bryar 2001). It has also been used to examine research utilization in specific groups of nurses, for example, community nurses (Bryar et al. 2003), specialist breast care nurses (Kirshbaum et al. 2004) and forensic mental health nurses (Carrion et al. 2004). The questionnaire identifies 29 items considered to be barriers to research utilization. Respondents are asked to rate on a 5 point Likert scale the extent to which they perceive each item to be a barrier. Factor analysis grouped the items around four factors, the nurse's research values, skills and awareness, the quality of the research, the way in which research is communicated and the characteristics of the organization (Funk et al. 1991b). International comparisons of published findings indicate that nurses experience broadly similar barriers to using research in terms of the ranking of individual items.

Some studies have sought to replicate the factor analysis. Whereas the original four factors identified by Funk et al. were confirmed by Hutchinson and Johnson (2004), other studies have identified different groupings of items: Retsas and Nolan (1999)– three factors, Marsh et al. (2001)– three factors, Kirshbaum et al. (2004)– three factors, Kuuppelomäki and Tuomi (2005)– six factors. Closs and Bryar (2001) and Marsh et al. (2001) undertook extensive testing of the instrument and independently raised questions about the content and construct validity of the scale for use in the UK.

Several other questionnaires have been developed to examine research utilization, however, they have not been used as extensively as the Barriers questionnaire in order to test the validity and reliability of the instruments in other settings (for example Lacey 1994, Rodgers 1994, Hicks 1996, Estabrooks 1998, McKenna et al. 2004). Moreover, within the context of evidence-based practice they focus on the use of research findings rather than a broader definition of evidence identified as important in the literature and referred to above. Although Estabrooks (1998) considered a broad range of sources of information that nurses draw upon, including professional and patient expertise, this was in order to examine the extent to which sources of research evidence were used rather than to acknowledge the contribution of diverse sources of evidence. Nevertheless, parts of Estabrooks instrument have the potential to be used to examine a broader definition of evidence-based practice.

From a review of the literature and existing instruments, there appeared to be a need for a questionnaire which would examine factors influencing evidence-based practice where in addition to research evidence, other forms of evidence were considered. The definition of evidence-based practice which informed the development of the questionnaire in this study was adapted from Sackett's definition referred to above which emphasizes the interplay of research evidence, clinical expertise and patient preferences. However, the definition of evidence was extended to include research products such as national guidelines and local information such as protocols and audit reports.

This paper reports the design and testing of the psychometric properties of a wide-ranging questionnaire designed to measure several aspects of evidence-based practice. Two surveys were undertaken in order to test the instrument, one involving hospital nurses and the second nurses working in the community.

The studies

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References

Aims

The aim of the two studies was

  • to develop and validate the Developing Evidence-Based Practice (DEBP) questionnaire as a comprehensive measure of evidence-based practice in England;
  • to determine the important factors influencing the development of evidence-based practice, using a composite measuring tool, the DEBP questionnaire.

The results of the first aim are presented in this paper. Results based on the application of the questionnaire will be published elsewhere.

Study 1: Hospital nurses

Study 1 provided an opportunity to survey two contrasting hospital sites. The nurse respondents were located in two acute hospitals in northern England [a university teaching hospital and a district general hospital (DGH)], within the same strategic health authority. It built upon earlier research (Nolan et al. 1998) undertaken in the teaching hospital which had developed an anglicized version of the ‘Barriers to Research Utilization’ scale. However, the current study took a much broader view of evidence-based practice and included the use of different sources of evidence and a self-appraisal of skills in finding and using evidence. Data were collected during 2002–2003.

Participants

The sample was drawn from the records of qualified nursing staff at each hospital. All nurses were included in the sample except those from two directorates at the teaching hospital that were participating in another research study related to evidence-based practice. This resulted in a sample of 728 at the teaching hospital, and 683 at the DGH. Of these, 330 were returned at the teaching hospital, and 274 at the DGH, a response rate of 45% and 40%, respectively. The useable achieved sample was 598, after the exclusion of questionnaires without information about the respondents’ grades.

Study 2: Community health nurses

In the second study, a slightly expanded version of the questionnaire was used, with an additional eight items which in each case simply increased the lists of sources of evidence; barriers and facilitators to employing evidence in practice, and personal skills (see Table 2 for all items including the additions, which are italicized). These minor modifications arose from testing the content validity of the instrument originally used for hospital nurses for use with community health nurses. The respondents were community health nurses in 12 primary care trusts (PCTs) in two strategic health authorities, one in northern England and the other in southern England. Data were collected during 2005.

Participants

A random sample of 1600 community health nurses was drawn from the records of qualified staff in the 12 PCTs. Equal numbers of health visitors, district nurses, community nurses, practice nurses and school nurses were sampled. The overall response rate was 47% with responses for each of the five community health nursing groups as follows: health visitors 57%, district nurses 55%, community nurses 40%, practice nurses 37%, school nurses 43%. The usable sample was 689, after the exclusion of questionnaires without information about the respondents’ post.

Construction of the questionnaire

The DEBP questionnaire has five main parts, each one derived from somewhat different sources. Twenty-two items, 16 of which are anglicized versions of items of the Estabrooks scale (Estabrooks 1998) about sources of knowledge, constitute the first section of the questionnaire. Each item was scored on a 5-point scale from never (score 1) to always (score 5). Permission to use these items was obtained from the author.

The second, third and fourth sections of the DEBP questionnaire examine barriers to achieving evidence-based practice. Feedback from the earlier study in the teaching hospital that had utilized an anglicized version of the North American ‘Barriers’ questionnaire identified a number of problems with this instrument that necessitated the development of a quite different set of items, albeit ones that still examined barriers. The new items took account of a broader understanding of evidence by including questions on organizational information (defined as care pathways, clinical protocols and guidelines) in addition to questions on research evidence. Emphasis was also placed on changing practice based on evidence rather than just the implementation of research findings. Additionally, in contrast to the original Barriers scale which asked respondents to comment on ‘the nurse’ in a generic sense, the new items used a personal ‘I’ or ‘my’ to ensure respondents were reporting their own experience rather than that of nurses in general. This new ‘Barriers’ scale consisted of 19 items with 5-point response scales. These are divided into two groups of barriers and one group concerned with colleague relations which facilitate evidence-based practice (scored in the opposite direction to the other two groups, but intended to reduce the apparent negativity of the ‘barriers’ items of the questionnaire). The scoring used the 5-point scale technique of section 1, with a score of 1 for ‘agree strongly’.

Finally, a fifth section was devised consisting of eight items asking nurses to rate themselves on skills of finding and reviewing evidence, and using evidence to effect change. Ratings on a 5-point scale ranged from ‘complete beginner’ (score 1) to ‘expert’ (score 5).

So the DEBP tool consisted of

  • Section 1. Bases of practice knowledge (22 items).
  • Section 2. Barriers to finding and reviewing evidence (10 items).
  • Section 3. Barriers to changing practice on the basis of evidence (five items).
  • Section 4. Facilitation and support in changing practice (four items).
  • Section 5. Skills in finding and reviewing evidence (eight items).

The core of the DEBP questionnaire hence consists of 49 items, designed as a paper-based tool for self-completion. It was initially piloted with 20 nurses who worked in hospital settings and minor modifications were subsequently made to two items to improve clarity. Prior to study 2 the content validity of the questionnaire was considered by a panel of four experts in community health nursing and minor modifications made to the questionnaire. This included four additional sources of knowledge, two additional barriers and two further skills considered to be relevant to the practice of community health nursing. The revised questionnaire was piloted with five community health nurses but no changes were required.

Data collection

Questionnaires were addressed individually and distributed via the external post or internal mail system at each site, depending on the preference of the organization. An addressed envelope was enclosed for return of the questionnaire. In study 1, reminders were posted around the hospital site to maximize response, but no individual reminders were sent to maintain anonymity of responders. Ward managers were asked to encourage completion of the questionnaires, but it was stressed that this was entirely voluntary. In study 2 targeted reminders were sent to non-respondents to maximize the response rate.

Ethical considerations

The study was approved by the relevant research ethics and governance committees at each site. A participant information sheet giving details of the study accompanied the questionnaire. Consent to participate was assumed on the basis of a returned, completed questionnaire.

Data analysis

The data for all items employed in both studies 1 and 2 were analysed using SPSS version 13. Initial analysis suggested no alteration in the structure of results over the timespan in which data were collected; this justified bringing together the findings of the two studies in validating the instrument.

Results

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References

Measurement characteristics of the ‘Developing Evidence-Based Practice’ Questionnaire

Responses were treated as five-point scale items, with ‘high’ and ‘low’ being assigned, as indicated above. The questionnaire has five major sections. The measurement features of the sections are given in Table 1. To be noted in particular is the column of values of reliability. Reliability in this context means internal consistency. It refers to the extent to which the scores on the items correlate with each other and, if they do, this means that we can regard the items within a scale as all being about the same thing. It then becomes justifiable to treat the items as constituting a Likert scale, since it is meaningful to sum up the scores of each item to give a scale score for the individual. The questionnaire was tested as to coherence of scales and subscales using intercorrelation of items and Cronbach's Alpha as indicators of reliability. The most widely used index of internal consistency, Cronbach's Alpha is equivalent to the average of all possible split-half correlation coefficients. The values of α for each scale and subscale can be seen in Table 1a. All α values are acceptable as estimates of reliability (the conventional value of α regarded as indicating a level which avoids false positive reliability estimates is 0·7). The five sections of the questionnaire can be assumed to be reliable. However, the pattern of intercorrelations in Table 1b is such that it would be inappropriate to employ all 49 items as a ‘scale’ (see also factor analytic evidence below).

Table 1.   Measurement characteristics of the Developing Evidence-Based Practice Questionnaire
(a) Descriptive statistics and reliability of sections of the questionnaire and overall questionnaire
SectionnNumber of items*MeansdReliability (Cronbach α)
112821859·9006·7180·788
21286928·6646·0840·843
31286516·5963·9900·805
41286310·2912·0140·730
51287617·1984·4480·913
Overall*127941132·67215·0010·874
(b) Intercorrelations between sections of the questionnaire (Pearson)
Section2345
  1. *Using only items employed in both studies.

  2. All correlations are significant at the 0·01 level, 2-tailed. n for each section is between 1281 and 1286.

10·1450·0870·1840·229
2 0·5820·1600·373
3  0·2480·211
4   0·197

It is to be noted that the five-point scoring technique, which is employed for all items of the DEBP questionnaire, is a deviation from the agree/disagree scoring originally suggested by Funk et al. (1991a). As a check on the effect of this modification, in study 1, the relevant items were recoded with 1 for ‘agree’ and ‘agree strongly’ and 0 for other responses. The results were equivalent though the five-point scoring technique is more sensitive (as one might expect).

Comparison of scores on section 1 of the DEBP questionnaire with Estabrooks (1998)

As part of the effort to validate the DEBP questionnaire, the results in study 1, for items of section 1 matching Estabrooks’ items were compared with those she reports in her original paper (1998). The correlation between the rank-orders of the means for the items in the two studies yields a value of Spearman's ρ = 0·897, which is significant at the P < 0·01 level (one tail). Tests of the difference between means in the two studies showed none to be significant at the 0·01 level using t (two-tailed for degrees of freedom of around 590). These findings indicate that the anglicized version of the Estabrooks questionnaire functioned in a manner akin to the original instrument, and that the responses of the nurses in this study were similar to those of the original Canadian sample. The comparability of the first section of the questionnaire with Estabrooks’ results gives evidence of construct validity.

Factor analysis

The mean and standard deviation for each item were calculated, and the Pearson correlation of each item with each other item was calculated. On this basis, an exploratory factor analysis (Lawley & Maxwell 1971, Pett 2003) was carried out. A factor analysis economises on the number of variables used to account for a matrix. So, in the present case, the very large number of intercorrelations between the 49 items of the questionnaire can be summarized by calculating the ‘position’ of a fewer number of imaginary parameters to which each of the actual items can be, as it were, correlated. These imaginary parameters are ‘factors’. The correlation of an item with a factor is the ‘loading’ of the item on the factor. The principle components algorithm was used to specify factors.

There are a large number of equivalent mathematical solutions to the question of where the imaginary parameters can go. The decision about the preferred (i.e. the specification of the rotation) was made using a conventional set of criteria. The solution was calculated in which factors (a) are not correlated with each other (orthogonal), and (b) have loadings which are as high as possible or near-zero.

The varimax (Kaiser 1958) rotated factor matrix is given in Table 2. The factor analysis was carried out using only the questionnaire items common to both studies. Factor analysis was based on the 10 principal components with initial eigenvalues greater than one. The relative strength of each factor within the matrix as a whole is indicated in the final row of Table 2.

Table 2.   Rotated factor matrix
 Factor
Questionnaire item12345678910
  1. Additional items included in the questionnaire for study 2 are in italics.

Section 1. Bases of practice knowledge
 1. Information I learn about each patient/client as an individual         0·687
 2. My intuitions about what seems to be ‘right’ for the patient/client      0·560  0·372
 3. My personal experience of caring for patients/clients over time      0·525  0·464
 4. What has worked for me for years      0·851   
 5. The ways I have always done it      0·822   
 6. Information my fellow practitioners share       0·556  
 7. Information senior clinical nurses share, e.g. clinical nurse specialists, nurse practitioners       0·458  
 8. What doctors discuss with me       0·797  
 9. New treatments and medications that I learn about when doctors prescribe them for patients       0·761  
10. Medication and treatments I gain from pharmaceutical or equipment company representatives       0·401  
11. Information I get from product literature  0·329    0·343  
12. Information I learned in my training     0·622    
13. Information I get from attending in-service training/conferences     0·719    
14. Information I get from local policy and protocols     0·767    
15. Information I get from national policy initiatives/guidelines  0·435  0·531    
16. Information I get from local audit reports     0·601    
17. Articles published in medical journals  0·763       
18. Articles published in nursing journals  0·758       
19. Articles published in research journals  0·733       
20. Information in textbooks  0·664       
21. Information I get from the internet  0·606       
22. Information I get from the media  0·589       
Section 2. Barriers to finding and reviewing evidence
23. I do not know how to find appropriate research reports 0·592 0·551      
24. I do not know how to find organisational information (guidelines, protocols, etc.) 0·722 0·373      
25. I do not have sufficient time to find research reports    0·806     
26. I do not have sufficient time to find organisational information (guidelines/protocols, etc.)    0·754     
27. Research reports are not easy to find   0·4640·511     
28. Organizational information (protocols, guidelines, etc.) is not easy to find 0·448 0·3160·363     
29. I find it difficult to understand research reports   0·810      
30. I do not feel confident in judging the quality of research reports   0·806      
31. I find it difficult to identify the implications of research findings for my own practice 0·426 0·683      
32. I find it difficult to identify the implications of organizational information for my own practice0·450  0·443      
Section 3. Barriers to changing practice on the basis of evidence
33. I do not feel confident about beginning to change my practice 0·818        
34. The culture of my team is not receptive to changing practice 0·847        
35. I lack the authority in the workplace to change practice 0·763        
36. There are insufficient resources (e.g. equipment) to change practice 0·433  0·613     
37. There is insufficient time at work to implement changes in practice 0·327  0·693     
Section 4. Facilitation and support in changing practice
38. Nursing colleagues are supportive of my changing practice        0·819 
39. Nurse managers are supportive of my changing in practice        0·837 
40. Doctors with whom I work are supportive of my changing practice        0·697 
41. Practice managers are supportive of my changing practice        0·564 
Section 5. Self-assessment of skills
42. Finding research evidence0·759         
43. Finding organizational information0·815         
44. Using the library to locate information0·622  0·393      
45. Using the internet to search for information0·594  0·337      
46. Reviewing research evidence0·818         
47. Reviewing organizational information0·873         
48. Using research evidence to change practice0·792         
49. Using organizational information to change practice0·787         
Percentage variance of matrix due to each factor10·62210·62210·62210·62210·62210·62210·62210·62210·62210·622

The version of the questionnaire employed in study 2 had eight items in addition to those in study 1. The factor analysis was carried out only on the items common to the two studies, and Table 2 reports the factor loadings of these items in regular font. The eight additional items also appear in Table 2 but are printed in italics. The data provided in the table for each of the additional items are not factor loadings but are correlations between the item scores and factor scores (Mulaik 1987). Correlations greater than +0·3 or less than −0·3 are reported in the table. The new items fall into the expected groupings.

The interpretation of the factors was undertaken by inspection of the items which had the highest loadings on each factor. The factors relate neatly to the five sections. There is overlap between sections 2 and 3 – which both have to do with barriers to evidence-based practice. The particular kind of evidence matters. And section 1 also generates factors specific to the kind of evidence which is under consideration.

The interpretability of the factors in the light of the intended meaning of the sections of the DEBP questionnaire constitutes construct validation. The interpretations are as follows:

  • Factor 1. Skill in finding, reviewing and using different sources of evidence The factor is construct validation for section 5.

  • Factor 2. Barriers to, or facilitators of, personal efficacy in the context of the organization, including team culture and personal authority. This factor includes part of each of the ‘barriers’ sections.

  • Factor 3. Published information as a source of knowledge used in practice. A subset of section 1.

  • Factor 4. Focal concern or interest in the effective use of research. Part of section 2.

  • Factor 5. The availability of formal information (research and organizational information), and disposable time to implement the recommendations. This factor includes part of each of the ‘barriers’ sections.

  • Factor 6. Knowledge gleaned from training, conferences, and local and national reports and audits. A subset of section 1.

  • Factor 7. Personal experience. A subset of section 1.

  • Factor 8. Informal information gleaned in the course of daily work, including interprofessional conversations. A subset of section 1.

  • Factor 9. The facilitating or hindering effect of colleagues in changing practice. This factor gives construct validity to section 4.

  • Factor 10. Client /patient contact and the nurse's personal knowledge and experience. A subset of section 1.

In Table 3, we present the results of a further analysis in which each of the factors is treated as a scale. The reliability of each factor, regarded as a Likert scale, is given as Cronbach's α. Values where α > 0·7 are generally regarded as indicating the reliability of a scale. However, α is sensitive to the number of items in a scale. Reliability values for factors 8 and 10 are likely to be low due to the small number of items contributing to the factor. Three additional items are associated with factor 8 in study 2.

Table 3.   Characteristics of the factors, treated as scales
FactorValues for factor as a scale
nNumber of items defining the factorMeansdCronbach's α
  1. *Reliability values as low as this may be due to the small number of items contributing to the factor. Three additional items are associated with factor 8 in study 2.

 11287617·204·1480·913
 21285930·346·6650·871
 31286720·954·0650·820
 41286722·865·2160·859
 51285618·264·2050·798
 61286414·612·4850·731
 71284412·672·3480·716
 81287310·571·7670·689*
 91286310·292·0140·730
101286311·711·6130·539*

Since it does seem that several of the factors cut across or subdivide the sections of the DEBP questionnaire, it was considered whether the 10 factors would function as sections, replacing the existing five sections. In the event, this would be premature: we are insufficiently confident of the reliability of factors 8 and 10 pending further data.

Discussion

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References

Study limitations

One of the main disadvantages of using a self-completed postal questionnaire is the potential for a low response rate (Robson 2002). Previous surveys examining barriers to research utilization have experienced relatively poor response rates. In study 1, the response rate of 45% for the teaching hospital and 40% for the DGH are not dissimilar to the response rate of 44% reported by Bryar et al. (2003) in a large study involving two hospitals and four community settings and the 40% response rate achieved by Funk et al. (1991b) in their original study of barriers to research utilization. In study 2 response rates were slightly higher at 47% which may reflect the effect of targeted reminders to non-respondents.

Although the response rates compared favourably with many similar studies, they may nevertheless conceal some response bias. It was noted, for example, that in the community study the response rate between different professional groups varied considerably (57% for health visitors compared with 37% for practice nurses). It is possible that nurses who were less favourable towards using evidence in their practice might have been less likely to respond, thus biasing the achieved sample. However, such response bias would be more of a concern in interpreting the overall findings of the study (to be reported elsewhere) rather than in assessing the reliability and validity of the tool.

A further limitation might be that the DEBP questionnaire was changed slightly between the two studies by the addition of some items. Although these changes were made to enhance face validity, after consulting users from community settings, it might be suggested that such changes would have altered the psychometric properties of the questionnaire. However, this was not found to be the case. The new items related well to the established factor structure.

Discussion of results

A large enough sample size (n = 1287 in total) was achieved for adequate testing of the tool. The psychometric properties of the DEBP suggest that it is a reliable instrument with 10 identifiable factors, although it is not a single scale. The conventions used to test the psychometric properties of the questionnaire were drawn from well-established sources and demonstrate high reliability (>0·7) for each of the five sections and for eight of the 10 factors when treated as scales. The lower values for the two remaining factors (8 and 10) are likely to be due to the small number of items defining the factors.

The 10 factors are in some cases consistent with the different elements and sub-sections of the tool, but in some cases provide over-arching concepts that are drawn from different elements of the tool. Factor 2, for example, highlights personal and organizational difficulties in using evidence-based practice which range from a lack of personal knowledge to a lack of empowerment to challenge established practice. Factors 7 and 8 emphasize the role of personal experience and informal sources of information in nurses’ application of evidence-based practice. This aspect of knowledge utilization has been disregarded in many previous tools measuring evidence-based practice.

The need to promote the use of appropriate evidence in nursing practice has been widely acknowledged, along with an associated need to test and evaluate the extent of evidence-based practice. As pointed out earlier in the paper, a variety of instruments have been used to do this, particularly the ‘Barriers’ scale (Funk et al. 1991a) but these tools have been either untested or found to be lacking in validity (Closs & Bryar 2001, Marsh et al. 2001). Many of the previously developed tools are also historically located in a time when information technology and access to electronic information in clinical settings was limited. Much has changed in the last decade, with computer access close to patient care increasingly available, and protocol-based care now integrated into many clinical areas. We therefore set out to develop and test a more comprehensive tool (DEBP).

The study has provided evidence of validation of the DEBP questionnaire for investigating factors associated with evidence-based practice among nurses in England. Notwithstanding the need for additional studies in the UK and beyond to further validate of the instrument, the inclusion of sources of knowledge and skills ratings alongside the ‘barriers’ scale adds considerably to its usefulness, and factor analysis suggests that the scales are consistent. Whereas the DEBP questionnaire has been shown in this study to be a valid instrument, with reliable scales, the questionnaire as a whole does not constitute a scale – the component sections are too diverse in meaning for this. One significant modification to earlier ‘barriers to research utilization’ questionnaires is the inclusion in the current questionnaire of organizational information as a source of evidence. This reflects the increased emphasis placed on nurses in the UK to draw upon national and local evidence-based guidelines and clinical protocols rather than assuming that nurses would, or indeed should, interpret the significance of findings in published research papers for their practice.

Conclusion

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References

The development and testing of the DEBP questionnaire reported in this article suggest that the instrument is a valid and reliable measure, although further testing is required to fully establish its validity and reliability. The generalizability of the DEBP questionnaire has been shown to extend to nurses in hospital and community settings in England. However, its validity in other countries remains to be demonstrated. Before adoption elsewhere it will be important to review the cultural appropriateness and content validity of items in the different sections of the questionnaire as different barriers to evidence-based practice may be important in some countries. For example, Oranta et al. (2002) identified that one of the greatest barriers to evidence-based practice for nurses in Finland was the fact that most research papers were published in a language other than Finnish (section 2 of questionnaire). Indeed, the preponderance of English language journals may present particular challenges for those whose first language is not English. Moreover, the high turnover of staff in some parts of South Africa is seen to mitigate against sustaining change in respect of evidence-based practice (section 3 of questionnaire) (Garner et al. (2004), McInerney 2004).

The questionnaire could be used as an outcome measure in ‘before and after’ intervention studies that aim to assess the impact of service development, training or other innovations on the extent of evidence-based practice. Organizations wishing to build research capacity will also find the tool useful in measuring progress. Because the tool has five sections and 10 identifiable factors, it may be possible to analyse the nature of the change being measured over time. Policies can then be tailored to address the particular barriers and organizational factors highlighted as being problematic. It would also be interesting to test its relevance to other professions such as allied health professions and social work. Comparisons between the professions regarding the implementation of evidence-based practice would then be possible.

Author contributions

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References

KG and AL were responsible for the study conception and design of the manuscript and KG, PA and AL were responsible for the drafting of the manuscript. KG, AL, JB, JC, SK and EM performed the data collection and KG, PA and AL performed the data analysis. KG, AL, JB and SK obtained funding and JB, EM and JC provided administrative support. PA provided statistical expertise. KG, PA, AL, JC, SK and EM made critical revisions to the paper.

Acknowledgements

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References

We are grateful to Professor Carole Estabrooks, University of Alberta, Canada, for granting permission to adapt and use some questions from an instrument she had developed to examine research utilisation.

References

  1. Top of page
  2. Abstract
  3. What is already known about this topic
  4. Introduction
  5. Background
  6. The studies
  7. Results
  8. Discussion
  9. Conclusion
  10. Author contributions
  11. Acknowledgements
  12. References
  • Bryar R., Closs S., Baum G., Cooke J., Griffiths J., Hostick T., Kelly S., Knight S., Marshall K. & Thompson D. (2003) The Yorkshire BARRIERS project: diagnostic analysis of barriers to research utilisation. International Journal of Nursing Studies 40, 7385.
  • Carrion M., Woods P. & Norman I. (2004) Barriers to research utilisation among forensic mental health nurses. International Journal of Nursing Studies 41, 613619.
  • Closs J. & Bryar R. (2001) The BARRIERS scale: does it ‘fit’ the current NHS research culture. NT Research 6, 853865.
  • Collett S. & Elliot P. (2000) Implementing clinical guidelines in primary care. In Implementing Evidence-Based Changes in Healthcare (EvansD. & HainesA., eds), Radcliffe Medical Press, Abingdon, pp. 235256.
  • DiCenso A. & Cullum N. (1998) Implementing evidence-based nursing: some misconceptions. Evidence-Based Nursing 1, 3840.
  • DiCenso A., Prevost S., Benefield L., Bingle J., Ciliska D., Driever M., Lock S. & Titler M. (2004) Evidence-based nursing: rationale and resources. Worldviews on Evidence-Based Nursing First Quarter 2004, 6975.
  • Dunn V., Crichton N., Roe B., Seers K., & Williams K. (1998) Using research for practice: a UK experience of the BARRIERS Scale. Journal of Advanced Nursing 27, 12031210.
  • Estabrooks C. (1998) Will evidence-based nursing practice make practice perfect? Canadian Journal of Nursing Research 30, 1536.
  • Fawcett J., Watson J., Neuman B., Hinton Walker P. & Fitzpatrick J. (2001) On nursing theories and evidence. Journal of Nursing Scholarship Second Quarter 2001, 115119.
  • Funk S.G., Champagne M.T., Wiese R.A. & Tornquist E.M. (1991a) The Barriers to Research Utilization Scale. Applied Nursing Research 4, 3945.
  • Funk S.G., Champagne M.T., Wiese R.A. & Tornquist E.M. (1991b) Barriers to using research findings in practice: the clinician's perspective. Applied Nursing Research 4, 9095.
  • Garner P., Meremikwu M., Volmink J., Xu, Q. & Smith H. (2004) Putting evidence into practice: how middle and low income countries ‘get it together’. British Medical Journal 329, 10361039.
  • Gerrish K. (2003) Evidence-based practice: unravelling the rhetoric and making it real. Practice Development in Health Care 2, 99113.
  • Gerrish K. (2006) Evidence-based practice. In The Research Process in Nursing (GerrishK. & LaceyA., eds), 5th edn Blackwell Science, London, pp. 491505.
  • Gerrish K. & Clayton J. (2004) Promoting evidence-based practice: an organisational approach. Journal of Nursing Management 12, 114123.
  • Glacken M. & Chaney D. (2004) Perceived barriers and facilitators to implementing research findings in the Irish practice setting. Journal of Clinical Nursing 13, 731740.
  • Harvey G., Loftus-Hill A., Rycroft-Malone J., Titchen A., Kitson A., McCormack B. & Seers K. (2002) Getting evidence into practice: the role and function of facilitation. Journal of Advanced Nursing 37, 577588.
  • Hicks C. (1996) A study of nurses’ attitudes towards research: a factor analytic approach. Journal of Advanced Nursing 23, 373379.
  • Hutchinson A. & Johnson L. (2004) Bridging the divide: a survey of nurses’ opinions regarding barriers to, and facilitators of, research utilisation in the practice setting. Journal of Clinical Nursing 13, 304315.
  • Ingersoll G. (2000) Evidence-based nursing: what it is and what it isn't. Nursing Outlook 48, 151152.
  • Kaiser H.F. (1958) The varimax criterion for analytic rotation in factor analysis. Psychometrika 23, 187200.
  • Kajermo K., Nordstrom G., Krusebrant A. & Bjorvell H. (1998) Barriers to and facilitators of research utilisation, as perceived by a group of registered Nurses in Sweden. Journal of Advanced Nursing 27, 798807.
  • Kirshbaum M., Beaver K. & Luker K. (2004) Perspectives of breast care nurses on research dissemination and utilisation. Clinical Effectiveness in Nursing 8, 4758.
  • Kuuppelomäki M. & Tuomi J. (2005) Finnish nurses’ attitudes towards nursing research and related factors. International Journal of Nursing Studies 42, 187196.
  • Lacey E.A. (1994) Research utilisation in nursing practice: a pilot study. Journal of Advanced Nursing 19, 987995.
  • Lawley D.N. & Maxwell A.E. (1971) Factor Analysis as a Statistical Method, 2nd edn. Butterworth, Oxford.
  • Liaschenko J. & Fisher A. (1999) Theorizing the knowledge that nurses use in the conduct of their work. Scholarly Inquiry for Nursing Practice. An International Journal 13, 2940.
  • Lomas J., Culyer T., McCutcheon C., McAuley L., Law S. (2005) Conceptualizing and Combining Evidence for Health System Guidance. Canadian Health Services Research Foundation (CHSRF). Retrieved from: http://www.chsrf.ca.
  • Marsh G., Nolan M. & Hopkins S. (2001) Testing the revised barriers to research utilisation scale for use in the UK. Clinical Effectiveness in Nursing 5, 6672.
  • McCormack B., Kitson A., Harvey G., Rycroft-Malone J., Titchen A. & Seers K. (2002) Getting evidence into practice: the meaning of ‘context’. Journal of Advanced Nursing 38, 94104.
  • McInerney P. (2004) Evidence-based nursing and midwifery: the state of the science in South Africa. Worldviews on Evidence-Based Nursing 1, 4207.
  • McKenna H., Ashton S. & Keeney S. (2004): Barriers to evidence-based practice in primary care. Journal of Advanced Nursing 45, 178189.
  • Mulaik, S.A. (1987) A brief history of the philosophical foundations of exploratory factor analysis. Multivariate Behavioural Research 22, 267305
  • Nolan M. (2005) Reconciling tensions between research, evidence-based practice and user participation: time for nursing to take the lead. International Journal of Nursing Studies 42, 503505.
  • Nolan M., Morgan L., Curran M., Clayton J., Gerrish K. & Parker K. (1998) Evidence-based care: can we overcome the barriers? British Journal of Nursing 7, 12731278.
  • Oranta O., Routasolo P. & Hupli M. (2002) Barriers to and facilitators of research utilisation among Finnish Registered Nurses. Journal of Clinical Nursing 11, 205213.
  • Pett M.A. (2003) Making Sense of Factor Analysis in Health Care Research: A Practical Guide. Sage, London.
  • Retsas A. & Nolan M. (1999) Barriers to nurses’ use of research: an Australian hospital study. International Journal of Nursing Studies 36, 335343.
  • Robson C. (2002) Real World Research: A Resource for Social Scientists and Practitioners. Blackwell, Oxford.
  • Rodgers S. (1994) An exploratory study of research utilisation by nurses in general medical and surgical wards. Journal of Advanced Nursing 20, 904911.
  • Royle J. & Blythe J. (1998) Promoting research utilisation in nursing: the role of the individual, organisation, and environment. Evidence-Based Nursing 1, 7172.
  • Rycroft-Malone J., Seers K., Titchen A., Harvey G., Kitson A. & McCormack B. (2004) What counts as evidence in evidence into practice? Journal of Advanced Nursing 47, 8190.
  • Sackett D., Rosenburg W., Muir Gray J., Haynes R. & Richardson W. (1996) Evidence-based medicine: what it is and what it isn't. British Medical Journal, 312, 7172.
  • Sackett D., Richardson S., Richardson S., Rosenberg W. & Haynes B. (2000) Evidence-Based Medicine: How to Practice and Teach EBM. Churchill Livingstone, Edinburgh.