SEARCH

SEARCH BY CITATION

Keywords:

  • Electronic health record;
  • nursing diagnosis;
  • nursing record;
  • terminology as topic;
  • validation study

Abstract

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Conclusion
  7. Acknowledgments
  8. References

Purpose

To assess the reliability and validity of Quality of Diagnoses, Interventions, and Outcomes (Q-DIO) in Brazil and in the United States.

Methods

The sample comprised 180 records: centers 1 (electronic records and standardized language) and 2 (paper-based records without standardized language in Brazil, and center 3 (electronic records without standardized language) in the United States. Reliability and discriminant construct validity was analyzed.

Findings

Cronbach's alpha for all 29 Q-DIO items were greater than or equal to 0.70 for all centers; construct validity was significantly different between the three study centers.

Conclusions

These results indicate that Q-DIO is valid and reliable for assessing the quality of nursing records.

Implications for Nursing Practice

The Q-DIO may be useful to assess the quality and accuracy of nursing records.

In clinical practice, nurses are required to systematize patient care as to ensure patient safety and quality care (Paans, Sermeus, Nieweg, & Van der Schans, 2010; Saranto & Kinnunen, 2009). A key aspect of this process is that nursing records be kept in a comprehensive and proper fashion and, especially, that this documentation be understood and valued. Within this context, the use of standardized terminology and electronic systems for nursing documentation have gained ground and helped enhance the quality of nursing records (Linch, Müller-Staub, & Rabelo, 2010).

In Brazil, although nursing documentation is a routine practice, nurses rarely use standardized language for this purpose (Ochoa-Vigo, Pace, & Santos, 2003). In the United States, 64% of nursing records are still paper based. Data from a U.S.-based integrative review of the literature designed to examine the relationship between electronic nursing documentation and quality of patient care suggest that these records lack the objectivity and reliability required to ensure their excellence (Kelley, Brandon, & Docherty, 2011). The authors of this review also suggest that the extent to which electronic nursing documentation improves the quality of care—an area in which research is lacking—needs to be explored.

The Quality of Diagnoses, Interventions and Outcomes (Q-DIO) instrument, a reliable and valid tool that can be used to assess the quality of nursing records regardless of documentation format, bridges this knowledge gap (Müller-Staub, Lunney, et al., 2008; Müller-Staub, Needham, et al., 2008; Müller-Staub et al., 2009). Within this context, in view of the heterogeneity of nursing documentation systems in Brazil, the United States, and, consequently, in similar countries, excellence in patient care requires assessment of the consistency of nursing documentation. Although the original Q-DIO instrument was developed in English, it has only been validated for use in Swiss hospitals.

Aim

To validate the Q-DIO for use in three different settings in Brazil and in the United States.

Method

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Conclusion
  7. Acknowledgments
  8. References

Study Design

This methodological instrument validation study was carried out at three centers (two in Brazil and one in the United States). The data collection period was from November to December 2011 in Brazil and from May to June 2012 in the United States.

Participants and Sampling Procedure

Three separate groups were established: (a) electronic records and standardized terminology for nursing diagnoses (NDs; center 1); (b) paper-based records and no standardized terminology (center 2); (c) electronic records and no standardized terminology (center 3). The sample comprised all nursing records of patients post cardiac surgery who had spent at least 48 hr in one of the study units and whose records contained history, progress notes, and nursing prescriptions for a period of at least 4 days, as recommended by the developers of the original Q-DIO instrument (Müller-Staub, Lunney, et al., 2008; Müller-Staub, Needham, et al., 2008; Müller-Staub et al., 2009).

The minimum sample size required for this study was calculated as 145 observations, as the literature recommends 5–10 observations per instrument item for calculation of Cronbach's alpha (Hair, Tatham, Rolph, & Black, 2009). The study sample ultimately comprised 60 patient records from each of the participating centers for a total of 180 records.

The sampling technique consisted of compilation of a list of all eligible patients who had undergone cardiac surgery within the year preceding data collection at each of the participating centers. This list was then randomly sampled (taking into account 20% sampling loss, e.g., incomplete records) with the aid of the SPSS 18.0 software (SPSS, Inc., Chicago, IL, USA). Furthermore, 20% of records (24 patients) from each center were randomized for assessment of intraobserver reliability (test and retest) and interobserver agreement.

Psychometric Test of the Brazilian Version of the Q-DIO

  • Reliability and stability: The reliability of the Q-DIO was assessed by Cronbach's alpha for the 29 items by the test and retest (intraobserver three weeks later).
  • Interobserver agreement: The interobserver agreement was assessed by two raters trained in the same records independently.
  • Construct validity: The divergent construct validity was assessed in the comparison between three groups of nursing records: electronic records and standardized terminology for NDs; paper-based records and no standardized terminology; and electronic records and no standardized terminology.

Validated Instrument: Q-DIO

The main purpose of the Q-DIO is to assess the accuracy of NDs, effectiveness of interventions, and the quality of patient outcomes in documentation, with a focus on issues recorded in the nursing history and on progress notes as a record of the nursing process, regardless of the use of standardized language. It is applicable to electronic and paper-based records alike (Müller-Staub, Lunney, et al., 2008; Müller-Staub, Needham, et al., 2008; Müller-Staub et al., 2009). The Q-DIO consists of 29 items divided across four domains: (a) NDs as process, (b) NDs as product, (c) nursing interventions, and (d) nursing-sensitive patient outcomes.

Both in the original instrument and in the Brazilian version, items are graded on a three-point Likert-type scale, with 0 being “not documented,” 1 being “partially documented,” and 2 being “comprehensive documentation.” Assuming that each of the 29 items receives a maximum score of two, the highest possible score for the total of the Q-DIO instrument is 58 points.

Assessing correctness and accuracy was defined in the Q-DIO based on the concepts and definitions of standardized terminology as presented in the nursing diagnosis classification NANDA-I, in the Nursing Interventions Classification (NIC), and Nursing Outcomes Classification (NOC).

Analysis of the Q-DIO for comparisons among different strata was based on assessment of means scores; the higher the score, the higher the quality of the nursing documentation assessed. Analysis can also be conducted separately for each Q-DIO domain or for the sum of all domains for the instrument as a whole.

Translation and Adaptation of the Instrument for Use in Brazil and the United States

In view of differences in culture and language, validation of the Q-DIO for use in Brazil first required translation and cross-cultural adaptation of the instrument, which have been published elsewhere (Beaton, Bombardier, Guillemin, & Ferraz, 2000; Linch, Müller-Staub, Moraes, Azzolin, & Rabelo, 2012). No translation or adaptation was required for use of the instrument in the United States, as the Q-DIO was originally developed in English (Müller-Staub, Lunney, et al., 2008).

Ethical Considerations

The study was approved by the institutional review boards of the participating centers. All authors signed a data use agreement affirming their commitment to safeguarding the confidentiality of patient data.

Data Analysis

All analyses were carried out in SPSS 18.0. The reliability of the Q-DIO and its four domains was assessed by means of Cronbach's alpha coefficient. Stability and agreement and interobserver were analyzed by means of intraclass correlation coefficients (ICCs). Discriminant construct validity among the three documentation groups was assessed by analysis of variance with Tukey's post-hoc test (Hair et al., 2009).

Results

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Conclusion
  7. Acknowledgments
  8. References

The sample comprised 180 nursing records of patients' status post cardiac surgery (60 records from each of the study centers).

Internal Consistency

Cronbach's alpha values for all 29 items were ≥0.70 at all centers (0.70, 0.89, and 0.82 at center 1, center 2, and center 3, respectively). Table 1 shows item–total correlation values and Cronbach's alpha if item excluded for each center.

Table 1. Cronbach's Alpha Coefficients
 Center 1Center 2Center 3
Cronbach's alpha = 0.70Cronbach's alpha = 0.89Cronbach's alpha = 0.82
Item-total correlationaCronbach's alpha if item excludedItem-total correlationaCronbach's alpha if item excludedItem-total correlationaCronbach's alpha if item excluded
  1. a

    Pearson correlation coefficients.

 1. Actual situation, leading to the hospitalization0.3540.6790.0620.8890.3090.812
 2. Anxiety and worries related to hospitalization, expectations, and desires about hospitalization0.5550.6580.0000.886−0.0090.820
 3. Social situation and living environment/circumstances0.3990.6710.0000.8860.0640.818
 4. Coping in the actual situation/with the illness0.4540.6680.0000.8860.0450.821
 5. Beliefs and attitudes about life (related to the hospitalization)0.3240.6810.0000.8860.1440.824
 6. Information of the patient and relatives/significant others about the situation0.3720.6770.0000.8860.0640.825
 7. Intimacy, being female/male0.3510.6760.0000.8860.0000.818
 8. Hobbies, activities for leisure0.4340.6690.0000.8860.0000.818
 9. Significant others (contact persons)0.3710.6740.0000.8860.4200.814
10. Activities of daily living0.1770.6920.0950.8860.1210.818
11. Relevant nursing priorities according to the assessment0.0700.7050.0950.8860.3940.809
12. Nursing diagnosis label is formulated0.0000.6960.6700.8770.0000.818
13. Nursing diagnosis label is formulated according to NANDA-I and is numbered0.0000.6960.6700.8770.0000.818
14. The etiology (E) is documented0.0840.6960.4630.8810.3640.810
15. The etiology (E) is correct, related/corresponding to the nursing diagnosis (P)0.0640.6990.4960.8810.5290.801
16. Signs and symptoms are formulated0.3120.6810.7110.8750.2400.815
17. Signs and symptoms (S) are correctly related to the nursing diagnosis (P)0.3530.6780.7490.8720.6190.797
18. The nursing goal relates/corresponds to the nursing diagnosis0.3610.6770.6500.8760.6550.796
19. The nursing goal is achievable through nursing interventions0.1920.6910.8030.8700.6100.797
20. Concrete, clearly named nursing interventions—according to Nursing Interventions Classification—are planned (what will be done, how, how often, who does it)0.0000.6960.2750.8870.1430.817
21. The nursing interventions effect the etiology of the nursing diagnosis0.2760.6840.7380.8720.5980.798
22. Nursing interventions carried out are documented (what was done, how, how often, who did it)−0.3180.7280.3670.8830.1430.817
23. Acute, changing diagnoses are assessed daily or from shift to shift/enduring diagnoses are assessed every fourth day0.0980.6950.7360.8720.0000.818
24. The nursing diagnosis is reformulated0.2250.6910.7560.8720.2540.815
25. The nursing outcome is documented−0.1230.6990.0000.8860.5140.803
26. The nursing outcome is observably/measurably documented0.0000.6960.0000.8860.5990.800
27. The nursing outcome shows
  • improvement in patient's symptoms
  • improvement of patient's knowledge state
  • improvement of patient's coping strategies
  • improved self-care abilities
  • improvement functional status
0.1580.6920.6890.8740.5300.804
28. There is a relationship between (or connection of) nursing outcomes + nursing interventions0.1730.6930.4460.8820.6820.795
29. Nursing outcomes and nursing diagnoses are internally related0.2140.6890.5910.8770.6720.799

Stability and Interobserver Agreement

ICCs ranged from 0.64 to 0.85 for intraobserver reliability and from 0.68 to 0.82 for interobserver agreement (Figure 1). Box plots A, B, and C in Figure 1 show the intraobserver ICC values for centers 1, 2, and 3, respectively, and plots D, E, and F show their interobserver counterparts.

figure

Figure 1. Intraobserver stability: A (center 1), B (center 2), and C (center 3). Interobserver stability: D (center 1), E (center 2), and F (center 3). Center 1, electronic records with standardized language; center 2, paper-based records without standardized language; center 3, electronic records without standardized language. Q-DIO, Quality of Diagnoses, Interventions and Outcomes

Download figure to PowerPoint

Validity

Regarding discriminant construct validity, there were statistically significant differences between the three study centers in means for the sum of all 29 Q-DIO items (Figure 2), with mean scores of 36.8 ± 4.5 (95% confidence interval [CI] 35.6–37.9) at center 1 (electronic records with standardized language), 11.5 ± 6.2 (95% CI 9.9–13.1) at center 2 (paper-based records and no standardized language), and 31.2 ± 5.3 (95% CI 29.8–32.6) at center 3 (electronic records and no standardized language). Assuming that each of the 29 items receives a maximum score of two, the highest possible score for the total of the Q-DIO instrument is 58 points.

figure

Figure 2. Quality of Diagnoses, Interventions and Outcomes (Q-DIO) scores on analysis of discriminant construct validity. *Tukey's post-hoc test. Center 1, electronic records with standardized language; center 2, paper-based records without standardized language; center 3, electronic records without standardized language

Download figure to PowerPoint

Discussion

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Conclusion
  7. Acknowledgments
  8. References

This was the first study designed to validate the Q-DIO for use in Brazil and in the United States. The Q-DIO has been used to assess the accuracy of NDs, effectiveness of interventions, and quality of patient outcomes in records (regardless of the use of standardized language), and to assess the impact of educational programs to implement standardized NDs, interventions, and outcomes into practice, as well as within the context of audit systems. Although it was originally developed in English, it had only been validated in Swiss hospitals thus far (Müller-Staub, Needham, et al., 2008; Müller-Staub et al., 2009).

Regarding the reliability of the Q-DIO, Cronbach's alpha coefficients were ≥0.70 at all three study centers when the 29 instrument items were assessed. These values suggest that the 29 component items of the Q-DIO are correlated and homogeneous in that they measure the same attribute. However, we observe that, especially at center 2 (domain—NDs as product), the correlations between eight items were zero, indicating that these items are not related. Conversely, if these items were to be excluded, internal consistency should remain unchanged. The lack of standardized language and use of paper-based records do not favor a situation in which these items are covered in the interview and duly recorded by the nurse.

Similar values were obtained during the pilot study phase of the original version of the instrument when the 29 instrument items were assessed, which Cronbach's alpha values ranged from 0.80 to 0.98 (Müller-Staub, Needham, et al., 2008). The first study to assess the reliability of the original instrument was conducted on a random sample of 60 nursing records from 12 units of a single hospital in Switzerland. Of these, 30 were described as high-quality records and 30 as low-quality records, with and without standardized language, respectively. Despite the high Cronbach's alpha values calculated, the investigators noted that further research with larger sample sizes was required, and they reiterated the pilot nature of the study (Müller-Staub, Needham, et al., 2008).

Reliability was also assessed during pretesting of the Brazilian Portuguese version of the Q-DIO, in which 40 nursing records (50% electronic and compliant with the NANDA-I terminology and 50% paper-based and not using any standardized language). The Cronbach's alpha coefficient calculated in this sample was 0.97 (Linch et al., 2012). These findings corroborate those of prior studies in which the component items of the Q-DIO were found to measure the same attribute of nursing documentation quality, regardless of the use of standardized terminology.

Regarding stability, ICC values at centers 1, 2, and 3, respectively, were 0.74, 0.64, and 0.85 for intraobserver reliability, and 0.70, 0.68, and 0.82 for interobserver agreement. Coefficients for centers 1 and 2 corresponded to satisfactory agreement, whereas those found in center 3 corresponded to excellent agreement. Centers 1 and 2 also differed with respect to the type of nursing records. The lowest ICC values were found at center 2, where nursing records were paper based and used nonstandardized language. In this setting, free text nursing records may have contributed to variation in inter- and intraobserver assessment due to the subjectivity of individual interpretations. Conversely, center 3 (electronic records and no standardized language) had the highest coefficients of agreement. At this center, even though assessment was conducted by a U.S. investigator (native speaker of English) and by a Brazilian observer (native speaker of Portuguese), comparison between their assessments demonstrated the stability of the instrument. The findings of inter- and intraobserver assessment in the original Q-DIO study also confirmed the stability of the instrument when administered by different raters (Fayers & Machin, 2000).

In this study, discriminant construct validity was used to assess the ability of the Q-DIO to discriminate between different types of nursing records (paper-based versus electronic, with versus without standardized language). This analysis showed significant differences in means for the sum of all 29 instrument items among the three study centers.

In a prior study performed by the instrument developer to assess the discriminant ability of the Q-DIO, a randomized clinical trial was conducted to assess improvement in the quality of nursing records after an educational intervention supporting NDs implementation into clinical practice. The intervention group took part in 1.5-hr weekly sessions, led by a nurse specialized in clinical reasoning and using NANDA-I NDs, the NOC, and the NIC, for a period of 5 months. These sessions included discussion of actual cases of hospitalized patients to facilitate critical thinking when using standardized terminology. Instead of this educational program, the control group received the conventional training provided by the institution, which consisted of case discussions with no questions or reflection; textbooks were provided. At the end of the study, 225 nursing records were assessed using the Q-DIO. The study results revealed highly significant differences in quality in all three Q-DIO domains—accuracy of NDs, effectiveness of nursing interventions, and quality of nursing-sensitive patient outcomes (Müller-Staub, Needham, et al., 2008). These findings suggest that the Q-DIO was able to discriminate between the quality of records kept by nurses in the intervention group and those kept by nurses in the control group.

Among other proposed uses for the Q-DIO, the instrument has also been employed as an audit indicator to ascertain the quality of nursing records. The first study, conducted at a Swiss hospital, sought to assess the impact of implementation of an educational program on the quality of NDs, interventions, and outcomes. A total of 36 nursing records were randomly selected from 12 units of the study hospital and assessed before and after an educational program that involved implementation of NANDA–NOC–NIC terminology. Among several implications for nursing practice, this study found a significant improvement in the quality of nursing records, as measured by the Q-DIO, after the implementation of the standardized terminologies (Müller-Staub, Needham, Odenbreit, Lavin, & van Achterberg, 2007).

The Q-DIO was developed for applicability to clinical practice settings, and it has proved its value as such. Studies conducted in a variety of different contexts have validated the Q-DIO as a feasible instrument for application in real-world practice.

Conclusion

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Conclusion
  7. Acknowledgments
  8. References

Having tested the psychometric properties of the Q-DIO on nursing records kept in Brazil and the United States, we conclude that the Brazilian Portuguese version of the instrument has good reliability (internal consistency and stability) as compared with the original instrument, which, in turn, had good reliability (as assessed by the same factors) when used in a U.S. setting.

Furthermore, the Q-DIO was able to discriminate quality across different modalities of nursing documentation, including paper-based and electronic records and records documented with and without standardized language.

Acknowledgments

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Conclusion
  7. Acknowledgments
  8. References

This work was supported by the Fundo de Incentivo à Pesquisa e Eventos (FIPE) do Hospital de Clínicas de Porto Alegre (HCPA) and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).

References

  1. Top of page
  2. Abstract
  3. Method
  4. Results
  5. Discussion
  6. Conclusion
  7. Acknowledgments
  8. References
  • Beaton, D., Bombardier, C., Guillemin, F., & Ferraz, M. (2000). Guidelines for the process of cross-cultural adaptation of self-report measures. Spine, 25(24), 31863191.
  • Fayers, P. M., & Machin, D. (2000). Quality of life: Assessment, analysis and interpretation. England: John Wiley & Sons.
  • Hair, J. F., Tatham, R. L., Rolph, A., & Black, W. (2009). Análise Multivariada de Dados (6th ed.). Porto Alegre, Brazil: Artmed.
  • Kelley, T. F., Brandon, D. H., & Docherty, S. L. (2011). Electronic nursing documentation as a strategy to improve quality of patient care. Journal of Nursing Scholarship, 43(2), 154162.
  • Linch, G. F. C., Müller-Staub, M., Moraes, M. A., Azzolin, K., & Rabelo, E. R. (2012). Cross-cultural adaptation of the Quality of Diagnoses, Interventions and Outcomes (Q-DIO) Instrument into Brazilian Portuguese. International Journal of Nursing Knowledge, 23(2), 18.
  • Linch, G. F. C., Müller-Staub, M., & Rabelo, E. R. (2010). Quality of nursing records and standardized language: Literature review. Online Brazilian Journal of Nursing, 9(2), 0106.
  • Müller-Staub, M. (2009). Evaluation of the implementation of nursing diagnoses, interventions, and outcomes. International Journal of Nursing Terminologies and Classifications, 20(1), 915.
  • Müller-Staub, M., Lunney, M., Lavin, M., Needham, I., Odenbreit, M., & van Achterberg, T. (2008). Testing the Q-DIO as an instrument to measure the documented quality of nursing diagnoses, interventions, and outcomes. International Journal of Nursing Terminologies and Classifications, 19(1), 2027.
  • Müller-Staub, M., Lunney, M., Odenbreit, M., Needham, I., Lavin, M., & van Achterberg, T. (2009). Development of an instrument to measure the quality of documented nursing diagnoses, interventions and outcomes: The Q-DIO. Journal of Clinical Nursing, 18(7), 10271037.
  • Müller-Staub, M., Needham, I., Odenbreit, M., Lavin, M., & van Achterberg, T. (2007). Improved quality of nursing documentation: Results of a nursing diagnoses, interventions, and outcomes implementation study. International Journal of Nursing Terminologies and Classifications, 18(1), 517.
  • Müller-Staub, M., Needham, I., Odenbreit, M., Lavin, M., & van Achterberg, T. (2008). Implementing nursing diagnostics effectively: Cluster randomized trial. Journal of Advanced Nursing, 63(3), 291301.
  • Ochoa-Vigo, K., Pace, A., & Santos, C. (2003). Retrospective analysis of nursing records at a specialized unit. Revista Latino-Americana de Enfermagem, 11(2), 184191.
  • Paans, W., Sermeus, W., Nieweg, R., & Van der Schans, C. (2010). Prevalence of accurate nursing documentation in patient records. Journal of Advanced Nursing, 66(11), 24812489.
  • Saranto, K., & Kinnunen, U. (2009). Evaluating nursing documentation—research designs and methods: Systematic review. Journal of Advanced Nursing, 65(3), 464476.