SEARCH

SEARCH BY CITATION

Keywords:

  • implementation;
  • point-of-care systems;
  • primary care;
  • quality assurance;
  • randomized controlled trial;
  • reminder systems

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Conclusion and discussion
  7. Conflict of interest
  8. Ethical approval
  9. Acknowledgements
  10. Author contributions
  11. References

Rationale, aims and objectives

This study aimed to evaluate the relative effectiveness of electronic and postal reminders for increasing adherence to the quality assurance programme for the international normalized ratio (INR) point-of-care testing (POCT) device in primary care.

Methods

All 213 family practices that use the Elective Laboratory of the Capital Region, Denmark, and regularly conduct INR POCT were randomly allocated into two similarly sized groups. During the 4-month intervention, these practices were sent either computer reminders (ComRem) or computer-generated postal reminders (Postal) if they did not perform a split test to check the quality of their INR POCT for each calendar month. The adherence of the practices was tracked during the subsequent 8 months subdivided into two 4-month periods both without intervention. Outcomes were measures of split test procedure adherence.

Results

Both interventions were associated with an increase in adherence to the split test procedure – a factor 6.00 [95% confidence interval (CI) 4.46–7.72] and 8.22 [95% CI 5.87–11.52] for ComRem and Postal, respectively – but there is no evidence that one of the interventions was more effective than the other. In the ComRem group, the expected number of split tests (out of four) was 2.54 (95% CI 2.33–2.76) versus 2.44 (95% CI 2.24–2.65) in the Postal group, P = 0.14. There was a slight decrease in adherence over the two follow-ups, but neither intervention was better than the other in achieving a lasting improvement in adherence.

Conclusion

Computer reminders are as efficient as postal reminders in increasing adherence to a quality assurance programme for the INR POCT device in primary care.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Conclusion and discussion
  7. Conflict of interest
  8. Ethical approval
  9. Acknowledgements
  10. Author contributions
  11. References

Point-of-care testing (POCT) in primary care is frequently used because of the prompt availability of the test results at the patient encounter [1–4]. A popular POCT device measures the international normalized ratio (INR) indicating the ability of blood to clot during treatment with vitamin K antagonists. However, a faulty INR POCT result may result in severe complications for the patient. Therefore, these POCT devices are often calibrated as part of quality assessment programmes [5,6]. In the Capital Region of Denmark, quality assessment of POCT is implemented as monthly split test procedures and annual visits by a consultant technician from the Elective Laboratory of the Capital Region (‘the Laboratory’). In the period leading up to the study start, the Laboratory recorded an alarmingly low number of split test procedures for INR POCT devices for the family physicians in the area. For example, in the 4-month baseline period, 68.5% did not conduct a split test. Therefore, the Laboratory and the Capital Region administration decided to add a reminder intervention to the general implementation activities (GIA) to promote adherence to the split test procedure. Because improving quality assessment adherence for practices conducting INR POCT was prioritized, a reminder scheme was implemented for all such practices. The reminders were either computer reminders (ComRem) embedded in the physicians' electronic medical records or computer-generated postal reminder letters. As the effectiveness of such schemes is unclear, the reminder schemes were evaluated in a randomized controlled trial (RCT) concurrent to their implementation.

In a parallel study on practices in the same area, performing haemoglobin and glucose POCT but not INR POCT, an electronic reminder scheme was implemented for only half the practices at a time. In this study we found that ComRem increased compliance to the quality assessment of these POCT devices beyond that due to the simultaneous GIA [7]. Therefore, ComRem can be assumed to increase the adherence to the INR POCT split test programme as well. This is in line with various systematic reviews that assess the effects of computer reminders on the behaviour of clinicians [8–11]. These reviews show a small-to-moderate behavioural effect of computer reminders, but as many different types of reminder scheme have been evaluated and results are heterogeneous, more research is needed to identify what type of reminder works and when [11]. Computer communication, often email, to clinicians has not been shown to be better than conventional postal communication in various settings. On the contrary, the response to postal invitations is often higher [12–14]. However, the electronic communication as implemented with ComRem in primary care demands active involvement of the physician. This may improve its effectiveness over postal reminders, which may be handled by staff not directly responsible for quality assurance. The aim of this RCT was to assess the effect of ComRem compared with postal reminder letters on family practices' adherence to clinical quality programmes regarding POCT.

Materials and methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Conclusion and discussion
  7. Conflict of interest
  8. Ethical approval
  9. Acknowledgements
  10. Author contributions
  11. References

Study participants

The study was conducted among the 567 family practices, comprising 739 family physicians, in the Capital Region of Denmark (population 1.1 million) serviced by the Laboratory during 2010–2011. We included all practices that conducted more than four POCT tests for INR in the 4-month baseline period January–April 2010. These practices were identified in the primary care database of the Capital Region. The practices were not asked for consent regarding inclusion because the study was an evaluation of formal quality assurance activities by the Laboratory and the Capital Region. The study protocol is presented in detail elsewhere [15].

The split test procedure

The quality assessment programme requires split test procedures for the INR POCT device each month. In this procedure, an INR is measured with the POCT device for a selected patient. Subsequently, a simultaneously drawn blood sample from the selected patient is sent to the Laboratory for analysis together with the INR POCT result. The quotient of the two values should be within the interval 0.74–1.26 [16]. The Laboratory requests a new split test if the quotient is outside this interval and will assist in finding the problem if the new test result is also outside the interval. Communication with the Laboratory uses a constructed person identifier to render the selected patient anonymous and to later recognize this test result as a split test in the electronic patient system. The responsibility of conducting the split test procedure lies with the practice, but the organization may vary at the practice level and may be delegated to a dedicated nurse, secretary or physician.

Intervention: ComRem and postal reminders

In both arms of the RCT, the intervention was the sending of a reminder at the start of a new calendar month to practices that did not do a split test procedure for their INR POCT device in the preceding month. These reminders were postal reminders in one arm of the RCT and ComRem in the other. The latter used the MedCom standard used in the Danish health care sector since the late 1990s. Using this standard, the reminder arrived directly in the electronic patient system of the practice in the form of a laboratory message for a person with a constructed identifier. The electronic patient system is the practice's communication hub to the rest of the health care system and receives, for example, discharge letters from hospitals and specialists, medicine prescriptions and laboratory test results. These messages have to be opened and approved by the physician. If the practice has multiple physicians, depending on the system, either all physicians or a dedicated physician opens these messages [15]. In contrast, letters arriving in the mail may not be opened timely, not opened by a physician and/or not distributed to the physician in charge of quality assurance of POCT. Figure 1 shows the reminder content, which was the same in the two reminder types.

figure

Figure 1. Content of both the computer reminder (ComRem) and the postal reminder.

Download figure to PowerPoint

General implementation activities

Coinciding with the start of the study in 2010, the Laboratory increased promotion and ease of access to the quality assessment programme in general practice. Specifically, the importance of performing the split tests was emphasized by the Laboratory consultant at the annual visit to the practice and also in three Laboratory newsletters. The consultants were ignorant to the allocation of the clinics. Moreover, a new procedure, introduced in April 2010, allowed the physicians to request split tests similarly to other tests in their electronic patient system using a constructed person identifier for all these tests. The results return to their system as other results and are gathered in the files for the constructed person identifiers.

Randomization and design

The randomization of practices into two groups of similar size was performed using computer-generated random numbers (SAS version 9.2, SAS Institute, Inc., Cary, NC, USA). This was carried out separately for group practices and solo practices, respectively, to ensure a balanced practice type distribution in the two arms. Randomization was performed by the data manager of the Research Unit of General Practice, who was blinded to the individual practice identification. Allocation was conducted by an organization independent of the study group: the Danish College of General Practitioners. The study group delivered practice identification and practice type to the College where designated staff allocated the practices and returned this, thereby concealing the allocation to the study group.

The interventions took place during the 4-month intervention period (September–December 2010), in which one group received postal reminders (the Postal group) and the other received ComRem (the ComRem group); and in two consecutive 4-month follow-up periods (January–April 2011 and May–August 2011), in which the reminder scheme interventions were discontinued. In each of the three intervention and follow-up periods, all practices were exposed to GIA in addition to the intervention.

Sample size

Laboratory data from 2007 showed that on average 1.5 split tests were performed for the INR POCT device in a 4-month period by the practices performing INR POCT with a standard deviation of 1. To detect an average difference of 0.5 split tests between the randomization arms with 90% power on a 5% significance level, at least 172 practices should be included in this study (SAS version 9.2 PROC POWER). Dropout (retirement of the family physician) and non-response (if a practice did not use the INR POCT device at all during the 4 months) was minimal as the study used data that were routinely collected for administrative purposes.

Data

Data on when split test procedures were performed and when reminders were sent were obtained from the Laboratory's database. This information cannot be related to individual patients because all split test procedures use artificial identification detached from the corresponding patient on whom the test was performed. Information on the practices and physicians included in the study, and the reimbursement claims for INR POCT tests, was supplied by the Capital Region's administrative databases.

Outcomes

Outcomes were measures of adherence to the split test procedure based on the number of split tests received by the Laboratory for each practice. Outcomes were compared between the randomization arms for three 4-month intervention and follow-up periods. The comparison over the first intervention period estimates the efficacy of ComRem versus postal reminders. The comparison in the second and third follow-up periods estimates the extent to which the respective reminder types induce a lasting improvement on quality assessment behaviour.

Primary outcome

The primary outcome was the number of split test procedures for the INR POCT device performed by the practice over 4 months. A split test is to be performed each month, and if the practice conducted at least one INR POCT analysis a month, the number of possible split test procedures for the INR POCT device over 4 months ranges from zero to four. For small practices that do not use the device every month, fewer split tests are required. A reminder is sent if the Capital Region's database has registered claims for INR POCT tests for the practice, but a split test is not registered in the Laboratory's database. Hence, up to four reminders were sent in the 4-month intervention period.

Secondary outcomes
  1. Whether the practice performed an acceptable number – at least 75%, that is, three or four – of the required split tests over 4 months.
  2. Whether the practice performed any of the required split tests over 4 months.

Statistical analysis

Differences in characteristics of the practice and physicians and outcomes in the baseline period between the arms of the RCT were tested by t-tests (for continuous characteristics or outcomes) and chi-square tests (for categorical characteristics or outcomes). Adherence to the split test procedure in the baseline period and the three intervention and follow-up periods for the two reminder types was analysed by Poisson (primary outcome) and logistic (secondary outcomes) regression. The method of generalized estimating equations was used to account for the repeated observation of practices. A P-value below 0.05 was taken as being statistically significant. Statistical analyses were performed in SAS version 9.2 (SAS Institute, Inc.).

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Conclusion and discussion
  7. Conflict of interest
  8. Ethical approval
  9. Acknowledgements
  10. Author contributions
  11. References

The study included 213 practices conducting at least four INR POCT tests during the baseline period. There were no imbalances between the randomization arms in physician and practice characteristics, and in the (low) degree of adherence to the split test procedure (Table 1) in the baseline period before study start. Dropout because of retirement or not using the INR POCT device was rare (4%; Fig. 2). In the intervention period the Laboratory sent 317 reminders: 154 in the ComRem group and 163 in the Postal group.

figure

Figure 2. Practices included in the analysis. GIA, general implementation activity; INR, international normalized ratio; POCT, point-of-care testing.

Download figure to PowerPoint

Table 1. Physician and practice characteristics, POCT use and quality assessment outcomes in the baseline period (January–April 2010)
  Allocation 
TotalComRemPostalP-valuea
  1. Values in italics are the number of analyzed physicians and/or practices (in each period).

  2. a

    P-value from a t-test (continuous variables) or chi-squared test (categorical variables).

  3. b

    Tests regarding physician characteristics are adjusted for clustering of physicians in practices.

  4. c

    A split test is required for each month in which at least one INR POCT is performed. For practices that did not perform INR POCT in one or more of the 4 months in the baseline period, the number of split tests performed is multiplied by the inverse of the fraction of months in which INR POCT is performed so that the measure of quality assessment adherence is comparable with that of practices that did perform INR POCT in each of the 4 months. INR, international normalized ratio; POCT, point-of-care testing; SD, standard deviation.

Physician characteristicsb(n = 318)(n = 167)(n = 151) 
Sex, n (%)   0.62
Male177.0 (55.7)95.0 (56.9)82.0 (54.3) 
Female141.0 (44.3)72.0 (43.1)69.0 (45.7) 
Age, mean (SD)53.1 (7.9)53.1 (8.0)53.1 (7.7)0.99
Years as family physician, mean (SD)11.3 (8.8)11.0 (9.0)11.8 (8.6)0.47
Practice characteristics(n = 213)(n = 107)(n = 106) 
Practice type, n (%)   0.81
Single handed139.0 (65.3)69.0 (64.5)70.0 (66.0) 
Group74.0 (34.7)38.0 (35.5)36.0 (34.0) 
Number of patients on practice list per physician, mean (SD)1674.0 (427)1651.0 (387)1697.0 (466)0.44
Performed point-of-care tests
Number of INR tests per practice over 4 months, mean (SD)86.2 (57.9)95.3 (64.8)77.0 (48.5)0.020
Primary outcome
Number (out of 4)c of split tests per practice, mean (SD)0.71 (1.11)0.82 (1.16)0.59 (1.04)0.13
Secondary outcomes
Practices with >0% of recommended split tests performed, n (%)67.0 (31.5)39.0 (36.5)28.0 (26.4)0.11
Practices with ≥75% of recommended split tests performed, n (%)0 (0.0)0 (0.0)0 (0.0)

Both the ComRem and the Postal intervention were associated with a sharp increase in adherence to the split test procedure compared with the baseline period – a factor 6.00 [95% confidence interval (CI) 4.46–7.72] and 8.22 (95% CI 5.87–11.52), respectively – but these increases were of similar size in the two allocation groups (P = 0.14; Fig. 3; Table 2). Hence, there is no evidence that one of the interventions was more effective than the other in increasing adherence to the INR POCT calibration procedure. Correspondingly, the slight, but significant, decrease in adherence over the two follow-up periods was also of similar size in the two groups; one intervention was not better than the other in achieving a lasting improvement in adherence to the quality assessment programme.

figure

Figure 3. Primary outcome – number of required split tests performed in each study period out of the maximum of four – through the baseline period, the intervention period and the two follow-up periods for the two allocation groups.

Download figure to PowerPoint

Table 2. Study outcomes through the baseline period, the intervention period and the two follow-up periodsa
 Allocation 
ComRemPostalP-valueb
  1. Values in italics are the number of analyzed physicians and/or practices (in each period).

  2. a

    The values in the table are the expected values of the Poisson regression model (primary outcome) or the logistic regression model (secondary outcomes) and their 95% confidence interval (CI) calculated with generalized estimating equation methods.

  3. b

    P-value of a test for the difference between ComRem and Postal (baseline period) or a test for the difference between ComRem and Postal beyond the difference in the baseline period (the intervention period and the two follow-up periods).

  4. c

    That is, the expected number of split tests for a practice that performs international normalized ratio point-of-care testing in each of the 4 months in the corresponding study period.

Baseline period (January–April 2010)(n = 107)(n = 106) 
Primary outcome
Split tests per practice, number (out of 4)c (95% CI)0.42 (0.33–0.55)0.30 (0.21–0.41)0.10
Secondary outcomes
Practices with >0% of required split tests performed, % (95% CI)36 (28–46)26 (19–36)0.12
Practices with ≥75% of required split tests performed, % (95% CI)00
Intervention period (September–December 2010)(n = 105)(n = 104) 
Primary outcome
Split tests per practice, number (out of 4)c (95% CI)2.54 (2.33–2.76)2.44 (2.24–2.65)0.14
Secondary outcomes
Practices with >0% of required split tests performed, % (95% CI)91 (84–95)91 (84–95)0.40
Practices with ≥75% of required split tests performed, % (95% CI)54 (45–64)50 (41–60)0.18
Follow-up period (January–April 2011)(n = 105)(n = 102) 
Primary outcome
Split tests per practice, number (out of 4)c (95% CI)2.27 (2.01–2.56)2.43 (2.19–2.69)0.052
Secondary outcomes
Practices with >0% of required split tests performed, % (95% CI)82 (73–88)88 (80–93)0.049
Practices with ≥75% of required split tests performed, % (95% CI)48 (38–57)54 (45–64)0.52
Follow-up period (May–August 2011)(n = 105)(n = 100) 
Primary outcome
Split tests per practice, number (out of 4)c (95% CI)2.17 (1.94–2.44)2.00 (1.76–2.27)0.22
Secondary outcomes
Practices with >0% of required split tests performed, % (95% CI)85 (76–90)81 (72–87)0.65
Practices with ≥75% of required split tests performed, % (95% CI)47 (37–56)37 (29–47)0.034

The secondary outcomes show a development similar to that of the primary outcome. In the first follow-up, the postal intervention maintained a significantly better minimal adherence, whereas the ComRem intervention maintained a significantly better acceptable adherence in the second follow-up, that is, 75%. Notably, for both interventions, only about half the practices adhered to the quality assurance programme to an acceptable level, and even in the intervention period, 9% of practices perform no split tests.

Conclusion and discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Conclusion and discussion
  7. Conflict of interest
  8. Ethical approval
  9. Acknowledgements
  10. Author contributions
  11. References

Summary of main findings

This RCT shows that the interventions increased adherence to the POCT quality assurance programme in primary care, but to the same extent for the two reminder types. Accordingly, no difference was shown in the efficacy of reminder messages implemented either as ComRem or as postal reminders. The differences in the way the interventions instigated organizational changes that last beyond the intervention period were only slight and ambivalent. The increase in adherence to the split test programme from baseline to intervention seen in both allocation groups cannot be ascribed to the interventions alone, but also to GIA, which was started in the baseline period for all practices in the Capital Region. However, only about 50% of practices achieved acceptable adherence to the programme, which is low relative to the importance of an accurate INR POCT result, and a small group of 9% hard line non-adhering practices seem to be impervious to the efforts to promote the quality assurance programme.

Comparison with other studies

A recent Cochrane review on the impact of computer reminders on processes of care [11] finds that computer reminders generally succeed in changing adherence to recommended protocol only marginally. The review concludes that more research is needed into features of reminder systems that consistently associate with clinically worthwhile improvements. Correspondingly, a review on prompting clinicians about preventive care measures [8] shows a similar, modest improvement for paper-based and computerized reminders. We also found that computer reminders in the present ComRem implementation did not perform better or worse than postal reminders [5,6].

Some studies comparing electronic and postal communication to clinicians [12–14] find higher response rates for the postal modality [12,13]. Whereas these studies evaluated standard email reminders, our study used electronic reminders that exploit the electronic medical record and found that these reminders performed as well as postal reminders and may well be cheaper. Although operation costs are clearly lower for ComRem, the cost of developing and evaluating ComRem is difficult to gauge and greatly depends on the setting. The other studies calculate lower costs for electronic communication in their respective settings [12–14].

The present analysis does not indicate what is best for a specific practice (e.g. review of POCT testing procedure, organizational restructuring) to increase future adherence when adherence is suboptimal. An older study attributed low response rates to electronic communication to technological ignorance [13], but this may no longer be a relevant explanation.

The strong increase in adherence coinciding with the intervention period in the present study is much larger than the effects reported by most of the studies in the aforementioned reviews. The primary reason is that concurrent with the intervention, GIA was started, which in itself has been shown to increase adherence [7]. Moreover, the studies in the other reviews often deal with reminders for directly optimizing care processes appearing online on the computer screen during the consultation with the patient. The reminders in the present study do not appear on the screen during the patient encounter and have the simpler aim of optimizing the quality of the POCT. This relative simplicity may also account for the larger effects seen in this study [17].

Strengths and limitations

The present prospective study is based on data from administrative databases; accordingly, the outcomes are not biased by subjective perceptions or recollection. Similarly, because study participation was not voluntary and included all eligible practices in the Capital Region, selection bias is avoided. There was minimal dropout, which was primarily caused by retirement of the physician during the study period. Limitations of the present study are that only the adherence to the quality assurance programme was measured and not the quality of the actual INR test result. It could be expected that POCT quality increases with better adherence to the split test programme, but we cannot conclude on this. Although ComRem is a low-cost solution and easily converted into a permanent reminder scheme in the present setting, other settings with different information technology infrastructure and software may find the implementation of some form of ComRem more problematic. Similarly, other settings have different organizational structures and different relations between practices and their associated laboratories so that also postal reminders are not as easily implemented as in the present setting where the quality assessment of a practice was performed by only one laboratory. The reminder (Fig. 1) mentions that reimbursement will be withheld if the practices do not adhere to the split test programme. However, this has never been enforced, and this was known by the family physicians in the Region. Enforcing this clause would probably lead to an increase in the effect of the reminder programme.

Implications for future research or practice

Previous studies [7,11] show that reminder schemes can increase adherence to quality assurance standards and call for studies that explore the optimal implementation. The present study shows that electronic reminders increase adherence to the same extent as postal reminders for the quality assurance programme of the INR POCT device in primary care. Although the introduction of a reminder scheme is seen as associated with a considerable increase in adherence to the split test programme, about half the practices did not reach an acceptable level of adherence and a small group performed no split test at all. Future research could investigate which type of practice neglects the quality assurance scheme and why this is the case in order to design an intervention more specifically for this group. Additionally, it could be investigated whether reminder schemes for one POCT device have a spillover effect on the adherence to quality assurance programmes for other POCT devices. Lastly, the effect on adherence of withholding reimbursement in cases of unacceptable quality assurance could be investigated.

Conflict of interest

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Conclusion and discussion
  7. Conflict of interest
  8. Ethical approval
  9. Acknowledgements
  10. Author contributions
  11. References

The study was initiated by Frans Boch Waldorff, who is a family physician in the study area. Peter Felding is working in the Elective Laboratory of the Capital Region that provided data for the study and sent out the reminders.

Ethical approval

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Conclusion and discussion
  7. Conflict of interest
  8. Ethical approval
  9. Acknowledgements
  10. Author contributions
  11. References

This study used blood samples that were not retraceable to specific patients in order to conduct the split tests. Patient identities were kept hidden through an artificial identification code used by all practices. The project was evaluated and approved by the Danish Data Protection Agency (j. nr. 2010-41-4680) and the Danish College of General Practitioners Study Committee (MPU 12-2010). The study was also submitted for assessment by the Scientific Ethics Committee for Copenhagen and Frederiksberg Municipalities (j. nr. H-1-2010_FSP/10). The committee found that approval was not necessary since the study was not clinical.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Conclusion and discussion
  7. Conflict of interest
  8. Ethical approval
  9. Acknowledgements
  10. Author contributions
  11. References

We thank all participating family physicians, the Capital Region and the Elective Laboratory of the Capital Region for providing data, Niels Hornum for programming the algorithm used to produce reminders, and Willy Karlslund for handling the randomization and data management. This study was funded by the Research Unit for General Practice, the Danish Research Foundation for General Practice and the Capital Region of Denmark.

Author contributions

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Conclusion and discussion
  7. Conflict of interest
  8. Ethical approval
  9. Acknowledgements
  10. Author contributions
  11. References

Siersma participated in the design of the trial, did the statistical analyses and drafted the manuscript. Waldorff initiated the trial and participated in its design and management. Kousgaard participated in the design of the trial. Felding participated in the design of the trial, provided data and managed the reminder procedures. Ertmann and Reventlow participated in the design of the trial. All authors have read, commented on and approved the manuscript.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Materials and methods
  5. Results
  6. Conclusion and discussion
  7. Conflict of interest
  8. Ethical approval
  9. Acknowledgements
  10. Author contributions
  11. References
  • 1
    Cals, J. W., Schot, M. J., de Jong, S. A., Dinant, G. J. & Hopstaken, R. M. (2010) Point-of-care C-reactive protein testing and antibiotic prescribing for respiratory tract infections: a randomized controlled trial. Annals of Family Medicine, 8 (2), 124133.
  • 2
    Gialamas, A., St John, A., Laurence, C. O. & Bubner, T. K. (2010) Point-of-care testing for patients with diabetes, hyperlipidaemia or coagulation disorders in the general practice setting: a systematic review. Family Practice, 27 (1), 1724.
  • 3
    Hobbs, F. D., Delaney, B. C., Fitzmaurice, D. A., Wilson, S., Hyde, C. J., Thorpe, G. H., Earl-Slater, A. S., Jowett, S. & Tobias, R. S. (1997) A review of near patient testing in primary care. Health Technology Assessment, 1 (5), iiv, 1–229.
  • 4
    Laurence, C. O., Gialamas, A., Bubner, T., Yelland, L., Willson, K., Ryan, P. & Beilby, J. (2010) Patient satisfaction with point-of-care testing in general practice. The British Journal of General Practice, 60 (572), e98e104.
  • 5
    Thue, G., Jevnaker, M., Gulstad, G. A. & Sandberg, S. (2011) Quality assurance of laboratory work and clinical use of laboratory tests in general practice in Norway: a survey. Scandinavian Journal of Primary Health Care, 29 (3), 171175.
  • 6
    Tirimacco, R., Glastonbury, B., Laurence, C. O., Bubner, T. K., Shephard, M. D. & Beilby, J. J. (2011) Development of an accreditation program for Point of Care Testing (PoCT) in general practice. Australian Health Review, 35 (2), 230234.
  • 7
    Kousgaard, M. B., Siersma, V., Reventlow, S., Ertmann, R., Felding, P. & Waldorff, F. B. (2013) The effectiveness of computer reminders for improving quality assessment for point-of-care testing in general practice-a randomized controlled trial. Implementation Science, 8, 47.
  • 8
    Dexheimer, J. W., Talbot, T. R., Sanders, D. L., Rosenbloom, S. T. & Aronsky, D. (2008) Prompting clinicians about preventive care measures: a systematic review of randomized controlled trials. Journal of the American Medical Informatics Association, 15 (3), 311320.
  • 9
    Garg, A. K. (2007) Automating communication: using technology to increase effectiveness and profitability. Dental Implantology Update, 18 (12), 8992.
  • 10
    Kawamoto, K. & Lobach, D. F. (2003) Clinical decision support provided within physician order entry systems: a systematic review of features effective for changing clinician behavior. AMIA Annual Symposium Proceedings, 2003, 361365.
  • 11
    Shojania, K. G., Jennings, A., Mayhew, A., Ramsay, C. R., Eccles, M. P. & Grimshaw, J. (2009) The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Systematic Review, (3), CD001096.
  • 12
    Crouch, S., Robinson, P. & Pitts, M. (2011) A comparison of general practitioner response rates to electronic and postal surveys in the setting of the National STI Prevention Program. Australian and New Zealand Journal of Public Health, 35 (2), 187189.
  • 13
    Raziano, D. B., Jayadevappa, R., Valenzula, D., Weiner, M. & Lavizzo-Mourey, R. (2001) E-mail versus conventional postal mail survey of geriatric chiefs. The Gerontologist, 41 (6), 799804.
  • 14
    Treweek, S., Barnett, K., MacLennan, G., et al. (2012) E-mail invitations to general practitioners were as effective as postal invitations and were more efficient. Journal of Clinical Epidemiology, 65 (7), 793797.
  • 15
    Waldorff, F. B., Siersma, V., Ertmann, R., Kousgaard, M. B., Nielsen, A. S., Felding, P., Mosbaek, N., Hjortso, E. & Reventlow, S. (2011) The efficacy of computer reminders on external quality assessment for point-of-care testing in Danish general practice: rationale and methodology for two randomized trials. Implementation Science, 6, 79.
  • 16
    Danish College of General Practice. (2002) Quality standards and quality assessment system for frequently performed clinical biochemistry and clinical microbiological tests in general practice. [Kvalitetskrav og kvalitetsvurderingssystem for hyppigt udførte klinisk biokemiske og klinisk mikrobiologiske analyser i almen praksis]. Report.
  • 17
    Francke, A. L., Smit, M. C., de Veer, A. J. & Mistiaen, P. (2008) Factors influencing the implementation of clinical guidelines for health care professionals: a systematic meta-review. BMC Medical Informatics and Decision Making, 8, 38.