How doctors practice evidence-based medicine

Abstract

Rationale and aims  Evidence-based medicine (EBM) has gained worldwide attention. Many studies have used questionnaires to discuss factors obstructing the practice of EBM. However, no large-scale data analysis has focused on who has practised EBM and when they practised it. This retrospective study aims to fill the research gap by applying nationally representative data to analyse EBM practice after the provision of new evidence regarding the prescription of rosiglitazone which has been shown to increase the risk of myocardial infarction.

Methods  We used the National Health Insurance Database in Taiwan to analyse the variations in rosiglitazone prescription among physicians. The study period was from the second quarter of 2007 to the fourth quarter of 2008. A total of 2536 physicians who prescribed rosiglitazone at least once were included in this study. We applied multivariate logistic analyses to predict the probability of physicians ceasing to prescribe rosiglitazone.

Results  We observed a significant improvement in EBM practice among specialists and experienced physicians. Endocrinologists were four times more likely to change rosiglitazone prescription habits than other specialists (odds ratio 4.129, 95% confidence interval 2.484–6.863). Doctors with more than 10 years of specialist experience performed better in EBM practice. Moreover, a prominent time lag with more than 6 months between EBM emergence and EBM practice was noticed.

Conclusions  Our study suggested that EBM was still not well practised, using rosiglitazone prescription as a study case. Further education and encouragement to strengthen physicians' EBM practice remain urgently needed within the medical community.

Introduction

Evidence-based medicine (EBM) uses statistical and epidemiological methods as evidence for guiding clinical care. Many factors have been identified through subjective questionnaires as barriers to the practice of acting on clear evidence by physicians [1–4]. Studies have shown that the compliance with EBM was not satisfactory but none of the studies explored the performance of doctors in detail [5,6]. In our study, the National Health Insurance Database in Taiwan was used to study how physicians' prescription behaviours changed after new and strong evidence on rosiglitazone emerged from meta-analysis.

Diabetes mellitus (DM) is an increasingly prevalent chronic disease worldwide [7]. Regular diet and medical management are required to prevent secondary complications. Thiazolidinedione is a group of oral hypoglycaemic agents that can decrease glycosylated haemoglobin, lower the incidence of treatment failure compared to metformin or sulfonylurea, improve the effectiveness of add-on therapy with hypoglycaemic agents, improve the insulin secretion capacity of beta-cells, preserve beta-cell mass and islet structure, and protect beta-cells from oxidative stress and apoptosis [8]. Rosiglitazone belongs to the thiazolidinedione group.

There have been concerns about rosiglitazone's effect on fluid retention and possible contribution to congestive heart failure since it was launched in the market. In June and September 2007, two meta-analyses indicated for the first time that rosiglitazone significantly increased the risks of myocardial infarction [9,10]. The American Diabetes Association revised their recommendations for diabetes care in January 2008 [11,12] to include concerns about myocardial infarction. The Food and Drug Administration of the United States also restricted its sale for the same reason in 2010 [13].

Although meta-analyses represent evidence of the highest reference level, there are still many potential barriers to the practice of new evidence according to the knowledge–attitude–behaviour (KAB) model [1]. Information penetration, attitudes to EBM and the resources needed are all potential areas of concern. In Taiwan, access to information from new medical studies and resources needed for EBM practice were not barriers for physicians. By 2008, more than 99% of the population in Taiwan was covered by national health insurance (NHI) and 92.5% of hospitals and local clinics were contracted with NHI [14,15]. Our goal was to explore the factors influencing the doctors' practice of EBM by reviewing their performance via nationally representative data in contrast to traditional questionnaires. Several hypotheses were developed prior to conducting this study. First, a specialist is defined as an expert in his field. Glaab et al. had studied the performance of compliance with evidence-based care on chronic obstructive pulmonary disease. They found that pulmonarists performed better than the primary care physicians [5]. For this study, endocrinologists were chosen as the representative specialists. At the same time, doctors of internal medicine should be more familiar with the treatment of DM than other specialists. Second, doctors working in medical centres should have the best knowledge of EBM. Third, EBM education has been vigorously promoted for 10 years in Taiwan so it should be more acceptable to junior doctors. Poolman et al. also found that younger surgeons, particularly those between 36 and 45 years old, as well as those with less than 10 years of professional experience, exhibited better competence in EBM [4]. Fourth, doctors practising in local clinics in counties with a better economic status and more information may be pushed to do more evidence-based service because (1) the educated consumers may demand more service [16]; (2) economic processes also influence quality expectations and the perceived value of goods and services [17]; and (3) cities allow for more information exchange. During this study, empirical data analyses were conducted and these hypotheses were examined.

Methods

We used nationally representative data to observe the prescription of rosiglitazone to patients with DM from the second quarter of 2007 to the fourth quarter of 2008. The use of the data was reviewed by the National Health Research Institute. The data used in this study came from 1 million randomly sampled people out of the 25.68 million NHI enrollees. There was no significant difference in the gender distribution of the subjects in the sample data and the original population [18]. As the physicians' perception of EBM was not available from a secondary database, we designed two models to reflect the true prescription behaviour within the medical community. In model A, we identified any doctors who stopped the prescription of rosiglitazone in subsequent patient visits during the study period. For example, if a doctor had 10 DM patients with rosiglitazone prescriptions then as long as the doctor stopped rosiglitazone prescription for at least one patient he/she was included in the study. We considered this to be the most sensitive way to identify doctors who changed their prescription behaviours as doctors were included in the study only if their prescription behaviour changed for at least one patient. For model B, we identified the doctors who had prescribed rosiglitazone and whose last prescription of rosiglitazone to any patient occurred during our study period. For example, if a doctor had 10 DM patients with rosiglitazone prescriptions, when the doctor stopped rosiglitazone prescriptions for all 10 patients then he/she was included in the study. We considered this to be the most specific way for identifying the doctors who stopped the prescription of rosiglitazone as a doctor should not prescribe any rosiglitazone after being included in the study. We assumed that the true behaviour of our medical community should fall somewhere between these two extremes. We used quarters as the basic time interval because the Bureau of NHI allowed patients with stable chronic diseases to receive 3-month prescriptions at each appointment. In model A, patients who had DM and took rosiglitazone were used to identify the doctors who made the prescription. If a patient visited the same doctor in at least two successive quarters and the prescriptions of rosiglitazone ceased on the latter visit, this was marked as an event. We excluded changes of prescription where the patients changed their doctors as the events happened at the same time. In model B, we identified the doctors who prescribed rosiglitazone during the study period. Doctors who continued to prescribe rosiglitazone in the fourth quarter of 2008 were excluded. The characters of the selected doctors were analysed and the results of the two models compared with each other.

Based on the hypotheses, we analysed three variables including the doctors' specialty, the type of the hospitals that the doctors worked in, and the years since the doctor became a specialist, to carry out univariate and multivariate analyses in both models.

We also focused on doctors working at local clinics and the criteria from the two models were applied again. The variables of specialty and age were retained while urbanization and economic status of the counties that the clinics were located in were added as new variables. The data for these two variables came from the Directorate-General of Budget, Accounting and Statistics [19,20]. We divided the counties into three groups: urban areas were those that had an average population density of 2635 persons per square kilometre in 2008, suburban areas were those with an average population density of 1458 persons per square kilometre, and rural areas were those with a population of density of 230 persons per square kilometre in 2008. We also divided the average discretionary income of every family in a county by the average number of family members in the county to reflect the economic status. The results were ranked and divided into four levels with a higher level indicating better economic status. The economic levels were used as dummy variables in the logistic regression.

Statistical analysis

We analysed the characters of the doctors in both models separately by univariate analysis using a chi-square test with a significant level of 0.05. The multivariate logistic regression model was also carried out to predict the probability of physicians ceasing to prescribe rosiglitazone.

Results

A total of 2536 doctors prescribed rosiglitazone at least once during the period of this study. In model A, univariate analysis found that doctors' specialty, type of hospital and professional experience were all significant factors in the changing rosiglitazone prescriptions. Among the specialties, endocrinologists were most likely to cease prescription (56.5%) followed by other internists (32.8%). Other specialists (21.7%) were the least likely group to cease prescription. As for the type of the hospitals, doctors working in regional hospitals were the most likely to change prescription (33.3%), followed by doctors working in medical centres (29.7%), local clinics (28.0%) and district hospitals (25.3%). In terms of professional experience, doctors with 10 to 20 years of specialist experience were most likely to change prescription (35.2%), followed by those with more than 20 years of experience (31.1%) then those with less than 10 years of experience (20.5%). Logistic regression results presented the same conclusions. In model B, the results were almost the same as model A except for a difference in the hospital type variable where no statistical significance was found. In terms of specialty, internists other than endocrinologists were the most likely to change prescription (17.5%), followed by endocrinologists (15.9%) and other specialists (12.9%). In terms of professional experience, doctors with 10 to 20 years of specialist experience were once again most likely to change prescription (20.4%) followed by those who with more than 20 years (14.4%) and those with less than 10 years (11.1%). Logistic regression reached the same conclusion as univariate analysis (Tables 1 & 2).

Table 1. Univariate analysis of all identified doctors in both models

	All doctors (n = 2536)	Model A		Model B
		Doctors changed prescription (n = 734)	P	Doctors changed prescription (n = 395)	P
In model A, we targeted a doctor who stopped prescribing rosiglitazone to at least one of the diabetes mellitus patients in their subsequent visits. In model B, a doctor was identified if he/she ceased to prescribe rosiglitazone to all of his/her diabetes mellitus patients.
Specialty	Endocrinologists (n = 69)	39 (56.5%)	<0.001	11 (15.9%)	0.009
	Internists other than endocrinologists (n = 1444)	473 (32.8%)		252 (17.5%)
	Other specialists (n = 1023)	222 (21.7%)		132 (12.9%)
Hospital type	Medical centre (n = 549)	163 (29.7%)	0.0107	89 (16.2%)	0.3573
	Regional hospital (n = 670)	223 (33.3%)		103 (15.4%)
	District hospital (n = 738)	187 (25.3%)		103 (14.0%)
	Clinic (n = 572)	160 (28.0%)		100 (17.5%)
Professional experience	<10 years (n = 758)	155 (20.5%)	<0.001	84 (11.1%)	<0.001
	10–20 years (n = 1039)	367 (35.3%)		212 (20.4%)
	>20 years (n = 655)	204 (31.2%)		94 (14.4%)

Table 2. Logistic regressions of all doctors identified in both models

	All doctors (n = 2536)	Model A			Model B
		OR	95% CI	P	OR	95% CI	P
In model A, we targeted a doctor who stopped prescribing rosiglitazone to at least one of the diabetes mellitus patients in their subsequent visits. In model B, a doctor was identified if he/she ceased to prescribe rosiglitazone to all of his/her diabetes mellitus patients. CI, confidence interval; OR, odds ratio.
Specialty	Endocrinologists	4.129	2.484, 6.863	<0.001	1.105	0.561, 2.175	0.773
	Internists other than endocrinologists	1.594	1.313, 1.935	<0.001	1.371	1.080, 1.740	0.009
	Other specialists	–	–	–	–	–	–
Hospital type	Medical centre	1.151	0.876, 1.513	0.313	0.921	0.665, 1.276	0.621
	Regional hospital	1.347	1.040, 1.746	0.024	0.845	0.617, 1.156	0.292
	District hospital	0.887	0.684, 1.150	0.366	0.741	0.542, 1.012	0.059
	Clinic	–	–	–	–	–	–
Professional experience	<10 years	–	–	–	–	–	–
	10–20 years	2.130	1.706, 2.659	<0.001	2.014	1.531, 2.651	<0.001
	>20 years	1.834	1.431, 2.349	<0.001	1.312	0.954, 1.804	0.095

We also analysed which physician groups changed their prescription earlier. After the second quarter of 2007 there was no significant difference in the average time needed for change of prescription among the variables studied in both models, with the exception of specialty in model A. For that particular variable, endocrinologists changed their prescriptions earliest with a mean of 2.23 quarters. Internists other than endocrinologists followed with a mean of 2.41 quarters. Other specialists took the longest time with a mean of 2.71 quarters (Table 3).

Table 3. Means of time periods needed to change doctors' prescription

	All doctors (n = 2536)	Model A		Model B
		Doctors changed prescription (n = 734)	P	Doctors changed prescription (n = 395)	P
In model A, we targeted a doctor who stopped prescribing rosiglitazone to at least one of the diabetes mellitus patients in their subsequent visits. In model B, a doctor was identified if he/she ceased to prescribe rosiglitazone to all of his/her diabetes mellitus patients.
Specialty	Endocrinologists	2.23 ± 1.51	0.0149	2.36 ± 1.63	0.2875
	Internists other than endocrinologists	2.41 ± 1.37		2.59 ± 1.43
	Other specialists	2.71 ± 1.42		1.80 ± 1.36
Hospital type	Medical centre	2.42 ± 1.35	0.1626	2.55 ± 1.29	0.5461
	Regional hospital	2.35 ± 1.40		2.54 ± 1.43
	District hospital	2.56 ± 1.42		2.77 ± 1.53
	Clinic	2.66 ± 1.43		2.74 ± 1.38
Professional experience	<10 years	2.57 ± 1.55	0.1437	2.71 ± 1.89	0.1552
	10–20 years	2.41 ± 1.32		2.43 ± 1.28
	>20 years	2.88 ± 1.48		2.98 ± 1.35

For doctors working in local clinics, logistic regression was carried out using doctors' specialty, age, urbanization and economic status as the variables in both models. There was no significant difference between the variables of urbanization and economic status in both models except for economic status in model B. Only the doctors on the third level of economic status were more likely to change prescription significantly (Table 4).

Table 4. Logistic regressions of local clinic doctors in both models

	All doctors (n = 2536)	Model A			Model B
		OR	95% CI	P	OR	95% CI	P
In model A, we targeted a doctor who stopped prescribing rosiglitazone to at least one of the diabetes mellitus patients in their subsequent visits. In model B, a doctor was identified if he/she ceased to prescribe rosiglitazone to all of his/her diabetes mellitus patients. CI, confidence interval; OR, odds ratio.
Specialty	Endocrinologists	5.431	2.297, 12.841	<0.001	2.586	0.967, 6.916	0.058
	Internists other than endocrinologists	1.451	0.978, 2.154	0.065	1.277	0.789, 2.043	0.309
	Other specialists	–	–	–	–	–	–
Economic status	Best	0.710	0.363, 1.388	0.317	1.796	0.730, 4.421	0.203
	Second	0.941	0.484, 1.829	0.857	2.090	0.888, 4.918	0.292
	Third	1.624	0.854, 3.087	0.139	3.513	1.537, 8.031	0.003
	Worst	–	–	–	–	–	–
Urbanization	Urban area	0.861	0.502, 1.477	0.587	0.407	0.219, 0.756	0.004
	Suburban area	1.059	0.456, 2.460	0.895	0.914	0.378, 2.209	0.841
	Rural area	–	–	–	–	–	–
Age	<40 years old	–	–	–	–	–	–
	40–50 years old	1.149	0.632, 2.089	0.648	1.324	0.661, 2.650	0.429
	>50 years old	1.418	0.798, 2.517	0.233	1.312	0.642, 2.481	0.499

Discussion

New and strong evidence from meta-analysis indicated that patients taking rosiglitazone may be at significantly higher risk of myocardial infarctions. We can use this to observe how new evidence modified our doctors' prescription behaviour. In model A, for sensitivity, we targeted doctors who stopped prescribing rosiglitazone to at least one of their patients with DM in their subsequent visits. In model B, doctors were identified for specificity if they ceased to prescribe rosiglitazone to every one of their DM patients. We assumed that true performance in our medical community should fall somewhere between the results of these two models.

In our study, doctors' specialty and professional experience were significant factors in determining whether EBM was practised or not. In model A, it was clear that endocrinologists were more likely to change prescription. When compared to other specialists, they were four times more likely to change prescription [odds ratio (OR) 4.129, 95% confidence interval (CI) 2.484–6.863]. This was also better than that of other internists (OR 1.594, 95% CI 1.313–1.935). However, only the group of internists other than endocrinologists performed significantly better than the other two groups in model B (OR 1.371, 95% CI 1.080–1.740). In terms of specialty, the results supported our hypothesis that internists should perform better with DM patients. Professional experience was also found to be a significant factor. Both models showed that doctors with more than 10 years of experience as specialists performed better than junior doctors. This was at odds with our hypothesis and also the result of the Dutch study that younger doctors are more accepting of EBM [4]. A possible reason may be that in our health care system, junior doctors have the most workload so they did not have enough time to notice or practise EBM. Studies have indicated that lack of time is an obstacle to the practice of EBM [1]. In terms of hospital type, only those in regional hospitals performed significantly better in model A, but there was no significant difference between the groups in model B. This also went against our hypothesis that doctors working in medical centres were supposed to have the best practice of EBM. Again, lack of time may be one of the major reasons. The doctors in medical centres are always busier than doctors at other types of hospitals. Meanwhile, too many subspecialties may be another barrier to the penetration and awareness of new evidence. Generally speaking, only 16–29% of all doctors changed prescriptions within the study period. The low percentage might be due to our exclusion of patients with poor compliance. Moreover, the average time to change prescription was between 6 and 9 months, slightly longer than the period of 5 months used by the PraCTice study in Scotland [21]. We also tried to find out whether patients living in counties with better information and better economies pushed doctors working in clinics to practise EBM or not. However, in our study, urbanization was not a significant factor and doctors working in relatively affluent areas did not actually practise better EBM. The reason may be that the threshold for patients to accept new EBM remains high. Other key factors may be a professional language barrier and lack of interest from the media.

Although this was a unique, large-scale empirical study to observe the practice of EBM, there were still some limitations. First, we had no information on doctors' subjective judgments from claim data. It was possible that a doctor changed their prescription of rosiglitazone due to other side effects, such as allergy, deterioration, congestive heart failure, or due to policy changes (e.g. global budget) which affect hospital management, and this could lead to overestimation. Second, we also excluded the events when the prescription change was made by a doctor other than the one whom the patient usually visited and this could lead to underestimation. Moreover, the Negelkerke R-square values of the logistic regressions in both models were low (0.062 in model A and 0.028 in model B). This could be compatible with the KAB model indicating that there are still many other barriers to practising EBM. This is a general limitation of applying claims data with limited information.

Conclusions

As time goes by, EBM can be expected to gain increasing emphasis worldwide. Our study, however, found that EBM was still not well practised among our doctors. The practice of new evidence among groups of doctors seemed different and may vary depending on their specialty or professional experience. The junior doctors and doctors working in medical centres seemed not to practise EBM well. The patients did not have enough influence to modify a doctor's prescription behaviour in clinics, despite the new evidence on rosiglitazone becoming available in 2008. A prominent time lag exists between the EBM emergence and EBM practice. Further education and encouragement to strengthen physicians' EBM practice remain urgently needed within the medical community.

Acknowledgements

This study is based in part on data from the National Health Insurance Research Database provided by the Bureau of NHI, Department of Health, and managed by the National Health Research Institute. The interpretation and conclusions contained herein do not represent those of the Bureau of NHI, Department of Health or National Health Research Institute. This study was partially supported by National Science Council of Taiwan grant NSC 99-2410-H-110-008-MY2 to YCLI.