The impact of changing provider remuneration on clinical activity and quality of care: Evaluation of a pilot NHS contract in Northern Ireland

OBJECTIVES
A pilot NHS dental contract was introduced in Northern Ireland between 2015 and 2016, which involved changing the method for paying general dental practitioners working in the NHS from fee-for-service (FFS) to capitation-based payments, providing an opportunity for a robust evaluation. We investigated the impact of a change in payment methods on clinical activity and the quality of care provided.


DESIGN
A difference-in-difference (DiD) evaluation was applied to clinical activity data from pilot NHS dental practices in Northern Ireland compared to matched control NHS practices and applied to a questionnaire survey of patient-rated outcomes of health outcomes and care quality. We estimated the impact on access to care, treatment activity levels, practice finances and patient-rated outcomes of care of a change from FFS to a capitation-based system for 1 year, as well as the impact of a reversion back to FFS at the end of the pilot period.


RESULTS
The monthly number of registered patients in the pilot practices increased more than the control practices during the capitation period, by 1.5 registrations per 1000 registered patients. The monthly reductions in the volumes of all treatments in the pilot practices during the capitation period were much larger than the control practices, with 175 fewer treatment items. All measures rapidly returned to baseline levels following reversion from capitation back to FFS. NHS income per month increased in pilot practices, by £5920 per month (calculated on FFS item cost basis) more than controls in the capitation period. The analysis of patient questionnaires suggest found that patients notice differences only in waiting times, skill-mix and number of radiographs, but not on other measures of healthcare process and quality.


CONCLUSION
General dental practitioners working in the NHS respond rapidly and consistently to changes in provider payment methods. A move from FFS to a capitation-based system had little impact on access to care, but did produce large reductions in clinical activity and patient charge income. Patients noticed little change in the service they received. This shows that changes in remuneration contracts have the potential to meet policy goals, such as meeting the expectations of patients within a predictable cost envelope. However, it is unlikely that all policy goals can be met simply by changing payment methods. Therefore, work is also needed to identify and evaluate interventions that can complement changes in remuneration to achieve desirable outcomes.

in the pilot practices during the capitation period were much larger than the control practices, with 175 fewer treatment items. All measures rapidly returned to baseline levels following reversion from capitation back to FFS. NHS income per month increased in pilot practices, by £5920 per month (calculated on FFS item cost basis) more than controls in the capitation period. The analysis of patient questionnaires suggest found that patients notice differences only in waiting times, skill-mix and number of radiographs, but not on other measures of healthcare process and quality.
Conclusion: General dental practitioners working in the NHS respond rapidly and consistently to changes in provider payment methods. A move from FFS to a capitation-based system had little impact on access to care, but did produce large reductions in clinical activity and patient charge income. Patients noticed little change in the service they received. This shows that changes in remuneration contracts have

| INTRODUC TI ON
Over the last 20 years, the National Health Service (NHS) dental services in the United Kingdom (UK) have faced significant criticism, largely about improving access to care, but also relating to their responsiveness to changing population needs over time, concerns about the quality of care provided, affordability of the service and persistent oral health inequalities. 1,2 Since the Steele review of 2009, policymakers across the UK have acknowledged the need to reform NHS dental contracts with providers to address these concerns. 3 The research literature suggests that NHS practices respond very quickly to changes made to the NHS dental contract, to ensure the viability of their practices. For example, changes to the NHS dental contract in England, in 2006, saw an immediate drop in the types of clinical activity that reduced profit margins for practices and an increase in clinical activity in areas where profit margins could be improved. 4,5 A 2013 systematic review on the effects of different methods of remuneration on the behaviour of primary care dentists concluded that financial incentives may influence the clinical activity provided, but a clear understanding of the relationship between a change in remuneration and the impact it has on activity and population health was lacking. 6 A 2011 systematic review also found there is insufficient evidence to determine the effect of financial incentives on the quality of health care provided. 7 In 2013, a change in the payment system for Primary Care Dentists (PCD) was considered by policymakers in Northern Ireland; one that meant GDPs would be paid based on the principles of capitation rather than the existing fee-for-service (FFS) system. The main reasons behind this initiative were to contain costs, promote prevention of disease, secure access to care and improve the quality of care provided to NHS dental patients across the province. The

Department of Health, Social Services and Public Safety in Northern
Ireland in conjunction with the Northern Ireland Health and Social Care Board (NIHSCB) made a commitment to pilot a new NHS dental contract between 2015 and 2016 and work collaboratively with academics to undertake a rigorous evaluation of the impact of the pilot. Dental practices in the pilot group were to switch from FFS to capitation-based payments (from August 2015) and then back to FFS after 12 months. The aim of this research was to evaluate the impact of a change in the system of provider payment on the productivity, quality of care and health outcomes of NHS dental services in Northern Ireland.

| ME THODS
A difference-in-difference (DiD) design was employed, which compares changes over time in the two groups (pilot and control NHS practices), with data from each group being recorded over two time periods. 8 Figure 1 is a diagrammatical representation of the design. Pilot practices were selected using a two-stage process overseen by the NIHSCB. An invitation to participate in the pilot was sent to all NHS practices in Northern Ireland. Practices that submitted an Expression of Interest were then reviewed by the NIHSCB panel, using criteria to ensure the final practices selected exhibited a range of characteristics other than provider payment mechanism that could influence the level of clinical activity (practice size, urban vs nonurban and extent of NHS commitment). The total number of practices selected was influenced by system affordability; the NIHSCB had a fixed budget for the pilot and had to ensure it could accommodate the possible fall in patient charge revenue (PCR) associated with any reduction in service volume.
PCR is the co-payment contribution made by NHS patients to their care and for fee-paying adults, and this equates to approximately 80% of the total NHS fee received by the dental practice. For exempt NHS patients (children, those on low income and pregnant and nursing mothers), the total NHS fee is paid by the NIHSCB.
The payment package for PCDs working under the pilot NHS contract was based on the total payments received under the previous year's FFS contract, and this total was divided into twelve equal payments for the pilot period. The number of practices that were included in the pilot NHS contract was fixed by the NIHSCB on the basis of potential loss in patient charge revenue (PCR), which the the potential to meet policy goals, such as meeting the expectations of patients within a predictable cost envelope. However, it is unlikely that all policy goals can be met simply by changing payment methods. Therefore, work is also needed to identify and evaluate interventions that can complement changes in remuneration to achieve desirable outcomes.

K E Y W O R D S
access, economics, program evaluation, quality of care, workforce Board under-wrote. This resulted in 11 practices being chosen to take part in the NHS pilot dental contract.
Control NHS practices were selected from all remaining practices in Northern Ireland using a two-stage process. Initially, stratified random sampling was used to identify potential control practices using the following strata: practice list size, proportion of children in the practices registered population, proportion of adult patients exempt from NHS patient charges in the practice and geographic location.
This initial process identified 45 potential control practices, 15 of which could not be used because of data inconsistencies, leaving 30 potential control practices. The final stage of selecting control practices involved matching the 11 pilot practices to control practices using a propensity score approach, which identified 18 matched control NHS practices (remained on the existing FFS system throughout the evaluation). Further details of this matching process are provided in the Appendix S1, as is a summary of the practice characteristics of the two groups at baseline including the number of registered patients and the types of patients (eg proportion patients exempt from fee-paying).
Data were collected longitudinally at a patient level in both pilot and control practices by the Business Services Organisation (BSO), which had not been possible in other NHS dental contract reform pilot studies at the time of the study. In addition to using a DiD design with the pilot and control NHS practices, we also analysed activ- which is when a patient is no longer registered at the practice due to them not receiving any treatment from a dentist over a 2-year period. Treatment activity levels captured the relative complexity of the different treatments provided under the NHS over the pilot period, for example direct restorations (fillings) and extractions, preventive care, for example examinations and fissure sealants. Finance was measured by assessing any changes to the total NHS income and PCR. These outcome measures were expressed relative to the practice list size (per 1000 patient registrations), to give each practice an equal weight in determining the comparison of group averages (such as changes between study periods within and between control and pilot practices). This was deemed appropriate, given the different sizes of the practices in both the pilot and control groups.
Analyses were performed at the practice level with clustered standard errors to adjust for estimates of the correlation over time.
We applied Sidak and Bonferroni corrections to the DiD estimates, to counteract the problem of multiple comparison. 9 The assumptions underlying a DiD approach, such as comparability of groups over time, should always be carefully scrutinized and can lead to erroneous conclusions if care is not exercised. To do this, a robustness analysis was performed using an interrupted time series (ITS) F I G U R E 1 The DiD design used in the study approach of selected activity outcomes, by fitting a linear model for each outcome over time grouped by practice. 10 The ITS analyses and the graphical depiction of the DiD results are presented in full in the Appendix S1.
The research team used a patient questionnaire to capture any changes in patient-reported oral health knowledge, attitudes and behaviour as well as patient-rated oral health outcomes and quality of care. It is presented in the Appendix S1. The questionnaire was iteratively developed, with public and patient involvement (PPI) in two focus groups (n = 7 and n = 8). Questionnaires (n = 3000) were distributed to eligible (registered) NHS patients in each study phase

| RE SULTS
The results for each domain of outcome measure (access to care, treatment activity levels and finance) are expressed per month per 1000 registered patients. Figures are found in the Appendix S1 and additional tables that summarize the outcomes prior to the DiD analysis. Table 1 shows the access outcomes results. The difference between pilot and control NHS practices in the number of registered patients increased during the capitation period, but by only 1.5 registrations per month, and reduced in the reversion period by 6.8 new registrations per month. However, the composition of the NHS practice lists had changed; the difference between pilot and control practices in the number of new patients and the number of patients leaving the practice list increased in reversion period. Table 2 shows treatment activity levels outcome results. All measures of clinical activity, bar one (mean number of treatments with a cost of £280 or more, eg applying crowns, dentures or bridges), saw a larger reduction in the pilot group of practices than controls as the practices moved from being paid FFS to capitation. However, there was no evidence of a long-term effect from the pilot, because the reduction in all of these treatments in the capitation period was offset by an increase in activity during the reversion period. Table 3 shows financial outcome results. The difference between pilot and control practices in NHS income per month changed between study periods calculated on FFS basis, which is not their capitation practice income but their income from the NHS if their activities observed under capitation had been paid on a FFS basis. The difference increased by £5920 per month in the capitation period compared to baseline FFS and decreased by £5248 in the reversion period. This was caused by a reduction of activity in the pilot practices in the capitation period from the level in baseline FFS. There was no evidence of a long-term effect from the pilot, because there was no difference between pilot and control practices in NHS income per month in the FFS reversion period than at baseline.
No differences were seen in the results between 'matched' practices (with propensity scores) and 'unmatched' practices. This suggests the findings were not influenced by the choice of the matching process. The ITS analyses were consistent with the DiD results; the estimated effects are in the same direction as those found under DiD (see Appendix S1 for details). The same direction of changes between study periods to the above analysis was seen at the ADs and PPs level (see Appendix S1 for details).
In total, 1215 patients in pilot practices responded (347 in the baseline period, 316 in the intervention period and 552 in the The coefficient is mean difference in outcome between groups (pilot and control practices) in the capitation phase compared to FFS.  (Table S3)  The coefficient is mean difference in outcome between groups (pilot and control practices) in the capitation phase compared to FFS.

DiD coefficient b P-value
c The coefficient is mean difference in outcome between groups (pilot and control practices) in the reversion FFS phase compared to capitation. d The coefficient is mean difference in outcome between groups (pilot and control practices) in the reversion FFS phase compared to Baseline FFS. The coefficient is mean difference in outcome between groups (pilot and control practices) in the capitation phase compared to FFS.

TA B L E 3 Overview of the DiD financial outcome results
c The coefficient is mean difference in outcome between groups (pilot and control practices) in the reversion FFS phase compared to capitation. d The coefficient is mean difference in outcome between groups (pilot and control practices) in the reversion FFS phase compared to Baseline FFS.
due to the remuneration incentives in the pilot contract. (e) There was some self-selection in the sample due to the requirement of practices to submit an expression of interest, although this is true of any study requiring informed consent to participate. Ideally, the project would have randomized practices into intervention and control groups, but this was not possible as the DHSSPS decided it would be barrier to recruitment.
A strength of this study is that it was pragmatic in that a 'real world' service was being provided to patients as the evaluation progressed, so the outcomes observed could be expected if the intervention was rolled out nationally. This study overcomes many of the serious methodological limitations that have restricted the completeness and generalizability of the evidence in order to understand how changing remuneration systems affected patient care.
These specific weaknesses were outlined in a review of reviews primarily focused on primary medical care that included 4 reviews and collectively included 32 studies. 14 First, the authors of the review could find no studies evaluating patient outcomes and none reporting on any adverse or unexpected outcomes. This study reported patient-rated outcome measures and patient-rated experience measures, although independent clinically assessed outcomes were not measured because, in the case of dental caries, this would have required a much longer follow up time and explicit consent from patients of the practices involved. The authors could also find no reviews of studies that systematically examined variable doses of financial intervention. This study addresses this weakness by comparing FFS with capitation, two interventions at opposite ends of the spectrum of possible remuneration systems.
The findings from this study reinforce the evidence from the literature that GDPs respond markedly and quickly to changes in how they are remunerated in terms of both the volume of care provided and how they provide care. 4-6 It is not surprising that a change in patient perceived quality was not evident (except for three questionnaire items), given that the 2006 change in contract in England also resulted in large reductions in specific treatments (eg root fillings, crowns and bridgework) and an increase in the number of extractions, without any perceptible increase in service users' concerns. 4,5 The most robust findings we report concern the impact of the intervention on activity of dentists. Whilst policymakers have a keen interest on securing access and controlling costs, the findings we report on health outcomes are more limited.
Care must be exercised in concluding that a reduction in treat-

| CON CLUS ION
Overall, the move to a capitation-based payment system from FFS suppressed clinical activity, including prevention. Equally, PCDs returning to an FFS remuneration system from capitation resumed activity levels to that seen in the baseline period. It is likely that a permanent change to capitation would lead to immediate changes like those found in the pilot, but that in the long term that behaviour in terms of access and activity would fall somewhere between FFS and capitation levels recorded in the pilot in response to pressures resulting from patients' ongoing needs and expectations. However, changing the way dentists are paid on its own is unlikely to achieve all policy goals and therefore research is required to develop and test supplementary interventions that can work in a complementary way alongside a remuneration system to achieve desired policy outcomes.

ACK N OWLED G EM ENTS
The authors thank all of the key stakeholders in the DHSSPS, in-