NHS arm's length bodies and regulatory networks in England: quantitative analysis

Authors


  • Public Policy PhD candidate.

SUMMARY

Health regulation is an area of English public policy that involves a plethora of different bodies operating independently of one another with the purported aim of monitoring standards, ensuring minimum standards are met and providing assurance for the public and government. The purpose of this research is to ascertain as well as possible whether regulation generally is having a positive effect on service provision as well as what the relationship between the data collected by these bodies is. The rationale for testing whether the effect of regulation is positive is obvious; it costs money, time and effort, so does it work? The rationale for testing the relationships is that providers who score well on quality measures should also be experiencing fewer deaths compared with what would be expected. They should also be the providers who are performing well in terms of finances and governance. If there is no relationship between these data sets, then surely something is going wrong; that is, does the various regulatory monitoring and scrutiny actually measure what it sets out to measure? Copyright © 2013 John Wiley & Sons, Ltd.

AIMS/RESEARCH QUESTIONS

  1. Has the overall average of hospital standardised mortality ratios (HSMRs) increased or decreased during the period 2005–2012?
  2. Are providers with poor HSMRs becoming worse?
  3. Is there a statistically significant correlation between HSMRs and Care Quality Commission (CQC) data on quality?
  4. Is there a significant correlation between HSMRs and Monitor data on finances and governance?
  5. Is there a significant correlation between CQC data on quality and Monitor data on finances and governance?
  6. To ascertain using linear regression whether it is possible to predict the values of the CQC and Monitor data sets based on the HSMR data set.

RESEARCH METHODS

The data sets explained

The data in this analysis are secondary and are extracted from a range of sources from the Care Quality Commission (CQC), Monitor and Dr Foster Intelligence. The CQC data come in the form of whether or not a provider is currently compliant with the five main ‘key standards’ and surveys of outpatients, inpatients and accident and emergency departments. The five key standards that the CQC measures for each provider are

  • ‘Standards of treating people with respect and involving them in their care
  • Standards of providing care, treatment & support which meets people's needs
  • Standards of caring for people safely & protecting them from harm
  • Standards of staffing
  • Standards of management’ (CQC, 2013a, 2013b, 2013c, 2013d)

For each provider, it is assessed whether they are meeting all the five key standards when last checked. It also will indicate whether at least one standard was not being met requiring improvements or whether at least one standard is not being met and enforcement action is being taken. Each provider in the sample has been given a score of 0–5 depending on how many of the five key standards are being met upon last inspection.

The CQC surveys on outpatients, inpatients and accident and emergency departments are based on a core of evaluative questions assessing the patients' experience at their respective departments. These questions are ‘core’ in the sense that they are those questions where results are available from every trust. The questions are scored on a scale from 0 to 10 where 0 represents a considerable scope for improvement and 10 refers to the most positive patient experience. These results are also based on standardised data because it is known that views can relate to certain demographic characteristics, and in these surveys, the data have been standardised by age, gender and method of admission (emergency or elective). In the sample, each provider has been given an overall score out of 10 for each survey, which is based on the mean for all the other questions in the survey.

The Monitor data is assigned a risk rating for finance and governance taken from their foundation trust directory. The risk rating for finance ranges from 1 to 5 where 1 represents the highest risk and 5 represents the lowest risk. This rating is derived from four financial criteria: achievement of plan, underlying performance, financial efficiency and liquidity.

The governance rating comes in four categories: red (likely or actual significant breach of terms of authorisation), red/amber (breach of terms of authorisation), amber/green (limited concerns surrounding terms of authorisation) and green (no material concerns). This overall rating is derived from performance against national measures, third parties, mandatory services, board statement factors and other factors.

The Dr Foster data is taken from their annual hospital guides based on hospital standardised mortality ratios (HSMRs). The HSMRs represent the ‘ratio of the observed number of in-hospital deaths with an HSMR diagnosis to the expected number of deaths, multiplied by 100’. These data are based on patients who were involved with an emergency spell with a primary dominant diagnosis of any of 56 Clinical Classification Software (CCS) groups. The HSMR basket of CCS groups accounts for approximately 80% of all in-hospital deaths in England. It must be made clear that mean HSMRs are not necessarily 100 each year. This is because they are based on expected number of deaths that are derived from logistic regression, adjusting for factors to indirectly standardise for differences in case mix. The way that these expected numbers of deaths are calculated is extremely complicated (please refer to Conclusions section for a fuller explanation).

Data were obtained from Dr Foster Intelligence on HSMRs for all the available years since data collection began. In terms of actual data that are available, this means the results are based on the years 2005, 2007, 2009, 2010, 2011 and 2012 (Dr Foster, 2013). In 2006 and 2008, there were no data available from Dr Foster.

The methods

The aim of this research is to use statistical techniques to answer the research questions on the basis of these secondary data. This involves deployment of Excel to collate and synchronise these data sets into one manageable and comparable data set. This information has then been processed into spss (IBM United Kingdom Limited, Hampshire, UK) where relationships and trends can be teased out.

FINDINGS

RO1: has the overall standard of hospital standardised mortality ratios increased or decreased during the period 2005–2012?

This analysis is based on all the available HSMR scores for 172 providers from 2005 to 2012. For each year with an available data set, the mean was worked out, and these means were compared over time. This is displayed in graph 1.The data for the unavailable years have been extrapolated on the basis of the other data.

Graph 1 illustrates that HSMRs have gone up over the period but not constantly. An increase in the HSMR over time means that the observed number of deaths versus the expected number of deaths has actually worsened over this period. From 2005 to 2012, there has been a 6% increase in mean HSMRs for all providers. The HSMR does fluctuate in 2010 with the score lowering; this is the only year when this occurs.

RO2: are providers with poor hospital standardised mortality ratios becoming worse?

To operationalise whether a provider was ‘good’ or ‘bad’, their mean HSMR in the period 2005–2012 was worked out, and then, the sample was ranked and split into two groups. ‘Bad’ trusts represent the highest 50% of the sample, and ‘good’ trusts represent the bottom 50%. Graphs 2 and 3 illustrate the bad and good trusts HSMRs over the period.

It is clear from these graphs that the increase in HSMRs in bad trusts is slightly more pronounced than in the good ones. In percentage terms, the increase in bad trusts HSMRs was 6.1%; the increase in HSMRs in good trusts was 5.7%.

RO3: is there a statistically significant correlation between hospital standardised mortality ratios and Care Quality Commission data on quality?

To correlate the HSMRs and CQC data sets, it first has to be determined whether or not the data are normally distributed. This was initially carried out using spss to create a histogram of the frequencies (with a line of normal distribution). These visual representations of the frequencies compared with a normal distribution show that some of the data look normally distributed and some do not. Because of the inconclusive nature of the histograms (as is always the case), they have been omitted from this paper. The HSMR and outpatient histograms show the data roughly fits the normally distributed line. The CQC five key standards, accident and emergency and inpatient survey histograms suggest that perhaps these data are not normal as they do not fit the projected normal frequency lines.

To test normality further (and more effectively), skewness and kurtosis are calculated for the variables' frequency distributions; this is shown in output 1. Skewness represents deviation from symmetry in a data set. Kurtosis is a measure of whether the data are peaked or flat relative to normal distribution.

The thresholds of normality for the purposes of this research for skewness and kurtosis are from −2 to 2. By this definition, the Dr Foster HSMR data fit the criteria for normality, as do the CQC outpatient survey data. This information supports the visual evidence on normality. This means that a standard Pearson's correlation can be measured to determine the relationship between the two variables. The skewness and kurtosis for the CQC five key standards, accident and emergency and inpatient surveys however do not fit the criteria for normally distributed data, which means that instead, a Spearman's rho correlation coefficient is necessary. Output 2 illustrates the Pearson's correlation between the HSMRs and the CQC outpatient survey. This indicates a very small positive correlation, but no correlation at even p > 0.05 (two tailed). It is therefore likely that there is no significant relationship between HSMRs and the CQC outpatient survey data sets.

Outputs 3–5 illustrate the Spearman's rho correlation coefficient between HSMRs and the CQC surveys for which the data are not normally distributed. The HSMR and accident and emergency data (output 3) show a very small positive correlation. There is no significant correlation at p > 0.05 (two tailed). It is therefore almost certain that there is no relationship between HSMRs and CQC accident and emergency survey data sets. Output 4 shows that there is a very small negative correlation between the HSMR and inpatient data, but there is no significant correlation at p > 0.05 (two tailed). It is again extremely likely that there is no correlation between the HSMR and CQC inpatient data sets. Output 5 displays no correlation whatsoever at any level of significance between the HSMR and the five key standard data sets. This clearly demonstrates that there is no relationship between HSMRs and the CQC five key standards.

RO4: is there a significant correlation between hospital standardised mortality ratios and Monitor data on finances and governance?

To test whether the Monitor data sets are normally distributed, frequency histograms with normal curves on (for comparison) have been created. Again, these are omitted from the research paper because of their inconclusive nature. The finance histogram illustrates that the frequencies are slightly skewed with more falling on the right-hand side, tailed off to the left. It is unclear whether this skewness is within the normal range. The governance histogram shows edge peak distribution due to the high numbers of ones and fours on the governance scores. On the basis of the visual representation, it appears that the governance data set is unlikely to be normally distributed. The frequency histograms show deviations from the normal line, but these could still be within the realms of normality. The skewness and kurtosis of the data are therefore worked out; this is displayed in output 6.

These figures show that the data are normally distributed, as the skewness and kurtosis are both within the threshold of −2 to 2. This means that to measure the correlation, a Pearson's correlation coefficient test is deployed (outputs 7 and 8). Output 7 indicated that there is no significant correlation at p > 0.05 (two tailed). This suggests that there is no relationship between HSMR and Monitor finance data sets. Output 8 indicates that there is a very small negative correlation but not a significant correlation at p > 0.05. This also suggests that there is no relationship between the HSMRs and Monitor governance data sets.

RO5: is there a significant correlation between Care Quality Commission data on quality and Monitor data on finances and governance?

It has already been established that the CQC data for outpatients and the Monitor data for governance and finances are normally distributed. It has also been established that the CQC data for the five key standards, inpatient surveys and accident and emergency surveys are not normally distributed. On this basis, the outpatient survey and Monitor data sets have been tested using Pearson's correlation coefficient (output 9). Similarly, the five key standards, inpatient surveys and accident and emergency surveys have been tested against the Monitor data by using Spearman's rho correlation coefficient (output 10).

Output 9 illustrates that there is no correlation between either Monitor finance or governance scores and the CQC outpatient surveys at even p > 0.05 (two tailed). This suggests that there is no relationship between outpatient and finance data, or outpatient and governance data.

Output 10 indicates that the Monitor data on finance and governance are correlated at a significant level. The correlation coefficient between finances and governance is positive, 0.617 at a significance at p > 0.01 (two tailed). This data may suggest that attempts by Monitor to measure these variables are more of a box-ticking process, because of the significant correlation. It may be the data have little relevance when it comes to actual quality in terms of service provision and safety. There are positive correlations between the Monitor and CQC data, but there is however no correlation of significance at p > 0.05 (two tailed) between the Monitor financial data and any of the CQC data sets (which are not normally distributed). There is also no correlation of significance at p > 0.05 (two tailed) between the Monitor data on governance and any of the CQC data (which are not normally distributed).

RO6: to ascertain using linear regression whether it is possible to predict the values of the Care Quality Commission and Monitor data sets based on the hospital standardised mortality ratio data set.

By using linear regression, it is possible to attempt to predict one variable from another; the dependent variable (outcome) is predicted using a model based on the independent variable(s) (predictors). So, measuring the Pearson correlation coefficients previously allowed to measure the relationships between variables, but regression allows this to be taken a step further, and say, for example, what is the impact of increasing a CQC standardised score by one on the HSMR for a provider? The assumption being that if the measures accurately portray quality, then a higher CQC score should result in a lower HSMR score. If the CQC rate a provider as safer on a particular area, it is logical to assume fewer patients in this area will die.

Because linear regression is a parametric test, it requires data that are normally distributed. Consequently, not all the CQC data sets are suitable for this analysis. The CQC outpatient surveys, however, as previously discussed, do fit the criteria of normality. The main assumption associated with linear regression is that of heteroskedasticity. This means, in general terms, differences in variances. This refers to the observations of the variance of the residuals in the analysis not being consistent or constant across the predictor variable, so the predictive power of the regression analysis should be roughly equal from low levels of the X value to high levels of the X value. This is tested using a histogram of the residuals of the dependent variable and is also tested using a normal probability plot. Also, the data is tested for heteroskedasticity. As earlier described, there is a Pearson correlation coefficient of 0.166 between the Dr Foster HSMR and CQC outpatient data sets. This is a very small positive correlation, which is not significant at even p > 0.05. Output 11 summarises the linear regression model for HSMRs and outpatient surveys.

The R score replicates the Pearson's correlation coefficient. The R-square score shows that 2.8% of the variability in HSMR can be explained by the scores on the outpatient surveys. So, this is not a meaningful predictor, and it accounts for a miniscule amount within this. Taking sample size into account, spss has adjusted the R squared to 0.020, which means that more realistically, only 2.0% of the variance can be explained by the outpatient surveys, a relatively trivial difference, but this negates the predictive nature of the relationship even further. Output 12 is the analysis of variance table related to the linear regression. This essentially tells us whether the correlation of 0.166 is significant. There is an F-value of 3.775 and a significance of 0.054.

Output 13 illustrates the coefficients with 95% confidence intervals. The standardised beta score is the same as the correlation coefficient, as is the statistical significance based on the t-distribution. So, it is known that the correlation is not statistically significant at p > 0.5. We now know that because unstandardised beta is 7.334 if you increase CQC outpatient survey score by one, then we expect a 7.334 increase in HSMR for the provider. The confidence intervals of 95% for beta illustrate that there can be no real certainty around this because both the constant (intercept) and CQC outpatient scores range from the negative (−29.5 and −0.132) to the positive (101.2 and 14.8). So, actually, if 95% confidence intervals are applied, the 7.334 increase in HSMR per one unit increase in outpatient survey is between −0.32 and 14.8.

Output 14 is a table for residual statistics, which are used to check heteroskedasticity. As the residual values for the mean are zero, the histogram looks normally distributed, and the normal P–P plot (output 15) looks to be relatively straddling the regression line, heteroskedasticity is present. Furthermore, the scatterplot (output 16) between regression standardised residual and regression standardised predicted values is like a bird's nest; that is, there is no associated pattern with the error of the variables, which confirms the heteroskedasticity of the data.

CONCLUSIONS

RO1: has the overall average of hospital standardised mortality ratios increased or decreased during the period 2005–2012?

From 2005–2012, there has been a 6% increase in mean HSMRs for all providers.

What exactly does this mean?

It means that if we are to believe that the Dr Foster HSMRs are true representations of the ratio between how many people die compared with how many people should die at hospitals, then more people are dying now in hospitals compared with how many should, compared with 2005.

It also means that we may doubt the data coding techniques of Dr Foster. Mohammed et al. (2009) published an influential paper titled ‘Evidence of methodological bias in HSMRs: retrospective database study of English hospitals’. This paper is heavily critical of HSMRs as an accurate measure, because of the coding and risk predictions that make them up. This article goes on to conclude that ‘variations in HSMRs from Dr Foster Unit reflecting differences in quality of care are less than credible’ (2009: 228; 780). Thus, it is possible that this increase in HSMRs over time does not truly represent a worsening of hospital quality.

In appendix 9 of the Francis Enquiry (vol. 1), there is a detailed review of mortality statistics produced by two Harvard academics. This also points out that there is no fool-proof method for quantifying health care quality. It is stated that

We accept that there is no single, perfect mechanism for assessing health care quality. We also agree that every statistical quality monitoring algorithm, including Dr Foster, should be critically examined by experts to determine its validity. (2010, p.440)

There is, however, significant evidence in this paper, which means that Dr Foster HSMRs should not be ignored or written off completely because of the methodological flaws. The conclusion on HSMRs is as follows:

The HSMR is a summary figure, designed to give an overview of mortality within a trust, and we accept it will hide a considerable number of differences in the risk profiles across different factors in the model, but we do not see why this should decrease the value of the HSMR as a summary figure used in conjunction with other measures. (2010, p.442)

Specifically referring to the Mohammed et al. (2009) research, it is stated that

We are disturbed by the final sentence summarising the author's conclusions: “In other words, quality of care should remain innocent until proven guilty”. This is a hospital-centric admonition, but certainly not one that would be acceptable to most patients or to the regulators entrusted with ensuring the quality of their care. (2010, p.446)

Despite the disclaimers by Mohammed et al. (which were in effect discussed by Francis), it still raises the question: Could HSMRs even at the time of the Mid Staffordshire National Health Service (NHS) Foundation Trust debacle (2005–2009) not have shown trends over time, or hospital to hospital, as cause for worry? This could perhaps be considered as a middle way between Mohammed's purism and the view that Mid Staffordshire did indeed have 1200 deaths too many. It is possible to concede that HSMRs do not show the whole picture, but they do certainly show an important part. The Francis report into the failures at Mid Staffordshire has identified that there were three types of data at the time that should have signalled the issues. This could not have helped matters, in terms of Mid Staffordshire being authorised as a foundation trust (which are supposed to be models of best practice). The three warning signs should have come from HSMRs, CQC (or Healthcare Commission previously) data and Monitor data. The balance of hard verses soft data has evidently not been right, and the mechanisms for triangulating the three types of data need to be enhanced, if these failures are to be avoided.

Clearly, Mid Staffordshire has been an example where HSMRs increasing was not noticed quickly enough and resulted in far too many people dying. This deficiency still remains a wide concern, as there are 14 trusts that have been identified by the Department of Health as having over than expected death rates, and one of the main possible causes is medical staffing levels.

The NHS Commissioning Board has identified the trusts, following the publication of the Francis report, and they are as follows: North Cumbria University Hospitals, United Lincolnshire Hospitals, George Eliot Hospital, Buckinghamshire Healthcare, Northern Lincolnshire and Goole Hospitals, The Dudley Group of Hospitals, Sherwood Forest Hospitals, Medway, Burton Hospitals, Colchester Hospital, Tameside Hospital, Blackpool Teaching Hospitals, Basildon and Thurrock University Hospitals and East Lancashire Hospitals (2013, p.1).

Bearing the validity of the HSMRs in mind, the fact that HSMRs have deteriorated each year apart from 2010 is worrying. A 6% increase in this period (2005–2012) is troublesome for policymakers even when the potential failures of quantifying health quality are taken into consideration.

RO2: are providers with poor hospital standardised mortality ratios becoming worse?

Not significantly, ‘bad’ providers are becoming worse at a slightly greater rate than ‘good’ providers, but there is not really much difference. More concerning than this is the fact that they have both gone up and not down.

RO3: is there a statistically significant correlation between hospital standardised mortality ratios and Care Quality Commission data on quality?

There is no significant correlation between HSMRs and any of the four CQC quality related data sets. This means that one of the two variables is not accurately measuring what they are supposed to and portraying quality in a useful way.

This is based on the rationale that if safety and quality outcomes in hospitals are measured accurately, then this should correlate with the amount of people dying there compared with how many should die (if measured accurately). On the basis of the previous discussion on HSMRs, it is assumed that these do represent quality to some extent at least. If we accept that HSMRs at least ‘give an overview of mortality’, then the CQC data may not properly represent quality in their measurements.

The five key standards include safety, care, respect, staffing and management. It must be the case that these sorts of measures, if accurate, would correlate with HSMRs. They do not however at any level of significance. The CQC accident and emergency, and outpatient and inpatient surveys all do not correlate with HSMRs. These are national surveys, which are undertaken periodically over hundreds of locations. It again seems logical to assume that if these measurements accurately portray quality, they would correlate with the HSMR (even with the HSMR critique considered). If these things are not obtaining the information they are supposed to, information that is relevant on quality and relevant for the regulation of hospitals for policy makers, then different techniques for data collection in this area should be deployed. It is possible that the CQC relies too heavily on ‘soft’ data on quality such as surveys, observations and self-reported information. HSMRs are based on actual numbers of people dying; this can be considered ‘hard’ data on quality (criticisms on the expected death algorithm considered). If the soft data sets collected on quality do not accurately portray quality, this renders the whole process pointless. If quality is not measured with hard data, it seems inevitable that providers in England will deteriorate without the CQC noticing.

RO4: is there a significant correlation between hospital standardised mortality ratios and Monitor data on finances and governance?

All the logic seems to point to there being an inextricable link between finances and governance on the one hand and quality on the other. But on the basis of the data sets, it seems that neither governance nor financial scores positively or negatively correlate with HSMRs. In this case, the evidence on finances and governance is less likely to be unreliable. Many of the elements of the Monitor weightings on risk are quantifiable variables, which are not open to interpretation (as with some of the CQC and Dr Foster data). Returns surpluses and liquidity ratios, for example, are unlikely to be false. The lack of correlation between HSMRs, finances and governance shows that it is quite possible to be performing well financially and governance wise but be performing poorly HSMR wise. The converse is equally true.

It is perhaps disappointing that there is no correlation. If organisations that were performing well financially and are effectively governed performed favourably on their HSMR scores, then this would add clout to the argument that the work of Monitor was driving up standards. There is, however, another argument that the Monitor assessing regime is merely a box-ticking process. If we are to doubt the reliability of the Monitor data at quantifying finance and governance robustness, then perhaps, there is an inseparable link between quality and finance, as well as quality and governance, but this is not clear from the data because the nature of Monitor regulation is bureaucratic and does not measure what it is supposed to.

RO5: is there a significant correlation between Care Quality Commission data on quality and Monitor data on finances and governance?

There is not, as with the Monitor data and the HSMRs. The reasons discussed previously explain why the CQC data may not accurately quantify quality, which may explain the lack of correlation. But equally, it may be that quality does not actually correlate with either finances or governance.

RO6: to ascertain using linear regression whether it is possible to predict the values of the Care Quality Commission and Monitor data sets based on the hospital standardised mortality ratio data set.

The linear regression between the CQC outpatient surveys and the Dr Foster HSMRs was revealing in several ways. The model predicts that for an increase in the CQC outpatient survey of one unit, there will be an increase in HSMR by 7.3. This is very troublesome because it is the opposite of what would be anticipated to be the case.

It is logical to assume that a better score for quality based on outpatients' surveys would be a predictor for the number of people dying and that as the outpatient scores increase (get better), the amount of people dying will decrease (improve). The fact that the converse is true suggests that there are serious limiting factors with the methodologies deployed by the CQC in their outpatient surveys, and there may well be issues with the HSMRs also (as has been discussed already). The 95% confidence intervals do illustrate that the figure of 7.3 may not be entirely accurate, but with a range of −0.132 to 14.8, it is appropriate to suggest that the opposite of what should be occurring is almost certainly occurring: As scores for outpatients improve, scores for death rates worsen, seriously limiting the validity and usefulness of the data sets involved.

What does it all mean for the arm's length bodies and their effectiveness at positively influencing health policy?

The fact is there are reasons to doubt all attempts of quantifying quality in any capacity. There are important lessons to be learnt for the NHS in England as well as internationally. Many countries around the world deploy regulation of a similar nature in some capacity at least, and these countries should take note of the trends of the NHS in England. There are certainly many deficiencies associated with the current regulatory regime in English health policy, which have been confirmed by the lack of the data sets to correlate with one another. If arm's length bodies are not measuring what they are supposed to, the service is likely to suffer. There is an overreliance on soft rather than hard data, which limits the effectiveness of regulation. There is also too much box ticking, which has no value to health policy, and actually creates additional bureaucracy.

A tempting conclusion may be that the composite regime of health regulation in England is having a neutral effect on quality at best. There is approximately no evidence of any link between providers graded highly by regulators and the normalised deaths among their patients. Indeed, it could be argued that the cost of the regulation, both in the regulators and servicing their requirements in the providers, uses resources that could be better allocated elsewhere. But this may be too superficial a view. The existence of the regulatory bodies and the public visibility of their findings may be exerting an overall upwards pressure on service quality, without which the HSMR figures would have worsened even faster. This possibility cannot be assessed through the kind of analysis contained in this paper. The jury is still out.

EVALUATION

One of the obvious criticisms of this research is that the data are not my own. This is something that comes with the nature of these types of research questions, in that I am attempting to assess the relationships between data sets that are not my own. This does bring the question of reliability of these data sets, but that is what I am seeking to investigate.

A further criticism of this research is that there are some missing data in the data sets; there is, however always a reason for this. For instance, some of the Dr Foster historical HSMR data are missing in some of the early years, when their remit was less than it now is. Furthermore, the Monitor data sets are only for foundation trusts, as that is who they regulate, and therefore, it has been compared with the corresponding data for foundation trusts, not all providers. There is also the issue of trusts merging into one another or closing down completely, which means there is no information for all years for all providers. That being said, these missing values have been taken into account in this quantitative analysis, and the number of missing values is minimal.

ACKNOWLEDGEMENTS

This research is a secondary research that is part of my PhD studentship funded by the Economic and Social Research Council and Keele University.

The authors have no competing interests.

APPENDICES

GRAPHS AND SPSS OUTPUTS

image
image
image

Output 1: Skewness and kurtosis for hospital standardised mortality ratios and Care Quality Commission data

 DR FThe CQC 5 key standardsCQC accident and emergency overallCQC outpatient overallCQC inpatient overall
  1. CQC, Care Quality Commission.

NValid145145139140140
Missing2828343333
Mean99.894.5046.7018.7515.819
Skewness−0.694−2.400−7.190−0.427−7.766
Standard error of skewness0.2010.2010.2060.2050.205
Kurtosis0.4346.21071.248−0.28379.235
Standard error of kurtosis0.4000.4000.4080.4070.407

Output 2: Hospital standardised mortality ratio and Care Quality Commission outpatient Pearson correlation coefficient

 DR FCQC outpatient overall
  1. CQC, Care Quality Commission.

DR FPearson correlation10.166
Significance (two tailed) 0.054
N145135
CQC outpatient overallPearson correlation0.1661
Significance (two tailed)0.054 
N135140

Output 3: Spearman's rho correlation for hospital standardised mortality ratios and accident and emergency surveys

 DR FCQC accident and emergency overall
  1. CQC, Care Quality Commission.

Spearman's rhoDR FCorrelation coefficient1.0000.124
Significance (two tailed)0.0000.154
N145134
CQC accident and emergency overallCorrelation coefficient0.1241.000
Significance (two tailed)0.1540.000
N134139

Output 4: Spearman's rho correlation for hospital standardised mortality ratios and inpatient surveys

 DR FCQC inpatient overall
  1. CQC, Care Quality Commission.

Spearman's rhoDR FCorrelation coefficient1.000−0.073
Significance (two tailed)0.0000.402
N145135
CQC inpatient overallCorrelation coefficient−0.0731.000
Significance (two tailed)0.4020.000
N135140

Output 5: Spearman's rho correlation for hospital standardised mortality ratios and the five key standards

 DR FThe CQC five key standards
  1. CQC, Care Quality Commission.

Spearman's rhoDR FCorrelation coefficient1.0000.006
Significance (two tailed)0.0000.947
N145138
The CQC five key standardsCorrelation coefficient0.0061.000
Significance (two tailed)0.9470.000
N138145

Output 6: Skewness and kurtosis for Monitor finance and governance scores

 Monitor finance scoresMonitor governance scores
NValid8686
Missing8787
Mean3.022.95
Skewness−1.002–0.667
Standard error of skewness0.2600.260
Kurtosis1.472−1.149
Standard error of kurtosis0.5140.514

Output 7: Hospital standardised mortality ratio and finance Pearson correlation

 DR FMonitor finance scores
DR FPearson correlation10.016
Significance (two tailed) 0.892
N14576
Monitor finance scoresPearson correlation0.0161
Significance (two tailed)0.892 
N7686

Output 8: Hospital standardised mortality ratio and governance correlation

 DR FMonitor governance scores
DR FPearson correlation1−0.034
Significance (two tailed) 0.769
N14576
Monitor governance scoresPearson correlation−0.0341
Significance (two tailed)0.769 
N7686

Output 9: Outpatient and Monitor Pearson's correlation coefficient

 CQC outpatient overallMonitor finance scoresMonitor governance scores
  • CQC, Care Quality Commission.

  • **

    Correlation is significant at the 0.01 level (two tailed).

CQC outpatient overallPearson correlation1−0.046−0.050
Significance (two tailed) 0.6990.673
N1407474
Monitor finance scoresPearson correlation−0.04610.617**
Significance (two tailed)0.699 0.000
N748686
Monitor governance scoresPearson correlation−0.0500.617**1
Significance (two tailed)0.6730.000 
N748686

Output 10: Spearman's rho correlation coefficients for Monitor data and Care Quality Commission data sets that are not normally distributed Correlations

   The CQC five key standardsMonitor finance scoresMonitor governance scoresCQC inpatient overallCQC a and e overall
  • CQC, Care Quality Commission.

  • **

    Correlation is significant at the 0.01 level (two tailed).

Spearman's rhoThe CQC five key standardsCorrelation coefficient1.000−0.065−0.1690.0560.063
Significance (two tailed)0.0000.5790.1440.5090.461
N1457676140139
Monitor finance scoresCorrelation coefficient−0.0651.0000.540**0.045−0.028
Significance (two tailed)0.5790.0000.0000.7060.814
N7686867474
Monitor governance scoresCorrelation coefficient−0.1690.540**1.0000.066−0.180
Significance (two tailed)0.1440.000.0.5750.125
N7686867474
CQC inpatient overallCorrelation coefficient0.0560.0450.0661.0000.448**
Significance (two tailed)0.5090.7060.5750.0000.000
N1407474140139
 CQC a and e overallCorrelation coefficient0.063−0.028−0.1800.448**1.000
Significance (two tailed)0.4610.8140.1250.0000.000
N1397474139139

Output 11

Model summaryb

ModelRR squareAdjusted R squareStandard error of the estimate
  1. a

    Predictors: (constant), Care Quality Commission outpatient overall.

  2. b

    Dependent variable: DR F.

10.166a0.0280.0209.621

Output 12

Analysis of variancea

ModelSum of squaresdfMean squareFSignificance
  1. a

    Dependent variable: DR F.

  2. b

    Predictors: (constant), Care Quality Commission outpatient overall.

1Regression349.4521349.4523.7750.054b
Residual12 311.54113392.568  
Total12 660.993134   

Output 13

Coefficientsa

ModelUnstandardised coefficientsStandardised coefficientstSignificance95.0% confidence interval for B
BStandard errorBetaLower boundUpper bound
  1. CQC, Care Quality Commission.

  2. a

    Dependent variable: DR F.

1(Constant)35.82033.039 1.0840.280−29.529101.169
CQC outpatient overall7.3343.7740.1661.9430.054−0.13214.799

Output 14

Residuals statisticsa

 MinimumMaximumMeanStandard deviationN
  1. a

    Dependent variable: DR F.

Predicted value95.96104.0299.991.615135
Residual−28.89021.1100.0009.585135
Standard predicted value−2.4992.4960.0001.000135
Standard residual−3.0032.1940.0000.996135

Output 15

image

Output 16

image