How do hospitalization experience and institutional characteristics influence inpatient satisfaction? A multilevel approach

Authors


ABSTRACT

Over the last several years, interest in benchmarking health services' quality—particularly patient satisfaction (PS)—across organizations has increased. Comparing patient experiences of care across hospitals requires risk adjustment to control for important differences in patient case-mix and provider characteristics. This study investigates the individual-level and organizational-level determinants of PS with public hospitals by applying hierarchical models. The analysis focuses on the effect of hospital characteristics, such as self-discharges, on overall evaluations and on across hospital variation in scores. Sociodemographics, admission mode, place of residence, hospitalization ward and continuity of care were statistically significant predictors of inpatient satisfaction. Interestingly, it was observed that hospitals with a higher percentage of Patients Leaving Against Medical Advice (PLAMA) received lower scores. The latter result suggests that the percentage of PLAMA may provide a useful measure of a hospital's inability to meet patient needs and a proxy indicator of PS with hospital care. © 2013 The Authors. International Journal of Health Planning and Management published by John Wiley & Sons, Ltd.

INTRODUCTION

The patient is increasingly involved in the evaluation of health services quality with indicators on patient satisfaction (PS) and experience now important parts of modern performance measurement (Arah et al., 2003). Comparing patients' experience across organizations may be interesting (i) to observe how each health institution performs (relative to non-health aspects, meeting or not meeting a population's expectations of how it should be treated by providers; World Health Organization, 2000) and (ii) to promote debate and discussion among policy makers and health providers about the patient-centeredness in the delivery of care.

However, benchmarking satisfaction scores across institutions that serve patients with a different demographic and health profiles may be misleading without a thoughtful approach to risk adjustment. It is the main reason unadjusted results on PS are often mistrusted by health managers and professionals, particularly when their organizations receive worse evaluations compared with others (Perneger, 2004). So that the patient perspective is considered a valid perspective on the quality of care, it is necessary to adjust comparisons of patient feedback indicators for important, potentially confounding factors.

A large number of studies have investigated how perceptions of health services quality are influenced by sociodemographic characteristics (Sitzia and Wood, 1997; Hargraves et al., 2001), patient experience (Jenkinson et al., 2002a; Danielsen et al., 2010; Garcı'a-Lacalle and Bachiller, 2011), and context where care is provided (Zastowny et al., 1989; Jha et al., 2008; Chen et al., 2012). These studies show a number of factors that could be a driving satisfaction, but in most cases, the methodological approaches adopted did not take into account important issues such as the considerable risk of interactions among factors and the overestimation of differences.

Recently, multilevel models have yielded important findings for performance measurement systems in health sector (Krumholz et al., 2006) allowing the creation of comparable measures adjusted for personal and institutional characteristics. But there are still relatively few studies that analyze the effects of institutional and other contextual factors on patients' evaluations, such as geography, service size and institutional status (e.g., teaching vs community hospitals) (Young et al., 2000; Hekkert et al., 2009), administrative characteristics (e.g., health jurisdictions) (Brown et al., 2008) and employees' satisfaction (Veenstra and Hofoss, 2003). Their results show that the use of a multilevel model contributes to the more conservative estimation differences in indicator scores for patient satisfaction.

This study aims at providing detailed and more useful information about patient experience and quality of hospital to the Italian health authorities' managers. We attempt to achieve this goal in two ways. First, we explore the determinants of inpatients' satisfaction, keeping separate the variables describing patient profiles and hospital characteristics. In this way, managers can observe how patient case-mix and institutional factors explain the patient satisfaction. Second, we measure the variability across patient and hospital levels. Investigating the variation in satisfaction scores within and between hospitals and their origins can be helpful for understanding the extent to which patient evaluations are affected by different elements. When it is known which level drives satisfaction, it should be easier to develop specific policies for quality improvement. Finally, performance indicators controlled for patient and hospital characteristics could be created to observe if real differences exist across hospitals.

METHODS

Setting

Previous researches on PS in Italy have mainly focused on groups of patients with specific diseases and described the properties of the instruments used (Gigantesco et al., 2002; Bredart et al., 1999). Therefore, relatively little is known about the evaluation of health services through the patient's perspective in Italy (Gallo et al., 2004) and its usefulness as a way of comparing performance across organizations and most studies describe a specific institution (Elia and Barburini, 2002).

Since 2004 in Tuscany (one of the 20 regions of Italy), it has been possible to compare the perceived quality of a large number of services provided by 12 local health authorities and five teaching hospitals. It is possible because the regional health system of Tuscany has been implementing a performance evaluation system (PES)—developed by Laboratorio Management e Sanità of the Scuola Superiore Sant'Anna in Pisa—which monitors the results achieved by providers through roughly 130 indicators (Nuti et al., 2010; Cacace et al., 2011). Fifteen indicators cover PS with health services (e.g., primary care, emergency department, home care, hospital service and maternal care) (Nuti, 2008; Nuti et al., 2009). This is one of the very few cases in Italy that supports comparison of patient experience with care provided by different institutions.

Survey

A stratified random sampling procedure was used to select participants for this study. The sampling frame was composed of inpatients discharged from Tuscan hospitals during the period September–December 2008. All 34 public general hospitals (excluding a pediatric hospital) were involved in the study. Patients hospitalized in medical, surgical and obstetric–gynecologic–pediatric (OGP) wards were involved, whereas newborn babies, patients treated in intensive care units or in a day hospital were excluded. When repeat admissions were recorded, only the last one was considered, because if a patient had more than one hospitalization (repeat admission) in the sampling period, then it was preferred to contact him only one time about the most recent admission.

The Computer Assisted Telephone Interviewing technique was used. It was preferred to a postal or Computer Assisted Web Interview-based survey because it allows the user to control data entry while conducting interviews, to obtain results quickly and to reach low literacy groups (Coulter et al., 2009).

Questionnaire

A brief Inpatient Experience Questionnaire was developed by considering the current literature (Jenkinson et al., 2002b; González et al., 2005) and based heavily on previous surveys undertaken and validated in the Tuscany region (Nuti, 2008). Twenty-eight questions covered the patient's relationship with doctors and nurses, communication process, information provided at discharge and overall evaluation of care. Seven questions covered the patient's sociodemographic characteristics (age, gender, educational level, self-reported health status, employment, chronic diseases and previous hospitalization). Report or objective style questions were used more often than rating or satisfaction style questions, because the more subjective nature of the PS concept may be affected by expectancies, pre-evaluation and previous experience (Cleary et al., 1992). (Copies of the survey and factor analytic results are available from the authors.)

Multilevel analysis

Data collected had a hierarchical structure where patient (level 1) data were nested in hospitals (level 2). This multilevel data framework allowed the investigation of whether (i) individual patient experience with hospital services varies among hospitals (heterogeneity), (ii) patients within a hospital have similar experiences with assistance and (iii) individual experience depends on both patient and hospital characteristics (Goldstein and Spiegelhalter, 1996).

To observe the hospital effect on patient satisfaction, a random intercept multilevel model was preferred to other statistical approaches because it manages more levels simultaneously and returns separate residual variance components for between and within-group variability.

Regression coefficients and variance components at hospital and patient levels were estimated for four indicators of satisfaction with (i) overall care, (ii) doctors' assistance, (iii) nurses' assistance and (iv) the extent that information and communication needs were met (Table 1). These indicators were chosen because relationships with health professionals and the communication process are potentially critical components of satisfaction (Sitzia and Wood, 1997). All indicators were calculated by averaging their subscale scores, following their transformation into a 0–100 scale with higher scores indicating better evaluations. A continuous scale was preferred to preserve the gradations of patients' evaluation (Brown et al., 2008). This method of scoring—already adopted in other studies (Weech-Maldonado et al., 2003)—differs from the method described by Jenkinson et al. (2002a) according to which items are classified as problematic or not.

Table 1. Patient satisfaction indicators and subscales
Indictors/subscalesMeanSD
  1. SD, standard deviation.

Doctors 84.917.4
 Courtesy86.018.6
Assistance85.819.3
How doctors and nurses work together83.719.4
Nurses 92.314.3
 Clear answers92.521.4
Trust94.118.4
Respect96.814.2
Timely answer to call button83.320.9
Communication 93.713.7
 Clear answers by doctors94.019.5
Info on care and treatment89.726.9
Respect (doctors)95.516.7
Privacy96.316.0
Concordance on information94.418.2
Between doctors and nurses94.319.7
Overall evaluation 88.219.5
 Assistance84.519.7
Willingness to recommend92.623.3

For each indicator, two models were fitted. The first model did not consider explanatory variables (empty model), and its intercept measured the overall mean of patients' scores. The second model measured the residual variance explained at lower and higher levels after adjusting for patient and hospital characteristics. Furthermore, intra-class correlation coefficients (ICCs) were calculated to observe the amount of variance in PS scores because of differences between hospitals.

Explanatory variables, patient level

Variables about sociodemographic characteristics and patient experience are used at the patient level. In details, they are the following: age (a), gender (b), educational level (c), self-reported health status (d), admission mode (e), length of stay (f), hospitalization ward (g), whether patient lived in the hospital's geographical area (h), whether either a doctor or a nurse was in charge of patient care (i), whether a chronic disease caused admission (l), whether a general practitioner (GP) was informed on patient hospitalization (m) and whether the patient had any previous stays (n). Variables (f), (g), (h) were derived from hospital's patient administrative records, whereas variables (a), (b), (c), (d), (e), (i), (l), (m), (n) were reported directly by patients during the interview (Table 2).

Table 2. Description of explanatory variables introduced in the random intercept model
VariableType of variableScaleReference group
  1. OGP, obstetric–gynecologic–pediatric; GP, general practitioner.

AgeContinuous0–99 
GenderDichotomousFemale, maleMale
EducationOrdinalPrimary, secondary, high, degreeDegree
Self-rated health statusOrdinalVery poor, poor, fair, good, excellentVery poor
Living in hospital areaDichotomousYes, noNo
Admission modeDichotomousPlanned, no plannedNo planned
Length of stayContinuous0–115 
Hospitalization wardCategoricalMedical, surgical, OGPSurgical
Hospitalization reasonDichotomousChronic disease, othersOthers
Previous staysCategoricalNo, onetime, more timesNo
Doctor in chargeDichotomousYes, noNo
Nurse in chargeDichotomousYes, noNo
GP informedDichotomousYes, noNo
Institutional statusDichotomousTeaching, no teachingNo teaching
% of voluntary dischargeContinuous0.1–6.2 
Hospital sizeCategoricalSmall, medium, largeSmall

Explanatory variables, hospital level

Hospital size, institutional status and percentage of voluntary discharges were introduced as predictors of PS at the hospital level. Hospital size was defined considering the percentile distribution of the beds' number (“small” ≤ 153 beds, “medium” ≤ 416, “large” > 416), whereas the variable on institutional status classified hospitals as teaching or non-teaching.

The voluntary discharge indicator measured the percentage of patients that left hospital against medical advice. Haywood et al. (2010) confirmed that patient trust and poor quality of interpersonal experiences with care may be associated with hospital self-discharge and that this outcome can be considered an indicator of the quality of care received. Moreover, dissatisfaction with treatment received is one of the most commonly cited reasons for leaving the hospital (Hwang et al., 2003). So a high percentage of patients leaving against advice could reflect low quality of assistance and a hospital's inability to meet patients' needs. For these reasons, it is interesting to determine if a relationship exists between PS and voluntary discharge rates. All data on hospital characteristics were derived from the regional administrative system.

RESULTS

About 25.510 patients were contacted by phone during a 4-month period, and 15.474 questionnaires were completed. This produced an overall response rate of 61%, varying from 52% to 84% across individual Tuscan health authorities. This response rate is consistent with previous studies using a telephone approach and confirms that questionnaires administered by phone ensure higher response rates than mail or Web questionnaires (Castle et al., 2005).

Descriptive analysis

The results described in this paragraph referred to 14.934 patients (users of teaching pediatric hospitals were not included in this study). As noted in Table 3, respondents were on average 54 years old, predominantly women, with a primary school certificate, with an average self-rated health status and experiencing their first hospitalization. The majority of the patients lived in the hospital area, did not have a planned admission, stayed in the hospital on average of 6 days, and the reason for hospitalization was generally a chronic condition. Patients were discharged by medical (36%), surgical (35%) or OGP wards (29%). Thirty per cent of patients reported that during the hospitalization, a specific doctor was in charge of his care, only 1% reported that they were followed by a specific nurse and 72% said that their GP was informed about the hospitalization.

Table 3. Principal characteristics of patients and hospitals
Level 1—patients characteristics
  1. SD, standard deviation; OGP, obstetric–gynecologic–pediatric.

Age, mean (SD, range) 53.8(22.7)(0–99)
Gender (%)
 Male38.8  
Female61.2  
Educational level (%)
 Primary39.3  
Secondary24.7  
High26.3  
Degree9.7  
Self-reported health status (%)
 Very poor1.7  
Poor13.2  
Fair49.5  
Good24.5  
Excellent11.1  
Admission mode (%)
 Planned38.3  
No planned61.7  
Length of stay (days), mean (SD, range) 5.7(5.7)(0–115)
Hospitalization Area (%)
 Medical35.7  
Surgical35.0  
OGP29.3  
Doctor in charge (%)
 Yes30.4  
No69.6  
Hospital size (%)
 Small35.3  
Medium32.4  
Large32.4  
Hospital institutional status (%)
 Teaching25  
No teaching75  
Voluntary discharges, % average (SD, range) 1.2(1.0)(0.1–6.2)

In 2008, the 34 surveyed hospitals had on average about 360 beds (range 33–1645), most of them were community hospitals (75%) and 1% (range 0.1–6.2%) of their users voluntarily decided to leave the hospital.

Patient satisfaction with Tuscan hospitals was generally positive (Table 1). In a score ranging from 0 to 100, patients rated hospital assistance and willingness to recommend a specific hospital as 85 and 93, respectively. Satisfaction with communication had the highest score: patients received clear answers to their questions; adequate and concordant information were given; during consultation, their privacy was respected; and they were treated as individuals. Again, the relationship with nurses was evaluated more positively than the relationship with doctors.

Multilevel model results

Variances at patient and hospital levels were first analyzed for each indicator (overall satisfaction, doctor and nurses assistance, and communication), without considering explanatory variables. In all empty models, a statistically significant variance was observed at hospital level, and it was larger for indicators of the overall satisfaction and the patient–doctor relationship.

However, the hospital level generally explained only a low percentage of overall variance (ICCs ranged from 1.1% to 1.9%) implying a moderate contextual effect, much smaller than the personal one. This result is not surprising and is in line with other studies (Veenstra and Hofoss, 2003).

  • Patient level—It was observed that gender had a significant statistical effect only on patient experience with nurses and communication; in particular, men were likely to provide higher scores than women (Table 4).
Table 4. Multilevel models' results
 Model 1Model 2Model 3Model 4
OverallDoctorsNursesCommunication
Fixed part—level 1 Coeff. SECoeff. SECoeff. SECoeff. SE
  • ICC. intra-class correlation coefficients.

  • °

    p ≤ 0.050.

  • +

    p ≤ 0.010.

  • *

    p ≤ 0.001.

Cons 69.89 1.7969.70 1.5884.12 1.3180.79 1.23
Age, mean 0.09°0.040.13*0.040.07+0.030.07+0.03
Age2, mean 0 00 00 00 0
Gender, male 0.53 0.400.21 0.351.62*0.300.60°0.28
Education
Primary4.48*0.724.55*0.643.02*0.533.36*0.50
Secondary3.80*0.694.18*0.611.84*0.512.62*0.48
High2.58*0.672.75*0.591.41+0.491.21+0.46
Self-reported health status
Fair9.04*1.466.20*1.294.75*1.087.13*1.01
Passable14.62*1.4110.51*1.247.07*1.0410.07*0.97
Good17.13*1.4513.45*1.287.47*1.0711.72*1.00
Excellent20.90*1.5117.51*1.348.86*1.1213.08*1.04
Place of residence, hospital area−1.40+0.55−1.94*0.49−0.58 0.41−0.33 0.38
Admission mode, planned1.68*0.401.46*0.361.41*0.301.10*0.28
Length of stay, mean−0.05 0.06−0.01 0.05−0.19*0.04−0.10+0.04
Length of stay2, mean0.001 0.0010.001 0.0010.003+0.0010.001 0.001
Hospitalization area
Med1.45*0.461.31*0.401.74*0.340.95+0.32
OGP1.94*0.592.41*0.520.99°0.440.82°0.41
Hospitalization for chronic disease, yes0.46 0.470.88°0.41−0.09 0.35−0.26 0.32
Previous stays
More−1.37°0.80−1.12 0.70−2.26*0.59−1.94*0.55
One time−0.56 0.49−0.22 0.43−0.42 0.36−0.64°0.34
Doctor in charge, yes4.23*0.405.09*0.350.81+0.292.52*0.27
Nurse in charge, yes2.90°1.760.64 1.550.20 1.3−1.49 1.22
GP informed0.81°0.420.98+0.370.10 0.311.00*0.29
Fixed part—level 2
Hospital size
Medium−2.57+0.86−1.91+0.78−2.76*0.60−1.85*0.56
Large−4.00*0.88−3.86*0.79−2.90*0.61−2.20*0.57
Teaching status, yes1.57 1.081.85°0.98−0.76 0.750.43 0.70
% voluntary discharges−1.08+0.36−0.64°0.32−0.31 0.25−0.21 0.23
Random part
Level 2 variance: hospitals, var(U0j)2.44 0.862.04 0.711.10 0.410.97 0.37
Level 1 variance: patients, var(Rij)349.55 4.57270.20 3.53190.31 2.49166.30 2.17
ICC (%)
var(U0j)/[var(U0j) + var(Rij)]0.690.750.570.58
−2*loglikelihood (level 2):102,191.999,008.2695,028.1693,500.18
−2*loglikelihood (level 1):102,238.399,059.4995,064.3693,537.87
 
 p < 0.0001p < 0.0001p < 0.0001p < 0.0001

As expected, results showed that age, education and self-reported health status were statistically significant predictors of PS with hospital care (Hekkert et al., 2009; Veenstra and Hofoss, 2003). Being older and in good health increased the probability of declaring a high subjective satisfaction. In contrast, less educated patients were more satisfied than those with a university education. As Table 4 shows, living in the hospital area negatively affected both patients' overall experience and their relationship with doctors, whereas the longer the patient stayed in the hospital, the lower the scores were on nursing assistance and communication.

On one hand, having had only one previous hospitalization experience decreased the probability of being satisfied with communication; on the other hand, being hospitalized more than once had a negative effect on overall experience and on the patient–nurse relationship. A planned admission (vs urgent or emergent admissions) was found to be a positive predictor for all four indicators. Whether patients were hospitalized in a medical or OGP ward, they generally judged hospital service more positively than patients hospitalized in a surgical ward.

When a referential doctor was reported by patients to be in charge of their care, all indicators' scores were higher. When the patient could refer to a specific nurse, only the ratings for the overall experience increased. It is important to specify that in Italian hospitals, only the physician can be in charge of all patient care (diagnosis, treatment, etc.), whereas nurses deal with nursing care.

Moreover, Table 4 shows that if the GP was informed about his patients' hospitalization, a positive effect was observed on all indicators with the exception of nursing care.

  • Hospital level—Having been hospitalized in medium or large hospitals (vs small hospitals) seemed to improve all the levels of patient satisfaction, whereas having been hospitalized in a teaching hospital positively affected only satisfaction with doctors. Last but not the least, as the percentage of discharges against medical advice increased, satisfaction with overall experience and doctor assistance declined.

The ICCs decreased for each patient satisfaction indicator after adjusting for patient and hospital characteristics.

DISCUSSION AND CONCLUSIONS

This study identified patient and hospital characteristics affecting satisfaction with hospitalization and noted whether and how individual experiences varied across and within hospitals. Most of the variance in ratings was observed at the patient level, confirming previous studies according to where variability is larger within organizations (hospitals or wards) than between organizations (Veenstra and Hofoss, 2003). Results highlighted that older, male, less educated and healthier patients tended to rate the hospital service more positively than others. In line with other studies, a planned admission tended to have a positive influence on patient evaluations (Veenstra and Hofoss, 2003).

Comorbidity and diagnosis data have not been considered in the analysis as the reason for hospitalization was not available because of privacy law restrictions. However, information on hospitalization ward (i.e., if it is medical, surgical or OGP) was considered, and some differences were observed across the four indicators. In particular, patients who had been hospitalized in a surgical ward tended to be generally unsatisfied. A possible explanation of this difference could be the characteristics of patients treated in the different kinds of wards. For example, patients in medical wards were generally older than those in surgical ones and more likely to suffer from chronic conditions; OGP patients were mostly pregnant women or new moms, whereas surgical patients were more likely to suffer from more acute exacerbations of conditions requiring necessitating surgical treatment. The first two groups of patients tended to give higher evaluation because older people are more positive—as Sitzia and Wood (1997) review brought out—and pregnant women fortunately and largely experience a happy event. Surgical patients may take a more negative view of their health and may have greater pain or other challenges with recovery.

Interesting results were observed when considering the continuity of care during the hospitalization and after the patient's discharge. The presence of a specific doctor in charge of care during the hospitalization improves the patients' experience and their evaluations, probably because they feel safer and followed up. Moreover, having had a GP informed about hospitalization positively affected ratings for doctors and for the overall experience. Positive ratings on satisfaction with doctors' care were also likely to be given when any chronic disease was the main reason for hospitalization. Chronic patients seemed to be more satisfied with doctor assistance, potentially because the hospital doctor is often the same specialist who is in charge of patient's care outside the hospital, creating greater continuity of care. Furthermore, receiving care in a hospital far from home had a positive impact on hospitalization experience (see model related to overall and doctors' assistance in Table 4). This result could be explained by taking into account that they are assumed to have made an informed decision specifically on the basis of their needs and preferences.

Small and teaching hospitals received higher scores than medium/large and community hospitals, respectively. Being hospitalized in a teaching hospital significantly and positively affected ratings on doctors' assistance. These findings about hospital peer group and academic status may appear conflicting because teaching hospitals usually have a larger number of beds than community hospitals. This suggests that patients' case-mix in teaching and community hospitals deserves additional consideration. Patients in small community hospitals do not always require complex care and may not have high expectations. These elements may make it easier to satisfy patients' demands. On the contrary, the need for complex care could be the main reason for being hospitalized in a teaching hospital (Messina et al., 2009); however, it was generally observed that sicker patients tended to give more negative evaluations. That said, the results obtained in this study, according to which ratings about doctors' assistance are slightly higher in teaching hospitals than in other large hospitals, could be in part explained by the reputation the academic status provides to doctors working in teaching hospitals. This in turn could positively influence patients' expectations of service and care, although the exact process by which this works is not clear. At this stage of analysis, the relationship between PS and hospital status remains a complex issue, which needs further research in both the teaching and non-teaching environment.

Finally, an important result for managers is the significant relationship between satisfaction and the percentage of voluntary discharges. Because the percentage of patients that leave hospital against medical advice may be considered as a proxy indicator of PS with hospital care, the observed result allows health care management to have at their disposal several measures of patient perspective even when more costly, more time consuming and potentially less timely survey data are not available.

Taken together, the results identify a clear profile of patients asking for more quality: they are young and with a higher education, with a poor perception of their health status, live close to home, do not feel to be followed by a health professional and are hospitalized in a medium or large institution. They appear to be more demanding and have a clear idea of what they want from a health service. To create a health service that is able to respond to all the patients' needs, policy makers and hospital managers have to take into account these factors. Moreover, differences in users' experiences should be considered in the evaluation of hospitals' performance, because the ranking across structures could change with adjustment for these characteristics (Figure 1).

Figure 1.

Hospital rank based on overall patient evaluations before and after adjusting for patient and hospital characteristics.

For these reasons, multilevel models were used to make more comparable reports on patient experiences with care across the hospitals of Tuscan health authority. The present study is the first attempt in a program of research that will more thoroughly investigate the relationship between PS and other indicators included in the Tuscan PES (such as job satisfaction and health outcomes).

ACKNOWLEDGEMENTS

This paper would not have been possible without the support of the whole team of Laboratorio Management e Sanità, Istituto di Management of Scuola Superiore Sant'Anna in Pisa. Furthermore, authors wish to thank all professionals of Tuscan health authorities for their collaboration during the patients' recruitment.

The authors have no competing interests.

Ancillary