Reliability and validity of the practice environment scale of the nursing work index for Japanese hospital nurses

Abstract Aims The aim of this study was to examine the reliability and validity of the Practice Environment Scale of the Nursing Work Index (PES‐NWI) for hospital nurses in Japan. Design A cross‐sectional mail survey. Methods Participants in this study were 1,219 full‐time ward nurses from 27 hospitals in Japan, using 31 items of the Japanese version of the PES‐NWI questionnaire, from December 2008‐March 2009. Construct validity, criterion‐related validity and internal consistency of the PES‐NWI were tested. Results The PES‐NWI showed reliable internal consistency. The five‐factor structure was supported by confirmatory factor analysis. The PES‐NWI correlated significantly with job satisfaction, burnout and the nurses’ intention to stay on the job, supporting criterion‐related validity.


| BACKG ROU N D
The PES-NWI, developed by Lake (2002) as one of the instruments to examine the practice environment for nurses, was based on the NWI. In the PES-NWI, "nursing practice environment" was defined as "the organizational characteristics of a work setting that facilitate or constrain professional nursing practice" (Lake, 2002, p. 178). It has been endorsed as a gauge of the quality of the nursing practice environment by several organizations in the USA, such as the National Quality Forum (NQF) (Warshawsky & Havens, 2011, p. 18). The PES-NWI consists of five subscales, which are "Nurse Participation in Hospital Affairs," "Nursing Foundations for Quality of Care," "Nurse Manager Ability, Leadership and Support of Nurses," "Staffing and Resource Adequacy," and "Collegial Nurse-Physician Relations" (Lake, 2002, p. 181). The five subscales have been shown to have acceptable internal consistency and reliability.
The PES-NWI has been translated into many languages, such as French, Chinese and Icelandic (Warshawsky & Havens, 2011). The PES-NWI has been, in many countries, examined for its relationship with other variables that reflect "nurse outcomes," such as job satisfaction, intent to leave job, empowerment and burnout level; "patient outcomes," such as mortality, nurse-rated quality of care and hospitalization; and "organizational variables," such as turnover rate (Warshawsky & Havens, 2011). Because items of the PES-NWI reflect the features of "magnet hospitals," it was expected that higher scores of the PES-NWI would lead to higher staff-nurse job satisfaction and retention of staff nurses and lower burnout level. Manojlovich (2005) identified, using the answers of 332 respondents from a random sample of 500 hospital nurses in Michigan, factors in the practice environment that contribute to nurses' job satisfaction. In Canada, "practice environment" had a positive correlation with personal accomplishment and a negative one with emotional exhaustion . A study of Registered Nurses (RNs) in US-based Army Medical Department hospitals showed that an unfavourable nursing practice environment was significantly correlated with job dissatisfaction, emotional exhaustion, intent to leave and poor quality of care (Patrician, Shang, & Lake, 2010).
The PES-NWI was translated into Japanese with both a forward translation and back-translation process, which included a method for confirming the equivalence between the original items of the PES-NWI and the items of the backward-translated version (Ogata, Nagano, & Akanuma, 2008). Since the PES-NWI translations have been available, it has continued to be used by researchers (Anzai, Douglas, & Bonner, 2014;Narita & Ishii, 2015;Ogata, Nagano, Fukuda, & Hashimoto, 2011;Ogata et al., 2010;Tei-Tominaga, Tsuchiya, & Sato, 2012), as well as by nurse administrators. This phenomenon indicates that an instrument that measures the nursing practice environment is also needed in Japan.
In a preliminary study for reliability and validity testing that used data from 282 RNs in six Japanese hospitals, test-retest reliability and internal consistency were found to be acceptable (Ogata et al., 2010). However, proving construct validity requires a study of the PES-NWI for a larger sample (Ogata et al., 2010). For international comparisons, confirmation of the reliability and validity of the original items, without modification, is needed. The aim of this study was to confirm the reliability and validity of the Japanese version of the PES-NWI with a larger sample than was used in the preliminary study.

| Method
This study is cross-sectional and consists of reliability and validity testing with data obtained from a mail survey.

| Participants
Invitations to nurse participants occurred in two steps. As the first step, 27 chief nursing officers out of the 696 candidate hospitals whose number of beds exceeded 50 notified us that their hospital would participate in the mail survey. The 696 hospitals were located in Japanese cities whose population is more than 500,000, such as Tokyo, Osaka and Kyoto. University hospitals were excluded, because their functions, which include education and research in addition to medical care, were different in kind. As the second step, 4,688 ward nurses from the 27 responding hospitals were delivered anonymous self-reported questionnaires from December 2008-March 2009. In this study, we considered only the questionnaires returned by full-time nurses.

| Measures
The self-reported questionnaire to nurses included the following: (1) demographic and professional experience; (2) the PES-NWI (Lake, 2002) translated into Japanese (PES-NWI-J); (3) the Work Satisfaction Instrument in Japanese (WSI-J) (Ozaki & Tadamasa, 1988); (4) the Japanese Burnout Scale (JBS) (Kubo & Tao, 1994;Tao & Kubo, 1996) and (5) an original item, "Intention to remain in or leave the workplace next year." The demographic and professional experience of participants included gender, age, educational background, years of nursing practice experience and marital status. The PES-NWI items included the following instruction (Lake, 2002): "For each item, please indicate the extent to which you agree that the item is PRESENT IN YOUR CURRENT JOB. Indicate your degree of agreement by circling the appropriate number." The four response categories (Lake, 2002) were from "strongly agree (=1)"-"strongly disagree (=4)." The subscales were calculated as mean scores after the reply values were reversed; for example, "strongly agree" became 4 and "strongly disagree became 1. The composite was calculated as the mean of the five subscale scores.
As the WSI-J has been used in many previous studies in Japanese nursing settings (Kubo et al., 2007;Nakagaki, 2011;Ueno, Syoji, Setsuko, Takashima, & Takama, 1999) and has been confirmed for its reliability and validity (Ozaki & Tadamasa, 1988), WSI-J was applied in this study. The Cronbach's alpha coefficient was 0.89 for the WSI-J in this study.
Second, the JBS was used for criterion-related validity testing, because the original PES-NWI scores were reported to have negative associations with nurse burnout (Gabriel et al., 2013;Hanrahan, Aiken, McClaine, & Hanlon, 2010;. The JBS was developed based on the setting of a Japanese human-services organization, including nursing units (Kubo & Tao, 1994). It consists of 17 items with three subdimensions that are the same as the Maslach Burnout Inventory (MBI), which was developed to assess three aspects of the burnout syndrome (Maslach & Jackson, 1981), namely, emotional exhaustion (EE), depersonalization (DP) and personal accomplishment (PA) (Tao & Kubo, 1996). The minimum and maximum values for the subscale of the JBS are 5 and 25 for EE, 6 and 30 for DP and 6 and 30 for PA. Higher EE scores and DP scores and lower PA scores, indicate a more serious burnout situation. The Cronbach's alpha coefficients measured by this study were 0.83 for EE, 0.84 for DP and 0.78 for PA.
Finally, an original item, "Intention to remain in or leave the hospital next year," with four answer options from "remain" to "resign," was used. In this study, "remain" and "probably remain" were interpreted as "intention to remain at the hospital in the next year," while "probably resign" and "resign" were interpreted as "intention to resign from the hospital within the next year."

| Analysis
Descriptive statistics were calculated regarding 27 hospital characteristics, nurses' characteristics, the PES-NWI-J scores and the other variables. To assess reliability of the PES-NWI-J, internal consistency, namely, Cronbach's alpha coefficients of the five subscales and the composite, were calculated. To assess construct validity, confirmatory factor analysis was conducted to determine whether the data of the participants would show the same structure as the 5-factor model theorized in the original PES-NWI. Model fitness was assessed with goodness-of-fit index (GFI), normed fit index (NFI), comparative fit index (CFI) and root-mean-square error of approximation (RMSEA). The model was built using the five subscales of the original PES-NWI.
To explore the criterion-related validity, we examined the relationships between the scores of the PES-NWI-J and WSI-J or JBS scores, using nurses' age as a control variable because satisfaction (Kubo et al., 2007) and burnout (Motomura & Yatsushiro, 2010) were related to respondents' age. The relationships between the PES-NWI-J scores and "Intention to remain in or leave the hospital next year" were also tested by logistic regression analysis, with the latter as the dependent variable and the composite and subscale scores of the PES-NWI-J and "nurse's age" as the independent variables.
Nurse's age was used as a control variable in logistic regression models. All statistical analyses were done using SPSS Statistics version 23.0 and AMOS version 23.0. A p-value of .05 was used as the cutoff for significance.

| Ethics
Study protocol and the process were reviewed and approved by the Ethics Committee of the Graduate School of Nursing, Chiba University, Japan. The return of the anonymous self-reported questionnaire from ward nurses in the mail survey was taken as the consent of participation by the nurses.

| Characteristics of hospitals and participants
Regarding the 27 hospitals' characteristics, the mean for number of beds was 311.7 beds (SD 159.3) and average length of stay was 16.9 days (SD 4.9). As for the fee category, which in Japan is based on the ratio of patients to nurses, the highest category, "seven-toone," comprised 15 hospitals (55.6%) and the second highest, "tento-one," comprised nine hospitals (33.3%).
Among 1,847 survey respondents, the 1,219 participants' answers that had no missing values for the variables used in the analysis to confirm the reliability and validity were considered. Their average age was 31.8 years old (SD 7.8) and the average amount of nursing experience was 9.3 years (SD 7.2). The average of their work experience at the current hospital was 6.3 years (SD 6.2). All of them are full-time nurses, not part-time. Of the 1,219 participants, 97.4% were female (N = 1,187) and 30.4% were married (N = 371). As for educational background, 80.5% (N = 981) finished nursing school only and 6.1% (N = 75) had achieved a BA or higher degree.

| Descriptive results
The average scores of the PES-NWI were 2.47 (SD 0.37) for

| Reliability and validity testing
The Cronbach's alpha coefficients were 0.78 for "Nurse Participation in Hospital Affairs," "Nursing Foundations for Quality of Care," and "Staffing and Resource Adequacy," 0.86 for "Nurse Manager Ability, Leadership and Support of Nurses," 0.81 for "Collegial Nurse-Physician Relations," and 0.79 for the composite.
The model of the confirmatory factor analysis was built using the five subscales of the original PES-NWI (Figure 1). All path coefficients were significant. The chi-square value was also signifi- Criterion-related validity was tested as the relationships between the PES-NWI-J and the WSI-J, the JBS and "Intention to remain in or leave the workplace." The composite of the PES-NWI-J indicated statistically significant positive correlation with the WSI-J (r = .75, p < .001) ( Table 1). The composite also showed statistically significant, negative correlation with EE (r = −.38, p < .001) and DP (r = −.40, p < .001) of the JBS and positive correlation with PA of the JBS (r = .28, p < .001) ( Table 1).
Participants' intention to remain in (=1) or leave (=0) the hospitals related significantly to nurses' age: younger groups (less than 25 years old, 26-29 years old, 30-34 years old and 35-39 years old) tended to leave more readily than the elder group (40 years old and over) (p < .001). In the logistic regression analysis (Table 2)

| D ISCUSS I ON
The PES-NWI in Japanese (PES-NWI-J) had already been confirmed for its equivalence with the original items of the PES-NWI (Ogata et al., 2008). This study confirmed that it was valid and reliable in a Japanese context with a large sample. As a result, the PES-NWI-J may be used in future studies to ascertain the nursing practice environment in Japan.

| Reliability
Internal consistency of the PES-NWI-J was confirmed, because Cronbach's alphas of the PES-NWI-J in this study, from 0.78 to 0.86, were acceptable for both the five subscales and the composite. Compared with the Cronbach's alphas found in previous studies that used the PES-NWI translated into other languages, they demonstrate satisfactory reliability of the PES-NWI-J; Cronbach's alpha were 0.71-0.84 in the original PES-NWI (Lake, 2002), 0.75-0.92 (Anzai et al., 2014;Ogata et al., 2011) in Japanese, 0.65-0.90 in Chinese (Chiang & Lin, 2009)

| Validity
In this study, validity was confirmed as "construct validity" founded on confirmatory factor analysis and as "criterion-related validity" based on the relations between the PES-NWI and the other variables.

| Construct validity
Because the chi-square test is affected by sample size, the model is easily rejected when the sample size is large (Hooper, Coughlan, & Mullen, 2008;Toyoda, 2011), as in this study. Therefore, goodnessof-fit in the confirmatory factor analysis has been comprehensively assessed by multiple indices. GFI (Toyoda, 2011) and RMSEA (Steiger, 2007) supported the model fit, although chi-square, CFI and NFI were marginal.
Goodness-of-fit indices of the model were improved by considering correlated errors for three pairs of items in Figure 1. The phenomenon was potentially caused by the correlation between factors 1 and 2 in Figure 1, namely, the subscales 1 and 2 of the PES-NWI, respectively. The correlation coefficients between those two factors were high compared with the relationships among the other factors in previous studies, such as 0.65 (Lake, 2002) and 0.74 (Hanrahan et al., 2010). The relationships among the structure's latent variables in the two factors were largest in both models used in this study, with trends the same as with the PES-NWI in Spanish (Pedro-Gómez et al., 2012). Therefore, the relationship between the two factors might be relatively greater in this study. In the context of this relationship, it is possible that the meanings of the items considered "correlated errors" are also similar. Although the two items that represent one of the three pairs considered correlated errors belonged to two different factors, their meanings were similar. Specifically, "item 5: Career development/clinical ladder opportunity" in factor 1 and "item 4: Active staff development or continuing education programs for nurses" in factor 2, both reflect staff development in the organization.
The goodness-of-fit of the confirmatory factor analysis can be improved by participant and organizational characteristics, too. Aged-care sector (Parker, Tuckett, Eley, & Hegney, 2010). Thus, future research might consider participants' "age mix" or "nurseexperience mix" and the different characteristics of organizational types.

| Criterion-related validity
Criterion-related validity testing showed that the PES-NWI composite score had relatively high correlations with the WSI-J, medium negative correlations with EE and DP and positive correlations with PA. The better that the PES-NWI composite score was, the more that nurses were satisfied, as in a previous study (Manojlovich, 2005). Better PES-NWI composite scores also correlated with lower burnout scores, as in previous studies (Aiken, Clarke, Sloane, Lake, & Cheney, 2008;Liu et al., 2012).
PES-NWI scores related significantly to the nurses' intention to remain. The better the practice environment was (higher PES-NWI scores), the more nurses intended to remain in their hospitals, as with previous studies (Gardner, Thomas-Hawkins, Fogg, & Latham, 2007;Ogata et al., 2011). The relationships seem to reflect that the PES-NWI was developed based on the characteristics of "magnet hospitals" that draw nurses to them.

| Limitations and future research implications
This study has several limitations. First, a self-report questionnaire cannot distinguish the influence of response bias. Second, the study included only nurses working in the following setting: a non-university hospital, located in a large city, with more than 50 beds. These participant hospitals may be more motivated to improve the practice environment, because the competition to hire nurses is more acute for city hospitals. Last, the relationships between the nursing practice environment and patient outcomes were not tested, although the environment is thought to substantially affect the quality of care. Future study is needed to consider variations in hospital characteristics, such as urban vs rural settings and to investigate the relationships between the nursing environment and patient outcomes.

ACK N OWLED G EM ENTS
We thank the nurses who participated in this research. We also thank Dr. Lake and Dr. Manojlovich for their support in the process of translating the PES-NWI into Japanese.

CO N FLI C T O F I NTE R E S T
No conflict of interest has been discovered by the authors.

AUTH O R CO NTR I B UTI O N S
All authors have agreed on the final version and meet at least one of the following criteria (based on those recommended by the ICMJE): • substantial contributions to conception and design, acquisition of data, or analysis and interpretation of data • drafting the article or revising it critically for important intellectual content.