Psychometric testing of the Older Patients in Acute Care Survey (OPACS) in Australian final year nursing students

Abstract Aim To assess the internal reliability and validity (content and criterion) of the Older Patients in Acute Care Survey (OPACS) as a measure of nurses’ knowledge, attitudes and practices regarding care of older hospitalized patients in Australia. Design Cross‐sectional survey. Method A convenience sample of final year nursing students at an Australian university in October 2014 completed the OPACS (N = 191). Internal reliability was assessed using Cronbach's alpha, content validity using exploratory factor analysis and criterion validity using correlations between the OPACS and Palmore's Facts on Aging Quiz and the Caring Efficacy Scale. Results Despite good internal reliability on both OPACS subscales, exploratory factor analysis of the 36 items representing behaviours and the 50 items on knowledge and attitudes failed to load strongly on their corresponding factors. Analyses of criterion validity suggested the OPACS scales are measures of attitude.


| INTRODUC TI ON
Nurses' knowledge, skills and attitudes are critical in determining how physical, psychological and emotional care are provided to older people, who are the largest patient group in most hospital environments (Rana & Upton, 2013). However, evidence from multiple studies in a diverse range of countries demonstrates that many nurses hold negative, or only marginally positive, attitudes about working with older patients and engage in undesirable practices as a result, which contributes to a failure to provide older patients with appropriate care (Dahlke, Phinney, Hall, Rodney, & Baumbusch, 2015;Rush, Hickey, Epp, & Janke, 2017;Wilson et al., 2017). Nurses' behaviour is influenced by their attitudes, and negative attitudes can adversely affect the quality of care provided (Kydd et al., 2014).
Hospitalization can be a life-changing event, even for a previously healthy older person, and can be associated with the onset of a new disability (Boltz, Resnick, Capezuti, & Shuluk, 2014;Dicks, Chaplin, & Hood, 2013). Older patients generally present to acute care with more complex chronic health issues than those who are younger (Lakhan et al., 2011), and the experience of hospitalization places older patients at risk of experiencing poor outcomes, including functional decline. For older patients, any loss of function has serious ramifications as it can result in iatrogenic infections, pressure injuries, falls, non-elective rehospitalization and increased risk of mortality (Boltz, Resnick, Capezuti, Shuluk, & Secic, 2012).
Nurses' attitudes towards older patients are established early in their professional identity formation. Student nurses have been shown to demonstrate more negative attitudes towards older patients than those younger and having many interactions with older patients may increase prejudice (Kydd et al., 2014;van Leeuwen, Oosterhuis, & Ruyter, 2016). These attitudes may reflect those of a youth-oriented society (Chrisler, Barney, & Palatino, 2016;Heise, Johnsen, Himes, & Wing, 2012) or may be promoted by Bachelor of Nursing (BN) programs that focus on a biomedical model of ageing and links extreme illness and death with ageing (Henderson, Xiao, Siegloff, Kelton, & Paterson, 2008). Students may enter the BN with positive or neutral attitudes towards older people, but their experiences in gerontological practical placements negatively influence their desire to work with older people as a career choice (Heise et al., 2012).
The stresses associated with the transition from university to the hospital environment are well documented in the literature and discrepancies between undergraduates' expectations based on their education, and the actual work environment can create a sense of instability for graduate nurses (Duchscher, 2009). Hospitals practise the traditional medical model of care, focusing on the medical diagnosis of the patient and overlooking the specific functional needs of older patients (Asmus-Szepesi et al., 2015;Buurman et al., 2011).
These issues may challenge and frustrate nurses' attempts to provide care for older patients in a hospital system that is not designed to manage their complex needs (Dahlke et al., 2015). Further, concern has been expressed that the culture of nursing has changed over the last few decades, with a decrease in nurses' willingness and motivation to help older patients (Koskenniemi, Leino-Kilpi, & Suhonen, 2013).
There remains a dearth of studies designed to investigate the attitudes (and the associated factors) of student nurses regarding older people (Liu, While, Norman, & Wenqin, 2012). One reason for this is the lack of suitable tools to measure ageism in nurses in the hospital sector. Courtney, Tong, and Walsh (2000) described the scarcity of literature examining nurses' knowledge, attitudes and practices towards older hospitalized patients as "extremely disturbing" (p. 95).
Kogan's Attitude towards Old People scale was at the time and continues to be commonly used (Dikken, Hoogerduijn, Lagerwey, et al., 2017;Neville & Dickie, 2014); however, the limitations of this survey are becoming increasing apparent. Social attitudes, the health care sector and nursing education have all evolved since it was developed in America in the middle of last century, and the focus of Kogan's scale is on caring for older people generally, rather than specifically hospitalized older patients (Runkawatt, Gustafsson, & Engström, 2013). Consequently, Courtney et al. (2000) developed the Older Patient in Acute Care Survey (OPACS) to address the need for a research instrument to assess nurses' knowledge, attitudes and practices towards older hospitalized patients. This study investigated the adequacy of the OPACS among final year Australian BN students to determine its potential for use in future studies as a measure of new graduates' preparedness to care for older patients.

| Background
Given the need for a reliable and valid measure of nurses' knowledge and attitudes towards older hospitalized patients, the OPACS has been the focus of studies undertaken in the Netherlands and the United States of America (USA). Studies have used both qualitative (Malmgreen, Graham, Shortridge-Baggett, Courtney, & Walsh, 2009;van Schelven et al., 2015) and quantitative methodologies Dikken, Hoogerduijn, Lagerwey, et al., 2017). These studies have clearly identified the importance of the cultural setting in assessing the validity of the survey, and whilst the OPACS was developed and has been used in Australia (Courtney et al., 2000;Deasey, Kable, & Jeong, 2016), no studies have examined the OPACS' reliability or validity in the Australian context.
The aim of the original developers of the OPACS was to design and trial an instrument to measure nurses' knowledge, attitudes and practices towards older patients. The OPACS includes 86 items relating to 13 different beliefs or practices identified in the literature and focus group interviews, believed to influence the quality of nursing care of older hospitalized patients (Courtney et al., 2000). The measure uses a five-point Likert-type scale of response options (ranging from 1 = never to 5 = very frequent). Most items are worded so that a high score indicates more negative attitudes (e.g., "I have difficulty..."; "Older patients are [negative attribute such as confused]"). Exceptions are items about involving patients in their own care, which are worded so that a high score indicates high intentions to involve patients and family members. The direction of scoring for some items is ambiguous; for example, on the item "I ask older patients whether they have incontinence problems" it is not apparent whether a high score represents a positive or a negative behaviour/attitude. Courtney et al. (2000) noted wording of items may be ambiguous and "results may not provide conclusive evidence of the identified findings" (Courtney et al., 2000, p. 100).

Courtney et al.'s (2000) study included Palmore's Facts on Aging
Quiz (PFAQ; Palmore, 1977) to allow criterion-related validity testing of the knowledge items of the OPACS. However, no results of any criterion-related validity testing have been published by this research team.
Attempts have been made to validate the OPACS in other countries. Malmgreen et al. (2009) aimed to establish validity of the OPACS in the USA and reported high content validity (CVI) for the survey (CVI = 0.92). However, when the Dutch version of the tool was studied in the Netherlands to establish content validity, the CVI obtained was 0.62 (van Schelven et al., 2015), which is well below the benchmark of 0.80, indicating poor overall content validity. Changes to the Dutch version to represent local language and culture may have contributed to differences in results from the USA study (van Schelven et al., 2015).
The reliability and structural validity of the Dutch version of the OPACS were further tested using a quantitative study design  . The results of a confirmatory factor analysis demonstrated good structural validity for section A (practice) and that it measured a single construct. Good  This study aimed to test the internal reliability (Cronbach's alpha) and validity (content and criterion) of the OPACS (Courtney et al., 2000) as a measure of student nurses' knowledge, attitudes and practices regarding care of older hospitalized patients in Australia.
It is intended that this study will add to the international discussion regarding developing a reliable and valid measure of nurses' knowledge, attitudes and practice towards older hospitalized patients (Dikken et al., 2016). 2 | ME THOD

| Research design
The research design is a cross-sectional survey.

| Sample
A convenience sample of Bachelor of Nursing (BN) students (N = 450) at an Australian university was invited to complete the survey at the end of their final lecture in the concluding week of their BN studies (in October 2014). One hundred and ninety-one students completed the survey (see Table 2 for participant characteristics).

| Survey
The paper-based self-report survey comprised the OPACS (Courtney et al., 2000) as well as the True/False/Don't know version of Palmore's Facts on Aging Quiz (PFAQ; Palmore, 1977) and the Caring Efficacy Scale (CES; Coates, 1997). The PFAQ and the CES were included to enable criterion-related validity testing of the knowledge and the attitude items of the OPACS, respectively.
The PFAQ is a widely used reliable and valid measure of knowledge about ageing (Baumbusch, Dahlke, & Phinney, 2012), comprising 25 items, with response options True, False or Don't Know (Palmore, 1977). It is a short test which relies on accurate knowledge of older people to assess ageist attitudes (Pachana, Helmes, & Gudgeon, 2013). PFAQ is scored by summing the number of correct responses: items with even numbers are True and items with odd numbers are False, with total scores ranging from 0-25. Higher scores reflect greater knowledge about ageing (Wang et al., 2010).
The CES has been found to be a reliable and valid measure of nurses' confidence regarding their ability develop caring relationships with patients (Reid, Courtney, Anderson, & Hurst, 2015). It is a self-report 30-item, 6-point (strongly disagree −3 to strongly agree +3) Likert-type scale and contains 23 positively worded and seven negatively worded items.

| Data analysis
Data analysis was undertaken using IBM SPSS version 22 on the 191 complete responses. Ambiguous items in the OPACS were scored in the direction consistent with most OPACS items, so that a high score indicated more negative attitudes.
Internal reliability was assessed for each of the OPACS scales using Cronbach's alpha. The structure of each of the OPACS scales (practice and general opinion) was assessed using exploratory factor analyses, using the maximum likelihood method of extraction with varimax rotation. While the sample size was relatively small, Guadagnoli and Velicer determined that a sample size of 150 is adequate to produce a stable solution if factors comprise 10 or more items with loadings >0.4 (Guadagnoli & Velicer, 1988).
Exploratory factor analysis was used to provide insight into the structure of the OPACS and is a method that explores the data set and tests predictions in an attempt to uncover complex patterns (Child, 2006). It enables identification of the underlying factor structure.
Three separate factor analyses were performed to maximize the number of participants per included item in each analysis.

| Reliability
The OPACS had good internal reliability on both scales (α = 0.88 practice scale; α = 0.90 general opinion scale) when used with Australian BN students. When the general opinion scale was further divided into knowledge and attitude subscales, good internal reliability was again demonstrated (α = 0.82 for knowledge; α = 0.84 for attitude).

| Exploratory factor analysis
Maximum likelihood factor analysis with varimax rotation was first Only 15 items had a factor loading above 0.45 and communalities >0.2 and the goodness-of-fit test was significant (χ 2 = 837.6, p < 0.000), suggesting poor fit.
The sampling adequacy of the 21 items of the knowledge subscale was verified by the Kaiser-Meyer-Olkin measure, KMO = 0.812 for the 21 items of the knowledge subscale. A one-factor solution accounted for 25.6% of the variance in items. Only 12 items had a factor loading above 0.45 and communalities >0.2 and the goodness-of-fit test was significant (χ 2 = 390.7, p < 0.000), suggesting poor fit for this subscale also.

| Criterion validity
Criterion validity was assessed using Pearson's correlation coefficients (Table 3). Strong correlations (all over 0.59) were observed between the OPACS subscales. There was no correlation between the PFAQ and OPACS practice and low negative correlations between the PFAQ and both the subscales of the OPACS general opinion (knowledge and attitude). Correlations between the CES and the OPACS subscales were low and negative (for practice and attitude) or non-existent (for knowledge; Table 4). tests of validity indicated unsatisfactorily low correlations with criterion measures.

| D ISCUSS I ON
The current study found acceptable internal reliability for the three OPACS subscales, measured using Cronbach's alpha. However, a high alpha may result from including a large number of items, rather than because the items cohere meaningfully (Field, 2014). A factor analysis is usually recommended given well-known limitations of coefficient alpha as a measure of unidimensionality or homogeneity in a sample of test items (Tavakol & Dennick, 2011).
Exploratory factor analyses indicated that the items of neither section A (practice) nor section B (attitude and knowledge) sample from distinct underlying constructs.
It was expected that correlations between the three subscales of the OPACS would be moderate in size, because each of the subscales should be predicted by, but not substantially overlap with, the other two. Higher correlations between the three subscales that might be expected on theoretical grounds were observed, suggesting that items from the three scales sample from a single underlying dimension.
Knowledge of ageing would be expected to have a low negative correlation with (poor) practice, a medium-to-high negative correlation with (poor) attitude and a high positive correlation with knowledge. The correlations between the OPACS practice and the criterion measure of knowledge were lower than expected and suggest the OPACS knowledge subscale is only minimally related to knowledge of ageing.
Similarly, a measure of positive attitudes towards caring for older patients would be expected to have a medium-to-high negative correlation with (poor) attitude and low negative correlations with (poor) practice and (low) knowledge. However, all correlations between the measure of caring self-efficacy and the OPACS subscales were low (although higher than correlations between the criterion measure of knowledge and OPACS subscales). This poor result may be because of the complex nature of attitude (Dikken, Hoogerduijn, Lagerwey, et al., 2017) and that attitude towards older patients is not a one-dimensional construct. Alternatively, it may be because some OPACS items are not good measures of attitudes towards older people. For example, it is difficult to see how the intention to involve younger patients in their hospital care is related to ageist attitudes, except when compared with the intention to involve older adults in their care.
Although originally described as a measure of knowledge, attitudes and practice of nurses regarding care of older hospitalized patients (Courtney et al., 2000;Malmgreen et al., 2009), more recent studies have largely concluded that revised versions of the OPACS are, at best, measures of attitudes only Dikken, Hoogerduijn, Lagerwey, et al., 2017;van Schelven et al., 2015). This study of the OPACS confirms these conclusions in the Australian context.

| Limitations
Generalizability of the results of this study may be limited by use of a convenience sample from the same university and the study's relatively small sample size. However, a more diverse sample may have resulted in lower estimates of internal reliability of the subscales and even lower assessment of factor structure and construct validity.
A further limitation is the study's reliance on self-report. Nurses' self-reports of their attitudes may not accurately predict their behaviours. Self-report scales are notoriously poor measures of behaviours, creating the potential for the introduction of bias because of participant self-selection and "faking good" in an attempt to conform to socially acceptable values (van de Mortel, 2008). For example, health professionals generally over-estimate the extent to which they comply with hand washing (Jenner et al., 2006). Such bias could have led to inflated estimates of internal reliability and construct validity.

| CON CLUS ION
Worldwide, older people account for an increasing proportion of patients in hospital; for many, the experience has negative outcomes including loss of functional capacity. Because nurses' knowledge and attitudes influence how and how well, care is provided (and to assess the effectiveness of any interventions to improve knowledge, attitudes and practice) having a means to measure these constructs is essential. Despite finding the OPACS has good internal reliability when trialed with Australian BN students, this study determined that the items of the OPACS do not sample from coherent underlying constructs as was expected, suggesting it is not a satisfactory measure of nurses' knowledge, attitudes or practices regarding older hospitalized patients. However, analyses of construct validity indicated that the OPACS more closely resembles a measure of attitudes than knowledge or practice. The complex multi-dimensional nature of attitudes towards older patients may contribute to the failure of the attitude items to load on a single factor. The current study confirms the findings of previous studies suggesting that the OPACS is a measure of attitudes only.
Similarly, to these previous studies, we cannot recommend the use of OPACS in its current form.

CO N FLI C T O F I NTE R E S T
No conflict of interest has been declared by the author(s).