This study aimed to assess the face, content, and construct validity and the internal reliability of a tool for assessing the generic needs of patients with cancer (the Supportive Care Needs Survey).
This study aimed to assess the face, content, and construct validity and the internal reliability of a tool for assessing the generic needs of patients with cancer (the Supportive Care Needs Survey).
A total of 1492 consecutive patients attending the surgical, radiation, or medical oncology departments of 9 cancer treatment centers in New South Wales, Australia, were asked to participate. Of the 1370 eligible patients, 1354 (99%) consented to participate and 888 (65%) completed the survey. Eligible consenting patients were given a Supportive Care Needs Survey to complete at home and return by mail within 7 days.
In the assessment of construct validity, the principal components method of factor analysis identified 5 factors with eigenvalues greater than 1, which together accounted for 64% of the total variance (patients' needs in the domains of psychologic, health system and information, physical and daily living, patient care and support, and sexuality). Face and content validity were found to be high following pilot tests and tests of reading ease. Internal reliability coefficients (Cronbach alpha) of all 5 factor-based scales were found to be substantial, ranging from 0.87 to 0.97.
These findings suggest that the Supportive Care Needs Survey provides a reliable and valid index of the global needs of oncology patients. The standardized and widespread application of this instrument is recommended following further refinement and evaluation. [See related article on pages 226−37, this issue.] Cancer 2000;88:217–25. © 2000 American Cancer Society.
Due to the rising incidence of cancer1, 2 and stabilizing mortality rates,2 the experience of living with cancer is increasingly prevalent. Research has shown that a diagnosis of cancer is often associated with high morbidity.3–9 Three primary methodologies have been applied in the study of morbidity outcomes for cancer patients: quality of life, patient satisfaction, and needs assessment.10 While quality-of-life evaluations have gauged the ramifications of the disease for different aspects of life experience,11–14 patient satisfaction surveys have more closely focused on perceived quality-of-care issues.10 In contrast, needs assessment spans both quality-of-life and quality-of-care issues when the impact of disease on patients' needs is assessed. Regardless of methodologies, the common goal is to acquire information about health outcomes pertinent to assisting in the improvement of the quality of patient care and the reduction of cancer-related morbidity.15
However, unlike other methodologies, needs assessments offer a number of advantages.10, 11, 16–19 First, they enable direct assessment of patients' perceived needs for help, thereby allowing a more direct indication of needed resources. Second, they allow the identification of the magnitude of need for help, thereby allowing some prioritization of service needs so that resources can be allocated where the need is most urgent. Third, needs assessment enables the identification of individuals and/or patient subgroups with higher level needs, thereby potentially enabling prevention or at least reduction of problems through appropriate early intervention.16–19
Despite this, the potential of needs assessment appears not to have been met.20, 21 A 1996 review of the psychometric properties of needs assessment tools used with oncology populations and published between 1985 and 1995 revealed that perhaps one of the reasons need assessment has not been commonly utilized is because of a lack of quality tools.21 The review of 71 articles rated each needs assessment tool against 6 criteria: 1) it measures the multidimensional impact of cancer on patients' needs; 2) it directly and comprehensively assesses subjective health-related needs for help; 3) it measures outcomes within a defined temporal context; 4) it demonstrates acceptable reliability and validity of assessment tools and methods; 5) it is user-friendly; and 6) it is system-friendly. The review found that few needs assessment tools met these criteria. A MEDLINE search of the literature published between 1996 and 1998 failed to reveal any advances in this area.
Arguably, one of the most critical of the criteria outlined above is that the needs assessment instrument provide demonstrable evidence of sound psychometric properties, such as validity and reliability.22 Without such evidence, an instrument's capacity to assess accurately and reliably that which it is designed to measure remains uncertain, and the resultant data is untrustworthy.
In practice, reliability is usually defined in terms of the consistency of scores that are obtained on the observed variable.23 An instrument is said to be reliable if it is shown to provide consistent scores on repeated administration or administration in different forms.24 The most popular administration methods are the internal consistency indices of reliability.24 One of the most widely used indexes of internal consistency reliability is coefficient alpha.25
Another critical psychometric property of an instrument is its validity, which refers to the ability of the instrument to measure that which it is intended to measure.23, 26 Some measures of validity include face validity, content validity, and construct validity. Face validity refers to whether items appear to be measuring issues consistent with the purpose of the instrument.27 Content validity directly relates to the comprehensiveness of the scale and can be facilitated by incorporating the perspectives of those who have actually experienced the problem.28 Construct validity refers to the extent to which an instrument measures the abstract concepts or constructs underlying the instrument's factors or subscales.29 This is most commonly assessed using factor analytic techniques.24
This article describes the development and psychometric evaluation of a subjective, self-administered instrument designed to measure the needs of patients with cancer. The instrument was designed to conform to the six criteria outlined above.21 Specifically, the study aimed to 1) assess the face, content, and construct validity of a needs assessment tool; and 2) assess the internal reliability, using Cronbach alpha, of a needs assessment tool.
The purpose of the instrument was to provide a direct and comprehensive assessment of the multidimensional impact of cancer (including assessment of health services and treatment delivery) on the lives of generic cancer patients. It was based on an existing Cancer Needs Questionnaire (CNQ),21, 30 which assesses cancer patients' current needs for help in multiple life areas with demonstrable reliability and validity. A priori estimates of scale constructs were based on the following CNQ constructs: 1) psychologic needs—assessing patients' perceived needs for help with psychologic and emotional issues, including counseling and support; 2) health information needs—information needs pertaining to diagnosis, investigative tests, psychologic issues, family issues, and financial issues; 3) physical and daily living–physical needs and adjustment tasks, including coping with physical symptoms and side effects of treatment, performing usual physical tasks and activities of daily living, and self-management of medical treatment routines and health care at home; 4) patient care and support—support needs of cancer patients in relation to family, friends, and health care professionals; and 5) interpersonal communication—interpersonal relationships and the interactional skills and communication styles of health care providers.
The Supportive Care Needs Survey was adapted from the existing CNQ,21, 30 which contains a total of 52 questions with a 5-point response scale. Thus, each item asked patients to consider their level of need for help with the item by choosing one of the following response options: no need—not applicable; no need—already satisfied; low need; moderate need; or high need. The instructions ask patients to identify needs they may have experienced in the last month as a result of having cancer. This instrument had been pilot-tested during development and its reliability and validity ascertained.21, 30
Using the CNQ as the initial pool of items, the survey was reviewed for appropriateness of content by a team of experts, consisting of oncology specialists, behavioral scientists, and consumer representatives. A sample of 200 patients with cancer were then asked to complete the draft survey and to provide written and verbal comment on its relevance, appropriateness, comprehensiveness and acceptability of items, and ease and comprehensibly of instructions and format. Items felt to be redundant were removed and complex language reduced. On the basis of patient and expert responses, a survey with 60 needs items was constructed. An additional question, stemming from the CNQ, about patients' desire for access to support services and people was also included.
Nine major public cancer treatment centers in an Australian state (New South Wales) participated. All treatment centers provided surgical and medical oncology treatment facilities and six provided radiation treatment facilities.
Respondents consisted of people diagnosed with cancer who were in consecutive attendance at either the surgical oncology department or the medical or radiation oncology outpatients clinics at the participating treatment centers during the survey period. Patient eligibility criteria included being between ages 18 and 85 years, having been diagnosed with cancer at least 3 months prior to the conduct of the study, being able to speak and write English, and being physically and mentally willing and able to complete a written questionnaire. If patients were receiving more than one type of treatment, they were asked to complete the survey only once.
After arriving at the clinic waiting room, each patient was introduced to a trained interviewer by a nurse and verbally informed about the study. After establishing the patient's eligibility to participate, the interviewer provided written information about the survey and asked for the patient's written consent. All eligible, consenting patients were given a Supportive Care Needs Survey and a reply-paid envelope to take home. Patients were asked to complete the questionnaire during the following 48-hour period and return it to the researchers by mail within 7 days. Patients who did not return the questionnaire to the researchers within 10 days received a follow-up reminder telephone call from the interviewer.
The study received approval from the New South Wales (NSW) Cancer Council Human Research Ethics Committee as well as the ethics committees of participating hospitals.
Of the nine participating treatment centers, patients were approached about the study in the surgical oncology department of one center, the radiation oncology departments of the six centers that provided it, and the medical oncology departments of all centers.
A total of 1492 cancer patients were approached. Of these, 122 patients were ineligible for the following reasons: 54 (4%) were too ill to participate, 3 (<1%) were younger than 18 or older than 85 years, 32 (2%) did not comprehend English, and 33 (2%) had already completed the survey. Of the 1370 eligible patients, 1354 (99%) consented to participate and 888 (65%) returned a completed survey. This provided a more than adequate sample size for the conduct of factor analyses, which usually require a 1:5 item:respondent ratio.24 There was no significant difference between consenters and nonconsenters with regard to gender.
Table 1 shows the demographic and treatment characteristics of participating patients in comparison to the NSW population of cancer patients,2 as reported by the NSW Central Cancer Registry. The study sample overrepresented females; breast, bowel, colon, and rectal carcinoma cases; and those between ages 31 and 60 years. Males; prostate carcinoma, lung carcinoma, and skin cancer/melanoma cases; and those ages 71–85 years are underrepresented in the sample compared with the NSW cancer population.
|Characteristics||Surveys sample (n=888)||NSW 1994 (n=26,186)a||P value|
|Primary cancer site|
|Colon and rectum||150||17||3,473||13||<0.001*|
|Treatment received in last moc|
|Surgical removal of cancer||107||11||**|
|Bone marrow treatment||6||<1|
The principal components method of factor analysis was calculated using the SAS program, PROC FACTOR command (SAS Institute, Cary, NC). The principal components model enables the exploration of dimensions underlying patients' perceptions by summarizing data into an optional number of independent factors. The program identified 5 factors with eigenvalues greater than 1, which together accounted for 64% of the total variance. These factors (and the proportion of the total variance explained by each) were identified as follows: 1) psychologic (21%)—needs related to emotions and coping; 2) health system and information (18%)—needs related to the treatment center and for information about the disease, diagnosis, treatment, and follow-up; 3) physical and daily living (10%)—needs related to coping with physical symptoms, side effects of treatment, and performing usual physical tasks and activities; 4) patient care and support (10%)—needs related to health care providers showing sensitivity to physical and emotional needs, privacy, and choice; and 5) sexuality (5%)—needs related to sexual relationships.
The five factors suggest that the scale's underlying constructs correspond closely, although not entirely, to those reported by Foot et al. (1996).30 Examining the relative magnitude of each factor suggests that the psychologic needs items and the health system and information needs items accounted for over one-half of the total variance.
|Psychologic (22 items)||Fears about losing your independence||56|
|Confusion about why this has happened to you||70|
|Feeling bored and/or useless||54|
|Feeling down or depressed||71|
|Feelings of sadness||78|
|Fears about the cancer spreading||78|
|Fears about the cancer returning||74|
|Fears about pain||56|
|Anxiety about having any treatment||64|
|Fears about physical disability or deterioration||66|
|Accepting changes in your appearance||65|
|Worry that the results of treatment are beyond your control||74|
|Uncertainty about the future||78|
|Learning to feel in control of your situation||75|
|Making the most of your time||57|
|Keeping a positive outlook||67|
|Finding meaning in this experience||62|
|Feelings about death and dying||73|
|Concerns about the worries of those close to you||56|
|Changes in usual routine and lifestyle||58|
|Concerns about the ability of those close to you to cope with caring for you||55|
|Health system and information (15 items)||Hospital staff to convey a sense of hope to you and your family||54|
|The opportunity to talk to someone who understands and has been through a similar experience||63|
|To be given written information about the important aspects of your care||76|
|To be given information (written, diagrams, drawings) about aspects of managing your illness and side effects at home||74|
|To be given explanations of those tests for which you would like explanations||81|
|To be adequately informed about the benefits and side effects of treatments before you choose to have them||82|
|To be informed about your test results as soon as possible||81|
|To be informed about cancer that is under control or diminishing (that is, remission)||78|
|To be informed about things you can do to help yourself get well||78|
|To be informed about support groups in your area||61|
|To have access to professional counselling (e.g., psychologist, social worker, counselor, nurse specialist) if you/family/friends need it||66|
|To be treated like a person, not just another case||70|
|To be treated in a hospital or clinic that is as physically pleasant as possible||70|
|To be given choices about when you go in for tests or treatment||68|
|To have one member of hospital staff with whom you can talk to about all aspects of your condition, treatment, and follow-up||70|
|Physical and daily living (7 items)||Pain||64|
|Lack of energy/tiredness||73|
|Not sleeping well||65|
|Work around the home||64|
|Not being able to do the things you used to do||65|
|Patient care and support (8 items)||Waiting a long time for clinic appointments||56|
|Family or friends to be allowed with you in hospital whenever you want||68|
|More fully protected rights to privacy when you're at the hospital||67|
|More choice about which cancer specialist you see||65|
|More choice about which hospital you attend||71|
|Reassurance by medical staff that the way you feel is normal||56|
|Hospital staff to attend promptly to your physical needs||70|
|Hospital staff to acknowledge, and show sensitivity to, your feelings and emotional needs||67|
|Sexuality (3 items)||Changes in sexual feelings||86|
|Changes in sexual relationships||86|
|To be given information about sexual relationships||67|
|No specific factor loadings (4 items)||Talking to other people about the cancer|
|Changes in other people's attitudes and behavior towards you|
|Concerns about your financial situation|
|Concerns about getting to and from the hospital|
Some preliminary evidence for the face and content validity of the needs items was confirmed in the results of the preliminary interviews and pilot study with the separate convenience sample of 200 patients who had various types of cancer. Patients were asked to rate the relevance, comprehensiveness, and acceptability of the items and survey format. All patients in the subsample commented favorably about the comprehensibility of the instrument. When suggestions were made for additional items by two or more respondents, these were incorporated into the instrument. The response format was well received and consistently rated as easy to follow and use. A panel of professional members were also consulted regarding the face validity of the instrument, and all supported the premise that the instrument appeared to measure that which it purported to measure. Overall, the verbal and written comments of both patients and professionals consistently attested to the face validity and content validity of the needs items in the modified instrument.
Tests of reading ease (Flesch-Kincaid formula, conducted using the computerized Rightwriter32 program) indicated that the survey could be readily comprehended by people with a minimal level of education (fourth to fifth grade education). The average length of time to complete the survey was 20 minutes.
Cronbach alpha was used to assess the internal consistency of items comprising each factor. In keeping with the recommended guidelines,33 an alpha coefficient of 0.7 was the criterion standard of internal consistency that the instrument aimed to achieve. As shown in Table 3, the reliability coefficients of all 5 factor-based scales were found to be substantial, ranging from 0.87 for the fifth factor to 0.97 for the first factor.
|Factor||Domain||Cronbach alpha coefficient|
|2||Health system and information||0.96|
|3||Physical and daily living||0.90|
|4||Patient care and support||0.87|
|5||Issues of sexuality||0.87|
The results of this study provide empiric evidence for the acceptability, reliability, and validity of a survey instrument designed to assess the supportive care needs of patients receiving treatment for cancer. The instrument meets most of the requirements for needs assessment instruments, as it measures multiple dimensions of cancer-related needs within a well- defined and recent time frame (1 month), using items generated from patient input and using a singular categoric response scale. Moreover, the self- administered instrument required less than 30 minutes to complete, and was rated as easy to understand and complete by oncology patient respondents. Quantitative tests of reading ease also confirmed the ease of readability and low comprehension requirements of the instrument.
Preliminary evidence was found for the instrument's ability to fulfill methodologic standards for achieving an accurate representation of patients' perceptions of need. Through the factor analytic process, it was possible to demonstrate statistically the different dimensions of need measured by the scale: psychologic, health information, physical and daily living, patient care, and issues of sexuality. These dimensions correspond with past research, which suggests that psychologic,8, 34 daily living,8, 35 and information36–38 issues are important to oncology patients. Four of the five constructs are similar to those underlying the CNQ30. Issues of sexuality is a new construct that evolved following patient consultation during the early stages of item generation. It may reflect changing attitudes toward this aspect of patient morbidity.38 Together, these 5 factors accounted for a psychometrically acceptable proportion of the total variance (64%). Furthermore, evidence for the structural reliability of the Supportive Care Needs Survey was also found, with Cronbach alpha coefficients exceeding 0.8 in all domains.
Although its development was based on the CNQ, the Supportive Care Needs Survey differs from its predecessor in a number of ways. First, following consultation with health professionals and people with cancer, a number of CNQ items were either not included in the Supportive Care Needs Survey or rephrased. Second, some new items were included (in particular, those dealing with issues of sexuality). As a result, despite underlying similarities between the two scales, the Supportive Care Needs Survey was treated as a new scale, and exploratory rather than confirmatory factor analyses were conducted.33
The response format of the survey was found to display considerable precision in measuring the level of perceived need. The survey is able to assess whether an issue has been experienced as a result of having cancer, the prevalence of met and unmet needs, and the magnitude of the unmet need within a simple, reliable 5-point response scale. The current method is conceptually consistent with the recommendation of the National Cancer Institute's working group for the development of methods that would allow measurement of “the relative importance or weight of particular quality of life dimensions for individual patients and incorporating these weights into the assessment of the extent to which quality of life is affected by treatment for cancer.”39 Asking people to indicate the magnitude of their need (low, moderate, or high) provides us with a direct categoric measure of the relative importance of one need item against others. The categoric information derived from the instrument also yields a useful quantitative data set that would facilitate ease of comparisons within and between groups as well as across time.10
Despite the complexity and wealth of information that the survey has the potential to yield, the findings show that oncology patients in this sample found the instrument easy to complete. The results attested to the ability of patients to self-administer the survey successfully without external assistance. Self-administered questionnaires have advantages over other interview or card-sort methods; they are relatively inexpensive to produce, require minimal time and energy input from staff, and yield quantifiable responses.10, 40 The ease of analyzing the data statistically is especially important to enable ongoing, efficient analyses of the changing needs of individual patients as a function of disease, treatment, intervention, or naturalistic factors.
The ability of the survey to assess the magnitude of needs may be especially important in clinical oncology contexts. By examining the unmet needs most commonly rated as “moderate” or “high” by different individuals, the survey could potentially be used as an assessment tool to alert practitioners of the need to connect patients with higher level needs to appropriate support resources.41–44 As group data is accumulated for patients from common diagnostic, stage, treatment, or sociodemographic groups, it may become possible to predict who will be risk of developing particular needs. In this way, the survey may be employed in a preventive capacity.
The ease of self-administration of the survey negates the need for expensive and time-consuming personal interviews,10, 40 thus increasing the practical feasibility and acceptability of conducting assessments of patient needs on an ongoing basis as a routine part of care. More recent trials have employed the survey using an interactive computer method, further decreasing expense and improving ease of use. The computer application has been found to be acceptable to patients with cancer in the treatment setting45 and shows promise as a quality assurance or patient education tool. For example, programs that incorporate patient and provider printouts of health status and advice are currently under evaluation.45
This article reports the preliminary evaluation of a new instrument that assesses the needs of people with cancer. Without doubt, there is room for further refinement of the instrument, including reduction of its length, continued testing with more diverse populations and young adults, and translation into other languages. A form of the instrument tailored specifically to people with terminal cancer is also under development. This instrument was designed to stand alone as a needs assessment tool. It would add depth to broader examinations of needs, quality of life, and other outcomes of interest if used as part of a battery of instruments. Together with other measures, an indication of the relations among patient outcomes may be made. However, before it may be applied in such a manner, it requires significant reductions in length in order not to overburden patients.
Finally, a number of methodologic issues need to be delineated. First, a comparison of the data on the study participants with the 1994 NSW Cancer Registry2 data indicated disparities. These may have been due to the large sizes of both samples, which may be magnifying small differences. Further, the NSW sample included patients attending private hospitals, whereas the current sample consisted of patients visiting public hospitals only. More generally, the patients who completed the survey did not adequately represent those younger than 30 or older than 85 years, were predominantly white, and did not include terminally ill people. Second, the data represents the views of cancer patients in one Australian state and may therefore further limit wider generalizability of the results. Third, the response rate (65%) was considerably lower than the consent rate (99%). The issue of low response rates associated with surveys requiring the mailing back of instruments by respondents has plagued similar research.46–50 Although telephone and in-clinic surveys have been found to produce higher response rates,46 the latter tend to be more costly.47 Different strategies for optimizing response rates of mail-back surveys have been tested, including telephone reminder calls and shortened surveys.48, 49 The study does, however, represent the largest (in terms of sample size) of its type reported in Australia, incorporating the views of patients undergoing various treatments for numerous types of cancer. Also, the length of the survey remains an issue that requires attention. An average of 20 minutes is required to complete the survey, which consists of 61 needs items. The research team is currently in the process of developing a short form of the survey to enhance further its ease of use.
Despite its limitations, the study outlines some of the pragmatic and methodologic advantages of a generic needs assessment tool for patients with cancer. The standardized application of the instrument across treatment centers may potentially yield a valuable pool of data on patients' needs. Such data will invariably prove valuable to policymakers and health care providers who strive toward meeting the needs of patients living with cancer.
The authors thank the staff of the participating hospitals for their assistance and cooperation in carrying out the project, and also Janet Anthes, Adam McLean for his assistance in undertaking this research, and Ian Clare and Ross Corkrey for statistical advice and support.