Development and content validation of the Telenursing Interaction and Satisfaction Questionnaire (TISQ)

Abstract Background Caller satisfaction with telephone advice nursing (TAN) is generally high, and the interaction is essential. However, a valid questionnaire exploring caller satisfaction in TAN with focus on perceived interaction is lacking. Objective To develop and assess content validity and test‐retest reliability of a theoretically anchored questionnaire, the Telenursing Interaction and Satisfaction Questionnaire (TISQ), that explores caller satisfaction in TAN by focusing on perceived interaction between the caller and the telenurse. Methods The study was performed in three stages. First, variables relevant for patient satisfaction in health care were identified through a literature search. Variables were then structured according to the Interaction Model of Client Health Behavior (IMCHB), which provided theoretical guidance. Items relevant for a TAN context were developed through consensus discussions. Then, evaluation and refinement were performed through cognitive interviews with callers and expert ratings of the Content Validity Index (CVI). Finally, test‐retest reliability of items was evaluated in a sample of 109 individuals using intraclass correlation coefficients (ICC). Results The TISQ consists of 60 items. Twenty items cover perceived interaction in terms of health information, affective support, decisional control and professional/technical competence. Five items cover satisfaction with interaction and five items overall satisfaction. Remaining items reflect singularity of the caller and descriptive items of the call. The TISQ was found to exhibit good content validity, and test‐retest reliability was moderate to good (ICC = 0.39‐0.84). Conclusions The items in the TISQ form a comprehensive and theoretically anchored questionnaire with satisfactory content validity and test‐retest reliability.


| INTRODUC TI ON
The field of telephone advice nursing (TAN) has expanded rapidly in western countries during the past decade, 1 and for many patients, the interaction with the nurse is the first contact with health care.
The easy access to professional advice in health matters is perceived as a reliable asset in daily life. 2 Research has provided support for its benefits, 1,3 and the service continues to grow.
In TAN, the interaction between the caller and the telenurse takes place during a relatively short and limited amount of time and is predominantly based on verbal communication. The interaction could further be described as a fundamental base within which the nursing process is accomplished. 4 In a recent concept analysis within a nursing care context, 4 it is suggested that nurse-patient interactions consist of following attributes: an overall aim towards facilitation of health; verbal or non-verbal exchange; dynamic adaptation; and multi-dimensionality such as physical, psychological, social or spiritual dimensions. The interaction and its meaning is perceived uniquely by each patient and nurse, and factors influencing the perception include health concerns, knowledge, interpersonal style, setting and expectations, as preferences for how the interaction will proceed.
Components of the interaction process and how they relate to outcomes such as patient satisfaction are described in the Interaction Model of Client Health Behavior (IMCHB) by Cox 5 (Figure 1). The object of this model is to 'identify and suggest explanatory relationships between client singularity, the client-provider relationship and subsequent client health-care behaviour'. 5 The model is generic for nursing purposes but according to its originator most useful in nursing situations when the client's personal responsibility and control of the health problem is large and the role of the health-care professional is more of an advisor, teacher or technician. 5 The IMCHB describes the interaction process as a major influence on health-care outcomes such as satisfaction. Four components define the content of the interaction process: health information; affective support; decisional control; and professional/ technical competence of the nurse. The professional nurse should ideally tailor the interaction with the patient depending on factors relating to the unique client and his or her expressed need for health care (client singularity), also described as a the dynamic qualities of the interaction by Evans. 4 Thus, the four components of interaction in the IMCHB work towards achieving health outcomes in terms of further use of health-care services, change in clinical health status, change in severity of the health-care problem, adherence to recommended care regimen and satisfaction with care. 5 High patient satisfaction rates have been considered a desired outcome and even a component of quality of care itself. 6 It is also considered a predictor of future behaviour. 7 In spite of the relatively large number of studies on patient satisfaction, according to Batbaatar et al 8 there is still no widely adopted definition of the concept within a health-care context, and study results trying to detect its determinants within health care are inconclusive and sometimes contradictory. The following is one way the nursing field defines pa- Affability refers to interpersonal manners of the medical staff, ability to health-care professional or technical quality, and availability to accessibility issues. According to a literature review by Batbaatar et al, 11 interpersonal care quality is the most important factor that influences satisfaction with care.
Since the general shift towards increased patient influences in health care, patient satisfaction has been widely studied and a large number of surveys to measure the trait have been developed.
Criticism of these measures includes a lack of conceptualization, low standardization, low reliability and uncertain validity, 12 which prevent meaningful comparisons between existing satisfaction assessments. Measures of patient satisfaction have been used interchangeably with measures of perceived service quality, a fact criticized by Gill and White, 12 who call for a separation of the two concepts. In a systematic review by Allemann Iseli et al, 13 16 published instruments measuring patient and caller satisfaction with out-of-hours services and teleconsultation and triage were examined. A majority of the reviewed instruments showed limitations in methodology and insufficient evaluation. For instance, only a few of the 16 instruments provided detailed information on item generation and content validation methodology, 13 which reduces possibilities to assess usability in other contexts.
In TAN, reported satisfaction with calls is generally high, 2,14 but, as described above, the degree of satisfaction is not necessarily a measure of high quality of care. It could, for example, be the result of low expectations and is affected by gender and age, as described by Chow et al 10 Parallel to this, there is in literature on TAN a documented need for improvements in health-care quality in terms of telenurses' communication competence, 15,16 and it has been suggested that patient satisfaction surveys designed for a TAN context should monitor improvements in telenurses' communication competence. 17 To our knowledge, there is no survey available that examines both the perception of and the satisfaction with the different parts of the interaction with the telenurses accompanied by the large number of potential influencing variables presented in the IMCHB. Thus, there is a need for a thoroughly developed questionnaire enabling systematic investigations on interactional matters, how they are perceived by callers and how they correlate to caller satisfaction. For content validity reasons, transparency in the development and validation process of such a questionnaire is needed. 18 Therefore, the aim of this study was to develop and assess content validity and test-retest reliability of items of a theoretically anchored questionnaire, the TISQ, that explores caller satisfaction in TAN with focus on perceived interaction between the caller and the telenurse.

| ME THODS AND RE SULTS
In this study, the person who makes the phone call is referred to as 'the caller' and could be either the patient or a person calling on behalf of the patient. All aspects of perceptions and satisfaction in this study refer to the person participating in the interaction with the telenurse, whether or not he or she is the patient.
The process of developing the Telenursing Interaction and Satisfaction Questionnaire (TISQ) was divided into three stages: development and judgement quantification, as suggested by Lynn,19 and evaluation of test-retest reliability. 20 In the first stage, a literature search was accomplished to identify the domain of satisfaction in TAN. Item generation was performed. 21 In the second stage, judgement quantification, the process was separated into two phases: cognitive interviews with callers 22 and evaluation by experts using the Content Validity Index (CVI). 23,24 The results from cognitive interviews and the CVI guided revisions of the entire questionnaire. In the third stage, test-retest reliability of items on perceived interaction and satisfaction was evaluated using intraclass correlation coefficients (ICC). The process is illustrated in Figure 2.

| Stage 1: development
Identification of the domain (steps 1a-d; Figure 2), item generation (step 1e; Figure 2) and assimilation of items into a useable form (step 1f; Figure 2) were performed, and content coverage was analysed (step 1g; Figure 2). The domain was therefore identified and defined by the structure of the existing theoretical model, with one exception: the items representing satisfaction were separated into two subcategories that were not present in the IMCHB-overall satisfaction and satisfaction with interaction.

| Item generation and assimilation of items into useable form
The next step in the development stage was converting variables into items. Wording was discussed in the author group with respect to interpretability in terms of reading level requirements, ambiguity, double-barrelled wording, jargon, value-laden words, and positive and negative wording. Options for response alternatives were discussed until a consensus was reached. Effort was put into ensuring a possible response alternative for every respondent and situation.
Items were then assembled into a usable form.
Content coverage 21 was checked according to the headings in IMCHB. Every subheading of client singularity (background and dynamic variables) and client-profession interaction was represented by at least one item in the questionnaire. Health outcome was represented by items on satisfaction, and other outcome variables were excluded. Content coverage was also checked in relation to a previously developed telenursing communication self-assessment tool 25 in order to ensure that aspects of nursing communication competence and phases of the nursing process were adequately covered.
This first version of the TISQ consisted of 75 items.

| Stage 2: judgement quantification
Content validity and understandability were evaluated from both caller and expert perspectives. First, cognitive interviews with callers were performed (steps 2a-c; Figure 2). Then, content validity was evaluated from a professional point of view using the Content Validity Index (CVI) (steps 2d-f; Figure 2). Revision of the questionnaire was guided by the results from both methods. Transcriptions were used to support revisions to improve the questionnaire. Miscomprehensions of wording and entirety were revealed, as were problems with memory recall, motivation and response processes. Further refinement of items, response options, headlines and instruction texts were discussed within the author group, and revisions were made with respect to the IMCHB. In all, six items were deleted due to perceived similarities and irrelevance: one on client singularity; one on expectations on support; one on overall satisfaction; and three on perceived affective support. One item on estimated number of previous calls to this service was added, and 35 items were reworded. The order of the questions was revised with respect to caller comments. After this refinement, the TISQ consisted of 70 items. Health outcome-satisfaction with care Satisfaction (with interaction) Overall, how satisfied were you with the advice and information you were given?  Table 1. The CVI of the entire questionnaire based on the experts' ratings was 0.92. In addition, scale CVI Average (S-CVI/Ave) was calculated as the mean of all I-CVIs. The S-CVI/Ave of the TISQ was also 0.92, which is above the acceptable level of 0.9. 23 No further evaluation of CVI was performed after the revision.

| Evaluation by experts-content validity index
Information letters to respondents and instruction texts were also revised due to expert comments as were headlines, response options and sequencing of items. All revisions were made after reaching consensus within the author group. Also, no revisions were implemented before checking in accordance with the IMCHB and results from previous stages in the development process.

| Stage 3: test-retest reliability of items on perceived interaction and satisfaction
For evaluation of test-retest reliability of items on perceived interaction and satisfaction (stage 3; Figure 2 (Table 1).

| The final version of the Telenursing Interaction and Satisfaction Questionnaire (TISQ)
After this revision process, the TISQ consisted of 60 items: 23 on client singularity, 20 on perceived interaction, ten on satisfaction and an additional seven items on the description of the call.
The items in the TISQ are sorted into four separate sections.
The first section includes items on the caller's appraisal of the situa- The fourth section includes descriptive items about the specific call (result of the call, timing, if the caller called on behalf of someone else, waiting time, preventive counselling and whether the call was carried out in Swedish or another language) and the caller's demography (sex, age, education, daily occupation, household economy, native tongue and general health condition).

| D ISCUSS I ON
This study describes the thorough process of developing a theoretically anchored content valid questionnaire exploring callers' perceptions of the interaction with the telenurse and caller satisfaction.
This is, to our knowledge, the first comprehensive questionnaire focusing on caller satisfaction and interaction between the caller and the telenurse. It derives from an identification of the domain and is structured according to the IMCHB, a nursing model that recognizes the interaction process as vital for health outcomes such as satisfaction.
The main purpose of the TISQ is to enable systematic investigations on interactional matters, how callers perceive these matters and how these matters correlate to caller satisfaction. Therefore, all potential influencing variables must be represented. The TISQ will not provide multi-item scales for measurement of satisfaction  Existing theory on determinants to patient satisfaction is, as described, complex and somewhat diverging. 11 The IMCHB by Cox 5 was chosen to provide a theoretical and sufficiently complex foundation for the content of the TISQ that at the same time provided guidance to identification of domains. Research studies have suggested the IMCHB to be a useful and comprehensive guide in nursing research. 27,28 In addition, the focus on interactional matters in the IMCHB is well adapted for the purposes of the TISQ. When using the model, it is recommended to focus on one or two of the outcomes, 27 which is the case in the TISQ, where all outcomes except satisfaction have been excluded.
One of the methodologies for judgement of the questionnairecognitive interviews with callers-added insight in addressing concerns experienced by the callers. This perspective is valuable for content validity reasons 18 but is also of importance for the future respondents' motivation to complete the questionnaire. 29 The verbal probing technique applied in the study gave insight to some beforehand important issues. For example, the interviews supported callers' ability to distinguish between desired and expected care and revealed divergent interpretations of key terms such as "severity", "anxiety" and "result of the call". These are everyday words that the callers most likely would not have reflected on otherwise.
Professional expert input contributed to the validation process through the method of CVI. This method is well documented and widespread in science. 23 It is recommended due to its ease of computation, understandability, focus on agreement of relevance, and provision of both item and questionnaire information. The procedure of letting experts share comments, especially on items with low-rated relevance, was helpful in the revision process as it provided explanations for low ratings and suggestions for revision.
The final version of the TISQ includes one item with I-CVI of 0.7 concerning caller expectations of influencing the result of the call. The issue of expectations as a predictor of satisfaction in TAN has support in theory 11,14,17,30 and is pointed out as being essential in the definition of patient satisfaction by Eriksen. 9 Therefore, this item was kept unrevised in spite of low I-CVI. In the IMCHB, expectations are integrated in client singularity, but this is not represented as one explicit factor. When expectations are not met, the telenurse's communication competence seems to have an important impact on satisfaction. 17 Reasons for met or unmet expectations could derive from the patient's perspective but could also be a result of telenurses' diverging understanding of professional responsibilities. 31 If a telenurse mainly focuses on optimising availability and 'gate-keeping', it is likely that potentially more time-consuming dialogues such as affective supporting and health-promoting dialogues will be avoided and vice versa. According to the instructions to respondents, the second questionnaire was to be completed and returned within 1-2 weeks from the first assessment, but many retest questionnaires were delayed and collected up to 4 weeks after the first assessment. According to Jackson et al, 32  In measurement of patient satisfaction, it is a well-known fact that satisfaction rates tend to be high 14,33 and dissatisfaction only emerges in situations where there are obvious reasons. In an attempt to minimize these routine high satisfaction ratings, fairly detailed items on perceived interaction are in the TISQ directly followed by satisfaction rating(s) on that specific interaction element. The purpose of this approach was to guide respondents into distinguishing between perceived quality of health care and satisfaction 12 and to elicit nuances of satisfaction if possible. As discussed in a review by Sitzia and Wood, 33 item construction in terms of general or detailed items may affect the result of satisfaction reports. There is a risk that respondents will assume questions are basically the same and maintain consistency in their answers, not really reading the questions. Comments on the relatively large number of items were collected from both callers and the group of experts. Nonetheless, callers participating in the cognitive interviews appreciated the opportunity to share a fair picture of the call, which has been described in theory. 29 The choice of a relatively large number of items on perceptions is further supported by Gill and White, 12 and therefore, no further deletion of items was performed at this stage. Parts of the TISQ will be further evaluated in terms of psychometric properties that might support further reduction of items.

| CON CLUS ION
This study describes the thorough process of developing and

ACK N OWLED G EM ENTS
The authors would like to thank all the participating experts and callers for their invaluable input to the development process of the TISQ.

CO N FLI C T O F I NTE R E S T
The authors declare that there is no conflict of interest.

E TH I C A L A PPROVA L
This study was performed according to the Principles of the

DATA AVA I L A B I L I T Y S TAT E M E N T
The data that support the findings of this study are available from the corresponding author upon reasonable request.