SEARCH

SEARCH BY CITATION

Keywords:

  • Questionnaire;
  • vulva disorders;
  • vulval intraepithelial neoplasia

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions
  8. References
  9. Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’
  10. Disclosure of interests
  11. Supporting Information

Objective

To develop and evaluate a questionnaire to assess the burden of vulval intraepithelial neoplasia (VIN) in women.

Design

A questionnaire development study.

Setting

Vulval Disorders Clinic serving a regional population.

Sample

Fifty-eight women with a histological diagnosis of VIN registered with the Vulval Disorders Clinic.

Methods

A 37-item questionnaire was developed through a comprehensive literature review, consultation with specialist clinicians and pretesting to assess the burden experienced by women. The questionnaire was assessed for validity and reliability against existing questionnaires used in related disease areas.

Main outcome measures

Spearman correlations were calculated between items in the VIN questionnaire with the scores of the Dermatology Life Quality Index (DLQI), Hospital Anxiety and Depression Scale (HADS), Sabbatsberg Sexual Self-Rating Scale (SSRS) and the Process Outcome Specific Measure (POSM) to assess the new questionnaire's validity. Internal consistency was measured using Cronbach's alpha. Test–retest reliability was calculated using quadratic weighted kappa.

Results

The VIN questionnaire had a high degree of internal consistency (Cronbach's alpha, 0.89). Test–retest reliability was assessed, with most questions showing a quadratic weighted kappa value of 0.5 or above. Most questions showed a stronger correlation with the corrected total VIN score than with HADS anxiety and depression subscales and the SSRS, indicating discriminant validity. Most questions correlated significantly with the DLQI and POSM scores, indicating convergent validity.

Conclusions

Initial assessment of the VIN questionnaire demonstrated that it is a valid and reliable measure of the burden of disease for women. The questionnaire could be used to compare new and existing treatments for VIN or to assess or monitor the impact of care.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions
  8. References
  9. Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’
  10. Disclosure of interests
  11. Supporting Information

Vulval intraepithelial neoplasia (VIN) is an uncommon premalignant skin condition affecting the vulva. Common symptoms include intense itching, burning or discomfort. The standard management of VIN is local excision of the affected tissue, which can be mutilating and disfiguring, and may be associated with significant psychosexual morbidity.[1]

A number of studies have identified poor health-related quality of life (HR-QoL) and poor sexual functioning in women diagnosed with VIN.[1-8] One prospective cohort study identified poor HR-QoL scores and sexual functioning in women, with some women reporting that the negative impact on sexual relations was the worst aspect of the condition for them.[5]

In a qualitative study, women reported that information about VIN was lacking and unhelpful, that they felt embarrassment, fear and confusion about the condition and their future health, and that sexual function and body image were both affected by VIN.[4]

Previous research has identified a wide range of issues relating to the symptoms, diagnosis, treatment and follow-up for VIN which can have a negative impact on women's lives. Many of these studies have used multiple questionnaires to measure these consequences of VIN [including the Dermatology Life Quality Index (DLQI), Sabbatsberg Sexual Self-Rating Scale (SSRS) and the Hospital Anxiety and Depression Scale (HADS)] because no single questionnaire is available which encompasses the wide range of difficulties experienced by women diagnosed with VIN. The aim of this study was to fill this gap by developing and evaluating a single questionnaire which addresses the issues of concern to women with a biopsy-confirmed diagnosis of VIN. The questionnaire addresses issues covering the symptoms, diagnosis, treatment and follow-up for VIN.

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions
  8. References
  9. Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’
  10. Disclosure of interests
  11. Supporting Information

Participant recruitment

The study population comprised the residents of the Grampian Region in Scotland, which covers the relatively affluent city of Aberdeen and a surrounding rural population. Women were recruited from the Vulval Disorders Clinic based in Aberdeen. A woman was eligible to participate if she had a biopsy-confirmed diagnosis of VIN, and was attending or had attended the Grampian Vulval Disorders Clinic for treatment and/or follow-up for VIN.

Questionnaire development

Questionnaire development was informed by the commonly accepted steps to develop HR-QoL questionnaires,[9, 10] and comprised four phases: (1) the generation of relevant issues; (2) the development of questionnaire items from the issues; (3) the pre-testing of the questionnaire items; and (4) field testing.

Generation of items

A comprehensive review of the literature was undertaken to identify potentially relevant items for inclusion in the questionnaire. Heathcare professionals experienced in the treatment and management of patients with VIN were also consulted to help generate and review items in terms of coverage and relevance.

The literature review and expert consultation generated 30 items, from which a draft questionnaire was developed focusing on six themes: (1) symptoms; (2) daily activities; (3) feelings; (4) sexual relations; (5) relationships; and (6) future health concerns. A copy of the items included in each section is shown in Table 1.

Table 1. Outline of each section in the pretest vulval intraepithelial neoplasia (VIN) questionnaire
SectionContent of each section
Symptoms

Frequency of:

 itch?

 pain?

Daily activities

Has VIN affected:

 choice in clothing?

 social and leisure activities?

 sleeping pattern?

Feelings

Has VIN made you feel:

 anxious, depressed, embarrassed?

 concerned about body image?

Sexual relations

Has VIN negatively affected:

 sexual activity?

 interest in sex?

 enjoyment of sex?

Relationships

Has VIN negatively affected:

 relationships with friends?

 relationships with partner/spouse?

Future health concerns

Have you been concerned about:

 future fertility?

 future treatment for VIN?

 getting vulval cancer?

The question stem ‘due to VIN and/or its treatment’ was repeated throughout the questionnaire to remind women to respond on the basis of their experience of VIN. All questions were time limited to 1 month, so that the responses reflected women's most recent experiences of VIN. Most response options were arranged as a four-point labelled categorical scale, ranging from ‘not at all’, ‘a little’, ‘quite a bit’ to ‘very much’.

Pretesting

The VIN questionnaire was pretested on 10 women attending the Vulval Disorders Clinic. Women were sent two copies of the questionnaire (one marked to be completed and returned and one marked to be kept for reference), a consent form and a prepaid and pre-addressed envelope to return these.

For the purposes of questionnaire development, a debriefing schedule was used to assess the content and design of the questionnaire.[10] The schedule included questions about the acceptability and intrusiveness of questions, how easily understood (or not) were the questions and whether the questionnaire missed out any important issues. The debriefing schedule was administered by the researcher face-to-face at clinic or by telephone as preferred by the participant. The questionnaire was redrafted in the light of the comments made during these interviews.

Field testing

A questionnaire booklet was developed which comprised socio-demographic and background questions, the redrafted VIN questionnaire and a number of other previously validated questionnaires, which had been used in this or related disease areas (see Appendix S1). Women indicating that they were willing to participate were sent the questionnaire booklet, a consent form and a pre-addressed, prepaid envelope to return these to the researcher.

In order to assess test–retest reliability of the VIN questionnaire, women were sent the questionnaire booklet on two occasions, with an interval of 7 days between each. The interval of 7 days was chosen as there was unlikely to be substantial changes in the participants' health in that time, but it would be sufficiently long to minimise participants remembering their responses to the first administration of the questionnaire. For each administration of the questionnaire booklet, a reminder letter was sent to women who did not respond within 2 weeks.

Socio-demographic, background and medical data collected included age, living status, parity, mode of child delivery, contraception and had women ever had treatment for an abnormal cervical smear test. Participants were asked what treatment they had undergone for VIN, which included creams, surgery under local anaesthetic and surgery under general anaesthetic. Laser treatment was included in the surgery under general anaesthetic response. A measure of deprivation was derived for each participant using the Scottish Index of Multiple Deprivation (SIMD). SIMD identifies small area concentrations of multiple deprivations across all of Scotland. The SIMD quintile is a score in the range 1–5, derived by a person's postcode, ‘1’ indicating the most deprived status and ‘5’ the least deprived status.

DLQI, a 10-item dermatology questionnaire, widely used in patients with a variety of skin conditions (including VIN), was employed to measure the negative impact of the disease.[11] The response options are arranged on a four-point categorical scale ranging from ‘not at all’ to ‘very much’. Question responses are summed to yield a total score from zero to 30, with a higher score reflecting greater impairment of HR-QoL.

SSRS is a 14-item questionnaire and was included to measure sexual functioning.[12] Each item is scored on a five-point scale from zero to four, and then summed to produce a total score which is expressed as a percentage, with 100 representing the best self-rated sexual functioning.

HADS was included to detect depressed and anxious mood.[13] It consists of 14 items, comprising two subscales, seven items measuring anxiety and seven items measuring depression. The items are summed, yielding two subscale scores each ranging from zero to 21.

The Process Outcome Specific Measure (POSM), a 15-item questionnaire originally designed to assess the psychosocial impact of the receipt and management of a low-grade abnormal cervical smear result, was adapted to be used for women with VIN.[14] Most response options are arranged on a six-point scale from ‘strongly agree’ to ‘strongly disagree’. The scores of the responses to the questions are standardised to be out of 100, with 100 indicating the highest distress.[15] POSM has been used in women with low-grade abnormal smears, and in women with cervical intraepithelial neoplasia (CIN) and micro-invasive cervical cancer.[16]

Sample size

We anticipated that there was unlikely to be substantial changes in the clinical condition of women with VIN over a 1-week period, and thus responses would be similar in both administrations of the questionnaire; we estimated a reliability coefficient of approximately 0.8.[9] In order to discriminate between groups of participants, it is commonly accepted that reliability should exceed 0.7.[10] Using these figures, our required sample size for reliability testing was 118. Based on previous clinical experience of the recruitment of women with VIN, it was anticipated that 80% of women would agree to participate and complete the questionnaire.[5] Thus, 147 would be invited to participate for field testing.

Scoring the VIN questionnaire

Four filter questions, which allowed women to skip questions which were not relevant to them, were excluded from the scoring. Question 11 (‘Is the reason you answered no to being sexually active because of VIN and/or its treatment?’) and question 18 (‘Are there any reasons why you have not told anyone about VIN and/or its treatment?’) were excluded from the VIN scoring as they had been included to help our understanding of women's responses to the sexual activity and support questions, and were not included as a measure of burden. The global HR-QoL question was also excluded from scoring as it was not VIN-specific; rather, it provided a general measure of HR-QoL. The remaining 30 questions were scored and summed to give a total score for the VIN questionnaire.

Each question was scored from one to two, four or five depending on the number of response categories. All questions were scored so that a higher score indicated a higher level of burden of disease. To overcome the problem of having different numbers of response options in the questionnaire, each question score was standardised to be out of 100, with 100 indicating the highest burden. This was achieved by multiplying an individual's raw score by 100, divided by the highest possible raw score.

Statistical analysis

The questionnaire data from field testing were entered manually and analysed using SPSS for Windows version 19 (IBM Corporation, USA). The score distributions of the questionnaire data from the field testing were examined for skewness, floor and ceiling effects, and missing values.

To assess whether a higher total VIN score corresponds to a higher disease burden, descriptive statistics were calculated to assess whether there was any differences in the total VIN score between women who had undergone different types of treatment. Participants were placed into one of four treatment groups: no treatment, creams, surgery and combined surgery and cream treatment. We might expect those women with a lower disease burden to have had no treatment and those with a higher disease burden to have undergone surgery.

To assess how women responded to the questionnaire, the distribution of the total VIN score was assessed.

Validity

Construct validity, by which we mean the extent to which the questionnaire measured the construct it was designed to measure,[10] was assessed in two ways. First, each question in the VIN questionnaire was correlated with the corrected total VIN score (i.e. the total score omitting the score of each question in turn) in order to assess to what extent the individual items were measuring a single construct. Second, to assess how well the VIN questionnaire was measuring something distinct to that being measured by the other questionnaires, each of the VIN questions was correlated with the total DLQI, SSRS and POSM scores and the HADS anxiety and depression subscale scores.

A priori, we anticipated the VIN questions relating to symptoms and daily activities to be strongly correlated with DLQI, and the question concerning women's future worries about getting cancer to be positively correlated with POSM. We also expected there to be a strong correlation between the total SSRS score and the VIN questions relating to sexual activity. Similarly, we expected only the VIN questions relating to feeling anxious or depressed to be correlated with the HADS anxiety and depression subscale scores.

Reliability

The reliability of the questionnaire was assessed in two ways. Cronbach's alpha was used to measure internal consistency (i.e. the extent to which the items were interrelated) using questionnaires with complete data.[9] Generally, reliablity over 0.7 is required to determine differences between groups of patients.[9] Second, the stability of individuals' responses to the questions over a period of time during which their health status would not be expected to change (i.e. test–retest reliability) was assessed by examining individuals' responses between both administrations of the questionnaire and calculating the level of agreement using quadratic weighted kappa.[9] Values of over 0.4 for quadratic weighted kappa reflect moderate agreement, and over 0.6 reflect good agreement.[9] The reliability of the overall VIN score between the two administrations of the questionnaire was measured using intraclass correlation coefficients.

Throughout all statistical analysis, a two-sided P value of ≤0.05 was used as the threshold for statistical significance.

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions
  8. References
  9. Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’
  10. Disclosure of interests
  11. Supporting Information

Pretesting

Ten women completed the pretest questionnaire and seven participated in the debriefing interviews. Four women commented that, because they were asked to answer in terms of their experiences of VIN in the last month, most of the questions were not relevant to them. One woman found the question concerning vulval cancer to be upsetting as she was unaware of a connection between VIN and vulval cancer. Two women commented that they had not told anyone about VIN, and suggested that it may be useful to explore this further. One woman commented that the questionnaire did not address concerns about starting a new relationship because of VIN. Six questions were added as a result of pretesting:

  1. ‘Due to VIN and/or its treatment, have you felt anxious about having sex in the last month?’
  2. ‘In the last month, due to VIN and/or its treatment, have you felt anxious about starting a relationship with a new partner?’
  3. ‘Have you told anyone about VIN?’
  4. ‘Are there any reasons why you have not told anyone about VIN?’
  5. ‘On a scale of 1–5, where 1 is no itch/irritation and 5 is extreme itch/irritation, how severe has the itch/irritation been in the last month?’
  6. ‘On a scale of 1–5, where 1 is no pain/discomfort and 5 is extreme pain/discomfort, how severe has the pain/discomfort been in the last month?’

A number of positive comments were made about the questionnaire, including that it was clear, concise and easy to understand. The questionnaire was quick to complete, taking under 10 minutes. All women made positive comments about the recognition of the issues experienced by women with VIN.

Field testing

From the 147 women invited to participate, 70 were recruited (Figure 1). In total, 58 women returned the first administration of the questionnaire. The socio-demographic and background characteristics of the participants are shown in Table 2.

Table 2. Socio-demographic, lifestyle and health characteristics of the study sample
 n = 58Valid%
  1. SIMD, Scottish Index of Multiple Deprivation; VIN, vulval intraepithelial neoplasia.

Age group (years)
30–39712
40–491628
50–591730
60–691323
>7047
Missing1
Marital status
Single814
Partnered611
Married2543
Divorced1322
Widowed610
Living circumstances
Alone2137
Partner/spouse3256
Other family47
Missing1
SIMD quintile
1 (most deprived)35
2814
31526
41628
5 (least deprived)1527
Missing1
Ever had children
Yes5086
No814
Number of children ( n  = 50)
11428
21938
31020
4+714
Contraception
None3563
Hormonal contraceptives712
Barrier contraceptives24
Sterilisation & partner – vasectomy1221
Missing2
Treatment for VIN
None24
Creams610
Surgery3460
Creams and surgery1526
Missing1
Sexually active
Yes2238
No3662
Treatment for an abnormal cervical smear test
Yes2239
No3155
Not sure36
Missing2
image

Figure 1. Flow chart of the invitation and recruitment process, and the response rates of both administrations of the questionnaire.

Download figure to PowerPoint

The mean age of the participants was 53 years (range, 30–82 years). Most women were married (43%), lived with a partner or spouse (56%), were not sexually active (62%) and had had children (86%). Over half (55%) of women were categorised into the SIMD quintiles of ‘4’ or ‘5’ (indicating the least deprived status). The majority of women had undergone surgical treatment for VIN (86%).

Score distributions

The score distributions of the VIN questionnaire were positively skewed, clustering around the bottom of the scale (indicating floor effects). Similar score distributions were also observed for the DLQI and SSRS questionnaires. Levels of missing data for most questions in the VIN questionnaire were low (≤5%). The exception was the question on concerns about having future treatment for VIN, with 8% missing data.

Table 3 highlights the descriptive statistics of the total VIN score for the different treatment groups. There were no differences in the VIN total score across the different treatment groups (P = 0.09). With regard to the total VIN score of all participants, the majority of women scored low [mean, 28.6 (standard deviation, 13.8); median, 24.5 (interquartile range, 19–34)].

Table 3. The different mean, standard deviation (SD), median and interquartile range (IQR) values of the total vulval intraepithelial neoplasia (VIN) score for the different treatment groups (n = 57)
Treatment(n = 57)MeanTotal VIN scoreIQR
 SDMedian 
None221.910.921.914–(–)
Creams631.712.027.323–44
Surgery3424.79.822.117–33
Creams and surgery1537.618.732.220–58

Item total correlations

Item total correlations for the VIN questionnaire and the correlations between the individual VIN questions and the total scores of the DLQI, SSRS, POSM, and HADS anxiety and depression subscales are shown in Table 4.

Table 4. Item total correlations of the vulval intraepithelial neoplasia (VIN) questions with the corrected total VIN score and the Sabbatsberg Sexual Self-Rating Scale (SSRS), Hospital Anxiety and Depression Scale (HADS) anxiety (A) and depression (D) subscale, Dermatology Life Quality Index (DLQI) and Process Outcome Specific Measure (POSM) total scores
VIN question n c Corrected VIN scoreSSRSHADS (A)HADS (D)DLQIPOSM
  1. Questions 1, 10, 17 and 25 were filter questions. Questions 11 and 18 were excluded as these questions were included to help our understanding of women's responses to the sexual activity and support questions.

  2. a

    P < 0.05.

  3. b

    P < 0.01.

  4. c

    Number of women who contributed to the analysis of the corrected total VIN score.

2Frequency of itch/irritation?580.54b−0.090.230.230.62b0.30a
3Severity of itch/irritation?580.48b−0.170.170.170.59b0.26a
4Frequency of pain/discomfort?580.57b−0.030.200.190.62b0.29a
5Severity of pain/discomfort?570.54b0.070.160.150.56b0.23
6aHas VIN affected your choice in clothing?580.39b−0.140.190.180.46b0.11
6bHas VIN affected your social and leisure activities?580.53b−0.060.33a0.30a0.50b0.25
6cHas VIN affected your daily activities?580.48b−0.020.30a0.30a0.45b0.36b
6dHas VIN negatively affected your work?580.54b−0.040.27a0.35b0.58b0.29a
6eHas VIN negatively affected your sleeping pattern?580.52b−0.020.27a0.28a0.61b0.31a
7aHave the symptoms, diagnosis or treatment of VIN made you anxious?580.60b0.040.33a0.240.47b0.34a
7bHave the symptoms, diagnosis or treatment of VIN made you depressed?580.44b−0.170.40b0.39b0.44b0.40b
7cHave the symptoms, diagnosis or treatment of VIN made you embarrassed?580.38b−0.100.28a0.250.43b0.27a
8Has VIN negatively affected your confidence?580.64b−0.170.57b0.50b0.67b0.53b
9Have you been concerned about the appearance of your body?580.58b−0.180.44b0.43b0.57b0.43b
12Has the frequency of your sexual activity been altered?580.51b0.64b0.29a−0.060.210.46b
13Has sex been painful in the last month?580.53b0.62b0.27a−0.060.220.44b
14Has your interest in having sex reduced?580.52b0.60b0.28a−0.040.210.46b
15Has your enjoyment of sex been negatively affected?580.54b0.60b0.30a−0.020.230.47b
16Have you felt anxious about having sex?580.54b0.60b0.27a−0.040.230.45b
19Has VIN negatively affected your relationship with your partner/spouse?580.33a0.220.210.110.38b0.39b
20Has VIN negatively affected your relationship with work colleagues?570.070.36a0.00−0.24−0.090.02
21Has VIN negatively affected your relationship with friends?580.32a0.120.140.080.130.21
22Have you felt anxious about starting a relationship with a new partner?580.17−0.170.220.250.170.17
23Have you had concerns about finding a long-term partner?570.07−0.080.150.000.030.12
24Have you had enough support from friends and/or family?550.41b0.170.260.170.240.19
26aHave you had concerns about your future fertility?580.230.210.240.050.030.19
26bHave you had concerns that you might have problems with childbirth?580.230.200.240.060.040.20
27aHave you had concerns about having future treatment for VIN?530.38b−0.050.39b0.40b0.30a0.21
27bHave you had concerns about your next check up at the vulval clinic?560.21−0.140.36b0.45b0.27a0.28a
27cHave you had concerns about getting vulval cancer?550.37b−0.120.220.27a0.200.24

All but six of the questions in the VIN questionnaire were correlated significantly with the total VIN corrected score (P < 0.05), with correlation coefficients ranging from 0.32 to 0.64. The exceptions were question 20 (‘Has VIN negatively affected your relationship with work colleagues?’) (r = 0.07), question 22 (‘Have you felt anxious about starting a relationship with a new partner?’) (r = 0.17) and question 23 (‘Have you had concerns about finding a long term partner?’) (r = 0.07).

Other questions which did not show statistically significant correlations were the questions concerning children (question 26a, ‘Have you had concerns about future fertility?’; question 26b, ‘Have you had concerns that you might have problems with childbirth?’), both of which had correlation coefficients of r = 0.23. Lastly, question 27b (‘Have you had concerns about your next check up at the vulval clinic?’) was not correlated significantly with the corrected total VIN score (r = 0.21).

Although there were some statistically significant correlations between the VIN questions and the total scores of the other questionnaires, most of the VIN questions correlated most strongly with the corrected total VIN score.

The VIN questions which related to symptoms and activities of daily life correlated strongly with the DLQI questionnaire, with correlations ranging from 0.45 to 0.62. The SSRS questionnaire correlated strongly with the VIN questions relating to sexual activity (correlations ranging from 0.60 to 0.64). The POSM questionnaire correlated positively with questions relating to concerns about getting cancer in the future (r = 0.24).

The correlations between the total VIN score and the total scores of the existing questionnaires were examined. The VIN questionnaire score was statistically significantly correlated with the DLQI (r = 0.69), POSM (r = 0.57) and HADS anxiety (r = 0.50) and depression (r = 0.27) total scores, but not with the SSRS questionnaire (0.19).

For the VIN questionnaire, Cronbach's alpha was 0.89, indicating an excellent degree of internal consistency. This value was similar to those for the DLQI (0.93), SSRS (0.97), POSM (0.69) and HADS anxiety and depression scales (0.89 and 0.89, respectively).

Test–retest reliability

The results of the test–retest reliability for the VIN questionnaire are shown in Table 5.

Table 5. Results from the test–retest reliability of the vulval intraepithelial neoplasia (VIN) questionnaire
VIN question number Quadratic weighted kappa (κ)n (number of respondents answering question both times)
  1. Question 18 was excluded from the analysis as this question was to be reported descriptively.

  2. The varying numbers are a result of some questions not being applicable to some women, and some questions contained missing data.

  3. a

    Only included women who responded positively to preceding filter questions (1, ‘Have you had any symptoms relating to VIN and/or its treatment in the last month?’; 10, ‘Are you sexually active?’; 25, ‘Do you intend to have children in the future?’).

  4. b

    Only included women who responded negatively to preceding filter questions (11, ‘Is the reason you answered no to being sexually active because of VIN and/or its treatment?’).

1Experienced symptoms?0.7532
2Frequency of irritation?0.8115a
3Severity of itch/irritation?0.3415a
4Frequency of pain/discomfort?0.5415a
5Severity of pain/discomfort?0.1415a
6aHas VIN affected your choice of clothing?0.4032
6bHas VIN affected your social and leisure activities?0.7832
6cHas VIN affected your daily activities?0.7332
6dHas VIN negatively affected your work?0.7832
6eHas VIN negatively affected your sleeping pattern?0.5232
7aHas VIN made you feel anxious?0.2932
7bHas VIN made you feel depressed?0.6732
7cHas VIN made you feel embarrassed?0.6732
8Has VIN negatively affected your confidence?0.6232
9Has VIN made you feel concerned about the appearance of your body?0.5332
10Are you sexually active?0.8632
11Reason answered no to being sexually active because of VIN?0.8320b
12Frequency of sexual activity?0.5311a
13Has sex been painful due to VIN?0.3911a
14Has your interest in sex reduced in the last month due to VIN?0.8211a
15Has your enjoyment of sex been negatively affected due to VIN?0.7111a
16Have you felt anxious about having sex in the last month due to VIN?0.5011a
17Have you told anyone about your VIN?0.9132
19Has VIN negatively affected your relationship with partner/spouse?0.7031
20Has VIN negatively affected your relationships with work colleague?0.7432
21Has VIN negatively affected your relationships with friends?0.7232
22Has VIN made you feel anxious about starting a new relationship?0.5532
23Have you had concerns about finding a long-term partner?0.8132
24Do you feel you have had enough support from friends and/or family?0.8332
25Do you intend to have children in the future?130
26aHas VIN given you concerns about future fertility?13a
26bHas VIN given you concerns about having problems with childbirth?13a
27aHave you had concerns about having future treatment for VIN?0.6730
27bHave you had concerns about your next check up at the vulval clinic?0.5230
27cHave you had concerns about getting vulval cancer?0.5829

All but five questions showed levels of reliability above 0.5 using quadratic weighted kappa. The exceptions were question 3 (‘On a scale of 1–5, where 1 is no itch/irritation and 5 is extreme itch/irritation, how severe has the itch/irritation been in the last month?) and question 5 (‘On a scale of 1–5, where 1 is no pain/discomfort and 5 is extreme pain/discomfort, how severe has the itch/irritation been in the last month?), with quadratic weighted kappa values of 0.34 and 0.14, respectively.

Three other questions displayed lower levels of agreement: question 6a (‘In the last month, has VIN and/or its treatment affected your choice of clothing?’) (κ = 0.40), question 7a (‘Has VIN made you feel anxious?’) (κ = 0.29) and question 13 (‘Due to VIN and/or its treatment, has sex been painful in the last month?’) (κ = 0.39). There was good agreement of the overall VIN scores between the two administrations of the questionnaire (intraclass correlation coefficient, 0.91; confidence interval, 0.79–0.96).

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions
  8. References
  9. Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’
  10. Disclosure of interests
  11. Supporting Information

Main findings

Responses to questions which show a skewed distribution (where most responses are at the bottom or top of the scale) are known as floor and ceiling effects, respectively.[10] The responses to the VIN questionnaire were positively skewed, indicating floor effects.

Cronbach's alpha for the VIN questionnaire was 0.89, which compared favourably with the internal consistency of each of the subscales of HADS (0.89 for the anxiety subscale and 0.89 for the depression subscale), SSRS (0.97), POSM (0.69) and DLQI (0.93). A value of greater than 0.7 is considered to be acceptable for these types of scale.[10]

The questionnaire seemed, for the most part, to be measuring a single construct, with all but three questions having corrected item total correlations above 0.2, which is the generally accepted cut-off level for keeping or discarding questionnaire items.[10] The exception was question 22 (‘In the last month, due to VIN and/or its treatment, have you felt anxious about starting a relationship with a new partner?’), which had been included after pretesting.

SSRS, DLQI and HADS have all been used in women with VIN in previous studies.[2, 5, 17] Although POSM was used for women with CIN, it also addressed issues of importance to women with VIN. Thus, we expected some degree of correlation between the VIN questions that were intended to measure similar constructs and those measured by these questionnaires. Our findings were consistent with our expectations, in that there were statistically significant correlations between some of the VIN questions and the SSRS, DLQI and POSM total scores and the HADS subscale scores (demonstrating convergent validity). However, the magnitude of the correlations between the individual VIN questions and the corrected total VIN score was generally greater (for most questions) than the correlation with the total scores of the HADS anxiety and depression subscales and other questionnaires (demonstrating discriminant validity).

Interpretation

The floor effects observed for the VIN questions could indicate that some questions were not relevant, but this conflicts with our findings from the literature and expert views. Instead, this may reflect the possibility that women have not experienced the problems described in the questionnaire in the time frame suggested (i.e. in the last month).

There were no differences in the VIN scores in women across the different treatment groups. Women with no treatment may have co-morbidities or may not have embarked on treatment, which may be reflected in a higher VIN score. Conversely, the use of topical cream with or without surgery may relate to extensive disease; therefore, we would expect a high score in these patients. The treatment groups are small and therefore the results should be interpreted with care.

With the exception of question 22, all questions had item total correlations of over 0.2. This question addressed whether women felt anxious about starting a relationship with a new partner. This issue had not been identified in the literature review or expert consultation, which may indicate that this issue is a concern for only a minority of women.

The high Cronbach's alpha result (0.89) suggests that the VIN questions may measure one construct and the questionnaire could be used as a scale. There was a strong positive relationship between the VIN questionnaire and DLQI and POSM. The HADS anxiety subscale correlated strongly with the overall VIN score (r = 0.50). Intuitively, we would expect that the higher the disease burden, the more anxious are the women. This is consistent with results from a similar VIN study, which found that increased anxiety was associated with worsening emotional and general health.[2]

Strengths and weaknesses

To our knowledge, this is the first study to develop and test a questionnaire to measure the negative impact of VIN. It can be completed quickly (generally taking less than 10 minutes), administered by post or in a clinical setting, and is relatively easy to score. The questionnaire underwent a rigorous developmental process, informed by previous research and by expert and patient consultation. Semistructured debriefing interviews identified problems and ambiguities in the questionnaire, and provided a further assessment of content validity.

The questionnaire has only been tested in the Grampian region; therefore, it is geographically limited. Most of the women participating in the study had been treated for VIN and were being followed up; therefore, the results may not be generalizable to women newly diagnosed with VIN. The study did not meet the estimated 118 participants needed for reliability testing, thereby limiting the robustness of the findings. The response rate for the first administration of the questionnaire was 40%, lower than the 80% achieved in a previous study,[5] and the 54% postal response rate achieved by a similar study in the VIN population.[2] The lower than expected response rate could have been caused by respondent fatigue, as the questionnaire booklet contained 96 questions. The lower response rate may indicate some degree of responder bias. We did not achieve the number for reliability statistics, and thus we may have under- (or over-) estimated the reliability of the questionnaire.

Conclusions

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions
  8. References
  9. Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’
  10. Disclosure of interests
  11. Supporting Information

We have developed a questionnaire which addresses issues experienced by women with VIN. The questionnaire requires further testing with a larger sample size, which is likely to require multiregional involvement. With a larger sample size, a factor analysis could be performed to assess whether the questionnaire measures one or more distinct constructs. Furthermore, the questionnaire could be tested with newly diagnosed women, and before and after treatment of VIN. The questionnaire is being tested in women with other vulval skin conditions (e.g. lichen sclerosus) to help to validate the questionnaire and to enable a comparison of vulval skin conditions in terms of the burden experienced by women.

The questionnaire is reliable with good convergent and discriminant validity. It could be used to monitor patients in the clinic and to compare new and existing treatments for VIN in terms of the burden experienced by women.

Disclosure of interests

None.

Contribution to authorship

JL performed the review of the literature, generated items for inclusion in the questionnaire and constructed the questionnaire. JL performed the debriefing interviews and posted out the questionnaires. JL analysed the data and contributed to the writing up of the work. NG helped to develop and write the project protocol and to construct the questionnaire, provided advice about the analysis and contributed to the writing up of the work. MC developed and wrote the project protocol, helped to recruit women for the study, provided advice about the questionnaire and analysis plan and contributed to the writing up of the work.

Details of ethics approval

This study was given ethical approval by the North of Scotland Research Ethics (NRES) Committee 2 (25 October 2011 – Reference: 11/NS/0038). A copy of the questionnaire was submitted to Committee 2 before field testing began.

Funding

None.

Acknowledgements

We are grateful to all the women who participated in this study.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions
  8. References
  9. Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’
  10. Disclosure of interests
  11. Supporting Information
  • 1
    Likes WM, Stegbauer C, Tillmanns T, Pruett J. Pilot study of sexual function and quality of life after excision for vulvar intraepithelial neoplasia. J Reprod Med 2007;52:237.
  • 2
    Shylasree TS, Karanjgaokar V, Tristram A, Wilkes AR, MacLean AB, Fiander AN. Contribution of demographic, psychological and disease-related factors to quality of life in women with high-grade vulval intraepithelial neoplasia. Gynecol Oncol 2008;110:1859.
  • 3
    Likes WM, Russell C, Tillmanns T. Women's experiences with vulvar intraepithelial neoplasia. J Obstet Gynecol Neonatal Nurs 2008;37:6406.
  • 4
    Cruickshank M, McFadden K. Report from VIN Qualitative Study. BSSVD Biennial Meeting, Cardiff, 2007.
  • 5
    McFadden KM, Sharp L, Cruickshank ME. The prospective management of women with newly diagnosed vulval intraepithelial neoplasia: clinical outcome and quality of life. J Obstet Gynaecol 2009;29:74953.
  • 6
    Thuesen B, Andreasson B, Bock JE. Sexual function and somatopsychic reactions after local excision of vulvar intra-epithelial neoplasia. Acta Obstet Gynecol Scand 1992;71:1268.
  • 7
    Andersen BL, Turnquist D, LaPolla J, Turner D. Sexual functioning after treatment of in situ vulvar cancer: preliminary report. Obstet Gynecol 1988;71:159.
  • 8
    Likes WM, Stegbauer C, Hathaway D, Brown C, Tillmanns T. Use of the female sexual function index in women with vulvar intraepithelial neoplasia. J Sex Marital Ther 2006;32:25566.
  • 9
    Fayers PMD. Quality of Life. Assessment, Analysis and Interpretation. Chichester: Wiley, 2000.
  • 10
    Streiner DLNG. Health Measurement Scales; A Practical Guide to their Development and Use, 3rd edn. Oxford: Oxford University Press, 1995.
  • 11
    Finlay AY, Khan GK. Dermatology Life Quality Index (DLQI): a simple practical measure for routine clinical use. Clin Exp Dermatol 1994;19:2106.
  • 12
    Garratt AM, Torgerson DJ, Wyness J, Hall MH, Reid DM. Measuring sexual functioning in premenopausal women. Br J Obstet Gynaecol 1995;102:3116.
  • 13
    Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983;67:36170.
  • 14
    Gray NM, Sharp L, Cotton SC, Avis M, Philips Z, Russell I, et al. Developing a questionnaire to measure the psychosocial impact of an abnormal cervical smear result and its subsequent management: the TOMBOLA (Trial of Management of Borderline and Other Low-grade Abnormal Smears) trial. Qual Life Res 2005;14:155362.
  • 15
    Rothnie K. Psychometric Properties of the Process and Outcome Specific Measure (POSM) and Predictors of Distress in Women with a Low Grade Abnormal Smear: A Secondary Analysis of Data from the TOMBOLA Trial. Aberdeen: University of Aberdeen, 2011.
  • 16
    Cairns M, Gray NM, Cruickshank ME. The impact of microinvasive cancer of the cervix on women during follow-up. Int J Gynecol Cancer 2008;18:128993.
  • 17
    Narayansingh GV, Cumming GP, Parkin DP, McConell DT, Honey E, Kolhe PS. Flap repair: an effective strategy for minimising sexual morbidity associated with the surgical management of vulval intra epithelial neoplasia. J R Coll Surg Edinb 2000;45:814.

Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions
  8. References
  9. Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’
  10. Disclosure of interests
  11. Supporting Information

How do we obtain the best information from study participants or patients to understand health? Clinicians and researchers ask questions to assess health status, but they are not always the best questions to solicit information that will inform clinical care or health policies. The article by Lockhart et al. (BJOG 2013;DOI:101111/1471-0528.12229) describes the process by which they developed a questionnaire to address ‘the symptoms, diagnosis, treatment and follow-up’ of vulval intraepithelial neoplasia, and demonstrates the need for a multistep approach to obtain the best information from participants involved in research. The process the authors used takes time, and it can vastly improve the reliability and validity of the data collected.

Their steps involved, first, gathering expert information about how the topic has been approached in the past to develop a conceptual basis for the questionnaire so that it accurately reflects the research goals. Then, the authors developed candidate questions and conducted qualitative data collection involving women with the condition. This step is important because, as researchers, we may think we understand the factors involved in a disease state and its effect on the patient, but, until patients are asked for their experiences, these assumptions are conjecture. After implementing revisions based on the feedback from patients, the next steps involved the administration of the instrument to a larger pool of patients and quantitative analysis of scale validity and reliability. Statistical tools assess whether the instrument has content validity, that is, how well the questions reflect the conceptual aims to be measured (DeVelllis, Scale Development: Theory and Application, 3rd edn. Thousand Oaks, CA: SAGE, 2012), and whether it has criterion validity, which compares new questions with other validated instruments, such as the development of a scale that is a shorter alternative to multiple scales, as is the case with the new scale described here. The authors also assessed reliability, or the consistency of the responses to questions over time (McDowell, Measuring Health: A Guide to Rating Scales and Questionnaires, 3rd edn. New York: Oxford University Press, 2006), by having participants complete the questionnaire twice, with a week between administrations, which was considered to be a short enough time such that changes in symptoms should not result in answer changes.

What more could the authors have done? First, the clarity of purpose of the instrument and the specificity of the construct(s) to be captured in the instrument could be improved. The authors indicate the instrument is intended to ‘assess the burden’ of vulval intraepithelial neoplasia and the construct intended to be measured is ‘symptoms, diagnosis, treatment and follow-up’. This suggests four broad constructs with sometimes unclear linkage to ‘burden’ and unclear utility (e.g. why assess ‘diagnosis’ with a self-report instrument among women already diagnosed via biopsy?). Second, an expanded discussion of the search terms used in their literature review would allow others to build on this work. In addition, the broad age range of participants might affect the validation, and so a more targeted age, perhaps of the women most affected, could potentially improve the clinical results of using the scale. Of particular concern is the low response rate (40%) and even lower repeat rate (26% of the targeted sample) that appear to have yielded a select group of participants and skewed distributions on multiple instrument items, as acknowledged by the authors. More information about the validation of the instruments that were used to test convergent and discriminant validity would also be useful.

The authors' efforts to create a scale that can be easily administered to women with vulval intraepithelial neoplasia and that provides high-quality data to clinicians could result in better health care and support for these patients. Taking a lesson from psychometricians, with some refinement, this approach should serve as a model for the development of all clinical assessments.

Disclosure of interests

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions
  8. References
  9. Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’
  10. Disclosure of interests
  11. Supporting Information

The authors declare no conflicts of interest.

  • N Dolea & C Tucker Halpernb

  • aCarolina Population Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

  • bDepartment of Maternal & Child Health & Carolina Population Center,

  • University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

Supporting Information

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Conclusions
  8. References
  9. Commentary on ‘The development and evaluation of the questionnaire to assess the impact of vulval intraepithelial neoplasia: a questionnaire study’
  10. Disclosure of interests
  11. Supporting Information
FilenameFormatSizeDescription
bjo12229-sup-0001-AppendiS1.pdfapplication/PDF202KAppendix S1. Vulval intraepithelial neoplasia questionnaire.

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.