To develop and evaluate a questionnaire to assess the burden of vulval intraepithelial neoplasia (VIN) in women.
To develop and evaluate a questionnaire to assess the burden of vulval intraepithelial neoplasia (VIN) in women.
A questionnaire development study.
Vulval Disorders Clinic serving a regional population.
Fifty-eight women with a histological diagnosis of VIN registered with the Vulval Disorders Clinic.
A 37-item questionnaire was developed through a comprehensive literature review, consultation with specialist clinicians and pretesting to assess the burden experienced by women. The questionnaire was assessed for validity and reliability against existing questionnaires used in related disease areas.
Spearman correlations were calculated between items in the VIN questionnaire with the scores of the Dermatology Life Quality Index (DLQI), Hospital Anxiety and Depression Scale (HADS), Sabbatsberg Sexual Self-Rating Scale (SSRS) and the Process Outcome Specific Measure (POSM) to assess the new questionnaire's validity. Internal consistency was measured using Cronbach's alpha. Test–retest reliability was calculated using quadratic weighted kappa.
The VIN questionnaire had a high degree of internal consistency (Cronbach's alpha, 0.89). Test–retest reliability was assessed, with most questions showing a quadratic weighted kappa value of 0.5 or above. Most questions showed a stronger correlation with the corrected total VIN score than with HADS anxiety and depression subscales and the SSRS, indicating discriminant validity. Most questions correlated significantly with the DLQI and POSM scores, indicating convergent validity.
Initial assessment of the VIN questionnaire demonstrated that it is a valid and reliable measure of the burden of disease for women. The questionnaire could be used to compare new and existing treatments for VIN or to assess or monitor the impact of care.
Vulval intraepithelial neoplasia (VIN) is an uncommon premalignant skin condition affecting the vulva. Common symptoms include intense itching, burning or discomfort. The standard management of VIN is local excision of the affected tissue, which can be mutilating and disfiguring, and may be associated with significant psychosexual morbidity.
A number of studies have identified poor health-related quality of life (HR-QoL) and poor sexual functioning in women diagnosed with VIN.[1-8] One prospective cohort study identified poor HR-QoL scores and sexual functioning in women, with some women reporting that the negative impact on sexual relations was the worst aspect of the condition for them.
In a qualitative study, women reported that information about VIN was lacking and unhelpful, that they felt embarrassment, fear and confusion about the condition and their future health, and that sexual function and body image were both affected by VIN.
Previous research has identified a wide range of issues relating to the symptoms, diagnosis, treatment and follow-up for VIN which can have a negative impact on women's lives. Many of these studies have used multiple questionnaires to measure these consequences of VIN [including the Dermatology Life Quality Index (DLQI), Sabbatsberg Sexual Self-Rating Scale (SSRS) and the Hospital Anxiety and Depression Scale (HADS)] because no single questionnaire is available which encompasses the wide range of difficulties experienced by women diagnosed with VIN. The aim of this study was to fill this gap by developing and evaluating a single questionnaire which addresses the issues of concern to women with a biopsy-confirmed diagnosis of VIN. The questionnaire addresses issues covering the symptoms, diagnosis, treatment and follow-up for VIN.
The study population comprised the residents of the Grampian Region in Scotland, which covers the relatively affluent city of Aberdeen and a surrounding rural population. Women were recruited from the Vulval Disorders Clinic based in Aberdeen. A woman was eligible to participate if she had a biopsy-confirmed diagnosis of VIN, and was attending or had attended the Grampian Vulval Disorders Clinic for treatment and/or follow-up for VIN.
Questionnaire development was informed by the commonly accepted steps to develop HR-QoL questionnaires,[9, 10] and comprised four phases: (1) the generation of relevant issues; (2) the development of questionnaire items from the issues; (3) the pre-testing of the questionnaire items; and (4) field testing.
A comprehensive review of the literature was undertaken to identify potentially relevant items for inclusion in the questionnaire. Heathcare professionals experienced in the treatment and management of patients with VIN were also consulted to help generate and review items in terms of coverage and relevance.
The literature review and expert consultation generated 30 items, from which a draft questionnaire was developed focusing on six themes: (1) symptoms; (2) daily activities; (3) feelings; (4) sexual relations; (5) relationships; and (6) future health concerns. A copy of the items included in each section is shown in Table 1.
|Section||Content of each section|
|Daily activities|| |
Has VIN affected:
choice in clothing?
social and leisure activities?
Has VIN made you feel:
anxious, depressed, embarrassed?
concerned about body image?
|Sexual relations|| |
Has VIN negatively affected:
interest in sex?
enjoyment of sex?
Has VIN negatively affected:
relationships with friends?
relationships with partner/spouse?
|Future health concerns|| |
Have you been concerned about:
future treatment for VIN?
getting vulval cancer?
The question stem ‘due to VIN and/or its treatment’ was repeated throughout the questionnaire to remind women to respond on the basis of their experience of VIN. All questions were time limited to 1 month, so that the responses reflected women's most recent experiences of VIN. Most response options were arranged as a four-point labelled categorical scale, ranging from ‘not at all’, ‘a little’, ‘quite a bit’ to ‘very much’.
The VIN questionnaire was pretested on 10 women attending the Vulval Disorders Clinic. Women were sent two copies of the questionnaire (one marked to be completed and returned and one marked to be kept for reference), a consent form and a prepaid and pre-addressed envelope to return these.
For the purposes of questionnaire development, a debriefing schedule was used to assess the content and design of the questionnaire. The schedule included questions about the acceptability and intrusiveness of questions, how easily understood (or not) were the questions and whether the questionnaire missed out any important issues. The debriefing schedule was administered by the researcher face-to-face at clinic or by telephone as preferred by the participant. The questionnaire was redrafted in the light of the comments made during these interviews.
A questionnaire booklet was developed which comprised socio-demographic and background questions, the redrafted VIN questionnaire and a number of other previously validated questionnaires, which had been used in this or related disease areas (see Appendix S1). Women indicating that they were willing to participate were sent the questionnaire booklet, a consent form and a pre-addressed, prepaid envelope to return these to the researcher.
In order to assess test–retest reliability of the VIN questionnaire, women were sent the questionnaire booklet on two occasions, with an interval of 7 days between each. The interval of 7 days was chosen as there was unlikely to be substantial changes in the participants' health in that time, but it would be sufficiently long to minimise participants remembering their responses to the first administration of the questionnaire. For each administration of the questionnaire booklet, a reminder letter was sent to women who did not respond within 2 weeks.
Socio-demographic, background and medical data collected included age, living status, parity, mode of child delivery, contraception and had women ever had treatment for an abnormal cervical smear test. Participants were asked what treatment they had undergone for VIN, which included creams, surgery under local anaesthetic and surgery under general anaesthetic. Laser treatment was included in the surgery under general anaesthetic response. A measure of deprivation was derived for each participant using the Scottish Index of Multiple Deprivation (SIMD). SIMD identifies small area concentrations of multiple deprivations across all of Scotland. The SIMD quintile is a score in the range 1–5, derived by a person's postcode, ‘1’ indicating the most deprived status and ‘5’ the least deprived status.
DLQI, a 10-item dermatology questionnaire, widely used in patients with a variety of skin conditions (including VIN), was employed to measure the negative impact of the disease. The response options are arranged on a four-point categorical scale ranging from ‘not at all’ to ‘very much’. Question responses are summed to yield a total score from zero to 30, with a higher score reflecting greater impairment of HR-QoL.
SSRS is a 14-item questionnaire and was included to measure sexual functioning. Each item is scored on a five-point scale from zero to four, and then summed to produce a total score which is expressed as a percentage, with 100 representing the best self-rated sexual functioning.
HADS was included to detect depressed and anxious mood. It consists of 14 items, comprising two subscales, seven items measuring anxiety and seven items measuring depression. The items are summed, yielding two subscale scores each ranging from zero to 21.
The Process Outcome Specific Measure (POSM), a 15-item questionnaire originally designed to assess the psychosocial impact of the receipt and management of a low-grade abnormal cervical smear result, was adapted to be used for women with VIN. Most response options are arranged on a six-point scale from ‘strongly agree’ to ‘strongly disagree’. The scores of the responses to the questions are standardised to be out of 100, with 100 indicating the highest distress. POSM has been used in women with low-grade abnormal smears, and in women with cervical intraepithelial neoplasia (CIN) and micro-invasive cervical cancer.
We anticipated that there was unlikely to be substantial changes in the clinical condition of women with VIN over a 1-week period, and thus responses would be similar in both administrations of the questionnaire; we estimated a reliability coefficient of approximately 0.8. In order to discriminate between groups of participants, it is commonly accepted that reliability should exceed 0.7. Using these figures, our required sample size for reliability testing was 118. Based on previous clinical experience of the recruitment of women with VIN, it was anticipated that 80% of women would agree to participate and complete the questionnaire. Thus, 147 would be invited to participate for field testing.
Four filter questions, which allowed women to skip questions which were not relevant to them, were excluded from the scoring. Question 11 (‘Is the reason you answered no to being sexually active because of VIN and/or its treatment?’) and question 18 (‘Are there any reasons why you have not told anyone about VIN and/or its treatment?’) were excluded from the VIN scoring as they had been included to help our understanding of women's responses to the sexual activity and support questions, and were not included as a measure of burden. The global HR-QoL question was also excluded from scoring as it was not VIN-specific; rather, it provided a general measure of HR-QoL. The remaining 30 questions were scored and summed to give a total score for the VIN questionnaire.
Each question was scored from one to two, four or five depending on the number of response categories. All questions were scored so that a higher score indicated a higher level of burden of disease. To overcome the problem of having different numbers of response options in the questionnaire, each question score was standardised to be out of 100, with 100 indicating the highest burden. This was achieved by multiplying an individual's raw score by 100, divided by the highest possible raw score.
The questionnaire data from field testing were entered manually and analysed using SPSS for Windows version 19 (IBM Corporation, USA). The score distributions of the questionnaire data from the field testing were examined for skewness, floor and ceiling effects, and missing values.
To assess whether a higher total VIN score corresponds to a higher disease burden, descriptive statistics were calculated to assess whether there was any differences in the total VIN score between women who had undergone different types of treatment. Participants were placed into one of four treatment groups: no treatment, creams, surgery and combined surgery and cream treatment. We might expect those women with a lower disease burden to have had no treatment and those with a higher disease burden to have undergone surgery.
To assess how women responded to the questionnaire, the distribution of the total VIN score was assessed.
Construct validity, by which we mean the extent to which the questionnaire measured the construct it was designed to measure, was assessed in two ways. First, each question in the VIN questionnaire was correlated with the corrected total VIN score (i.e. the total score omitting the score of each question in turn) in order to assess to what extent the individual items were measuring a single construct. Second, to assess how well the VIN questionnaire was measuring something distinct to that being measured by the other questionnaires, each of the VIN questions was correlated with the total DLQI, SSRS and POSM scores and the HADS anxiety and depression subscale scores.
A priori, we anticipated the VIN questions relating to symptoms and daily activities to be strongly correlated with DLQI, and the question concerning women's future worries about getting cancer to be positively correlated with POSM. We also expected there to be a strong correlation between the total SSRS score and the VIN questions relating to sexual activity. Similarly, we expected only the VIN questions relating to feeling anxious or depressed to be correlated with the HADS anxiety and depression subscale scores.
The reliability of the questionnaire was assessed in two ways. Cronbach's alpha was used to measure internal consistency (i.e. the extent to which the items were interrelated) using questionnaires with complete data. Generally, reliablity over 0.7 is required to determine differences between groups of patients. Second, the stability of individuals' responses to the questions over a period of time during which their health status would not be expected to change (i.e. test–retest reliability) was assessed by examining individuals' responses between both administrations of the questionnaire and calculating the level of agreement using quadratic weighted kappa. Values of over 0.4 for quadratic weighted kappa reflect moderate agreement, and over 0.6 reflect good agreement. The reliability of the overall VIN score between the two administrations of the questionnaire was measured using intraclass correlation coefficients.
Throughout all statistical analysis, a two-sided P value of ≤0.05 was used as the threshold for statistical significance.
Ten women completed the pretest questionnaire and seven participated in the debriefing interviews. Four women commented that, because they were asked to answer in terms of their experiences of VIN in the last month, most of the questions were not relevant to them. One woman found the question concerning vulval cancer to be upsetting as she was unaware of a connection between VIN and vulval cancer. Two women commented that they had not told anyone about VIN, and suggested that it may be useful to explore this further. One woman commented that the questionnaire did not address concerns about starting a new relationship because of VIN. Six questions were added as a result of pretesting:
A number of positive comments were made about the questionnaire, including that it was clear, concise and easy to understand. The questionnaire was quick to complete, taking under 10 minutes. All women made positive comments about the recognition of the issues experienced by women with VIN.
From the 147 women invited to participate, 70 were recruited (Figure 1). In total, 58 women returned the first administration of the questionnaire. The socio-demographic and background characteristics of the participants are shown in Table 2.
|n = 58||Valid%|
|Age group (years)|
|1 (most deprived)||3||5|
|5 (least deprived)||15||27|
|Ever had children|
|Number of children ( n = 50)|
|Sterilisation & partner – vasectomy||12||21|
|Treatment for VIN|
|Creams and surgery||15||26|
|Treatment for an abnormal cervical smear test|
The mean age of the participants was 53 years (range, 30–82 years). Most women were married (43%), lived with a partner or spouse (56%), were not sexually active (62%) and had had children (86%). Over half (55%) of women were categorised into the SIMD quintiles of ‘4’ or ‘5’ (indicating the least deprived status). The majority of women had undergone surgical treatment for VIN (86%).
The score distributions of the VIN questionnaire were positively skewed, clustering around the bottom of the scale (indicating floor effects). Similar score distributions were also observed for the DLQI and SSRS questionnaires. Levels of missing data for most questions in the VIN questionnaire were low (≤5%). The exception was the question on concerns about having future treatment for VIN, with 8% missing data.
Table 3 highlights the descriptive statistics of the total VIN score for the different treatment groups. There were no differences in the VIN total score across the different treatment groups (P = 0.09). With regard to the total VIN score of all participants, the majority of women scored low [mean, 28.6 (standard deviation, 13.8); median, 24.5 (interquartile range, 19–34)].
|Treatment||(n = 57)||Mean||Total VIN score||IQR|
|Creams and surgery||15||37.6||18.7||32.2||20–58|
Item total correlations for the VIN questionnaire and the correlations between the individual VIN questions and the total scores of the DLQI, SSRS, POSM, and HADS anxiety and depression subscales are shown in Table 4.
|VIN question||n c||Corrected VIN score||SSRS||HADS (A)||HADS (D)||DLQI||POSM|
|2||Frequency of itch/irritation?||58||0.54b||−0.09||0.23||0.23||0.62b||0.30a|
|3||Severity of itch/irritation?||58||0.48b||−0.17||0.17||0.17||0.59b||0.26a|
|4||Frequency of pain/discomfort?||58||0.57b||−0.03||0.20||0.19||0.62b||0.29a|
|5||Severity of pain/discomfort?||57||0.54b||0.07||0.16||0.15||0.56b||0.23|
|6a||Has VIN affected your choice in clothing?||58||0.39b||−0.14||0.19||0.18||0.46b||0.11|
|6b||Has VIN affected your social and leisure activities?||58||0.53b||−0.06||0.33a||0.30a||0.50b||0.25|
|6c||Has VIN affected your daily activities?||58||0.48b||−0.02||0.30a||0.30a||0.45b||0.36b|
|6d||Has VIN negatively affected your work?||58||0.54b||−0.04||0.27a||0.35b||0.58b||0.29a|
|6e||Has VIN negatively affected your sleeping pattern?||58||0.52b||−0.02||0.27a||0.28a||0.61b||0.31a|
|7a||Have the symptoms, diagnosis or treatment of VIN made you anxious?||58||0.60b||0.04||0.33a||0.24||0.47b||0.34a|
|7b||Have the symptoms, diagnosis or treatment of VIN made you depressed?||58||0.44b||−0.17||0.40b||0.39b||0.44b||0.40b|
|7c||Have the symptoms, diagnosis or treatment of VIN made you embarrassed?||58||0.38b||−0.10||0.28a||0.25||0.43b||0.27a|
|8||Has VIN negatively affected your confidence?||58||0.64b||−0.17||0.57b||0.50b||0.67b||0.53b|
|9||Have you been concerned about the appearance of your body?||58||0.58b||−0.18||0.44b||0.43b||0.57b||0.43b|
|12||Has the frequency of your sexual activity been altered?||58||0.51b||0.64b||0.29a||−0.06||0.21||0.46b|
|13||Has sex been painful in the last month?||58||0.53b||0.62b||0.27a||−0.06||0.22||0.44b|
|14||Has your interest in having sex reduced?||58||0.52b||0.60b||0.28a||−0.04||0.21||0.46b|
|15||Has your enjoyment of sex been negatively affected?||58||0.54b||0.60b||0.30a||−0.02||0.23||0.47b|
|16||Have you felt anxious about having sex?||58||0.54b||0.60b||0.27a||−0.04||0.23||0.45b|
|19||Has VIN negatively affected your relationship with your partner/spouse?||58||0.33a||0.22||0.21||0.11||0.38b||0.39b|
|20||Has VIN negatively affected your relationship with work colleagues?||57||0.07||0.36a||0.00||−0.24||−0.09||0.02|
|21||Has VIN negatively affected your relationship with friends?||58||0.32a||0.12||0.14||0.08||0.13||0.21|
|22||Have you felt anxious about starting a relationship with a new partner?||58||0.17||−0.17||0.22||0.25||0.17||0.17|
|23||Have you had concerns about finding a long-term partner?||57||0.07||−0.08||0.15||0.00||0.03||0.12|
|24||Have you had enough support from friends and/or family?||55||0.41b||0.17||0.26||0.17||0.24||0.19|
|26a||Have you had concerns about your future fertility?||58||0.23||0.21||0.24||0.05||0.03||0.19|
|26b||Have you had concerns that you might have problems with childbirth?||58||0.23||0.20||0.24||0.06||0.04||0.20|
|27a||Have you had concerns about having future treatment for VIN?||53||0.38b||−0.05||0.39b||0.40b||0.30a||0.21|
|27b||Have you had concerns about your next check up at the vulval clinic?||56||0.21||−0.14||0.36b||0.45b||0.27a||0.28a|
|27c||Have you had concerns about getting vulval cancer?||55||0.37b||−0.12||0.22||0.27a||0.20||0.24|
All but six of the questions in the VIN questionnaire were correlated significantly with the total VIN corrected score (P < 0.05), with correlation coefficients ranging from 0.32 to 0.64. The exceptions were question 20 (‘Has VIN negatively affected your relationship with work colleagues?’) (r = 0.07), question 22 (‘Have you felt anxious about starting a relationship with a new partner?’) (r = 0.17) and question 23 (‘Have you had concerns about finding a long term partner?’) (r = 0.07).
Other questions which did not show statistically significant correlations were the questions concerning children (question 26a, ‘Have you had concerns about future fertility?’; question 26b, ‘Have you had concerns that you might have problems with childbirth?’), both of which had correlation coefficients of r = 0.23. Lastly, question 27b (‘Have you had concerns about your next check up at the vulval clinic?’) was not correlated significantly with the corrected total VIN score (r = 0.21).
Although there were some statistically significant correlations between the VIN questions and the total scores of the other questionnaires, most of the VIN questions correlated most strongly with the corrected total VIN score.
The VIN questions which related to symptoms and activities of daily life correlated strongly with the DLQI questionnaire, with correlations ranging from 0.45 to 0.62. The SSRS questionnaire correlated strongly with the VIN questions relating to sexual activity (correlations ranging from 0.60 to 0.64). The POSM questionnaire correlated positively with questions relating to concerns about getting cancer in the future (r = 0.24).
The correlations between the total VIN score and the total scores of the existing questionnaires were examined. The VIN questionnaire score was statistically significantly correlated with the DLQI (r = 0.69), POSM (r = 0.57) and HADS anxiety (r = 0.50) and depression (r = 0.27) total scores, but not with the SSRS questionnaire (0.19).
For the VIN questionnaire, Cronbach's alpha was 0.89, indicating an excellent degree of internal consistency. This value was similar to those for the DLQI (0.93), SSRS (0.97), POSM (0.69) and HADS anxiety and depression scales (0.89 and 0.89, respectively).
The results of the test–retest reliability for the VIN questionnaire are shown in Table 5.
|VIN question number||Quadratic weighted kappa (κ)||n (number of respondents answering question both times)|
|2||Frequency of irritation?||0.81||15a|
|3||Severity of itch/irritation?||0.34||15a|
|4||Frequency of pain/discomfort?||0.54||15a|
|5||Severity of pain/discomfort?||0.14||15a|
|6a||Has VIN affected your choice of clothing?||0.40||32|
|6b||Has VIN affected your social and leisure activities?||0.78||32|
|6c||Has VIN affected your daily activities?||0.73||32|
|6d||Has VIN negatively affected your work?||0.78||32|
|6e||Has VIN negatively affected your sleeping pattern?||0.52||32|
|7a||Has VIN made you feel anxious?||0.29||32|
|7b||Has VIN made you feel depressed?||0.67||32|
|7c||Has VIN made you feel embarrassed?||0.67||32|
|8||Has VIN negatively affected your confidence?||0.62||32|
|9||Has VIN made you feel concerned about the appearance of your body?||0.53||32|
|10||Are you sexually active?||0.86||32|
|11||Reason answered no to being sexually active because of VIN?||0.83||20b|
|12||Frequency of sexual activity?||0.53||11a|
|13||Has sex been painful due to VIN?||0.39||11a|
|14||Has your interest in sex reduced in the last month due to VIN?||0.82||11a|
|15||Has your enjoyment of sex been negatively affected due to VIN?||0.71||11a|
|16||Have you felt anxious about having sex in the last month due to VIN?||0.50||11a|
|17||Have you told anyone about your VIN?||0.91||32|
|19||Has VIN negatively affected your relationship with partner/spouse?||0.70||31|
|20||Has VIN negatively affected your relationships with work colleague?||0.74||32|
|21||Has VIN negatively affected your relationships with friends?||0.72||32|
|22||Has VIN made you feel anxious about starting a new relationship?||0.55||32|
|23||Have you had concerns about finding a long-term partner?||0.81||32|
|24||Do you feel you have had enough support from friends and/or family?||0.83||32|
|25||Do you intend to have children in the future?||1||30|
|26a||Has VIN given you concerns about future fertility?||1||3a|
|26b||Has VIN given you concerns about having problems with childbirth?||1||3a|
|27a||Have you had concerns about having future treatment for VIN?||0.67||30|
|27b||Have you had concerns about your next check up at the vulval clinic?||0.52||30|
|27c||Have you had concerns about getting vulval cancer?||0.58||29|
All but five questions showed levels of reliability above 0.5 using quadratic weighted kappa. The exceptions were question 3 (‘On a scale of 1–5, where 1 is no itch/irritation and 5 is extreme itch/irritation, how severe has the itch/irritation been in the last month?) and question 5 (‘On a scale of 1–5, where 1 is no pain/discomfort and 5 is extreme pain/discomfort, how severe has the itch/irritation been in the last month?), with quadratic weighted kappa values of 0.34 and 0.14, respectively.
Three other questions displayed lower levels of agreement: question 6a (‘In the last month, has VIN and/or its treatment affected your choice of clothing?’) (κ = 0.40), question 7a (‘Has VIN made you feel anxious?’) (κ = 0.29) and question 13 (‘Due to VIN and/or its treatment, has sex been painful in the last month?’) (κ = 0.39). There was good agreement of the overall VIN scores between the two administrations of the questionnaire (intraclass correlation coefficient, 0.91; confidence interval, 0.79–0.96).
Responses to questions which show a skewed distribution (where most responses are at the bottom or top of the scale) are known as floor and ceiling effects, respectively. The responses to the VIN questionnaire were positively skewed, indicating floor effects.
Cronbach's alpha for the VIN questionnaire was 0.89, which compared favourably with the internal consistency of each of the subscales of HADS (0.89 for the anxiety subscale and 0.89 for the depression subscale), SSRS (0.97), POSM (0.69) and DLQI (0.93). A value of greater than 0.7 is considered to be acceptable for these types of scale.
The questionnaire seemed, for the most part, to be measuring a single construct, with all but three questions having corrected item total correlations above 0.2, which is the generally accepted cut-off level for keeping or discarding questionnaire items. The exception was question 22 (‘In the last month, due to VIN and/or its treatment, have you felt anxious about starting a relationship with a new partner?’), which had been included after pretesting.
SSRS, DLQI and HADS have all been used in women with VIN in previous studies.[2, 5, 17] Although POSM was used for women with CIN, it also addressed issues of importance to women with VIN. Thus, we expected some degree of correlation between the VIN questions that were intended to measure similar constructs and those measured by these questionnaires. Our findings were consistent with our expectations, in that there were statistically significant correlations between some of the VIN questions and the SSRS, DLQI and POSM total scores and the HADS subscale scores (demonstrating convergent validity). However, the magnitude of the correlations between the individual VIN questions and the corrected total VIN score was generally greater (for most questions) than the correlation with the total scores of the HADS anxiety and depression subscales and other questionnaires (demonstrating discriminant validity).
The floor effects observed for the VIN questions could indicate that some questions were not relevant, but this conflicts with our findings from the literature and expert views. Instead, this may reflect the possibility that women have not experienced the problems described in the questionnaire in the time frame suggested (i.e. in the last month).
There were no differences in the VIN scores in women across the different treatment groups. Women with no treatment may have co-morbidities or may not have embarked on treatment, which may be reflected in a higher VIN score. Conversely, the use of topical cream with or without surgery may relate to extensive disease; therefore, we would expect a high score in these patients. The treatment groups are small and therefore the results should be interpreted with care.
With the exception of question 22, all questions had item total correlations of over 0.2. This question addressed whether women felt anxious about starting a relationship with a new partner. This issue had not been identified in the literature review or expert consultation, which may indicate that this issue is a concern for only a minority of women.
The high Cronbach's alpha result (0.89) suggests that the VIN questions may measure one construct and the questionnaire could be used as a scale. There was a strong positive relationship between the VIN questionnaire and DLQI and POSM. The HADS anxiety subscale correlated strongly with the overall VIN score (r = 0.50). Intuitively, we would expect that the higher the disease burden, the more anxious are the women. This is consistent with results from a similar VIN study, which found that increased anxiety was associated with worsening emotional and general health.
To our knowledge, this is the first study to develop and test a questionnaire to measure the negative impact of VIN. It can be completed quickly (generally taking less than 10 minutes), administered by post or in a clinical setting, and is relatively easy to score. The questionnaire underwent a rigorous developmental process, informed by previous research and by expert and patient consultation. Semistructured debriefing interviews identified problems and ambiguities in the questionnaire, and provided a further assessment of content validity.
The questionnaire has only been tested in the Grampian region; therefore, it is geographically limited. Most of the women participating in the study had been treated for VIN and were being followed up; therefore, the results may not be generalizable to women newly diagnosed with VIN. The study did not meet the estimated 118 participants needed for reliability testing, thereby limiting the robustness of the findings. The response rate for the first administration of the questionnaire was 40%, lower than the 80% achieved in a previous study, and the 54% postal response rate achieved by a similar study in the VIN population. The lower than expected response rate could have been caused by respondent fatigue, as the questionnaire booklet contained 96 questions. The lower response rate may indicate some degree of responder bias. We did not achieve the number for reliability statistics, and thus we may have under- (or over-) estimated the reliability of the questionnaire.
We have developed a questionnaire which addresses issues experienced by women with VIN. The questionnaire requires further testing with a larger sample size, which is likely to require multiregional involvement. With a larger sample size, a factor analysis could be performed to assess whether the questionnaire measures one or more distinct constructs. Furthermore, the questionnaire could be tested with newly diagnosed women, and before and after treatment of VIN. The questionnaire is being tested in women with other vulval skin conditions (e.g. lichen sclerosus) to help to validate the questionnaire and to enable a comparison of vulval skin conditions in terms of the burden experienced by women.
The questionnaire is reliable with good convergent and discriminant validity. It could be used to monitor patients in the clinic and to compare new and existing treatments for VIN in terms of the burden experienced by women.
JL performed the review of the literature, generated items for inclusion in the questionnaire and constructed the questionnaire. JL performed the debriefing interviews and posted out the questionnaires. JL analysed the data and contributed to the writing up of the work. NG helped to develop and write the project protocol and to construct the questionnaire, provided advice about the analysis and contributed to the writing up of the work. MC developed and wrote the project protocol, helped to recruit women for the study, provided advice about the questionnaire and analysis plan and contributed to the writing up of the work.
This study was given ethical approval by the North of Scotland Research Ethics (NRES) Committee 2 (25 October 2011 – Reference: 11/NS/0038). A copy of the questionnaire was submitted to Committee 2 before field testing began.
We are grateful to all the women who participated in this study.
How do we obtain the best information from study participants or patients to understand health? Clinicians and researchers ask questions to assess health status, but they are not always the best questions to solicit information that will inform clinical care or health policies. The article by Lockhart et al. (BJOG 2013;DOI:101111/1471-0528.12229) describes the process by which they developed a questionnaire to address ‘the symptoms, diagnosis, treatment and follow-up’ of vulval intraepithelial neoplasia, and demonstrates the need for a multistep approach to obtain the best information from participants involved in research. The process the authors used takes time, and it can vastly improve the reliability and validity of the data collected.
Their steps involved, first, gathering expert information about how the topic has been approached in the past to develop a conceptual basis for the questionnaire so that it accurately reflects the research goals. Then, the authors developed candidate questions and conducted qualitative data collection involving women with the condition. This step is important because, as researchers, we may think we understand the factors involved in a disease state and its effect on the patient, but, until patients are asked for their experiences, these assumptions are conjecture. After implementing revisions based on the feedback from patients, the next steps involved the administration of the instrument to a larger pool of patients and quantitative analysis of scale validity and reliability. Statistical tools assess whether the instrument has content validity, that is, how well the questions reflect the conceptual aims to be measured (DeVelllis, Scale Development: Theory and Application, 3rd edn. Thousand Oaks, CA: SAGE, 2012), and whether it has criterion validity, which compares new questions with other validated instruments, such as the development of a scale that is a shorter alternative to multiple scales, as is the case with the new scale described here. The authors also assessed reliability, or the consistency of the responses to questions over time (McDowell, Measuring Health: A Guide to Rating Scales and Questionnaires, 3rd edn. New York: Oxford University Press, 2006), by having participants complete the questionnaire twice, with a week between administrations, which was considered to be a short enough time such that changes in symptoms should not result in answer changes.
What more could the authors have done? First, the clarity of purpose of the instrument and the specificity of the construct(s) to be captured in the instrument could be improved. The authors indicate the instrument is intended to ‘assess the burden’ of vulval intraepithelial neoplasia and the construct intended to be measured is ‘symptoms, diagnosis, treatment and follow-up’. This suggests four broad constructs with sometimes unclear linkage to ‘burden’ and unclear utility (e.g. why assess ‘diagnosis’ with a self-report instrument among women already diagnosed via biopsy?). Second, an expanded discussion of the search terms used in their literature review would allow others to build on this work. In addition, the broad age range of participants might affect the validation, and so a more targeted age, perhaps of the women most affected, could potentially improve the clinical results of using the scale. Of particular concern is the low response rate (40%) and even lower repeat rate (26% of the targeted sample) that appear to have yielded a select group of participants and skewed distributions on multiple instrument items, as acknowledged by the authors. More information about the validation of the instruments that were used to test convergent and discriminant validity would also be useful.
The authors' efforts to create a scale that can be easily administered to women with vulval intraepithelial neoplasia and that provides high-quality data to clinicians could result in better health care and support for these patients. Taking a lesson from psychometricians, with some refinement, this approach should serve as a model for the development of all clinical assessments.
The authors declare no conflicts of interest.
N Dolea & C Tucker Halpernb
aCarolina Population Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
bDepartment of Maternal & Child Health & Carolina Population Center,
University of North Carolina at Chapel Hill, Chapel Hill, NC, USA