‘Potency’: the validation of information from a self-administered questionnaire using objective measurements of night-time erections and test-retest reliability

Authors

  • Helgason,

    1. Southern Division of Community Medicine, Tobacco Prevention Center, Novum, Huddinge, Sweden
    2. Karolinska Institute, Department of Cancer Epidemiology and Department of Cancer Prevention, Radiumhemmet, Stockholm, Sweden
    3. Icelandic Cancer Society, Reykjavik, Iceland
    Search for more papers by this author
  • Arver,

    1. Karolinska Hospital, Department of Molecular Medicine and Department of Gynaecology and Obstetrics, Stockholm, Sweden
    Search for more papers by this author
  • Adolfsson,

    1. Karolinska Institute, Department of Urology, Huddinge University Hospital, Stockholm, Sweden
    Search for more papers by this author
  • Dickman,

    1. Karolinska Institute, Department of Cancer Epidemiology and Department of Cancer Prevention, Radiumhemmet, Stockholm, Sweden
    2. University of Newcastle, Department of Statistics, Australia
    Search for more papers by this author
  • Granath,

    1. Karolinska Institute, Department of Cancer Epidemiology and Department of Cancer Prevention, Radiumhemmet, Stockholm, Sweden
    Search for more papers by this author
  • Steineck

    1. Karolinska Institute, Department of Cancer Epidemiology and Department of Cancer Prevention, Radiumhemmet, Stockholm, Sweden
    Search for more papers by this author

Drs Helgason and G. Steineck, Department of Cancer Epidemiology, Radiumhemmet, Karolinska Hospital, S-171 76 Stockholm, Sweden.

Abstract

Objective

 To assess the validity and reliability of a questionnaire assessing ‘physiological potency’.

Patients and methods

 The study comprised 89 patients with prostate cancer and 43 men without; the latter were attending a consultation clinic because of problems with erection. All men answered three questions assessing erectile rigidity during sexual activity, morning and spontaneous erections. In the questionnaire, ‘potency’ was defined as erectile rigidity ‘sufficient for intercourse most of the time’ or better. ‘Potency’ in one or more of the three aspects of erection was defined as ‘physiological potency’. The patients with prostate cancer answered the questionnaire twice with a 3-week interval. The men attending the consultation clinic underwent two nights of erectile monitoring (using the RigiScan device) and the minimum criterion for RigiScan potency was defined as 55% rigidity at both tip and base.

Results

 The test-retest assessment showed 93% conformity in the questionnaire diagnosis of ‘physiological potency/impotence’ between the tests. The sensitivity and specificity of the questionnaire assessment compared with the RigiScan method were 40% and 100%, respectively, when the question assessing sexually stimulated erectile rigidity was used alone. Using ‘physiological potency’, the sensitivity increased to about 60% without jeopardizing the specificity, and when men reporting depression were excluded from the analysis, the sensitivity increased to about 80%.

Conclusions

 The test-retest reliability of the questionnaire was satisfactory. Using questions in a self-administered questionnaire, ‘physiological impotence’ can be diagnosed with complete and ‘physiological potency’ with 60–80% sensitivity. The sensitivity of the self-assessment for ‘potency’ depended on the number of questions asked and the proportion of men reporting depression.

Introduction

An increased prevalence of decreasing erectile rigidity has been documented among patients with certain diseases, including prostate cancer, diabetes and cardiovascular disorders [ 1, 2]. However, there are conflicting results about the prevalence of ‘impotence’ in these patients. For prostate cancer, the reported prevalence of intact ‘potency’ after treatment with curative intent ranges from 21% [ 3] to 86% [ 4] for EBRT and 14% [ 5] to 68% [ 6] for retropubic radical prostatectomy using a nerve-sparing technique. One reason for this wide range is probably the variation in definitions of ‘potency’ and a lack of valid standardized methods of grading erectile rigidity that can be used efficiently in large populations, and which are relatively sensitive for the diagnosis of ‘physiological potency’. Most studies assessing erectile rigidity before and after treatment for prostate cancer use subjective assessments, including clinical interviews and self-administered questionnaires. In population-based studies and clinical trials of many patients, a questionnaire is generally preferred for practical reasons. Consequently, it is of interest to develop valid and reliable questions for the assessment of ‘physiological potency’ that may replace more time-consuming and expensive clinical assessments.

 Erectile rigidity may be assessed objectively during sleep using the RigiScan device [ 7[8][9]–10] and the subjective assessment of a trained observer correlates strongly with this formal rigidity measurement [ 9, 10]. In assessing ‘organic impotence’, a base rigidity of 55% measured by the RigiScan had the highest sensitivity and specificity using ‘trained-observer determinations of functional erection as the gold standard in a controlled clinical setting’ [ 10]. To our knowledge, no study has compared the self-assessment of erectile capacity with an objective assessment of erectile function. The aim of the present study was to investigate the reliability, sensitivity and specificity of a ‘potency’ assessment using a self-administered questionnaire.

Patients and methods

The study comprised 89 patients with prostate cancer selected from a group of patients diagnosed in the Stockholm region in Sweden in 1993, and 43 men without prostate cancer visiting a sexual consultation clinic at the Karolinska University Hospital in Stockholm in 1996, because they had erectile problems. All the men investigated completed a questionnaire containing three questions assessing erectile rigidity during sexual activities, morning erections and spontaneous erections. Erectile rigidity was graded on an eight-category ordinal scale for each of the three aspects of erection (Table 1). Questions assessing factors that might affect erectile rigidity and the quality of the information were included. ‘Potency’ in the questionnaire assessment was defined as erectile rigidity ‘sufficient for intercourse most of the time’ or better (responses 1–4, Table 1). Potency on at least one of the three questions assessing stiffness (sexual, morning or spontaneous erections) was defined as ‘physiological potency’.

Table 1.  The erectile rigidity scale used in the potency questionnaire Thumbnail image of

 The 89 patients with prostate cancer completed the questionnaire twice with a 3-week interval between to determine the stability (test-retest reliability) of the diagnosis of ‘potency’ against ‘impotence’ of the three questions assessing erectile rigidity, as well as the test-retest reliability of assessing ‘physiological potency’.

 All 43 men attending the sexual consultation clinic completed the questionnaire before they underwent an objective measurement of erectile rigidity using the RigiScan device, which assessed the duration, frequency and rigidity of erections during sleep [ 7]. Two loops are placed on the penis and connected to a registration unit attached to the thigh; tumescence and rigidity are then automatically recorded with time. In the present study, the patients slept with the device operating for two nights (two sessions). The men were asked to abstain from using alcohol and sleeping pills during the days preceding the recordings. A rigidity of 55% at both the tip and base of the penis was defined as the minimum criterion for RigiScan ‘potency’, in accordance with previous findings [ 10]. In the present analysis, only those men registering 55% rigidity at the tip and base lasting at least 10 min during the same session were defined as ‘potent’.

 The sensitivity and specificity for ‘potency’ were calculated to compare the RigiScan and questionnaire assessments.Sensitivity and specificity refer to the proportion of men defined as potent or impotent, respectively, by the RigiScan who were also defined as potent and impotent, respectively, in the questionnaire assessment.

 To assess if changing the definition for ‘potency’ as assessed by the questionnaire improved the sensitivity and specificity of that assessment, an alternative potency criterion was defined as erectile rigidity ‘seldom sufficient for intercourse’ or better (responses 1–5; Table 1). The sensitivity and specificity of ‘questionnaire physiological potency’ and ‘potency’ during sexually stimulated erections only (sexual potency) was assessed for both definitions of potency on the erectile rigidity questions. The 95% CI are presented for the sensitivity and specificity calculations and, when appropriate, Fisher’s exact test was used to test for statistical significance between two proportions. The rigidity criterion for RigiScan potency was adjusted from 55% to 60% and 70% to assess whether this affected the sensitivity and specificity of the questionnaire assessment. An overview of the different definitions of ‘potency’ used in the analysis is presented in Table 2

Table 2.  Different definitions of ‘potency’ used in the present analysis Thumbnail image of

Results

All 89 patients with prostate cancer were included in the test-retest analysis; of the 43 men without prostate cancer attending the consultation clinic, three were not eligible because they had technically invalid RigiScan recordings and one did not answer the questions about erectile rigidity, leaving 39 patients for evaluation. The test-retest reliability of the questions assessing erectile rigidity (same diagnosis in both assessments) was 93% when assessing ‘physiological potency A’ (defined in Table 2), 92% for ‘sexual potency A’, 91% for ‘morning erections’ and lowest for ‘potency’ during spontaneous erections (88%).

 The comparison between the questionnaire assessment and the RigiScan measurement is presented in Table 3. Using a rigidity of 55% as the reference, the sensitivity of the questionnaire diagnosis of ‘physiological potency’ was 67% (18 of 27) when the threshold in the questionnaire was responses 1–4 (potent) and 70% (19 of 27) when responses 1–5 were used. Complete specificity (11 of 11) was obtained when using a 55% rigidity criterion and responses 1–4 from the erectile rigidity questions. The specificity decreased to 67% (8 of 12) when using responses 1–5 to indicate potency. Using 60% or 70% rigidity criterion had little effect on the sensitivity but impaired the specificity; with a threshold of responses 1–5, the sensitivity surpassed the specificity. The sensitivity decreased but the specificity remained relatively stable when the potency assessment was based on sexually stimulated erections only (sexual potency).

Table 3.  A comparison of the sensitivity and specificity (95%CI) of the RigiScan and questionnaire assessment in 39 patients with erectile dysfunction, using one or a combination of three questions in the questionnaire assessing rigidity and with three thresholds of potency Thumbnail image of

 When 11 men who stated that they had been ‘unusually depressed during the last year’ were excluded from the analysis (leaving 28 men) the sensitivity of the questionnaire ‘physiological potency’ assessment increased to about 80% (15 of 18) using responses 1–4 as the criterion and 55% rigidity as the reference without jeopardizing the specificity. This also increased the sensitivity of the questionnaire ‘sexual potency’ assessment to about 60% (11 of 18).

 Among the 11 depressed men the sensitivity and specificity for questionnaire ‘physiological potency’ was 38% (three of eight) and complete (all three), respectively, using responses 1–4 as the threshold and 55% rigidity. For ‘sexual potency’ the sensitivity was zero (none of eight) but the specificity remained complete (all three). The sensitivity of the ‘sexual potency’ assessment was significantly better for the men not reporting depression compared to the depressed men, 0.61 vs 0.00 respectively (P=0.008; Fisher’s exact test). For ‘physiological potency’ the difference was not quite significant, 0.83 vs 0.38 respectively (P=0.06). Of the depressed men, three had received treatment for their depression and were taking antidepressive medicine. They were all potent according to the RigiScan measurement (at 55% and 60% rigidity) but ‘physiologically impotent’ when assessed by the questionnaire using responses 1–4 as the threshold. Of the remaining eight men not treated for their depression, three were potent and three impotent in both assessments, and two were potent according to the RigiScan measurement but impotent in the questionnaire assessment.

Discussion

The test-retestreliability of the questionnaire assessment for all three aspects of erectile rigidity was about 90%. This was also true for the ‘collapsed’ variable, ‘physiological potency’. Defining potency as responses 1–4 (Table 1) and the 55% rigidity criterion, the questionnaire assessment tended to misclassify some men as ‘physiologically impotent’ who were potent according to the RigiScan. This was reflected in the sensitivity assessment. Using the definitions of potency in Tables 1 and 2, men defined as ‘impotent’ in the RigiScan assessment were in all cases ‘physiologically impotent’ in the questionnaire assessment, giving complete specificity. Increasing the criterion for RigiScan potency to 60% or 70% rigidity impaired the specificity but had little effect on the sensitivity. Likewise, changing the definition for physiological potency in the questionnaire assessment from responses 1–4 to include response 5 resulted in a much lower specificity without improving the sensitivity to any great extent.

 Questionnaire ‘potency’ during sexually stimulated erections only (sexual potency) was a poor indicator of RigiScan potency; using 55% rigidity, the sensitivity decreased from 67% (using responses 1–4) 41%, but the specificity remained complete. The high specificity indicates that the present questions are probably the most relevant indicators for the assessing erectile dysfunction.

 When men reporting to have been unusually depressed during the last year were excluded from the analysis, the sensitivity of ‘physiological potency’ (responses 1–4) increased from 60% to 80% without impairing the specificity, indicating that depressed men have a tendency to underestimate their erectile capacity. Interestingly, none of the eight depressed men who were defined as potent on the RigiScan assessment were potent according to the questionnaire when assessing sexually stimulated erections only (sexual potency). Although depression clearly affected the self-assessment of sexually stimulated erections, there was no indication that the depression or its treatment affected the physiological ability to have an erection. Of the five depressed men defined as ‘physiologically impotent’ (responses 1–4) but potent on the RigiScan (55% and 60% rigidity), three were taking medication for their depression and they were all potent according to the RigiScan assessment. In accordance with these findings, a previous study detected no relationship between depression and nocturnal tumescence [ 11]. The men included in the present RigiScan assessment were, of course, not representative of the general male population; there was probably a preponderance of men with psychological rather than physiological impotence in the present study. The prevalence of treated depression in men aged 50–80 years in the Stockholm region (men reporting to have been treated for depression at some time in their lives) has been estimated at 3% [ 2]; of the present patients, 8% reported treated depression, indicating an over-representation of depressed men. The prevalence of depression in the present patients may be partly caused by erectile problems, but only three of the 11 men reporting depression were ‘truly’ impotent according to the RigiScan.

 Assessing nocturnal erectile activity with a RigiScan device has previously been shown to be a valid and reliable measure of the physiological ability to obtain an erection [ 7[8][9]–10]. In the present study, the nocturnal erectile activity was assessed at home by the patients and may therefore be more prone to measurement error than assessment in a controlled clinical setting. However, in one study, the home assessment of nocturnal penile tumescence was reported to be as accurate as hospital recordings [ 8]

 There are varying definitions in the literature for potency; this confusing terminology may be responsible for the conflicting results about the preservation of potency after treatment for prostate cancer. According to the present definition, potency refers to the physiological ability to attain erectile rigidity ‘sufficient for intercourse most of the time’ or better. Table 4 summarizes the definitions of potency from several studies assessing the effect of prostate cancer and its treatment on erectile function. Some authors combine erectile function and aspects of sexual activity, like frequency of intercourse [ 3, 16, 28] in their definition of potency. Others include different aspects of sexual function in the definition, e.g. orgasm [ 6, 14, 15] and ejaculation [ 3, 17]. In one report, erection is graded on a stiffness scale but the threshold used is not mentioned [ 13] and in three reports, no definition of potency is given [ 4, 12, 18]. Finally, most authors defining potency as erectile capacity exclusively appear only to use one question for their potency assessment [ 5, 19, 21[22][23][24][25]–26].

Table 4.  Examples of different definitions of potency by authors assessing erectile capacity in patients with prostate cancer Thumbnail image of

 In conclusion, the test-retest reliability of the questionnaire assessment was satisfactory. ‘Physiological impotence’ may be diagnosed with complete and ‘physiological potency’ with 60–80% sensitivity using three questions assessing different aspects of erection in a self-administered questionnaire. A recommended definition for potency when using a subjective measurement of erectile rigidity and verbal categories is ‘rigidity sufficient for intercourse most of the time’ or better. The sensitivity of the potency assessment depends on the number of questions assessing erectile rigidity in different situations. Depression influences the quality of information and should always be included in studies as a covariant and adjusted for to improve the validity. Further research is required to determine if adding more questions assessing erectile rigidity would improve the sensitivity of the ‘physiological potency’.

Acknowledgements

We thank the Swedish Cancer Society and the Stockholm Cancer Foundation for financial support.

Ancillary