Cumulative response curves to enhance interpretation of treatment differences on the Self-Esteem And Relationship questionnaire for men with erectile dysfunction

Authors


Joseph C. Cappelleri, Pfizer Inc, 445 Eastern Point Road, MS 8260-2502, Groton, CT 06340, USA. e-mail: joseph.c.cappelleri@pfizer.com

Abstract

What's known on the subject? and What does the study add?

Studies on erectile dysfunction (ED) therapies rely heavily on patient-reported outcomes (PROs) to measure efficacy on treatment response. A challenge when using PROs is interpretation of the clinical meaning of changes in scores. A responder analysis provides a threshold score to indicate whether a change in score qualifies a patient as a responder. However, a major consideration with responder analysis is the sometimes arbitrary nature of defining the threshold for a response. By contrast, cumulative response curves (CRCs) display patient response rates over a continuum of possible thresholds, thus eliminating problems with a rigid threshold definition, allowing for a variety of response thresholds to be examined simultaneously, and encompassing all data.

With respect to the psychosocial factors addressed in the Self-Esteem And Relationship questionnaire in ED, CRCs clearly, distinctly, and meaningfully highlighted the favourable profiles of responses to sildenafil compared with placebo. CRCs for PROs in urology can provide a clear, transparent and meaningful visual depiction of efficacy data that can supplement and complement other analyses.

OBJECTIVE

  • • To use cumulative response curves (CRCs) to enrich meaning and enhance interpretation of scores on the Self-Esteem And Relationship (SEAR) questionnaire with respect to treatment differences for men with erectile dysfunction (ED).

PATIENTS AND METHODS

  • • This post hoc analysis used data from all patients who took at least one dose of study drug and had at least one post-baseline efficacy evaluation in a previously published 12-week, multicentre, randomized, double-blind, placebo-controlled trial of flexible-dose (25, 50, or 100 mg) sildenafil citrate (Viagra) in adult men with ED who had scored ≤ 75 out of 100 on the Self-Esteem subscale of the SEAR questionnaire.
  • • CRCs were used on the numeric change in transformed SEAR scores from baseline to end-of-study for each SEAR component.
  • • The horizontal axis of the CRC represented change from baseline on the SEAR score, and the vertical axis represented the percentage of patients experiencing that change or greater.
  • • The differences between CRCs for the sildenafil group vs the placebo group were assessed using the Kolmogorov–Smirov test.

RESULTS

  • • For each of the SEAR components, there was essentially no overlap in the CRCs between the sildenafil group (n= 113) and placebo group (n= 115 or 116, depending on the component), showing that a greater percentage of sildenafil recipients compared with placebo recipients had a more favourable change across the spectrum of response thresholds (P≤ 0.01).
  • • Previous research showed that a 10-point score increase is the minimal clinically meaningful improvement for most SEAR components. In the sildenafil vs placebo groups, a ≥10-point score increase occurred in 72 vs 37% of patients, respectively, on the Sexual Relationship Satisfaction domain, 71 vs 41% on the Confidence domain, 76 vs 49% on the Self-Esteem subscale, 60 vs 44% on the Overall Relationship Satisfaction subscale, and 75 vs 38% on the Overall score.

CONCLUSIONS

  • • With respect to the psychosocial factors addressed in the SEAR questionnaire, CRCs clearly, distinctly, and meaningfully highlighted the favourable profiles of responses to sildenafil compared with placebo.
  • • CRCs for patient-reported outcomes in urology can provide a clear, transparent, and meaningful visual depiction of efficacy data that can supplement and complement other analyses.
Abbreviations
CRC

cumulative response curve

SEAR

Self-Esteem And Relationship

ED

erectile dysfunction

PRO

patient-reported outcome

FDA

Food and Drug Administration

CDF

cumulative distribution function

INTRODUCTION

Patient-reported outcomes (PROs) are often used as measures of efficacy in clinical trials. In some disease states, PROs may be the primary assessment. Studies on erectile dysfunction (ED) therapies rely heavily on PROs to measure efficacy (e.g. treatment response and treatment satisfaction) [1].

The Self-Esteem And Relationship (SEAR) questionnaire is a validated, 14-item psychometric PRO measure specific to ED that is used to assess ED treatment efficacy in relation to psychosocial benefit [2]. The five components are the Overall score, the Sexual Relationship Satisfaction domain (items 1–8), the Confidence domain (items 9–14), and the latter's Self-Esteem (items 9–12) and Overall Relationship Satisfaction subscales (items 13–14) [2]. Items are scored on a 5-point Likert scale, with higher scores indicating better outcome. To facilitate interpretation, scores of the five SEAR components are standardized by transforming onto a scale of 0 to 100, with higher scores being more favourable. Sildenafil citrate (Viagra®, Pfizer Inc, New York, NY, USA) treatment for men with ED produced substantial and meaningful psychosocial benefits as assessed using the SEAR component and individual item scores [3–5].

A typical challenge when using PROs is interpretation of the scores, especially in urology where PROs abound. Often, because of the novelty of the measure, it can be difficult to interpret the clinical meaning or meaningful import of statistically significant changes in scores. Responder analyses provide a threshold score to indicate whether a change in a patient score has clinical meaning or, at least, a meaningful interpretation [6,7]. A responder analysis is based on a binary measure of whether a patient achieved a particular efficacy threshold and consequently qualified as a responder [6,7]. The US Food and Drug Administration (FDA) final guidance document on PROs promotes using an a priori responder definition [8], as does the Committee for Medicinal Products for Human Use in Europe [9]; however, a major consideration with responder analysis is the sometimes arbitrary nature of defining the threshold for a response [10].

Cumulative response curves (CRCs) display patient response rates over a continuum of possible thresholds, thus eliminating problems with a rigid threshold definition, and are recommended in the FDA PRO guidance document [8]. CRCs should be labelled and specified clearly to allow easy and clear interpretation with respect to the directionality of score changes (e.g. do positive changes indicate improvement or deterioration?) and their associated percentages of patients. Such cumulative distribution response curves, one for each treatment group, would allow a variety of response thresholds to be examined simultaneously and would encompass all data.

The objective of the present study was to generate distribution curves to enrich meaning and enhance interpretation of scores on the SEAR questionnaire with respect to treatment differences for men with ED.

PATIENTS AND METHODS

DATASET

Data were from a previously published 12-week, multicentre, randomized, double-blind, placebo-controlled trial in adult men (aged ≥ 18 years) with ED who had scored ≤75 out of 100 on the Self-Esteem subscale of the SEAR questionnaire and had the same partner for the duration of the trial [3]. Exclusion criteria were reported in the previous publication and included previous use of >6 doses of sildenafil, use of any other phosphodiesterase type 5 inhibitor for the treatment of ED or concurrent use of any other commercially available treatment for ED, use of nitrates or nitric oxide donors or of ritonavir, and any condition or circumstances that would impair the ability to participate reliably in the study or increase risk to the patient (i.e. blood pressure >170/110 mmHg, hypotension, significant cardiovascular disease in the last 3 months, or a history of retinitis pigmentosa) [3].

Patients were randomized to receive either sildenafil or a matching placebo. All patients initiated therapy with 50 mg sildenafil or a matching placebo. At weeks 2, 4 and 8, the investigator could adjust the dose to 100 mg or 25 mg based on criteria for insufficient efficacy or poor tolerability.

The SEAR questionnaire [2] was completed by the patient at baseline, week 8, and week 12 of the trial. The primary efficacy variable for the original trial was the mean change in score from baseline to week 12 in the SEAR Self-Esteem subscale.

DATA ANALYSES

For the current post hoc analysis, CRCs were constructed for each SEAR component (Sexual Relationship Satisfaction domain, Confidence domain, Self-Esteem subscale, Overall Relationship Satisfaction subscale, and Overall score). Data were from the full-analysis set, which consisted of all patients who took at least one dose of study drug and had at least one post baseline efficacy evaluation. Last-observation-carried-forward was applied for missing data at the week 12 visit, as was the case in the original analysis. Original SEAR scores were transformed onto a 100-point scale using the following equation:

Transformed score = 100 ×[(actual raw score-minimum of raw score)/(raw score range)].

Cumulative response curves were used on the numeric change in transformed scores (from baseline to end of study) for each SEAR component. The horizontal x-axis represented change from baseline on the SEAR score, and the vertical y-axis represented the percentage (or proportion) of patients experiencing that change or greater (better) (Fig. 1). Horizontal non-overlap of CRCs depicts the consistency with which patients experience a benefit. Vertical non-overlap shows the difference in responder rates for a variety of thresholds.

Figure 1.

Illustrative CRCs showing the active and control treatments. The horizontal x-axis represents change from baseline on the score of the PRO, in which a greater numeric change from baseline is more favourable. The vertical y-axis represents the percentage (or proportion) of patients experiencing that change or greater. Vertical non-overlap shows the difference in responder rates for a variety of thresholds. Horizontal non-overlap of CRCs depicts the consistency with which patients experience a benefit.

In the present report, we use the generic term ‘cumulative response curve’ rather than the more specific term ‘cumulative distribution function’ (CDF) because CDF technically refers to the proportion (or percentage) of patients who achieved less than or equal to each possible change score [6,11], whereas the CRC may be defined in the same way or in the way defined in this report, that is, as the proportion (or percentage) of patients who achieved greater than or equal to each possible change score. Thus, because higher scores on the SEAR questionnaire indicate a more favourable response, we chose to illustrate those scores using this type of CRC. Used in this manner, the CRC is downward shifting; it equals one minus the CDF and is a mirror image of the upward-shifting CDF.

The differences between CRCs for the sildenafil group vs the placebo group were analysed using the Kolmogorov–Smirov test. All analyses were performed in SAS Version 9.2. (SAS Institute Inc., Cary, NC, USA [12]).

RESULTS

DATASET

Overall, 256 patients were randomized, and 128 (sildenafil) and 125 (placebo) took at least one dose of study medication. SEAR change scores were available for 113 patients receiving sildenafil and for 115 or 116 patients receiving placebo, depending on the component. Patient characteristics were similar across groups (Table 1). Mean (sd) duration of ED was 4.6 (4.3) years in the sildenafil group and 3.8 (4.2) years in the placebo group.

Table 1. Patient characteristics at baseline
 Placebo n= 125Sildenafil n= 128
  1. IIEF EF, Erectile Function domain of the International Index of Erectile Function. Reproduced with permission of O'Leary MP, Althof SE, Cappelleri JC, Crowley A, Sherman N, Duttagupta S. Self-esteem, confidence and relationship satisfaction of men with erectile dysfunction treated with sildenafil citrate: a multicenter, randomized, parallel group, double-blind, placebo controlled study in the United States. J Urol 2006; 175(3 Pt 1): 1058–62.

Mean (sd; range) age, years55 (13; 23–81)56 (12; 30–83)
Race, %  
 White6658
 Black2524
 Asian22
 Other616
Mean (sd; range) weight, kg93 (18; 65–148)92 (16; 60–142)
Mean (sd; range) ED duration, years3.8 (4.2; 0.1–29.5)4.6 (4.3; 0.1–25.4)
Primary ED aetiology, n (%)  
 Organic62 (50)61 (48)
 Psychogenic15 (12)18 (14)
 Mixed48 (38)49 (38)
Mean (sd; range) SEAR scores  
 Sexual Relationship Satisfaction domain40 (21; 0–88)38 (21; 0–88)
 Confidence domain45 (24; 0–96)48 (23; 0–100)
  Self-Esteem subscale40 (24; 0–94)44 (23; 0–100)
  Overall Relationship Satisfaction subscale52 (29; 0–100)55 (31; 0–100)
 Overall score42 (21; 0–82)42 (20; 0–82)
Mean (sd; range) IIEF EF domain score13 (6;1–24)13 (6; 3–28)

At end-of-treatment, patients receiving sildenafil had significantly greater increases from baseline in scores on each SEAR component compared with patients receiving placebo, as previously reported [3]. Differences in mean change between groups (95% CI) for each SEAR component were as follows: Sexual Relationship Satisfaction domain, 21.5 (15.3–27.7), Confidence domain, 20.0 (13.7–26.2), Self-Esteem subscale, 19.4 (12.9–25.9), Overall Relationship Satisfaction subscale, 18.8 (11.4–26.2), and Overall score, 20.7 (14.9–26.6) (each P < 0.001, from t-test in an ancova model that adjusted for baseline score and centre).

CRCs

There was essentially no overlap in curves between the sildenafil and placebo groups for each of the SEAR components (Figs 2–6); Kolmogorov–Smirov P values were <0.05 for the Overall Relationship Satisfaction subscale and were <0.001 for all other components, thereby rejecting the null hypothesis of equality between the two distributions. The change in SEAR scores was distinct across all thresholds of relevance. This means that, at each of these thresholds, the group of patients receiving sildenafil had a greater response rate (percentage of patients with that amount of improvement or better) than patients receiving placebo.

Figure 2.

SEAR Overall score.

Figure 3.

SEAR Sexual Relationship Satisfaction domain score.

Figure 4.

SEAR Confidence domain score.

Figure 5.

SEAR Self-Esteem subscale score.

Figure 6.

SEAR Overall Relationship Satisfaction subscale score.

A change in score of at least 10 points is considered to be the clinically relevant response threshold on improvement for all components of the SEAR questionnaire except for its Overall Relationship Satisfaction subscale, where the evidence is inconclusive [13]. As shown in Table 2, for the Sexual Relationship Satisfaction domain, 72% of patients in the sildenafil group and 37% of patients in the placebo group had at least a 10-point score improvement and, therefore, met the response threshold. Similarly, in the sildenafil vs placebo groups, 71 vs 41% of patients had at least a 10-point score improvement on the Confidence domain, 76 vs 49% on the Self-Esteem subscale, 60 vs 44% on the Overall Relationship Satisfaction subscale, and 75 vs 38% on the Overall score.

Table 2. Percentage of patients experiencing a response threshold for each SEAR component
Response threshold (numerical change from baseline in SEAR component score)Percentage of patients experiencing response threshold, placebo (n= 115 or 116) vs sildenafil (n= 113)
Sexual Relationship Satisfaction domainConfidence domainSelf-Esteem subscaleOverall Relationship Satisfaction subscaleOverall score
  1. *P < 0.05; †P < 0.01; ‡P < 0.001; §P < 0.001.

 057 vs 82§60 vs 88§69 vs 8962 vs 8161 vs 84§
1037 vs 72§41 vs 71§49 vs 76§44 vs 60*38 vs 75§
2020 vs 63§30 vs 59§30 vs 64§29 vs 5021 vs 62§
3016 vs 52§18 vs 4024 vs 51§17 vs 3317 vs 47§
4010 vs 33§12 vs 2715 vs 27*11 vs 2710 vs 28
505 vs 24§7 vs 17*10 vs 22*11 vs 274 vs 16
602 vs 122 vs 9*4 vs 112 vs 123 vs 8

Relative to a given percentage of patients in the placebo group, the same percentage of patients in the sildenafil group experienced generally much higher (more favourable) change scores. On the Self-esteem subscale, for instance, 70% of the patients in the placebo group had a score improvement of 0 points or more, whereas 70% of the patients in the sildenafil group experienced a score improvement of 20 points or more (Fig. 5).

DISCUSSION

To illustrate the entire spectrum of treatment differences between sildenafil and placebo, post hoc analyses of efficacy data (SEAR score improvement) were conducted by constructing CRCs, which use all available data. These CRCs clearly show that response rates to sildenafil are superior to those of placebo over a continuum of response intervals or thresholds.

Cumulative response curves provide more detailed information than conventional responder analysis, which typically focuses only on one or two response thresholds. The latter is similar to a snapshot of the data; although it is easy to use and understand, it is best used when the responder definition is predetermined and has clinical meaning [7,8,14]. Instead of setting a single response threshold (e.g. ≥10-point improvement in score from baseline), CRCs encompass a range of response thresholds, such as the percentage of patients who have at least a 5-point improvement from baseline or those who have achieved at least a 20-point improvement from baseline. The vertical non-overlap of the curves shows the difference in responder rates for a variety of thresholds.

The advantage of cumulative distribution curves for PROs is the ability to intimately examine all of the data and show descriptive data from all available individuals. These curves are best used to supplement and complement, but not replace, primary analyses, which are typically based on a comparison of mean scores between treatments or a comparison of responder rates between treatments (with a predetermined threshold score for response).

The FDA recommends the use of CRCs to differentiate the responses between treatment groups in characterizing treatment effects and to help determine the extent to which the mean improvement reflects different response rates between groups [8]. A CRC can be plotted in multiple ways – for example, as the proportion of patients less than or equal to each possible change score (referred to specifically as the CDF) and as the proportion of patients greater than or equal to each possible change score (one minus the CDF). The choice of which type of score (original or transformed) and type of metric (percent or numeric change), as well as how the cumulative data are to be graphed, should be based on how a PRO was developed, analysed, interpreted and reported [15].

In the present paper, we chose the second method of plotting a CRC because higher SEAR scores indicate a more favourable response; therefore, use of the second method would be expected to provide a self-contained, intuitive, graphic display to best understand and visually appreciate treatment differences on change scores. However, if, in contrast to the SEAR, the PRO under examination indicated a more favourable response with a lower score, then we would have favoured the customary CDFs with their upward slopes.

To our knowledge, this is the first published report on the use of CRCs to describe treatment effects on a PRO in sexual medicine. Nonetheless, CDFs (one of the several types of CRCs) have been used to describe treatment effects on a PRO used to assess other diseases or conditions. For example, a previously published report used CDFs, in which lower scores indicate a more favourable response, to describe treatment effects on a PRO assessing quality of life in patients with chronic constipation [16].

Cumulative response curves can be useful in understanding and interpreting data from different kinds of urological conditions and studies, not just those in sexual medicine. In addition to being generally useful in characterizing dose–response curves in pharmacology [17], as well as in its actual use in depicting male sexual functioning as related to age (instead of treatment) in community men [18], CRCs can succinctly and meaningfully characterize symptoms of urinary urgency and associated health-related quality-of-life aspects for patients with overactive bladder and its treatment using, for example, the patient-reported Overactive Bladder Questionnaire [19]. Such CRCs could serve to complement and supplement previous research on the responsiveness of and important differences on the Overactive Bladder Questionnaire [20,21].

Cumulative response curves can also enrich the interpretation of patient-reported scores in the disease burden and treatment of various types of urological cancers. For example, CRCs could serve to enhance the interpretation of scores on the Expanded Prostate Cancer Index Composite for research [22] and its abbreviated form for clinical practice [23], thereby providing a comprehensive descriptive display to measure a broad spectrum of urinary, bowel, sexual, and hormonal symptoms in prostate cancer management. Although CRCs were not used in these psychometric evaluations in overactive bladder and prostate cancer [19–23], the application of CRCs would have been beneficial in interpreting change scores over time with respect to treatment intervention or simply absolute (raw) scores at a given time with respect to subgroups of interest. In general, CRCs in urology deserve consideration for many quantitative measures, whether patient-reported or not.

A limitation of the present study is that, although cumulative response analysis avoids the restriction of a fixed response threshold, it does not obviate the need to define a clinically meaningful response threshold [7]. Some understanding of what qualifies as a meaningful response is necessary to interpret cumulative distribution data because, as more of the range of the distribution is considered, the curves for the treatments will approach one another and eventually the curves will meet.

For the SEAR questionnaire, previous work established that a 10-point increase in score represents the minimal clinically meaningful improvement for all SEAR components, with the exception of the Overall Relationship Satisfaction subscale, and that the mean SEAR score of the superior treatment group should be at least 10 points higher than that of the inferior group to demonstrate clinical meaning [13]. Given that the results of the CRCs showed that, across the SEAR components, 60–76% of sildenafil-treated patients compared with 37–49% of placebo-treated patients had at least a 10-point score improvement (each P≤ 0.01), the CRC results support the ability of sildenafil treatment to achieve a clinically meaningful and robust improvement in SEAR components.

In conclusion, with respect to the psychosocial factors addressed in the SEAR questionnaire, CRCs clearly, distinctly and meaningfully highlighted the favourable profiles of responses to sildenafil compared with placebo. CRCs provide information that can supplement and complement a primary analysis based on responders or on mean scores. CRCs of PROs have relevance beyond any specific pair of treatment comparisons. Such curves can enrich meaningful interpretation of scores on PROs in general and, as illustrated in the present analysis, in sexual medicine and other aspects of urology.

ACKNOWLEDGEMENTS

This study was sponsored by Pfizer Inc. All authors actively participated in the design of the study, data collection, or interpretation of the results. Editorial support was provided by Deborah M. Campoli-Richards of Complete Healthcare Communications, Inc., and was funded by Pfizer Inc.

CONFLICT OF INTEREST

Joseph C. Cappelleri, Kelly H. Zou, Andrew G. Bushmakin, and Martin O. Carlsson are employees of Pfizer Inc. Tara Symonds is an employee of Pfizer Ltd.

Ancillary