SEARCH

SEARCH BY CITATION

Keywords:

  • acne;
  • patient reported outcome;
  • benefit assessment;
  • Patient Benefit Index;
  • Individualized weighted outcome assessment;
  • quality of life

Summary

  1. Top of page
  2. Summary
  3. Introduction
  4. Goals
  5. Methods
  6. Results
  7. Discussion
  8. Conflicts of interest
  9. References

Background: Benefit assessment of drugs and medical products has become a legally established feature of medical research. A standardized assessment of benefits using scientifically sound and valid methods is essential.

Objective: Development, validation and practical evaluation of an instrument to record patient benefit in treatment of acne.

Patients and Methods: In open interviews with n = 50 patients, possible benefits of the therapy from the patients' point of view were recorded. The item pool thus generated was reviewed by a panel of dermatologists, psychologists and patients and transferred to a 23-item questionnaire. This is used prior to therapy to assess patients' desired benefits and after therapy to record the perceived benefits. The therapy goals and the resulting benefits are then used to generate a weighted ‘Patient Benefit Index’ (PBI). The procedure has been tested for its validity and feasibility in n = 923 patients with acne.

Results: Patients accepted the instrument and deemed it to be easily understandable. Additionally, the method proved itself to be internally consistent, constructively valid and sensitive to changes.

Conclusions: The Patient Benefit Index (PBI) is a valid and highly accepted practical instrument for recording patient benefit. The PBI permits an individualized, patient-weighted assessment of the benefits of acne therapy.


Introduction

  1. Top of page
  2. Summary
  3. Introduction
  4. Goals
  5. Methods
  6. Results
  7. Discussion
  8. Conflicts of interest
  9. References

Numerous studies have shown that patients with acne can experience a dramatic reduction in quality of life (QoL) [1, 2, 3, 4, 5, 6]. The most important factors for the handicap are the stigmatization experienced or anticipated by the patient, disturbing subjective symptoms as well as the chronic, oftentimes therapy-refractory course. Even treatment itself can constitute a burden. The assessment of treatment benefit of drugs has become a central legislative and administrative procedure [7]. In many western nations, the benefit of drug treatment to the patient is one criterion for the reimbursement of treatment [8]. In Germany, the Statutory Health Insurance Modernization Act (GMG) 2004 and the Act to Strengthen Competition in Statutory Health Insurance (GKV-WSG) 2007serve as the legal basis for a central benefit evaluation in medicine. The Federal Joint Committee (Gemeinsamer Bundesausschuss, GBA) is responsible for benefit evaluation; it can commission the Institute for Quality and Economic Efficiency (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, IQWiG) to perform scientific benefit evaluation. Windeler (2006) defines “therapeutic benefit” as “all effects of an intervention that result in an improvement of prognosis and/or symptoms/quality of life of patients in more than an insignificant degree”[9]. In the law itself as well as in the regulations of the GBA and IQWiG, therapeutic benefit is defined as primarily “patient-relevant benefit”[10, 11, 12]. Primary features of patient-relevant benefit in the regulations and in SGB V (Book V of the Social Code) are reduction of mortality and morbidity, improvement in quality of life, patient satisfaction as well as a reduction of burdens of treatment. Recent studies on skin diseases show that patient-relevant benefit can only reliably be assessed by the patient [13]. In acne, clinical findings (lesions, severity) as well as disease-specific quality of life are important parameters in evaluating benefit. In practice, instruments used up to now to evaluate benefit often have not reflected the broad and individual spectrum of benefit to patients. Beyond a multitude of possible benefits, conventional instruments failed to include a patients-weighted assessment. Against this backdrop the Patient Benefit Index (PBI) was developed as a system for benefit assessment from the patient's perspective, where the patient chooses treatment goals which are relevant for him before starting from a standardized list of possible benefits in a weighted fashion and evaluates the achievement of these self-chosen goals after treatment. The first clinical trial of this novel concept was performed in patients with chronic wounds [14]. In the present study this instrument was applied to acne therapy and subsequently validated in clinical use.

Goals

  1. Top of page
  2. Summary
  3. Introduction
  4. Goals
  5. Methods
  6. Results
  7. Discussion
  8. Conflicts of interest
  9. References
  • 1
    Development and validation of an instrument to assess patient-defined benefit in acne therapy.
  • 2
    Testing the suitability of the instrument under clinical conditions.

Methods

  1. Top of page
  2. Summary
  3. Introduction
  4. Goals
  5. Methods
  6. Results
  7. Discussion
  8. Conflicts of interest
  9. References

Creating a pool of items of potential benefits on the basis of open patient and physician interviews

According to usual international standards in developing psychometric and biometric tests an item pool of personally relevant treatment benefits was generated first in an open patient interview (n = 50) and supplemented with additional items of individual burdens caused by the disease.

Development of a questionnaire by a board of experts with participation of patients

The resulting item pool was reduced by an expert board (two dermatologists, two psychologists and two patients) to 23 benefits of acne therapy particularly relevant to patients. The selected items were transferred to a questionnaire. To answer the items a five grade Likert scale was expanded by the possible answer “does not apply to me”.

Development of Patient Benefit Index (PBI)

Arriving at parameters of patient-defined benefits was by way of further development of goal-oriented measurement of results taking criticism into consideration [15, 16]. First, this method allows the patient to choose individual personal benefit preferences out of a standardized list of items (“Patient Needs Questionnaire”– PNQ, see Figure 1). This results from rating the 23 items according to their individual importance (0 =“not at all important”, “does not apply to me” up to 4 =“very important”). The “Patient Benefit Index” derives from the sum of the benefit items weighted by their respective relevance divided by the number of relevant items. Values between 0 = no benefit and 4 = maximal patient-defined benefit are possible. Analogously the achievement of the individually selected benefits (“Patient Benefit Questionnaire – BPQ, see Figure 2) is measured during the course of therapy (0 = therapy did not help at all” up to 4 =“therapy helped very much”) are measured. Finally, the preferences before therapy (PNQ) and the achieved benefit after therapy (PBQ) are converted into a weighted index value, the “Patient Benefit Index” (PBI). This is calculated by taking the individually determined importance for each item (PNQ) divided by the sum of all individual importances and multiplied by the respective benefit achieved (PBQ). The sum of these products is the PBI (Figure 3). This can have a value between 0 (no benefit) up to 4 (maximal benefit). On the basis of our own pilot studies, a minimum PBI value of ≥1 can be viewed as “relevant benefit”. The goal criterion “response” is the percentage of patients achieving a PBI ≥1.

image

Figure 1. Excerpt from the Patient Needs Questionnaire (PNQ), instructions and items 1–5.

Download figure to PowerPoint

image

Figure 2. Excerpt from the Patient Benefit Questionnaire (PBQ), introductory instructions and items 1–5.

Download figure to PowerPoint

image

Figure 3. Formula of Patient Benefit Index (PBI) with k need-items (w, range 0–4) and benefit-items (b, range 0–4).

Download figure to PowerPoint

Validation and test under clinical conditions

To validated the method and test its suitability under clinical conditions it was employed in a prospective observational study on 925 patients with acne. The patients were treated with a topical combination product containing 5% benzoyl peroxide and 1% clindamycin. The clinical results of this study are not subject of the present publication.

(1) Study design: Prospective, open, uncontrolled, multicenter cohort study in line with the recommendations of the Federal Institute for Drugs and Medical Devices as well as the German Society for Medical Informatics [17] for performing post-marketing surveillance studies.

(2) Patients and study centers: Questionnaires for up to 1,025 patients were distributed to n = 205 dermatology offices in Germany. Data from n = 925 patients were collected between September 2005 and March 2006. Patients of both genders aged 12–30 years with mild to moderate acne (physician's global assessment) were included.

(3) Collection of data: Data were collected from the physician and patient respectively at three time points: at the beginning of the study (V1), after 4–6 weeks (V2) and at the end of the study after 10–12 weeks (V3).

In addition to measuring patient-defined benefit and PBI, the quality of life was assessed in accordance with the appropriate AWMF guideline [18]. As an instrument to assess disease-specific quality of life in acne, the ADI (Acne Disability Index [19]) was employed, an internationally widely used instrument with 5 items which assess social and mental aspects of quality of life.

Methods to test validity of PBI

In accordance with the AWMF guideline 2004 of the German Society of Dermatology [18], the following criteria for validation were considered: (1) Clinical use (“feasibility”) (2) Item characteristics (distribution, distinction, internal consistency) (3) Constructive validity with respect to patient satisfaction, to self and foreign assessment of treatment success as well as to the Acne Disability Index [2, 19] (4) Sensitivity to change during topical therapy

Results

  1. Top of page
  2. Summary
  3. Introduction
  4. Goals
  5. Methods
  6. Results
  7. Discussion
  8. Conflicts of interest
  9. References

Feasibility

The good practicability of the method in clinical use is especially reflected in the low rate of missing responses to the items (missing values per item 1.7%, median = 5, range: 2–14 missing values). In a sample of n = 50 patients 95% of patients responded that the questionnaire was principally well understandable and easy to complete and from its content took benefit needs of acne patients well into consideration.

Patient-defined benefit preferences

From the response to the importance of treatment goals of the patients at time point V1 (begin of therapy) a broad spectrum of patient-defined benefit preferences is seen (n = 832, see Table 1). The distribution shows that acne patients formulate treatment goals of varying relevance which should not be neglected when assessing benefit relevant for the patient.

Table 1.  Results on the importance of therapy goals in the patients' view at time point V1 (beginning of the treatment, descending order, n = 832).
Therapy goalMeanSDApplies to meDoes not apply to meMissingPortion important/very important (%)
  1. 0 = not at all, 1 = somewhat, 2 = moderately, 3 = quite, 4 = very important.

To be healed of all skin lesions3.620.77788202488.5
To have faith in the treatment3.221.04751612074.6
To find a clear diagnosis and treatment2.991.22718922262.8
To want to show oneself more2.961.21712962461.1
To have no fear of disease progression2.841.326801292355.5
To be more cheerful2.811.286401652751.9
To lead a normal daily life2.741.305772292645.3
To be less burdened in a partnership2.681.354653452235.9
To have more contacts to other people2.641.335902182442.9
To spend less time on daily therapy2.621.27746612553.1
To be able to pursue normal recreational activities2.561.375312812037.6
To be less depressed2.561.356331772244.9
To no longer experience burning of the skin2.511.345212793235.2
To have less side effects2.491.396231852441.5
To need less physician or hospital consultations2.461.306801302244.5
To lead a normal sex life2.441.474133922727.9
To have less therapy costs2.431.426321752541.2
To no longer experience itching2.351.415132972232.9
To lead a normal professional life2.321.453924182223.9
To be free of pain2.271.425093022130.3
To be more productive in daily life2.261.364643442426.7
To be less of a burden on family and friends2.061.404893202325.5
To be able to sleep better1.881.473584522216.5

Internal consistency of PNQ

In the validation study the internal consistency of PNQ showed good characteristics. With respect to all cases, Cronbachs alpha was in a very good range with α= 0.96. The distinction between items when building sum in the PNQ was satisfactory (Table 2), which again underscores the variability of individual treatment goals and confirms the inventory-like nature of the method as well as the need for individual weighting in determining the PBI.

Table 2.  Item characteristics at time point V1 and with the respective item distinction Cronbachs alpha.
Therapy goalCorrected item-scale correlationCronbachs alpha, when item is excluded
To be free of pain0.6120.960
To no longer experience itching0.6600.960
To no longer experience burning on the skin0.7160.959
To be healed of all skin lesions0.3700.962
To be able to sleep better0.6690.960
To be less depressed0.7960.958
To be more cheerful0.7630.959
To have no fear of disease progression0.7120.959
To lead a normal daily life0.8020.958
To be more productive in daily life0.7990.958
To be less of a burden on family and friends0.7290.959
To be able to pursue normal recreational activities0.7630.959
To lead a normal professional life0.7960.958
To have more contacts to other people0.7560.959
To want to show oneself more0.6790.960
To be less burdened in a partnership0.7460.959
To lead a normal sex life0.7530.959
To need less physician or hospital consultations0.7480.959
To spend less time on daily therapy0.6500.960
To have less therapy costs0.6940.959
To have less side effects0.7450.959
To find a clear diagnosis and treatment0.6360.960
To have faith in the treatment0.5730.961

Constructive validity of PNQ

At time point V1 (begin of the study) the sum of importances in the PNQ correlates to r = 0.59 (p < 0.01, n = 764) with the LQ score of ADI which suggests a convergent validity. The PNQ does display test-specific variance, pointing to a discriminant validity of the PNQ.

Patient-defined Benefit Index

Distribution characteristics of the PBI: At the time point V2 (after 4–6 weeks) 90.2% of patients achieved a PBI > 1; at time point V3 (after 10–12 weeks) 92% of patients did so (Figures 4, 5). The corresponding mean PBI was 2.45 (SD 1.02, n = 780) and 2.79 (SD 1.04, n = 731) at time point V3. Thus, a majority of patients achieved a high patient-defined benefit.

image

Figure 4. Frequency distribution of Patient Benefit Index (PBI) after 4–6 weeks acne therapy. Mean = 2.45; Std. Dev. = 1.02; n = 780.

Download figure to PowerPoint

image

Figure 5. Frequency distribution of Patient Benefit Index (PBI) after 10–12 weeks acne therapy. Mean = 2.79; Std. Dev. = 1.04; n = 731.

Download figure to PowerPoint

Constructive validity of the PBI:Table 3 summarizes the results with convergent validity of the PBI at time point V2 (after 4–6 weeks of therapy) and V3 (after 10–12 weeks of therapy). In general middle to high degrees of correlation between the PBI and relevant parameters of patient satisfaction, efficacy of therapy out of the physician's and patient's perspective as well as quality of life were observed. This suggests convergent validity of the PBI, as it correlates to relevant validation constructs as well dis-criminant validity of the PBI, as other variances are obviously also included.

Table 3.  Construct validity of the PBI: Correlation with other outcome parameters at time point V2 and V3.
  1. r: Pearson correlation coefficient; p: Significance (2-sided each); n: Sample size: V2: Visit 2 after 4–6 weeks; V3: Visit 3 after 10–12 weeks.

Correlation of the PBI with …r =pn =
Further recommendation at V20.5320.01776
Global patient assessment at V20.5410.01774
Global physician assessment at V20.3860.01771
Global physician assessment at V30.4570.01725
Physical well-being at V30.5380.01728
Mental well-being at V30.6320.01724
Productivity in profession and daily life at V30.4570.01708
Social contacts at V30.5470.01726
Recreational activities at V30.5250.01719
QoL in general at V30.6270.01725

Sensitivity to change: In the course of therapy the PBI displayed parallel to further clinical improvement between time points V2 and V3 good sensitivity to change (Figures 4, 5).

Discussion

  1. Top of page
  2. Summary
  3. Introduction
  4. Goals
  5. Methods
  6. Results
  7. Discussion
  8. Conflicts of interest
  9. References

The goal of the present study was the development, validation and testing of the clinical suitability of an instrument to assess patient-defined benefit in acne therapy.

The development of the method was in accordance with internationals standards for developing psychometric and biometric instruments [20, 21, 22, 23, 24, 25, 26, 27], the AWMF guidelines of the DDG (German Society for Dermatology) to assess quality of life in dermatology [18] and already takes benchmarks of IQWiG and the GBA on measuring patient benefit into consideration. Development of the questionnaire which can be applied to other skin diseases was done by an interdisciplinary team that included patients.

The results of the PNQ show that acne patients expect a broad spectrum of possible benefits from therapy. Determining the PBI originally was modeled on the not well-standardized methods of “goal attainment scaling” used in rehabilitative medicine [28, 29, 30, 31, 32] and psychiatry [33, 34] to verbally formulate treatment goals and to check their attainment at intervals. This method has also been applied in nursing [35, 36]. The limitations of “goal attainment scaling” are, among others, the low-degree of standardization of the method and the lack of external validation [37]. As a further, more structured method goal-oriented result measurement (ZOE = zielorientierte Ergebnismessung) was introduced [15] also termed goal attainment scaling by other authors [16]. Here patient goal selection is done in a systematic and prestructured form. The PBI is a further modification of this method including a dimensionally scaled assessment of individual patient-defined therapy goals and benefits which are weighted and summarized by a single benefit index. The PBI allows for the depiction of benefit functions over the course of time or in the comparison of cohorts. With the PBI it is possible to avoid major criticism of ZOE [16]; by, for example, assessing two differing constructs (importance and benefit) a regression to the middle is blocked and a positive distortion of effects is minimized by the dimensional conception of weighting. The questionnaire PBI for acne was shown to be valid and reliable in the validation study. Is practicability in terms of patient acceptance and understandability was good. In a clinical study on patients with mild to moderate acne, the utility of the PBI was shown on over 900 patients as well as substantially a high degree of benefit of therapy with a combination product containing benzoyl peroxide and clindamycin. The reduction in case numbers in the course of the study in comparison to the time point of inclusion is not unusual in an observational study. It might lead to a clinically relevant selection effect but does not affect the validation test. In conclusion, the PBI is a benefit instrument with which the patient benefit of acne therapy can simply and reliably be assessed. While it can be an advantage for a differential benefit evaluation and for health services planning to depict a broad spectrum of benefit relevant to the patient – as the PNQ and PBQ do, outcome research requires a bundled benefit parameter, which summarizes patient benefit in a single value, if possible. Being an index, this is possible with the PBI. The distribution of characteristic values of the PBI of different samples can also directly be compared and be used in outcome analyses. The combined use of several outcomes methods with a) clinical score, b) quality of life questionnaire and c) patient-defined benefit evaluation (PBI) would be advisable.

Conflicts of interest

  1. Top of page
  2. Summary
  3. Introduction
  4. Goals
  5. Methods
  6. Results
  7. Discussion
  8. Conflicts of interest
  9. References

This study was supported by a research grant from Stiefel GmbH, Offenbach/-Germany.

References

  1. Top of page
  2. Summary
  3. Introduction
  4. Goals
  5. Methods
  6. Results
  7. Discussion
  8. Conflicts of interest
  9. References
  • 1
    Lasek RJ, Chren MM. Acne vulgaris and the quality of life of adult dermatology patients. Arch Dermatol 1998; 134(4): 454458.
  • 2
    Salek MS, Kahn GK, Finlay AY. Questionnaire techniques in assessing acne handicap: reliability and validity study. Quality Life Res 1996; 5: 131138.
  • 3
    Mallon E, Newton JN, Klassen A, Ryan TJ, Finlay AY. Standard patient assessed quality of life instruments can be used to measure the benefits of acne treatment. Br J Dermatol 1995; 133(Suppl. 45): 35.
  • 4
    Simpson N. Effect of Isotretinoin on the Quality of Life of Patients with Acne. Pharmacoeconomics 1994; 6(2): 108113.
  • 5
    Simpson NB. Social and economic aspects of acne and the cost-effectiveness of isotretinoin. J Dermatol Treatment 1993; 4, Suppl. 2: 69.
  • 6
    Motley RJ, Finlay AY. How much disability is caused by acne? Clin Exp Dermatol 1989; 14: 194198.
  • 7
    Rychlik R, Rusche H, Augustin M. Systematik der Nutzenbewertung von Arzneimitteln. Gesundh Ökon Qual Manag 2004; 9: 245252.
  • 8
    Zentner A, Busse R. Kriterien der Nutzenbewertung von Arzneimitteln im internationalen Vergleich. HTA-Bericht, DIMDI 2005.
  • 9
    Windeler J. [Benefit and benefit assessment] Dtsch Med Wochenschr. 2006 May 12; 131(19 Suppl. 1): S1215.
  • 10
    Bastian H, Bender R, Ernst AS, Kaiser T, Kirchner H, Kolominsky-Rabas P, Lange S, Sawicky PT, Weber M. Methoden. Version 2.0 vom 19.12.2006. Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen, Köln 2007.
  • 11
    Gemeinsamer Bundesausschuss. Verfahrensordnung vom 20.09.2005. Bundesanzeiger 2005; 242: 16998.
  • 12
    Gesetz zur Stärkung des Wettbewerbs in der gesetzlichen Krankenversicherung vom 26.03.2007. Bundesanzeiger 2007; 66: 3613.
  • 13
    Zschocke I, Hammelmann U, Augustin M. Therapeutischer Nutzen in der dermatologischen Behandlung. Hautarzt 2005; 56: 839846.
  • 14
    Augustin M, Zschocke I. Nutzenbewertung der ambulanten und stationären VAC-Therapie aus Patientensicht: Multizenterstudie mit Patientenrele-vanten Endpunkten. MMW-Fortschr. Med. Originalien 2006; 148, Nr. I: 2532.
  • 15
    Steffanowski A, Lichtenberg S, Schmidt J, Huber C, Wittmann WW, Nübling R. Ergebnisqualität psychoso-matischer Rehabilitation: Zielerrei-chungsskalierung auf der Basis einer strukturierten Therapiezielliste. Rehabilitation 2004; 43: 219232.
  • 16
    Zwingmann C. Zielorientierte Ergebnismessung (ZOE) mit dem IRES-Patientenfragebogen – Eine kritische Zwischenbilanz. Rehabilitation 2003; 42: 226235.
  • 17
    Victor N, Windeler J, Hasford J, Köpcke W, Linden M, Michaelis J, Röhmel J, Schäfer H. Empfehlungen zur Durchführung von Anwendungs-beobachtungen. Informatik, Biometrie und Epidemiologie in Medizin und Biologie 1997; 28: 247252.
  • 18
    Augustin M, Amon U, Braathen L, Bullinger M, Gieler U, Klein GF, Schultz-Amling W. Erfassung von Le-bensqualität in der Dermatologie (Leit-linie). JDDG 2004; 9: 802806.
  • 19
    Motley RJ, Finlay AY. Practical use of a disability index in the routine management of acne. Clin Exp Dermatol 1992; 17: 13.
  • 20
    Bowling A. Measuring disease: a review of disease-specific quality of life measurement scales. Open University Press, Buckingham 1995.
  • 21
    Brazier J, Dixon S. The use of condition specific outcome measures in economic appraisal. Health Econ 1995; 4(4): 255264.
  • 22
    Brock D. Quality of Life Measures in Health Care and Medical Ethics. In: SpilkerB (Hg.): Quality of Life and Pharmacoeconomics in Clinical Trials. 2nd Edition. Philadelphia : Lippincott-Raven, 1996: 487510.
  • 23
    Drummond M, McGuire A. Economic Evaluation in Health Care. Oxford University Press 2001.
  • 24
    Feeny DH, Torrance GW, Labelle R. Integrating Economics Evaluations and Quality of Life Assessments. In: SpilkerB (Hg.): Quality of Life and Pharma-coeconomics in Clinical Trials. 2nd Edition. Philadelphia : Lippincott-Raven, 1996: 8595.
  • 25
    Guyatt GH, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chron Dis 1987; 40: 171178.
  • 26
    Lienert G. Testaufbau und Testanalyse. 5. Auflage, Beltz , Weinheim, 1994.
  • 27
    SmithGT (Hg.). Measuring Health: a Practical Approach. John Wiley, Chichester, NY, Brisbane, Toronto, Singapore , 1988.
  • 28
    Clark MS, Caudrey DJ. Evaluation of rehabilitation services: the use of goal attainment scaling. Int Rehabil Med 1982; 5: 4145.
  • 29
    Ottenbacher KJ, Cusick A. Goal attainment scaling as a method of clincal service evaluation. Am J Occup Ther 1990; 44: 519525.
  • 30
    Donnelly C, Carswell A. Individualized outcome measures: A review of the literature. Can J Occup Ther. 2002 Apr; 69(2): 8494.
  • 31
    Fisher K, Hardie RJ. Goal attainment scaling in evaluating a multidisciplinary pain management programme. Clin Rehabil 2002; 16: 871877.
  • 32
    Rushton PW, Miller WC. Goal attainment scaling in the rehabilitation of patients with lower-extremity amputations: A pilot study. Arch Phys Med Rehabil 2002; 83: 771775.
  • 33
    Rockwood K, Howlett S, Stadnyk K, Carver D, Powell C, Stolee P. Responsiveness of goal attainment scaling in randomized controlled trial of comprehensive geriatric assessment. J Clin Epidemiol 2003; 56: 736743.
  • 34
    Steinbook RM, Jacobson AF, Mosher JC, Davies DL. The goal attainment scale: An instructional guide for the delivery of social reinforcement. Arch Gen Psychiatry 1977; 34: 923926.
  • 35
    Gordon JE, Powell C, Rockwood K. Goal attainment scaling as a measure of clinically important change in nursing-home patients. Age Ageing 1999; 28: 275281.
  • 36
    Turnbull J. Evaluating health care using goal attainment scaling. Nursing Standard 1998; 12: 3538.
  • 37
    Boothroyd RA, Banks SM, Evans ME, Greenbaum PE, Brown E. Untangling the web: An approach to analyzing the impacts of individually tailored, multi-component treatment interventions. Mental Health Services Research 2004; 6: 143153.