Drs. Ruof and Huelsemann contributed equally to this work.
Patient-reported health care utilization in rheumatoid arthritis: What level of detail is required?
Article first published online: 11 OCT 2004
Copyright © 2004 by the American College of Rheumatology
Arthritis Care & Research
Volume 51, Issue 5, pages 774–781, 15 October 2004
How to Cite
Ruof, J., Huelsemann, J. L., Mittendorf, T., Handelmann, S., Von Der Schulenburg, J. M., Zeidler, H., Aultman, R. and Merkesdal, S. (2004), Patient-reported health care utilization in rheumatoid arthritis: What level of detail is required?. Arthritis & Rheumatism, 51: 774–781. doi: 10.1002/art.20686
- Issue published online: 11 OCT 2004
- Article first published online: 11 OCT 2004
- Manuscript Accepted: 3 DEC 2003
- Manuscript Received: 18 AUG 2003
- Kompetenznetz Rheumatologie. Grant Number: C5.1
- German Ministry of Education and Research
- Health care utilization;
- Rheumatoid Arthritis
To investigate the level of detail required in self-reported health care utilization questionnaires for administration to patients with rheumatoid arthritis (RA).
A preliminary questionnaire was developed on the basis of existing tools for use in rheumatic conditions and in-depth interviews with 10 RA patients. Data gathered over 1 year of administration in a clinical setting were then matched to a comprehensive database of payer-reported information. Kappa statistics were calculated for each health care utilization domain. For domains where disaggregation into metric data was potentially preferable, histograms of difference were assessed visually and the strength of association examined using Spearman's rank correlation coefficient.
Patients (n = 136) included in the base case analysis determined the preferred levels of detail for each domain. Physician visits: occurrence of physician visits (yes/no; κ not applicable) and their number (r = 0.42, P < 0.001). Medication use of the following drug classes (yes/no): disease-modifying antirheumatic drug (DMARD; κ = 0.68), nonsteroidal antiinflammatory drug (κ = 0.64), osteoporosis medication (κ = 0.56), analgesic (κ = 0.38), and steroid (κ = 0.83). Further disaggregation into different DMARD classes was recommended (κ ranging between 1 [use of biologics: yes/no] and 0.67 [use of azathioprine: yes/no]. Imaging: imaging of bones and chest (yes/no; κ = 0.20). Hospitalization: inpatient episodes (yes/no; κ = 0.64) and number of inpatient days (r = 0.80, P < 0.001). Transport: costs incurred (yes/no; κ = 0.13) and amount (r = 0.39, P < 0.001).
The use of highly aggregated items to assess health care utilization in RA is supported. Dichotomous assessment (yes/no) was the preferred level of detail for items in the domains covering medication and diagnostic procedures or tests. Metric data is appropriate in 3 areas: number of physician visits, number of inpatient days, and total expenditure on transportation.
The value of the information provided by economic evaluations of health care depends largely on the use of valid and comparable measures of both resource consumption and disease consequences (1, 2). Although considerable attention is paid to ensuring valid and reliable measurement of clinical outcomes, assessment of health care utilization has yet to be standardized. It is therefore no surprise that economic evaluations are often incomplete and differ markedly with regard to the domains they consider, the costing methods applied (3), and consequently, the health care use they report (4).
The most frequently used data on health care uptake are derived from patient-reported questionnaires. However, analysis of the components of internationally accepted instruments for use in rheumatic conditions reveals considerable differences in the level of detail required for each item (5). The number of items ranges from 3 to 113 per questionnaire, and only 4 of 15 published instruments include an assessment of their own validity and reliability (6–9).
The cost of medication is one of the most relevant resource domains in rheumatoid arthritis (RA), yet the level of detail sought varies enormously. The questionnaire by Ritter et al (9) does not cover medication at all, whereas Goossens et al (8) and Guzman et al (7) include a free-text item asking patients to list their prescribed medication by name. The Arthritis Research Project (10) questionnaire lists more than 60 drug names on 9 pages and includes Likert response items for dosage and number of days and months of drug use. To our knowledge, no investigator has yet determined whether the high level of detail requested in comprehensive health care utilization questionnaires results in more accurate cost estimates.
The goal of the present study was to determine the most appropriate level of detail (or item aggregation) for use in health care utilization surveys in RA.
PATIENTS AND METHODS
A preliminary health care utilization questionnaire was developed and applied in a cohort of RA patients. The data generated were then compared with a matching set of objective, payer-reported data on a patient-by-patient microcosting level to identify the optimum degree of aggregation.
Construction of the preliminary health care utilization questionnaire.
The basis of the current analysis is a matrix of relevant health care utilization domains developed by the authors specifically for the purpose and reported in detail elsewhere (11, 12). Particular attention was paid to domains for which corresponding, objective payer-reported data were available (i.e., physician visits, outpatient surgery, medication, diagnostic procedures and tests, inpatient care, transportation). Domains for which matching objective data were not available (i.e., use of nonphysician services, devices and aids, home health care services, productivity costs) were excluded from the analysis. Patients often contribute to the costs of such services, and it was therefore not considered appropriate to validate patient-reported health care use against payer reports.
The second step included the collection and analysis of all internationally applied health care utilization questionnaires in RA (detailed results reported elsewhere) (5). The items used were then grouped according to the matrix described above and systematically analyzed with regard to level of detail required, length, type and wording of items, and availability of psychometric characteristics.
Domain by domain, a pool of potential items was generated and used by 2 of the authors (JR, SM) as a framework for in-depth interviews with 10 RA patients (5 per interviewer). Each interview was structured according to the matrix of health care utilization domains (11). Domain by domain, patients were exposed to the pool of potential items and to various aggregation levels of items (e.g., to determine the preferred level of aggregation for medication, patients were exposed to both a free-text item that allowed them to actively list their medication and, as an alternative, to a comprehensive list of drugs where they just had to mark type, formulation, and dosage of drugs they received). The feedback of the patients was collected and compared, and any controversial areas resolved by discussion with the other authors. There was an agreement to include a more detailed aggregation level in case of controversy.
An appropriate level of item aggregation and wording was agreed upon and a preliminary questionnaire was compiled.
Application of the questionnaire.
The newly developed instrument was completed by a cohort of RA patients every 4 months for 1 year. The data generated was compared with objective, payer-reported health care use information on a patient-by-patient basis.
The patients included in this study were recruited through the private offices of 14 rheumatologists in Lower Saxony, 1 of 16 regional states in Germany. Inclusion criteria were as follows: 1) RA according to accepted diagnostic criteria (13); 2) age >18 years; 3) membership of the Allgemeine Ortskrankenkasse Niedersachsen (AOKN; the largest regional statutory sickness fund in Lower Saxony) (14); 4) at least 1 previous consultation with the same rheumatologist; and 5) written consent. Baseline clinical and demographic variables were collected by the rheumatologists.
The preliminary questionnaire was completed by patients at baseline and then once per quarter throughout 2001 to give a total of 4. The first form was filled in at the physician's office, and the others were sent by mail. A system was developed to ensure that patients were reminded to return forms if necessary.
Matching objective health care utilization data for the same patients were obtained from the payers: the AOKN and the Kassenärztliche Vereinigung Niedersachsen (KVN; the regional doctors' association). The AOKN covers medical care for 2.3 million members in Lower Saxony (14). Data received from AOKN covered the domains of medication, inpatient health care, and transportation. Payment of physicians for outpatient care in Germany involves 2 steps. First, the sickness funds pay the regional physicians' associations for all affiliated doctors. The total amount is negotiated as a capitation for each member or for each insured person. Second, the associations distribute this lump sum among their members according to a systematic points system: the “Uniform Value Scale” (EBM) (15). Because medical services approved for reimbursement are listed in the EBM, all outpatient cost data in Germany are collected and managed by the physicians' associations. The transmission of those data for outpatient services from the physicians' associations to the sickness funds is prohibited by federal law. Hence, the KVN was asked for information on all outpatient health care costs incurred. Data from AOKN and KVN were matched in a single database. The matching was performed on a patient-by-patient basis using anonymous codes. Details of the microcosting approach used are reported elsewhere (16).
The course of the study was covered by a contract between the AOKN, KVN, and the Hannover Medical School, and the study design was approved by the school's local ethics committee. Data transfer procedures and data protection measures were approved by the Social Ministry of Lower Saxony, Germany. Each patient signed an informed consent form.
Health care utilization as reported by patients versus payers was structured according to the relevant domains (i.e., physician visits, outpatient surgery, medication, diagnostic procedures and tests, inpatient care, and transportation).
Cross tables were generated to examine the association between payer and patient reports in a dichotomous yes/no format (i.e., patients/payers reported consumption or nonconsumption of health care in a particular domain). Kappa statistics (κ) were calculated and interpreted as suggested by Kramer and Feinstein (17): values of 0.41–0.60 were considered to indicate moderate agreement; 0.61–0.80, substantial agreement; and >0.80, almost perfect agreement.
A full set of descriptive statistics (mean, median, standard deviation, standard error, maximum, minimum, sum) was developed for all cost domains where metric data were collected (i.e.. physician visits, bone and chest imaging, laboratory tests, hospital stays, transportation). Histograms of difference (patient-reported minus payer-reported values) were visually assessed as suggested by Bland and Altman (18). The association between patient-reported and payer-reported values was examined using Spearman's rank correlation coefficient.
Payer-reported data were complete, but some patient followup information was missing. Because it was not considered appropriate to impute missing values, data were analyzed from 2 populations: all patients recruited (including those who failed to return questionnaires) and the subset who returned all 4 questionnaires. The latter were used as the base case and compared with the former to check for a patient-selection bias.
Data were managed using Microsoft ACCESS software (version 8.0; Redmond, WA). Analyses were conducted in SPSS (Chicago, IL) (19).
Design of the preliminary questionnaire.
The in-depth interviews revealed a clear trend toward a relatively high aggregation level of items. This trend and its impact on the assessment of individual cost domains was discussed among the group of authors. The preliminary health care utilization questionnaire designed on the basis of literature review and in-depth interviews consisted of 12 items subdivided into 3 major categories: outpatient care, medication, and inpatient care (copy of questionnaire may be provided by authors upon request). It took patients about 7 minutes to complete and no complaints were received about wording or the feasibility of any item.
Demographics and data collection.
At the 2001 baseline visit, 227 questionnaires were returned; 160 were returned at quarter 2; 204 at quarter 3; and 195 at quarter 4. Complete sets of questionnaires were returned by 136 respondents. An exploratory sensitivity analysis was conducted on all 227 patients. Demographic characteristics of the initial group of 227 and the 136 complete responders are shown in Table 1.
|Variable||Completers (n = 136)||All participants (n = 227)|
|Age, mean ± SD years||57.4 ± 12.5||58.4 ± 11.8|
|Disease duration, mean ± SD years||8.2 ± 8.4||8.4 ± 8.6|
|No. swollen joints, mean ± SD||5.5 ± 6.2||5.2 ± 6.1|
|ESR, mean ± SD mm/hour*||16.2 ± 17.3||17.7 ± 17.1|
|Rheumatoid factor positivity, %||60.3||63.4|
|Erosive changes, %†||59.6||56.8|
|Social variables, %|
|Apprenticeship as highest educational level||42.6||42.3|
|University degree as highest educational level||1.5||1.8|
Determination of preferred aggregation level.
Payer-reported data were matched with the corresponding questionnaire items. The cross tables and kappa statistics are displayed in Table 2. Table 3 shows additional results for all cost domains where more than a simple yes/no response was required. An overview of the suggested aggregation level of items is shown in Table 4. Data for physician visits, outpatient surgery, medication, and diagnostic procedures and tests refer to outpatient care utilization only. Inpatient costs were collected through highly aggregated summary items that did not discriminate specific utilization that occurred during the inpatient episodes. Results for individual domains follow.
|Domain||Payer reported||Patient reported||Spearman's rank correlation coefficient|
|Mean (SEM)||Median||Minimum/ maximum||Mean (SEM)||Median||Minimum/ maximum|
|Physician visits||27.9 (1.7)||23||0/90||21.7 (1.5)||17||3/113||0.42*|
|Inpatient days||5.5 (1.0)||0||0/57||5.7 (1.6)||0||0/183||0.80*|
|Transportation costs, ε||56.7 (17.4)||0||0/1,536||62.5 (9.9)||10||0/644||0.39*|
|Domain||Suggested level of aggregation||Source of evidence|
|Outpatient health care utilization (physical units)|
|Physician visits||Physician visits yes/no and number of visits||Data matching|
|Outpatient surgery||No separate data collection recommended||Interview, data matching|
|Medication||DMARD yes/no (subaggregation into DMARD subgroups possible, i.e., methotrexate, biologics, leflunomide, sulfasalazine, antimalarials)||Interview, data matching|
|Osteoporosis medication yes/no|
|No disaggregation into dosing regimens|
|Diagnostic procedures and test imaging||Imaging yes/no (disaggregate into CT and MRI scans)||Data matching|
|Laboratory||Laboratory investigations yes/no; no disaggregation into number and type|
|Inpatient health care utilization (physical units)|
|Inpatient episodes||Hospitalization yes/no and number of inpatient days||Interview, data matching|
|Nonmedical health care utilization (monetary units)|
|Transportation trips||Total expenditure||Interview, data matching|
On a dichotomous yes/no level, physician visits were reported by all 136 patients, and by payers on behalf of 133 of 136 patients (κ not applicable), suggesting that disaggregation to determine the number of physician visits would be appropriate. The correlation between patient-reported and payer-reported visits was r = 0.42 (P < 0.001). Histograms of difference (patient-reported minus payer-reported data) indicate that patients tend to underreport visits. Further analyses were conducted to determine whether patients accurately report the specialties of physicians consulted (general practitioner, rheumatologist, gastroenterologist, etc.). Respective kappa statistics were consistent at κ < 0.2. The Spearman correlation matrix revealed only nonsignificant correlation coefficients, with r < 0.1.
The in-depth interviews had already indicated that collection of health care utilization data in this domain would be a challenge, a view that was confirmed by empirical data collection. The cross tables indicated a mismatch between payers' and patients' reports in both directions, i.e., patients reported procedures that were not confirmed by the payers (n = 14) and vice versa (n = 8). In 4 cases, outpatient procedures may have been confused with inpatient episodes.
RA-specific medications were categorized as disease-modifying antirheumatic drugs (DMARDs), nonsteroidal antiinflammatory drugs (NSAIDs), steroids, osteoporosis medications, and analgesics. Within each drug class, patients were asked to list the names of the specific medications they received. Information was not sought on dosing regimens because only 2 of 10 patients correctly recalled their DMARD dosages during the in-depth interviews. Cross tables indicated moderate (analgesics; κ = 0.38) to almost perfect (steroids; κ = 0.83) agreement between patient and payer reports. Further subclassification of DMARDs was successful. The kappa statistics were as follows: biologics κ = 1.0, methotrexate κ = 0.81, antimalarials (chloroquine and hydroxychloroquine) κ = 0.83, azathioprine κ = 0.67, leflunomide κ = 0.93, sulfasalazine κ = 0.91. Both cyclosporine and gold kappas were not applicable because only 1 patient each received it.
Diagnostic procedures and tests.
During the in-depth interviews, some patients recalled their imaging procedures in such detail that a comprehensive list was included in the preliminary questionnaire. However, both the cross tables (κ = 0.20) and the correlation coefficient (r = 0.19; P < 0.05) indicated a weak association between payer and patient reports. Histograms of difference did not reveal a systematic error. When only computed tomography and magnetic resonance imaging scans were considered, the kappa statistic was higher (κ = 0.53). In-depth interviews indicated that patients were unable to recall details of laboratory investigations performed. This issue was therefore addressed using a simple yes/no question. Because almost all patients (97.1%) reported laboratory investigations that were confirmed by payers, the kappa statistics were not applicable. When information was sought about the number of investigations performed, the correlation coefficient (number of investigations as reported by patients versus payers) was r = 0.18 (P < 0.05).
Inpatient health care utilization.
The in-depth interviews indicated a relatively high level of accuracy in recall of inpatient episodes and length of hospitalization. This was confirmed when comparing patient and payer data (κ = 0.64, r = 0.80, P < 0.001). The histograms of difference were normally distributed with a dominant central value, indicating no difference between patient and payer reports. However, these statistics are achieved when aggregating all inpatient episodes into one variable. Patients were not able to discriminate between acute inpatient and nonacute inpatient (i.e., rehabilitation) settings or between surgical and nonsurgical inpatient episodes (results not presented here because they were negative).
During the in-depth interviews, patients clearly preferred to report transportation costs in terms of money. Cost of travel is often covered by patients themselves initially, with some subsequent refunding by payers. Comparing the patient- and payer-reported transportation costs revealed κ = 0.13, but with a reasonable correlation coefficient of r = 0.39 (P < 0.001). In half of the 60 patients who reported transportation costs that were not confirmed by the payers, costs were less than €13.
Analysis for all 227 patients.
Results for the 227 patients who filled in at least 1 questionnaire were very similar to those for the 136 who completed all 4. The level of underreporting of physician visits was slightly lower in the 227 patients and Spearman's correlation coefficient was slightly higher (r = 0.48). Kappa statistics for the utilization of medication were as follows: DMARD κ = 0.52, NSAID κ = 0.58, steroid κ = 0.76, osteoporosis medication κ = 0.51, and analgesics κ = 0.43. Outcomes for the 2 subdomains of diagnostic procedures and tests (imaging and laboratory investigations) were almost identical. The kappa statistic for inpatient health care utilization was 0.64; the correlation coefficient between hospital days as reported by patients and payers was r = 0.73 (P < 0.001; base case r = 0.80). In the transportation costs domain, the correlation was r = 0.35 (P < 0.001; base case r = 0.39).
Generally speaking, the present analysis supports the trend toward the use of less-detailed items when assessing health care utilization in RA. Dichotomous (yes/no) assessment was preferred for items in the domains covering medication and diagnostic procedures and tests. More detail was considered appropriate in 3 areas: number of physician visits, number of inpatient days, and total travel expenditure.
In accord with Ritter et al (9), we do not recommend differentiating between specialties when collecting data on physician visits. The diary-based approach used by Goossens et al (8) discriminated between general practitioner and specialist consultations, but validation data were available only for the latter, making it impossible to determine whether the distinction was successful. The data presented by Guzman et al (7) indicate that kappa statistics (interviewer-administered questionnaire compared with physicians' records) diminish when patients are asked for information about specialist consultations.
Underreporting of physician visits by patients has been reported elsewhere (9). Reasons for this are unclear, but the total number of consultations may be a factor (20). This issue is scheduled for future investigation (Table 5).
|Topics related to individual cost domains|
|Examine models to explain why patients are underreporting physician visits|
|Validity of patient-reported non-RA medication|
|Translation of physical into monetary units (e.g., create annual reference costs per specific drug class)|
|Diagnostic procedures and tests|
|Further exploration of which specific imaging procedures are reported most accurately|
|Reproduce data in a patient sample where only part of the population received laboratory examinations (in our study it was >97%; therefore kappa statistics were not applicable)|
|Inpatient care utilization|
|Translation of physical into monetary units (e.g., determine reference cost per inpatient day)|
|Examination of validity of patient-reported reasons for hospitalization (e.g., by comparing against the discharge diagnosis)|
|Develop validation settings for cost domains where payer-reported data are not gold standard (devices and aids, nonphysician visits, etc.)|
|Further psychometric testing of items (test–retest reliability, sensitivity to change, determination of best recall period, etc.)|
|Impact on validity of patient reports of focusing on RA-related versus general health care utilization|
|Generalizability of preferred aggregation level in international settings|
The preliminary questionnaire reported here included items covering outpatient surgery and emergency room visits. Information explicitly referring to outpatient surgery is collected in only 2 of 15 questionnaires in the literature, and 4 of the 15 specifically address emergency room visits (5). The present results do not suggest that patient-reported data on outpatient surgery provide valid information. Due to the lack of an appropriate validation setting (emergency room visits are not separately accounted for in Germany), we were not able to examine validity of patient-reported emergency room visits (16).
A key finding of our study is the validity of highly aggregated patient self-reported medication data. Given that medication accounts for 44% of the total direct costs in RA (16), the strong association between payer and patient reports is very important. Within that domain, self reports of the consumption of DMARDs and steroids were highly accurate. Furthermore, disaggregation into separate DMARDs (methotrexate, sulfasalazine, etc.) seems appropriate. The authors found no published evidence to support the collection of more detailed medication data (e.g., dosing regimes). Clarke et al (6) reported a 92% agreement between government data and medication self reports, but did not specify the item aggregation level. Goossens et al (8) included medication in their diary system, but did not provide validation data.
The majority (9 of 15) of health care utilization questionnaires in RA collect patient-reported data on diagnostic and therapeutic procedures and tests (5). However, only 1 provides any information on their validity (7). That study showed a good association between patient self-reported information and physicians' records on the use of computed tomography scans and blood tests. However, in the present cohort, 33 patients (24%) reported outpatient imaging procedures that were not confirmed by the payers. Seven of the 33 had had an inpatient episode during the study year. Although the questionnaire specifically asked for outpatient procedures, further research should clarify whether patients are able to discriminate between inpatient and outpatient imaging. The future research agenda (Table 5) includes further exploration of that domain.
Inpatient care accounts for 24% of total direct expenditure in RA, making it the second most important cost domain (16). Subjects here accurately reported its use. Inpatient data should be collected on a dichotomous (yes/no) level as well as quantitatively (i.e., number of inpatient days). The preliminary item pool in this study discriminated between acute and nonacute care and surgical and nonsurgical settings, but the comparison with payer data did not support that level of disaggregation. Ritter et al (9) and Clarke et al (6) confirm the high validity of patient self-reported data for the highest aggregation level (i.e., hospitalization yes/no). Another approach to estimate inpatient costs more precisely might be based on reasons for hospitalization. However, this would require further research comparing patient-reported reasons for hospitalization with the hospital discharge diagnoses.
Transportation costs are rarely included in health care utilization questionnaires (6 of the 15 tools described above explicitly cover them) (5). Furthermore, the validity of patient-reported transportation data has yet to be investigated. The present findings indicate that patients have a moderate recall of their expenses. However, the validity of our approach was limited due to the fact that patients pay a certain amount (total of €13) themselves. This limitation might explain the low kappa statistic.
This analysis provides an initial insight into the optimal aggregation level of patient-reported health care utilization data. However, as the future research agenda (Table 5) indicates, several areas require further exploration.
In such domains as devices and aids, nonphysician visits, and to a certain extent, transportation (i.e., low-cost trips), payer-reported data are not the gold standard because patients often contribute. For that reason, the 2 domains devices and aids and nonphysician visits were not included in the present analysis.
Further psychometric testing such as test–retest reliability and sensitivity of items to detect change in health care utilization is required. Other challenges are the approach toward missing values and the impact of recall period on the validity of patient self reports. The review of questionnaires (5) indicated a variance in recall period between 1 week and 1 year.
RA patients incur a considerable amount of non-RA expense (21, 22). That the present study design considered only RA-related costs may be considered an important weakness. The authors therefore aim to generate a dataset that will permit them to address the question of whether the discrimination of disease-related and non–disease-related costs based on patient self reports is appropriate.
The in-depth interviews were conducted in 10 randomly chosen RA patients that visited the outpatient care facilities of the Division of Rheumatology at the Hanover Medical School. The outcome of the interviews should be reproduced in a larger representative sample of RA patients in Lower Saxony.
Finally, the generalizability of these findings to an international setting has still to be determined.
In conclusion, the present analysis supports a trend toward the use of increasingly aggregated items to assess health care utilization in RA. Dichotomous assessment (i.e., health care used, yes/no) was preferred in the domains covering medication and diagnostic procedures and tests. Disaggregation to provide quantitative data appears to be appropriate in 3 areas: number of physician visits, number of inpatient days, and total expenditure on transportation.
The authors thank Brigitte Käser from Allgemeine Ortskrankenkasse Niedersachsen (AOKN) and Ernst Weinhold from Kassenärztliche Vereinigung Niedersachsen for their encouragement and support in conducting the costing study. The study team also gratefully acknowledges Volker Kück from AOKN for his continuous support with data transfer and management.
- 1Critical assessment of economic evaluation. In: DrummondMF, StoddartGL, TorranceGW, editors. Methods for the economic evaluation of health care programmes. Oxford, UK: Oxford University Press; 1990. p. 18–35., , .
- 2Department of Clinical Epidemiology and Biostatistics, McMaster University Health Sciences Centre. How to read clinical journals: VII. To understand an economic evaluation (part A). Can Med Assoc J 1984; 130: 1428–34.
- 14AOK. Die Gesundheitskasse für Niedersachsen. Geschäftsbericht 2000. Hannover: Grütter; 2001.
- 15BMAE, E-GO, EBM: Gegenueberstellung mit Abrechnungshinweisen. Dachau: Zeuner, Druck-und Verlags GmbH; 2000..
- 18Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1989; I: 307–10., .
- 19SPSS base system and professional statistics. Koesel, Kempten: International Thomson Publishing; 1995., .