To evaluate the validity of the Cutaneous Lupus Erythematosus Disease Area and Severity Index (CLASI) for use by rheumatologists via reliability testing, and to extend the validation for dermatologists.
To evaluate the validity of the Cutaneous Lupus Erythematosus Disease Area and Severity Index (CLASI) for use by rheumatologists via reliability testing, and to extend the validation for dermatologists.
Fourteen subjects with cutaneous lupus erythematosus (CLE; n = 10), a mimicker skin disease only (a cutaneous lesion that may appear clinically similar to CLE; n = 1), or both (n = 3) were rated with the CLASI by academic-based dermatologists (n = 5) and rheumatologists (n = 5).
The dermatology intraclass correlation coefficient (ICC) was 0.92 for activity and 0.82 for damage; for rheumatology the ICC was 0.83 for activity and 0.86 for damage. For intrarater reliability, the dermatology Spearman's rho was 0.94 for activity and 0.97 for damage; for rheumatology the Spearman's rho was 0.91 for activity and 0.99 for damage.
Our data confirm the reliability of the CLASI when used by dermatologists and support the CLASI as a reliable instrument for use by rheumatologists.
In a review of activity measurement systems, Liang et al noted that ∼60 outcome measures have been developed to measure disease activity in systemic lupus erythematosus (SLE) (1). Although the usefulness of such outcome measures in the evaluation of cutaneous lupus erythematosus (CLE) has been reported, in general the practicality of their use by the dermatology community has been questioned (2). Overall, the design and evaluation of such outcome measures has been heavily influenced by nondermatologists, potentially limiting their usefulness in skin-specific disease. Albrecht et al (3), along with a team of dermatology/rheumatology experts, designed the Cutaneous Lupus Erythematosus Disease Area and Severity Index (CLASI), a quantitative outcome measure that assesses both the activity and damage components of CLE.
The CLASI (Appendix A, available at the Arthritis Care & Research Web site at http://www.interscience.wiley.com/jpages/0004-3591:1/suppmat/index.html) divides the evaluation of CLE lesions according to activity and damage. The activity measurement attempts to quantify the level of active inflammation in the skin, scalp, and oral mucosa. For example, lesions with greater visual intensity (dark purple is worse than faint erythema) and scaling are considered to be more active and would be scored appropriately higher. Recent hair loss, nonscarring alopecia, and mucous membrane involvement are also represented within the activity assessment. In contrast, the damage measurement attempts to quantify the “footprint” of destruction that previous inflammation has left behind. Dyspigmentation, scarring, and panniculitis as visualized in the skin, scalp, nails, or mucous membranes count toward the damage assessment. The total possible scores for activity and damage are 70 and 56, respectively. The relative weight of body surface area in the CLASI was designed to emphasize areas of the body that are more apparent in daily life. For example, the nose and malar area counts the same as the entire back/buttocks region.
Previously published literature supports the construct validity of the CLASI for use by dermatologists in assessing CLE (3). First, both content validity and face validity of the CLASI have been evaluated and confirmed by a group of 7 expert dermato-rheumatologists and the American College of Rheumatology Response Criteria Committee on SLE (3). Second, the internal structure evaluation (reliability testing) of the CLASI demonstrated good interrater reliability (agreement) and intrarater reliability (temporal stability) when used by dermatologists (3). Because the CLASI is currently the only outcome measure specific to CLE, no direct comparisons with other outcome measures have been made. A recent study also supports the CLASI as being responsive to changing CLE lesions in the midst of implementing a treatment regimen (4).
Because the skin is the second most commonly involved organ in persons with SLE, affecting up to 85% of patients, dermatologists have both a role in and a meaningful research tool (the CLASI) for studying SLE/CLE (5). Although isolated cutaneous manifestations are more frequent in chronic CLE and subacute CLE, approximately 10% and 50% of these patients, respectively, meet the criteria for SLE (6). Given their interest and expertise in studying SLE, rheumatologists may also find the CLASI to be a useful research tool in assessing patients with cutaneous features of lupus erythematosus (LE). Because the prior validation studies of the CLASI were performed with dermatologists, we sought to determine if the CLASI also meets acceptable criteria for validity when used by rheumatologists, specifically with respect to internal structure reliability (inter- and intrarater reliability) (7).
Although considered a prerequisite for validity, reliability alone does not prove an instrument's validity (7). Besides reliability, the usefulness of the CLASI as an appropriate outcome measure for CLE also depends on raters being able to accurately identify the proper subset of cutaneous lesions on which evaluations should be performed, i.e., lesions of CLE. It is theoretically possible, for example, for the CLASI to perform with excellent inter- and intrarater reliability when used to assess a different skin disease such as cutaneous T cell lymphoma or psoriasis, even though this use does not fit with the intended design and use of the CLASI. Evaluation of the response process, or the nature of the fit between the ideal use of the CLASI (only in patients with CLE) and the raters' thought processes (“I am rating CLE lesions”), may be useful in assessing whether raters are appropriately trained to use the CLASI (7). Given the differences in training, we hypothesized that differential response process performance might occur in the dermatology and rheumatology groups. We sought to characterize such theoretical differences in response process performance by comparing the diagnostic accuracy of the 2 groups in identifying cutaneous pathology. Specifically, one of the study objectives was to calculate and compare the sensitivity and specificity of diagnosing CLE between the dermatology and rheumatology groups in the context of a minipilot study. The specificity of CLE diagnosis is important, because a low specificity signifies that non-CLE lesions are both inappropriately counted and integrated into the CLASI evaluation.
In summary, this study evaluated the validity of the CLASI for use by rheumatologists via internal structure assessment (inter- and intrarater reliability) and response process analysis (diagnostic skill evaluation) (7). Validation testing of the CLASI for use by dermatologists was also extended via reevaluation of reliability testing and response process analysis.
Ten subjects with biopsy-proven CLE as the primary and only cutaneous disease process were recruited from the academic practice of the principal investigator (VPW). Three subjects with coexisting lesions of biopsy-proven CLE and a CLE-mimicker disease and 1 subject with 2 mimicker skin diseases only (dermatomyositis and acne) were also recruited. A mimicker skin lesion was defined as a cutaneous lesion that may appear clinically similar to CLE. The study was designed with 13 subjects (excluding the 1 subject without CLE) and 5 raters to provide 80% power for detection of increases in inter- and intrarater reliability ranging from 0.33 (against a null value of 0.3) to 0.18 (against a null value of 0.7), using a 1-sided Type I error rate of 5%. Five subjects were chosen for the diagnostic assessment pilot substudy to uncover trends in differences between dermatologists and rheumatologists and generate hypotheses for future, larger-scale studies, while recognizing that we were not likely to have sufficient power to detect statistically significant differences in sensitivity and specificity.
Subjects who met inclusion and exclusion criteria gave informed consent in accordance with the regulations set forth by the Institutional Review Board at the University of Pennsylvania. The inclusion criteria included the following: age ≥17 years, a diagnosis of CLE according to the modified Gilliam criteria (8), or a diagnosis of a CLE-mimicker skin disease. Subjects without CLE or a CLE-mimicker skin disease were excluded from the study. Subjects were allowed to use prescribed medical therapies leading up to the study, including topical steroids and/or oral immune modulating therapies.
The physician rating group comprised 5 attending-level, academic-based dermatologists; 4 attending-level, academic-based rheumatologists; and 1 academic-based, second-year rheumatology fellow. Upon arrival in the morning of the study day, raters viewed a 30-minute training slide presentation (complete with photographic examples) by the principal investigator (VPW) on the appropriate use of the CLASI outcome measurement tool. Raters were also given printed instructions regarding the study guidelines. A separate set of instructions on how to complete the CLASI was not provided. A list of possible CLE diagnoses according to Sontheimer (8) was provided to assist in the completion of the diagnostic assessment. No glossary, however, was provided for the dermatologic diagnoses nor was an image bank of cutaneous pathology provided for reference purposes. During the introduction, raters were also informed that subjects may or may not have CLE. Questions relating to a subject's diagnosis were prohibited, excluding those specifically required by the CLASI (presence of alopecia, duration of dyspigmentation). Raters were evenly divided into 2 groups, and each rater initiated ratings with one hemicohort of subjects and then completed the ratings upon evaluation of the second hemicohort. Raters evaluated each subject and filled out the respective CLASI assessment individually and without assistance.
During the assessment of 5 of the 14 subjects (subjects 1, 2, 3, 8, and 9), each rater was also asked to perform the following evaluation in the context of a minipilot study within the larger study: 1) classify up to 3 prespecified lesions of the skin, scalp, or nails as either caused by CLE or not, and 2) provide a more specific diagnosis of the lesion, as outlined by Sontheimer's revised classification of CLE, if the rater considered the lesion CLE specific (8).
After completing this sequence of evaluations, physician raters turned in the scoring sheets and were then asked to rescore the CLASI on any 2 subjects of their choice from the hemicohort with which he or she initiated the ratings, excluding subject 3, who had a pure CLE mimicker. Before this point in the study, to minimize recall bias, raters were kept unaware that they would be re-rating 2 subjects. The gold standard for dermatologic diagnoses was designated as the consensus opinion of the dermatologists as provided during the diagnostic assessment portion of the study. Because no histologic, laboratory, or examination findings are pathognomonic for CLE, the consensus opinion among this specialty group was thought to be the most accurate determination of the gold standard. CLASI reliability testing was performed on the same group of dermatologists that was used to designate the gold standard. We do not consider this to be inconsistent or a source of bias because reliability data correspond to internal consistency of the CLASI and the diagnosis data correspond to the suitability of the response process, 2 separate elements of construct validity.
The original data, originally hand recorded onto paper by each physician, were transcribed into Excel spreadsheets (Microsoft, Redmond, WA). With respect to missing or unclear data points (excluding unclear entries corresponding to duration of damage/dyspigmentation), the primary data assessment was carried out according to the following rules: 1) if no response was provided for the question “Report duration of dyspigmentation after active lesions have resolved” listed under damage/dyspigmentation, the same score given by the other raters was inserted if ≥80% of the other raters were in agreement; otherwise, a 0 was inserted; 2) if a given entry on the CLASI was left blank, excluding the damage/dyspigmentation question mentioned above, a score of 0 was assumed; 3) if a given entry was indiscriminate (e.g., a response of 0–1, or “+”), then the score was rounded up to assume that the rater intended to demonstrate that some abnormality was present. The question regarding duration of damage/dyspigmentation (symptoms >12 months or <12 months) represents a nonsubjective, rater-independent response. As such, we considered the substitution of unclear entries with the consensus of other raters to be satisfactory because the response is an unknown truth rather than subjective or rater dependent. After the initial data entry, data from the CLASI were verified by comparing a manual summation of total activity and damage scores with those computed by formula in Excel. Discrepancies in the transcribed data were checked against the primary data and resolved appropriately. Depersonalized data from the CLASI data set were then sent to the Center for Clinical Epidemiology and Biostatistics at the University of Pennsylvania for analysis. Interrater agreement was assessed using Pearson's correlation coefficient and the intraclass correlation coefficient (ICC); intrarater agreement was assessed using paired t-tests and Spearman's correlation coefficient. Diagnostic accuracy among the rater groups was assessed by estimating sensitivity and specificity. The 95% confidence intervals are provided; all analyses were conducted using Stata software, version 9 (StataCorp, College Station, TX). All analyses were repeated excluding lesions with missing or indecipherable responses to check the robustness of our method of handling missing data.
The internal structure analysis data for both inter- and intrarater agreement are shown in Table 1. Of the 14 subjects who participated in the study, 93% were female, 71% were African American, and the average age was 39.6 years, ranging from 17 to 51 years (Table 2). All 14 subjects were evaluated with the CLASI by the group of 5 dermatologists during the first phase of the study. Seven subjects were evaluated a second time during the second phase of the study in order to complete the intrarater reliability (temporal stability) assessment. Similarly, for the group of rheumatology raters, 14 subjects were evaluated during the first phase and 6 of these were evaluated for a second time during the second phase.
|Raters, no.||Subjects, no.||Score (95% CI)|
|Interrater reliability (ICC)|
|Intrarater reliability (Spearman's ρ)|
|Subject number||Age, years||Race||Sex||Diagnosis|
|1||27||AA||M||CCLE, discoid type; melanonychia striata|
|2||35||AA||F||CCLE, discoid type; psoriasis|
|3||17||White||F||Dermatomyositis and acne|
|4||49||AA||F||CCLE, discoid type|
|6||29||AA||F||CCLE, discoid type|
|7||47||AA||F||CCLE, discoid type|
|8||47||White||F||CCLE, LE tumidus|
|9||45||AA||F||CCLE, discoid and hypertrophic types|
|10||46||White||F||CCLE, discoid type|
|11||48||AA||F||CCLE, discoid type|
|12||34||White||F||CCLE, discoid type|
|13||51||AA||F||CCLE, discoid type|
|14||43||AA||F||CCLE, discoid type|
|17||37||AA||F||CCLE, discoid type|
When calculating interrater agreement, data from the subject without CLE (subject 3) were ignored. The interrater agreement for both the dermatology and rheumatology groups was high (Table 1).
The dermatology group on average scored the activity ratings 1.7 points higher than the rheumatology group (Table 3), but this was not significant (P = 0.35). In contrast, the rheumatology group on average scored the damage ratings 2.0 points higher than the dermatology group, but this was not significant (P = 0.39).
|Low score (subject no.)||High score (subject no.)||Mean ± SD|
|Dermatology||3.4 (7)||32.8 (1)||15.6 ± 10.6|
|Rheumatology||4.6 (4)||31.4 (1)||14.1 ± 9.1|
|Dermatology||0.2 (8)||38.8 (14)||20.2 ± 13.1|
|Rheumatology||2.2 (3)||36.0 (11)||22.0 ± 13.1|
Data collected from the subject without CLE (subject 3) were excluded from the intrarater reliability analysis. The differences between the first and second ratings for total activity score ranged from −7 to 5 in the dermatology group and from −10 to 3 in the rheumatology group. The mean differences between the first and second ratings for the total activity score were −0.8 and −1.1 for the dermatology and rheumatology groups, respectively; these differences were nonsignificant (dermatologists: t = −0.78 by paired t-test, P = 0.45; rheumatologists: t = −0.95, P = 0.37). For the total damage score the results were similar. The differences between the first and second ratings ranged from −9 to 4 in the dermatology group and from −12 to 4 in the rheumatology group. The mean differences between the first and second ratings for the total damage score were −0.5 and −0.8 for dermatologists and rheumatologists, respectively. These differences were also found to be nonsignificant (dermatologists: t = −0.44 by paired t-test, P = 0.67; rheumatologists: t = −0.52, P = 0.62).
The intrarater reliability for both the activity and damage scales in each rating group was high (Table 1). When assessed by regression models relating the first activity score (outcome) to the second activity score (predictor), the slope was close to 1 among both dermatologists and rheumatologists (β1 = 0.94 and 0.87, respectively) and the intercept was close to (and not significantly different from) 0 for both the dermatology and rheumatology groups (β0 = 0.05, P = 0.98 and β0 = 1.15, P = 0.57, respectively). Results were similar for the damage score, namely, regression slopes were close to 1 (β1 = 0.91 and 0.98 for the dermatologists and rheumatologists, respectively) and intercepts were close to (and not significantly different from) 0 (β0 = 1.18, P = 0.55 and β0 = −0.28, P = 0.92 among dermatologists and rheumatologists, respectively).
A complete case analysis was performed in which all data sets with missing or indecipherable responses were excluded. The complete sensitivity analysis resulted in no substantive changes from the results presented in Table 1.
The gold standard diagnoses and rater responses for the cutaneous diagnosis assessment are listed in Table 4. Three lesions were assessed in each of the 5 subjects (subjects 1, 2, 3, 8, and 9). On a few occasions raters listed multiple possible diagnoses for a given lesion, so the total number of diagnoses listed in Table 4 may exceed the number of physician raters in a given group. The gold standard for diagnostic assessment was considered to be the consensus diagnosis from the dermatology group.
|Subject||Lesion||Gold standard diagnosis||Rheumatology diagnoses||Dermatology diagnoses|
|1||Right ear||Generalized DLE||2/5||3/5 classic DLE||4/5||1/5 Seborrheic dermatitis|
|Fingernails||Not lupus; melanonychia striata||1/5||1/5 classic DLE||1/5 Hyperpigmented; 1/5 medication; 1/5 NL||2/5 Melanonychia striata; 2/5 hyperpigmented/linear nail streaks; 1/5 LE|
|Right arm||Generalized DLE||2/5 annular||3/5 classic DLE||2/5||3/5|
|2||Scalp||DLE||4/5 classic DLE; 1/5 hypertrophic||5/6||1/6 Seborrheic dermatitis|
|Right superior ear||Not lupus; eczematous process||3/6 localized; 1/6 DLE & SLE; 1/6 hypertrophic||1/6 SLE||1/5||1/5 Eczema/psoriasis; 2/5 seborrheic dermatitis; 1/5 NL|
|Right axilla||Not lupus; inverse psoriasis||1/5||2/5 lichenoid DLE; 1/5 hypertrophic||1/5 NR||1/5||1/5 Lichen sclerosis chronicus/steroid atrophy; 1/5 psoriasis; 1/5 seborrheic dermatitis; 1/5 tinea corporis/atopic dermatitis|
|3||Hands||Not lupus; DM||1/5 verrucous||4/5 Gottron's papules||1/6||1/6 DLE overlap with DM||2/6 DM; 2/6 Gottron's papules|
|Left elbow||Not lupus; DM||2/5 classic DLE||1/5 Postinflammatory; 1/5 NR; 1/5 NL||1/5||1/5 Discoid lupus scar||1/5 DM; 2/5 Gottron's papules|
|Forehead||Not lupus; acne||2/5 localized DLE; 1/5 lichenoid DLE||2/5 Acne||1/6 Classic DLE||5/6 Acne|
|8||Left shoulder||Tumid lupus||4/5||1/5 lupus profundus/panniculitis||1/6 annular SCLE; 1/6 SCLE||4/6 Tumid lupus|
|Right shoulder||Tumid lupus||1/7||1/7 papulosquamous||1/7 classic DLE; 2/7 papulomucinous; 1/7 lupus profundus/ panniculitis||1/7 Classic SLE||1/7||4/7 Tumid lupus||2/7 LE|
|Left ankle||Not lupus; post-inflammatory hyperpigmentation||1/5 lichenoid DLE||1/5 Postinflammatory; 2/5 NR; 1 NL||2/5 Stasis dermatitis; 2/5 postinflammatory; 1/5 NR|
|9||Scalp||DLE||5/6 DLE||1/6 SLE||5/5 DLE|
|Chest||DLE||3/6 DLE; 2/6 hypertrophic/ verrucous||1/6 SLE||5/5 DLE|
|Right knee||Hypertrophic DLE||2/6||1/6 classic DLE; 2/6 hypertrophic||1/6 SLE||1/6||3/6 hypertrophic; 2/6 DLE|
A sensitivity and specificity analysis to assess the diagnostic accuracy of both the dermatology and rheumatology groups was performed (Table 5). The data were analyzed according to the following questions as proposed to the raters: 1) Is the lesion primarily caused by CLE or not? 2) Is the lesion primarily caused by discoid lupus erythematosus or not? 3) Is the lesion primarily caused by tumid LE or not? Because some respondents provided more than 1 diagnosis per lesion, the data had to be interpreted under special rules to complete the analysis. If a rater listed the correct diagnosis as at least one of the responses, then the response was scored as true positive. If, however, at least 1 of the responses was listed incorrectly as the true diagnosis, then the response was listed as false positive.
|Is lesion secondary to CLE?|
|Positive predictive value||0.82||0.68|
|Negative predictive value||1||1|
|Is lesion secondary to DLE?|
|Positive predictive value||0.79||0.58|
|Negative predictive value||0.94||0.91|
|Is lesion secondary to tumid LE?|
|Positive predictive value||1||1|
|Negative predictive value||0.97||0.87|
Rheumatologists may find the CLASI to be a useful and reliable outcome measure for studying the cutaneous manifestations of LE. Given the excellent intra- and interrater reliability scores, this study demonstrates that rheumatologists can reliably use the CLASI to measure both disease activity and severity when presented with subjects affected with CLE. The reliability results for the dermatology group in this study are also excellent and confirm those of the prior study by Albrecht et al (3). The high level of correlation between the rheumatologists and dermatologists in reliability testing further demonstrates that the internal structure design of the CLASI is good. Overall, this study confirms the reliability of the CLASI when used by dermatologists and rheumatologists in subjects with CLE.
The ability to reliably assess subjects with CLE is different from identifying for which subjects or specific lesions the CLASI should be used. Because validity depends in part on the response process of a given rater group, the presence of an ill fit between the actual use of the CLASI and the intended goal of its use, i.e., evaluation of subjects with CLE, may suggest an incomplete validation. In general, the rheumatologists were able to identify CLE with good sensitivity. Often, however, nonlupus diseases were misclassified as CLE (see Table 2, subject 1, lesion 2, and subject 2, lesions 2 and 3), as evidenced by lower specificity of the rheumatology group. Misclassification of nonlupus lesions as CLE may underestimate disease responsiveness via inaccurate and biased CLASI ratings. For example, if rheumatologists misclassify psoriasis lesions as CLE while completing CLASI assessments in the setting of a therapeutic trial, the true effect on CLE lesions may be masked if psoriasis lesions are unresponsive to or, even worse, activated by the new treatment. The relative overdiagnosis of CLE in the rheumatology group suggests that rheumatology-directed studies utilizing the CLASI may benefit from assistance with subject selection and ongoing monitoring.
Although some subjects may have been undergoing topical corticosteroid therapy at the time of the study, it is unlikely that this played any significant role in the study outcomes. Because the subjects were all rated on the same day by all raters, any changes in erythema, scale, hypertrophy, etc. induced by corticosteroids or other therapies should not affect results of either the intra- or interrater reliabilities.
Although the diagnostic accuracy of both groups was not 100%, the CLASI scores corresponding to partially incorrectly diagnosed subjects should be included in the analysis for several reasons. First, use of the CLASI in this study was meant to serve as an approximation to real-life use of the instrument in that physicians completing CLASI assessments of subjects in future studies may not attain 100% accuracy in lesional identification either. Second, the evaluation of the diagnostic capabilities was meant to serve more as a measure of the appropriateness of the CLASI for use by rheumatologists, which addresses a separate aspect of construct validity than the internal structure evaluation (intra- and interrater reliabilities). Furthermore, the secondary non-CLE lesions that were designated for identification generally accounted for a very small surface area of the body. In other words, even in situations where the diagnosis was incorrect, the vast majority of affected body surface area was affected with CLE and as such the ultimate CLASI score was most likely affected only to a small extent.
These results are not entirely surprising, given the differences in training between the 2 fields. The data are also consistent with other studies that demonstrated that dermatologists' diagnostic ability and skill in scoring cutaneous outcome measures, even for a more common and less complex disease such as atopic dermatitis, are better than those of nondermatologists (9, 10). Although this study was not powered well enough to assess the difference in diagnostic ability between dermatologists and rheumatologists to properly identify CLE lesions, our data suggest that such a difference may exist. Further testing may clarify such a difference, but the data imply that future studies in which the CLASI would be used by rheumatologists may benefit from incorporating input from dermatologists or experts in dermatology/rheumatology in both subject screening and ongoing evaluation. In summary, the CLASI shows excellent internal structure reliability when used by both dermatologists and rheumatologists, yet differential response processing is seen during assessments by the 2 different groups of raters.
This study is partly limited by the lack of a true gold standard. Because there are no absolute guidelines for the diagnosis of CLE, the consensus opinion from 5 dermatologists was used. The dermatologists, however, were not given access to pathologic specimens or the results of such evaluations. The diagnostic assessment data, overall, should be interpreted with caution given the small sample sizes with respect to the minipilot portion of the overall study. With respect to intrarater reliability, recall bias may have played a role in falsely increasing the ICC because little time elapsed between first and second ratings and raters were allowed to choose which subjects of a given hemicohort to re-rate instead of being assigned subjects at random. Notwithstanding, raters were required to select subjects whom they initially rated in the first group of subjects rated in the morning. We think it is unlikely that the raters remember how they rated a particular subject from the morning session given the total number of CLASI ratings (16) and data points to fill in (n = 856). Finally, it may seem inconsistent to validate the CLASI with responses provided by raters who are unable to identify the correct diagnosis in 100% of cases. Nonetheless, we consider the inclusion of all ratings, whether applied appropriately or not to CLE and other cutaneous lesions, to be a more accurate representation of what the real-life performance of the CLASI would be when used by rheumatologists with a non-preselected group of subjects who may have a mixture of CLE and non-CLE lesions.
Dr. Werth had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study design. Krathen, Propert, Werth.
Acquisition of data. Krathen, Dunham, Gaines, Junkins-Hopkins, Kim, Kolasinski, Kovarik, Kwan-Morley, Okawa, Rogers, Rose, Thomas, Van Voorhees, Von Feldt, Werth.
Analysis and interpretation of data. Krathen, Propert, Troxel, Von Feldt, Werth.
Manuscript preparation. Krathen, Gaines, Junkins-Hopkins, Kovarik, Kwan-Morley, Propert, Troxel, Von Feldt, Werth.
Statistical analysis. Krathen, Propert, Troxel, Weber.