Comparison of three screening tools to detect psoriatic arthritis in patients with psoriasis (CONTEST study)

Authors

Errata

This article is corrected by:

  1. Errata: Corrigendum Volume 168, Issue 6, 1376, Article first published online: 5 June 2013

  • Funding sources
    An unrestricted investigator initiated study grant from Pfizer.

  • Conflicts of interest
    None declared.

Philip Helliwell.
E-mail: p.helliwell@leeds.ac.uk

Summary

Background  Multiple questionnaires to screen for psoriatic arthritis (PsA) have been developed but the optimal screening questionnaire is unknown.

Objectives  To compare three PsA screening questionnaires in a head-to-head study using CASPAR (the Classification Criteria for Psoriatic Arthritis) as the gold standard.

Methods  This study recruited from 10 U.K. secondary care dermatology clinics. Patients with a diagnosis of psoriasis, not previously diagnosed with PsA, were given all three questionnaires. All patients who were positive on any questionnaire were invited for a rheumatological assessment. Receiver operating characteristic (ROC) curves were used to compare the sensitivity, specificity and area under the curve of the three questionnaires according to CASPAR criteria.

Results  In total, 938 patients with psoriasis were invited to participate and 657 (70%) patients returned the questionnaires. One or more questionnaires were positive in 314 patients (48%) and 195 (62%) of these patients attended for assessment. Of these, 47 patients (24%) were diagnosed with PsA according to the CASPAR criteria. The proportion of patients with PsA increased with the number of positive questionnaires (one questionnaire, 19·1%; two, 34·0%; three, 46·8%). Sensitivities and specificities for the three questionnaires, and areas under the ROC curve were, respectively: Psoriatic Arthritis Screening Evaluation (PASE), 74·5%, 38·5%, 0·594; Psoriasis Epidemiology Screening Tool (PEST), 76·6%, 37·2%, 0·610; Toronto Psoriatic Arthritis Screen (ToPAS), 76·6%, 29·7%, 0·554. The majority of patients with a false positive response had degenerative or osteoarthritis.

Conclusion  Although the PEST and ToPAS questionnaires performed slightly better than the PASE questionnaire at identifying PsA, there is little difference between these instruments. These screening tools identify many cases of musculoskeletal disease other than PsA.

Psoriatic arthritis (PsA) manifests clinically in several ways including arthritis, enthesitis, dactylitis, axial disease and skin/nail involvement. The majority of people with this condition have pre-existing psoriasis.1 A period of preclinical disease may occur,2 and cases of established disease may remain unidentified for some time.3 The reasons why cases of established PsA remain unidentified have not been elucidated but one possible cause is the lack of musculoskeletal expertise among primary care physicians and treating dermatologists. It follows that a simple method of screening for PsA in people with psoriasis would prevent unnecessary suffering and enable earlier treatment of this potentially disabling disease.4 Indeed, recent consensus guidelines for managing psoriasis [both the Scottish Intercollegiate Guidelines Network guidelines for psoriasis and PsA5 and those from the National Institute for Health and Clinical Excellence, which are currently in draft format (see http://guidance.nice.org.uk/index.jsp?action=byID&o=12344)] recommend using questionnaires to screen for PsA.

Several screening tools have been developed. All are patient-completed questionnaires, although their method of development and purpose are slightly different. The Psoriatic Arthritis Screening Evaluation (PASE) assesses likely articular symptoms and, in addition, assesses the impact of these on the patient in terms of disability.6 The PASE was developed in a hospital dermatology setting and in development had a sensitivity of 0·82 and specificity of 0·73. The Toronto Psoriatic Arthritis Screen (ToPAS) is a tool used to screen for both psoriasis and PsA, which has been validated in both dermatology and rheumatology clinics and can be used in community settings.7 The ToPAS questionnaire has an overall sensitivity of 87% and a specificity of 93%. Recently, a new questionnaire, the Psoriasis Epidemiology Screening Tool (PEST) has been developed in a primary care-based population with psoriasis.8 The new instrument consists of five questions supported by the addition of a manikin. In the development of this instrument the questionnaire had a sensitivity of 0·94 and a specificity of 0·78.

The aims of the current study were to validate the PEST screening questionnaire in a hospital-based sample of people with psoriasis and to compare the performance of this questionnaire with the PASE and ToPAS instruments using CASPAR (the Classification Criteria for Psoriatic Arthritis) as the gold standard.

Methods

The study was approved by local ethical committees in five centres in England and one in Scotland. Local collaborating dermatologists invited consecutive patients with psoriasis attending dermatology hospital clinics to participate. Patients were excluded if they had an established diagnosis of PsA.

Each patient was given an envelope that included a cover letter, a study information sheet, the three screening questionnaires and a return envelope. All current versions of the screening questionnaires were used in this study, including the updated version 2 of the ToPAS. The three screening questionnaires were presented in a random order to avoid completion bias. Those who scored positively on any of the three screening forms were contacted by the local study team and invited to attend for further evaluation by a rheumatologist. Those scoring negative on all three screening forms were thanked for their participation and took no further part in the study. Missing data were not interpolated – only questions answered were scored for the purpose of recording a positive response.

The rheumatological evaluation consisted of the following: demographics, medications, 78 tender joint count and 76 swollen joint count, assessment of psoriasis [Psoriasis Area and Severity Index (PASI)9 score], assessment of the nails of the hand (modified Nail Psoriasis Severity Index10 score), and assessment of enthesitis, which included 31 entheses (medial and lateral epicondyles of the humerus, supraspinatus insertion, first costochondral joints, seventh costochondral joints, fifth lumbar process, posterior superior iliac spines, iliac crest, anterior superior iliac spines, greater trochanters, medial condyle femur, superior pole of patellar, inferior pole of patellar, tibial tubercle, Achilles insertion and plantar fascia). Assessment of dactylitis consisted of a simple count of tender and nontender digits, and spinal measurements, if there was historical evidence of axial involvement, including tragus to wall, lumbar flexion, lumbar side flexion, intermalleolar distance, cervical rotation and chest expansion, all of which enabled calculation of the Bath Spinal Metrology Index (BASMI).11 No investigations were performed as part of the protocol but results were noted if available from case notes. Patients also completed the Health Assessment Questionnaire (HAQ)12 and a PsA quality of life measure (PsAQoL).13 CASPAR criteria14 were used to classify patients.

Sample size and statistical methods

For the purpose of sample size estimation, the sensitivity was chosen as the comparator. Given the published sensitivities of PEST, ToPAS and PASE (92%, 89% and 82%, respectively), the calculation was based on the comparison likely to give the largest sample size. The McNemar test for significance was used with a β of 0·8 and an α of 0·05 and the sensitivities of PEST and ToPAS were compared. The sample size was calculated as 203 people examined.15,16 Given a population prevalence of PsA in people with psoriasis of 15–40% it was estimated that a total of up to 1360 people would have to be screened to give sufficient positive responders for this study.

For people examined, results are given by number with true positive, false positive, true negative and false negative test results. Figures are given for the proportion of people with a true positive test among those with PsA according to CASPAR criteria (an estimate of sensitivity) and those with a true negative test among those without PsA (an estimate of specificity). The questionnaires were also compared using receiver operating characteristic (ROC) curves. ROC curves examine the ability of a measure to distinguish between two states, plotting sensitivity against (1 − specificity). A straight line joining the bottom left [sensitivity = 0, (1 − specificity) = 0] and top right corners would be obtained if the measure had no ability to discriminate between the two states, and would have an area of 0·5. A curve passing further away and to the left of this straight line approaches an area of 1·0 and better discriminates between groups. For the area of the ROC curve 95% confidence intervals (CIs) contained above 0·5 indicate significance. The sensitivity and specificity of the questionnaires were also compared using the nonparametric McNemar and Friedman tests (SPSS v. 16.0; SPSS, Chicago, IL, U.S.A.).

Results

In total 938 patients were given the questionnaires and 657 returned them (Fig. 1). Of these, 318 were positive for at least one of the questionnaires and these people were invited to attend for an examination; 195 attended. There were 47 cases of PsA according to the CASPAR criteria and these cases are compared with the other cases in Table 1. Patients with PsA were younger and had a shorter duration of psoriasis but these differences were not significant (as reflected by the 95% CI). Significant differences were seen for the PASI and enthesitis scores, and tender or swollen joint counts, these all being higher in the patients with PsA. In addition, examining rheumatologists were asked to report a new diagnosis of PsA irrespective of CASPAR scores: all patients classified by CASPAR were included plus an additional eight cases who did not fulfil CASPAR criteria (mostly because a test for rheumatoid factor was not available).

Figure 1.

 Flow diagram for the study. PsA, psoriatic arthritis; CASPAR, Classification Criteria for Psoriatic Arthritis.

Table 1. Comparison of people examined according to fulfilment of the Classification Criteria for Psoriatic Arthritis (CASPAR)
 CASPAR criteria assessment of PsA
No PsA (= 148)PsA (= 47)Whole group
Mean95% CIMin./max.Mean95% CIMin./max.Mean95% CIMin./max.
  1. PsA, psoriatic arthritis; CI, confidence interval; PASI, Psoriasis Area and Severity Index; mNAPSI, modified Nail Psoriasis Severity Index; TOTAL, total enthesitis count; LEI, Leeds Enthesitis Index; HAQ, Health Assessment Questionnaire; PsAQoL, PsA quality of life instrument.

Age (years)4846–5019–764642–4921–684745–4919–76
Sex (M/F), n69/7929/1898/97
Duration of psoriasis (years)23·220·8–25·60–6320·016·1–23·90–4822·520·5–24·50–63
PASI5·464·52–6·390–36·409·777·11–12·420–45·406·505·52–7·470–45·40
mNAPSI8·856·94–10·750–6214·139·89–18·370–5910·168·37–11·950–62
TOTAL3·072·27–3·860–247·705·50–9·910–314·193·34–5·030–31
LEI0·490·30–0·680–61·511·01–2·020–60·740·54–0·930–6
Tender dactylitis total0·01−0·01–0·040–20·32−0·12–0·760–100·09−0·02–0·190–10
HAQ0·450·36–0·530–2·250·650·46–0·830–2·380·500·42–0·570–2·38
PsAQoL5·965·0–6·920–208·276·34–10·200–206·505·64–7·370–20
Tender joint count4·213·02–5·400–5110·266·37–14·140–575·674·33–7·000–57
Swollen joint count0·390·13–0·660–102·090·82–3·350–280·800·43–1·170–28

The majority of cases of PsA had peripheral articular involvement only (= 32). Enthesitis was recorded as the sole musculoskeletal manifestation in eight cases. Spinal involvement was recorded in 14 cases, in seven of whom it was the only musculoskeletal manifestation. BASMI scores were low for those with and without spinal involvement, mean (SD), 2·4 (1·6) and 2·7 (1·3), respectively.

Systemic treatments were used in a minority of patients who were examined. Overall, biologics were used in 14 patients (7%), methotrexate in six (3%) and systemic agents for psoriasis, including retinoids and ciclosporin, in eight (4%). The corresponding percentages for people classified with PsA were biologics (13%), methotrexate (11%) and systemic psoriasis agents (6%).

Performance of the questionnaires

The questionnaires are compared in Table 2. For each questionnaire about three-quarters of the cases of PsA were identified but there was a high false positive rate for each of them, resulting in low specificity. For both PEST and PASE, the 95% CI of the area under the ROC curve exceeded 0·5, but this was not the case for ToPAS. Comparing the sensitivity and specificity of the questionnaires, no significant differences were found using the McNemar test [PEST vs. PASE, χ2 = 0·05, not significant (ns); PEST vs. ToPAS, χ2 = 1·3, ns; PASE vs. ToPAS, χ2 = 1·72, ns]. Further comparison of the three questionnaires together, using the Friedman test, indicated no significant difference between the three (χ2 = 2·53, d.f. = 2, = 0·283). Performing these analyses by physician diagnosis of PsA, rather than by CASPAR classification, gave the same result (data not shown).

Table 2. Comparison of the three questionnaires
QuestionnaireTrue positive, nFalse positive, nTrue negative, nFalse negative, nSensitivitySpecificityAUC (95% CI)
  1. AUC, area under receiver operating curve; CI, confidence interval; PASE, Psoriatic Arthritis Screening Evaluation; PEST, Psoriasis Epidemiology Screening Tool; ToPAS, Toronto Psoriatic Arthritis Screen.

PASE359157120·7450·3850·594 (0·505–0·683)
PEST369355110·7660·3720·610 (0·516–0·704)
ToPAS3610444110·7660·2970·554 (0·455–0·653)

Of interest were the false negative cases. These were not the same cases for all the questionnaires, indicating differences in their performance at the patient level. Further, the prevalence of PsA after administration of the questionnaires increased according to the number of positive instruments: the prevalence was 19·1%, 34·0% and 46·8% with one positive, two positive and three positive scores, respectively.

Clinical subgroups of patients with psoriatic arthritis

The majority (32 out of 47) of patients fulfilling the CASPAR criteria had peripheral joint involvement; seven (15%) had pure spinal involvement and eight (17%) pure entheseal involvement. It is possible that the nonarticular subgroups are more likely to be ‘missed’ by the screening questionnaires. This appeared to be the case: for both PEST and ToPAS 72% of missed cases had entheseal or spinal involvement only. For PASE the figure was lower, at 33%, and not substantially different from the whole group. However, the PASE questionnaire ‘missed’ more cases of pure articular disease (33%) in contrast to PEST and ToPAS, which ‘missed’ 18% each.

Other diagnoses identified in those patients examined

With high false positive rates, physician diagnoses of those not classified as PsA are of interest. Where alternative diagnoses were recorded, 42 had osteoarthritis or ‘degenerative arthritis’, four fibromyalgia, four hypermobility syndrome and one each had avascular necrosis, connective tissue disorder, trauma and crystal disease.

Discussion

In this secondary care-based study, the PEST and ToPAS questionnaires performed slightly better than the PASE questionnaire at identifying PsA but overall discriminatory ability, as reflected by the area under the ROC curve, was best for the PEST instrument. However, any differences between the questionnaires were small and probably insignificant. In addition, the high false positive rate reflects the fact that these screening tools identify many cases of musculoskeletal disease other than PsA, and these cases were mainly noninflammatory arthritis in this study.

In the development phase, these screening tools performed much better in terms of sensitivity and specificity. This is not surprising, particularly in the case of PEST, given that the questionnaire derived from the data on which it was tested. This is the reason why it is important to test these instruments in new populations and datasets, and to compare them ‘head to head’ if possible. It is worth noting that both the PASE and ToPAS were thought appropriate for use in secondary care but that the PEST was developed in a primary care setting and thus required further validation in secondary care. Although the sensitivities of all three instruments were less than originally published, they were acceptable. However, the specificities were poor, and much worse than the original figures.17 The poor specificities reflected the identification of many cases of noninflammatory articular disease. However, the current study design, where only respondents were invited for examination, is likely to have underestimated the specificity and to have overestimated the sensitivity of all three instruments. A further limitation of this study comes from the setting in which the study was conducted. Patients were recruited largely from dedicated severe psoriasis clinics rather than general dermatology outpatients. Therefore, many cases of PsA would already have been identified, particularly the ‘obvious’ ones, and these would have been excluded from this study. This may have reduced the specificity of the instruments in this population compared with an incident case cohort.

As only 195 patients were examined, the study failed to achieve the required sample size of 203. In fact, the response rate to the questionnaires, both in terms of instrument completion and willingness to attend for examination, was better than anticipated. That the recruitment target fell short by eight cases is unlikely to have made a difference to the results, given the prevalence of PsA in those patients attending for examination (at a rate of about 25% this would equate to two patients at most). Had these two patients been unusually identified by only one of the instruments the magnitude of difference would still not have influenced the outcome (e.g. the ToPAS sensitivity would have increased from 77% to 78%).

Another recently published ‘head-to-head’ study18 revealed results that were completely opposite to this study: the PEST, PASE and ToPAS had sensitivities of 27·5%, 24% and 41%, and specificities of 98%, 94% and 90%, respectively. Some of the differences were due to differences in study design – the study of Haroon et al. examined all respondents irrespective of their questionnaire results – and some due to differences in patients, as the study makes no mention of the prevalence of noninflammatory arthritis in their cohort. The authors of the Haroon et al. study found that the poor instrument sensitivities were in part due to the poor performance of the questionnaires in detecting nonpolyarticular (oligoarticular, spinal and entheseal) disease and, to some extent, the current study would support this.

It is of interest to note that the prevalence of PsA in the current study increased with increasing questionnaire positivity, suggesting that it might be possible to identify key elements among all three instruments that would have better discriminatory capacity in identifying PsA. Further analysis is planned to attempt to develop a ‘composite’ questionnaire that might perform better than any of the individual instruments.

What implications does this study have for these tools as screening questionnaires to be used in patients with psoriasis? The results suggest that if patients test positive on these questionnaires the majority of patients with PsA involving the peripheral joints will be identified. However, the two caveats are, firstly, other subgroups of PsA (such as axial or entheseal) may not be so readily detected, and, secondly, quite a lot of other articular pathology will also be identified at the same time. This means that dermatologists using these questionnaires as an indication of which patients to refer for a full rheumatological examination will also be referring many cases of noninflammatory arthritis. Although this represents an increased burden on rheumatology services, the majority of patients did have some form of musculoskeletal disease and so a single assessment by rheumatology would still be appropriate. There may be other ways to select referrals. From Table 1 it is clear that PsA has a greater impact on the patient, as measured by HAQ and PsAQoL scores, and this may be used as a guide for referral. After all, even if the patient has some other rheumatological disease, if that disease is impacting severely on the patient then rheumatological referral is appropriate anyway. Table 1 also indicates that the PsA cases had more additional disease features that might distinguish them to the dermatologist: more tender and swollen joints, more enthesitis and dactylitis and worse skin and nail disease. However, this presupposes a certain amount of musculoskeletal examination skill by the dermatologist, which may not be appropriate in all centres despite one study showing reasonably good reliability for ‘expert’ dermatologists assessing articular disease.19

In summary, this study has shown that the PEST and ToPAS questionnaires performed slightly better than the PASE questionnaire at identifying PsA, although there is probably little difference between these instruments. If choice of one were required the PEST is quickest and easiest to complete from a patient point of view, and easiest to score from a physician point of view. These screening tools do identify many cases of musculoskeletal disease other than PsA and therefore require further refinement. One possible way forward is to add other ‘filters’ such as the degree of impact of the arthritis. Another is to attempt to distil the most discriminatory features from all three questionnaires and to use those as a new improved screening tool and this study is currently underway.

Acknowledgments

We would like to acknowledge the contribution of an unrestricted investigator-initiated study grant by Pfizer and the support of the following people: Joyce Leman, Consultant Dermatologist, Western Infirmary, Glasgow; Dr Ian Gaywood and Dr Ira Pande, Consultant Rheumatologists, Nottingham University Teaching Hospitals; Joanne Llewellyn and Sue Davies Jones, Dermatology CLRN nurses, University of Nottingham Centre of Evidence Based Medicine.

Ancillary