To determine the agreement and reliability of the Nail Psoriasis Severity Index (NAPSI) in the assessment of nail involvement in patients with psoriatic arthritis (PsA) when performed by rheumatologists with no experience in using this instrument.
In total, 3 women with PsA, satisfying the Classification of Psoriatic Arthritis Study Group criteria, with nail involvement were selected from an outpatient clinic devoted to PsA. The assessors consisted of 2 groups: 8 expert rheumatologists in the field of PsA who were members of the Group for Research and Assessment of Psoriasis and Psoriatic Arthritis and had extensive experience of >10 years, and 69 rheumatologists who had never previously used the NAPSI. A video showing the nail of each selected patient (patient A, patient B, and patient C) with the most nail PsA dystrophy was shown to these 2 groups. The 8 assessors of the first group, previously trained in using the NAPSI, evaluated the 3 videos independently by using the NAPSI score. The second group scored the NAPSI after an educational session. This evaluation was repeated after 6 hours with a different sequence of videos (unpaired fashion). Interreader and intrareader reliability were estimated by calculating intraclass correlation coefficients (ICCs) and associated 95% confidence intervals (95% CIs).
The interreader reliability showed ICC 0.934 (95% CI 0.7504–0.9983). Intrareader reliability showed ICC 0.463 (95% CI 0.134–0.668), ICC 0.148 (95% CI 0.3767–0.4722), and ICC 0.354 (95% CI 0.0425–0.600) for patient A, patient B, and patient C, respectively.
These results show that the NAPSI may be an unreliable instrument to assess nail involvement when used by untrained rheumatologists in clinical practice.
Comprehensive assessment of patients with psoriatic arthritis (PsA) involves evaluation of joints, dactylitis, skin, and nails (1–5). Nail lesions are very common and help distinguish between patients who have PsA and those who have rheumatoid arthritis (6). Nail lesions occur in ∼40–45% of patients with psoriasis uncomplicated by arthritis, and in ∼87% of patients with PsA (7). The Nail Psoriasis Severity Index (NAPSI) is a numerical, reproducible, objective, and simple scale for the evaluation of nail bed psoriasis and nail matrix psoriasis. The NAPSI was useful during clinical trials for evaluating response to treatment of psoriatic nails (8), and a modified version (mNAPSI) was developed to enhance the face validity and feasibility of this tool (9). In 2009, a study aimed to determine whether assessment of the skin and joints in patients with PsA by rheumatologists and dermatologists was reproducible. An excellent agreement was obtained (intraclass correlation coefficient [ICC] >0.80) between dermatologists and rheumatologists on the mNAPSI, whereas the agreement for other parameters was moderate or fair (10).
Although the mNAPSI showed excellent interrater reliability (9), it has not yet been determined whether the assessments of the NAPSI or the mNAPSI are reliable in real life by rheumatologists who are not involved in clinical trials. In fact, the mNAPSI, which is very useful in clinical research, considers semiquantitative scores of some features (i.e., onycholysis, oil drop dyschromia, pitting, and nail plate crumbling) that could be difficult to grade in real life by untrained rheumatologists.
Therefore, the purpose of this study was to determine the agreement and reliability of the NAPSI in the assessment of nail involvement in patients with PsA when performed by rheumatologists without any experience using this instrument.
Significance & Innovations
Nail involvement is very common in psoriatic arthritis (PsA), and there is an unmet need on how to measure this important aspect of daily practice for patients with PsA.
The Nail Psoriasis Severity Index (NAPSI) was investigated as a reliable tool.
The NAPSI may be an unreliable instrument to assess nail involvement when used by untrained rheumatologists in clinical practice.
Patients and methods
In total, 3 women with PsA with nail involvement were selected from the outpatient clinic devoted to PsA at the Academic Rheumatology Unit of Policlinico Umberto I at Sapienza University in Rome, Italy. All of the patients also had psoriatic skin lesions, and all met the Classification of Psoriatic Arthritis Study Group criteria for the classification of PsA (11).
The assessors consisted of 2 groups. The first group included 8 expert Italian rheumatologists in the field of PsA who were members of the Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA) and had >10 years of experience in the assessment of PsA. The second group included 69 rheumatologists attending an educational meeting on an update of PsA according to the Italian regulation on Continuing Medical Education. The educational meeting was open to rheumatologists working in hospital settings who had never used the NAPSI for the assessment of the nail involvement in patients with PsA.
To make the assessment simple, based on a visual scoring of nail involvement, the authors agreed to record a video. Therefore, one of the authors (RS) recorded the video, lasting no more than 2 minutes, showing the nail of each selected patient (patient A, patient B, and patient C) with the most nail PsA dystrophy. The 8 assessors in the first group independently evaluated the 3 videos using the NAPSI score. Following the methods outlined by Rich and Scher (8), the nail was divided into quadrants with imaginary horizontal and longitudinal lines. Each nail was given a score for nail bed psoriasis (score 0–4) and nail matrix psoriasis (score 0–4) depending on the presence of any of the features of nail psoriasis in that quadrant.In each nail quadrant, nail matrix psoriasis was evaluated by the presence of any of the nail matrix features (pitting, leukonychia, red spots in the lunula, and crumbling) and scored on a scale of 0–4, where 0 = not present in any nail quadrants, 1 = present in 1 quadrant, 2 = present in 2 quadrants, 3 = present in 3 quadrants, and 4 = present in 4 quadrants. Nail bed psoriasis was evaluated by the presence of any of the nail bed features (onycholysis, splinter hemorrhages, subungual hyperkeratosis, and oil drop [salmon patch dyschroma]) and scored on a scale of 0–4, where 0 = not present in any nail quadrants, 1 = present in 1 quadrant, 2 = present in 2 quadrants, 3 = present in 3 quadrants, and 4 = present in 4 quadrants. Each nail was given a nail matrix score and a nail bed score, the total of which was the score for that nail (range 0–8).
After this first step, the experts discussed the abnormalities of each patient, and they defined the final score for each video using a Delphi technique. The second group was then divided into 4 subgroups. Four expert Italian rheumatologists (1 for each subgroup) explained the components of the NAPSI score for 30 minutes to each subgroup. After this educational session, the assessors of each subgroup evaluated, by a standard NAPSI grading sheet, the 3 videos with psoriatic nails projected on a screen within 30 minutes. This evaluation was repeated after 6 hours with a different sequence of videos (unpaired fashion).
Interreader and intrareader reliability were estimated by the 2-way mixed-effects model calculating ICCs and associated 95% confidence intervals (95% CIs). According to Chandran et al (10), we interpreted the ICC values in the following way: values between 0.80 and 1.00 represented excellent agreement beyond chance, between 0.60 and 0.80 represented substantial agreement, between 0.40 and 0.60 represented moderate agreement, between 0.20 and 0.40 represented fair agreement, and between 0.0 and 0.20 represented poor agreement beyond chance. All statistical analyses were performed using SPSS (11.0.0) statistical software.
The characteristics of the 69 rheumatologists who participated in the study were as follows: there were 31 men and 38 women (ratio 0.82), the median age was 45 years (range 25–60 years), the median professional activity duration was 9.5 years (range 1–29 years), 17 rheumatologists (24.6%) had <10 PsA-related visits per month, and 52 rheumatologists (75.4%) had >10 PsA-related visits per month.
The total NAPSI scores for patient A, patient B, and patient C, as graded by the 69 rheumatologists in the second group, are shown in Figure 1, Figure 2, and Figure 3, respectively. Twenty-one (30.4%), 24 (34.8%), and 18 (26.1%) of the 69 rheumatologists recorded the same final NAPSI score calculated by the 8 expert rheumatologists for patient A, patient B, and patient C, respectively.
The interreader reliability showed an ICC of 0.934 (95% CI 0.7504–0.9983). The intrareader reliability showed ICCs of 0.463 (95% CI 0.134–0.668), 0.148 (95% CI 0.3767–0.4722), and 0.354 (95% CI 0.0425–0.600) for patient A, patient B, and patient C, respectively. The rheumatologist's assessment was not dependent on any variables, such as sex, age, professional activity duration, or number of PsA-related visits.
During Outcome Measures in Rheumatology 8, a survey on outcome measures was carried out by GRAPPA. In particular, the role of nail involvement in patients with psoriasis and PsA was discussed, and it was deemed a common and important problem (4). Therefore, the assessment of nails was considered to be recommended but not mandatory for randomized controlled trials or for longitudinal observational studies; it was allocated to the “outer core” (5). Allocation to the “outer core” was the result of breakout groups and based on the paucity of validated instruments for some of the domains (5).
Reliable assessment of disease is important in patient care, clinical trials, and longitudinal observational studies. Many studies have evaluated the effectiveness of various therapies for nail psoriasis, assessing clinical improvement in a global manner or in a target nail (8). The NAPSI is a scale that is simple and quick to calculate; it is used to evaluate the severity of nail psoriasis, and it was developed to evaluate the response to the treatment of psoriatic nails in clinical trials (8). Recently, a review on the role of the NAPSI to assess the use of biologic agents in the treatment of nail psoriasis showed that it measures the improvement induced by treatment (12), meaning it has a good sensitivity to change. In a previous study, there was substantial to excellent agreement among expert dermatologists and rheumatologists on the assessment of nails (10), but it was not determined whether the assessments of the NAPSI by rheumatologists who were not involved in clinical trials were reliable in real life. In fact, in the article by Chandran et al, all of the assessors were members of GRAPPA, which represents a group of rheumatologists and dermatologists with both clinical and scientific expertise in clinimetry of PsA and psoriasis, respectively. Conversely, the group of assessors in the present study was characterized by clinicians treating only patients with PsA as practitioners and without a metrology approach. Indeed, these rheumatologists could represent the majority of physicians working in the outpatient clinics in Italy.
Our results showed that one-third of the rheumatologists that had never used the NAPSI for the assessment of nail involvement in patients with PsA agreed with the score of the expert rheumatologists. Moreover, the interreader reliability was high, and this result was also shown in other studies (4). Nevertheless, intrareader reliability showed a variable agreement in the different patients. In fact, the ICCs were moderate (patient A), poor (patient B), and fair (patient C), according to Chandran et al (10). These results could be explained by the fact that the different pattern of nail lesions could affect the reliability of the NAPSI. In fact, the scoring process for nails using the NAPSI could be relatively easy when classic lesions are present, but very difficult in the case of concomitant lesions where a ceiling effect of the NAPSI could be obtained. In other words, when used in complicated nail conditions, the NAPSI could be too difficult for an untrained rheumatologist to use, therefore confirming that it is an unreliable instrument in these situations.
Another explanation could be the insufficient training of the assessors; longer training could have positively affected the final score. We tried to be comprehensive in the training course. Of course, a more thorough study would have consisted of real patient evaluations and untrained doctors with a Latin square design, with more lengthy training and more examples as a test before the exercise. Nevertheless, the large number of rheumatologists did not allow us to adopt this statistical approach. However, GRAPPA also has an educational mission to implement the knowledge of all aspects involving patients with PsA, and this study could suggest pursuing this goal, since there is a need to obtain more reliable and feasible instruments to assess all the domains of PsA in daily clinical practice.
In conclusion, objective measurements of clinical improvement or worsening of nail psoriasis are of value in guiding medical therapy and standardizing clinical trials, suggesting that the development of an educational training program using the NAPSI in rheumatologic settings could be useful. Therefore, in order to use the NAPSI in real life, an improvement of the intrareader reliability and sensitivity to change of this instrument is required.
All authors were involved in drafting the article or revising it critically for important intellectual content, and all authors approved the final version to be published. Dr. Lubrano had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study conception and design. Lubrano, Scarpa, Spadaro.
Acquisition of data. Scrivo, Marchesoni, Spadaro.
Analysis and interpretation of data. Cantini, Mathieu, Olivieri, Salvarani.