Validity of the personality disorder diagnosis in the Swedish National Patient Register

The Swedish National Patient Register (NPR) is an exceptional source of information in clinical epidemiological research. The register is, however, not validated for the diagnosis of personality disorder (PD). We therefore assessed its validity in this patient group, and the group with emotionally unstable PD (EUPD)/borderline personality disorder (BPD).


Introduction
The idiom 'The chain is no stronger than its weakest link' is particularly relevant in epidemiological studies in which the validity of underlying data is crucial for any intellectual interpretation. The large healthcare databases and registries that have been created in the past few decades have been the basis for health surveillance and have also given ample opportunities for epidemiological research. National longitudinal registers of healthcare contacts are increasingly common, with several countries now maintaining registers with mandatory individual-level reporting of details from all healthcare contacts.
Since 1964, Sweden has such a National Patient Register (NPR) (1), which contains information on an individual basis for patients in public care, including medical diagnoses using the International Classification of Diseases (ICD) system (2). The population-based public healthcare system has enabled this register to develop into a highly valuable source of information for many epidemiological studies. The issue of the NPR's validity has been approached for some diagnostic entities (1), but to a surprisingly low extent for diagnoses in psychiatric care. Despite this formal weakness, the register has often been used in studies of psychiatric disorders. Public attempts to validate psychiatric disorders have been restricted to a few studies and types of disorders. Thus, the validity of a schizophrenia diagnosis (3,4) or bipolar disorder (5) has been ensured. The validity and reliability of chronic tic disorder and obsessive-compulsive disorder have also been confirmed (6). In a recent study, a positive predictive value of 63% was reported for the diagnosis of borderline personality disorder (BPD) (7). In general, however, the validity of the NPR for the important group of personality disorders (PDs) has not been determined.
A PD is a way of thinking, feeling and behaving that deviates from the expectations of the culture, causes distress or problems in functioning and lasts over an extended period (8). Different PDs have several common features, largely because the core feature of PD is the inability to form and sustain satisfactory interpersonal relationships (9,10). The wide differences in expression have motivated the description of PD as 'A disease in disguise' (9) and categorization into eight PD groups in the ICD-10 (11) and 10 groups in both the DSM-IV and DSM-5 (8,12).
PDs are highly maladaptive, affect the performance of expected occupational and social roles and might lead to withdrawal from peers, co-workers and family. PDs constitute a global health problem and are expected to occur in more than five per cent of adults worldwide (13). High comorbidity between PDs and other psychiatric (14,15) and somatic conditions is known to exist (16). From this, it follows that PDs are common in psychiatric health care, with rough estimates close to or above 50% of all psychiatric patients (10,15,17). Studies show that individuals with one or more PDs have higher mortality and a shorter life expectancy than the background population (18)(19)(20)(21)(22)(23). Above all, there is an increased risk of death due to murder (24), suicide or accident (21,22,25). Further and more in-depth studies on the epidemiology and consequences of having a PD require that the underlying data are validated.

Aims of the study
The present study aims to fill a knowledge gap in the NPR by validating, or rather substantiating, the registered diagnosis of PD by comparing it with available information in patient records. We also validated the diagnosis of the most frequent subtype diagnosis in the register, the emotionally unstable personality disorder (EUPD), which corresponds to borderline personality disorder in the DSM system. Finally, we took the opportunity to validate the register as to how well the registered diagnosis matched a diagnosis based on DSM criteria.

Methodological approach
The Swedish NPR includes all individuals admitted, and all admissions, to any psychiatric or general hospital, has partial coverage between 1973 and 1986 and has almost complete coverage since 1987 (1). Since 2001, all out-patient visits to specialized psychiatric care are also registered. Patients were classified according to  between 1987 and 1996 and to ICD-10 after 1996. PDs in the NPR are quantitatively dominated by those with EUPD, as well as those who have been diagnosed with 'other PD' or 'PD unspecified' (e.g. (21,22)).

Method
After the approval from the Regional Ethics Review Board in Uppsala (Dnr 2016/234), we requested information from the NPR for 100 randomly selected patients who had been registered with any of the defined diagnoses from two counties in Sweden between and including the years 1987 and 2015. We did not differentiate between primary and secondary diagnoses. Randomization was restricted to those patients who were between 15 and 65 years old at the time when they were first given the diagnosis in question. Stratification was done to establish representation in the sample regarding the time of diagnosis, sex and whether the diagnosis was obtained in an in-patient or outpatient setting. We requested personal identifiers (PIDs), county of care, the date for the selected registration and all registered diagnostic codes for that registration. The stratification and randomization procedure is described in the Supplement. Both counties gave prompt and full access to all requested medical records.

Process
A diagnosis of PD is based on general diagnostic guidelines in the ICD classification system (11), as well as on fulfilment of six defined criteria, named A to F, in the DSM system (8,12). On top of these general guidelines/criteria, which should be met for any diagnosis of PD, there are supplementary diagnostic guidelines for each of the subtypes in the ICD system and defined specific criteria in the DSM system. In this study, a prestudy protocol was developed based on the above-mentioned diagnostic requirements both for PD as such, that is what is required for an unspecified PD diagnosis and the subtype EUPD/BPD. The full protocol is shown in the Supplement.
This protocol was used in reviewing the patients' records for documentation related to the general criteria for PD and the specific criteria for EUPD, anywhere in the record up to the date when the diagnosis was reported to the NPR. Fulfilment of each of the diagnostic requirements, both according to the ICD and the DSM systems, was evaluated separately, individually and blindly. Each of the general diagnostic criteria was assessed as '0' when the criterion in question was not met, '1' when there is a good but not certain agreement with the criterion and '2' when the criterion was fulfilled. Based on this assessment, the result was registered for each patient as 'not sufficient evidence for a diagnosis of PD', 'probable diagnosis of PD' or 'verified diagnosis of PD'. Subsequently, the reviewers discussed their assessments, went back to patient records when necessary, and ultimately agreed on a consensus diagnosis that was used to validate the NPR.
As an add-on, patients who were reported with a subtype of PD in the NPR, or patients with records that gave strong suspicion for a certain subtype, were also validated regarding fulfilment of criteria for a subtype diagnosis.
The process was conducted by two senior psychiatrists with clinical experience and training in personality diagnostics, authors EK and LE. After initial calibration, the kappa value for interrater agreement between the two reviewers was 0.85 for the ICD-10 diagnosis of any PD and 1.0 for EUPD.
Details about specific diagnostic procedures undertaken as part of the care process were also registered.

Statistics
Data on individuals are presented as raw values and percentages. Age is reported as mean AE standard deviation (SD) and median (range). To overcome weaknesses in Cohen's kappa (27), interrater reliability was assessed by the prevalence-and bias-adjusted Kappa according to in Byrt et al. (28). These authors proposed an adjusted kappa that assumes 50% prevalence of the condition and absence of any bias. Interrater agreement was calculated by dividing the number of individuals with validated diagnoses of either PD or EUPD by the reviewers, with the total number of assessable individuals with a corresponding diagnosis obtained from the NPR. All data were processed with IBM SPSS Statistics version 25 for Windows (IBM, Armonk, New York, USA).

Results
Of the 100 randomized patients, 10 were classified according to ICD-9 and 90 to ICD-10. Twentyseven patients were registered with a diagnosis of EUPD and 50 with a diagnosis of an unspecified PD (Table 1). Sixty-two per cent of the patients were women, and the mean age of all patients at the date of registration was 36 years. Forty-two per cent of the patients had a main diagnosis of PD, while 58% of the PD diagnose were registered as comorbidity. The main diagnoses in the 58 persons who had a comorbid diagnosis of PD were affective spectrum (n = 22), anxiety spectrum (n = 16) and psychotic spectrum (n = 3) (Table S2).
Of the 100 PIDs obtained, 95 could be linked to patients that were available for assessment (Table 1). For three PIDs, no patient record could be found, and for two PIDs, the records contained insufficient content for any analysis, like only a preface.

ICD-10 diagnoses in the register
The reviewers identified documented support for a diagnosis of PD in 88 (93%) of the 95 evaluable patient records ( Table 1). The agreement rate was equally high for those who presented with a primary diagnosis of PD (35 of 39 patients or 90%) and those with a secondary diagnosis (53 of 56 patients or 95%) ( Table 2). Moreover, agreement was similarly high for those given the diagnosis in out-patient care (36 of 36 patients) and those given the diagnosis in in-patient care (52 of 59 patients or 88%). There were 26 evaluable patients with a diagnosis of EUPD in the cohort, of which only one was a man (Table 2). In addition, the reviewers found that another three, who were registered as unspecified PD also fulfilled specific criteria for EUPD.
Specific PD diagnoses other than EUPD were recorded in 23 of the 100 patients, of whom three had no evaluable records (Table 1). Of the remaining 20 patients registered with other specific PD diagnoses, the reviewers could ascertain general criteria for PD in only 17. In eight of those, there were only criteria for an unspecified, but not for a specified, PD in the records.
A structured diagnostic process could be discerned in 34 (36 %) of the 95 patient records. The reviewers could not confirm the diagnosis in only one of these 34 patients.
There were only minor differences between the two counties (Table S3).  ). Using DSM-IV criteria, the reviewers only identified 25 of 29 patients who were identified with EUPD using ICD-10 criteria ( Table 2).

Discussion
The results of this study show that the PD diagnoses reported in the Swedish NPR generally have high validity for fulfilment of the general criteria. Such high validity provides support for the use of the register in epidemiological research, which contains issues related to the contribution of PDs. The high validity also holds for women with the most common specific PD registered in health care, namely EUPD. The validation process used in this study was restricted to a retrospective review of patient records for those registered with the diagnoses in question. Other, more elaborate modes of validation (e.g. comparisons with other independent sources) were not available. Thus, from a strict standpoint, our study did not validate the diagnosis of PD as such, but rather corroborated that the observations in the register corresponded to criteria fulfilment in the patient records. Nevertheless, the results obtained are based on similar approaches as for other diagnostic entities, which have shown that, from this perspective, the overall quality of the NPR and its functionality are good (1).
Regrettably, this validation study does not offer any information on individuals who fulfil PD criteria but who are not registered as such in the clinical records. This group is probably considerably large given that previous studies have shown that the true incidence rate of PD in psychiatric populations is close to 50% (10,15,17), whereas it is less than four per cent among those with a psychiatric diagnosis in the NPR (29). It was therefore not judged possible to identify a control group of psychiatric patients that would be virtually negative for PD. A hypothesis, based on the obvious underreporting of diagnoses in the NPR, is that those reported have well-fulfilled criteria and a comparatively clear clinical pattern, which should support the fairly high level of agreement.
There are several mechanisms responsible for the underreporting of PD in NPR. One is that limited time in encounters with patients prevents extensive, personality-focused interviews. Another mechanism is that very few patients seek help for personality dysfunction. Rather, PDs are characteristically identified while evaluating other psychiatric states or exacerbation of symptoms, for example severe depressive or anxiety states. Also, the risk of stigmatization could act as a deterrence for diagnosis in the early phases of clinical contact. Furthermore, in clinical work, there is a tendency to focus on current psychiatric symptoms even in cases where PD seems to be the main cause of the patients suffering from the very beginning. Moreover, fulfilment of the general criteria of endurance and inflexibility may not be obvious until after patients have undergone extensive clinical contact or after treatment failure for coexisting conditions. The current reviewers perceived such a time factor in many of the evaluated patients. It was noted that the clinical pattern was increasingly obvious over time. For example, in four patients, the reviewers were not fully confident with the diagnosis at the time when it was reported to the NPR but found good documentation based on later encounters.
A secondary purpose of this study was to assess whether the diagnoses in the NPR reflect the documentation in the records for a PD diagnosis based on DSM criteria. An issue to consider is that while the DSM and ICD seem to have a good concordance concerning PDs as such (30), there are some differences in diagnostic guidelines. For instance, ICD clinical descriptions permit a higher degree of freedom than the operationalized criteria in the DSM. In Sweden, the DSM is mostly used in clinical praxis, although all coding is done according to the ICD, like in the rest of Europe. This approach has important consequences for the diagnoses registered in the NPR. Our validation process revealed that the documentation required by the DSM system was not met to the desired extent. Above all, there was insufficient information on whether the enduring problem pattern could better be a manifestation or consequence of another mental disorder, or whether it was due to physiological effects of a substance or a general medical condition (i.e. criteria E and F) (8,12).
The observations made in this study support previous notions that insufficient documentation for high-quality validation of the diagnosis of PD is not uncommon (31). The validation process in this study was entirely dependent on the documentation in the patient records. The first (and strongest) impression was that the overall quality of the patient records was low, with key information related to diagnostic criteria not presented in a systematic and structured manner. Rather, reading long narratives, which sometimes contained nonrelevant information for the patient's care, was required to obtain enough information. The quality of information was considerably better in the cases in which structured methods (e.g. SCID II) were used. However, this could be an obfuscation in that using a structured diagnosis does not per se indicate that it is more valid than one based on good clinical assessment (32). Finally, most cases lacked an expressed differential diagnostic argumentation regarding the possibility of other explanations for the symptoms, something that is required in DSM, but not explicitly expressed in the clinical descriptions and diagnostic guidelines in ICD-10 (11).
This validation study had its focus on fulfilment of the general criteria for PDs, which, above all, reflects the presence of personality dysfunction. It therefore represents the view in the upcoming ICD-11, which focuses on the dimensional description regarding the severity of the disturbed functioning rather than on the type of clinical presentation (9,33,34). Consequently, this validation for fulfilment of the general criteria of PD is more relevant than a validation that has its focus on the type of clinical presentation but ignores the general criteria.
The present study sought to validate PD diagnoses across ICD-9 and ICD-10, two separate classification systems which, regarding PDs, contained differences regarding subtypes of the disorder. In 2 years, ICD-11, which classifies PDs dimensionally by severity rather than by type of symptoms, will be in operation (35). ICD-11 is largely similar to the classification of PD in an alternative model in the current DSM-5 (36) while the DSM-5 main document maintains the old categorization. A dimensional classification is a radical shift in terms of personality pathology, which has been considered appropriate (35). However, it creates new challenges when using healthcare databases for longitudinal studies crossing diagnostic borders between the classification systems.
An issue not coveredand outside the scope of our studyis that previous studies have shown that the trajectory of patients fulfilling criteria for PD may change over time (37,38). These findings suggest that a diagnosis of PD is not set in stone but may change overtime. Nevertheless, this validation study had its focus on whether there was fulfilment of diagnostic criteria for PD in the patient records at the time of registering that motivated a diagnosis.

Strengths and Limitations
One of the strengths of this study is that as many as 95 of the 100 patients could be assessed. Yet, the relatively small sample size prevented validation of most specific PDs.
The inclusion of patients from only two (one predominantly rural and one predominantly urban) of 21 Swedish counties represents only 6.5% of the total population. Documentation routines in the two chosen counties may differ from that of the remaining counties, which is a weakness of the study. However, documentation routines in Sweden are highly regulated by authorities and therefore vary little from county to county. Indeed, we found no differences between the two counties.
Furthermore, the quantitative dominance of EUPD/BPD and unspecified PD limits the validity of the study for other specific PDs. Also, the EUPD/BPD group contained only one man and thus no conclusions as to the validity of the register can be drawn for this subgroup of patients.
Lastly, with the present study design, it is only possible to control for false-positive cases, which means that nothing can be said about the falsenegative cases, that is all those patients with a clinical picture of PD who are not formally diagnosed and registered.
To conclude, this study indicates that the quality of the Swedish NPR regarding a diagnosis of PD as such and for the subtype diagnosis of EUPD in women is good. Therefore, the register should be considered a valid source of data in research for these diagnostic groups.

Supporting Information
Additional Supporting Information may be found in the online version of this article: Table S1. Reported and validated diagnoses in the 100 patient records. Table S2. Main non-PD diagnoses and all comorbidities. Table S3. Distribution of validated diagnoses.