Target condition being diagnosed
Schizophrenia is a psychotic disorder that can occur as a single episode of illness, although the majority of sufferers have remissions and relapses, and for many sufferers the condition becomes chronic and disabling (Bustillo 2001). The most effective method of treatment is antipsychotic medication: first- and second-generation antipsychotics. However, although second-generation antipsychotics are used for those with a poor response to first-generation antipsychotics, both produce various side effects (Kane 2001). As there is evidence to suggest that early intervention can be beneficial, early and accurate diagnostic techniques would have particular utility (Marshall 2011).
The index test being evaluated in this review is a set of clinical symptoms called Schneider's first-rank symptoms, which include: auditory hallucinations; thought withdrawal, insertion and interruption; thought broadcasting; somatic hallucinations; delusional perception; and feelings or actions as made or influenced by external agents (Schneider 1959; Table 1). These are positive symptoms, i.e. they are symptoms not usually experienced by people without schizophrenia (as opposed to negative symptoms that are deficits of emotional responses or other thought processes) and are usually given priority among other positive symptoms, such as other forms of delusions or hallucinations. These are currently incorporated into the major operationalised diagnostic systems of the International Classification of Diseases (ICD-10) ( Table 2) and Diagnostic and Statistical Manual of Mental Disorders (DSM-III-IV) ( Table 3), which go beyond the relatively simple list produced by Schneider. The presence of even one of these first-rank symptoms is said to be strongly suggestive of schizophrenia, in the absence of organic brain disease (Schneider 1959); it is postulated that this may be symptomatically sufficient for a diagnosis of schizophrenia. We will examine whether the presence of any one, or multiple first-rank symptoms, is a useful diagnostic tool to differentiate schizophrenia from other psychotic disorders. However, first-rank symptoms have been described in subsequent studies in people with other psychiatric diagnoses as well, such as mood disorders with psychotic symptoms, thus raising doubts about their specificity in the diagnosis of schizophrenia (Koehler 1978; Koehler 1979).
The onset of schizophrenia usually occurs in adolescence or early adulthood, and around seven people out of 1000 will be affected during their lifetime (McGrath 2008). This means that the lifetime prevalence of the illness is around 0.5% to 1%. The confirmation of the diagnosis of schizophrenia is largely determined by symptom stability (of psychosis and of first-rank symptoms) and (at least in a majority) a deteriorating course (not reaching pre-morbid levels of functioning).
Role of index test(s)
Schneider's efforts did help make diagnoses more operational - although use of the checklist was never free of criticism, with issues around false positive diagnoses (Koehler 1978) and, therefore, potentially damaging mislabelling of people (Koehler 1979). Nevertheless a simple checklist such as Schneider's first-rank symptoms could continue to be of value for use in situations where healthcare workers are not highly trained. First-rank symptoms could be used to screen out the seriously mentally ill for further consideration by more specialised services.
Largely operational criteria that have superseded Schneider's list confirm the diagnosis of schizophrenia by determining symptom stability (of psychosis and of first-rank symptoms) and (at least in a majority) a deteriorating course (not reaching pre-morbid levels of functioning). These operational criteria that incorporate first-rank symptoms as well as confirm longitudinally are likely to be the current reference standard. The new DSM-5 is, however, moving away from special treatment of Schneiderian first-rank symptoms (Tandon 2013) to very diagnostic stipulations, "raising the symptom threshold" and necessitating considerably more skill to illicit than the relatively simple first-rank symptoms.
In this comparison it is likely that reference standards may well incorporate first-rank symptoms and hence potentially dilute the specificity of the sole use of first-rank symptoms. Also, conversely, in many cases using first-rank symptoms may also include taking a history and clinical examination, again contaminating the uniqueness of either approach. Differences between first-rank symptoms and the reference standards lie in i) utilisation of a longitudinal framework in addition to the cross-sectional assessment of specific symptoms of psychosis, such as first-rank symptoms (reflecting limbic system abnormalities), and ii) less specific symptoms of psychosis, such as the consequences of having acute psychotic symptoms and the deleterious effects of psychosis.
Over half a century ago Schneider described 11 symptoms as being characteristic of schizophrenia and therefore exhibiting a "first-rank" status in the hierarchy of potentially diagnostic symptoms (Schneider 1959). First-rank symptoms have played an extremely important role in more recent diagnostic systems: the DSM-III, DSM-IV and the ICD-10. These international classifications include first-rank symptoms but also involve additional parameters, which involve more time and skills to assess when compared with the relatively simple checklist of Schneider. A diagnosis of schizophrenia is usually made by clinical examination and patient's history, often using these diagnostic systems, which incorporate first-rank symptoms. Schneider's first-rank symptoms can be considered a simpler form of the usual method of diagnosing schizophrenia. Because schizophrenia is equally prevalent worldwide, most people with this illness live in middle or low-income countries where access to highly trained professionals is often limited. Easy diagnostic techniques may have particular utility in these situations.
Five subtypes of schizophrenia have been described: paranoid, disorganised, catatonic, undifferentiated and residual type, but none are clearly discrete and allow confident prediction of the long-term course of the disease. However, insidious slow onset of illness lasting for several months is associated with a poor prognosis when compared with acute onset linked to stress and lasting only a few weeks (Lawrie 2004). Early and accurate diagnosis and treatment of schizophrenia may have long-term advantages for the patient (De Haan 2003); there is also evidence that the longer psychosis goes unnoticed and untreated the more severe the repercussions for relapse and recovery (Bottlender 2003). If schizophrenia is not really the diagnosis, embarking on a schizophrenia treatment path could be very deleterious, due to the stigma associated with a diagnosis of schizophrenia and having intrusive treatment with considerable physical, social and psychological adverse effects. Furthermore, if the correct diagnosis is another psychotic disorder with some symptoms similar to schizophrenia - the most likely being bipolar disorder – treatment tailored to schizophrenia may cause symptoms to be ignored and appropriate treatment delayed, with possible severe repercussions for the person involved and their family.
There is widespread uncertainty about the diagnostic specificity and sensitivity of first-rank symptoms; we will examine whether they are a useful diagnostic tool to differentiate schizophrenia from other psychotic disorders.
To determine the diagnostic accuracy of one or multiple first-rank symptoms for diagnosing schizophrenia, verified by clinical history and examination by a qualified professional (e.g. psychiatrist, nurse, social worker), with or without the use of operational criteria and checklists, in people thought to have non-organic psychotic symptoms.
Criteria for considering studies for this review
Types of studies
We will include studies of all designs, except case control studies that use healthy controls. Studies will be included that evaluate the diagnostic accuracy of first-rank symptoms (one or multiple) for the diagnosis of schizophrenia compared with the reference standard, irrespective of publication status and language.
Participants will include adolescents and adults presenting with psychotic symptoms, which may include symptoms such as hallucinations, delusions, disordered thinking and speech, grossly disorganised or catatonic behaviour, or negative symptoms (i.e. affective flattening, alogia or volition). We will not exclude on the grounds of co-morbidities. In addition, if a study reports all admissions to a psychiatric ward instead of only those with psychosis, the study will not be excluded. We will exclude if participants have organic source of psychosis, such as psychosis triggered by an existent physical disease or alcohol and drug abuse.
For all participants, we will pay particular attention to their history, their current clinical state (acute, post-acute or quiescent), stage of illness (prodromal, early, established, late) or whether there are predominant clinical issues (negative or positive symptoms). In addition, the setting and referral status of people in the study will be of note. Those in psychiatric hospital have already experienced a considerable degree of screening compared with those in community settings. People referred to a specialist centre treating only those with schizophrenia are different to those in general care.
Schneider first-rank symptoms ( Table 1). The presence of any one of these symptoms, or multiple symptoms, would be indicative of a diagnosis of schizophrenia.
Kurt Schneider proposed that the presence of any one of these symptoms was diagnostic of schizophrenia as long as the person was free of other organic causes, such as substance misuse, epilepsy or tumours. The different value of one symptom over another is not the focus of this review.
All types of schizophrenia disorder regardless of descriptive subcategory (e.g. paranoid, disorganised, catatonic, undifferentiated and residual). For studies reporting combined results for schizophrenia, schizoaffective or schizophreniform disorders where data cannot be separated, we will include the study and investigate potential heterogeneity.
The reference standard is history and clinical examination collected by a qualified professional (e.g. psychiatrist, nurse, social worker), which may or may not involve the use of operational criteria or checklists of symptoms such as:
- International Statistical Classification of Diseases (ICD-9 or ICD-10) ( Table 2);
- Diagnostic and Statistical Manual of Mental Disorders (DSM-III or DSM-IV) ( Table 3);
- Feighner (Feighner 1972);
- Research Diagnostic Criteria (RDC) (Spitzer 1978);
- Carpenter criteria (Carpenter 1973);
- New Haven (Astrachan 1974);
- Taylor Abrams (Taylor 1978).
The more modern of these criteria involve some degree of follow-up.
Where studies report more than one reference standard, for example where multiple operational criteria are applied, we will firstly use DSM if reported, then ICD, RDC and finally other operational criteria as the reference standard for these studies.
Search methods for identification of studies
We will conduct searches in MEDLINE, EMBASE and PsycINFO using OvidSP (see Table 4 for details).
We will not apply any restrictions based on language or type of document in the search. We will use the multipurpose search command for the OvidSP interface (.mp.) to search both text and database subject heading fields. To capture variations in suffix endings, we will use the unlimited truncation symbol '*'.
Searching other resources
We will identify additional references by manually searching references of included and excluded studies.
Data collection and analysis
We will follow the available guidelines provided in the Cochrane Diagnostic Reviewer’s Handbook (DTA Handbook 2011).
Selection of studies
NM and KSW will independently screen all titles and abstracts for eligibility. We will retrieve full papers of potentially relevant studies, as well as review articles, if they are relevant, for a manual reference search. NM and KSW will independently review the full papers for eligibility according to the inclusion criteria detailed above. Abstracts in the absence of a full publication will be included if sufficient data are provided for analysis. Any disagreements will be resolved by discussion between NM and KSW and we will document all decisions. If a consensus cannot be reached, CEA or CD will make the final decision on these studies.
Data extraction and management
We will develop data extraction forms using DistillerSR software and pilot these on a small selection of studies. NM and KSW, again working independently, will complete the data extraction form for all included studies. Agreements and disagreements will be recorded and resolved by discussion between NM and KSW. If the issue cannot be resolved we will not add these data to analyses and label them as 'unable to use' with the reason given pending further information. CEA will make the final decision in these cases.
We will extract the information on study characteristics listed in Table 5.
We will record the number of true positive (TP), true negative (TN), false positive (FP) and false negative (FN) to construct a 2 x 2 table for each study for differentiating schizophrenia from other diagnoses, from other psychotic diagnoses and from non-psychotic diagnoses. If such data are not available we will attempt to derive them from summary statistics such as sensitivity, specificity and/or likelihood ratios, if reported. We will treat data as dichotomous, with a high-risk result (as defined by each individual study) being regarded as test positive (suggestive or diagnostic of schizophrenia), and a low-risk result being regarded as test negative (suggestive of absence of schizophrenia). Where data are available for one and/or multiple first-rank symptoms, or at several time points, we will record these.
Assessment of methodological quality
Key issues to be considered in study design are:
- verification of disease status;
- independence of assessments; and
- completeness of information.
Eligible studies will be assessed by NM and KSW, again working independently using a form that we will pilot on a small selection of studies. Any disagreements will be solved by consensus with CEA and CD.
We will use QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies), an updated version of the original QUADAS tool for the assessment of quality in systematic reviews of diagnostic accuracy studies (Whiting 2011). The QUADAS-2 tool is made up of the four domains: patient selection, index test, reference standard, and flow and timing. We have tailored the tool to our review (see Appendix 1) and will use it to judge the risk of bias and applicability of the studies. We will first pilot the tool on a small number of studies, measure the inter-rater agreement and adapt the tool if necessary. We will then apply it to the other included studies.
We will use the results of the quality assessment to describe the overall quality of the included studies and to evaluate the validity of the included studies. We will also use the results to make recommendations for the design of future studies. We are aware that quality rating is important but also that it is problematic to pre-define cut-off points beyond which inclusion of data would be contraindicated. We, therefore, do not propose to use QUADAS-2 other than to help the qualitative commentary.
Statistical analysis and data synthesis
We will plot study-specific estimates of sensitivity and specificity in ROC space and forest plots for visual examination of variation in the accuracy of first-rank symptoms. When adequate data are available, we will undertake meta-analyses using the hierarchical summary ROC (HSROC) model (Rutter 2001). The model estimates the position and shape of the SROC curve whilst also accounting for variation within and between studies. To produce a SROC curve using all available studies irrespective of the cut-off used, if a study reports multiple cut-offs and hence multiple 2 x 2 tables, we will select one 2 x 2 table for inclusion in the meta-analysis based on the cut-off most commonly used in included studies. We recognise the limitation of this data-driven approach but there are no standard cut-offs used in practice. If we encounter problems with model convergence due to few studies or sparse data, we will simplify the HSROC model by assuming a symmetrical shape for the SROC curve. For meta-analysis of studies that used the same or similar cut-offs, we will use parameter estimates from the models to derive summary sensitivities and specificities and corresponding 95% confidence regions, and summary likelihood ratios. All models will be fitted in SAS using the macro MetaDAS (Takwoingi 2010) or the NLMIXED procedure as proposed by Macaskill (Macaskill 2004).
Investigations of heterogeneity
We will create covariates based on the factors outlined under investigation of sources of heterogeneity. In the first instance, we will investigate heterogeneity by using the covariates for subgroup analyses on forest plots and SROC plots in RevMan. If the data permit, we will add covariates to the HSROC model to assess their effect on test accuracy, threshold and/or shape of the SROC curve. We will evaluate the statistical significance of differences in test performance using likelihood ratio tests.
We will investigate the following possible sources of heterogeneity.
- Whether operational criteria were used as part of the reference standard.
- Whether first-rank symptoms were used as part of the reference standard.
- All admissions to a psychiatric ward or people with specific psychoses.
- Whether the definition of schizophrenia in the study included schizoaffective and/or schizophreniform.
If possible, we will compare the results as follows.
- Schizophrenia from all other diagnoses.
- Schizophrenia from other types of psychosis.
- Schizophrenia from non-psychotic disorders.
We will undertake sensitivity analyses to investigate the impact of blinding when conducting the tests.
Assessment of reporting bias
Standard funnel plots and tests for publication bias are likely to be misleading for meta-analysis of test accuracy studies (Deeks 2005) and so we will not assess publication bias.
The authors would like to thank the Diagnostic Test Accuracy Editorial Team for their support throughout the protocol development, in particular Yemisi Takwoingi for advice on statistical analysis methods.
Appendix 1. QUADAS 2
Contributions of authors
Karla Soares-Weiser - led project and drafted the protocol.
Nicola Maayan - helped draft protocol.
Clare Davenport - helped guide and draft protocol.
Amanda Kirkham - statistical support.
Clive E Adams - helped draft protocol.
Declarations of interest
Karla Soares-Weiser - currently works for Enhance Reviews Ltd, a company that carries out systematic reviews mostly for the public sector, it currently does not provide services for the pharmaceutical industry.
Nicola Maayan - currently works for Enhance Reviews Ltd, a company that carries out systematic reviews mostly for the public sector, it currently do not provide services for the pharmaceutical industry.
Clare Davenport - none known.
Amanda Kirkham - none known.
Clive E Adams - none known.
Sources of support
- University of Nottingham, UK.
- NIHR Cochrane Programme Grant 2011, UK.Reference number: 10/4001/15, UK