First-rank symptoms for schizophrenia

  • Protocol
  • Diagnostic

Authors


Abstract

This is the protocol for a review and there is no abstract. The objectives are as follows:

To determine the diagnostic accuracy of one or multiple first-rank symptoms for diagnosing schizophrenia, verified by clinical history and examination by a qualified professional (e.g. psychiatrist, nurse, social worker), with or without the use of operational criteria and checklists, in people thought to have non-organic psychotic symptoms.

Background

Target condition being diagnosed

Schizophrenia is a psychotic disorder that can occur as a single episode of illness, although the majority of sufferers have remissions and relapses, and for many sufferers the condition becomes chronic and disabling (Bustillo 2001). The most effective method of treatment is antipsychotic medication: first- and second-generation antipsychotics. However, although second-generation antipsychotics are used for those with a poor response to first-generation antipsychotics, both produce various side effects (Kane 2001). As there is evidence to suggest that early intervention can be beneficial, early and accurate diagnostic techniques would have particular utility (Marshall 2011).

Index test(s)

The index test being evaluated in this review is a set of clinical symptoms called Schneider's first-rank symptoms, which include: auditory hallucinations; thought withdrawal, insertion and interruption; thought broadcasting; somatic hallucinations; delusional perception; and feelings or actions as made or influenced by external agents (Schneider 1959; Table 1). These are positive symptoms, i.e. they are symptoms not usually experienced by people without schizophrenia (as opposed to negative symptoms that are deficits of emotional responses or other thought processes) and are usually given priority among other positive symptoms, such as other forms of delusions or hallucinations. These are currently incorporated into the major operationalised diagnostic systems of the International Classification of Diseases (ICD-10) (Table 2) and Diagnostic and Statistical Manual of Mental Disorders (DSM-III-IV) (Table 3), which go beyond the relatively simple list produced by Schneider. The presence of even one of these first-rank symptoms is said to be strongly suggestive of schizophrenia, in the absence of organic brain disease (Schneider 1959); it is postulated that this may be symptomatically sufficient for a diagnosis of schizophrenia. We will examine whether the presence of any one, or multiple first-rank symptoms, is a useful diagnostic tool to differentiate schizophrenia from other psychotic disorders. However, first-rank symptoms have been described in subsequent studies in people with other psychiatric diagnoses as well, such as mood disorders with psychotic symptoms, thus raising doubts about their specificity in the diagnosis of schizophrenia (Koehler 1978; Koehler 1979).

Table 1. Schneider first-rank symptoms
First-rank symptomDefinitionExample
Auditory hallucinations

Auditory perceptions with no cause

These auditory hallucinations have to be of particular types:

 
hearing thoughts spoken aloud"I hear my thoughts outside my head."
hearing voices referring to himself/herself made in the third person"The first voice says 'He used that fork in an odd way' and then the second replies 'Yes, he did'".
auditory hallucinations in the form of a commentary"They say 'He is sitting down now talking to the psychiatrist'".
Thought withdrawal, insertion and interruptionA person's thoughts are under control of an outside agency and can be removed, inserted (and felt to be alien to him/her) or interrupted by others"My thoughts are fine except when Michael Jackson stops them."
Thought broadcastingAs the person is thinking everyone is thinking in unison with him/her"My thoughts filter out of my head and everyone can pick them up if they walk past."
Somatic hallucinationsA hallucination involving the perception of a physical experience with the body"I feel them crawling over me."
Delusional perceptionA true perception, to which a person attributes a false meaningA perfectly normal event such as the traffic lights turning red may be interpreted by the patient as meaning that Martians are about to land
Feelings or actions experienced as made or influenced by external agentsWhere there is certainty that an action of the person or a feeling is caused not by themselves but by some others or other force"The CIA controlled my arm."
Table 2. ICD criteria for schizophrenia
Although no strictly pathognomonic symptoms can be identified, for practical purposes it is useful to divide symptoms into groups that have special importance for the diagnosis and often occur together, such as:
athought echo, thought insertion or withdrawal, and thought broadcasting; 
bdelusions of control, influence or passivity, clearly referred to body or limb movements or specific thoughts, actions or sensations; delusional perception; 
challucinatory voices giving a running commentary on the patient's behaviour, or discussing the patient among themselves, or other types of hallucinatory voices coming from some part of the body; 
dpersistent delusions of other kinds that are culturally inappropriate and completely impossible, such as religious or political identity, or superhuman powers and abilities (e.g. being able to control the weather, or being in communication with aliens from another world); 
epersistent hallucinations in any modality, when accompanied either by fleeting or half-formed delusions without clear affective content, or by persistent over-valued ideas, or when occurring every day for weeks or months on end; 
fbreaks or interpolations in the train of thought, resulting in incoherence or irrelevant speech, or neologisms; 
gcatatonic behaviour, such as excitement, posturing or waxy flexibility, negativism, mutism and stupor; 
h"negative" symptoms such as marked apathy, paucity of speech, and blunting or incongruity of emotional responses, usually resulting in social withdrawal and lowering of social performance; it must be clear that these are not due to depression or to neuroleptic medication;
ia significant and consistent change in the overall quality of some aspects of personal behaviour, manifest as loss of interest, aimlessness, idleness, a self absorbed attitude and social withdrawal.
The normal requirement for a diagnosis of schizophrenia is that a minimum of 1 very clear symptom (and usually 2 or more if less clear-cut) belonging to any one of the groups listed as (a) to (d) above, or symptoms from at least 2 of the groups referred to as (e) to (h), should have been clearly present for most of the time during a period of 1 month or more. Conditions meeting such symptomatic requirements but of duration less than 1 month (whether treated or not) should be diagnosed in the first instance as acute schizophrenia-like psychotic disorder and are classified as schizophrenia if the symptoms persist for longer periods.
Table 3. DSM-IV criteria for schizophrenia
A*

Characteristic symptoms: 2 or more of the following, each present for a significant portion of time during a 1-month period:

  • delusions

  • hallucinations

  • disorganised speech (e.g. frequent derailment or incoherence)

  • grossly disorganised or catatonic behaviour

  • negative symptoms (i.e. affective flattening, alogia or avolition).

B Social/occupational dysfunction: since the onset of the disturbance, 1 or more major areas of functioning, such as work, interpersonal relations or self care, are markedly below the level previously achieved
C Duration: continuous signs of the disturbance persist for at least 6 months. This 6-month period must include at least 1 month of symptoms (or less if successfully treated) that meet Criterion A
D Exclusion of schizoaffective disorder and mood disorder with psychotic features
E Substance/general medical condition exclusion: the disturbance is not due to the direct physiological effects of a substance (e.g. a drug of abuse, a medication) or a general medical condition
F Relationship to a pervasive developmental disorder: if there is a history of autistic disorder or another pervasive development disorder, the diagnosis of schizophrenia is made only if prominent delusions or hallucinations are also present for at least a month (or less if successfully treated)
* Only 1 Criterion A symptom is required if delusions are bizarre or hallucinations consist of a voice keeping up a running commentary on the person's behaviour or thoughts, or 2 or more voices conversing with each other

Clinical pathway

The onset of schizophrenia usually occurs in adolescence or early adulthood, and around seven people out of 1000 will be affected during their lifetime (McGrath 2008). This means that the lifetime prevalence of the illness is around 0.5% to 1%. The confirmation of the diagnosis of schizophrenia is largely determined by symptom stability (of psychosis and of first-rank symptoms) and (at least in a majority) a deteriorating course (not reaching pre-morbid levels of functioning).

Prior test(s)

Before the use of the symptoms stipulated in the list of those designated to be of first rank (Schneider 1959; Table 1) diagnosis tended to be made by clinical observation.

Role of index test(s)

Schneider's efforts did help make diagnoses more operational - although use of the checklist was never free of criticism, with issues around false positive diagnoses (Koehler 1978) and, therefore, potentially damaging mislabelling of people (Koehler 1979). Nevertheless a simple checklist such as Schneider's first-rank symptoms could continue to be of value for use in situations where healthcare workers are not highly trained. First-rank symptoms could be used to screen out the seriously mentally ill for further consideration by more specialised services.

Alternative test(s)

Largely operational criteria that have superseded Schneider's list confirm the diagnosis of schizophrenia by determining symptom stability (of psychosis and of first-rank symptoms) and (at least in a majority) a deteriorating course (not reaching pre-morbid levels of functioning). These operational criteria that incorporate first-rank symptoms as well as confirm longitudinally are likely to be the current reference standard. The new DSM-5 is, however, moving away from special treatment of Schneiderian first-rank symptoms (Tandon 2013) to very diagnostic stipulations, "raising the symptom threshold" and necessitating considerably more skill to illicit than the relatively simple first-rank symptoms.

In this comparison it is likely that reference standards may well incorporate first-rank symptoms and hence potentially dilute the specificity of the sole use of first-rank symptoms. Also, conversely, in many cases using first-rank symptoms may also include taking a history and clinical examination, again contaminating the uniqueness of either approach. Differences between first-rank symptoms and the reference standards lie in i) utilisation of a longitudinal framework in addition to the cross-sectional assessment of specific symptoms of psychosis, such as first-rank symptoms (reflecting limbic system abnormalities), and ii) less specific symptoms of psychosis, such as the consequences of having acute psychotic symptoms and the deleterious effects of psychosis.

Rationale

Over half a century ago Schneider described 11 symptoms as being characteristic of schizophrenia and therefore exhibiting a "first-rank" status in the hierarchy of potentially diagnostic symptoms (Schneider 1959). First-rank symptoms have played an extremely important role in more recent diagnostic systems: the DSM-III, DSM-IV and the ICD-10. These international classifications include first-rank symptoms but also involve additional parameters, which involve more time and skills to assess when compared with the relatively simple checklist of Schneider. A diagnosis of schizophrenia is usually made by clinical examination and patient's history, often using these diagnostic systems, which incorporate first-rank symptoms. Schneider's first-rank symptoms can be considered a simpler form of the usual method of diagnosing schizophrenia. Because schizophrenia is equally prevalent worldwide, most people with this illness live in middle or low-income countries where access to highly trained professionals is often limited. Easy diagnostic techniques may have particular utility in these situations.

Five subtypes of schizophrenia have been described: paranoid, disorganised, catatonic, undifferentiated and residual type, but none are clearly discrete and allow confident prediction of the long-term course of the disease. However, insidious slow onset of illness lasting for several months is associated with a poor prognosis when compared with acute onset linked to stress and lasting only a few weeks (Lawrie 2004). Early and accurate diagnosis and treatment of schizophrenia may have long-term advantages for the patient (De Haan 2003); there is also evidence that the longer psychosis goes unnoticed and untreated the more severe the repercussions for relapse and recovery (Bottlender 2003). If schizophrenia is not really the diagnosis, embarking on a schizophrenia treatment path could be very deleterious, due to the stigma associated with a diagnosis of schizophrenia and having intrusive treatment with considerable physical, social and psychological adverse effects. Furthermore, if the correct diagnosis is another psychotic disorder with some symptoms similar to schizophrenia - the most likely being bipolar disorder – treatment tailored to schizophrenia may cause symptoms to be ignored and appropriate treatment delayed, with possible severe repercussions for the person involved and their family.

There is widespread uncertainty about the diagnostic specificity and sensitivity of first-rank symptoms; we will examine whether they are a useful diagnostic tool to differentiate schizophrenia from other psychotic disorders.

Objectives

To determine the diagnostic accuracy of one or multiple first-rank symptoms for diagnosing schizophrenia, verified by clinical history and examination by a qualified professional (e.g. psychiatrist, nurse, social worker), with or without the use of operational criteria and checklists, in people thought to have non-organic psychotic symptoms.

Methods

Criteria for considering studies for this review

Types of studies

We will include studies of all designs, except case control studies that use healthy controls. Studies will be included that evaluate the diagnostic accuracy of first-rank symptoms (one or multiple) for the diagnosis of schizophrenia compared with the reference standard, irrespective of publication status and language.

Participants

Participants will include adolescents and adults presenting with psychotic symptoms, which may include symptoms such as hallucinations, delusions, disordered thinking and speech, grossly disorganised or catatonic behaviour, or negative symptoms (i.e. affective flattening, alogia or volition). We will not exclude on the grounds of co-morbidities. In addition, if a study reports all admissions to a psychiatric ward instead of only those with psychosis, the study will not be excluded. We will exclude if participants have organic source of psychosis, such as psychosis triggered by an existent physical disease or alcohol and drug abuse.

For all participants, we will pay particular attention to their history, their current clinical state (acute, post-acute or quiescent), stage of illness (prodromal, early, established, late) or whether there are predominant clinical issues (negative or positive symptoms). In addition, the setting and referral status of people in the study will be of note. Those in psychiatric hospital have already experienced a considerable degree of screening compared with those in community settings. People referred to a specialist centre treating only those with schizophrenia are different to those in general care.

Index tests

Schneider first-rank symptoms (Table 1). The presence of any one of these symptoms, or multiple symptoms, would be indicative of a diagnosis of schizophrenia.

Kurt Schneider proposed that the presence of any one of these symptoms was diagnostic of schizophrenia as long as the person was free of other organic causes, such as substance misuse, epilepsy or tumours. The different value of one symptom over another is not the focus of this review.

Target conditions

All types of schizophrenia disorder regardless of descriptive subcategory (e.g. paranoid, disorganised, catatonic, undifferentiated and residual). For studies reporting combined results for schizophrenia, schizoaffective or schizophreniform disorders where data cannot be separated, we will include the study and investigate potential heterogeneity.

Reference standards

The reference standard is history and clinical examination collected by a qualified professional (e.g. psychiatrist, nurse, social worker), which may or may not involve the use of operational criteria or checklists of symptoms such as:

The more modern of these criteria involve some degree of follow-up.

Where studies report more than one reference standard, for example where multiple operational criteria are applied, we will firstly use DSM if reported, then ICD, RDC and finally other operational criteria as the reference standard for these studies.

Search methods for identification of studies

Electronic searches

We will conduct searches in MEDLINE, EMBASE and PsycINFO using OvidSP (see Table 4 for details).

Table 4. Search strategies
  1. All databases searched from inception.

    All searches were undertaken, added to a common database and duplicates deleted.

Database Phase and date Search strategy
MEDLINE (OvidSP)

Phase I

Date: 13-04-11

1          first-rank.mp.

2          first rank.mp.

3          first?rank.mp.

4          FRS$.mp.

5          Schneiderian.mp

6          1 or 2 or 3 or 4 or 5

Phase II

Date: 01-06-11

1          exp "International Classification of Diseases"/ (3305)

2          exp "Diagnostic and Statistical Manual of Mental Disorders"/ (9349)

3          "Research Diagnostic Criteria".mp. (1325)

4          Feighner.mp. (147)

5          ICD.mp. (13774)

6          DSM.mp. (29660)

7          RDC.mp. (1143)

8          schneider.mp. (1529)

9          bleuler.mp. (215)

10        kraepelin.mp. (495)

11        "international pilot study of schizophrenia".mp. (47)

12        IPSS.mp. (1436)

13        "new haven schizophrenia index".mp. (13)

14        NHSI.mp. (14)

15        "present state examination".mp. (412)

16        PSE.mp. (1394)

17        "operational criteria".mp. (431)

18        (operation$ adj3 criteri$).mp. [mp=protocol supplementary concept, rare disease supplementary concept, title, original title, abstract, name of substance word, subject heading word, unique identifier] (1077)

19        "Sensitivity and Specificity"/ (233760)

20        Diagnosis/ (15703)

21        1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18 or 19 or 20 (303793)

22        exp Schizophrenia/ (74720)

23        schizophren$.mp. (93319)

24        22 or 23 (93528)

25        21 and 24 (7401)

26        (animals not (humans and animals)).sh. (3503422)

27        25 not 26 (7398)

Phase III

Date: 17-07-11

1          *schizophrenia/di [Diagnosis]
EMBASE (OvidSP)

Phase I

Date: 13-04-11

1          first-rank.mp.

2          first rank.mp.

3          first?rank.mp.

4          FRS$.mp.

5          Schneiderian.mp

6          1 or 2 or 3 or 4 or 5

Phase II

Date: 01-06-11

1          exp "International Classification of Diseases"/ (5070)

2          exp "Diagnostic and Statistical Manual of Mental Disorders"/ (17196)

3          "Research Diagnostic Criteria".mp. (1436)

4          Feighner.mp. (161)

5          ICD.mp. (20919)

6          DSM.mp. (38725)

7          RDC.mp. (1310)

8          schneider.mp. (2302)

9          bleuler.mp. (293)

10        kraepelin.mp. (661)

11        "international pilot study of schizophrenia".mp. (40)

12        IPSS.mp. (2881)

13        "new haven schizophrenia index".mp. (12)

14        NHSI.mp. (22)

15        "present state examination".mp. (447)

16        PSE.mp. (1625)

17        "operational criteria".mp. (557)

18        (operation$ adj3 criteri$).mp. [mp=title, abstract, subject headings, heading word, drug trade name, original title, device manufacturer, drug manufacturer, device trade name, keyword] (1358)

19        "Sensitivity and Specificity"/ (139513)

20        Diagnosis/ (544275)

21        1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18 or 19 or 20 (755543)

22        exp Schizophrenia/ (110049)

23        schizophren$.mp. (120515)

24        22 or 23 (121863)

25        21 and 24 (12378)

26        Human/ (12332237)

27        nonhuman/ (3642333)

28        26 and 27 (656673)

29        27 not 28 (2985660)

30        25 not 29 (12368)

Phase III

Date: 17-07-11

1          *schizophrenia/di [Diagnosis]
PsycINFO (OvidSP)

Phase I

Date: 13-04-11

1          first-rank.mp.

2          first rank.mp.

3          first?rank.mp.

4          FRS$.mp.

5          Schneiderian.mp

6          1 or 2 or 3 or 4 or 5

Phase II

Date: 01-06-11

1          exp "International Classification of Diseases"/ (747)

2          "Research Diagnostic Criteria".mp. (1393)

3          Feighner.mp. (169)

4          ICD.mp. (4427)

5          DSM.mp. (42426)

6          RDC.mp. (392)

7          schneider.mp. (1283)

8          bleuler.mp. (464)

9          kraepelin.mp. (736)

10        "international pilot study of schizophrenia".mp. (68)

11        IPSS.mp. (53)

12        "new haven schizophrenia index".mp. (20)

13        NHSI.mp. (7)

14        "present state examination".mp. (811)

15        PSE.mp. (440)

16        "operational criteria".mp. (462)

17        (operation$ adj3 criteri$).mp. [mp=title, abstract, heading word, table of contents, key concepts, original title, tests & measures] (810)

18        Diagnosis/ (25820)

19        "Research Diagnostic Criteria".mp. (1393)

20        exp "Diagnostic and Statistical Manual"/ (4184)

21        exp Research Diagnostic Criteria/ (122)

22        1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 or 11 or 12 or 13 or 14 or 15 or 16 or 17 or 18 or 19 or 20 or 21 (72691)

23        exp SCHIZOPHRENIA/ (62498)

24        schizophren$.mp. (88734)

25        23 or 24 (88734)

26        22 and 25 (10364)

27        limit 26 to human (10180)

MEDIONDate: 24-02-11(schizophrenia or schizophrenic or schizophreniform in title or abstract) or (psychosis or psychoses or psychotic in title or abstract)

We will not apply any restrictions based on language or type of document in the search. We will use the multipurpose search command for the OvidSP interface (.mp.) to search both text and database subject heading fields. To capture variations in suffix endings, we will use the unlimited truncation symbol '*'.

Searching other resources

We will identify additional references by manually searching references of included and excluded studies.

Data collection and analysis

We will follow the available guidelines provided in the Cochrane Diagnostic Reviewer’s Handbook (DTA Handbook 2011).

Selection of studies

NM and KSW will independently screen all titles and abstracts for eligibility. We will retrieve full papers of potentially relevant studies, as well as review articles, if they are relevant, for a manual reference search. NM and KSW will independently review the full papers for eligibility according to the inclusion criteria detailed above. Abstracts in the absence of a full publication will be included if sufficient data are provided for analysis. Any disagreements will be resolved by discussion between NM and KSW and we will document all decisions. If a consensus cannot be reached, CEA or CD will make the final decision on these studies.

Data extraction and management

We will develop data extraction forms using DistillerSR software and pilot these on a small selection of studies. NM and KSW, again working independently, will complete the data extraction form for all included studies. Agreements and disagreements will be recorded and resolved by discussion between NM and KSW. If the issue cannot be resolved we will not add these data to analyses and label them as 'unable to use' with the reason given pending further information. CEA will make the final decision in these cases.

We will extract the information on study characteristics listed in Table 5.

Table 5. Study characteristics
Study detailsFirst author, year, publication status, country, aim of study
Patient characteristics and setting

Number of participants included in study and number in analysis

Description of participants in the study (age, gender, ethnicity, comorbid disorders, duration of symptoms and concurrent medications used)

Predefined inclusion and exclusion criteria

Study aim

Previous treatment for schizophrenia

Clinical setting

Country

Index test

Description of first-rank symptoms used

Professionals performing test

Resolution of discrepancies

How first-rank symptoms used in study

Target condition and reference standard(s)

Reference standard

Target condition(s)

Professionals performing test

Resolution of discrepancies

Flow and timing

Study process

Follow-up

We will record the number of true positive (TP), true negative (TN), false positive (FP) and false negative (FN) to construct a 2 x 2 table for each study for differentiating schizophrenia from other diagnoses, from other psychotic diagnoses and from non-psychotic diagnoses. If such data are not available we will attempt to derive them from summary statistics such as sensitivity, specificity and/or likelihood ratios, if reported. We will treat data as dichotomous, with a high-risk result (as defined by each individual study) being regarded as test positive (suggestive or diagnostic of schizophrenia), and a low-risk result being regarded as test negative (suggestive of absence of schizophrenia). Where data are available for one and/or multiple first-rank symptoms, or at several time points, we will record these. 

Assessment of methodological quality

Key issues to be considered in study design are:

  • representativeness;

  • verification of disease status;

  • independence of assessments; and

  • completeness of information.

Eligible studies will be assessed by NM and KSW, again working independently using a form that we will pilot on a small selection of studies. Any disagreements will be solved by consensus with CEA and CD.

We will use QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies), an updated version of the original QUADAS tool for the assessment of quality in systematic reviews of diagnostic accuracy studies (Whiting 2011). The QUADAS-2 tool is made up of the four domains: patient selection, index test, reference standard, and flow and timing. We have tailored the tool to our review (see Appendix 1) and will use it to judge the risk of bias and applicability of the studies. We will first pilot the tool on a small number of studies, measure the inter-rater agreement and adapt the tool if necessary. We will then apply it to the other included studies.

We will use the results of the quality assessment to describe the overall quality of the included studies and to evaluate the validity of the included studies. We will also use the results to make recommendations for the design of future studies. We are aware that quality rating is important but also that it is problematic to pre-define cut-off points beyond which inclusion of data would be contraindicated. We, therefore, do not propose to use QUADAS-2 other than to help the qualitative commentary.

Statistical analysis and data synthesis

We will plot study-specific estimates of sensitivity and specificity in ROC space and forest plots for visual examination of variation in the accuracy of first-rank symptoms. When adequate data are available, we will undertake meta-analyses using the hierarchical summary ROC (HSROC) model (Rutter 2001). The model estimates the position and shape of the SROC curve whilst also accounting for variation within and between studies. To produce a SROC curve using all available studies irrespective of the cut-off used, if a study reports multiple cut-offs and hence multiple 2 x 2 tables, we will select one 2 x 2 table for inclusion in the meta-analysis based on the cut-off most commonly used in included studies. We recognise the limitation of this data-driven approach but there are no standard cut-offs used in practice. If we encounter problems with model convergence due to few studies or sparse data, we will simplify the HSROC model by assuming a symmetrical shape for the SROC curve. For meta-analysis of studies that used the same or similar cut-offs, we will use parameter estimates from the models to derive summary sensitivities and specificities and corresponding 95% confidence regions, and summary likelihood ratios. All models will be fitted in SAS using the macro MetaDAS (Takwoingi 2010) or the NLMIXED procedure as proposed by Macaskill (Macaskill 2004).

Investigations of heterogeneity

We will create covariates based on the factors outlined under investigation of sources of heterogeneity. In the first instance, we will investigate heterogeneity by using the covariates for subgroup analyses on forest plots and SROC plots in RevMan. If the data permit, we will add covariates to the HSROC model to assess their effect on test accuracy, threshold and/or shape of the SROC curve. We will evaluate the statistical significance of differences in test performance using likelihood ratio tests.

We will investigate the following possible sources of heterogeneity.

  1. Whether operational criteria were used as part of the reference standard.

  2. Whether first-rank symptoms were used as part of the reference standard.

  3. All admissions to a psychiatric ward or people with specific psychoses.

  4. Whether the definition of schizophrenia in the study included schizoaffective and/or schizophreniform.

If possible, we will compare the results as follows.

  1. Schizophrenia from all other diagnoses.

  2. Schizophrenia from other types of psychosis.

  3. Schizophrenia from non-psychotic disorders.

Sensitivity analyses

We will undertake sensitivity analyses to investigate the impact of blinding when conducting the tests.

Assessment of reporting bias

Standard funnel plots and tests for publication bias are likely to be misleading for meta-analysis of test accuracy studies (Deeks 2005) and so we will not assess publication bias.

Acknowledgements

The authors would like to thank the Diagnostic Test Accuracy Editorial Team for their support throughout the protocol development, in particular Yemisi Takwoingi for advice on statistical analysis methods.

Appendices

Appendix 1. QUADAS 2

DOMAIN 1: PATIENT SELECTION
Risk of bias: Could the selection of patients have introduced bias?
Signalling question1Was a consecutive or random sample of patients enrolled?
‘Yes’ if a random sample of patients with suspected psychotic symptoms were included, or consecutive patients were enrolled
‘No’ if the patients were specifically selected (not random sample) to be included in the study
‘Unclear’ if insufficient information is provided
2Was a case-control design avoided?
‘Yes’ if all participants had not been previously diagnosed with schizophrenia.
‘No’ if healthy controls were also included, or participants with other symptoms
‘Unclear’ if insufficient information is provided
3Did the study avoid inappropriate exclusions?
‘Yes’ if all patients recruited were given both the index test and reference standard
‘No’ if some patients were excluded if they were deemed “difficult-to-diagnose” patients
‘Unclear’ if insufficient information is provided
Applicability
Signalling question1Are there concerns that the included patients and setting do not match the review question?
‘No’ if the inclusion criteria of the study are patients with a positive clinical diagnosis of (non-organic psychosis) of undetermined cause
‘Yes’ if the inclusion criteria for the study are patients without a positive clinical diagnosis of (non-organic psychosis) of undetermined cause
DOMAIN 2: INDEX TEST
Risk of bias: Could the conduct or interpretation of the index test have introduced bias?
Signalling question1Were the index test results interpreted without knowledge of the results of the reference standard?
‘Yes’ if the index test was conducted before the reference standard, or if the person applying the index test was blinded to the results of the reference standard
‘No’ if the index test operator knew the results of the reference standard
‘Unclear’ if insufficient information is provided
2Did the study pre-specify whether they were using one or multiple first-rank symptoms (FRSs)?
‘Yes’ if the study states the number of FRSs needed to be present to diagnose schizophrenia
‘No’ if the study does not state the number of FRSs they considered necessary to diagnose schizophrenia
Applicability
Signalling question1Are there concerns that the index test, its conduct or interpretation, differ from the review question?
‘No’ if FRSs are used for diagnosing schizophrenia
‘Yes’ if the study is not using FRSs for the diagnosis of schizophrenia, e.g. the prognosis of patients, or the prevalence of FRSs
‘Unclear’ if insufficient information is provided to judge the purpose of applying FRSs
DOMAIN 3: REFERENCE STANDARD
Risk of bias: Could the reference standard, its conduct or its interpretation have introduced bias?
Signalling question1Is the reference standard likely to correctly classify the target condition?
‘Yes’ if the history and clinical examination is conducted by a qualified professional (psychiatrist, nurse, social worker)
‘No’ if the history and clinical examination is conducted by insufficiently qualified individuals
‘Unclear’ if insufficient information is provided
2Were the reference standard results interpreted without knowledge of the results of the index test?
‘Yes’ if the reference standard was conducted before the index test, or if the person applying the reference standard was blinded to the results of the index test
‘No’ if the reference standard operator knew the results of the index test
‘Unclear’ if insufficient information is provided
Applicability
Signalling question1Are there concerns that the target condition as defined by the reference standard does not match the question?
‘No’ if the paper specifically looks at diagnosing schizophrenia (regardless of subtypes)
‘Yes’ if the paper also includes schizophrenia-like illnesses
‘Unclear’ if insufficient information is provided
DOMAIN 4: FLOW AND TIMING 
Risk of bias: Could the patient flow have introduced bias?
Signalling question1

Was there an appropriate interval between index test and reference standard?

'Yes' if reference standard and index text were applied in the same interview or within 4 weeks (applied more than once for chronic schizophrenia)
'No' if reference standard or index test were applied in different interviews beyond 4 weeks
'Unclear' if not enough information is given to assess whether there was an appropriate interval

2

Did all patients receive a reference standard?

'Yes’ if all patients had details of history and clinical examination, with or without operational criteria
‘No’ if not all patients had a description of history and clinical examination
‘Unclear’ if insufficient information is provided

3Did all patients receive the same reference standard?
‘Yes’ if all patients were diagnosed with history and clinical examination; and if any operational criteria were used, the same ones were applied to all patients and all received the same clinical follow-up
‘No’ if all patients received history and clinical examination but only some received operational criteria, or different operational criteria
‘Unclear’ if insufficient information is provided
4Were all patients included in the analysis?
'Yes' if there are no patients excluded from the analysis
'No' if there are patients excluded from the analysis
'Unclear' if not enough information is given to assess whether any patients were excluded from the analysis

Contributions of authors

Karla Soares-Weiser - led project and drafted the protocol.

Nicola Maayan - helped draft protocol.

Clare Davenport - helped guide and draft protocol.

Amanda Kirkham - statistical support.

Clive E Adams - helped draft protocol.

Declarations of interest

Karla Soares-Weiser - currently works for Enhance Reviews Ltd, a company that carries out systematic reviews mostly for the public sector, it currently does not provide services for the pharmaceutical industry.

Nicola Maayan - currently works for Enhance Reviews Ltd, a company that carries out systematic reviews mostly for the public sector, it currently do not provide services for the pharmaceutical industry.

Clare Davenport - none known.

Amanda Kirkham - none known.

Clive E Adams - none known.

Sources of support

Internal sources

  • University of Nottingham, UK.

External sources

  • NIHR Cochrane Programme Grant 2011, UK.

    Reference number: 10/4001/15, UK