Diagnostic Test Accuracy Protocol

You have free access to this content

First-rank symptoms for schizophrenia

  1. Karla Soares-Weiser1,*,
  2. Nicola Maayan1,
  3. Clare Davenport2,
  4. Amanda J Kirkham2,
  5. Clive E Adams3

Editorial Group: Cochrane Schizophrenia Group

Published Online: 16 JUL 2013

DOI: 10.1002/14651858.CD010653


How to Cite

Soares-Weiser K, Maayan N, Davenport C, Kirkham AJ, Adams CE. First-rank symptoms for schizophrenia (Protocol). Cochrane Database of Systematic Reviews 2013, Issue 7. Art. No.: CD010653. DOI: 10.1002/14651858.CD010653.

Author Information

  1. 1

    Enhance Reviews Ltd, Wantage, UK

  2. 2

    University of Birmingham, Department of Public Health, Epidemiology and Biostatistics, Birmingham, UK

  3. 3

    The University of Nottingham, Cochrane Schizophrenia Group, Nottingham, UK

*Karla Soares-Weiser, Enhance Reviews Ltd, Central Office, Cobweb Buildings, The Lane, Lyford, Wantage, OX12 0EE, UK. karla@enhance-reviews.com. ksoaresweiser@gmail.com.

Publication History

  1. Publication Status: New
  2. Published Online: 16 JUL 2013

SEARCH

 

Background

  1. Top of page
  2. Background
  3. Objectives
  4. Methods
  5. Acknowledgements
  6. Appendices
  7. Contributions of authors
  8. Declarations of interest
  9. Sources of support
 

Target condition being diagnosed

Schizophrenia is a psychotic disorder that can occur as a single episode of illness, although the majority of sufferers have remissions and relapses, and for many sufferers the condition becomes chronic and disabling (Bustillo 2001). The most effective method of treatment is antipsychotic medication: first- and second-generation antipsychotics. However, although second-generation antipsychotics are used for those with a poor response to first-generation antipsychotics, both produce various side effects (Kane 2001). As there is evidence to suggest that early intervention can be beneficial, early and accurate diagnostic techniques would have particular utility (Marshall 2011).

 

Index test(s)

The index test being evaluated in this review is a set of clinical symptoms called Schneider's first-rank symptoms, which include: auditory hallucinations; thought withdrawal, insertion and interruption; thought broadcasting; somatic hallucinations; delusional perception; and feelings or actions as made or influenced by external agents (Schneider 1959;  Table 1). These are positive symptoms, i.e. they are symptoms not usually experienced by people without schizophrenia (as opposed to negative symptoms that are deficits of emotional responses or other thought processes) and are usually given priority among other positive symptoms, such as other forms of delusions or hallucinations. These are currently incorporated into the major operationalised diagnostic systems of the International Classification of Diseases (ICD-10) ( Table 2) and Diagnostic and Statistical Manual of Mental Disorders (DSM-III-IV) ( Table 3), which go beyond the relatively simple list produced by Schneider. The presence of even one of these first-rank symptoms is said to be strongly suggestive of schizophrenia, in the absence of organic brain disease (Schneider 1959); it is postulated that this may be symptomatically sufficient for a diagnosis of schizophrenia. We will examine whether the presence of any one, or multiple first-rank symptoms, is a useful diagnostic tool to differentiate schizophrenia from other psychotic disorders. However, first-rank symptoms have been described in subsequent studies in people with other psychiatric diagnoses as well, such as mood disorders with psychotic symptoms, thus raising doubts about their specificity in the diagnosis of schizophrenia (Koehler 1978; Koehler 1979).

 

Clinical pathway

The onset of schizophrenia usually occurs in adolescence or early adulthood, and around seven people out of 1000 will be affected during their lifetime (McGrath 2008). This means that the lifetime prevalence of the illness is around 0.5% to 1%. The confirmation of the diagnosis of schizophrenia is largely determined by symptom stability (of psychosis and of first-rank symptoms) and (at least in a majority) a deteriorating course (not reaching pre-morbid levels of functioning).

 

Prior test(s)

Before the use of the symptoms stipulated in the list of those designated to be of first rank (Schneider 1959;  Table 1) diagnosis tended to be made by clinical observation.

 

Role of index test(s)

Schneider's efforts did help make diagnoses more operational - although use of the checklist was never free of criticism, with issues around false positive diagnoses (Koehler 1978) and, therefore, potentially damaging mislabelling of people (Koehler 1979). Nevertheless a simple checklist such as Schneider's first-rank symptoms could continue to be of value for use in situations where healthcare workers are not highly trained. First-rank symptoms could be used to screen out the seriously mentally ill for further consideration by more specialised services.

 

Alternative test(s)

Largely operational criteria that have superseded Schneider's list confirm the diagnosis of schizophrenia by determining symptom stability (of psychosis and of first-rank symptoms) and (at least in a majority) a deteriorating course (not reaching pre-morbid levels of functioning). These operational criteria that incorporate first-rank symptoms as well as confirm longitudinally are likely to be the current reference standard. The new DSM-5 is, however, moving away from special treatment of Schneiderian first-rank symptoms (Tandon 2013) to very diagnostic stipulations, "raising the symptom threshold" and necessitating considerably more skill to illicit than the relatively simple first-rank symptoms.

In this comparison it is likely that reference standards may well incorporate first-rank symptoms and hence potentially dilute the specificity of the sole use of first-rank symptoms. Also, conversely, in many cases using first-rank symptoms may also include taking a history and clinical examination, again contaminating the uniqueness of either approach. Differences between first-rank symptoms and the reference standards lie in i) utilisation of a longitudinal framework in addition to the cross-sectional assessment of specific symptoms of psychosis, such as first-rank symptoms (reflecting limbic system abnormalities), and ii) less specific symptoms of psychosis, such as the consequences of having acute psychotic symptoms and the deleterious effects of psychosis.

 

Rationale

Over half a century ago Schneider described 11 symptoms as being characteristic of schizophrenia and therefore exhibiting a "first-rank" status in the hierarchy of potentially diagnostic symptoms (Schneider 1959). First-rank symptoms have played an extremely important role in more recent diagnostic systems: the DSM-III, DSM-IV and the ICD-10. These international classifications include first-rank symptoms but also involve additional parameters, which involve more time and skills to assess when compared with the relatively simple checklist of Schneider. A diagnosis of schizophrenia is usually made by clinical examination and patient's history, often using these diagnostic systems, which incorporate first-rank symptoms. Schneider's first-rank symptoms can be considered a simpler form of the usual method of diagnosing schizophrenia. Because schizophrenia is equally prevalent worldwide, most people with this illness live in middle or low-income countries where access to highly trained professionals is often limited. Easy diagnostic techniques may have particular utility in these situations.

Five subtypes of schizophrenia have been described: paranoid, disorganised, catatonic, undifferentiated and residual type, but none are clearly discrete and allow confident prediction of the long-term course of the disease. However, insidious slow onset of illness lasting for several months is associated with a poor prognosis when compared with acute onset linked to stress and lasting only a few weeks (Lawrie 2004). Early and accurate diagnosis and treatment of schizophrenia may have long-term advantages for the patient (De Haan 2003); there is also evidence that the longer psychosis goes unnoticed and untreated the more severe the repercussions for relapse and recovery (Bottlender 2003). If schizophrenia is not really the diagnosis, embarking on a schizophrenia treatment path could be very deleterious, due to the stigma associated with a diagnosis of schizophrenia and having intrusive treatment with considerable physical, social and psychological adverse effects. Furthermore, if the correct diagnosis is another psychotic disorder with some symptoms similar to schizophrenia - the most likely being bipolar disorder – treatment tailored to schizophrenia may cause symptoms to be ignored and appropriate treatment delayed, with possible severe repercussions for the person involved and their family.

There is widespread uncertainty about the diagnostic specificity and sensitivity of first-rank symptoms; we will examine whether they are a useful diagnostic tool to differentiate schizophrenia from other psychotic disorders.

 

Objectives

  1. Top of page
  2. Background
  3. Objectives
  4. Methods
  5. Acknowledgements
  6. Appendices
  7. Contributions of authors
  8. Declarations of interest
  9. Sources of support

To determine the diagnostic accuracy of one or multiple first-rank symptoms for diagnosing schizophrenia, verified by clinical history and examination by a qualified professional (e.g. psychiatrist, nurse, social worker), with or without the use of operational criteria and checklists, in people thought to have non-organic psychotic symptoms.

 

Methods

  1. Top of page
  2. Background
  3. Objectives
  4. Methods
  5. Acknowledgements
  6. Appendices
  7. Contributions of authors
  8. Declarations of interest
  9. Sources of support
 

Criteria for considering studies for this review

 

Types of studies

We will include studies of all designs, except case control studies that use healthy controls. Studies will be included that evaluate the diagnostic accuracy of first-rank symptoms (one or multiple) for the diagnosis of schizophrenia compared with the reference standard, irrespective of publication status and language.

 

Participants

Participants will include adolescents and adults presenting with psychotic symptoms, which may include symptoms such as hallucinations, delusions, disordered thinking and speech, grossly disorganised or catatonic behaviour, or negative symptoms (i.e. affective flattening, alogia or volition). We will not exclude on the grounds of co-morbidities. In addition, if a study reports all admissions to a psychiatric ward instead of only those with psychosis, the study will not be excluded. We will exclude if participants have organic source of psychosis, such as psychosis triggered by an existent physical disease or alcohol and drug abuse.

For all participants, we will pay particular attention to their history, their current clinical state (acute, post-acute or quiescent), stage of illness (prodromal, early, established, late) or whether there are predominant clinical issues (negative or positive symptoms). In addition, the setting and referral status of people in the study will be of note. Those in psychiatric hospital have already experienced a considerable degree of screening compared with those in community settings. People referred to a specialist centre treating only those with schizophrenia are different to those in general care.

 

Index tests

Schneider first-rank symptoms ( Table 1). The presence of any one of these symptoms, or multiple symptoms, would be indicative of a diagnosis of schizophrenia.

Kurt Schneider proposed that the presence of any one of these symptoms was diagnostic of schizophrenia as long as the person was free of other organic causes, such as substance misuse, epilepsy or tumours. The different value of one symptom over another is not the focus of this review.

 

Target conditions

All types of schizophrenia disorder regardless of descriptive subcategory (e.g. paranoid, disorganised, catatonic, undifferentiated and residual). For studies reporting combined results for schizophrenia, schizoaffective or schizophreniform disorders where data cannot be separated, we will include the study and investigate potential heterogeneity.

 

Reference standards

The reference standard is history and clinical examination collected by a qualified professional (e.g. psychiatrist, nurse, social worker), which may or may not involve the use of operational criteria or checklists of symptoms such as:

The more modern of these criteria involve some degree of follow-up.

Where studies report more than one reference standard, for example where multiple operational criteria are applied, we will firstly use DSM if reported, then ICD, RDC and finally other operational criteria as the reference standard for these studies.

 

Search methods for identification of studies

 

Electronic searches

We will conduct searches in MEDLINE, EMBASE and PsycINFO using OvidSP (see  Table 4 for details).

We will not apply any restrictions based on language or type of document in the search. We will use the multipurpose search command for the OvidSP interface (.mp.) to search both text and database subject heading fields. To capture variations in suffix endings, we will use the unlimited truncation symbol '*'.

 

Searching other resources

We will identify additional references by manually searching references of included and excluded studies.

 

Data collection and analysis

We will follow the available guidelines provided in the Cochrane Diagnostic Reviewer’s Handbook (DTA Handbook 2011).

 

Selection of studies

NM and KSW will independently screen all titles and abstracts for eligibility. We will retrieve full papers of potentially relevant studies, as well as review articles, if they are relevant, for a manual reference search. NM and KSW will independently review the full papers for eligibility according to the inclusion criteria detailed above. Abstracts in the absence of a full publication will be included if sufficient data are provided for analysis. Any disagreements will be resolved by discussion between NM and KSW and we will document all decisions. If a consensus cannot be reached, CEA or CD will make the final decision on these studies.

 

Data extraction and management

We will develop data extraction forms using DistillerSR software and pilot these on a small selection of studies. NM and KSW, again working independently, will complete the data extraction form for all included studies. Agreements and disagreements will be recorded and resolved by discussion between NM and KSW. If the issue cannot be resolved we will not add these data to analyses and label them as 'unable to use' with the reason given pending further information. CEA will make the final decision in these cases.

We will extract the information on study characteristics listed in  Table 5.

We will record the number of true positive (TP), true negative (TN), false positive (FP) and false negative (FN) to construct a 2 x 2 table for each study for differentiating schizophrenia from other diagnoses, from other psychotic diagnoses and from non-psychotic diagnoses. If such data are not available we will attempt to derive them from summary statistics such as sensitivity, specificity and/or likelihood ratios, if reported. We will treat data as dichotomous, with a high-risk result (as defined by each individual study) being regarded as test positive (suggestive or diagnostic of schizophrenia), and a low-risk result being regarded as test negative (suggestive of absence of schizophrenia). Where data are available for one and/or multiple first-rank symptoms, or at several time points, we will record these. 

 

Assessment of methodological quality

Key issues to be considered in study design are:

  • representativeness;
  • verification of disease status;
  • independence of assessments; and
  • completeness of information.

Eligible studies will be assessed by NM and KSW, again working independently using a form that we will pilot on a small selection of studies. Any disagreements will be solved by consensus with CEA and CD.

We will use QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies), an updated version of the original QUADAS tool for the assessment of quality in systematic reviews of diagnostic accuracy studies (Whiting 2011). The QUADAS-2 tool is made up of the four domains: patient selection, index test, reference standard, and flow and timing. We have tailored the tool to our review (see Appendix 1) and will use it to judge the risk of bias and applicability of the studies. We will first pilot the tool on a small number of studies, measure the inter-rater agreement and adapt the tool if necessary. We will then apply it to the other included studies.

We will use the results of the quality assessment to describe the overall quality of the included studies and to evaluate the validity of the included studies. We will also use the results to make recommendations for the design of future studies. We are aware that quality rating is important but also that it is problematic to pre-define cut-off points beyond which inclusion of data would be contraindicated. We, therefore, do not propose to use QUADAS-2 other than to help the qualitative commentary.

 

Statistical analysis and data synthesis

We will plot study-specific estimates of sensitivity and specificity in ROC space and forest plots for visual examination of variation in the accuracy of first-rank symptoms. When adequate data are available, we will undertake meta-analyses using the hierarchical summary ROC (HSROC) model (Rutter 2001). The model estimates the position and shape of the SROC curve whilst also accounting for variation within and between studies. To produce a SROC curve using all available studies irrespective of the cut-off used, if a study reports multiple cut-offs and hence multiple 2 x 2 tables, we will select one 2 x 2 table for inclusion in the meta-analysis based on the cut-off most commonly used in included studies. We recognise the limitation of this data-driven approach but there are no standard cut-offs used in practice. If we encounter problems with model convergence due to few studies or sparse data, we will simplify the HSROC model by assuming a symmetrical shape for the SROC curve. For meta-analysis of studies that used the same or similar cut-offs, we will use parameter estimates from the models to derive summary sensitivities and specificities and corresponding 95% confidence regions, and summary likelihood ratios. All models will be fitted in SAS using the macro MetaDAS (Takwoingi 2010) or the NLMIXED procedure as proposed by Macaskill (Macaskill 2004).

 

Investigations of heterogeneity

We will create covariates based on the factors outlined under investigation of sources of heterogeneity. In the first instance, we will investigate heterogeneity by using the covariates for subgroup analyses on forest plots and SROC plots in RevMan. If the data permit, we will add covariates to the HSROC model to assess their effect on test accuracy, threshold and/or shape of the SROC curve. We will evaluate the statistical significance of differences in test performance using likelihood ratio tests.

We will investigate the following possible sources of heterogeneity.

  1. Whether operational criteria were used as part of the reference standard.
  2. Whether first-rank symptoms were used as part of the reference standard.
  3. All admissions to a psychiatric ward or people with specific psychoses.
  4. Whether the definition of schizophrenia in the study included schizoaffective and/or schizophreniform.

If possible, we will compare the results as follows.

  1. Schizophrenia from all other diagnoses.
  2. Schizophrenia from other types of psychosis.
  3. Schizophrenia from non-psychotic disorders.

 

Sensitivity analyses

We will undertake sensitivity analyses to investigate the impact of blinding when conducting the tests.

 

Assessment of reporting bias

Standard funnel plots and tests for publication bias are likely to be misleading for meta-analysis of test accuracy studies (Deeks 2005) and so we will not assess publication bias.

 

Acknowledgements

  1. Top of page
  2. Background
  3. Objectives
  4. Methods
  5. Acknowledgements
  6. Appendices
  7. Contributions of authors
  8. Declarations of interest
  9. Sources of support

The authors would like to thank the Diagnostic Test Accuracy Editorial Team for their support throughout the protocol development, in particular Yemisi Takwoingi for advice on statistical analysis methods.

 

Appendices

  1. Top of page
  2. Background
  3. Objectives
  4. Methods
  5. Acknowledgements
  6. Appendices
  7. Contributions of authors
  8. Declarations of interest
  9. Sources of support
 

Appendix 1. QUADAS 2


DOMAIN 1: PATIENT SELECTION

Risk of bias: Could the selection of patients have introduced bias?

Signalling question1Was a consecutive or random sample of patients enrolled?
‘Yes’ if a random sample of patients with suspected psychotic symptoms were included, or consecutive patients were enrolled
‘No’ if the patients were specifically selected (not random sample) to be included in the study
‘Unclear’ if insufficient information is provided

2Was a case-control design avoided?
‘Yes’ if all participants had not been previously diagnosed with schizophrenia.
‘No’ if healthy controls were also included, or participants with other symptoms
‘Unclear’ if insufficient information is provided

3Did the study avoid inappropriate exclusions?
‘Yes’ if all patients recruited were given both the index test and reference standard
‘No’ if some patients were excluded if they were deemed “difficult-to-diagnose” patients
‘Unclear’ if insufficient information is provided

Applicability

Signalling question1Are there concerns that the included patients and setting do not match the review question?
‘No’ if the inclusion criteria of the study are patients with a positive clinical diagnosis of (non-organic psychosis) of undetermined cause
‘Yes’ if the inclusion criteria for the study are patients without a positive clinical diagnosis of (non-organic psychosis) of undetermined cause

DOMAIN 2: INDEX TEST

Risk of bias: Could the conduct or interpretation of the index test have introduced bias?

Signalling question1Were the index test results interpreted without knowledge of the results of the reference standard?
‘Yes’ if the index test was conducted before the reference standard, or if the person applying the index test was blinded to the results of the reference standard
‘No’ if the index test operator knew the results of the reference standard
‘Unclear’ if insufficient information is provided

2Did the study pre-specify whether they were using one or multiple first-rank symptoms (FRSs)?
‘Yes’ if the study states the number of FRSs needed to be present to diagnose schizophrenia
‘No’ if the study does not state the number of FRSs they considered necessary to diagnose schizophrenia

Applicability

Signalling question1Are there concerns that the index test, its conduct or interpretation, differ from the review question?
‘No’ if FRSs are used for diagnosing schizophrenia
‘Yes’ if the study is not using FRSs for the diagnosis of schizophrenia, e.g. the prognosis of patients, or the prevalence of FRSs
‘Unclear’ if insufficient information is provided to judge the purpose of applying FRSs

DOMAIN 3: REFERENCE STANDARD

Risk of bias: Could the reference standard, its conduct or its interpretation have introduced bias?

Signalling question1Is the reference standard likely to correctly classify the target condition?
‘Yes’ if the history and clinical examination is conducted by a qualified professional (psychiatrist, nurse, social worker)
‘No’ if the history and clinical examination is conducted by insufficiently qualified individuals
‘Unclear’ if insufficient information is provided

2Were the reference standard results interpreted without knowledge of the results of the index test?
‘Yes’ if the reference standard was conducted before the index test, or if the person applying the reference standard was blinded to the results of the index test
‘No’ if the reference standard operator knew the results of the index test
‘Unclear’ if insufficient information is provided

Applicability

Signalling question1Are there concerns that the target condition as defined by the reference standard does not match the question?
‘No’ if the paper specifically looks at diagnosing schizophrenia (regardless of subtypes)
‘Yes’ if the paper also includes schizophrenia-like illnesses
‘Unclear’ if insufficient information is provided

DOMAIN 4: FLOW AND TIMING 

Risk of bias: Could the patient flow have introduced bias?

Signalling question1Was there an appropriate interval between index test and reference standard?

'Yes' if reference standard and index text were applied in the same interview or within 4 weeks (applied more than once for chronic schizophrenia)
'No' if reference standard or index test were applied in different interviews beyond 4 weeks
'Unclear' if not enough information is given to assess whether there was an appropriate interval

2Did all patients receive a reference standard?

'Yes’ if all patients had details of history and clinical examination, with or without operational criteria
‘No’ if not all patients had a description of history and clinical examination
‘Unclear’ if insufficient information is provided

3Did all patients receive the same reference standard?
‘Yes’ if all patients were diagnosed with history and clinical examination; and if any operational criteria were used, the same ones were applied to all patients and all received the same clinical follow-up
‘No’ if all patients received history and clinical examination but only some received operational criteria, or different operational criteria
‘Unclear’ if insufficient information is provided

4Were all patients included in the analysis?
'Yes' if there are no patients excluded from the analysis
'No' if there are patients excluded from the analysis
'Unclear' if not enough information is given to assess whether any patients were excluded from the analysis



 

Contributions of authors

  1. Top of page
  2. Background
  3. Objectives
  4. Methods
  5. Acknowledgements
  6. Appendices
  7. Contributions of authors
  8. Declarations of interest
  9. Sources of support

Karla Soares-Weiser - led project and drafted the protocol.

Nicola Maayan - helped draft protocol.

Clare Davenport - helped guide and draft protocol.

Amanda Kirkham - statistical support.

Clive E Adams - helped draft protocol.

 

Declarations of interest

  1. Top of page
  2. Background
  3. Objectives
  4. Methods
  5. Acknowledgements
  6. Appendices
  7. Contributions of authors
  8. Declarations of interest
  9. Sources of support

Karla Soares-Weiser - currently works for Enhance Reviews Ltd, a company that carries out systematic reviews mostly for the public sector, it currently does not provide services for the pharmaceutical industry.

Nicola Maayan - currently works for Enhance Reviews Ltd, a company that carries out systematic reviews mostly for the public sector, it currently do not provide services for the pharmaceutical industry.

Clare Davenport - none known.

Amanda Kirkham - none known.

Clive E Adams - none known.

 

Sources of support

  1. Top of page
  2. Background
  3. Objectives
  4. Methods
  5. Acknowledgements
  6. Appendices
  7. Contributions of authors
  8. Declarations of interest
  9. Sources of support
 

Internal sources

  • University of Nottingham, UK.

 

External sources

  • NIHR Cochrane Programme Grant 2011, UK.
    Reference number: 10/4001/15, UK

References

Additional references

  1. Top of page
  2. Abstract
  3. Background
  4. Objectives
  5. Methods
  6. Acknowledgements
  7. Appendices
  8. Contributions of authors
  9. Declarations of interest
  10. Sources of support
  11. Additional references
Astrachan 1974
  • Astrachan BM, Brauer L, Harrow M, Schwartz C. Symptomatic outcome in schizophrenia. Archives of General Psychiatry 1974;31:155-63.
Bottlender 2003
  • Bottlender R, Sato T, Jager M, Wegener U, Wittmann J, Strauss A, et al. The impact of the duration of untreated psychosis prior to first psychiatric admission on the 15-year outcome in schizophrenia. Schizophrenia Research 2003;62(1-2):37-44. [PUBMED: 12765741]
Bustillo 2001
Carpenter 1973
De Haan 2003
  • De Haan L, Linszen DH, Lenior ME, De Win ED, Gorsira R. Duration of untreated psychosis and outcome of schizophrenia: delay in intensive psychosocial treatment versus delay in treatment with antipsychotic medication. Schizophrenia Bulletin 2003;29(2):341-8. [PUBMED: 14552508]
Deeks 2005
  • Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. Journal of Clinical Epidemiology 2005;58:882-93.
Feighner 1972
  • Feighner JP, Robins E, Guze SB, Woodruff RA Jr, Winokur G, Munoz R. Diagnostic criteria for use in psychiatric research. Archives of General Psychiatry 1972;26(1):57-63.
Kane 2001
  • Kane JM, Marder SR, Schooler NR, Wirshing WC, Umbricht D, Baker RW, et al. Clozapine and haloperidol in moderately refractory schizophrenia: a 6-month randomized and double-blind comparison. Archive of General Psychiatry 2001;58(10):965-72.
Koehler 1978
Koehler 1979
Lawrie 2004
  • Lawrie SM, Johnstone EC. Schizophrenia and related disorders. Companion to Psychiatric Studies. London: Churchill Livingstone, 2004.
Macaskill 2004
  • Macaskill P. Empirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis. Journal of Clinical Epidemiology 2004;57:925-35.
Marshall 2011
McGrath 2008
Rutter 2001
Schneider 1959
  • Schneider K. Clinical Psychopathology. New York: Grune & Stratton, 1959.
Spitzer 1978
  • Spitzer RL, Endicott J, Robins E. Research diagnostic criteria: rationale and reliability. Archives of General Psychiatry 1978;35(6):773-82.
Takwoingi 2010
  • Takwoingi Y, Deeks JJ. METADAS: A SAS macro for meta-analysis of diagnostic accuracy studies. User Guide Version 1.3. 2010 July. http://srdta.cochrane.org (accessed 7 June 2012).
Tandon 2013
  • Tandon R. Schizophrenia and other psychotic disorders in DSM-5. Clinical Schizophrenia & Related Psychoses 2013;7(1):16-9. [DOI: 10.3371/CSRP.TA.032513]
Taylor 1978
Whiting 2011
  • Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Annals of Internal Medicine 2011;155(8):529-36.