Cardiometabolic risk prediction algorithms for young people with psychosis: a systematic review and exploratory analysis

Cardiometabolic risk prediction algorithms are common in clinical practice. Young people with psychosis are at high risk for developing cardiometabolic disorders. We aimed to examine whether existing cardiometabolic risk prediction algorithms are suitable for young people with psychosis.


Introduction
Cardiometabolic disorders broadly include cardiovascular diseases (CVD), disorders of adiposity such as obesity and disorders of glucose-insulin homeostasis such as type 2 diabetes mellitus (T2DM) (1). They impose a huge societal burden costing an estimated £30 billion and accounting for over 190 000 deaths each year in the UK alone (2). A particularly high-risk group for the development of cardiometabolic disorders are people with psychotic disorders such as schizophrenia, who make up around 0.8% of the population (3) and have up to a 30% increased incidence of cardiometabolic disorders than the general population (4). Indeed, increased physical comorbidity is a leading cause for significantly increased mortality rates and reduced life expectancy for people with schizophrenia compared with the general population (5)(6)(7). We therefore need clinical tools to predict cardiometabolic risk in this group in order to optimize care and improve long-term outcomes. Yet, a recent report of a small sample of people with chronic schizophrenia suggests that some commonly used cardiometabolic risk prediction algorithms return differing risk prediction scores when tested on the same participants. This calls into question the reliability and suitability of such algorithms for relatively older people with chronic schizophrenia, let alone young people with or at risk of psychosis (8).
Recent evidence suggests that the physical comorbidity associated with schizophrenia starts early. Markers of developing cardiometabolic disorders are a feature that distinguish cases of first-episode psychosis from matched general population controls (9,10) and are associated with young adults at risk of developing psychosis (11). The field of early intervention in psychosis rests on a premise that intervening early could improve longer-term outcomes, and this premise applies equally to the treatment of cardiometabolic disorders. Therefore, cardiometabolic risk prediction algorithms may be a useful tool for healthcare professionals to help tailor treatment plans for young people with psychosis that could help to reduce both long-term physical and psychiatric morbidity. However, such a tool could only be clinically useful if the predictions it makes are accurate. It is unclear as to whether this may or may not be the case.

Aims of the study
We conducted a systematic review to identify and compare existing cardiometabolic risk prediction algorithms developed for the general or psychiatric populations and consider their suitability for young people with psychosis. Next, we performed an exploratory analysis using data from a large UK birth cohort to examine the predictive ability of any algorithms highlighted as potentially suitable by the review, in a sample of young adults with or at risk of psychosis. To explore the impact of age on risk estimates, we reassessed model performance after artificially increasing the age of participants to the mean age of the original algorithm development study, leaving all other predictors unchanged.
We applied the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-analyses) guidelines (13). The systematic review was registered on PROSPERO (CRD42019150377).
Study selection. The inclusion criteria were as follows: (i) studies reporting the development and/or validation of cardiometabolic risk algorithms designed for either the general or psychiatric populations; (ii) studies which reported in combination the development and validation (internal or external) of an original algorithm; reported the development but not validation of an algorithm; reported the first validation of a previously developed but not validated algorithm; or reported a new recalibration of a previously developed algorithm; (iii) cardiometabolic risk was defined as CVD (stroke, myocardial infarction, hypertension, unstable angina) and its common predeterminants including T2DM, prediabetes, obesity or dyslipidaemia; (iv) studies reported in any language; (v) published and unpublished research, conference proceedings and academic theses. The exclusion criteria were as follows: (i) algorithms designed specifically for other defined health groups (i.e. postoperative patients or patients with any physical health diagnoses at baseline) and (ii) studies reporting validation without recalibration of previously validated algorithms.
Titles and abstracts were screened independently by four authors (BIP, EL, IM and EC) prior to full-text screening. Any discrepancies were resolved in consultation with a senior author (GMK). Data were extracted by three authors (BIP, OC and SJ) from studies that met the inclusion criteria. Searches were re-run immediately prior to the final analyses, and further studies retrieved for inclusion using the processes outlined above.
Data extraction and synthesis. We extracted data on general characteristics (e.g. population, location, study type, type of risk predicted), the characteristics of included participants (e.g. age, sex, ethnicity) and characteristics of the developed/validated algorithms (e.g. included predictors, algorithm performance). Risk of bias was assessed using the 'Prediction model Risk Of Bias Assessment Tool' (PROBAST) (14), which aims to identify shortcomings in study design, conduct or analysis that could lead to systematically distorted estimates of model predictive performance. PRO-BAST includes four domains for potential sources of bias in prediction model studies (participants, predictors, outcome and analysis) which are then summarized by an overall judgement, either low risk, high risk or unclear risk of bias (14). We plotted the range and frequency of predictors included in studies. We illustrated the relative weighting of different predictors in one included study which featured psychiatric predictors. Algorithm performance was compared using statistics relating to model discrimination (how well an algorithm discriminates people at higher risk from people at lower risk, e.g. Harrell's C Statistic, where a score of 1.0 indicates perfect discrimination, and a score of 0.5 indicates the model is no better than chance) and model calibration (the accuracy of absolute risk estimates, e.g. calibration plots) (15). We also examined the events-per-variable ratio (EPV) (the ratio of outcome events: predictors considered in algorithm development) of each study to assess the potential risk of model overfitting (16). An EPV of 10 or more had previously been considered satisfactory (17), though higher ratios have more recently been advised (18). Where an EPV ratio was not reported, we calculated it where possible from the information available in the study. Finally, we considered the likely suitability of all included algorithms for young people with psychosis. We summarized and compared studies with a narrative synthesis (19).

Exploratory analysis
Data source. The Avon Longitudinal Study of Parents And Children (ALSPAC) birth cohort initially recruited 14 541 pregnant women resident in a geographically defined region in southwest of England, with expected dates of delivery 1 April 1991 to 31 December 1992, resulting in 14 062 live births (20)(21)(22). Following further periods of recruitment over time, 913 additional participants were recruited. See http://www.bris.ac.uk/alspac/ researchers/data-access/data-dictionary/ for a fully searchable data dictionary. Study data were collected and managed using REDCap electronic data capture tools hosted at University of Bristol (23,24). Ethical approval for the study was obtained from the ALSPAC Ethics and Law Committee and Local Research Ethics Committees. All participants provided informed consent.
Study sample. We included participants who at age 18 or 24 years were identified as experiencing definite psychotic symptoms or psychotic disorder. In ALSPAC, psychotic symptoms were identified through the face-to-face, semi-structured Psychosis-Like Symptom Interview (PLIKS) conducted by trained psychology graduates and coded according to the definitions in the Schedules for Clinical Assessment in Neuropsychiatry, version 2.0. See Supplementary Data for further information. We excluded participants who already met the outcome criteria at age 18 years and participants who had missing data on all included variables. Additionally, we conducted a post hoc sensitivity analysis to examine the potential impact of sample size; we performed the analysis again including all participants from the total ALSPAC sample at age 18 years who did not meet the criteria for the outcome at age 18 years and who did not have missing data on all included variables. See Figures S1-S2 for flow charts of included participants.
Outcome. We used the harmonized definition (25) of the metabolic syndrome measured at age 24y as the outcome, in which it is an established precursor of T2DM (26) and CVD (27). Metabolic syndrome is a more appropriate outcome for a sample of relatively young participants. The follow-up period was six years. The binary outcome was coded present for participants meeting ≥3 factors from the following: ethnicity-specific waist circumference (≥94 cm in males and ≥80 cm in females for Caucasians; ≥90 cm in males and ≥80 cm in females for other ethnic groups (25)); elevated triglycerides (≥1.7 mmol/L); reduced high-density lipoprotein (HDL (<1.0 mmol/L in males and <1.3 mmol/L in females); elevated seated blood pressure (systolic ≥ 130 mmHg); and elevated fasting plasma glucose (FPG) (≥5.7 mmol/L). See Supplementary Methods for further detail on biochemical measurements.
Predictors. We included all available predictors from QRISK3, QDiabetes and PRIMROSE, which were the three algorithms highlighted as being potentially the most suitable for young people with psychosis. These included age, Townsend deprivation score, body mass index (BMI), ethnicity, smoking, antipsychotic medication use, antidepressant use, corticosteroid use, psychosis, depression, family history of cardiovascular disease or type 2 diabetes, hypertension, FPG, cholesterol:HDL ratio, systolic blood pressure, total cholesterol, HDL, alcohol intake and year of assessment. For a full list of predictors for each algorithm and details on how they were measured, see Table S1 and Methods S1.
Statistical analysis. Estimated six-year risk estimates for metabolic syndrome were calculated for QDiabetes (28), QRISK3 (29) and PRIMROSE (30), by applying the published fully specified algorithms to our sample. QDiabetes and PRIMROSE comprise different models depending on the availability of blood test results; thus, we used the model which performed best in the original model development studies (28,30). For QDiabetes, the best performing model included FPG; for PRIM-ROSE, the best performing model included lipids. QDiabetes and QRISK3 estimate risk separately for males and females. We used multiple imputation using chained equations (31) to address the impact of missing predictor data. See Methods S1 for further details. Algorithm performance was assessed using measures of discrimination (Harrell's C statistic and R 2 ) and a measure of calibration (calibration plots). Calibration plots included grouped observations, which were split at each 0.2 of predicted risk. First, we calculated model performance using actual participant age (18 years). To assess the impact of age on model performance, we artificially substituted every participants' age in ALSPAC to the mean age from the original algorithm development study (QDiabetes = 44.9 years; QRISK3 = 42.9 years; and PRIMROSE = 49.5 years), leaving all other predictors unchanged. We re-ran each algorithm and compared the model performance statistics described above. Statistical analysis was carried out in R version 3.6.0 (32).

Systematic review
Study selection and quality assessment. The literature search returned 7744 results after removing duplicates. We reviewed 362 full texts, of which 110 studies met inclusion criteria (28)(29)(30). See Fig. 1 for the PRISMA diagram. Three studies were not contained within peer-reviewed journals and were published either as conference proceedings (108), a thesis (93) or a preprint (106). Reporting quality was relatively poor across the majority of studies, with 108 studies (98%) either at unclear or high risk of bias following assessment with the PROBAST tool (14). See Table S2.
Study characteristics. Table S3 reports the characteristics of included studies. All studies were conducted on general population samples of healthy adults, except one which was conducted on patients with severe mental illness, defined as either schizophrenia, other psychotic disorder or bipolar disorder (30). The majority of included studies were conducted in high-income or upper-middleincome countries, with the UK, USA and China best represented. Eleven studies were conducted in lower-or middle-income countries. Sample sizes were highly variable in both development (n = 100 participants (120) to n = 8 136 705 participants (28)) and validation cohorts (n = 90 participants (104) to n = 2 671 298 participants (29)). Sixtyone studies (55%) assessed the risk of fatal or nonfatal CVD; 31 studies (28%) assessed the risk of T2DM; five studies (5%) assessed the risk of either prediabetes or T2DM; three studies (3%) assessed the risk of metabolic syndrome or obesity; and three studies (3%) assessed the risk of stroke or transient ischaemic attack.
Lengths of predicted risks ranged from one (119) to 30 (80,123) years. The most common risk prediction timeframes were either ten-year risk (38 studies, 35%) or five-year risk (14 studies, 13%). Thirty-nine studies (35%) performed external validation of an original algorithm. Forty studies (36%) performed internal validation, either by subsetting the initial cohort or by bootstrap methods. All algorithms were designed using either Cox proportional hazards or logistic regression analysis. Most studies selected variables for inclusion from previous research or clinical importance (50 studies, 45%), or using statistical methods, that is forward or backward selection (31 studies, 28%). Seventeen studies (15%) used simple univariable analysis of each considered predictor, which is least preferable since it cannot assess interactions between two or more variables. Eleven studies (10%) used machine learning techniques.
Participant characteristics. All studies were conducted in adults. The mean age of participants based on the 76 studies that reported mean age was 50.50 years (SD 9.31). No studies included a mean age of participants below 35 years. Eightynine studies (81%) reported the sex distribution of The majority of studies included roughly equal sex distribution, apart from nine studies which included only (121,127) or mostly females (82,83,85,98,120,122,128) and 12 studies which included only (41,71,94,102,103,112,119,132,136) or mostly males (69,80,81). Thirty-three studies (30%) reported the ethnic makeup of their sample, where samples ranged from being ethnically completely homogenous in 18 studies (16%) to relatively heterogeneous, with less than 66% of participants falling into the most common ethnic group (63,72,84,125). See Table S3.
Potential applicability of existing cardiometabolic risk algorithms for young people with psychosis. Psychiatric illness and treatment were taken into account in three studies (28)(29)(30) predicting risk of CVD (29,30) or T2DM (28). Two of these studies (QRISK3 and QDiabetes (28,29)) were conducted on large general population samples, and one (PRIM-ROSE) was conducted in people with severe mental illness (30). QRISK3 and QDiabetes (28,29) included diagnosis of severe mental illness as a single predictor, whereas PRIMROSE included separate predictors for bipolar disorder and psychosis (30). QRISK3 and QDiabetes included the presence of any atypical antipsychotic as a predictor (28,29); PRIMOSE included first-or second-generation antipsychotics as separate predictors, along with antidepressants as another predictor (30). All three studies were conducted on middle-aged adults (mean ages QDiabetes: 42.9 years (28), QRISK3: 44.9 years (29), PRIMROSE: 49.5 years (30)). In PRIMROSE, age was applied as a nonlinear term with a log transformation and was weighted heavily in comparison to other risk factors. See Figure S3. In both QRISK3 and QDiabetes, age was applied as a fractional polynomial, also implying a non-linear impact on risk. QRISK3 and QDiabetes both included a number of interactions between age and other predictors, further amplifying the relative importance of age in the algorithms. QRISK3, QDiabetes and PRIMROSE were taken forward for the exploratory analysis, on the basis of the following: large samples used in development and validation; strong performance statistics; low risk of bias in three domains; and inclusion of psychiatric predictors/development in a psychiatric sample.

Exploratory analysis
Baseline characteristics. The six-year observed risk of metabolic syndrome at age 24 years in our sample of participants with or at risk of psychosis was 14.21% in females and 11.88% in males. In our sensitivity analysis (all available ALSPAC participants), the six-year observed risk was 7.54% for females and 5.76% for males. In our primary analysis, we included 3030 person-years of observation.
In our sensitivity analysis, we included 19 020 person-years of observation. Characteristics of included participants for both the primary and sensitivity analyses are presented in Table 1 and  Table S6 respectively. Associations between algorithm predictors and outcome are reported in Table S7. After substituting participant ages to the mean age of the original studies, Harrell's C statistics mildly improved for each algorithm. Similarly, at age 18 years, R 2 statistics were marginally higher in females than males in QDiabetes and QRISK3 and improved mildly after substituting participant ages to the mean age of the original studies. See Table 2.
Calibration. At age 18 years, calibration was poor across all three algorithms, with observed risk estimates consistently higher than predicted risk estimates. After substituting participant ages to the mean age of the original studies, calibration improved markedly in all three algorithms. See Figure 3.
Sensitivity analysiswhole ALSPAC sample. Discrimination. QDiabetes and QRISK3 performed better in the overall sample than the psychosis sample. PRIMROSE performed better in the psychosis sample. Harrell's C Statistics were as  Similarly, at age 18 years, R 2 statistics were marginally higher in females than males in QDiabetes, but marginally higher in males in QRISK3. After substituting age to the mean age of the original studies, Harrell's C statistics and R 2 improved in all three algorithms. See Table S8.
Calibration. In a similar pattern to the psychosis sample, at age 18 years, calibration was poor across all three algorithms, with observed risk estimates consistently higher than predicted risk estimates. After substituting participant ages to the mean age of the original studies, calibration improved markedly in all three algorithms. See Figure S4.

Main findings
We performed a systematic review of cardiometabolic risk prediction algorithms developed either for the general or psychiatric populations and considered their potential suitability for young people with psychosis. We also used data from a sample of relatively young adults to first explore whether existing cardiometabolic risk prediction algorithms may be suitable for young people with or at risk of psychosis and second to explore the impact of the manner in which age is weighted in existing cardiometabolic risk prediction algorithms. We do not present the results of our exploratory analysis as an external validation of the three algorithms, since the algorithms we tested were not developed to predict metabolic syndrome. Rather, we present our findings as a means to explore the likely suitability of these algorithms for a population of individuals who may be at higher cardiometabolic risk compared with the general population. It should be made clear from the outset that the three algorithms we tested, as we show in the results of our systematic review, were developed and validated on large samples and perform well in the populations they were designed for.

Systematic review
We identified a substantial number of cardiometabolic risk prediction algorithms, yet most have not been integrated into clinical practice. Predicted outcomes ranged from prediabetes and T2DM, CVD or transient ischaemic attack and stroke. The five most commonly included predictors across all algorithms were age, smoking, systolic blood pressure, sex and BMI. One included algorithm (PRIMROSE) was developed in a population of people with severe mental illness (30), which predicted risk of CVD. Two (QRISK3 and QDiabetes) were developed in the general population and included psychiatric predictors (28,29) such as a diagnosis of schizophrenia. All included algorithms were developed in samples of middle-to older-age adults. One might traditionally consider this proportionate, since cardiometabolic disorders are traditionally considered diseases of advancing age. Yet, cardiometabolic risk still exists in the absence of advancing age; even in the general population, there is an increasing prevalence of early-onset T2DM (140) and childhood obesity (141), likely related to the shift towards a more sedentary lifestyle and unhealthy diet in recent decades. The absence of an algorithm developed for younger populations is an important finding, since early intervention may reduce the risk of young people forming part of a future generation of patients with chronic cardiovascular diseases (142). This finding suggests the need for either new or recalibrated versions of currently existing cardiometabolic risk algorithms tailored to the younger generations.  Primary prevention is the best means with which to address the personal and societal burden attributed to T2DM, CVD and its complications (143).
Whilst this message is important for the general population, it is particularly important for young people with/at risk of psychosis, who are at a higher risk of precipitant cardiometabolic disorders. This population may be more likely to smoke (144), exercise less (145) and eat a more unhealthy diet (145) than their peers and yet may also be prescribed medication that in itself can adversely and severely impact cardiometabolic indices (146). Further, they may be faced with inappropriate barriers to accessing healthcare (147), diagnostic overshadowing (148) and may have an intrinsic biological propensity for altered cardiometabolic function (149). Meta-analyses featuring mostly antipsychotic-na€ ıve young people with first-episode psychosis have consistently reported an increased incidence of insulin resistance, impaired glucose tolerance (9, 10) and dyslipidaemia (9,150,151) compared with matched controls from the general population, after adjusting for anthropometric and sociodemographic factors. Each is predeterminants of cardiometabolic disorders such as T2DM and obesity. These factors may not be adequately captured by currently existing algorithms. Additionally, meta-analyses of cross-sectional studies suggest that psychosis is associated with higher levels of circulating inflammatory markers (152)(153)(154)(155), and evidence from some longitudinal studies suggests an association between inflammatory markers at baseline and psychosis at follow-up (156)(157)(158), although other longitudinal studies have reported negative findings (159). Inflammatory states are also associated with cardiometabolic disorders (160)(161)(162)(163). Whilst 15 relatively newer algorithms from our systematic review did include inflammatory predictors, none also included psychiatric predictors. Each of the three algorithms that did include psychiatric factors featured an antipsychotic-related predictor. Antipsychotic-associated weight gain can occur relatively quickly after initiation (164) and is associated with altered eating behaviours (165) and sedentariness (166). However, whilst there are some efficacy differences between antipsychotics, these are gradual rather than discrete (167). Differences in side-effects are more marked, and each has an inherently different impact upon cardiometabolic risk (168). This may be explained by differing affinities to receptors other than the dopamine-2 (D2) receptor, for example the histamine-1 (H1) receptor, serotonin-2c (5-HT2c) and adrenergic receptors (a2 and b3) (169), which may have a role in the regulation of food intake (170). The varied impact upon cardiometabolic risk by different antipsychotics does not abide by the traditional distinctions of either typical/atypical or first/second generation, which were the binary distinctions of the included algorithms. A more appropriate antipsychotic predictor may instead model antipsychotics based on their relative cardiometabolic risk.
We used the PROBAST tool (14) to examine the risk of bias of included studies in our systematic review. Only two studies were rated as low risk of bias, with all others rated as either unclear or high risk of bias. This may be a reflection of the relatively recent introduction of the 'Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis' (TRIPOD) guidelines for prediction model studies (139). Nevertheless, the results suggest that the reported performance statistics and therefore clinical validity of the majority of included studies should be accepted with extreme caution.
The EPV ratio also varied widely between studies. A low EPV ratio can be an indicator of model overfit (17) which can bias results. We identified 20 studies with an EPV ratio of likely < 10, and therefore, the performance reported in those studies should be interpreted with caution. Finally, it is striking that whilst many included studies promoted the use of their algorithms in clinical practice, there appears to have been relatively little follow-up to assess either clinical or economic impact. A notable exception was PRIMROSE (30), which was the only algorithm developed and validated on a sample of people with mental illness. A cost-effectiveness analysis (171) found it improved quality of life and reduced healthcare-related costs in comparison with using no algorithm.
A previously published systematic review (172) examining cardiovascular risk prediction algorithms in the general population also identified a very large number of studies. The review similarly concluded the methodological shortcomings of most risk prediction algorithms likely limit their suitability for clinical practice. The previous review differs from our own since we were interested in identifying original or recalibrated algorithms and assessing their suitability for young people with psychosis. Therefore, we did not include studies reporting new validations in a similar population to already validated algorithms. The previous review also presented sexstratified algorithms as distinct entities, increasing the apparent number of algorithms they reported. For ease of simplicity and in consideration of our overarching research question, we did not take this step. Finally, a large number of new algorithms have been developed since the previous review, which we were able to include in our own.

Exploratory analysis
We considered three algorithms for exploratory analysis: QRISK3, QDiabetes and PRIMROSE. These were selected due to the large sample sizes in model development and validation, model performance statistics, relatively low risk of bias and the inclusion of psychiatric predictors/development in a psychiatric population.
We found that discrimination statistics were relatively good at age 18 years for QDiabetes and PRIMROSE and improved further when substituting to the mean age of original studies. This means that QDiabetes and PRIMROSE were able to predict higher risks in 'cases' than 'non-cases', even in relatively young adults. This did not apply to QRISK3, particularly in males, where the algorithm was little better than chance at discriminating higher and lower cardiometabolic risk in young adults with or at risk of psychosis.
For all three algorithms, however, the discriminative ability in our sample was attenuated compared with the original published studies (28)(29)(30). This may be because our sample included younger participants than the original studies. For example, both QRISK3 and QDiabetes were developed and validated in participants aged 25 and over, and PRIMROSE was developed and validated in participants aged 30 and over. QRISK3 and QDiabetes set a minimum age of 25 when using their online calculators, although PRIMROSE sets a minimum of age 18 years. Additionally, in our primary analysis, we tested a sample of participants with or at risk of psychosis, whereas QDiabetes and QRISK3 were designed for use in the general population. Furthermore, we tested a different outcome compared with the original algorithms. We tested metabolic syndrome since it is an established precursor of both T2DM and CVD (26,27) and is a more suitable outcome for younger populations. The improvement in discrimination statistics after substituting age provides some face validity to our choice of outcome.
However, discriminative ability is only half the story, since discrimination statistics cannot assess the accuracy of the amount of risk apportioned by a model; this represents a test of absolute risk estimates and is examined with a measure of calibration. Our calibration plots at 18 years showed that observed risk was systematically greater than predicted risk in all models, suggesting a notable underprediction of risk in younger participants. Calibration plots improved markedly in all algorithms when we artificially substituted age to the mean age of the original studies. This suggests that the manner with which age is modelled in current algorithms is a major limiting factor in applying them to younger populations. This is likely because many cardiometabolic risk factors are cumulative over time (173); thus, age becomes increasingly important with regard to cardiometabolic risk as one gets older. This notion is elegantly painted by all three algorithms, which modelled age as either a non-linear function, included interactions between age and other predictors, or both.

Strengths and limitations
Strengths of this systematic review include following PRISMA reporting guidelines (13), as would be expected for a high-quality review. Alongside the review, we were able to complement our findings with an exploratory analysis using data from a large birth cohort of young adults. We were able to test three validated cardiometabolic risk prediction algorithms which are commonly used in clinical medicine in the UK, on a different population who are in clear and crucial need of a suitable tool.
Limitations of the study first and foremost relate to the exploratory analysis. The three algorithms we tested were not designed for use in young adults, though this in itself should not be a barrier to explore potential suitability in a different population. Nevertheless, our results should not be seen to cast doubt on the predictive ability of such algorithms when applied to the populations intended by the authors. We were unable to include every predictor from the algorithms we tested, which may have impacted upon performance statistics. That said, the impact of this limitation on our results may not have been uniform for each predictor we could not include. For example, even if we had the data, it is unlikely that many participants in our relatively young cohort would have diagnosed CVD or chronic kidney disease, a history of gestational diabetes or be prescribed statins. Also, our measured outcome differed from the outcome of the algorithms we tested. Whilst three algorithms included in the systematic review did aim to predict risk of metabolic syndrome, we did not consider them for our exploratory analysis since they did not include psychiatric predictors, were at relatively high risk of bias, and study authors did not publish their fully specified algorithm equations. Nevertheless, metabolic syndrome is a precursor of T2DM (26) and CVD (27), and the relatively good performance of the algorithm when we artificially substituted age to the mean age of the original study suggests face validity to our chosen outcome. Our sample size was relatively small compared with the original studies. However, by testing a more encompassing outcome, we were able to include a greater number of cases and reduce the impact of model overfit.
Other limitations relate to the systematic review. We were unable to follow a meta-analytic approach to the synthesis of results due to study heterogeneity. The lack of meta-analytic approach meant we were unable to examine the risk of publication bias, which may have played a part in the configuration of studies we included in our synthesis, since only three included studies were not published in peer-reviewed journals.
In conclusion, young people who are at higher risk than the general population of developing psychosis are also at higher risk of developing cardiometabolic disorders. A suitable cardiometabolic risk prediction algorithm for this population would be highly beneficial to general and psychiatric practitioners to help them to tailor treatment plans with the aim of reducing long-term physical and psychiatric morbidity. Existing cardiometabolic risk algorithms cannot be recommended for this purpose since they likely underestimate the cardiometabolic risk of all young people, let alone a group already at significantly higher risk than the general population. Existing algorithms require recalibration to suit younger populations, and, better still, a new cardiometabolic risk prediction algorithm is required which is specifically developed for young people with psychosis. A well-designed algorithm may include a more appropriate distinction of metabolically active antipsychotics; should more appropriately weight the predictors for the specific characteristics of young people with psychosis; and may include a more age-appropriate outcome, such as metabolic syndrome. Further, particular attention should be paid to patient acceptability, to ensure the algorithm is actually used in clinical practice rather than simply buried in a research database. In lieu of a suitable algorithm, simple lifestyle interventions such as smoking cessation, encouraging a healthy diet and increasing physical activity must be offered to all young people with or at risk of psychosis. Indeed, encouraging results are emerging from studies of primary prevention in this population (174,175), who may not have yet developed chronic and pervasive lifestyle behaviours which are associated with chronic illness. Figure S2. Flow-diagram of included participants in sensitivity analysis of all participants at age 18 years. Figure S3. Relative weighting of age vs other predictors in PRIMROSE(6). Figure S4. Calibration plots of algorithms tested in ALSPAC at age 18 years and at mean age of original study (whole sample). Table S1. Predictors included in QDiabetes, QRISK3 and PRIMROSE. Table S2. Risk of bias assessment using PROBAST. Table S3. Participant characteristics of studies included in systematic review. Table S4. Algorithm characteristics of studies included in systematic review. Table S5. Algorithm performance of studies included in systematic review. Table S6. Characteristics of ALSPAC participants included in exploratory analysis (whole sample). Table S7. Odds ratio and 95% CI for the association between predictors included in algorithms measured at 18 years and metabolic syndrome at 24 years in the ALSPAC Cohort. Table S8. Discrimination statistics for algorithms tested on whole sample at age 18 years and mean age of original study.