Assessment of metacognitive beliefs in an at risk mental state for psychosis: A validation study of the Metacognitions Questionnaire‐30

Aim The Metacognitions Questionnaire‐30 (MCQ‐30) has been used to assess metacognitive beliefs in a range of mental health problems. The aim of this study is to assess the validity of the MCQ‐30 in people at risk for psychosis. Methods One hundred eighty‐five participants meeting criteria for an at risk mental state completed the MCQ‐30 as part of their involvement in a randomized controlled trial. Confirmatory and exploratory factor analyses were conducted to assess factor structure and construct validity. Results Confirmatory factor analyses confirmed the original five‐factor structure of the MCQ‐30. Examination of principal component analysis and parallel analysis outputs also suggested a five‐factor structure. Correlation analyses including measures of depression, social anxiety, and beliefs about paranoia showed evidence of convergent validity. Discriminant validity was supported using the normalizing subscale of the beliefs about paranoia tool. Conclusions The MCQ‐30 demonstrated good fit using the original five‐factor model, acceptable to very good internal consistency of items was evident and clinical usefulness in those at risk for psychosis was demonstrated.

The CAS is linked to underlying metacognitive knowledge (beliefs) that compromise flexible control of the syndrome (Wells, 2009 Wells & Cartwright-Hatton, 2004). In a recent meta-analysis of MCQ studies, metacognitive beliefs were confirmed as transdiagnostic factors across psychopathologies (Sun, Zhu, & So, 2017). Robust and reliable positive associations have been demonstrated between MCQ metacognition domains and symptoms of anxiety (Wells, 2005) and mood disorders (Papageorgiou & Wells, 2003), and more recently, this has been extended to psychotic symptoms (Morrison, French, & Wells, 2007;Sellers, Gawęda, Wells, & Morrison, 2016). However, although the psychometric properties and the construct validity of the MCQ is reasonably well established in nonpatients and those with emotional disorder, relatively little is known about its properties in patients with psychosis or at risk of psychosis, and further advances in this area depend on the interpretability of the measure in psychosis groups. Although the subscales of the MCQ relate to positive and negative beliefs about worry and this measure should be relevant to those experiencing psychosis or who are at risk for psychosis in keeping with the S-REF model, the latent structure should be confirmed. Paranoia, for example, can be conceptualized as a type of worry with those experiencing such thoughts having positive beliefs about the benefits of worrying (e.g., to protect oneself from harm). Engaging in such thoughts could lead to unhelpful ways of coping (e.g., avoidance of social situations to protect the self).
An important step in testing the metacognitive model and treatment applied to psychosis is to determine the properties of the MCQ as an appropriate tool that can be interpreted in the usual way.
The MCQ (Cartwright-Hatton & Wells, 1997) was originally constructed as a 65-item measure of metacognitive beliefs and monitoring. The MCQ was developed using data obtained from individuals with generalized anxiety disorder, obsessive compulsive disorder, hypochondriasis, and panic disorder (Cartwright-Hatton & Wells, 1997). The MCQ has five subscales: positive beliefs about worry (e.g., "Worrying helps me avoid problems in the future."); negative beliefs about uncontrollability and danger of worry (e.g., "My worrying is dangerous for me."); negative beliefs about thoughts in general including items relating to superstition, punishment, and responsibility (e.g., "If a bad thing happens which I have not worried about, I feel responsible."); cognitive self-consciousness (e.g., "I am constantly aware of my thinking."); and cognitive confidence (e.g., "I have a poor memory.").
The original MCQ had limited use due to its length and some items were found to be unclear to participants (Wells & Cartwright-Hatton, 2004). The Metacognitions Questionnaire-30 (MCQ-30;Wells & Cartwright-Hatton, 2004), a shortened 30-item version of the MCQ, was developed as a result. The number of items was reduced by removing any items that participants questioned on the original MCQ and by keeping the highest loading items for each subscale (Wells & Cartwright-Hatton, 2004 (Terwee et al., 2007). The improved internal consistency combined with the more efficient length makes the MCQ-30 the measure of choice for metacognitive beliefs.
At risk mental state (ARMS) refers to people who are at risk for psychosis. Enhanced interest in the presence of metacognitive beliefs in the early stages of psychosis has led to increased use of the MCQ-30 in research (Cotter, Yung, Carney, & Drake, 2017;Morrison et al., 2014;Palmier-Claus, Dunn, Taylor, Morrison, & Lewis, 2013;Welsh, Cartwright-Hatton, Wells, Snow, & Tiffin, 2014). A systematic review and meta-analysis of metacognitive beliefs in those with an ARMS (Cotter et al., 2017) found those at risk of psychosis have significantly higher scores compared to healthy controls on all metacognitive belief domains. No significant differences were found between ARMS and those experiencing psychosis on any of the metacognitive belief subscales.
Research in this area is helpful in building a picture of the presence of metacognitive beliefs in ARMS and established psychosis, which could help clinicians and researchers work out improved targets for intervention.
It is important, therefore, that the tool used to measure metacognitive beliefs is appropriate for the population in which it is being used.
Although the MCQ-30 has been validated in non-clinical (Spada, Mohiyeddini, & Wells, 2008;Wells & Cartwright-Hatton, 2004), obsessive-compulsive disorder (Grøtte et al., 2016), and physical health (Cook, Salmon, Dunn, & Fisher, 2014;Fisher, Cook, & Noble, 2016) populations, no studies have validated the MCQ-30 in those at risk for psychosis. With the increased use of this measure with those at risk for psychosis, it is important to explore the validity the MCQ-30 in this population. This study aims to do so by examining the construct validity via the factor structure of the MCQ-30 and its internal consistency in those with an ARMS.
To examine convergent and discriminant validity, correlations with related measures are useful (DeVon et al., 2007). Although the MCQ measures beliefs about repetitive negative thinking in the form of worry, other forms of similar thinking have been identified in psychosis, such as paranoid ideation, that would be conceptualized as a

Key Practitioner Message
• Multicomponent five-factor structure of MCQ-30 was confirmed in an ARMS sample.
• Principal components analysis and parallel analysis suggested retaining a five-factor solution.

| Participants
Data from 185 participants meeting criteria for an ARMS were used to conduct this study. Participants were taking part in either the Early Detection and Intervention Evaluation (EDIE; Morrison et al., 2004) or the Early Detection and Intervention Evaluation 2 (EDIE-2) trial (Morrison et al., 2012). Both were randomized controlled trials testing the efficacy of cognitive therapy in preventing transition to psychosis.
Participants who took part in the EDIE-2 trial were a completely separate sample to those who took part in EDIE (i.e., checks at entry to the EDIE-2 trial ensured that there was no chance any EDIE participants also took part in EDIE-2). Participants were recruited from primary (e.g., psychological services and general practitioners) and secondary (e.g., early intervention for psychosis or community mental health teams) care National Health Service (NHS: publicly funded healthcare providers in the UK) services, as well as other non-NHS services such as university counselling services or voluntary agencies. Research assistants trained in administering all measures collected data for both studies. Thirty-two participants were drawn from the EDIE trial and 153 from EDIE-2. The male to female ratio was 112:73.
Ethical approval for EDIE and EDIE-2 was sought from UK-based ethical committees. Please refer to the full texts for EDIE and EDIE-2 for more information (Morrison et al., 2004;Morrison et al., 2012). All participants voluntarily consented to take part in the studies and for anonymous data to be collected and used in publications. Research procedures in both trials were in accordance with the Declaration of Helsinki and Good Clinical Practice guidelines.

| Measures and procedures
The current study utilized data from two separate samples (EDIE and EDIE-2). In EDIE-2, the MCQ-30 (Wells & Cartwright-Hatton, 2004) was used, whereas in EDIE, the 65-item MCQ was used (Cartwright-Hatton & Wells, 1997 Respondents on these measures are required to select a number ranging from 1 Do not agree to 4 Agree very much for each of the items in the measure. A score is calculated for each of the five subscales as well as a total score for the whole measure. Internal consistency (as measured by Cronbach's alpha) for the original five-factor structure in the current study was as follows: cognitive confidence α = 0.88, positive beliefs about worry α = 0.85, cognitive self-consciousness α = 0.82, negative beliefs about uncontrollability and danger α = 0.83, negative beliefs about the need to control thoughts α = 0.75, and for the full measure α = 0.90. These results demonstrate that the internal consistency for the original five-factors in an ARMS sample was acceptable to high.
The Comprehensive Assessment of At Risk Mental States (CAARMS; Yung et al., 2005) was administered in the EDIE-2 trial to assess for at risk for psychosis status. Four of the six subscales in the CAARMS are used to determine ARMS status: unusual thought content (e.g., thought insertion, feeling controlled by something other than self); nonbizarre ideas (e.g., paranoid thoughts, feeling that one does not exist or is dead); perceptual abnormalities (e.g., visual, auditory, or sensory hallucinations); and disorganized speech (e.g., trouble finding the right word, tangential speech). In the version of the CAARMS used in EDIE-2 (Yung et al., 2005), a Global Assessment of Functioning score was also calculated as problems with functioning was part of the criteria for ARMS. This version of the CAARMS was found to have very good validity and reliability (Yung et al., 2005). Further, interrater reliability checks were conducted eight times throughout the EDIE-2 trial with good reliability found between raters (intraclass correlation coefficient = 0.90, SD = 0.03; Morrison et al., 2012).
The Positive and Negative Symptoms Scale (PANSS; Kay, Fiszbein, & Opler, 1987) was used in the EDIE trial to assess for at risk for psychosis status. Scores on the hallucinations, delusions, suspiciousness, and conceptual disorganisation subscales of the PANSS were used to determine if participants met criteria for an ARMS. This measure has been found to be reliable and valid (Kay et al., 1987).
The BAPS is an 18-item self-report assessment tool used in the EDIE-2 trial to assess metacognitive beliefs about paranoia. The initial version of the BAPS had four subscales (Morrison et al., 2005). The revised three subscale version of this measure (Gumley et al., 2011) was used with participants included in this study. Internal consistency for the current sample for each subscale were negative beliefs about paranoia α = 0.88, survival beliefs about paranoia α = 0.87, and normalizing beliefs about paranoia α = 0.88.
The SIAS is a 20-item self-report questionnaire used to measure social anxiety (Mattick & Clarke, 1998). The SIAS was used for measurement of social anxiety in the EDIE-2 trial. The Cronbach's alpha for the current sample was α = 0.90 demonstrating high reliability.
The BDI-7 includes seven self-report items designed to measure depression (Beck, Guth, Steer, & Ball, 1997) and was the depression measure used in EDIE-2. Internal consistency for this study was very good: α = 0.86. analytic technique used to reduce data into meaningful groups or factors. In this study, it was used to explore potential alternative solutions of the MCQ-30 and their factor loadings. Oblique rotation (direct oblimin) was used to allow for more flexibility in the position of factors (Kline, 1994) because past validation research on the MCQ-30 (Wells & Cartwright-Hatton, 2004) has demonstrated items on this measure correlate. Eigenvalues above 1 were retained for the initial exploration of the measure. We examined the scree plot (Cattell, 1966) to determine the number of factors to extract.

| Data analyses
Parallel analysis is an alternative statistical method to determine the optimal number of factors to extract. It is recommended that this method is used in addition to the scree plot (O'Connor, 2000). Parallel analysis compares the eigenvalues of raw data to randomly selected data. Random data matches raw data in terms of the number of variables and observations (O'Connor, 2000). O'Connor (2000) suggests that random data are generated using the 95th percentile of the distribution of these randomly generated eigenvalues. Where the eigenvalue for the raw data is larger than the eigenvalue of the randomly generated data, the factor or component can be retained. A parallel analysis was run in SPSS using the O'Connor (2000) syntax; number of data sets input as 1,000; percentile set at 95; option "1" for PCA; and option "1" for normally distributed random data generation parallel analysis.
A further CFA was conducted in AMOS to test the model fit of an alternative factor structure identified by the parallel analysis to allow us to compare the results to the original five-factor structure. Only participants that did not have any missing data in the MCQ-30 (or MCQ-30 items extracted from the MCQ) were included in factor analyses (i.e., no estimates were created for missing data).
SPSS was also used to calculate means and SDs of measures and to conduct one-way MANOVA analyses to check if any statistical differences existed between males and females on the MCQ-30 subscales and total scores.
Cronbach alphas are calculated in SPSS to measure internal consistency of measures (i.e., how correlated items are in a subscale to assess how much items measure the same construct).
SPSS was used to run Pearson correlations to test for convergent and discriminant validity. Only EDIE-2 data (N = 153) was used for correlation analyses as the EDIE data did not have these measures.
Correlations were generated using the pairwise option in SPSS to prevent complete exclusion of participants from correlation analyses where they had missing data from only some measures.

| Sample
The mean age of the combined EDIE and EDIE-2 sample (N = 185) was 20.54 years, minimum 14 years, maximum 34 years, and SD 4.06 years.

| Confirmatory factor analysis
The chi-square was found to be significant X 2 (395) = 683.45, p < 0.001, which indicates a poor fitting model. However, the chisquare statistic is very sensitive to sample sizes (Garver & Mentzer, 1999;Hair, et al., 2014) and models with larger number of observed variables (individual items; Hair et al., 2014). The CMIN/DF (relative chi-square/degrees of freedom) score for this analysis was 1.73, well below the threshold of <3 (Hoe, 2008). A score of <2 on this statistic indicates a very good fitting model (Hair et al., 2014). The RMSEA was also found to be within acceptable parameters at 0.06, as it was <0.08 (Garver & Mentzer, 1999). On the CFI, GFI, and TLI, a score of more than 0.90 indicates a good fitting model (Garver & Mentzer, 1999).
CFI and TLI scores were just below the cut-off indicating a fair fitting model, whereas the GFI was further below the cut-off. Finally, the SRMR score is recommended to be below 0.08 to indicate good fit (Hu & Bentler, 1999), and this was achieved with a value of 0.07.
In this model, most of the standardized regression weights were above the recommended cut-off of 0.50 (Hair et al., 2014) with a range from 0.50 to 0.79. Only one item was found to be less than 0.50 from the cognitive self-consciousness subscale ("I am aware of the way my mind works when I am thinking through a problem" = 0.44).
Small to moderate correlations existed between most of the latent factors (ranges between 0.24 and 0.52). Larger correlations were found for the following subscales: need to control thoughts and cognitive self-consciousness (0.66) and need to control thoughts and negative beliefs about uncontrollability and danger (0.68).
Modification indices suggested the correlation of a number of errors within the same subscale. This improved the model as follows: X 2 (383) = 582.72, p < 0.001; CMIN/DF was 1.52; RMSEA = 0.05; CFI = 0.91; TLI = 0.90; GFI = 0.84; and SRMR = 0.07. Figure 1 shows the final model with correlated errors. Examination of the scree plot suggested a five-factor model of the MCQ-30 in the ARMS population. The PCA was rerun setting the fixed number of factors to five. Loadings on the pattern and structure matrices were checked. Where an item loaded on more than one factor and the difference between the loadings was equal to or more than 0.20, the highest loading item was retained on the factor it loaded highest on.

| Principal component analysis
Loadings at 0.40 or above were retained on each factor (Hair et al., 2014). Table 2 shows the structure matrix for all five factors.
Factors 1, 2, and 3 of the analysis included all the items that matched exactly the original MCQ-30 subscales: negative beliefs about uncontrollability and danger, cognitive confidence, and positive beliefs about worry, respectively. Where double loadings (i.e., items loading on another factor) were present on these three factors, the differences between the loadings was ≥0.20, and the higher loading was always on the original subscale. In this circumstance, the higher scoring item was retained and the lower scoring item ignored.
Factors 4 and 5 are related to subscales cognitive self-consciousness and negative beliefs about the need to control thoughts, respectively. All items related to the original subscale of the MCQ-30 loaded on these two factors. However, three variables (Questions 3, 13, and 27) loaded on two factors of the MCQ-30.
The differences between the two loadings in each case was <0.20.
Loadings for Questions 3 and 27 were slightly higher for the original Loadings that score lower on their original subscale than another factor. subscale, and for Question 13, it was slightly higher for the factor that did not relate to the original subscale.
Further investigation into the content of these questions and where they were loading suggested that the double loadings were logical and made theoretic sense. Question 13 ("I should be in control of my thoughts all of the time"), for example, loaded slightly higher on Factor 4 (cognitive self-consciousness) rather than its original subscale negative beliefs about the need to control thoughts (Factor 5). This double loading is consistent with the high correlation between the need to control thoughts and cognitive self-consciousness latent factors in the CFA. It makes sense that if someone believes they should be in control of their thoughts, they are likely to constantly monitor them so that this may be achieved.

| Parallel analysis
The parallel analysis output suggested a four-factor solution. Due to this, a PCA was rerun in SPSS using oblique rotation, setting the fixed number of factors to four. Factors 2 and 3 included items that Where cross loadings were present, they were minor and seemed theoretically coherent.
Although the parallel analysis suggests a four-factor structure, examination of the loading of the items did not suggest an alternative structure that was theoretically coherent. However, as a final test, a further CFA was run to examine the construct validity of this four-factor structure. The chi-square was found to be significant X 2 3.5 | MCQ-30 descriptive statistics for original fivefactor structure Means and SDs were calculated for the five original MCQ-30 subscales. Table 3 shows the results of this analysis for the total sample and split across males and females. A one-way MANOVA found no significant differences between males and females on any of the five MCQ-30 subscales or the MCQ-30 total score.

| Convergent and discriminant validity
(2.6%). Descriptives for measures are shown in Table 4. The correlation results are shown in Table 5.

| DISCUSSION
With the increased use of the MCQ-30 measure in at risk for psychosis research, it is important to examine the validity of this measure in this sample. The fit and structure of the original five-factor model was tested using CFA. Although the chi-square result indicated a poor fitting model, the chi-square is very sensitive to sample size (Garver & Mentzer, 1999;Hair et al., 2014), with larger samples leading to the increased likelihood of a significant chi-square. Further, the chances of the chi-square being significant is increased the more observed variables (i.e., individual items that make up each subscale or overall measure) there are in a model (Hair et al., 2014). It is, therefore, recommended that a range of fit indices are interpreted. The CMIN/DF (modified chi-square) was used in addition to assess model fit and suggested a very good fit as it was well below the cut-off. Further, the RMSEA was used in this study because it corrects for both sample size and complexity of model issues of the chi-square and better shows how well a model fits a sample (Hair et al., 2014). The RMSEA fit index also demonstrated a good model fit for this sample. The TLI is not as sensitive to changes in sample size, and the CFI is less sensitive to the complexity of models (Hair et al., 2014). Initially, both these indices were just below the cut-off indicating a fair fitting model. However, after correlating the errors both were within the cut-off indicating a good model. Overall, the majority of the CFA output indicated a good fit of the original five-factor structure of the MCQ-30 in this ARMS sample using the most commonly used and recommended indices (Hair et al., 2014;Hoe, 2008;Hu & Bentler, 1999).
To explore other possible latent structures for the MCQ-30, a PCA was run with a five-factor solution suggested by the scree plot, which was almost identical to the original solution. However, a fourfactor solution was suggested by the results of parallel analysis. A  four-factor solution was, therefore, specified and the PCA rerun. All items loaded under their original subscales except the negative beliefs about the need to control thoughts subscale. The items for this measure were split between Factor 1, negative beliefs about uncontrollability and danger of thoughts (four items) and Factor 4, cognitive self-consciousness (two items). However, these loadings made theoretical sense (e.g., if someone is feeling the need to control thoughts, they will likely increase monitoring of their thoughts hence the loading of negative items on the cognitive self-consciousness subscale). Further, the CFA results established that the original five-factor model had a better fit than the four-factor model recommended by the parallel analysis. All fit indices for the five-factor model showed a better fit including the SRMR, which is the statistic that is recommended for comparing models (Hair et al., 2014). Despite the good fit illustrated by the comprehensive examination of various tests of validity and reliability, some limitations exist for this study. The use of secondary data meant that other tests of validity and reliability could not be run. Test-retest reliability, for example, was not examined because the study involved a psychological intervention that lasted for 6 months. This could have led to different responses at the 6-month time point from participants who received treatment. With regards to criterion validity, which is described as how much scores on a measure relate to the gold standard (Terwee et al., 2007), as the MCQ-30 is considered to be the gold standard measure of metacognitive beliefs, this test could not be carried out. Although measures that assess some related concepts such as the Thought Control Questionnaire (Wells & Davies, 1994) could have been utilized instead, the use of secondary data meant that this could not be tested in the current study.
Although the internal consistency scores for all MCQ-30 subscales were between acceptable and very good, and males and females had similar Cronbach's alpha scores on four out of five of the subscales, there was a noticeable difference on the negative beliefs about the need to control thoughts subscale. Females scored higher on this subscale than males. It is not possible to determine the reasons for this from this study; however, past research has found that females tend to ruminate more than males (Bahrami & Yousefi, 2011;Johnson & Whisman, 2013). Due to this, it is possible that females may feel the need to control their thoughts more than males.
The male internal consistency score was still within acceptable parameters, so it seems the difference does not have an effect on the overall reliability of the subscale in the context of this study. However, it would be worth exploring this difference in future studies to better understand the variance.
In summary, psychometric analysis appears to confirm that the original five-factor structure of the MCQ-30 is valid for measuring metacognitive beliefs in those with an ARMS. The MCQ-30 with ARMS samples can be interpreted in the same way as in other psychological disorders. Future studies might find it useful to validate the MCQ-30 in other samples across the psychosis spectrum (e.g., first episode psychosis) and consider testing theoretical models using this tool. Further tests of reliability and validity that could not be conducted in the current study should also be examined in future work.
It might also be useful for CFA analyses to be conducted in larger samples to better establish the latent structure of the instrument and for closer examination of any gender differences on individual subscales of the measure.