Using network analysis to validate domains of the modified telephone interview for cognitive status

The modified Telephone Interview for Cognitive Status (TICS‐M) is a widely used tool for assessing global cognitive functions and screening for cognitive impairments. The tool was conceptualised to capture various cognitive domains, but the validity of such domains has not been investigated against comprehensive neuropsychological assessments tools. Therefore, this study aimed to explore the associations between the TICS‐M domains and neuropsychological domains to evaluate the validity of the TICS‐M domains using network analysis.


| INTRODUCTION
The modified Telephone Interview for Cognitive Status (TICS-M) 1 is a well-validated assessment tool used to assess global cognitive function in older adults. The TICS-M is suitable for screening for cognitive impairment 2,3 and monitoring changes in cognition over time such as due to an effective intervention. [4][5][6] This tool, which can be administrated by telephone or face-to-face, is often used in large epidemiological studies by the virtue of its brevity. 4,7 The TICS-M, while brief, consists of 13 items grouped into four cognitive domains, broadly including orientation, memory, attention and language according to factor analytic data from the original authors and a subsequent key paper. 1,5,8 Table S1 displays a detailed description of the TICS-M questionnaire, including its domains and individual items as well as the maximum score of each item. Previous studies have demonstrated high internal and temporal reliability of this measure, 2,[9][10][11] while its validity was supported by strong correlations with the widely used Mini-Mental State Examination. 8,12 In addition, the TICS-M has been validated against more comprehensive neuropsychological assessment tools, such as tests of general intellectual function, memory and language. 13,14 However, to the best of our knowledge, such validation studies have only focused on the total score of the TICS-M rather than the scores of its cognitive domains.
Network analysis offers a rigorous and comprehensive statistical framework to evaluate the validity of the TICS-M cognitive domains in relation to neuropsychological test performance. 15,16 Network analysis can be used to understand unique associations between a set of components by integrating single components into a global network, providing a clearer picture to enhance understanding of the complex relationship between components. 15,17 Network analysis graphically illustrates the presence and strength of associations between different variables using 'nodes' and 'edges'. 18 The nodes represent variables in the network and the edges represent unique statistical associations between the nodes (often in the form of partial correlations). Network analysis estimates all associations within the network without dividing variables into dependent (i.e. caused by other variables) and independent (i.e. affecting the dependent variable) variables. 19 This is preferable when no clear causal pathways are evident, making it a preferred method of analysis to determine the associations between the TICS-M domains and neuropsychological test performance. Furthermore, both the TICS-M domains and various cognitive tests capture different aspects of a person's broader cognitive functioning and thus are expected to be interrelated. By estimating their unique associations within a single network, these complex interrelations can be disentangled and unique links between different domains outlined.
This study aimed to explore the associations between the TICS-M domains and neuropsychological domains to evaluate the convergent validity of the TICS-M domains in a large sample of older adults over a 6-year period. Exploratory analyses using data collected at baseline, also called 'wave 1', aimed to explore unique associations between TICS-M domains and their associated cognitive domains as captured by standardised neuropsychological testing. Such analyses could provide preliminary evidence about convergent validity of individual TICS-M domains. Confirmatory network analyses were then conducted using data collected after 2, 4, then 6 years, or three follow-up waves (i.e. waves 2, 3 and 4), which allowed us to discard spurious exploratory findings and identify whether associations between domains captured by the TICS-M and neuropsychological tests remained stable over the 6-year period. To do this, we derived hypotheses after performing exploratory analyses, which were then tested in confirmatory analyses across waves.

| Participants
Participants were from the Sydney Memory and Ageing Study (MAS) which included 1037 older adults (572 females) aged 70-90 (M age = 78.84, standard deviation [SD] = 4.82) at the baseline assessment (i.e. wave 1). 20 Participants were living in the Eastern suburbs of Sydney, Australia, and were fluent in English such that they could provide written consent and complete psychometric assessments and self-report questionnaires. 20 The ethnicity of the MAS participants was predominantly European (98%). Participants were followed up every 2 years after the baseline assessment over the 6-year period. Exclusion criteria were prior dementia diagnosis, major psychological or neurological disorder, or progressive malignancy at baseline. More details about inclusion and exclusion criteria for MAS, as well as participants' baseline demographics, have been previously published elsewhere. 20 It should be noted that no inclusion/exclusion criteria were applied to participants at follow-up waves. Missing data at each followed wave were due to participants being unwell, global cognition, modified telephone interview for cognitive status, network analysis, neuropsychological domains having passed away or being unreachable at that wave. In some instances, participants were unable to participate in the assessment or were in advanced stages of dementia, rendering them unable to answer questions during the follow-up waves. All participants provided written informed consent to participate in this study, which was approved by the University of New South Wales Human Ethics Research Committee (HC: 05037, 09382, 14327, 190962). Figure 1 presents the consort diagram of how participants were selected at each wave for data analyses. We excluded participants who did not have complete assessments for each wave. Figure 1 also displays some basic demographic details of participants (i.e. age and sex) at each wave. The results of independent samples t-tests indicated that there was no significant difference in the ages of female and male participants across all waves (all p's > .05).

| Global cognition assessment
The TICS-M 1 is a 13-item test of cognitive functioning conducted via telephone or face-to-face. The scale theoretically captures four domains: orientation, memory, attention and language. 8 Each TICS-M item is scored differently with scores ranging from 1 to 10 and higher scores indicate better cognition function. Individual domain scores are computed by adding responses of relevant item(s) together, and thus their maximum scores are different (Table S1). For example, the memory domain has a maximum score of 22, while attention has a maximum score of 6. It should be noted that the TICS-M assessments were conducted 1 year after neuropsychological assessments were done at each wave.

| Neuropsychological domains
At each wave, participants completed a full neuropsychological test battery that captured the domains of attention/ processing speed, language, executive function, visuospatial ability and memory. These cognitive domains were evaluated because they assess cognitive abilities directly relevant for the diagnosis of dementia and predementia syndromes in the MAS study. 20 Domains were formed based on participants' scores across 10 neuropsychological tests according to the principal cognitive function each test represented 20 (Table S2). Neuropsychological domain scores were computed as quasi z-scores as follows. First, F I G U R E 1 CONSORT diagram for participants selected at each wave. raw test scores were converted to z-scores using baseline means and SDs of a reference group comprised of 723 MAS participants classified as cognitively healthy at baseline. Second, composite domain scores were formed by averaging the z-scores of the component tests, apart from the visuospatial domain which was represented by a single test. Each domain composite was standardised by transforming it so that the mean and SD of the baseline cognitively healthy group were 0 and 1, respectively. Higher z-score represented better performance/ability.

| Data analyses
IBM SPSS Statistics 28 software was used to compute descriptive statistics including mean, SD, skewness and kurtosis for the domain/node scores. We estimated a network that included the TICS-M domains and the neuropsychological test domains. Each variable/domain was represented as a node in the graphical network and unique associations between variable pairs were represented as edges (i.e. lines). The networks were Gaussian graphical models. In a network, nodes are connected with edges, and edges represent statistically significant associations. In this study, edges were estimated using partial correlations which can range from −1 (perfect negative association) to +1 (perfect positive association). The strength of each edge is reflected by the thickness of the line. Blue lines represent positive associations and red lines represent negative associations. We coloured the nodes according to whether they were included in the TICS-M or whether they were a cognitive domain measured using neuropsychological tests.
The network analyses were conducted in R (version 4.0.4; R Core Team, 2021) using the package Bayesian Gaussian graphical models (BGGM) which can estimate BGGM. 21 We used copula Gaussian graphical model estimation, which can estimate linear associations (edges) between a set of variables with different distributions because some variables were expected to be non-normally distributed. Unique edges between nodes were estimated as partial correlations, that is, each partial correlation represented the edge while accounting for all other linear relationships in the full set of nodes. Using a Bayesian theory approach, credible intervals (CIs) for each edge were used to control for false-positive rate. 22 Edges whose 95% CI did not include zero were considered statistically significant. A 95% CI indicates the lower and upper limits of an interval where an unobserved parameter is expected to fall 95% of the times. 23 The R library qgraph and the Fruchterman-Reingold algorithm 24 were used to plot the networks. This algorithm allows the most interconnected nodes to be central in the diagram as well as placing strongly related nodes closer to each other while avoiding overlap of nodes. 25 To facilitate comparisons, this study used an average network layout that was based on estimates from all four waves. That is, the placement of each node was kept the same in all networks while the edge thickness differed. 24 Moreover, predictability of nodes represented by Bayes R 2 were also conducted to indicate which nodes can be influential in the network. 26 A node with higher predictability means that such a node is more central or influential in comparison to other nodes in their relationships. 16,26 This study involved exploratory and confirmatory analyses. The exploratory analyses were carried out with the sample of participants at baseline (wave 1) to explore the network relations between nodes. In other words, this analysis was used to explore edges (associations) between nodes (domains) of the TICS-M and neuropsychological assessments. Post hoc tests were subsequently conducted to further test whether network patterns were statistically significant (e.g. that one edge is statistically significantly stronger than another edge), again using wave 1 data. Confirmatory analyses were then conducted with samples of participants at each follow-up wave to examine whether the network patterns that had been established with the exploratory sample at baseline could be confirmed using new data at future waves. The posterior probability (PP) method was used to test hypotheses. 27 PP is the probability of a pre-specified event occurring; for instance, that the edge between nodes A and B will be larger than between nodes A and C. To calculate PPs and CIs, we used 5000 posterior estimates, which aims to approximate the posterior distribution of parameters. A PP above .95 (95%) was used as an indicator of statistical significance. That is, if the PP for edge A-B > edge A-C, is above .95, we consider this difference to be statistically significant and the hypothesis confirmed. Table 1 presents descriptive statistics for the domains of TICS-M and the neuropsychological assessments domains across the four waves. Because skewness and kurtosis values for some domains/nodes indicated non-normality, 28 associations were estimated using copula Gaussian graphical model estimation.  Post hoc tests were used to examine whether these edges were confirmed. That is, whether the edges between the TICS-M attention node and the neuropsychological executive function and visuospatial ability nodes would be significantly stronger than between the TICS-M attention node and the other neuropsychological nodes. Also, whether the edge between the TICS-M attention node the neuropsychological attention node would be significantly weaker than between the TICS-M attention domain and the two other neuropsychological nodes (i.e. memory and language). The results confirmed the edges between the TICS-M attention node and the neuropsychological executive function and visuospatial ability nodes were statistically significant with all PPs > 98.8%, but not the edge between the TICS-M attention node and the neuropsychological attention node. The TICS-M language node had significant edges with the neuropsychological language (edge weight of .29, 95% CI [.106, .288]) and attention nodes (edge weight of .26, 95% CI [.041, .233]). Further tests were conducted to determine whether these edges were significantly stronger than edges between the TICS-M language node and the three remaining neuropsychological nodes (i.e. executive functions, visuospatial and memory). The results confirmed this, with all PPs > 95.1%. In addition, the TICS-M memory node had a significant edge with neuropsychological memory (edge weight = .47, 95% CI [.270, .395]) and this edge was significantly stronger than all other edges between TICS-M memory and the other neuropsychological domains (all PPs = 100%).

| Exploratory analyses
In sum, the results from the exploratory network analyses allowed us to derive hypotheses that (1) there would be weak links between the TICS-M nodes internally; (2) the TICS-M attention node would be more strongly associated with the neuropsychological executive function and visuospatial nodes than other neuropsychological domain nodes; (3) the TICS-M memory node would be more strongly related to neuropsychological memory node than to the other neuropsychological domain nodes; (4) the TICS-M language node would be more strongly related to the neuropsychological language and attention nodes than to the other neuropsychological domain nodes; and (5) the TICS-M orientation node would not have an associations with any of the neuropsychological domain nodes. These hypotheses would be tested in confirmatory analyses using data collected at waves 2-4. between the TICS-M and some neuropsychological nodes emerged in the followed up waves as comparison to wave 1 (e.g. edges between TICS-M orientation and neuropsychological visuospatial and memory nodes), the relative differences between edges were not affected by the presence of these new edges. Specifically, the network for wave 2 partly confirmed our hypotheses that weak internal associations between TICS-M nodes would remain (i.e. edge weights ranged from .12 to . 27), and that the edge between the TICS-M memory and neuropsychological memory nodes, and that the edge between TICS-M attention and neuropsychological visuospatial nodes, would remain significant. The hypothesis that no associations between TICS-M orientation and neuropsychological nodes was also confirmed at wave 2. The network at wave 3 confirmed most of our hypotheses, except the hypothesis that no associations between TICS-M orientation and neuropsychological nodes was not confirmed as there was an association between TICS-M orientation and neuropsychological visuospatial nodes. The hypotheses that were supported at wave 4 were the edge between TICS-M memory and neuropsychological memory node, and the edge between TICS-M language and neuropsychological language nodes. Besides that, the network analysis at wave 4 revealed stronger internal associations between TICS-M nodes (i.e. some edge weights were .30 or above) compared to the wave 1 findings. However, these TICS-M node interconnections were incoherent, and thus this finding confirmed the TICS-M internal associations found at wave 1. Therefore, the confirmatory results indicated that the majority of associations between nodes that emerged during the exploratory network analyses at wave 1 were verified by confirmatory network analyses with at least one follow-up wave during the 6-year period. This also means that there were significant and enduring relationships between TICS-M nodes and neuropsychological nodes. Predictability of nodes was also analysed across waves, which is presented in Figure 4. As can be seen, all neuropsychological nodes had high predictability, with the TICS-M memory node showing the highest predictability compared to other TICS-M nodes. This suggests that all neuropsychological domain nodes and the TICS-M memory node, specifically, reflected cognitive abilities that were more influential across all networks.

| DISCUSSION
The present study investigated the convergent validity of the domains of the TICS-M in a large ageing sample of older adults over a 6-year period using network analyses. The networks showed consistent associations between particular TICS-M domains and neuropsychological domains across four waves of assessment. It might be expected that TICS-M domains would be significantly associated with their corresponding neuropsychological domains at baseline and over time. It could also be expected that the TICS-M orientation domain would not be associated with any of the neuropsychological nodes because the orientation domain was exclusive to the TICS-M and was added to increase the accuracy of the brief cognitive screen for detecting older people with cognitive impairments. 5,12,29 However, the results did not fully meet these expectations. While the TICS-M memory and orientation domains remained significantly and consistently associated with their affiliated neuropsychological domains over 6-year, the remaining TICS-M domains (i.e. language and attention domains) did not. For example, instead of being clearly linked to the neuropsychological domains of attention, the TICS-M attention domain was significantly related to the executive function and visuospatial neuropsychological domains instead and thus seems to capture important aspects of broader cognitive functioning. This pattern of results is not entirely surprising given attention, executive function and visuospatial ability are not mutually exclusive and do overlap to some extent for most tasks. These tasks are both numerical (which can overlap with spatial processing type functions mediated by the parietal lobe), and also involve some executive aspects (holding information in mind while manipulating it) where multiple circuits are exercised to complete each task. 30 It is noteworthy that the majority of the associations that did emerge between TICS-M domains and F I G U R E 4 Predictability of nodes measured by Bayes R 2 across waves. neuropsychological domains were observed at least over two waves within the 6-year period. Specifically, the association between the TICS-M memory domain and the neuropsychological memory domain remained significant across all four waves. The associations between the TICS-M attention domain and neuropsychological visuospatial domain, and between the TICS-M language domain and the neuropsychological language domain, remained significant across three waves. The associations between the TICS-M attention domain and the neuropsychological executive function, and between the TICS-M language domain and the neuropsychological attention domain, remained significant across two waves. Interestingly, the TICS-M orientation domain did not have significant associations with most neuropsychological domains across all waves, except associations with neuropsychological visuospatial and memory at wave 3 and wave 4, respectively. Our findings regarding these consistent associations between the TICS-M domains internally and with neuropsychological domains over time can add more evidence to support the overall structural validity of the TICS-M screener in assessing enduring cognitive function, and are consistent with previous findings. 2 Moreover, the neuropsychological domains were interconnected to each other and each node had high predictability within each wave, which was expected and in line with previous research 31 and contrasts with the TICS-M domains, which were not. This finding may also raise concerns regarding internal consistency and validity of the individual domains of the TICS-M as these were expected to have stronger links with each other, and each TICS-M node had high predictability similar to neuropsychological tests. This is possibly due to the fact that the constructs of individual TICS-M domains were not cognitively comprehensive. For example, the TICS-M attention domain only has two items that are more likely focused on mathematical capacity rather than on the actual attention skill, and hence is only a proxy measure of such a skill. Therefore, the findings of this study revealed that global cognitive abilities captured by the TICS-M as a whole appeared more reliable compared to its individual domains. These findings are in-line with a previous study, which suggested that the TICS-M may be more useful as a neuropsychological screener of global cognitive performance than as an indicator of domain-specific ability. 32 Thus, the individual TICS-M domains should be used cautiously as indicators of specific cognitive abilities.
We observed changes in the relationship between TICS-M domains and neuropsychological domains over the course of 6 years. Several factors could potentially explain these changes. First, the ageing process of participants during the MAS study may play a role, as cognitive abilities can decline or change with age. 12 Second, the presence of medical conditions or neurological disorders, which may emerge or progress over time, can impact the associations between cognitive measures. 33 Third, differences between the assessment tools could contribute to these observed changes. For example, one assessment tool might be more sensitive to enduring aspects of cognition, while the other might capture more temporal fluctuations in cognitive performance. This would lead to differences in how the tools measure the associations between cognitive domains over time. Lastly, other external factors such as lifestyle changes, socio-economic status and environmental factors may also influence the cognitive domains and their relationships over time. Taking these factors into account is important when interpreting the relationship between TICS-M domains and assessed neuropsychological domains at different time periods.
Our study is not without limitations. Our sample was not representative as participants were predominately Caucasian and well-educated and were recruited from a moderately affluent area of Sydney, Australia. Future studies should replicate our results using a more culturally, linguistically and educationally diverse sample of older adults. In addition, the confirmatory analyses were conducted using the same sample but at later time points. It may be worth mentioning that there was nonrandom attrition between waves 1 and 4. That is, participants who were younger, healthier and more cognitively intact made it to wave 4 compared to those who dropped out, meaning, the confirmatory analyses could be biased by this. Therefore, it would be preferred to conduct the confirmatory analyses with independent samples of older persons. Moreover, we experienced a considerable decrease in sample size, with almost 50% of participants lost at the final wave. This reduction in sample size may have impacted the statistical power of our study, and hence caution should be exercised when interpreting our findings. Future research should consider strategies to enhance participant retention and minimise attrition, which could strengthen the validity and generalisability of the findings.

| CONCLUSION
The weak internal links between the TICS-M domains remained across all four waves, and most associations between TICS-M and neuropsychological domains were observed at least over two waves within the 6-year period. These results support the overall structural validity of the TICS-M screener in assessing enduring global cognitive function. However, the results also suggest that individual TICS-M domains should not be used as equivalent to the analogous neuropsychological domains.

AUTHOR CONTRIBUTIONS
QT lead and designed the study, analysed data and wrote the manuscript. MC supervised data analysis and edited the manuscript. CC supervised the study and edited the manuscript. KN managed data, collaborated with the study design and edited the manuscript. BA, AM and VF collaborated with designing the study and editing the manuscript. PS supervised data collection, the study design and edited the manuscript. NK collected data and edited the manuscript. HB sourced funding, collected data and edited the manuscript. OM supervised the study, data analysis and edited the manuscript.

ACKNO WLE DGE MENTS
The Sydney Memory and Ageing Study has been funded by three National Health & Medical Research Council (NHMRC) Program Grants (ID No. ID350833, ID568969 and APP1093083). We thank the participants and their informants for their time and generosity in contributing to this research. We also acknowledge the MAS research team: https://cheba.unsw.edu.au/research-projects/ sydney-memory-and-ageing-study. Open access publishing facilitated by The University of Waikato, as part of the Wiley -The University of Waikato agreement via the Council of Australian University Librarians.