Policy effects of the expansion of telehealth under 1135 waivers on intentions to seek counseling services: Difference-in-difference (DiD) analysis

Practicing counselors swiftly shifted to telehealth after insurance payers changed their policy to cover telehealth counseling under the expansion of telehealth with 1135 waivers, which has led to concerns over the policy effects on clients’ continuing counseling service-seeking intentions. To determine if this shift to telehealth was associated with a change in clients’ counseling service-seeking intentions while adjusting for unmeasured confounders and time trends, we performed a difference-in-difference analysis (a quasi-experimental design) to compare changes between the client (treated) and non-client (control) groups through a large national sample of emerging adults ( N = 52,237). The study ﬁndings indicated that the shift to telehealth was not associated with a change in clients’ counseling service-seeking intentions, providing support for the utility of telehealth counseling. Implications of these ﬁndings should be considered when assessing the merit of telehealth counseling to inform public policies and counseling practice in the post-pandemic era.


INTRODUCTION
Mental health professionals faced unprecedented challenges in keeping their services available for individuals with pressing mental health needs due to the highly infectious severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), responsible for coronavirus disease 2019 (COVID-19) (Wosik et al., 2020;Zhou et al., 2020).Many mental health providers suspended traditional face-to-face services to reduce the risk for transmission of COVID-19 between clients and counselors (Smith et al., 2020;Whaibeh et al., 2020;Wosik et al., 2020).To mitigate the risk of service disruptions, the Centers for Medicare & Medicaid Services shifted policy by broadening access to telehealth under the expansion of telehealth with 1135 waivers, and commercial health plans started to cover telehealth counseling services.
Many practicing counselors rapidly responded to the policy change amid lockdowns/shelters-in-place and the threat of COVID-19, shifting their services from in-person to telehealth counseling (Hall et al., 2022;Smith et al., 2020).The first aim of this current study is to assess whether this policy change/shift to telehealth was associated with a change in clients' intentions to seek counseling services.
Delivering counseling services through technologies at a distance (known as telehealth) became critical during the first year of this pandemic (Smith et al., 2020;Whaibeh et al., 2020;Wosik et al., 2020;Zhou et al., 2020).Telehealth is crossing the gap between use by a few visionaries and implementation by many pragmatists due to the clinical benefits of telehealth and the explosive growth and development of technological infrastructure over the years (Dorsey & Topol, 2020).Research on telehealth counseling has found no decline in therapeutic outcomes compared with face-toface treatment to date (Everitt et al., 2019;Hammersley et al., 2019;Nauphal et al., 2021;Ngai et al., 2015).Given the pivotal role of telehealth in protecting clients from a disruption of care during lockdowns/shelters-in-place periods, there also has been a growing body of literature assessing the feasibility and acceptability of telehealth counseling, indicating that telehealth counseling appears to be acceptable for clients (Childs et al., 2021;Cuthbert et al., 2022;Nauphal et al., 2021;Zhou et al., 2020).
Despite these clinical and technological benefits of telehealth counseling, some studies suggested that a preference for in-person counseling (Hall et al., 2022), mis/distrust of telehealth (Rovner et al., 2021), and barriers to telehealth solutions (Zhai, 2021) might have affected the continuity of care and clients' intentions to continuously seek counseling services.The internet and digital devices seem ubiquitous nowadays, yet many individuals with pressing mental health concerns still encountered a variety of barriers to telehealth in the early phase of this pandemic, such as poor technological infrastructures in many regions (Struminger & Arora, 2019), inequalities around digital literacy (Hargittai, 2002;Humphry, 2019;Rhoades et al., 2017;Scheerder et al., 2019), insufficient telehealth training for counselors (Johnson & Rehfuss, 2021;Scharff et al., 2020), and regulatory/reimbursement restrictions (Whaibeh et al., 2020;Zhai, 2021).We wondered if these barriers and mixed attitudes toward telehealth counseling might have impeded clients' intentions to seek counseling services during the first year of this pandemic.

Racial/ethnic disparities in mental health
Prior to the outbreak of COVID-19, racial disparities in intentions and access to counseling services were well documented (Shannon et al., 2022).The uneven psychosocial effects of the COVID-19 pandemic on racial/ethnic minorities might be expected to exacerbate racial disparities in intentions to engage in counseling and also mental health outcomes because of poor social determinants of health attributed to prolonged sociopolitical and economic inequalities (Chowkwanyun & Reed, 2020;Shannon et al., 2022).Research has shown that the COVID-19 pandemic may have placed racial/ethnic minorities at elevated risk for mental health problems (Thomeer et al., 2022;Zhai & Du, 2022a).Before the pandemic, racial/ethnic minorities, such as members from Asian, Black/African, and Latinx communities, reported greater psychosocial stress and worse well-being than White individuals, experiencing various mental health issues (Thomeer et al., 2022).As typical mental health issues, clinical depression, anxiety, and eating disorders are among the most critical causes of disability, morbidity, and mortality (Wang et al., 2020;Ward et al., 2019;Weissman & Hay, 2022).These mental health issues can substantially impact life expectancy and disrupt physical, cognitive, psychological, and social functioning (Kessler et al., 2022;Taquet et al., 2021;Weissman & Hay, 2022;Zipfel et al., 2022).Severe conditions will also require intensive treatment and support, which can be costly (Kessler et al., 2022;Zipfel et al., 2022).Prior empirical studies (Cooper et al., 2022;Kessler et al., 2022;Weissman & Hay, 2022) indicated that clinical depression, anxiety, and eating disorders have become a more prominent health burden during the early days of the pandemic because many factors associated with COVID-19 had the potential to contribute to the development and risk of depression, anxiety, and eating disorders.During the pandemic, racial/ethnic minorities were more susceptible to public health, social, and economic turmoil, including heightened discrimination, racially motivated hate crimes, and food insecurity, rooted in this COVID-19 crisis (Thomeer et al., 2022;Zhai & Du, 2020, 2022b).The disproportionate impact of the pandemic might be anticipated to worsen racial disparities in mental health outcomes; however, there remains a paucity of evidence on racial/ethnic differences in these critical mental health outcomes (i.e., clinical depression, anxiety, and eating disorders).In responding to the initiatives to address racial health disparities (the American Counseling Association [ACA], 2022; the U.S. Centers for Disease Control and Prevention [CDC], 2022), the second aim of this study is to examine racial differences in the likelihood of experiencing clinically significant mental health symptoms (i.e., clinical depression, anxiety, and eating disorders) before and during the pandemic.
Studies conducted before the COVID-19 pandemic indicated that racial/ethnic minorities (e.g., Asian, Black/African, Indigenous, and Latinx American) were less likely than White/European Americans to seek counseling services when experiencing mental health concerns (Ballesteros & Tran, 2020;Cheng et al., 2018).Research shows that stigmatization associated with mental illness and mental health services can hinder racial/ethnic minorities from seeking professional help (Alang, 2019;Shannon et al., 2022).Racial/ethnic minorities also experience racial microaggressions that penetrate mental health care settings, such as racist assumptions and stereotypes against the experiences of people of color (Alang, 2019).Additionally, racial/ethnic minorities have confronted various challenges that emerged before and during the pandemic, such as socioeconomic inequalities, systemic racism, and civil unrest, and these challenges were likely to worsen racial disparities in seeking counseling services (Shannon et al., 2022).
Although there has been a growing body of research providing a snapshot of psychological help-seeking among racial/ethnic minorities during the pandemic (Banks, 2022;Singh et al., 2021), a considerable gap in knowledge still exists around the difference of intentions to seek counseling services between pre-and peri-pandemic periods among White individuals and racial/ethnic minorities.To fill the gap, the third aim of this current study is to explore racial differences in intentions to seek counseling services before and during the COVID-19 pandemic.

Intentions to seek counseling services
Individuals' intentions to seek counseling services reflect their attitudes and actions of pursuing professional services in responding to mental health changes (Ajzen, 1985;Carlson et al., 2022;Cornally & McCarthy, 2011;Li et al., 2014).According to Ajzen's Theory of Planned Behavior (Ajzen, 1985(Ajzen, , 2011)), individuals' intentions play a determinative role in their actions and behaviors, which is subject to attitudes, subjective norms, and perceived behavioral control.An emerging body of literature (Carlson et al., 2022;Rabideau et al., 2014;Sylvia et al., 2012) has shown that individuals' service-seeking intentions strongly predict their actual behaviors, namely, attendance at their appointments.When coupled with these empirical findings (Carlson et al., 2022, Rabideau et al., 2014), Ajzen's Theory of Planned Behavior (Ajzen, 1985(Ajzen, , 2011) ) lends theoretical support for the importance of examining clients' counseling service-seeking intentions, given a growing body of research indicating positive associations between attendance in counseling services and mental health outcomes (Erekson & Lambert, 2015;Erekson et al., 2015;Gutner et al., 2016;Xiao et al., 2017).
Specifically in counseling research, current empirical evidence suggests that intentions to seek counseling services serve as a strong predictor and determinant of engagement in counseling that is linked to mental health outcomes (Erekson et al., 2015;Li et al., 2014).Individuals' lack of intention to seek counseling services often delays their actual behaviors of seeking professional help, which exacerbates their mental health problems (Li et al., 2017;Li et al., 2014).For those who experienced counseling, their lack of intention to seek counseling services predicted their treatment adherence and premature termination, affecting therapeutic and mental health outcomes (Erekson & Lambert, 2015;Erekson et al., 2015;Gutner et al., 2016;Hammer & Vogel, 2013;Xiao et al., 2017).To date, however, there has been a lack of empirical evidence about whether the policy change (i.e., the shift to telehealth under the expansion of telehealth with 1135 waivers) may have affected clients' intentions to seek counseling services in the United States.Therefore, assessing the change in clients' intentions to seek counseling services before and after the policy change is necessary to inform public health policies and counseling practices contributing to mental health outcomes.Wing et al. (2018) suggested that natural experiments can be used to practically examine causal effects of policies instead of large-scale randomized controlled trials (RCTs) due to the complexities of funding and logistics.Following Wing et al. (2018) for this study, a quasi-experimental design (i.e., a difference-in-difference [DiD] analysis) was used to examine the effect of the policy change (i.e., the shift to telehealth under the expansion of telehealth with 1135 waivers) on counseling service-seeking intentions among clients in counseling amid the COVID-19 pandemic, through a large national sample.

CURRENT STUDY
To our knowledge, this is the first study of its kind evaluating the association of this policy with the intention of seeking counseling services among individuals nationwide.Because of the need to understand racial disparities in mental health outcomes and services over time, we also used adjusted multivariable logistic regression to examine the likelihood of having clinically significant mental health symptoms (i.e., clinical depression, anxiety, and eating disorders), intentions of seeking counseling services, and the utilization of counseling services between racial/ethnic minorities and White individuals before and during the pandemic.The following hypotheses guided our research design and analysis:

METHOD Participants
The present study was approved by the institutional review board (IRB) of the University of Alabama at Birmingham.We analyzed secondary, de-identified data from the Healthy Minds Study, which surveyed a random sample of individuals from 144 US higher education institutions from September 2018 to June 2020.The Healthy Minds Study was approved by IRB at all participating institutions, and all participants provided written informed consent.The original purpose of the Healthy Minds Study and survey was to investigate the prevalence of mental health outcomes, knowledge and attitudes about mental health and services, serviceseeking intentions, and barriers to mental health services.The inclusion criteria for this study stipulated that participants must be over the age of 18. Study sites are diverse across institution characteristics, including institutional type (e.g., Historically Black Colleges and Universities, community colleges, private/public, 2-/4-year colleges, undergrad/grad schools), enrollment size, and geographic location (i.e., urban and rural campuses representing schools from all nine census regions).At larger US institutions (i.e., more than 4000 individuals), a random sample of 4000 individuals from the full population was invited to participate.At smaller US institutions, all individuals were invited to participate.When recruited via email, participants were informed of their eligibility for winning a gift card (i.e., 1 of 10 $100 prizes, or 1 of 2 $500 prizes).
Differences between individuals who responded and those who did not respond were adjusted through sample weights based on institutional data on sex, race, academic level, and grade point average.Although individuals who were invited to participate were representative of the entire population at each institution, it was possible that certain demographic characteristics, such as biological sex and race, of individuals who completed the survey might be different from those of individuals who did not participate.Therefore, the aforementioned institutional data were used to estimate the response propensity of each type of participant, and then, each participant was assigned a response propensity weight accordingly.Namely, weights are larger for participants with underrepresented characteristics to ensure that estimates represent the full population at each institution.

Treatment versus control group (clients vs. non-clients)
In the current study, participants were assigned to the treatment or control group based on the status of their use of counseling services.Participants who met all three criteria (i.e., [a] received counseling services before and/or since starting college; [b] received counseling services in the past 12 months; and [c] currently receive counseling services) were assigned to the client group (i.e., the treatment group) because they were exposed to the "treatment" (i.e., the policy change/telehealth shift that started on March 15, 2020).Participants who reported never receiving counseling services before and/or since starting college were assigned to the non-client group (i.e., the control group), given that they were very unlikely to be affected by and exposed to the "treatment."Demographic characteristics of participants (N = 52,237, mean [SD] age, 19.8 [1.3] years, 52.8% women) are presented in Table 1.Among them, 9843 (18.8%) individuals were assigned to the client group, and 42,394 (81.2%) individuals were assigned to the non-client group.Participants self-identified as non-Hispanic White (n = 32,111, 61.5%),American Indian/Alaskan Native (n = 118, 0.2%), Asian (n = 5243, 10.0%), Black/African (n = 4504, 8.6%), Latinx (n = 4160, 8.0%), Middle Eastern/Arab Americans (n = 651, 1.2%), Native Hawaiian/Pacific Islander (n = 51, 0.1%), and other racial/ethnic groups, including multiracial (n = 5307, 10.2%).Further, 12,795 (24.5%) selfidentified as first-generation college students.Overall, the client group had a lower percentage of racial/ethnic minorities (i.e., Asian, Black/African, Indigenous [i.e., American Indian/Alaska Native and Native Hawaiian/Pacific Islander], Latinx, Middle Eastern/Arab American, and multiracial; 2810 [28.6%] vs. 17,223 [40.7%]) relative to the non-client group.Likewise, the group that indicated counseling serviceseeking intentions had a lower percentage of racial/ethnic minorities (5502 [32.2%] vs. 14,532 [41.4%]) relative to the group that did not indicate counseling service-seeking intentions.

Outcome variables
Participant-reported intention to seek counseling services was measured based on participants' responses to a survey question: "If you were experiencing serious emotional distress, whom would you talk to about this?"This binary outcome variable was coded 1 for participants who selected professional clinicians, such as counselors and psychologists, and coded 0 for those who did not select this answer.Three other binary outcome variables were also included defined by exceeding cutoff scores for clinically significant depression (≥15 on the Patient Health Questionnaire-9 [PHQ-9; Kroenke et al., 2001]), anxiety (≥15 on the Generalized Anxiety Disorder-7 [GAD-7; Spitzer et al., 2006]), and eating disorders (≥3 on the Sick, Control, One, Fat, Food [SCOFF] questionnaire [Morgan et al., 1999]), respectively.
PHQ-9: The PHQ-9 is a nine-item screening tool for a major depressive episode, and it uses a four-point Likert scale ranging from 0, Not at all, to 3, Nearly every day.Clinical depression is warranted when the total score is greater than or equal to 15 (Kroenke et al., 2001).The PHQ-9 has demonstrated consistently strong validity and reliability across studies before and during the pandemic (Shevlin et al., 2022).Of note, receiver-operating characteristic (ROC) curve analysis is essential to evaluate and determine the diagnostic accuracy (i.e., sensitivity, specificity) of a screening test at a specific cutoff (Zou et al., 2007).At the cutoff of 15, the PHQ-9 showed the highest specificity with less sensitivity (i.e., 0.95 and 0.68; Kroenke et al., 2001), which assisted in ruling diagnoses in.Namely, a screening test must be highly specific to lower the number of false-positive results when the prevalence of a certain condition is relatively low (Power et al., 2013).Overall, the PHQ-9 showed good construct and factorial validity (Kroenke et al., 2001;Schuler et al., 2018).Further, the PHQ-9 demonstrated excellent test-retest reliability at 14 days (r = 0.94; Zuithoff et al., 2010).It also showed high interrater reliability (ICC = 0.94; Indu et al., 2018).McDonald's omega reliability of the PHQ-9 was 0.90 in the current sample.
GAD-7: The GAD-7 is a seven-item screening tool for generalized anxiety disorder, and it uses a four-point Likert scale ranging from 0, Not at all, to 3, Nearly every day  (Spitzer et al., 2006).Clinical anxiety is warranted when the total score is greater than or equal to 15.The GAD-7 has demonstrated consistently strong validity and reliability across studies before and during the pandemic (Shevlin et al., 2022;Spitzer et al., 2006).At the cutoff of 15, the GAD-7 showed the highest specificity with less sensitivity (i.e., 0.95 and 0.48; Spitzer et al., 2006).Overall, the GAD-7 had good construct, factorial, and procedural validity (Spitzer et al., 2006).Additionally, the GAD-7 demonstrated good test-retest reliability within 7 days (r = 0.83; Spitzer et al., 2006).McDonald's omega reliability of the GAD-7 was 0.92 in the current sample.SCOFF: The SCOFF is a five-item screening tool with adequate diagnostic accuracy for eating disorders, covering anorexia nervosa or bulimia nervosa, and it uses a two-point scale from 0, No, to 1, Yes (Feltner et al., 2022;Morgan et al., 1999).Participants were likely to have an eating disorder when the total score was greater than or equal to three (Mor-gan et al., 1999).The SCOFF has been widely adopted in studies before and during the pandemic, demonstrating strong validity and reliability (Cerniglia & Cimino, 2022;Tavolacci et al., 2021;Wan Wahida et al., 2017).At the cutoff of three, the sensitivity and specificity of the SCOFF were 99.1% and 95.8%, respectively, when DSM-IV-based clinical interview was utilized as the reference standard (Hill et al., 2010).In a recent meta-analysis (Kutz et al., 2020), the pooled sensitivity and specificity of the SCOFF were 86% and 83%, respectively.Overall, the SCOFF showed good construct and factorial validity (Richter et al., 2017;Wan Wahida et al., 2017).The SCOFF demonstrated high test-retest reliability at 10-15 days (r = 0.97; Garcia-Campayo et al., 2005).It also showed strong interrater reliability when comparing four raters (Fleiss' kappa = 0.89; Piacentino et al., 2017).McDonald's omega reliability of the SCOFF was 0.60 in the current sample, which was due to the susceptibility of internal reliability to the binary answer layout (yes or no) and brevity of the scale (five items in the SCOFF) (Leung et al., 2009;Siervo et al., 2005).

Covariates (demographic variables)
Demographic variables, as covariates in our statistic models, included participants' race/ethnicity, gender, sex (female, male, and intersex), and socioeconomic status (SES; firstgeneration student status).Race/ethnicity was a categorical variable with eight categories (i.e., American Indian/Alaska Native, Asian, Black/African, Latinx, Middle Eastern/Arab, Native Hawaiian/Pacific Islander, non-Hispanic White, and other races/ethnicities including multiracial).Gender was a categorical variable with six categories (i.e., man, woman, transgender man, transgender woman, queer, and additional genders).

Statistical analysis
A quasi-experimental study design (i.e., a DiD analysis) was adopted to evaluate the association between the shift to telehealth under the expansion of telehealth with 1135 waivers and counseling service-seeking intentions.DiD is an econometric approach and a quasi-experimental alternative to an RCT design (Wing et al., 2018).It has been an effective means to investigate causal relationships that are of interest and importance to public health and has been widely used to examine changes that occur due to policy changes, natural experiments, or sociopolitical events (Dimick & Ryan, 2014;Wing et al., 2018).
Noteworthy is the notion that a simple pre-post comparison among clients is no longer sufficient to assess the association between the shift to telehealth and clients' continuing intentions to seek counseling services because unmeasured confounders (e.g., environmental change and group-specific and time-specific effects) might also have shifted over time (Dimick & Ryan, 2014;Rajaram et al., 2014;Wing et al., 2018).As a way to adjust for such unmeasured confounders and time trends, this study included individuals who had never received counseling as a control group as it was very unlikely that the shift to telehealth counseling affected counseling service-seeking intentions in people who had never been in counseling (Dimick & Ryan, 2014;Rajaram et al., 2014;Wing et al., 2018).
One example of unmeasured confounders is the psychosocial effect of the pandemic that might have affected both the client and non-client groups; however, only the client group (i.e., those who had been in counseling) was exposed to the policy change/shift of telehealth counseling.Thus, prepost differences in non-clients reflected any changes found in clients in the absence of the shift to telehealth (Dimick & Ryan, 2014;Rajaram et al., 2014;Wing et al., 2018).In order to identify the association between the shift to telehealth and the percentage of counseling service-seeking, we compare the difference between clients and non-clients in pre-post differences in counseling service-seeking intentions (i.e., the difference in differences).
Specifically, we performed DiD logistic regression with robust standard errors to examine changes in the percentage of clients reporting counseling service-seeking intentions surrounding the shift to telehealth beyond changes that occurred in non-clients.To detect pre-post time differences, we included two dummy variables to indicate (a) whether individuals had been receiving counseling (clients vs. non-clients), and (b) whether they reported counseling service-seeking intentions before (December 16, 2019 to March 15, 2020) or after (March 16-June 15, 2020) the shift to telehealth.Also included was a DiD estimator, which is an interaction term of these two dummy variables.The coefficient of this DiD estimator reflected the magnitude of association between the shift of telehealth and counseling service-seeking intentions (or the "true" effects of the shift to telehealth on clients).In this study, DiD logistic regression models adjusted for demographic characteristics (i.e., race/ethnicity, gender, sex, and SES).
To illustrate the DiD modeling approach using logistic regression, including the aforementioned variables, the DiD model is written as follows: In this equation, i indexes participants.Y i refers to the study outcome of interest (i.e., counseling service-seeking intentions).Covariates indicate demographic characteristics of participants.s are the estimated effect of one unit change in corresponding variables on the log odds of Y.The coefficient (i.e.,  3 ) of the interaction term, as the primary variable of interest, is the DiD estimator of the effect of the shift to telehealth on the log odds of Y (i.e., the "difference in differences").A significant DiD estimator indicates the effect of telehealth transition on counseling service-seeking intentions.
Lastly, to detect the association between race/ethnicity and clinically significant mental health symptoms (i.e., depression, anxiety, and eating disorders) and counseling service-seeking and utilization, sample-weighted multivariable logistic regression was performed to compute adjusted odds ratios (aORs) and 95% CIs, adjusting for covariates.We used complete case analysis (i.e., listwise deletion) to handle missing data.A two-sided p-value with the level of statistical significance was set to 0.05.

The common/parallel trends assumption
DiD designs rely on the assumption that unmeasured confounders are either time-invariant group factors or groupinvariant time-varying factors, and researchers often perform two types of assessments to check this assumption (Dimick 15566676, 2023, 3  & Ryan, 2014; Wing et al., 2018).The first approach is the visual inspection of the trends in outcomes over multiple time points (i.e., graphical evidence), and a set of parallel lines in a graph of the trends in outcomes between the treatment and control groups is required (Dimick & Ryan, 2014;Wing et al., 2018).In Figure 1, we plot counseling service-seeking intention rates for the control and treatment groups to visually examine the period before the policy change.
Second, formal statistical tests empirically examine the credibility of the parallel trend assumption (Acharya & Dhakal, 2021;Autor, 2003;Wing et al., 2018).In this study, a dynamic event study approach for pooled DiD was used, including three indicators/leads (i.e., respective interactions of dummy variables for November 2018 to February 2019 [lead], February-May 2019 [lead], and September-December 2019 [lead]) in a separate regression model for periods before the policy change to detect significant differences of the trends prior to the intervention period (March-June 2020), and the first period before the policy change was considered the baseline period (i.e., December 2019 to March 2020) (Acharya & Dhakal, 2021;Autor, 2003;Wing et al., 2018).

Sensitivity analyses
Sensitivity analyses were performed to test the robustness of our findings and whether there were differential policy associations in racial minority and non-Hispanic White populations (Delaney & Seeger, 2013;Wing et al., 2018).DiD was, respectively, performed to compare the difference between clients and non-clients in pre-post differences in counseling service-seeking intentions, stratifying by race/ethnicity (racial/ethnic minorities vs. non-Hispanic White).We recoded race/ethnicity into a binary variable with two categories: 0 = non-Hispanic White, 1 = racial/ethnic minorities (i.e., Asian, Black/African American, Indigenous [i.e., American Indian/Alaska Native and Native Hawaiian/Pacific Islander], Latinx, Middle Eastern/Arab, and multiracial).We also performed sensitivity analysis by replacing the categorical covariate of race/ethnicity with the dichotomized covariate of race/ethnicity to test the robustness of our findings from the DiD model (Delaney & Seeger, 2013).

The common/parallel trends assumption tests
The graphic evidence (see Figure 1) suggested a set of parallel trends in counseling service-seeking intentions between the client group and the non-client group in the preintervention period.Additionally, results from the event study model for pooled DiD (see Figure 2) revealed that three DiD indicators/leads were not statistically significantly associated with a change in the likelihood of counseling service-seeking intentions (i.e., 95% CIs included zero) before the policy change, indicating that the trends were not significantly different prior to the intervention period (Acharya & Dhakal, 2021;Autor, 2003;Wing et al., 2018).Namely, counseling serviceseeking intentions among clients and non-clients changed  approximately in parallel before the universal shift to telehealth counseling, with no significant differential change before the policy change (i.e., November 2018 to March 2020).Clients' intentions increased or decreased in parallel with non-clients' intentions before the adoption of the expansion of telehealth with 1135 waivers.Given the results of these tests, the parallel trends assumption was met in this current study.

Counseling service-seeking intentions in difference-in-difference models (H1)
Estimates from DiD are presented in Table 2 (n = 16,304, 47.3% women, 14.9% clients [vs.85.1% non-clients], 58.5% White, 28.0% first-generation college students).Results from multivariable logistic regression showed that the shift to telehealth was not associated with a change in the likelihood of seeking counseling services (aOR: 1.08; 95% CI: 0.81-1.45;p = 0.61), although counseling service-seeking intentions increased from pre-telehealth shift to post-telehealth shift in both clients (80.3%-85.5%)and non-clients (11.8%-15.2%).Figure 3 shows the percentage of clients and non-clients with counseling service-seeking intentions before and after the shift to telehealth, adjusted for race/ethnicity, gender, sex, and first-generation student status.
Results from sensitivity analyses (adjusted for gender, sex, and first-generation student status) revealed that among racial/ethnic minorities (e.g., people of color and Indigenous people), the shift to telehealth was not associated with a change in the likelihood of seeking counseling services (aOR: 1.42; 95% CI: 0.91-2.23;p = 0.12), though counseling service-seeking intentions increased from pre-telehealth shift to post-telehealth shift in both clients (82.3%-89.8%)and non-clients (17.5%-22.0%).Among non-Hispanic White individuals, the shift to telehealth was not associated with a change in the likelihood of seeking counseling services (aOR: 0.87; 95% CI: 0.59-1.27;p = 0.46), albeit counseling service-seeking intentions increased from pre-telehealth shift to post-telehealth shift in both clients (79.7%-82.5%)and non-clients (9.2%-12.4%).Additionally, in the DiD model with the dichotomized covariate of race/ethnicity, the shift to telehealth was not associated with a change in the likelihood of seeking counseling services (aOR: 1.06; 95% CI: 0.79-1.42;p = 0.69).

Mental health outcomes and services seeking and utilization (H2 and H3)
Overall, we observed increases in the likelihood of experiencing clinical depression and anxiety and decreases in the likelihood of counseling service-seeking and utilization among some racial/ethnic minorities when compared with non-Hispanic White individuals.

Before the shift to telehealth and the pandemic
Results from multivariable logistic regression models revealed that after controlling for sex, gender, and firstgeneration student status, individuals who self-identified as TA B L E 2 Association between the shift to telehealth under the 1135 waivers and counseling service-seeking intentions from the difference-in-difference analysis (DiD), 2019(DiD), -2020 (n = 16,304) (n = 16,304).

Variable
No  After the shift to telehealth around the pandemic

Main findings
In this study, the shift to telehealth under the policy expansion of telehealth with 1135 waivers responding to the national emergency concerning COVID-19 was not associated with a change in counseling service-seeking intentions among clients receiving counseling.To our knowledge, this is one of the first national empirical studies to examine the effects of this policy change.Our results may be considered when evaluating the merit of this transition to inform future organizational and social policies for telehealth counseling.Previous studies found that mental health providers struggled to keep their in-person services available for their clients due to the highly infectious nature of SARS-CoV-2 (Wosik et al., 2020;Zhou et al., 2020).Responding to the national emergency concerning COVID-19, Medicare/Medicaid 15566676, 2023, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jcad.12468,Wiley Online Library on [27/11/2023].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License programs and commercial health plans began to cover telehealth services provided by professional counselors.Correspondingly, many counselors swiftly shifted to telehealth counseling, as a viable approach during lockdowns/shelterin-place (Smith et al., 2020).Although a growing body of literature has suggested that telehealth counseling offers clinical outcomes comparable to face-to-face treatment (Everitt et al., 2019;Hammersley et al., 2019;Ngai et al., 2015), only a few studies indicated that barriers to telehealth solutions (Zhai, 2021), mis/distrust of telehealth counseling (Rovner et al., 2021), and a preference for in-person care (Hall et al., 2022) might affect the continuity of care and clients' intentions of seeking counseling services.
We did not find a significant change in counseling serviceseeking intentions among clients, suggesting that clients were likely to continue seeing their counselors via telehealth counseling.Namely, the rapid shift to telehealth might have protected clients from a disruption of their counseling services during a time of public health emergency.Moving beyond the pandemic, many clients may still experience difficulties accessing in-person services due to poor social determinants of health (e.g., low SES, geographic distances, and lack of reliable transportation), and some may prefer telehealth as a viable solution to receive ongoing counseling services (Childs et al., 2021;Cuthbert et al., 2022;Nauphal et al., 2021).In light of these clinical and technological benefits of telehealth counseling, our findings lend support for considering the use of telehealth and preserving insurance coverage for telehealth counseling in the post-pandemic era.

Racial/ethnic disparities in mental health and services
Findings of this study suggested that members from certain racial/ethnic minority communities were more likely than White individuals to experience one or more clinically significant mental health problems (i.e., clinical depression, anxiety, eating disorders) during the pandemic, which is consistent with previous studies indicating such mental health disparities (McKnight-Eily et al., 2021;Thomeer et al., 2022).Given the disproportionate impact of the pandemic (e.g., unequal living conditions, poor medical care, economic inequality, and inequitable policies), racial/ethnic minorities were more susceptible to mental health issues compared with their White counterparts (McKnight-Eily et al., 2021;Thomeer et al., 2022).Specifically, Middle Eastern/Arab individuals were more susceptible to all study mental disorders (i.e., clinical depression, anxiety, and eating disorders) than White individuals during the pandemic.Further, we found that Asian and Black/African identities were associated with lower odds of experiencing clinical anxiety and eating disorders, respectively, which might be attributed to heightened levels of resilience among these racial/ethnic minority communities when they responded to societal racism, discrimination, and xenophobia in the context of the pandemic (Rivas- Drake et al., 2021).
In addition to the findings tied to the peri-pandemic period, it is worthy to note some changes in the likelihood of experiencing certain mental health problems among several racial/ethnic minority groups between preand peri-pandemic periods.For example, Black/African and Middle Eastern/Arab identities were significantly associated with higher odds of having clinical depression in the peri-pandemic period; however, these associations were not statistically significant in the pre-pandemic period.When compared with results from the pre-pandemic period, Black/African identities were no longer significantly associated with lower odds of having clinical anxiety.These findings further support recent studies indicating the pronounced decline in mental health among racial/ethnic minorities (McKnight-Eily et al., 2021;Thomeer et al., 2022).
Moreover, our findings indicated that Asian, Black/African, Latinx, and Middle Eastern/Arab individuals were less inclined than White individuals to use and seek counseling services before or during the COVID-19 pandemic, which corroborates previous findings suggesting racial disparities in utilization and intentions to seek professional help and services (Alang, 2019;Hunt et al., 2015;Rosenthal & Wilson, 2008;Thomeer et al., 2022).Given the distinct cultural norms and beliefs between people of color and White individuals, racial/ethnic minorities, such as Asian, Black/African, and Latinx Americans, are more inclined to seek help from social support networks, such as family, friends, tight-knit communities, and spiritual/religious communities (Balkin et al., 2022;Crowe et al., 2022;Han & Pong, 2015).During the first year of this prevailing pandemic, social isolating preventive measures (e.g., social distancing, shelter-in-place) might have reduced the ability of racial/ethnic minorities to access their traditional social support networks that served as a strong protective factor for mental health, and therefore, they might experience increased mental health problems (Thomeer et al., 2022).In light of racial disparities in mental health outcomes and protective factors for mental health, the pandemic became a perfect storm that exacerbated mental health inequalities (Shim & Compton, 2020).

Implications for public health policy and clinical practice
Implications emerged from the study findings.First, there was no evidence suggesting declines in counseling service-seeking intentions and utilization after mental health providers universally shifted to telehealth counseling around March 15, 2020.Given the benefits of telehealth, current policies tied to telehealth should continue beyond the pandemic.However, federal regulations and medical-necessity criteria that govern insurance policies can create financial barriers to telehealth for counseling clients in the post-pandemic era.In fact, on March 15, 2022, the Biden administration enacted the Consolidated Appropriations Act, 2022, including provisions that extended telehealth waivers and flexibilities for only an additional 151-day following the end of the federal public health emergency (PHE).Namely, insurance payers may discontinue coverage for telehealth counseling in the foreseeable future regardless of clients' preferences and clinical benefits of telehealth.As dates for ending PHEs could vary across different states, counselors should be aware of changing regulations and coverage guidelines to prepare their practices and clients for such changes.
Although the battle against COVID-19 called for a joint effort and a multi-faceted approach, the implementation of telehealth is a critical step forward toward improving access to timely mental health counseling and eliminating barriers to mental health promotion and prevention.Barriers to mental health care and services persist in many US regions, contributing to the disproportionately psychosocial effects of the COVID-19 pandemic on marginalized communities (Drake et al., 2019;Rodriguez et al., 2021;Zhai, 2021).Considering those clients who prefer telehealth counseling (Shklarski et al., 2021;Sugarman et al., 2021), federal and state officials may consider updating public health policies and allocating funds to address telehealth infrastructure disparities in certain including Health Professional Shortage Areas (North, 2020;Park et al., 2018), lacking public transportation (Beatty et al., 2022), larger swathes of rural areas (Drake et al., 2019;Rodriguez et al., 2021), and higher poverty rates (Park et al., 2018;Rodriguez et al., 2021).
Second, findings of this study highlighted racial disparities in mental health outcomes and service-seeking intentions.The COVID-19 pandemic poses an uneven threat to marginalized communities, and racial/ethnic minorities have been at elevated risk for mental health problems (Thomeer et al., 2022).Despite the greater prevalence of mental health issues found in racial/ethnic minorities, they are less likely than their White counterparts to receive counseling services and express intentions to seek mental health care (Alang, 2019;Hunt et al., 2015;Shannon et al., 2022).The expectation of experiencing discrimination based on race and stigmatization associated with mental illness and mental health services creates barriers to timely care for racial/ethnic minorities (Alang, 2019;Shannon et al., 2022).Members from racial/ethnic minority communities experience racial microaggressions that penetrate healthcare settings, such as racist assumptions and stereotypes against the experiences of people of color (Alang, 2019;Shannon et al., 2022).Subsequently, systemic racism distresses racial/ethnic minorities whose lack of trust in the health system may outweigh the benefits of health care, preventing racial/ethnic minorities from accessing mental health services.
Practicing counselors are trained to be multiculturalism and social justice advocates, therefore having the onus to bridge the service gap and address racial inequalities in mental health care (Clark et al., 2022).In light of ACA's ongoing efforts to address health disparities and racism, counselors and ACA may continue to help disseminate stigma reduction messages and rally to support racial/ethnic minorities by advocating for non-discrimination (ACA, 2022).Given that racially uneven mental health needs do not occur in vacuum, counselors should consider making collaborative efforts with other health professionals (e.g., social workers and case managers) and policymakers to address socioeconomic inequalities confronted by racial/ethnic minorities.Meanwhile, counselor educators and supervisors should integrate the framework of the multicultural and social justice counseling competencies into teaching and supervision to prepare practicing counselors for anti-racism and decolonization of counseling practice (Singh et al., 2020).

Limitations and future research directions
Limitations of this study include the use of self-reported data.Future research might use clinical data from medical records across diverse clinical settings (e.g., private practice, clinics, and hospitals) to increase the validity of a study.Second, the findings may not generalize beyond current participants who were emerging adults enrolled in higher education institutions.Older adults may not have adequate technological/digital literacy compared with emerging adults to use telehealth counseling, so they may be less inclined to seek counseling in this format.Nonetheless, the current body of literature has suggested good acceptability of telehealth across different age groups (Childs et al., 2021;Cuthbert et al., 2022).Future research should increase age diversity when recruiting participants.Third, we were only able to assess the 3 months following the policy change (i.e., the shift to telehealth with the 1135 waivers).There might be differences in counseling service-seeking intentions that were evident several more months after the implementation and shift to telehealth.Future studies using data collected after the study period may help examine the robustness of current study findings.Fourth, although we were able to control for unmeasured confounding effects and changes via a DiD design, this study was observational.Unobserved confounding factors, if any, might bias the results.Given that it is not plausible to replicate the real-world effect of COVID-19 in RCTs, and largescale RCTs are not common in practice (Wing et al., 2018), researchers may consider qualitative research approaches to deepen the understanding of potential confounding factors.Fifth, the SCOFF for detecting eating disorders only demonstrated marginally acceptable internal reliability (McDonald's omega; Kalkbrenner, 2021); however, high covariance amongst the few scale items was not expected because the sum of the SCOFF was essentially a count of the number of risk factors (i.e., scale items).Under this circumstance, ROC curve analysis is more suitable to evaluate the merit of the SCOFF, namely, the diagnostic performance of screening tests, as the SCOFF differentiated respondents based on the number of risk factors marked as opposed to a total score analyzed for internal reliability (Power et al., 2013).Thus, this issue does not appear to undermine the merit of the SCOFF when this measurement demonstrated adequate diagnostic accuracy across clinical settings (Feltner et al., 2022).Last, lower numbers of participants in the client group (vs. the nonclient group) might have led to DiD models with relatively 15566676, 2023, 3, Downloaded from https://onlinelibrary.wiley.com/doi/10.1002/jcad.12468,Wiley Online Library on [27/11/2023].See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions)on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License inadequate power to detect changes in clients' counseling service-seeking intentions.Despite these limitations, our study was one of the first to investigate the effects of the policy change regarding the shift to telehealth in a large national sample.
Results from this current study support the utility of telehealth counseling with respect to clients' intentions to continuously seek counseling services.Future outcome research is needed to assess therapeutic outcomes between in-person and telehealth counseling.Building on an emerging body of research that has provided preliminary support for the effectiveness of telehealth counseling (Hall et al., 2022;Nauphal et al., 2021), researchers could conduct single-/multicentered RCTs (with at least two active [non-placebo] intervention arms) to examine the efficacy of specific counseling/therapeutic interventions between in-person and telehealth counseling.To test the real-world effectiveness of these interventions among delivery formats, researchers should consider comparative effectiveness research to analyze real-world data in order to provide real-world evidence to support clinical trials and to further inform clinical practices and regulatory decisions (The U.S. Food and Drug Administration [FDA], 2018).To this end, researchers may consider using quasi-experimental alternatives to RCT, such as DiD and interrupted time series analysis, to evaluate treatment outcomes by analyzing real-world data in counseling research.This present study shed light on the use of DiD to examine whether there were changes in outcomes that occurred due to interventions.Researchers may also examine the effect of interventions on changes in client outcomes via a single-armed interrupted time series analysis wherein participants served as their own control (Matowe et al., 2003;Warton, 2020).With more research approaches attained, future investigators are poised to conduct outcome studies that inform public policies and advance the counseling profession (Watson et al., 2021).

CONCLUSION
In this study, the shift to telehealth under the policy expansion of telehealth with 1135 waivers responding to the national emergency concerning COVID-19 was not associated with a change in counseling service-seeking intentions among clients receiving counseling; thus, the findings provided support for the utility of telehealth counseling.The implications of these results should be considered when evaluating the merit of the shift to telehealth counseling to inform public policies and counseling practice in the future.The results also underscored racial/ethnic disparities in mental health, counseling utilization, and counseling service-seeking intentions before and during the COVID-19 pandemic, contributing to the current literature on the disproportionately negative mental health impact of the pandemic on racial/ethnic minorities.

A C K N O W L E D G M E N T S
We thank the Healthy Minds Network team for providing access to data.

C O N F L I C T O F I N T E R E S T S TAT E M E N T
We declare no competing interest.

E T H I C S S TAT E M E N T
The current study  was approved by the University of Alabama at Birmingham Institutional Review Board.

F
I G U R E 1 Percentage of individuals with intentions of seeking counseling services before the policy change from common trends assumption test.

F
I G U R E 2 Difference-in-difference (DiD) estimates from dynamic event study model for pooled DiD for common trends assumption test.Note: The first survey wave (December, 2019 to March, 2020) before the policy change served as the reference.

F I G U R E 3
Percentage of individuals with intentions of seeking services before and after the shift to telehealth under the 1135 waivers.Note: The graph displays the percentage of individuals with intentions of seeking counseling services before and after the shift to telehealth with the 1135 waivers.They depict adjusted probabilities from the sample-weighted multivariable logistic regression model.The model was adjusted for race/ethnicity, gender, sex, and socioeconomic status.
TA B L E 1Abbreviation: SES, socioeconomic status.