Changes in otolaryngology application requirements and match outcomes: Are we doing any better?

Abstract Objectives Otolaryngology‐specific requirements were piloted to minimize applicant and program burdens. We investigated the impact of introducing and then removing these requirements on Match outcomes. Methods 2014–2021 National Resident Matching Program® data were examined. The primary outcome was the impact of Otolaryngology Resident Talent Assessment (ORTA; prematch 2017, postmatch 2019) and Program‐Specific Paragraph (PSP; implemented 2016, optional 2018) on applicant numbers and match rates. Secondary survey analysis assessed candidate perceptions of PSP/ORTA. Results Applicant numbers declined significantly during PSP/ORTA (18.9%; p = 0.001). With the optional PSP and postmatch ORTA, applicant numbers increased significantly (39.0%; p = 0.002). Examined individually, mandatory PSP was associated with a significant decline in applicants (p = 0.007), whereas postmatch ORTA was associated with significant increases in applicants (p = 0.010). ORTA and PSP negatively impacted the decision to apply to otolaryngology in 59.8% and 51.3% of applicants, respectively. Conversely, match rate success improved significantly from 74.8% to 91.2% during PSP/ORTA (p = 0.014), followed by a significant decline to 73.1% after PSP was made optional and ORTA moved to postmatch (p = 0.002). Conclusions ORTA and PSP correlated with decreased applicant numbers and increased match rate success. As programs seek ways to remove barriers to applying to otolaryngology, the potential consequences of an increasing pool of unmatched candidates must also be considered.


INTRODUCTION
Since its transition from an "early match" coordinated by the San Francisco match to the Main Residency Match ® ("the Match") in 2006, otolaryngology has remained one of the most competitive specialties in medicine. 1,2 The otolaryngology residency application and selection process is plagued by hyperinflation, wherein applications far outnumber available positions. Among all specialties, otolaryngology has the second highest ratio of graduating medical students ranking it first in the Match compared to available positions in that specialty (ratio: 1.18), second only to plastic surgery (ratio: 1.27). 3 Under such supply-demand discord, a proportion of graduating medical students risk an unsuccessful match, perpetuating the reputation that matching into otolaryngology is "impossible" or "near-impossible." 4 Previous studies evaluating the otolaryngology Match have attributed its competitiveness to a complex interplay between applicant factors (e.g., number of programs applicants applied to) and program factors (e.g., United States Medical Licensing ExamⓇ [USMLE] score and Alpha Omega Alpha [AOA] status screening, research requirements). [4][5][6] In the context of these factors, between 2007 and 2016 otolaryngology applicants' mean USMLE Step 1 scores increased by 10 points (average score 248 in 2016); percent AOA membership increased by more than 5%; and the average number of abstracts, presentations, and publications per applicant more than doubled. 5,7 Fueled by shotgun approaches to applying, there has been a 250% increase in the mean number of applicationsper-candidate over the last two decades. 7,8 In the 2015 Match cycle, the Otolaryngology Program Director Organization (OPDO) required applicants to write a separate paragraph for each program (the Program-Specific Paragraph or PSP) as a medium for candidates to express their specific interest in a program and reduce the number of applications submitted per applicant, thereby improving match rate success. 9,10 In the subsequent application year, the Otolaryngology Resident Talent Assessment (ORTA) was implemented as a concurrent prerequisite. The ORTA is a structured, telephone-based interview developed to assess noncognitive attributes of applicants that are not systematically evaluated through traditional requirements, such as USMLE board exam scores, AOA membership, and letters of recommendation. The ORTA was intended to yield psychometric-based predictions regarding which applicants would excel as otolaryngologists. 11 Since the PSP and ORTA were enacted in 2015 and 2016, respectively, their implementation and characteristics have evolved: the PSP became optional in 2018, and starting in 2019 the ORTA was conducted postmatch. While the PSP and ORTA have been suggested to contribute to declining applicant numbers (Figure 1), the implications of these interventions have not been thoroughly investigated. In this study, we sought to evaluate the impact of introducing and then removing these prematch requirements on Match outcomes between the years 2014-2021. We hypothesized that the introduction of the prematch PSP and ORTA led to a decline in applicant numbers and that medical students perceived the PSP and ORTA as barriers to otolaryngology, contributing to the downward trend observed in applicant numbers. A secondary survey was designed to assess otolaryngology resident perceptions on the impact of PSP, ORTA phone interview, reputation that it is difficult to match into otolaryngology, number of medical school classmates applying to otolaryngology, and program director (PD) advice on the decision to apply to otolaryngology. In addition, otolaryngology residents were asked about their perceptions of these factors on medical school classmates that considered otolaryngology but applied to another specialty instead ("nonotolaryngology"). Following Institutional Review Board approval, the survey was circulated to all otolaryngology PDs for distribution to

Applicant perceptions
A total of 118 of 610 (19.3%) otolaryngology residents participated in the survey. Among residents who pursued otolaryngology, 51.3% (n = 58/ 113) regarded the PSP as a negative influence on the decision to apply to otolaryngology, of which 9.7% (n = 11) qualified the PSP as a major negative influence ( Figure 2). The ORTA phone interview was regarded as a negative influence in 59.8% (n = 64/107) of otolaryngology residents, of which 19.6% (n = 21) qualified it as a major negative influence ( Figure 3).
For the classmates who considered otolaryngology but applied to a different specialty, otolaryngology residents estimated that the PSP and ORTA as major negative influences (Figures 2 and 3). Comparing the impact of the PSP and ORTA on otolaryngology residents and the estimated impact on the medical school classmates that considered otolaryngology but did not apply, the PSP impact was similar whereas the ORTA interview was estimated to have a more negative influence on otolaryngology residents than on those who ultimately applied to a different specialty (p = 0.050).
The reputation that it is difficult to match in otolaryngology was viewed as a negative influence among 45.2% (n = 52/115) of otolaryngology residents, of which 12.2% (n = 14) qualified it as a major negative influence (Figure 4). Reputation was estimated to be a negative influence in 78.9% (n = 82/104) of medical students who ultimately applied to a different specialty. Reputation had a stronger negative influence on medical school classmates that considered otolaryngology but did not apply compared to otolaryngology residents (p < 0.001).
Advice from PDs was viewed as a negative influence in 6.5% (n = 7/107) of residents who matched into otolaryngology, and 31.4% (n = 27/86) of those who applied to a different specialty ( Figure 5 5,13,14 Although otolaryngology applicants are high-achieving in each of these domains, more than 90% of programs report having to remediate residents due to unprofessional behavior, insufficient medical knowledge, or poor clinical judgment. 15 Alternatively, qualities that otolaryngologists do highly value, such as integrity, empathy, and surgical dexterity, are not captured by these academic metrics. 3 Perceived competitiveness motivates candidates to submit large numbers of applications as a mechanism to increase the likelihood of a successful match. 16 As a result, over the past two decades, the mean number of applications-per-candidate for otolaryngology has increased by nearly 250%. 6 Among 150 otolaryngology residents surveyed, 90.6% acknowledged applying to programs in which they had no specific interest to improve their chances of matching. 6 Programs inundated by these application numbers are left grappling to understand candidates' genuine interest in specific programs.
In response to candidates' shotgun approaches to the Match, limitations on the number of applications-per-candidate to between 10 and 20 programs have been recommended. 16  there is currently no method available to limit application numbers. 8,17 Beyond instituting a limitation on applications-per-candidate, numerous proposals have been made to improve the otolaryngology residency application and selection process. 9 A preference signaling system piloted in 2018 was successfully implemented in the 2021 otolaryngology Match, and will be continued in otolaryngology and appended to dermatology, general surgery, and internal medicine in the 2022 Match cycle. 18,19 Named "the Star System," this approach provides each applicant a predetermined number of "stars" or "signals" to send to programs of particular interest. 9,18 This enables applicants to easily and transparently indicate interest in a select few programs and addresses the current system that leaves programs grappling to understand candidates' genuine interests. Another signaling approach known as the Consortia Match utilizes a hybrid early-and conventional-match system in which residency programs are grouped into "baskets" based upon qualities, including program caliber, reputation, and geography, and applicants are limited to one program "basket" in the early match. 1,9 By limiting the number of The third change, a coordinated interview invite release date, was suggested by the OPDO in the 2021 Match cycle, to decrease applicant stress and anxiety surrounding interview scheduling and to improve the efficiency of the interview scheduling process for programs. 26 Programs were also advised by the OPDO to limit their number of interview invitations to the number of interview slots available. 26 However, these changes were merely suggestions, not mandates, and thus were not universally implemented. 30,31 Data regarding how many programs complied with these OPDO recommendations is limited given their recency, thus the efficacy of such interventions is yet to be formally investigated.
The otolaryngology residency application process is clearly in flux and has adapted well throughout the phases of the COVID-19 pandemic. From this analysis of otolaryngology match trends and applicant perspectives, and an extensive review of Match process reform during the pandemic, there appears value in adopting strategies involving preference signaling and interview caps. Preference signals enable applicants to demonstrate an interest in a select few programs and encourage programs to holistically review candidates that may not have been as thoroughly considered otherwise. 25 As only a small minority of applicants receive a large portion of interview offers, yet few complete more than 20 of those interviews, 24 an interview cap would decrease interview hoarding to improve match rate success and optimize the alignment of program and applicant preferences.
In addition to application and selection process reform, pregraduate curriculum development and otolaryngology exposure and mentorship early on in medical school, must be considered.
Opportunities such as shadowing, resident mentorship, and interest group involvement 32 allow a greater breadth of students to explore otolaryngology as a specialty and enable departments to identify who would be "best-fit" for the specialty. Decreasing or supplementing the emphasis on scholastic achievements in lieu of more holistic or

CONCLUSIONS
With recent changes in the USMLE Step 1 scoring system from numerical scores to pass/fail, programs may be compelled to seek new ways to differentiate applicants. However, programs must bear in mind the consequence(s) of deterring applicants when implementing any new requirement(s) for residency applications.
Our study suggests that the PSP and ORTA are perceived as barriers to applying to otolaryngology and were associated with

ACKNOWLEDGMENT
There is no funding and acknowledgment for this study.

CONFLICT OF INTEREST
The authors declare no conflicts of interest.

DATA AVAILABILITY STATEMENT
Annual National Resident Matching Program ® data are available from Charting Outcomes in the Match available at https://www.nrmp.org/ match-data-analytics/residency-data-reports/. Postgraduate year 1/ 2 otolaryngology resident survey data are available from the corresponding author upon reasonable request.

ETHICS STATEMENT
The above manuscript is the author's own original work, which has not been previously published and is not in submission elsewhere.
The paper reflects the author's own research and analysis in a truthful and complete manner and properly credits the meaningful contributions of coauthors and coresearchers.