Research productivity of radiation therapy physics faculty in the United States

Abstract Purpose Research productivity metrics are important for decisions regarding hiring, retention, and promotion in academic medicine, and these metrics can vary widely among different disciplines. This article examines productivity metrics for radiation therapy physicists (RTP) in the United States. Methods and materials Database searches were performed for RTP faculty at US institutions that have RTP residencies accredited by the Commission on Accreditation of Medical Physics Education Programs (CAMPEP). Demographics, academic rank, number of publications, academic career length, Hirsch index (h‐index), m‐quotient, and history of National Institutes of Health (NIH) funding as a principal investigator (PI) were collected for each RTP. Logistic regression was performed to determine the probability of academic rank as a function of h‐index and m‐quotient. Statistical tests used included the Wilcoxon ranked sum test and the Pearson χ 2 test. Results A total of 1038 faculty and staff were identified at 78 institutions with CAMPEP‐accredited residencies. The average RTP academic career duration is 13.5 years, with 46.7 total publications, h‐index of 10.7, and m‐quotient of 0.66. Additionally, 10.5% of RTP have a history of NIH funding as a PI. Large disparities were found in academic productivity of doctoral‐prepared physicists compared to those with a terminal master's degree. For differences in junior and senior faculty, statistical tests yielded significance in career duration, number of publications, h‐index, and m‐quotient. Gender disparities were identified in the overall distribution of RTP consistent with the membership of the American Association of Physicists in Medicine. Further gender disparities were found in the number of doctoral‐prepared RTP and physicists in senior faculty roles. Conclusions This manuscript provides objective benchmark data regarding research productivity of academic RTP. These data may be of interest to faculty preparing for promotion, and also to institutional leadership.


INTRODUCTION
skewed by a single major publication. Alternatively, the Hirsch index (h-index) quantifies both the productivity and citation impact of a scientist's publications. 5 Simply put, an author's h-index is defined as the author having published h papers that have each been cited at least h times. 5,6 The h-index can vary widely across academic disciplines; smaller scientific disciplines have fewer scientists, fewer publications, fewer citations, and correspondingly, lower h-indices than larger disciplines. Furthermore, clinical scientists often have clinical duties in addition to fulfillment of their academic mission, and these clinical scientists typically have lower h-indices than those in pure science who may have primarily research and teaching responsibilities. It could be inequitable to compare academic output for faculty in clinical science to metrics for faculty in pure science, and therefore, it is important to understand benchmarks for specific subfields before using these metrics to quantify an individual's academic productivity. Hirsch noted that for researchers in traditional subfields of physics a value for h of 12 is typical for promotion to associate professor at major US research universities, and a value of 18 is typical for promotion to full professor. 5 Multiple studies have been performed examining bibliometric benchmarks for various medical specialties. 4,[7][8][9] A meta-analysis was performed in 2020 that included data on 14 567 academic physicians. 10 In this study, it was shown that the h-index increases with academic rank with a mean h-index of 5.2 for assistant professors, 11.2 for associate professors, and 20.8 for full professors. Of particular interest, several studies have examined academic productivity benchmarks for academic radiation oncology departments. 1,2,11 The most recent study found that for academic radiation oncologists in 2016 the mean h-index was 14.5, and h-index greater than 21 was correlated with senior faculty status versus junior. 11 While the h-index is an important objective metric, it has limitations and there are other metrics that can help quantify academic output. For example, the hindex increases as citations accumulate during a career, and thus depends on a researcher's career length. The m-quotient attempts to weigh the length of academic career and is defined as the h-index divided by the number of years since the researcher's first publication. 11 m-quotient may allow better comparison of researchers across time,as h-index is greatly affected by career length and not necessarily rate of academic productivity. 12 Higher m-quotient has been weakly correlated with senior faculty status for academic radiation oncologists, 11 but the m-quotient could still be an important indicator for promotion of mid-career faculty. Another important metric of productivity in academic medicine is the history of extramural funding from the National Institutes of Health (NIH) as a principal investigator (PI). 13 In this manuscript, we provide an analysis of the academic output of faculty from accredited radiation therapy physics (RTP) residency programs in the United States in 2020, using the total number of publications, h-index, m-quotient, and a history of being an NIH funded PI as metrics of research productivity.

Departmental and faculty inclusion criteria
Departments with RTP residencies listed on the Commission on Accreditation of Medical Physics Education Programs (CAMPEP) website were included for analysis (https://www.campep.org/campeplstres.asp). The available departmental websites were accessed in December 2020 to determine the gender, highest academic degree, and academic rank or title of each RTP listed. Not all programs explicitly separate clinical faculty from research faculty or faculty from staff, so all RTP staff listed on departmental websites were included in this analysis. This included all RTP with either a doctorate or terminal master's degree. Designations as full professor, chairperson, chief, or director were classified as senior faculty. Associate professors, assistant professors, and clinical instructors as well as other faculty titles not fitting the previously mentioned categories were classified as junior faculty. While not necessarily a standard definition of junior faculty, this stratification was made for easy comparison to previously published metrics for radiation oncologists. 11 No distinction was made between clinical, research, or tenure-track faculty since these designations are not always easily determined from departmental websites.

Academic productivity data collection
For each RTP, the Scopus database (Elsevier BV, Amsterdam, The Netherlands) was queried to determine the h-index, total number of publications, and year of first publication. Scopus currently has over 76 million records from journals, book series, and conference materials that have an International Standard Serial Number assigned to them. Because many conference abstracts are published in journals, the total publication count includes some peer-reviewed abstracts. While the number of published conference abstracts is not indicative of research productivity, these cannot be easily filtered from the data. It also includes articles in press from more than 3850 journals. [14][15][16] Therefore, the total publication number may seem high compared to a particular author's list of peer-reviewed publications. The h-index is not generally affected by the inclusion of TA B L E 1 Characteristics of RTP at CAMPEP-accredited residency programs

Statistical analysis
Descriptive statistics were calculated, including mean, SD, median, and interquartile range. h-index and mquotient by academic rank and individual characteristics were also analyzed. Logistic regression analysis was used to generate predictive probability curves for professorial faculty ranks as a function of h-index. The quality of the final regression was assessed for discrimination using the area under the receiver-operatorcharacteristic (ROC) curve (AUC). This analysis of the ROC curve serves as the model validation. 17 The hindex and m-quotient data were not normally distributed; therefore, the Wilcoxon rank sum test with continuity correction was used for comparison of continuous variables and the Pearson χ 2 test was used for categorical variables. For family-wise error rate correction, the Bonferroni method was applied. This method, known for its conservative approach, would deter any false positives for trends affecting faculty promotion. 18 Although a single comparison study may consider p-values less than 5% significant, this threshold was adjusted to 0.3% given 15 comparisons over the course of the study.

RESULTS
A total of 1038 RTP were identified at 78 institutions with CAMPEP-accredited residencies.  Table 2 provides descriptive statistics for academic productivity metrics as a function of academic rank. The academic ranks recognized were instructor (or lecturer), assistant professor (or assistant faculty), associate professor (or associate faculty), professor, and director (or chair or chief). Any other designation was placed into a category of "no traditional academic rank."

Academic productivity characteristics
All academic productivity metrics increase with increasing academic rank. On average, over a 7.7-year academic career, instructors have produced 7. Additionally, 42.1% of full professors have been an NIHfunded PI, while only 13.6% of associate professors and 3% of assistant professors have a history of NIH funding as a PI. For those in director positions, the average career duration, h-index, m-quotient, and percentage with NIH funding as a PI are all slightly lower than those for full professors. Figure 1 shows predictive probability curves for faculty ranks of assistant, associate, and full professor, with the h-index as the independent variable. Figure 2 shows similar curves with the m-quotient as the independent variable. Table 3 summarizes the parameters used for the logistic regression. P, the probability of achieving a minimum rank of the tested professorship level follows  the equation: where C 2 is the index coefficient and C 1 describes the intercept. The increase in probability of a particular professorial designation per unit increase in the covariate, in this case the h-index or m-quotient, can be calculated from: The values of the h-index and m-quotient that maximize the AUC in an ROC analysis were determined. Using this methodology, the h-index values that maximize AUC for designation as an assistant professor, associate professor, or full professor are 5.5, 9.7, and 16.8, respectively, and the corresponding m-quotient values are 0.46, 0.67, and 0.77. While these values cannot be considered cutoff values for specific academic designations, the h-index values in this model for associate and full professor have 81%-82% predictive accuracy. Table 4 summarizes research productivity of the top 10% of medical physics academic faculty ranked by h-index. This allows comparison with metrics previously published for radiation oncologists. 2 The top 111 academic producers had an h-index of 25 or higher. The mean h-index in the group was 36.2,and the highest was 91. Additionally, 53.2% of the top 10% ranked by h-index have been an NIH-funded PI. The p-values reported are for the comparison of junior to senior faculty for each parameter.

Characteristics of RTPs with the highest h-indices
The group was separated into junior and senior faculty. Of the top 10%, 88 are senior faculty and 33 are junior. All metrics are statistically higher for the senior faculty group than for the junior faculty group, except for gender and the m-quotient, which is the same for both groups. There are possible explanations for the seemingly large number of junior faculty in the top 10%. Firstly, four of these researchers were in medical physics as a second career, and much of their academic output was in a field other than therapy physics. These individuals have very high h-indices, but their academic rank is either assistant or associate professor. Additionally, one researcher is at an institution that does not have traditional faculty titles and was placed in the junior faculty group. Similar trends are evident when one examines the characteristics of the top quartile by h-index (Table 5).
Additionally, it is seen that there is a large disparity in the history of NIH funding between the senior and junior faculty in this cohort, with 68% of senior faculty having been an NIH-funded PI compared to only 18% of junior faculty. These results compare favorably with previously published data showing increases in NIH funding for older AAPM members. 20 While disparities exist, a significant correlation between NIH PI status and faculty promotion was not evident.

Gender characteristics of senior faculty
We further examined the gender characteristics of senior faculty (Table 6). Of the 172 senior RTP, 144 are male (83.7%) and 28 are female (16.3%). This is statistically different than the overall composition of RTP faculty, which is approximately 25% female. The h-index was slightly higher for male senior faculty and approached statistical significance. The history of having been an NIH-funded PI was statistically higher for male faculty. These results corroborate the comprehensive study on NIH research funding for AAPM members by Whelan et al., showing that the number of grants awarded to different genders within AAPM are disparate,

F I G U R E 4 Predictive probability of faculty ranks as a function of Google Scholar Hirsch index
with 11% held by females in a 22% female population. 20 Similarly, our study found that 13.4% of female PIs held NIH grants in a 24.7% female population.

Comparison of h-index in GS and Scopus
While GS is a convenient tool for literature searches and determining citation impact, individual investigators need to have created a GS page to determine their per-sonal bibliometric metrics. GS pages were found for only 214 of the 1038 RTP in this study.The GS h-index is consistently higher than the Scopus h-index; linear regression finds that the GS h-index is 1.27 times higher than the Scopus h-index (Figure 3). It has been shown that GS overestimates citations due to inclusion of non-peerreviewed publications and duplicate records. [21][22][23][24][25] It can still be a useful tool for gauging academic productivity when used with the appropriate benchmarks, and Figure 4 shows the predictive probability for faculty ranks as a function of the GS h-index. Further comparisons are discussed in the work by Bar-Ilan. 26 For the purposes of our study, we focused on Scopus h-indices because of its availability for all published researchers.

DISCUSSION
Many factors are considered in hiring, tenure, and promotion decisions, including clinical proficiency, teaching evaluations, institutional service, service to the profession, and academic achievement. When assessing research achievement alone, many faculty contributions are considered, including extramural funding, scientific presentations, organizing and hosting workshops, participation in clinical trials, task group membership, and publication record. Historically, a researcher's total number of publications and/or citations has been used to assess the publication record. The h-index has become increasingly recognized as an objective indicator of academic productivity, since it provides a more complete overview of the author's publication record and overall research impact. 1,2,7,11 Additional metrics have also been used to supplement the h-index. In this study, increased h-index, m-quotient, and career duration, along with a history of NIH funding as a PI, were all associated with increased academic rank for RTP associated with CAMPEP-accredited residency programs. Academic department directors have a slightly lower average h-index than full professors do. There are many possible reasons for this, including the possibility that administrative duties displace research activities, leading to lower research output. Over 13% of RTPs associated with CAMPEPaccredited programs have published zero publications. The majority of these were physicists with terminal master's degrees, who are often in primarily clinical roles. Conversely, among the top 10% of RTP ranked by hindex, all were doctoral-level physicists and 88 (79%) were senior faculty. A gender discrepancy was found, with just 15% of these top RTPs (ranked by h-index) being women.
An interesting point to note is that the h-indices for RTP tend to be lower than those for physician faculty in radiation oncology.  27 Having a history of being a PI with the NIH, however, is similar for senior medical physics faculty in comparison to physician colleagues in radiation oncology; 42.1% of RTP full professors and 41.6% of RTP directors have been PIs on NIH grants, compared to 32.3% of radiation oncology professors and 38.7% of department chairs. Outside of the most senior RTP, however, NIH funding is rare enough to be considered an anomaly, with only 13.6% of RTP associate professors having served as PIs. The relative scarcity of RTPs successfully competing for NIH funding makes this a questionable metric for assessing RTP. The data are included here, however, because obtaining extramural funding, and NIH funding specifically, is considered in promotion and tenure decisions at some academic institutions.
Gender disparities have been found to exist in many medical specialties, 11,19,[28][29][30][31] and our study showed similar results with men having longer career durations, more publications, and higher h-indices than women. However, in our study these results are somewhat skewed because of a disproportionate number of women who have terminal master's degrees and who are nonpublishers. While it still exists, the gender disparity in academic productivity is much less when considering only senior faculty, and the disparity is even less when considering only the top 10% and top quartile of academic producers using the h-index. However, the large gender disparity in RTP leadership positions seen in other studies 19 was also evident in our study. Further study is warranted to better understand the reasons for these disparities and develop strategies to increase the gender diversity in the RTP workforce and leadership. 32 Our study includes a large and comprehensive sample size of academic RTP and provides important benchmarking data for RTP. There are limitations, however. We relied on the CAMPEP-listed departmental websites to determine RTP affiliation and title. If these are not updated regularly, the results could be incomplete or incorrect. Throughout the course of data collection, separate staff performed several spot checks to ensure the accuracy of the data. Furthermore, RTP represents a very heterogeneous population with great diversity in terminal degree and responsibilities. Many RTPs, even at academic institutions, focus on clinical service and teaching, with little or no responsibility for the research mission of their institution. Since departmental websites do not necessarily delineate responsibilities, however, we made no attempt to stratify staff members into different bins such as tenure track, research track, clinical track, or staff. Including those with no research responsibilities skews the analysis downward for the overall population, However, any physicist who had no academic title listed was grouped into the "no rank" and "junior" categories in our analysis. Hence, this analysis provides valid benchmarks for those with traditional academic titles and those classified as senior faculty. We relied primarily on Scopus and the NIH online reporting tools for the data collection, and there are potential errors, including the potential to incorrectly attribute a researcher's work to another individual with the same name. We also did not correct for the potential for name changes related to maiden name, married name, nicknames, or initials. These inconsistencies could also affect the results, and many of these can disproportionately affect women. For example, one study showed that female faculty in radiation oncology often published under more than one last name, and this affected their h-index in 8% of cases. 31 Scopus is the largest curated citation database, and has consistently improved its coverage and author profiles. [14][15][16] It is reasonable to expect that bibliometric databases will continue to improve. Furthermore, it is recommended that individual authors review their profiles to make sure that they are accurate and complete. This will additionally improve the accuracy of the databases and any future benchmarks generated from these databases. There are also limitations to the academic metrics studied. For example, the h-index grows over time, and faculty early in their career will have a low h-index. The m-quotient was used to account for this, but it is a much weaker predictor of academic success than the h-index. Viewing Table 3, one may note that the accuracies for the h-index logistic model predicting the ranks of associate and full professorship are higher than the accuracies for the m-index logistic model. There is confidence in this assessment because the SDs do not overlap, with a minimum expected advantage of 0.05 and 0.11 in predictive power for the h-index model. Additionally, there is error introduced by using time from first publication to determine career duration, as many RTP have published papers during their training. This affects both the career duration and m-quotient.

CONCLUSION
This report compiles objective metrics describing research productivity for RTPs associated with CAMPEP-accredited residency programs. These metrics include total number of publications, the h-index, m-quotient, and history of obtaining NIH funding as a PI. While these data alone cannot be used to make any personnel decisions, they can be used as an easily obtained, quantifiable, and objective benchmark describing a small portion of an academic RTP's overall productivity. To our knowledge, this is the first study to compile these data, which may be of interest to faculty preparing for promotion, and also to institutional leadership.