Scholarly Productivity and National Institutes of Health Funding of Foundation for Anesthesia Education and Research Grant Recipients: Insights from a Bibliometric Analysis

The American Society of Anesthesiologists established the Foundation for Anesthesia Education and Research (FAER) with the goal of facilitating “continuous improvements in anesthesiology by fostering and encouraging education, research, and scientific progress in the field.”¹  The FAER grant program provides fellows and junior faculty members interested in pursuing basic science, clinical, translational, or educational research with “seed money” as they begin their academic careers with the expectation that many of the recipients will develop into productive independent investigators capable of obtaining competitive awards from the National Institutes of Health (NIH). FAER grants were initially funded through donations provided by the Burroughs Wellcome and Parker B. Frances foundations before the FAER endowment fund was established. The American Society of Anesthesiologist and its state component societies, 9 subspecialty anesthesiology societies, 15 anesthesiology-related private corporations, dozens of academic anesthesiology departments and group practices, and several hundred individuals are the major donors to the FAER endowment.²  According to the organization’s 2013/14 Annual Report and its website,^1,2  currently, the FAER endowment holds assets of $24.88 million and has awarded more than $31 million in research and educational grants to date. The subsequent success of FAER grant recipients has been anecdotally described, but the influence of FAER funding on scholarly productivity and the recipients’ ability to earn NIH grants has not been systematically quantified. Accordingly, we conducted a bibliometric analysis of all recipients of FAER grants since the organization began funding research in 1987. We quantified scholarly output using h-index, a bibliometric statistic that has been used extensively to describe productivity in academic anesthesiology^3–12  and other medical specialties.^13–22  The h-index is defined as the number of an investigator’s publications that have been cited at least h times and, despite its well-documented limitations,^8,21  is an established indicator of the relative strength and consistency of an investigator’s collective work based on the assumption that publications of less overall value are not cited as frequently.^20,23–25  We also quantified the ability of FAER grant recipients to obtain NIH funding by identifying the number and types of NIH grants, the years of funding, and the total amounts of these awards. We tested the hypothesis that FAER grant recipients remain in academic anesthesiology, become productive scholars, and are successful at obtaining extramural funding from the NIH.

All data were collected between December 2014 and January 2015. FAER grant recipients were identified in the program alumni database.²⁶  FAER research grants included the “Cerebral Function Monitoring Study,” “Clinical Research Starter Grant,” “Fellowship Award Grant,” “GEMSSTAR NIH Aging R03 Award,” “Mentored Research Training Grant” (Basic Science, Clinical Research, and Clinical/Translational Research), “New Investigator Award” (Basic Science and Clinical Research), “Research Fellowship Grant,” “Research Starter Grant,” “Research Training Grant,” and “Young Investigator Award.” FAER education grants included the “Education Grant” and the “Research in Education Grant.” Participants in the FAER “Resident Scholar” program were excluded from the analysis unless these individuals were subsequently awarded a FAER grant. The current academic (including faculty rank), private practice, or industry affiliation of each FAER grant recipient was identified using an Internet search combined with the keyword “anesthesiology.”²⁷  Academic anesthesiology practice was defined as a full-time appointment as noted on the corresponding department’s website. The duration of scholarly activity, number of publications, publications per year, number of citations, citations per publication, and h-index for each FAER grant recipient were obtained using the Scopus^® database.²⁸  The number of publications was verified using PubMed to reduce possible inaccuracies in h-index values.²⁹  The m-index (rate of increase of h-index) was calculated from these data as the ratio of h-index to the years of scholarly activity.³⁰  Each FAER grant recipient’s history of NIH funding was defined using NIH Research Portfolio Online Reporting Tools^® (NIH RePORTER; National Institutes of Health, USA).³¹  The number and types of NIH grants (mentored basic or clinical scientist development awards [K-series such as K01, K08, and K23], research grants [R-series including R01, R03, R21, and R29], research training grants [T32], and program project grants [P01]), the years of grant funding, and the amount of grant awards were identified for each FAER grant recipient as the principle investigator. Affiliation history and primary research interests in the health sciences were used to distinguish grant recipients with similar names.

Categorical variables are presented as numbers with percentages. Continuous variables are expressed as median (interquartile range [range]) because they are not normally distributed (Kolmogorov–Smirnov test). Comparison of continuous variables was performed using the Mann–Whitney U test for two independent samples or the Kruskal-Wallis test for multiple independent samples where appropriate. The null hypothesis was rejected when the P value was less than 0.05. Statistical calculations were performed using StatPlus:macLE software (AnalystSoft, Canada).

Four hundred eight physicians received a total of 441 FAER grants. Thirty-one investigators were awarded two grants, and one individual received three grants. Eleven recipients (who received one FAER grant each) could not be uniquely identified in the Scopus^® database because of common names; these individuals were excluded from the analysis. Thus, a total of 397 recipients who received 430 FAER grants were evaluated. FAER grant recipients published 19,647 manuscripts that have been cited 548,563 times in the peer-reviewed literature (table 1). Recipients received 391 NIH grants as principle investigators, including 78 (to 73 individuals) and 291 (to 124 individuals) K- and R-series awards, respectively, while amassing a total of $448.44 million in support. Recipients also earned 12 program project grants and 10 research training grants (data not shown). Twenty institutions received nearly two thirds (65.9%) of FAER grants (table 2). Currently, three hundred fourteen FAER grant recipients hold academic anesthesiology appointments (79.1%; table 3); 83 (20.9%) now work in private practice (n =80) or industry (n = 3). Sixteen FAER recipients (0.4%) serve as Deans (n = 2) of U.S. medical schools or Chairpersons of academic departments (n = 14), whereas 121 (30.5%), 105 (26.5%), 68 (17.1%), and 4 (0.1%) recipients are Professors, Associate Professors, Assistant Professors, and Instructors, respectively (table 3). The number of publications, publication rate, total citations, h-index, and m-index were academic rank dependent (P < 0.0001). The number of R-series, but not K-series, NIH grants received, total NIH grants, years of funding, and amount of support were also dependent on faculty rank (P < 0.0001). The number of individuals receiving NIH support was also faculty rank dependent (e.g., 57.6% of Professors vs. 27.9% of Assistant Professors; table 3). When comparing years of academic activity independent of faculty rank, the number of publications, citations, citations per publication, h-index, number of R-series NIH grants, and total amount of NIH funding increased in a time-dependent manner between groups with less than or equal to 10, 11 to 20, and more than 20 yr of experience (table 4). Individuals with more experience were also more likely to have NIH funding (53.9% of those with >20 yr of activity) compared with their colleagues with less experience (13.8% in those with ≤10 years of activity; table 4). These data indicate that the productivity of FAER grant recipients consistently increases over time.

FAER grant recipients who acquired NIH grants (40.3% of the sample) had greater scholarly productivity than those who did not (59.7%; table 5). Of those FAER alumni who went on to earn NIH awards, scholarly output was dependent on the cumulative monetary value of NIH grants received (table 6). Of note, 25 FAER grant recipients, each of whom received more than $5 million from the NIH, amassed the majority of NIH financial support ($248.05 million). FAER recipients with more publications were also more likely to receive NIH funding than those with fewer publications (table 7). Individuals who received FAER research grants had higher scholarly output and were more likely to earn NIH funding compared with those who were awarded education grants (table 8). Indeed, the majority of NIH grants subsequently received by FAER grantees were awarded to individuals who received research rather than education grants. Women had fewer publications and citations than men (table 9). The h-index of women who received FAER grants was also lower than men concomitant with fewer years of scholarly activity, but m-index was similar between genders. There were no differences in the number of NIH grants, years of NIH funding, and the amount of funding between male and female FAER grant recipients (table 9). Notably, a smaller percentage of women than men individually received NIH funding (34.1 vs. 41.9%, respectively; table 9), but NIH-funded women were more likely to receive multiple awards than their male counterparts. No differences in scholarly output, h-index, and m-index were observed between FAER grant recipients with medical degrees alone compared with those who also had PhD degrees (table 10). Combined MD and PhD degree holders were more successful at obtaining NIH grants, had more years of funding, and received greater monetary support than those with medical degrees alone.

The 397 FAER grant recipients examined here established an impressive record of scholarly activity and successful NIH funding, producing nearly 20,000 peer-reviewed publications cited more than one-half million times and obtaining more than $440 million in NIH grants since the FAER program’s inception. More than three quarters of FAER grant recipients remain in academic anesthesiology, and many have become national leaders. Approximately two thirds of all FAER grants were awarded to fellows or early-career faculty members from 20 academic anesthesiology departments. These departments have established research programs with solid track records of consistent NIH funding,³²  thereby providing a suitable environment and experienced mentors to assist FAER grant recipients with their projects.

We did not formally compare FAER grant recipients with a matched cohort of anesthesiologists who did not receive a FAER award, but we⁵  and others¹²  previously reported that the median h-index of U.S. academic anesthesiologists ranges between 1 and 3. In contrast, the h-index of FAER alumni in the current study was 13 (7 to 22 [0 to 72]). This difference may be related to the preponderance of senior faculty members here compared with typical academic departments. We noted a disproportionate number of Professors and Associate Professors (57%) in the current sample of FAER alumni compared with our previous survey of 24 academic departments⁵  in which only 35% held these ranks. Thus, the median h-index observed here would expected to be higher than that observed in our previous survey⁵  that incorporated a larger percentage of junior faculty members because h-index is academic rank dependent.^8,21,33–36  Alternatively, our data suggest that receiving a FAER grant may be associated with accelerated advancement in rank resulting from greater scholarly output and acquisition of NIH funding. We did not examine the number of years required for promotion between successive ranks in our current and previous⁵  studies, but our results suggest that FAER grant recipients may be more productive than their peers at each rank, and greater productivity is strongly associated with academic promotion.^14,17,18,37  For example, we found that the h-index of Professors who received FAER grants was 24 (18 to 31 [3 to 72]) compared with 9 (5 to 15 [0 to 44]) in our previous investigation. Similarly, Professors in the current study had an average h-index of 25 ± 11 (mean ± SD), whereas those working in accredited adult cardiothoracic anesthesia fellowship training programs had an h-index of 14 ± 8.⁸  Another study of productivity in anesthesiology residency program directors further supports this contention. Professors and Associate Professors are responsible for 73% of these programs, yet the median h-index in this cohort was 3 (95% CI, 2 to 6).¹²  The major educational commitment required by residency program directors may limit this group’s scholarly output, but the recipients of FAER education grants alone (who presumably have similar interests in education) were more productive (h-index of 10 [5–18 {0–51}] vs. 3 [95% CI, 2–6]) than residency program directors.¹² 

Differences in productivity and NIH funding were also dependent on the duration of scholarly activity, as the number of publications, h-index, the number of R-series NIH grants, and the total value of NIH funding increased in a temporal manner. These results suggest that the career trajectories of many FAER grant recipients include consistent time-dependent scholarly output and NIH funding. Similar findings were described in otolaryngology.²⁵  However, the current study’s observational, retrospective design precludes establishing a cause-and-effect relationship between a FAER grant and subsequent scholarly productivity or NIH support. Despite this limitation, there is some evidence that such a relationship may be present. For example, an actuarial analysis of 2,784 mentored K-series NIH grant recipients predicted that approximately 42.5% of this cohort would earn a R01 award within 10 yr.³⁸  A postal survey of 589 mentored K-series NIH grant recipients also reported that each of these individuals was highly likely to obtain at least one R01 grant and publish more than 35 papers during his or her career.³⁹  However, a prospective study will be required to definitively establish whether FAER grant funding predicts later scholarly productivity or NIH support.

The FAER program’s success mirrors the achievements of a similar initiative in otolaryngology. The Centralized Otolaryngology Research Efforts (CORE) grant program was created to support the research efforts of early-career otolaryngology investigators and to educate them about the preparation of NIH grant applications.⁴⁰  Otolaryngologists who were awarded CORE grants had higher h-indices than their colleagues without funding,⁴¹  and 39.6% of CORE-funded otolaryngologists subsequently obtained K- or R-series NIH awards.²⁵  These findings are nearly identical to those observed in FAER grant recipients (40.3% obtained NIH funding after a FAER award). Not surprisingly, FAER alumni who earned NIH grants had greater scholarly output than those who did not, and conversely, those with more publications were more likely to secure NIH funding. Similar findings were observed when comparing CORE-funded otolaryngologists with or without NIH awards.²⁵  A strong link between NIH funding and scholarly productivity was also reported in other medical specialties,^42–45  but the relative impact of NIH awards on overall scholarly output may be less important than previously thought because such grants are highly competitive and alternative sources of funding may be available.⁴⁶ 

Our results provided insights about differences in scholarly output and NIH funding between men and women who received FAER grants. Overall, men had greater numbers of publications and higher h-indices than women, but men were also academically active for more years. Notably, the publication rate and m-index were similar between groups. These data are consistent with those of Eloy et al.,⁴⁷  who demonstrated that men and women had equivalent rates of productivity in a large survey of academic physicians. Other studies of academic anesthesiologists,¹¹  neurosurgeons,⁴⁸  otolaryngologists,²²  and radiation oncologists⁴⁹  showed that scholarly output of women initially lags behind men early during their careers, but subsequently increases to equal or exceed that of men. Our findings also indicated that there were no differences in NIH grant acquisition, the number of years of funding, or total NIH support between men and women who received FAER grants. These findings contrast with those of other studies that demonstrated gender differences in NIH funding. For example, men received more R-series NIH grants and had greater funding than women in otolaryngology.⁵⁰  Similar findings were observed in radiation oncologists.⁴⁹  Our results suggest that an early-career FAER grant may provide some of the experience needed to allow women and men to become equally competitive NIH grant applicants.

Our data further indicate that FAER grant recipients with combined MD and PhD degrees had similar scholarly productivity as those with MD degrees alone (table 10). In contrast, Hurley et al.⁵¹  reported that academic anesthesiology faculty members with MD and PhD degrees had more publications than those with only medical degrees (2.59 ± 4.42 vs. 1.08 ± 2.92, respectively; P < 0.0001) during a 2-yr period (2006–2008) in their study of 6,143 U.S. faculty members. The reasons for this discrepancy are unclear to us, but it would appear likely that the FAER grant recipients with MD degree examined here represent a small but highly research-motivated subset of all MD anesthesiology faculty members. Notably, the current data also show that FAER grant recipients with MD and PhD degrees received more total NIH grants with more years of funding and greater financial support than their peers with MD degrees alone. These observations were not entirely surprising considering the additional years of research experience gained while obtaining a PhD degree.

In addition to the shortcomings already mentioned, several other potential limitations should be considered when interpreting the current results. The use of h-index as a measure of scholarly output certainly has widely recognized limitations as a bibliometric statistic^{8,14,16,20,21,23–25,33,35,52–54}  that we will not reiterate in detail here. Despite these concerns, it is also clear that h-index is positively linked with academic standing, scholarly output, and the ability to successfully obtain financial support for research from NIH across all of the medical specialties, including anesthesiology, in which it has been studied to date.^3–22  We did not quantify other sources of funding available to investigators (e.g., Veterans Affairs, National Science Foundation) and may have underestimated the total amount of extramural support obtained by FAER grant recipients. However, the NIH is the major source of extramural funds for biomedical researchers in the United States. Currently, the NIH Research Portfolio Online Reporting Tools^® provide public access to NIH awards dating from 1991 to present, but FAER program began awarding grants in 1987. Thus, it is possible that NIH grants awarded between these dates were not recorded in our analysis. It is unclear to us why less than 50% of FAER grant recipients went on to obtain NIH funding. From our perspective, the current bibliometric survey cannot identify specific factors associated with subsequent success or failure to obtain a NIH grant, as the reasons for this observation are most likely multifactorial. The current data are also not intended to influence FAER policy/practice or provide insight into how FAER decides which applicants should or should not be awarded a grant.

In summary, our results indicate that the majority of FAER grant recipients remain in academic anesthesiology and have established a consistent record of scholarly output as measured using h-index. The output of FAER grant recipients appears to substantially exceed previously reported productivity for average academic anesthesiology faculty identified in other studies.^5,8,12  Many FAER alumni have also received research awards from the NIH, which totals more than $440 million since the program’s inception. These data point to the success of the FAER program as an important stimulus of academic anesthesiology careers in the United States.

Support was provided solely from institutional and/or departmental sources.

The authors declare no competing interests.