When it comes to funding scientific research, everyone wants the process to be fair – ensuring the most innovative and promising projects receive support, regardless of where the scientists work. However, research has shown that reviewers can be unconsciously influenced by the prestige of an applicant’s institution. Dr Anne Hultgren and her colleagues at The Arnold and Mabel Beckman Foundation and the Health Research Alliance investigated whether blinding reviewers to applicants’ institutional affiliations could help create a more equitable review process for their Young Investigator awards.

The mission of the Arnold and Mabel Beckman Foundation is to fund innovative research projects, especially those that open new avenues of research in chemistry and life sciences. The foundation’s review process is multi-staged. After the initial Letter of Intent (or LOI) review, successful applicants are invited to submit more detailed full applications, which include a six-page research proposal, budget, timeline, recommendation letters, and institutional support documentation. These full applications are evaluated by panels of three reviewers each, with separate panels for biology and chemistry-related proposals. The most promising candidates are then invited for interviews before final selections are made.

In 2020, the Arnold and Mabel Beckman Foundation introduced a significant change to their grant review process – applicants were required to submit anonymous proposals that didn’t include their name, gender, pronouns or institutional information. The researchers then analysed four years of data before this change (from 2017-2020) and four years after (from 2021-2024) to see how blinding affected which scientists were invited to submit full grant applications and ultimately received funding.

Dr Hultgren’s study focused on the Beckman Young Investigator program, which supports early-career faculty in chemistry and life sciences who are within their first four years of a tenure-track position. The program begins with an open call for LOIs. Previously, these LOIs included the applicant’s full information, but under the new system, reviewers could only see the scientific proposal itself.

To measure institutional prestige bias, the researchers developed a comprehensive system for categorizing universities. They used eight different ranking systems, including federal research funding data, global university rankings, and their own foundation’s historical funding patterns. Universities were divided into five categories: the top 10 institutions, ranks 11-25, ranks 26-50, ranks 51-100, and all others.

The results were striking. During the unblinded period, applications from the top-ranked institutions had a significant advantage. Scientists from the top 10 institutions were 1.4-1.7 times more likely than average to be invited to submit a full application. Those from institutions ranked 11-25 were 1.2-1.4 times more likely to advance. Meanwhile, scientists from institutions outside the top 100 were at a considerable disadvantage, being only 0.6-0.8 times as likely as average to move forward.

After implementing blind review, these disparities decreased substantially. The advantage for top-10 institutions dropped to just 1.0-1.3 times the average, and similar decreases were seen for the 11-25 category. Most importantly, scientists from institutions outside the top rankings saw their chances improve significantly, rising to 0.81-0.93 times the average likelihood of advancing.

This pattern continued through to the final awards. Before blinding, about 75% of grants went to scientists at the top 25 ranked institutions. After blinding was implemented, this dropped to about 45%, allowing more opportunities for excellent scientists at a broader range of institutions.

Interestingly, when Dr Hultgren and her colleagues examined potential gender bias in their review process, they found no significant differences in success rates between male and female applicants either before or after implementing blind review. This suggests that their previous review process was already relatively balanced in terms of gender considerations.

The researchers note that while blinding reduced institutional bias, it didn’t eliminate it completely. They suggest this could be because there are some genuine advantages of applications from prestigious universities. This could be partially related to the quality of the candidates themselves and the assistance that well-resourced institutions provide to their faculty, such as better research infrastructure and more support for grant writing.

Additionally, the research team informed reviewers at the beginning of the process that the purpose in blinding the applications was in part to study if there was institutional prestige bias in their reviews. This awareness may have impacted the scoring of proposals if the reviewers were more willing to advance LOIs or full applications that had weaknesses attributed to lack of mentorship or access to resources, which might indicate that the applicant was from an institution that did not have the same level of research support infrastructure. The COVID-19 pandemic also affected the total number of applications during the study period, though the proportional distribution across institutional categories remained consistent.

The study also revealed some practical benefits of blind review. Reviewers reported that blinded materials were easier to read and evaluate, leading to more efficient review meetings where discussions could focus solely on the scientific merit of proposals. This feedback, combined with the evidence of reduced bias, led the Arnold and Mabel Beckman Foundation to permanently adopt blind review for their Young Investigator program and extend it to their Postdoctoral Fellowship program as well.

For the scientific community, these findings have important implications. Many funding organizations are looking for ways to make their review processes more equitable, and this study provides strong evidence that blind review can help achieve this goal. The researchers suggest that reducing institutional prestige bias is particularly important for organisations with missions similar to their own, such as assisting early-career scientists who might otherwise be disadvantaged simply because they work at less prestigious institutions.

The success of this initiative demonstrates how relatively simple changes to review processes can help level the playing field in scientific funding. By focusing reviewers’ attention solely on the quality of the scientific ideas being proposed, rather than where those ideas originate, funding organizations can better fulfill their mission of supporting the most innovative and promising research, regardless of institutional affiliation.

Looking ahead, the foundation plans to continue monitoring their application metrics and expanding their outreach to institutions that may have fewer internal resources. They are also working to diversify their reviewer pool to ensure multiple perspectives are included at all levels of the review process. Through these ongoing efforts, they hope to continue building a more equitable system for supporting the next generation of scientific innovators.