Reviewing grant proposals for federal agencies has proven to be a boon to researchers at the USC School of Social Work.
A perfect example is Suzanne Wenzel, a professor and current chair of the school’s Research Council. Since 2001, she has served as a reviewer for the National Institutes of Health on multiple occasions. She recently accepted an invitation to join a four-year review panel tasked with examining proposals related to the multiple personal, social and community factors that influence HIV risk and preventive behaviors.
The level of precision and scientific rigor that is expected by the reviewers and the institutes is really quite pronounced.
“I think it has helped me to craft stronger proposals and to give my colleagues and doctoral students better advice on writing their own,” she said. “The level of precision and scientific rigor that is expected by the reviewers and the institutes is really quite pronounced.”
A stepping-stone to excellence
The grant review process can be mystifying for many in academia. Application guidelines are long and complex, arguments and approaches have to be framed in a certain way, and funding priorities are constantly shifting.
Having an insider’s perspective on how federal agencies such as the NIH and National Science Foundation assess grant applications is critical to putting together a successful proposal, said Charles Kaplan, associate dean of research at the USC School of Social Work.
You can send proposals in, but you can’t get them funded unless you are very aware of the dynamics of the review process.
“You can send proposals in, but you can’t get them funded unless you are very aware of the dynamics of the review process,” he said. “You really have to have your finger on the pulse of what of what NIH and NSF are doing in terms of their review and to try to get on those committees. That’s the best way of learning. It’s a stepping-stone to excellence.”
Room for improvement
The school has increasingly emphasized mentoring and internal review of grant proposals before they are submitted. Each of the school’s research clusters — which bring together faculty, student and postdoctoral scholars with similar research interests to stimulate intellectual productivity — has at least one faculty member with extensive experience reviewing federal grant proposals. That knowledge appears to be paying off.
Three recent proposals earned perfect scores during the review process, a relatively rare occurrence in federal funding circles. However, Kaplan said he would like to see more leadership among faculty members with review experience in terms of mentoring younger or less experienced researchers.
“We definitely have lots of room for improvement,” he said.
The most effective way to gain knowledge of the review process, at least at the NIH, is to be invited to join a study section at the federal Center for Scientific Review. Members of these study sections or peer review groups typically serve for four years and meet three times a year to review proposals that correspond with their area of expertise.
Due to her extensive research in the area, Wenzel has primarily assessed applications for funding that focus on HIV/AIDS prevention and treatment and behavioral health issues. During a typical review period, she is assigned as the primary reviewer of a handful of proposals.
It is important that you address whether the application is answering an important question within an area of science.
“Your role as a reviewer is to review the applications carefully, present your point of view to the study section, listen to the other reviewers and as a group evaluate the applications in terms of their overall impact on the field and beyond,” she said. “It is important that you address whether the application is answering an important question within an area of science and how the aims, if successful, will move the field forward.”
Before meeting with other reviewers, however, Wenzel meticulously reads each proposal, including sections on the research and data analysis strategy, procedures for protecting participants, the budget and biographical information for the research team.
She then drafts a written review and assigns a preliminary score to proposals that warrant further discussion at the group level. After she presents the application to the study section, other reviewers weigh in and vote on a final score.
Recommendations are then forwarded on to a separate advisory council, which helps the federal agency decide which proposals to fund. Adding to the complexity of the process is the varying set of guidelines for different types of proposals. Some focus on the significance and innovation of the research, the rigor of the methods and analysis plan, and the scientific environment in which the project will occur. Others are focused on early career development and may emphasize training opportunities for less experienced researchers.
“If I had to describe my mindset when I approach review of any given application, it would be objectivity,” Wenzel said. “Fairness. Open-mindedness. Impartiality must be a hallmark of review. Without that, the process is doomed.”
A staggering commitment
The commitment required to be a federal grant reviewer is staggering. John Brekke, Frances G. Larson Professor of Social Work Research, has been consistently involved as a scientific reviewer at the federal level since 1989, serving as a standing member of a mental health services and epidemiology review committee at the National Institute of Mental Health for six years and as an ad hoc reviewer on many occasions.
Although each round of review involves at least two weeks of intensive work prior to gathering with other committee members to discuss the proposals, Brekke said he never questioned whether he should dedicate his time to the process.
“It really is a notable time commitment, but it’s just something that people who are in the research community and take the research and scientific world seriously know is something we have to do,” he said.
“It’s all about shaping a scientific agenda, and that’s a wonderful thing to be a part of.”
He also noted that the position is considered to be very prestigious. Only individuals who have exemplary records of scientific exploration and productivity are invited to serve as reviewers, Brekke said.
He is thankful for several senior mentors who offered guidance during the early years of his career, and he has since created a template for others to follow in terms of building a successful grant application.
“I think it’s absolutely perilous for someone to attempt an NIH proposal without mentoring from someone who has gone through the process,” Brekke said. “That can mean the difference between funding and no funding.”
Capitalizing on experience
By capitalizing on the strength and knowledge of experienced researchers, he said the USC School of Social Work is setting up its junior faculty members for success. It’s also helping senior researchers maintain their edge in an increasingly competitive funding environment and adapt to new funding streams.
Brekke recently applied for and received a $1.2 million award from the newly established Patient-Centered Outcomes Research Institute, a nonprofit organization created by Congress through the Patient Protection and Affordable Care Act. He said advice and guidance from colleagues who had submitted successful proposals to the institute in earlier rounds of funding was critical to his success.
“I’ve had good success following a really disciplined approach,” he said. “You can tell the proposals that have done this and those that haven’t. It’s immediately obvious, just in reading the abstract, who has had some mentoring and who has not.”
The power of mentoring
Another strong believer in the power of mentoring and peer review of proposals prior to submission is Eric Rice, an assistant professor at the USC School of Social Work.
As an early career researcher at UCLA, he wanted to pursue a specific NIH funding mechanism designed for career development known as a K award. Because none of his close colleagues had experience with that award, his mentor referred him to a friend on a federal review committee, Concepción “Concha” Barrio, now an associate professor at the USC School of Social Work.
Barrio, who several years prior had applied for and received a K award on her first submission and had been reviewing grant proposals related to mental health services, was more than happy to provide feedback on Rice’s application.
“No hyperbole — I got that grant because of Concha,” Rice said. “Knowing what one of these things should look like and how it should read and having someone who has sat on those review committees is invaluable. It’s an inside bit of knowledge you wouldn’t otherwise have.”
He has since served as an ad hoc reviewer called in by federal officials because of his expertise in social network research. He said the experience has given him a better sense of how to craft his own proposals, and he is sharing that knowledge with postdoctoral scholars and colleagues.
However, Rice acknowledges that he still has plenty to learn. He turned to Brekke for advice on how to leverage data from a pilot study into a larger project on social networks among young men and women experiencing homelessness.
Wenzel, who also specializes in homelessness issues, helped him refine his application, and he successfully secured a $2 million grant from the National Institute of Mental Health.
“I completely attribute my early career successes to having been in an environment with people like Concha and Suzanne and John who were willing to help me,” Rice said. “The resources are there; you just have to reach out and ask for that help.”