Do taxpayers get their money's worth from the National Institutes of Health?

As proposed cuts loom, new research digs into the return on investment for public funding of the National Institutes of Health. Photo by Andrew Councill/MCT/MCT via Getty Images

Two weeks ago, a budget proposal from the Trump administration, to the surprise of many scientists, called for a 19 percent reduction in funding for the National Institutes of Health. Congress must approve the final terms of fiscal year 2018 budget, but the proposed cuts raise questions about the value of the America's health research organization.

Founded in 1887, the NIH comprises 27 different institutes and centers, though most of its $33 billion annual budget goes toward supporting 300,000 scientists spread across 2,500 universities and organizations in every state and other places around the world. The institutes disperse these funds through grants, which back research into myriad human diseases but also fuel the educational training of doctors, scientists and undergraduates.

On Wednesday, Secretary of Health and Human Services Tom Price defended the "skinny budget"'s call for a $5.8 billion reduction in NIH funding by arguing the trims would not target research, but instead go after direct "overhead" costs like lab equipment or utilities.

Can the NIH survive without its indirect costs? And what might a 19 percent slash mean for the NIH's return on public investment?

A new study published Thursday in the journal Science tries to suss out an answer by calculating how many NIH grants lead to new patents and medications. The numbers portray the NIH's spillover influence into the private sector, but also shine a light on research's inherent propensity for failure.

What they studied

  • Grant data. Three economists — Harvard's Danielle Li, Pierre Azoulay of the Massachusetts Institute of Technology and Columbia University's Bhaven N. Sampat — amassed data on grants issued by 21 of the NIH's 24 institutes between 1980 to 2007. (It stopped there because it takes about 10 years for a drug to be developed and approved).
  • The team examined 365,380 grants in total to determine how many directly led to patents or FDA-approved drugs. They also tracked whether or not these grants were cited by patent or drug applications.
  • They excluded three institutes — the National Library of Medicine, the National Institute of Nursing Research and the National Institute of Minority Health and Health Disparities — because of their limited influence on private-sector patents.
  • Their analysis focused primarily on marketed drugs — think pills in bottles or injected medicines — and didn't assess medical device patents.

What they found

  • 8.4 percent of these grants (approximately 30,000) were directly responsible for a patent, most of which were "Bayh-Dole" patents held by the university or hospital where the innovation was made.
  • A larger proportion of the grants — 31 percent — had been cited by 81,642 by private-sector patents, suggesting the research played a role in the development of those concepts.
  • However, fewer than 1 percent of these grants — 4,414 — were directly tied to FDA-approved drugs, while 5 percent were mentioned in a patent for a marketed drugs.

Why it matters

Let's unpack these figures. Does an 8.4 percent influence on patents and a less than 1 percent impact on FDA-approved drugs represent a solid rate of return?

Pierre Azoulay argues yes. He told the NewsHour:

Okay, first let's be clear. This is very important. This is not a return on investment. This article doesn't tell you what's the return on investment in publicly funded research. This article provides evidence that the research done in the public sector is useful — it's relevant for the research done by private-sector firms. They actually build on it.

So say there's a scientist funded in academia, funded by NIH. They do research. They publish it, and at some later point — sometimes a much much later point — that research will become relevant to private sector efforts in the biopharmaceutical industry. They will use it. They will build on it. What we found is that is a much higher proportion NIH funded grants that are relevant in this slightly more indirect way.

Moreover, the figures in this paper do not represent the full return on investment for NIH research. That's a much broader question because the NIH's goal is not to increase patenting, but "to seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life and reduce illness and disability."

Hence, assessing the NIH's full return on investment should also include looking into their influence over public well-being and how those actions contribute subsequently to the economy.

  • Take for example the Framingham Heart Study, Azoulay said. Much of what is known about what causes hypertension and heart failure comes from the Framingham Heart Study, which has been running for 69 years.
  • "The rate of return on just this study must be in like the gazillions of dollars, right?" Azoulay said. "Statin drugs wouldn't have been developed, if we hadn't had the results from the Framingham Heart Study. Just think about the number of Americans who take statins every year."
  • In a separate, preliminary analysis, Azoulay and his colleagues estimate that every $10 million of public investment in the NIH can expect $14.7 million million in subsequent drug sales. Note: This research has not been officially published or peer-reviewed, so the final numbers might change.

What happens next

The NIH isn't perfect. In the upcoming book Rigor Mortis, NPR science reporter Richard Harris dives into the many flaws of biomedical research, namely the replication crisis. This catchall term refers to a growing trend whereupon biomedical researchers fail to reproduce each other's results. Anyone familiar with the scientific method recognizes that's a serious problem.

Here's a clip from Harris' book:

There has been no systematic attempt to measure the quality of biomedical science as a whole, but Leonard Freedman, who started a nonprofit called the Global Biological Standards Institute, teamed up with two economists to put a dollar figure on the problem in the United States.

Extrapolating results from the few small studies that have attempted to quantify it, they estimated that 20 percent of studies have untrustworthy designs; about 25 percent use dubious ingredients, such as contaminated cells or antibodies that aren't nearly as selective and accurate as scientists assume them to be; 8 percent involve poor lab technique; and 18 percent of the time, scientists mishandle their data analysis. In sum, Freedman figured that about half of all preclinical research isn't trustworthy.

Experts debate the veracity of these numbers, but the calculations raise serious questions about the efficiency of biomedical research. Azoulay recognizes the problems.

"It's a human enterprise with human flaws," Azoulay said. "But the [NIH] grant system itself, which is very stringent, does tend to screen out ideas that are high risk."

Yet the nature of science is failure. Millions of biomedical experiments are conducted each year just to prep drug candidates for human testing, yet only one in 10 will survive clinical trials. Harris estimates "of the 7,000 known diseases, only about 500 have treatments." Not all of those remedies came from the NIH, but countless did.

Azoulay argues that instantly shutting off the NIH's funds for indirect costs would undoubtedly have repercussions for medical schools and research programs at universities.

"They'd have to shrink significantly and would have to review all kinds of priorities if overhead costs got cut," Azoulay said. "You need to build those buildings. You need to staff them. There's a lot of infrastructure that's needed for research. It's not a gift."

Recently in Science