Name: What is a P-Value?
Uploaded: undefined
Duration: undefined

Nova

Science's most important (and controversial) number has its origins in a British experiment involving milk and tea.

Science's most important number has its origins in a British experiment involving tea.

What is a P-Value?

Simons Foundation Science Sandbox

NOVA Science Trust Logo

<div class="rich-text">Science's most important (and controversial) number has its origins in a British experiment involving milk and tea.
</div>

<div class="rich-text">Science's most important (and controversial) number has its origins in a British experiment involving milk and tea.</div>

Physics & Math

<div class="rich-text"><h3>
 What is a P-Value?
</h3>

 Published March 7, 2018



 Onscreen:
 
 What if a woman claimed to tell the difference between milk poured into tea vs. tea poured into milk? How could you test her claim?


 British scientist Ronald A. Fisher designed an experiment using statistics and probability to test her claim. He proposed that a reasonable test of her ability would be 8 cups: four with milk into tea and four with tea into milk, each presented randomly. The lady then would separate them back into the two groups.


 Why eight?


 That produced 70 possible combinations of the cups, but only one with them separated correctly. If she got it right, that wouldn’t prove she had a special ability. But Fisher could conclude if she was just guessing it was an extremely unlikely result.


 A probability of just 1.4%


 Thanks mainly to Ronald A. Fisher, that idea became enshrined in experimental science as the p-value. P for probability. If you assume your results were just chance, what’s the probability of those results or something even more rare?



 Rebecca Goldin:
 
 If you assume that there’s a process that is completely random, and you find that it’s pretty unlikely to get your data, then you might be suspicious that something was happening. You might conclude that it’s not a random process. That this is interesting to look at what else might be going on and it passes some kind of sniff test.


 Fisher also suggested a benchmark. Only experimental results with a p-value under .05, a probability of less than 5% were worth a second look. If you assume your results were just due to chance, you’d see them less than 1 time out of 20. Not very likely. He called those results statistically significant.



 Jordan Ellenberg:
 
 Statistically significant. Now, this is a terrible word. It could be quite insignificant. You could be detecting a very, very, very small effect. But it would be called, in the mathematical lingo, significant.


 Since Ronald A. Fisher’s day, p-values have been used a convenient yardstick for success by many, including most scientific journals. Since they prefer to publish successes, and getting published is critical to career advancement, the temptation to massage and manipulate experimental data into a good p-value is enormous.


 There’s even a name for it: P-hacking.



 Regina Nuzzo:
 
 P-hacking is when researchers consciously or unconsciously guide their data analysis to get the result that they want. And since .05 is the bar for being able to publish, and call something real, get all your grant money, it’s usually guiding the results so that you arrive at that P of .05.


 How much p-hacking really goes on is hard to know. What may be more important is to remember what was originally intended by a p-value.



 Ellenberg:
 
 The p-value was always meant to be a detective, not a judge. If you do an experiment and find a result that is statistically significant, that is telling you, that is an interesting place to look and research and understand further what’s going on. Not, don’t study this further because the matter is settled.


 In a sense, a low p-value is an invitation to reproduce the experiment, to help validate the result. But that doesn’t always happen. In fact, there are few career incentives for it. Journals and funders prefer novel research. There is no Nobel Prize for replication.
</div>

Canada

United States

Northern Mariana Islands

Puerto Rico

American Samoa

Guam

U.S. Virgin Islands

U.S. Minor Outlying Islands

US, US Territories, Canada

US & Territories

NOVA investigates the story of cannabis from the criminalization that has disproportionately harmed communities of color to the latest medical understanding of the plant. What risks does cannabis pose to the developing brain? How much do we know about its potential medical benefits? As cannabis becomes socially accepted, scientists are exploring its long-term health consequences. (Premiered September 29, 2021)Check out a previously recorded panel discussion and audience Q&amp;A featuring experts and producers from "The Cannabis Question" <a href="https://youtu.be/nSTT5RZ_hxA">here</a>.

As state-legalized cannabis spreads, NOVA explores its little-known risks and benefits.

The Cannabis Question

A 2006 composite image from NASA's Hubble Space Telescope and Spitzer Telescope shows the chaos of baby stars in the Orion nebula some 1,500 light-years away. Image Credit: NASA/JPL-Caltech/STScI

With the upcoming launch of the James Webb Space Telescope, the Hubble era is gradually drawing to a close. Here are some highlights from the countless wonders Hubble has shown us during its 31 years in space.

10 spectacular Hubble Space Telescope images

A statue of Zheng He at the Sam Poo Kong Temple in Java, Indonesia. Image Credit: Budianto Santosa, Shutterstock

In the 1400s, Zheng He sailed thousands of miles around Asia and Africa in ships the size of soccer fields, spreading Chinese innovations like compasses and gunpowder in the process.

Transcript

Credits

Support Provided By