Name: What is a P-Value?
Uploaded: undefined
Duration: undefined

Nova

Science's most important (and controversial) number has its origins in a British experiment involving milk and tea.

Science's most important number has its origins in a British experiment involving tea.

What is a P-Value?

Simons Foundation Science Sandbox

Koch Foundation

NOVA Science Trust Logo

Brilliant.org

<div class="rich-text">Science&#39;s most important (and controversial) number has its origins in a British experiment involving milk and tea.
</div>

<div class="rich-text">Science's most important (and controversial) number has its origins in a British experiment involving milk and tea.</div>

Physics & Math

<div class="rich-text"><h3>
 What is a P-Value?
</h3>

 Published March 7, 2018


 
 Onscreen:
 
 What if a woman claimed to tell the difference between milk poured into tea vs. tea poured into milk? How could you test her claim?


 British scientist Ronald A. Fisher designed an experiment using statistics and probability to test her claim. He proposed that a reasonable test of her ability would be 8 cups: four with milk into tea and four with tea into milk, each presented randomly. The lady then would separate them back into the two groups.


 Why eight?


 That produced 70 possible combinations of the cups, but only one with them separated correctly. If she got it right, that wouldn’t prove she had a special ability. But Fisher could conclude if she was just guessing it was an extremely unlikely result.


 A probability of just 1.4%


 Thanks mainly to Ronald A. Fisher, that idea became enshrined in experimental science as the p-value. P for probability. If you assume your results were just chance, what’s the probability of those results or something even more rare?


 
 Rebecca Goldin:
 
 If you assume that there’s a process that is completely random, and you find that it’s pretty unlikely to get your data, then you might be suspicious that something was happening. You might conclude that it’s not a random process. That this is interesting to look at what else might be going on and it passes some kind of sniff test.


 Fisher also suggested a benchmark. Only experimental results with a p-value under .05, a probability of less than 5% were worth a second look. If you assume your results were just due to chance, you’d see them less than 1 time out of 20. Not very likely. He called those results statistically significant.


 
 Jordan Ellenberg:
 
 Statistically significant. Now, this is a terrible word. It could be quite insignificant. You could be detecting a very, very, very small effect. But it would be called, in the mathematical lingo, significant.


 Since Ronald A. Fisher’s day, p-values have been used a convenient yardstick for success by many, including most scientific journals. Since they prefer to publish successes, and getting published is critical to career advancement, the temptation to massage and manipulate experimental data into a good p-value is enormous.


 There’s even a name for it: P-hacking.


 
 Regina Nuzzo:
 
 P-hacking is when researchers consciously or unconsciously guide their data analysis to get the result that they want. And since .05 is the bar for being able to publish, and call something real, get all your grant money, it’s usually guiding the results so that you arrive at that P of .05.


 How much p-hacking really goes on is hard to know. What may be more important is to remember what was originally intended by a p-value.


 
 Ellenberg:
 
 The p-value was always meant to be a detective, not a judge. If you do an experiment and find a result that is statistically significant, that is telling you, that is an interesting place to look and research and understand further what’s going on. Not, don’t study this further because the matter is settled.


 In a sense, a low p-value is an invitation to reproduce the experiment, to help validate the result. But that doesn’t always happen. In fact, there are few career incentives for it. Journals and funders prefer novel research. There is no Nobel Prize for replication.
</div>

Canada

United States

Northern Mariana Islands

Puerto Rico

American Samoa

Guam

U.S. Virgin Islands

U.S. Minor Outlying Islands

US, US Territories, Canada

US & Territories

Follow along as NASA launches the Mars 2020 Mission, perhaps the most ambitious hunt yet for signs of ancient life on Mars. In February 2021, the spacecraft blazes into the Martian atmosphere at some 12,000 miles per hour and lowers the Perseverance Rover into the rocky Jezero Crater, home to a dried-up river delta scientists think could have harbored life. Perseverance will comb the area for signs of life and collect samples for possible return to Earth. Traveling onboard is a four-pound helicopter that will conduct a series of test flights—the first on another planet. During its journey, Perseverance will also test technology designed to produce oxygen from the Martian atmosphere, in hopes that the gas could be used for fuel—or for humans to breathe—on future missions. (Premiered February 24, 2021)

NASA launches its most ambitious search yet for traces of ancient life on Mars.

Looking for Life on Mars

A person is tested for COVID-19 at a drive-thru testing site in Florida in July 2020. Image Credit: Paul Hennessy/NurPhoto, Getty Images

Figuring out when to get tested after exposure requires understanding what happens once the virus enters your body. We’ve got you covered.

Figuring out when to get tested after exposure requires understanding what happens once the virus enters your body. We’ve got you covered.

I’ve been exposed to Covid-19. When should I get tested?

A statue of Zheng He at the Sam Poo Kong Temple in Java, Indonesia. Image Credit: Budianto Santosa, Shutterstock

In the 1400s, Zheng He sailed thousands of miles around Asia and Africa in ships the size of soccer fields, spreading Chinese innovations like compasses and gunpowder in the process.

In the 1400s, Zheng He sailed thousands of miles around Asia and Africa in ships the size of soccer fields, spreading Chinese innovations like compasses and gunpowder in the process.

Transcript

Credits

Support Provided By