Visit Your Local PBS Station PBS Home PBS Home Programs A-Z TV Schedules Watch Video Donate Shop PBS Search PBS
Photo of Bill Moyers Bill Moyers Journal
Bill Moyers Journal
Bill Moyers Journal
Watch & Listen The Blog Archive Transcripts Buy DVDs
Margins of Error. Poll Reading Tips
Graph
Watch Video
Read Transcript
Comment
October 24, 2008

Like a sentence by Dr. Suess, the presidential polls seem to be up, down, all around and potentially nonsensical. As the election nears, the frenzied polling only increases each day. The famous Gallup Organization updates its presidential prognostications numerous times daily. How is the average non-statistician to make sense of them?

The Internet has been a boon for pollsters — everybody's results are touted worldwide seemingly instantaneously. But for observers it sometimes leads to confusion. How can the AP be calling the race as near dead even, when CNN has Obama with a significant lead at the same time? Well, poll experts say that it has little to do with strict numbers and everything to do with who you ask...and what you ask them. The same pollsters publish very different numbers for samples of registered and "likely" voters (those who have voted regularly in the past.) But it's even more complex. For example, some polling organizations use cell phone numbers in their samples, some don't. Cell phone users tend to be younger and more affluent — factors that also play into voting patterns.

The Internet is a great place to sort these discrepancies out though poll aggregators — sites which gather the disparate polls and re-calculate all the results offering up some sort of believable average. The serious analysts don't just add and divide, but weight different pollsters based on past accuracy and other factors. However, the aggregators often don't agree among themselves. There's been a blog-based battle between two of the most popular sites, RealClearPolitics.com and FiveThirtyEight.com over which polls should be in, or out, of the sample. Then last week there were the numerous articles about American racial attitudes and "The Bradley Effect" — which suggests that predicting whether people are telling the truth to either themselves or pollster is always a losing game. Our best advice is self-education — read the tips below and read the each site's FAQ.

In addition, get advice from Daniel Yankelovich, the founding father of public opinion research in an excerpt from the MOYERS DIGITAL ARCHIVE:

"I think that if I had to make one single suggestion it would be to ask yourself the question, when you look at the poll results, is this an issue where people have made up their minds? You may not know, but if you see inconsistencies, if the wording of the question changes the response. And you can ask yourself, have you made up your mind about Social Security, or Medicare or drugs for seniors, or more money for schools and things of that sort. If you haven't made up your mind, the poll the people that are polling are like you and they haven't made up their minds, you can't rely on the poll results." -- Daniel Yankelovich

Poll Reading Tips

The National Council on Public Polls has a list of questions all journalists should ask before citing a poll. They are useful questions for a media-savvy public as well.

  • Who did the poll?
  • Who paid for the poll and why was it done?
  • How many people were interviewed for the survey?
  • How were those people chosen?
  • What area (nation, state, or region) or what group (teachers, lawyers, Democratic voters, etc.) were these people chosen from?
  • Are the results based on the answers of all the people interviewed?
  • Who should have been interviewed and was not?
  • When was the poll done?
  • How were the interviews conducted?
  • What about polls on the Internet? (There's no sample control on the Web.)
  • What is the sampling error for the poll results?
  • Who's on first? (The ordering of choices, especially candidates, can affect responses.)
  • What other kinds of factors can skew poll results? What questions were asked? These questions are annotated at the National Council on Public Polls web site. You can also find an Ethics and Standards guide to polling at the American Association for Public Opinion Research.
You can often obtain the full poll from the polling institution.

Daniel Yankelovich
Bill Moyers interviewed Daniel Yankelovich, founder of the non-profit organization Public Agenda and author of THE MAGIC OF DIALOGUE in 2004. Recognized the world over for careful and credible research on American values and public opinion, Yankelovich explains the need to be careful when reading poll results.

Published October 24, 2008.

Related Media:
Katherine Newman, photo by Robin HollandGoing Negative
Bill Moyers checks in with JOURNAL contributor and director of the Annenberg Public Policy Center Kathleen Hall Jamieson on how dirty politics will play out in this final stretch to the election. (October 10, 2008)

George Soros, photo by Robin HollandRed, White and Blue Media
The JOURNAL takes an in-depth look at the news of the week to sort out the media-frenzied hype from the facts the public needs to know. Factcheck.org's Kathleen Hall Jamieson and ON THE MEDIA's Brooke Gladstone dissect the campaign coverage. (October 3, 2008)

YouTubePolitics 2.0
Bill Moyers talks with Kathleen Hall Jamieson about how the Internet has transformed the political campaign in the United States.
References and Reading:
Poll Science
National Council on Public Polls
The National Council on Public Polls (NCPP) is an association of polling organizations established in 1969. Its mission is to set the highest professional standards for public opinion pollsters, and to advance the understanding, among politicians, the media and general public, of how polls are conducted and how to interpret poll results.

Public Agenda
Best Estimates: A Guide to Sample Size and Margin of Error There are about 209 million adults in America, of every imaginable background and circumstance. So how can a survey of only 800 or 1,000 adults reflect what the entire country is thinking? Public Agenda crunches the numbers.

American Association for Public Opinion Research
The American Association for Public Opinion Research (AAPOR) is the leading professional organization of public opinion and survey research professionals in the U.S.

Poll Aggregators
Real Clear Politics
Real Clear Politics keeps track of and aggregates hundred of state and national polls and allows the viewer to sort by pollster, state and date. PBS and NPR use Real Clear Politics data in some of their electoral maps.

Five Thirty Eight
This site is the work of baseball statistician Nate Silver, who updates his own take on the polls several times daily. Silver. The site has an extensive FAQ on the methods used to weight each pollster's accuracy.

Pollster.com
This aggregator has a group of six bloggers who comment on poll trends.

Polls
BBCNews U.S. Elections Polltracker
The BBC offers a frequently-updated interactive feature following four key pollsters over time.

Gallup Organization
The famed pollster offers frequent updates on the premiere polls on its homepage. You can dig deeper for their analysis of the economy, election issues and even the view from overseas.

Quinnipiac Polling
Quinnipiac College offers good basic lessons in poll design and interpretation.

Published October 24, 2008

Also This Week:

JAMES K. GALBRAITH
Bill Moyers sits down to talk about the economic future with with James K. Galbraith, Lloyd Bentsen, Jr. Chair in Government/Business Relations at the LBJ School of Public Affairs at the University of Texas at Austin. Galbraith is the author of six books, the most recent, THE PREDATOR STATE: HOW CONSERVATIVES ABANDONED THE FREE MARKET AND WHY LIBERALS SHOULD TOO.

PLAYING FOR CHANGE
Bill Moyers talks with Mark Johnson, the producer of a remarkable documentary about the simple but transformative power of music.

MARGIN OF ERROR: POLL READING TIPS
As the election frenzy heightens, get some tips on reading polls from experts.

MOYERS ON 2008
Add your voice to our election year map. Plus, get perspective on pressing election-year issues from JOURNAL guests.

TALK BACK: THE MOYERS BLOG
Our posts and your comments
OUR POSTS
YOUR COMMENTS
For Educators    About the Series    Bill Moyers on PBS   

© Public Affairs Television 2008    Privacy Policy    DVD/VHS    Terms of Use    FAQ   
SIGN UP FOR BLOG UPDATES AND PODCASTS EMAIL US