TOPICS > Politics

Polls offer a snapshot, but not always reliable predictions

December 9, 2015 at 6:35 PM EDT
Early polling often doesn't reflect who goes on to become president (think Howard Dean or Newt Gingrich). These days pollsters acknowledge they are wrestling with a sea change in polling methods that's making it harder to collect a sample opinion of Americans. Judy Woodruff reports as part of a collaboration between The Atlantic and the PBS NewsHour.

JUDY WOODRUFF: But, first, in this presidential race, as with any of the previous ones over the past few decades, potential voters have been inundated with a seemingly nonstop wave of opinion polls.

But what do polls really tell us? Historically, at this point in the race, they aren’t always a good predictor of the eventual party nominee. So, how can Americans decipher which ones are more accurate than others?

In a report that’s part of our collaboration with “The Atlantic” magazine, we break down the art and science of polling.

JESSE JACKSON, Former Presidential Candidate: Who would have thought that I would be in front?

JUDY WOODRUFF: Remember President Jesse Jackson? President Mario Cuomo? No? How about President Howard Dean or Presidents Gingrich, Cain and Santorum?

RICK SANTORUM, Republican Presidential Candidate: Thank you so much Iowa.

JUDY WOODRUFF: Not ringing any bells? Of course, none of these men actually became president. But each was once deemed a front-runner in the polls, however briefly, as he sought his party’s nomination.

Early polls sometimes do reflect the final numbers. Think Ronald Reagan in 1980 and Al Gore in 2000. Both led early and became party nominees. And who knows, maybe 2016 will be one of those years, as the Trump juggernaut shows little sign of slowing.

But, often, the early leader doesn’t hold onto that lead.

BILL MCINTURFF, Public Opinion Strategies: I don’t think Peter and I, or any pollster, would tell you that a poll today has much or any predictive power for what October of 2016 will look like.

JUDY WOODRUFF: Republican pollster Bill McInturff partners with Democratic pollster Peter Hart to conduct the NBC/Wall Street Journal poll.

PETER HART, Research Associate: Judy, four years from now, you’re going to come right at the two of us and you’re going to say, you had Ben Carson and Donald Trump winning this election.

No, we didn’t. We told you that this is where the American public is in the odd year before we’re there.

JUDY WOODRUFF: The most reputable pollsters stand by the accuracy of their work, but acknowledge they’re wrestling with a sea change in polling methods.

BILL MCINTURFF: There’s no question it’s harder and much more difficult to collect an accurate opinion, a sample of opinion of Americans. I don’t think industry has figured out all of our methodological problems, and what we know is, like everything else, to fix something, it takes a lot of money.

JUDY WOODRUFF: One problem is cell phones.

MAN: I have a landline at home for the Internet, and I don’t answer it ever.

MAN: My landline isn’t plugged in. I just use a cell phone.

JUDY WOODRUFF: Pollsters actually do reach out to those people on cell phones, lots of them, but it’s expensive, far more expensive than calling a landline. That’s because pollsters calling landlines can use auto-dialers, computer programs that dial random numbers.

But the FCC prohibits auto-dialers for cell phones, so every call to a mobile device must be made by a person.

LEE MIRINGOFF, Marist Poll: This is a state-of-the-art facility, but all the interviews are being done manually in terms of punching in the numbers of the people we want to reach. There is no auto-dialing going on. And we don’t even have the capacity to do that.

JUDY WOODRUFF: Lee Miringoff directs the Marist Institute for Public Opinion.

LEE MIRINGOFF: Good scientific, rigorous polls have become more difficult. They have become more expensive. And, unfortunately, we also have seen a proliferation of lower-quality polls that also get picked up in the media. So, for the public as a whole, it gets a little confused trying to sort out all the numbers.

BILL MCINTURFF: In bad polls that I have seen, and mostly in a partisan setting, they interview anybody who answers the phone. Well, guess who answers the phone now? It’s all people over 50.

JUDY WOODRUFF: So polls that avoid the expense of calling cell phones exclude a significant segment of the population.

BILL MCINTURFF: Forty-five percent of the people in the country have a cell phone only, and those people who are most likely to have a cell phone, they are Latinos, some of our America’s poorer respondents, who that’s the one phone they have, and younger people. So guess what? If you under-represent in our politics, Latinos, the poor, younger people, you are systematically undercounting Democrats.

JUDY WOODRUFF: What about online-only polls?

PETER HART: Trouble.

I mean, I think that the work that SurveyMonkey is doing is exceptionally good. But I think there are an awful lot of people who are in this, and, essentially, unless you really understand the business of polling, online polling can be exceptionally dangerous.

BILL MCINTURFF: But if you’re a local news organization, and someone says, hey, I can give you a headline for literally $5,000 or $7,000 for a poll, compared to our telling you, wow, that’s just going to be the interviewing cost for just a little fraction, that’s — I mean, we have had an explosion of that kind of polling.

JUDY WOODRUFF: The major polling organizations like ABC/Washington Post, NBC/Wall Street Journal, Marist, Pew, and others continue to produce excellent, accurate polls.

But because of their expense, many in the industry are working hard to improve the accuracy of less costly nontraditional methods like online polling.

Meanwhile, as the presidential campaign heats up, and polls proliferate, the lesson for voters is buyer beware and to understand that not all polls are created equal.

At this stage in an election season, the main targets in the efforts to reach voters are the early primary and caucus states, like Iowa. Here, every four years, voters are flooded with phone calls, not just at home, but wherever they carry a cell phone.

WOMAN: I would we say we probably get them twice a day, everyday.

WOMAN: Two or three a day. And do I answer them? No.

TINA SHAW: Our landline rings nonstop.

JUDY WOODRUFF: I sat down with these Iowa voters at Drake University in Des Moines to hear firsthand what it’s like to be the target of so many polling calls.

TINA SHAW: It’s become somewhat of a game to figure out who’s conducting the poll.

OLIVIA O’HEA: Typically, I kind of screen them and don’t answer if I don’t recognize the number. But when I pick up, I do like to participate.

RAYMOND STARKS, Voter: I like to kind of decipher, you know, what is their agenda, why are they trying to push this issue, because that’s what you’re going to be seeing in the next two weeks to a month.

J. ANN SELZER, Selzer & Company: The assignment that Iowans are given is to look at more candidates than any other state does, and, really, embedded in that assignment is to keep an open mind.

JUDY WOODRUFF: Ann Selzer runs the Des Moines Register/Bloomberg poll. She says polling for caucuses presents special challenges.

J. ANN SELZER: Finding the people who are actually going to show up on caucus night, and getting a good cross-section of them, is difficult from the get-go. It always has been difficult. What has added to it is that, with the proliferation of polls, there’s a greater likelihood of polling burnout.

JUDY WOODRUFF: What is it about polling — is there something about polling that annoys you?

DENNIS OLSON, Voter: I don’t like the negative ones, where they will say, did you know this person backs — is not — is in favor of abortion? And then they go into these things. You can tell right away, right when it starts. I hang up on those types of ones.

TINA SHAW: I think when they start manipulating the questions, it turns me off. So, it’s very skewed or trying to lead me down a garden path, I feel like that’s when I click off.

J. ANN SELZER: It is become far more complicated for the public. And I feel for the public, because there are so many polls, and even I have a hard time weeding through and finding what methodology was used.

JUDY WOODRUFF: Determining the source and purpose of a poll can be daunting for voters. Public polls like those conducted by large media organizations mainly measure which candidate is up and which is down, and what voters think of them.

But the campaigns themselves are also polling, often in an effort to change voters’ opinions.

JEFF LINK, Link Strategies: We’re really doing a lot of message development and message testing.

JUDY WOODRUFF: Jeff Link directed polling in 25 states for the 2008 Obama campaign.

JEFF LINK: The horse race information is not really the most important thing for a campaign. It shouldn’t be. What we’re really doing with our polling internally is trying to figure out, what are the issues that we need to emphasize the most, what are the aspects of a candidate’s background that are most appealing to voters?

JUDY WOODRUFF: And what to say about an opponent?

JEFF LINK: And, certainly yes, what to say about an opponent, what people find most disturbing about an opponent’s record, or what’s something they have said.

Former Chairman, Iowa Republican Party: Polling ranks near the top of any decision when it comes to resource allocation within a campaign.

JUDY WOODRUFF: Matt Strawn is a former chairman of the Iowa Republican Party.

MATT STRAWN: Where are your weaknesses and deficiencies? Do you have weaknesses with female voters in the eastern part of the state? You need polling to guide those decisions.

J. ANN SELZER: People are often asking me, are these numbers going to hold through caucus night, and I always reply, I hope not, and it’s hard to imagine they would. The campaigns are spending a lot of time, and money, and effort, and bringing their candidate into our state to change those numbers.

So one would hope that they would. And if they were to hold, why would we continue to poll?

PETER HART: I think what you really want to understand is the mood and the direction of the electorate. And the fact that we have Donald Trump ahead, don’t get caught up with that. Understand the anger behind that and what that is expressing.

J. ANN SELZER: Imagine how you would be thinking about this race — and let’s take the Republican side — if you didn’t know what the polls were saying. You have a field of 15, 16 candidates. How would you know who is doing well and who is not doing well if you didn’t have a poll to tell you? Would you predict Donald Trump? Would you predict Ben Carson?

So I think those polls are really very helpful at shaping the debate, and helping people decide how they want to vote.

JUDY WOODRUFF: As Iowa voters head into another caucus season, they will endure the onset of winter, as well as the continued barrage of polling calls, wanted or not.

Being Iowans, they will take it all in stride, and on a wintry February night, they will caucus for their candidate of choice, bearing out or not what the polls were showing.