David Moore is Vice-President and senior analyst at the Gallup Organization.
He is the managing editor of the Gallup Poll. He is the author of The Super Pollsters: How They Measure and Manipulate Public Opinion in America.
New River Media Interview with: David Moore
QUESTION: How important was George Gallup in the development of the modern opinion poll?
DAVID MOORE: George Gallup was probably the most important early pollster because of several things. Number one is he launched his "America Speaks" syndication in 1935. In 1936 he predicted the [presidential] race correctly. Now, Elmo Roper and Archibald Crossley also did, but George Gallup went out on the line more than they did. He actually guaranteed the newspapers that were subscribing to his poll that if he was wrong he would refund all their money. Now that was a lot of polling that he had been doing that whole year, so for him to agree to refund the money if he was wrong was a real big gamble.
And what he meant by being wrong is that he said he would be closer than the Literary Digest. In 1932 the Literary Digest had done an absolutely fantastic job of predicting the election within a couple percentage points, had predicted correctly 44 out of the 48 states. So he was really taking on what at that time appeared to be the giant of polling. That was another reason why Gallup was so correct because the stakes were so large, and he won.
And finally the other thing that he did at that time is he predicted what the Literary Digest poll would show using their technique, and he predicted that they would show Landon ahead of Roosevelt. And he was right, right within a couple of percentage points. That, of course, infuriated the people at the Literary Digest, but it made for good copy and it certainly pushed George Gallup's name out there to the forefront.
QUESTION: The Literary Digest's poll was also a pretty massive operation. Was this sort of a David and Goliath situation?
DAVID MOORE: One of the features of the Literary Digest poll is that the Literary Digest would send out around 10 to 15 million ballots. Actually they got back 10 to 15 million. They got back I think in 1936 they just got back about 2 million. But still George Gallup was doing polling on the basis of approximately 2,000 to 2,500 people across the country. So a lot of politicians at the time and said, "How can you believe George Gallup? I know the Literary Digest is polling because I know of people who have gotten the ballot, but I never see a George Gallup interviewer." So, you know, what's going on here?
And the Literary Digest not only sent out millions of ballots, but, of course, having been so accurate in the 1932 election when it came within a couple percentage points of predicting the election, was correct in 45 of the 48 states, was now going against this upstart former academic, who was saying, "Oh no, you're going to be wrong, because you're using a system of sampling from telephones and auto lists, people who are richer. You're using this sampling procedure that is going to lead you to an incorrect conclusion."
Well, at the time in 1936 not a whole lot of people believed George Gallup and a lot of people believed the Literary Digest, much to George Gallup's glee when the final results came forth.
QUESTION: Why was Gallup right and the Digest so wrong?
DAVID MOORE: Gallup was right in 1936 because the structure of the voting had changed from 1932. In 1932 rich people and poor people were all dissatisfied with the Republicans, because of the Great Depression, because of the fall of the stock market and things of that nature. But by 1936 it had become a class election so that disproportionately the poorer people were voting for the Democrats and the more wealthy people were voting for the Republicans.
So when the Literary Digest was sampling based upon people who had telephones and who add autos, they were disproportionately getting the richer people who were going to vote for the Republicans and so their results reflected the notion that Alf Landon was going to beat FDR, whereas Gallup intended to get people of all economic levels.
And when he sent his interviewers out across the country they had quotas they had to fill to make sure that they had people with high, low and of course medium income levels. So by ensuring that he had a proper distribution among the income levels he got a more accurate sample of the general public, and therefore was accurate in his final analysis.
QUESTION: How was he able to predict what the Digest's results would be?
DAVID MOORE: In order to predict what the Literary Digest might come up with, George Gallup almost casually said, "You know, I think I will use a tel-auto lists, the way in which the Literary Digest does", and he didn't use as many, and he didn't send out as many millions, but just sent out, you know, maybe a couple thousand, figuring that he would get some idea of how different the tel-auto lists would be from his own sample, which was more systematic.
And when he got the results back, he tabulated them and it showed that Alf Landon would beat FDR in this tel-auto list by about the same proportion that the millions of tallies that the Literary Digest did also showed later on that year. So in the spring of the election year George Gallup was saying, "This is what the Literary Digest is going to find." And the editor of the Literary Digest was almost apoplectic, because they hadn't even sent out their first ballot yet. And so he was very critical of this unknown person named George Gallup, but ultimately of course the Literary Digest folded and George Gallup did not.
QUESTION: What did Gallup's success in predicting this election mean for him and for public opinion polling in America?
DAVID MOORE: The success of the 1936 election meant that the new scientific pollsters, which included Elmo Roper and Archibald Crossley, along with George Gallup now had credibility. It also meant that polling expanded into a lot of commercial areas. It's also used sociologically. This election demonstrated that the so-called scientific polling was much more accurate. For George Gallup it meant that, of course, he had more subscribers. It meant that the Gallup Poll became famous and it meant that people had more belief now in these polls.
Still, as there is even today, there was skepticism that a poll of only about two or three thousand people nationwide could really reflect what everybody was thinking. But at least the 1936 election said, "Well, it's a lot better than getting ten million if you got the wrong ten million."
QUESTION: Tell me about the presidential election of 1948. What happened and why did Gallup and other scientific pollsters miss the boat so badly?
DAVID MOORE: 1948 was a big disaster for the polling industry, for the scientific pollsters. The major problem with the polling in 1948 is that the pollsters did not believe that much could change. After all, they had done polling in 1936, 1940, 1944. They had done polling early on in the presidential year and it hadn't really changed very much in the whole election year. There was no real volatility of the electorate as we see today.
Now, one possible explanation for that is that they didn't really have as serious campaigning as we have today, and more important it was the same presidential candidate every time, FDR. But in 1948 it was the first time that FDR wasn't in the race, so people had two new candidates to consider. And Elmo Roper was probably the most eloquent in his misperception, because he said, "I think what we ought to do is do one poll sometime in the fall and then not do anymore because nothing could possibly change between now and the election." Gallup also did not poll all the way up until the election, but stopped about a week and a half or 10 days early, thinking that not much could change.
Now today we would really be surprised if they did that, because, after all, at that point Dewey was leading Truman by only about 6 or 7 percentage points. With a 6 or 7 percentage point lead today no one would say, "The race is over." We'd say, "A lot can happen" because we have the experience, but at that time they didn't know.
And a lot did happen and probably the most significant thing is that Harry Truman's whistle-stop campaign on the train in the last closing weeks had enough of an impact on public opinion to help switch the vote over in his favor.
But that was an explanation that came after the election. Before the election everybody - not just the pollsters - the politicians, the Broadway pamphlet came out asking, "What are we going to do during the Dewey administration?" Newsweek, Time, everybody came out expecting that Dewey was going to win. And it was in large part because the polls had shown that and because the pollsters were so confident.
So the net consequence of this disaster was a real setback for the polling industry, not only public opinion polling but commercially as well.
QUESTION: In addition to stopping the polling too soon, what about the type of the sampling method he was using?
DAVID MOORE: Most of the advances in sampling were being done by academics. And one of the advances had been that instead of doing the quota sampling, which George Gallup had been doing since 1936, they ought to do what they call probability sampling. Quota sampling is just saying, "Let's go out to the various states, to the various regions and make sure we've got so many women, so many men, so many high income, so many medium, so many low income people, so many of this age and so on." So you make sure that you've got a distribution.
Random sampling is different. It says everybody has an equal chance of being selected and as a consequence you'll get the right proportions of men and women, of people by age and income.
Some people felt that the 1948 election polling debacle was a result of Gallup and other pollsters at the time continuing to use the quota method rather than the probability sampling. And by the way today everybody uses probability sampling. As a matter of fact, the very next election even the public pollsters used it. They weren't going to get caught again.
Still, a very close analysis of the results suggests that that wasn't really the problem. Quota sampling can be okay. Even, you know, throughout the 1970s and 1980s and most of the 1990s quota sampling, if done correctly, can come pretty close. So it wasn't really a sampling problem. Still, I wouldn't want to use quota sampling today, because we have a better method, and particularly with telephone interviewing you can do the right method. But a lot of people felt that that was possibly the reason for the error in 1948.
QUESTION: Can you really get valid data from just a small number of people? Are these random probability samplings really accurate?
DAVID MOORE: When trying to explain sampling, Gallup always used this analogy. He said, "Assume you've got a pool outside and all the water has been removed and instead what's been dumped into there are jillions and jillions of marbles, a certain number of red marbles, a certain number of white marbles and they've all been mixed up. And your task is to find out what's the proportion of red and what's the proportion of white marbles in this great big pool. How are you going to do it?
"You're not going to count every single one of those marbles. What you're probably going to do is go in and get a bucket, which will be a sample, and count out the distribution. Now, if they're all completely mixed you've got a good example. The problem, of course, is if most of the red marbles are on one side and most of white marbles are on the other, then you've got a problem of trying to make sure that you pull out a proportionate number."
In any case, it's the same concept. You don't need to poll everybody. If you can get a good representative sample, small number of people but have all of the characteristics of a larger group, then you don't need a large sample. In fact, typically the samples that are used today are about a thousand nationwide. In most states pollsters tend to use about 500. Nationwide, if we use a thousand we say that we'll come in with what we would have gotten had we polled everybody; 95 percent of the time we'll come in within about 3 percentage points. If you do 500 you're up to a margin of error of about 5 percentage points. So, the sample has been tested by statisticians, not just pollsters but statisticians. It's a good technique if it's done correctly.
QUESTION: As a result of the 1948 debacle, what then happened to public opinion of polling then and through the 1950s and 1960s?
DAVID MOORE: The consequence of the 1948 debacle was probably greater for some of the commercial clients. For example, up until that time Gallup had been doing a lot of movie research. The Best Years of Our Lives was a movie, was a 1947 movie that won the Academy Award that had been tested by Gallup, all the characters and things like that. But after 1948 a lot of the artists in Hollywood, who didn't like the whole concept of concept testing anyway, said, "Look, you can't even trust Gallup anymore." So the movie business for Gallup really went down.
Oddly enough there weren't that many cancellations of the newspapers, because even if you make a mistake, you know, you still say, "Well, what else is there? If you want to find out what the public's thinking, how are you going to do it? You're going to go back to Literary Digest or are you going to hope that the pollsters figure out what they did was wrong and improve it?"
So while there was some initial setback, I think in the long run it didn't really hurt the advancement of polling. Perhaps it helped it by forcing all of the public opinion pollsters now to move to the new methodology of sampling rather than the old antiquated one, and as a consequence it essentially grew.
QUESTION: Can't polls be used to manipulate public opinion as well as measure it?
DAVID MOORE: I think it's much more difficult today to say that polls can be used to manipulate public opinion, because there are so many polls. The advantage of having all the major network polls is that if there is a bias question other polling organizations who are competing with each other are going to point that out or ask their own questions. And I think one can safely say that among the major media polling organizations that are publishing results on a regular basis, national polling organizations, that they want to be accurate. They don't want to manipulate and they constantly check each other.
For example, during that whole period of [Clinton's] impeachment I would say that that was probably one of the best polled set of events that we've had, because you had so many polling organizations trying to figure out what was really going on, trying to understand how the public might continue to support a president who was getting all of this other negative kind of publicity. So the polls really tried to look at the issue from a lot of different angles. And because there were so many polling organizations it wouldn't have mattered, if one organization had had a bias for or against them it wouldn't have mattered. There were four or five others that were always going at it at the same time.
So from my way of thinking it was a dangerous period during the 1960s and 1970s when there were only two polling organizations [Gallup and Harris]. Too much emphasis was placed upon the results of those polls, too little recognition of how polling results can change with subtle changes in question wording and order. And so that now with more polls, more competition, we're in a better situation so that no one polling organization can dominate and no real manipulation of the public and more important of public opinions appearance so that it has an influence in Washington in a negative way can happen because of all these organizations.
QUESTION: What kind of criticisms do you hear about our reliance on public opinion polling today, and are some of them valid?
DAVID MOORE: Some people have argued that there are too many polls, that as a consequence politicians rely on polls and that's the question, what are they supposed to rely on? Well, one could say they should rely on their own views rather than what the public thinks or they should rely on their own views rather than what polls show. They should be leaders.
I'm a political scientist first and foremost. My sense is this: Politicians will always find a way to cater to the masses. If there are no polls, that didn't mean that politicians didn't try to cater to the masses, they just found different ways of doing that. When they would go back to their districts, they would go to, you know, Lions Club meetings and all the other kinds of meetings, they'd find out what the people would say. People who wrote to them were given undue influence, because, "Oh, I got letter from this constituent or I got a campaign of letters from these constituents," and he might pay more attention to those.
So I think those politicians who are not going to exercise leadership, who are going to try to kow-tow too much to the public are going to do it whether it's by polls or some other way. Those who believe that there are certain principles, that probably in the long run would help their constituency and they're going to follow them, will use polls to find out what the public is thinking and then maybe say, "It looks like I've got an educational problem on my hands. I've got to go out and explain my point of view to the public given what I know and they don't know." So I don't think polls change the character of people. It's just an instrument that can be used for good and for ill.
QUESTION: Is there anything about the use of polls today that gives you pause?
DAVID MOORE: One of the problems the polling industry is facing right now is a lower response rate. More and more people are refusing to respond, or if they don't refuse directly they make it difficult for us to get a hold of them. They've got answering machines. They've got cell phones. They've got, you know, screening devices and things of that nature. And so one is that the refusal rate has gone up. Once you actually get a hold the people, the refusal rate is greater than it used to be. It's harder to get a hold of people because of all the various other ways in which they can not be available.
[Something] that the polling industry would like to solve is the problem of telemarketing. We feel that the telemarketers are our problem, because people pick up a phone and you've got somebody who's trying to sell you something, so now you have the good pollsters, us, the legitimate pollsters who call you up and say, "Oh, we want your opinion. We're not trying to sell you anything." But you don't believe us anymore because that's what a lot of these other people are doing, they're selling under the guise of polling, which we call slugging. So they're out there slugging and we're trying to do something legitimate and the consumers are getting conflicted messages.
QUESTION: What was George Gallup's view of the role of public opinion polling in a democracy and what did he see as its potential?
DAVID MOORE: Gallup's vision for polling was that it would be a way to monitor the pulse of democracy. Up until then even with Literary Digest, which only did a poll around election time or maybe once every two years, there was no way to ascertain what the public in general was thinking about a lot of public issues of the day. You just had to wait until an election when people would go and vote in our vote out those people who had made the decisions, but there was no way really to figure out what the meaning of an election was without knowing why people voted a certain way.
So he felt that by having polling continuously we would essentially get the reaction of the public to a lot of various issues. He was also very attuned to the notion that you can ask people questions about issues that they aren't aware of, and so he came up with his five-step method, that's called his quintessential method for making sure that we ask people what they've heard about it, how involved are they about it, and then once we ask about the issue and then, you know, how intensely they feel about it. There simply isn't enough time in any given poll for every single issue to do it that way, but he was attuned to it.
I think today a lot of our polling asks people about issues that they don't know anything about. We don't always do the question that says, "Do you favor, oppose, or are you unsure?" We often, even though we don't give them projective questions, we often give them forced choice questions, thinking that whatever they're inclined to at the moment, at least has some relevance for what we're reporting.
In any case, I would say that Gallup's vision of the polling industry that helps to monitor the pulse of democracy has largely been fulfilled, that the industry is, in fact, providing as accurate information as we can about opinion that is constantly in flux about a lot of different issues, but nevertheless is at least of some relevance in a democratic form of government.
QUESTION: Jumping back to 1935, the very first question in the very first Gallup poll found that 60 percent of the people thought government was spending too much on relief and recovery. This runs counter to what most people would think about New Deal America. Do you have any thoughts on why he got the result?
DAVID MOORE: In 1935 people had a lot of assumptions about what the public was thinking. Although it's true that the very first question that Gallup asked about whether the government was spending too much, too little or about the right amount for the New Deal, and he showed that people felt the government was spending too much, that viewpoint, that kind of skeptical viewpoint about the government spending money has remained a very important part of American politics for these last sixty-five years.
I think in retrospect it's not as surprising as it was at the time, because at the time people thought, "Well, the government is helping a recovery." Now, it wasn't absolutely clear to all of the people necessarily that the government spending was always beneficial.
We also know that question wording could play a part. For example, today if we asked whether there should be more or less government spending on some things, we often, particularly on welfare, if we use the term "welfare", you get a strong majority say no. But if you ask whether or not the government should provide more, about the same, or less assistance to the poor, you get more people saying support for assistance to the poor.
So, I'm not sure myself when I look back at that question whether it was a question wording or a problem, and had it been worded differently he might have gotten more support, or whether it was just a part of the general American skepticism about government spending, one or the other.
QUESTION: Just a few months later he asked the question of whether people favored old age pensions, and over 90 percent said yes. Is that not contradictory with the previous poll?
DAVID MOORE: I think what Gallup was finding in 1935, 1936 is what we find today, and that is on the one hand people have a certain set of ideas about what government role should be. We find that, for example, with respect to guns, we say, "Do you think there should be more laws regulating guns or not?" And a slight majority of people say, "No, we don't need more laws." And then we say, "Would you favor or oppose" and we mentioned a whole bunch of new laws such as registration of handguns and trigger locks and things of that nature, and an overwhelming majority say yes.
So there is oftentimes a conflict in the public opinion results between a conceptual, general matter on the one hand and very specific items on the other. What Gallup found in 1935 is that while people felt too much spending for relief by the government, that most people felt there was too much spending, that when it came down to a very specific program they supported it by again strong majorities. And we find that kind of conflict today, apparent conflict today in public opinion polling.
Sometimes the reasons we get would appear to be conflicting responses in poll questions, where people support one thing and then seem to support exactly the opposite is that we're addressing issues that are very complex. And sometimes when you hit it from one angle what you get is people's general feeling that, yes, we ought to spend some money on that item, but over here, no, we ought to have a balanced budget. And so sometimes questions can appear to be conflicting.
And what we've discovered now is that the only way to try to resolve those apparent conflicts is to do trade-off questions, where we ask people, "Well, would you rather have A or B", and sometimes in that approach we can take what otherwise seem to be very conflicting ideas among the American public and make them more understandable.
Sometimes one of the reasons we have a conflict is that too much is going on. The issue itself is very complex. There's a lot of changes that are going on, and there's no single set of questions, maybe one or two, three questions that could really address the full complexity of the issue. So we report it out, but somebody else asking it a little bit differently might come up with a very different viewpoint. And both could be correct, but still apparently conflict.