Obama or Romney? How Accurate Are Economic Models?

BY Paul Solman  October 19, 2012 at 4:08 PM EDT

Last week, Paul Solman examined how polls and election markets parse data to determine who will win an election, in this case, the 2012 presidential race between President Barack Obama and former Mass. governor Mitt Romney.

One such pollster, featured in last week’s broadcast is Ray Fair, a professor of economics at Yale University, who has come up with his own model to predict elections.



Here is the extended conversation between Ray Fair and Paul Solman:

PAUL SOLMAN: So what does the Fair model tell us right at the moment about the race to Election Day?

RAY FAIR: The model says it’s a very close election. If the economy were booming, the equation would be predicting a pretty substantial Obama victory. If the economy were going into a recession, it would be a substantial Romney victory. The economy’s more or less in between. The growth rate is not negative; it’s rising at a rate of a couple of percentage points, so it’s kind of a mediocre economy, and combining that with Obama having the incumbency advantage, I believe the equation to be predicting, essentially, 50/50.

src="http://www.pbs.org/newshour/rundown/ms_logo_homepage_blog_horizontal.gif"
width="92" height="92" alt="Making Sense" />

PAUL SOLMAN: And what are the key variables?

There’s an output growth variable. You can either think of it as output growth, or employment growth, or the change in the unemployment rate. The variable I use is per capita GDP growth in the year of the election.

Inflation is also a variable. You and I are old enough to remember inflation as a bad thing. People don’t like inflation much. Students nowadays [have] never experienced inflation. When I tell them it’s a bad thing, they have no idea what I’m talking about. But inflation, historically, is a negative for voters.

PAUL SOLMAN: And then you do get an advantage to being an incumbent, right?

RAY FAIR: You get an advantage to be the person running again, yes, and that helps Obama some.

PAUL SOLMAN: And then the fourth variable: there’s a penalty for being a Democrat?

RAY FAIR: Penalty for being a Democrat. This is a sample that goes back to 1916. There seems to be a bias in favor of the Republicans, for reasons that I don’t know, but it sure is factored into the estimates. It’s not large, but it’s not zero, either.

PAUL SOLMAN: But it means that when you look at every election back to 1916, and you try to account for everything else, there’s still an advantage to being a Republican?

RAY FAIR: There is an advantage to being a Republican, yes. If everything else is held constant, the Republicans, over the elections since 1916, have done slightly better than the Democrats, and that’s what’s picked up by the equation.

PAUL SOLMAN: How accurate has the model been?

RAY FAIR: Well, the ex-post error is about 2.5 percent. If you look at the ex-ante errors — the actual errors [in] the predictions I made right before the elections — so truly an outside information prediction — the errors are more like 3 to 3.5 percent.

So the average mistake is somewhere between 2.5 and 3.5 percent, depending on which of these various measures you look at.

PAUL SOLMAN: And how many times have you called the wrong winner?

RAY FAIR: I’m not interested in that question — and it’s hard to answer that question, because of the electoral college, among other things — because what I’m interested in is testing this hypothesis: does the economy have an effect on voting behavior? And the best measure of that, probably, is vote share, not who wins or loses.

PAUL SOLMAN: What did the model say was going to happen in 2000?

RAY FAIR: The Democrats got 50.3 percent, and I was predicting 50.8 percent, so an error of 0.5. So, a terrific prediction, right? And yet I got the winner wrong, because I was predicting Gore was going to win with 50.8, and he lost with 50.3. But the 0.5 [error] is really good!

PAUL SOLMAN: And 2004 was the second George W. Bush election.

RAY FAIR: Kerry got 48.8 percent and was predicted to get 42.3 percent, so that was an error of 6.5 percent. There I got the winner right, but the error was actually not trivial. About twice the size of the standard error.

PAUL SOLMAN: I remember when I interviewed you then, you said: “Look, the formula isn’t taking a bunch of other things into account,” and I remember, if I’m correct, your saying that you thought Kerry would get more than the 42 percent that was being predicted.

RAY FAIR: Yes, because you always come to this time of year [when] we only have a month left. There’s a lot of information by that time that’s not in the economic variables that I’m using.

PAUL SOLMAN: So 2008, what happened?

RAY FAIR: That was a very good prediction for me, in that election, Obama got 53.7 percent and the prediction was 51.9 percent. The error then was 1.8 [percent], so that’s within the 2.5 percent error margin. That was a well predicted election. The winner was correctly predicted, but also the vote share was close to what actually happened.

PAUL SOLMAN: And that’s what you like.

RAY FAIR: That’s what I like, yes.

PAUL SOLMAN: So everybody else has a horse in this race, meaning either Obama or Romney. Your horse in this race is your equation?

RAY FAIR: I’m an econometrician, not a pundit, right? So the horse you’re interested in, is my equation.

But I have another side of me which I’m not talking to you about, which is what I personally would like to see.

PAUL SOLMAN: And you’re not going to tell me what that is.

RAY FAIR: I’m not going to tell you what that is, no.

PAUL SOLMAN: So, right now what’s Fair model say, right at this moment?

RAY FAIR: Right at this moment, it’s 49.5 percent of the two-party vote for Obama. Could be 50.1 percent, if the economy really does quite a bit better in the third quarter than I’m predicting it will, but somewhere around 50 percent. And so that’s not going to change.

PAUL SOLMAN: And 2.5 to 3 percentage points of error – something like that?

RAY FAIR: Yes.

PAUL SOLMAN: So you wouldn’t be surprised if it were 53/47 Obama/Romney, or 53/47 Romney/Obama popular vote?

RAY FAIR: Right.

It looked like before the debate perhaps Obama was going to get 54 to 55 percent of the two party vote. If I’m predicting 50 or 49.5, that’s a substantial error. So that would be an example where the error was large, but at the moment it doesn’t look that way, especially after the first debate. So if it is again roughly 50/50, then the equation will have done well.

So, it’s not like this explains everything, but it… what is interesting about it is it explains as much as it does, the economy.

PAUL SOLMAN: And if you were a betting man, would you bet on the Fair model, or would you bet on the prediction markets?

RAY FAIR: I would bet on the prediction markets. They have information in those markets that I don’t have in my equation. I did beat the [prediction markets] in 2000 when it was close. On average though, I think they have more information, and so I use them as well.

We at Making Sen$e are working on a story to explain where the monthly unemployment numbers come from. To do so, we are looking for interviewees who have worked on the Current Population Survey (CPS; household survey) and/or the Current Employment Statistics survey (CES; establishment survey). Are you a former surveyor? Do you know one? If so, we want to hear from you! Please email us at businessdesk@newshour.org. Be sure to include your contact information. Very much obliged.

This entry is cross-posted on the Making Sen$e page, where correspondent Paul Solman answers your economic and business questions.