Why do the polls seem to be showing such different results?

Why do the polls seem to be showing such different results?
Mark Blumenthal
10/20/04

First Rule of Poll Interpretation: Otherwise smart people tend to forget about sampling error as numbers approach fifty (or margins approach zero). But that would be a cop out. More important, it would be wrong, because despite all outward appearances, the Ohio numbers are actually amazingly consistent.

To understand how that could be, let us revisit the Incumbent Rule, the notion that the most important number in a race featuring an incumbent is that candidate's percentage of the vote. This rule itself is not an immutable law of physics, of course, but it relies on sound theories of psychological decision-making. Voters know incumbents better. Their attitudes toward sitting Presidents are especially well defined and firmly held, and the first stage of their decision is whether to support the incumbent. Regarding George W. Bush, at this stage in the campaign, the first part of the decision has largely been made.

By contrast, challengers are less well known and attitudes toward them tend to be more in flux, even at the end of the campaign. Vulnerable incumbents do their best to raise doubts about the challenger. Undecided voters are typically conflicted; unhappy with the incumbent's performance but also uncertain about the alternative. The challenger is typically perceived, almost by definition, as less qualified to be president, less experienced in the conduct of foreign affairs, harder to imagine as commander in chief. So some voters who are ready to "fire" the incumbent are hesitant to support the challenger, some may not feel truly comfortable with their decision until the moment they cast their ballot in the voting booth. More often than not, their first instinct is to tell pollsters they are undecided. As such, their attitudes are more susceptible to what pollsters call "measurement error" - how questions are phrased, whether the question offers a third candidate as a choice, how hard interviewers push for an answer, whether an interviewer or a recorded computer message asks the questions.

Now, let's look at the Ohio numbers and see how well these principles apply. The following table shows the presidential vote (with Nader included) on the four polls released today. George Bush's percentage is remarkably, almost impossibly consistent: 47%, 47%, 47, , 47% and 46%. All of the variation is between the Kerry, Nader and undecided. That is the incumbent rule in action.

Nader
Ohio Bush Kerry Other Und
ABC News 47% 50% 2% 1%
SurveyUSA 47% 49% 2% 1%
Rasmussen Reports 47% 47% 3% 2%
Fox/Opinion Dynamics 47% 45% 3% 5%
Ohio Poll 46% 48% 1% 5%

[Note: Although the issue remains in the courts, as of writing, Ralph Nader has been ruled off the Ohio ballot. For Fox News, the results above are based on a their first question, a three-way choice including Nader. They followed with a two-way question that put Bush ahead 49%-44%. Neither ABC, the Ohio Poll nor Rasmussen asked a three-way question, although all three did accept responses for other candidates when volunteered. Even at 49%, the Fox number is still under 50% and still quite close to the other surveys. More on this later...]

Looking back at the national Fox polls over the course of the year and notice that their undecided percentage is always higher. This difference is almost certainly about the text and structure of the vote question (as well as how the pollsters train their interviewers to ask it). The ABC News Poll pushes the initially undecided to say, "which candidate are you leaning toward?" Fox/Opinion Dynamics and Ohio Poll questions do not. The Ohio Poll is the only one that does not offer Ralph Nader as a choice. The surveys conducted by SurveyUSA and Rasmussen used a recorded voice and automated computer response system rather than an interviewer in a way that arguably simulates the solitary decision making of the voting booth (Survey USA also uses a subtle push that involves pausing briefly before presenting the undecided option: "For Bush, press 1, for Kerry press 2, for Nader press 3....[ pause] if you are undecided press 4" [Correction: SurveyUSA does not probe for Nader in Ohio].

Does the same pattern hold at the national level? Not quite as consistent as Ohio, but close in the surveys released today. The table below shows the results of the six surveys that released results today. Bush's percentages nationally are only slightly more varied: 51%, 49, 48%, 48%, 48%, and 45%. ABC News is a little higher tonight, Zogby a littler lower. Once again, Fox News and Zogby have higher numbers for undecided, neither pushes them to say how they lean.

National Bush Kerry Nader Other Undecided
ABC/Wash Post 51% 46% 1% 2%
Fox/Opinion Dynamics 49% 42% 2% 7%
NBC/WSJ 48% 48% 1% 3%
IBD/TIPP 48% 46% 2% 4%
Rasmussen 48% 47% 2% 3%
Zogby 45% 45% 2% 8%

Incidentally, in Ohio, you can see a similar pattern in Ohio's U.S. Senate race. Four polls put George Voinvich's support within a range of four points: 62%, 60%, 58%, 58%. By contrast, Eric Fingerhut's support is significantly lower on the Fox/Opinion Dynamics survey (28%) than in the other three (34-35%), because the Fox undecided percentage is higher (14% vs. 4-7%).

Voino- Finger - Ohio Senate vich hut Und
Ohio Poll 62% 35% 4%
ABC News 60% 35% 6%
SurveyUSA 58% 34% 7%
Fox/Opinion Dynamics 58% 28% 14%

There is a second possible explanation for the variation in Ohio. The Fox/Opinion Dynamics survey is the only one of the five that sets regional quotas in its sample to match past voting patterns. The other polls (Rasmussen being a possible exception) make sure their initial sample of adults matches the population regionally, but let their screens and models determine the regional distribution of likely voters. Ordinarily, I prefer the Fox/Opinion Dynamics approach, but the unprecedented efforts to register new voters in Ohio make me a bit uncomfortable relying on it blindly. If the Democrats have registered more new voters as the anecdotal evidence suggests, and those voters get through the likely voter models of the other polls, then the Fox/Opinion Dynamics may be sampling relatively fewer voters in Ohio's large urban counties. However - big hedge here - absent regional profiles of the various samples, we have no way of knowing for sure. Given the consistency of Bush's support in Ohio, I would guess that the regional differences are small.

Main point: you are watching the underlying principles of the Incumbent Rule in action. With Bush at roughly 47%, the race in Ohio is close. If the President's support remains at this level, he is likely headed for an Ohio defeat.