Get yer polls, bets, numbers & pretty graphs! Elections 2008

OK, that was a lot of rambling.

For clarity's sake: I guess the first post was more interesting in terms of reading the polls and testing the various interpretations, without ever reaching a straightforward explanation. While the second post is more about the meat of Obama's dilemma now, the way I see it anyhow.

[Edwards] could do okay, winning perhaps 40% of the the Dem votes, leaving 60% to be fought over by Obama and Clinton (I am making these numbers up, while waiting for nimh).

Right now, I would bet on Clinton.

(The inside joke is that fbaezer-from Mexico- and Johnboy-who doesn't have a tv, blindly make picks in the A2K Pro Football League games here. And we do about as good as the other 18 players).

Thank you. That is pretty much clear to me now. What, though, is "vig?"

McCain, a former Vietnam prisoner of war, argues that people in a state with a long military tradition, several bases and a large number of veterans could seek commander in chief qualities primarily in a candidate. He said of his decades of experience on military matters, "I certainly think it's helpful with a lot of South Carolinians."

At the same time, McCain hopes to benefit from a population spurt; South Carolina's numbers rose by nearly 10 percent since the decade began, with many people settling along the coastal region that McCain won in 2000. These transplants in retirement communities and resort towns tend to be more moderate on social issues. The newcomers have made the state even more Republican than it already was, but not necessarily more conservative, and that could benefit McCain.

New Hampshire: So What Happened?

There is obviously one and only one topic on the minds of those who follow polls today. What happened in New Hampshire? Why did every poll fail to predict Hillary Clinton's victory?

Let's begin by acknowledging the obvious. There is a problem here. Even if the discrepancy between the last polls and the results turns out to be about a big last minute shift to Hillary Clinton that the polls somehow missed (and that certainly sounds like a strong possibility), just about every consumer of the polling data got the impression that a Barack Obama victory was inevitable. One way or another, that's a problem.

For the best summary of the error itself, I highly recommend the graphics and summary Charles Franklin posted earlier today. Here's a highlight of how the result compared to our trend estimates:

What we see for the Democrats is quite stunning. The polls actually spread very evenly around the actual Obama vote. Whatever went wrong, it was NOT an overestimate of Obama's support. The standard trend estimate for Obama was 36.7%, the sensitive estimate was 39.0% and the last five poll average was 38.4%, all reasonably close to his actual 36.4%. It is the Clinton vote that was massively underestimated . . .Clinton's trend estimate was 30.4%, with the sensitive estimate even worse at 29.9% and the 5 poll average at 31.0% compared to her actual vote of 39.1%.

So what went wrong? We certainly have no shortage of theories. See Ambinder, Halperin, Kaus, and, for the conspiratorially minded, Friedman. The pollsters that have weighed in so far (that I've seen at least) are ABC's Gary Langer (also on video), Gallup's Frank Newport, Scott Rasmussen and John Zogby. Also, Nancy Mathiowetz, president of the American Association for Public Opinion Research (AAPOR) has blogged her thoughts on Huffington Post. [For the links to all the above, see original article - nimh]

Figuring out what happened and sorting through the possibilities is obviously a much bigger task than one blog post the morning after the election. But let me quickly review some of the more plausible or widely repeated theories and review what hard evidence we have, for the moment, regarding each.

1) A last minute shift? - Perhaps the polls had things about "right" as of the rolling snapshot taken from Saturday to Monday, but missed a final swing to Hillary Clinton that occurred over the last 24 hours and even as voters made their final decisions in the voting booth. After all, we knew that a big chunk of the Democratic electorate remained uncertain and conflicted, with strong positive impressions of all three Democratic front-runners. The final CNN/WMUR/UNH poll showed 21% of the Democrats "still trying to decide" which candidate they would support, and the exit poll showed 17% reported deciding on Election Day with another 21% deciding within the last three days. Polls showed Clinton polling in the mid to upper 30s during the late fall and early winter before a decline in December. Perhaps some supporters simply came home in the final hours of the campaign.

I did a quick comparison late last night of the crosstabs from the exit polls and final CNN/WMUR/UNH survey. Clinton's gains looked greatest among women and college educated voters. That pattern, if it also holds for other polls (a big if) seems suggestive of a late shift tied to the intense focus on Clinton's passionate and emotional remarks, especially over the last 24 hours of the campaign.

2) Too Many Independents? - One popular theory is that polls over-sampled independent voters who ultimately opted for a Republican ballot to vote for John McCain. I have not yet seen any hard turnout data on independents from the New Hampshire Secretary of State, but the exit poll data does not offer promising data for this theory. As I blogged yesterday, final Democratic polls put the percentage of registered independents (technically "undeclared" voters) at between 26% and 44% (on four polls that released the results of a party registration question). The exit poll reported the registered independent number as 42%, with another 6% reporting they were new registrants. So if anything polls may have had the independent share among Democrats too high.

On Republican samples, pre-election pollsters reported the registered independent numbers ranging between 21% and 34%. The exit poll put it at 34%, with 5% previously unregistered. So here too, the percentage of independents may have been too low.

Apply those percentages to the actual turnout, do a little math, and you get an estimate of how the undeclared voters split: roughly 60% took a Democratic ballot and 40% a Republican. That is precisely the split that CNN/WMUR/UNH found in their last poll although other

Keep in mind that the overall turnout was over 526,671 (or 53.3% of eligible adults). Eight years ago (the last time both parties had contested primaries) it was 396,385 (or 44.4% of eligible adults at the time). That helps explain why we may have seen an increase in independents in both parties.

Of course, we are missing a lot of data here: Nothing yet on undeclared voter participation from the Secretary of State, and roughly half the pollsters never released a result for party registration.

3) Wrong Likely Voters? OK, so maybe they had the independent share right, but perhaps pollsters still sampled the wrong "likely voters" by some other measure. The turnout above means that pollsters had to try to select (or model) a likely electorate that amounted to roughly half the adults in New Hampshire, they reached with a random digit dial sample.

Getting the right mix is always challenging, possibly more so because the Democratic turnout was so much higher than in previous elections. That's an argument blogged today by Allan McCutcheon of Edison Research:

In 2004, a (then) record of 219,787 voters turned out to vote--the previous record for the Democratic primary was in 1992, when 167, 819 voters participated. This year, a record shattering 287,849 voters participated in the New Hampshire Democratic primary--including nearly two thirds (66.3%) of the state's registered Democrats (up from 43.3% in 2004). Simply stated, the 2008 New Hampshire Democratic primary had a voter turnout rate that resembled a November presidential election, not a usual party primary, and the likely voter models for the polling organizations were focused on a primary--this time, that simply did not work.

One way to assess whether polls sampled the wrong kinds of voters would be to look carefully at their demographics (gender, age, education, region) and see how they compared to the exit poll and vote return data. Unfortunately, as is so often the case, only a handful of New Hampshire pollsters reported demographic composition.

4) The Bradley/Wilder effect? The term, as wikipedia tells us, derives from the 1982 gubernatorial campaign of Tom Bradley, then the long time African-American mayor of Los Angeles. Bradley led in pre-election polls but lost narrowly. A similar effect, in which polls understated the support for the opponents of African-American candidates seemed to hold in various instances during the 1980s. Consider this summary of polls compiled by the Pew Research Center for a 1998 report, which they updated in February 2007:

(click to enlarge)


Note that, in almost every instance, the polls were generally about right in the percentage estimate for African-American candidate but tended to underestimate the percentage won by their white opponents. The theory is that some respondents are reluctant to share an opinion that might create "social discomfort" between the respondent and the interviewer, such as telling a stranger on the telephone that you intend to oppose an African-American candidate.

Of course, the Pew Center also looked at six races for Senate and Governor in 2006 that featured an African-American candidate and did not see a similar effect. Also keep in mind that that all of the reports mentioned above that show the effect were from general election contests, not primaries.

What other evidence might suggest the Bradley/Wilder effect operating in New Hampshire in 2008? We might want to consider whether the race of interviewer or the use of an automated (interviewer-free) methodology would have an effect, although these kinds of analyses are difficult, because other variables can confound the analysis. For what it's worth, the final Rasmussen automated survey had Obama leading by seven points (37% to 30%), roughly the same margin as the other pollsters. We might also look at whether pushing undecided voters harder helped Clinton more than other candidates.

Update: My colleagues at AAPOR have made three relevant articles from Public Opinion Quarterly available to non-subscribers on the AAPOR web site.

5) Non-response bias? We would be crazy to rule it out, since even the best surveys are getting response rates in the low twenty percent range. If Clinton supporters were less willing to be interviewed last weekend than Obama supporters, it might contribute to the error. Unfortunately, it is next to impossible to investigate, since we have little or no data on the non-respondents. However, if pollsters were willing to be completely transparent, we might compare the results among those with relatively high response rates to those with lower rates. We might also check to see if response rates declined significantly over the final weekend.

6) Ballot Placement? Gary Langer's review points to a theory offered by University Prof. Jon Krosnick, that Clinton's placement near the top of the New Hampshire ballot boosted her vote total. Krosnick believes that ballot order netted Clinton "at least 3 percent more votes than Obama."

7) Weekend Interviewing? I blogged my concerns on Sunday. Hard data on whether this might be a factor are difficult to come by, but it is certainly an issue worth pursuing.

8) Fraud? As Marc Ambinder puts it, some are ready to believe "[t]here was a conspiracy, somehow, because pre-election polls are just so much more valid than actual vote counts." Put me down as dubious, but Brad Friedman's Brad Blog has the relevant Diebold connections for those who are interested.

Again, no one should interpret any of the above as the last word on what happened in New Hampshire. Most of these theories deserve more scrutiny and I agree with Gary Langer that "it is incumbent on us - and particularly on the producers of the New Hampshire pre-election polls - to look at the data, and to look closely, and to do it without prejudging." This is just a quick review, offering what information is most easily accessible. I am certain I will have more to say about this in coming days.

-- Mark Blumenthal

Trust on Issues
Unaffiliated Voters Often Trust Democrats Over Republicans; Domestic Concerns Trump Foreign Policy

Voters continue to trust Democrats more than Republicans on most key issues, according to the latest data from a Rasmussen Reports tracking poll.

Democrats get the nod on nine of ten issues we asked about. They lead by double digits on six of the ten: the Economy, Immigration, Government Ethics and Corruption, Health Care, Social Security, and Education.

On no issue are Republicans more widely trusted than Democrats. The best the GOP can currently achieve is parity: 43% to 43% on National Security and the War on Terror [..].

When the War in Iraq is isolated as a separate foreign-policy issue, Democrats enjoy an eight-point advantage of 47% to 39% [..].

The new data show an increase in pro-Democratic feeling on several fronts. For example, the GOP has slipped further on their once-signature issue of Taxes, where a plurality (48%) now trusts Democrats more than they trust Republicans (40%).

Republicans in November had a nominal edge of 39% to 38% on Abortion. Now Democrats have a seven-point advantage of 45% to 38%. On the more electorally potent issue of the Economy, Democrats now lead 48% to 35%; in November they led by only five percentage points. [..]

On Health Care, 57% of unaffiliated voters prefer Democrats, only 24% prefer Republicans. [..] 19% are not sure either is more deserving of their trust [..].

On Social Security, the Democrats' 20-point edge among unaffiliated voters is practically identical to the size of the margin they enjoy with all voters. [A]lmost a quarter (24%) of unaffiliated voters are Not Sure, versus 13% of all voters.

Of the ten issues we asked about, the five most important to all likely voters are the Economy (Very Important to 73%), Government Corruption (73%), Health Care (63%), Social Security (63%) and National Security and the War on Terror (62%).

For unaffiliated voters, the five top issues are Government Corruption (69%), the Economy (66%), Health Care (63%), Education (60%), and Social Security (59%).

Just 57% of unaffiliated voters say National Security and the War on Terror is a Very Important issue affecting their vote. Taxes are Very Important to 48%, Abortion to 27% [..].

Democrats continue to enjoy a double digit lead on the Generic Congressional Ballot.

But FWIW, for now Obama and Hillary are pretty much matched up 1 to 1 as both their numbers bounce up and down between 30-45% from poll to poll (with one poll directly after 'Iowa' giving Obama 50%).

Pollsters have a plan for Nevada: Skip it

Most say we're too green at caucusing, too transient
By Brian Eckhouse, Michael Mishak

Fri, Jan 11, 2008 (2 a.m.)

National public opinion pollsters, fresh off a glaring failure to pick the winner in New Hampshire's presidential primary, are now violently queasy about trying to predict a winner in Nevada.

In fact, for a variety of reasons, major news organizations are taking a pass on polling before Nevada's Jan. 19 caucus.

The concerns stem from the New Hampshire mistake and from knowledge that Nevada has a large transient population not familiar with the workings of a big-time caucus.

By comparison, New Hampshire is a state with a long history of political participation, a stable population and a history of respected surveys that have accurately gauged the preferences of the electorate. Nevertheless, the polls had Illinois Sen. Barack Obama leading New York Sen. Hillary Clinton on election day by an average margin of more than 8 percentage points.

Clinton won by 2 points.

In the world of pollsters, that's outside the statistical margin of error a train wreck.

The disparity in New Hampshire added to the anxiety of pollsters already uneasy about Nevada. Even NBC, whose cable news network, MSNBC, will broadcast Tuesday's Democratic debate in Las Vegas, has no plans to poll Nevada, said Peter Hart, the Washington, D.C.-based pollster whose firm typically conducts the NBC-Wall Street Journal poll.

Pollsters who do dare to try to gauge not only the preferences of Nevadans but also the likelihood they will turn out for a first-time prime-time caucus face a set of challenges.

Voters must show up at particular location at 11:30 a.m. on caucus day and publicly declare their support for a candidate in a process that could take more than an hour. In New Hampshire, voters had all day to cast a secret ballot in a voting booth a process familiar to citizens everywhere.

In polling language, a caucus is a "low incidence" event, meaning a relatively small percentage of eligible voters will show up to caucus.

Pollsters have to work harder to find a representative sample of likely caucusgoers, sometimes burning through thousands of households to find an acceptable pool of voters, at significant cost.

"It's very tricky," said J. Ann Selzer, whose firm contracts with the Des Moines Register newspaper to run the highly respected Iowa Poll every four years. "It should be impossible to ever get it right, even in Iowa."

For the record, Selzer's firm was spot on about Obama's victory, just one point off. But that accuracy came after decades of experience. Iowa has held a high-profile caucus since 1972. Selzer knows a likely caucusgoer when she sees one.

Larry Harris, a principal with Mason-Dixon Polling & Research, one of two national firms that conduct polling for Nevada media on occasion, acknowledged the difficulty. "It requires a lot more energy than somebody pulling a lever in a secret ballot," he said.

Nevada's highly transient and increasingly diverse population is another complication for pollsters. New Hampshire and Iowa are on the opposite end of that spectrum, with nearly all-white populations and relatively static population numbers.

"It's very difficult to know who to sample, in terms of likely voters," said Glen Bolger, a respected Republican pollster who has conducted surveys in Nevada.

The Pew Research Center, a respected public policy foundation, regards the circumstances here as so foreboding that it opted to leave Nevada out of its pre-election poll in December.

"It's an ascending chain of difficulty: general elections, primaries and caucuses," said Scott Keeter, Pew's director of survey research. "Caucuses burn up a lot of resources (for pollsters), and we thought our resources could be put to better use elsewhere."

Nevada's last poll was taken a month ago by Mason-Dixon, which says it will survey voters again before the caucus. (In the most recent poll, Clinton led in the single digits.) Infrequent polling is a problem, said Susan Pinkus, director of the Los Angeles Times Poll.

"My concern is that you really don't know what's going on in your state," she said. "Everybody's flying under the radar there. So much is predicated on speculation."

Even so, Pinkus returned to the problem of identifying likely caucusgoers in a state that has never caucused in a significant way. "You'd have to poll up to wazoo to find these caucusgoers."

Then there is the issue of race, which Andrew Kohut, president of the Pew Research Center, raised in The New York Times on Thursday.

The New Hampshire results were thrown, in part, by race. Polls frequently overstate support for black candidates among white voters, particularly white voters who are poor, he wrote. Poorer, less educated whites refuse to respond to surveys more often than affluent, better-educated whites and those who won't answer tend to have more unfavorable views of blacks than respondents who do the interviews.

The upshot: Obama's support was overstated.

Even before the New Hampshire primary, questions arose about the accuracy of polls anywhere this election season.

In past election cycles, presidential primary voters tended to be older. This year has ushered in a wave new voters, mostly younger, Harris noted.

Those young voters are more likely to have cell phones but not land lines and pollsters prefer land lines because they connect the voter to a specific location. So gauging the sentiment of younger voters is difficult.

Quote:

Pollsters have a plan for Nevada: Skip it

Most say we're too green at caucusing, too transient
By Brian Eckhouse, Michael Mishak

Fri, Jan 11, 2008 (2 a.m.)

The New Hampshire results were thrown, in part, by race. Polls frequently overstate support for black candidates among white voters, particularly white voters who are poor, he wrote. Poorer, less educated whites refuse to respond to surveys more often than affluent, better-educated whites and those who won't answer tend to have more unfavorable views of blacks than respondents who do the interviews.

The upshot: Obama's support was overstated.