17
   

Get yer polls, bets, numbers & pretty graphs! Elections 2008

 
 
nimh
 
  1  
Reply Tue 1 Jan, 2008 11:37 pm
Just to continue my eternal references to that other site I'm obsessed with (pollster.com), they had a very interesting post a couple days back too - on which pollsters are the most trusted.

It's a question that's kept returning in discussions here: which ones are good, which ones should be mistrusted? All very subjective of course, everyone has their faves it seems. Rasmussen, for example, is belittled by some, praised by others. Zogby is much maligned from different quarters. When the name of one of the alphabet networks appears on the title of a poll, some conservatives are instantly suspicious; when it's Fox some liberals are.

All of which is curious, since in the end, the differences between what the different pollsters show are relatively small. That is to say, divergences between any one poll by one pollster and a subsequent one by another can be huge; but they all show the same trends over time. Moreover, "house effects" that are demonstrated rarely pair up neatly with expected political preferences.

For example, Fox News polls once got some flak here for consistently showing a higher job approval, and lower disapproval, for President Bush than other polls did. But Thomas found out -- twice, even - that actually, it also always had President Clinton's ratings higher than the other polls. Seems like "house effects" are more likely due to methodological differences than to political bias.

That said, there are more and less experienced and reputable polling agencies. Gallup is sometimes mentioned as the golden standard because of its long history, though I'll note that one odd thing is how its numbers often bounce up and down more than average.

But how do you measure how good a poll is? Obvious way to check seems to simply compare last polls out with the eventual election outcome. And for what it's worth, after the '04 elections I listed which pollsters did the best - and which the worst - in getting the nearest to the results Bush and Kerry got. (Pew, the Tarrance Group, and TIPP did best.)

But that's not really fair, is it? After all, even polls done in the last week of campaigning don't claim to predict the result, anyhow; they merely claim to represent public opinion at the time they were done, and there can always be last-moment changes in preference on the day of polling itself or the last two-three days. Also, for the pollsters that focus on nationwide polls, there is only the one or two results for each election cycle to test them on, so there is some arbitrariness involved.

So Pollster.com did something else - they did a survey of pollsters, focusing specifically on the Iowa polls. And the pollsters turned out to be pretty critical of each others work:

Poll of Pollsters: Rating the IA Polls

Quote:
We sent out invitations to just over a hundred pollsters and had 46 complete the entire survey, although a few more (49) completed the questions about the reliability of the polls in Iowa. Of those, 22 are media pollsters and and 27 campaign pollsters (16 Democrats and 11 Republicans). There is no margin of error because the results represent nothing more or less than the views of the pollsters that participated. [..]

We started with a simple question asked about each of the 16 pollsters that have released public polls in Iowa: "How reliable do you consider surveys of IOWA CAUCUS goers done by each of the following organizations, very reliable, somewhat reliable, not very reliable or not reliable at all?" [..]

We left "reliability" in the eye of the beholder, but it is fair to assume that few are in a position to evaluate the performance of each organization in past Iowa caucuses. [..] Instead, it is safe to assume that most based their judgements on the reputation of each organization and its methods. As you will see, the pollsters had little trouble making such judgements.

As the following table shows, the Des Moines Register "Iowa Poll" conducted by Selzer and Company easily earns the highest marks, with virtually all rating it either very (36%) or somewhat (50%) reliable. The other pollsters with the highest scores are nationally known media surveys: ABC/Washington Post, the Pew Research Center and CBS/New York Times.

http://www.pollster.com/blogs/12-28%20PP%20All%20Pollsters.png

The pollsters receiving the lowest scores are Zogby International, the American Research Group and Rasmussen Reports. In the case of Zogby, four out of five pollsters rated their surveys as not very (28%) or not at all reliable (52%).

Check the original item if you're interested in how the judgements differed between the media pollsters and their campaign consultant colleagues; the latter were more critical.

They also asked, specifically, which one pollster was deemed the best - and worst:

Quote:
We also asked our pollster-respondents to select from the same list the "pollsters you consider MOST and LEAST reliable in Iowa." As the table below shows, the Des Moines Register/Selzer survey easily stands out as the favorite, especially among the campaign pollsters:

http://www.pollster.com/blogs/12-28%20PP%20most.png

As for the least reliable pollster in Iowa, once choice easily led the pack. One third (33%) of the campaign pollsters and just less than half (45%) of the media pollsters picked Zogby International:

http://www.pollster.com/blogs/12-28%20PP%20least.png

Again, reputation cannot tell us everything we need to know about the quality of the numbers a pollster produces. Pollsters with poor reputations may conduct quality polls and even the best pollsters are fallible. However, when a pollster earns the respect of their colleagues, it should tell us something.


There were some interesting comments as well, though - raising the question how much stock one should put in reputation, exactly?

Quote:
I wonder whether these results correlate with anything in the survey results. I agree that the judgments are at least primarily based on reputation, but it would be interesting to know whether the most highly regarded pollsters are getting different answers.

Just at first glance, Zogby seemed pretty much in line with the averages. My impression is that Zogby has damaged its reputation in lots of ways that have little to do with its capacity to field a decent pre-election poll. [..] I have to admit, what I remember from 2004 is not what Zogby's pre-election polls did or didn't show, but the way Zogby came up with a late-afternoon Election Day "call" that gave Kerry a comfortable EV win. Zogby's last Ohio poll apparently gave Bush a 6-point lead in Ohio, yet on Election Day he confidently assigned it to Kerry. [..]

Posted by: Mark Lindeman

Funny. Everybody hates Zogby but his final poll in the 2004 Iowa caucuses was nearly identical to the Des Moines Register/Seltzer poll, which everyone loves. [..]

In addition to those two, Democracy Corps, Pew, Research 2000, the LA Times, the Quad-City Times [..] and Survey USA all polled the caucus race during the 2004 season and they all generally agreed with one another fairly well except for Survey USA [..].

Posted by: CalD
0 Replies
 
nimh
 
  1  
Reply Tue 1 Jan, 2008 11:55 pm
Quote:
As the following table shows, the Des Moines Register "Iowa Poll" conducted by Selzer and Company easily earns the highest marks, with virtually all rating it either very (36%) or somewhat (50%) reliable. The other pollsters with the highest scores are nationally known media surveys: ABC/Washington Post, the Pew Research Center and CBS/New York Times.

Unfortunately, the last Pew poll out is from mid-November, and the last CBS poll in Iowa was done in early November. The last ABC/WaPo poll was done on 13-17 December, and showed Obama at 33%, Hillary at 29%, and Edwards at 20%.

The only pollster from this top list that's had a poll out in the last two weeks is Selzer & Co. for the Des Moines Register. It was conducted on 12/27-30. So perhaps that one should be accorded extra attention - which is good news for Obama fans!

Butrflynet already posted the results for the Democratic race on the Obama thread, and you'll see why:

32% Obama
25% Clinton
24% Edwards
6% Richardson
4% Biden

Here's the Republican race:

32% Huckabee
26% Romney
13% McCain
9% Thompson
9% Paul
5% Giuliani ( Exclamation )
0 Replies
 
Cycloptichorn
 
  1  
Reply Wed 2 Jan, 2008 12:12 pm
Pew national poll.

http://www.talkingpointsmemo.com/images/2008-01-02_pew_chart_gop.jpg

Few things have been as satisfying to me as watching Rudy's numbers decline as traditional Republicans realize that he's basically a corrupt Democrat hawk. Exactly as predicted by yours truly over a year ago.

Cycloptichorn
0 Replies
 
nimh
 
  1  
Reply Wed 2 Jan, 2008 02:27 pm
Cycloptichorn wrote:
Few things have been as satisfying to me as watching Rudy's numbers decline as traditional Republicans realize that he's basically a corrupt Democrat hawk.

It couldnt happen to a more deserving guy.. Twisted Evil
0 Replies
 
nimh
 
  1  
Reply Wed 2 Jan, 2008 03:08 pm
On a state-by-state level, Hillary more electable than Obama, polls suggest

There has been a lot of discussion about who is the more "electable" Democratic candidate. Hillary, Obama and Edwards all have claimed or suggested that they were the most electable candidate, using different arguments: experience, cross-over appeal and occasionally, how they are polling.

In blogs and the like, discussions about how the polls show that this or that candidate is the more electable are more explicit. We've had them here too. But usually, the only references are to national match-up polls. There is little reference to how the candidates' appeal against possible Republican contenders plays out region by region, state by state. How does Hillary do, in relative terms, out West? How does Obama do in the South? When you dig into the state-by-state polling that's been done, it looks like Hillary might, surprisingly, be the more electable candidate.


There are many ways to assess electability. Personality, ideology, political baggage. Questions about how the gender and race of the candidate plays out. And to be sure, a year out from the general elections, opinion polls are of modest significance. You're not going to go just on polls, at this point in time. Moreover, qualifications go both ways.

For example, Obama and Edwards supporters point to Hillary's high unfavourability ratings; but Hillary supporters counter that her negatives are only so high because she's already been in the crosshairs of the Republicans and the conservative media for a decade and a half. Once those folks are done with a nominee called Obama or Edwards, his negatives would be equally high.

On the other hand, Obama and Edwards supporters might claim that their candidate will still have a chance to present himself to the public that Hillary, already so familiar a figure, will not have. For now, they might argue, interest in the presidential race has been superficial at best, beyond political enthusiasts and those in the early primary states. Even taking the media blitz that surrounded Obama's arrival on the scene and Edwards' prior run in 2004 into account, public judgements of these candidates will for now still be superficial, and receptive to change.

Everyone has heard such qualifications of the significance of polls. However, those polls are often cited anyway. Notably national polls. We discussed how Obama claimed that "right now among all Democratic nominees or Democratic candidates I do better in a general election match ups than the other candidates." To evaluate claims like those, I tracked the national match-up polls since March. A set of resulting graphs from November can be found here.

But state-by-state polling turns out to provide a different picture. I was already tracking how Hillary was matching up in state-level polls against the Republican frontrunners, I posted some tables here. Now I've gathered data for Obama and Edwards as well. Data on Edwards is too scarce to say anything much, but a comparison between Hillary and Obama surprisingly seems to show Hillary doing better.

Below is a first table, which lists all the state-level match-up polls I could find for the last three months pitting Hillary and Obama against Rudy Giuliani. Other tables for how they're doing against McCain, Romney and Huckabee are forthcoming. Giuliani, of course, appears to have almost become a non-factor in the Republican race; but since he was long the apparent frontrunner, there is simply much more data available for how the Democrats stack up against him than against any of the others.

Most of the polls are by Survey USA, Rasmussen or Quinnipiac, so that's how I've organised the table. Survey USA often has the Democrats doing better than Rasmussen. The Quinnipiac polls seem to mostly be in between. In addition to those three, I found nine other pollsters that each did one poll; I've inserted those data pragmatically in the tables where there was place, but each is indicated by a lettered reference in the cell to the left or right.


http://img84.imageshack.us/img84/2990/demsvsgiulianistates2xd4.png


How to read this table: on the left are the states for which polling was available, ranked from the state where Kerry led Bush by the largest margin to the most Bush-friendly state. The margin with which Kerry or Bush led is given in the first column. The colours represent how "blue" or "red" the state was in those elections.

Then come the state-level match-up polls: the number given is the margin with which the Democratic candidate leads or trails. The cells are coloured in accordingly. If there were two polls by the same pollster in the same month, both numbers are given divided by a |, and the colour represents the average.

Finally, on the right of all the data for Hillary vs Giuliani and of all the date for Obama vs Giuliani, I've weighed out the colour that represents the balance of the polls for each state. Basically, the colour is based on the average of the polls, though in borderline cases I've taken into account that Survey USA may lean optimistically and Rasmussen pessimistically, and that polls from December are more significant than polls from October. Yellow represents a complete toss-up.

WARNING! You will notice that the variations from one poll to the next are large. When looking at match-up polls, keep in mind how the margin between two candidates is significantly more sensitive to volatility than the numbers polled for each candidate, as margin of errors double when looking at the margin between two numbers. This is how I tried to explain it (but I'm no expert)...

This is a simplified side-by-side view of how the Hillary vs Giuliani and the Obama vs Giuliani numbers compare with each other and with Kerry's score against Bush:


http://img161.imageshack.us/img161/3672/demsvsgiulianistatessimeg8.png
0 Replies
 
nimh
 
  1  
Reply Wed 2 Jan, 2008 05:50 pm
This is what the electoral maps to go along with those data would look like. Made according to the same recipe as these here were, extrapolating the result for states that have not been polled at all yet on the basis of how comparable states are showing up on the polls.

http://img46.imageshack.us/img46/6238/hillarygiulianimap03010iq0.png

http://img46.imageshack.us/img46/2198/obamagiulianimap030108sc0.png
0 Replies
 
Cycloptichorn
 
  1  
Reply Wed 2 Jan, 2008 05:58 pm
Nimh, as it looks increasingly like Giuliani won't be the Republican candidate, how valid are these comparisons?

Cycloptichorn
0 Replies
 
nimh
 
  1  
Reply Wed 2 Jan, 2008 06:09 pm
Cycloptichorn wrote:
Nimh, as it looks increasingly like Giuliani won't be the Republican candidate, how valid are these comparisons?

Fair point. Like I just wrote, Giuliani is of course almost a non-factor in the Republican race now. But because he was so long the apparent frontrunner, there is simply much more data available for how the Democrats stack up against him than against the others.

You have to look back over the last three months to get at least a fair number of polls (which shouldnt necessarily be a problem, as the Democratic race has been fairly stable during this time, outside the early primary states), and for most of those three months Giuliani was still considered the #1 Republican to poll for.

That said, I will post equivalent tables and maps for how Hillary and Obama have been stacking up against McCain, Romney and Huckabee as well. The tables will show that there's fewer polls for those, but there's still a fair number, especially from Survey USA. (Only Huckabee is a bit of a special case as he started performing much better over these three months. There's been little consistent change for any of the other candidates, Dem or Rep).
0 Replies
 
nimh
 
  1  
Reply Wed 2 Jan, 2008 08:14 pm
Here's the table for how Hillary and Obama have been polling against John McCain in state polls.

It's a good reminder of how it's McCain who's been polling the best against the Democrats for a while now. In the still unlikely case the Republicans pick him, he would be a formidable enough opponent.


http://img137.imageshack.us/img137/2838/demsvsmccainstateswf8.png


Interesting stuff. I'd like to come back later to not so much focus on how Hillary and Obama compare in these match-ups, but on some of the regional trends that are hinted at in these numbers, especially for Hillary's races where there are more polls.

In the meantime, I'll also post the accompanying maps to these match-ups with McCain - but those come with a warning. Compared to the Giuliani maps, more states here are extrapolated rather than based on polls. For the match-ups against Giuliani, there was at least one recent poll for 32 states in the case of Hillary, and 25 states in the case of Obama. With McCain, it's 26 and 19. All the rest is extrapolated. The extrapolated ones are mostly obvious ones, concentrated at the far ends of the spectrum (DC, Vermont, Idaho, Wyoming), but still.

Plus, for those states where there are polls, there are fewer here than with the race against Giuliani. But the biggest difference here is not between Giuliani and McCain as opponent, but between Hillary and Obama. There's far more match-up polls involving Hillary than with Obama. In fact, the total of individual state polls in the last three months was, respectively:

107 - for the Giuliani vs Hillary race
92 - for McCain vs Hillary
62 - for Giuliani vs Obama
44 - for McCain vs Obama.

The fewer polls, the more tentative the map. So take the Obama maps with a bigger grain of salt than the Hillary ones. For example, there's only two polls from the last three months matching Obama up against McCain in Massachusetts, and since they had Obama at respectively -3 and +2, the state is drawn in as neutral here. Whereas I dont doubt that Obama would win it.

The Obama maps do suggest, however, that he doesnt necessarily have an edge over Hillary when you start looking at the individual battleground states; and that he might be especially vulnerable in the South. If you look back at the tables for the Giuliani and McCain matchups as well, you see that in Virginia, Kentucky and to a lesser extent Tennessee, Hillary Clinton appears to have real chances, and do much better than Kerry did against Bush. For Obama, this seems to be much less the case or not at all. Legacies of racism after all?


http://img252.imageshack.us/img252/790/hillarymccainmap030108bh9.png

http://img230.imageshack.us/img230/4645/obamamccainmap030108ps3.png
0 Replies
 
Cycloptichorn
 
  1  
Reply Wed 2 Jan, 2008 09:30 pm
Bah, by tomorrow night these maps will all be useless! Fickle voters.

Cycloptichorn
0 Replies
 
realjohnboy
 
  1  
Reply Wed 2 Jan, 2008 09:41 pm
You will have to admit though, Cyclo, that the maps and charts while meaning little are pretty.

I have made my bets. Anyone else care to venture in?
0 Replies
 
nimh
 
  1  
Reply Wed 2 Jan, 2008 09:46 pm
Useless, dont think so. Depends on what you expect from them.

If you expect some early answer about what the end result of the elections will be, then yeah. Or if you want a prediction of where the race will go in the next six months, also.

But if you wanna know where possible regional strengths and weaknesses of an individual candidate lie; if you want to get a feeling of how the different Republican candidates, or Democratic candidates, compare to their peers in their regional appeal; or how successful or unsuccessful they have been so far in creating an appeal for themselves beyond their own party's core voters; then no, I dont think so.

All depends on to what level you're interested. <shrugs>
0 Replies
 
nimh
 
  1  
Reply Wed 2 Jan, 2008 09:55 pm
For example, it wasnt until I fleshed out all these numbers that I got a feeling for how surprisingly vulnerable the Republicans appear to be in the South this year. Much more so than in the mountains and plains of the West, to my surprise. Especially in Virginia and even more Kentucky, where the shift to the Democrats is especially pronounced (perhaps because of the travails around Gov. Fletcher etc?)

Turned my perception around a bit, really, as I'd pretty much given up on the South and was on the other hand expecting much from the West, which was showing distinct signs of purplification in '06.

But, again - much of the vulnerability of the Republicans in the South appears to evaporate when Obama's the Dem name on the form. Thats a more tentative appearance, for sure, but also interesting (and kinda depressing).
0 Replies
 
realjohnboy
 
  1  
Reply Wed 2 Jan, 2008 10:12 pm
Nimh, don't ever take anything Realjohnboy says too seriously. He means no offense and he switches to writing in the third person when he wants to step back a bit from a topic.

Shrug? Shrug not. Your knowledge of U.S. politics is awesome, and I enjoy seeing your pretty charts and graphs. And I will look forward to seeing this all continue throught the elections in November.

Meanwhile, I seem to be the only one predicting who will finish 1,2 and 3 tomorrow night.
0 Replies
 
nimh
 
  1  
Reply Wed 2 Jan, 2008 10:34 pm
Below is the table for how Clinton and Obama do against Romney in the state-by-state polls of the last three months. Plus just one map: for how Hillary does against Mitt (there's not enough data to make much of a map for Obama).

Also an interesting set, this table and map, if only just because of how it again shows that Romney, at least so far, has a far weaker cross-over appeal to independent voters than Giuliani or McCain. Couple weeks ago I already noted that Romney's weak performance in these polls could still just be a function of lower name recognition, but how that argument holds less water as the campaign continues, his name recognition increases, and he still does no better in these match-ups. Confirms my suspicion that Romney is probably the easiest of the Republican frontrunners to beat.

Noteworthy with both Romney's and Giuliani's numbers is also how the numbers compare if you look at the difference between the 2004 results and current polling. There's a lot of variation, but over time a clear trend has caught my attention: Giuliani and especially Romney do worse than Bush did in '04 pretty much across the board, but it's in the core red states, especially in the South, that they lose the most ground, also in relative terms, from the way it looks.

In the blue states, on the other hand, they do less badly. Giuliani in some cases does better than Bush did in '04, mostly in the Northeast, and Romney at least loses a lot less ground on Bush's results in the blue states than he does in the South. (Romney's Mormon background does mean he holds up well in some parts of the West, Utah of course, perhaps Nevada too.)

Is it because Rudy's and Mitt's relatively unorthodox backgrounds meet some solid resistance in the most traditional conservative states? Mitt's Mormonism, in particular, might hit bigger resistance in the evangelical South than in the more individualist West. Or is it simply because the hard core Republicans are feeling disillusioned and unmotivated this year, and that shows up most starkly in the hard core red states? 2004 was a year of massive partisan mobilisation, and the core red states mobilised en masse for Bush. With that fervour gone, the drop-off might be steeper there than in MOR states. Or does the shift from terrorism and war as the main themes of the '04 elections to the economy and economic insecurity now hurt the Republicans especially in the poorer red states of the south and midwest?

The pattern should be a nice ground for other speculation too. For example, in spite of all the talk of how a Hillary candidacy would be uniquely polarising, state polls this last half a year or so seem to show an opposite phenomenon. In a race between Romney or Rudy and Hillary, red states would be less red, and blue states would be less blue (in the case of Giuliani) or at least less starkly blue in comparison with purple and red states (in the case of Romney). Rudy and Romney as very much unintentional vehicles for the political 'desegregation' and purplefication of the country?

Anyhow, back to the data, here's the table and map:


http://img185.imageshack.us/img185/7056/demsvsromneystatesug5.png


http://img81.imageshack.us/img81/5660/hillaryromneymap030108dn1.png
0 Replies
 
Cycloptichorn
 
  1  
Reply Wed 2 Jan, 2008 10:34 pm
nimh wrote:
Useless, dont think so. Depends on what you expect from them.

If you expect some early answer about what the end result of the elections will be, then yeah. Or if you want a prediction of where the race will go in the next six months, also.

But if you wanna know where possible regional strengths and weaknesses of an individual candidate lie; if you want to get a feeling of how the different Republican candidates, or Democratic candidates, compare to their peers in their regional appeal; or how successful or unsuccessful they have been so far in creating an appeal for themselves beyond their own party's core voters; then no, I dont think so.

All depends on to what level you're interested. <shrugs>


hah, Didn't write clearly enough.

I've just been a little obsessive over graphs and polls and figures lately. And when I study the historical data (at least the last 12 years or so) I can see how much small events can cause (or signal?) large shifts quickly.

So it's difficult for me to look at elect ability maps, for example, without being frustrated. I've given up until tomorrow.

Cycloptichon
0 Replies
 
nimh
 
  1  
Reply Wed 2 Jan, 2008 10:35 pm
realjohnboy wrote:
Meanwhile, I seem to be the only one predicting who will finish 1,2 and 3 tomorrow night.

I'd like to join you in a bet, but to be honest I really, really dont have a clue how things will turn out tomorrow night! They sure know how to build up the suspense over there in Iowa...
0 Replies
 
Cycloptichorn
 
  1  
Reply Thu 3 Jan, 2008 11:27 am
Per TPM.

Quote:
Rumblings

I'm just pulling together the different threads here, but it's really starting to seem like the second choice factor could end up being the big story tonight in the Democratic side of the ledger. It's starting to seem like almost all the candidates are either choosing to expressly direct their supporters to caucus for Obama as their second choice, are implicitly doing so or are simply expecting that that's what they'll do. Probably almost 20% of the caucus electorate remains either undecided or supporting a candidate not likely to reach the viability threshold. So that's a lot of support that could potentially swing in Obama's direction.

This morning Eric Kleefeld called up Zogby and asked what his tracking numbers looked like when the second-choices were factored in and reallocated amongst the candidates the numbers went from ...

Obama 31%, Edwards 27%, Clinton 24%

to

Obama 37.5%, Edwards 33.7%, Clinton 28.8%


--Josh Marshall


Gobama!

Cycloptichorn
0 Replies
 
sozobe
 
  1  
Reply Thu 3 Jan, 2008 11:43 am
I know Kucinich did that, I didn't know that others did too.

Interesting.
0 Replies
 
ehBeth
 
  1  
Reply Thu 3 Jan, 2008 11:50 am
Cycloptichorn wrote:
Per TPM.

Quote:
This morning Eric Kleefeld called up Zogby and asked what his tracking numbers looked like when the second-choices were factored in and reallocated amongst the candidates the numbers went from ...





Is that second-choices preferred by the candidates, or second-choices as indicated by the folks who'll be caucusing?
0 Replies
 
 

Related Topics

 
Copyright © 2024 MadLab, LLC :: Terms of Service :: Privacy Policy :: Page generated in 0.15 seconds on 12/23/2024 at 01:10:53