Get yer polls, bets, numbers & pretty graphs! Elections 2008

An update of the graphs on the polling data from Iowa, for the Republicans and the Democrats.

http://img253.imageshack.us/img253/4666/iademsonlymine271207be5.png

http://img169.imageshack.us/img169/4504/iarepsonlymine271207zc2.png

WARNING

These graphs are based on six polls each for the second half of December. That sounds OK - it's no less than the number of polls for some of the previous periods in the graphs - but this time there is an added complication. Two of the six polls in each race come from ARG, the American Research Group. That means that ARG polls make up one third of the polling data for the last stretch in each graph.

Why is this a potential problem? Every pollster has its own methodology, and the characteristics of its methodology - not so much in the phrasing of the questions as rather in the weighing of the results - can squeeze results a bit this way or that, in what are called "house effects".

in particular, each pollster has its own measures to screen for "likely caucusgoers" among the many people they call. This is a tricky process, potentially involving questions about whether the respondent him/herself thinks (s)he is likely to go, whether he has been following election news closely, whether he took part in previous caucuses, whether he already knows where the caucus in his area will be held, etc. Some pollsters end up with larger "screens" of likely voters than others, which basically means they are predicting a higher turnout than others. And some candidates do better, and others worse, when turnout is high.

ARG has in the past shown a clear house effect, though not consistently. Is results have regularly, though not always, tended to show McCain and Hillary doing significantly better in their respective races than in other polls, and Obama worse. Notes Pollster.com:

Quote:

A new American Research Group (ARG) poll of Iowa has caused quite a debate in the comments at Pollster.com. [So] let's look at the track records of the pollsters in Iowa.

The chart above shows the polling of the Democratic nomination race in Iowa since January. In 2007, twenty four different polling organizations conducted Democratic caucus polls. Of these, only 7 have conducted three polls or more. That means that [..] we can at least look at [those] organizations [..], and see how they compare to the trend estimates. [..]

[..] ARG generally showed a much better performance for Clinton than the trend in the first half of the year. From January through June, ARG usually had Clinton some 6-10 points above the trend estimate, with one exception in which ARG agreed almost exactly with the Clinton trend. During this period, ARG was the most discrepant of all pollsters from the Clinton trend.

In the second half of the year, ARG's polling has generally been much closer to the trend estimates, usually less than 4 points away from the Clinton trend. During this time ARG has mixed in with other polling pretty consistently.

That is, until the most recent December 20-23 poll, which again shows a large Clinton deviation of about +6 points from the trend. (I'm using the standard blue trend line. The upturn in the red "sensitive" estimate is interesting, but it is also sensitive to the ARG poll, an example of why you might not entirely trust the red estimator.) This is the first large ARG deviation recently, and quite a change from ARG's previous poll of Dec 16-19 which was close to trend.

In an inversion of the Clinton effect, ARG consistently underestimated Obama's support, compared to the trend, in the first half of the year, but their polling fell much closer to the trend during the second half. At least until the latest poll which has Obama some 10 points below the standard trend. (Again, note the downturn of the sensitive estimator, which would be turning down even without ARG, but which is turning down somewhat more because of ARG.)

ARG's polling for Edwards has been more variable. [R]ecent ARG results for Edwards has generally been a few points below trend, with the latest result about as much below trend as has been "normal" for ARG recently.

ARG's results this year have been more heterogeneous than some oversimplifications claim. There was a substantial overestimate of Clinton early in the year, along with an underestimate of Obama's support. That was important during this period because there were relatively few other polls available for Iowa in this period. But in the second half of the year, the ARG results for Clinton and Obama have been much closer to trend estimates, though still with a small average advantage for Clinton and small disadvantage for Obama.

In light of this, the ARG poll for December 20-23 does look out of line with their own previous polling, certainly their polling of the last 4-5 months.

The same warning goes for the Republican ARG results:

Quote:

On the Republican side, ARG's results have been especially favorable to John McCain, and to a lesser but still substantial degree to Giuliani. Unlike the Democratic results, these effects have not diminished much in the second half of the year. [See this chart.) McCain has often been as much as 10 points above trend, with only one poll below trend and one more right on the trend. No other pollster has been so consistently far from the trend for McCain.

For Giulinai, the results are less far from trend, but still quite consistently above trend. Only 2 of the last 10 ARG polls have Giuliani below trend. [..]

[T]he reasons for these discrepancies are largely matters of speculation. But the consistency of the ARG house effects are pretty clear in these data. The ARG results currently stand on the same side as their long term house effects: above trend for Clinton, Giuliani and McCain, and below trend for Obama, Edwards and Romney. Compared to other pollsters, these house effects for ARG appear to be the largest of any polling firm in Iowa.

0 Replies

Having said (or rather, quoted) all that, however, not all of the eye-catching last-moment developments in my graphs above can be dimissed purely on the basis of ARG house effects.

A downturn of sorts for Huckabee, upturns for Romney and McCain; a slight downturn for Obama, a slight upturn for Hillary; all those things also show up if you simply ignore the ARG polls and go only on the other four polls that have been done in the second half of December so far.

Four polls is of course a very small sample, but the below table does show how these possible last-moment trends appear without ARG results as well (and in the case of seeming upturns for Romney and Edwards, all the more strongly so):

Code:



            Nov II   Dec I  Dec II  Dec II

                             - ARG   + ARG



# of polls     7       9       4       6



Hillary      27.1%   26.9%   28.0%   29.2%



Obama        27.0%   29.6%   28.0%   26.0%



Edwards      22.1%   22.2%   25.3%   23.2%



Richardson    8.3%    7.3%    6.3%    6.2%



Biden         4.7%    4.6%    4.3%    5.8%



Other/       10.8%    9.4%    8.1%    8.7%

Don't Know

Code:



            Nov II   Dec I  Dec II  Dec II

                             - ARG   + ARG



# of polls     7       8       4       6

			

Romney       26.0%   22.6%   26.3%   23.8%



Huckabee     25.9%   33.6%   29.3%   28.0%



McCain        6.1%    6.0%    9.5%   12.5%



Giuliani     12.4%    9.3%    7.5%    9.5%



Thompson     11.3%    9.9%   11.0%    8.7%



Paul          5.1%    5.5%    5.8%    6.2%



Other/       13.2%   13.1%   10.6%   11.3%

Don't Know

Here's some improvised graphs to go with it; same ones as above, but with the alternative, no-ARG course of the trends included as "shadow" lines - click to enlarge:

http://img168.imageshack.us/img168/6713/iademsmine271207bqm7.th.png

http://img168.imageshack.us/img168/3659/iarepsmine271207bij1.th.png

0 Replies

Even that's not the end of it yet, though, because there is another complication. One that should be primarily interesting for polling geeks like me who want to know all about how these things work, and for Obama supporters, to whom it might provide some additional relief.

Pollster.com's Mark Blumenthal explains this one:

Polling on the Dark Side of the Moon.

Basically, the point is that it's very difficult to poll during these festive days, when many people can not be reached because they're away from home. That adds an extra degree of statistical volatility. This is why normally, polling slows down in the second half of December and shuts down altogether between Christmas and New Years.

As a result, "we risk getting skewed or biased results if the missing respondents are both numerous and different in terms of their political views from those at home when we call." Basically, depending on the demographic appeal of a candidate (whether his or her voters are young or old, highly or less highly educated), his or her voters are more or less likely to be travelling, and thus to be left out from the poll.

There is some data showing that those travelling for the holidays have "a very distinctive demographic profile": they're much more likely to be younger and better educated. Which of course is exactly the group Obama does best in. And thus, a "sample with fewer voters under 45 or fewer with a college education will skew in Clinton's favor."

Check Blumenthal's post for an explanation of how pollsters may choose to "weight" their samples differently to counter this effect, but how that can only lessen the bias, not correct it entirely, and also increases the statistical error.

Rasmussen's national daily tracking poll appears to have shown the result around Thanksgiving. Blumenthal produces a graph and describes how "there was a very unusual and precipitous plunge" in the Obama trend line "from what should have been a plateau around 24-26 to 17%" just before and just after Rasmussen took a five-day break from interviewing.

So in short, if Obama is dropping in polls that come out during this festive period, it may simply be because more of his supporters are unreachable.

Nevertheless, taking into account that we know little to nothing about ARG's methods, as it is not one of the pollsters who provide their methodological details, there is still a warning for Obama supporters as well:

"It is [..] worth remembering [..] that real changes may be occurring in vote preference this week even if surveys may be severely challenged in their ability to measure it. Clinton may be gaining and Obama falling. So it is quite a leap for anyone to say they know conclusively that the ARG result is either right or wrong. The hard truth is that we are behind the dark side of the moon this week, and we may not know much with certainty until next Wednesday night."
Here is that graph of the Rasmussen tracking poll, btw:

http://www.pollster.com/blogs/12-26%20rasmussen%20tracking400.png

0 Replies

News alert!

There's three new polls out on the Iowa caucuses since I posted the graphs above.

One is by LA Times/Bloomberg, and comes with some question marks: it's based on a small sample size, and more importantly, was carried out over the holidays (20-26 December), and thus potentially suffers from the problems laid out in the above two posts. For example, it has Obama very low, at 22%, just like the ARG poll right before Christmas, and unlike the polls before - and after.

After, because we now also have the first two polls taken completely after Christmas. And they are very interesting, especially regarding the Democratic race.

Here they are!

Research 2000
12/26-27/07

29% Edwards
29% Obama
28% Clinton

Strategic Vision (R)
12/26-27/07

30% Obama
29% Clinton
28% Edwards

Look at that Exclamation

This must definitely be as close as a race could possibly be. And who'd have thought Edwards would, at the very last moment, seem to catch up with the other two entirely!

Just two polls of course, but here's me hoping...

0 Replies

nimh wrote:

News alert!

There's three new polls out on the Iowa caucuses since I posted the graphs above.

Here's an update of the graphs taking those into accounts.

What I've done for the sake of convenience is to chart two lines for the second half of December - one that takes the ARG polls into account, and one that doesnt.

Here's the Democrats:

http://img151.imageshack.us/img151/9205/iademsonlymine281207fp0.png

Basic observations:

Obama's numbers seem to have dropped off somewhat from their high earlier in the month, whether you count the ARG polls or not.

Keep in mind, though, that neither line takes into account that vagaries of holiday season polling, which might negatively impact Obama's numbers in particular.

However, even with that in mind, it does seem like his numbers may be levelling off. For example, if you exclude not just the ARG polls but the LA Times/Bloomberg poll that was done over Christmas as well, his new average still only goes up to 28.5%.
Hillary seems to be rebounding a little bit, putting her ever so slightly over Obama again, or if you take all the above into account, equal with him.
Edwards seems to be experiencing an unexpected late surge of sorts. The appearance holds up when doing apples-to-apples comparisons, looking at how his latest poll numbers compare to previous polls by the same pollster. Comparing polls now with polls from the first half of the month, Research 2000 has him up 5%, Strategic Vision has him up 3%, and Rasmussen (back on the 17th) had him stagnant.
Richardson keeps on falling (no difference whether you count ARG or not), while Biden remains stable at a very low level.

Here's the Republicans:

http://img151.imageshack.us/img151/465/iarepsonlymine281207gr6.png

All kinds of interesting dynamics going on here too. Basic observations:

Huckabee too seems to be dropping back a little bit from his high earlier in the month. He does so whether you take the ARG polls into account or not.

Before the more libertarian-minded Republicans cheer, though, here another warning applies. In the case of Obama's and Edwards' latest movements, their appearance in the graph is echoed when doing apples-to-apples comparisons of how they fared from poll to poll by the same pollster. This is not the case with Huckabee, with whom such comparisons have conflicting results. So his downturn might just be a result of a different mix of pollsters in the numbers.
Romney appears to be rebounding. Like the other last-minute movements described here, his 'rebound' doesnt appear in the more cautious pollster.com graph. But his polling average this second half of the month is a few points higher than in the first half, all the more so if you skip the ARG results; and he is (back) up within individual pollsters' numbers too. Research 2000 has him up 5 compared to earlier this month; the latest Strategic Vision poll has him up 2-3 points compared to the previous three SV polls this month; Rasmussen has him up 4 compared to earlier in the month. Only ARG dissents.
McCain seem to be enjoying a late surge; one echoed even more strongly in the New Hampshire numbers, by the way. He even looks well-positioned to take third place now. His numbers appear among the hardest to predict, though: polls held within the span of the last 12 days have had him as low as 8% and as high as 20%.
Fred Thompson poses another possible wild card. Counting all polls, he has at least stabilised at 10% these last two weeks. But again, his polling varies starkly, from 3% in an ARG poll held less than a week ago to 16% in a Strategic Vision poll held less than two weeks ago.

The last three polls out have him at 11%, 15% and 10%, and all three represent a slight improvement over earlier this month, so he might do a little better than expected. Likely to outdo Giuliani, perhaps even grab third place. Only the latter might keep his campaign alive though.
Talking about Rudy, he just keeps on dropping. The graph here has him at 8.6%, but the last three polls out have him at 8%, 4% and 8%. If he ends below Ron Paul that would be a spectacular humiliation, but as Paul will probably fail to get many second preferences in the caucus votes he should be able to escape that fate.

0 Replies

nimh wrote:

News alert!

There's three new polls out on the Iowa caucuses since I posted the graphs above.

One is by LA Times/Bloomberg, and comes with some question marks: it's based on a small sample size, and more importantly, was carried out over the holidays (20-26 December)

Given the calendar, Nimh, it is too late too get any meaningful polls.

You will recall that Obama controversially courted college kids who are eligible to participate in the Iowa caucuses even though they live out of state. Hotels are offering special rates if they come back to Iowa and stay at the hotels when the dorms are not yet open (oh, to be young again. What fun that would be).
Students may be clustered in a small number of districts, or precincts.

But I would repeat, the time for polls is over.

0 Replies

Finally, here are new graphs for the polling in New Hampshire. Most striking bits: McCain's surge; and Obama approaching Hillary very closely (in the pollster.com graphs they're practically equal now).

http://img149.imageshack.us/img149/8627/nhdemsonlymine281207nv4.png

http://img149.imageshack.us/img149/747/nhrepsonlymine281207kl1.png

0 Replies

realjohnboy wrote:

Given the calendar, Nimh, it is too late too get any meaningful polls. [..] I would repeat, the time for polls is over.

Hm. I disagree.

There are certainly many problems involved with polling over the holiday season - as described at too much length in the previous posts.

But "the time is over", period, is too short. Consider this. Remember Iowa in '04? Dean was ahead all the time, until almost the very end of the campaign, right? Gephardt was a close second. As we know, it didnt end up that way. Kerry and Edwards took advantage of a surge in voter preferences at the very last minute. We're talking about the last week or two before voting here.

And it was only the very last Des Moines Register poll, published the very weekend before the caucuses (!), that done caught it.

There have been drastic last-moment turnabouts before; there's always a chance of there being one again now. And just like that very last DMR poll caught the turnabout then, any last-moment development now might just be caught by late polls again, in spite of the holiday-season dilemmas.

--------

(Pollster.com actually polled the pollsters on this. They collected the opinion of 46 pollsters working for the media or campaigns on which is more true: "polling in this [holiday] period will be challenging but can be done reliably", or "polls done in this period will be so unreliable that they should be ignored". Respondents could choose on a scale of 1-10 between the two statements. Result: "They divide almost evenly [..], with 48% placing themselves closer to the notion that polls "can be done reliably" this week, and 50% placing themselves closer to the statement that polls this week are "so unreliable they should be ignored"; the campaign pollsters were less skeptical than the media pollsters.)

0 Replies

Okay, Nimh, so we should all watch for the Des Moines Register poll this Sunday. I can live with that.

0 Replies

realjohnboy wrote:

Okay, Nimh, so we should all watch for the Des Moines Register poll this Sunday. I can live with that.

Well, it could be any other poll too. The DMR poll has a reputation to live up to, for sure. But my point was rather that the DMR poll from '04 showed that last-moment polls can have value. Doesnt necessarily depend on who does it, I mean, as long as it's a reputable pollster with experience in the state.

0 Replies

Too little, too late, Nimh in my mind, as far as polling. I simply can't see how a poll over the holidays can be valid.

0 Replies

so nimh, are you going to make any picks at this stage of the game in Iowa? Main questions as I see them are, can Obama actually win and will he? And can Huckabee retain his lead and win, or will Romney prevail, or will McCain pull off a late upset with a wild surge in Iowa, then onto New Hampshire?

I think its probably Romney and as much as I hate to see it, Clinton. I don't think Huckabee will sustain his lead and McCain does not have enough upside potential, and I think the Democrats will hold their nose and go with Clinton, figuring that is their best chance in the general.

Revision: Watch out for Edwards in Iowa, but what matters is how Obama places compared to Clinton.

0 Replies

okie wrote:

so nimh, are you going to make any picks at this stage of the game in Iowa?

Hi Okie, well it's as close to a crapshoot as it can be in Iowa now, isnt it? What, with Obama's support apparently peaking a little too early and experiencing a slight downturn, Hillary holding steady, and Edwards surging, it's become a photofinish.

Same in the Republican race, where Huckabee's support is levelling off or even dropping off a little and Romney is making a good last-moment comeback - another photofinish.

And the race for third place in the Republican contest is another neck-to-neck race. McCain doing suddenly surprisingly well, Thompson steadying a little bit; those two are looking the best for position #3. And it's important which one wins; appearances count, and even if the difference is small, a third place for Fred will count as a recovery and keep his candidacy alive, while a fourth place would pretty much kill it off. In my view, anyway.

Whereas a third place would make McCain look like a guy with momentum, especially if Huckabee and Romney end up practically tied for first place, and position him well for NH, where he is really surging. Depending on how the Iowa results are played out in the media, it could make the difference between him surging into second place in NH like he already is, or becoming an instant contender for first place - especially if Romney dissappoints in Iowa.

Well, in short I guess I'm agreeing with you so far, just in more words Smile

okie wrote:

Revision: Watch out for Edwards in Iowa, but what matters is how Obama places compared to Clinton.

Yeah, Edwards is obviously the big story - suddenly I see everyone on this forum talk about him. So I guess those polls I posted here on the last page that showed him moving up werent all that meaningless after all, eh?

And the thing about how this works is, of course, the more the media talk of Edwards surging, the more actual waverers will give him another look too - it can be self-confirming.

I think it would be great if Edwards won. It would instantly make him the main challenger to Hillary, and considering the way the increasingly resentful to-and-fro between Edwards' and Obama's campaigns is playing out (the fighter vs the negotiator), I'm all the more on Edwards' side.

0 Replies

Fun statistic of the day: in the last five polls out in Iowa (by Strategic Vision, Research 2000, ARG, Mason-Dixon and Zogby respectively), Rudy Giuliani and Ron Paul registered identical scores Exclamation

In something of a statistical fluke, both received, respectively, 4%, 8%, 6%, 5% and 8%. That put Rudy in a shared last place with Paul every last five times. (In one poll each he also shared this last place with Thompson and McCain.)

0 Replies

Your analysis is pretty much right on I think. Thanks for all the poll information, etc.

0 Replies

okie wrote:

Your analysis is pretty much right on I think. Thanks for all the poll information, etc.

My pleasure! Thank you for chatting. Makes for a nice break from the usual to-and-fro on the Politics forum - and it was getting a little lonely on this thread Smile

0 Replies

Polls, polls,polls, raining down on us. Especially from Iowa. I see them mentioned on all the political threads, but alas not here -- and I havent been able to keep up with posting myself.

But Pollster.com - them again - have a great new graph up, which shows the results of the last 15 polls out on Iowa. That is - all the polls completed since 17 December. 15 for the price of 1! Smile

One annoying thing is that if two polls on the same day have a candidate with the same percentage, there's just one dot, so some of the emphasis is lost.

Here we are - I created a copy that's vertically rather than horizontally aligned so it wont stretch the screen - but go here for the original thing.

Oh, while the black line just tracks from one poll result to the next, the blue and red lines represent the pollster.com trendlines - the blue one is the more cautious one, the red one the more sensitive one, but as you can see there's not much difference between them at the moment. At the most, the sensitive one shows McCain's upturn and the levelling off of Huckabee's support a bit more.

http://img143.imageshack.us/img143/8315/0iowaendgamelargehx7.png

It's a sobering graph, because one thing it shows clearly is that the differences between individual polls are often much larger than those that separate the frontrunners within the individual polls. Which is a very graphic way of saying, "the race is very volatile and too close to tell".

Says Prof. Franklin:

Quote:

There have been seeming sharply contradictory polls since Christmas. [..] And none of this looks to be because of very strong late trends. Depending on when you start and how hard you squint you can see some modest trends or no trends at all in the polling since mid-December. Our trend estimators, both standard blue and sensitive red, see small trends over each day, but the noise of the individual polls around these trends is large.

0 Replies

Right, lots of noise which makes using the trend lines for predictive outcomes problematic. I looked at the original posts and don't see where they post the r-square values associated by their lines but from the look of the noise I would guess they are very small. I'll throw out my general caution of using data to draw conclusions that it doesn't support (not that you've done so).

I agree with your conclusion that the race is very volatile and too close to call. That's the beauty of election day.

0 Replies

Bets:
Dems: Obama. Edwards, Clinton
Repubs: Huckabee, McCain, Romney

All by margins too close to preclude them from moving on.

0 Replies

New side-by-side comparisons: the pollster.com trendlines on the left - with each individual poll result represented by a dot; and my graphs with the periodic averages on the right.

(The latest average, for the second half of December, has by now yielded 15 polls for both the Republican and the Democratic race - counting the first three days of the Zogby rolling-average tracking survey as one, taking only the last day out.)

http://img167.imageshack.us/img167/2776/iademsboth010107di6.png

It's interesting, by the way, to look back at what the same side-by-side looked like a little over two weeks ago. Most striking developments: Hillary's slight rebound, Obama's last-moment downturn, Edwards' last-moment uptick, and Richardon's continuing slide.

Despite the crudeness of the method of using periodic averages, all these things actually appear in much the same way in the pollster.com trendlines as they do in those averages.

http://img160.imageshack.us/img160/1438/iarepsboth010107vb2.png

To go two weeks back in time with this side-by-side for the Republicans, go back here. Most eye-catching changes: Huckabee's levelling off of support (in the pollster trendlines) or even downturn (in the half-monthly averages); Romney's at least partial rebound; and McCain's unexpected return to the front of the second tier. And pollster actually has Rudy trending below Ron Paul now!

0 Replies

Get yer polls, bets, numbers & pretty graphs! Elections 2008

Related Topics

Quick Links

My Account

able2know