Of the active ones, it's no surprise that ARG ranks almost last. But I was surprised that Mason-Dixon ranked almost as low, and LA Times even lower.
Rasmussen and Zogby didnt do too well either, but then both are relatively controversial.
Gallup and Quinnipiac near the top is no surprise, but Research 2000 ranking as high is. And Survey USA's self-praise for ranking first is deserved, especially after the flak it got for its election-eve California poll.
Another measure of pollster accuracy/reliability (with the caveat that pollsters dont actually claim to
predict the result, though with last-minute polling the distinction is slight), is to see how often they've really been off. How likely were they to get a result really wrong? Using SUSA's table,
- ARG was off on the winner's leading margin 10 points or more 9 out of 20 times.
- LA Times polls, meanwhile, were off 10 points or more 2 out of 6 times. The reason the LA Times polls ranked even below ARG's ones in the SUSA listing is because those two times (in Iowa and NH), they were off a lot.
- Mason-Dixon was off 10 points or more 5 out of 19 times.
Let's make a full list of that:
How often more than 10 points off?
1.
Quinnipiac: 0 out of 6 times
2.
Research 2000: 1 out of 11 times
3.
Survey USA: 4 out of 22 times
4.
Gallup: 1 out of 5 times
5.
PPP: 2 out of 9 times
6.
InsiderAdvantage: 3 out of 12 times
7.
Mason-Dixon: 5 out of 19 times
8.
Strategic Vision: 3 out of 11 times
9.
LA Times: 2 out of 6 times
10.
Rasmussen: 11 out of 31 times
11.
Zogby: 7 out of 17 times
12.
Suffolk Univ: 3 out of 7 times
13.
ARG: 9 out of 20 times
14.
Marist Univ: 2 out of 4 times
All in all a rather sobering list. For all but the top 4 pollsters in this list, any poll you may have seen on the eve of the primaries had at least a chance of 1 in 4 of being completely out of whack...
So how often did they really get close? Let's say, how often did they get the winning margin down to 3 points from the actual results at most?
1.
Gallup: 4 out of 5 times
2.
Survey USA: 13 out of 22 times
3.
Marist Univ: 2 out of 4 times
4.
Suffolk Univ: 3 out of 7 times
5.
Research 2000: 5 out of 11 times
6.
Zogby: 7 out of 17 times
7.
Quinnipiac: 2 out of 6 times
8.
Mason-Dixon: 6 out of 19 times
9.
ARG: 6 out of 20 times
10.
InsiderAdvantage: 3 out of 12 times
11.
PPP: 2 out of 9 times
12.
Rasmussen: 6 out of 31 times
13.
Strategic Vision: 2 out of 11 times
14.
LA Times: 1 out of 6 times
ANother quite sobering list... Most polls get the margin roughly in the ballpark it will actually be in, but this is a good warning not to take the numbers too precisely: chances that it will actually be very close remain quite small but for the most qualified/lucky pollsters.
And I mean, you cant blame them at all for being off 5 or 6 points, in any case. The numbers the SUSA table gives are about how many points the poll was off on the margin
between the two leading candidates, meaning that a margin of error applies twice the size of that for any individual candidate's number - so in many cases, 2 x 3% = 6%.
Interesting to see that there are some pollsters that have a lot of variation, coming in either very close or very far off (Marist and Suffolk), whereas others (InsiderAdvantage, PPP, Strategic Vision) have tended to be reliably mediocre.
Any which way you slice it, Gallup and Survey USA are near the top and Research 2000 and Quinnipiac in the top half in all of the three ways to rank the data here. And Rasmussen and especially ARG and the LA Times polls are in the bottom half in any of the three ways.