okie wrote:Pollsters don't have agendas? I will admit to more skepticism concerning this than you may have.
A pollster's overriding agenda is getting the numbers right - not out of any kind of idealist scruples, but because their livelihood depends on it.
The more they get it right, the more media and experts will cite them, the more they will attract publicity and clients, the more money they will make.
If they get numbers that later turn out to be drastically off-kilter, they will lose customers and money.
Money is always the bottom line.
Of course you are right - the wording of questions can skew results, and sometimes does. But fundamentally, getting skewed results is not in the pollster's business interest.
There is one blatant exception, of course: polls that are commissioned by, for example, an interest group. Again its money that counts. If its the NRA paying, or NOW, and the pollster wants to attract it as a custumer for the next poll too, it is wise to get a result that would enable NRA or NOW to do good lobby work and get publicity with. Ie a result showing that people think gun rights are really important or the right of abortion should be protected.
So beware of commissioned, topic-specific polls. But for the regular ones, whatever the pollster's personal convictions, it is not in his business interest to skew results. I doubt conviction wins out over $$ often.
Of course there's some mucking about in the margins. I saw one poll of possible Democratic contenders that was commissioned by Fox News, and they put Ted Kennedy on it - who, of course, got very high negatives. Sheer flippancy, of course, and that's the pollster catering for a political interest of the medium in question. But they can only really do that with marginal mucking about like that, because if they'd get any of their core results totally out of kilter with what the other polls are saying (or what the subsequent election result turns out to be), it hurts them directly.
If pollsters
do have an overriding bias - in Holland, where there are three main polling agencies, that seems true for De Hond, at least - it's towards getting results that will be newsworthy. The more coverage for the results, the better for business, after all. Spectacular results = good; boring, predictable results = fine if you're Gallup and you're so well established that you're gonna get covered whichever way, but not so good for more minor pollsters or new kids on the block.
But overall the meme of pollsters being nothing more than political hacks in disguise, seems to be disproven by the fact that they really pretty much tend to all get roughly the same results - whether it's the Fox News or the NYTimes poll.
Look at
this GIF I created through the years leading up to the 2004 elections, for example, about Bush's job approval rating. All the different pollsters are represented in the graph, as you can see. And though there's a bandwidth of variation, they really pretty much all showed the same month-on-month developments, regardless of whether they are commonly labelled liberal or conservative-leaning. There's just one or two exceptions. The Fox poll consistently had Bush's disapproval a coupla points lower than the others. But it had Clinton's disapproval a coupla points lower than the other polls too (as Thomas has pointed out a few times). AP/Ipsos and ARG (American Research Group) are the only other two exceptions - they consistently had Bush's disapproval rating higher than the others. Otherwise, the numbers are, over time, practically indistinguishable.
And its not just on Bush's approval ratings. Look
at this GIF I made at the time that tracked the Bush-vs-Kerry poll ratings for all the main pollsters. Week on week, polls would vary widely with each other - but not in any systematic way. Over time, there isnt a single poll, not Fox nor Democracy Corps, that "favoured" Bush or Kerry in its results.