Obama '08?

JPB wrote:

hmmm... I'm not so sure that averaging polls is more accurate than using a pre-selected poll, Thomas.

Even if this was true, it wouldn't affect the main point of my post: that pre-selecting one poll is much more precise than cherry-picking it from a set of polls after the results are in. Joe had implied that the two are analogous, and the main point in my response to him was that they are not. Can we agree on that?

About the other comparison, I agree with you and Joe that averaging is not necessarily more accurate than pre-selecting just one poll, although it can be. Granted, you first have to make sure that the polls you average measure the same statistical signal. Perhaps I was naive in implicitly assuming that this is done before the averaging. However, once you do make sure of that, averaging will improve the accuracy of the result. After all, it's no different in practice than running the same poll several times, each time with different people.

0 Replies

JPB wrote:

hmmm... I'm not so sure that averaging polls is more accurate than using a pre-selected poll, Thomas.

I haven't spent any time looking into those who average polls but, unless they are taking multiple errors into account, the errors associated with the averages are much greater than those of individual polls.

I am either not following you or not strongly disagreeing with you. When statisticians average over several polls that ask about the same question at about the same time, that's analogous to a meta-analysis of several medical studies. Are you saying that meta-analyses are increasing the error bars around the averages? If so, how do you explain that reputable statisticians are doing it?

JPB wrote:

Usually, a poll result is expressed in terms of a margin of error at 95% confidence (say, 3 percent).

This is true in medical research, but I doubt it's true for political polls. From comparing the number of people polled with the reported sampling error, I find it more likely that they report the standard deviation. Not an important point, but worth mentioning.

JPB wrote:

But If you average 5 polls, each with a MOE of 3% at 95% confidence, the MOE on the average is more like 15%.

I'm not sure I understand what you're saying here. In experimental physics, what we would have done with the five polls is this:

Apply whatever mathematical trickery is necessary to make the data comparable.
Throw the data into one big pot and stir.
Calculate a new average and a new margin of error from the data in the pot.

The details of doing this can get horribly complicated. But for illustration, let's assume the simple case that all five polls sampled about the same amount of people, so produced about the same amount of sampling error. In this case, the new margin of error would be sqrt(5) times that of a simple poll in absolute terms (number of people polled). But as a percentage, it would be smaller by a factor of sqrt(5). In practice, of course, the mathematical trickery in step 1 introduces new errors, so you would up with a lesser improvement than the simple case suggests. Still, if you do it responsibly, the result of the meta-poll would still be more reliable than each of its constituent polls.

Is this consistent with what you said?

0 Replies

The problem with using the results of a poll to perform further analysis is that the original result is an interval, not a point estimate. Polls are presented as a single number with a margin of error. What that means is that ANY value within the interval defined by the margin of error is as equally likely to represent the true outcome (with 95% confidence) as ANY other value within the interval. Most people take the midpoint of the interval and start manipulating it. When they do that, they've lost the error band that was associated with that midpoint value.

Although the results of a poll are oftentimes used as a point estimate, that's not what they are. If it was a point estimate then the error band would reflect a decreasing likelihood as it approached the limits.

When you average multiple point estimates then you end up with a point estimate. How do you average multiple intervals? You could take the upper and lower limits of each interval and manipulate them into an average, but you have no more idea that the limits represent the true number than you have that the midpoint is correct.

0 Replies

nimh wrote:

If one poll substantiates your assertion, and another concurrent poll refutes it, than to say that "the polls substantiate my assertion" is misleading.

Same as with any cherry picking of data.

Agreed?

Well, let me be clear here. Is your objection that Obama used (cherry-picked) one poll and said that it was accurate, or that he said that the polls (plural) had him ahead when he should have said poll (singular)? If it's the latter, then I don't have a problem with your objection. It's a minor point, to be sure, and it wouldn't constitute "cherry-picking," but you'd be correct: if only a single poll showed him leading his Democratic rivals, then it was misleading to suggest that multiple polls came to the same conclusion.

On the other hand, if you're arguing that Obama shouldn't have cited the results of one poll when other polls had different results, then I return to my original query: so what?

nimh wrote:

Polls are extremely informative things, but their nature means that yes, any one single poll is, if not quite meaningless, an irresponsible basis for any assertion that reaches beyond the most blatant fundamentals (if a mainstream poll shows you ahead 60 to 30, you can safely say you're looking pretty good, but that's about it). There's ways to validly use polls, and ways to abuse them; cherrypicking the one that fits your agenda from among dissenting ones is the latter.

Sorry, I don't buy that. I'm not sure how one poll is unreliable but ten polls are. It's a mystery to me how ten untrustworthy polls yield one trustworthy result. Furthermore, polls already have a built-in statistical degree of error based on the sample size. That degree of error, however, is not significantly decreased by sampling a larger population. In other words, a sample of 10,000 is not necessarily ten times more accurate than a sample of 1,000. Taking the results of ten polls and averaging them, therefore, does not yield a result that is ten times more accurate than a single poll. Indeed, as JPB pointed out, it might actually be less accurate, even if the polls ask the same question and use the same methodologies (which is rarely the case).

In the end, the polling firms, the media, the bloggers, the political commentators, and a significant slice of the electorate all consider the results of individual polls to be significant. On the other hand, it's only the statistical geeks who are saying that the results of individual polls are unreliable and misleading. I don't begrudge the politicians if they side with the former rather than the latter. After all, the geeks represent a far less influential and numerous constituency.

0 Replies

Thomas wrote:

JPB wrote:
hmmm... I'm not so sure that averaging polls is more accurate than using a pre-selected poll, Thomas.

I haven't spent any time looking into those who average polls but, unless they are taking multiple errors into account, the errors associated with the averages are much greater than those of individual polls.

I am either not following you or not strongly disagreeing with you. When statisticians average over several polls that ask about the same question at about the same time, that's analogous to a meta-analysis of several medical studies. Are you saying that meta-analyses are increasing the error bars around the averages? If so, how do you explain that reputable statisticians are doing it?

Reputable statisticians are pooling polls? If so, then I assume they are extremely careful on the polls they choose to pool and on the conclusions they are drawing. Polls are oftentimes performed with a self-selected sample (as opposed to a scientifically selected sample) and generally should not be combined (or at least the conclusions should be used with caution).

Thomas wrote:

JPB wrote:
Usually, a poll result is expressed in terms of a margin of error at 95% confidence (say, 3 percent).

This is true in medical research, but I doubt it's true for political polls. From comparing the number of people polled with the reported sampling error, I find it more likely that they report the standard deviation. Not an important point, but worth mentioning.

No, it is true, the sample size chosen for the poll to control for the MOE is determined at a 95% confidence level, but I don't want to argue with you about it.

Thomas wrote:

JPB wrote:
But If you average 5 polls, each with a MOE of 3% at 95% confidence, the MOE on the average is more like 15%.

I'm not sure I understand what you're saying here. In experimental physics, what we would have done with the five polls is this:
Apply whatever mathematical trickery is necessary to make the data comparable.

Throw the data into one big pot and stir.

Calculate a new average and a new margin of error from the data in the pot.
The details of doing this can get horribly complicated. But for illustration, let's assume the simple case that all five polls sampled about the same amount of people, so produced about the same amount of sampling error. In this case, the new margin of error would be sqrt(5) times that of a simple poll in absolute terms (number of people polled). But as a percentage, it would be smaller by a factor of sqrt(5). In practice, of course, the mathematical trickery in step 1 introduces new errors, so you would up with a lesser improvement than the simple case suggests. Still, if you do it responsibly, the result of the meta-poll would still be more reliable than each of its constituent polls.

Is this consistent with what you said?

You can't throw the data into one big pot and stir unless the data are generated from consistent protocols and include proper randomization. If the samples are not properly generated than you can't assume the error is equally distributed across the samples.

Here are a couple good articles (one and two) on using poll data, but even here we are concerned with the problems with comparing polls, not averaging them and then drawing conclusions from the averages. Polls are valid indicators of public opinion within the constraints from how they are performed. Using them to do further mathematical manipulations and draw inferences from the outcomes potentially magnifies the risk of being wrong.

0 Replies

Ah -- I found a pollster.com article that discusses averaging polls...

http://www.pollster.com/blogs/a_surrender_of_judgment_conclu.php

The comments and discussions at the end are also interesting.

and here

Quote:

4. Don't be seduced by averages.

Poll averages are all the rage this year, even ones that purport to show candidates' standings measured down to tenths of a percentage point. (Don't get us started.) This may fill political junkies' seemingly insatiable desire for a minute-by-minute assessment, as if this were the Nasdaq average or our rich Uncle Leo's EKG. In fact, there's a lot less to these averages than meets the eye.

Averaging across polls with different methodologies can easily obscure rather than clarify. If you take a state with few polls -- one good-quality survey, say, and three methodological clunkers -- averaging may well do more harm than good. Averaging polls done across different time periods, with different sampling methodologies, different procedures to estimate "likely voters" (some reasonable, some not) and different numbers of alleged "undecideds" all assumes that these differences make no difference. With this approach, you might as well throw a little Ouija in as well.

The reality is that a good poll is a good estimate. All else being equal (and it never is), a collection of good polls will be an even better estimate, but a collection of good and bad polls won't. more

Sorry for the sidebar, soz...

0 Replies

Obama hints he won't run a 2nd time
BY MICHAEL SAUL
DAILY NEWS STAFF WRITER

Saturday, December 29th 2007, 4:00 AM

WILLIAMSBURG, Iowa - Barack Obama told Iowa voters Friday that if he doesn't get elected President this time, he might not try again.

Turning the "experience" argument with Hillary Clinton on its head, the 46-year-old first-term senator said eight more years in Washington could make him a less desirable candidate.

It's now or perhaps never, Obama and his wife, Michelle, concluded, because, "We still remember what it's like to be normal," he told a crowd here six days before Iowa's first-in-the-nation caucuses.

The notion that Obama would shoot his own rising star out of the political sky if he doesn't win the White House on his first attempt troubled one voter leaning toward him.

"I'm very disappointed because I think he's young and I think he has a future," said Lue Sykes, 64, a retired government worker, after Obama spoke in Coralville, Iowa. "He's a person that should continue to present to this country what he has to offer."

The Obamas don't seem to see it that way.

"My wife and I were talking the other day, and she said, 'We're not doing this again,' " Obama said. "Those of you who have met my wife or heard my wife, you know she doesn't mince words. I mean, she's a tough cookie."

Obama said their stance has less to do with the grueling campaign schedule that separates the family for large swaths of time, and more with the couple's belief that eight years from now, they wouldn't be the "same people."

Washington insiders, Obama said, lose touch with reality because "you think your worth is tied up with a title, or a chauffeur or people opening doors for you." Obama regularly portrays Clinton as a Washington insider.

Just five years ago, the family was living in a condo that was getting to be too small. Obama, not yet a senator, regularly bought the groceries and his wife was "still shopping at Target." (Obama noted that "she still does.")

"Eight years from now, we will have lost a little bit of touch with what ordinary families are going through," Obama said Friday.

Margaret Street, 45, a supporter of Bill Richardson who lives in Coralville, said she believes Obama will change his mind.

"If he lost this time around, he would give it another shot," she said.

Corey Stoglin, 33, a Target employee who works in Iowa City, said Obama is right.

"If this country would sit there and not elect him when he's clearly the obvious person for the job, that shows we're just dysfunctional as a society, that we don't really want change," he said.

Elect me now or I will take my bat and ball and go home

0 Replies

Not exactly the wisest statement made by Obama.

0 Replies

...a matter of opinion.

I think that there is something to be said about seizing the moment, and I think there is truth in the sentiment he expressed that staying in Washington for longer might mkae him a less attractive candidate to some.

0 Replies

There is honesty, and there is political strategy. I'm not sure Obama gains much, if anything, with "his" statement about not running again.

0 Replies

I understand. You don't think it was a strategically wise thing to say. I don't think that's necessarily true - it could have a galvanizing effect on those who are supporting him now, and those on the fence (they might think "now, or never"). It's a matter of opinion which scenario is the case.

0 Replies

Rephrase that, snood. Talk slower. Break it down into little words, snood.

0 Replies

Galvanizing. I have always hated that word.

Sounds like closure.

0 Replies

snood wrote:

...a matter of opinion.

I think that there is something to be said about seizing the moment, and I think there is truth in the sentiment he expressed that staying in Washington for longer might mkae him a less attractive candidate to some.

True he would fill his rather thin resume and people will know what he is and believes. That is what is missing now since IMO he is rather a blank slate.

0 Replies

au1929 wrote:

snood wrote:
...a matter of opinion.

I think that there is something to be said about seizing the moment, and I think there is truth in the sentiment he expressed that staying in Washington for longer might mkae him a less attractive candidate to some.

True he would fill his rather thin resume and people will know what he is and believes. That is what is missing now since IMO he is rather a blank slate.

You're definitely entitled to your opinion. I don't know how anyone could listen to the man or read his policies or his books or see his record of community service and still see him as a black slate. Unless they had simply prejudged and closed their mind.

0 Replies

snood
I said blank not black.

I think there is a vast difference between being a community organizer and a state senator and being the president of the US.

0 Replies

JPB wrote:

Ah -- I found a pollster.com article that discusses averaging polls...

http://www.pollster.com/blogs/a_surrender_of_judgment_conclu.php

The comments and discussions at the end are also interesting.

and here

Quote:
4. Don't be seduced by averages.

[..] If you take a state with few polls -- one good-quality survey, say, and three methodological clunkers -- averaging may well do more harm than good [..]

This bit stood out for me, and made me look at the date of the post.

Pollster.com is, by the way, an excellent resource, not just about all the polls coming out but also about polling itself. It's very addictive, and it's taught me, an unqualified dilettant enthusiast, a great much. And yes, it has especially taught me about the various needs to be very, very cautious when interpreting results.

That said, the site does pool the results from all the different polls in order to identify trends, if with copious qualifications. It does so by using regressive trend lines rather than crude rolling averages, for sure. But that it does; collect all the polling results on the Republican race in Iowa, say, and identify the trends over time. Apparently the authors of the site - Mark Blumenthal, who's on the Executive Council of the American Association for Public Opinion Research, and Prof. Charles Franklin, who teaches statistical analysis of polls, public opinion and election results - do consider that to have a value that's above a mere turn of the ouija board. I dont know, of course, I'm no expert, but I've come to trust their analyses.

Now like I said, that one sentence above stood out for me, and made me look at the date. It's November 2006. But now it's December 2007. The concern the comment expresses, that in "a state with few polls", averaging out "one good-quality survey, say, and three methodological clunkers" leads to trouble, of course still holds true for states like Nevada, say, or any of the Feb. 5 states, where polling is still relatively scarce. But in Iowa, there have been 18 polls this month alone, and in New Hampshire there have been 14, and the overwhelming majority is from experienced, reputable pollsters.

Prof. Franklin's regressive trend lines are devised in such a way that two or three outliers will not impact it at all. This is not true for averages, of course; if you calculate an average, two or three starkly divergent results will leave their mark. That's why, in the last graphs I posted for Iowa this week on the Polling thread, I drew in diverging lines for the second half of this month, in order to highlight the difference in results if you include or exclude two controversial ARG polls - and in the posts before added other important qualifications, such as those about the vagaries of holiday season polling. Yet even when averaging, the impact of two or three "methodological clunkers" is ever more muted as the number of polls increases.

Not to say that suddenly one should take trendlines and moving averages and such as some to the point exact truth rather than as rough representations of overall trends, of course. But this particular concern should now be much muted - as should the ouija comparison that comes with it.

0 Replies

au1929 wrote:

snood
I said blank not black.

I think there is a vast difference between being a community organizer and a state senator and being the president of the US.

I read correctly, but mis-wrote. I knew you said blank.

The job of the presidency renders all prior experience sort of moot, IMO - so the case could easily be made that there is a "vast difference" between being the POTUS and all the respective experiences of the candidates.

0 Replies

Waitress runs for president---no experience necessary!!

0 Replies

George F. Will in today's Washington Post about Shelby Steele's book:

Misreading Obama's Identity

0 Replies

Related Topics

Quick Links

My Account

able2know