Memo to media, science museums, homo ‘sapiens’: Enough with the online polls!

Especially the poorly worded ones

First, of course, please vote in yet another Economist online poll with biased wording, “This house believes that innovation works best when government does least.”

Two weeks ago it was another poorly worded Economist poll, “This house believes that creating green jobs is a sensible aspiration for governments” (see “You can support Van Jones and clean energy jobs“).

Then, of course, the US News pitted me vs. Big Oil on climate science with the triply biased, “Did Climategate Expose Global Warming Fears as Unfounded?” I guess you can keep voting on that one forever (click here)!

Of course, you can vote for Climate Progress  every day through April 2 at Treehugger (click here — you know you want to).

Oh, and let’s not forget that online voting made WattsUpWithThat the 2008 Weblog awards winner for “best science blog” (!) — see “Weblog Awards duped by anti-science disinformers “” again!

And, of course, Watts freeped the London Science Museum poll and claims he helped persuade them to consider dumbing down their Big-Oil-funded climate exhibit.  A commenter noted the voting on the question was bifurcated between those who attended the exhibit gallery and those who voted on the web.  The “question” was “I’ve seen the evidence. And I want the government to prove they’re serious about climate change by negotiating a strong effective fair deal at Copenhagen.”  And yes, that question also seems worded to get a certain outcome.  Here’s the voting on “Count me in” vs. “Count me Out”:

Gallery Counted in = 3408 Counted out = 626
Web Counted in = 2650 Counted out = 7612
Total Counted in = 6058 Counted out = 8238

I understand why the websites do it — they want to drive up traffic, which translates into ad revenues and maybe ongoing readers.

The problem of course is that these polls are meaningless and unscientific, wholly gamable by whichever group decides to put the biggest effort into online voting.  Yet, since the results are visible for millions to see, and have the imprimatur of credible organizations like The Economist and USNews and the London science Museum, the parties involved have little choice but to participate in the charade. am, however, relatively confident that frogs — even slowly boiling brainless frogs — do not subject themselves to online polls.  Maybe they aren’t so dumb after all.

14 Responses to Memo to media, science museums, homo ‘sapiens’: Enough with the online polls!

  1. Leif says:

    I agree, I hate polls that it are confusing which way a “yes” or “no” is even interpreted. I even see ballot measures that I must fill out with a “cheat sheet” reminding myself which way to vote lest I get confused in the ballot box. Criminal. Then there are those “push polls” where the out come leaves no reasonable vote. The “have you stopped beating your wife?” type. yes or no

  2. prokaryote says:

    Online stats are always to taken with a grant of salt.
    Because it can be manipulated in diffrent ways, such as the question itself. Concerted – organized votes to swing a poll, you could have diffrent ip’s to vote a dozen times.

    To take a poll to make decision is therof very salty ;)

  3. mike roddy says:

    Krugman wrote a good column in the Times today about the Republican Party being totally hijacked by fanatical and potentially violent extremists. These kinds of people like to vote as a bloc (as in Congress), and especially love to click their heels and go when given directions from their leaders.

    This hostile takeover does not seem likely in the United States, but I saw it succeed in the Comments section of Dot Earth. One of the pontificators from the Far Right would make a completely wooly statement about the climate topic of the day, and he would be echo chambered and reader recommended as if he were a genius. This was obviously orchestrated, with a think tank arm of the oil companies throwing these guys some quarters. WUWT and Climate Audit appear to be more voluntary, but equally bizarre.

    I see the same phenomenon at the Science Museum, where a piddling sum from a company like Shell can reap significant dividends. Facts and scientific evidence become not only optional, but irrelevant.

    Apart from the danger we are facing from slow action on global warming, the combined strength of the fossil fuel companies and the looney tune right has created a very gnarly beast, strangely out of place in a technically advanced society.

  4. Jeff Huggins says:

    I spent much of my career working with market research (“qualitative” and “quantitative”) — interpreting it, designing it, implementing it, making decisions based on results, learning from successes and mistakes, and so forth … in and for organizations such as McKinsey and Disney, working with folks from P&G and top ad agencies, and so forth.

    It is hard enough to do valid and helpful market research even if you are very careful, combine approaches, connect the dots, listen carefully, and so forth.

    The vast majority of the sorts of online polls you mention are so ridiculous and invalid, in a whole bunch of ways, that the results they give are worthless and often downright misleading. That’s the bottom line. Anybody who places any importance on the results of these sorts of polls (beyond the very narrow and almost useless point that they indicate how the particular “silly” people who responded to the silly poorly-worded question responded, some of whom may have participated on their own and some of whom, with a known position in advance, may have been encouraged to participate) is not being wise in doing so, and is deceiving himself or herself.

    Cheers for now,


  5. Jeff Huggins says:

    (In other words, I agree with the essence of the post.)

  6. If we take the phrase literally, no one believes that “innovation works best when government does least.”

    For government to do “least,” it would have to stop providing military defense, stop providing police protection, stop providing local streets. Not even the looniest tea-partyer believes that government should stop building streets.

    The wording is not only biased. It is so absurd that you can only vote for it if you don’t think about what it means.

  7. paulina says:

    Regular polls are frequently problematic because of how they are worded and interpreted. People end up receiving the “message” that some part X of the population believes/wants/supports/will-do Y, but the message may have little to actually back it up. Sometimes we call this propaganda.

    Online polls skip what for regular polls may sometimes be a mere pretense (that of finding out stuff about public opinion), and go straight to the message part. But this message (ie the one beyond the “useless” one mentioned above by Jeff) *never* has anything to back it up. By design, no matter how well an online poll may be worded, it has nothing to back it up.

    It would be interesting to know results from social psychology research on the propaganda value, or perceived-social-norm influence, of online poll results.

  8. David B. Benson says:

    Charles Siegel (6) — On the web I’ve encountered some self-proclaimed libertarians who want government services at all beyond police and military.

    Some Dunning-Kruger there, methinks.

  9. Jeff Huggins says:

    I agree with paulina (Comment 7) and second her “it would be interesting” point at the end. Indeed, that would be very interesting.

    (I wouldn’t be surprised if the main influence is a self-affirming one: If the poll results agree with someone’s pre-existing view, that person probably thinks or feels “see, I’m right”. On the other hand, if the poll results disagree, that same person would be likely to dismiss the results as being invalid. Often, when it’s possible, people believe what they want to believe. Of course, some people might persuade themselves to change their minds based on such polls, but because these polls are not valid in the first place, even such changing-of-minds isn’t grounded in good reasoning.)



  10. bill says:

    Joe – you have told us that nuclear is very expensive but why exactly? is the imported equipment ? the redundancy in safety controls/ the fuel? the training? the regulations? the materials?

  11. richard pauli says:

    53% of Americans think Congress should repeal the 2nd Law of Thermodynamics.

    Or let the Supreme Court deal with it.

  12. paulm says:

    Dr. Hayhoe really does have a way with words. I bet she can gently convert most skeptics over….

    Obama should consider her a role on his Climate Action team.

    Its definitely worth listening to her talk here…

    Conservative Evangelicals embrace God and green

    Farley, who’s a conservative Republican, says he was “a very hard-nosed skeptic” about climate change until he began discussing the subject with Hayhoe, a supporter of action. “She would show me the data,” he recalls. “And after being presented with the data, I would lose the argument.” After a while, “I became increasingly convinced [and said to myself] ‘I see it now.’ “

  13. Leif says:

    Bill, #10: First I would say yes to all of the above.

    I would like to add a few thoughts if I might.

    I agree with all the negative things about Nuclear power and am still convinced that some faction there of will be required. My criterion is that Nuclear compete on a level playing field, paying for all it’s costs via rates including waste and decommission and operating costs and insurance etc. I strongly feel that the Nuclear industry killed itself by having taxpayer’s support in the first place. It became too large too soon. To wit, I would point out that a Nuclear powered Aircraft Carrier has 5000 people living within a thousand feet of a Nuclear reactor, many within a hundred feet or less, and no one gives it a second thought. That same ship can tie up in the down town of most any port in the world, ( except perhaps NZ and Japan, etc.) and again not a word. Nuclear subs pass within a mile of my house and tie up often less than two miles. Not a peep! That single small reactor would be enough power for all of a small town and even the surrounding area. The whole thing would take less space than the new fire house. Hundreds of reactor could be built in a central construction area, shipped and installed in a heartbeat compared with the behemoths the industry tries to shove down societies throat. Cooling could heat adjacent homes or greenhouses opposed to rivers and estuaries.

    But NO! Big Money only cared about making big money as always, not the well-being of society. Then they blame their failure on environmental regulations and tree huggers. What a crock.

    Again the problem arises because big money works only for big money, NOT for HUMANITY.

    Humanity First, Status Quo NO!

    Fist held high,

    Gray Panther

  14. Larry Coleman says:

    It’s not the wording of the polls. It’s the non-random respondents, stupid. (no offense, Joe, or anyone else…the phrase is in the vernacular now, an idiom). The point is that the polls are invalid no matter how perfectly they are worded because the respondents are not randomly selected, or even almost randomly selected (the typical “good poll”). Under this circumstance a perfectly worded question is no better than any other…they are all worthless.