Thursday, June 04, 2009

Poll Results!

Come on in, slip off your skin, and rattle around in your bones! 22 (11%)
There once was a man from Nantucket. 60 (30%)
Kitties!!! 70 (35%)
Hey, nonny nonny. 44 (22%)

Votes so far: 196
Poll closed

The Kitties have won!!!11eleven!!

Ok, now the fun part. Statistics. I mean, sure, Kitties won, and if this were a simple election, we would be welcoming our new Kitty overlords with catnip and balls of yarn. But what would be the fun in that?

These were clearly nonsense answers—one might be tempted to treat them as random choices. But, like “I’m thinking of an odd number between one and one hundred”, all choices are not equal. Though this be madness, yet there’s method in’t.

First, let’s see if this is a statistically significant difference. Yes, the kitties won, but somebody had to—is it possible that this distribution of votes was simply random? (“Yes”, says the null hypothesis; “No”, retorts the alternative hypothesis.)   To test this, we need a limerick, courtesy of the good people at the OEDILF:

chi-square test, chi-squared test by Cuttlefish

Chi-square testing compares Fexpected
To the real Fobtained you've collected:
Square, divide by Fe,
Sum these up, and you'll see
If Hypothesisnull is rejected.

The chi-square test, a robust statistical test appropriate for nominal (categorical) data, calculates a ratio of observed frequencies Fo to expected frequencies Fe. The assumption of the null hypothesis is that the categories are unrelated; a significant value for chi-square allows one to conclude that there is a relationship between the categorical variables.

The formula is: χ2 = ( (Fe – Fo)2 / Fe )

Close, but in this case we are using a Chi-square goodness of fit test rather than a test of independence. No biggie; the formula is the same. All we need to know is that, with 196 responses, the expected frequency under the null hypothesis is 49 votes per answer.

Chi-square obtained, in this case, is 26.86. At three degrees of freedom, the probability of these results occurring by chance alone is less than one in ten thousand. Ok, for all practical purposes we can conclude it’s not random.

But why? Could be the order the answers were presented in; I could repeat the poll with randomized presentation. Could be collusion by participants. Could just be that people like kitties. Could be that kitties across the world were holding voters’ loved ones hostage. Sadly, no independent variable was manipulated, so we are at a loss to explain our results other than descriptively.

No big deal, of course; it’s just a nonsense poll. But for fans of experimental methodology and measurement, it is just one more example of the methodological concerns involved in poll-making.

Bottom line—the appearance of randomness is not randomness. Remember that when you look at psychic predictions, or remote viewing, or polls on cuttlefish blogs.


Podblack said...

I voted KITTIES!! I THE WIN!!!

Which is better than my results on that fracking Dr Richard Wiseman 'be psychic on Twitter' poll. I have bombed out every time... :p

Podblack said...


Dr Wiseman responded to your post!!!

Anonymous said...

I voted for Nantucket because, well, I know the limerick and it made me chuckle.