Sunday, August 28, 2011

Statistical Doublespeak

It is a wonder why so many distinguished personalities in history have sceptical views of statistics. Benjamin Disraeli once remarked “...there are three kinds of lies; lies, damned lies and statistics”, the ebullient Winston Churchill chided an assistant that when he asked for statistics they should be favourable to his watch in Parliament, from Mark Twain Get your facts first, and then you can distort them as much as you please.” -  “Facts are stubborn, but statistics are more pliable.)” And from Evan Esar “Statistics: The only science that enables different experts using the same figures to draw different conclusions.
It is an unfair view of statistics. The science of statistics is on solid ground as its theories and practices are based on logical frameworks and borrows from numerical arguments. Statistics is the touchstone of progress as it seeks the betterment of things through trials, comparisons, observations. Statistics, whether applied to marketing or scientific endeavours that better the human condition is a vital and noble occupation. Advancements in most of the things that people now enjoy have been, in one way or the other, helped out by the use of statistics. It is a very powerful tool in the pursuit of new knowledge and the improvement of existing wisdom. What gives statistics a bad name is in how it is abused by those who use it to bolster weak arguments, by those who interpret the numbers to suit their purposes, the selective exposition of results, by those who try to make meaningful insignificant figures, the manufacture of numbers to prove a point and in so many other ruses statistics are made pliable and contorted to support untruths.
When you are invited to sit in on a presentation of the results of a research project you should be on the alert to the phraseology being used by the presenter in order to make out what is really being said and to separate what is factual from the speculated, what is deduction from conjecture, a significant find from a tendential outcome. You know that the presenter has trapped himself in a corner when he starts spouting a lot intelligible technical of terms making it appear that these are accepted facts by all in marketing. Anyone who would seem clueless to what is being explained would be made to look like a marketing troglodyte. These verbal cues are more effective when it comes with articulate body language such as squirms, twists, head scratches, looking out of the window and random twitches.
Here are some samples picked up from internet sources and some from my own exaggerations of true incidents during my years in market research.
"In an earlier study done twenty years ago..."
I didn't look up the original reference.

"A definite trend is evident in the sub group..."
The data is practically useless.

"While it has not been possible to provide definite answers to the questions..."
An unsuccessful experiment, but still the design is considered an elegant one by...

"Three of the samples were chosen for detailed study..."
The other results didn't make any sense.

"Typical results are shown..."
This is the prettiest graph.

"These results will be in a subsequent report..."
I might get around to this sometime, if pushed/funded.

"The most reliable results are obtained by xxxxx..."
Xxxxx was my graduate student; his grade depended on this.

"In my experience..."

"In case after case..."

"In a series of cases..."

"It is believed that..."
I think.

"It is generally believed that..."
A couple of other guys think so too.

"Correct within an order of magnitude..."

"According to the statistical analysis..."
Rumor has it.

"A statistically oriented projection of the significance of these findings..."
A wild guess.

"A careful analysis of obtainable data..."
Three pages of notes were obliterated when I knocked over a glass of beer.

"It is clear that much additional work will be required before a complete understanding of this phenomena occurs..."

I don't understand it.

"After additional study by my colleagues..."
They don't understand it either.

"Thanks are due to Mr. Smith for assistance with the experiment and Miss Jones for valuable discussions..."
Harry Smith did the experiment and Mary Jones explained to me what it meant.

"A highly significant area for exploratory study..."
A totally useless topic selected by my committee.

"It is hoped that this study will stimulate further investigation in this field..."
I quit.

“It is slightly below the action standard, about 9 percent, only a single digit differential on target...”
Hey stop moving the goal post!!!

“Wilcoxon test may not be appropriate, now let’s try Kruskal-Wallis, then the Friedman test...”
If you torture the data enough it will begin to confess and yield favourable results.

 “I need 20 more additional readings before I can arrive at any conclusive findings.”
By the time we complete the research they would have completely lost interest or I will be 10,000 miles away from here.

“Let’s do more Varimax rotations on the factors”
I’m dead meat if I don’t get a new factor pattern that would help explain my interpretation.

“Research shows that our competitor has more buyers than us but we have a subgroup whose willingness to buy scores are above threshold.”
A sample taken in the Calamian Group expressed willingness to buy if our product is made available to them.

“The respondents showed a dislike for the test products.”
We should have known better than to test our pork loaf product in Maguindanao.

“Secondary data from industry sources have shown a negligible market potential for our new brand”
Who was the moron who insisted on this expensive market study?

“The Simulated Test market predicted a 25% market share on the first year.”
Yeah, but it assumed that competitors will just stand by and watch.

“Casting research has shown that Rossana Roces has got a big following and would be an effective endorser.”
But we are selling tickets for the World Youth Day.

“A survey conducted in September of 2010 have shown dismal performance ratings for P’noy.”
The sample was taken from 4 towns in Lubao, Pampanga.

No comments: