Mark Twain famously said, “there are three kinds of lies: lies, damned lies, and statistics”.
All too often, statistics are used to mislead people. We are taught that numbers are to be trusted as facts, so they make powerful weapons in the hands of people who want to twist the facts.
Hamilton City Council staff published a survey headlined “Increased confidence in Council decision-making”. From that headline, the casual reader might be tricked into thinking “Hamiltonians are now much more confident in Hamilton City Council’s decision-making and have a greater sense of community in their neighbourhood” – which is actually the first paragraph in the Council report just in case you didn’t know what to think.
Indeed, Mayor Southgate “took heart” from this, while the council’s PR manager was “pleased with the upward result”.
But a closer look at the survey reveals a very different picture. The confidence in council had increased from a pathetic 30% in the previous survey to a dismal 42%. The majority of residents do not have confidence in the council. This should have been the headline.
A meagre 370 people out of 160,000 residents filled in the on-line survey, which results in a high margin of error. Mathematically, this is calculated at 5%, which applies to both surveys. At best, a proper statistician could only state the previous survey may have been as high as 35%, and the new survey may have been as low as 37%, so the much-vaunted increase is a negligible 2%. Not really something to shout about.
Now consider rounding errors. 42% of 370 is 155.4 people. I am not sure how 0.4 of a person can answer a survey. Even the 2% increase is looking shaky.
But it gets worse. The non-mathematical effects are significant. The results were “weighted” (i.e. altered) by age, gender, and ethnicity to be “representative” of people who live in Hamilton. Who knows what effect this had on the percentages. The survey questions themselves were leading (i.e. intended to get a specific outcome). By nature, on-line surveys are highly selective, and who exactly knew about the survey?
The whole system is meaningless. It is incapable of providing guidance to the council on decision making, which suggests it is a waste of time. So clearly its only purpose is for marketing. A good result can be trumpeted while a bad result is simply not reported, or, as in this case, has so much spin applied to it that it becomes a damned lie.
But the real question remains. What is the point of councils doing marketing? They are not businesses. They have nothing to sell. It is just another waste of ratepayers’ money.
If you enjoyed reading this BFD article why not share it with others?