Lies, damned lies … and percentage growth
One of the many banes of a technology journalist’s life is a tech company gushing about its percentage growth – while refusing to divulge actual numbers.
This is almost always cause for alarm. Invariably, it means the base figure is so low, it would be an embarrassment to the company to reveal its true performance. But anyone can claim 678 percent customer or user growth if they only had 2 customers to start, and anyone can truthfully declare 200 percent revenue growth if revenue started at a few hundred or even thousand dollars.
Let’s be straight about this: it is an insult to present journalists with these kinds of numbers. That many of these claims make it into the media is not a reflection of credibility, but of poor journalism and inability of journalists to think through the claims they report.
This is only marginally more offensive than the use of old data to back up current arguments. Especially in the mobile arena, and especially in Africa, growth is so fast, that old data is all but irrelevant – and yet many companies still base both business decisions and claims on a long gone status quo.
A rule of mobile Internet resrarch has even been coined in South Africa to warn against this form of sloppy data use: Gray’s Rule, named for veteran Internet marketer Scott Gray:
“Research around mobile typically has a relevant life of around three months. Decrease the relevance by about a third for every six months on top of that.”
The rule has been bandied about among South African tech journalists frustrated with the relevance given to old data by large corporations. It’s been modified somewhat by refinements and corrolaries, to allow for the difficulty in accessing fruit-tree-fresh data.
For example, there has been consensus that, if such data is accompanied by solid forecasts based on proven methodologies, it may be used to extrapolate data by a further 12–18 months. However, this does not apply to data more than 18 months old.
The underlying assumption is that the research is based on sampling that is broad enough to be generally representative of a population in question.
In April, during a Virtual Indaba to thrash out further rules around acceptance of statistical claims, Brainstorm editor Samantha Perry pointed to a third broad category of data abuse: the use of undated statistics in press releases or marketing material, in an attempt to avoid having to vouch for their veracity. The refusal to accept such data was accepted as Perry’s Corrolary to Gray’s Rule.