For those who are loyal listeners to Tim Harford’s More or Less programme on Radio 4, Professor David Spiegelhalter will be a familiar name. He is often interviewed to help listeners make sense of controversial or contradictory statistical results. In a similar field, Dr Ben Goldacre (www.badscience.net/) and Alan Smith (@theboysmithy) work to help us understand statistics. Their work is necessary now more than ever given the growth of big data and the Internet of Things (IoT). David Spiegelhalter’s new book, The Art of Statistics: Learning from Data, is a must-read for anyone who relies on data to make decisions in their work (which, let’s face it, is more and more of us).
Spiegelhalter outlines some of the fundamental principles of statistics that are often misused, even by those working in the industry. Take the word “average”, for example. This must surely be one of the most overused words – the ‘average’ call handling time (AHT), the ‘average’ hours spent on social media, the ‘average’ hours slept, the ‘average’ house price, etc. In his book, Spiegelhalter explains the crucial difference between the mean, median and mode.
He also pulls up the media for sensationalizing stories about a disease (let’s call it X), with reports of “a 100% increase in the risk of contracting X” when the actual risk of becoming ill has only gone up from 0.5% of the population to 1%. He discredits the misuse of bogus causation statements such as “Waitrose puts £36k on the value of your house”. He also describes how statistical evidence shows the probability that the skeleton dug up in a Leicester car park in 2012 was really that of Richard III.
But just how reliable are statistics and data? The Reproducibility Project cast doubt on the reliability of many accepted psychological studies from academia. It revisited and tested 100 such studies but with larger sample sizes, and found that only 36% had significant results – a somewhat shocking finding, and it was reported that the remaining 64% were either false claims or fake science. In fact, Spiegelhalter points out that only 23% of the results were significantly different from the original – a much less newsworthy finding but more accurate reporting of the academic research. He also notes that statistical significance shouldn’t be ‘black or white’ – for instance, called either ‘significant’ or ‘not significant’, when it is, in fact, a continuum.
The Art of Statistics reminds us that when using algorithms, accuracy is vital. But transparency is also important, as we need to understand how algorithms work so that we can detect bias or discrimination. Or prove to a financial regulator that the right information was provided. Spiegelhalter argues that the Kaggle test encourages ‘black box’ and complicated algorithms that no one can understand once they are operating! He offers some sound advice for machine learning engineers, data scientists, data engineers, data analysts, machine learning researchers and indeed anyone who is using data in their work:
1. Worry about data quality
2. Make sure the results of the research can be replicated
3. Ask how rigorously has the study been done
4. Consider the statistical uncertainty
5. Ask yourself ‘What am I not being told?’
6. Consider how the claim fits with what else is known.
At Davies Hickman, we work with data and statistics day in, day out – generating, analysing and interpreting data to identify trends and guide our clients’ decisions. Always mindful of quality, we aim to arrive at results that can help you make the right decisions for your business to succeed.