The COVID-19 numbers game
03 June 2020
One of the defining characteristics of the coronavirus crisis has been the widespread use of statistics. From prime ministerial messages to NHS board meetings, numerous stats and graphs have been rolled out to support many policy decisions and debating positions.
The daily death toll has formed a grim backdrop to almost every news outlet’s narrative throughout the crisis. The world fixated on graphs showing the exponential growth curve of new infections, especially in the early days of the pandemic.
Then, towards the end of April, the UK daily test numbers became the focus of our attention. The question of whether the government would hit its self-imposed target of 100,000 tests a day before the end of the month became all-consuming for the media and for Heath Secretary Matt Hancock’s allies and foes alike.
Dangerous game
But that last number shows exactly why this obsession with statistics is a dangerous game. In some ways, it’s completely meaningless. Without the context of the government’s overall aspirations for testing numbers, the number floats uselessly in space. Without some knowledge of what happened before the end of April, and what has followed in May, it gives us nothing more than a snapshot in time. And looking at the UK number without comparing it with those from other countries doesn’t provide any sense of the context or scale of the government’s ambition.
There’s also the veracity of the number itself. The 100,000 target was duly hit on the last day of April, prompting warm words of congratulation – and some relief – from many quarters, including some of the government’s opponents. But then it emerged that the 122,000 tests counted for that day included around 40,000 that had been posted to residential and other addresses and so hadn’t actually been taken. It also included multiple tests on the same people.
There was also a dip back below 100,000 two days later – a dip that continued for eight consecutive days. The government’s rather blithe response to these anomalies being pointed out was to set a daily test target for 200,000 for the end of May.
By this stage, the progress that was genuinely being made in rolling out more extensive testing had been completely overshadowed by a ludicrous statistical sideshow.
We need numbers
Our appetite for statistics is understandable. There’s something reassuring about the certainty of numbers in a time of chaos and crisis. But have the charts and league tables distracted us from what is really needed? The British economist Henry Clay once said: ‘Statistics are no substitute for judgment’.
Of course, an accurate understanding of infection, testing and recovery rates is crucial in the fight against the virus. This is one of the reasons why the eminent statistician Sir David Spiegelhalter called it ‘extraordinary’ that the Office for National Statistics (ONS) was only asked on 17 April to carry out a survey to establish how many people in the UK have the virus and how many have recovered from it.
Spiegelhalter has called the communication of statistics during the pandemic ‘embarrassing’ and a ‘missed opportunity’. Speaking on the BBC’s Andrew Marr show, he said the British public were largely supportive of the government’s measures to tackle the virus and were hungry for information but instead they were being fed ‘number theatre’. He added: “I wish the data was being presented by people who really understand its strengths and limitations and could treat the audience with some respect.”
Risk management
At a board as well as a national level, we must be careful to avoid being seduced by statistics that can’t be relied on or don’t really help. Crucial to the success of the recovery phase of this pandemic is a proper understanding of risk. Spiegelhalter was correct when he said the pandemic has moved from being a societal threat to what is effectively a national risk management exercise. So it’s clearly vital that we have the most accurate possible picture of what those risks are.
Spiegelhalter used the example of the risks to different age groups revealed by ONS death statistics, which show roughly 30,000 COVID-19 deaths in the four weeks up to 24 April. There are 10 million under-15s in England and Wales, but they account for just two of those 30,000 deaths. Of the 17 million under-25s there were 26 deaths recorded. These are negligible risks. But it’s a very different story for the over-90s – more than 1% of whom died from COVID-19 in that four weeks. That means they face 10,000 times the risk of younger people. This is the sort of information that should be shaping the government’s – and NHS boards’ – responses to COVID-19.
Martin Thomas
Copywriter
GGI
