select search filters
briefings
roundups & rapid reactions
Fiona fox's blog
The Office for National Statistics (ONS) have released the latest data from their COVID-19 Infection Survey.
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“The latest weekly bulletin on the ONS infection survey takes the analysis up to the week 17-23 January. The news isn’t particularly encouraging, though it could be worse. ONS presents the results separately for the four UK countries. In England, Scotland and Wales the overall broad picture is that the number of infections is roughly level compared to the previous week (ending 16 January). In Northern Ireland, ONS are reporting that the number of infections has levelled off, after a period where it was increasing. This position is better than if infection rates were increasing, though I’d certainly be much happier if the rates were falling faster than the ONS estimates show. In very broad terms, the ONS conclusions for England do roughly match the latest conclusions from the other big infection survey, REACT-1, which were published earlier this week1 – some small hint of a fairly slow decrease in infections towards the end of the period covered. But they appear not to match the daily counts of new confirmed cases, as given on the dashboard at coronavirus.data.gov.uk. They fell by around 40% between the first week of 2021 and the latest week covered in the ONS survey, while the ONS official estimates for England fell by only about 10% in the most recent week compared to two weeks before that (3-9 January), and barely fell at all in the most recent week compared to the week before. However, there are reasons for that apparent discrepancy, which I’ll come to.
“The trends, and the amount of detail available, are less clear in the other three countries than in England. That’s mainly because the statistical uncertainty is less in England than in the other countries, which is essentially because more people are surveyed in England than in the other countries, and in turn that’s largely because the population of England is so much larger than in the other countries. Because if that, I’ll concentrate on the results for England in the rest of my comments.
“The weekly revisions to the estimated ranges for the R number and the growth rate of the infection, from SAGE and the Government Office for Science (GOS) have also been published today. As always, these estimates are given as ranges rather than single numbers, because there is always uncertainty about the exact numbers. This week’s range for the R number for the UK is 0.7 to 1.1. That’s centred in exactly the same place as last week’s range (0.8 to 1.0), so there’s no clear signal from SAGE that R is moving either up or down. But the upper end of the range is higher than last week, and the lower end is lower. That indicates that SAGE are less certain about the exact R value than they were last week. It’s disappointing news that the upper limit is above 1, because that means that SAGE think that the possibility that R is above 1 is more likely than it was last week. But it’s still true that most of the interval is below 1, so that SAGE are saying that it’s considerably more likely than not that R is less than 1 and so that the number of infections is falling rather than rising. Just for England, the R number range is 0.7 to 1.0, so SAGE are saying that it’s very likely that R is below 1 in England, but probably not very far below 1. So again, in very broad terms, these R ranges are saying roughly the same thing as the ONS survey (and REACT-1) – infections are probably falling, but not falling fast, and the evidence for falling infection rates is not overwhelmingly strong. Comparing the SAGE estimates of the growth rate of new cases with last week, the message is much the same as with R. The range for growth rate for the UK is centred in the same place as last week, but the range is wider indicating more uncertainty about what the growth rate exactly is. This week’s range runs from -5% per day to 0%, so SAGE are saying that the number of cases each day might be 5% lower than the previous day, or it might be the same as the previous day (0% change) or anywhere in between. Last week’s range for the UK was -4% to -1% per day, so SAGE have increased the upper limit and reduced the lower one this week, making the interval wider (more uncertainty) rather than moving where it is centred.
“The main reason why the ONS infection survey results are so important is that they aren’t affected by changes in the numbers of routine tests (done within programmes like Test and Trace) that are carried out, or by the reasons why people that decide to seek those tests. If the numbers being routinely tested change a lot, in a way that’s not directly connected with the spread of infection, or if different types of people are being routinely tested (perhaps because of more availability of tests for symptom-free people), then the numbers of new confirmed cases could change even if the underlying rate of infection has not changed. The ONS survey involves a representative sample of people, across the country, who are tested solely to produce estimates of the progress of infections. So the ONS survey results would not be affected by the sort of biases that might make the numbers of confirmed cases difficult to interpret. (The same goes for the REACT-1 survey in England, though the details of the sampling of people and the collection of data are not the same as in the ONS survey.)
“The ONS ‘official’ estimate is that, in the week ending on 23 January, 1.87% of the community population of England, aged 2 and over, would test positive if they had a swab test for the vaccine. That’s more than a million people (about 1,018,700 is the ONS central estimate), or about 1 in 55 of the population. Because the results come from a survey, there’s a certain amount of statistical uncertainty, but it is a very large survey testing over 80,000 people a week, so the statistical uncertainty is not very great. Allowing for the uncertainty, ONS estimate that the proportion of people who would test positive is very likely to be somewhere between 1 in 50 and 1 in 55, rounded to the nearest 5. The ONS estimate for the previous week was a 1.88% rate of positive tests, so really hardly any different to the estimate for the latest week, and again that amounts to somewhere between 1 in 50 and 1 in 55 of the population when rounded to the nearest 5. Both of these weekly estimates are rather lower than the week ending 9 January, when the estimated rate of positive tests was 2.08%, or between 1 in 45 and 1 in 50 of the population. So rates have come down a little since the first week of the year, but only a little, and they remain very high. These are the numbers for people who would test positive if everyone had a swab test. Like all tests for infection (or for almost anything else), the PCR test on the swabs is not perfect (though it’s very good), and can lead to false negative and false positive errors. With levels of infection this high, and with what’s known about the chances of the two kinds of error, false negatives are likely to exceed false positives by a considerable margin. Therefore the true percentage of people who are infected would be rather higher than these ONS figures, though it’s difficult to say exactly how much higher – and this issue of possible test errors won’t really affect the patterns of trends at all.
“As well as these ‘official’ estimates of the numbers who would test positive, ONS also publish two other sets of estimates. These are a set of daily ‘modelled’ estimates, which are based on a statistical model that takes into account the way the results are collected and the general patterns in which they would be expected to change, and a set of estimates of rates for non-overlapping fortnightly periods, which are calculated using a statistically simpler method. The ‘official’ estimates are largely based on the ‘modelled’ estimates. Sometimes the three sets of estimates show rather different trends, but that is not really the case for the latest week or fortnight. All of them show a very slow decline in rates of positive tests, which is slightly clearer in the daily modelled estimates than in the others. But in any case the decline is so slow that it’s not even completely clear that there is a decline at all, when the statistical uncertainty is taken into account. Again, this is very broadly similar to what was reported by the REACT-1 team, and usefully ONS have provided a chart in another weekly report2 that shows the estimates from REACT-1 and the ONS survey on the same diagram. This shows that the two results do differ, though it’s not straightforward to compare them because they cover different time periods. That chart does not show the slow decline in infections that the REACT-1 researchers said was ‘suggested’ towards the end of their latest period, in their preprint two days ago. But fundamentally both surveys are saying there is some evidence of a decline in infections in England, but the decline is slow and the evidence is not statistically really strong.
“As always, ONS also give estimates of the level and trends of infections in different age groups and in different regions of England. In the English regions, the infection rate is highest in the London region, where between 1 in 35 and 1 in 40 people would test positive, and in the North West, where between 1 in 40 and 1 in 50 would test positive. In most of the other regions, the rate of positive tests isn’t clearly statistically distinguishable from the English average rate, though that’s partly because there is greater statistical uncertainty about the regional rates, because the number of swabs tested in any single region will be a lot less than the national total. But rates do seem to be lower than the national level in the East Midlands (between 1 in 55 and 1 in 80), the South West (between 1 in 60 and 1 in 85), and in Yorkshire & the Humber (between 1 in 70 and 1 in 100). However, all these rates, even in Yorkshire, are still high. The current estimate for Yorkshire is about the same as it was for the whole of England in the middle of December, when there was already considerable concern about the number of infections. We’ve got a long way to go still. Vaccination should help, but it hasn’t had a noticeable impact yet.
“On trends in the regions, ONS consider that there’s evidence of a decrease in the percentage of people testing positive in four regions in the most recent week – London, the North West, the West Midlands, and the South East. They say that rates in all the other regions ‘appear to be level’. You might be wondering how these findings are compatible with the overall finding that numbers barely changed in the most recent week compared to the week before, if rates went down in four regions where more than half the population of England live, and remained level in the other regions. It’s mostly because of the statistical uncertainty about the details of the regional figures – ONS warn about “over-interpreting small movements” – but also because the regional estimates are ‘modelled estimates’, and the decrease in the modelled estimate for the whole of England is a bit larger than the decrease in the ‘official’ estimate. (But the decrease in the modelled estimate for the whole of England is still very slow.) These regional figures from the ONS survey show some larger differences with the REACT-1 results than is the case for the figures for the whole of England, but again this is probably largely due to the increased statistical uncertainty about the regional figures. The R and growth rate estimates from SAGE also provide data for the regions of England, though they are rather harder to compare with the results of the surveys because different regions are used. (These SAGE estimates use the NHS regions, whereas the ONS survey (and REACT-1) use what are sometimes called Government Office Regions, which are standard for many purposes. The biggest differences are that there is only one Midlands NHS region, rather than treating East and West Midlands separately, and that the NHS region of North East and Yorkshire combines what are two different regions in the ONS results. But there are differences in detail of the boundaries as well – for instance, where I live, Milton Keynes, is in the South East for ONS purposes, but the East of England for NHS purposes and for the SAGE R estimates. Then again, lots of people don’t know where Milton Keynes is.)
“On different age groups, the good news is that ONS consider the positivity rate to have fallen recently for all age groups apart from the youngest group they consider (age 2 up to the end of primary school), and in that youngest age group they consider that infections are showing signs of levelling off. Again this looks odd at first sight – how can infections be falling in every age group but one, and level in that group, if overall they are hardly changing? – but again this is largely because of the difference between ‘modelled’ and ‘official’ estimates. It’s good that rates appear to be falling in almost all age groups, but except for secondary school ages where there’s been quite a considerable fall, they are falling really pretty slowly, and there’s quite a lot of statistical uncertainty about the trends.
“For the past couple of weeks, ONS have been presenting estimates connected with the new variant of the virus (the one that was first found in Kent, not the other variants from Brazil and South Africa). The PCR test that they use with their swabs does not allow exact identification of the variant, because that requires full sequencing of the genetic material from the virus. So they can report only which swabs are ‘compatible with’ the new variant. Some of these might not be the new variant at all, though the great majority of them will be. The way these results are presented in the ONS bulletin has been improved this week, in a way that I think makes the results much less problematic to understand. Last week, some of the detailed results did look rather strange, which seems to be because swab results were classified into just two groups – those that are compatible with the new variant, and those that are not, where data were reported as if those that were not compatible must have come from another variant. But for some swabs, there really wasn’t enough evidence to classify the variant, and from this week there is a third category. So there are data on cases compatible with the new variant, cases not compatible with the new variant, and cases where it was not possible to determine which variant was involved. Where the variant cannot be determined, this would generally because virus levels in the sample were too low to give a clear enough signal. This change makes the patterns much clearer to explain, in my view, and is a big improvement – though there is inevitably quite a lot of uncertainty about the position on the variants in the UK countries other than England, because the sample sizes are smaller. But in all four countries, the indication is that there are more infections with the new variant than with other variants, and this is very clear for England. The good news about this, though, is that in all four countries (and very clearly in England particularly), cases known to be from the new variant are falling (as are cases known to be from another variant). That’s excellent, because it indicates that the current lockdowns and restrictions can reduce infections with the new variant as well as previous variants, even though the new variant is easier to transmit to other people than previous variants. When less was known about the new variant, there had been some concern that current lockdowns might not be enough to contain it, but thankfully that seems not to have happened overall. The same is true within each of the English regions, because cases compatible with the new variant have been decreasing in every region except the East Midlands. There must be a certain amount of concern that cases compatible with the new variant may still be increasing in the East Midlands, but that’s just one region, and the data for individual regions is subject to quite a lot of statistical uncertainty. So we can’t be entirely sure that new variant cases are really increasing in the East Midlands, and overall I’m not hugely concerned about this finding.
“I said I’d return to the question of why the decrease in infection rates, in these ONS infection survey results as well as in the REACT-1 results, seems to be going much more slowly than the decline in the daily new confirmed cases (for England and for the whole of the UK). The ONS results show infection rates falling by around 10% over two weeks, between the first week of the year and the latest week for which figures are available, while confirmed cases fell by around 40% over that period. One reason may be because of biases introduced into the confirmed case counts by changes in the numbers of tests being carried out and the types of people being tested, perhaps because of changes in the amount of use of rapid lateral flow tests. But another very important point is that the two sets of figures are measuring different things. The infection survey estimates the percentage of people, on a given day, who would test positive for the virus. In the jargon, these are estimates of the ‘prevalence’ of infections. People testing positive today include those who would test positive for the first time, that day, if they were tested every day, but also people who would have tested positive the day before for the first time, or two days before for the first time, and so on. It’s estimated that once someone has tested positive, they will continue to test positive for several days – perhaps ten days on average. The counts of new confirmed cases include only people who tested positive for the first time on the day in question – so they are counting what’s called ‘incidence’ in the jargon. When infection rates are changing, incidence and prevalence don’t change at the same speed. In present conditions I’d expect incidence (as measured, roughly, by new confirmed cases) to change more rapidly than incidence (as measured by the main ONS estimates). So maybe the rate of new infections is falling rather faster than the total rate of positive tests – but I can’t be sure of that.
“In the past, ONS used to provide estimates of the incidence rate of new infections, as well as estimates of prevalence. They could do this because people swabbed for the ONS survey are generally swabbed more than once, at intervals of a week initially, so that there is some information on roughly when someone first became infected. (REACT-1 does not do this repeat testing and so cannot directly measure incidence of new infections.) However, ONS stopped publishing estimates of the incidence rate in late November, citing a need to review their estimation method, and they have not yet finished this review and estimated of incidence are still not available. If and when they restart publishing incidence estimates, it should be easier to compare the ONS figures with numbers of new confirmed cases on the dashboard.”
1 https://spiral.imperial.ac.uk/handle/10044/1/85703
Prof James Naismith, Director of the Rosalind Franklin Institute, and University of Oxford, said:
“Today’s release from ONS is not good news. Whilst cases are clearly down from their peak, (also seen in hospital admissions) over the last two weeks the decline may have stopped, next week will confirm this. The prevalence of infection rates remain high and this will translate to a high number of deaths in the weeks ahead. The new variant has as we feared proven very difficult to suppress with UK lockdown measures. With over 100,000 dead, I realise the certainty of thousands more deaths is bleak news. Families and friends will have paid an awful price for the failure to suppress the virus.
“The data from ONS echo the REACT data published earlier. These types of studies are more helpful than the official number of positive tests, it is clear that these can underestimate the prevalence of the infection. This is a long standing and perhaps insoluble problem given where we are, but the mismatch in the number people actually infected and the number of people who are actually tested is what dooms the track and trace system.
“The vaccine campaign is realistically the only way out of the bind we are in.
“If Britain can give lessons to the world on vaccination then we most assuredly have lessons to learn in how to suppress the virus.
“SARS-COV-2 is a global problem, we will not be safe while the virus rages anywhere. The new variant, if it spreads to poorer countries, will be devastating simply because it appears to be so much more infectious and will therefore overwhelm health services.
“It is crucially important that we support and aid vaccination efforts in poorer countries, that we continue to develop effect broad spectrum treatments and that we eliminate the virus across the whole UK population.”
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Kevin McConway: “I am a Trustee of the SMC and a member of its Advisory Committee. I am also a member of the Public Data Advisory Group, which provides expert advice to the Cabinet Office on aspects of public understanding of data during the pandemic. My quote above is in my capacity as an independent professional statistician.”
None others received.