November 13, 2020

expert reaction to the latest ONS Infection Survey and government R value and growth rates

The Office for National Statistics (ONS) have released the latest data from their COVID-19 Infection Survey, and the government have released the latest estimates for the COVID-19 growth rate and R value.

On the ONS infection survey only:

Prof Sheila Bird, Formerly Programme Leader, MRC Biostatistics Unit, University of Cambridge, said:

“Response-rates matter: This resume where ONS infection Survey ends its account: with response-rates when the address-file has been used for inviting participation from households. Even if the household-participation rates in Scotland and Wales achieve the dizzy height of England’s 11%, random-sample surveys via the address files in all three nations is not to be recommended.  The ONS Infection Survey reimburses participants, which may in part explain participants’ consistently high fidelity as 95% of those who provide a first swab continue (for weeks or months).

“Re-weighting in REACT-1 versus ONS infection Survey:  Unlike REACT-1, ONS Infection Survey is now shy  – even in its data-files – to explain its number of participants by region, age-group, deprivation quintile, etc so that reader can appreciate the before/after impact of the re-weighting that underlies much of the output we read. Let’s have better transparency, please, so that reverse-engineering from the width of 95% confidence intervals is unnecessary (albeit do-able).

My reading of the confidence intervals about ONS Infection Survey’s weekly reported estimates of SARS-CoV-2 incidence for England is, broadly, acceleration between the first and second fortnights. Last week, I suggested that it was too early to call it otherwise. Confirmation has followed for 31 October to 6 November.

“My third table compares Scotland and England in terms of weighted percentage (95% confidence interval) testing positive by non-overlapping 14-day periods.

England’s sample size is around 30 times greater in the most recent two fortnights and, consequently, England’s 95% confidence interval would be expected to be about a fifth the width of Scotland’s.

“Even so, as seen by essentially non-overlapping confidence intervals, Scotland’s SARS-CoV-2 prevalence was lower in the second fortnight than was England’s but currently cannot be distinguished as both share the same upper limit of 1.4%.

“See ONS Infection Survey’s Table 1i for corresponding information for English regions such as shown below for the North East and the North West but without my being able to cite (because missing in action): number in sample. In the most recent two fortnights, prevalence was clearly higher in the North West than in North East. And, in each fortnight, prevalence in the North East was higher than in Scotland (compare NE’ grey lower bound with Scotland’s corresponding upper bound).

“See ONS Infection Survey’s Table 1h for its story-line by age-group. Number in sample again missing in action and so readers need to pay heed to 95% confidence intervals.”

Prof Paul Hunter, Professor in Medicine, The Norwich School of Medicine, University of East Anglia, said:

“The recent report by ONS shows that the epidemic was continuing to level out at least in the week up to the 6th November. The modelling based on these data suggest that they incidence of new infections was even declining, albeit slowly. There are still regional variations in trend with the areas of the North West that were in the generally in tier 3 showing the greatest decline. It is also notable that the most obvious decline was in younger age groups (ages 7 to 24) whilst in older age groups the prevalence was still increasing. These results are in line with other sources of data that suggest that the tier system was having a beneficial impact on bringing down the transmission rate of the infection, and it is notable that the region with the greatest number of local authorities in tier 3 (The North West) has seen the most obvious decline in prevalence.

“Given that these data were collected during the week up to the 6th November, it is still too early to see any impact of the current lock down on trends. On the 12th November there was a dramatic increase in reports of new COVID infections in England and it looks like this increase was first obvious in samples collected on the 9th November. We have to be careful about reading too much into a single days figures but it has been suggested that the increase was due to increased socialisation in the days between the before the start of the new lockdown. If this increase in the daily reports of new cases  continues over the next few days then it is indeed likely that increased socialisation during the five days between the announcement and the eventual lockdown has driven a surge in infections and thereby reduced any benefit from the current lockdown. Again today’s ONS report does not contain data recent enough to identify this recent surge. We will have to wait for the next ONS report to know for certain what has been happening in the past few days.”

On the ONS data and other studies:

Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:

“Today (as usual for a Friday) we’ve had two different reports on the progress of the pandemic in the UK – the latest from the ONS Infection Survey, and the updated estimates  of R and the growth rate from SAGE. Yesterday we had results from another survey, this time covering England only – the REACT-1 survey from Imperial College. How do they compare, and why do they (in some cases) not say the same as one another? I’ll give my views on this only in relation to England – many apologies to people in the other UK countries, but REACT-1 covers only England, and most of the detailed results from the ONS survey are also only for England (because the survey has been running for longer in England than elsewhere and has a bigger, and so more informative, number of respondents in England).

“Both REACT-1 and the ONS survey should give more reliable estimates of the number of infected people that can be obtained from Test and Trace data and the daily counts of new confirmed cases on the gov.uk dashboard. That’s because they both test reasonably representative samples of the community population of England, and test them only to estimate infection rates, not because they have symptoms or work in certain jobs or live in certain places where a particular effort is being made to stem an outbreak.

“The latest ONS infection survey estimates take the data up to the week ending 6 November. For that most recent week, ONS estimate that 1.20% of the English community population would test positive for the virus, which comes to about 654,000 people in all, or about 1 in every 85 people. Like any estimate based on data from a sample of people, there is some statistical uncertainty about this estimate, and ONS report that the true figure could plausibly be somewhere between about 619,000 and 690,000. That’s a pretty high rate, and it has increased since the previous week. The good news about it is that the number of infected people seems to be increasing more slowly than in was in late September and early October, but it is still increasing. An important point about these estimates is that they estimate the number of people who would test positive, if everyone in the English community population could have a swab tested. The RT-PCR test that is used for the survey, like any other test, can have false positive and false negative errors – that is, people who have a positive test even though they aren’t really infected, and people who have a negative test event though they are really infected. At this level of infection, and given what is known about the chances of these two types of errors, it’s pretty well certain that then number of false negatives will considerably exceed the number of false positives. That means that the number of infected people in the country will be larger than the 654,000 estimate. ONS make some calculations to indicate what the number would be, using some reasonable estimates of the probabilities of false negatives and false positives. On this basis, the central estimate of the number of infected people would be about 800,000, or about 1 in 70 of the population, but taking into account the statistical uncertainty, the figure could be somewhere between about 730,000 and 880,000. Those numbers are high.

“The REACT-1 estimates for the proportion of people who would test positive, in round 6 of their survey, are rather higher than this. For the whole of round 6 (which covers swabs taken between 18 October and 2 November), they estimated that 1.3% of the population would test positive, but with an interval indicating the statistical uncertainty from about 1.2% to 1.4%. (They also provided results for just the first half and just the second half of Round 6, but they do not differ hugely from the figure for the whole round.). Turning these into estimates of the number of infected people, that translates into a range from about 900,000 to about 1,040,000 – in round figures, roughly a million. These figures are for a rather earlier period than the latest week of ONS estimates. So why are they higher than the latest ONS estimate, given than ONS (and some other sources) report that the number of infections is increasing? One reason is that the REACT-1 researchers and ONS use different estimates of the false negative rates of the test. That may be justified by the fact that two surveys use slightly different testing procedures, the overall issue is that the false negative rates of the test, under the conditions used in the surveys, are not known precisely. But probably a more important reason is that the underlying estimates of the percentage of people who would test positive are not the same for the two surveys. Usually, comparing the results from previous REACT-1 rounds as well as the latest one, the REACT-1 estimates of percentages testing positive have been higher than those from the ONS survey. This could be because the two surveys use different lists from which to draw the people that they ask to take part – REACT-1 uses lists of people registered with GP practices, and ONS uses list of addresses. It could also be because of differences in ways that the results are processed statistically, in particular in how they deal with the fact that the people who actually take part do not exactly match the English population in terms of age, gender, and other factors. It’s important in any survey to allow for such differences, and the size of the differences, as well as the way they are allowed for statistically, do differ between the two surveys. Because there’s no better source of data that the two surveys, it’s not possible to say which one is more accurate, but it’s important to bear in mind that the measures of statistical uncertainty that they both provide may not allow for all the possible uncertainty.

“The ONS survey also provides estimates of the number of new infections each day. That’s different from the number of people who are infected at any one time, because that number will include people who have already been infected for several days as well as new infections.  The ONS infection survey can do this estimation of new infections, because it tests people more than once, so has a reasonably accurate measure of when an infection actually began. REACT-1 tests people only once, so cannot do this more precise estimation of new infections. REACT-1 did publish estimates of the number of new infections, but they are based on the rather simplistic assumption that people remain infectious for 10 days on average. Because of that, I think the ONS estimates for new infections are likely to be more accurate. For the latest week (ending 6 November), ONS estimate that there were about 875 new infections each day for every million people in the population. That comes to 47,700 new infections each day, but the interval showing the uncertainty runs from 39,500 to 59,600. (There’s proportionally rather more uncertainty than for the total numbers infected, because the estimate for new infections is based on smaller numbers, because the total number of infected people has to be bigger than the number of newly infected people.). The central estimate is slightly up on the week before, but I’d very strongly caution against over-interpreting small changes, particularly for new infections. The survey just can’t be precise enough to make accurate measurements of changes from one week to the next. That’s one reason why it’s impossible to attribute these short-term changes to changes in people’s behaviour – but an even more important reason is that, even if we knew that a change in infection rates really did happen at the same time as a change in behaviour, we just can’t tell from data like this that the change in behaviour was the cause of the change in infections. It might be just a coincidence that the behaviour change and the infections change happened at about the same time. On new infections, then it’s safest not to concentrate on small changes, and look at broad patterns and trends. ONS report (as they did last week) that, as in a few recent weeks, the number of new infections remains at roughly 50,000 a day. The good news on this is that new infections aren’t increasing as fast as they were in late September and early October, and may in fact not even be increasing at all. But 50,000 new infections a day is still a very substantial number, and there’s certainly no clear sign that that number has started to fall yet.

“Both the ONS and REACT-1 also produce estimates for different regions and different age groups. Those figures are inevitably less certain than the totals for all ages across the whole country, because the number of people tests for the surveys in each single region or each single age groups is much smaller than for the whole country. In very, very broad terms, the two surveys roughly agree on the details for age groups and regions, though there are certainly differences between them. ONS report today that the infections rates remain highest in much of the North (North West, and Yorkshire and The Humber), though there’s some indication that the level of infections is levelling off or perhaps even falling in those two regions. Infections are also higher than the English average in the North East and in both Midlands regions (East and West), and they are lower than the English average in the regions in the South (including London). But differences between regions are generally smaller than they were a few weeks ago. The same could be said about differences in infection rates between age groups. ONS report that infection rates for older teenagers and young adults are decreasing, and although the rates for those age groups are still higher than for younger and older age groups, the differences between age groups have got smaller.

“Again as usual on a Friday, the Government has published revised estimates from SAGE for R and the infection growth rate, for the whole UK and also for England and the English regions. All these estimates are given as ranges, because they are based on statistical and epidemiological modelling and simply can’t be known very precisely. For the whole UK, the interval for R goes from 1.0 to 1.2, indicating that (on average) every 10 infected people will infect somewhere between 10 and 12 others. Last week’s range for R was 1.1 to 1.3. Although this week both ends of the interval are lower than they were last week, that doesn’t even mean that R has decreased for certain. It could possibly have been 1.1 both weeks, for instance. The growth rate range is from +1% to +3% per day – that is, the number of new cases is estimated to be rising by somewhere between 1% and 3% each day. If the growth rate is 3% per day, the number of new cases would double in around 3 weeks. At 1% a day, the number of new cases would double in about 10 weeks. Last week’s growth rate range for the UK went from +2% to +4% per day. Again we can’t be sure from these ranges that the growth rate is lower this week, because the ranges overlap. It is encouraging that the ranges for R and the growth rate have moved downwards a small amount compared to the week before, but you must bear in mind that R is still estimated to be at 1 or above 1, and the growth rates are positive, so the number of infections is still estimated to be increasing rather than decreasing. Any increasing trend in infections is bad news – we need R below 1 and negative growth rates (that is, decreases rather than growth in the number of infections). The SAGE range for R for England was the same as for the whole UK, 1.0 to 1.2, while their range for the growth rate in England was slightly wider, +1% to +4% each day.

“REACT-1 also produces estimates for R and the growth rate, for the whole of England as well as for the regions. Yesterday, on their Round 6, they reported statistical evidence that R was changing quite considerably during that round, and so they did not give estimates for the whole of the round, but instead for the two halves separately (16-25 October and 25 October-2 November). For the first half, their range for R was about 1.2 to 1.4, but for the second half, it was about 0.7 to 1.0, so they were reporting that it was probably (though not certainly) below 1 for the second half of round 6. Their ranges for the growth rate in England were about +4% to +11% for the first half, and -5% (so a decreasing number of new infections) to about 0% (no growth or decrease) for the second half of the round. The fact that these growth rate ranges are much wider than the SAGE estimates probably reflects that the REACT-1 estimates are based on relatively small amounts of data for each day within the round. My suspicion is that the SAGE ranges are rather more trustworthy, because they are based on wider sources of data that just a survey of swab tests, even though that survey is large. But an issue with the SAGE estimates is always that some of the data involved (for example on deaths, and on hospital admissions) do involve time delays after the people involved first became infected, and because of this, it’s probably best to think of the SAGE estimates as being averages over a couple of weeks, and not ranges for the true value right now. On this basis, the SAGE ranges are pretty similar to an average of the REACT-1 estimates for the two halves of their latest round. But I think we’ll have to wait for more time to pass and more data before we can be really confident that R and the growth rate really did move in an encouraging direction over the time period of REACT-1 round 6.”

Prof James Naismith FRS FRSE FMedSci, Director of the Rosalind Franklin Institute, and University of Oxford, said:

“The ONS data for week ending 6th Nov shows 48, 000 (40, 000 to 60, 000) new cases per day, the previous week was 46, 000 (38, 000 to 60, 000) new cases per day and the week before 52, 000 (38, 500 to 79, 200). 

“The Nowcast release estimates that the virus is infecting 64, 000 (43, 000 to 89, 000) people per day. Looking back to last week 77, 000 (53, 000 to 113, 000) and the week before 56, 000 (38, 000 to 82, 000). There are notable drops in North West and North East. The Midlands, London and East of England remained constant.

“The REACT survey reports significantly higher numbers, but points to a slowdown in spread with a hint of levelling off in early November.

“It is important to note the national lockdown does not yet show in any of these figures.

“The ZOE KCL system, which will show the national lockdown first, has reported evidence of cases beginning to very slowly fall.

“Today’s R-value numbers show the rate of increase in the virus has slowed, it is still spreading but much more slowly.

“Taken together, we can conclude that the local lockdowns have slowed the spread to around 50, 000 new cases per day at the start of this week. Stabilisation is not deliverance, 50, 000 cases per day will result in 100’ s of deaths every day. These deaths will be heart-breaking.

“Yesterday’s blip in positive test cases is unlikely to change this assessment. Rather the high level of positivity in pillar 2 seen over the last weeks meant testing was no longer accurately measuring the pandemic. This was obvious from looking at ONS, REACT, Nowcast and ZOE data, all of which had showed around twice as many cases per day as the testing reports. Yesterday’s number of pillar 2 tests processed showed a significant increase and I think the increase in cases reflects a more accurate measurement of the pandemic not a surge in infection. The episode should provide a reality check that we are not out of the woods and cherry picking data to suit an argument can lead to the wrong decision.

“Social restrictions have been proven again and again to slow the spread of the virus. The virus took off in September in the UK, it has slowed its increase because of social restrictions. The rapid and steep rise of covid19 cases in the winter has also been the fate of France, the USA and many other countries.

“There is no doubt that social restrictions and lockdowns do result in significant mental, financial and health harm. The costs are especially high for the young, the most deprived,  those in insecure employment and the elderly in care home isolated from families. It was to avoid this, that so many scientists urged a functioning track and trace system. Track and trace has failed to make any impact upon the spread of the virus.”

“By tonight, over 2000 of our fellow citizens will have lost their lives to covid19 this week alone. These 2000 people will have meant the world to someone, I feel huge sympathy for and share in the sorrow of those left behind. These deaths were predictable and predicted, they are simply a mathematical consequence of infections. The simple truth is that we remain unable to shield the elderly and vulnerable in the UK, nor has any European country. These 2000 deaths happened despite our efforts to reduce the number of cases. Therefore had we followed the policy prescription outlined in the Great Barrington Declaration when it was announced, data we now show the additional deaths would be measured in tens of thousands.

“Similarly asserting the local lockdowns were clearly working well enough two weeks ago is also not grounded in data. Two weeks ago there was evidence that the virus was still growing quite rapidly. There was some evidence which suggested the growth was slowing, perhaps in some high incidence areas declining. There was no evidence at the national level it was falling.

“Had the cases continued to grow nationally at these rates, the hospital were almost certain to overfill. Even with levelling off in the number of new cases per day, the number of people hospitalised with covid19, has continued to rise this month. By Monday, we will be over 100, 000 new infections. It is inconsistent with data to state that had we relied only on local measures there was no possibility that the hospitals would over fill. The only way to ensure the hospitals will not over fill, is to reduce the number of infections, stabilisation at 50, 000 may not be enough.

“It is important to understand the consequences of over filling the hospitals. Every time an aeroplane lands with fuel in the tank, we do not berate the pilot for wasting our money by carrying unused fuel. Few if any of us would fly were we told that that as the reverse thrust on landing stopped, the fuel would be exhausted. Instinctively we understand the concept of a safety margin. We know that running out of fuel at 30, 000 feet is catastrophic because should it happen it’s too late to do anything.  This is a risk that is never even contemplated. Over filling hospitals, which are not simply beds, but nurses, doctors and equipment would see a catastrophic rise in the death rate, not just from covid19, but from heart attacks, cancers and strokes. We can never know for sure that we would over fill hospitals until it was too late to stop it.

“The Prime Minister is elected and in a democracy, it must be the elected politicians who make these decisions. I recognise he carries a massive burden and responsibility. Science’s responsibility is to provide its best analysis based on the best data of the risks and costs of these choices.

I believe the SAGE system is doing this as well as can be expected. In a fast moving pandemic, we will get things wrong and some of data will point in different directions, this is science in action. Science works by learning as much from errors as from getting things right.

“I was skeptical of the ZOE data that first showed a levelling of cases, I preferred the more established ONS data, accepting the delay was the price to be paid for reliability. However, ZOE data does appear to be an useful more up to date measurement of the state of the pandemic, meaning I was incorrect. Only a fool or a charlatan, claims to be have right about everything or never to change their mind.

“Those who advocate loosening social restrictions, should present evidence that the costs of these restrictions is lower than the number of covid19 deaths that would result from over filling hospitals.

This is a hard argument to make, but I strongly believe it should be listened to respectfully and that those making it are sincere. However, those advancing arguments for reducing social restrictions with cases this high and saying there is no risk of over filling hospitals are deceiving themselves or the public or perhaps both. I do not believe such deception helps anyone.

“There is really good news on the vaccine front, reducing the number of cases now is likely to save even more lives because avoiding infecting people before Christmas means they will live to be vaccinated in the New Year.”

https://www.ons.gov.uk/releases/coronaviruscovid19infectionsurveyuk13november2020

https://www.gov.uk/guidance/the-r-number-in-the-uk

All our previous output on this subject can be seen at this weblink:

www.sciencemediacentre.org/tag/covid-19

Declared interests

Prof Kevin McConway: “I am a Trustee of the SMC and a member of the Advisory Committee, but my quote above is in my capacity as a professional statistician.”

None others received.