select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
A preprint, an unpublished non-peer reviewed study, reports on the latest data from the REACT-1 study on COVID-19 spread across England.
This Roundup accompanied an SMC Briefing.
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“These latest results from the REACT-1 infection survey, run by Imperial College with collaborators from other universities and Ipsos MORI, do show clearly the strengths and limitations of this kind of data. In England, we’re lucky enough to have not one, but two, regular surveys that test reasonably representative samples of people from the general population, to see whether they test positive for SARS-CoV-2. (The other one is the ONS Infection Survey.) I don’t know of any other country that is doing this on such a regular basis. Without a survey like this, we would still have data on people who have tested positive for the virus after they go for a test because they have symptoms that might indicate an infection, or because they are in hospital, or because of the job they have, or because a mass testing initiative is happening where they live. That’s where the data come from for the daily and weekly counts of new confirmed cases in different places, as reported on the dashboard at coronavirus.data.gov.uk. That’s helpful, but not always reliable because the number of people who test positive will depend on how many people are tested, and why they are tested. If those things change, the rate of positive tests could change too, even if infection levels in the community have not actually changed – or the changes in confirmed cases might be different in important ways from the real change in infection levels. That’s unlikely to cause big problems when looking at changes over short periods of time, but without some check, we can’t be sure. One important thing that the infection surveys provide is a check that the confirmed case numbers aren’t misleading. But a survey like REACT-1 or the ONS survey can do more than that, by allowing comparisons to be made between infection rates in different subgroups of the population, that don’t necessarily show up in the data from routine testing, such as differences between people living in households of different sizes, or living in areas at different levels of deprivation. That can provide valuable information of what might be driving changes in the pandemic’s progress, though it can’t give direct information on what causes what.
“Because of its role as a check for what has been seen in counts of confirmed cases, it’s good to see that the broad patterns in the latest REACT-1 results do match what has been seen in other data. The new preprint covers round 7 of the survey, which uses data from over 168,000 swabs collected between 13 November and 3 December. Results from the first half of round 7, up to 24 November, were previously published in an interim preprint1. Broadly, the new results show a decrease in the estimated number of people in England who would test positive, compared to round 6 (16 October to 2 December). That showed up in the interim round 7 report – the decrease has continued in the second half of round 7, though at a slower rate than the first half. There are similar patterns in the numbers of confirmed cases, as far as about 3 December when this REACT-1 round ended, and in the ONS infection survey, for which the latest results go up to 5 December. The REACT-1 researchers report that there are large differences between the trends of infections in different regions – with increasing infection rates, against the overall national trend, in London and some parts of the East and South East of England near London. Again that matches other data sources for the same period of time. According to REACT-1, rates fell fastest in the North West and the West Midlands, and again those falls appear in the other data sources. REACT-1 is somewhat less definite about the falls in infection rates in the other regions of the North (North East, Yorkshire & The Humber), though it certainly indicates falls since the peak level around the start of November. Also, the patterns of changes in infections in different ages groups do broadly match other data.
“I must make it clear, though, that REACT-1 can’t tell us everything we need to know. That isn’t because it’s a bad survey – it is being run by a team involving academics from Imperial College and several other universities, who definitely know what they are doing, and it involves collaboration with Ipsos MORI on the logistics and other matters, and again they are experts. It is also a very large survey – during the 22 days of Round 7 they obtained and tested, on average, over 7,600 swabs per day. One reason it can’t tell us everything is that it’s inevitably backward-looking, to a certain extent. It’s remarkable that the latest preprint is published just a week and a half after the last swabs for round 7 were taken – there is a huge quantity of lab work to be done, data to be analysed, and interpretations to be decided and reported on in that time. But, even if we just consider the part of round 7 that wasn’t covered in the interim report, the midpoint of that time was just over two weeks ago, and (for instance) confirmed cases in London have gone up by more than a third since then, up to the latest date available on the gov.uk dashboard. There are much clearer patterns of increase in the South East and the East of England, in confirmed cases, than could be seen in this REACT-1 report, and that appears to be because those increases have already got faster and much clearer since the last REACT-1 round. The issues that I’ve mentioned about potential unreliability of confirmed case data do still apply, and we’ll have to wait for later data from REACT-1 and the ONS survey to be more certain of some of the details, though I’d be amazed if the surveys don’t confirm substantial rates of increase in the affected areas.
“What else can’t REACT-1 tell us? Like all surveys, the accuracy of its estimates is affected by the sample size – that is, the number of people who are tested. The overall national total is very large, so that the estimate of the number of people who would test positive, if everyone in England were tested, will be pretty accurate. However, the estimates for subgroups of the population won’t be as accurate, because the number of people tested in the survey within the subgroups isn’t as large. That’s probably why the decreases in infection rates in the North East and in Yorkshire and the Humber don’t show up as clearly in the REACT-1 results as they do in some other data sources. In the second part of round 7, REACT-1 tested only about 2,300 people in the North East, and just under 4,000 in Yorkshire and the Humber, compared to over 14,000 in the South East and nearly 7,000 in London. That’s largely just because the population of those regions in the North is considerably smaller than that of London and the South East, so the number of people sampled is proportionally smaller there – but it means that the estimates of infection rates are less precise there, and they are less precise still for some other subgroups that were considered, such as some ethnic groups. Also, fundamentally, all that is known from the survey data is which people (in the sample) test positive and negative, and what some of their personal characteristics (age, gender, the area where they live, and so on) are. So it can tell us, for example, about differences in infection rates between people in different kinds of job, but it can’t tell us exactly how people in some jobs came to be infected more than others. It can tell us that infection rates are increasing in London, but it can’t directly tell us exactly why, or who might be infecting whom. It can’t tell us how much pressure infections might be putting on the NHS in different places, because it doesn’t measure that, but it can provide very useful information for planning the response of the NHS, alongside many other kinds of data. REACT-1 can give estimates of the R number and growth rate (or rate of decrease) in different regions, but the estimates are based only on the rates of infections measured in the survey’s sample. Estimates of R and growth rates that go into the SAGE figures that are revised weekly also include data on admissions to hospitals and to intensive care, on numbers of deaths, and from various other sources, so can take into account wider information than just how many are infected. So overall these REACT-1 results are very useful and valuable, and it’s very good that the findings are available (and available so quickly) – but they can’t answer every question and many other source of information need to be considered too.”
Preprint (not a paper): ‘REACT-1 round 7 updated report: regional heterogeneity in changes in prevalence of SARS-CoV-2 infection during the second national COVID-19 lockdown in England’ by Steven Riley et al. was posted online at 00:01 UK time on Tuesday 15 December 2020. This work is not peer-reviewed.
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Kevin McConway: “I am a Trustee of the SMC and a member of the Advisory Committee, but my quote above is in my capacity as a professional statistician.”