select search filters
briefings
roundups & rapid reactions
Fiona fox's blog
The Office for National Statistics (ONS) have released the latest data from their COVID-19 Infection Survey.
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“This week’s report on the ONS Infection Survey extends the data up to 23 October. There isn’t much in it in the way of good news, though there are some small encouraging points. The estimate of the percentage of people in the whole English community population that would test positive for the virus has continued to increase, and the latest week’s estimate is almost a third higher than the previous week. More or less the same goes for the ONS estimate of the daily number of new infections. The estimate for the latest week has gone up by almost half compared to last week’s estimate. The most encouraging thing I can find to say about these national figures is that both these numbers seem to be increasing at a slightly slower rate than if the increases were exponential – that is, if they were doubling in a given number of days. But they are still both increasing rapidly.
“Overall, ONS estimate that, in the week ending 23 October, 1 in 100 of the English community population (excluding people who live in communal establishments like care homes or some university halls of residence) would test positive for SARS-CoV-2. (The previous week’s figure was 1 in 130.) Like all estimates from surveys, there’s some uncertainty about it, and the data are consistent with a rate between 1 in 90 and 1 in 100. The central ONS estimate of the number of people who would test positive is 568,100, with a credible interval (plausible range) from 536,500 to 600,400. The central estimate is nearly 135,000 higher than last week’s figure – that’s a lot more people who would test positive. These are all estimates of the numbers who would test positive. At this level of infection, it’s pretty well certain that the number of false negatives (people who are actually infected but would test negative) considerably outnumber the number of false positives (people who are not infected but would test positive), so that the actual numbers of infected people will be higher than those estimates. This week, ONS have again done some estimation of the percentage of people who are actually infected, allowing for false positives and false negative, and they indicate that the percentage of people infected, on average over the fortnight from 10 to 23 October, might well be somewhere between 1.07% and 1.19%. corresponding to somewhere between about 1 in 80 and 1 in 90 people being infected with the virus.
“ONS estimate that, in the week from 17 to 23 October, there were 952 new infections per million people each day, with a credible interval from 706 to 1,453. In numbers across the English community population, that’s 51,900 new infections each day (with a credible interval from 38,500 to 79,200). That’s a very considerable increase on last week, when the central estimate was 646 new infections per million people each day. These figures, as always, are much higher than the daily reported ‘new cases’ on the Government dashboard, for which the daily rate (as an average over 7 days) was about 20,000 for England for the week in question. The difference is partly because people without symptoms would be unlikely to be tested under the system that provides these daily dashboard figures, so they would not show up as a confirmed case. People with no symptoms are still tested if they are in the ONS survey sample, and some of them will actually be infected and test positive. That’s part of the reason why the results of the ONS survey can tell us more, and more accurately, than the information from general swab testing that goes into the dashboard.
“As has been the case for several weeks now, there are big differences in the rates of positive tests between different English regions. Rates are again highest in the three regions of the North (North East, North West, Yorkshire & the Humber). In the North East region, as for the last two weeks, there’s evidence that the rate of increase may be levelling off or perhaps even falling, and the central estimate of the rate of positive tests in this region is now not much above the national average rate for England. However the rate remains high even in the North East, with an estimated 1 in 80 people testing positive, and there’s a lot of statistical uncertainty. Rates in the other two North regions are higher, indeed alarmingly high, with estimates that roughly 1 in 40 people in the North West would test positive and about 1 in 50 in Yorkshire & the Humber. Rates in other parts of the country are lower, though in all those regions the rates do appear to be increasing. That wasn’t the case last week, where there were some signs of a slowing up in a couple of regions, so this is more bad news.
“In terms of different age groups, ONS are estimating that the number of infections is growing in every age group. The rates are still highest in the younger age groups, particularly those aged about 17 (school year 12) to age 24, where ONS estimate that about 1 in 35 would test positive. But the rates of infection are growing more slowly in these younger age groups than in other groups. In people aged over 70, and indeed in children aged from 2 to about 10, the rate more than doubled in the most recent week compared to the week before.
“The ONS infection survey isn’t the only source of new information about infections in England this week. In particular, interim results from Imperial College’s REACT-1 survey1 were published yesterday, and several of the REACT-1 details look considerably different from the ONS results. It’s good that such surveys are running here at all. I don’t know of any other country where a regular, large, country-wide survey of current SARS-CoV-2 infections is being run, and here in England we have two of them. Results from this type of survey should provide more reliable estimates of infection rates than, for example, the figures that come from testing in the Test and Trace programme. That’s because both REACT-1 and the ONS survey use data from reasonably representative samples of the community population. They are tested only to provide these estimates, and not because they have asked for a test because they have symptoms, or have been advised to have a test by health professionals.
“But, in that case, why do the results from the two surveys differ? Over roughly the same period, REACT-1 estimated the number of currently infected people in England as 960,000, with an interval from 860,000 to 1,050,000 because of statistical uncertainty, while the ONS survey estimated the number of people who would test positive as 568,100, with an interval allowing for the uncertainty from 536,500 to 600,400. These ONS estimates are for the number who would test positive, not the number who are infected, which would be larger because of the inclusion of quite a few false negatives. So it makes more sense to compare the figures from both surveys for the percentage of the population that would test positive. REACT-1 estimate that, for the period from 16 to 25 October, as 1.28%, with an interval showing the uncertainty from 1.15% to 1.41%, The ONS survey figures, for a very similar period (17 to 23 October), are 1.04% with a credible interval from 0.98% to 1.10%. Still a very considerable difference, and the intervals that are intended to show the statistical uncertainty in the estimates do not even overlap. Some of the results for groups of the population, such as for regions or age groups, differ even more, though the overall patterns of differences between age groups and regions do approximately match.
“It might help to think of these estimates as being, to some extent, like a more familiar set of estimates from surveys – the estimates from opinion polls on voting intentions in an election. Usually several sets of poll results are published covering roughly the same period. The true percentages that would vote for each party are not known, and never will be. (Of course we do know the percentages on election day, but polls carried out a week or a month before that are estimating how many would vote for each party on those days, and that can’t be known.) The results from different polling companies do differ. The companies publish margins of error for their findings, but often the differences between different results are greater than those margins of error. That’s because the margins of error describe how far the results would have been different, if a different sample of people had been used, but using the same methods as the actual method and with the same method of data analysis. So they don’t allow for differences between polling companies in the way they obtain the samples of people or the ways of analysing the data, and those do differ between companies. With political polls, it’s known that results from one company might tend to give higher percentages for, say, the Conservatives, than do results from another company, and it’s not possible to tell which company is more accurate at the time the polls are taken. (Some light might be thrown on those things after the actual election.)
“Could something similar be going on with the ONS survey and REACT-1? It’s possible. The infection surveys involve much bigger numbers of people than a typical political poll, and the data comes from swab tests on the people, not from asking them who they might vote for. But the two surveys draw the people that they test from different lists – ONS use samples of addresses from a standard list of addresses, and REACT-1 uses list of people registered with GPs. The way the samples are obtained from those lists are, I believe, appropriate, and both surveys involve organisations that are very experiences in running sample surveys (ONS themselves, and Ipsos MORI for the REACT surveys) as well as academic partners (who also do know about surveys). But the lists are different, and may be representative of the population in rather different ways.
“In any national sample survey, however careful the organisers are to draw samples of people that represent the population fairly, the people who actually agree to provide data may not be so representative. Therefore it is standard practice to adjust the results statistically, by processes often known as weighting, to allow for differences in demographic characteristics that have occurred. Both the infection surveys do this, though in somewhat different ways. Their main headline results are weighted (or adjusted in similar ways), but they do also publish some unweighted results for comparison. The weighting tends to increase the estimated prevalence of infection, in both surveys, but the effect of the weighting is generally greater in REACT-1 than in the ONS survey, which would tend to indicate that the imbalances in the unweighted samples are greater for REACT-1 than for the ONS survey. That’s not necessarily a bad thing, but it could, just possibly, indicate that there are other imbalances that can’t be dealt with by weighting. Weighting could make up for certain age groups being less likely to agree to take part in the surveys, because the percentages of people in different age groups in the English population is known. But, if some population groups are less likely to agree to be swabbed than others, in a way that relates to their infection risk but isn’t clearly associated with known demographic characteristics like their age, gender, and where they live, that might not be dealt with by the weighting.
“Is there an equivalent, for these surveys, of what’s called ‘company bias’ in political polling, where the results from one company tend always to give a higher percentage vote to a certain political party, compared to the results of another company? The point here is that this bias generally is not allowed for in the published margins of error – because, if the companies know it was occurring and, importantly, they knew which of them is nearer the truth, they could and would adjust their methods. It arises because of different ways of obtaining the samples, and different ways of analysing the data, including different weighting methods. So it could be happening with the ONS and REACT-1 results, because they differ in the lists they use for sampling and in the ways they do the weighting or adjustment. They also differ, a bit, in the way they obtain the swabs for testing. Comparing the REACT-1 estimates of the percentage of positive tests, over its six rounds so far, the REACT-1 estimates are higher than the ONS estimates for similar time periods for four of the six rounds, and generally considerably higher. For the two rounds where the REACT-1 estimates are lower than the corresponding ONS figures, the REACT-1 estimates were not much lower than the ONS estimates, and these were both at times when infection rates were much lower than they are now. But again, this can’t tell us which estimates are nearer the truth, only that they seem to be systematically different. There is no better or more accurate data source with which to compare them.
“The position is different on estimates from the two surveys of the daily rate of new infections (as opposed to infections that might have been going on for some time). The point is that the ONS survey can estimate this directly, because it takes swabs from the same people, roughly weekly, for several weeks. If someone is infected on their first swab test, ONS cannot tell whether that is a new infection or not, so they do not use that result for estimating the rate of new infections. But if someone is not infected on their first test, but shows up as infected on a later test, the ONS team do know, approximately, when the new infection actually occurred, and it’s these figures that they use to estimate the daily rate of new infections. REACT-1 cannot do that because they test people only once. REACT-1 do publish an estimate of the daily number of new infections – for their interim report on Round 6, the number was estimated as 96,000 new infections each day (with an interval from 86,000 to 105,000). But they get these figures by simply assuming that people are infected for 10 days on average, and therefore dividing their estimate of the total number of infected people by 10. That does give an estimate, but it is unlikely to be as accurate as the ONS estimate, and also the interval showing the possible range is probably too narrow, because it does not allow for any variability in the average length of time for which people are infected. So on new infections, I have a strong preference for the ONS numbers (51,900 infections each day, as a central estimate, with a range from 38,500 to 79,200). Those figures are also similar to the figures for new infections, across England, published yesterday by the MRC Biostatistics Unit at Cambridge University2, which give an estimate of 55,600 new infections a day (with a range from 38,400 to 81,600). The MRC unit estimates do not come from a survey, but are based on a very different modelling process using different data, so I consider the agreement between their results and the ONS results on new infections to be a helpful triangulation, and an indication from a different source that the ONS results are reasonably reliable.”
1 https://www.imperial.ac.uk/news/207534/coronavirus-infections-rising-rapidly-england-react/
2 https://www.mrc-bsu.cam.ac.uk/nowcasting-and-forecasting-29th-october-2020/
Prof James Naismith FRS FRSE FMedSci, Director of the Rosalind Franklin Institute, and University of Oxford, said:
“ONS data suggest 1% of the English population is infected with covid19. Scotland, Wales and Northern Ireland are also estimated. Scotland does appear to have a slightly lower percentage infected. The uncertainty in NI and Wales precludes a firm conclusion.
“For England ONS estimates a headline of 52,000 (38,000 to 79,000) new cases per day to the week ending 23rd Oct.
“For the previous week, the equivalent was 35,200 (29,800 to 46,600).
“Taking the headline this 17,000 average day increase is just under 50% rise in a week. This suggests the spread of the virus has accelerated, the previous week the rise was around 25% in a week. These new data would estimate a doubling time of 12 to 14 days.
“We have three estimates of the number of new infections per day up to last week:
ONS 52,000 (38,000 to 79,000)
React 96,000 (86,000 to 105,000)
Nowcast 55,000 (38,400 to 81,600)
“We can’t simply average or say one is right. They are all well planned and carried out by experts. They all measure slightly different things and therefore have uncertainty.
“What is concerning is the numbers and trajectories reported by these three surveys, differ significantly from the average of 18,000 detected by track and trace system (week ending 21st Oct).
“Taking the headline numbers, if we were to catch all positive cases and keep the positivity rate below the 5% recommended by WHO, this would need:
1.04 million PCR tests a day (if we use the ONS numbers)
1.9 million (if we use the React numbers)
1.1 million (if we use the Nowcast numbers).
“Such high numbers are of course not realistic and only new DIY kits could ever reach this level of throughput. Whilst the increase in testing throughput in the UK has been impressive, these estimates show how a large number of positive cases stretches the testing system. The sign of stress in the testing system is where the ratio of the people who test positive in a day to the total number of people in that day tested exceeds 5%.
“The data also help understand why track and tracing has not worked. Using the ONS data, we would estimate 330,000 new cases week ending 21st Oct. This means (assuming the contact number per person are right, which is a big assumption), 1 million people need to be contacted.
“Of the 1 million people to trace, 330, 000 are outside the home. An effective system would reach 800,000 in 24 hrs from identification of a positive test and they would then isolate. The test and trace system reached 172,000 (that is 17% of the possible contacts). Of those less than 60,000 were reached with 24 hrs (6 % of contacts).
“If we assume that we reached those outside the home as well as we reached those inside the home, then of the 330,000 people out of the home we are trying to find (the whole value add of tracing), then 17% is 52,000 people and 6% is 17,000 people. The fact that a minority seem to isolate further undermines the system.
“Setting up an effective track and trace is an extremely difficult task, with the current level of infections it’s probably impossible. I wish to stress the UK is not alone in this struggle, the virus is spreading rapidly across the EU and the USA , they have also failed to implement effective tracing. The German Chancellor and French Presidents have, in particular been very clear that for their countries, increasing social restrictions remain the effective means to control the virus.
“The UK was one of the hardest hit countries in the world in the first wave. It would be particularly tragic if we repeat this experience. These three new surveys all suggest that the number of deaths will continue to climb for in the weeks ahead. These will bring terrible pain to families across the UK.
“The breakthroughs in science and medicine mean the toll will be less, but this absolutely depends on not overloading the hospital system (beds, medicines and staff). I am confident that there will be an effective vaccine to end this pandemic. We can do our part by washing our hands, wearing masks and distancing.”
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Kevin McConway: “I am a Trustee of the SMC and a member of the Advisory Committee, but my quote above is in my capacity as a professional statistician.”
None others received.