select search filters
briefings
roundups & rapid reactions
factsheets & briefing notes
before the headlines
Fiona fox's blog

expert reaction to latest data from the ONS COVID-19 Infection Survey (4th May – 17th May)

The latest Office for National Statistics (ONS) Infection Survey data, estimates the number of people currently testing positive (via swab ‘have-you-got-it’ test) for COVID-19 between 4th and 17th May.

 

Prof Sir David Spiegelhalter FRS, Chair, Winton Centre for Risk and Evidence Communication, University of Cambridge, said:

“The ONS survey results are very welcome, but their finding that infection rates do not vary between age groups should be interpreted with caution. While they are technically correct that there is no statistically significant differences, the results for younger people are only based on a handful of positive cases: the wide confidence intervals suggest there were perhaps three who tested positive out of around 1,150 2-11 year-olds tested, and only one positive case out of around 1,700 12-19 year-olds tested.”

 

Prof Carl Heneghan and Dr Jason Oke, Centre for Evidence-Based Medicine and Nuffield Department of Primary Care Health Sciences, University of Oxford, said:

“From the ONS’s report: ‘Out of the 14,599 participants’ swab tests included in this analysis, 35 individuals in 32 households tested positive for COVID-19.

‘At any given time between 4 May and 17 May 2020, it is estimated that an average of 0.25% of the community population had COVID-19 (95% confidence interval: 0.16% to 0.38%). This equates to an average of 137,000 people in England (95% confidence interval: 85,000 to 208,000); a similar level to the previous estimate indicating that the number of people with COVID-19 is relatively stable.’

These results are from self-swab testing with PCR analysis: ‘The survey involves all participants over the age of two years. We test whether they currently have the virus using self-administered throat and nose swabs, where parents or carers take swabs from younger children.’

“Given the above, we want to know what the probability is of an individual having COVID given a positive test result?

“The likelihood that you have had COVID-19 even after testing positive is low.

“If we test 10,000 people and we assume that the prevalence of COVID-19 in the population is 0.25% (the ONS estimate), then 25 people have the disease and 9,975 do not.

“Using test accuracy results taken from The BMJ ‘As current studies show marked variation and are likely to overestimate sensitivity, we will use the lower end of current estimates from systematic reviews, with the approximate numbers of 70% for sensitivity and 95% for specificity for illustrative purposes.’ – of the 25 people with COVID-19, 18 will test positive and 7 will not (sensitivity 70%). Of the 9,975 people without COVID-19, 499 people without COVID-19 will test positive and 9,476 will not (specificity 95%).

 

Sensitivity 70%

Specificity 95%

Had Covid-19 

Not had COVID-19

 

Swab test +

18

499

Positive Predictive Value = 3.5%

Swab test –

7

9476

Negative Predictive Value = 99.9%

 

25

9975

10,000

“At a specificity of 99%, 100 people without COVID will still test positive at a prevalence of 0.25% (30 positive tests would occur even in the presence of no disease if test specificity was 99.7%).

“Because we do not know how accurate the test is, we cannot comment on the results.

“We do not consider the test has 100 specificity and neither do ONS:

‘The estimates provided in this analysis are for the percentage of the private-residential population testing positive for COVID-19, otherwise known as the positivity rate. We do not report on the prevalence rate. To derive estimates for the prevalence rate instead, we would need to adjust for imperfect tests results. Since we do not have accurate information on the rate of false-positive and false-negative results, we are not providing estimates for prevalence at this time.’

“The extrapolation beyond the sample to provide estimates of COVID-19 in the wider population is too uncertain to provide meaningful estimates.

“The probability of having COVID given a positive test is 3.5% (assuming a prevalence of 0.25%, the sensitivity of 70% and specificity of 95%).

“Also, the accuracy of tests derived from secondary care populations tends to vary in primary care populations with a lower prevalence of the disease.Care should be taken when comparing the performance of tests developed and evaluated in different populations and using different methods.’

“The negative test result is helpful given the high negative predictive value, so long as the swab is done properly, it is highly likely these individuals do not have COVID.”

 

Dr Thomas House, Reader in Mathematical Statistics, University of Manchester, said:

“The current results would be expected to show the effects of changes in measures announced over a week ago if these changes were very dramatic, but to see more subtle effects we expect to require slightly longer. The higher levels of infection in health and social care workers reported earlier were statistically significant, and so we should hope that the observed reduction is the result of better control of the epidemic in these sectors, although caution in interpretation due to the small number of positive cases is still appropriate. Provided the system is sufficiently well resourced and cooperated with, we would expect that tracing and testing can have an appreciable effect on the epidemic at these levels of incidence.”

 

Prof Paul Hunter, Professor in Medicine, UEA, said:

“One concerning factor is that the rate does not seem to have declined much since the previous report so does this mean that the decline in the epidemic is perhaps stalling as people start to get out more? Too early to say but we need to monitor this. Hospitalisations and deaths (rolling 7-day averages) from COVID19 are still declining though maybe not as rapidly over the past few days so nothing to be too concerned about yet.

“Also note the similar rate in health and care workers to the general population. But the confidence intervals on H & SC workers are wide so difficult to draw conclusions. Also the epidemic in care homes and elsewhere is now on retreat, as measured by deaths in the week ending 8th May ONS. Deaths occur on average about 23 days after infection so the occupational risk will have dropped substantially in May when compared to April and there may not have been that many new infections in the period of today’s ONS study.

“So the bottom line is don’t draw too may conclusions from this.”

 

Prof Sheila Bird, formerly Programme Leader, MRC Biostatistics Unit, University of Cambridge, said:

“This is a fascinating update on ONS’s Coronavirus (COVID-19) Infection Survey: as much for what is not said as for what is reported.

“For example, there is still no transparency about response rates. Last week’s estimates were based on 10,705 swabs taken during 27 April to 10 May from people in 5,276 households. This week’s estimates are based on 14,599 swabs taken during 4 to 17 May from folk in 7,054 households.

“We may suppose that the set-up week recruited fewer households than in subsequent weeks and so I shall assume 1,749 households recruited in week 1; and 3,527 in each of weeks 2 and 3. The ratio of swabs to households is around 2 in both periods, slightly higher during 4 to 17 May than previously.

“Participation by a household in the ONS COVID-19 Infection Survey means prior consent by an index member of the household (who had participated in a previous ONS survey and given permission to be re-contacted). Hence, in the most recent 2-week period, the 14,599 swabs are, presumably, 7,054 from index-participants and 7,545 from the estimated 10,581 other household-members. If so, then co-habitees’ consent-rate would be a highly creditable 71%. Please, don’t keep us guessing ONS – transparency is so much better.

“Newly estimated is the new infection-rate, a remarkable achievement in so brief a time-scale.

“How was this achieved? ONS considers the subset of respondents who gave additional consent for 4 weekly swabs after their first. We are not told how many gave this extra-consent. If these extra-consenters were PCR-negative on their first swab, ONS considers whether their next weekly-swab had become PCR-positive (interpreted as “new infection”). Some extra-consenters will have provided two subsequent weekly swab-tests and so each of their between-swab-weekly-intervals is considered in the ONS-estimation. Very roughly, based on 3,500 participants in week 1 with 2 extra swabs each plus 7,200 participants in week 2 with one extra swab and common additional consent-rate of A%, ONS’s 6,862 additional-consent participants suggest that the additional-consent-rate (A%) was around 6,862/10,700 or 64%: again, highly credible given that four extra swabs is quite an ask!

“Strictly, the new infection rate for additional-consenters, estimated at 1.1 new infections per 1,000 household members followed for 1 week (95% CI: 0.5 to 2) should be compared with the period-prevalence for additional-consenters rather than for all participants, namely: 2.5 per 1000 (95% CI: 1.6 to 3.8). Can this be interpreted as there being 1.1 new infections per 1.4 non-new prevalent infections and hence centrally akin to a reproduction number of 0.8 but with wide uncertainty.

“As the ONS Infection Survey gathers pace, the data become ever more intriguing. And, unless I’m very much mistaken, shyness about response-rates is unwarranted.”

 

https://www.ons.gov.uk/peoplepopulationandcommunity/healthandsocialcare/conditionsanddiseases/bulletins/coronaviruscovid19infectionsurveypilot/latest

 

All our previous output on this subject can be seen at this weblink:

www.sciencemediacentre.org/tag/covid-19

 

Declared interests

Dr Thomas House: “I provide statistical and mathematical support for the ONS study and the SAGE SPI-M sub-committee; this is not an official statement from either body, and remains my opinion as an independent academic, but is subject to these declared interests.”

None others received.

 

in this section

filter RoundUps by year

search by tag