select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
The Office for National Statistics (ONS) have released initial estimates of prevalence of ongoing symptoms following COVID-19 infection in staff and pupils from the COVID-19 Schools Infection Survey (SIS).
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“Today’s ONS bulletin on the Schools Infection Survey (SIS), that it has been running in England in collaboration with Public Health England and the London School of Hygiene and Tropical Medicine, is based on the final round of the survey, that took place in July this year. It concentrates on ‘ongoing symptoms’ that are still present four or more weeks after someone originally had Covid-19. It’s important to be clear that this is not necessarily what’s commonly called ‘long Covid’, though there isn’t a clear accepted definition of that. NICE, in its current guideline1 on the long-term effects of Covid-19, distinguishes between ‘ongoing symptomatic Covid-19’, when the symptoms are present 4 to 12 weeks after the initial infection, and ‘post-Covid-19 syndrome’ when the symptoms have gone one 12 weeks or more. This ONS bulletin doesn’t make a distinction between these two types, and just includes anyone whose symptoms have gone on for more than 4 weeks. So it does include some people who might not be considered to have long Covid because their symptoms have not (yet) gone on long enough.
“However, that’s controversial, to some extent, because there’s disagreement about almost every definition involving long Covid and/or ongoing symptoms. Indeed, one might even argue that the main things that this bulletin demonstrates are the difficulty of counting cases of ongoing symptoms after Covid-19, and the difficulty of comparing measures from different sources of data and estimation methods.
“One problem with the sampling for the SIS is that it was never intended to provide estimates that applied to the whole of England, and all the previous ONS SIS bulletins have prominently displayed warnings telling us not to assume that the results apply outside the relatively small number of local authority areas that were included. Those local authority areas were, in any case, never intended to represent accurately local authority areas in general – they were chosen to over-sample areas that had high infection levels when the survey was planned. Today’s bulletin also carries warnings about this aspect. Since the main results are for people (pupils or staff) who had Covid-19, one might perhaps think that maybe having had Covid-19 is the key point, not where someone lives or goes to school, and that maybe the unrepresentative sample of local authorities doesn’t matter so much. But actually we don’t know that at all – it could be that the amount of infection around in an area does affect the chance of having ongoing symptoms in some way, so that these new results really aren’t representative of the general position across England.
“Another sampling issue is that the results on the percentages of pupils and staff reporting ongoing symptoms are based on relatively small numbers of people answering the survey. The SIS as a whole does not have a huge sample size, though it is adequate for its main initial aims of tracking current infection levels in schools. However, only people who said they had had Covid-19, either confirmed by a test or suspected, were included here as people who could have had ongoing symptoms. There were in all 432 staff who reported that they had had Covid-19 before, and 434 secondary pupils. Because those numbers are not large for this sort of survey, the statistical margins of error around the estimates are rather wide – particularly when the ‘confirmed’ and ‘suspected’ cases are considered separately.
“The ONS bulletin does carry a recommendation that data from the ONS Covid-19 Infection Survey (CIS) should be used instead of these new results to give estimates of the prevalence of ongoing symptoms at national level. I’d agree with that – though it does point out another set of problems. ONS have published various different kinds of estimates of the prevalence of ongoing symptoms, and none of them really matches what was recorded in the schools survey. ONS have been publishing one set of estimates at (roughly) monthly intervals – the last publication was on 2 September2 and another one is promised for 7 October. These figures are based on similar questions to those asked in the schools survey, but they are asked, not just of pupils and staff in a sample of schools in an unrepresentative set of local authorities, but of a large representative sample of people across the UK.
“For this particular set of results, CIS participants are asked whether they had ‘long Covid’ described still having symptoms more than 4 weeks after they first had Covid-19, that are not explained by anything else, and they are also asked about limitations to their daily activities and about a list of 21 specific symptoms. The SIS questions are similar, though not absolutely identical, and also include a list of 22 symptoms which is the same as the CIS list with the addition of one more (prickling, tingling or burning skin sensations). So you might expect the results so be similar, taking age into account.
“However, it’s actually pretty well impossible to compare them. That’s because these estimates based on the bigger CIS give the numbers who self-report long Covid as a percentage of the whole community population, while the SIS results give the numbers who self-report long Covid as a percentage of people who say they have previously had Covid. Since most people in the population have not had Covid, these percentages are bound to differ a great deal, and it makes little sense to compare them.
“For instance, the schools survey estimates that in July about 12%, so about 1 in every 8, secondary school pupils who previously had Covid-19, confirmed or suspected, still had ongoing symptoms more than four weeks after they first had Covid-19. The CIS survey, also covering July this year, reports that 0.65%, or about 1 in 150, of all people aged 12 to 16 in the whole community population of the UK had ongoing symptoms related to Covid-19 more than four weeks after they first had the disease. The age 12 to 16 doesn’t quite match secondary school age, but it’s quite near – and clearly 1 in 8 and 1 in 150 are very different estimates. But they are estimates of rates in completely different populations so it makes no sense to compare them anyway.
“ONS did recently publish some estimates3, based on CIS results, of the percentage of people, who had had confirmed Covid-19, that had ongoing symptoms four (or more) weeks after infection. This is the ONS bulletin that is linked from today’s SIS bulletin. In this bulletin, ONS used three different approaches of making the estimates, but the only one that it makes sense to compare with the schools survey estimates is the third estimation method, because that also uses the questions from the CIS that are very similar to the SIS questions. But again, the results are quite a lot different. For the 12-16 age group, ONS estimated that 4.7% of people with previous confirmed Covid-19 had ongoing symptoms – that’s around 1 in 20. Still a lot smaller than the SIS estimate for secondary pupils of 1 in 8, and there are similar differences for older people that might correspond to school staff.
“Today’s ONS SIS bulletin does mention these differences, and suggests some reasons, mainly to do with whether the survey participants had symptoms at the time when they first had Covid. Participants in the large CIS are tested for the virus regardless of whether they have any symptoms, and over half of those who test positive in that survey did not have symptoms around the time they tested positive. Participants in the SIS self-reported whether they had previously had Covid, which might have been on the basis of a positive test or just on suspicion based on having symptoms when they first believed they had Covid. Even if they tested positive, whether people go for a routine test can depend on whether they have symptoms. So this could indeed make a substantial difference to the chance of testing positive. Maybe that is indeed the explanation for the differences in rates of testing positive between the SIS and the CIS – but there are other differences too. And, as today’s SIS bulletin points out, there are even bigger differences between the SIS results and those from some other research studies that have tried to measure the prevalence of long-lasting symptoms.
“In my view, it would be good to make some progress on sorting out the reasons for these large differences, and on agreeing a definition of what’s meant by long Covid and ‘ongoing symptoms’. But I’m not sure that’s the overwhelming need, compared to ensuring that there are appropriate services available to people who are suffering badly from these symptoms. In any case, the strange sample design of the SIS would limit the applicability of today’s results quite seriously.”
1 https://www.nice.org.uk/guidance/ng188
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Kevin McConway: “I am a Trustee of the SMC and a member of its Advisory Committee. I am also a member of the Public Data Advisory Group, which provides expert advice to the Cabinet Office on aspects of public understanding of data during the pandemic. My quote above is in my capacity as an independent professional statistician.”