select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
The Office for National Statistics (ONS) have released data from the COVID Test Trace Cases Insights Survey on self-isolation after testing positive for COVID-19 between 8 and 13 March 2021.
Alan Champneys, Professor of Engineering Mathematics at the University of Bristol, said:
“Based on our published mathematical model, in closed environments like workplaces, not only is infection suppressed when self-isolation is strictly adhered to, but productivity also increases in the long run. Therefore, although only based on preliminary data, these findings that the vast majority of people testing positive in England are complying is greatly encouraging.”
Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:
“There are some fascinating details in these survey results from ONS, some of them encouraging and others not. It’s encouraging that a big majority, 82%, reported that they fully adhered to the requirements of self-isolation, but then again, that does mean that 18%, or between 1 in 5 and 1 in 6, did not fully adhere. And although most people did understand what the requirements actually are, almost a third (30%) did not, and that level of misunderstanding was considerably higher in people aged 55 and over compared to younger people (37% compared to 27%). But a lot of care is needed in interpreting the results.
“The survey appears to have been carefully done and analysed, though ONS do point out that the fieldwork had to be done in a limited time so that it was “difficult to reach a large number of people and therefore the overall sample size for the survey is limited.” But the accuracy of the results from any survey is affected by two constraints. First, not everyone in the population begin surveyed is interviewed, and the results would be different if , by chance, a different sample had been contacted instead. The effect of this sampling uncertainty is often quantified by giving what are sometimes called margins of error around the estimates, and technically the ends of the margin of error are called confidence limits. For some reason, ONS do not give this margin of error for the estimates, apart from for an overall measure of life satisfaction. However, in the case of this survey, these sampling uncertainties are likely to be reasonably small in most cases. They will almost certainly be very much smaller than what are sometimes called non-sampling errors, or biases. These can arise in many ways – perhaps the population from which the sample of people was obtained is not really the population you’re interested in, or something to do with the way the sample was chosen means that it cannot be typical of the whole population, or people’s answers don’t necessarily mean what they appear to mean. It’s pretty well impossible to get numerical estimates of the amount of uncertainty that can result from these things; you just have to be aware that they might exist. ONS do a good job of listing the main potential sources of biases like this in the “Strengths and limitations” section of the bulletin.
“The sample was drawn from lists held by the Test and Trace database of people who tested positive for the virus that causes Covid-19, and had recently come to the end of their self-isolation period. That’s reasonable, but people who didn’t get tested, even though they had symptoms and might be infected, obviously would not be on the database and could not be included. It’s possible that people with mild symptoms might avoid even asking for a test, in case the resulting legal requirement to self-isolate would mean they had to stop working and lose income. People who are on the database could only be contacted if they had supplied a valid phone number to Test and Trace, and not everyone does that. For about a third (36%) of the potential respondents, the interviewers could not make contact at all. That’s not unusual for a telephone survey, but it does bring up the question of whether the people who could be contacted are typical enough of all the people on the database. Then, slightly under half (45%) of the potential respondents who were contacted by an interviewer in fact failed to provide a response – so overall, only 16% of potential respondents, whom the interviewers tried to contact, actually provided a response. That’s not such a low figure as these things sometimes go, and as ONS point out, many potential respondents would have been ill with Covid-19 when they were contacted, which would reduce the response rate. But it again raises the question of whether the people who did respond were typical of those who should have been self-isolating. Most potential respondents, who were actually contacted, would know that the survey involved potentially illegal behaviour (of not complying with the self-isolation rules). Some may well have chosen not to give answers at all, rather than giving answers that would indicate they had done something illegal, despite the fact that the interviewers would have made it very clear that all individual responses are kept confidential. And the responses were all self-reported – so there’s no way to know whether the respondents did actually do what they said they did. People on the whole do tend to respond pretty honestly to survey interviewers, but some doubt must remain.”
Prof James Rubin, Professor of Psychology & Emerging Health Risks, King’s College London, said:
“Broadly speaking, these latest data are good news. Self-isolation is a crucial part of our defences against the COVID-19 pandemic, so the high levels of adherence found by the ONS are encouraging. We do need to be a little careful in how we interpret the results though. First, the data tell us how well adherence to self-isolation is going amongst people who have taken a COVID-19 test and provided their contact details to NHS Test and Trace. People who have a cough, fever or loss of sense of smell or taste but who do not take a test or decide not to give their contact details aren’t included in this survey. It’s reasonable to assume that self-isolation is lower in people who don’t recognise that they need to get a test or who decide not to get one for whatever reason.
“Second, the response rate for this survey is low. Out of every six people who the ONS tried to call, they had a response back from only one person. We need to ask ourselves whether there’s a systematic bias at play. Perhaps people who don’t adhere to self-isolation also tend not to respond to surveys asking about adherence?”
Prof Robert West, Department of Behavioural Science and Health, UCL, said:
“There are three important reasons to be cautious about these results. First, the rate of reported self-isolation is much higher than that found in repeated surveys using a different sampling method. Secondly, the data rely on self-report in circumstances where the pressure to report having self-isolated is very high. Thirdly, this is a sample of people who have chosen to get tested, and responded to the survey and are therefore likely to be much more compliant and the average person. I therefore think there is a high risk that the headline figure is a significant overestimate.”
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Kevin McConway: “I am a Trustee of the SMC and a member of its Advisory Committee. I am also a member of the Public Data Advisory Group, which provides expert advice to the Cabinet Office on aspects of public understanding of data during the pandemic. My quote above is in my capacity as an independent professional statistician.”
Prof James Rubin: “I participate in several government advisory groups including SAGE and provided advice for this survey.”
None others received.