select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
The government have announced the purchase of 1 million home antibody tests for nationwide COVID-19 surveillance.
Dr Hayley Jones, Senior Lecturer in Medical Statistics, University of Bristol, said:
This comment was amended on 22 October 2020 to correct a statement about samples excluded from the Ulster University study.
“We worked with PHE on an evaluation of the accuracy of the UK Rapid Test Consortium’s “AbC-19 Rapid Test” for SARS-CoV-2 antibodies, which involved testing 4,842 blood samples. The results will be released shortly, and we’ll release a Press statement at that point.
“In the meantime, the only published evidence on the accuracy of this test appears to be from a pre-print article, not yet peer reviewed, reporting results from a study led at Ulster University. I have major concerns about some of the methods used in this study. The authors estimate accuracy of the AbC-19 test by comparing test results with three laboratory-based antibody tests. However, they estimated AbC-19 test specificity (how often the test is correct among true negatives) based only on blood samples that were negative on all three laboratory tests, and sensitivity (how often the test is correct among true positives) based only on blood samples that were positive on the EuroImmun test and one other. This means that any samples for which it was considered unclear, based on the laboratory-based tests, whether they were real positives or not, were excluded from the evaluation. In doing so, the authors essentially excluded all of the “difficult to classify/diagnose” samples. This approach will always over-estimate test accuracy.
“When using antibody tests for surveillance purposes (to estimate what proportion of a population has been infected or has antibodies), which is what the Government press release indicates this test will be used for, errors in the accuracy of a test can in principle be corrected for. Test errors could be dangerous, however, if individuals receive their test results and change their behaviour based on this – due to a belief that they are immune from the virus. Regardless of test errors, of course, for this to be a sensible response to a positive test result we would need strong evidence that the presence of antibodies implies immunity. For now, any individual who receives a positive antibody test result should interpret it with caution and not change their behaviour as a result.
“The AbC-19 test is a “pregnancy-type” test that produces a test band if positive. An important issue raised by the Ulster team is that laboratory scientists performing the test in their study often found the test very difficult to read. They note that there was often “a faint test band, visible to a trained laboratory scientist but perhaps difficult to identify as positive by the general public conducting a self-test”. This highlights that further study is also needed of how accurate the test is as performed in “real life”.”
Prof Jon Deeks, Professor of Biostatistics and head of the Biostatistics, Evidence Synthesis and Test Evaluation Research Group, University of Birmingham, said:
“The UK Rapid Test Consortium has been backed by the UK Government since April to develop a home antibody test, and its imminent arrival has been given high profile. However, whilst we are waiting to see valid evidence of how well it performs, the Government has jumped the gun in purchasing the test before the independent studies evaluating it are finished or available. The PHE evaluation is complete and submitted for publication, but the authors have not been able to post a preprint of their report, so their results are still a secret. The REACT team at Imperial are still finishing their evaluation. The only available study of this test was undertaken by UK RTC members, and was shown to be critically flawed several weeks ago.
“The test has not been approved for home use by the MHRA. It can only be used by health professionals for diagnostic purposes, although it can be used by members of the public for research purposes, such as in a surveillance study. Thus it is much less useful than was hoped.
“There are many other similar tests developed by UK and overseas companies that have been available for months, for which there is good evidence of their accuracy, and have been sold around the world but not in the UK. Without the completed studies of the AbC test, we do not know whether it is better, worse or the same as these other tests. If we waited a bit, we would know whether this is really the best test for the country to purchase. For unstated reasons the Government are choosing not to do this.
“Antibody tests have a very limited role in managing COVID-19 and there are accumulating examples that they are being wrongly sold to the public. The Government must clearly articulate its policy on what they will be used for and why it is spending millions of pounds on these tests. We wasted millions on poor Chinese tests in March, we wasted millions buying many more tests than were needed from Roche (which the Government wrongly told us were 100% accurate) and now the Government is failing to show us any valid evidence at all that suggests that spending millions buying these tests is sensible.”
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Jon Deeks: “Jon Deeks is Professor of Biostatistics at the University of Birmingham and fully funded by the University of Birmingham. He leads the international Cochrane COVID-19 Diagnostic Test Accuracy Reviews team summarising the evidence of the accuracy of tests for Covid-19; he is a member of the Royal Statistical Society (RSS) Covid-19 taskforce steering group, and co-chair of the RSS Diagnostic Test Advisory Group; he is a consultant adviser to the WHO Essential Diagnostic List; and he receives payment from the BMJ as their Chief Statistical advisor.”
None others received.