October 14, 2021

expert reaction to study looking at lateral flow test accuracy at detecting SARS-CoV-2 infection and infectiousness

A study published in Clinical Epidemiology looks at lateral flow test (LFT) sensitivity.

Dr Thomas House, Reader in Mathematical Statistics, University of Manchester, said:

“This study considers how we should compare the performance of the rapid ‘LFD’ tests that are currently distributed in packs of seven in the UK for at-home testing and laboratory ‘PCR’ tests.  In particular, the authors argue that because PCR is capable of detecting levels of virus that are too low to be associated with an appreciable probability of infection, it is not appropriate to treat tests that would be LFD negative but PCR positive as false negatives when the test is used for the purposes of reduction of infection in the community.  They propose a method to calculate accuracy of LFDs that accounts for their intended use more accurately.  The method proposed seems perfectly reasonable.  This work highlights that the considerations around testing as a public health intervention during an exceptional pandemic are not the same as those in usual diagnostic settings.”

Dr Hayley Jones, Senior Lecturer in Medical Statistics, University of Bristol, said:

“The accuracy of lateral flow tests for SARS-CoV-2 is usually estimated by comparison with PCR test results.  If a substantial proportion of the people who tested PCR positive in a test evaluation study were no longer infectious, then the sensitivity of lateral flow tests to detect “infectiousness” (i.e. the chance that someone who is currently infectious will test positive) would be under-estimated.  The authors claim that this proportion may be more than 50% in some test evaluation studies.  Since the majority of people with the virus develop symptoms at some point, this problem could be easily minimised by limiting participation in test evaluation studies to people who have not recently had a positive PCR result or symptoms that have recently cleared.  Whether or not existing studies had these exclusion criteria, I don’t know.  Importantly however, even if the true sensitivity of lateral flow tests to detect infectiousness were as high as the authors claim it could be (possibly “above 80%” according this paper) a negative lateral flow test result isn’t sufficient to rule out carrying the virus or being currently infectious, especially during periods when there is a lot of SARS-CoV-2 in circulation.  People should be careful not to treat a negative lateral flow test result as a definitive “green light”, especially if they have symptoms.”

Dr Alexander Edwards, Associate Professor in Biomedical Technology, Reading School of Pharmacy, University of Reading, said:

This stuy:

“Essentially this study aims to calculate what proportion of people with high levels of virus at the time of testing could be detected using lateral flow tests.  The models and maths used work out new ways to compare the accuracy of PCR vs lateral flow.  A direct head-to-head comparison is the standard method, but this doesn’t take into account the amount of virus detected by the different methods.  The strength of this study is it uses maths to correct for this difference.  The authors quite fairly also point out limitations of the study – for example we also know that how you perform lateral flow tests can affect the accuracy.  For example sometimes the positive line isn’t very clear so not everyone will spot a weak positive; this study does not aim to address any variation in performance when lateral flow tests are used in different ways.  It also doesn’t tell us if regular testing reduces transmission, or which test is more cost-effective.

Why is it hard to define accuracy of tests?:

“Overall, there are several different ways to work out the accuracy of diagnostic tests, leading to plenty of potential for confusion.  None of these methods are perfect, and most diagnostic tests are not perfect either.  It is often true to say that one particular test has different accuracy, depending on how it is used, and depending on the situation.  This is why some people might state that lateral flow tests are “only 40% sensitive” yet others suggest they are “over 90% accurate” or “99.9% specific”.  Often all these numbers are all technically correct but seem very conflicting – so what does it actually mean?  In general, here is plenty of good science and data (and plenty of very useful tests), hidden underneath a puzzling and alarmingly large range in the percentage values presented.

Testing limitations and practicalities:

“There have been extensive efforts to understanding of strengths and limitations of different diagnostic tests for Covid-19 but also before covid-19 appeared.  For example, over the past decade people have worked very hard to make best use of lateral flow tests to diagnose infections such as HIV and malaria.  Neither lateral flow or PCR are perfect.  In general PCR has much higher “analytical sensitivity” than lateral flow, meaning PCR tests are better at detecting tiny amounts of virus – this is both a strength and a potential weakness of the method.  Cheap but imperfect tests used at scale in the community, can often be just as useful as more powerful lab methods.  Most PCR and lateral flow tests rely on swabbing, and the analytical tool is only as good as the swab – the amount of virus can change quickly, there are different swabbing techniques, and it’s so quite possible for a swab to ‘miss’ collecting any virus, so false negatives cam happen for PCR and lateral flow.

Big picture – how good is Covid-19 testing?:

“What is vital is that the general public understands several key points.  1: The covid-19 tests we are using are not perfect but are still extremely useful, and are correct far more often than they are wrong.  2: If you get a negative results from a lateral flow test, this does not prove that you are ‘safe’ – if nothing else we all know how quickly you can fall ill with any respiratory infection: feeling fine in the morning but developing a terrible cough by night-time.  3: Improvement in testing technology, and our methods to evaluate tests, are both needed.  4: Right now there are a lot of people infected with Covid-19 and a lot of tests being done, so you’d expect plenty of examples where the tests appear wrong, just from the sheer scale.  When there are lots of cases, you’d expect higher levels of false negatives.  Back when case numbers were low, you’d expect to see a higher proportion of positive results being false positives.

“Without improved technology, we will keep making the best of the tests we do have.”

‘Recalibrating SARS-CoV-2 Antigen Rapid Lateral Flow Test Relative Sensitivity from Validation Studies to Absolute Sensitivity for Indicating Individuals Shedding Transmissible Virus’ by Irene Petersen et al. was published in Clinical Epidemiology at 00:01 UK time on Thursday 14 October 2021.

DOI: 10.2147/CLEP.S311977

All our previous output on this subject can be seen at this weblink:

www.sciencemediacentre.org/tag/covid-19

Declared interests

Dr Thomas House: “Have worked on modelling TTI strategy during the pandemic.”

Dr Hayley Jones: “None.”

Dr Alexander Edwards: “I am co-investigator in a AHRC funded research project researching the design of instructions for Covid-19 community testing.  I am shareholder and co-founder in Capillary Film Technology Ltd, a diagnostic technology company which does not produce or sell Covid-19 tests.”