select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
A U.S. Food and Drug Administration (FDA) safety communication warns the public to stop using the Innova SARS-CoV-2 Antigen Rapid Qualitative Test for diagnostic use.
Prof Sheila Bird, Formerly Programme Leader, MRC Biostatistics Unit, University of Cambridge, said:
“The Food and Drug Administration issued its STOP notice on Innova’s lateral flow antigen tests in USA on 10th June, by coincidence one day after a Royal Statistical Society (RSS) Working Group had issued its report on diagnostic tests which made 22 recommendations – 10 on study-design, six for regulators and six on transparency. See RSS-Diagnostic-tests-report-FINAL.pdf
“Innova lateral flow antigen tests, as distributed in UK, have Department of Health and Social Care (DHSC) as their “manufacturer”. The UK’s regulator, Medicines and Healthcare products Regulatory Agency (MHRA), is conducting an investigation into the extent of read-across to Innova’s UK deployment from FDA’s stop notice across-the-pond.
“Information for users of diagnostic tests needs to be transparent about the evidential-basis for claims, for example on sensitivity and specificity in respect of a well-defined diagnostic-goal. Moreover, claims should be qualified by confidence intervals or other relevant measure of uncertainty. Goals have shifted for the INNOVA lateral flow antigen test from the initial use i) by trained personnel to diagnose SARS-CoV-2 infection in persons with symptoms to ii) assisted-screening of asymptomatic persons to diagnose SARS-CoV-2 infection or so-called “infectiousness” iii) SARS-CoV-2 infections associated with cycle-threshold < 25 or iv) SARS-CoV-2 infections associated with high viral load, e.g. more than 1 million copies or v) home-use for twice-weekly screening by asymptomatic secondary pupils for one of the aforementioned goals (infection; infectiousness defined by ct-value < 25; infectiousness defined by viral load in excess of 1 million copies) but with PCR-adjudication.
“Shifts in diagnostic-contexts and diagnostic-goals may be more numerous than my list above. A previous RSS Working Party on the Measurement of Unemployment (1995) tracked multiple shifts in the then-government’s definition of unemployment, almost all of which reduced the headline-count.
“The UK’s regulator should have tracked each shift in context-of-use for DHSC’s deployment of the Innova/DHSC lateral flow antigen test in the UK, the shift-associated evidence-base and corresponding information for users – to ensure fidelity of claims by the “manufacturer”. The Royal Statistical Society’s Working Group had considered undertaking this tracking – but lacked the powers that the MHRA should have in terms of duty on the manufacturer of a diagnostic-test to inform MHRA of substantive changes. In addition, much information for the public by DHSC and NHS Test & Trace is online with external links to other government websites so that tracking of the changes made to these linked websites is non-trivial – without either full co-operation from DHSC/NHS Test & Trace or right-of-access to the changes that were pre-notified for MHRA approval.
“Speed does not always hasten best. MHRA may need some time to address fully the plethora of study-design, regulatory and transparency issues raised for UK by FDA’s STOP notice.
“Needing some time may explain why MHRA accorded DHSC an interim 2-month extension and has yet to make public the basis for its interim judgement, see https://www.gov.uk/government/news/following-a-satisfactory-review-mhra-extends-authorisation-of-nhs-test-and-trace-lateral-flow-devices.
“Apparent failure of transparency (on methodology and findings) in how MHRA’s review addressed the public’s legitimate interest in how DHSC/Innova lateral flow antigen tests perform in different contexts of use must surely be redressed before the 2-month extension comes into force with effect from 22 June 2021.”
Prof Jon Deeks, Professor of Biostatistics and head of the Biostatistics, Evidence Synthesis and Test Evaluation Research Group, University of Birmingham, said:
“The FDA explain that they have withdrawn the Innova test for four main reasons: (a) distribution of the test in the US without approval; (b) erroneous claims on product leaflets; (c) concerns about the reliability of the evidence; and (d) failure to implement appropriate quality assurance processes to ensure that only inspected and approved devices are distributed. The FDA state that the decision is informed by inspection visits to the distribution sites in the US. Innova have responded stating that they have already or are fixing the issues which concerned the FDA and will reapply for approval.
“Given the issues raised, it is important that risk assessments are urgently undertaken in the UK by the manufacturer (which includes the Department of Health and Social Care for the repackaged NHS self-test) in consultation with the MHRA (the regulator) to assess whether similar risks occur with the supply and distribution of this test in the UK. Whilst several of the issues raised by the FDA relate only to distribution of the test in the US, some of the concerns may have broader impact. The MHRA have indicated that this process has started and will issue safety notices in due course if necessary.
“It is important to note that the approval of the NHS branded self-test in the UK is a time-limited Emergency Use Approval which expires on the 22nd June 2021, and only for using the test for “test-to-find” purposes. Thus one would expect the review of these issues to be done quickly to allow decisions about ongoing use of the test to be made.
“There have been multiple issues with lack of transparency in the evidence to support the Government’s policies for use of this lateral flow test, which negatively impacts on uptake. Given the more serious concerns identified by the FDA, it is essential that full explanations and data are provided to explain decisions made about its continued use, if that is the decision made.
“However, wider concerns about the evidence base need to be addressed. Most of the UK data about sensitivity are from using the test in laboratory settings, and it is well established that tests in the real world tend to give lower estimates of sensitivity than those does in the lab. Currently there are no data available showing how well the test can detect cases in children, and when used as a self-test – two key components of current Government testing policy. Regardless of the outcome of the MHRA investigation it is essential that greater efforts are put in place to ensure that strong evidence exists before policies are implemented, and that it is fully and transparently reported in a timely manner.”
Dr Alexander Edwards, Associate Professor in Biomedical Technology, Reading School of Pharmacy, University of Reading, said:
“It’s hard to work out how the tests named by the FDA notice relate to the DHSC home tests distributed widely across the UK, which have their own packaging and instructions. It’s also not clear from reviewing the FDA notice what exactly is the source of concern. Generally speaking, the FDA has very high standards for diagnostic tests – they are also very experienced in evaluating COVID-19 tests.
“Lateral flow tests are easy to distribute, can be mass produced, and can measure important clinical targets. One well-known limitation is they are not able to detect such low levels of a target as many laboratory tests. What is less obvious is that any test operated outside a laboratory has an extra limitation: it’s much harder to be sure that it works as intended, because it’s used in an uncontrolled environment. One consequence of this is it usually takes a lot longer to validate an out-of-lab test, because so much work is needed to refine the design and be certain that it always works reliably without needing supervised operation (this can take years). During a pandemic, time is against us.
“Laid on top of this, asymptomatic screening for infection is far harder than it might seem, and generally we test people with symptoms. Of course, with COVID-19, this can cause problems if people spread the virus before they become symptomatic and seek testing.
“In theory, the lateral flow tests distributed by DHSC that were manufactured by Innova have been validated in UK trials. At least some of the evidence behind this has been published, and this showed some variation in accuracy depending on who uses the test, and on which people. But the widespread home, school and workplace testing with lateral flow remains controversial because of remaining limitations in evidence that it is actually effective at preventing virus spread. Because the tests are being used so widely, it’s also far harder to check that all the tests are functioning correctly in all locations.
“Hopefully the guidance for use of DHSC lateral flow tests has always been clear – and the public has understood: a negative home lateral flow result does not rule out being infected (or infectious). Thus, these home tests should only be used to try to help pick up community cases early: a negative test does not prove anyone is “safe”.
“The FDA notice talks about ‘for diagnostic use’ and the DHSC home test kits are not exactly intended for diagnostics. If you do have COVID-19 symptoms, then you need a PCR test to help confirm if you are infected with COVID-19. Asymptomatic testing is not really a diagnostic process, because no-one is ill. This may be a reason to be cautious about rejecting Innova tests based on the US FDA recall. In other words, it’s possible that the Innova products flagged by the FDA are being sold for a different purpose. It’s also possible that whatever problems the FDA noticed, will be fixed rapidly by Innova. Let’s hope this is the source of the notice from the US regulator.”
Prof Sheila Bird, Formerly Programme Leader, MRC Biostatistics Unit, University of Cambridge, said:
“The report by the Royal Statistical Society Working Group on Diagnostic Tests (https://rss.org.uk/RSS/media/File-library/Policy/2021/RSS-Diagnostic-tests-report-FINAL.pdf ) was issued on 9 June 2021, presciently but coincidentally one day ahead of the US Food and Drugs Administration’s stop of the Innova SARS-Cov-2 antigen rapid qualitative test. The RSS report made 22 recommendations on: study-design matters (10), regulation matters (6) and transparency matters (6). The FDA action addresses all three elements.
“The UK’s regulator, the Medicines and Healthcare products Regulatory Authority, needs urgently to clarify the extent of read-across from across the pond to the 7-kit lateral flow test-packs which DHSC has made available to UK citizens for twice-weekly asymptomatic screening and which currently feature in the randomized controlled trial of Daily Contact Tracing versus self-isolation that Michael Gove announced his participation in.
“The RSS Working Group calls for stricter regulation of in vitro diagnostic tests and stronger heed to statistical science – notably study-design and statistical reporting standards – in their evaluation.”
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/covid-19
Declared interests
Prof Sheila Bird: “SMB was proposer and is a member of the Royal Statistical Society’s Working Group on Diagnostic Tests and serves on RSS’s COVID-19 Taskforce. Since January 2021, SMB also serves on NHS Test & Trace/PHE’s Testing Initiatives Evaluation Board.”
Prof Jon Deeks: “JD is an external advisor to the MHRA and the WHO, and leads the Cochrane reviews of the accuracy of tests for SARS-CoV-2 supported by funding from the Foundation for new Innovative Diagnostics.”
Dr Alexander Edwards: “No conflicts.”
Prof Sheila Bird: “SMB was proposer and is a member of the Royal Statistical Society’s Working Group on Diagnostic Tests and serves on RSS’s COVID-19 Taskforce. Since January 2021, SMB also serves on NHS Test & Trace/PHE’s Testing Initiatives Evaluation Board.”