select search filters
briefings
roundups & rapid reactions
before the headlines
Fiona fox's blog
A paper published in the New England Journal of Medicine (NEJM) looks at the Pfizer-BioNTech COVID-19 vaccine in a nationwide mass vaccination setting.
Prof Stephen Evans, Professor of Pharmacoepidemiology, London School of Hygiene & Tropical Medicine, said:
“This is an interesting observational study using a matched cohort design (nothing to do with a case-control design). Each day on which the study was conducted, someone who was vaccinated was matched with someone as similar as possible who was unvaccinated. There was information available on each individual from their past medical history that enabled the matching to take into account many conditions that made people vulnerable to Covid-19 outcomes.
“It has a number of strengths, but the results may have unaccounted-for biases and there will be greater uncertainty in the results than are implied by the statistical confidence intervals.
“As I noted in a previous comment “However non-randomised (observational) studies of the effects of vaccines are subject, even in the best-designed and analysed studies, to potential biases making the results less certain than the statistical analysis suggests. These biases can occur at the initiation of treatment, during follow-up and in the ascertainment of the outcome, resulting in the observed effect appearing larger or smaller than the true effect.”
“The authors have made efforts to deal with some biases, probably close to as many as are possible in any observational study, and this has been done very carefully. The results from this study are likely to be reliable, and the arguments made by the authors supporting the way they have addressed possible biases are very good.
“The key findings from the study are similar to those from the randomised trial in terms of vaccine efficacy and they are able to extend their findings to groups not – or not well – represented in the trials. They also included outcomes ranging from asymptomatic infection to death, though the numbers of events were smaller for the more serious events and so the uncertainty in the benefits seen for them was greater. There is no discussion of any adverse events.
“In spite of any limitations, these results provide further encouragement for the efficacy of this vaccine in use outside of a clinical trial.”
Comment corrected 08/03/2021 to reflect that the study is observational, not experimental, as had originally been said in the comment: Dr Peter English, Consultant in Communicable Disease Control, Former Editor of Vaccines in Practice Magazine, Immediate past Chair of the BMA Public Health Medicine Committee, said:
This is another early paper on the initial “real world” effectiveness of the Pfizer-BioNTech (BNT162b) vaccine in Israel.1 Despite it having been published quickly, the quality of the paper is of the highest. It is rigorous and thorough, with appropriate statistical analyses, and enough data has been provided to allow others to interrogate the data in different ways, should they choose to do so.
Like previous recently published papers from Israel, this is an observational study.2 3 This one, however, is a form of cohort study, which used comparison groups or “cohorts”, to allow for differences, other than their vaccination status, to be accounted for more clearly, giving greater confidence that differences between the groups are due to their vaccination status, and not to some other (“confounding”) factor.
Each person who was vaccinated from 20 December 2020 to 01 Feb 2021 was “matched” to a person of similar age, sex, geographical, clinical and other characteristics. (The paper and appendix provide extensive information about the matching process, which was robust and appropriate.) Those in the “unexposed” cohort were NOT vaccinated, while those in the “exposed” cohort were.
This allowed the investigators to compare a number of outcomes between the two cohorts; and the matching allows them to infer that differences are likely to be a consequence of vaccination.
The study “outcomes” were “documented infection with the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), symptomatic Covid-19, Covid-19-related hopitalization, severe illness, and death”.
As each person in the exposed cohort was matched to an individual in the unexposed cohort, there were the same number of participants in each “arm” of the study – 596,618 in each, a total of nearly 1.2 million participants.
It is not mentioned in the abstract (it is covered in the full paper), but we know from other papers that this trial was undertaken at a time when Covid-19 incidence was high, so a large number of “events” (the study outcomes) were to be expected, allowing large enough numbers to obtain robust estimates of vaccine efficacy; and that the more transmissible B.1.1.7 variant of the virus was prevalent at this time, showing that the vaccine is effective against this variant.
The outcomes were recorded at days 14-20 after the first dose (the second dose will have been given on day 21, which is why they stopped at day 20) and at 7 or more days after the second dose. Previous studies have shown that the first dose of vaccine has little efficacy before day 14; and it seems likely that the booster effect of the second dose will have kicked in by day 7 after the second dose.
As they only started vaccinating on 20 December, they will only have started giving second doses to people 21 days later, on 10 January; so the period in which they will have been collecting data after the second dose will have been the relatively short (two-week) period of 17 Jan to 01 Feb 2021 (if I’ve got my sums right!).
Vaccine efficacy was estimated to be as shown in Table 1 on page 2 (my table, based on the data in the Results section of the abstract).
You can see that the confidence intervals get wider as you get further down the table – this is to be expected, as there are progressively fewer cases with more severe disease. The absence of data for deaths ≥7 days after the second dose is likely to be that there were too few deaths in that short period. The findings will become more precise if the study continues and collects a lot more data – the estimated vaccine efficacy will change, and the confidence intervals will become smaller.
But this is more great news, confirming that the vaccine is around 90% effective at preventing documented infection of any degree of severity from 7 days after the second dose.
This is consistent with the findings of the phase-III trials carried out before the vaccine was marketed.
The paper is clear that, as they do not systematically test asymptomatic people in Israel, they are likely to have failed to detect some asymptomatic cases; and we know that people without symptoms can still transmit the infection. The authors acknowledge this: the appendix to the paper, on p15, shows cumulative incidence of recorded asymptomatic infection, but adds that “In the absence of systematic periodic testing… among asymptomatic people… documented asymptomatic infections do no account for all asymptomatic infections, and likely cannot account accurately capture vaccine effectiveness for this outcome”.
If the vaccine is as effective at preventing asymptomatic infection as it is at preventing symptomatic infection (around 90%), then this would be very good news indeed: if people don’t get infected, they are unable to transmit the infection to others (other than, eg, through surface contamination – they touch a contaminated surface and then somebody’s face). If the vaccine is 90% effective at preventing infection and transmission, there is a much greater chance that as the proportion of the population which has been vaccinated increases, the vaccine will be able to drive the effective R number down. These findings give us hope that vaccination alone will, eventually, get the R number below 1. If it can do this – and this is the big question – we will no longer need restrictions and behavioural measures (lockdown, masks…) to interrupt spread.
References
‘BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting’ by Noa Dagan et al. was published in the New England Journal of Medicine at 22:00 UK time on Wednesday 24 February 2021.
DOI: 10.1056/NEJMoa2101765
All our previous output on this subject can be seen at this weblink:
www.sciencemediacentre.org/tag/COVID-19
Declared interests
Prof Stephen Evans: “No conflicts of interest. I am funded (one day per week) by LSHTM. They get funding from various companies, including Astra Zeneca and GSK but I am not funded by them, I have no involvement in obtaining funding from them and I am not an investigator on any grants obtained from them. I am the statistician to the ‘meta-Data Safety and Monitoring Board’ for CEPI. I am paid for my attendance at those meetings and will be paid expenses for travel if that occurs. I am a participant in the Oxford/Astra Zeneca trial, and on 13th January 2021 learnt I had received the active vaccine.”
Dr Peter English: “No interests to declare.”