May 7, 2020

expert reaction to preprint looking at factors associated with COVID19-related hospital deaths using the OpenSAFELY platform

A pre-print, not a published or peer-reviewed paper uploaded to medRxiv, looks at factors associated with COVID-19 related hospital deaths in the UK using the OpenSAFELY platform.

This Roundup accompanied an SMC Briefing 

Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:

“This is an impressive piece of work.  One thing it clearly demonstrates is the value of linking data from different sources.  There can be technical issues in linking data, but there are also barriers to doing it because of concerns about matters such as privacy, so it’s good to see that these have been overcome in this case, by using methods of anonymisation.

“This is not the only new publication using data linkage about possible risk factors for death from COVID-19; the report from the Office for National Statistics (ONS) examining deaths from COVID-19 by ethnic group also linked data sets (but not the same data sets as this research).  Both studies investigated differences in death rates between ethnic groups.  Both looked at how far those differences might be explained by a range of differences between ethnic groups on factors that have previously been found affect the outcome of COVID-19.  But they did not both look at the same set of other factors, largely, I expect, because of the data available to them.  You can’t use statistical methods, however clever, to investigate something that you have no data for.  So, for example, the ONS study linked data from the 2011 Census with current death registrations, and so it could allow for what people reported about their health and disability in 2011.  But that was a long time ago, and some people who have died would not have a record from the 2011 Census.  There is plenty of data from across the world showing that certain pre-existing medical conditions have a large effect on the risk of death from COVID-19, and detail of such conditions would not have been in the data available to ONS.  The OpenSAFELY study used data from GP practices, and so could take into account much better data about pre-existing conditions.  This was done very thoroughly.  But it did not take into account some of the other factors that the ONS statisticians investigated – for example some aspects of where people lived.  The ONS research took into account the regions where they lived, and whether they lived in a rural or urban area.  The OpenSAFELY research did take region into account, in a rather different way, but not the urban/rural factor, which may be important.  Both pieces of research did take into account a measure of the socio-economic deprivation of where people lived – indeed the same measure in both studies.  And both pieces of research concluded that, even after taking into account several possible factors that might explain the variation in death rates between different ethic groups, there were still some differences between the groups.

“So there is still a question to answer about how and why these remaining differences arise.  What is it about different ethnic groups that leads to many of the minority groups having worse prospects that the majority UK White group, if they contract COVID-19, even allowing for a number of known risk factors?  This research seems to imply that it isn’t just poverty or deprivation, it may not just be where people live, and it may not simply be connected to their general health, disability or pre-existing health conditions.  So something else is involved as well, it appears.  One can speculate about other possible reasons, but without data we can’t really tell.

“Despite the generally impressive nature of the OpenSAFELY research, I should point out some limitations.  One broad one is that it is an observational study, and so can really only establish correlations and associations, rather than what actually causes death rates to vary.  More specifically, the numbers of death that they analysed are only those in NHS hospitals in England, and where tests had confirmed that the patient was infected with the new coronavirus.  Problems with this measure of deaths have been raised repeatedly – one issue is that someone who has not been tested will not be included, even if in fact they were infected.  And deaths in care homes, for instance, are not included.  The research report says that in the future, the project will look at other data on deaths.  (The ONS research used data on death registrations, which is not without problems, but generally has been rather less criticised than the hospital death counts.)  Another point to note is that, though the research is based on records from over 17 million people, they are not necessarily a random sample of the population of England.  The researchers used the data available to them, which come from just one of the electronic systems of patients records available to GP practices.  They report that the use of this particular system varies considerably from one place to another, and that in London, where a major proportion of COVID-19 cases so far have occurred, only 17% of GP practices use this system.  This may have biased the findings to an unknown extent.  And finally, I must point out that the OpenSAFELY research report has not yet been subject to peer review by other scientists.”

Preprint (not a paper): ‘OpenSAFELY: factors associated with COVID-19-related hospital death in the linked electronic health records of 17 million adult NHS patients’ by The OpenSAFELY Collaborative is on medRxiv.  This work is not peer-reviewed.

https://www.medrxiv.org/content/10.1101/2020.05.06.20092999v1

All our previous output on this subject can be seen at this weblink:

www.sciencemediacentre.org/tag/covid-19

Declared interests

Prof Kevin McConway: “Prof McConway is a member of the SMC Advisory Committee, but his quote above is in his capacity as a professional statistician.”