July 23, 2020

expert reaction to paper looking at weather and COVID-19

A paper, published in Environmental and Resource Economics, has looked at the possibility of links between the weather and the spread of COVID-19.

Prof Ian Jones, Professor of Virology, University of Reading, said:

“This white paper like study provides some clear thinking on a murky subject, virus seasonality, in particular the role of the weather in the spread of SARS-CoV-2.  Some important points emerge, for example that the weather could affect the rate of testing rather than the true incidence of infection, and the general conclusion that we should not assume that risk has gone away if the sun is shining is well made.  But the fact is that respiratory viruses are generally seasonal, probably as viruses that transmit on water droplets do so less well if the droplet dries up faster, and temperature, humidity and UV may be part of the lull in transmission we are now seeing.  The flip side, alas, is that the opposite will be true in the autumn and beyond.”

Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, said:

“This new paper gives a strong warning about any data analysis that uses “existing COVID-19 data or comparable epidemiological data” to investigate the relationship between COVID-19 and the weather, on the grounds that there are important limitations and potential biases in the data.  I’d certainly agree that any researchers using data on COVID-19 should be suitably cautious in considering potential problems with the data they want to use.  There have been many warnings and examples, going back almost to the start of the pandemic, about issues with the data, differences between countries and differences over time in what counts as a case or an infection or a death related to or caused by the virus, and so on, so there’s not really anything new in suggesting that there might well be data difficulties.  It’s not even very surprising that such difficulties have arisen, in the face of a new disease caused by a new virus that was spreading very rapidly.  Because of that, I’d agree with the top line on the press release, ‘We still don’t know if warmer weather slows down the spread of COVID-19′.

“But it’s still worth looking at the basis for this recommendation in this particular study.  What is based on the data that the researchers examined, and what isn’t?  The researchers initially set out to make a statistical analysis of the relationship between weather and the spread of COVID-19.  There has been a lot of research in the past on the relationship between weather events and health events, looking (for instance) at the association between cold weather snaps and deaths from various diseases, or between heat waves and deaths, and methods to carry out these investigations are well established.  The researchers on this new study write that they used similar methods to some of these previous studies.  But, importantly in my opinion, the specific previous studies that they refer to did not look at numbers of cases of particular diseases – they looked at numbers of deaths, and their main analyses were of total numbers of deaths from any cause.  Particularly in richer countries, registration and recording of total numbers of deaths is generally pretty accurate, and certainly not subject to the sort of biases and inconsistencies that have arisen with counts of COVID-19 cases.  I do wonder why the researchers on this new study chose to look at case counts and not counts of deaths in relation to weather.  There have been issues of inconsistencies between countries with definitions of which deaths are related to COVID-19 – but that’s why many epidemiologists and statisticians have recommended instead looking at excess deaths, that is, the difference between the total deaths (from any cause) occurring on particular dates and the average number of deaths on those dates in previous years.  I strongly suspect that, if these researchers had looked at some measures of deaths rather than COVID-19 cases, they might have been able to find more useful results – though, because neither they nor I have done that data analysis, I cannot be sure of this.

“The new research points out a series of issues with data on case counts that, in their view, makes the data unusable for the kind of analysis they set out to do.  (That does not prevent them reporting the results of that analysis, however, in the supplementary materials associated with their report, which seems slightly odd given that the main report says that that supplementary analysis may be next to useless – they say “these results could be highly misleading” – because of the data issues.)  Some of those issues arise because, in many countries and in this research, a case is defined by the person testing positive for the disease (using the standard rt-PCR test).  The number of cases that could have tested positive was limited by testing capacity, particularly in the early part of the crisis, and those limitations changed over time at different speeds in different places.  Also, there have been some concerns about the accuracy of the testing, again varying over time and between places.  These concerns are spelled out clearly in the new research, but the concerns themselves are certainly not new.  What is new in this report is the statement that the weather conditions themselves might influence the numbers of people being tested, and hence the number of cases that are counted.  That’s certainly plausible, and the researchers give several reasons why it might happen (in their Table 1).  But it’s important to understand that those reasons are not based directly on the data.  The potential reasons are simply that, potential reasons, and there is no data on whether in fact they occur.  So what we have is a plausible statement that weather conditions might lead to biases in case counts, and some possible reasons why that might occur, but nothing to show that in fact it does occur.

“In summary, what we have here is a warning that there are problems with the data on COVID-19 case counts (which was, in general, already known), a plausible suggestion (but without supporting data) that weather might affect the case counts without necessarily affecting the underlying numbers of cases, a statistical analysis of relationships between weather and case counts which the researchers themselves say might be highly misleading, and no discussion of whether there exists other data on the spread of the virus, such as counts of COVID-19 deaths or excess deaths, that might be less prone to these issues and might be able to say something useful about relationships between coronavirus spread and the weather.  (To be clear, I’m not saying that analysis of deaths would in fact provide useful information, just that this research says nothing about whether it would or not.)”

‘The Challenge of Using Epidemiological Case Count Data: The Example of Confirmed COVID-19 Cases and the Weather’ by Francois Cohen et al. was published in Environmental and Resource Economics at 00:01 UK time on Thursday 23 July 2020.

All our previous output on this subject can be seen at this weblink:

www.sciencemediacentre.org/tag/covid-19

Declared interests

Prof Kevin McConway: “Prof McConway is a member of the SMC Advisory Committee, but his quote above is in his capacity as a professional statistician.”

None others received.