February 11, 2019

expert reaction to use of AI in evaluation and diagnosis of childhood diseases

Reactions to research published in Nature Medicine which demonstrates the application of AI and deep learning techniques to assist physicians in data analysis and diagnosis.

Prof Duc Pham, Chance Professor of Engineering and Head of School, University of Birmingham, said:

“This is another excellent application of deep learning.  As is the case with other similar applications of machine learning in medicine, the work has the potential to improve healthcare by assisting clinicians in making rapid and accurate diagnoses.  Although the authors’ results show that on average their system performed better than junior doctors, it will not replace clinicians.  Like other forms of machine learning, deep learning is inductive, i.e. it forms general rules and principles from specific training examples.  Inductive systems cannot be guaranteed to produce 100% correct results, no matter how many training examples they used or how much training they received.   Thus, critical judgments or decisions must always be left to qualified human experts to make.”

Dr Paul Tiffin, Reader in Psychometric Epidemiology and Honorary Consultant in the Psychiatry of Adolescence, University of York and Fellow of the Royal College of Psychiatrists, said:

“This is a technically impressive study. In particular, it would have taken considerable work to turn the huge volume of ‘unstructured data’, such as medical case notes, into structured data (i.e. numbers) that can be understood by a machine.

“Certainly the authors have shown the potential for machine learning to help support rapid diagnosis of illnesses in children. However, it should be stressed that the machine learning system used still relied on the quality of the recording of symptoms and other findings by clinicians. Thus, human health practitioners are not likely to be made redundant any time soon.

“Further research would be required in order to evaluate whether these approaches would translate into more effective diagnosis and care for children in the real world. Moreover, although overall levels of accuracy were relatively high, they did vary across conditions. Therefore, such systems would have to be evaluated, in comparison to usual care, for any unintended consequences related to missed or false diagnoses. It would also be important to understand how both clinicians, patients and carers responded to such automatic diagnoses, and the extent to which they trusted and acted on them”.

Prof Kazem Rahimi, Director of the Oxford Martin Programme on Deep Medicine, University of Oxford, said:

“The title of the press release implies that this could be the first major paper that provides evidence for an AI-based solution to diagnose childhood diseases. However, after reading the paper, I was disappointed not to find such a solution. Probably a title like “AI models accurately extract information from medical records and using that information they are able to make the same diagnosis as doctors” would be more appropriate. The subtle but fundamental difference between these two titles/statements is that the former implies that the models are capable of predicting the future but the latter just says that they are good at extracting important features or concepts from medical notes around the time of diagnosis. From a technical point of view, the paper has much to offer and provides an elegant example of a combination of human interpretation with several machine learning and statistical methods for classification. However, given the limitation of study design that as far as I can see seems to ignore the factor time, I have doubts about its practical applications, even as a proof-of-concept study. The key task for diagnostic algorithms is not to ‘read’ the entire notes but to assist with the process of diagnosis making, which involves predicting which questions to ask, which tests to order and then based on a narrowed range of differential diagnoses, which treatments to offer. I don’t see the present study doing any of these.”

‘Evaluation and accurate diagnoses of pediatric diseases using artificial intelligence’ by Huiying Liang et al. will be published in Nature Medicine at 16:00 UK time on Monday 11th February.

All our previous output on this subject can be seen at this weblink: http://www.sciencemediacentre.org/tag/ai-robotics/

Declared interests

Prof Duc Pham: No conflicts of interest

None others received.