Poorly Trained AI Models Could Misdiagnose Kids' Sleep Issues

A dearth of paediatric data collected using artificial intelligence (AI) tools could compromise the understanding of early sleep patterns.

“Sleep stages are primarily determined by analysing the brain waves during sleep, which traditionally is done manually (or semi-automated),” said the University of Adelaide’s Associate Professor Mathias Baumert, School of Electrical and Mechanical Engineering.

“There has been a big push to automate this labour-intensive and expensive process using deep learning – an AI method that teaches computers to process data in a way that is inspired by the human brain.

“We have found evidence that errors in sleep classification generated using AI could lead to misdiagnosis of sleep disorders, particularly in children.”

Nerve impulses in the brain cause electrical activity. An electroencephalogram (EEG) records the brain’s electrical activity on a person’s scalp. Collected overnight, it provides the input data to deep learning systems.

“Deep learning systems need big EEG databases to learn the brain wave patterns associated with various sleep stages to give reliable results,” said Associate Professor Baumert.

“However, most available sleep EEG databases contain adult data only and therefore, the systems developed on those databases seem to have issues when dealing with kids’ sleep because they have never “seen” and “learnt” it.

“This means sleep disorders in children could be misdiagnosed due to using adult-biased data sets.”

The association between sleep problems, cognitive development and mental health has long been studied and recognised. Early identification of sleep problems, especially in young people, is vital to maximising the effectiveness of appropriate treatment.

Generally, sleep comprises rapid eye movement (REM) sleep and non-rapid eye movement (NREM) sleep. NREM is categorised into three stages (N1, N2, N3). Prior to the introduction of AI methods for classifying sleep patterns, a technician would examine overnight EEG, 30 seconds at a time, and assign one of the sleep stages to it.

“Sleep follows a typical cyclical pattern of sleep stages that make up the ‘sleep architecture’ and can be seen in a hypnogram - a graph that represents the stages of sleep over time,” said Associate Professor Baumert.

“Clinicians consider several metrics from hypnograms, such as sleep onset latency, sleep efficiency, time spent in each of the sleep stages, and time spent awake after sleep onset, to help them diagnose sleep disorders.”

On adult EEG, AI systems assign sleep stages correctly about nine out of ten times – on par with human experts. A similar performance can be achieved for children if paediatric data are included in the deep learning process.

Dr Huy Phan, Alan Turing Institute, UK, who was co-author of the study said improvements are needed in AI systems to avoid bias towards population groups and ensure accurate diagnoses for all patients.

“Even if trained with paediatric data, current AI systems don’t work equally well in all patients. While sleep stage classification accuracy appears sufficient when looking at EEG databases as a whole, errors in the sleep metrics of individual patients can be substantial, which is unacceptable for any diagnostic purpose.”

“Effective strategies are needed to avoid bias in the learning process and also to identify unusual behaviour of AI systems and mitigate the risk of misdiagnoses.”

The study was published in the journal Sleep Medicine.

Reference: Baumert M, Hartmann S, Phan H. Automatic sleep staging for the young and the old – Evaluating age bias in deep learning. Sleep Med. 2023;107:18-25. doi: 10.1016/j.sleep.2023.04.002

This article has been republished from the following materials. Note: material may have been edited for length and content. For further information, please contact the cited source.