Page 24 - IOE_Winter 24
P. 24
FURTHER DEVELOPMENTS
IN MEDICAL ELECTRONICS
AI Semantic Decoder gives Paralysed a Voice Transforming Heart Health with Smart Data
Assistant Professor of Neuroscience and Researchers at Duke University, in collaboration
Computer Science Alex Huth, along with doctoral with computational scientists at the Lawrence
student in computer science Jerry Tang at Livermore National Laboratory, have made a
the University of Texas, Austin, have together major breakthrough in heart health monitoring
developed an AI semantic decoder that can by developing the Longitudinal Hemodynamic
translate brain activity into a continuous stream Mapping Framework (LHMF).
of text.
Utilising the concept of ‘digital twins’ to create a
The system utilises a transformer model similar to detailed 3D model of a patient’s blood flow, the
those behind Open AI’s ChatGPT and Google’s Bard model is based on data collected over a period
and allows brain activity to be generated while of time covering over 700,000 heartbeats. This
listening to a story or imagining telling a story. It does contrasts with the present standard for heart
not require surgical implants and is envisaged to be disease evaluation, which typically relies on
extremely useful for those who are mentally alert but snapshots of single moments in time that are
unable to speak, for example following a stroke. insufficient for the monitoring of heart disease
because it tends to develop gradually over months
Brain activity is measured using an fMRI scanner
or years.
once the decoder has been extensively trained
whilst individuals listen to hours of podcasts in the One of the primary challenges for creating
scanner. Once trained the decoder can interpret these detailed simulations was the immense
brain activity from listening to a new story or computational power required. This was addressed
imagining one, and produce corresponding text by means of a software package known as HARVEY,
that closely matches the intended text. developed by Duke University, which allows for more
efficient processing of the complex simulations.
The technology only works effectively with
participants who have trained the scanner and the By simulating heartbeats in parallel rather than
system’s practical use is restricted to laboratory sequentially, and breaking the task up among many
settings due to the dependency on the fMRI different computing nodes, the team managed to
scanner, although the researchers believe that it reduce what previously took almost a century of
could in future be adapted to more portable brain- simulation time to just 24 hours. This was achieved
imaging systems such as functional near-infrared by making reasonable assumptions about the lack
spectroscopy (fNIRS). of impact of certain time-specific coronary flows on
others, thus allowing for the simulation of different
Alex Huth states:
time chunks simultaneously prior to reassembling
“ For a non-invasive method this is a real leap them.
forward compared to what’s been done before,
The LHMF was tested on both Duke University’s
which is typically single words or short sentences.
computer cluster and on cloud computing systems
We’re getting the model to decode continuous
such as Amazon Web Services. This found that
language for extended periods of time with
errors in the LHMF simulations were negligible
complicated ideas.”
compared to traditional methods.
A PCT patent application has been filed for this work.
Applications for the LHMF include using it to
[ Reference: Fowle, H., Electronic Specifier, 14th. determine whether a patient requires a stent for
November 2023 ] the treatment of arterial plaque or lesions and
24
