Towards reconstructing intelligible speech from the human auditory cortex

Reconstructing speech from the human auditory cortex creates the possibility of a speech neuroprosthetic to establish a direct communication with the brain and has been shown to be possible in both overt and covert conditions.

To advance the state-of-the-art in speech neuroprosthesis, we combined the recent advances in deep learning with the latest innovations in speech synthesis technologies to reconstruct closed-set intelligible speech from the human auditory cortex.

Reconstructing speech from the neural responses recorded from the human auditory cortex6 opens up the possibility of using this technique as a speech brain-computer interface to restore speech in severely paralyzed patients.

Expanding from the closed-set intelligible speech in this work to continuous, open-set, natural intelligible speech requires additional research, which will undoubtedly benefit from a larger amount of training data, higher-resolution neural recording technologies72, and the adaptation of regression models73 and the subject to improve the BCI system26,27.

In summary, we present a general framework that can be used for speech neuroprothesis technologies that can result in accurate and intelligible reconstructed speech from the human auditory cortex.

Comparison of time-frequency responses and the event-related potential to auditory speech stimuli in human cortex.

Speech reconstruction from human auditory cortex with deep neural networks.

This article was summarized automatically with AI / Article-Σ ™/ BuildR BOT™.

Original link