A decade ago, speech was first decoded from human brain signals. Still, today the accuracy and speed of translation from neural activity to speech remain far below that of natural speech. Regardless, researchers have taken things a step further and a step closer to making mind reading a reality. A team of scientists from the University of California, San Francisco, has developed an artificial intelligence (AI) that can translate brain activity into text. The study has been published in the journal Nature Neuroscience.
For now, the vocabulary is limited to a set of 250 words, and the system only works by interpreting neural patterns when someone speaks aloud. Still, they hope that eventually, the technology will help people who have lost the ability to speak or type due to paralysis, or even those with locked-in syndrome.
Joseph Makin, a co-author of the research, told The Guardian:
We are not there yet, but we think this could be the basis of a speech prosthesis.
The Study
- The team employed deep learning algorithms to analyze the brain signals of four participants who have epilepsy. They all had electrodes attached to their brains already to monitor seizures.
- Each participant read aloud a set of 50 sentences several times over. There was a total of 250 different words included in sentences such as: “Those thieves stole 30 jewels” and “Tina Turned a pop singer.”
- The team tracked the participants’ neural activity as they spoke each word and fed the data to the algorithm along with recordings of the actual speech.
- The AI system then converted the data into a string of numbers.
- That numerical representation was then sent into a second neural network – the one that deals with turning the spoken sentence into text.
The Results
- In the beginning, the system wasn’t able to translate well, and spat out nonsense sentences such as: “The spinach was a famous singe” instead of what was spoken, which was “Those musicians harmonize marvelously.”
- But the more the system compared word sequences with sentences read aloud, it began to improve. Eventually, it learned how the string of numbers corresponded to words, and it began to recognize which words tend to follow each other.
- The accuracy of the system varied between participants. However, for one of them, only 3% of their spoken sentences needed correction.
As far as mind-reading goes, the system is nowhere near being able to do that. Although, if a person has brain electrodes implanted, it may be possible someday. It will be complicated, however, because imagined speech is very different from the inner voice.
