Doctors have found a way to extract a person’s speech directly from their brain! Their radical new approach is the first to demonstrate how a person’s intention to say specific words can be gleaned from brain signals and turned into text fast enough to keep pace with natural conversation. This breakthrough is the first step towards a more powerful system that can decode the words a person intends to say in real-time.
Edward Chang, a neurosurgeon and lead researcher on the study published in the journal Nature Communications, said:
To date there is no speech prosthetic system that allows users to have interactions on the rapid timescale of a human conversation.
For now, the brain-reading software works only for stock sentences it has been trained on. The challenge to create this software was taken on by doctors at the University of California in San Francisco and funded by Facebook. They wanted to create a product that allows paralyzed people to communicate more fluidly, unlike existing devices that have to pick up eye movements and muscle twitches to control a virtual keyboard.
The study involved three epilepsy patients who were about to have neurosurgery for their condition. All the patients could speak normally and agreed to take part in Chang’s research. All three of them had a small patch of tiny electrodes placed directly on the brain at least one week before their operations went ahead. The patch mapped the origins of their seizures so the doctors would know where exactly to operate. Chang used these electrodes to record brain activity while each patient was asked nine set questions and asked to read a list of 24 potential responses.
Chang and his team then used the recordings and built computer models that learned to match particular patterns of brain activity to the questions the patients heard and the answers they spoke. After running all the recordings through, the trained software was able to identify almost instantly, and from brain signals alone, what question a patient heard and what response they gave, with an accuracy of 76% and 61% respectively.
David Moses, a researcher on the team, said:
This is the first time this approach has been used to identify spoken words and phrases. It’s important to keep in mind that we achieved this using a very limited vocabulary, but in future studies we hope to increase the flexibility as well as the accuracy of what we can translate.
The system they created was rudimentary. Nevertheless, it allowed patients to answer questions about the music they liked; how well they were feeling; whether their room was too hot or cold, or too bright or dark; and when they would like to be checked on again.
Next, the researchers need to improve the software so it can translate brain signals into more varied speech quickly. Such ability will require algorithms trained on a huge amount of spoken language and corresponding brain signal data, which may vary from patient to patient. Then, they want to figure out how to get the software to read “imagined speech”, or sentences spoken in the mind. The current system detects “the machinery of speech” – brain signals that are sent to move the lips, tongue, jaw, and larynx.
Some people were concerned about the ethical issues such systems might raise in the future. For example, Winston Chiong, a neuroethicist at UCSF who was not involved in the latest study, was worried that maybe a “speech neuroprosthesis” could unintentionally reveal people’s private thoughts. However, Chang explained that “decoding what someone was trying to say was hard enough, and that extracting their inner thoughts was virtually impossible”.
He said:
I have no interest in developing a technology to find out what people are thinking, even if it were possible. But if someone wants to communicate and can’t, I think we have a responsibility as scientists and clinicians to restore that most fundamental human ability.
