A team of Moscow Institute of Physics and Technology researchers, in collaboration with the Russian corporation Nerobotics, have developed artificial intelligence that can draw what a person is looking at in real-time by noninvasively reading and decoding their brainwaves! The technique works through a headpiece that is covered in electrodes called a electroencephalography (EEG) headset. The study has been published in bioRxiv.
Grigory Rashkov, one of the researchers involved in the study, said in a press release:
Researchers used to think that studying brain processes via EEG is like figuring out the internal structure of a steam engine by analyzing the smoke left behind by a steam train. We did not expect that it contains sufficient information to even partially reconstruct an image observed by a person. Yet it turned out to be quite possible.
The Study (Phase 1)
- A cap of electrodes was placed on the participant’s scalps to record their brainwaves.
- Each participant was instructed to watch 20 minutes worth of video fragments that were 10 seconds long.
- There were five categories of content. The subjects of the fragments fell into one of these categories.
The Results
- The researchers were able to recognize which category of video fragment a participant was watching by looking at their EEG data.
The Study (Phase 2)
- The team developed two neural networks for this part of the study.
- They trained one to use visual “noise” to generate images in three of the five tested categories.
- They trained the other to turn EEG data into comparable noise.
- Then, they paired the networks together to generate an image.
The Result
- When the neural networks worked together, the artificial intelligence was able to draw shockingly accurate images of what a participant was looking at exclusively from their real-time EEG data.
- The system was able to pass convincingly accurate representations that could easily be categorized 90% of the time.
Rashkov said:
What’s more, we can use this as the basis for a brain-computer interface operating in real-time. It’s fairly reassuring. Under present-day technology, the invasive neural interfaces envisioned by Elon Musk face the challenges of complex surgery and rapid deterioration due to natural processes — they oxidize and fail within several months. We hope we can eventually design more affordable neural interfaces that do not require implantation.
The team hopes their “mind-reading” system will enable new post-stroke rehabilitation devices controlled by brain signals.
