The human brain has a mind of its own and it has ways of playing tricks on you! Granted, its true intention is to help you clarify situations. When your senses get bombarded with information your mind can get confused and so the brain tries to fill in the gaps retroactively. This phenomenon is a form of postdiction – an explanation after the fact.
Postdiction can happen in a variety of different ways, either physiologically or illusory, within the process of registering multiple things at the same time creating optical illusions. Physiologically, it causes people to claim predictions of significant events (like a plane crash or natural disaster) foretelling them after they’ve occurred. The “I knew this was going to happen” scenario. Illusory, it causes people to sense something that hasn’t happened. For example, seeing something that was never even there.
In a research paper called What you saw is what you will hear: Two new illusions with audiovisual postdictive effects author Noelle Stiles states, “Illusions are a really interesting window into the brain. By investigating illusions, we can study the brain’s decision-making process.” This paper is the result of a group of researchers wanting to find out how the brain “determines reality” when a couple of your senses (in this case, sight and hearing) are overloaded with noisy and conflicting information. The work was done in the laboratory of Shinsuke Shimojo, Gertrude Baltimore Professor of Experimental Psychology and affiliated faculty member of the Tianqiao and Chrissy Chen Institute for Neuroscience at Caltech.
Through experimentation, they came to realize that when your mind isn’t sure of what’s going on, it essentially makes up information. “The brain uses assumptions about the environment to solve this problem,” Stiles said. “When these assumptions happen to be wrong, illusions can occur as the brain tries to make the best sense of a confusing situation. We can use these illusions to unveil the underlying inferences that the brain makes.”
The Illusory Rabbit Illusion (stemmed from one of the experiments in the research paper) will be our example here to show you how this works. Watch and follow along with the video below to see how your audition can dominate over your vision:
As you can see, an illusory flash is positioned post-dictively between a previous and future flash location. This indicates that what you hear can modify what you see via postdictive processing. The brain seems to be using prior information stored, together with your direct attention, to integrate multimodal cues and effectively predict future stimuli.
“When the final beep-flash pair is later presented, the brain assumes that it must have missed the flash associated with the unpaired beep and quite literally makes up the fact that there must have been a second flash that it missed,” explains Stiles. “This already implies a postdictive mechanism at work. But even more importantly, the only way that you could perceive the shifted illusory flash would be if the information that comes later in time—the final beep-flash combination—is being used to reconstruct the most likely location of the illusory flash as well.”
“Neuroscience research has begun to investigate the emerging area of multimodal postdiction, where later stimuli can affect prior stimuli. The AV Rabbit Illusions add to this growing body of postdictive effects bridging the senses, and point to a new mechanism for sensory combination within short perceptual time scales.”
In addition to Stiles and Shimojo, co-authors are former Caltech undergraduate Monica Li, Carmel Levitan of Occidental College, and former Caltech postdoc Yukiyasu Kamitani of Kyoto University and ATR Computational Neuroscience Laboratories.