We usually feel, and talk, as if each sense is
independent of the others.
But really they are connected. How could it be otherwise? The sense we make of our senses has evolved to deal with a single external reality and all the information we receive is pooled into a single model of what’s going on in the world about.
This post is two experiments in one. The first is tactile. The second involves sight and sound.
1. This is the tactile experiment and it’s called the ‘Cutaneous rabbit‘.
What is this strange animal? Feel it for yourself.
With the finger of one hand, tap quickly on the opposite arm. 3 x 2 taps, as fast as you can say 1-2-3-4-5-6. Make two taps at the wrist; two on the upper surface of the elbow; two on the shoulder. The sketch below shows where to tap.
If you’re like me, you will experience tap 5 as being somewhere on your upper arm, between elbow and shoulder – at a position where there was no actual tap! Under laboratory conditions where the ‘taps’ can be mechanized and several can be applied rapidly to each point, people feel a spectral rabbit hopping up their arm, from wrist to shoulder.
2. But this strange effect in which a series of rapid stimuli can deceive the brain is not confined to touch, and not to just a single sense. Sound can affect visual perception. We saw earlier that vision can influence what your hear. But what you hear can also affect what you see.
In the Sound-Induced Visual Rabbit illusion an object flashes a number of time as it moves across the screen. If a series of rapid clicks is played at the same time then the object seems to flash at a greater number of positions.
Yukiyasu Kamitani has produced an animation that demonstrates the illusion. I don’t think everyone’s response will be the same. But I find what I hear does influence what I see.
The sound and vision illusion is certainly very remarkable and I don’t think any of us would have expected it. But is not unique. An article by Väljamäe and Soto-Faraco points out that similar effects have been intuitively recognized by film makers, and also advances an explanation of why sound-vision cooperation might have survival value.
They call to mind the example of a cat running around a furniture-filled room, so we see glimpses and catch snatches of sound, but what we experience is seeing a streaking cat. (I have paraphrased this in a way that I think is more evocative!)
V & S then ask us think of tracking an animal moving rapidly through undergrowth. We see it in snatches and can often hear it rustling, and so we might fill in the animal’s progress. We perceive its motion as continuous, even though we only see it occasionally. It’s easy to think how this would have survival value.
The article then goes on to give examples of sound-vision effects in films. One I like, a well known example, is from The empire strikes back by George Lucas. A spaceship door appear to slide open. We see it closed: then open: and we hear a spaceship door slide open. So that is what we see.
V & S also mention ‘Mickey Mousing’, where a director makes a cartoon character’s movements synchronize with their movements, and so gives life to them. Here is an example of Micky Mousing, from Popeye.