Hearing

In his book "Auditory Scene Analysis", author Albert S. Bregman likens the ear canals to two narrow channels on the edge of a lake and sound waves to water waves:

"Your friend digs two narrow channels up from the side of the lake. Each is a few feet long, and a few inches wide, and they are spaced a few feet apart. Halfway up each one, your friend stretches a handkerchief and fastens it to the sides of the channel. As waves reach the side of the lake they travel up the channels and cause the two handkerchiefs to go into motion. You are allowed to look only at the handkerchiefs and from their motions to answer a series of questions: How many boats are there on the lake, and where are they? Which is the most powerful one? Which one is closer? Is the wind blowing? Has any large object been dropped suddenly into the lake?"

As impossible as this task sounds, it is analogous to the work performed by your auditory system.

Here is a small experiment. Listen to this first recording and try to guess what it is:



No, it is not some wild animal or an alien. It is just human speech (singing) and some instruments, slowed down. Here is the original, if you are curious:



Now try to listen to the first recording again and see if you can figure out what the words are and where they start and end. People in general do not have a good appreciation of how difficult the problems of perception are (unless they are trying to build a machine to solve these problems). We have been working on speech recognition for decades but the best programs still do not perform very well except in very restricted contexts. Yet we recognize speech so effortlessly that it is difficult to see what the big deal is. Trying to recognize the words in the first recording may help you appreciate the computer's difficulty.

The thing that struck me about the first recording when I first heard it was how different it "felt" than human speech. Although it contained exactly the same information as the original its "quale" was different in philosopher-speak. This reminded me of a famous thought experiment devised by philosopher Frank Jackson: Mary the color scientist.

Mary lives in the far future when neuroscience is complete and scientists know everything there is to know about the physical processes in the brain. She has studied and learned everything there is to know about color perception: the optics of the eye, the properties of colored objects, the processing of color information in the visual system, and how this information leads to actions, memories, feelings etc. But Mary has been brought up all her life in a black and white room, she has never seen any colors at all. One day Mary is let out of her black and white room and sees colors for the first time. What happens? Does she learn anything new?

Frank Jackson argues that she obviously learns something fundamentally new: what red is like, its raw feel, its quale.

When I listen to the first recording I think of an alien speech scientist, trying to decipher the message hidden in the signal. The alien can train itself and become an expert at recognizing the words upon hearing the signal. But will it ever get the same "quale" as we get when we listen to the original recording?

Philosopher Dan Dennett argues that there are no such things as qualia and Mary will not learn anything new when she sees the colors for the first time. That is, of course, if we take the premises seriously: that she knows EVERYTHING there is to know about color perception. As counterintuitive as this sounds, I find that when the subject is the "mind", familiar and intuitive is usually wrong.