Eyes Do Precision Digital Sampling
What is the shutter speed of the eye? Have you ever considered this question? After all, the eye functions like a camera in some respects. Shutterbugs know that shutter speed and aperture are factors in proper exposure. Most of us know that the iris of the eye controls the aperture, but what controls the shutter speed?
The question is much more complicated for the eye, because it is a motion picture system. Movies are typically shot at 24 frames per second, yet our brain perceives the train of still images as a continuous stream of motion. Does this mean our eyes use a shutter speed less than 24 fps? That can’t be it, because we notice the jitter when a movie pans across the scene too fast. Where are the controls for shutter speed in our visual system? And if the eye is similar to a camcorder, is it analog or digital?
To find out, a team of scientists from Harvard, Cornell, State University of New York and University of Connecticut examined the response of neurons in the mammalian eye when watching static uniform noise versus a movie of natural motion. Their results, published in Nature,1 were surprising: the eye has a variable shutter speed in the millisecond range2, and our visual system is digital. Their conclusion will sound familiar to audiophiles and HD geeks familiar with CD/DVD sampling rates:
Relative precision may be a general feature of sensory neuron communication, in which an analogue input (the sensory stimulus) is encoded by what is essentially a digital signal (the neuron’s spike train). In this context, temporal precision of neuronal responses is conceptually similar to the problem of digital sampling, in which encoding frequencies must be at least double that of the analogue signal information because of the Nyquist limit.3 From this perspective, the mechanisms that generate neuronal precision … which seem to make the encoding of visual information more complicated, may actually serve to provide easier means for downstream neurons to decode this information.
The sampling rate of the visual system, in other words, is more than twice as precise as the incoming signal. This is necessary to allow the brain to extract the maximum amount of information from the input. A high-performance CD or DVD will sound or look better at a high sampling rate. The eye, likewise, samples the visual field appropriately to preserve the maximum amount of information from the input. Audiophiles know that a high sampling rate, while good, has trade-offs; the CPU or player has to be able to keep up with the corresponding higher data rate. Since the mammalian visual field can vary from static noise to a fast-moving field packed with information, neurons automatically adjust with a variable “shutter speed” to match the information content of the scene. As a result, we get optimum performance within the physical constraints of cell biology: “the frequency content of the stimulus determines the temporal scale at which the response must be specified to reconstruct the stimulus faithfully.”
The paper said nothing about how this system could have evolved. Instead, the abstract made it clear that the scientists were approaching the problem with a focus on purpose, information, and function. Indeed, information was one of the most frequent words in the paper, used 36 times:
Using information-theoretic techniques, we demonstrate a clear role of relative precision, and show that the experimentally observed temporal structure in the neuronal response is necessary to represent accurately the more slowly changing visual world. By establishing a functional role of precision, we link visual neuron function on slow timescales to temporal structure in the response at faster timescales, and uncover a straightforward purpose of fine-timescale features of neuronal spike trains.
A layman’s summary of this complex paper was published on PhysOrg entitled, “Brain’s timing linked with timescales of the natural visual world.”
1Daniel A. Butts et al, “Temporal precision in the neural code and the timescales of natural vision,” Nature 449, 92-95 (6 September 2007) | doi:10.1038/nature06105.
2The authors said, “This remarkable precision at millisecond timescales has been observed in the retina, the lateral geniculate nucleus (LGN)and the visual cortex, as well as in many other sensory systems such as the fly visual system, the electrosensory system of the weakly electric fish, and the mammalian somatosensory and auditory systems.”
3Wikipedia explains, “The sampling theorem tells us that aliasing can be avoided if the Nyquist frequency [i.e., half the sampling frequency of a discrete signal processing system] is greater than the bandwidth, or maximum frequency, of the signal being sampled…. In principle, a Nyquist frequency just larger than the signal bandwidth is sufficient to allow perfect reconstruction of the signal from the samples.”
Science Daily picked up on this story two days after we did, with much less detail.
This finding becomes more amazing the more you think about it. It shows that comparing the eye to a camera is way too simplistic. We have a digital sampling studio in our heads! Maybe you’ve thought sometimes that the eye can’t be too great if it perceives 24 frames per second as continuous motion. Well, think about all that’s going on. The eye is a physical system – subject to physical and molecular constraints. The rhodopsin in the rod or cone in the retina (one pixel) must reset itself in a finite amount of time, because chemical reactions (protein rearrangements) cannot be instantaneous. Similarly, each neuron in the visual system requires a reset before the next firing. A neural axon contains a train of complex “ion pumps” (01/17/2002) that transmit a chemo-electric signal down the membrane to the tip. There, neurotransmitters must be delivered across a synapse to the next neuron. Though the response is rapid, it does take measurable time.
Now, multiply this constraint by the 120 megapixels in each eye that are all having to simultaneously intake photons from the incoming visual field, fire a bit to the brain, and reset (to see what’s involved there, see the 12/30/2003 entry). It’s incredible that our 3-pound jelly-like brains can keep up with it, while simultaneously monitoring our heart, breathing, and every other input coming from all the senses from head to toe. To handle this torrent of information, the visual system samples the field and digitizes it. Each neuron firing event is an element in a code. The brain does not receive an actual picture, like a projection on a screen. It receives a continuous train of neuronal signals rich with information. Because all the information has been encoded with the optimum sampling rate, the brain has all it needs to reconstruct the continuously moving scene with high fidelity. High-def TV and MPEG-4 is nothing compared to this.
Even beyond that (if you are still struggling to keep up with this mind-boggling discussion), the neuronal pattern has a temporal structure that the brain interprets to get the time-based information out of the signal. We shouldn’t think of a single shutter speed for the eye, in other words; there are hundreds of millions of individual shutters going off at their own variable rates. Each rod or cone, each neuron in the optic nerve, and each neuron in the visual cortex is automatically adjusting its firing to provide the brain with a continuous pattern, containing both spatial and temporal structure, that maximizes the amount of useful information from the scene.
So the analog-to-digital sampling is not just two-dimensional, but four-dimensional: we get a stereo image from two 2-D sources (combining the information from each eyeball), yielding 3-D, and the temporal structure makes it 4-D. This all happens within the constraints of physical chemistry.
A complex 4-D field of information, therefore, is represented in code, where each neuron firing is a bit (“the response of the neuron … consists of discrete firing ‘events’”). The temporal structure of the digitally-sampled code is optimized to preserve the maximum amount of information from the scene, without swamping the brain with unnecessary bits (TMI, too much information). Lest this commentary cause cognitive overload from TMI, we won’t remind you of another amazing fact, that the eye also does on-the-fly imaging processing (05/22/2003, 05/27/2003). Try building a robot with all this that can dive into a swimming pool.
We saw a somewhat similar encoding/decoding technology in the olfactory sense (see 11/07/2001, 06/07/2005); an almost infinitely varying input can be represented by finite neurons using a combinatorial code. Update: A new paper in Current Biology explored this very thing on Sept. 17: “spike-timing dependent plasticity” apparently is responsible for the precision in the olfactory sense of locusts. The authors said this has been found also in sight, learning and memory formation in vertebrates. It is likely that all the senses employ digital sampling to some degree. What a concept: humans have digital senses, and analog-to-digital conversion is built into our ostensibly analog anatomy. This is true for all other animals, too. What are the chances that locusts dreamed up this technology by evolution?
The authors here, again, used an information-theoretic approach to understand the role and purpose of the phenomenon under investigation. Doesn’t that sound like Intelligent Design at work? Who needs Charlie to do science? The eye only gave him cold shudders, and well it should have (12/30/2003, 05/22/2003, 11/10/2006 commentaries). He didn’t know a hundredth of the problem. 1859 was way before people knew anything about digital sampling, analog-to-digital conversion, millisecond precision, combinatorial code representation of 4-D signals, motion pictures, image processing, neuronal and genetic codes, and much, much more. Sorry, Charlie; the Darwinian revolution has been rendered obsolete by the information revolution. Get with the picture.
An evolutionary theorist from Australia responded,
The article “Eyes do Precision Sampling” attracted my attention and drew me to your site. You see (no pun intended) in New Scientist (11 August 2007, No. 2616) there was a rather derisive feature: Life’s Biggest Blunders – The design flaws that prove evolution is blind. And as expected the eye come under the writer’s critical gaze concluding “back-front retinas are a mistake”. So now comparing these articles I realized that is essential for the Darwinist to downplay design features in order to detract from anything that remotely looks design – impeccable design at that! I am agnostic but I can never deny that organic life (except human) is doing a wonderful job at functioning at optimum capacity. Thank you for this sight – sorry – site!