The brain's visual data-compression algorithm

Your vision stack has a keyframe interval!

When the primary visual cortex processes sequences of complete images and images with missing elements -- here vertical contours -- it "subtracts" the images from each other (the brain computes the differences between the images). Under certain circumstances, the neurons forward these image differences (bottom) rather than the entire image information (upper left). [...]

"We have now demonstrated that the visual cortex suppresses redundant information and saves energy by frequently forwarding image differences," similar to methods used for video data compression in communication technology. The study was published in Cerebral Cortex. [...]

If these individual images were presented at 33Hz (30 milliseconds per image), the neurons represented complete image information. But at 10Hz (100 milliseconds), the neurons represented only those elements that were new or missing, that is, image differences.

Tags:

5 Responses:

  1. Gianteye says:

    Well, since there are fewer neural fibers in the optic nerve than there are photoreceptors in the retina, some kind of compression has to take place. Chances are there are a lot of different compression processes going on, some that chunk information about edges and planes, some about movement, some about color, that kind of thing. After image illusions are probably breakdowns of the mechanisms solely contained in the photoreceptor system, but other ones like the reverse phi motion illusion are probably banking on compression and analysis happening on a much broader level.

    If you look at ways data is manipulated in the eye like lateral inhibition or opponent process you can start to picture how things could be bundled together to make for data feeds that are really excellent at peaking when two different slices in time are different in a way that implies sudden motion.

    I gave a talk a while back about similar kinds of stuff - https://www.youtube.com/watch?v=CzM1q6YgYtQ

  2. If you haven't read it yet, "On Intelligence" is a fascinating book about this sort of stuff at a very general level of hypothetical neurology.

  3. William Krick says:

    I smell a patent lawsuit coming.

  4. I wonder if this could be exploited, displaying image sequences at the right timing that make you believe you see something other than what you are actually seeing... This could be possibly useful on newer TVs to make a less annoying frame smoothing system, or simply a novelty trick, where you see the original images at one speed, then as the rate changes, the frame-difference calculation fills in the blanks, and you process it as something other than what is actually displayed.

  5. Ian Dutton says:

    on the 23 of dec i had a large benign mass removed from my head. it was found ne cau it was messing with my vision . on the washome from the hospital inside the car was refreshing normal but outside the car was sporatic and glitchy thre are now fewer artifacts adn ghosting i still get extra branches in the treeswhenni am outside. when i can handle looing at a screen fro longet prriods of time will definatly look in to cybernetics .