The system can automatically recover wide-angle views of what people are looking at, including panoramic details to the left, right and even slightly behind them. It can also calculate where people are gazing - for instance, at a single smiling face in a crowd.
The detailed wide-angle information recovered by the new system is possible because the image reflected by the cornea is broader than that captured on the retina. The retinal field of view is considerably less than a hemisphere - 160 degrees horizontally and 130 degrees vertically. But the corneal image is roughly about a hemisphere or more, permitting objects to the side and behind the person to be seen so long as the person is not looking away from the camera at an extreme angle.
The crucial algorithm in the system automatically computes the relative position and orientation of the cornea in relation to the camera, using the elliptical shape of the limbus, or border, between the cornea and the white of the eye. "The shape of the limbus tells you where the eye is in the three-dimensional scene and which direction the eyeball is pointing," Dr. Nayar said. The wide-angle image can then be created from this information.