If Apple Computer was interested in Augmented Reality systems (AR), I believe these would be the essential classes of invention that they would need to make a iPhone quality of device, which I will call iGlasses for the sake of this discussion.
- iPod-Nano style controller
Has a few IR LEDs to tell the glasses where it is in 3D space, a battery pack, on/off switch, and GPS/CPU/Network/Storage for the iGlasses. Would be used like a WiiMote or Sony Motion Controller. Other IR generating accessories could be used (ie, a glove with several LED lights). The IR on the controller would blink in a specific per-device pseudo-random pattern to prevent confusion with ambient IR sources.
An alternate configuration could be to have IR LED emitters on the frames, and IR reflective paint strips, dots, or ARID patterns on the controller or accessories.
- Accelerometers (one per temple)
- Tri-color laser projector (already available from MicroVision)
- LCD shutter glasses, switching between partially reflective and fully reflective on the inside of the glasses.
This is needed to provide both AR (overlay) and cinematic 3D (full screen) modes.
To prevent possible violation of local driving laws, the fully reflective mode cannot be used if the user is in a moving vehicle. (GPS, video analysis, accelerometers, etc can detect this).
- Micromirror laser scanner
This directs the laser light to reflect off of the inside of the glasses. (Adaptation of Texas Instruments MEMs technology, spinning mirror, or many other solutions).
- Miniature adjustable focus optics.
This is one of the two critical technologies. By defocusing the incoming laster light on a per-pixel basis, the AR overlay can be placed at the correct depth for a comfortable focus by the eye. (Off-the-shelf models are available from VariOptic). This will also be a killer feature, enabling the Hyperreal 3D and Optometrist App features described below.
- Binocular 4-color cameras (IR+RGB), fixed infinity focus
Two cameras mounted on the eye frames will provide the data for AR and other applications (ie Mood Snaps). RGB for normal video, and a separate IR channel for interacting with the iPod-Nano style controller/battery in 3D space.
- Piezoelectric microphone/heart rate/skin temp & conductivity pickups
At the temples of the iGlasses would be gold plated piezoelectric pickups, designed to pick up the jawbone sounds, heart rate, and skin conductivity at the wedge at the top of the human ear. This is the other criticial technology, which when combined with voice stress analysis, allows the Mood Snaps application to be created.
- External microphone/earphone
Over the aural canal itself would be the earbuds/external microphones. Used as expected, with built-in noise cancelling technology.
- Mesh networking
Mesh networking would allow low-latency communication between multiple close proximity users, without the need for a WiFi base station or using cell network bandwidth. This is essential for outdoor events such as live-action gaming. (Think "Virtual LaserTag")
- Mood Snaps - "Never miss another moment"
The iGlasses would take a 3D video of the last 15 seconds (adjustable) and following 10 seconds when a 'mood inflection' is sensed. Rapid change in hear rate, changes in vocal stress, very loud noises, changes in skin temperature, etc would be used to compute an 'inflection' point. Users could edit and post these automatically snapped videos at their leisure, or individuals in high risk occupations could have them automatically uploaded to a private location.
I expect this to be the primary driver behind most sales in the first year
- Optometrist App - "Your prescription is always correct"
In fully reflective mode, the iGlasses paints the incoming camera image on the inside of the glasses, and uses the dynamic focusing to automatically correct for the user's nearsighted or farsightedness. For most users with mild correctable vision, they would not need to wear their prescription glasses nor contact lenses while wearing the iGlasses.
- Hyperreal 3D - "Be There"
This feature destroys the 3D television market. By using the laser projected 3D with dynamic focus, the most vivid and comfortable to the eye 3D projection is possible. Combined with 'fully 3D' machinama technologies and the head mounted accelerometers, the user can now 'look around' during 3D movies, for a truly immersive experience. (And a new raft of headaches for the Hollywood editing room!)
- AR Anywhere
The feature that will grow the slowest, but be the real driver of sales after the first year. Augmented Reality video games; 'pop up' labelling of stores, products, (people in bars!); an arbitrarily infinite number of virtual monitors for the workplace; and even more are possible. Huge opportunities for work and play applications, that are impossible for any other platform.
