Demonstrations of augmented-reality typically involve tricking you into seeing animated content such as monsters and robots that aren’t really there. Microsoft wants its HoloLens headset to mess with reality more believably. It has developed a way to make you see photorealistic 3D people that fit in with the real world.
With this technology, you could watch an acrobat tumble across your front room or witness your niece take some of her first steps. You could walk around the imaginary people just as if they were real, your viewpoint changing seamlessly as if they were actually there. A sense of touch is just about the only thing missing.
That experience is possible because Microsoft has built a kind of holographic TV studio at its headquarters in Redmond, Washington. Roughly 100 cameras capture a performance from many different angles. Software uses the different viewpoints to create a highly accurate 3D model of the person performing, resulting in a photo-real appearance.
The more traditional approach of using computer animation can’t compare, according to Steve Sullivan, who works on the project at Microsoft. He demonstrated what Microsoft calls “video holograms” at the LDV Vision Summit, an event about image-processing technology, in New York on Tuesday. More details of the technology will be released this summer.
“There’s something magical about it being real people and motion,” he said. “If you have a HoloLens, you really feel these performances are in your world.”