Text input could be the killer app for VR

Source: NVIDIA blog post. Experiencing VR.

A presentation at the GPU Technology Conference by Morgan McGuire, an Associate Professor of computer science who will soon join NVIDIA as a Distinguished Research Scientist, has touched on the technical challenges for virtual reality (VR).

If VR becomes the main interface for consumer computer use, it will one day replace keyboards, making text input the killer app for VR, McGuire was quoted as saying in a blog post.

NVIDIA will also be improving the rendering process to increase throughput, reduce latency and create a better visual experience for users. “We’re about five or six magnitudes of order [between] modern VR systems and what we want,” McGuire said. “That’s not an incremental increase.”

As VR graphics increase in resolution, the number of processes undertaken to create an image also increases, which fuels latency. “You can’t process the first pixel of the next stage until you’ve completed the final pixel in the previous stage,” McGuire said.

Future graphics systems need to be able to process 100,000 megapixels per second, up from the 450 megapixels per second available today to approach the maximum perceptive abilities of humans, McGuire said. Latency is currently at around 150 milliseconds and needs to be cut to 20 milliseconds, with a goal of getting that down under 1 millisecond, he said.

To bring latency down, McGuire said NVIDIA is, and will be, experimenting in many areas:

• Removing Path Tracer, the film industry’s primary rendering tool, from the process and replacing it with a combination of rasterisation and GPUs, speeds up the rendering process.

• Eliminating post-rasterisation tools such as shading and post-FX increases throughput and reduces latency, although it also reduces image quality.

• Eye-tracking software enables VR systems to deliver the sharpest resolution to whatever parts of an image the user is looking at, allowing the rendering process to deliver lower resolution imagery of for the rest of the display.

• Maintaining many versions and angles of an image brings down latency, but requires a lot of throughput, as the system is basically processing many images simultaneously.

• NVIDIA researchers also have been testing the effectiveness of using a sheet of holographic glass that replaces the assortment of lenses and filters that a camera uses, enabling focus-on-the-fly by moving back and forth as the user’s eyes move to different parts of an image.

Hashtag: #GTC17