Super-resolution scaling AI to improve performance for Oculus Quest

March 24, 2020

Google and NVIDIA’s Super-resolution scaling AI are real winners, turning low quality images into more detailed ones, proving this software is game changing. Oculus have shown great interest and plan on using it themselves in the not too distant future.

After some research Facebook’s AI division have found a way to provide developers with more rendering power with a simple software update. It gives up to 67% more rendering power and is suitable for mobile VR hardware. Though implementation of any sort of image rendering AI can be complex and have a range of issues to address. This latest update sees clear improvements in performance for the current generation of VR hardware.

A document from Facebook AI researchers states that they are using a “super-resolution” algorithm to turn poor quality images into higher resolution, clearer images. This means for example Oculus Quest apps can be rendered at 70% of their current resolution which in turn drastically increases performance of the app allowing higher frame rates for a better experience. This super-resolution algorithm will take lower resolution images and convert them into new images utilizing the devices mobile processing architecture. The end result is much sharper and more detailed images, without using as much processing power as it normally would rendering high quality frames.

Using AI to improve image quality is nothing new, NVIDIA used its own AI-upscaling algorithm for Shield TV last year and google used it’s Super Res Zoom on the Pixel 3’s camera back in 2018. This technology is especially good for older, slower mobile chipsets like the Snapdragon 835. It’s also a huge benefit for developers who wish to port their PC VR titles over to the Quest, As well as making it easier to develop apps to run across multiple platforms.

Once, this concept of turning low quality images into high resolution ones was just stuff of science fiction. But with advances in AI, what once seemed impossible has become reality. Facebook’s super-resolution algorithm isn’t too different from previous algorithms of its kind. It teaches machines to detect common objects and then the AI imagines how the full object is meant to look. NVIDIA’s developed a similar AI algorithm for its RTX line of graphics cards to enhance images while not suffering much of a loss in performance.

For now, while there is much anticipation, there is no set timeline for rolling out the updates for Oculus Quest, or for any headsets. Dynamic Fixed Foveated Rendering was added to Oculus development kit last year, which greatly improved the performance of the Quest’s apps with no real loss in visual quality. We hope to see this update rolled out sooner rather than later.

Connect with iTRA to discuss your next project.