spatialcomputing.info

This week Epic Games released the latest version of its next-gen game engine, Unreal Engine 5. Though the new version brings improvements in many areas, its most notable features are Lumen (global illumination) and Nanite (micro-polygon geometry), which could be game-changers for VR immersion. Unfortunately the company says neither feature is ready for VR developers.

Available as of this week for all developers, Unreal Engine 5 promises to usher in a new era of game development which makes it easier for developers to create games with extremely high quality assets and realistic lighting. That’s thanks to the engine’s two new key features, Nanite and Lumen.

Nanite

Nanite is what Epic calls a “virtualized geometry” system which radically improves the geometric detail in game scenes.

Previously developers would create high quality 3D models as a sort of ‘master’ reference which would eventually have their geometry greatly simplified (leading to a reduction detail and complexity) before being pulled into the game engine. The same model generally gets several versions with increasingly reduced detail which ‘pop’ between each other depending upon how far the game camera is away from the model (known as ‘level of detail’ or ‘LOD’). This allows the game to show higher quality up close while using the reduced quality models when they are further away to save performance.

Nanite essentially functions like a continuous LOD system that draws detail from the original ‘master’ model, instead of relying on pre-built models with reduced detail. In each frame the system references the master model and pulls out the maximum level of detail needed for the given camera distance. Not only does this eliminate the need to create discrete LOD models, it also means that the range of detail for a model can be much greater, allowing players to see incredibly fine detail—right down to the original polygons of the ‘master’ model—if they get close enough.

Lumen

Meanwhile, Unreal Engine 5’s new lighting system, Lumen, greatly simplifies game lighting thanks to global illumination.

Realistic lighting can be very computationally expensive; without Lumen, many games use a combination of lighting techniques to achieve the look they want while maintaining game performance. A given scene might use pre-calculated ‘baked’ lighting (which isn’t interactive with the rest of the scene) along with a small number of real-time light-sources that cast shadows on certain objects in the scene, and various ‘screen-space’ effects to emulate realistic lighting.

Lumen unifies lighting into a single approach called global illumination, which aims to make every light in the scene—even the Sun—into a real-time light that is interactive with other lights and the rest of the scene. This includes realistic bounced light which spreads color throughout the scene based on the color of the objects that the light hits. For instance, white sunlight shining into a white room with a red floor will cast some red light onto the walls as it bounces from the red floor. Such bounced lighting is an essential component of photo-real lighting.

Both Nanite and Lumen could massively improve immersion in VR thanks to their ability to hugely increase geometric detail in nearby objects (which is especially noticeable with the stereoscopic capability of VR headsets) and to create more realistic and interactive real-time lighting.

“No Timeframe” for Nanite or Lumen in VR

Unfortunately Epic says that neither Nanite nor Lumen in UE5 are ready for VR yet.

“While we have no timeframe to share in terms of Lumen and Nanite support for VR experiences, we are exploring how to bring those UE5 features to additional platforms,” the company tells Road to VR.

But, Epic says, that doesn’t mean VR developers shouldn’t use UE5.

“VR developers can leverage most of Unreal Engine 5’s production-ready tools and features, such as the new UI, the new suite of modeling tools, creator tools such as Control Rig, MetaSounds, and World Partition for large open environments.”

What’s the Holdup?

Though both Nanite and Lumen are capable of creating incredible looking scenes, they aren’t ‘free’ from a performance standpoint.

“Although the advantages [of Nanite] can be game-changing, there are practical limits that still remain. For example, instance counts, triangles per mesh, material complexity, output resolution, and performance should be carefully measured for any combination of content and hardware,” the company warns developers. “Nanite will continue to expand its capabilities and improve performance in future releases of Unreal Engine.”

Lumen, meanwhile, is only designed to target 60 FPS for large outdoor scenes and 30 FPS for indoor scenes on the very latest console hardware. That’s a far cry from the 90 FPS minimum for most PC VR headsets. And with Quest 2 being significantly less powerful than the latest consoles, there’s just no way it’ll be able to handle those kinds of demands. Which may mean that the ultimate limitation in bringing these features to VR is simply performance (or lack thereof).

The same scene rendered for a flat screen compared to being rendered for VR is often less performant in VR due to the need for stereoscopic rendering (and usually higher resolutions). Tricks like single-pass stereo and foveated rendering help reduce this overhead, but may not yet work in conjunction with the likes of Nanite and Lumen.

So it may be matter of optimization and more powerful hardware before it’s practical to bring these features to VR experiences. From Epic’s perspective, Unreal Engine has just a small fraction of VR developers compared to the likes of the Unity engine, on which the vast majority of VR games today are built. Especially with the trajectory of Meta’s Quest 2 as the most popular target platform for developers (and its lack of power compared to consoles and PCs), it seems likely that optimizing Nanite and Lumen for VR is very low on Epic’s priority list.

Hopefully we’ll see these next-gen engine features in VR eventually, but it might not happen for some time yet.