Varjo Unlocks Next Level of Extended Reality

Chromakey technology takes immersion to new heights

One of the continued challenges with extended reality — a term that refers to virtual, augmented and mixed reality — lies in seamlessly blending real and virtual worlds. Often, the digital overlay onto the physical world relies on projections and displays, which work in a relatively limited field of view, making the experience less immersive than it could be.

Finnish company Varjo is aiming to re-establish people’s understanding of extended reality entirely, proposing to do this by integrating chromakey technology into its XR-1 headset. Chromakey is a visual effects technique, perhaps better known to most of us as “green screen” technology, allowing for alternative images to be projected over a coloured space. The difference in the case of Varjo’s headset is that the effect works in real time, rather than post-production, thanks to the pass-through cameras on the XR-1.

This essentially means that the XR-1 headset can track physical objects as well as identify a green screen, using its cameras. The chromakey integration then allows the XR-1 to project computer-generated imagery onto the green screen. The wearer, thanks to the headset, can see the real world and the computer-generated imagery simultaneously. Varjo’s ultrahigh-resolution displays are especially well-suited to this as they include what the company calls a Bionic Display, which it claims offers the equivalent of 20/20 vision. Thanks to this visual quality and chromakey integration, the XR-1 offers an incredible real-time blending of the real and digital worlds in an entirely new way.

Varjo’s promotional video puts this into a simpler context. Watch below:

I was fortunate enough to experience the technology in action in London last week. A demonstration that hit home the value of this innovation was a virtual render of the inside of a house. I was able to physically step from the real world (the “normal” part of the room) into the digital one (the part of the room covered in a green screen), and look around at the environment. However, other people could also physically step into the green-screen space, and I could see them in real time inside the virtual environment.

Similarly, other objects such as a pair of shoes could be placed down on the green-screen space and would simply appear within the virtual environment in real time. After a little while, it became genuinely difficult to distinguish between the real chair that had been placed in the virtual environment and the computer-generated ones. The quality of the XR-1’s display and cameras is such that I could even check the time on my watch without a problem.

The experience is unlike anything I’ve ever tried in any part of extended reality. Its ability to make other people and objects appear so effortlessly within a virtual environment is revolutionary.

I also tried demonstrations such as a Google Earth integration, where I could step right into a map and examine the world around me, and a flight simulator, where I was able to hold and see a real joystick as a controller while enjoying a high-resolution aerial experience. Varjo also demonstrated visual markers, essentially printed QR codes, which allow for digital objects to render for a wearer; these can be placed anywhere in the physical environment and easily moved around and manipulated.

The potential uses for this technology are vast, but the prices of the headsets may limit them to enterprise users in the immediate future. Varjo’s headsets cost nearly €10,000, so it will take some time for this technology to trickle down to most users.

Varjo’s technology isn’t without weaknesses, and there are some ways that it could be improved. For example, the ability to interact with the digital environment would elevate the experience further. This could be enhanced with feedback such as haptic response. Varjo works closely with Ultraleap — a merger of Ultrahaptics and Leap Motion — specifically on mid-air haptic feedback experiences, an area that I believe has huge potential to create new ways of interacting with extended reality content. However, this is a brand-new venture and I don’t doubt further refinements will follow, from developers using the XR-1, who have access to the integration, and from Varjo itself.

Varjo’s XR-1 headset is already an impressive example of the art of the possible for extended reality headsets. However, I believe the addition of chromakey technology represents a glimpse of the future of extended reality content more broadly, where real and virtual worlds are increasingly enmeshed. The ability to step from one to another without barriers is a revelation and I can’t wait to see the range of ways this will be used. There are clear barriers to adoption to overcome, but the technology has staggering potential to redefine the way we think of extended reality.