Apple Vision Pro – what it is and how it works

Before talking about the Apple headset, let’s briefly clarify what virtual reality (VR) and augmented reality (AR) are:

• virtual reality is an immersive technology that allows the user to be projected into a simulated 3D environment using special headsets that isolate the user from the real world;
• augmented reality is the integration of real-time digital information into the user’s surroundings.

Apple invested heavily in AR and VR technologies. Recall the acquisition of some companies such as FlyBy Media (space mapping), PrimeSense (3D sensing sensors), FaceShift (animated avatars created by scanning people’s faces), and Metaio (augmented reality). After years of research and development, Apple finally unveiled Vision Pro, a headset for augmented and virtual reality, with its operating system called VisorOS.

ARKit 6 is a development framework that enables the creation of augmented reality apps for iOS. Using this technology and together with special sensors, such as the LiDAR Scanner, Apple devices can map their surroundings and add 2D or 3D digital elements in space.

Spatial computing is a technology that uses AR and VR, to create interactive and immersive digital environments that integrate with the real world.
After the environment around the user is mapped, thanks to both LiDAR Scanner and True Depth camera technology, digital contents are displayed in the user’s field of view as if they were positioned in the surrounding space.

Vision Pro also provides some unique features:

• navigate with eyes. It’s enough to look at an app or button to select them;
• scroll and select with fingers. Thanks to the headset’s sensors, it is sufficient to tap with two fingers to activate the app or to do a small scroll with the hand (a patented gesture called flick) to move files and possibly enlarge the displayed screen;
• dictating with voice and sending commands. For example, by looking at the microphone in the search bar, it activates and it’s possible to dictate or to activate Siri by asking her to open or close apps.

In addition to being modular and lightweight, the hardware features innovative, miniaturized and powerful technologies:

• aluminum alloy frame, slightly curved at the front that wraps around the face;
• a layered display that allows for both internal projection (such as seeing apps in the space in front of oneself) and external projection (when the user’s eyes are revealed to the interlocutor);
• the padding is lightweight and also flexes to fit the face so that no external light enters;
• the headband is elastic, breathable, and adjustable;
• a small, high-performance, external battery with a 2-hour battery life is connected via a cable to the headset and can be stored in a pocket;
• a micro-Oled system enables ultra-high-resolution viewing.

Design dual-chip:

• M2 gives power to the Vision Pro and processes projects high-resolution images;
• R1 processes data from inputs such as cameras, sensors, and microphones.

The design study was aimed at creating a slim and elegant product that is ideal for wearing on any occasion and offers immersive and engaging content viewing.

Apple addresses and solves the problem of user isolation during use. Thanks to EyeSight technology, the headset lets people around the user know whether they are using apps or fully immersed in Virtual Reality. This technology can automatically switch modes from immersion in VR to viewing the world around them. If a person approaches the user, the front display shows the user’s gaze at the person. This is a big plus for the Apple house, as well as advancement since so far other types of headsets do not allow this.

The headset is also designed for use in everyday life. Apple gives examples showing how the technology does not exclude the user from the external environment, making the experience of connection and interaction even more immersive and accessible.

Many activities can be carried out with the headset, including:

• record videos and take photos;
• bring the cinema home: with the “Pro Display XDR” display mode, you get the highest color rendering;
• listen with spatial audio: Vision Pro analyzes the space around the user, location, and materials. With the Ambient Spatial Audio system, the speakers generate sound that seems to come from outside, but at the same time allows the user to maintain awareness of their surroundings;
• immersive gaming for an even more realistic videogame experience;
• projecting environments and landscapes, for a feeling of traveling to high places.

The world of work is also being revamped. Now it is easier to connect with colleagues and work on the same document or work on separate documents without having to leave the call:

• it’s possible to open multiple apps simultaneously “around the room” and have them interact with each other (like having multiple tabs open on the desktop);
• by connecting to your Mac, its small screen is projected and you can resize it to a larger screen;
• a system scans the users and represents them perfectly. This system is used for video calling and secure authentication for Face ID;
• the ability to work on the same document together with colleagues or otherwise be able to work on other tools while on video calls.

Let’s briefly look at some differences between the Vision Pro and its main competitors:

• Meta Quest Pro – features a single display and has a controller. Therefore the hardware of Apple Vision Pro is more complex and more expensive;
• Microsoft HoloLens – the design is completely different: HoloLens is designed for work and has a narrower field of headset. On the other hand, it allows you to see a possible person in front of you.

As Apple stated at WWDC, Vision Pro will cost $3,499 and its release date will be early 2024 in the United States and then the rest of the world.

To learn more about the Vision Pro, watch Apple’s presentation video: