Hands on with Apple’s Vision Pro: bringing the metaverse to life

Apple has done it again. The $3,500 Vision Pro headset takes all the major computing trends of the past two decades, places them around your eyes in a way that looks sleek and feels comfortable, and offers an intuitive interface that is novel and intimate.

Following Apple’s two-hour presentation on Monday, I was among the first people to receive a private, 30-minute demo of the Vision Pro and had a chance to try a variety of features — including one “killer app” not mentioned in chief executive Tim Cook’s keynote address watched by millions online.

Moving from app to app using the device could hardly be more intuitive thanks to eye- and hand-tracking. Click a button with your right hand, and an iPhone-like home screen appears. Glance at a photo or icon, then pinch your fingers to “double-click”. You can scroll through photos with a swiping gesture, or zoom in as if a giant smartphone were projected in front of your face.

The device can easily move between virtual reality, in which the wearer is fully immersed in a digital world, and “augmented reality,” which overlays images upon the real surroundings. An Apple Watch-like dial allows you to manually fade between these two modes or, in some settings, the effect is automatic: if a person stands beside you, just look at them and their image will slowly appear and become clearer with time.

Among the features Apple could not show in its presentation were the 3D photos and videos that the headset could capture. In my private demo, I could sit around a fire with friends or have a seat at the table as children blew out birthday candles in uncanny depth.

Gene Munster, portfolio manager at Deepwater Asset Management, said this part of the demo blew him away. “3D memories are going to change how we remember things,” he said. “I’m not going to want to take a birthday party video again, unless it’s like that.”

Apple proclaimed a “new era” in “spatial computing”, suggesting the Vision Pro could do for AR/VR what the iPhone did to revolutionise mobile computing.

Wall Street, however, shrugged. Apple’s stock fell less than 1 per cent after the headset was unveiled with its high price tag that was likely to make it unaffordable to many people. Still, industry experts were taken aback by the headset’s sophistication.

“All other VR companies are in deep trouble because Apple has raised the threshold,” said Rony Abovitz, the founder and former chief of Magic Leap, a maker of augmented reality glasses. “They just laid down a gauntlet for companies like HTC and Samsung and Meta to chase. They have surpassed all of them in one shot.”

Just days before the demo, I attended AWE, a major conference for mixed reality in Santa Clara, where start-ups showcased all manner of cutting-edge technology that pointed towards a post-smartphone future. But nothing was a must-have, few devices were geared towards consumers and they were not necessarily any cheaper.

Magic Leap 2 glasses cost $3,200, while top-end headsets from Finnish group Varjo cost $6,500. I came away thinking this sort of technology had a future, but a distant one. The Apple event changed that.

Apple succeeded in demonstrating a vision of AR/VR that felt here and now, in stark contrast to the avatar-filled “metaverse” envisioned by Meta chief Mark Zuckerberg.

“To many, the metaverse concept sounds and feels too far out, driving a lack of belief of when that will ever materialise,” said Sam Cole, CEO of immersive fitness app FitXR. “What we saw today felt natural, felt obvious, it felt accessible.”

Munster from Deepwater Asset Management said he was initially “shocked” by the $3,500 price point and drafted a note to clients emphasising his disappointment. After using it, he conceded his perspective had “totally” changed. “I think it’s priced right,” he said.

Analysts pointed out that the headset felt “familiar”. The design of the device has elements of the Apple Watch and AirPods Max headphones, while the visionOS software resembles that of the iPhone and iPad. It is equipped with “Apple silicon” chips, further underscoring how Apple’s push to bring the design and production of much of its components in-house gives it another advantage over rivals.

“The thing that immediately grabbed me was the fact that anyone who used an Apple product will have instant familiarity with the device,” said Ben Wood, analyst at CCS Insight.

Impressive as it is, though, it is difficult to make the case that any consumer “needs” this device. It was entertaining to watch movie clips, view photos and take a call, and I was surprised by the clarity and comfort of simply reading a PDF document. But Apple still has some major challenges to convince a mainstream audience to invest.

Staging thousands of demos to potential consumers at Apple retail stores around the world will prove difficult. But I was unable to find anyone who came away disappointed after trying it. Most testers were wildly enthusiastic.

“I saw the presentation and thought it looked good but figured that when we tried it, we’d see the glitches,” said Francisco Jeronimo, analyst at IDC. “But my demo was completely perfect. Everything worked, like the product was ready to hit the stores. I was really impressed.”

Jeronimo added that after 20 minutes, he was ready to take it off. Despite Apple’s “EyeSight” tech — which shows the wearer’s eyes to others in real life so the device does not look antisocial — he was not sure he would wear it in a social setting.

“Even if the battery lasted all day long, I don’t see people interacting with others in their office, with a screen in front of your eyes,” he said.

Some analysts argued that the Vision Pro did not truly offer AR because unlike Magic Leap, for example, the Apple device was immersive — everything was seen through cameras, even the room one was in, instead of digital images overlaid on top of a real-world view.

But the semantics of this distinction are unlikely to matter. The latency of the video feed is just 12 milliseconds — eight times faster than the blink of an eye — and your brain will find it difficult to discern the difference between the screen inside the headset and your physical surroundings.

During my demo, I conversed with two Apple employees in the same room, and a third appeared in a moveable window through a FaceTime call. She was wearing the Vision Pro, but Apple had rendered it invisible so I could see her entire face. Apple calls this a “persona”, which sounds cartoonish, but even when I asked her to dart her eyes back and forth or made her laugh, her reactions were lifelike.

To my embarrassment, I even yelped when a dinosaur emerged from the wall in the demo room, recognised my presence and tried to bite my hand. I was told this had happened all day long.

Apple also developed proprietary cameras to take 3D videos of sports games and events such as a studio concert, enabling the wearer to feel like the action was directly in front of them. It was impressive enough to wonder if Ticketmaster was about to get disrupted.

Disappointment that the headset would not go on sale until “early next year” was palpable. Akash Nigam, CEO of Genies, an avatar tools company, said he was surprised that Apple made little to no attempt to gear the device towards Gen Z consumers. There was nothing about social media or dating apps, for example.

But millions of developers now have months to build content. And once they do, Vision Pro’s potential could emerge in ways not even Apple understands.