05 Mar 2022

Arcane, Apple, and Watching Movies in VR

I was blown away by Arcane: League of Legends, an animated series set in the universe of Riot Games’ hugely popular video game. I haven’t played LoL, but I’ve always been intrigued by video games as a medium for story telling. I’ve even played some games just for their stories. In the end, though, video games’ stories tend to fall short of realizing their full potential. At some point, the story must take a backseat to the actual game.

Arcane realizes this full potential. It’s like if you took a video game, removed all of the gameplay, enhanced the aesthetics, and crafted a thoughtful story with fully formed characters. I’ve never seen anything quite like it. There’s something new going on here, and I think the root of it lies in how immersive it is.

You don’t feel like you’re watching Arcane; you feel like you’re participating in it. The show drops you into a richly realized world: the twin cities of Piltover and Zaun. They’re deliciously detailed. Steam comes out of vents on the ground, intricate architecture unfolds to the horizon, airships float by overhead, and a full populace ebbs and flows in the streets. All of this texture leaves the impression that you’re inhabiting a real, breathing world.

But what really instills the sense that you’re inside the story, instead of just watching it on a TV, is the way Arcane plays with perspective. Every scene is shot with a depth of field that yields a kind of three dimensional effect. Focal points are established so that you feel like you are there: like you’re looking up at a monster hulking over you, or peering down a narrow hallway straining to see what’s around the corner.

Most strikingly, the perspective frequently shifts from third-person to first-person. We might watch big sister and all around badass Violet sprinting across a rooftop and hurling herself into the air, and then cut to first-person as she/we look down to see the alleyway we’re leaping over rushing beneath our feet, and then look up just in time to see our hands bracing for impact as we roll and land on the neighboring rooftop.

See what I did there? That perspective shift, where you suddenly find yourself thinking you are that character, happens all the time in Arcane. We go from seeing a character looking at something, to looking at that something through their eyes. This is interesting beyond just the exhilaration of action sequences unfolding in first-person. It conveys emotional import. You feel bad when you watch a bigger kid picking on Violet’s little sister Powder. But you feel devastated when the scene suddenly switches to Powder’s perspective, and you see just how big and scary that bully is to her.

All of this makes me wonder: could this be a possible path for VR?

Arcane’s immersive effect would only be enhanced when viewed in virtual reality. In the simplest case, Arcane could remain exactly as is, but instead of watching it on your TV, you watch it inside a VR headset. Now it fills your entire field of view. In the same way that going from a small TV to a large one makes a movie more immersive, going from a large TV to a VR headset would take that effect even further.

It gets even more interesting though. Arcane could let you look around. Now you’re actually participating in it, controlling the perspective yourself. When Violet and Powder are walking down a crowded street, you could look over your shoulder to get a better look at a passerby. As they scramble the rooftops of Piltover, you could look up to watch that airship float overhead. And if the narrative has you worried that they’re being followed, you could look behind you to put your mind at ease.

Watching a show like Arcane in VR would be insanely cool… assuming there was a VR headset that I actually wanted to use.

Apple is widely rumored to be working on a VR headset. It’s surprising to me that there isn’t more buzz about this. If true, it could be a category defining product. Just as the iPhone completely changed what we thought we knew from Blackberries, it’s easy to imagine a VR headset from Apple causing us to look back and chuckle at today’s options. There’s still ample room for Apple to realize the ideal form of a VR headset. It’s early days.

Many of the issues that have plagued VR headsets to date are exactly the kinds of problems that Apple is exceptionally well-suited to solving. Today’s headsets are hot, heavy, and uncomfortable to wear for extended periods. They’re especially uncomfortable if you wear eyeglasses. The displays don’t have a high enough resolution to look visually stunning, and a combination of pixelation and low refresh rates can induce headaches (or worse).

If you do get a headset, unless it’s Facebook’s Meta’s Quest, you need a PC to drive it. And the specs are constantly outpacing themselves, such that you can’t simply get a VR headset, some new VR game, and assume it will “just work”.

And, frankly, the entire experience is awkward. Wearing a VR headset today completely isolates you from your surroundings. It’s disorienting. Some people can acclimate to this, but I bet most people won’t. It’s also easy to feel self-conscious, since the devices look dorky and their marketing leans weirdly into this.

Today’s VR headsets just aren’t very nice. But Apple could make one that is. They could draw upon their expertise from Apple Watch and Airpods to design a headset that’s light, breathable, and comfortable to wear for long periods of time. Just as they found a unique harddrive from Toshiba that made the iPod possible, it’s plausible that they could procure curved, Super Retina XDR-like displays that look phenomenal in a headset. And they could bring all of Apple Silicon to bear, building the computer right into the headset, so that no wires and no external computer is required. No specs to figure out. It just works.

Beyond making the headset itself nice, though, Apple is well-poised to make it nice to use. This may be what truly separates whatever Apple creates from the competition. It’s exactly the kind of design challenge – sitting at the intersection of human-computer interaction, technology, and the liberal arts – that is deep in Apple’s DNA.

Perhaps there’s a Transparency Mode to seamlessly switch from displaying VR to displaying your surroundings, just as AirPods can switch from noise cancelling to amplifying what you’re hearing around you. As with touch on the iPhone, it’s likely that Apple will discover novel ways to arrange and navigate UI inside VR, be it with controllers, gestures, or voice commands. And a VR headset could finally provide a good use case for Spatial Audio, so that if you wear your AirPods with the headset, the soundscape adjusts depending upon where you’re looking. That would be immersive.

All of these pieces could come together in a breakthrough product. Apple’s headset could finally bring VR from feeling unnatural to natural, so that it better fits into our lives. If the device looks good, feels good, and does cool things, VR could finally go mainstream. It helps that Apple’s product marketing is among the best in the world; the positioning would be less like this and more like this.

But even supposing Apple achieves all of this and ships an amazing VR headset, what are we going to do with it?

There will, of course, be many use cases. I hope the headset will serve as a tool for content creation: 3D graphics and modeling, video editing, coding… infinite Vim, anyone?! But in order to achieve the mainstream appeal that Apple will be aiming for, the headset will have to be great for content consumption.

Until I saw Arcane, I’d been scratching my head about this. Most VR content to date is in the form of video games, which Apple has been historically bad at supporting on their platforms.¹ And most of Apple’s customers aren’t serious gamers. Video games just don’t seem like a viable path for Apple to take their VR headset mainstream.

But Arcane showed me that VR movies just might be. Arcane is the demo that proves VR movies would be super cool, and my sense is that people will really like them. They would be way less awkward than VR games, which often degrade to waving your arms like a lunatic until ultimately knocking a lamp off the table. With VR movies, you’d just recline on the sofa, pull on the headset, and immerse yourself in another world for a couple hours. This seems like a fairly natural thing to do. For better or worse, people like passively consuming content. Instead of asking most people to do something completely new, VR movies would improve upon something they already do.

So I can see customers wanting to watch movies in VR.

I can also see Apple and the studios they’d partner with wanting to make them:

• Apple has the potential to make VR ubiquitous. To date, VR headsets just aren’t prevalent enough to merit investment from major film studios in a new content category like VR movies. But it’s feasible that Apple’s headset could eventually sell in volumes similar to, say, the iPad. ² This would be ubiquitous enough to cement the idea of VR movies in consumers’ minds, and to persuade film studios to make them.

• Apple TV+ is only expanding and would be a natural place for Apple to release “Made for VR” movies. This seems like a good strategy for them. In the streaming wars, exclusive content is key; this would be an exclusive category of content. And the “regular” version of these VR movies could still be viewed in TV+ on a normal TV, iPad, etc. Arcane demonstrates that you can still get a sense of virtual reality on a regular display. It wouldn’t be so unlike the 3D movies in theaters that also have a regular 2D option. Supporting both formats would avoid limiting marquee content to only those with a VR headset. It would also help create buzz to entice more people into getting one. I think this may be a theme for Apple’s VR headset overall: you can do something more immersively in VR, but there’s still a regular way to do it on your Mac, iPad, or iPhone.

• VR movies would fit into Apple’s design philosophy of tight vertical integration. They would have total control over the content, its distribution, and the hardware it runs on in order to create the best possible experience. Interestingly, this may appeal not only to Apple but also to some of Hollywood’s most progressive directors like James Cameron, Peter Jackson, and Christopher Nolan, who are famous for pushing the technical boundaries of their films and wanting total control over how they’re screened. It’s a stretch, but they might go for an exciting new technology platform with strong guarantees over what the end viewer will experience. And Apple may now actually have the connections to court filmmakers of their stature, thanks to these past years working with Hollywood for TV+.

• The video game industry only continues to grow. It surpassed the film industry a long time ago. There’s a ton of content to leverage in video games, and a ton of money being spent on that content (look no further than the recent acquisitions of Weta, Activision, and Bungie). So there’s certainly desire to maximize revenue from that content. VR movies could grow the market by bringing all that content to a broader audience, beyond gamers. Just as Arcane came out of League of Legends, video games seem like a great source for VR movies, at least initially. Animated films more naturally fit the medium than live action, game engines are probably the right technology for rendering movies where the viewer can control the camera, and game studios have the talent and domain expertise in creating immersive worlds.

• There’s been a shift from movie-theater going to home streaming. If this is a trend that continues past the pandemic, then Hollywood will need to figure out a way to monetize releasing blockbuster films straight to the home, and a new high-end VR movie platform could be one answer.

I have no idea if VR movies will actually become a thing. There are so many open questions, starting with if Apple is actually going to make a VR headset. And if they do, am I right that people will want to watch movies while wearing it? What if it’s family movie night – does everyone wear a headset? That gets expensive fast. What if it’s date night? That’s just weird.

I think these questions point to how VR isn’t going to be instead. It’s going to be in addition. I hope VR movies will be one such addition, and I’m curious what the others will be. It’s exciting and a little wild that an entirely new mode of computing could be right around the corner.