Why Apple bet on Pixar tech to make the metaverse work

In June, Apple finally revealed its long-rumored Vision Pro headset, which lets users see immersive 3D worlds on an extremely premium device. There’s just one problem: someone needs to build those worlds, and building them requires a lot of work.

But that’s not only going to be on Apple; the company’s also looking to developers to make great apps for the headset. So to help make it easier to build that 3D content, Apple has teamed up with a bunch of big companies working in 3D tech to announce an alliance that could change the way 3D projects are made. 

The Alliance for OpenUSD — a group formed by Apple, Pixar, Adobe, Autodesk, and Nvidia — aims to standardize and develop a technology called Universal Scene Description (USD) that was originally made by Pixar. The technology lets developers move their work across a world of 3D creation tools, making it much easier for developers to build 3D experiences for everything from VFX and animation (where it’s already been widely adopted) to gaming, VR, and metaverse applications. 

USD is, essentially, a file format for encoding 3D scenes, but describing it in such simple terms undersells its significance. Creating 3D content for movies and games involves generating, storing, and transmitting large quantities of data — known as “scene description.” Various applications are required in this pipeline for things like modeling, lighting, animation, shading, and rendering, and these typically use their own specialized form of scene description that can’t be edited or even read by other apps.

The USD format solves this problem by enabling 3D tools across the development pipeline to “communicate,” allowing various formats to be compiled together. Creators can use USD to collaborate on the same scene or project simultaneously, and any changes made to part of that environment will be reflected across everything else that references it. 

Apple in particular has a lot to gain from the widespread adoption of OpenUSD. Should creative tools adopt it en masse, it could help developers more easily build 3D apps for the Vision Pro headset, a rumored cheaper version down the line, and, should they ever come out, some kind of AR glasses. And if Apple has the apps that convince people to buy its headsets over others — say, those from Meta — it could become a leader in yet another big technology. (Though you probably won’t hear Apple use the word “metaverse” to describe anything you can experience with its products.)

USD isn’t the only “standardized” format available, nor is it a new development. It is, however, the frontrunner for actually being adopted as the industry standard over rivaling formats like Alembic because it can be scaled for incredibly complex development pipelines. Many industry standard applications like Autodesk’s Maya, Adobe’s Substance 3D, and Nvidia’s Omniverse platform already support USD, and these companies have thrown their weight behind its standardization.

Apple started experimenting with the USD format years ago, back when its adoption was more focused on the film industry. At WWDC 2018, the company announced a new AR file format called USDZ for AR content in iOS apps like Messages and Safari. At the time, Apple was working with Pixar, Adobe, and Autodesk on the new format. (And Apple and Pixar go way back thanks in part to Steve Jobs.) Now, as Apple tries to find ways to support expansive 3D content for the Vision Pro, getting more involved with standardizing USD seems like a logical step.

USD itself has been around for a while. Pixar open sourced USD in 2016, but even prior to that, people were optimistic about it becoming the industry standard format in the 3D content creation pipeline. “Proprietary 3D formats [are] still a pretty big problem,” one commenter wrote in a Hacker News thread from nearly a decade ago. “Anything that helps that is good.” By 2022, USD was going more mainstream. In a blog post, Nvidia wrote that it believed “USD should serve as the HTML of the metaverse: the declarative specification of the contents of a web site.”

The new alliance could make some kind of HTML-level adoption more feasible. The group says it will “develop written specifications detailing the features of OpenUSD,” which it says will “enable greater compatibility and wider adoption, integration, and implementation, and allows inclusion by other standards bodies into their specifications.” 

There are some absences in Tuesday’s announcement, however: Meta, which makes Quest headsets, and the triumvirate of Google, Samsung, and Qualcomm, who are working on a new XR platform.

Meta is a particularly notable missing party. The company is burning cash to make the metaverse happen, and its support would go a long way toward standardization around 3D asset creation. Plus, since Meta CEO Mark Zuckerberg has signaled that he wants the company to represent the Android of the metaverse — “the open ecosystem for the next generation of computing around virtual and augmented reality in the metaverse more broadly,” he told our colleague Alex Heath last year — you’d think Meta would be jumping up and down to be a part of this. Meta didn’t reply to our request for comment.

Perhaps that all comes down to the fact that Apple is such a big part of the alliance. Meta is already pushing for the adoption of standard formats for building virtual worlds and metaverses. The Metaverse Standards Forum — a group formed last year by Meta, Maxon, Epic Games, Autodesk, Nvidia, Adobe, and several other companies with a vested interest in metaverse technology — is even already reviewing USD’s current capabilities (pdf).

There seems to be momentum behind what the Alliance for OpenUSD is working on. Epic Games, which makes the popular Unreal Engine suite of development tools, is “excited to witness the [alliance’s] formation,” according to Marc Petit, the company’s VP of the Unreal Engine ecosystem. Unity is on board, too, according to Allan Poore, SVP of Unity Wētā Tools.

The future does seem bright for USD. And if it does win the race for 3D standardization, it’ll be bright across every industry it can be applied to.