Microsoft is using its new Surface launch and this week’s Build developer conference as a platform to launch its new “Copilot+” PC initiative, which comes with specific hardware requirements that systems will need to meet to be eligible. Copilot+ PCs will be able to handle some AI-accelerated workloads like chatbots and image generation locally instead of relying on the cloud, but new hardware will generally be required to run these workloads quickly and power efficiently.

At a minimum, systems will need 16GB of RAM and 256GB of storage, to accommodate both the memory requirements and the on-disk storage requirements needed for things like large language models (LLMs; even so-called “small language models” like Microsoft’s Phi-3, still use several billion parameters). Microsoft says that all of the Snapdragon X Plus and Elite-powered PCs being announced today will come with the Copilot+ features pre-installed, and that they’ll begin shipping on June 18th.

But the biggest new requirement, and the blocker for virtually every Windows PC in use today, will be for an integrated neural processing unit, or NPU. Microsoft requires an NPU with performance rated at 40 trillion operations per second (TOPS), a high-level performance figure that Microsoft, Qualcomm, Apple, and others use for NPU performance comparisons. Right now, that requirement can only be met by a single chip in the Windows PC ecosystem, one that isn’t even quite available yet: Qualcomm’s Snapdragon X Elite and X Plus, launching in the new Surface and a number of PCs from the likes of Dell, Lenovo, HP, Asus, Acer, and other major PC OEMs in the next couple of months. All of those chips have NPUs capable of 45 TOPS, just a shade more than Microsoft’s minimum requirement.

The problem for Intel and AMD is that none of their current-generation chips come anywhere close to meeting this requirement, even the ones that have NPUs in the first place—Intel’s Meteor Lake-based Core Ultra NPUs top out (ha) at 10 TOPS, and a handful of AMD’s Ryzen 7000 and Ryzen 8000 desktop and laptop processors with NPUs offer between 12 and 16 TOPS. This is particularly embarrassing for Intel, which for decades has been so synonymous with Windows PCs that their portmanteau has a fairly thorough Wikipedia entry.

Both Intel and AMD have products on their roadmap that will boost NPU performance to the level that Microsoft is asking for with Copilot+ PCs, but none of the hundreds of millions of active x86-based Windows PCs, up to and including models you can buy today, will qualify for the label.

NPUs that meet Microsoft’s Copilot+ PC requirement will be used to power (among other things) a group of features that Microsoft is calling “Recall,” a collection of AI features that will try to make helpful suggestions by keeping track of everything you’ve done on your PC, including attending meetings, opening files, and doing web searches. Perhaps aware of the fact that this sounds like a privacy and security nightmare, Microsoft EVP Yusuf Medhi says that the NPU will handle all of this processing locally, allowing user data to stay “private, local, and secure on just the device.” On a Copilot+ PC with the minimum 256GB SSD, Microsoft says Recall will take up about 25GB of disk space and store around three months’ worth of events.

Intel and AMD’s CPUs and GPUs can also contribute to the total number of TOPS that any given system can perform. But NPUs are specialized hardware, made to run AI workloads more quickly and while using less power. You could use a CPU to render 3D games or other workloads, but you typically wouldn’t; you’d leave that work to the GPU. Dedicated video encoding and decoding hardware allows slow, dirt-cheap ARM processors in Roku boxes or Fire sticks to play back 4K video streams. The NPU fills a similar role for AI workloads—it’s not dishonest to put out TOPS figures that bundle the CPU, GPU, and NPU together, but achieving that level of performance using a CPU and GPU means using a lot more power and reducing battery life.

Microsoft sees Copilot+ PCs as “premium” devices that exist above older Windows PCs or Intel, AMD, and Qualcomm-powered systems with lower-performance NPUs. Windows 11’s core system requirements aren’t changing, and all of the cloud-powered AI features (and existing NPU-accelerated workloads like Windows Studio Effects for webcams) will continue to work on existing PCs as they currently do.

There’s risk for Microsoft in tying the Copilot+ PC branding to the Arm version of Windows in all of these launch devices. Though compatibility has gotten dramatically better in just the last couple of years, and though Windows 11 24H2’s new Prism emulation layer improves app translation speeds by as much as 20 percent compared to Windows 11 23H2 running on the same hardware, there are still some compatibility gaps where Arm-based Windows PCs won’t be able to do the same things that an Intel or AMD PC can. Hardware that requires specialized drivers is one sticking point; games that require the installation of anti-cheat software are another.

Copilot+ PCs could be a new phase in the PC’s development, one that we eventually look back on as an inflection point. They could also be Windows RT all over again, a trend-chasing flash-in-the-pan that comes and goes without making a dent in the Wintel monolith.

https://arstechnica.com/?p=2025348