AI Phones: Qualcomm Gets It
Nuvia, Oryon for Smartphones, Qualcomm's AI Phone Strategy
Qualcomm shared product updates at its Snapdragon Summit in Maui last week, but Arm interrupted the party. That was unfortunate. This article will give Qualcomm the credit it deserves; when it comes to AI phones, Qualcomm gets it.
Last week’s smartphone-related announcements demonstrated a sound business strategy backed by impressive performance gains. Should we be surprised? Not at all. To understand, let’s wind back the clock to 2019.
Nuvia
Nuvia was founded by industry veterans in 2019 and quickly raised a $53M series A to bring performant Arm chips to the data center.
John Bruno, Manu Gulati and Gerard Williams III founded Nuvia, and combined have been granted more than 100 patents related to system engineering and silicon design. Besides Apple and Google, the chipmakers previously held engineering roles at Arm, Broadcom and AMD.
The nine-month-old Santa Clara-based startup’s mission is to “reimagine silicon design to deliver industry-leading performance and energy efficiency for the data center.”
By 2020, Nuvia raised another $240M to make their Arm-based data center SoC come to life. Take special notice of their custom Arm approach:
Nuvia isn’t licensing a predesigned Arm core as part of its CPU; instead, it is designing its own core under a custom Arm license. “We’re designing our own core from the ground up,” Williams said. “Our secret sauce is really physical design capabilities and microarchitecture. And that’s where you see the differentiation coming into play for almost anybody that builds a processor.”
Nuvia focused on performance efficiency and obtained an Arm Architecture License Agreement (ALA), allowing them to develop a custom Arm CPU similar to Apple's approach.
In a blog post, the company said it is aiming for performance-per-watt leadership with its Phoenix CPU core. “We are going to look very much like a mobile core in terms of size and power, but performance is going to be substantially higher,” Williams said.
Those are not-so-subtle hints that Nuvia felt it could compete in mobile, too.
Might any large companies desire a power-efficient custom Arm core that could also be deployed in smartphones and laptops?
You guessed it. In 2021, Qualcomm announced it would acquire Nuvia for $1.2B.
NUVIA comprises a proven world-class CPU and technology design team, with industry-leading expertise in high performance processors, Systems on a Chip (SoC) and power management for compute-intensive devices and applications. The addition of NUVIA CPUs to Qualcomm Technologies’ already leading mobile graphics processing unit (GPU), AI engine, DSP and dedicated multimedia accelerators will further extend the leadership of Qualcomm Snapdragon platforms, and positions Snapdragon as the preferred platform for the future of connected computing.
NUVIA CPUs are expected to be integrated across Qualcomm Technologies’ broad portfolio of products, powering flagship smartphones, next-generation laptops, and digital cockpits, as well as Advanced Driver Assistance Systems, extended reality and infrastructure networking solutions.
From the outset, Qualcomm was transparent about its strategy — Nuvia was seen as the missing link to deliver leadership performance and power efficiency across smartphones, laptops, automobiles, XR, and more.
Two years later, led by the Nuvia team, Qualcomm unveiled the Oryon CPU.
Who is Qualcomm Oryon’s fearless leader? Nuvia’s CEO Gerard Williams.
Qualcomm CEO Cristiano Amon: The Oryon CPU is the new leader in mobile computing. Qualcomm designed it from the ground up with one thing in mind – an unprecedented level of performance at an extremely low power. That’s what we do.
This has been an incredible journey. Please give a warm round of applause to the leader who designed the CPU with his incredible team, Gerard Williams.
Oryon’s first deployment was in Windows AI PCs. From Ben Bajarin,
Qualcomm has finally shared extensive details of their Oryon custom architecture purpose-built to take Arm-based Windows PCs to the next level. This is a result of years of work via their acquisition of Nuvia, and represents a significant moment for Qualcomm as they return to a proprietary custom Arm architecture.
These Oryon CPUs led to a massive scoop for Qualcomm. In May, Microsoft announced that Qualcomm’s Snapdragon X Elite was the exclusive silicon powering Microsoft Copilot+ at launch.
The first Copilot+ PCs will launch with both the Snapdragon® X Elite and Snapdragon® X Plus processors and feature leading performance per watt thanks to the custom Qualcomm Oryon™ CPU, which delivers unrivaled performance and battery efficiency.
Yes, you read that right. Not Intel. Not AMD. Microsoft’s big bet on generative AI PCs launched with Windows on Arm, powered by Qualcomm.
Unfortunately, Copilot+ stumbled out of the gate, and Qualcomm’s PC sales are likely disappointing. As previously discussed,
Qualcomm made a big splash back in May as Microsoft's exclusive Copilot+ PC supplier, but the Copilot+ excitement quickly faded when Recall was rescinded over security concerns. Microsoft finally released Recall on October 1st, but that’s too little, too late for Qualcomm; Copilot+ is coming to Intel and AMD in November, just in time for holiday shopping.
Nevertheless, Qualcomm made a strong entrance into the PC market and the Oryon CPU played a leading role.
This brings us to the present, where Qualcomm introduced the Oryon CPU on a smartphone last week with the introduction of the Snapdragon 8 Elite.
I wasn’t in Maui (maybe next year?) but eagerly watched the recording.
Chris Patrick, Qualcomm SVP and GM for Mobile Handset, led with a bold claim: Oryon offers desktop performance within a smartphone’s power envelope and form factor.
Chris Patrick: We’ve arrived at a significant moment for the industry. We have created a desktop-class CPU with mobile-level power efficiency. We built the first and only modern Android SoC that’s built from the ground up with a custom CPU.
Manju Varma, Director of Product Management, reiterated that Nuvia’s CPU is the final piece of the puzzle.
Manju Varma: We build our IPs from scratch and Qualcomm Oryon is that final piece of the puzzle.
This is our second-generation CPU architecture. We made significant design changes and performance improvements since our first generation Qualcomm Oryon CPU. This is a brand-new microarchitecture that is designed for mobile.
Proliferation of Oryon across form factors — this was the plan all along.
This second-generation CPU is at the heart of the Snapdragon 8 Elite, with incredible performance gains. Qualcomm claims performance and power efficiency gains of more than 40% across the CPU, GPU, and NPU. The entire SoC’s power consumption was reduced by ~30%, adding many hours of additional battery life for consumers.
Future Proof
Why does the Oryon-based Snapdragon 8 Elite launch matter for customers and Qualcomm?
First, the across-the-board CPU, GPU, and NPU improvements improve the customer experience for today's and tomorrow’s use cases. Qualcomm highlighted faster app launches, a snappier browser, improved video rendering and battery life for gaming, and enhanced local generative AI support.
The showcase example was the Asus ROG 9 phone (ROG stands for Republic of Gamers). With the 8 Elite, ROG achieved a 30% power efficiency improvement in the most demanding games. Talk about Qualcomm doing right by its OEM customers! The massive 30% battery life improvement will help Asus retain and grow customers and likely encourage early upgrades. This was a solid focus on present use cases.
Simultaneously, Qualcomm demonstrated a sound strategic approach to generative AI: future-proof, but don’t oversell too early.
Don’t get me wrong — Qualcomm definitely talked about on-device generative AI, and they have a lot to talk about.
They shared that the NPU saw up to 100% improvement in tokens/sec to 70+ tokens per second. That’s a useful amount of tokens/sec and suggests architectural and software improvements beyond the bump from TSMC 4nm to 3nm. On the other hand, was that on a helpful LLM or some tiny toy LLM? Speed only matters if the LLM is intelligent.
Qualcomm also showed the importance of workload orchestration at the edge; sending the right job to the suitable processor (CPU, GPU, NPU, ISP) is essential to maximize performance efficiency. This heterogenous compute orchestration is no-regrets work and will help Qualcomm differentiate when Gen AI liftoff takes place.
By the way, this orchestration happens on the CPU. Didn’t Jensen say CPUs were dead? Not in smartphones!
Qualcomm emphasized the privacy benefits of running generative AI locally on the phone. After all, mobile phones have access to all sorts of useful personal context for generative AI, including location, contacts, app usage history (YouTube viewing history, etc), health data, and more. This is an example of a strategy credit and aligns with Apple’s approach. Apple has already done the heavy lifting, allowing Qualcomm to draft behind them. Frankly, all smartphone makers should cash in on this credit. What’s Sam doing with your data? Don’t give it to him. Cloud AI bad! Local AI good.
The future of generative AI is coming to mobile; it’s just not quite here. We’ll see a proliferation of useful generative AI use cases sooner on mobile than on the laptop, from simple voice commands (the Siri we always wanted) to new apps that leverage multimodal inputs and outputs (touch, text, audio, video).
Qualcomm is appropriately investing in preparing for generative AI liftoff; better to be a bit early and tempered than late. This approach differs from Intel’s AI PC strategy and Microsoft’s Copilot+ strategy, where the marketing hype was turned to 11 before the software and use cases were ready. Yeah, Apple might have been a bit early with Apple Intelligence too. All the better for Qualcomm as they draft in Apple’s wake.
And hey, in the meantime, why not let Apple fund market development too?