Qualcomm inevitably focuses on AI with new flagship Snapdragon SoC

While the return to designing its own CPU cores, rather than Arm's designs, was the headline news, the ability to process as much AI work locally as possible was a major sub-theme. The new Oryon CPU certainly does seem to raise the bar a fair bit, manufactured in the latest 3nm process node, it offers significant (45%) performance increases over its predecessor. Meanwhile the latest iteration of the Hexagon AI NPU (neural processing unit) also has lots of juicy metrics.

"Today, our second generation of the Qualcomm Oryon CPU debuts in our flagship mobile platform – it's a major leap forward and we expect consumers to be thrilled with the new experiences enabled by our CPU technology,” said Chris Patrick, Qualcomm’s GM of mobile handsets. "With leading CPU, GPU and NPU capabilities, the Snapdragon 8 Elite delivers dramatic performance enhancements and power efficiency^.

“In addition, it revolutionizes mobile experiences by offering personalized, multi-modal generative AI directly on the device enabling the understanding of speech, context, and images to enhance everything from productivity to creativity tasks while prioritizing user privacy."

The subtext of that comment about speech is a bet on the latest generative AI technology finally making the voice (as opposed to touch) interface a mainstream proposition. Leaving aside the cultural friction that comes with talking to a machine as you would a human being, ubiquitous voice UI seems inevitable, especially in physically constrained environments such as the car. We will always need mobile screens for video, but communication and a lot of internet activity could be conducted much more seamlessly by using voice and leaving your phone in your pocket.

"However, voice adds a novel intolerance to latency which in my opinion means that the LLM that powers the voice must run locally on the device or the conversation will be stilted and unrealistic,” notes RFM. “This is one reason (other than economics) why inference at the edge is important meaning that a migration to voice as the MMI plays directly to Qualcomm’s strength as an edge processor company.”

As Nvidia shareholders will be gleefully aware, genAI requires a hell of a lot of processing. Granted, Nvidia GPUs are mainly being used for large language model training but the inference part, in which the pre-trained LLM receives a natural language request and responds to it, is a separate bit of processing. If people are expected to converse with their devices, it needs to be done at a natural pace and cadence, which is best done without the exchange travelling to the cloud and back.

"This latest innovation marks a transformative leap in performance, heralded by a new name and striking visual design,” wrote independent Analyst Paolo Pescatore on his Substack. “Its unprecedented performance merits a premium variant, thus introducing the 8 Elite. This is truly cutting-edge technology, which boasts a vast array of IPs and key architectural upgrades. Make no mistake, this elevates it beyond a mere SOC with major enhancements in image processing and overall functionality.”

Here's the full keynote session if you’ve got two-and-a-half hours to kill.