Qualcomm’s Snapdragon 8 Gen 3 moves AI out of the cloud and onto your phone

Qualcomm’s new flagship mobile chipset is an on-device AI fest from top to bottom. The Snapdragon 8 Gen 3 brings generative AI capabilities right to the chipset, vastly speeding up the processing-intensive activities that are usually outsourced to the cloud. It’s one more leap forward in a year that has seen a lot of leaping when it comes to AI, and it will come sooner than later: the first phones with the new chipset are expected to debut in the coming weeks.

The 8 Gen 3 supports a chatbot trained on Meta’s Llama 2, and it can accept text, image, and voice input. It can also talk back to you as well as generate an image or text. The chipset also runs the AI image generator Stable Diffusion on-device, something that Qualcomm demoed earlier this year. This time around, the company says it can generate an image in less than one second — the previous tech generated an image in about 15 seconds, which is still faster than the couple of minutes it can take on even a well-equipped laptop.

All of this is housed in what Qualcomm calls its AI engine, utilizing the company’s Hexagon neural processor. Meanwhile, the Sensing Hub uses OpenAI’s Whisper for speech recognition. The Sensing Hub also supplies the AI engine with information about the user, including location, for more personalized responses. It’ll even relay your favorite activities, age, and “fitness level,” which sounds slightly dystopian if you ask me.

Generative AI also plays a major role in the 8 Gen 3’s new image processing capabilities. It will support generative fill for image expansion so you can zoom out and re-crop photos — again, right on the device. The video features are even wilder: there’s an object eraser for video — just tap and an unwanted subject disappears! — and on-device night mode recording at up to 4K / 30p. Google announced its own Night Sight for video feature coming to the Pixel 8 Pro in the near future, but it will run in the cloud.

There’s also a feature called Vlogger’s View that will layer video from the selfie and rear cameras together into one view. It’s not a picture-in-picture thing — this feature uses improved image segmentation to remove the background from the selfie video to make it look like you’re standing in front of whatever your rear camera sees.

There’s good reason to be concerned about the misuse of these tools, and to that end, Qualcomm is working with a company called Truepic. The tech it’s using is compliant with the Coalition for Content Provenance and Authenticity’s open standard, which is kind of like metadata to guarantee the authenticity of a photo or video. It cryptographically binds authentication to the digital asset — like a photo or video — so that it can’t be tampered with as easily as EXIF data can.

It’s not all AI on the Snapdragon 8 Gen 3, though. Qualcomm claims a modest 10 percent bump in power savings compared to the 8 Gen 2 thanks to efficiency gains from the Kryo CPU and Adreno GPU. Dolby’s HDR photo format is supported, echoing Apple and Google’s push toward high dynamic range image capture in their latest devices. There’s the X75 modem-RF with more support for 5G carrier aggregation (remember 5G?), and the GPU supports improved hardware-based ray tracing for more realistic light reflections in mobile games. The system will support up to 240Hz refresh rates on compatible external displays.

There’s also a new system called Snapdragon Seamless, which is designed to make it easy to pair laptops and phones with peripherals across manufacturers and OS platforms. Qualcomm says this will enable easier switching between devices — like switching to audio to your PC for a video call and back to music playing on your phone, for example.

This seems mainly aimed at getting Android and Windows devices talking to each other more easily, at least at launch. Qualcomm says it’s an open platform that anyone can join, although initially the technology will be largely accessed by Qualcomm’s direct OEM and OS partners. Every garden has to start somewhere — even an open one.