Introduction, AI, Imaging
Qualcomm today (25 October 2023) officially unveiled the chipset that will power our 2024 premium flagship Android smartphones: the Snapdragon 8 Gen 3 mobile platform (SD8G3).
According to Qualcomm, the 4nm-by-TSMC chipset is made such that generative artificial intelligence (AI) tasks and features are possible with 2024 premium flagship Android phones. Qualcomm is extremely confident about its new processor, touting it as “the titan of on-device intelligence”.
10 billion parameters, all on-device
To run AI efficiently, a chipset would need libraries of information it can tap into quickly.
Qualcomm said that its AI Engine now supports multi-modal generative AI models, which include popular large language models (LLMs), language vision models (LVM), and transformer network-based automatic speech recognition (ASR), totalling up to 10 billion parameters, all on-device. Of the various models, Qualcomm also has Meta’s Llama 2 LLM on board.
In English, that means Qualcomm squeezed multiple libraries of machine-learning information. The info covers decision-making, visuals, and speech recognition to make the processor able to give up to 10 billion “solutions” without needing to look beyond the chip (and that’s a gross understatement).
Besides having the volume, the brand also upped its AI speed and efficiency. Qualcomm’s SD8G3 neural processing unit (NPU), called Qualcomm Hexagon NPU, is now up to 98% faster than its predecessor, with 40% better performance per watt.
Qualcomm claims up to 3.5x AI performance increase on the Qualcomm Sensing Hub (which identifies and processes streams of data), on top of a maximum of 20 tokens per second (which makes your AI voice assistant more personalised and faster than ever). The brand also claimed generative AI for images taking only 1 second to complete, versus the conventional 15-second processing time.
For folks who work in AI, you’ll also be pleased to know that the chipset supports INT4 precision. Qualcomm AI Stack Models now come with 20 models at launch (all on HuggingFace).
Better, smarter photos and videos
Noprizes for guessing correctly — yes, there’s AI working on Qualcomm’s imaging processing deck, called Spectra ISP, on the SD8G3 too.
The chipset supports up to three 18-bit Cognitive ISPs, has an engine for Visual Analytics 4.0, has hardware acceleration for iToF (indirect time-of-flight) depth sensors, and 12 layers of real-time semantic segmentation for photo and video processing.
This means that your SD8G3-powered phone can technically perform the following AI-powered features and tasks, if the phone manufacturer wishes so:
Video Object Eraser (by Arcsoft), removing unwanted people from your videosDual always-sensing cameras for front and back cameras (for QR code, face unlock etc.)Zoom Anyplace (if using Samsung’s 200MP image sensor optimised for Snapdragon), which lets you capture multiple videos, and do object tracking, in 2x or 4x zoom, all while recording in 4K resolutionVloggers View, which lets you livestream both selfie camera and rear camera video at the same timePhoto Expansion, which extends photographs beyond what the image sensor captured using AITruepic photo capture with C2PA standard, which proves the photos you took are real and not AI-made
All this is on top of old-but-still-awesome features, like triple HDR video capture, and support for many HDR image sensors (DCG, QDOL, Less Blanking, Multi-frame).
< PrevPage 1 of 2 – Introduction, AI, ImagingPage 2 of 2 – CPU, GPU, Connectivity, Audio, phones with SD8G3Page 1 of 2 – Introduction, AI, ImagingPage 1 of 2 Page 1 of 2 – Introduction, AI, ImagingPage 2 of 2 – CPU, GPU, Connectivity, Audio, phones with SD8G3Next >