AMD new Instinct MI300X GPU and MI300A APU: A step change in generative AI
The landscape of generative artificial intelligence (AI) is a testament to the blistering pace of technological advancement, and AMD's Chair and CEO, Dr Lisa Su, speaking at the company’s Advancing AI event keynote shone a light on this rapid progression. With the introduction of its new flagship AI GPU accelerator, the Instinct MI300X, and the new APU MI300A, AMD positions itself at the forefront of this wave.
"Reviewing the last four months, the adoption of generative AI in various sectors has evidently accelerated, underscoring a burgeoning demand that's gaining momentum," Dr Su explained.
While most of its mainstream customers are familiar with its Ryzen and Radeon processors, AMD is also coming on strong in the global race to develop microchips and semiconductors that power AI technologies, and the MI300X is described as the world's highest-performing accelerator for generative AI while the MI300A is the world’s first data centre APU, meaning that it combines both a CPU and GPU in the same package.
"To enable AI infrastructure, it certainly starts with the cloud but extends into the enterprise," Dr Su added, outlining the expansive vision AMD holds for AI's integration into every aspect of our digital infrastructure.
Strategic priorities in the AI era
AMD's strategy revolves around three key imperatives – performance, platform accessibility, and partnerships. "Firstly, we aim to offer an extensive portfolio of high-performance, energy-efficient GPUs, CPUs, and adaptive computing solutions for AI training and inference," Dr Su emphasised, highlighting the need for a diverse suite of technologies to cater to the varied demands of AI workloads.
But it’s not just about hardware. AMD is making strides in ensuring their software platform is open, proven, and developer-friendly. "It's really about expanding our open proven and being very developer friendly in our software platform," she said, stressing the importance of ease of use and compatibility with leading AI frameworks, libraries, and models.
The spotlight on partnerships was particularly bright, with AMD acknowledging the role of collaborations in driving AI forward. The company enjoys a particularly strong relationship with Microsoft, described by Microsoft's CTO Kevin Scott as a "special partnership" that has evolved significantly over the years.
"Our collaboration with AMD spans many years and has seen a gamut of developments, from personal computing advancements to bespoke silicon projects for Xbox, and now to the sophisticated realm of high-performance computing and AI compute," Scott said.
His commentary navigated through the past, present, and future of the Microsoft-AMD alliance with a focus on the substantive, real-world impact of their joint efforts. "What we are witnessing is not just a fleeting surge in technology but a sustained and strategic expansion of capabilities that define our era."
He further elaborated on the integration of AMD's technologies within Microsoft's infrastructure. "The breakthroughs we've achieved in AI compute, in particular, have been a result of our concerted efforts towards full-system optimisation – not just at the silicon level, but across the entire stack," Scott added. This holistic approach, according to Scott, is what underpins their ability to deliver generative AI applications like ChatGPT to the broader public.
Acknowledging the significance of the newly launched MI300X, Scott said: "This level of compute performance, coupled with the efficiencies it brings to AI workloads, is a testament to what our partnership can achieve. It's about crafting a foundation robust enough to propel us into the next wave of AI innovation."
Unpacking the Instinct MI300X
Technically, the MI300X is an extraordinary achievement. "It's the most advanced product we've ever built, and it is the most advanced AI accelerator in the industry," Dr Su claims. It is built on the CDNA 3 architecture, optimised for both performance and power efficiency: it houses 153 billion transistors and employs a multi-chiplet design, featuring a mixed of 12 5nm and 6nm chiplets that leverage the most advanced packaging techniques in the industry.
The specifications of the MI300x are indeed compelling – it integrates four I/O die at its base, each festooned with 256MB of Infinity Cache. It also supports the latest in next-gen I/O, including 128 lanes of PCIe Gen 5 interface and AMD's fourth-generation Infinity Fabric, which allows for a robust interconnect between multiple MI300x units, ensuring a throughput of 896GB/s. On top of this, the MI300x stacks eight CDNA 3 accelerator chiplets, which deliver computing might at the rate of 1.3 petaflops for FP16 operations and 2.6 petaflops for FPGA performance. This ensemble is linked together with dense through-silicon vias (TSVs), enabling bandwidths reaching up to 17TB/s.
While the raw numbers are impressive, Dr Su also grounded these in a real-world context, indicating how they translate into substantial benefits for AI applications. For instance, this formidable array of specifications means that the MI300x is well-equipped to handle the demands of extensive AI model training sessions and complex inference tasks, significantly enhancing the speed and efficiency with which AI systems can operate.
Unpacking the Instinct MI300A
The MI300A is AMD's strategic answer to the nuanced demands of high-performance computing (HPC) and AI workloads, employing a design philosophy that mirrors its sibling, the MI300X, with a few calculated adjustments. At its core, the MI300A incorporates three 5nm core compute dies (CCDs), each hosting eight Zen 4 CPU cores identical to those found in the EPYC and Ryzen series, thereby replacing two of the MI300X’s XCD GPU chiplets.
This alteration equips the MI300A with an assembly of 24 multi-threaded CPU cores and 228 CDNA 3 compute units, arrayed across six XCD GPU chiplets. Like the MI300X, the MI300A utilises a hybrid bonding technique to connect all compute chiplets to four I/O dies. This approach yields a remarkable leap in bandwidth, latency, and energy efficiency, surpassing the capabilities of conventional chip-packaging methods.
In a move to align with its intended application spectrum, the MI300A scales down its memory to eight 8Hi HBM3 stacks, reducing the total capacity from the MI300X's 192GB to 128GB. Yet, it maintains the memory bandwidth of 5.3TB/s. AMD clarifies that this reduction is not a compromise born of power or thermal constraints but a deliberate choice to optimise the chip's architecture for its target environments. This tailored configuration results in the MI300A boasting 1.6 times the memory capacity and throughput of NVIDIA's H100 SXM GPU.
Industry impact and considerations
Dr Su also addressed the broader implications of the MI300X and MI300A's launch. It's clear that AMD envisions both as significant contributors to the AI field – not through sheer might, but through its potential to streamline AI operations, make AI systems more energy-efficient, and enable the handling of increasingly intricate models.
In a market that's as competitive as it is dynamic, AMD's approach with the MI300X and MI300A appears to be one of careful calibration. "Our goal is to facilitate as seamless an adoption as possible," Dr Su said, reinforcing AMD's commitment to accessibility and industry standards.
With the AI revolution well underway, AMD's new AI GPU and APU is not just keeping pace; it’s setting the tempo.
< PrevPage 1 of 1 – AMD new Instinct MI300X GPU and MI300A APU: A step change in generative AIPage 1 of 1 – AMD new Instinct MI300X GPU and MI300A APU: A step change in generative AIPage 1 of 1 Page 1 of 1 – AMD new Instinct MI300X GPU and MI300A APU: A step change in generative AINext >