HBM memory and AI processors: Happy together
High-bandwidth memory (HBM) is again in the limelight. At GTC 2025, held in San Jose, California, from 17 to 21 March, SK hynix displayed its 12-high HBM3E devices for artificial intelligence (AI) servers. The Korean memory maker also showcased a model of its 12-high HBM4, currently under development, claiming that it’s now completing the preparatory works for large-scale production of the 12-high HBM4 in the second half of 2025.
Micron, another leading memory supplier, is signaling strong demand for its HBM chips in AI and high-performance computing (HPC) applications. Micron’s chief business officer, Sumit Sadana, told Reuters that all of Micron’s HBM chips are sold out for the calendar year 2025.
Figure 1 HBM is creating a new “near memory” space between cache and main memory. Source: IDTechEx
HBM—essentially a 3D structure of vertically stacked DRAM dies on top of a logic die—relies on advanced packaging technologies like through silicon vias (TSVs) while using a silicon interposer for interconnection with the processor. It’s proving highly suitable in parallel compute environments such as HPC and AI workloads.
That’s because it can handle multiple memory requests simultaneously from various cores in GPUs and AI accelerators to facilitate parallel workload processing. In fact, HBM has become the main venue for overcoming memory bottlenecks in data-intensive HPC and AI workloads. Otherwise, these memory bottlenecks lead to underutilization of AI processors.
What’s also pivotal about HBM devices is their continued development to improve AI accelerator performance. For instance, the current generation HBM3E devices use thermal compression with micro-bumps and underfills to stack DRAM dies. Next, HBM makers like Micron, Samsung and SK hynix are transitioning toward HBM4 devices, which employ advanced packaging technologies such as copper-copper hybrid bonding to increase input/outputs, lower power consumption, improve heat dissipation, and reduce electrode dimensions.
Market research firm IDTechEx’s report “Hardware for HPC, Data Centers, and AI 2025-2035: Technologies, Markets, Forecasts” assesses the key developments and trends in HBM devices serving AI and HPC workloads. It also projects that compared to 2024, the unit sales of HBM are forecast to increase 15-fold by 2035.
Figure 2 The booming AI and HPC hardware is forecast to increase HBM sales 15-fold by 2035. Source: IDTechEx
HBM was a prominent technology highlight in 2024 for its ability to overcome the memory wall for AI processors. With the emergence of HBM4 memory devices, that trend is likely to continue in 2025 and beyond.
Related Content
- HBM Flourishes, But HMC Lives
- AI boom and the politics of HBM memory chips
- Memory Bottlenecks: Overcoming a Common AI Problem
- HBM memory chips: The unsung hero of the AI revolution
- High-bandwidth memory (HBM) options for demanding compute
The post HBM memory and AI processors: Happy together appeared first on EDN.
