HBM memory chips: The unsung hero of the AI revolution

Memory chips like DRAMs, long subject to cyclical trends, are now eying a more stable and steady market: artificial intelligence (AI). Take the case of SK hynix, the world’s second-largest supplier of memory chips. According to its chief financial officer, Kim Woo-hyun, the company is ready to grow into a total AI memory provider by leading changes and presenting customized solutions.

The South Korean chipmaker has been successfully pairing its high-bandwidth memory (HBM) devices with Nvidia’s H100 graphics processing units (GPUs) and others for processing vast amounts of data in generative AI. Large language models (LLMs) like ChatGPT increasingly demand high-performance memory chips to enable generative AI models to store details from past conversations and user preferences to generate human-like responses.

Figure 1 SK hynix is consolidating its HBM capabilities to stay ahead of the curve in AI memory space.

In fact, AI companies are complaining that they can’t get enough memory chips. OpenAI CEO Sam Altman has recently visited South Korea, where he met senior executives from SK hynix and Samsung, the world’s largest memory chip suppliers followed by Micron of the United States. OpenAI’s ChatGPT technology has been vital in spurring demand for processors and memory chips running AI applications.

SK hynix’s HBM edge

SK hynix’s lucky break in the AI realm came when it surpassed Samsung by launching the first HBM device in 2015 and gained a massive head start in serving GPUs for high-speed computing applications like gaming cards. HBM vertically interconnects multiple DRAM chips to dramatically increase data processing speed compared with earlier DRAM products.

Not surprisingly, therefore, these memory devices have been widely used to power generative AI devices operating on high-performance computing systems. Case in point: SK hynix’s sales of HBM3 chips have increased by more than fivefold in 2023 compared to a year earlier. A Digitimes report claims that Nvidia has paid SK hynix and Micron advanced sums ranging from $540 million to $770 million to secure the supply of HBM memory chips for its GPU offerings.

SK hynix plans to proceed with the mass production of the next version of these memory devices—HBM3E—while also carrying out the development of next-generation memory chips called HBM4. According to reports published in the Korean media, Nvidia plans to pair its H200 and B100 GPUs with six and eight HBM3E modules, respectively. HBM3E, which significantly improves speed compared to HBM3, can process data up to 1.15 terabytes per second.

Figure 2 SK hynix is expected to begin mass production of HBM3E in the first half of 2024.

The Korean memory supplier calls HBM3E an AI memory product while claiming technological leadership in this space. While both Samsung and Micron are known to have their HBM3E devices ready and in the certification process at AI powerhouses like Nvidia, SK hynix seems to be a step ahead of its memory rivals. Take, for example, HBM4, currently under development at SK hynix; it’s expected to be ready for commercial launch in 2025.

What’s especially notable about HBM4 is its ability to stack memory directly on processors, eliminating interposers altogether. Currently, HBM stacks integrate 8, 12, or 16 memory devices next to CPUs or GPUs, and these memory devices are connected to these processors using an interface. Integrating memory directly onto processors will change how chips are designed and fabricated.

An AI memory company

Industry analysts also see SK hynix as the key beneficiary of the AI-centric memory upcycle because it’s a pure-play memory company, unlike its archrival Samsung. It’s worth noting that Samsung is also heavily investing in AI research and development to bolster its memory offerings.

AI does require a lot of memory and it’s no surprise that South Korea, housing the top two memory suppliers, aspires to become an AI powerhouse. SK hynix, on its part, has already demonstrated its relevance in designs for AI servers and on-device AI adoption.

While talking about memory’s crucial role in generative AI at CES 2024 in Las Vegas, its CEO Kwak Noh-Jung vowed to double the company’s market cap in three years. That’s why it’s now seeking to become a total AI memory provider while seeking a fast turnaround with high-value HBM products.

VIDEO AD

–><!–

div-gpt-ad-inread

–>

<!–
googletag.cmd.push(function() { googletag.display(‘div-gpt-ad-native’); });
–>

The post HBM memory chips: The unsung hero of the AI revolution appeared first on EDN.

HBM memory chips: The unsung hero of the AI revolution

Become a member

Become a member

Become a subscriber

Become a sponsor