A sneak peek at HBM cold war between Samsung and SK hynix

As high-bandwidth memory (HBM) moves from HBM3 to its extended version HBM3e, a fierce competition kicks off between Samsung and SK hynix. Micron, the third largest memory maker, has also tagged along to claim stakes in this memory nirvana that is strategically critical in artificial intelligence (AI) designs.

HBM is a high-value, high-performance memory that vertically interconnects multiple DRAM chips to dramatically increase data processing speed compared to conventional DRAM products. HBM3e is the fifth generation of HBM following HBM, HBM2, HBM2E and HBM3 memory devices.

HBM helps package numerous AI processors and memories in a multi-connected fashion to build a successful AI system that can process a huge amount of data quickly. “HBM memory is very complicated, and the value added is very high,” Jensen Huang, Nvidia co-founder and CEO, said at a media briefing during the GPU Technology Conference (GTC) held in March 2024 at San Jose, California. “We are spending a lot of money on HBM.”

Take Nvidia’s A100 and H100 processors, which commanded 80% of the entire AI processor market in 2023; SK hynix is the sole supplier of HBM3 chips for these GPUs. SK hynix currently dominates the market with a first-mover advantage. It launched the first HBM chip in partnership with AMD in 2014 and the first HBM2 chip in 2015.

Figure 1 SK hynix currently dominates the HBM market with nearly 90% of the market share.

Last month, SK hynix made waves by announcing to start the mass production of the industry’s first HBM3e chip. So, is the HBM market and its intrinsic pairing with AI processors a case of winner-takes-all? Not really. Enter Samsung with a 12-layer HBM3e chip.

Samsung’s HBM surprise

Samsung’s crosstown memory rival SK hynix has been considered the unrivalled HBM champion since it unveiled the first HBM memory chip in 2014. It’s also known as the sole HBM supplier of AI kingpin Nvidia while Samsung has been widely reported to be lagging in HBM3e sample submission and validation.

Then came Nvidia’s four-day annual conference, GTC 2024, where the GPU supplier unveiled its H200 and B100 processors for AI applications. Samsung, known for its quiet determination, once more outpaced its rivals by displaying 12-layer HBM3e chips with 36 GB capacity and 1.28 TB/s bandwidth.

Figure 2 Samsung startled the market by announcing 12-layer HBM3e devices compared to 8-layer HBM3e chips from Micron and SK hynix.

Samsung’s HBM3e chips are currently going through a verification process at Nvidia, and CEO Jensen Huang’s note “Jensen Approved” next to Samsung’s 12-layer HBM3e device on display at GTC 2024 hints that the validation process is a done deal. South Korean media outlet Alpha Biz has reported that Samsung will begin supplying Nvidia with its 12-layer HBM3e chips as early as September 2024.

These HBM3e chips stack 12 DRAMs, each carrying 24-GB capacity, leading to a peak memory bandwidth of 1.28 TB/s, 50% higher than 8-layer HBM3e devices. Samsung also claims its 12-layer HBM3e device maintains the same height as the 8-layer HBM3e while offering 50% more capacity.

It’s important to note that SK hynix began supplying 8-layer HBM3e devices to Nvidia in March 2024 while its 12-layer devices, though displayed at GTC 2024, are reportedly encountering process issues. Likewise, Micron, the world’s third largest manufacturer of memory chips, following Samsung and SK hynix, announced the production of 8-layer HBM3e chips in February 2024.

Micron’s window of opportunity

Micron, seeing the popularity of HBM devices in AI applications, is also catching up with its Korean rivals. Market research firm TrendForce, which valued the HBM market approximately 8.4% of the overall DRAM industry in 2023, projects that this percentage could expand to 20.1% by the end of 2024.

Micron’s first HBM3e product stacks 8 DRAM layers, offering 24 GB capacity and 1.2 TB/s bandwidth. The Boise, Idaho-based memory supplier calls its HBM3e chip “HBM3 Gen2” and claims it consumes 30% less power than rival offerings.

Figure 3 Micron’s HBM3e chip has reportedly been qualified for pairing with Nvidia’s H200 Tensor Core GPU.

Besides technical merits like lower power consumption, market dynamics are helping the U.S. memory chip supplier to catch up with its Korean rivals Samsung and SK hynix. As noted by Anshel Sag, an analyst at Moor Insights & Strategy, SK hynix already having sold out its 2024 inventory could position rivals like Micron as a reliable second source.

It’s worth mentioning that Micron has already qualified as a primary HBM3e supplier for Nvidia’s H200 processors. The shipments of Micron’s 8-layer HBM3e chips are set begin in the second quarter of 2024. And like SK hynix, Micron claims to have sold all its HBM3e inventory for 2024.

HBM a market to watch

The HBM market will continue to remain competitive in 2024 and beyond. While HBM3e is positioning as the new mainstream memory device, both Samsung and SK hynix aim to mass produce HBM4 devices in 2026.

SK hynix is employing hybrid bonding technology to stack 16 layers of DRAMs and achive 48 GB capacity; compared to HBM3e chips, it’s expected to boost bandwidth by 40% and lower power consumption by 70%.

At the International Solid-State Circuits Conference (ISSCC 2024) held in San Francisco on February 18-21, where SK hynix showcased its 16-layer HBM devices, Samsung also demonstrated its HBM4 device boasting a bandwidth of 2 TB/s, a whopping 66% increase from HBM3e. The device also doubled the number of I/Os.

HBM is no longer the unsung hero of the AI revolution, and all eyes are on the uptake of this remarkable memory technology.

<!–

VIDEO AD

–>

<!–

div-gpt-ad-inread

–>

<!–
googletag.cmd.push(function() { googletag.display(‘div-gpt-ad-native’); });
–>

The post A sneak peek at HBM cold war between Samsung and SK hynix appeared first on EDN.

A sneak peek at HBM cold war between Samsung and SK hynix

Become a member

Become a member

Become a subscriber

Become a sponsor