Edge AI in a DRAM shortage: Doing more with less

Memory is having a difficult year. As manufacturers prioritize DDR5 and high-bandwidth memory (HBM) for data centers and large-scale AI workloads, availability has tightened and costs have risen sharply: up to 3–4x compared to Q3 2025 levels and market signals suggest the peak has not yet been reached.

Even hyperscalers—typically at the frontline—are reportedly receiving only about 70% of their allocated volumes, and analysts expect tight conditions to persist well into 2026 and possibly even 2027.

The strain isn’t evenly distributed, with the steepest price hikes and longest lead times concentrated in higher-capacity modules. Those components sit directly in the path of cloud infrastructure demand, and their pricing reflects it. On the other hand, lower-capacity modules (1-2 GB) have remained accessible and far more stable.

This trend is now influencing how teams think about system design. AI workloads built around large memory footprints now run into procurement challenges; systems engineered to operate within modest memory baselines avoid both the price spikes and the uncertainty. The outcome is important: in a shortage, architecture built for efficiency gives teams more strategic freedom compared to architectures built for abundance.

The most effective solution: DRAM-less AI accelerator

In a constrained memory market, the most robust solution is also the simplest: remove the dependency on external DRAM entirely. Take the case of Hailo-8 and Hailo-8L AI accelerators. By keeping the full inference pipeline on-chip, Hailo-8/8L eliminate the most expensive and supply-constrained component in the system.

In practical terms, avoiding DRAM can reduce bill of materials by up to $100 per device, while also improving power efficiency, latency, and system reliability. Though not every AI application can avoid DRAM.

Generative AI workloads inherently require more memory, and systems that run them will continue to rely on external DRAM. But even in this case, memory constraints strongly favor moving inference closer to the edge.

Running generative AI on the edge allows teams to work with smaller, domain-specific models rather than large, general-purpose ones designed for the cloud. Smaller models translate directly into smaller DRAM requirements, reducing cost, easing procurement, and improving power efficiency. This is where edge-focused accelerators come into play, enabling efficient generative AI inference while keeping memory footprints as lean as possible.

Privacy and latency have long shaped the case for running intelligence on the device. In 2025, another factor cemented it: the expectation that generative AI simply be there. Users now rely on transcription, summarization, audio cleanup, translation, and basic reasoning often with no tolerance for startup delays or network dependency.

Recent cloud outages from AWS, Azure and Cloudflare underscored how fragile cloud-only assumptions can be. When the networks faced disruptions, everyday features across consumer apps and enterprise workflows failed. Even brief interruptions highlighted how a single infrastructure dependency can take down tools that users now rely on dozens of times a day.

As AI moves deeper into everyday workflows and users expect agentic AI capabilities to be available instantly, a hybrid approach proves far more resilient. Keep frequently used intelligence local, either on the device or in a nearby gateway, while using the cloud for heavier or less frequent tasks. And crucially, when models are small enough to operate within 1-2 GB of memory, that hybrid approach becomes far easier to implement using memory configurations that are still readily sourced.

Small models change the equation

Until recently, generative AI required the memory and compute scale of the cloud. A new class of small language models (SLMs) and compact vision language models (VLMs) now deliver strong instruction following, reliable tool use, and competitive benchmark performance at a fraction of the parameters.

Releases like IBM’s Granite 4.0 Nano line demonstrate how far efficient architectures have come. These models show that some generative AI tasks and applications no longer need massive, expensive system memory—they need well-defined domains, optimized inference paths, and efficient pre- and post-processing.

For hardware teams, this evolution has many practical benefits. Smaller models reduce the “memory tax” that has been baked into AI design for years. When an entire intelligence pipeline can operate in 1-2 GB of DRAM, several constraints loosen simultaneously:

Costs fall as systems avoid the inflated pricing of high-capacity DRAM.
Supply-chain risk drops as lower-capacity memory chips remain easier to procure.
Power consumption improves because smaller models with hardware-assisted offload (NPU or AI accelerator) run cooler and more efficiently.
System reliability increases as local inference keeps essential features online even during network outages.

An AI architecture designed for efficiency rather than abundance fits squarely within the ethos of edge computing. Many high-value agentic AI tasks—summarizing a conversation, describing an image, or translating speech—do not require massive models. In narrow domains, compact models can deliver faster, more private and consistent results because they operate with fewer unknowns.

The path forward

If the DRAM shortage proves anything, it’s that the most resilient AI systems are the ones designed around constraints, not excess. Teams are re-evaluating assumptions about model size, memory baselines, and what “good enough” looks like for common tasks. They’re recognizing that domain-specific intelligence often performs better than brute-force scale—especially in environments that demand consistency, privacy, and low power draw.

Edge AI fits naturally within this moment. Its memory profile lines up with the DRAM capacities that remain accessible, and its deployment model brings stability to the tasks users rely on most. As supply tightness continues, organizations that invest in leaner model design and hybrid deployment strategies will be better positioned to deliver stable, responsive AI without absorbing high memory costs.

Avi Baum is chief technology officer (CTO) and co-founder of Hailo.

Special Section: AI Design

The post Edge AI in a DRAM shortage: Doing more with less appeared first on EDN.

Become a member

Become a member

Become a subscriber

Become a sponsor