Two new runtime tools to accelerate edge AI deployment

While traditional artificial intelligence (AI) frameworks often struggle in ultra-low-power scenarios, two new edge AI runtime solutions aim to accelerate the deployment of sophisticated AI models in battery-powered devices like wearables, hearables, Internet of Things (IoT) sensors, and industrial monitors.

Ambiq Micro, the company that develops low-power microcontrollers using sub-threshold transistors, has unveiled two new edge AI runtime solutions optimized for its Apollo system-on-chips (SoCs). These developer-centric tools—HeliosRT (runtime) and HeliosAOT (ahead-of-time)—offer deployment options for edge AI across a wide range of applications, spanning from digital health and smart homes to industrial automation.

Figure 1 The new runtime tools allow developers to deploy sophisticated AI models in battery-powered devices. Source: Ambiq

The industry has seen numerous failures in the edge AI space because users dislike it when the battery runs out in an hour. It’s imperative that devices running AI can operate for days, even weeks or months, on battery power.

But what’s edge AI, and what’s causing failures in the edge AI space? Edge AI is anything that’s not running on a server or in the cloud; for instance, AI running on a smartwatch or home monitor. The problem is that AI is power-intensive, and sending data to the cloud over a wireless link is also power-intensive. Moreover, the cloud computing is expensive.

“What we aim is to take the low-power compute and turn it into sophisticated AI,” said Carlos Morales, VP of AI at Ambiq. “Every model that we create must go through runtime, which is firmware that runs on a device to take the model and execute it.”

LiteRT and HeliosAOT tools

LiteRT, formerly known as TensorFlow Lite for microcontrollers, is a firmware version for TensorFlow platform. HeliosRT, a performance-enhanced implementation of LiteRT, is tailored for energy-constrained environments and is compatible with existing TensorFlow workflows.

HeliosRT optimizes custom AI kernels for the Apollo510 chip’s vector acceleration hardware. It also improves numeric support for audio and speech processing models. Finally, it delivers up to 3x gains in inference speed and power efficiency over standard LiteRT implementations.

Next, HeliosAOT introduces a ground-up, ahead-of-time compiler that transforms TensorFlow Lite models directly into embedded C code for edge AI deployment. “AOT interpretation, which developers can perform on their PC or laptop, produces C code, and developers can take that code and link it to the rest of the firmware,” Morales said. “So, developers can save a lot of memory on the code size.”

HeliosAOT provides a 15–50% reduction in memory footprint compared to traditional runtime-based deployments. Furthermore, with granular memory control, it enables per-layer weight distribution across the Apollo chip’s memory hierarchy. It also streamlines deployment with direct integration of generated C code into embedded applications.

Figure 2 HeliosRT and HeliosAOT tools are optimized for Apollo SoCs. Ambiq

“HeliosRT and HeliosAOT are designed to integrate seamlessly with existing AI development pipelines while delivering the performance and efficiency gains that edge applications demand,” said Morales. He added that both solutions are built on Ambiq’s sub-threshold power optimized technology (SPOT).

HeliosRT is now available in beta via the neuralSPOT SDK, while a general release is expected in the third quarter of 2025. On the other hand, HeliosAOT is currently available as a technical preview for select partners, and general release is planned for the fourth quarter of 2025.

Related Content

The post Two new runtime tools to accelerate edge AI deployment appeared first on EDN.

Two new runtime tools to accelerate edge AI deployment

Become a member

Become a member

Become a subscriber

Become a sponsor