RISC-V Tensor Unit claims to turbocharge AI applications
A new RISC-V Tensor Unit, based on fully customizable 64-bit cores, claims to provide a huge performance boost for artificial intelligence (AI) applications compared to just running software on scalar processors. The Tensor Unit provides matrix multiplications required by AI applications via a hardware design that delivers massive compute performance while keeping energy consumption low.
Machine learning models—such as LLaMa-2 or ChatGPT—comprise billions of parameters and require massive computation power in the order of several trillions of operations per second. That’s because the bulk of computations in large language models (LLMs) is built as fully connected layers that can be efficiently implemented as matrix multiplication.
The Tensor Unit unveiled by Semidynamics provides hardware tailored specifically for matrix multiplication workloads. It does that by leveraging the Vector Unit capabilities as well as the company’s Gazzillion technology to fetch the data it needs from memory swiftly. The Vector Unit, which is constantly fed data by the Gazzillion technology, ensures that there are no data misses.
A Vector Unit comprises several vector cores—roughly equivalent to a GPU core—that perform multiple calculations in parallel. And each vector core has arithmetic units capable of performing addition, subtraction, fused multiply-add, division, square root, and logic operations.
Figure 1 The Tensor Unit is optimized for 64-bit RISC-V cores. Source: Semidynamics
The Tensor Unit, built on top of the company’s Vector Processing Unit, leverages the existing vector registers to store matrices (see Figure 1). As a result, it can be used for layers that require matrix multiply capabilities. Moreover, the Tensor Unit can use the Vector Unit for the activation function layers, which is a significant improvement over stand-alone NPUs that usually have trouble dealing with activation layers.
According to Roger Espasa, founder and CEO of Semidynamics, while other solutions rely on difficult-to-program direct memory access (DMA) to solve this problem, his company’s solution seamlessly integrates the Tensor Unit into its cache-coherent subsystem, opening a new era of programming simplicity for AI software.
Furthermore, the Tensor Unit uses the vector registers to store its data and does not include a new, architecturally visible state. As a result, it seamlessly works under any RISC-V vector-enabled Linux without any changes.
Figure 2 Here is an overall ensemble with the Atrevido-423 RISC-V core, the Gazzillion Unit, the Vector Unit, and the Tensor Unit. Source: Semidynamics
Semidynamics, a European supplier of RISC-V cores, specializes in high-bandwidth, high-performance cores with vector units. The Barcelona, Spain-based company will provide more details about its Tensor Unit solution at the RISC-V North America Summit in Santa Clara on 7 November 2023.
Related Content
Top five fallacies about RISC-V
RISC-V Climbs Software Mountain
DAC 2023: RISC-V is not in the future, it’s now
RISC-V Summit 2022: All Your CPUs Belong to Us
Startups Help RISC-V Reshape Computer Architecture
RISC-V venture in Germany to accelerate design ecosystem
googletag.cmd.push(function() { googletag.display(‘div-gpt-ad-native’); });
–>
The post RISC-V Tensor Unit claims to turbocharge AI applications appeared first on EDN.
