AI Inference

Altera FPGAs Solve the Memory Problem

The memory wall is here, and growing.

Programmable orchestration at the hardware level feeds xPUs at line rate: decompressed, in the model's native format,
every clock cycle. Model changes ship as a new bitstream, not new hardware.

Why FPGAs

Deterministic at the Memory Wall

CPU-scheduled vs FPGA Decode is bound by memory bandwidth. An FPGA moves weights and KV cache in a hardware pipeline at line rate: no interrupts, no context switches, no jitter. P99 tracks P50.

LINE RATE · P99 ≈ P50

Memory in the Format the Model Needs

Interfaces, Network on Chip, NoC An FPGA sits in the memory path and reshapes data as it moves. Compress in, decompress out at line rate; convert and quantize in flight. Fit more in the memory pool and storage, support any model's format, and deliver it in any xPU's native language, with no extra pass through software.

COMPRESS · CONVERT · QUANTIZE IN FLIGHT

New Model, New Bitstream, Same System

bitstream comparison Support the model architectures of yesterday, today, and tomorrow. Keep the same system, reflash in milliseconds, run any model. Keep your ASIC refresh cycle; programmable FPGAs keep your devices relevant and efficient with every model.

REFLASH IN MILLISECONDS • ANY MODEL

Why Altera

Hardware orchestration, AI Tensor Blocks in the fabric, and a chiplet architecture that fits the I/O to the workload.

Every xPU Fed, Every Cycle

Three memory tiers, one device. In-package HBM2e for the hot set, DDR5 for capacity, a CXL pool for rack-scale KV cache, all through the Agilex® 7 NoC. Agilex schedules the tiers in nanoseconds, in the data path's clock domain, so every xPU gets its data on the cycle it was scheduled for.

HBM2e · DDR5 · CXL · DETERMINISTUC LATENCY

AI Tensor Blocks in the DSP

Agilex DSP blocks carry AI Tensor Blocks: dense INT8 and block floating point matrix math in the fabric, next to the memory path. Decode GEMV runs beside the KV cache, not across a bus, with the precision the model asks for.

AI TENSOR BLOCK · INT8 · BLOCK FLOATING POINT

Chiplet Architecture, Built for Your I/O

Agilex 7 connects its core fabric to transceiver tiles over EMIB: F-Tile 400G Ethernet for KV cache between nodes, R-Tile PCIe 5.0 and CXL for the memory pool. Pick the tile mix for prefill, decode, or the network, and use the fabric as a low-latency, pipelined, deterministic data path.

EMIB · F-TILE 400GE · R-TILE PCIe 5.0 · CXL

Get Started

Agilex 7 FPGAs and SoC FPGAs M-Series

Memory-class • in package HBM2e

Agilex 7 FPGAs and SoC FPGAs I-Series

I/O class • scale-out networking

Agilex 5 FPGAs and SoC FPGAs

Mid-range • performance per watt

FPGA AI Suite

Software • OpenVINO™ flow

Altera Ships the Industry's Highest Memory Bandwidth FPGA

Press release - Agilex 7 M-Series with HBM2e, Position deployment

Addressing the Greatest Memory and Compute Challenges with Agilex M-Series FPGAs

Technical blog

Altera and Arm Expand Collaboration for AI Data Center Infrastructure

Press release - FPGA plus Arm AGI CPU for heterogeneous AI infrastructure

FPGA AI Suite Handbook

Documentation - model conversion and inference IP generation

HBM2E Interface Agilex 7 M-Series IP Design Example

HBM2e via the hard memory NoC - fabric NoC use case

Agilex 7 M-Series Network-0n-Chip (NoC) Example Designs

NoC initiators, traffic generations, HBM2e and EMIF targets

External Memory Interfaces Agilex 7 M-Series IP Example Designs

DDR5 through the NoC

Agilex 7 M-Series HBM2e Kit: PCIe Root Port and HPS GSRD

Embedded design examples on the M-Series HBM2e development kit

Open FPGA Stack (OFS)

PCIe-attack FIM / AFM reference for AI NICs and accelerator cards

Agilex 7 M-Series Development Kit, HBM2e Edition

3x F-Tile, 1x R-Tile

Agilex 7 I-Series Development Kit

2x R-Tile, 1x F-Tile

FPGA AI Suite

Open VINO-based inference IP generation

P4 Suite for FPGA

P4-programmable packet processing for AI NIC designs

HBM2e • CXL • F-Tile Ethernet • R-Tile PCIe

Applications

Memory Orchestration

View Details ›

Prefill-Decode Disaggregation

View Details ›

Scale-Out Inference Networking

View Details ›

Compressed in. Exact format out. Every expert group fed at line rate.

The Agilex FPGA sits between the memory tiers and the xPUs as a Memory Processing Engine. Weights and KV cache live compressed in DDR5, a CXL pool holds the rack-scale KV tier, and 400GE brings remote KV and activations over RDMA. The FPGA routes, decompresses, prefetches, and DMAs in flight, so each xPU receives its expert group in native format and never stalls. New model, new bitstream, same system.

Key Benefits:

Dynamic expert routing. Route to experts in one clock; prefetch the hot experts.
In-flight conditioning. Decompress KV, dequantize, and convert formats on the way to the xPU.
Hardware scheduling. The FPGA schedules in nanoseconds in the data path's clock domain; a CPU schedules in microseconds.
Zero idle cycles. Double-buffered compute and weight load with predictive next-layer prefetch and deterministic QoS in logic.

Production Grade IP:

Device:

Agilex 7 FPGAs and SoC FPGAs M-Series

Learn More:

Two phases. Two kinds of silicon. One shared KV cache.

LLM inference has two phases: prefill (compute-bound) and decode (memory-bound). Match each phase to the silicon built for it: GPUs on prefill, xPUs on decode, with an Altera FPGA feeding the decode xPUs through a KV Cache Scheduler across HBM and DDR and AI Tensor Blocks for in-line GEMV. With shared KV cache, prefill and decode scale independently across nodes, so decode capacity grows where the workload needs it.

Key Benefits:

KV Cache Scheduler. Tiered across HBM2e and DDR5 in hardware.
Shared cache over Ethernet. Prefill and decode nodes exchange KV state at 400G.
Memory wall, addressed. Decode xPUs run at memory bandwidth, fed by the FPGA.
Scales across nodes. Add decode capacity independently of prefill.

Production Grade IP:

Device:

Agilex 7 FPGAs and SoC FPGAs M-Series

Learn More:

Vendor-agnostic 400G between prefill, decode, and memory.

Inference clusters scale by splitting prefill and decode across nodes and distributing KV cache over a shared memory fabric. Altera FPGAs provide 400G Ethernet with programmable MAC and routing, enabling vendor-agnostic, low-latency connectivity. In-network processing runs at line rate, offloading CPUs and adapting in place as architectures evolve.

Key Benefits:

GPU-agnostic AI NIC. One NIC design across xPU vendors and generations.
Programmable MAC and routing. RoCE today, the next fabric protocol by bitstream.
In-network processing. KV cache movement and collectives at line rate.
Deterministic. No software stack in the path; p99 tracks p50.

Production Grade IP:

Devices:

Learn More:

Partner Ecosystem

Inference-class boards, AI NICs, and memory IP from the Altera Solutions Partner network.

Find all partners

Accelerator Boards

All NICs & DPUs

Memory & Networking IP

Customer Success Stories

“

Agilex 7 M-series enables Positron to utilize a memory-optimized architecture, achieving over 93% bandwidth utilization—far surpassing the typical 10-30% seen in GPU-based systems.

Thomas Sohmers, CTO & Founder, Positron

“

Altera Agilex 7 FPGAs are selected to be used in our Vollo platform because they enables a wide variety of custom models, from financial trading to wireless telecoms to cybersecurity achieving lower latencies than any other vendor platforms.

Myrtle.ai

Products

High-Performance

Mid-Range

Power and Cost-Optimized

AI Development Tools

FPGA Design & Simulation Tools

Embedded Design Tools & Software

IP Development Tools

High-Performance

Mid-Range

Power and Cost-Optimized

Interfaces

Memory Controllers

Digital Signal Processing & AI

Soft Embedded Processors

Transceivers & Basic Functions

Solutions

Technology

Applications

Design

FPGA Development Tools

Embedded Tools

Add-On Development Tools

Simulation Tools

Utilities

Licensing

Design Hubs

Developer Centers

Training

FPGA Developer Site

Design Store

All FPGA Documentation

Support

About

Contact Us

AI Inference

Altera FPGAs Solve the Memory Problem

Why FPGAs

Deterministic at the Memory Wall

Memory in the Format the Model Needs

New Model, New Bitstream, Same System

Why Altera

Every xPU Fed, Every Cycle

AI Tensor Blocks in the DSP

Chiplet Architecture, Built for Your I/O

Get Started

Applications

Memory Orchestration

Prefill-Decode Disaggregation

Scale-Out Inference Networking

Memory Orchestration

Prefill-Decode Disaggregation

Scale-Out Inference Networking

Partner Ecosystem

Accelerator Boards

All NICs & DPUs

Memory & Networking IP

Customer Success Stories

Talk to an Altera AI Inference expert