.webp){kind=small}
Data Center
AI Inference
Memory-class FPGAs for AI inference and memory orchestration at data center scale.
Altera FPGAs Solve the Memory Problem
Programmable orchestration at the hardware level feeds xPUs at line rate: decompressed, in the model’s native format, every clock cycle. Model changes ship as a new bitstream, not new hardware.
Agilex® 7 M-Series pairs in-package HBM2e with DDR5 and a high-bandwidth network-on-chip, so memory-bound inference workloads run at memory speed rather than bus speed. Reprogrammable fabric lets teams update AI pipelines as models and precisions evolve, without a hardware redesign.
The Challenges, and Why FPGA
The Challenge
New Model, Same System
Model architectures and precisions change faster than a hardware refresh cycle.
Why FPGA
New Bitstream, Same Hardware
Reflash in milliseconds and run any model; keep the system and keep every device relevant.
The Challenge
Memory Wall
Memory-bound inference workloads are limited by bus speed, not compute.
Why FPGA
In-Package HBM2e + DDR5
Agilex 7 M-Series runs memory-bound workloads at memory speed via in-package HBM2e, and as a NIC at line rate: pipelined, quantize in flight.
The Challenge
Thermal & Power Envelopes
Rack power and cost-per-inference targets cap how far scaling by adding parts can go.
Why FPGA
Efficient Inference per Watt
Deterministic hardware pipelines deliver predictable throughput per watt, with no idle cycles.
The Challenge
Prefill and Decode Scale Differently
Prefill is compute-bound and decode is memory-bound; one node mix cannot serve both phases.
Why FPGA
Disaggregated, Deterministic Scale-Out
A shared KV cache fabric: low latency, line rate, pipelined, no software in the data path, so prefill and decode scale independently.
Why Altera
Highest Memory Bandwidth FPGA
Agilex 7 M-Series pairs in-package HBM2e with DDR5 and a 7.52 Tb/s network-on-chip.
FPGA AI Suite
Sequential mode and spatial dataflow map AI models into FPGA fabric for high throughput.
Deterministic, Low-Latency Inference
Agilex retains high performance at high utilization, scheduling in nanoseconds in the data path.
Leading Performance per Watt
Hardware pipelines with no idle cycles deliver more inference within the power budget: efficiency and power consumption are the metrics that matter.
Get Started
Data Center Applications
Storage
NVMe-oF, inline crypto and compression, and array processing on one Agilex 7 device.
Platform Management
BMC, SCM, and Platform Root of Trust consolidated on one field-upgradable device.
Common Questions
The KV cache stores attention state reused during token generation and can consume substantial memory capacity and transfer bandwidth. FPGA orchestration can manage supported movement and formatting across memory tiers; the benefit depends on model and workload characteristics.
Offload memory movement when profiling shows transfer, formatting or orchestration limits the inference system. Keep compute on the GPU when its execution flow fits; evaluate whether an FPGA removes the measured bottleneck after including integration overhead.
Disaggregation places prefill and decode on separate resources, so the required KV state must reach the decode side. Size the transport for model, context length, concurrency and transfer timing, including congestion and any supported compression overhead.
Place frequently accessed state in the tier whose capacity, bandwidth and latency meet the access pattern. HBM2e favors high bandwidth, while DDR5 or a supported CXL pool can add capacity; actual placement needs topology and contention measurements.
Production context lengths expose realistic KV cache capacity, transfer and contention costs. Benchmark concurrency and token latency at those lengths so a small-context demonstration does not hide the deployment's memory bottleneck.