Hero Banner image

AI Inference

AI Inference Compute

AI inference isn’t a monolithic workload. It alternates between compute-bound and memory-bandwidth-bound phases — each with fundamentally different scaling limits. Altera FPGAs are built to accelerate the phases that matter most, while delivering the memory fabric and high-speed interconnect needed to scale disaggregated inference clusters efficiently.

Why Altera FPGAs

Built for Change

Model architectures evolve every quarter — dense transformers to MoE to SSM hybrids. An FPGA retargets via bitstream in seconds. ASICs take 18–24 months to tape out. When the model changes, the silicon adapts.

Low Latency at Scale

Hardened 400GE, PCIe Gen5, and DDR5 interfaces deliver deterministic, wire-speed data movement with no software stack in the path. At 90%+ utilization, latency stays bounded — not average-case, every-case.

Proven in the Cloud

Deployed by leading CSPs to power DPUs and AI NICs. Agilex 7 FPGAs are in production today alongside leading GPUs and accelerators for memory orchestration and scale-up networking.

Get Started

Agilex™ 7

Agilex™ 5

Applications
AI Inference Compute 

Altera’s DSP architecture accelerates AI with optimized Tensor Modes, efficient dataflow, and high compute density per watt. Combined with high-bandwidth memory, it excels at decode-phase token generation. In disaggregated systems, FPGAs handle decode while GPUs focus on prefill—reducing GPU needs and lowering TCO.

dam image
Prefill-Decode Disaggregation

LLM inference has two phases: prefill (compute-bound) and decode (memory-bound). Running both on GPUs wastes compute during decode. Altera FPGAs offload decode, using AI Tensor Blocks for high-bandwidth GEMV and a KV Cache Scheduler across HBM and DDR. With shared KV cache over Ethernet, GPUs stay focused on prefill while FPGAs handle decode more efficiently—reducing cost and power while scaling across nodes.

dam image
Scale-Out Inference Networking

Inference clusters scale by splitting prefill and decode across nodes and distributing KV cache over shared memory fabric. Altera FPGAs provide 400G Ethernet with programmable MAC and routing, enabling vendor-agnostic, low-latency connectivity. In-network processing runs at line rate, offloading CPUs and adapting in place as architectures evolve.

dam image