|#||Precedes a command that indicates the command is to be entered as root.|
|$||Indicates a command is to be entered as a user.|
|This font||Filenames, commands, and keywords are printed in this font. Long command lines are printed in this font. Although long command lines may wrap to the next line, the return is not part of the command; do not press enter.|
|<variable_name>||Indicates the placeholder text that appears between the angle brackets must be replaced with an appropriate value. Do not enter the angle brackets.|
Hardware accelerator implemented in FPGA logic that accelerates or intends to accelerate an application.
|AFU||Accelerator Functional Unit||The supplied implementation of an accelerator, typically in HDL.|
|API||Application Programming Interface||A set of subroutine definitions, protocols, and tools for building software applications.|
|ASE||AFU Simulation Environment||
Co-simulation environment that allows you to use the same host application and AF in a simulation environment. ASE is part of the .
|CCI-P||Core Cache Interface||CCI-P is the hardware-side signaling interface between the AFU and the FPGA Interface Unit (FIU).|
|CL||Cache Line||64-byte cache line|
|DFH||Device Feature Header||Creates a linked list of feature headers to provide an extensible way of adding features.|
|FIM||FPGA Interface Manager (FIM)||The compiled bitstream containing the FPGA Interface Unit (FIU) and other interfaces such as external SDRAM.|
|FIU||FPGA Interface Unit (FIU)||
The FIU connects the host and the AFU.
|MPF||Memory Properties Factory||Optimizes CCI-P traffic traffic before it reaches FIU.|
|Msg||Message||Message - a control notification|
|NLB||Native Loopback||The NLB performs reads and writes to the CCI-P link to test connectivity and throughput.|
|RdLine_I||Read Line Invalid||
Memory Read Request, with FPGA cache hint set to invalid. The line is not cached in the FPGA, but may cause FPGA cache pollution.
Note: The cache tag tracks the request status for all outstanding requests on Intel® Ultra Path Interconnect ( Intel® UPI). Therefore, even though RdLine_I is marked invalid upon completion, it consumes the cache tag temporarily to track the request status over UPI. This action may result in the eviction of a cache line, resulting in cache pollution. The advantage of using RdLine_I is that it is not tracked by CPU directory; thus it will prevent snooping from CPU.
|RdLine-S||Read Line Shared||Memory read request with FPGA cache hint set to shared. An attempt is made to keep it in the FPGA cache in a shared state.|
|WrLine_I||Write Line Invalid||
Memory Write Request, with FPGA cache hint set to Invalid. The FIU writes the data with no intention of keeping the data in FPGA cache.
|WrLine_M||Write Line Modified||
Memory Write Request, with the FPGA cache hint set to Modified. The FIU writes the data and leaves it in the FPGA cache in a modified state.
|Acceleration Stack for Intel® Xeon® CPU with FPGAs||Acceleration Stack||
A collection of software, firmware and tools that provides performance-optimized connectivity between an Intel® FPGA and an Intel® Xeon® processor.
|Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA||Intel® PAC with Arria® 10||
PCIe® accelerator card with an Intel® Arria® 10 FPGA. Programmable Acceleration Card is abbreviated PAC.
Contains a FPGA Interface Manager (FIM) that pairs with an Intel® Xeon® processor over PCIe® bus.
|Intel® Xeon® Processor with Integrated FPGA||Integrated FPGA Platform||
Intel® Xeon® plus FPGA platform with the Intel® Xeon® and an FPGA in a single package and sharing a coherent view of memory via Quick Path Interconnect (QPI).
- fpgadiag requires NLB mode 0 AF for the lpbk1 test.
- fpgadiag requires NLB mode 3 AF for the trput, read, and write tests.
The $DCP_LOC/hw/green_bits/nlb_400/ directory stores source code for the NLB sample AFU. Refer to the Intel Acceleration Stack for Intel® Xeon® CPU with FPGAs Getting Started Guide to understand the NLB sample AFU source code structure and how to build it.
The release package provides the following two sample AFs:
- NLB mode 0 AF: available from the hello_fpga or fpgadiag utility
- NLB mode 3 AF: available from the fpgadiag utility.
The following files implement the loopback function shown in the figure above:
|nlb_lpbk.v||Top-level wrapper for NLB that instantiates the requestor and arbiter.|
|arbiter.sv||Instantiates the test AF.|
|requestor.sv||Accepts requests from the arbiter and formats the requests according to the CCI-P specification. Also implements flow control.|
|nlb_csr.sv||Implements a 64-bit read/write Control and Status (CSR) registers. The registers support both 32- and 64-bit reads and writes.|
|nlb_tram_sdp.sv||Implements a generic dual-port RAM with one write port and one read port.|
NLB is a reference implementation of an AFU compatible with the Core Cache Interface (CCI-P) Reference Manual. NLB’s primary function is to validate host connectivity using different memory access patterns. NLB also measures bandwidth and read/write latency. The bandwidth test has the following options:
- 100% read
- 100% write
- 50% read and 50% writes
|Byte Address (OPAE)||Word Address (CCI-P)||Access||Name||Width||Description|
|0x0000||0x0000||RO||DFH||64||AF Device Feature Header|
|0x0008||0x0002||RO||AFU_ID_L||64||AF ID low|
|0x0010||0x0004||RO||AFU_ID_H||64||AF ID high|
|0x0018||0x0006||Rsvd||CSR_DFH_RSVD0||64||Mandatory Reserved 0|
|0x0020||0x0008||RO||CSR_DFH_RSVD1||64||Mandatory Reserved 1|
|0x0100||0x0040||RW||CSR_SCRATCHPAD0||64||Scratchpad register 0|
|0x0108||0x0042||RW||CSR_SCRATCHPAD1||64||Scratchpad register 2|
|0x0110||0x0044||RW||CSR_AFU_DSM_BASEL||32||Lower 32-bits of AF DSM base address. The lower 6 bits are 4x00 because the address is aligned to the 64-byte cache line size|
|0x0114||0x0045||RW||CSR_AFU_DSM_BASEH||32||Upper 32-bits of AF DSM base address|
|0x0120||0x0048||RW||CSR_SRC_ADDR||64||Start physical address for source buffer. All read requests target this region.|
|0x0128||0x004A||RW||CSR_DST_ADDR||64||Start physical address for destination buffer. All write requests target this region|
|0x0130||0x004C||RW||CSR_NUM_LINES||32||Number of cache lines|
|Controls test flow, start, stop, force completion|
|0x0140||0x0050||RW||CSR_CFG||32||Configures test parameters|
|0x0148||0x0052||RW||CSR_INACT_THRESH||32||Inactivity threshold limit|
|0x0150||0x0054||RW||CSR_INTERRUPT0||32||SW allocates Interrupt APIC ID and Vector to device|
|DSM Offset Map|
|0x0040||0x0010||RO||DSM_STATUS||32||Test status and error register|
|CSR_SRC_ADDR||[63:<N>]||RW||2^(N+6)MB aligned address points to the start of the read buffer.|
|CSR_DST_ADDR||[63:<N>]||RW||2^(N+6)MB aligned address points to the start of the write buffer.|
Number of cache lines to read or write. This threshold may be different for each test AF.
Ensure that source and destination buffers are large enough to accommodate the <N> cache lines.
CSR_NUM_LINES should be less than or equal to <N>.
For the following values, assume <N>=14. Then, CSR_SRC_ADDR and CSR_DST_ADDR accept 2^20 (0x100000).
|CSR_SRC_ADDR||[31:14]||RW||1MB aligned address.|
|CSR_DST_ADDR||[31:14]||RW||1MB aligned address.|
Number of cache lines to read or write. This threshold may be different for each test AF.
Note: Ensure that source and destination buffers are large enough to accommodate the <N> cache lines.
|||RW||Force test completion. Writes test completion flag and other performance counters to csr_stat. After forcing test completion, the hardware state is identical to a non-forced test completion.|
|||RW||Starts test execution.|
|||RW||Active low test reset. When low, all configuration parameters change to their default values.|
|CSR_CFG||||RW||cr_interrupt_testmode tests interrupts. Generates an interrupt at the end of each test.|
|cr_interrupt_on_error sends an interrupt when upon error detection.|
cr_test_cfg configures the behavior of each test mode.
|[13:12]||RW||cr_chsel selects the virtual channel.|
cr_rdsel configures the read request type. The encodings have the following valid values:
cr_delay_en enables random delay insertion between requests.
|[6:5]||RW||Configures test mode,cr_multiCL-len. Valid values are 0,1,3.|
configures test mode. The following values are valid:
For more information about the test mode, refer to the Test Modes topic below.
c_cont selects test rollover or test termination.
cr_wrthru_en switches between WrLine_I and Wrline_M request types.
Inactivity threshold limit. Detects the duration of stalls during a test run. Counts the number of consecutive idle cycles. If the inactivity count > CSR_INACT_THRESHOLD, no requests are sent, no responses are received, and the inact_timeout signal is set.
Writing 1 to CSR_CTL activates this counter.
|CSR_INTERRUPT0||[23:16]||RW||The Interrupt Vector Number for the device.|
|[15:0]||RW||apic_id is the APIC OD for the device.|
|DSM_STATUS||[511:256]||RO||Error dump form Test Mode.|
|[191:160]||RO||Number of Writes.|
|[159:128]||RO||Number of Reads.|
|[127:64]||RO||Number of Clocks.|
|[63:32]||RO||Test error register.|
|[31:16]||RO||Compare and exchange success counter.|
|[15:1]||RO||Unique ID for each DSM status write.|
|||RO||Test completion flag.|
CSR_CFG[4:2] configures the test mode. The following four tests are available:
- LPBK1: This is a memory copy test. The AF copies CSR_NUM_LINES from the source buffer to the destination buffer. Upon test completion, the software compares the source and destination buffers.
- Read: This test stresses the read path and measures read bandwidth or latency. The AF reads CSR_NUM_LINES starting from the CSR_SRC_ADDR. This is only a bandwidth or latency test. It does not verify the data read.
- Write: This test stresses the write path and measures write bandwidth or latency. The AF reads CSR_NUM_LINES starting from the CSR_SRC_ADDR. This is only a bandwidth or latency test. It does not verify the data written.
- TRPUT: This test combines the reads and writes. It reads CSR_NUM_LINES starting from CSR_SRC_ADDR location and writes CSR_NUM_LINES to CSR_SRC_ADDR. It also measures read and write bandwidth. This test does not check the data. The reads and writes have no dependencies
The following table shows the CSR_CFG encodings for the four tests. This table sets and CSR_NUM_LINES, <N>=14. You can change the number of cache lines by updating the CSR_NUM_LINES register.
|Test Mode||Encoding CSR_CFG[4:2]||Cache Line Threshold CSR_NUM_LINES||Cache Line Threshold for <N>=14|
The fpgadiag utililty includes several tests to diagnose, test, and report on the FPGA hardware. Use the fpgadiag utility to run all the test modes. For more information about using the fpgadiag utility, refer to the fpgadiag section in the OPAE FPGA Tools Guide.
- Software initializes Device Status Memory (DSM) to zero.
- Software writes the DSM BASE address to the AFU.
CSR Write(DSM_BASE_H), CSRWrite(DSM_BASE_L)
- Software prepares source and destination memory buffer. This preparation is test specific.
- Software writes CSR_CTL[2:0]= 0x1. This write brings the test out of reset and into configuration mode. Configuration can proceed only when CSR_CTL=1 & CSR_CTL=1.
- Software configures the test parameters, such as src, destaddress, csr_cfg, num lines, and so on.
- Software CSR writes CSR_CTL[2:0]= 0x3. The AF begins test execution.
- Test completion:
- Hardware completes when the test completes or detects an error. Upon completion, the hardware AF updates DSM_STATUS. Software polls DSM_STATUS[31:0]==1 to detect test completion.
- Software can force test completion by writing CSR writes CSR_CTL[2:0]=0x7. Hardware AF updates DSM_STATUS.
|December 2017||2017.12.22||Initial release.|