SLP1 Firmware Development Project Overview

SLP1 firmware development Christos Gentsos, AUTH FTK IAPP 8/3/2014

Overview Scope of the SLP1 project FPGA firmware structure Components to adapt from FTK Considerations on resource usage Firmware status Simulation status Work done so far - summary Future plans

SLP1 goals: integration Main goal is to adapt the FTK system to do L1 tracking Tighter integration will help with that by improving performance First step will be to put an AMChip, an FPGA and a memory working together in a board Then the devices can be merged in a single package That configuration will function like a miniature of a current FTK AMB + AUX The integration steps resemble the evolution of computing structures from discrete devices, to chips integrated on a motherboard, to end up in a single SoC package

SLP1 goals: flexibility The main target will be the ATLAS detector L1 trigger The end architecture must be flexible, and the resulting chip easy to adapt to various HPC applications such as medical imaging applications without need for major redesign cycles

FPGA firmware structure The firmware will include components that provide Communication (GTX serial transceivers) Data buffering structures (e.g. Data Organizer for FTK) Data flow and instruction handling for the AMChip Complex processing operations on the AMChip output (e.g. track fitting) Formatting of the input and output data The FPGA device chosen for the implementation is a Xilinx Kintex-7 XC7K325T

Components to adapt from FTK There is an existing DO design we can use There is also an existing Track Fitter Those are the two largest components of the FPGA firmware

Considerations on resource usage The memory requirements of the existing DO when adapted for the specifications exceed the device capacity The TF structure when ported to that device also requires more DSP slices than available The most significant components of the firmware require redesign The AMMap is too big (18Mb) to fit in the FPGA device so an external RAM interface is necessary

Considerations on resource usage The following memory components were compared in terms of random access bandwidth, size, power and price DDR3-1066 (MT41J64M16JT-125) QDRII+ SRAM (CY7C2165KV18-550BZC) RLDRAMII (MT49H8M36BM-25) We must compare to the 50 MT/s from the AMchip Type Size / data width [Mb/b] DDR3-1066 1024 / 16 Rand-burst data read BW [144MT/s] 18 - 20 Power [W, V] Price [$] 4 - 6 0.3 0.6, 1.5 QDRII+ SRAM 18 / 36 275 1.3 2.0, 1.8 51 - 66 RLDRAMII 288 / 36 50 - 200 0.9 1.4, 1.8 18 - 32

Considerations on resource usage The DDR3 option is much too slow for the application The QDRII+ is very fast but consumes a lot of power and is expensive The RLDRAMII is the most balanced choice Type Size / data width [Mb/b] 1024 / 16 Rand-burst data read BW [144MT/s] 18 - 20 Power [W, V] Price [$] 4 - 6 DDR3-1066 0.3 0.6, 1.5 QDRII+ SRAM 18 / 36 275 1.3 2.0, 1.8 51 - 66 RLDRAMII 288 / 36 50 - 200 0.9 1.4, 1.8 18 - 32

Firmware status Data Organizer Stores the full resolution hits for each event according to SSID Reads the AMChip output and translates the RoadID to SSID for each layer using a LUT called AMMap Recovers the full resolution hits and forwards the data to the Track Fitter Has to support the ternary function of the AMChip and non- ordered input flow of hits

Firmware status Data Organizer When increasing the SSID width of current design to 16 bits (max AMchip hit width) the memory demands rise to 16.200Kb which is 101% of the total resources Also there is a requirement for a register file of 64k registers total An alternative implementation was proposed that requires 12000Kb and a register file of just 16k About 4000Kb are left for the TF and other components

Firmware status Data Organizer The proposed architecture operates at a higher clock rate of 400MHz storing 8 hits every 16 clocks, for an input data rate of 200Mhits/sec To achieve that clock rate, careful design and application of certain techniques are necessary Deep pipeline especially on memory buses Signal fan-out limit on selected signals Manual CLB resource instantiation Manual floorplanning of certain components

Firmware status Data Organizer The AM at its current state supports 100MHits/sec for reasons of consumption reduction but that speed could be easily increased The firmware can work at that rate with reduced power consumption It will be able to support a possible future doubling of the AMChip speed virtually without need for modifications

Firmware status Track Fitter Available implementation demands more DSP units than available on the device It uses 3 types of fitters depending on the existence of a missing layer and its type (nominal, pixel, sct) This is a complex structure, would get even more so to handle two missing hits The option of porting the Gigafitter for CDF should be considered

Firmware status Track Fitter Very fast implementation, solving the problem of the missing layers by loading different sets of constants Is implemented for the same type of DSP so there is less need to change things The memory requirements of the adapted Gigafitter should be estimated

Simulation status Necessary modifications were made to the AMChip code to simulate properly in Modelsim Tcl scripts were written to load data to the AM cells in the Modelsim environment The serial link core models on the AM side needed NDAs to use, so the models from the FPGA serial links were used on both sides of the link to be practical A simulation of the AM basic functionality was performed

Simulation status The results were as expected, observing pattern matches correctly More simulation will be necessary after the DO firmware is complete

Work done so far - summary A serial link interface to communicate with the AMChip was implemented in RTL and simulated The AMChip code was slightly modified to simulate properly in Modelsim A simple AM simulation was done with results being the expected ones The architecture of the existing components was studied

Work done so far - summary Projections of resource usage were done Options for the external memory were evaluated and the best option was selected SystemVerilog was picked as the language to do the design and basic verification The implementation of the core part of the DO architecture is almost complete

Future plans Complete the DO component core imminent Design the RAM and combiner component of the DO Build automated testbench with random-constrained stimulus, assertions for functional verification of the completed DO Evaluate Gigafitter architecture more thoroughly and adapt it After completing the FPGA firmware maybe build a board- level testbench using realistic data from FTK A lot of work left to do but again 9 months of secondment is enough time to do it