Enhancing Network Performance Monitoring with Co-Design Software and Hardware

Co-Designing Software and Hardware for Declarative Performance Measurement Srinivas Narayana MIT CSAIL October 7, 2016

An example: High tail latencies Delay completion of flows (and applications)

An example: High tail latencies Where is the queue buildup? How did queues build up? UDP on-off traffic? Fan-in? Which other flows cause queue buildup? Throttle UDP? Traffic pattern?

An example: High tail latencies What measurement support do you need? Where is the queue buildup? How did queues build up? UDP on-off traffic? Fan-in? Which other flows cause queue buildup? Throttle UDP? Traffic pattern?

Existing measurement support Sampling (NetFlow, sFlow) May not sample packets/events you care about Counting (OpenSketch, UnivMon, ) Only traffic counting Time granularity too coarse Network Packet capture (Endace, Niksun, Fmadio, ) Too much data to collect everywhere & always Endpoint data collection (Pingmesh, ) Data distributed over several hosts Insufficient network visibility Host

Network performance questions Flow-level packet drop rates Queue latency EWMA per connection Route flapping incidents Persistently long queues TCP incast and outcast Interference from bursty traffic Incidence of TCP reordering and retransmissions Understanding congestion control schemes Incidence and lengths of flowlets High end to end latencies ...

Can we build better performance monitoring tools for networks?

An example: High tail latencies Switches have precise visibility into network conditions

Performance monitoring on switches (+) Precise visibility of performance (e.g., queue buildup, flows, ) (+) Speed (line rates of 10-100 ports of 10-100 Gb/s/port) (-) Costly and time-consuming to build hardware (2-3 years) Ideally, new hardware should suit diverse measurement needs

Co-design software and hardware (1) Design declarative query language to ask performance questions (2) Design hardwareprimitives to support query language at line rate

Performance query system Declarative performance queries Network operator Accurate query results with low overhead Diagnostic apps

(1) Declarative language abstraction Write SQL-like performance queries on an abstract table: (Packet headers, Queue id, Queue size, Time at queue ingress, Time at queue egress, Packet path info) For all packets at all queues

(1) Example performance queries Queues and traffic sources with high latency SELECT srcip, qid FROM T WHERE tout - tin > 1ms Traffic counters per source & destination SELECT srcip, dstip, count, sum_len GROUPBY srcip, dstip User-defined fold functions Queue latency EWMA SELECT 5tuple, qid, ewma GROUPBY 5tuple, qid def ewma (lat_est, (tin, tout)): lat_est = alpha * lat_est + (1 - alpha) * (tout-tin) Packet ordering matters (recent queue sizes more significant)

(2) Hardware primitives Good news: Many existing primitives prove useful! Selection: Match-action rules Per-packet latency and queue data: In-band network telemetry (INT)

(2) Hardware support for aggregation SELECT 5tuple, ewma GROUPBY 5tuple 5tuple EWMA Run a key-value store on switches? K-V store supports read-modify-write operations

(2) Challenges in building switch K-V Run at line rate 1GHz packet rate Scale to millions of keys e.g., number of connections No existing memory is fast and large enough!

(2) Split key-value store Off-chip Backing store (DRAM) On-Chip Cache (SRAM) Key Value Update / Initialize Merge Value Key key evicted key

Need more information? See our upcoming HotNets 16 paper!

Why performance monitoring? Determine network performance bottlenecks Exonerate network as source of problems New proposals for scheduling, congestion control, what works?

Semantically useful language primitives Per-packet performance attributes: latency, loss Isolate traffic with interesting performance Aggregate performance stats over sets of packets Simultaneous occurrences of multiple conditions Compose results to pose more complex questions

The SRAM cache

Does caching lead to correct results? In general, no Forget keys and their values when evicted Keys previously evicted can come back! Merging SRAM and DRAM may eventually produce correct results

Merging example SELECT COUNT, SUM(pkt_len) GROUPBY srcip, dstip Off-chip Backing store (DRAM) Srcip, Dstip Count Sum(pkt_len) On-Chip Cache (SRAM) Key Value Update / Initialize Merge Value Key key evicted key True COUNT = COUNTSRAM + COUNTDRAM True SUM(pkt_len) = SUMSRAM + SUMDRAM

Linear-in-state condition State updates of the form S = A * S + B A, B: functions on a bounded history of packets

Linear-in-state update correctDRAM = finalSRAM An*defaultSRAM + An*previousDRAM previousDRAM correctDRAM State defaultSRAM finalSRAM Updates

Current work: Compiler Detecting linear-in-state condition Queries that use multiple key-value stores Network-wide queries Nested queries An equivalent of linear-in-state for multiple tables?

How YOU can help Useful performance diagnostics questions? How would you evaluate this system? What s a reasonable prototype?