Evolution of Data Center Networking Trends and Protocols

Link State on Data Center Fabrics Russ White, LinkedIn http://rule11.tech

Past Trend BGP BGP BGP BGP

Why BGP? Requirement OSPF IS-IS BGP Prefix distribution Yes Yes Yes Prefix filtering Limited Limited Extensive Traffic Engineering Limited Limited Extensive Traffic Tagging Basic Basic Extensive Multi-vendor stability Yes Yes Even more so (think about the Internet) https://forums.juniper.net/t5/Data-Center-Technologists/BGP-in-the-Data-Center-Why-you-need-to-deploy-it-now/ba-p/227547 REQ1: Select a topology that can be scaled "horizontally" by adding more links and network devices of the same type without requiring upgrades to the network elements themselves. REQ2: Define a narrow set of software features/protocols supported by a multitude of networking equipment vendors. REQ3: Choose a routing protocol that has a simple implementation in terms of programming code complexity and ease of operational support. REQ4: Minimize the failure domain of equipment or protocol issues as much as possible. REQ5: Allow for some traffic engineering, preferably via explicit control of the routing prefix next hop using built-in protocol mechanics. https://tools.ietf.org/html/rfc7938

Current Trend Open/R (+ BGP) BGP BGP BGP Openfabric (maybe + BGP) BGP Firepath (+ others) BGP

What changed? Our perception of policy is changing SDN Complexity of mixing policy and reachability Our perception of configuration is changing Single source of truth versus minimal or no configuration ZTP/DevOps versus autonomic

Layered Control Plane Controller needs full network view LSDB is easiest Hence link state reachability and topology preferred Overlay is policy only Removes filtering, security, and traffic engineering configuration from fabric devices

Shifting Trendlines Requirement OSPF IS-IS BGP Prefix distribution Yes Yes Yes Prefix filtering Limited Limited Extensive Traffic Engineering Limited Limited Extensive Traffic Tagging Basic Basic Extensive Multi-vendor stability Yes Yes Even more so (think about the Internet) versus Requirement OSPF IS-IS BGP Prefix distribution Yes Yes Yes Prefix filtering Handled by the SDN overlay Traffic Engineering Handled by the SDN overlay Traffic Tagging Handled by the SDN overlay Multi-vendor stability Reduced importance through disaggregation Autoconfiguration Yes Yes Can be (largely) modified to work Full View of Topology Yes Yes Can be modified to work

What do I need to do to BGP? Autocompute router ID and AS number Peer on link locals Autodiscover local peers through some secondary protocol Peer with anyone who tries to open Use something like BGP-LS to gather a topology at the controller But I still Need to configure remote peers, policies, etc. Need to carry policy in the distributed control plane These are made more challenging by the mix of local autoconfiguration and controller based single source of truth

Why am I doing this? BGP gives me policy But mixing policy, reachability, and topology discovery has proven complex BGP is well known and widely implemented This is valid if you are relying vendor driven software in a mixed vendor environment Disaggregation (white box) changes the game BGP scales better Let s think about this one a little

Scaling Issues BGP can handle more prefixes Mitigating factors Modern processors Optimized SPF Most changes are leaf only Link state can carry the number of prefixes required in a DC fabric BGP can handle more routers in a single network This comes down to flooding

Why IS-IS? Easier to modify than OSPF Flooding domains are less entangled Packet format is TLV based Easier incremental/partial SPF Does not require link local addressing

The Flooding Challenge A B C D E 1 2 3 4 Each Is receives at least 4, and potentially more than 8 copies, of each modified LSP

Reverse Flooding Optimzation Distributed Solution A B C D E A1 runs SPF C1-4, A2-4 are two hop neighbors B1 chosen as flooder Flooded to B1 in normal LSP Flooded to others in link local LSP (RFC7356) draft-white-openfabric 1 2 3 4

Reverse Flooding Optimization Distributed Solution A B C D E Do not flood to any neighbor on any shortest path towards the originator draft-white-openfabric 1 2 3 4

Centralized Flooding Firepath Jupiter Rising research.googlstatic.googleusercon tent.com/media/research.google.co m/en//pubs/archive/43837.pdf draft-li-dynamic- flooding

The Scaling Problem Route count is not a huge problem There are solutions in this space if needed Node count primarily comes down to flooding The flooding problem can be solved What else do we need? Autoconfiguration bits A policy overlay

Autoconfiguration IS net ID Can be calculated From a local MAC address Link local IPv6 through SLAAC Loopback for management Interface addresses are not needed Local label pool for traffic engineering Can be calculated

Openfabric Fabric Location hop count == spf with all metrics set to 1 x = max hop count from my perspective y = max path from the perspective of someone who is T0 location == y x draft-white-openfabric Advertised in tier TLV from shen-isis-spine- leaf-ext A B C D E

Fabric Location A B C D E If connected to T0, then I must be T1 connected to T1, then I must be T2 Etc. draft-shen-isis- spine-leaf-ext Advertised in tier TLV from shen-isis- spine-leaf-ext

Policy Overlay There are many choices here PCEP I2RS BGP as a southbound Publish/Subscribe transport is (probably) best Every operator and/or vendor could choose a different solution

Summary BGP is a good DC fabric protocol For some specific situations Such as when you need Policy mingled with reachability Little or no visibility into the topology Strong multivendor support Single source of truth with strong DevOps automation environment

Summary Each of these points is being challenged Policy should be separated from reachability and topology Strong requirements for topology visibility Minimize configuration/autonomic operation Link state protocols fit these requirements better than BGP