Unicast and Static Discovery for Efficient Application Communication

Beyond Discovery Shaun Foley Principal Applications Engineer, RTI

Topics Unicast Discovery How can applications discover each other when multicast is unavailable or undesirable? Static Endpoint Discovery How can I minimize the configuration burden when using RTI Limited Bandwidth Static Discovery Plugins? User Discovery How does DDS discovery fit into the broader picture of system architecture?

Topics Unicast Discovery How can applications discover each other when multicast is unavailable or undesirable? Static Endpoint Discovery User Discovery

Hotel Reception Angela Nathan Tell Nathan I am in room 205 Where is Angela? Track guests and room numbers. Angela : room 205 Call room 205 Communication established: decide where to meet for beer

Discovery Proxy Application A Application B Tell everyone I am address 205 Where is everyone? Track applications and addresses A: 205 B: 408 Contact 205 Contact 202 Contact 408 Communication established: Begin app biz-logic

Discovery Proxy Application A Application B Contact proxy at well-known address Contact proxy at well-known address Tell everyone I am address 205 Where is everyone? Track applications and addresses Call add_peer() on received addresses A: 205 B: 408 Contact 205 Contact 202 Contact 408 Communication established: Begin app biz-logic

An Implementation Aside: Peer Indices and Addresses athena Domain Participant 0 Remote Participant peer list: 3@athena Domain Participant 1 Domain Participant 2 Domain Participant 3 Remote Participant peer list: 1@athena Domain Participant 4 discovery announcements

Data Types Apps only care about the highest participant index at a given address: add_peer( 2@192.168.44.1 ); remove_peer( 2@192.168.44.1 ); struct PeerEntry { string addr; //@key ulong pidx; //@key }; Publishes unneeded data. Resource limits depend on # of expected DPs on node. Must choose arbitrary sequence bound. struct PeerEntry { string addr; //@key sequence<ulong> pidxs; }; Last value cache. Relies on idempotent add_peer(). Only removes peer when last DP on machine dies. struct PeerEntry { string addr; //@key ulong max_pidx; };

Initial peers: 0@128 Multicast rx addrs: empty Discovery Proxy Address 128 Participant Idx 0 Peer Map published durably Addr PIdx

Initial peers: 0@128 Multicast rx addrs: empty Discovery Proxy Address 128 Participant Idx 0 128 1 Peer Map published durably Addr 128 Application PIdx 1 Address 128 Participant Idx 1

Initial peers: 0@128 Multicast rx addrs: empty Discovery Proxy Address 128 Participant Idx 0 128 1 Peer Map published durably Addr 128 128 Application PIdx 1 2 Address 128 Participant Idx 1 128 2 Application Address 128 Participant Idx 2

Discovery Proxy Application Contact proxy at well- known address Normal application functions When proxy tells us about new applications, call add_peer() proxy_client.h Three lines of code to change normal application to use unicast discovery proxy

Initial peers: 0@128 Multicast rx addrs: empty Discovery Proxy Address 128 Participant Idx 0 128 1 Peer Map published durably Addr 128 128 Application PIdx 1 2 Address 128 Participant Idx 1 128 2 Application Address 128 Participant Idx 2

Initial peers: 0@128 Multicast rx addrs: empty Discovery Proxy Address 128 Participant Idx 0 128 1 Peer Map published durably Addr 128 128 1 Application PIdx 1 2 0 Address 128 Participant Idx 1 1 0 128 2 rtiddsspy Application Address 1 Participant Idx 0 Address 128 Participant Idx 2

Topics Unicast Discovery Static Endpoint Discovery How can I minimize the configuration burden when using RTI Limited Bandwidth Static Discovery Plugins? User Discovery

Why Static Discovery? Unicast Discovery Static Endpoint Discovery How can I minimize the configuration burden when using RTI Limited Bandwidth Static Discovery Plugins? User Discovery

Why Static Discovery? Participant B Participant A A Participant A sends to initial peers Participant Announcement ~400 bytes, but varies from - Propagated properties - Entity name - # of Locators B Participant B responds Endpoint Announcement Exchange DW/DR info R ~800 bytes, but varies from - Propagated properties - Entity name - Type description W

Federation with Routing Service Fully meshed discovery within cluster Routing Service Routing Service Routing Services act as gateways for cluster May be difficult to choose meaningful clusters All cross-cluster traffic passes through Routing Services

Endpoint Discovery with Limited Bandwidth Plugins Participant B Participant A Read LBED configuration to learn about remote participant s datareaders and datawriters A Participant A sends to initial peers B R Participant B responds W R No endpoint data sent on the wire!

LBED Configuration participant name = Y reader rtps_object_id = 3 topic_name = A qos = ... reader rtps_object_id = 4 topic_name = B qos = ... participant name = X writer rtps_object_id = 1 topic_name = A qos = ... writer rtps_object_id = 2 topic_name = B qos = ... DW B roid 2 DW A roid 1 DR B roid 4 DR A roid 3 Participant X Participant Y X Look up participant by name X Assert X s endpoints into discovery database A B Discovery data: QoS, Type description, User samples

Configuration Challenges Applications do not use the LBED configuration to create DDS entities Actual configuration and LBED configuration must remain synchronized DDS relies on each datawriter or datareader to have a unique object ID Object ID assignment must be deterministic If created serially: maintain DW/DR count If created in parallel: hash topic name

LBED Misconfiguration participant name = Y reader rtps_object_id = 3 topic_name = A qos = ... reader rtps_object_id = 4 topic_name = B qos = ... participant name = X writer rtps_object_id = 1 topic_name = A qos = ... writer rtps_object_id = 2 topic_name = B qos = ... DW B roid 20 DW A roid 10 DR B roid 4 DR A roid 3 Participant X Participant Y X Look up participant by name X Assert X s endpoints into discovery database A X B Who is ROID 10 and 20??? Discovery data: QoS, Type description, User samples

Generating LBED Configuration A B Endpoint A Endpoint B X Participant X Discovery Monitor Collect Builtin Topic data and save to XML file Y C Endpoint C Endpoint D D Participant Y Discovery data: QoS, Type description,

Two Steps to Use LBED Configuration Tell applications to use load LB plugins Select previously generated LBED configuration These can be set in participant QoS section via XML

Topics Unicast Discovery Static Endpoint Discovery User Discovery How does DDS discovery fit into the broader picture of system architecture?

Selective Discovery DDS participants communicate with and store data about all endpoints, even those they do not match. How can we avoid this?

Proxied Participant Location Discovery Proxy process data X Endpoint A Endpoint B Y Participant X process metadata Y X Y B C A X D normal communication process metadata Endpoint C Endpoint D User data Participant Y Discovery metadata: Address, DP index, Topic name, Discovery data: QoS, Type description,

Proxied Topic Data - no match scenario Discovery Proxy X process data B A Endpoint A Endpoint B f(A,B) Participant X process metadata Y Y C f(C,D) D X process metadata Endpoint C Endpoint D User data Participant Y Discovery metadata: Address, DP index, Topic name, Discovery data: QoS, Type description,

Proxied Topic Data - match scenario Discovery Proxy X process data B A Endpoint A Endpoint B f(A,B) C Participant X process metadata Y Y C f(C,D) A D X normal communication process metadata Endpoint C Endpoint D User data Participant Y Discovery metadata: Address, DP index, Topic name, Discovery data: QoS, Type description,

Discovery Proxy X process data B A Endpoint A Endpoint B A Control Y Participant X process metadata Y B C X D A normal communication Data process metadata Endpoint C Endpoint D User data Participant Y Discovery metadata: Address, DP index, Topic name, Discovery data: QoS, Type description,

Discovery Control Control information need not be distributed the same way as data LBED plugin uses files to describe endpoints, DDS to communicate WSDL to describe services, SOAP to communicate DNS to find, {HTTP, DDS, } to communicate How can we maximize control plane flexibility?

Discovery Data Fixed data type Necessary for DDS interoperability Uses [internal] DDS writers and readers Has associated QoS Participant Built-in Data Writer Participant Built-in Data Reader Publication Built-in Data Writer Publication Built-in Data Reader Subscription Built-in Data Writer Subscription Built-in Data Reader Participant Built-in Data Reader Publication Built-in Data Writer Publication Built-in Data Reader Subscription Built-in Data Writer Subscription Built-in Data Reader Participant Built-in Data Writer Best Effort Reliable

Bridging Discovery Data to Normal DDS Topics Application participant Plugin Userdisc Plugin participant Discovery database Publication DataWriter Subscription DataWriter Publication DataReader Subscription DataReader

Application participant Plugin Userdisc Plugin participant core calls afterLocalWriterEnabled Hello Writer Publication DataWriter Topic: Hello key participant_key ... Publication DataReader Subscription DataWriter call into core assertRemoteReader Subscription DataReader Topic: Hello key participant_key ...

Inherit, not to reuse, but to be reused 1 Routing Service System federation and bridging Interacting with non-DDS data Mediation and transformation Recording and Replay Services Persistence Service Implement DDS durability Build more complex messagine on top of DDS User code Infrastructure frameworks, programmer familiarity 1: Sutter and Alexandrescu, C++ Coding Standards

Todays way Endpoint Description Key Topic name Type name QoS Type description Does not change for static discovery Unused by 95% of applications The Right Way Endpoint Header Key Topic name Type name Published separately Accessible via pub-sub or request-reply Endpoint QoS Key QoS reference or data Easily reconstructed on other side Endpoint Type Key Type description

Fin