Understanding Modularity in Procedures and Processes

Remote Procedure Call Landon Cox Landon Cox February February 7 7, 2018 , 2018

Modularity so far Procedures Procedures as modules Private vs shared between procedures Private vs shared between procedures Local variables are private Stack, heap, global variables are shared as modules Module 1 Module 2 Code Code Private state Shared state Private state

Modularity so far Procedures Procedures as modules Transferring control between procedures Transferring control between procedures Caller adds arguments and RA to stack, jumps into callee code Callee sets up local variables, runs code, jumps to RA as modules Module 1 Module 2 Code Code Private state Shared state Private state

Modularity so far Procedures Procedures as modules Is isolation between procedures enforced? Is isolation between procedures enforced? No, either module can corrupt the other No guarantee that callee will return to caller either as modules Module 1 Module 2 Code Code Private state Shared state Private state

Modularity so far MULTICS processes MULTICS processes as modules Private vs shared between MULTICS processes Private vs shared between MULTICS processes Address spaces are private Segments can be shared as modules Module 1 Module 2 Code Code Private state Shared state Private state

Modularity so far MULTICS processes MULTICS processes as modules Transferring control between MULTICS processes Transferring control between MULTICS processes Use synchronization primitives from supervisor Lock/unlock, wait/notify as modules Module 1 Module 2 Code Code Private state Shared state Private state

Modularity so far MULTICS processes MULTICS processes as modules Is isolation between MULTICS processes enforced? Is isolation between MULTICS processes enforced? Yes, modules cannot corrupt private state of the other Isolate shared state inside common segments as modules Module 1 Module 2 Code Code Private state Shared state Private state

Modularity so far UNIX processes UNIX processes as modules Private vs shared between UNIX processes Private vs shared between UNIX processes Address spaces are private File system and pipes are shared as modules Module 1 Module 2 Code Code Private state Shared state Private state

Modularity so far UNIX processes UNIX processes as modules Transferring control between UNIX processes Transferring control between UNIX processes Use synchronization primitives from kernel Block by reading from pipe, notify by writing to pipe as modules Module 1 Module 2 Code Code Private state Shared state Private state

Modularity so far UNIX processes UNIX processes as modules Is isolation between UNIX processes enforced? Is isolation between UNIX processes enforced? Yes, modules cannot corrupt private state of the other Protect shared state using pipe buffer and FS access control as modules Module 1 Module 2 Code Code Private state Shared state Private state

Network programming Now say two modules are on different machines Now say two modules are on different machines What is the standard abstraction for communication? What is the standard abstraction for communication? Sockets Each end of socket is bound to an <address, port> pair Module 1 Module 2 Code Code Private state Shared state Private state

Network programming Now say two modules are on different machines Now say two modules are on different machines Sockets look familiar Sockets look familiar Very similar to pipes Use read/write primitives for synchronized access to buffer Downsides of socket programming Downsides of socket programming Adds complexity complexity to a program Blocking conditions Blocking conditions depend on data received Data structures copied copied into and out of messages or streams All of this work can be tedious and error-prone Idea: programmers are used to local procedures Idea: programmers are used to local procedures Try to make network programming as easy as procedure calls

Remote procedure call (RPC) RPC makes request/response look local RPC makes request/response look local Provides the illusion of a function call RPC isn t a really a function call RPC isn t a really a function call In a normal call, the PC jumps to the function Function then jumps back to caller This is This is similar Stream of control goes from client to server And then returns back to the client similar to request/response though to request/response though

The RPC illusion How to make send/ How to make send/recv recv look like a function call? look like a function call? Client wants Client wants Send to server to look like calling a function Reply from server to look like function returning Server wants Server wants Receive from client to look like a function being called Wants to send response like returning from function

Implementing RPC Primary challenges Primary challenges How to name, locate the remote code to invoke? How to handle arguments containing pointers? How to handle failures?

RPC architecture Client Server Client code Server code Import Import Export Export Interface Interface Client stub Server stub Export Export Import Import Network RPC runtime RPC runtime Who imports and who exports the interface? Who imports and who exports the interface?

RPC architecture Client Server Client code Server code Import Import Export Export Interface Interface Client stub Server stub Export Export Import Import Network RPC runtime RPC runtime Who defines the interface? Who defines the interface? The programmer

RPC architecture Client Server Client code Server code Import Import Export Export Interface Interface Client stub Server stub Export Export Import Import Network RPC runtime RPC runtime Who writes the client and server code? Who writes the client and server code? The programmer

RPC architecture Client Server Client code Server code Import Import Export Export Interface Interface Client stub Server stub Export Export Import Import Network RPC runtime RPC runtime Who writes the stub code? Who writes the stub code? An automated stub generator (rmic in Java)

RPC architecture Client Server Client code Server code Import Import Export Export Interface Interface Client stub Server stub Export Export Import Import Network RPC runtime RPC runtime Interface precisely defines behavior. What data comes in, what is returned. Why can stub code be generated automatically? Why can stub code be generated automatically?

RPC architecture Client Server Client code Server code Import Import Export Export Interface Interface Client stub Server stub Export Export Import Import Network RPC runtime RPC runtime Where else have we seen automated control transfer? Where else have we seen automated control transfer? Compilers + procedure calls

RPC stub functions call call send send Client Client stub stub recv recv return return send send return return Server Server stub stub call call recv recv

RPC stub functions Client stub Client stub 1) Builds request message with server function name and parameters 2) Sends request message to server stub Transfer control to server stub: clients Transfer control to server stub: clients- -side code is paused 8) Receives response message from server stub 9) Returns response value to client side code is paused Server stub Server stub 3) Receives request message 4) Calls the right server function with the specified parameters 5) Waits for the server function to return 6) Builds a response message with the return value 7) Sends response message to client stub

Binding What is binding? What is binding? Establishing map from symbolic name object In an RPC system what needs to be bound? In an RPC system what needs to be bound? Client code uses interface as a symbolic name RPC system must bind those names to code instances residing in some address space In Cedar what managed this mapping? In Cedar what managed this mapping? The Grapevine distributed database Types are listed as symbolic names Instances are listed as machine addresses

Binding Is anyone allowed to export any interface? Is anyone allowed to export any interface? No, this is regulated through Grapevine access controls Users allowed to export an interface are explicit in group Only group owner can allow someone to export Grapevine Is anyone allowed to import an interface? Is anyone allowed to import an interface? Yes, clients authorized at higher level Group map: interfaces user ids Individual map: user id network address What other distributed database is Grapevine like? What other distributed database is Grapevine like? Domain name service (DNS) Contains mapping from names to IP addrs

Binding Is anyone allowed to export any interface? Is anyone allowed to export any interface? No, this is regulated through Grapevine access controls Users allowed to export an interface are explicit in group Only group owner can allow someone to export Grapevine Is anyone allowed to import an interface? Is anyone allowed to import an interface? Yes, clients authorized at higher level Group map: interfaces user ids Individual map: user id network address Are permissions same or different than DNS (reads and writes)? Are permissions same or different than DNS (reads and writes)? Basically the same DNS updates are controlled DNS retrievals are not

Shared state What is the shared state of the RPC abstraction? What is the shared state of the RPC abstraction? Arguments passed through function call What is the actual shared state in RPC? What is the actual shared state in RPC? The underlying messages between client and server Client Server Code Code Private state Shared state Private state

Shared state Why is translating arguments into messages tricky? Why is translating arguments into messages tricky? Data structures have pointers Client and server run in different address spaces Need to ensure that pointer on client = pointer on server Client Server Code Code Private state Shared state Private state

Shared state How do we ensure that a data structure is safely transferred? How do we ensure that a data structure is safely transferred? Must know the semantics of data structure (typed object references) Must then replace pointers on client with valid pointers on server Requires explicit help of programmer to get right Cannot just pass arbitrary C-style structs and hope to work correctly Client Server Code Code Private state Shared state Private state

Shared state What about after server code completes? What about after server code completes? Must synchronize updates to arguments Changes by server must be reflected in client before returning Client Server Code Code Private state Shared state Private state

Faults With procedures, what happens if a module faults? With procedures, what happens if a module faults? No isolation, program crashes Result of sharing the same address space With pipes, what happens if a module faults? With pipes, what happens if a module faults? Faulting module (process) crashes OS makes pipe unreadable and unwritable Cannot just return an error code through client stub Cannot just return an error code through client stub Bad idea to overload errors Want to distinguish network failures from incorrectness

Faults How are RPC faults handled in practice? How are RPC faults handled in practice? Usually through a software exception Often supported by language So how pure is the RPC abstraction? So how pure is the RPC abstraction? Not totally pure Programmer still knows which calls are local vs remote Have to write code for handling failures So is RPC a good abstraction? So is RPC a good abstraction? In some cases yes, hides a lot of the complexity However, it often comes at a steep performance penalty What part of RPC is slowest? What part of RPC is slowest? Argument packing and unpacking Java class introspection for shipping data structures is particularly painful

Structuring a concurrent system Talked about two ways to build a system Talked about two ways to build a system

Alternative structure Can also give cooperating threads own address spaces Can also give cooperating threads own address spaces Each thread is basically a separate process Use messages instead of shared data to communicate Why would you want to do this? Why would you want to do this? Protection Each module runs in its own address space Reasoning behind micro Reasoning behind micro- -kernels Each service runs as a separate process Mach from CMU (influenced parts Mac OS X) Vista/Win7 s handling of device drivers kernels

Eduardo Cuervo - Duke Aruna Balasubramanian - U Washington Dae-ki Cho - UCLA Alec Wolman, Stefan Saroiu, Ranveer Chandra, Paramvir Bahl Microsoft Research

Li-Ion Energy Density 250 200 150 Wh/Kg 100 Just 2X in 15 years 50 0 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 Year CPU performance during same period: 246X A solution to the battery problem seems unlikely

Not on par with desktop counterparts Slow, Limited or Inaccurate Power Intensive Interactive Games Speech Recognition and Synthesis Too CPU intensive Limited Augmented Reality

Remote execution can reduce energy consumption Challenges: What should be offloaded? How to dynamically decide when to offload? How to minimize the required programmer effort?

MAUI Contributions: Combine extensive profiling with an ILP solver Makes dynamic offload decisions Optimize for energy reduction Profile: device, network, application Leverage modern language runtime (.NET CLR) To simplify program partitioning Reflection, serialization, strong typing

Motivation MAUI system design MAUI proxy MAUI profiler MAUI solver Evaluation Summary Beyond MAUI

Maui Runtime Maui Runtime RPC Server Proxy Client Proxy Application Application Profiler Profiler RPC Solver Solver Maui Controller Maui server Smartphone

Goal: make it dead-simple to MAUI-ify apps Build app as a standalone phone app Add .NET attributes to indicate remoteable Follow a simple set of rules

Portability: Mobile (ARM) vs Server (x86) .NET Framework Common Intermediate Language Type-Safety and Serialization: Automate state extraction Reflection: Identifies methods with [Remoteable] tag Automates generation of RPC stubs

Maui Runtime Maui Runtime Handles Errors RPC Provides runtime information Server Proxy Client Proxy Application Intercepts Application Calls Application Synchronizes State Profiler Profiler RPC Solver Chooses local or remote Solver Maui Controller Maui server Smartphone

CPU Cycles State size Device Profile Execution Time Network Latency Profiler Callgraph Network Bandwidth Annotated Callgraph Network Power Cost Network Delay Computational Delay Computational Power Cost Computational Delay

A sample callgraph C 5000 mJ 3000 ms B 1000mJ 900 mJ 15ms A Energy and delay for state transfer Computation energy and delay for execution D 15000 mJ 12000 ms

Yes! This simple example from Face Recognition app shows why local analysis fails. InitializeFace Recognizer 5000 mJ 1000mJ User Interface FindMatch 900 mJ Cheaper to do local DetectAndExtract Faces 15000 mJ

Yes! This simple example from Face Recognition app shows why local analysis fails. InitializeFace Recognizer 5000 mJ Cheaper to do local 1000mJ User Interface FindMatch 900 mJ DetectAndExtract Faces 15000 mJ Cheaper to do local

InitializeFace Recognizer 1000mJ User Interface 25900mJ FindMatch Cheaper to offload DetectAndExtract Faces

Adapt to: Network Bandwidth/Latency Changes Variability on method s computational requirements Experiment: Modified off the shelf arcade game application Physics Modeling (homing missiles) Evaluated under different latency settings