Cloud Infrastructure Overview Lecture

14-848: CLOUD INFRASTRUCTURE AWS OVERVIEW * LECTURE 24 * FALL 2019 * KESDEN

MICROSERVICES, DEFINED Small, light-weight, network-interfaced software components representing program logic intended to be integrated into a solution Most commonly implemented via REST

SERVICE-ORIENTED ARCHITECTURE (SOA) Service-oriented architecture (SOA) is a style of software design where services are provided to the other components by application components, through a communication protocol over a network. The basic principles of service-oriented architecture are independent of vendors, products and technologies. A service is a discrete unit of functionality that can be accessed remotely and acted upon and updated independently, such as retrieving a credit card statement online. A service has four properties according to one of many definitions of SOA: It logically represents a business activity with a specified outcome. It is self-contained. It is a black box for its consumers. It may consist of other underlying services. -- Quoted from: https://en.wikipedia.org/wiki/Service-oriented_architecture, 12/2/2019

MICROSERVICES VS SOA May be viewed as a type of SOA But, doing so may lose the bigger picture Distinctions in practice: SOA tends to have very heavily architected interfaces, e.g. SOAP, WSDL, etc Microservices have light-weight interfaces, e.g. HTTP(S)/REST SOA tends to have heavier-weight components, e.g. middleware Microservices have very little build up SOA focus is often on ecosystem, integration, etc Microservices tend to be small bits of encapsulated logic needed for an application or single service.

MICROSERVICES VS SOA Loosely coupled SOA Encapsulating data with business logic Fine-grained SOA

WHY MICROSERVICES Consider goal of delivering single service or Web/Mobile app. Why use microservices? Short answer: Modularization is always good Lighter-weight components have many developmental benefits: Less runway needed to deliver Small units of work for developers Less tangle among developers Greater scalability resulting from smaller building blocks, each one can be scaled/replicated as needed.

WHY MICROSERVICES, CONT? Stepping back Microservices are a big win because they are small, independent, and light-weight Natural model for serverless computing Easy to scale out what is needed, when it is needed Developers can work independently Developers can be responsible for their service from dev through production without needing to worry about bigger picture. Wasyh to containerize

HOW MICRO IS MICRO? As small as a few lines of code As large as a single logical function As individual function are integrated, starts to approach SOA Functions may impose different loads and scale differently Different teams may be responsible for different functions Monitoring across integration isn t straight-forward for the developer of a single concern

FROM MICRO TO SOLUTION Microservices need to be integrated to become a solution If all integration is at the front-end, the front end can become a monster UIs are at risk of being monolithic If UI functions can be distributed across the microservices, this may be best. UIs can be modular, even if not microservices. Integration can be a microservice, within reason Communication is an important consideration

COMPLEXITY AND COST The integration of microservices presents complexity Profiling Debugging Communications overhead Cognitive overhead Logging, event tracking, etc. Easy to have either not enough information surface or chaotic information overload Can be worse if deployed across geographically distant locales

UNDERSTANDING FAILURE Horizontally scaled systems, almost by definition, are increasingly prone to failure as they scale Microservice failure scales as the deployment size grows Some failures are compartmentalized, e.g. within a container Some failures are bucketed, e.g. all containers sharing a host Microservice failures scale as the communication grows Because of interaction, often in excess of host scaling Unmanaged, integration means failures are infectious.

UNDERSTANDING RESILIENCE Availability depending upon an instance is fragile Needing a particular, single instance from a pool of n-instances Availability depending upon a service pool is resilient Ability to use any instance means all instances need to fail to lose availability Failure can be soft, and degrade performance Longer latencies Lower resolution/incomplete results Remember Long tail of response problem

MICROSERVICES OVERVIEW Think of a Web Services API in a SOA Containerize the functions to make them independent Each piece can be scaled and distributed Each instance can be interchanged Less interaction among developers Easier for a developer to monitor dev-to-production Increased interaction can be expensive Communications complexity overhead Cost of monitoring, debugging, profiling, and managing Failure is manageable, but needs to be managed

SERVERLESS COMPUTING Also known as Function as a Service (FaaS) Ephemeral instance of a containerized function is instantiated for each request Enabled by containerization Natural model for scalability But, overhead can be costly Compute time may be only 40% of runtime

EASY WINS Highly parallelizable functions, e.g. true lambdas Highly independent functions E.g. Orchestration functions Little would be shared across instances Composition functions Aggregations without a lot of data to move or long communications pipelines

HARD CASES Benefitting from recurrent state Caching, etc. Heavy interactions, data pipelines Additive latency and multiplicative complexity Anything that needs to be started up, except RAM Costly

SERVERLESS COMPUTING AND CONTAINERIZATION Wouldn t be A Thing without containers VMs would be way too latent to launch and costly in overhead. Raw execution isn t really a thing anymore and would lack protections and environment isolation Often called Lambdas or Functions, e.g. AWS Lambdas, Microsoft Azure Functions, Google Cloud Functions, etc JavaScript Node.js solutions are also possible anywhere Node.js runs E.g., Google Cloud, etc.

WHAT TEMPERATURE? Strong isolation guarantee requires new instance for each call Latency likely can be measured in whole seconds Concurrent requests Strong isolation requires concurrent containers or new container for each request Relaxed requirements may allow for a queue or concurrent execution Reusing containers is a possibility Relaxes isolation guarantees First access is still cold Containers can be launched in advance of anticipated need Takes up resources before needed Requires good understanding of when and how many to pre warm

THINKING ABOUT THE ISOLATION MODEL Isolation within an application vs outside of an application Stronger guarantee likely required across applications Communication within an application may not need to be the same as across applications Network calls much more expensive that local function calls or even IPC calls Maybe this suggests more efficient organization?

STRATEGY: ISOLATE APPLICATIONS, NOT FUNCTIONS Containerize applications, leave functions raw Isolation where needed Efficient instantiation of functions within a single application

STRATEGY: HIGHER-LEVEL COMMUNICATION Consider communication as high-level messaging Implement via network only where needed Networking across containers, not within containers More efficient IPC within a container

SERVERLESS COMPUTING SUMMARY Excellent for auto-scaling Inefficient for start-up But can use pre-warming Inefficient w.r.t.communications overhead Especially painful, for example, for pipelined communication But, can be worked out with specialized communication mechanisms under the hood