CLOUD FINAL

Challenges of Virtualization

(1)Performance overhead (2)Special hardware compatibility (3)Portability and vendor lock (4)Network load

Benefits of Cloud Computing

(1)Reduced investments and proportional costs (2)Increased scalability (3)Increased availability and reliability

Issues with Cloud & Trust Boundary

(1)Security flaws (2)Accountability (3)Lack of user control (4)Unauthorized secondary usage

When Not To Virtualize

(1)special hardware / licensing requirements, (2)applications with MAC address identifiers, (3)applications requiring high graphic rendering, (4)some unsupported applications, (5)enterprise overhead depending on scale

Number of Nines Availability

526 minutes of downtime per year -> 99.9% (three nines), 5 minutes of downtime per year -> 99.999% (five nines)

High Availability Algorithm

A = MTBF / (MTBF + MTTR)

Load Balancing Virtual Server Instances

Architecture establishes a capacity watchdog system that dynamically calculates virtual server instances and associated workloads, before distributing the processing across available physical server hosts

Bare Metal Provisioning

Architecture establishes a central deployment system that utilizes special service deployment/discovery agents capable of discovering and provisioning physical systems remotely

Resource Reservation

Architecture establishes a system whereby one of the following is set aside exclusively for a given cloud consumer: A single IT resource, a portion of an IT resource, multiple IT resources

Non-Disruptive Service Relocation

Architecture triggers the duplication or migration of a cloud service at runtime, avoiding any disruptions

Workload distribution Support Mechanisms

Audit monitor, Cloud usage monitor, Hypervisor, Logical network perimeter, Resource cluster, Resource replication

Resource Pooling Support Mechanisms

Audit monitor, Cloud usage monitor, hypervisor, Logical network perimeter, Pay-per-use monitor, Remote admin system, Resource management system, Resource replication

[Language Support] Azure Functions

C#, JS, F# (PHP, Python, Batch, PS)

Dynamic Scalability Support Mechanisms

Cloud usage monitor, Hypervisor, Pay-per-use monitor

Elastic Resource Capacity Support Mechanisms

Cloud usage monitor, Pay-per-use monitor, Resource replication

Service Load Balancing Support Mechanisms

Cloud usage monitor, Resource cluster, Resource replication

Elastic Disk Provisioning Support Mechanisms

Cloud usage monitor, Resource replication

FaaS Disadvantages

Decreased transparency, Tough to debug, Autoscaling of costs, Tough to keep track of many functions

4 Uses of Containers

Distributed applications, Microservices, Batch processing, Continuous integration

Functions of Load Balancing

Distributes client requests or network load across multiple servers, ensures HA and reliability through sending requests to only available servers, and Provides flexibility to add or remove servers based on demand

4 Benefits of Containers

Environment consistency, Operational efficient, Increased developer productivity, Version control

Cloud Balancing

Establishes a specialized model in which virtualized resources can be load balanced across multiple clouds or availability zones

Dynamic Failure Detection and Recovery

Establishes a watchdog system to monitor and respond to a wide range of pre-defined failure scenarios. It notifies and escalates the failure conditions that it cannot automatically resolve itself

FaaS Advantages

Fewer developer logistics, More time focused on writing code, Inherently scalable, Cost reduction, Built in availability & fault tolerance

5 Benefits of Serverless Computing

Fully managed service, Event-driven, Scalability & Availability, Less-Ops, Pay for execution time

Architectural Considerations For Serverless Computing

Functions are stateless (will not exist after functions cease to exist) & ephemeral (containers are started when event is triggered for X duration)

IP Hash

IP of client is used to determine which server receives the new request; a deterministic algorithm - results will be the same if no other elements are involved in the hash computation; the hash is applied on the source IP of the incoming request, though the hash may be changed if a server in the pool goes down/up, or if there are server weight changes

Workload Distribution

IT resources horizontally scaled by adding n number of additional identical resource; centrally managed thru load balancer, reduces over-utilization and under-utilization, equally distributed load

Risks and Challenges of Cloud Computing

Increased security vulnerabilities, Responsibility over data security is shared with the cloud provider, Organizational trust boundaries must include the cloud, Trust boundaries may overlap with other orgs, Increased attack surface, Reduced optional governance control, Level of governance control lower than on-prem resources, Cloud provider SLAs may not actually be met, causing resources to be unreliable, Geographical distances between provider and consumer may introduce latency and/or BW constraints , Limited portability between cloud providers, Compliance and legal issues

Drawbacks of Running Microservices in Full VMs

Individual VM for each service (wastes resources & has a performance penalty), Multiple services into a single VM (incr risk of app conflict, similar problem as running them on a single physical server

[Language Support] Google functions

JS

vSwitch

L2 Ethernet switch software abstraction, physical links are uplinks to vSwitch, each VM emulates at least one vNIC, vNIC generates Ethernet frames and sending them to vSwitch through bridge which then gets checked by vSwitch to send frame to physical NIC or a different VM connected to the same VLAN, if a local VM is the destination the data exchanged between the 2 VMs in the same host only require a memory-based operation

LXD

Linux container management extension. LXD container that gives you full machine functionality not just a single process, really fast hypervisor that lets you operate at container level speeds

LXC

Linux container; uses cgroups, namespaces, AppArmor profiles and SECCOMP, bare-metal performance, precise QoS, snapshots and live migration, rapid provisioning, guest instant boot, remote image service, extensible storage

MTBF

Mean Time Between Failures (incr to high)

MTTR

Mean Time to Recover (reduce to low)

[Language Support] AWS Lambad

Node.js, Python, Java

Guest OS

OS running on a VM

6 Cloud characteristics

On-demand usage, Ubiquitous access, Multitenancy & resource pooling, Elasticity, Measured usage, Resiliency

Evolution of serverless Computing

Physical > VM > Containers > Serverless

5 Levels of Redundancy

Physical components, Virtual resources, Availability zones, Regions, Clouds

Environments Appropriate to Run Microservices

Physical server (wasteful), Within a VM, Individual VMs (running multiple services on a single OS = conflicting library versions / app components, one service failure could affect others)

Zero Downtime

Provides a sophisticated failover system that allows virtual servers to be moved dynamically to a different physical server hosts in the event of a host failure

99.9999% Availability (6 9's)

Services are replicated to on-site spare hardware, Software automatically recovers from failures through the use of HA and replication from frameworks (hot failover), Optimized recovery speed

Achieving 99.9% Availability (3 9's)

Spare inventory is held on site for quick repair/swap, Technician dispatched when failures detected, Cold failover to another location if needed

99.999% Availability (5 9's)

Spare inventory is held pre-emptively on-site for quick repair/swap, Software automatically recovers from failures through the use of a HA (High Availability) framework, Hot failover to another data centre if needed

Not Attached

Standalone VM

Other Cloud delivery Models

Storage-as-a-Service, Database-as-a-Service, Security-as-a-Service

Bridged Adapter

VM is assigned an IP address on physical NIC and hypervisors uses net filter driver to creat a new software driver

Function-as-a-Service (Faas)

Webapps, Backends, Data/stream processing, Chatbots, Scheduled tasks, IT automation

Load Balancer in Cloud

a piece of hardware or virtual hardware/software that acts like a reverse proxy to distribute network and/or application traffic that is used to improve concurrent user capacity and overall reliability of apps; helps improve these by distributing workload across multiple servers, decreasing overall burden

Service Load Balancing

a specialized variation of the workload distribution architecture that is geared specifically for scaling cloud service implementations (redundant/duplicate cloud service deployments are pooled/created with load balancing system and added dynamically to distribute workload)

VM

abstracted set of resources running onto a VMM that creates logical computing

IaaS -> PaaS -> FaaS (Function-as-a-Service)

abstracts the entire programming runtime, deploy and forget

Heartbeat Algorithm

accurate instance monitoring, system must be able to quickly and accurately synchronize resources during failover; graceful shutdown of failing systems, accurate information about resources available, and makes information available to recovering instances

Fault Tolerance Systems

achieve 100% uptime, guaranteed resource uptime and availability

Match Strategy

adding capacity in small increments as demands increase

Lead Strategy

adding capacity to a resource in anticipation of demand

Lag Strategy

adding capacity when the resource reaches full capacity

Cost Reduction

aims to reduce these two major costs related to IT infrastructure (1) cost of acquiring new resources (2) cost of ongoing ownership and operational overhead (e.g. technical personnel, upgrades and patches, utility bills, security and access control measures, administrative and accounts staff)

Version Control

all environments are defined by files

Development

any programming language/tool/framework

High-Availability Systems

attempt to maximize resource up time and availability but cannot guarantee 100% (realistic for cloud providers, uptime can sometimes approach 100%)

Continuous Integration

automatically deploy environments using files without requiring developer intervention

Distributed Applications

break apart a web service into its components which will all scale independently

Elastic Disk Provisioning

cloud consumer is granularly billed for the exact amount of storage it actually uses; thin provisioning software is installed on virtual servers that process dynamic storage allocation via the hypervisor

On-Demand Usage

cloud consumers are given freedom to self provision IT resources; those resources should be provisioned automatically without human intervention (on-demand self-service usage)

Ubiquitous Access

cloud services should be accessible almost everywhere and should support a large variety of configuration devices, transport protocols, interfaces, security technologies & service platforms (providers tend to tailor environments to the needs of clients)

Hybrid Cloud

comprised of two/more cloud deployment models; creation/maintenance is difficult and complex due to environment/management disparity (Increased flexibility w resources, Good to transition to public cloud, Control over sensitive data, Rules/procedures may be difficult to define/eforce, Data retention, process, and transit may be difficult to control, Security compliance may be difficult to control due to organizational trust boundaries)

Batch Processing

computer-intensive applications can deploy thousands of container nodes to provide grid computing environments

Grid Computing

computing resources placed into logical pools providing high performance; hardware and resource types may vary due to use of middleware layer (load balancing logic, failover controls, autonomic configuration management) deployed on all machines - tend to be more distributed and loosely coupled than in clustering

High Availability

configured to provide nearly full-time availability, uses redundant hardware and software, avoids single point of failure, failover process moves processing tasks to backup components (milliseconds and transparent)

Weighted Round Robin

considers criteria like traffic handling capacity (requests/s)

Public Cloud

consumable by the general public and are owned by third-party providers, where the provider creates, maintains, and monitors all resources (Availability, Scalability, Accessibility, Cost savings, Integration limitations, Reduced flexibility & forced downtime)

Docker

container-based "virtualization" that assists developers with quickly developing, deploying and running applications, containers are isolated, may or may not have resource restrictions, their own file system, and share the host's kernel and binaries (using namespaces and cgroups); Supports DevOps and PaaS deployments

Operational. Efficiency

containers are grouped strategically since each is an application space

Hypervisor Clustering

creates a high availability cluster of hypervisors across multiple physical servers; if a hypervisor or underlying physical hardware goes

NAT

default configuration, no additional config on host machine or network, hypervisor uses physical network adapter as a NAT router with VMs connecting to this NAT router using a software based data link interface

[Benefits of Serverless Computing] Fully Managed Service

developers don't need to worry about underlying infrastructure, OS, middleware etc.

Unplanned

difficult to predict, disruptive

Thin VM

disk size is equal to the amount of data stored by the Guest OS, up to a max threshold

Round Robin

distributed across group of servers sequentially

Cloud Bursting

dynamic scaling that scales or "bursts out" on-premise It resources into a cloud whenever predefined capacity thresholds have been reached; pre-deployed, but remain inactive until bursting - after they are no longer needed, they burst back to the on-prem equipment - automated scaling listener and resource replication mechanisms are in play for higher usage demands

Load Balancing

efficiently distributing incoming traffic across a group of backend servers (server farm/pool, adding more server, load balancer sits in front of servers and router client requests across capable servers)

VMM (Hypervisor)

enables the creation of / access to a virtual resource that emulates and underlying physical resource (interfacing physical resources, emulating virtual resources, isolation, deallocating/constraining virtual resources)

Increases Developer Productivity

environment consistency removes cross-service dependencies for applications

[Benefits of Serverless Computing] Event-Driven

functions are triggered based on events

[Benefits of Serverless Computing] Scalability & Availability

functions scale horizontally in automatic/elastic fashion managed by cloud service provider

Clustering

group of interconnected and independent IT resources working together as a single system; hardware must be similar and interconnected with high speed communication links - incl. built in redundancy & failover, greater availability, & reliability

Microservices

hyper-scaling horizontally through reproducibility

Architectural/Infrastructural

improves scalability, horizontal scaling solutions like load balancing and messaging

Under-Provisioning

inability to meet the needs of users

Over-Provisioning

inefficient use of resources

Cloud Service Owner

legal entity owning a service running on the cloud

Community Cloud

like public clouds, however access is limited to a specific cloud community; jointly owned/managed by a group of consumers or third party (Costs shared between members, Multitenancy enable economics, Support and maintenance requirements can be shared, Ownership may not be clearly defined, Responsibilities for maintenance/support may be ambiguous, Security compliance may be different for each organization leveraging the resources)

Trust Boundary

logical perimeter that spans beyond physical boundaries to represent which IT resources are trusted in a system

Capacity Planning

maximum amount of deliverable resources available at any instance in time

VM Migration [Cold]

migration of a powered-off virtual machine between 2 hypervisor nodes (doesn't require a SAN)

Container

modular environment allowing resource-isolated processes to run together to deliver applications using configuration files, web application, clusters, & DevOps support (All VMs have their own resources & OS, and Containers share host OS kernel, host binaries, have their own file system, & share underlying physical resources

VM Migration [Live]

movement of virtual machine from one physical host to another while its powered on (great for load/resource balancing across datacenters, uses centralized shared storage (SAN))

Least Connections

new requests are sent to servers with fewest active clients

Serverless

no server management, computing model or code execution where underlying infrastructure: physical, virtual hosts, OS, containers are abstracted from the developer and service consumer (logic is encapsulated in functions, if load increases = more functions are executed, scalability and capacity benefits, provider manages underlying infrastructure

Microservices and Containers

offers between execution isolation, runs within its own environment, uses cgroups for isolation, leads to server consolidation, faster initialization/execution than VMs, better for erratic workloads

Resource Pooling

one or more resource pools of identical IT resources grouped and maintaned by a system that automatically ensures they remain synced; (CPU pool, memory pool, storage pool, network pool) - can form parent, sibling, and nest pools if needed

OS-Assisted Virtualization

operates at OS (kernel) level, guests share a common OS kernel that supports software to act as a VMM; guest OS is same type as the host and knows its being virtualized (Very low VMM overhead, Extreme server density, Does not support guest OS of different types, Security of main kernel impacts all other containers)

[Benefits of Serverless Computing] Less-Ops

operational tasks like debugging testing troubleshooting is outsourced to cloud service provider

Cloud Consumer

organization or individual utilizing IT resources made available by the cloud provider

Organizational Agility

organization's responsiveness to change

Private Cloud

owned and consumed by a single organization; both the consumer & provider centralize access to themselves, making trust boundaries less worrisome (Easier support to troubleshooting access, Granular control over upgrade and downtime schedules, Monitoring can be expanded based on organization needs, Substantial upfront costs, Ongoing maintenance costs, Must ensure software compatibility, Requires a breadth of knowledge to deploy/maintain)

Cloud Provider

party that provides access to cloud-based resources

Cloud Consumer

party that uses the remote cloud-based resources

[Benefits of Serverless Computing] Pay for Execution Time

pay for only duration of execution of a function and number of functions executed, idle = no charge

Organizational Boundary

physical perimeter surrounding a set of IT resources owned by an organization

Host-Only

physical port need not connect to outside network, virtual loop-back port is created on host for communication between VMs that reside on that host

Environment Consistency

portability/reproducibility from the environmental configuration files

Dynamic Scalability

pre-defined scaling conditions that trigger the dynamic allocation of IT resources from resource pools; can be dynamic horizontal/vertical scaling or dynamic relocation

VLAN

provides VM isolation from one host to another, Layer 3 routing, Layer 2 bridging

Hardware Based

provides proprietary software and specialized processors (if traffic increases you gave to buy more or bigger machines from vendor)

Elastic Resource Capacity

provisions virtual servers using a system that allocates and reclains CPU and RAM in immediate response to fluctuating process requirements of hosted IT resources

Platform-as-a-Service (PaaS)

ready-made environment built upon already deployed and configured IT resources (consumers have pre-packaged tools and products, helps them replace/extend on-premise to cloud with scalability and cost reduction) ex. Google's app engine (Python & Java), webhosting environments

Cloud

refers to an It environment facilitating the remote provisioning of, and access to, scalable & measurable computing resources; internet's access component is a cloud, but a cloud has finite resources and boundaries whereas the internet does not

Measured Usage

refers to the ability of a cloud platform to track a cloud consumer's usage of the cloud provider's underlying IT resources (consumers billed on usage and/or time)

Multitenancy & Resources Pooling

refers to the ability of a software application to enable instances of the program to serve different tenants while transparently providing isolation from one another (. Pooling allows providers to combine underlying physical resources into a virtual hardware abstraction layer (both features can be assigned and reassigned to consumers based on requirements & demands)

Elasticity

refers to the automated scaling of IT resources depending on the runtime requirements of the cloud consumer (reduces consumer investment $ while allowing billed per usage, main design foundation of cloud computing)

Sibling Pool

regionally grouped It resources that are isolated and independent

Cloud Resources

resources are remote to the organization

On-Premise Resources

resources fall within the local operational boundaries of an organization

Microservice Application Examples

ride share app, online store app, large teams working on a large monolithic project vs. smaller teams developing the services using the tools they are most comfortable with

Planned

routine operations, periodic maintenance, upgrades

Software Based

run on commodity hardware (less expensive / more flexible, run locally on cloud VM or container)

Server Consolidation

running/installing multiple OS onto a single physical server which will incr hardware utilization, load balancing, and optimization of resources (On-demand usage, Resource pooling, Elasticity, Scalability, Resiliency)

Full Virtualization

runs directly on hardware, code directly executed on processor for high speed performance; VMM emulates all underlying systems needed by a VM, guests don't know they are being virtualized (High performance, Reduced VMM management overhead, Requires explicit CPU support, Consolidation incr in proportion to CPU traps, Increased resourced resource usage due to emulated devices and OS kerver)

Resource Scaling [Horizontal]

scaling in and out, allocating or releasing of IT resources of the same type

Resource Scaling [Vertical]

scaling up and down, resource is replaced by another with higher or lower capacity

Redundant Storage

secondary duplicate cloud storage device that is part of a failover system that syncs its data with data in the primary cloud storage device; storage gateway diverts cloud consumer requests to secondary device whenever primary fails

Infrastructure-as-a-Service (IaaS)

self-contained IT environment comprised of infrastructure-related resources that are accessible and manageable by cloud service-based interfaces and tools (hardware, network resources, connectivity, operating systems, consumers have high-level control over environment including administration) (e.g. Amazon EC2, Microsoft Azure)

Microservices

several independent applications that can run on their own, uses different programming languages/platforms, building complex applications with simpler independent programs that execute by themselves, building blocks of larger apps, each service runs its own process

Cloud Service Consumer

software application programmatically interfacing with the cloud service's remote API

Software-as-a-Service (Saas)

software program positioned as a shared cloud service and made available as a product (reusable and made available as a consumable) ex. Office 365, Google Apps for Business

Resiliency

the ability of a cloud platform to sustain uptime during an underlying outage (consumers increase availability and reliability by leveraging resilience of cloud platforms)

Hardware Independence

ties both the OS and the installed software's configuration to a single unique platform (unique hardware into standardized emulated sources that are transferable/cloneable/manipulatable)

Thick VM

total disk size is pre-allocated when VM is created

Virtualization

underlying technology platforms used to create virtual instances of various IT resources; hardware requirements are simulated and managed by underlying virtualization software & hardware (hypervisors)

Resource Replication

virtual disk images used to create copies of hard disk content to help w/ moving/duplicating/backing up virtual servers, rapid scaling, snapshotting, and managing virtual servers

Nested Pool

when pools are divided into smaller logical pools that are provided to a customer