Cloud Computing as a Source of Latency

Besides the two-tier and three-tier architecture, there are numerous other ways to struc-ture applications [FLR⁺14]. Cloud computing is quickly becoming the major backbone of novel technologies across application fields such as web and mobile applications, Internet of Things (IoT), smart cities, virtual and augmented reality, gaming, streaming, data sci-ence, and Big Data analytics. Cloud computing delivers on the idea ofutility computing introduced by John McCarthy in 1961 that suggests that computing should be a ubiquitous utility similar to electricity and telecommunications [AG17]. In the context of cloud com-puting, there are several sources of latency across all types of application architectures.

In this section, we will summarize the architecture-independent latency bottlenecks that contribute to the overall performance of cloud-based applications.

In the literature, cloud computing has been defined in various different ways [LS13, YBDS08, MB16, FLR⁺14, BYV⁺09, MG09, TCB14]. Throughout this thesis, we will use the widely accepted NIST definition [MG09]. It distinguishes between five characteristics of cloud offerings and groups them into three service models and four deployment models.

The nature of the service and deployment models motivates, why latency is of utmost relevance in cloud computing.

Characteristics

The characteristics of cloud offerings explain how cloud computing is desirable for both customers and providers. Providers offer on-demand self-service, which means that con-sumers can provision services and resources in a fully automated process. Broad network accessenables the cloud services to be consumed by any client technology that has Internet access. Cloud providers applyresource pooling(multi-tenancy) to share storage, network-ing, and processing resources across tenants to leverage economies of scale for reduced costs. Rapid elasticity demands that resources can be freed and allocated with minimal delay, building the foundation for scalability. The provider exposes ameasured servicethat is used for pay-per-use pricing models with fine-grained control, monitoring and reporting of resource usage to the consumer. In practice, the major reasons for companies to adopt cloud computing is the ability to replace capital expenditures (CAPEX) that would have been necessary to acquire hardware and software into operational expenditures (OPEX) incurred by the usage of pay-per-use cloud services. The major incentive for providers is the ability to exploit economies of scale and accommodate new business models.

Service Models

Based on increasing degree of abstraction, three high-level service models can be distin-guished:

Infrastructure-as-a-Service (IaaS). In an IaaS cloud, low-level resources such as com-puting (e.g., containers [Mer14] and virtual machines [BDF⁺03]), networking (e.g., subnets, load balancers, and firewalls [GJP11]) and storage (e.g., network-attached storage) can be provisioned. This allows deploying arbitrary applications in the cloud while leaving control of the infrastructure to the IaaS provider. In IaaS clouds, latency is particularly relevant for cross-node communication, potentially across different data centers (e.g., between an application server and a replicated database). Example offerings are Amazon Elastic Compute Cloud (EC2) [Ama17b], Softlayer [Sof17], Joyent [Joy17], and Google Compute Engine (GCE) [Goo17a].

Platform-as-a-Service (PaaS). Consumers of PaaS clouds run applications on a technol-ogy stack of services, programming languages, and application platforms defined by the provider including explicit support for developing, testing, deploying and host-ing the application. In addition to the infrastructure, a PaaS provider also manages operating systems and networks. The role of latency in a PaaS is critical: as there is no control over native computing and storage resources, data management has to be consumed as a service either from the same provider or an external DBaaS. Ex-amples of PaaS vendors are Microsoft Azure [Azu17], Amazon Beanstalk [AWS17], IBM Bluemix [IBM17], Google App Engine [App17], and Heroku [Clo17b].

Software-as-a-Service (SaaS). A SaaS provides a specific cloud-hosted application to users (e.g., email, word processors, spreadsheets, customer relationship manage-ment, games, virtual desktops). The provider completely abstracts from the cloud

infrastructure and only allows customization and configuration of the application.

Almost all SaaS offerings are consumed as web applications via HTTP, so that client-server latency is crucial for both initial loads and performance of interactions. Ex-amples include Microsoft Office 365 [Off17], Salesforce [Onl17], and Slack [Sla17].

Besides the above three models, other “XaaS” (Everything-as-a-Service) models have been proposed, for example, Storage-as-a-Service, Humans-as-a-Service and Function-as-a-Ser-vice amongst many others [KLAR10, DLNW13, TCB14, MB16, HYA⁺15]. Database-as-a-Service and Backend-as-a-Database-as-a-Service as discussed in Section 2.2.6 cut across the three canon-ical levels of IaaS, PaaS, and SaaS and can be employed in each of the models.

Deployment Models

Deployment models describe different options for delivering and hosting cloud plat-forms.

Public Cloud. A public cloud is operated by a business, academic, or government orga-nization on its infrastructure and can be used by the general public. Commercial cloud offerings such as Amazon EC2, Google App Engine, and Salesforce fall in this category. In public clouds, latency to users and third-party services is critical for performance.

Private Cloud. A private cloud provides exclusive use for one organization and is hosted on-premises of the consumer. This implies that the hardware resources are mostly static and in order to gain elasticity, public cloud resources may be added on de-mand, e.g., during load spikes (cloud bursting[GSW⁺12]). Besides commercial so-lutions such as VMWare vCloud [Clo17c], various open-source platforms for private PaaS and IaaS clouds have been developed, including OpenStack [BWA13], Euca-lyptus [NWG⁺09], and Cloud Foundry [Clo17a]. As private clouds usually cannot exploit a globally distributed set of data centers, tackling wide-area latency to end users is a key challenge.

Hybrid Cloud. In a hybrid cloud (also called multi-cloud deployment), two or more clouds are composed to combine their benefits. There are frameworks for ad-dressing multiple clouds through a common API, e.g., jclouds [Apa17a] and lib-Cloud [Apa17b] as well as commercial providers for multi-cloud deployments, scal-ing and burstscal-ing such as RightScale [Rig17], Scalr [Sca17], and Skytap [Sky17].

Any communication between different cloud platforms is highly latency-sensitive.

When offloading critical components like data storage to a different cloud, incurred latency can be prohibitive and outweigh the advantages. On the other hand, if data management makes use of the broader geographic reach of multiple providers through caching or replication [WM13], latency can be reduced substantially as we will show in the next chapters.

The NIST definition [MG09] also defines a community cloud, as a cloud shared between organizations with common concerns. Though the model is not in common use, the same

latency challenges apply: composed backends and remote users are subject to latency bottlenecks.

Latency in Cloud Architectures

In cloud-based applications, latency stems from various sources introduced by the com-position of different service and deployment models. We group the latency into three categories:

1. Round-trip times within a data center network or LAN are usually in the order of single-digit milliseconds.

2. Latencies between twoco-located data centersare in the order of 10 ms.

3. For hosts from twodifferent geographical locations, latency often reaches 100 ms and more.

Figure 2.3 illustrates the typical latency contributions of several communication links within a distributed web application. In the example, the client is separated from the backend by a high-latency wide area network (WAN) link. The application’s business logic is hosted on an IaaS platform and distributed across multiple servers interconnected via local networks. The data tier consists of a database service replicated across different availability zones. For a synchronously replicated database system, the latency between two data centers therefore defines the response time for database updates (for example in the Amazon RDS database service [VGS⁺17]).

Most complex applications integrateheterogeneous servicesfor different functions of the application. For example, an external DBaaS might be consumed from the main applica-tion over a high-latency network, since it is shared between two different applicaapplica-tions or provides a level of scalability that a database hosted on an IaaS could not provide. Parts of the application might also be developed with a service model that fits the requirements better, for example by offloading user authentication to microservices running on a PaaS.

A BaaS could be integrated to handle standard functions such as push notifications. High latency also occurs if third-party services are integrated, for example, a social network in the frontend or a SaaS for payments in the backend. Overall, the more providers and services are involved in the application architecture, the higher the dependency on low latency for performance. As almost all interactions between services evolve around ex-changing and loading data, the techniques proposed in this thesis apply to the latency problems in the example.

For further details on cloud models, please refer to Murugesan and Bojanova [MB16], who provide a detailed overview of cloud computing and its foundational concepts and technologies. Bahga and Madisetti [BM13] review the programming models and APIs of different cloud platforms.

In the following, we will provide detailed background on backend and network perfor-mance to highlight the different opportunities for tackling latency across the application stack.

External Database-as-a-Service ApplicationLogic

Client

IaaS Data Center DC1

Data Center DC2 Data Center DC3

Replicated Database Service

Third-Party Services (e.g., Social Networks)

External Cloud Services (e.g., Payment Provider) PaaS

Files (HTML, CSS, JS, ...) Data (JSON, XML, …)

Microservice (e.g., Authentication)

BaaS

Real-Time Data Synchronization Static File Hosting Push Notifications

~1 mslatency ~10 mslatency ~100 mslatency

Figure 2.3: Potential sources of latency in distributed, cloud-based applications.