Introduction
Scope of this Thesis
We will have a detailed look on concurrency in three distinct areas of web architectures–connection handling, application logic and backend persistence. For each stage, we will discuss its main challenges and issues and explain existing approaches to tackle concurrency.
Web Applications
distinguish two separate types of web applications – web sites and web services.
Web Sites
some popular types for web sites that are interesting in terms of scalability and concurrency
-
Example: Social Network Sites
-
Example: Collaborative Web Applications
-
Example: E-Commerce Sites
Web Services
-
XML-RPC
-
SOAP, WSDL and WS-*
-
Representational State Transfer
Concurrency
three fundamental ways how the concurrent execution of an application can improve its performance:
-
Reduce latency
-
Hide latency
Multiple long-running tasks are executed together by the underlying system. This is particularly effective when the tasks are blocked because of external resources they must wait upon, such as disk or network I/O operations.
-
Increase throughput
This is not limited to the web server that must handle multiple client connections in parallel. Also the application that executes the associated business logic of a request and the backend data storage are confronted with concurrency.
Concurrency vs. Parallelism
Concurrency is a conceptual property of a program, while parallelism is a runtime state. Concurrency of a program depends on the programming language and the way it is coded, while parallelism depends on the actual runtime environment. Given two tasks to be executed concurrently, there are several possible execution orders. They may be performed sequentially (in any order), alternately, or even simultaneously. Only executing two different tasks simultaneously yields true parallelism. In terms of scheduling, parallelism can only be achieved if the hardware architecture supports parallel execution, like multi-core or multi-processor systems do. A single-core machine will also be able to execute multiple threads concurrently, however it can never provide true parallelism.
Concurrent Programming vs. Parallel Programming
Differentiating concurrent and parallel programming is more tedious, as both are targeting different goals on different conceptual levels. Concurrent programming tackles concurrent and interleaving tasks and the resulting complexity due to a nondeterministic control flow. Reducing and hiding latency is equally important to improving throughput. Instead, parallel programming favors a deterministic control flow and mainly reaches for optimized throughput. High-performance computing is another important area of parallel programming. It takes advantage of computer clusters and distributes sub-tasks to cluster nodes, thus speeding up complex computations.
Multiple Processes and Multiple Threads
So far, our considerations of concurrency were based on computer architectures and programming mode
A process is a heavyweight task unit and owns system resources such as memory and file descriptors that are allocated from the operating system. Threads are lightweight task units that belong to a certain process. All threads allocated within the same process share memory, file descriptors and other related resources. Creating threads is a relatively cheap operation compared to the creation of new processes.
-
multiple threads
Most concurrent applications make heavy use of multiple threads.
Concurrency, Programming Languages and Distributed Systems
Consider the strong relationship between concurrent programming, programming languages and distributed systems when building large architectures. Programming distributed systems introduces a set of additional challenges compared to regular programming.
From a software engineering perspective, the major challenges are fault tolerance, integration of distribution aspects and reliability. As we have already seen before, distributed systems are inherently concurrent and parallel, thus concurrency control is also essential.
Programming languages to be used for distributed systems must either incorporate appropriate language idioms and features to meet these requirements. Otherwise, frameworks are necessary to provide additional features on top of the core language.
the strengths of general purpose languages do not cover the main requirements of distributed systems to a great extent. Middleware systems providing distributability compensate for features missing at the core of a language. Thus, the systems actually meet the necessary requirements, but they are often also cumbersome to use and introduce superfluous complexity. So we need Go language.
Scalability
a system is described as scalable, if it will remain effective when there is a significant increase in the number of resources and the number of users.
Scalability competes with and complements other non-functional requirements such as availability, reliability and performance.
Horizontal and Vertical Scalability
There are two basic strategies for scaling–vertical and horizontal.
In case of vertical scaling, additional resources are added to a single node. As a result, the node can then handle more work and provides additional capacities. Additional resources include more or faster CPUs, more memory or in case of virtualized instances, more physical shares of the underlying machine. In contrast, horizontal scaling adds more nodes to the overall system.
Both scaling variants have different impacts on the system. Vertical scaling almost directly speeds up the system and rarely needs special application customizations. However, vertical scaling is obviously limited by factors such as cost effectiveness, physical constraints and availability of specialized hardware. Horizontal scaling again requires some kind of inherent distribution within the system. If the system cannot be extended to multiple machines, it could not benefit from this type of scaling. But if the system does support horizontal scaling, it can be theoretically enlarged to thousands of machines. This is the reason why horizontal scaling is important for large-scale architectures. Here, it is common practice to focus on horizontal scaling by deploying lots of commodity systems. Also, relying on low cost machines and anticipating failure is often more economical than high expenses for specialized hardware.
Considering a web server, we can apply both scaling mechanisms. The allocation of more available system resources to the web server process improves its capacities. Also, new hardware can provide speed ups under heavy load. Following the horizontal approach, we setup additional web servers and distribute incoming requests to one of the servers.
Scalability and other Non-functional Requirements
In software engineering, there are several important non-functional requirements for large software architectures. We will consider operational (runtime) requirements related to scalability: high availability, reliability and performance.
A system is available, when it is capable of providing its intended service. High availability is a requirement that aims at the indentured availability of a system during a certain period. It is often denoted as percentiles of uptime, restricting the maximum time to be unavailable.
Reliability is a closely related requirement that describes the time span of operational behavior, often measured as meantime between failures.
Scalability, anticipating increasing load of a system, challenges both requirements. A potential overload of the systems due to limited scalability harms availability and reliability. The essential technique for ensuring availability and reliability is redundancy and the overprovisioning of resources. From a methodical viewpoint, this is very similar to horizontal scaling. However, it is important not to conflate scalability and availability. Spare resources allocated for availability and failover can not be used for achieving scalability at the same time. Otherwise, only one requirement can be guaranteed at once.
Performance of software architectures can have multiple dimensions such as short response times (low latencies) and high throughput along with low utilization. Again, increasing load of an application may affect the requirement negatively. Unless an application is designed with scalability in mind and there are valid scaling options available, performance may degrade significantly under load.
Note that in most web architecture scenarios, all of the requirements mentioned are desirable. However, especially when resources are limited, there must be some kind of trade-off favoring some of the requirements, neglecting others.
Scalability and Concurrency
concurrency is a feature that can make an application scalable.
Summary
concurrency is not just a trait of a web architecture. It is also a mandatory and crucial principle when programming and implementing large-scale web applications in order to utilize hardware capacities to the fullest.
As the vertical scalability of a single node is limited, we must take into account horizontal growth, which also brings in distribution compulsively.
The Quest for Scalable Web Architectures
load-balancing, an established and important mechanism for scalability in web architectures.
Traditional Web Architectures
Cloud Architectures
An Architectural Model for Scalabale Web Infrastructures
Scaling Web Applications
Web Server Architectures for High Concurrency
4.1 Overview 4.2 Server Architectures 4.3 The Case of Threads vs. Events 4.4 Summary
Concurrency Concepts for Applications and Business Logic
5.1 Overview 5.2 Concurrency Based on Threads, Locks and Shared State 5.3 Concurrency via Software Transactional Memory 5.4 Actor-based Concurrency 5.5 Event-driven Concurrency 5.6 Other Approaches and Concurrency Primitives 5.7 Summary
Concurrent and Scalable Storage Backends
6.1 The Challenge of Distributed Database Systems 6.2 Internals of Distributed Database Systems 6.3 Types of Distributed Database Systems 6.4 Summary
Recommendations
a guideline for choosing a decent approach or concept for different scenarios.
7.1 Selecting a Web Server Architecture 7.2 Picking the Right Concurrency Concepts for Application Logic 7.3 Choosing a Storage Backend 7.4 Summary
Discussion Result
- Outlook 9.1 Emerging Web Architecture Trends 9.2 Trends in Programming Languages 9.3 Summary