Hands-On Docker for Microservices with Python

Preface

For the technical aspects, we will use well-tailored tools, including the following:

• Python, to implement RESTful web services
• Git source control, to track the changes in an implementation, and GitHub, to share those changes
• Docker containers, to standardize the operation of each of the microservices
• Kubernetes, to coordinate the execution of multiple services
• Cloud services, such as Travis CI or AWS, to leverage existing commercial solutions to problems

We will also cover practices and techniques for working effectively in a microservice oriented environment, the most prominent being the following:

• Continuous integration, to ensure that services are of a high quality and ready to be deployed
• GitOps, for handling changes in infrastructure
• Observability practices, to properly understand what is happening in a live system
• Practices and techniques aimed at improving teamwork, both within a single team and across multiple teams

Chapter 1: Making the Move – Design, Plan, and Execute

The traditional monolith approach and its problems

Python Django application for our monolith example.

The monolith example can be found at: https://github.com/PacktPublishing/Hands-On-Docker-for-Microservices-with-Python/tree/master/Chapter01/Monolith

In the context of web applications, that means creating deployable code that can be replicated so that requests can be directed to any of the deployed copies.

• one or more servers (physical boxes, virtual machines, and cloud instances such as EC2 and more) running a web server application (such as NGINX or Apache) to direct requests directed to HTTP port 80 or HTTPS port 443 toward one or more Python workers (normally, through the WSGI protocol), run by mod_wsgi—https://github.com/GrahamDumpleton/mod_wsgi (Apache only), uWSGI, GNUnicorn, and so on.

problems:

• The code will increase in size, the complexity naturally tends to increase, making it more difficult to change the code in certain ways.
• Inefficient utilization of resources
• Issues with development scalability, the development team grows, once several teams are working on the same code base, the probability of clashing will increase.
• Deployment limitations
• Interdependency of technologies
• A bug in a small part of the system can bring down the whole service.

The characteristics of a microservices approach

A system following a microservices architecture is a collection of loosely coupled specialized services (一组松散耦合的专用服务) that work in unison to provide a comprehensive service.

• A collection of specialized services, meaning that there are different, well defined modules.
• Loosely coupled, meaning that each of the microservices can be independently deployed.
• That work in unison—each microservice is capable of communicating with others.
• To provide a comprehensive service, because our microservice system will need to replicate the same functionalities that were available using a monolith approach. There is an intent behind its design.

Note that there may be multiple workers per microservice.

There are several advantages and implications to this architecture:

• If the communication between microservices is done through a standard protocol, each microservice can be programmed in different languages.
• Better resource utilization—if Microservice A requires more memory, we can reduce the number of worker copies.
• Each individual service is smaller and can be dealt with independently. That means fewer lines of code to maintain, faster builds, and a simpler design, with less technical debt to maintain.
• Some services can be hidden from external access. In some scenarios, that can improve security, reducing the attack surface area for sensitive data or services.
• As the systems are independent, a stability problem in one won’t completely stop the system.
• Each service can be maintained independently by different developers.

Docker containers

Containers are a packetized bundle of software that encapsulates everything that is required to run, including all dependencies. It only requires a compatible OS kernel to run autonomously.

Working with Docker containers has two steps. First, we build the container, executing layer after layer of changes on the filesystem, such as adding the software and configuration files that will be executed. Then, we execute it, launching its main command.

An important factor for dealing with containers is that containers should be stateless (Factor VI—https://12factor.net/processes). Any state needs to be stored in a database and each container stores no persistent data.

Another advantage of Docker is the availability of a lot of ready-to-use containers.

Container orchestration and Kubernetes

Though Docker presents on how to deal with each of the individual microservices, we will need an orchestrator to handle the whole cluster of services.

Parallel deployment and development speed

The single most important element is the capacity to deploy independently. The objective is to increase the number of deployments and the speed of each of them.

The microservice architecture is strongly related to Continuous Integration and Continuous Deployment principles.

As deploying a microservice should be transparent for dependent services, special attention should be paid to backward compatibility. Some changes will need to be escalated and coordinated with other teams to remove old, incorrect functionality without interrupting the system.

Challenges and red flags

Systems get started as monoliths, as it is simpler and allows for quicker iteration in a small code base. As more and more developers get involved, the advantages of a monolith start to become less evident, and the need for long- term strategy and structure becomes more important. More structure doesn’t necessarily mean moving toward a microservice architecture. A great deal can be achieved with a well- architected monolith.

Moving to microservices also has its own problems:

• Migrating to microservices requires a lot of effort, actively changing the way an organization operates, and a big investment until it starts to pay off.
• Debugging a request that moves across services is more difficult than a monolithic system. Monitoring the life cycle of a request is important and some subtle bugs can be difficult to replicate and fix in development.
• A bad division line can make two services tightly coupled, not allowing independent deployment. A red flag in that means almost any change to one service requires a change in the other, even if, normally, it could be done independently.
• Solid communication between teams is required to allow consensus and the reuse of common solutions. Be careful when introducing (引入) shared code across services. If the code grows, it will make services dependent on each other. This can reduce the independence of the microservices.
• The best approach is to create self-documenting services. how to use tools to allow documenting how to use a service with minimal effort.
• Each call to another service, such as internal microservices calling each other, can increase the delay of responses, as multiple layers will have to be involved. This can produce latency problems, with external responses taking longer. They will also be affected by the performance and capacity of the internal network connecting the microservices.

Analyzing the current system

The objective of this phase is to determine whether a change to microservices will actually be beneficial and to get an initial idea of what microservices will be the result of the migration.

• Searching: Search is also read-only, so it may be a good idea to detach.
• Authentication: Splitting it will allow us to keep on track for new security issues and have a team specifically dealing with those issues.
• the new microservice will be the frontend, which will be served as static, precompiled code, and calls a backend API to render the pages.

These are just some ideas, which will need to be discussed and evaluated. What are the specific pain points for your monolithic app? What is the roadmap and the strategic future? What are the most important points and features for the present or the future?

The system will be divided into the following modules:

• Users backend: This will have the responsibility for all authentication tasks and keep information about the users. It will store its data in the database.
• Thoughts backend: This will create and store thoughts.
• Search backend: This will allow searching thoughts.
• A proxy that will route any request to the proper backend. This needs to be externally accessible.
• HTML frontend: This will replicate the current functionality. This will ensure that we work in a backward-compatible way and that the transition can be made smoothly.
• Allowing clients to access the backends will allow the creation of other clients than our HTML frontend. A dynamic frontend server will be created, and thereare talks with an external company to create a mobile app.
• Static assets: A web server capable of handling static files. This will serve the styling for the HTML frontend and the index files and JavaScript files for the dynamic frontend.

Preparing and adapting by measuring usage

The ability to know how a live system is working is called observability. The main tools for it are metrics and logs. The problem you’ll find is that they will normally be configured to reflect external requests and give no information about internal modules.

Remember that the most important element for the long-term success of the move to microservices is to allow teams to be independent. If you split across modules that constantly need to be changed in unison, deployments won’t be truly independent, and, after the transition, you’ll be forced to work with two tightly coupled services.

Strategic planning to break the monolith

Being realistic, the company’s business activities will not stop. That’s why a plan should be in place to allow a smooth transition between one state and the other.

This is known as the strangler pattern (https://docs.microsoft.com/en-us/azure/architecture/patterns/strangler)—replacing parts of a system gradually until the old system is “strangled” and can be removed safely.

The replacement approach

This black-box approach completely replaces existing functionality coding with an alternative from scratch. Once the new code is ready, it gets activated and the functionality in the old system is deprecated.

The divide approach

If the system is well structured, maybe some parts of it can be cleanly split into its own system, maintaining the same code.

• Copy the code into its own microservice and deploy it.
• The old calling system is using the old embedded code.
• Migrate a call and check that the system is working fine.
• Iterate until all old calls are migrated to the new system.
• Delete the divided code from the old system.

Change and structured approach

These three approaches can be combined to generate full migration. In this phase of the project, the objective is to have a clear roadmap.

• As a prerequisite, a load balancer will need to be in front of the operation. Changing the configuration of this element, we will be able to route the requests toward the old monolith or any new microservice.

• A static web server, it will be deployed as an independent microservice.

• The code for authentication will be replicated in a new service.It will use a RESTful API to log in and generate a session, and to log out.

  Generate a package, shared across the externally faced microservices, which will allow checking to see whether a session has been generated with our own service. This will be achieved by signing the session cryptographically and sharing the secret across services.

  The Users Backend needs to be able to allow authentication using OAuth 2.0 schema, which will allow other external services, not based on web browsers, to authenticate and operate, for example, a mobile app.

• The new API will be added externally to the load balancer and promoted as externally accessible.

Executing the move

we want to keep the services running and not have outages that interrupt our business.

The single most important idea during this phase is backward compatibility.

Web services’ best friend – the load balancer

For our purposes, a group of old web services that are behind a load balancer can add one or more replacement services that are backward compatible, without interrupting the operation. The new service replacing the old one will be added in small numbers (maybe one or two workers) to split the traffic in a reasonable configuration, and ensure that everything is working as expected. After the verification, replace it completely by stopping sending new requests to the old services, draining them, and leaving only the new servers.

Any web server capable of acting in reverse proxy mode, such as NGINX, is capable of working as a load balancer, but, for this task, probably the most complete option is HAProxy (http://www.haproxy.org/).

Because a load balancer is a single point of failure, you’ll need to load balance your load balancer. The easiest way of doing it is creating several identical copies of HAProxy, as you’d do with any other web service, and adding a cloud provider load balancer on top.

Keeping the balance between new and old

The pilot phase – setting up the first couple of microservices The consolidation phase – steady migration to microservices The final phase – the microservices shop

Chapter 2: Creating a REST Service with Python

Technical requirements

Analyzing the Thoughts Backend microservice

Understanding the security layer

This security layer will come in the shape of a header. This header will contain information that is signed by the user backend, verifying its origin. It will take the form of a JSON Web Token (JWT), https://jwt.io/introduction/, which is a standard for this purpose.

A JWT is not the only possibility for the token, and there are other alternatives such as storing the equivalent data in a session cookie or in more secure environments using similar modules such as PASETO (https://github.com/paragonie/paseto). Be sure that you review the security implications of your system, which are beyond the scope of this book.

With an understanding of how the authentication system is going to work, we can start designing the API interface.

Designing the RESTful API

It’s always good to have a brief reminder about REST before starting an API design, so you can check https://restfulapi.net/ for a recap.

Specifying the API endpoints

Defining the database schema

Working with SQLAlchemy

The first approach is well represented by the Python database API specification (PEP 249—https://www.python.org/dev/peps/pep-0249/), which is followed by all major databases, such as psycopg2 (http://initd.org/psycopg/) for PostgreSQL.

For the second approach, the best-known example is probably the Django ORM (https://docs.djangoproject.com/en/2.2/topics/db/).

SQLAlchemy (https://www.sqlalchemy.org/) is quite flexible and can work on both ends of the spectrum. it can take an existing, complicated legacy database and map it, allowing you to perform simple operations easily and complicated operations in exactly the way you want.

Implementing the service

Introducing Flask-RESTPlus

Flask-RESTPlus adds some very interesting capabilities that allow for good developing practices and speed of development.

We will use the connector for SQLAlchemy, Flask-SQLAlchemy (https://flask-sqlalchemy.palletsprojects.com/en/2.x/). Its documentation covers most of the common cases, while the SQLAlchemy documentation is more detailed and can be a bit overwhelming. To run the tests, the pytest-flask module (https://pytest-flask.readthedocs.io/en/latest/) creates some fixtures ready to work with a Flask application. We will talk more about this in the Testing the code section.

Handling resources

Parsing input parameters

Serializing results

Performing the action

Authenticating the requests

Note that we are using a private/public key schema, instead of a symmetric key schema, to encode and decode the tokens. This means that the decoding and encoding keys are different.

In our microservice structure, only the signing authority requires the private key. This increases the security as any key leakage in other services won’t be able to retrieve a key capable of signing bearer tokens. We’ll need to generate proper private and public keys, though.

To generate a private/public key, run the following command:

1

$ openssl genrsa -out key.pem 2048

Then, to extract the public key, use the following:

1

$ openssl rsa -in key.pem -outform PEM -pubout -out key.pub

Reading them in text format will be enough to use them as keys for encoding/decoding the JWT token:

1
2
3
4

>> with open('private.pem') as fp:
>> .. private_key = fp.read()
>> generate_token_header('peter', private_key)
'Bearer <token>'

inject this secret using an environment variable, and in Chapter 11, Handling Change, Dependencies, and Secrets in the System, we’ll see how to properly deal with secrets in production environments.

Testing the code

Defining the pytest fixtures

Understanding test_token_validation.py

test_thoughts.py

Chapter 3: Build, Run, and Test Your Service Using Docker

Technical requirements

Building your service with a Dockerfile

Executing commands

Understanding the Docker cache

This also means that an image will never get smaller in size, adding a new layer even if the layer removes data, as the previous layer is still stored on the disk.

Keeping your containers small is quite important. In any Docker system, the tendency is to have a bunch of containers and lots of images. Big images for no reason will fill up repositories quickly. They’ll be slow to download and push, and also slow to start, as the container is copied around in your infrastructure.

There are several practices for keeping your images small. Other than being careful to not install extra elements, the main ones are creating a single, complicated layer that installs and uninstalls, and multi-stage images. Multi-stage Dockerfiles are a way of referring to a previous intermediate layer and copying data from there. Check the Docker documentation (https://docs.docker.com/develop/develop-images/multistage-build/).

Compilers, in particular, tend to get a lot of space. When possible, try to use precompiled binaries. You can use a multi-stage Dockerfile to compile in one container and then copy the binaries to the running one.

You can learn more about the differences between the two strategies in this article: https:/ /pythonspeed.com/articles/smaller-python-docker-images/. A good tool to analyze a particular image and the layers that compose it is dive (https://github.com/wagoodman/dive). It will also discover ways that an image can be reduced in size.

Building a web service container

Configuring uWSGI

Refreshing Docker commands

Operating with an immutable container

Testing the container

Creating a PostgreSQL database container

Configuring your service

Deploying the Docker service locally

Pushing your Docker image to a remote registry

Obtaining public images from Docker Hub

Using tags

Pushing into a registry

Chapter 4: Creating a Pipeline and Workflow

Technical requirements Understanding continuous integration practices Producing automated builds Knowing the advantages of using Docker for builds Leveraging the pipeline concept Branching, merging, and ensuring a clear main build

Configuring Travis CI Adding a repo to Travis CI Creating the .travis.yml file Working with Travis jobs Sending notifications

Configuring GitHub Pushing Docker images from Travis CI Setting the secret variables Tagging and pushing builds Tagging and pushing every commit

Chapter 5: Using Kubernetes to Coordinate Microservices

Technical requirements Defining the Kubernetes orchestrator Comparing Kubernetes with Docker Swarm

Understanding the different Kubernetes elements Nodes Kubernetes Control Plane Kubernetes Objects

Performing basic operations with kubectl Defining an element Getting more information Removing an element

Troubleshooting a running cluster

Chapter 6: Local Development with Kubernetes

Technical requirements Implementing multiple services Describing the Users Backend microservice Describing the Frontend microservice Connecting the services

Configuring the services Configuring the deployment Configuring the service Configuring the Ingress

Deploying the full system locally Deploying the Users Backend Adding the Frontend

Chapter 7: Configuring and Securing the Production System

Technical requirements Using Kubernetes in the wild Creating an IAM user

Setting up the Docker registry Creating the cluster Creating the Kubernetes cluster Configuring the cloud Kubernetes cluster Configuring the AWS image registry Configuring the usage of an externally accessible load balancer Deploying the system

Using HTTPS and TLS to secure external access Being ready for migration to microservices Running the example

Deploying a new Docker image smoothly The liveness probe The readiness probe Rolling updates

Autoscaling the cluster Creating a Kubernetes Horizontal Pod Autoscaler Deploying the Kubernetes metrics server Configuring the resources in deployments Creating an HPA Scaling the number of nodes in the cluster Deleting nodes Designing a winning autoscaling strategy

Chapter 8: Using GitOps Principles

Technical requirements Understanding the description of GitOps Managing configuration Understanding DevOps Defining GitOps

Setting up Flux to control the Kubernetes cluster Starting the system Configuring Flux

Configuring GitHub Forking the GitHub repo Adding a deploy key Syncing Flux

Making a Kubernetes cluster change through GitHub Working in production Creating structure Using GitHub features Working with tags

Chapter 9: Managing Workflows

Understanding the life cycle of a feature Features that affect multiple microservices Implementing a feature

Reviewing and approving a new feature Reviewing feature code Approving releases

Setting up multiple environments Scaling the workflow and making it work Reviewing and approving is done by the whole team Understanding that not every approval is the same Defining a clear path for releases Emergency releases Releasing frequently and adding feature flags Using feature flags Dealing with database migrations

Chapter 10: Monitoring Logs and Metrics

Technical requirements Observability of a live system Understanding logs Understanding metrics

Setting up logs Setting up an rsyslog container Defining the syslog pod log-volume syslog container The front rail container Allowing external access Sending logs Generating application logs Dictionary configuration Logging a request ID Logging each request Searching through all the logs Detecting problems through logs Detecting expected errors Capturing unexpected errors Logging strategy Adding logs while developing

Setting up metrics Defining metrics for the Thoughts Backend Adding custom metrics Collecting the metrics Plotting graphs and dashboards Grafana UI Querying Prometheus Updating dashboards

Being proactive Alerting Being prepared

Chapter 11: Handling Change, Dependencies, and Secrets in the System

Technical requirements Understanding shared configurations across microservices Adding the ConfigMap file Using kubectl commands Adding ConfigMap to the deployment Thoughts Backend ConfigMap configuration Users Backend ConfigMap configuration Frontend ConfigMap configuration

Handling Kubernetes secrets Storing secrets in Kubernetes Creating the secrets Storing the secrets in the cluster Secret deployment configuration Retrieving the secrets by the applications

Defining a new feature affecting multiple services Deploying one change at a time Rolling back the microservices

Dealing with service dependencies Versioning the services Semantic versioning Adding a version endpoint Obtaining the version Storing the version in the image Implementing the version endpoint Checking the version Required version The main function Checking the version

Chapter 12: Collaborating and Communicating across Teams

Keeping a consistent architectural vision Dividing the workload and Conway’s Law Describing Conway’s Law Dividing the software into different kinds of software units Designing working structures Structuring teams around technologies Structuring teams around domains Structuring teams around customers Structuring teams around a mix

Balancing new features and maintenance Regular maintenance Understanding technical debt Continuously addressing technical debt Avoiding technical debt

Designing a broader release process Planning in the weekly release meeting Reflecting on release problems Running post-mortem meetings