Learning Erlang, and the OTP framework in particular, has given me a better understanding of distributed systems and their fundamental building blocks. Before this, my experience with distributed systems was solely in the realm of Kubernetes, which we use at work for deploying scalable, distributed web services.
I was introduced to Erlang through Elixir. The Phoenix framework leverages Elixir to build performant, functional web applications and it often comes up in the Rails community as a better-performing alternative to Ruby. Elixir depends on the decades of development behind the Erlang VM and in many ways is just syntactic sugar on top of Erlang/OTP. For that reason I wanted to get a better understanding of the underlying technologies so I picked up the book you see above, Erlang And OTP In Action.
Building Blocks of a Distributed System
In Erlang one can take distributed computing for granted. To start off, all code within Erlang runs in processes. You can look at a process in Erlang and Elixir as a basic building block of the language much like one might view a class within an object-oriented language. An application is thus a tree of interconnected processes. Because processes are so native to Erlang, all of these individual processes can automatically be spread across the available cores of the system running them. And because processes are each isolated environments which depend on a built-in system of message passing, the Erlang environment can treat a distributed network of systems the same way it treats a single one. This may sound like magic, but in reality it is leveraging the work that experts have put into building possibly the most scalable environment currently available for development.
Although Kubernetes may immediately seem to solve a very different problem than Erlang (for one, Kubernetes isn't a programming language), there are some important similarities when it comes to the structure of its distributed implementation. Much like Erlang uses the basic building block of a process, Kubernetes uses the container, or more generally the pod, as a base. Every node in a kube cluster can run one or more pods, just the same way that every node in an Erlang cluster can run one or more processes.
Comparing APIs
If we want to discuss implementation details it would be helpful to introduce OTP. OTP is a built-in framework within Erlang for building applications. Building on top of the process-based system, OTP (Open Telecom Platform) offers a further abstraction known as a GenServer. GenServer adds message sending and receiving, storage, TCP, and a ton of other functionality to a process, allowing you to spin up a server that reacts to input and performs useful functions very quickly. Furthermore, OTP adds the concept of a supervisor, a parent process that manages the lifecycle of the child GenServers. In cases where a child process fails unexpectedly, the supervisor can restart that individual process without having it affect the other process environments.
Kubernetes leverages many of the concepts we just discussed. To name a few, Kubernetes has an ApiServer running on the master which acts as the supervisor for its cluster. Like OTP, this API will ensure that the pods which are running are healthy based on their underlying deployment (very similar to a child specification in Erlang). And if a pod goes down unexpectedly, it will be restarted based on the restart strategy; same with OTP!
To wrap up, it's evident that much of the work that went into Erlang has inspired the work of Kubernetes. You can look at Kubernetes as a language-agnostic implementation of the same kind of distributed system that's found in the Erlang environment. It's inspiring to see the way open source projects have built, incremented on and in many cases directly inspired each other over the years.