Subscribe to
Elixir Alchemy
A true alchemist is never done exploring. And neither are we. Sign up for our Elixir Alchemy email series and receive deep insights about Elixir, Phoenix and other developments.
In today’s post, we’ll go over what continuous integration and continuous delivery are, the benefits that come along with employing CI/CD, and some best practices that you should follow. We’ll also explore a wide array of Elixir ecosystem tools that can help you create top-notch CI pipelines. In order to experiment with a handful of the tools that we will be discussing, we’ll use a Git hooks Elixir library to execute our CI/CD validation steps, but on our local machine.
Let’s jump right in!
Continuous integration is the process by which new features and bug fixes are frequently merged into mainline branches. For every change to the code base, the project’s validation suite is executed to ensure it’s up to team standards and doesn’t introduce any regressions.
Continuous delivery is the process by which validated code, once merged, generates a new build artifact. The build artifact with the new code is then automatically deployed to your non-production environments. Note that continuous delivery deploys to non-production environments. The act of deploying to production directly is called, confusingly enough, continuous deployment.
CI and CD are critical tools to have at your disposal as they allow you and your team to move faster and with a higher degree of confidence. They enable you to catch bugs early on and ensure that they don’t impact customers.
With the vocabulary explained, let’s move on to discussing the 12 Factor App. This is a collection of techniques and guidelines that can be used to create sustainable and scalable web services. As the name implies, there are 12 concepts that make up the manifesto. For the purposes of CI/CD, we’ll focus on “Config” and “Codebase”, but I highly recommend reading up on the 12 Factor App if you are unfamiliar with it.
The Codebase section states that a single code repository should contain only a single application. If your entire system is made up of multiple applications, then each of those applications should be contained within their own repositories. This is particularly relevant to CI/CD since, by following this convention, it is very easy to ensure that only the necessary applications are built, tested, and deployed. If multiple applications are contained within the same repository, it becomes a bit more difficult to ascertain which application should be built, tested, and deployed.
The Config section states that any kind of configuration data, credentials and secrets need to be kept out of the code. Instead, these bits of data should be set in the application’s environment via environment variables. This is important from a CI/CD perspective since you will be able to spin up supporting services for testing, and can easily point your application to those services via runtime configuration. You can do this with a wide array of services such as Postgres, Redis and RabbitMQ. Not only will it make your application portable between higher up environments, but it will also make it easy to test.
Before jumping into some Elixir ecosystem tooling, let’s review some best practices that we should adhere to when designing our CI/CD pipelines (for clarification, a CI/CD pipeline is defined as a series of steps that are performed in order to take code from commit to deployment):
Luckily, there are many tools at our disposal that come out of the box with Elixir. Some of these tools include:
mix compile --warnings-as-errors
— Running this will return a non-zero exist status if your code contains any warnings.mix xref unreachable --abort-if-any
— Running this will return a non-zero exit status if your code makes any references to functions/modules that do no exist. If you are using Elixir 1.10+ this Mix task has been deprecated and its functionality has been rolled into mix compile --warnings-as-errors
.mix xref deprecated --abort-if-any
— Running this will return a non-zero exit status if your code leverages any functions that have been marked as deprecated. If you are using Elixir 1.10+ this Mix task has been deprecated and its functionality has been rolled into mix compile --warnings-as-errors
.mix format --check-formatted
— Running this will return a non-zero exist status if any of your source files do not adhere to the format configuration specified in your .formatter.exs
. This helps ensure that the codebase has a uniform look and feel for all team members.mix test
— Elixir has an amazing built-in testing framework called ExUnit. By running the preceding command you can execute all of your project’s tests and the exit status will be non-zero if any tests failed.After you’ve incorporated the aforementioned items into your CI flow, it is time to reach for some community tools. Luckily, the Elixir ecosystem is packed full of great tools! Below is a list of a few of the tools that I use day in and day out:
Once your application has been tested and statically analyzed, you’ll want to deploy it somehow. Below are a few options that are available to you for doing so:
In order to play around with some of the ideas presented here without trying to learn a new CI/CD system, we’ll instead experiment with validating a sample Elixir project using Git hooks. Leveraging Git hooks is a good habit to get into as it will help you validate your code locally before pushing it to your team’s repository. It makes fixing any errors easier, given that you don’t need to dig through logs to figure out why builds failed.
To begin, start by cloning a sample repository that I put together on GitHub:
1 | $ git clone https://github.com/akoutmos/sample_math.git |
Once we have our project cloned locally, we’ll want to open up our mix.exs
file and add the following dependency:
1 2 3 4 5 6 | defp deps do [ ... {:git_hooks, "~> 0.4.0", only: [:test, :dev], runtime: false} ] end |
With that in place, we can now open up our config/config.exs
file and add the following (feel free to omit the mix xref
tasks if you are using Elixir 1.10+):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | ... if Mix.env() != :prod do config :git_hooks, verbose: true, hooks: [ pre_commit: [ tasks: [ "mix clean", "mix compile --warnings-as-errors", "mix xref deprecated --abort-if-any", "mix xref unreachable --abort-if-any", "mix format --check-formatted", "mix credo --strict", "mix doctor --summary", "mix test" ] ] ] end |
With that in place, let’s fetch our new dependency and attempt to commit our code via the terminal:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | $ mix deps.get $ mix git_hooks.install $ git add . $ git commit -m "Added git hooks to sample math project" ↗ Running hooks for :pre_commit ✔ `mix clean` was successful Compiling 1 file (.ex) warning: variable "num_2" is unused (if the variable is not meant to be used, prefix it with an underscore) lib/sample_math.ex:10: SampleMath.sum/2 Compilation failed due to warnings while using the --warnings-as-errors option × pre_commit failed on `mix` ** (exit) 1 lib/mix/tasks/git_hooks/run.ex:175: Mix.Tasks.GitHooks.Run.error_exit/1 lib/mix/tasks/git_hooks/run.ex:128: Mix.Tasks.GitHooks.Run.run_task/3 (elixir) lib/enum.ex:783: Enum."-each/2-lists^foreach/1-0-"/2 (elixir) lib/enum.ex:783: Enum.each/2 lib/mix/tasks/git_hooks/run.ex:63: Mix.Tasks.GitHooks.Run.run/1 (mix) lib/mix/task.ex:331: Mix.Task.run_task/3 (mix) lib/mix/cli.ex:79: Mix.CLI.run_task/2 |
As you may have guessed, there is an issue with our project that requires fixing! Luckily, our commit will not go through until we are able to rectify the issue. I’ll leave the fixing of issues in your capable hands, but as we can see there is a problem with the compile step because of the presence of an unused variable. Our validation steps worked just as expected!
While we only performed the validation steps on our local machine, taking these same validation steps to an actual CI/CD pipeline is a relatively simple task. The Git hooks library that we leveraged allows us to run all of our configured steps via mix git_hooks.run all
. In other words, we can run this command in our CI/CD solution of choice and validate that our code changes pass team standards. The benefit of this is that our CI/CD validation steps can be easily run locally for quick and easy debugging.
Thanks for sticking with me to the end! Hopefully, you learned a thing or two related to CI/CD and how to go about doing it with an Elixir application.
Guest author Alex Koutmos is a Senior Software Engineer who writes backends in Elixir, frontends in VueJS and deploys his apps using Kubernetes. When he is not programming or blogging he is wrenching on his 1976 Datsun 280z.
P.S. If you’d like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!