While Elixir is frequently associated with web development, this is not where its capabilities end. As a general-purpose language, it can be used for virtually anything. You don't have to take my word for it — projects such as Nerves, Nx, Scenic, or LiveBook speak for themselves.
But today, we will focus on something different: writing a command-line application in Elixir and preparing it for distribution.
Before we get going, let's touch on why we'd write a CLI app and what we'll cover in this tutorial.
Why Write a CLI App in Elixir?
CLI (command-line interface) applications have a wide variety of purposes. You probably already use some of
them often — think of git, asdf, or even mix.
In our daily dev life, we usually write such applications to automate
things that are otherwise tedious — formatting code, preparing deployments, running backups, synchronizing
data, or... generating code. Yes, think about creating language-specific files from .proto definitions. You probably
would use protoc or buf for that. Both of them are, in fact, CLI applications.
In this tutorial, we are going to build something a lot simpler. Let's set our requirements as follows: a script should take a path to a directory as an argument. We will (non-recursively) get all the files from there and calculate their mean size.
During this journey, you will also learn about tools that will help you achieve your goal. This kind of endeavor
deserves a special name, so we will call it MFSC — Mean File Size Calculator.
ElixirScript: Let's Start Simple
Elixir already has a tool that is enough to get started. It's called ElixirScript and is really simple to use:
you write code in a file with an .exs extension, and then you run it with elixir my_file.exs. I'm sure you have
seen some .exs files before — for example, in Phoenix's config directory or in tests. These are examples of
ElixirScript.
So, without further ado, let's create the first version of our MFSC:
defmodule MFSC do def call do [directory] = System.argv() with true <- File.dir?(directory), {:ok, files} <- list_files(directory) do Enum.reduce(files, 0, fn file, size -> size + calculate_file_size(file) end) |> Kernel.div(length(files)) |> IO.puts() end end defp list_files(directory) do case File.ls(directory) do {:ok, files} -> files = files |> Enum.map(&Path.join(directory, &1)) |> Enum.reject(&File.dir?/1) {:ok, files} error -> error end end defp calculate_file_size(file) do file |> File.stat!() |> Map.get(:size) end end MFSC.call()
We define a module called, unsurprisingly, MFSC with a call/0 function. This lists the files in the directory, then
iterates on them to find their size before finally calculating the average. The only piece relevant to a command-line script is this line:
[directory] = System.argv()
System.argv/0 here returns a list of arguments passed to the script. We optimistically assume it's always just one
argument and assign it to the directory variable.
That's it. If you run the script with an existing directory as an argument, it will return the mean length in bytes:
> elixir mfsc.exs ~/some_directory 1184 > elixir mfsc.exs ~/none_existing_dir >
As you can see, for an incorrect input (a directory that does not exist), it will just return nothing. Let's fix
it by adding else to the with statement.
with true <- File.dir?(directory), {:ok, files} <- list_files(directory) do Enum.reduce(files, 0, fn file, size -> size + calculate_file_size(file) end) |> Kernel.div(length(files)) |> IO.puts() else _ -> IO.puts("Invalid input") System.halt(1) end
Note that I didn't only add an error message but also a System.halt(1) call. This is to return a non-zero
(i.e., non-success) exit code from a script (you can check the exit code of the last run command with echo $?).
It is important to do so to be a 'good CLI land citizen.'
Why? Command-line applications often do not work on their own but are chained together or called by other programs. An exit code helps to detect if a given part of a chain was run successfully (and we can expect the output to be in some predefined format) or not (in which case, we probably shouldn't even try to parse output).
We have a working script, except for one small issue. When you run it on an empty directory, it will crash because of division by zero:
** (ArithmeticError) bad argument in arithmetic expression: div(0, 0) :erlang.div(0, 0) mfsc.exs:8: MFSC.call/0 (elixir 1.13.3) lib/code.ex:1183: Code.require_file/2
That's not nice. But what should we do on an empty directory? It's not obvious. We could return zero. Or alternatively, halt the application with a non-zero exit code.
Or maybe — just a thought — we should leave the choice to the user?
OptionParser: More Robust Input Handling in Elixir
Remember how we assumed that there would always be only one input argument to our script? We're about to ditch this assumption. We want to introduce an option for the user to decide what should happen when an empty directory is found.
By default, we want to return an error, but there will be a flag to alter this behavior:
> elixir mfsc.exs /my/empty/dir Empty directory > echo $? 10 > elixir mfsc.exs /my/empty/dir --allow-empty 0 > echo $? 0
Fortunately, Elixir has a tool readily waiting for us in its standard library: OptionParser.
Let's first play with it a bit in iex:
iex(1)> OptionParser.parse(["/my/empty/dir", "--allow-empty"], strict: [allow_empty: :boolean]) {[allow_empty: true], ["/my/empty/dir"], []} iex(2)> OptionParser.parse(["/my/empty/dir"], strict: [allow_empty: :boolean]) {[], ["/my/empty/dir"], []}
The first argument to the OptionParser.parse/2 function is a list of arguments that we normally get from
System.argv/0. In this case, we are simulating it manually.
The second one is the definition of our expected input.
It is a keyword list with three possible keys: strict, aliases, and switches. For now, we will concentrate on strict
as it is the basic one. In this example, we defined an expectation of one argument of the boolean type.
The return value is a tuple of three elements: the first is a list of parsed options, the second is for other (positional) arguments, and the third one is for errors. Let's see some more examples:
iex(3)> OptionParser.parse(["/my/empty/dir", "/tmp/second/dir"], strict: [allow_empty: :boolean]) {[], ["/my/empty/dir", "/tmp/second/dir"], []} iex(4)> OptionParser.parse(["/my/empty/dir", "--allow-empty=3"], strict: [allow_empty: :boolean]) {[], ["/my/empty/dir"], [{"--allow-empty", "3"}]}
In the first example, we have two positional arguments and no parsed options. In the second, we violate
the constraint that --allow-empty should be a boolean switch by passing the number 3 to it. As a result, it doesn't
appear in parsed options but instead lands in errors.
Knowing how OptionParser works, we can add it to our app:
defmodule MFSC do def call do parsed = System.argv() |> OptionParser.parse(strict: [allow_empty: :boolean]) with {options, [directory], []} <- parsed, true <- File.dir?(directory), {:ok, files} <- list_files(directory) do Enum.reduce(files, 0, fn file, size -> size + calculate_file_size(file) end) |> print_size(files, options) else _ -> IO.puts("Invalid input") System.halt(1) end end defp print_size(0, _, options) do case Keyword.get(options, :allow_empty) do true -> IO.puts("0") _ -> IO.puts("Empty directory") System.halt(1) end end defp print_size(total_size, files, _) do IO.puts(div(total_size, length(files))) end # rest of the code does not change end
Nice! We support our first input option.
There is still a lot of work to do to make this app's interface friendlier, but I'll leave it up to you at this point. Some ideas on what you could do include:
- Returning different error messages and exit codes for various errors (non-existing directory vs. empty directory
without
--allow-emptyvs. some random access error) - An option to print a result in a different format than just the number of bytes, for example:
--output-format [B|K|M|smart]
- Allowing recursive counting, i.e., also calculate file sizes in subdirectories and their subdirectories. Beware of symbolic links leading to cycles!
The Limitations of ElixirScript
So far, we've been using ElixirScript to create our CLI apps, but as our code grows, you will soon notice some limitations in this approach.
Splitting code into multiple files is inconvenient, and you cannot really use all
the awesome packages published at hex. You might overcome this by using Mix.install/2 inline:
Mix.install([ {:ecto_sql, "~> 3.8"} ]) # some code using Ecto
There is, however, one last problem: to run your application, people need to have Elixir installed on their computers. This is what makes ElixirScript a nice tool to run something along an existing larger Elixir application (like tests in a Phoenix project), but not for our use case.
Elixir is still not a standard language. Some system repositories might have really old versions of the language. Generally, the fewer dependencies the user needs to install themselves, the more likely they will actually try out your application.
How can we do better, then?
Using Escript for Your Elixir App
Escript is a built-in way to distribute whole Elixir applications as a single-file executable. This is, of course, one level better than using ElixirScript. Let's look closely at how to include it in our application.
To use escript, we need to have a regular mix project, not just a single file. But this is fine, as we want to split our
application into multiple files and use dependencies anyway. Let's create a project then:
mix new mfsc
This will create a bunch of files in an mfsc directory. Among them is lib/mfsc.ex, where
we will put the code we built in the previous section.
Theoretically, you should be able to run it via mix run now,
but here's an unpleasant surprise: mix run resets System.argv/0, and you'll always get an empty array as the input. We are not
going to focus on hacking this. Instead, we will introduce escript right away.
The first step: remove MFSC.call() from the bottom of the file — we only want to define a module for now.
Now we will add support for escript to our mix.exs file. In the project function that returns a keyword list of the
project configuration, add a relevant line for escript:
def project do [ app: :mfsc, # bunch of other things escript: [main_module: MFSC] ] end
We are almost there. Now we need to refactor the call/0 function to the main/1 function accepting one argument — args — which will be the equivalent of System.argv/0.
defmodule MFSC do def main(args) do parsed = args |> OptionParser.parse(strict: [allow_empty: :boolean]) # rest of the file remains unchanged
That's all, folks! We build the escript file with mix escript.build. This will create an mfsc executable file in the
project's root directory. We can call it like a regular executable:
> mix escript.build Generated escript mfsc with MIX_ENV=dev > ./mfsc ~/empty_dir Empty directory > ./mfsc ~/empty_dir --allow-empty 0
This is a huge step forward from ElixirScript — and the user does not need to have Elixir installed on their machine. Note, however, that they still need to have Erlang. On the other hand, this is a much more common dependency than Elixir, and chances are they already have it. In the worst-case scenario, it's just one thing less to install.
If that's not enough, we can improve things even further.
Bakeware and Burrito for Elixir
Bakeware is a relatively new project (created in mid-2020) aiming to
create dependency-less executables from Elixir code. Let's try it with MFSC. Bakeware leverages Elixir releases, so
we need to add support for them to mix.exs. On top of that, we need the bakeware dependency and to add our entry
module to application. The complete mix.exs should look like this:
defmodule Mfsc.MixProject do use Mix.Project def project do [ app: :mfsc, version: "0.1.0", elixir: "~> 1.13", start_permanent: Mix.env() == :prod, deps: deps(), escript: [main_module: MFSC], releases: [ # here we define cli release cli: [ steps: [:assemble, &Bakeware.assemble/1] ] ] ] end def application do [ extra_applications: [:logger], mod: {MFSC, []} # adding our entry module ] end defp deps do [ {:bakeware, "~> 0.2.4"} # bakeware dependency ] end end
After that, we need to adjust our MFSC module by adding Bakeware scripting support:
defmodule MFSC do use Bakeware.Script @impl Bakeware.Script def main(args) do # rest of the file
And with that in place, we just need to run mix release cli. This will take some time, but we end up with our standalone CLI application with
a _build/dev/rel/bakeware/cli file. Let's take it for a ride!
> _build/dev/rel/bakeware/cli ~/empty_dir Empty directory > _build/dev/rel/bakeware/cli ~/empty_dir --allow-empty 0
This definitely looks familiar. However, you should know one thing — these files will only work on a similar operating system as the one they were built on. For example, I created mine on Linux, and when I sent it to another Linux machine, it worked without any issues. But it did not work on Intel Mac, not to mention M1 Mac.
We might try to address this issue with an even newer project than Bakeware. It's called
Burrito and the first public commit is just from October 2021. It
promises cross-platform standalone releases and is aimed at CLI applications. Burrito has a few additional dependencies,
namely Zig and xz. When we install them, we add another release to mix.exs (remember to add the burrito dependency too!):
burrito_cli: [ steps: [:assemble, &Burrito.wrap/1], burrito: [ targets: [ macos: [os: :darwin, cpu: :x86_64], macos_m1: [os: :darwin, cpu: :aarch64], linux: [os: :linux, cpu: :x86_64], windows: [os: :windows, cpu: :x86_64] ], ] ]
Now, run the release process:
mix release burrito_cli
After a while, you should have the executables for different operating systems waiting under the burrito_out directory,
for example, burrito_out/burrito_cli_linux. Note that Burrito is still a work in progress, and you may experience some
bumps on the road. For me, at the time of writing this article, building for macOS from Linux did not work.
If you want to learn more about Burrito, there's a great talk by its author, Digit, from the EPEX conference. It's called Wrap your app in a BEAM burrito! In this talk, you'll learn why the project was born and the main hurdles along the way, followed by a live coding session.
The Downsides of Using Elixir for CLI Applications
Now that we've seen how easy it is to build a standalone CLI application in Elixir, let me warn you about one thing: the startup time will be slow. In my tests, it always takes about half a second.
This is not a problem for long-running
or interactive applications but might be a showstopper if you need a small tool to execute multiple times — for example, the find command:
find . -maxdepth 1 -type f -exec ./my_elixir_app {} \;
The startup time of half a second is multiplied by the number of files, which may add up to quite significant delays. You can experience this pain in the real world when you generate protobuf files for Elixir with protobuf-elixir.
Should I Use Elixir for CLI Apps?
If some of the problems outlined in the previous sections of this post do not scare you or do not apply to your use case, by all means use Elixir for CLI applications. It can be a lot of fun, especially combined with its concurrency abilities.
I personally did in my simple stress-tester application, which spawns multiple processes and attempts to flood a given URL with requests, gathering statistics on how it goes. And I used it for work more than once, where a full-blown solution was not required.
Wrapping Up
In this post, we built a simple CLI application using ElixirScript, learned to use OptionParser, then released it using
escript as an executable, which only requires Erlang to run.
After that, we looked into creating more standalone releases with Bakeware or Burrito. Finally, we explored some downsides of choosing Elixir for certain types of CLI tools.
I hope you found this a good starting point for writing a CLI app in Elixir.
Happy coding!
P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!
