elixir

# Write a Standalone CLI Application in Elixir

Paweł Świątkowski on

While Elixir is frequently associated with web development, this is not where its capabilities end. As a general-purpose language, it can be used for virtually anything. You don't have to take my word for it — projects such as Nerves, Nx, Scenic, or LiveBook speak for themselves.

But today, we will focus on something different: writing a command-line application in Elixir and preparing it for distribution.

Before we get going, let's touch on why we'd write a CLI app and what we'll cover in this tutorial.

## Why Write a CLI App in Elixir?

CLI (command-line interface) applications have a wide variety of purposes. You probably already use some of them often — think of git, asdf, or even mix.

In our daily dev life, we usually write such applications to automate things that are otherwise tedious — formatting code, preparing deployments, running backups, synchronizing data, or... generating code. Yes, think about creating language-specific files from .proto definitions. You probably would use protoc or buf for that. Both of them are, in fact, CLI applications.

In this tutorial, we are going to build something a lot simpler. Let's set our requirements as follows: a script should take a path to a directory as an argument. We will (non-recursively) get all the files from there and calculate their mean size.

During this journey, you will also learn about tools that will help you achieve your goal. This kind of endeavor deserves a special name, so we will call it MFSC — Mean File Size Calculator.

## ElixirScript: Let's Start Simple

Elixir already has a tool that is enough to get started. It's called ElixirScript and is really simple to use: you write code in a file with an .exs extension, and then you run it with elixir my_file.exs. I'm sure you have seen some .exs files before — for example, in Phoenix's config directory or in tests. These are examples of ElixirScript.

So, without further ado, let's create the first version of our MFSC:

defmodule MFSC do
def call do
[directory] = System.argv()

with true <- File.dir?(directory),
{:ok, files} <- list_files(directory) do
Enum.reduce(files, 0, fn file, size -> size + calculate_file_size(file) end)
|> Kernel.div(length(files))
|> IO.puts()
end
end

defp list_files(directory) do
case File.ls(directory) do
{:ok, files} ->
files =
files
|> Enum.map(&Path.join(directory, &1))
|> Enum.reject(&File.dir?/1)

{:ok, files}

error ->
error
end
end

defp calculate_file_size(file) do
file
|> File.stat!()
|> Map.get(:size)
end
end

MFSC.call()

We define a module called, unsurprisingly, MFSC with a call/0 function. This lists the files in the directory, then iterates on them to find their size before finally calculating the average. The only piece relevant to a command-line script is this line:

[directory] = System.argv()

System.argv/0 here returns a list of arguments passed to the script. We optimistically assume it's always just one argument and assign it to the directory variable.

That's it. If you run the script with an existing directory as an argument, it will return the mean length in bytes:

> elixir mfsc.exs ~/some_directory
1184
> elixir mfsc.exs ~/none_existing_dir
>

As you can see, for an incorrect input (a directory that does not exist), it will just return nothing. Let's fix it by adding else to the with statement.

with true <- File.dir?(directory),
{:ok, files} <- list_files(directory) do
Enum.reduce(files, 0, fn file, size -> size + calculate_file_size(file) end)
|> Kernel.div(length(files))
|> IO.puts()
else
_ ->
IO.puts("Invalid input")
System.halt(1)
end

Note that I didn't only add an error message but also a System.halt(1) call. This is to return a non-zero (i.e., non-success) exit code from a script (you can check the exit code of the last run command with echo $?). It is important to do so to be a 'good CLI land citizen.' Why? Command-line applications often do not work on their own but are chained together or called by other programs. An exit code helps to detect if a given part of a chain was run successfully (and we can expect the output to be in some predefined format) or not (in which case, we probably shouldn't even try to parse output). We have a working script, except for one small issue. When you run it on an empty directory, it will crash because of division by zero: ** (ArithmeticError) bad argument in arithmetic expression: div(0, 0) :erlang.div(0, 0) mfsc.exs:8: MFSC.call/0 (elixir 1.13.3) lib/code.ex:1183: Code.require_file/2 That's not nice. But what should we do on an empty directory? It's not obvious. We could return zero. Or alternatively, halt the application with a non-zero exit code. Or maybe — just a thought — we should leave the choice to the user? ## OptionParser: More Robust Input Handling in Elixir Remember how we assumed that there would always be only one input argument to our script? We're about to ditch this assumption. We want to introduce an option for the user to decide what should happen when an empty directory is found. By default, we want to return an error, but there will be a flag to alter this behavior: > elixir mfsc.exs /my/empty/dir Empty directory > echo$?
10
> elixir mfsc.exs /my/empty/dir --allow-empty
0
> echo \$?
0

Fortunately, Elixir has a tool readily waiting for us in its standard library: OptionParser.

Let's first play with it a bit in iex:

iex(1)> OptionParser.parse(["/my/empty/dir", "--allow-empty"], strict: [allow_empty: :boolean])
{[allow_empty: true], ["/my/empty/dir"], []}
iex(2)> OptionParser.parse(["/my/empty/dir"], strict: [allow_empty: :boolean])
{[], ["/my/empty/dir"], []}

The first argument to the OptionParser.parse/2 function is a list of arguments that we normally get from System.argv/0. In this case, we are simulating it manually.

The second one is the definition of our expected input. It is a keyword list with three possible keys: strict, aliases, and switches. For now, we will concentrate on strict as it is the basic one. In this example, we defined an expectation of one argument of the boolean type.

The return value is a tuple of three elements: the first is a list of parsed options, the second is for other (positional) arguments, and the third one is for errors. Let's see some more examples:

iex(3)> OptionParser.parse(["/my/empty/dir", "/tmp/second/dir"], strict: [allow_empty: :boolean])
{[], ["/my/empty/dir", "/tmp/second/dir"], []}
iex(4)> OptionParser.parse(["/my/empty/dir", "--allow-empty=3"], strict: [allow_empty: :boolean])
{[], ["/my/empty/dir"], [{"--allow-empty", "3"}]}

In the first example, we have two positional arguments and no parsed options. In the second, we violate the constraint that --allow-empty should be a boolean switch by passing the number 3 to it. As a result, it doesn't appear in parsed options but instead lands in errors.

Knowing how OptionParser works, we can add it to our app:

defmodule MFSC do
def call do
parsed =
System.argv()
|> OptionParser.parse(strict: [allow_empty: :boolean])

with {options, [directory], []} <- parsed,
true <- File.dir?(directory),
{:ok, files} <- list_files(directory) do
Enum.reduce(files, 0, fn file, size -> size + calculate_file_size(file) end)
|> print_size(files, options)
else
_ ->
IO.puts("Invalid input")
System.halt(1)
end
end

defp print_size(0, _, options) do
case Keyword.get(options, :allow_empty) do
true ->
IO.puts("0")

_ ->
IO.puts("Empty directory")
System.halt(1)
end
end

defp print_size(total_size, files, _) do
IO.puts(div(total_size, length(files)))
end

# rest of the code does not change
end

Nice! We support our first input option.

There is still a lot of work to do to make this app's interface friendlier, but I'll leave it up to you at this point. Some ideas on what you could do include:

• Returning different error messages and exit codes for various errors (non-existing directory vs. empty directory without --allow-empty vs. some random access error)
• An option to print a result in a different format than just the number of bytes, for example:
--output-format [B|K|M|smart]
• Allowing recursive counting, i.e., also calculate file sizes in subdirectories and their subdirectories. Beware of symbolic links leading to cycles!

## The Limitations of ElixirScript

So far, we've been using ElixirScript to create our CLI apps, but as our code grows, you will soon notice some limitations in this approach.

Splitting code into multiple files is inconvenient, and you cannot really use all the awesome packages published at hex. You might overcome this by using Mix.install/2 inline:

Mix.install([
{:ecto_sql, "~> 3.8"}
])

# some code using Ecto

There is, however, one last problem: to run your application, people need to have Elixir installed on their computers. This is what makes ElixirScript a nice tool to run something along an existing larger Elixir application (like tests in a Phoenix project), but not for our use case.

Elixir is still not a standard language. Some system repositories might have really old versions of the language. Generally, the fewer dependencies the user needs to install themselves, the more likely they will actually try out your application.

How can we do better, then?

## Using Escript for Your Elixir App

Escript is a built-in way to distribute whole Elixir applications as a single-file executable. This is, of course, one level better than using ElixirScript. Let's look closely at how to include it in our application.

To use escript, we need to have a regular mix project, not just a single file. But this is fine, as we want to split our application into multiple files and use dependencies anyway. Let's create a project then:

mix new mfsc

This will create a bunch of files in an mfsc directory. Among them is lib/mfsc.ex, where we will put the code we built in the previous section.

Theoretically, you should be able to run it via mix run now, but here's an unpleasant surprise: mix run resets System.argv/0, and you'll always get an empty array as the input. We are not going to focus on hacking this. Instead, we will introduce escript right away.

The first step: remove MFSC.call() from the bottom of the file — we only want to define a module for now.

Now we will add support for escript to our mix.exs file. In the project function that returns a keyword list of the project configuration, add a relevant line for escript:

def project do
[
app: :mfsc,
# bunch of other things
escript: [main_module: MFSC]
]
end

We are almost there. Now we need to refactor the call/0 function to the main/1 function accepting one argument — args — which will be the equivalent of System.argv/0.

defmodule MFSC do
def main(args) do
parsed =
args
|> OptionParser.parse(strict: [allow_empty: :boolean])

# rest of the file remains unchanged

That's all, folks! We build the escript file with mix escript.build. This will create an mfsc executable file in the project's root directory. We can call it like a regular executable:

> mix escript.build
Generated escript mfsc with MIX_ENV=dev
> ./mfsc ~/empty_dir
Empty directory
> ./mfsc ~/empty_dir --allow-empty
0

This is a huge step forward from ElixirScript — and the user does not need to have Elixir installed on their machine. Note, however, that they still need to have Erlang. On the other hand, this is a much more common dependency than Elixir, and chances are they already have it. In the worst-case scenario, it's just one thing less to install.

If that's not enough, we can improve things even further.

## Bakeware and Burrito for Elixir

Bakeware is a relatively new project (created in mid-2020) aiming to create dependency-less executables from Elixir code. Let's try it with MFSC. Bakeware leverages Elixir releases, so we need to add support for them to mix.exs. On top of that, we need the bakeware dependency and to add our entry module to application. The complete mix.exs should look like this:

defmodule Mfsc.MixProject do
use Mix.Project

def project do
[
app: :mfsc,
version: "0.1.0",
elixir: "~> 1.13",
start_permanent: Mix.env() == :prod,
deps: deps(),
escript: [main_module: MFSC],

releases: [                          # here we define cli release
cli: [
steps: [:assemble, &Bakeware.assemble/1]
]
]
]
end

def application do
[
extra_applications: [:logger],
mod: {MFSC, []}                      # adding our entry module
]
end

defp deps do
[
{:bakeware, "~> 0.2.4"}              # bakeware dependency
]
end
end

After that, we need to adjust our MFSC module by adding Bakeware scripting support:

defmodule MFSC do
use Bakeware.Script

@impl Bakeware.Script
def main(args) do

# rest of the file

And with that in place, we just need to run mix release cli. This will take some time, but we end up with our standalone CLI application with a _build/dev/rel/bakeware/cli file. Let's take it for a ride!

> _build/dev/rel/bakeware/cli ~/empty_dir
Empty directory
> _build/dev/rel/bakeware/cli ~/empty_dir --allow-empty
0

This definitely looks familiar. However, you should know one thing — these files will only work on a similar operating system as the one they were built on. For example, I created mine on Linux, and when I sent it to another Linux machine, it worked without any issues. But it did not work on Intel Mac, not to mention M1 Mac.

We might try to address this issue with an even newer project than Bakeware. It's called Burrito and the first public commit is just from October 2021. It promises cross-platform standalone releases and is aimed at CLI applications. Burrito has a few additional dependencies, namely Zig and xz. When we install them, we add another release to mix.exs (remember to add the burrito dependency too!):

burrito_cli: [
steps: [:assemble, &Burrito.wrap/1],
burrito: [
targets: [
macos: [os: :darwin, cpu: :x86_64],
macos_m1: [os: :darwin, cpu: :aarch64],
linux: [os: :linux, cpu: :x86_64],
windows: [os: :windows, cpu: :x86_64]
],
]
]

Now, run the release process:

mix release burrito_cli

After a while, you should have the executables for different operating systems waiting under the burrito_out directory, for example, burrito_out/burrito_cli_linux. Note that Burrito is still a work in progress, and you may experience some bumps on the road. For me, at the time of writing this article, building for macOS from Linux did not work.

If you want to learn more about Burrito, there's a great talk by its author, Digit, from the EPEX conference. It's called Wrap your app in a BEAM burrito! In this talk, you'll learn why the project was born and the main hurdles along the way, followed by a live coding session.

## The Downsides of Using Elixir for CLI Applications

Now that we've seen how easy it is to build a standalone CLI application in Elixir, let me warn you about one thing: the startup time will be slow. In my tests, it always takes about half a second.

This is not a problem for long-running or interactive applications but might be a showstopper if you need a small tool to execute multiple times — for example, the find command:

find . -maxdepth 1 -type f -exec ./my_elixir_app {} \;

The startup time of half a second is multiplied by the number of files, which may add up to quite significant delays. You can experience this pain in the real world when you generate protobuf files for Elixir with protobuf-elixir.

## Should I Use Elixir for CLI Apps?

If some of the problems outlined in the previous sections of this post do not scare you or do not apply to your use case, by all means use Elixir for CLI applications. It can be a lot of fun, especially combined with its concurrency abilities.

I personally did in my simple stress-tester application, which spawns multiple processes and attempts to flood a given URL with requests, gathering statistics on how it goes. And I used it for work more than once, where a full-blown solution was not required.

## Wrapping Up

In this post, we built a simple CLI application using ElixirScript, learned to use OptionParser, then released it using escript as an executable, which only requires Erlang to run.

After that, we looked into creating more standalone releases with Bakeware or Burrito. Finally, we explored some downsides of choosing Elixir for certain types of CLI tools.

I hope you found this a good starting point for writing a CLI app in Elixir.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!