In this third and final part of this series, we will look at what happens during an application upgrade.
Let's get going!
Set-up: Create an Appup File in Elixir
As with a single module, we need new compiled code for a fresh version of an application.
But an application can consist of many modules and have running processes. So we need a scenario to specify what to upgrade and how to do it.
These scenarios are called application upgrade files (.appup files).
Let's try to make an appup file for our application. Imagine that we want to upgrade our counters so that we can specify counter increment speed.
Prepare a New Version of the Application in Elixir
First, we'll add our application to git and tag the old version:
git add . git commit -m 'initial commit' git tag 'v0.1.0'
Now we should make a new version.
In mix.exs, update the version to 0.1.1:
... def project do [ app: :our_new_app, version: "0.1.1", ...
We want interval to be an external parameter, not a module attribute. Make the following changes in lib/our_new_app/counter.ex:
defmodule OurNewApp.Counter do ... def start_link({start_from, interval}) do GenServer.start_link(__MODULE__, {start_from, interval}) end ... def init({start_from, interval}) do Process.flag(:trap_exit, true) st = %{ current: start_from, timer: :erlang.start_timer(interval, self(), :tick), terminator: nil, interval: interval } {:ok, st} end ... def handle_info({:timeout, _timer_ref, :tick}, st) do :erlang.cancel_timer(st.timer) new_current = st.current + 1 if st.terminator && rem(new_current, 10) == 0 do # we are terminating GenServer.reply(st.terminator, :ok) {:stop, :normal, %{st | current: new_current, timer: nil}} else new_timer = :erlang.start_timer(st.interval, self(), :tick) {:noreply, %{st | current: new_current, timer: new_timer}} end end ... end
Now add a code_change callback to the same file:
defmodule OurNewApp.Counter do ... def code_change(_old_vsn, st, new_interval) do {:ok, Map.put(st, :interval, new_interval)} end end
And change supervisor specs in lib/our_new_app/counter_sup.ex:
@impl true def init(start_numbers) do children = for start_number <- start_numbers do # We can't just use `{OurNewApp.Counter, start_number}` # because we need different id's for children Supervisor.child_spec({OurNewApp.Counter, {start_number, 200}}, id: start_number) end Supervisor.init(children, strategy: :one_for_one) end
Let's construct our_new_app.appup file.
We need to update our supervision specs and pass a new tick interval (250) to change_code callback:
{ "0.1.1", [{"0.1.0", [ {update, 'Elixir.OurNewApp.CounterSup', supervisor}, {update, 'Elixir.OurNewApp.Counter', {advanced, 250}} ]}], [{"0.1.0", []}] }.
Note that this file is written in Erlang syntax.
Now tag your new version:
git add . git commit -m 'added customizable intervals' git tag 'v0.1.1'
Try to upgrade 0.1.0 to 0.1.1.
Checkout, compile, and run 0.1.0 version:
git checkout v0.1.0 iex -S mix
In a separate shell and different directory, checkout the new version and put the appup file in the appropriate place:
... git checkout v0.1.0 mix compile cp our_new_app.appup _build/dev/lib/our_new_app/ebin
Run the Application Upgrade in Elixir
We are ready to upgrade the application. In the running iex session, do the following:
iex(1)> Application.spec(:our_new_app) [ description: 'our_new_app', id: [], vsn: '0.1.0', modules: [OurNewApp, OurNewApp.Application, OurNewApp.Counter, OurNewApp.CounterSup], maxP: :infinity, maxT: :infinity, registered: [], included_applications: [], applications: [:kernel, :stdlib, :elixir, :logger], mod: {OurNewApp.Application, []}, start_phases: :undefined ]
You can see that the old application is running.
Run an "orphaned" counter outside the supervision tree (we will need it later):
iex(2)> {:ok, pid} = OurNewApp.Counter.start_link(30000) {:ok, #PID<0.147.0>}
Check that your appup file is correct and the OTP knows how to upgrade your application:
iex(3)> :release_handler.upgrade_script(:our_new_app, '/path/to/new/version/of/our_new_app/_build/dev/lib/our_new_app/') {:ok, '0.1.1', [ {:load_object_code, {:our_new_app, '0.1.1', [OurNewApp.CounterSup, OurNewApp.Counter]}}, :point_of_no_return, {:suspend, [OurNewApp.CounterSup]}, {:load, {OurNewApp.CounterSup, :brutal_purge, :brutal_purge}}, {:code_change, :up, [{OurNewApp.CounterSup, []}]}, {:resume, [OurNewApp.CounterSup]}, {:suspend, [OurNewApp.Counter]}, {:load, {OurNewApp.Counter, :brutal_purge, :brutal_purge}}, {:code_change, :up, [{OurNewApp.Counter, 250}]}, {:resume, [OurNewApp.Counter]} ]}
Check your counter processes and their pids:
iex(4)> Supervisor.which_children(OurNewApp.CounterSup) [ {20000, #PID<0.143.0>, :worker, [OurNewApp.Counter]}, {10000, #PID<0.142.0>, :worker, [OurNewApp.Counter]} ]
Upgrade the application!
iex(5)> :release_handler.upgrade_app(:our_new_app, '/path/to/new/version/of/our_new_app/_build/dev/lib/our_new_app/') {:ok, []} iex(6)> 02:46:48.286 [info] terminating with {{:badkey, :interval, %{current: 33478, terminator: nil, timer: #Reference<0.948322908.3672375303.146224>}}, [{OurNewApp.Counter, :handle_info, 2, [file: 'lib/our_new_app/counter.ex', line: 52]}, {:gen_server, :try_dispatch, 4, [file: 'gen_server.erl', line: 680]}, {:gen_server, :handle_msg, 6, [file: 'gen_server.erl', line: 756]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}]}, counter is 33478 02:46:48.291 [error] GenServer #PID<0.147.0> terminating ...
Let's see what happened:
iex(1)> Application.spec(:our_new_app) [ description: 'our_new_app', id: [], vsn: '0.1.1', modules: [OurNewApp, OurNewApp.Application, OurNewApp.Counter, OurNewApp.CounterSup], maxP: :infinity, maxT: :infinity, registered: [], included_applications: [], applications: [:kernel, :stdlib, :elixir, :logger], mod: {OurNewApp.Application, []}, start_phases: :undefined ] iex(2)> Supervisor.which_children(OurNewApp.CounterSup) [ {20000, #PID<0.143.0>, :worker, [OurNewApp.Counter]}, {10000, #PID<0.142.0>, :worker, [OurNewApp.Counter]} ] iex(3)> pids = for {_, pid, _, _} <- Supervisor.which_children(OurNewApp.CounterSup), do: pid [#PID<0.143.0>, #PID<0.142.0>] iex(4)> OurNewApp.Counter.get(Enum.at(pids, 0)) 27225 iex(5)> OurNewApp.Counter.get(Enum.at(pids, 1)) 17235 iex(6)> :sys.get_state(Enum.at(pids, 0)) %{ current: 27468, interval: 250, terminator: nil, timer: #Reference<0.948322908.3672375311.146315> } iex(7)> :sys.get_state(Enum.at(pids, 1)) %{ current: 17476, interval: 250, terminator: nil, timer: #Reference<0.948322908.3672375311.146330> } iex(8)> :sys.get_state(OurNewApp.CounterSup) {:state, {:local, OurNewApp.CounterSup}, :one_for_one, {[20000, 10000], %{ 10000 => {:child, #PID<0.142.0>, 10000, {OurNewApp.Counter, :start_link, [{10000, 200}]}, :permanent, 5000, :worker, [OurNewApp.Counter]}, 20000 => {:child, #PID<0.143.0>, 20000, {OurNewApp.Counter, :start_link, [{20000, 200}]}, :permanent, 5000, :worker, [OurNewApp.Counter]} }}, :undefined, 3, 5, [], 0, OurNewApp.CounterSup, [10000, 20000]}
Our upgrade process was successfully completed.
You can see that:
- The new version of the application is now running.
- Our child counters are alive — they updated their state and are functioning.
- From the internal state of
OurNewApp.CounterSup, we see those child specifications for our counters updated too! Now, if they die, they will properly restart.
But what about the error GenServer #PID<0.147.0> terminating?
Recall that
#PID<0.147.0> is the pid of our "orphaned" counter, which was running outside of the supervision tree. As the application upgrade process traverses the supervision tree and updates processes,
the "orphaned" counter's state was not updated. But the code of the OurNewApp.Counter
did update, so the "orphaned" counter process died: new code met its old state.
We've seen how to upgrade a single running application.
We needed only two special tools for that: an .appup file and :release_handler.upgrade_app/2 function.
It was also crucial for us to follow OTP principles.
Wrap-up and What to Learn Next
I hope you've enjoyed this whirlwind ride through production code upgrades in Elixir! We started with my guide to hot code reloading, followed by the best use of supervisors when building applications.
This final article has demonstrated how following OTP principles can show us the way to powerful application code upgrades.
It's worth noting that the application code upgrade I've demonstrated here still has disadvantages. The critical issue is that if the whole OS beam process restarts, this will load our application's old code (unless we take some action).
How can we handle this potential problem, I hear you ask? With so-called release upgrades. This awesome article from 'Learn you some Erlang' is a good starting point to dive into release upgrades.
Thanks again for taking the time to read this series — and happy coding!
P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!
