If you're using Oban for managing background jobs in your Elixir application and want to gain a deeper data-driven understanding of how they perform, you've come to the right place.

AppSignal for Elixir now automatically instruments Oban, meaning you can now monitor the performance of your background jobs through an AppSignal Magic Dashboard, which gives you detailed information on queue times, processing times, and notifies you of any exceptions. Best of all, because Oban is instrumented automatically, all you need to do to access this data is perform a fuss-free upgrade of your AppSignal for Elixir integration to our 2.5.0 package.

In this blog post, we'll show you how your Elixir application can benefit from AppSignal's Oban background job monitoring.

What's Oban for Elixir?

Oban is a job queueing and processing system for Elixir, like Ruby's Sidekiq and ActiveJob. Oban can schedule tasks your application needs to run in the future, check if your application runs those jobs successfully, and retry them if not.

For example, say you have a newsletter publishing platform. When one of your users presses send on their latest campaign, your application will need to email all their subscribers. Sending thousands of emails can take some time, and some of the emails might bounce and need to be re-sent later, but you want to make sure your user gets all of these newsletters delivered successfully to their recipients.

So you create one pending job for every email your application needs to send. Once created, those jobs are put in a queue. Oban runs in the background picking up the jobs from the queue one by one, sending an email to each of your user's subscribers, and retrying any that bounce.

If you've ever sent an email campaign on a platform like Mailchimp, you'll likely be familiar with the process we've just described.

Should I Monitor Oban Background Jobs?

Yes, you should!

Let's go back to our newsletter publishing platform scenario. Imagine a user presses send, and the background jobs designed to send everyone their newsletter fail when sending every other email. These failures will not be obvious to your end users and won't be picked up by manual testing. Without adequate monitoring, you may not find out about the issue until weeks after it first occurs. This could have serious implications for your application.

That's not the only reason to monitor your background jobs. Queueing systems can also be finicky and benefit from fine-tuning. Too much processing power, and you're paying your provider a lot of money to run machines that are doing nothing. Not enough processing power, and your jobs will start to bottleneck in queues, taking longer to run than desired and impacting the user experience. You may want to run several job processors in parallel for better performance. Even then, it can be tricky to figure out the perfect amount to run in parallel to ensure that any performance improvements make up for the overhead of parallelization.

These tweaks and many others, such as job prioritization, job scheduling, and how to handle failing jobs, require not only trial and error but data that can inform your decisions. AppSignal can provide that data, telling you which jobs are failing, how long jobs have been waiting in the queue, and how long it takes for each successful job to run.

How Does AppSignal Support Oban?

Starting with AppSignal for Elixir 2.5.0, our integration will listen to events emitted by your Oban setup in your application, trace their execution, and send that information to AppSignal, using it to provide metrics, as well as performance and error samples.

No setup is necessary. Simply upgrade your AppSignal for Elixir integration to the latest version. You can do this by running the following command:

mix update appsignal

Once you've updated your integration you can sit back, and enjoy the show 🍿!

Job Execution: Performance and Error Samples in AppSignal

For each job that is performed, you can see:

• The performance when processing a job, in the "Performance" > "Issue list" panel, under the Oban namespace.
• The error emitted when failing to process a job, in the "Errors" > "Issue list" panel, under the Oban namespace.

For each of these performance and error samples, you'll be able to see key information about the event, such as the queue in which it was scheduled, the job's priority, the job's arguments, and the result it emitted. You can also filter performance and error samples by the tags and meta information you added to your Oban jobs.

You'll also see other events instrumented by AppSignal that take place while processing the job. For example, if a database query is performed using Ecto at the same time, you'll see that database query in the sample's event timeline as well.

Oban Magic Dashboard in AppSignal: Queue and Job Performance Metrics

As soon as your application starts processing jobs, the Oban Magic Dashboard will appear. In this dashboard, you can see information about the performance of your job queueing system, such as:

• Job duration: For each of the workers (think "job types") in your system, a graph of the time the job took to complete.
• Job queue time: For each of the queues in your system, a graph of the time the jobs currently executed have spent waiting in the queue.
• Job exceptions: For each worker in your system, a graph of how many jobs failed at a given point in time.

This is just a slice of what you can do with the metrics you send to AppSignal. Be sure to play around with our dashboard graph editor to find the metric combination that gives you the exact information you need!

Wrap Up

Thanks to the addition of Oban support to the Elixir integration, you can now gain valuable insights into the performance and metrics of your Elixir application's background jobs. With the additional data AppSignal can provide, you'll be able to monitor the health and stability of your background jobs and make informed decisions to optimize their performance.

When It Comes to Background Jobs, We've Got Your Back

If you have any questions about how AppSignal works with Elixir and Oban, or you'd like to see something not already available on AppSignal, please do get in touch with us, we'd love to hear what you have to say! We provide developer-to-developer support, and if you're a trial user in need of a sweet pick-me-up; we've got some sweet stroopwafels to send your way 😋 🍪