Distributed Tracing Is Now in Beta for Ruby, PHP, and Python

A request comes in, enqueues a job, and returns. Twenty seconds later the job runs, and it’s slow. You have a trace of the request and a trace of the job, and nothing joining them. Time Detective has always helped you reconstruct what happened. Now we join it up for you, across applications, services, background jobs and infrastructure, even when they’re built in different languages.

Distributed tracing, a feature more commonly known to developers interested in OpenTelemetry, is now part of our platform capabilities. It’s in beta, with Ruby, Python, and PHP supported immediately.

NetNodes have been running it in production:

The distributed tracing features now give us end-to-end visibility across our entire stack, allowing us to see and follow the more complex cross-service interactions - this is a game changer for us, and has saved me hours already

Steve Hill, NetNodes

What It Does

A single trace now shows the signals from every service that took part in one request, with the web app, the internal API, and the background job each reporting its own piece.

We’ve added two new views to the trace detail page.

• The service map shows a trace from the perspective of the services that took part: which services the request reached, and how long each call between them took. Edges are colored by how each call rates against the trace’s own baseline, so the slowest hops stand out without you reading every duration. It appears once a trace crosses more than one service.
• The trace timeline lays every span out over time, including spans that came from a different service, so you can see where the time actually went.

Above: one checkout, followed across four services. The two red edges are the slow hops, and the dashed one is the handoff into Sidekiq.

“We Run a Monolith. Does This Do Anything for Us?”

If your application really is a single service end to end — one Rails process, no background jobs, nothing calling out — then distributed tracing has nothing to connect, and it won’t show you anything you can’t already see. You’d still pick up some niceties from running in collector mode, but the tracing itself would be solving a problem you don’t have.

That’s rarely the actual shape of a Rails monolith, though. The usual shape is at the very least a Rails website and a Sidekiq worker. The moment a request enqueues a job, you have two processes and a handoff between them, and that handoff is exactly where the story goes missing today.

A job now runs in the same trace as the code that enqueued it, appearing under the request or job that scheduled it, so you can trace a slow job back to what caused it to run. For Active Job the trace rides along in the job’s serialized arguments; for Sidekiq it’s carried on the job itself, alongside its other data.

You don’t have to have split anything up to benefit.

What’s Connected for You

You don’t pass anything between your services by hand. Once you’re in collector mode, these integrations carry the trace on their own:

• Background jobs. Jobs run in the same trace as the code that enqueued them: Active Job, Delayed Job, Que, Resque, Shoryuken, Sidekiq.
• HTTP clients. Outgoing requests carry the current trace to whatever they call: Excon, Faraday, HTTP.rb, Net::HTTP.
• HTTP servers. Incoming requests that arrive carrying a trace continue it: Grape, Hanami, Padrino, Rack, Rails, Sinatra, Webmachine.

How the trace reaches a job differs per library, and each integration’s docs page describes its own behavior and limits. The full list lives under distributed tracing for Ruby.

Turning It On in a Rails App

Distributed tracing is built on OpenTelemetry, and for Ruby that means running the gem in collector mode: reporting over OTLP/HTTP to an AppSignal Collector instead of through the bundled agent.

Two requirements first: Ruby 3.1 or newer, and a hosted collector. Distributed tracing is only available through a collector AppSignal hosts for you; a self-hosted collector won’t light it up. Collector mode also doesn’t support JRuby, because of how it handles forking.

Create your collector on the Hosted Collector page in your organization’s settings, then add the OpenTelemetry gems. The appsignal gem doesn’t install them, so pull in appsignal-opentelemetry alongside it:

# Gemfile
gem "appsignal", ">= 5.0.0.rc1", "< 6"
gem "appsignal-opentelemetry", ">= 5.0.0.rc1", "< 6"

Yes, that’s a pre-release. appsignal-opentelemetry does nothing on its own; it exists so you don’t have to track which OpenTelemetry gem versions collector mode supports. Its version stays in lockstep with appsignal.

Then point the gem at your collector and name the service:

# config/appsignal.rb
Appsignal.configure do |config|
  # ... your existing settings ...
  config.collector_endpoint = "https://collector.example"
  config.service_name = "web-server"
end

Or with environment variables:

APPSIGNAL_COLLECTOR_ENDPOINT="https://collector.example"
APPSIGNAL_SERVICE_NAME="web-server"

service_name is the one people skip, and it’s the one that matters here. It’s how a trace tells your web app apart from your worker apart from your internal API. Without it, a cross-service trace is technically correct and completely unreadable.

Set both in each app that takes part. For a monolith with a worker, that means the Rails process and the Sidekiq process each get their own service_name. Run it, and traces start arriving connected.

If the gems are missing or on an unsupported version, AppSignal logs a warning at startup and keeps reporting through the agent, so a bad Gemfile degrades rather than goes dark.

On Python and PHP

Python works the same way as Ruby: set collector_endpoint to a hosted collector and give each application a service_name. Collector configuration for Python has the details.

PHP has nothing to configure. The PHP package reports through a collector already, so distributed tracing works once your app is reporting to AppSignal. The same is true of a custom OpenTelemetry setup — exporting to a collector is the only way those report, so traces connect on their own.

One thing Ruby and Python don’t get yet: when you add a PHP, Go, Java or OpenTelemetry app through the UI, AppSignal provisions a hosted collector for you. For Ruby and Python you create one yourself on the Hosted Collector page.

What Isn’t Here Yet

• Elixir and JavaScript aren’t supported yet. Elixir is next on our list. Follow along in AppSignal Labs and join our Discord to hear when we start work on adding it.
• Hosted collector only, and the Ruby gem is pre-release. Both will change.
• Front-end spans aren’t in the trace yet. Browser monitoring is already in AppSignal Labs, and the @appsignal/browser SDK behind it is what will feed your front end into a trace.

Note: any OpenTelemetry-instrumented app can join a trace by exporting to a hosted collector, through an AppSignal integration like the PHP package, or your own OpenTelemetry SDK setup.

Full setup details are in the distributed tracing docs, with the Ruby specifics under collector configuration for Ruby.

Every one of those pages is also served as plain Markdown at the same URL with a .md extension, and listed in llms.txt. So if you would rather not do the setup by hand, point your coding agent at https://docs.appsignal.com/ruby/configuration/collector.md and let it read the same instructions you would. More on that tomorrow.

Find Us at Rails World

At the booth: come meet us for a demo, especially if you want to see the service map on something messier than a two-service example.

• Wednesday, at Tacos at Sunset: Ashley Orr, our UX researcher, is signing people up for feedback sessions. Half an hour, your real trace, our design team listening.
• Not at Rails World: Discord works just as well, and it’s where the team building this reads.

Everything we’re running this week is in one place on appsignal.com/connect.

If you missed it, yesterday we announced expert context, built with GoRails.