appsignal

Microservices Monitoring: Using Namespaces for Data Structuring

Tomas Fernandez

Tomas Fernandez on

Microservices Monitoring: Using Namespaces for Data Structuring

What Is the Microservice Architecture?

Microservice architecture is a software design pattern in which we write applications by combining several small programs. These programs, which are called microservices, work together for a common goal. For some teams, it takes a lot less time and effort to write several small applications than a single large one.

A microservice-oriented project consists of a constellation of small applications, each doing its part, running in a separate process, and communicating with the rest through standardized interfaces. This approach lets teams choose the best tool for each problem without committing to one language or framework. It also allows us to split the work among more specialized groups.

From Monolith to Microservice

In the first part of this series, we talked about monoliths. Monoliths are easy to start with. Development pace, however, scales poorly, mainly because everything is so tightly coupled. Even a small code change forces us to rebuild and test the whole project, which leads to frustratingly long release cycles.

How do we go from monolith to microservice? Take the case of Amazon. A while back, they started as a monolith and over time switched to microservices. Their initial design may have looked like this:

Monolith Design

Of course, I’m oversimplifying here, but I believe I covered most of the basics. What if they had chosen to follow the microservice pattern from the start? They would have split the application by function, having each component focus on one problem.

They would also have needed to define interfaces and protocols for inter-service communication, typically with lightweight mechanisms like RESTful APIs.

Microservice Design

What Are Namespaces

A microservice design has a unique set of challenges. The main one is perhaps instrumentation and error reporting. Think about it, we are monitoring tens or hundreds of components spread across different platforms and languages. And we must somehow keep an eye on all of them, and at the same time, avoid losing the big picture. Namespaces can help us focus a group of loosely-coupled microservices into a coherent picture.

In AppSignal, namespaces are containers for collected metrics. AppSignal uses three namespaces by default (web, background, and frontend), but we can create our own by adding a few lines of code. We’ll see how they work next.

One Application to Rule Them All

When setting up a microservice component, the first thing to do is to configure a common application name and environment. Hence, AppSignal presents all the collected metrics and alerts on the same dashboard.

Every microservice reports to the same dashboard

Specific details of how to configure these values depend on the language and the integration. For example, to configure a Ruby on Rails application to use the name “Nozama”:

ruby
# config/appsignal.yml production: active: true push_api_key: "YOUR APPSIGNAL API KEY" name: "Nozama"

Which is very similar to how we configure Elixir integrations:

elixir
# config/config.exs config :appsignal, :config, active: true, name: "Nozama", push_api_key: "YOUR APPSIGNAL API KEY", env: "production"

In Node.js, on the other hand, we use:

javascript
const { Appsignal } = require("@appsignal/nodejs"); const appsignal = new Appsignal({ active: true, name: "Nozama", apiKey: "YOUR APPSIGNAL API KEY", });

For frontent JavaScript integration, we use @appsignal/javascript instead:

javascript
import Appsignal from "@appsignal/javascript"; export default new Appsignal({ name: "Nozama", key: "YOUR FRONTEND API KEY", });

You can find information on installing and configuring AppSignal here:

Using Namespaces in Microservices

Let's see how we would go about coding each microservice. Let’s start with the billing system; we’ll use Elixir and Phoenix for this part.

Once we've followed the Phoenix integration setup, we can start working on the controllers. The following snippet sets the namespace to billing:

elixir
# in a Phoenix controller, we use plug to run the namespace initialization defmodule BillingPageController.PageController do use BillingPageController, :controller plug :set_appsignal_namespace defp set_appsignal_namespace(conn, _params) do # Sets all actions in this controller to report in the "billing" namespace Appsignal.Transaction.set_namespace(:billing) conn end # rest of the controller ... end

Data should start appearing in the dashboard once the microservice is running and the controller sees some activity.

Showing the billing namespace

Of course, billing won’t get us far unless people buy something. This is a problem that we can tackle in a separate microservice. Following the same pattern, we’ll write a brand new Phoenix application with a PayButtonController controller that begins like this:

elixir
defmodule PayButtonController.PageController do use PayButtonController, :controller plug :set_appsignal_namespace defp set_appsignal_namespace(conn, _params) do Appsignal.Span.set_namespace(Appsignal.Tracer.root_span(), "pay_button") conn end # rest of the controller ... end

We now have two namespaces in the dashboard. Using the same name and environment guarantees that the data from PayButtonController is shown together with BillingPageController, even if they are separate applications running on different machines.

Added paybutton namespace

The next component is the recommendation engine. We’ll implement the API endpoint that shows product suggestions with Express. We set the namespace in Node.js as shown:

javascript
app.get("/", (req, res) => { const tracer = appsignal.tracer(); tracer.withSpan( tracer.createSpan({ namespace: "recommendations" }), (span) => { // code to measure goes here span.close(); } ); });

We’re up to three namespaces now.

Recommendation service

The mobile and frontend teams may want to log errors in the dashboard. AppSignal’s JavaScript integration automatically assigns incoming data with the frontend namespace. But we can change it like this:

javascript
try { // code that might fail } catch (error) { // handle the error // send error to AppSignal appsignal.sendError(error, {}, "Mobile"); }

After a while, data should start appearing in the Mobile namespace.

Mobile namespace

The example shows plain JavaScript, but there may be additional setup steps required if you’re using a frontend framework like React or Angular.

For the website, let’s try Ruby on Rails, a very well known MVC framework which AppSignal integrates with out-of-the-box. We’ll start the Rails controllers with the following snippet to set the namespace to homepage:

ruby
# in Rails we use before_action callback to change # the namespace before the request starts class HomepageController < ApplicationController before_action :set_appsignal_namespace def set_appsignal_namespace Appsignal.set_namespace("homepage") end # controller actions ... end
Search API and homepage

Next, we could use API endpoints to serve data to the website and mobile applications. For this, we could use Grape, a lightweight REST API framework for Ruby. This time, configuring AppSignal takes a bit more work.

After configuring the Ruby integration in ´config/appsignal.yml´, as we did earlier, you can start logging events and metrics with:

ruby
Appsignal.start_logger Appsignal.start

Then, insert the AppSignal middleware in the error handler chain:

ruby
require "appsignal" require "appsignal/integrations/grape" class API < Grape::API insert_before Grape::Middleware::Error, Appsignal::Grape::Middleware resource :search do desc 'return a product search' before do Appsignal.set_namespace("search") end get :product do # product search logic end end end

For more examples, check the Grape integration docs.

To complete the picture, we’ll end with a Sidekiq background job. Sidekiq is a popular job processor for Ruby and this is how we can start it up in standalone mode after configuring ´config/appsignal.yml´:

ruby
# config.ru require 'appsignal' Sidekiq.on(:startup) do Appsignal.start end Sidekiq.on(:shutdown) do Appsignal.stop('Sidekiq shutdown') end

AppSignal automatically assigns data from jobs to the background namespace. We may want to change it to a more specific namespace.

ruby
require 'sidekiq' require 'appsignal' class PlainOldRuby include Sidekiq::Worker def perform() Appsignal.set_namespace("urgent_background") # job logic end end

Collecting Metrics With the Standalone Agent

The standalone agent collects resource utilization metrics from any Ubuntu, RedHat, or CentOS machine. We can use the agent to monitor satellite servers that provide facilities like databases, gateways, or message brokers to the microservice applications.

In AppSignal we use the agent to monitor our own Kafka servers. The agent is also handy for creating custom instrumentation on languages and frameworks not directly supported.

To get started with the agent, download and install it following the installation instructions.

Once it’s running, we’ll need to edit the configuration file to set the API key, the application name, and the environment. Use the same values used in the rest of the microservice to get everything onto one dashboard.

shell
# /etc/appsignal-agent.conf push_api_key = "YOUR APPSIGNAL API KEY" app_name = "Nozama" environment = "production"

Conclusion

Namespaces are a great way of organizing data distributed among multiple systems. They also allow us to configure notifications and fine-tune alert handling. To see how that works, check the first part of this series.

Additional reads:

Tomas Fernandez

Tomas Fernandez

Our guest author Tomas spent 10 years working at IBM, where he did a bit of everything - development, service delivery, database administration, and cloud engineering. He’s now an independent consultant and a technical writer.

All articles by Tomas Fernandez

Become our next author!

Find out more

AppSignal monitors your apps

AppSignal provides insights for Ruby, Rails, Elixir, Phoenix, Node.js, Express and many other frameworks and libraries. We are located in beautiful Amsterdam. We love stroopwafels. If you do too, let us know. We might send you some!

Discover AppSignal
AppSignal monitors your apps