AppSignal's Standalone Agent

The standalone agent is based on the same software with which we usually instrument Ruby, Elixir, or JavaScript applications. This software can also run in standalone mode.

The standalone agent can be used to monitor:

• Infrastructure: machines that are part of our system but do not run application code.
• Background jobs: such as intensive cron jobs or long-running data-processing scripts. If these background jobs are written in a supported language (Ruby, Elixir, or Node.js) you can use the standard integration.
• More languages: programs written in languages other than those supported out of the box.

For instance, with the standalone agent, we could track a machine learning model written in Python, instrument backup scripts, monitor Kafka brokers, or collect host metrics in our web farms. You can view all this information on AppSignal to complement the metrics you already have for your main applications.

How It Works

The agent is shipped as a deb or rpm package and doesn’t have any language dependencies. It runs in any Debian/Ubuntu or Red Hat-based systems. For detailed installation instructions, check the agent documentation.

Once installed, the agent is configured in a few minutes and keeps running forever as a daemon, silently monitoring your infrastructure. What is more, the agent includes a StatsD server that relays any custom data you log into your AppSignal dashboard.

KEY:VALUE|TYPE

Sending Data to StatsD

The standalone agent listens for UDP packets on port 8125. We can send StatsD-formatted strings from the command line using netcat:

echo -n "myscript.myevent.counter:1|c" | nc -4u -w0 localhost 8125

Since we’re using UDP, we don’t have to wait for a response.

Socat also works:

echo -n "myscript.myevent.counter:1|c" | socat - udp:localhost:8125

This makes it easy to instrument any batch or cron jobs. For instance, the following lines use a gauge to log how much data a backup job has generated:

backup_size=$(du -m /backups | cut -f1)
echo -n "backup.data:$backup_size|g" | nc -4u -w0 localhost 8125

We’re not limited to integers. StatsD also works with floating-point numbers:

echo -n "network.latency:0.2|g" | nc -4u -w0 localhost 8125

Using Tags

You can add tags to your metrics. The StatsD server supports optional tags at the end of the message:

KEY:VALUE|TYPE|#TAGS

We can apply several tags in the same message and assign values for filtering later:

echo -n "backup.data:$backup_size|g|#backups,env:production" | nc -4u -w0 localhost 8125

We’ll learn how to see the data in AppSignal in a bit.

Instrumenting Languages

The StatsD server is compatible with any language that can send UDP packets. Let’s see a few examples.

Python is a popular language for data mining and machine learning. We can add instrumentation into Python applications using the build-in socket library:

import time
 
# measure time taken for function
start = time.process_time()
train_my_model()
training_time = time.process_time() - start
 
# send value to statsd
import socket
 
UDP_IP = "127.0.0.1"
UDP_PORT = 8125
MESSAGE = b"model.training.time:" + bytes(str(training_time), 'utf-8') + b"|t"
 
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(MESSAGE, (UDP_IP, UDP_PORT))

We could achieve similar results in Java with the java.net library:

byte[] buffer = "mybatch.exception.counter:1|c".getBytes();
InetAddress address = InetAddress.getByName("127.0.0.1");
DatagramPacket packet = new DatagramPacket(
    buffer, buffer.length, address, 8125
);
DatagramSocket datagramSocket = new DatagramSocket();
datagramSocket.send(packet);

PHP is another example of an incredibly popular language. We can send UDP packets with socket_sendto:

$msg = "mywebsite.active_users:$ACTIVE_USERS|g";
$len = strlen($msg);
 
$sock = socket_create(AF_INET, SOCK_DGRAM, SOL_UDP);
socket_sendto($sock, $msg, $len, 0, '127.0.0.1', 8125);
socket_close($sock);

Viewing Host Data in AppSignal

The Host Metrics dashboard shows your machine’s resource utilization:

AppSignal adds an entry for every machine running the standalone agent.

The dashboard shows the load and CPU averages disk, network, memory, and swap usage:

Creating Dashboards

AppSignal doesn’t automatically generate dashboards for the StatsD value you sent—you’ll need to create custom dashboards for this.

First, click on Add dashboard under the dashboard menu:

Give a name to the dashboard:

Clicking the Add graph button shows the graph builder:

Give a name to the graph, and click on Add metric:

Pick the metric you’re interested in from the menu.

Optionally, use tags for filtering:

Next, click Back to overview. You might want to try different graph types and value units to find out which one best suits the data you want to represent.

Timing data looks better with area graphs because the mean and percentiles are shown more clearly.

Line graphs work great for counters and gauges.

Once you’re happy with the result, click on Create graph.

Note that you can also add a dashbaord for any other automatically instrumented or measured metrics if you have AppSignal set of as your performance monitoring tool (APM) for Node.js, or your Ruby (on Rails) APM or Elixir APM.

Wrapping Up

We’ve learned how to use AppSignal’s standalone server to watch your machines and satellite code. With its built-in StatsD server, you can record arbitrary performance data and instrument any process.

Check the following links to learn more about using the standalone agent:

• StatsD reference.
• How to send custom metrics.
• Creating dashboards in AppSignal.
• Identifying and Resolving a Kafka Issue the Standalone Agent.