Extend ActiveStorage for Ruby on Rails with Custom Previewers

In part one of this series, we looked at how to extend the ActiveStorage ingest process with custom analyzers.

In this post, we will reverse the procedure and explore how to utilize ActiveStorage previewers to display data.

What Are ActiveStorage Previewers for Ruby on Rails?

Not every uploaded blob is an image. Nonetheless, some non-image blobs can be converted into an image preview. ActiveStorage itself provides built-in previewers for videos and PDFs (via MuPDF or Poppler).

Displaying a preview in your ERB template works exactly the same as creating variants of an image:

<%= image_tag song.recording.preview(resize_to_fill: [640, 160]) %>

Here, we lazily create and display a song preview, taken from the previous article's example use cases. Observe that preview takes the same arguments as variant, which you would use to transform an image.

The Skeleton of an ActiveStorage Previewer

To understand how a previewer is constructed, let's look at an example. Before we dive deep into the code, though, let's examine which previewers are available in the Rails console:

(dev)> ActiveStorage.previewers
=>
[ActiveStorage::Previewer::PopplerPDFPreviewer,
  ActiveStorage::Previewer::MuPDFPreviewer,
  ActiveStorage::Previewer::VideoPreviewer]

These are the two flavors of PDF previewers (depending on the present backend), as well as the video previewer mentioned in the introduction. We are going to study the VideoPreviewer to get a better grasp of what is required to come up with such a solution.

As of today, the implementation looks like this (I've omitted some helper code for clarity):

module ActiveStorage
  class Previewer::VideoPreviewer < Previewer
    class << self
      def accept?(blob)
        blob.video? && ffmpeg_exists?
      end
 
      def ffmpeg_exists?
        # somehow check if ffmpeg is present
      end
    end
 
    def preview(**options)
      download_blob_to_tempfile do |input|
        draw_relevant_frame_from input do |output|
          yield io: output, filename: "#{blob.filename.base}.jpg", content_type: "image/jpeg", **options
        end
      end
    end
 
    private
      def draw_relevant_frame_from(file, &block)
        # invoke a ffmpeg command to extract the first image from the video
      end
  end
end

If you get a feeling of déjà vu, that's understandable, because we encountered a very similar pattern in our previous post. Let's inspect the two important parts:

• An accept? method checks whether the previewer is applicable. In this case, it is, if the MIME type is correct and the tooling to transform video (aka ffmpeg) exists on the machine.
• The preview method does all the heavy lifting. Specifically, it's used to draw a preview image and store this as a new blob. In the above case, it just uses the first still video image as the preview.

Note: download_blob_to_tempfile is a helper method defined in the ActiveStorage::Previewer base class.

A Use Case for a Custom Previewer

Building on our example from the previous article, we will build a custom previewer for audio waveforms. Let's get started!

Audio data is usually displayed using an abstract waveform or envelope representation (where each bar stands for the loudness at a certain point in time). There are other forms, like spectrograms, but for this demonstration, let's stick with a simple example.

First of all, we are going to produce previews in the PNG format. We'll add ChunkyPNG, a library that helps with PNG manipulation, to our application:

$ bundle add chunky_png

Now, let's start with a minimal WaveformPreviewer scaffold:

# lib/active_storage/waveform_previewer.rb
 
require "chunky_png"
 
module ActiveStorage
  class WaveformPreviewer < ActiveStorage::Previewer
    class << self
      def accept?(blob)
        blob.audio? && blob.metadata[:waveform].present?
      end
    end
 
    def preview(**options)
      waveform = blob.metadata.fetch(:waveform, [])
 
      # somehow transform the stored waveform into a PNG image
    end
  end
end

The accept? class method checks if we're dealing with an audio blob. Additionally, we check whether waveform data has been stored in metadata, as we did in the previous article. In the preview method, we will have to implement a way of painting an image representing that waveform and provide it as an attachable IO stream. Let's go and do this now.

# lib/active_storage/waveform_previewer.rb
 
require "chunky_png"
 
module ActiveStorage
  class WaveformPreviewer < ActiveStorage::Previewer
    # class methods omitted
 
    def preview(**options)
      waveform = blob.metadata.fetch(:waveform, [])
 
      width = 640
      height = 240
      center = height / 2
 
      png = ChunkyPNG::Image.new(width, height, ChunkyPNG::Color::WHITE)
 
      frame_width = waveform.size / width
 
      waveform.each_with_index do |v, idx|
        next unless idx % frame_width == 0
 
        x = idx / frame_width
 
        y_val = (v * center).to_i
        y1 = center - y_val
        y2 = center + y_val
 
        png.rect(x, y1, x, y2, 0x4b0082ff, 0x4b0082ff)
      end
 
      # Write to in-memory buffer
      io = StringIO.new
      png.write(io)
      io.rewind
 
      yield io: io, filename: "#{blob.filename.base}.png", content_type: "image/png", **options
    end
  end
end

To simplify things, we assume a fixed width and height (640 by 240 pixels). We store a new instance of ChunkyPNG::Image with a white background in the png variable. Now, we have to iterate over each of the datapoints in the waveform and decide whether to draw a line for it or not. Presumably, our stored waveform has more than 640 datapoints, so we calculate 640 frames containing several datapoints. The frame_width is determined by dividing the waveform's size by 640.

When drawing the waveform, we can skip every index that is not an integer multiple of the frame width. For the ones that match, we draw a vertically centered rectangle (a line, really, since it's only one pixel wide) whose height is proportional to the value v.

Finally, we have to wrap our composed PNG image in an in-memory buffer so it's ready for ActiveStorage to be attached. That's what the yield call does: it simply creates a new attachment that's tied to the blob as a preview image.

The result looks something like this:

Beautiful, isn't it? I have one adjustment to propose, though. Drawing a waveform true to size like this looks very technical and might fit an audio editor. But if we consider a simplified version for a social media preview, we can tweak it a bit.

To make it more visually appealing, we are going to do two things:

• Make the bars 5 pixels wide (that's why we have to multiply our frame size by 5).
• Display only every other frame.

We only need to make a few adaptations to the drawing logic:

# produce bars 5 pixels apart
frame_width = (5 * waveform.size / width).floor
 
waveform.each_with_index do |v, idx|
  next unless idx % frame_width == 0
 
  frame_even = (idx / frame_width).even?
 
  x = idx * width / waveform.size
 
  y_val = (v * center).to_i
  y1 = center - y_val
  y2 = center + y_val
 
  png.rect(x, y1, x+6, y2, 0x4b0082ff, 0x4b0082ff) unless frame_even # we skip every other frame for a nice visual effect, and make the bar a bit wider
end

In addition to making the frame width bigger, we have to decide if the frame index is even or odd. We do that by dividing the index by the frame width, and if it's odd, we simply skip drawing the rectangle. To compensate for the diminished width, we increase the width by 6 pixels, so there's a bit of an overlap.

The result is a nice interleaved graph of approximate sound loudness:

To actually activate it, we have to prepend it to the list of previewers in our active_storage initializer:

# config/initializers/active_storage.rb
 
require_relative "../../lib/active_storage/waveform_analyzer.rb"
require_relative "../../lib/active_storage/waveform_previewer.rb"
 
Rails.application.config.active_storage.analyzers.prepend ActiveStorage::WaveformAnalyzer
Rails.application.config.active_storage.previewers.prepend ActiveStorage::WaveformPreviewer

Let's now look at a second use case: blurhash image previews.

Blurhash Image Previews

Full disclosure: when writing this article, I assumed that what follows below would also be attainable using a previewer. And it is, however, it also interferes with the normal variant processing logic for images. This makes it unfeasible for a vanilla Rails application.

To understand what's going on, let's look at the image transformation logic employed by ActiveStorage:

1. When applying transformations (e.g., blob.variant(...)), Rails always uses ActiveStorage::Blob#representation.
2. This method prefers to create a preview when possible, so any attempt to create a variation will be rerouted to a previewer that returns true on self.accept?(blob).
3. The only accessible parameter in this method is the blob itself. It is not possible to distinguish whether the developer meant to generate a preview or a variant using an option applied to the representation generation logic.

This limits the usability of ActiveStorage previewers for our use case: preparing a blurhash preview as a placeholder for lazily loading an image. In fact, every call to variant would be overridden by the previewer, so we could not produce scaled down thumbnails, for example.

That doesn't mean that we cannot obtain a similar result though: we just have to use different semantics. We will be able to achieve this using a new, as yet unreleased feature which will be part of Rails 8.1 (introduced in this pull request) — a custom image transformer. Let's try and put this together!

We'll line out our desired API first. We would like to be able to compute a variant from a blurhash:

<%= image_tag image.variant(blurhash: image.blob.metadata["blurhash"]) %>

Rails will now complain because we have just made up the blurhash: ... image processing method, but that is expected:

ActiveStorage::Transformers::ImageProcessingTransformer::UnsupportedImageProcessingMethod (One or more of the provided transformation methods is not supported.)

What we have to do, therefore, is subclass an image processing backend and adapt it for blurhash image calculation. In our case, we can reuse the ImageMagick transformer and insert a conditional that generates our blurhash image if it is present in the transformations hash:

# lib/active_storage/transformers/image_magick_with_blurhash.rb
 
require "blurhash"
require "chunky_png"
 
module ActiveStorage
  module Transformers
    class ImageMagickWithBlurhash < ImageMagick
      private
 
      def process(file, format:)
        if transformations[:blurhash]
          generate_blurhash_png(transformations[:blurhash])
        else
          super(file, format:)
        end
      end
 
      def generate_blurhash_png(blurhash)
        raise ArgumentError, "Missing blurhash metadata" unless blurhash
 
        width = 600
        height = 400
 
        pixels = decode_blurhash(blurhash, width, height)
 
        png = ChunkyPNG::Image.new(width, height)
        pixels.each_with_index do |(r, g, b), i|
          x = i % width
          y = i / width
          png[x, y] = ChunkyPNG::Color.rgb(r, g, b)
        end
 
        tempfile = Tempfile.new(["blurhash", ".png"], binmode: true)
        png.write(tempfile)
        tempfile.rewind
 
        tempfile
      end
 
      # decode_blurhash method omitted
    end
  end
end

We decode the blurhash, use ChunkyPNG to create an image again, and fill it with the respective pixels. We then need to store it in a temp file so that the ActiveStorage variant processor can pick it up.

I'll save you the dire details of how to produce a PNG from a blurhash, but if you're interested, you can look at this example repository.

That's about it for the image processing part. What's left to do is to register it:

# config/initializers/active_storage.rb
 
require_relative "../../lib/active_storage/blurhash_analyzer.rb"
require_relative "../../lib/active_storage/transformers/image_magick_with_blurhash.rb"
 
Rails.application.config.active_storage.analyzers.prepend ActiveStorage::BlurhashAnalyzer
 
ActiveSupport.on_load(:active_storage_blob) do
  ActiveStorage.variant_transformer = ActiveStorage::Transformers::ImageMagickWithBlurhash
end

Because the default transformer is lazily loaded, this time we have to register it in a load hook, ActiveSupport.on_load(:active_storage_blob). Once that is done, we can give it a go — for example, in the _post partial:

<!-- app/views/posts/_post.html.erb -->
 
<div id="<%= dom_id post %>">
  <p>
    <strong>Title:</strong>
    <%= post.title %>
  </p>
 
  <p>
    <strong>Body:</strong>
    <%= post.body %>
  </p>
 
  <% post.images.each do |image| %>
    <%= image_tag image.variant(resize_to_fit: [600, 400]), width: 600, height: 400 %>
    <%= image_tag image.variant(blurhash: image.blob.metadata["blurhash"]) %>
  <% end %>
</div>

Now we get a scaled-down variant of the original, side by side with the calculated blurhash image:

Writing the necessary JavaScript to swap out the blurhash preview for the real image is left as homework for the reader.

Important note: We are in Rails alpha territory here. While it's pretty certain that this functionality will land in Rails 8.1, there might be breaking changes.

Wrap Up

In this post, we looked at how to make any content stored in an ActiveStorage blob previewable using a custom class. This comes in handy to graphically represent arbitrary media content like audio waveforms, especially when preparing social media previews like open graph images or X (formerly Twitter) cards.

We extended ActiveStorage with a custom waveform previewer based on precomputed audio intensity data. Then we created our own variant transformer capable of converting blurhash data into a downloadable PNG file.

During this deep dive, we learned a lot about ActiveStorage's internals and the interfaces it provides to craft a customized experience. What other analyzers, previewers, and transformers for your media files can you think of?

Happy coding!