ruby

Ruby Templating: Baking an Interpreter

Benedikt Deicke

Benedikt Deicke on

Ruby Templating: Baking an Interpreter

We hope you’ve got your stroopwafels warmed on top of your coffee because today we’re gluing things up with sticky stroop (the syrup that makes the two halves of a stroopwafel stick together). In the first two parts of our series, we baked a Lexer and a Parser and now, we’re adding the Interpreter and gluing things together by pouring stroop over them.

Ingredients

Alright! Let’s get the kitchen ready for baking and put our ingredients on the table. Our interpreter needs two ingredients or pieces of information to do its job: the previously generated Abstract Syntax Tree (AST) and the data we want to embed into the template. We’ll call this data the environment.

To traverse the AST, we’ll implement the interpreter using the visitor pattern. A visitor (and therefore our interpreter) implements a generic visit method that accepts a node as a parameter, processes this node and potentially calls the visit method again with some (or all) of the node’s children, depending on what makes sense for the current node.

ruby
module Magicbars class Interpreter attr_reader :root, :environment def self.render(root, environment = {}) new(root, environment).render end def initialize(root, environment = {}) @root = root @environment = environment end def render visit(root) end def visit(node) # Process node end end end

Before continuing, let’s also create a small Magicbars.render method that accepts a template and an environment and outputs the rendered template.

ruby
module Magicbars def self.render(template, environment = {}) tokens = Lexer.tokenize(template) ast = Parser.parse(tokens) Interpreter.render(ast, environment) end end

With this in place, we’ll be able to test the interpreter without having to construct the AST by hand.

ruby
Magicbars.render('Welcome to {{name}}', name: 'Ruby Magic') # => nil

To no surprise, it currently doesn’t return anything. So let’s start implementing the visit method. As a quick reminder, here’s what the AST for this template looks like.

Simple AST

For this template, we’ll have to process four different node types: Template, Content, Expression, and Identifier. To do this, we could just put a huge case statement inside our visit method. However, this will become unreadable pretty quickly. Instead, let’s make use of Ruby’s metaprogramming capabilities to keep our code a bit more organized and readable.

ruby
module Magicbars class Interpreter # ... def visit(node) short_name = node.class.to_s.split('::').last send("visit_#{short_name}", node) end end end

The method accepts a node, gets its class name, and removes any modules from it (check out our article on cleaning strings if you’re interested in different ways of doing this). Afterward, we use send to call a method that handles this specific type of node. The method name for each type is made up of the demodulized class name and the visit_ prefix. It’s a bit unusual to have capital letters in method names, but it makes the method’s intent pretty clear.

ruby
module Magicbars class Interpreter # ... def visit_Template(node) # Process template nodes end def visit_Content(node) # Process content nodes end def visit_Expression(node) # Process expression nodes end def visit_Identifier(node) # Process identifier nodes end end end

Let’s start by implementing the visit_Template method. It should just process all the statements of the node and join the results.

ruby
def visit_Template(node) node.statements.map { |statement| visit(statement) }.join end

Next, let’s look at the visit_Content method. As a content node just wraps a string, the method is as simple as it gets.

ruby
def visit_Content(node) node.content end

Now, let’s move on to the visit_Expression method where the substitution of the placeholder with the real value happens.

ruby
def visit_Expression(node) key = visit(node.identifier) environment.fetch(key, '') end

And finally, for the visit_Expression method to know what key to fetch from the environment, let’s implement the visit_Identifier method.

ruby
def visit_Identifier(node) node.value end

With these four methods in place, we get the desired result when we try to render the template again.

ruby
Magicbars.render('Welcome to {{name}}', name: 'Ruby Magic') # => Welcome to Ruby Magic

Interpreting Block Expressions

We wrote a lot of code to implement what a simple gsub could do. So let’s move on to a more complex example.

ruby
Welcome to {{name}}! {{#if subscribed}} Thank you for subscribing to our mailing list. {{else}} Please sign up for our mailing list to be notified about new articles! {{/if}} Your friends at {{company_name}}

As a reminder, here’s what the corresponding AST looks like.

Complex AST

There’s only one node type that we don’t yet handle. It’s the visit_BlockExpression node. In a way, it is similar to the visit_Expression node, but depending on the value it either continues to process the statements or the inverse_statements of the BlockExpression node.

ruby
def visit_BlockExpression(node) key = visit(node.identifier) if environment[key] node.statements.map { |statement| visit(statement) }.join else node.inverse_statements.map { |statement| visit(statement) }.join end end

Looking at the method, we notice that the two branches are very similar, and they also look similar to the visit_Template method. They all handle the visiting of all nodes of an Array, so let’s extract a visit_Array method to clean things up a bit.

ruby
def visit_Array(nodes) nodes.map { |node| visit(node) } end

With the new method in place, we can remove some code from the visit_Template and visit_BlockExpression methods.

ruby
def visit_Template(node) visit(node.statements).join end def visit_BlockExpression(node) key = visit(node.identifier) if environment[key] visit(node.statements).join else visit(node.inverse_statements).join end end

Now that our interpreter handles all node types, let’s try and render the complex template.

ruby
Magicbars.render(template, { name: 'Ruby Magic', subscribed: true, company_name: 'AppSignal' }) # => Welcome to Ruby Magic! # # # Please sign up for our mailing list to be notified about new articles! # # # Your friends at AppSignal

That almost looks right. But on a closer look, we notice that the message is prompting us to sign up for the mailing list, even though we provided subscribed: true in the environment. That doesn’t seem right…

Adding Support for Helper Methods

Looking back at the template, we notice that there’s an if in the block expression. Instead of looking up the value of subscribed in the environment, the visit_BlockExpression is looking up the value of if. As it is not present in the environment, the call returns nil, which is false.

We could stop here and declare that we’re not trying to imitate Handlebars but Mustache, and get rid of the if in the template, which will give us the desired result.

ruby
Welcome to {{name}}! {{#subscribed}} Thank you for subscribing to our mailing list. {{else}} Please sign up for our mailing list to be notified about new articles! {{/subscribed}} Your friends at {{company_name}}

But why stop when we are having fun? Let’s go the extra mile and implement helper methods. They might come in handy for other things as well.

Let’s start by adding a helper method support to simple expressions. We’ll add a reverse helper, that reverses strings passed to it. In addition, we’ll add a debug method that tells us the class name of a given value.

ruby
def helpers @helpers ||= { reverse: ->(value) { value.to_s.reverse }, debug: ->(value) { value.class } } end

We use simple lambdas to implement these helpers and store them in a hash so that we can look them up by their name.

Next, let’s modify visit_Expression to perform a helper lookup before trying a value lookup in the environment.

ruby
def visit_Expression(node) key = visit(node.identifier) if helper = helpers[key] arguments = visit(node.arguments).map { |k| environment[k] } return helper.call(*arguments) end environment[key] end

If there is a helper matching the given identifier, the method will visit all the arguments and try to lookup values for them. Afterward, it will call the method and pass all the values as arguments.

ruby
Magicbars.render('Welcome to {{reverse name}}', name: 'Ruby Magic') # => Welcome to cigaM ybuR Magicbars.render('Welcome to {{debug name}}', name: 'Ruby Magic') # => Welcome to String

With that in place, let’s finally implement an if and an unless helper. In addition to the arguments, we’ll pass two lambdas to them so that they can decide if we should continue interpreting the node’s statements or inverse_statements.

ruby
def helpers @helpers ||= { if: ->(value, block:, inverse_block:) { value ? block.call : inverse_block.call }, unless: ->(value, block:, inverse_block:) { value ? inverse_block.call : block.call }, # ... } end

The necessary changes to visit_BlockExpression are similar to what we did with visit_Expression, only this time, we also pass the two lambdas.

ruby
def visit_BlockExpression(node) key = visit(node.identifier) if helper = helpers[key] arguments = visit(node.arguments).map { |k| environment[k] } return helper.call( *arguments, block: -> { visit(node.statements).join }, inverse_block: -> { visit(node.inverse_statements).join } ) end if environment[key] visit(node.statements).join else visit(node.inverse_statements).join end end

And with this, our baking is done! We can render the complex template that started this journey into the world of lexers, parsers, and interpreters.

ruby
Magicbars.render(template, { name: 'Ruby Magic', subscribed: true, company_name: 'AppSignal' }) # => Welcome to Ruby Magic! # # # Thank you for subscribing to our mailing list. # # # Your friends at AppSignal

Only Scratching the Surface

In this three-part series, we covered the basics of creating a templating language. These concepts can also be used to create interpreted programming languages (like Ruby). Admittedly, we glossed over a couple of things (like proper error handling 🙀) and only scratched the surface of the underpinnings of today’s programming languages.

We hope you enjoyed the series and if you want more of that, subsribe to the Ruby Magic list. If you are now hungry for stroopwafels, drop us a line and we might be able to fuel you with those as well!

Benedikt Deicke

Benedikt Deicke

Guest author Benedikt Deicke is a software engineer and CTO of Userlist. On the side, he’s writing a book about building SaaS applications in Ruby on Rails.

All articles by Benedikt Deicke

Become our next author!

Find out more

AppSignal monitors your apps

AppSignal provides insights for Ruby, Rails, Elixir, Phoenix, Node.js, Express and many other frameworks and libraries. We are located in beautiful Amsterdam. We love stroopwafels. If you do too, let us know. We might send you some!

Discover AppSignal
AppSignal monitors your apps