Learning EventMachine

This post is part of my weekly tech learning series, where I take one hour each week to try out a piece of technology that I’d like to learn.

This week I decided to try out EventMachine: a fast, simple event-processing library for Ruby.

I have to say, I was pretty impressed. Both by its documentation and also how easy it was to get started.

Running Log

I decided to stick with my running log idea from last week. It’s a simple app idea that is different enough from the standard todo list examples used everywhere.

Getting started

Right away I found links to EventMachine’s wiki which was filled with an introduction and some code snippets. I did get stuck for a bit, but that is completely my fault and my own personal bias towards reading code instead of a description of that code.

Simple Prototype

To prevent getting stuck like last week with backbone.js I decided to try a quick prototype before I started on the app. That way there would be less code to debug if there was a problem.

But there was a problem.

I was trying to build a simple server that would accept input and return it reversed (e.g. “Hello” turns into “olleH”). I copied the first example from the code snippets, started the server, and tried to telnet to it:

$ telnet 127.0.0.1 8081
Trying 127.0.0.1...
telnet: Unable to connect to remote host: Connection refused

Uh oh. I checked the server, it was still running, no exceptions raised. I tried telnet again, still nothing.

That’s when I pulled out my old sysadmin knowledge and ran `netstat -luntp`:

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:17500           0.0.0.0:*               LISTEN      10838/dropbox
tcp        0      0 0.0.0.0:24800           0.0.0.0:*               LISTEN      20984/synergys
tcp        0      0 0.0.0.0:60198           0.0.0.0:*               LISTEN      8382/skype
# Truncated a bit

This command lists every process that is listening on a TCP or UDP socket, its interface address (localhost or ip addresses), as well as the process id and program name.

That’s when I noticed that my server wasn’t even listening on a port.

Going back to my code I started inspecting line by line to see what was wrong. I even tried to copy some other example EventMachine servers and ran them (most of them worked).

Finally I found my problem.

I used the first example from the Code Snippets which is under the Client Example.

After cursing I then realized that my intuition was correct at the beginning when I though EventMachine.connect was an odd method name for starting a server.

Lesson #1: Read ALL of the documentation, not just the code.

Lesson #2: If an API seems confusing, double check that you are not reading it incorrectly.

So one change from #connect to #start_server and my server booted, showed up in netstat, and I could telnet to it.

#!/usr/bin/env ruby
require 'rubygems'
require 'eventmachine'
 
class Reverser < EventMachine::Connection
  def post_init
    send_data ">> Ready\n"
  end
 
  def receive_data(data)
    send_data data.strip.reverse + "\n"
  end
 
end
 
EventMachine.run {
  EventMachine.start_server '127.0.0.1', 8081, Reverser
  puts "Running on port 8081"
}

Building the Running Log

Now that I’ve prototyped and got EventMachine running completely it was time to start on my Running Log app. I knew I wouldn’t have enough time to implement a complete CRUD type of system so I settled on the ability to add a run and then list all of the runs that have been added (the CR__ in CRUD).

Tests

Since I was working in Ruby, I also decided to use TDD to make sure I don’t misstep along the way. I’m running 1.9.3 so using minitest was the perfect fit. I also didn’t want to mess with multiple files so I embedded the test directly into the main server file. Not only will this keep the implementation and test in one place but it also serves as some basic internal documentation.

The only problem was that I usually used the __FILE__ == $0 trick to embed tests but that wouldn’t work with EventMachine because I was executing the file directly.

[Sidebar]

__FILE__ == $0 checks if the file is getting executed directly, such as ruby your_file.rb. The technique is to code your library like normal and wrap your tests in that check. So running the file directly will run the tests while using require 'your_file' will load the library without the tests.

class YourClass
end
 
if __FILE__ == $0
  # Your testing code
end

[End sidebar]

Instead I just decided to uses ARGVs so the server starts when the first argument is “start”, otherwise to run the test suite.

Since EventMachine lets you separate your implementation logic into a module, I was able to easily include that module in a test class and not have to worry about testing much of EventMachine itself. This made testing cycle a lot faster.

Running Log Methods

Since I was only concerned with two public APIs for the server I was able to keep the implementation pretty slim:

  • #receive_data is the main method EventMachine uses when it gets input. I used a classic condition in here to see which API the client was calling.
  • #list_runs is used to run a simple text list of the runs the server has saved.
  • #add_run_from_user is used to create a new run. I didn’t want to spend much time with a data format so I went with a simple line based CSV format. Making it line based makes it easy to enter into a telnet client.

You can see the implementations below, #add_run_from_user is the most complex because I’m doing the data parsing and validation inline. In actual production code I’d probably split it up to be clearer.

All in all there was about 50 lines of implementation code and close to 100 lines of test code which covered some of the common use cases.

Screencast

Below is a screencast of me walking through the code and how the app works.

Summary

Working with EventMachine was quite fun. I know I’m not really using it to it’s full power but it’s nice to know that there wouldn’t be that much else I’d have to do to scale it up in production.

My favorite part was how well it separated the network server part from the application code. You can see in my test, the RunLogWrapper class is not using EventMachine but manages to pull in my entire implementation for testing.

I could see EventMachine being used for running web services and acting as glue servers for other systems. I’m definitely happy I spent I hour to play with EventMachine and learn the basics.

Code

Here is the full code for the server along with the embedded tests.

#!/usr/bin/env ruby
require 'rubygems'
require 'eventmachine'
 
module RunLog
  def receive_data(data)
    if data.match(/list/i)
      list_runs
    else
      add_run_from_user(data)
    end
  end
 
  def list_runs
    output = runs.inject("") do |o, run|
      o += "Date: #{run[:date]} | Distance: #{run[:distance]} | Duration: #{run[:duration]} | Pace: #{run[:pace]} | Comment: #{run[:comment]}\n"
      o
    end
    send_data output
  end
 
  def add_run_from_user(data)
    run_data = {
      :date => "",
      :distance => "",
      :duration => "",
      :pace => "",
      :comment => ""
    }
    date, distance, duration, pace, comment = data.split(',')
    run_data[:date] = date.strip if date
    run_data[:distance] = distance.strip if distance
    run_data[:duration] = duration.strip if duration
    run_data[:pace] = pace.strip if pace
    run_data[:comment] = comment.strip if comment
 
    if run_data.any? {|key, value| !value.nil? && value != ""}
      add_run(run_data)
      send_data "OK\n"
    else
      send_data "INPUT ERROR\n"
    end
 
  end
 
  def runs
    @runs || []
  end
 
  def add_run(run)
    @runs ||= []
    @runs << run
  end
end
 
 
 
command = ARGV.shift
 
case command
when "start"
  EventMachine.run {
    EventMachine.start_server '127.0.0.1', 8081, RunLog
    puts "Running on port 8081"
  }
else
  require 'minitest/autorun'
 
  class RunLogWrapper
    include RunLog
 
    attr_reader :output_buffer
 
    # Used to stub EM's methods
    def send_data(*args)
      @output_buffer ||= ""
      @output_buffer << args.join("\n")
    end
 
  end
 
  class TestRunLog < MiniTest::Unit::TestCase
    def setup
      @runlog = RunLogWrapper.new
    end
 
    def test_sanity
      assert_equal 4, 2 + 2
    end
 
    def test_receive_data_with_list_command
      @runlog.add_run({
        :date => "2012-09-14",
        :distance => "3mi",
        :duration => "30:00",
        :pace => "10:00",
        :comment => "Nice and easy run"
      })
 
      @runlog.receive_data("list")
 
      assert_equal "Date: 2012-09-14 | Distance: 3mi | Duration: 30:00 | Pace: 10:00 | Comment: Nice and easy run\n", @runlog.output_buffer
    end
 
    def test_receive_data_with_good_input
      @runlog.receive_data("2012-09-14, 3mi, 30:00, 10:00, Nice and easy run")
 
      assert_equal 1, @runlog.runs.length
      expected_run = {
        :date => "2012-09-14",
        :distance => "3mi",
        :duration => "30:00",
        :pace => "10:00",
        :comment => "Nice and easy run"
      }
      assert_equal expected_run, @runlog.runs.first
 
    end
 
    def test_receive_data_with_empty_input
      @runlog.receive_data("\n")
 
      assert_equal 0, @runlog.runs.length
    end
 
    def test_receive_data_with_missing_fields
      @runlog.receive_data("2012-09-14, 3mi, 30:00")
 
      assert_equal 1, @runlog.runs.length
      expected_run = {
        :date => "2012-09-14",
        :distance => "3mi",
        :duration => "30:00",
        :pace => "",
        :comment => ""
      }
      assert_equal expected_run, @runlog.runs.first
 
    end
 
    def test_receive_data_with_malformed_input
      @runlog.receive_data("This input, isn't quite, right, but, it still is accepted, for now")
 
      assert_equal 1, @runlog.runs.length
      expected_run = {
        :date => "This input",
        :distance => "isn't quite",
        :duration => "right",
        :pace => "but",
        :comment => "it still is accepted"
      }
      assert_equal expected_run, @runlog.runs.first
 
    end
 
  end
 
end