This post is part of my weekly tech learning series, where I take one hour each week to try out a piece of technology that I'd like to learn. Make sure to read to the end, where I have a screencast overview of the final application.

This week I decided to learn a bit more about Redis. I've used Redis in Chirk HR already but only as part of a non-critical service to see how it performs under load (I'd like to screencast this later). Today I wanted to use Redis as an actual data store but first, what is Redis?

What is Redis?

Redis is an open source key-value store. I think of it as a light nosql database with a lot of flexibility.

Today's application

My idea for today was to build my running journal but this time as a CRUD style web app. I want to stay close to the metal so I can really see how Redis works so I picked my second favorite web framework: Sinatra.

Basically I wanted to mimic a Rails scaffold inside Sinatra using Redis as the data storage. I'm also using the redis gem because it is very similar to the actual Redis API. This means I'll be implementing the data layer myself, without an ORM.

(Normally I'd use ActiveRecord or Datamapper for production applications instance but I'm more concerned with learning and exploration with this.)

Since I will be doing a lot of data layer work I'm also opting for heavy TDD with this app, using minitest and autotest.

Here is the Gemfile I ended up with:

source :rubygems

gem "sinatra", :require => "sinatra/base" gem "shotgun" gem "redis"

group :test do gem "minitest" gem "rack-test" gem "autotest" gem "autotest-notification" end

Use TDD for the application behavior

After getting everything hooked up I decided to split the tests into two files:

test_redis_running.rb to test the RedisRunning Sinatra application at the integration layer. This would be the controller and view if you're thinking about it as a MVC app.
test_run.rb to test the Run class, which is used as the data layer between the application and Redis.

I like to start with integration testing and write pending tests for each major workflow or feature. I came up with the following integration tests to get started with:

test_get_root_response
test_get_root_list_runs
test_get_root_add_form_present
test_post_new_with_valid_run
test_post_new_with_invalid_run
test_get_edit_response
test_get_edit_show_form
test_put_update_with_valid_update
test_put_update_with_invalid_update
test_delete_response
test_delete_should_delete_record

This is very similar to Rails scaffold except I decided to put the new form on the list of runs (:index) and to skip the details of a run (:show). With such a small data model, a list provides a perfect summary.

The first test I wrote was to check that getting the index was successful, which isn't about Redis at all so I won't bore you with the details of it.

The second test on the other hand (test_get_root_list_runs), is the important one (and as you'll see later is all that I had time for). Here I wanted to do a few things:

Create a few runs and save them to Redis
Load the index page
Check that the runs were displayed

In order to implement this test I have to do a full cycle of saving and loading runs from Redis. This meant I'd need to build the data layer now.

Here is the full integration test.

def test_get_root_list_runs run1 = Run.new(:date => "2012-09-28", :distance => "3mi", :duration => "30:00", :pace => "10:00", :comment => "A run") assert run1.save, "Save failed"

run2 = Run.new(:date => "2012-09-29",
               :distance => "3mi",
               :duration => "27:00",
               :pace => "9:00",
               :comment => "Another run")
assert run2.save, "Save failed"

get '/'

assert last\_response.body.include?("A run")
assert last\_response.body.include?("Another run")

end

RedisRunning Data Layer

The integration test needed three things from my Run class:

Create an object with some attributes
Save the object to Redis
Load the saved object out of Redis

I started with the first, using standard Ruby accessors.

Run accessors and attributes

A run has five attributes that a user should be able to set:

date
distance
duration
pace
comment

Additionally, I'm going to need some way to uniquely identify each run. In Postgres I'd just use an id field as a primary key but from what I see, Redis doesn't have primary keys. Only keys.

At first I thought of using the date as the primary key, but many runners will run multiple times per day so that won't work. I could use a combination of the date and another field like distance or duration but there is also a chance those will conflict also (e.g. two runs per day, around a 1 mile track at a specific pace).

Redis doesn't have primary keys so you might need to roll your own.

So I'll need to be creating my own unique ids for each run. I've used UUIDs before and they seem like a good fit here.

So using attr_accessor and initialize I can easily fulfill these requirements.

The test:

class RunTest < MiniTest::Unit::TestCase def test_has_attributes run = Run.new

\['date','distance','duration','pace','comment'\].each do |attribute|
  assert run.respond\_to?(attribute), "Run does not respond to #{attribute}"
  assert run.respond\_to?(attribute + "="), "Run does not respond to #{attribute}="
end

end

def test_should_allow_setting_attributes_on_initialize run = Run.new(:date => '1', :distance => '2', :duration => '3', :pace => '4', :comment => '5')

assert\_equal '1', run.date
assert\_equal '2', run.distance
assert\_equal '3', run.duration
assert\_equal '4', run.pace
assert\_equal '5', run.comment

end end

The implementation:

class Run attr_accessor :id attr_accessor :date attr_accessor :distance attr_accessor :duration attr_accessor :pace attr_accessor :comment

def initialize(attributes={}) attributes.each do |attribute, value| setter = "#{attribute}=" self.send(setter, value) if self.respond_to?(setter) end end end

Generating unique ids with UUID

I've used some libraries to generate UUIDs in Ruby but I just found out that Ruby 1.9.3 now has UUID support in the standard library as part of SecureRandom. This makes the UUID code simple.

The test.

def test_id_should_be_a_uuid run = Run.new

assert run.id.is\_a?(String)
assert\_equal 36, run.id.length
assert\_equal 4, run.id.count('-')

end

The implementation.

# Way up top outside the Run class... require 'securerandom'

In the Run class...

def id @id ||= SecureRandom.uuid @id end

By using ||= I can make sure that if a Run already has an id then I won't generate it again (existing record).

I also wrote a brute force test to make sure Ruby's UUIDs were unique and that I wasn't misreading the docs. It's not really production quality but good for exploration of a new API:

def test_id_should_be_unique # Lets create a bunch of runs and check their uniqueness. # TODO: not the best solution but time is limited runs = 100.times.collect { Run.new.id }

assert\_equal 100, runs.uniq.length, "Duplicate ids found"

end

Saving a run to Redis

Now that I have a data format in Ruby, I need to save it to Redis. Since Redis is a key value store, I decided to save my runs where the key is the id (uuid) and the data is a hash of strings. This way it would be easy to load the data back into a Ruby object.

Since the integration test uses the #save method, that's what I need to implement first. After writing two quick tests I'm ready to figure out how to save to Redis:

def test_save_with_valid_attributes run = Run.new(:date => "2012-09-28", :distance => "3mi", :duration => "30:00", :pace => "10:00", :comment => "A run")

assert run.save, "Save failed"

assert $db.keys.include?(run.id), "Run not found in Redis"

end

def test_save_with_mostly_empty_attributes run = Run.new(:date => "", :distance => "", :duration => "", :pace => "", :comment => "")

assert run.save, "Save failed"

assert $db.keys.include?(run.id), "Run not found in Redis"

end

Redis uses a #set method to save data. It takes the key and value of the data and will return an "OK" when it succeeds.

One thing that tripped me up for a minute was that if I just sent a Ruby hash to Redis it would get saved but as the string interpolation of the hash like

'{"f"=>"g"}'

This wouldn't work because I'd need to use eval to turn that back into a Ruby hash if I wanted to load it into a Run object.

So I turned to my old standby, JSON. By calling #to_json on the Ruby hash before sending it to Redis, I can be sure that I can unserialize it on the way back out. This is what Redis stored when I encoded the hash above into JSON.

'{"f":"g"}'

That figured out, the #save method became easy. Just use #set to save the JSON using a run's uuid as the key.

def save response = $db.set(id, { "id" => id, "date" => @date, "distance" => @distance, "duration" => @duration, "pace" => @pace, "comment" => @comment }.to_json)

return response == "OK"

end

Why not use the native hash support in Redis?

Redis natively supports hashes with operations like hset, hget, and hkeys. I probably should have use them instead of JSON but I forget all about them until after I was done. Switching to a native Redis hash wouldn't be difficult at all, because all of that logic is hidden inside two methods of the Run class. The Sinatra application itself doesn't even know that Redis is used at all.

Loading a Ruby object out of Redis

The next step in the integration test is to load an object out of Redis and into Ruby, to be used by erb when creating the HTML page.

Mimicking ActiveRecord's API again, I decided to name this method #find and only support finding a run by it's uuid. The test is simple:

Create a run object
Save it to Redis
Load the run object out of Redis
Make sure its data is correct

def test_find_existing_run run = Run.new(:date => "2012-09-28", :distance => "3mi", :duration => "30:00", :pace => "10:00", :comment => "A run")

assert run.save, "Save failed"

redis\_run = Run.find(run.id)
assert\_equal run.id, redis\_run.id
assert\_equal "2012-09-28", redis\_run.date
assert\_equal "3mi", redis\_run.distance
assert\_equal "30:00", redis\_run.duration
assert\_equal "10:00", redis\_run.pace
assert\_equal "A run", redis\_run.comment

end

Notice how I'm using an actual run object and its attributes this time instead of a raw hash from Redis. This is what the Run data model class is providing me.

Thinking about the #find method, it needs to do two things:

Load the raw data from Redis, if it's available
Convert the raw data into a Run object

Loading the data is quite easy. Redis uses the #get method to fetch a single object, all you have to do is give it the key. If it finds an object, it is returned. If Redis doesn't find the object, you'll get a nil back. This makes it simple to use an if statement for control flow between these two cases.

Since the raw data is JSON, I needed to feed it through a JSON parser to convert it to a Ruby hash. This isn't all I needed though, I wanted a Run class so I can interact with it right away. Since my earlier work created an initialize method that could set attributes based on a hash, I could easily send the data there and get a Run object back out.

In less words, here is the code:

def self.find(uuid)

record = $db.get(uuid)

if record
  return new(JSON.parse(record))
else
  return nil
end

end

Where are we now?

Going back up the stack to the integration test, the data layer (Run) works for initialing, saving, and loading a record. That means all I have to do is to load all of the Runs from Redis and show them on the index page.

But, I'm out of time now.

I already have an idea of how I'd go about listing the Runs, using Redis's keys method which returns all keys and then using mget to get multiple keys in one call. After that it's just a delete action and the HTML.

Summary

Even though I didn't get any user interface written, I created most of the data model and had some good tests to make sure the data was behaving the way I wanted. Finishing up the application would be straight-forward and would just take some more time than I have available.

Like I said in the introduction, I have Redis running in production for Chirk HR while I test how easy it is to keep running. As of this writing, the Redis server has been online and receiving data for over 16 days and 16 hours with no downtime (probably longer, my monitoring system lost its network connection for a bit).

Given this exploration and my production test I've gotten very comfortable with Redis and will be adding it to my developer toolbox for future apps. I think it's great when you need a simple and stable key/value store.

Screencast

Full code

Gemfile

source :rubygems

gem "sinatra", :require => "sinatra/base" gem "shotgun" gem "redis"

group :test do gem "minitest" gem "rack-test" gem "autotest" gem "autotest-notification" end

RedisRunning Sinatra application

require 'rubygems' require 'sinatra' require 'securerandom' require 'json'

$db = Redis.new $db.select(1) # To get a different database

class Run attr_accessor :id attr_accessor :date attr_accessor :distance attr_accessor :duration attr_accessor :pace attr_accessor :comment

def initialize(attributes={}) attributes.each do |attribute, value| setter = "#{attribute}=" self.send(setter, value) if self.respond_to?(setter) end end

def id @id ||= SecureRandom.uuid @id end

Save a run to Redis

def save response = $db.set(id, { "id" => id, "date" => @date, "distance" => @distance, "duration" => @duration, "pace" => @pace, "comment" => @comment }.to_json)

return response == "OK"

end

def self.find(uuid)

record = $db.get(uuid)

if record
  return new(JSON.parse(record))
else
  return nil
end

end end

class RedisRunning < Sinatra::Base get '/' do "Hi" end end

Application tests

require 'rubygems' require 'bundler'

Bundler.require

require './redis_running' require 'minitest/autorun' require 'rack/test'

ENV['RACK_ENV'] = 'test'

class RedisRunningTest < MiniTest::Unit::TestCase include Rack::Test::Methods

def app RedisRunning end

def setup # Select a test database and clean it $db.select(2) assert_equal "OK", $db.flushdb assert_equal [], $db.keys end

def test_sanity assert_equal 2, 1 + 1 end

def test_get_root_response get '/'

assert last\_response.ok?

end

def test_get_root_list_runs run1 = Run.new(:date => "2012-09-28", :distance => "3mi", :duration => "30:00", :pace => "10:00", :comment => "A run") assert run1.save, "Save failed"

run2 = Run.new(:date => "2012-09-29",
               :distance => "3mi",
               :duration => "27:00",
               :pace => "9:00",
               :comment => "Another run")
assert run2.save, "Save failed"

get '/'

assert last\_response.body.include?("A run")
assert last\_response.body.include?("Another run")

end

def test_get_root_add_form_present get '/'

assert last\_response.body.include?("form")
assert last\_response.body.include?("Add a new run")

end

def test_post_new_with_valid_run end

def test_post_new_with_invalid_run end

def test_get_edit_response end

def test_get_edit_show_form end

def test_put_update_with_valid_update end

def test_put_update_with_invalid_update end

def test_delete_response end

def test_delete_should_delete_record end end

Data storage tests