Jan 07 2009

Testing with the help of machinist, forgery, cucumber, webrat and rspec

etienne @ 6:23 pm

I’ve been using rspec for my testing for some now and have played around the edges with rspec user stories. When I started working on a new application a month or two ago I thought it would be a great opportunity to revisit my testing approach and my testing toolkit. After reading and researching the current trends I have settled on the following:

  • machinist - After trying many fixture replacement plugins I’m happy with machinist, so far it does everything I need.
  • forgery - Machinist includes Sham which allows me to create dummy data for my fixtures, I use forgery which is a very nice fake data generator to supply the data to Sham,
  • cucumber - Is the replacement for rspec user stories, it allows you to write plain text stories (features) and execute them as functional tests.
  • webrat - It lets you write expressive and robust acceptance tests for a Ruby web application.
  • rspec - Is a Behaviour Driven Development (BDD) framework for writing executable code examples.

After reading lot’s of articles and looking at lots of example code I was finding it all very daunting, I found myself suffering from a severe case of coders block. I really did not know where to start, how to structure my code, how to write the features and so on. I was hoping that the Rspec book might have been released before I had to start development to give me some direction but the beta release has been delayed and I had to face my fears and dive in.

Once I started, I was pleasantly surprised how natural the whole process felt, after writing my first half dozen features or so I was really starting to enjoy it! In this article I want to give a brief overview of what I did to get started, show some example features and steps, as well as demonstrating how all the tools in my toolkit mesh together to form a very nice testing framework. I’m still very new to the process of writing features and steps and am still becoming familiar with all the tools, so I’m sure that I will make adjustments as I become more experienced in using the framework.

I will not go into how to install the plugins as this is well documented for each plugin.

Creating my fixtures

I create my fixtures using machinist. To use machinist you use a class method called blueprint which is an extension of ActiveRecord::Base which means you can use it on any of your ActiveRecord models. In my application I have an "Account" link that allows the user to modify their account details including the password and email address which is the first feature I wanted to write. So the first thing I did was to create some blueprints for my user models in the blueprint.rb file which resides in the spec directory. The first thing I do is define some standard sham’s that I will use in my blueprints, I use forgery to populate these shams with some good dummy data.

require ‘forgery’

# Shams
# We use forgery to make up some test data

Sham.name  { NameForgery.full_name }
Sham.login  { InternetForgery.user_name }
Sham.email  { InternetForgery.email_address }
Sham.password  { BasicForgery.password }
Sham.string { BasicForgery.text }
Sham.text { LoremIpsumForgery.text }

# Blueprints

Role.blueprint do
  name { ‘guest’ }
end

SiteUser.blueprint do
  user_type { ‘SiteUser’ }
  login { Sham.login }
  name { Sham.name }
  email = Sham.email
  email { email }
  email_confirmation { email }
  pwd = Sham.password
  password { pwd }
  password_confirmation { pwd }
  accept_terms { ‘true’ }
  time_zone { ‘Melbourne’ }
end

OpenidUser.blueprint do
  user_type { ‘OpenidUser’ }
  time_zone { ‘Melbourne’ }
  email { Sham.email }
end

These blueprints allow me to easily create some objects using make, i.e.

user = SiteUser.make

Where to put my test files

There are any number of ways to structure your files and directories, after reading this article I decided to create a new sub-directory for each model that I wanted to test, so my directory structure looks something like this:

Directory structure

 

So for model User I created a users directory which contains a user.feature file and a steps_definitions sub-directory where the step files will be located, in this case we have user-steps.rb.

Setting up my env.rb

Before we start writing features we need to tweak the cucumber env.rb file. I’ve added code to load webrat and machinist, my file looks like this:

# Sets up the Rails environment for Cucumber
ENV["RAILS_ENV"] = "test"
require File.expand_path(File.dirname(__FILE__) + ‘/../../config/environment’)
require ‘cucumber/rails/world’
Cucumber::Rails.use_transactional_fixtures

# Add webrat
require "webrat"
Webrat.configure do |config|
  config.mode = :rails
end

# Comment out the next line if you’re not using RSpec’s matchers (should / should_not) in your steps.
require ‘cucumber/rails/rspec’

# Add machinist
require File.join(RAILS_ROOT, ’spec’, ‘blueprints’)

Writing my first feature

The first feature I wanted to write was the ability for the user to display their account details once they had logged in. I initially started writing the feature using my own words to describe it, for example:

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I go to the "Account" page
  Then I should see my account details
    And I should see a "Change Password" link
    And I should see "Change Email" link

I then implemented each of these in my steps file using webrat. It’s only after some time that I realized that cucumber comes with a webrat steps definition file, i.e. webrat_steps.rb. Which contains a lot of standard webrat steps which I could directly call from my feature definition. So I could rewrite my scenario using the webrat steps:

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

The steps in bold are all webrat steps that I do not have to implement in my own steps file as they are implemented in the webrat steps file! There are only two steps that I have to define myself.

Step 1: Given I am logged in as a SiteUser

This step is quite involved as I have to figure out how to login a user using a user name/password, I also need to cater for those users that use an OpenID. I decided to create a step_helper.rb file which is located in the /features/step-definitions sub-directory and will be automatically loaded by cucumber, which will contain any shared steps. It currently contains only one step that handles the login process for a user and looks like this:

# Login
# e.g.
# Given I login as a SiteUser in the guest role
# Given I login as a OpenidUser in the admin role
#
Given /I login as a (.*) in the (.*) role/i do |user_type, role|
  role = Role.make
  @user = Object::const_get(user_type).make_unsaved(:roles => role) 
  if (!@user.openid)
    @user.activate!
    visit login_path
    fill_in("login", :with => @user.login)
    fill_in("password", :with => @user.password)
  else
    # identity_url not allowed to be set via mass assignment in blueprint
    @user.identity_url = ‘https://good.openid.url/’
    @user.activate!
    visit login_with_openid_path
    fill_in("openid_identifier", :with => @user.identity_url)   
  end 
  click_button("Log in")
  Then ‘I should see "Dashboard"’
end

 To use it I create a separate step within my steps file. One important point to note is that you can reuse steps by calling them directly from within other steps. In the example below we are calling the shared login step from within the step by calling Given "I login as a #{user_type} in the guest role".

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

This step can then be used in my feature like this:

Given I am logged in as a SiteUser

Step 2: And I should see my account details

The last step we need to implement is the one that checks to make sure that the users account details are actually being displayed. The step do this looks like this:

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

Running my features

Before we run the feature here is the content of our feature and step file so far.

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

And my user-steps.rb looks like this:

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

When I run the the feature assuming I’ve implemented the code I get the following output:

Feature result

 

More example features

Here are a few more example features and the steps.

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

Scenario: Display my account information if I am an OpenidUser
  Given I am logged in as a OpenidUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should not see "Change Password"
    And I should not see "Change Email"

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in my new password details
    And I press "Change your password"
  Then my password should be changed
    And I should see "Account"
    And I should see "Password successfully updated"

Scenario: Not allow me to change my password if the old password incorrect
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in an incorrect old password
    And I press "Change your password"
  Then my password should not be changed
    And I should see "You password was not changed, your old password is incorrect."

Scenario: Not allow me to change my password when the password and confirmation is not the same
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in an incorrect password confirmation
    And I press "Change your password"
  Then my password should not be changed
    And I should see "New password does not match the password confirmation."

Scenario: Allow me to change my account details
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I fill in my new account details
    And I press "Save"
  Then my account details should be changed
    And I should see "Account details updated."

Scenario: Allow me to change my email
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Email"
    And I fill in my new email address
    And I press "Change your email"
  Then my email should be changed
    And I should see "Account"
    And I should see "Email successfully updated"

Scenario: Not allow me to change my email when the email and confirmation is not the same
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Email"
    And I fill in an incorrect email confirmation
    And I press "Change your email"
  Then my email should not be changed
    And I should see "New email does not match the email confirmation."

Steps in user-steps.rb

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

When /I fill in my new password details/ do
  When "I fill in \"old_password\" with \"#{@user.password}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_123"’
end

When /I fill in an incorrect old password/ do
  When "I fill in \"old_password\" with \"#{@user.password + ‘xyz’}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_123"’
end

When /I fill in an incorrect password confirmation/ do
  When "I fill in \"old_password\" with \"#{@user.password}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_1234"’
end

When /I fill in my new account details/ do
  When ‘I fill in "user_name" with "Fred Flintstone"’
  When ‘I fill in "user_company" with "Acme Rocks"’
  When ‘I select "(GMT+10:00) Canberra" from "user_time_zone"’
end

Then /my password should be changed/ do
  SiteUser.find_by_id(@user.id).authenticated?(’test_123′).should == true
end

Then /my password should not be changed/ do
  SiteUser.find_by_id(@user.id).authenticated?(’test_123′).should == false
end

Then /my account details should be changed/ do
  u = SiteUser.find_by_id(@user.id)
  u.name.should == ‘Fred Flintstone’
  u.company.should == ‘Acme Rocks’
  u.time_zone.should == ‘Canberra’
end

When /I fill in my new email address/ do
  When ‘I fill in "email" with "test@isp.com"’
  When ‘I fill in "email_confirmation" with "test@isp.com"’
end

When /I fill in an incorrect email confirmation/ do
  When ‘I fill in "email" with "test@isp.com"’
  When ‘I fill in "email_confirmation" with "test123@isp.com"’
end

Then /my email should be changed/ do
  SiteUser.find_by_id(@user.id).email.should == ‘test@isp.com’
end

Then /my email should not be changed/ do
  SiteUser.find_by_id(@user.id).email.should_not == ‘test@isp.com’
end 

Feature Writing Style

There seems to be two styles that you can use when writing your features, "Imperative" and "Declarative" which is described in detail in the article Imperative vs Declarative Scenarios in User Stories written by Ben Mabey. I understand the extreme cases as demonstrated by the examples given by Ben, but what about a feature such as this one, can this be classed as being written using the Imperative style?

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in my new password details
    And I press "Change your password"
  Then my password should be changed
    And I should see "Account"
    And I should see "Password successfully updated"

Does the Imperative style dictate that all steps appear in the feature, the example above do have some steps that are defined in the steps file, but there are a few other details that could be moved to the steps file as well. So if the one above is Imperative does this modified version make it declarative?

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I go to the "Change Password" page
    And I fill in my new password details
  Then my password details should be changed

At this stage I’m not sure, for now I will stay with the first style as it provides details that may be important to someone reading the feature but hides the non-essential detail in the steps file.


Feb 14 2008

refresh_to plugin rspec matcher

etienne @ 10:00 pm

We have been using the refresh_to plugin in one of our projects. For those who do not know, the refresh_to plugin adds a refresh_to method to the ActionController::Base which allows us to avoid the IE security warnings when moving from a secure SSL page to an insecure page.

We are using rspec for our testing and when I went to write some tests to test some code that uses the refresh_to method I found out that the standard redirect_to rspec matcher does not work with refresh_to. The same problem exist for the standard test method assert_response :redirect and assert_redirected_to my_url, but the plugin does supply replacement tests for these methods. The problem is that it does not supply a replacement for the rspec matcher so I had to write my own. 

# The standard rspec redirect_to matcher does not work with the refresh_to
# plugin methods, i.e. refresh_to and refresh_back_or_default.
# This is also the case with the with the standard Rails functional
# tests - assert_response :redirect and assert_redirected_to my_url.
# The plugin has replacement test methods for these but they do not
# have an rspec replacement for redirect_to so I’ve had to write my own.
#
module Spec
  module Rails
    module Matchers
     
      class RefreshTo  #:nodoc:

        def initialize(controller, expected)
          @expected = expected
          @controller = controller
        end

        def matches?(response)
          match_data = response.body.match(/<meta http-equiv="refresh" content="0;url=([\S]+)">/)
          return false if (!match_data)
          parts = match_data[1].split("http://test.host")
          @actual = parts.last
          return match()
        end
         
        def match
          case @expected.class.to_s
            when ‘String’
              parts = @expected.split("http://test.host")                      
              @expected = parts.last
              return @actual == @expected
            when ‘Hash’
              @expected = @controller.url_for(@expected)
              return match()
          end         
        end

        def failure_message
          return %Q{expected redirect to #{@expected.inspect}, got redirect to #{@actual.inspect}}
        end

        def negative_failure_message
          return %Q{expected not to be redirected to #{@expected.inspect}, but was}
        end

        def description
          "redirect to #{@actual.inspect}"
        end
      end
     
      # :call-seq:
      #   response.should refresh_to(url)
      #   response.should refresh_to(:action => action_name)
      #   response.should refresh_to(:controller => controller_name, :action => action_name)
      #   response.should_not refresh_to(url)
      #   response.should_not refresh_to(:action => action_name)
      #   response.should_not refresh_to(:controller => controller_name, :action => action_name)
      #
      # Passes if the response is a redirect to the url, action or controller/action.
      # Useful in controller specs (integration or isolation mode).
      #
      # == Examples
      #
      #   response.should refresh_to("path/to/action")
      #   response.should refresh_to("http://test.host/path/to/action")
      #   response.should refresh_to(:action => ‘list’)
      def refresh_to(opts)
        RefreshTo.new(controller, opts)
      end
    end

  end
end