Jan 07

Testing with the help of machinist, forgery, cucumber, webrat and rspec

etienne @ 6:23 pm

I’ve been using rspec for my testing for some now and have played around the edges with rspec user stories. When I started working on a new application a month or two ago I thought it would be a great opportunity to revisit my testing approach and my testing toolkit. After reading and researching the current trends I have settled on the following:

  • machinist - After trying many fixture replacement plugins I’m happy with machinist, so far it does everything I need.
  • forgery - Machinist includes Sham which allows me to create dummy data for my fixtures, I use forgery which is a very nice fake data generator to supply the data to Sham,
  • cucumber - Is the replacement for rspec user stories, it allows you to write plain text stories (features) and execute them as functional tests.
  • webrat - It lets you write expressive and robust acceptance tests for a Ruby web application.
  • rspec - Is a Behaviour Driven Development (BDD) framework for writing executable code examples.

After reading lot’s of articles and looking at lots of example code I was finding it all very daunting, I found myself suffering from a severe case of coders block. I really did not know where to start, how to structure my code, how to write the features and so on. I was hoping that the Rspec book might have been released before I had to start development to give me some direction but the beta release has been delayed and I had to face my fears and dive in.

Once I started, I was pleasantly surprised how natural the whole process felt, after writing my first half dozen features or so I was really starting to enjoy it! In this article I want to give a brief overview of what I did to get started, show some example features and steps, as well as demonstrating how all the tools in my toolkit mesh together to form a very nice testing framework. I’m still very new to the process of writing features and steps and am still becoming familiar with all the tools, so I’m sure that I will make adjustments as I become more experienced in using the framework.

I will not go into how to install the plugins as this is well documented for each plugin.

Creating my fixtures

I create my fixtures using machinist. To use machinist you use a class method called blueprint which is an extension of ActiveRecord::Base which means you can use it on any of your ActiveRecord models. In my application I have an "Account" link that allows the user to modify their account details including the password and email address which is the first feature I wanted to write. So the first thing I did was to create some blueprints for my user models in the blueprint.rb file which resides in the spec directory. The first thing I do is define some standard sham’s that I will use in my blueprints, I use forgery to populate these shams with some good dummy data.

require ‘forgery’

# Shams
# We use forgery to make up some test data

Sham.name  { NameForgery.full_name }
Sham.login  { InternetForgery.user_name }
Sham.email  { InternetForgery.email_address }
Sham.password  { BasicForgery.password }
Sham.string { BasicForgery.text }
Sham.text { LoremIpsumForgery.text }

# Blueprints

Role.blueprint do
  name { ‘guest’ }
end

SiteUser.blueprint do
  user_type { ‘SiteUser’ }
  login { Sham.login }
  name { Sham.name }
  email = Sham.email
  email { email }
  email_confirmation { email }
  pwd = Sham.password
  password { pwd }
  password_confirmation { pwd }
  accept_terms { ‘true’ }
  time_zone { ‘Melbourne’ }
end

OpenidUser.blueprint do
  user_type { ‘OpenidUser’ }
  time_zone { ‘Melbourne’ }
  email { Sham.email }
end

These blueprints allow me to easily create some objects using make, i.e.

user = SiteUser.make

Where to put my test files

There are any number of ways to structure your files and directories, after reading this article I decided to create a new sub-directory for each model that I wanted to test, so my directory structure looks something like this:

Directory structure

 

So for model User I created a users directory which contains a user.feature file and a steps_definitions sub-directory where the step files will be located, in this case we have user-steps.rb.

Setting up my env.rb

Before we start writing features we need to tweak the cucumber env.rb file. I’ve added code to load webrat and machinist, my file looks like this:

# Sets up the Rails environment for Cucumber
ENV["RAILS_ENV"] = "test"
require File.expand_path(File.dirname(__FILE__) + ‘/../../config/environment’)
require ‘cucumber/rails/world’
Cucumber::Rails.use_transactional_fixtures

# Add webrat
require "webrat"
Webrat.configure do |config|
  config.mode = :rails
end

# Comment out the next line if you’re not using RSpec’s matchers (should / should_not) in your steps.
require ‘cucumber/rails/rspec’

# Add machinist
require File.join(RAILS_ROOT, ’spec’, ‘blueprints’)

Writing my first feature

The first feature I wanted to write was the ability for the user to display their account details once they had logged in. I initially started writing the feature using my own words to describe it, for example:

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I go to the "Account" page
  Then I should see my account details
    And I should see a "Change Password" link
    And I should see "Change Email" link

I then implemented each of these in my steps file using webrat. It’s only after some time that I realized that cucumber comes with a webrat steps definition file, i.e. webrat_steps.rb. Which contains a lot of standard webrat steps which I could directly call from my feature definition. So I could rewrite my scenario using the webrat steps:

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

The steps in bold are all webrat steps that I do not have to implement in my own steps file as they are implemented in the webrat steps file! There are only two steps that I have to define myself.

Step 1: Given I am logged in as a SiteUser

This step is quite involved as I have to figure out how to login a user using a user name/password, I also need to cater for those users that use an OpenID. I decided to create a step_helper.rb file which is located in the /features/step-definitions sub-directory and will be automatically loaded by cucumber, which will contain any shared steps. It currently contains only one step that handles the login process for a user and looks like this:

# Login
# e.g.
# Given I login as a SiteUser in the guest role
# Given I login as a OpenidUser in the admin role
#
Given /I login as a (.*) in the (.*) role/i do |user_type, role|
  role = Role.make
  @user = Object::const_get(user_type).make_unsaved(:roles => role) 
  if (!@user.openid)
    @user.activate!
    visit login_path
    fill_in("login", :with => @user.login)
    fill_in("password", :with => @user.password)
  else
    # identity_url not allowed to be set via mass assignment in blueprint
    @user.identity_url = ‘https://good.openid.url/’
    @user.activate!
    visit login_with_openid_path
    fill_in("openid_identifier", :with => @user.identity_url)   
  end 
  click_button("Log in")
  Then ‘I should see "Dashboard"’
end

 To use it I create a separate step within my steps file. One important point to note is that you can reuse steps by calling them directly from within other steps. In the example below we are calling the shared login step from within the step by calling Given "I login as a #{user_type} in the guest role".

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

This step can then be used in my feature like this:

Given I am logged in as a SiteUser

Step 2: And I should see my account details

The last step we need to implement is the one that checks to make sure that the users account details are actually being displayed. The step do this looks like this:

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

Running my features

Before we run the feature here is the content of our feature and step file so far.

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

And my user-steps.rb looks like this:

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

When I run the the feature assuming I’ve implemented the code I get the following output:

Feature result

 

More example features

Here are a few more example features and the steps.

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

Scenario: Display my account information if I am an OpenidUser
  Given I am logged in as a OpenidUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should not see "Change Password"
    And I should not see "Change Email"

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in my new password details
    And I press "Change your password"
  Then my password should be changed
    And I should see "Account"
    And I should see "Password successfully updated"

Scenario: Not allow me to change my password if the old password incorrect
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in an incorrect old password
    And I press "Change your password"
  Then my password should not be changed
    And I should see "You password was not changed, your old password is incorrect."

Scenario: Not allow me to change my password when the password and confirmation is not the same
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in an incorrect password confirmation
    And I press "Change your password"
  Then my password should not be changed
    And I should see "New password does not match the password confirmation."

Scenario: Allow me to change my account details
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I fill in my new account details
    And I press "Save"
  Then my account details should be changed
    And I should see "Account details updated."

Scenario: Allow me to change my email
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Email"
    And I fill in my new email address
    And I press "Change your email"
  Then my email should be changed
    And I should see "Account"
    And I should see "Email successfully updated"

Scenario: Not allow me to change my email when the email and confirmation is not the same
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Email"
    And I fill in an incorrect email confirmation
    And I press "Change your email"
  Then my email should not be changed
    And I should see "New email does not match the email confirmation."

Steps in user-steps.rb

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

When /I fill in my new password details/ do
  When "I fill in \"old_password\" with \"#{@user.password}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_123"’
end

When /I fill in an incorrect old password/ do
  When "I fill in \"old_password\" with \"#{@user.password + ‘xyz’}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_123"’
end

When /I fill in an incorrect password confirmation/ do
  When "I fill in \"old_password\" with \"#{@user.password}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_1234"’
end

When /I fill in my new account details/ do
  When ‘I fill in "user_name" with "Fred Flintstone"’
  When ‘I fill in "user_company" with "Acme Rocks"’
  When ‘I select "(GMT+10:00) Canberra" from "user_time_zone"’
end

Then /my password should be changed/ do
  SiteUser.find_by_id(@user.id).authenticated?(’test_123′).should == true
end

Then /my password should not be changed/ do
  SiteUser.find_by_id(@user.id).authenticated?(’test_123′).should == false
end

Then /my account details should be changed/ do
  u = SiteUser.find_by_id(@user.id)
  u.name.should == ‘Fred Flintstone’
  u.company.should == ‘Acme Rocks’
  u.time_zone.should == ‘Canberra’
end

When /I fill in my new email address/ do
  When ‘I fill in "email" with "test@isp.com"’
  When ‘I fill in "email_confirmation" with "test@isp.com"’
end

When /I fill in an incorrect email confirmation/ do
  When ‘I fill in "email" with "test@isp.com"’
  When ‘I fill in "email_confirmation" with "test123@isp.com"’
end

Then /my email should be changed/ do
  SiteUser.find_by_id(@user.id).email.should == ‘test@isp.com’
end

Then /my email should not be changed/ do
  SiteUser.find_by_id(@user.id).email.should_not == ‘test@isp.com’
end 

Feature Writing Style

There seems to be two styles that you can use when writing your features, "Imperative" and "Declarative" which is described in detail in the article Imperative vs Declarative Scenarios in User Stories written by Ben Mabey. I understand the extreme cases as demonstrated by the examples given by Ben, but what about a feature such as this one, can this be classed as being written using the Imperative style?

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in my new password details
    And I press "Change your password"
  Then my password should be changed
    And I should see "Account"
    And I should see "Password successfully updated"

Does the Imperative style dictate that all steps appear in the feature, the example above do have some steps that are defined in the steps file, but there are a few other details that could be moved to the steps file as well. So if the one above is Imperative does this modified version make it declarative?

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I go to the "Change Password" page
    And I fill in my new password details
  Then my password details should be changed

At this stage I’m not sure, for now I will stay with the first style as it provides details that may be important to someone reading the feature but hides the non-essential detail in the steps file.

9 Responses to “Testing with the help of machinist, forgery, cucumber, webrat and rspec”

  1. Ben Mabey says:

    Great overview, thanks for sharing. The more recent scenarios that I have been writing have been somewhat of a hybrid of declarative and imperative. As you said with the imperative approach you get a better idea of what is going on, and so I try use that style when writing about the parts where the details are important to the story. The ability of doing step tables in cucumber has also changed how I write scenarios, since communicating a lot of information isn’t as cumbersome.

  2. Billee D. says:

    Thanks for sharing your work. I have been using Shoulda and rcov along with the integrated tests for quite some time, but I have been looking at other alternatives; something more robust. I was researching rspec one day recently and found Cucumber, which really piqued my interest. What a really great way to write tests. With Machinist and SHAM it also seems to make dummy data and fixtures easier to manage. Wiring fixtures together is not my idea of fun. But your approach seems to take some of the drudgery out of this portion of the testing cycle. Thanks for sharing! :-)

  3. etienne says:

    Hi Billee,

    Thanks for the feedback. I’m really enjoying the cucumber experience, the whole process of writing the feature then writing rspec steps and implementing the code feels very natural. The ease of reusing and sharing steps between features is one of the great benefits that I have found with cucumber.

  4. Robby on Rails : 20 articles on Cucumber and a free beverage recipe! says:

    […] Testing with the help of machinist, forgery, cucumber, webrat and rspec, Etienne van Tonder […]

  5. Marcello Nuccio says:

    Hi Billee,

    Thanks for sharing your experience.
    One question. I think the following assertion is… well, not an assertion:

    SiteUser.find_by_id(@user.id).authenticated?(’test_123′) == false

    and should be:

    SiteUser.find_by_id(@user.id).authenticated?(’test_123′).should == false

    or it will do nothing.

  6. etienne says:

    Hi Marcello,

    Thanks for picking up the error, I’ve made the change.

  7. Ennuyer.net » Blog Archive » I am way behind on my rails link blogging. Link dump and reboot. says:

    […] Testing with the help of machinist, forgery, cucumber, webrat and rspec | IT.Signals […]

  8. Kiran says:

    This was a very useful post, thanks for sharing. I did not have to add machinist to the env.rb file and it still worked. I also used it with Faker and it seems to work great. The only issue i am really having is that it is really slow to run cucumber features. I only have one feature with one scenerio and the run time is like 30-50 seconds which is crazy long. Any hints on what might be the issue?

  9. Delicious Bookmarks for October 24th from 09:20 to 10:57 « Lâmôlabs says:

    […] Testing with the help of machinist, forgery, cucumber, webrat and rspec | IT.Signals – October 24th ( tags: ruby rails testing example machinist forgery cucumber rspec webrat tutorial guide ) […]

Leave a Reply