Jan 07 2009

Testing with the help of machinist, forgery, cucumber, webrat and rspec

etienne @ 6:23 pm

I’ve been using rspec for my testing for some now and have played around the edges with rspec user stories. When I started working on a new application a month or two ago I thought it would be a great opportunity to revisit my testing approach and my testing toolkit. After reading and researching the current trends I have settled on the following:

  • machinist - After trying many fixture replacement plugins I’m happy with machinist, so far it does everything I need.
  • forgery - Machinist includes Sham which allows me to create dummy data for my fixtures, I use forgery which is a very nice fake data generator to supply the data to Sham,
  • cucumber - Is the replacement for rspec user stories, it allows you to write plain text stories (features) and execute them as functional tests.
  • webrat - It lets you write expressive and robust acceptance tests for a Ruby web application.
  • rspec - Is a Behaviour Driven Development (BDD) framework for writing executable code examples.

After reading lot’s of articles and looking at lots of example code I was finding it all very daunting, I found myself suffering from a severe case of coders block. I really did not know where to start, how to structure my code, how to write the features and so on. I was hoping that the Rspec book might have been released before I had to start development to give me some direction but the beta release has been delayed and I had to face my fears and dive in.

Once I started, I was pleasantly surprised how natural the whole process felt, after writing my first half dozen features or so I was really starting to enjoy it! In this article I want to give a brief overview of what I did to get started, show some example features and steps, as well as demonstrating how all the tools in my toolkit mesh together to form a very nice testing framework. I’m still very new to the process of writing features and steps and am still becoming familiar with all the tools, so I’m sure that I will make adjustments as I become more experienced in using the framework.

I will not go into how to install the plugins as this is well documented for each plugin.

Creating my fixtures

I create my fixtures using machinist. To use machinist you use a class method called blueprint which is an extension of ActiveRecord::Base which means you can use it on any of your ActiveRecord models. In my application I have an "Account" link that allows the user to modify their account details including the password and email address which is the first feature I wanted to write. So the first thing I did was to create some blueprints for my user models in the blueprint.rb file which resides in the spec directory. The first thing I do is define some standard sham’s that I will use in my blueprints, I use forgery to populate these shams with some good dummy data.

require ‘forgery’

# Shams
# We use forgery to make up some test data

Sham.name  { NameForgery.full_name }
Sham.login  { InternetForgery.user_name }
Sham.email  { InternetForgery.email_address }
Sham.password  { BasicForgery.password }
Sham.string { BasicForgery.text }
Sham.text { LoremIpsumForgery.text }

# Blueprints

Role.blueprint do
  name { ‘guest’ }
end

SiteUser.blueprint do
  user_type { ‘SiteUser’ }
  login { Sham.login }
  name { Sham.name }
  email = Sham.email
  email { email }
  email_confirmation { email }
  pwd = Sham.password
  password { pwd }
  password_confirmation { pwd }
  accept_terms { ‘true’ }
  time_zone { ‘Melbourne’ }
end

OpenidUser.blueprint do
  user_type { ‘OpenidUser’ }
  time_zone { ‘Melbourne’ }
  email { Sham.email }
end

These blueprints allow me to easily create some objects using make, i.e.

user = SiteUser.make

Where to put my test files

There are any number of ways to structure your files and directories, after reading this article I decided to create a new sub-directory for each model that I wanted to test, so my directory structure looks something like this:

Directory structure

 

So for model User I created a users directory which contains a user.feature file and a steps_definitions sub-directory where the step files will be located, in this case we have user-steps.rb.

Setting up my env.rb

Before we start writing features we need to tweak the cucumber env.rb file. I’ve added code to load webrat and machinist, my file looks like this:

# Sets up the Rails environment for Cucumber
ENV["RAILS_ENV"] = "test"
require File.expand_path(File.dirname(__FILE__) + ‘/../../config/environment’)
require ‘cucumber/rails/world’
Cucumber::Rails.use_transactional_fixtures

# Add webrat
require "webrat"
Webrat.configure do |config|
  config.mode = :rails
end

# Comment out the next line if you’re not using RSpec’s matchers (should / should_not) in your steps.
require ‘cucumber/rails/rspec’

# Add machinist
require File.join(RAILS_ROOT, ’spec’, ‘blueprints’)

Writing my first feature

The first feature I wanted to write was the ability for the user to display their account details once they had logged in. I initially started writing the feature using my own words to describe it, for example:

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I go to the "Account" page
  Then I should see my account details
    And I should see a "Change Password" link
    And I should see "Change Email" link

I then implemented each of these in my steps file using webrat. It’s only after some time that I realized that cucumber comes with a webrat steps definition file, i.e. webrat_steps.rb. Which contains a lot of standard webrat steps which I could directly call from my feature definition. So I could rewrite my scenario using the webrat steps:

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

The steps in bold are all webrat steps that I do not have to implement in my own steps file as they are implemented in the webrat steps file! There are only two steps that I have to define myself.

Step 1: Given I am logged in as a SiteUser

This step is quite involved as I have to figure out how to login a user using a user name/password, I also need to cater for those users that use an OpenID. I decided to create a step_helper.rb file which is located in the /features/step-definitions sub-directory and will be automatically loaded by cucumber, which will contain any shared steps. It currently contains only one step that handles the login process for a user and looks like this:

# Login
# e.g.
# Given I login as a SiteUser in the guest role
# Given I login as a OpenidUser in the admin role
#
Given /I login as a (.*) in the (.*) role/i do |user_type, role|
  role = Role.make
  @user = Object::const_get(user_type).make_unsaved(:roles => role) 
  if (!@user.openid)
    @user.activate!
    visit login_path
    fill_in("login", :with => @user.login)
    fill_in("password", :with => @user.password)
  else
    # identity_url not allowed to be set via mass assignment in blueprint
    @user.identity_url = ‘https://good.openid.url/’
    @user.activate!
    visit login_with_openid_path
    fill_in("openid_identifier", :with => @user.identity_url)   
  end 
  click_button("Log in")
  Then ‘I should see "Dashboard"’
end

 To use it I create a separate step within my steps file. One important point to note is that you can reuse steps by calling them directly from within other steps. In the example below we are calling the shared login step from within the step by calling Given "I login as a #{user_type} in the guest role".

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

This step can then be used in my feature like this:

Given I am logged in as a SiteUser

Step 2: And I should see my account details

The last step we need to implement is the one that checks to make sure that the users account details are actually being displayed. The step do this looks like this:

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

Running my features

Before we run the feature here is the content of our feature and step file so far.

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

And my user-steps.rb looks like this:

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

When I run the the feature assuming I’ve implemented the code I get the following output:

Feature result

 

More example features

Here are a few more example features and the steps.

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

Scenario: Display my account information if I am an OpenidUser
  Given I am logged in as a OpenidUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should not see "Change Password"
    And I should not see "Change Email"

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in my new password details
    And I press "Change your password"
  Then my password should be changed
    And I should see "Account"
    And I should see "Password successfully updated"

Scenario: Not allow me to change my password if the old password incorrect
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in an incorrect old password
    And I press "Change your password"
  Then my password should not be changed
    And I should see "You password was not changed, your old password is incorrect."

Scenario: Not allow me to change my password when the password and confirmation is not the same
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in an incorrect password confirmation
    And I press "Change your password"
  Then my password should not be changed
    And I should see "New password does not match the password confirmation."

Scenario: Allow me to change my account details
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I fill in my new account details
    And I press "Save"
  Then my account details should be changed
    And I should see "Account details updated."

Scenario: Allow me to change my email
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Email"
    And I fill in my new email address
    And I press "Change your email"
  Then my email should be changed
    And I should see "Account"
    And I should see "Email successfully updated"

Scenario: Not allow me to change my email when the email and confirmation is not the same
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Email"
    And I fill in an incorrect email confirmation
    And I press "Change your email"
  Then my email should not be changed
    And I should see "New email does not match the email confirmation."

Steps in user-steps.rb

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

When /I fill in my new password details/ do
  When "I fill in \"old_password\" with \"#{@user.password}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_123"’
end

When /I fill in an incorrect old password/ do
  When "I fill in \"old_password\" with \"#{@user.password + ‘xyz’}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_123"’
end

When /I fill in an incorrect password confirmation/ do
  When "I fill in \"old_password\" with \"#{@user.password}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_1234"’
end

When /I fill in my new account details/ do
  When ‘I fill in "user_name" with "Fred Flintstone"’
  When ‘I fill in "user_company" with "Acme Rocks"’
  When ‘I select "(GMT+10:00) Canberra" from "user_time_zone"’
end

Then /my password should be changed/ do
  SiteUser.find_by_id(@user.id).authenticated?(’test_123′).should == true
end

Then /my password should not be changed/ do
  SiteUser.find_by_id(@user.id).authenticated?(’test_123′).should == false
end

Then /my account details should be changed/ do
  u = SiteUser.find_by_id(@user.id)
  u.name.should == ‘Fred Flintstone’
  u.company.should == ‘Acme Rocks’
  u.time_zone.should == ‘Canberra’
end

When /I fill in my new email address/ do
  When ‘I fill in "email" with "test@isp.com"’
  When ‘I fill in "email_confirmation" with "test@isp.com"’
end

When /I fill in an incorrect email confirmation/ do
  When ‘I fill in "email" with "test@isp.com"’
  When ‘I fill in "email_confirmation" with "test123@isp.com"’
end

Then /my email should be changed/ do
  SiteUser.find_by_id(@user.id).email.should == ‘test@isp.com’
end

Then /my email should not be changed/ do
  SiteUser.find_by_id(@user.id).email.should_not == ‘test@isp.com’
end 

Feature Writing Style

There seems to be two styles that you can use when writing your features, "Imperative" and "Declarative" which is described in detail in the article Imperative vs Declarative Scenarios in User Stories written by Ben Mabey. I understand the extreme cases as demonstrated by the examples given by Ben, but what about a feature such as this one, can this be classed as being written using the Imperative style?

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in my new password details
    And I press "Change your password"
  Then my password should be changed
    And I should see "Account"
    And I should see "Password successfully updated"

Does the Imperative style dictate that all steps appear in the feature, the example above do have some steps that are defined in the steps file, but there are a few other details that could be moved to the steps file as well. So if the one above is Imperative does this modified version make it declarative?

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I go to the "Change Password" page
    And I fill in my new password details
  Then my password details should be changed

At this stage I’m not sure, for now I will stay with the first style as it provides details that may be important to someone reading the feature but hides the non-essential detail in the steps file.


Jan 06 2009

Rails and Sybase Adapative Server Anywhere (SQLAnywhere) - Part 3

etienne @ 8:28 am

For anyone struggling to get SQL Anywhere working with Rails there is some good news, I’ve received and email from Eric Farrar from Sybase to let me know that he has been doing a lot of work on developing a new adaptor for SQL Anywhere. I’ve included the content of his email below for anyone interested.

There is now a ActiveRecord adapter specifically for SQL Anywhere. It is available on RubyForge at sqlanywhere.rubyforge.org/. The source is hosted on GitHub and is available at github.com/sqlanywhere. This driver does not go through ODBC, but instead uses the new C API library to directly interface with SQL Anywhere databases. Please note that this driver can only be used with SQL Anywhere 10 and greater.

 


May 20 2008

Rails and Sybase Adapative Server Anywhere (SQLAnywhere) - Part 2

etienne @ 10:26 am

As discussed in the Rails and Sybase Adapative Server Anywhere (SQLAnywhere) article I had to modify the odbc_adapter.rb code to allow the adapter to be used with my SybaseASA DB. Making the change to the adapter code directly is not an acceptable solution and until the developers of the adapter apply the change I had to come up with an alternate solution.

The solution I came up with was to monkey-patch the method responsible for checking the DB type to allow SybaseASA to be seen as a valid DB. I added the following code to my environment.rb:

# Monkeypatch the ODBC Adapter so that it will work with SybaseASA
# see
http://odbc-rails.rubyforge.org/index.html
module ActiveRecord
  module ConnectionAdapters
    class ODBCAdapter < AbstractAdapter 
      alias :orig_dbmsNameToSym :dbmsNameToSym

      def dbmsNameToSym(dbmsName, dbmsVer)   
        begin
          # Call original method
          symbl = orig_dbmsNameToSym(dbmsName, dbmsVer)
        rescue ActiveRecord::ActiveRecordError
            # Catch exception if db not found in existing list, check
            # if SybaseASA, if not raise exception
            if dbmsName =~ /adaptiveserveranywhere/i
              symbl = :sqlanywhere
            else
              raise ActiveRecord::ActiveRecordError, "ODBCAdapter: Unsupported database (#{dbmsName})"
            end
        end
        return symbl
      end

    end
  end
end


May 19 2008

Rails and Sybase Adapative Server Anywhere (SQLAnywhere)

etienne @ 1:13 pm

In a previous article (Sybase DB Adapter) I wrote how I tried to install the Sybase adapter in an attempt to get access to an existing SybaseASA 9.0.2 DB. After doing some research I found that this adapter only works for the Enterprise edition of Sybase (i.e. SybaseASE).

So I needed to find an alternative that would work with SybaseASA or SQLAnywhere. After a lot of searching I came across a group dedicated to this task, at this stage there is not a lot of activity. One of the postings from Bryan Lahartinger proved very helpful, in the article he mentions "odbc-rails". I did a google search and found there is a Ruby project ODBC Adapter for Ruby on Rails / Active Record that has been developing an ODBC adapter for use with Rails.

The latest release has added support for SQLAnywhere so I dowloaded and installed it. There is a home page for this project which provides detailed installation instructions. I do my Rails development on Windows using InstantRails, a quick check confirmed that Christian Werner’s ODBC module comes preloaded with InstantRails so I did not have to install it separately. I downloaded the gem file for V2.0 from rubyforge and installed it using:

gem install -l activerecord-odbc-adapter-2.0.gem –include-dependencies

I setup my database.yml file to use the DSN I had previously defined to access the DB, i.e.

development:
  adapter: odbc
  dsn: mydatabase
  username: uid
  password: pwd

I started my server which loaded without any problem but when I tried to run my application I got the following error:

ODBCAdapter: Unsupported database (adaptiveserveranywhere)

As SybaseASA is the same as SQLAnywhere I changed the following line in odbc_adapter.rb (line #:1585) to get me going from:

elsif dbmsName =~ /SQLAnywhere/i

to:

elsif dbmsName =~ /SQLAnywhere/i or dbmsName =~ /adaptiveserveranywhere/i

So far everything seems to working without any problems, I’ve asked if it will be possible to get this change added to the gem, if not I will need to monkeypatch the source. Next I will attempt to do an install under Suse Linux as this is where the application will ultimately reside.

Update: I’ve now added the monkey-patch to my application see Rails ODBC Adapter with Sybase Adaptive Server Anywhere (SybaseASA)


May 15 2008

Sybase DB Adapter

etienne @ 12:26 pm

 Update: This adapter only works with the enterprise edition of Sybase (i.e. Sybase ASE) not with SQLAnywhere (Sybase ASA)

One of my Rails projects require me to connect to a Sybase ASA database, I quickly figured out that I needed to install the sybase adapter as in Rails 2.0.2 these and some of the other databases have been pulled out of Rails core. According to the information in this article you can achieve this by using this command:

gem install activerecord-sybase-adapter

When I tried this I got the following error:

ERROR:  could not find activerecord-database-adapter locally or in a repository

The only way I got it to work was to use the -s option ("Use URL as the remote source for gems"), i.e.

gem install activerecord-sybase-adapter -s http://gems.rubyonrails.org


Feb 14 2008

refresh_to plugin rspec matcher

etienne @ 10:00 pm

We have been using the refresh_to plugin in one of our projects. For those who do not know, the refresh_to plugin adds a refresh_to method to the ActionController::Base which allows us to avoid the IE security warnings when moving from a secure SSL page to an insecure page.

We are using rspec for our testing and when I went to write some tests to test some code that uses the refresh_to method I found out that the standard redirect_to rspec matcher does not work with refresh_to. The same problem exist for the standard test method assert_response :redirect and assert_redirected_to my_url, but the plugin does supply replacement tests for these methods. The problem is that it does not supply a replacement for the rspec matcher so I had to write my own. 

# The standard rspec redirect_to matcher does not work with the refresh_to
# plugin methods, i.e. refresh_to and refresh_back_or_default.
# This is also the case with the with the standard Rails functional
# tests - assert_response :redirect and assert_redirected_to my_url.
# The plugin has replacement test methods for these but they do not
# have an rspec replacement for redirect_to so I’ve had to write my own.
#
module Spec
  module Rails
    module Matchers
     
      class RefreshTo  #:nodoc:

        def initialize(controller, expected)
          @expected = expected
          @controller = controller
        end

        def matches?(response)
          match_data = response.body.match(/<meta http-equiv="refresh" content="0;url=([\S]+)">/)
          return false if (!match_data)
          parts = match_data[1].split("http://test.host")
          @actual = parts.last
          return match()
        end
         
        def match
          case @expected.class.to_s
            when ‘String’
              parts = @expected.split("http://test.host")                      
              @expected = parts.last
              return @actual == @expected
            when ‘Hash’
              @expected = @controller.url_for(@expected)
              return match()
          end         
        end

        def failure_message
          return %Q{expected redirect to #{@expected.inspect}, got redirect to #{@actual.inspect}}
        end

        def negative_failure_message
          return %Q{expected not to be redirected to #{@expected.inspect}, but was}
        end

        def description
          "redirect to #{@actual.inspect}"
        end
      end
     
      # :call-seq:
      #   response.should refresh_to(url)
      #   response.should refresh_to(:action => action_name)
      #   response.should refresh_to(:controller => controller_name, :action => action_name)
      #   response.should_not refresh_to(url)
      #   response.should_not refresh_to(:action => action_name)
      #   response.should_not refresh_to(:controller => controller_name, :action => action_name)
      #
      # Passes if the response is a redirect to the url, action or controller/action.
      # Useful in controller specs (integration or isolation mode).
      #
      # == Examples
      #
      #   response.should refresh_to("path/to/action")
      #   response.should refresh_to("http://test.host/path/to/action")
      #   response.should refresh_to(:action => ‘list’)
      def refresh_to(opts)
        RefreshTo.new(controller, opts)
      end
    end

  end
end 


Jan 09 2008

SMERF Released!

etienne @ 5:21 pm

 

Update 11 January 2008: SMERF web site completed.

I’m happy to announce the release of SMERF V0.0.1!

I’m still in the process of setting up the official web site but the source is now available for download. The README file contains a lot of information about the plugin including installation and configuration notes. You can also checkout the documentation in the rdoc directory which includes the API docs and the README in a more readable format.

There are several ways to download SMERF:

Direct download

Get it from RubyForge

Subversion

**Create a new smerf directory in your /vendor/plugins directory and change into the new directory

Check out the stable release

svn checkout http://smerf.rubyforge.org/svn/stable .

or Check out a particular release

svn checkout http://smerf.rubyforge.org/svn/tags/REL-0.0.1 .

or Check out the development code

svn checkout http://smerf.rubyforge.org/svn/trunk .

See the README for details of how to install and configure the plugin.

Until I get my bug tracker sorted out if you have any problems please feel free to contact me on smerf at cascadia dot com dot au.

Enjoy!

 


Jan 02 2008

Announcing SMERF - Simple MEta Rails Form

etienne @ 6:18 pm

 

Update 11 January 2008: SMERF web site completed.

Update 9 January 2008: SMERF Released! 

Update 8 January 2008: I’ve completed the documentation and testing. I’ve created a project on RubyForge which is waiting approval. If this is not ready by tomorrow I will make an alternate link available for download, so if all goes well I should have a download available tomorrow.

I am putting the final touches to a new plugin called SMERF which stands for Simple MEta Rails Form. The plugin is a result of me extracting the code I used for one of my projects which I discussed in an earlier article Ruby on Rails Survey Builder.

I’ve spend the past few weeks rewriting parts of it to make it more generic so that it can be used for all sorts of things including survey’s, questionnaires, data collection forms and so on. Some of the important changes I’ve made include

  • Allow question validations to be written using Ruby code. You can now create your own validation methods as required, there are also a couple of standard system validation methods that come with the plugin. You can specify any number of validation methods for a question by comma separating each method name, e.g. validation: validate_mandatory_question, validate_date_format
  • User responses are now stored in a DB table allowing you to perform analysis on the question responses using SQL.

ToDo V0.2

  • Currently there are no tests
  • Allow forms to be broken into smaller forms, currently you get all questions on a single form
  • Allow the save of user responses as you move from form to form

I’m currently writing a plugin generator that will make installation of the plugin easier, hopefully I’ll have this done very soon. I then need to finish the documentation which is mostly done.

I will keep you informed and let you know when the plugin will be available.

 


Nov 28 2007

Cleaning up Rails Session Records

etienne @ 8:07 pm

Similar:
  • None Found
  • In the Rails Cookbook there is an article describing how to clean up stale session records periodically, i.e. 13.14 Cleaning Up Residual Session Records. When I attempted to set this up as described in the book there was a couple of problems.

    1. I have a few additional require statements in my config/environment.rb which includes some of my custom classes which I’m serializing.

    require ‘app/models/survey_item’
    require ‘app/models/survey_group’

    The problem was that the cron job was failing as it could not find these files so I had to modify the cron job from the suggested

    */10 * * * * ruby /srv/www/htdocs/apps/<appname>/script/runner \
    script/runner -e production SessionCleanup.nuke_old_db_session

    to

    */10 * * * * cd /srv/www/htdocs/apps/<appname> && ruby script/runner \
    -e production SessionCleanup.nuke_old_db_sessions

    2. Now my cron job was running but if was deleting all my sessions, it was as if it was comparing the wrong time so the condition was always true. I use UTC time in my application and realised I had to make a slight modification to the function in the environment.rb that deletes the session records to take this into consideration, so I changed the function from:

    # Clean up session
    class SessionCleanup
      def self.nuke_old_db_sessions
        CGI::Session::ActiveRecordStore::Session.destroy_all(
          [’updated_at < ?’, 20.minutes.ago]
        )
      end
    end

    to

    # Clean up session
    class SessionCleanup
      def self.nuke_old_db_sessions
        CGI::Session::ActiveRecordStore::Session.destroy_all(
          [’updated_at < ?’, 20.minutes.ago.utc]
        )
      end
    end
     

    By adding the .utc to 20.minutes.ago fixed the problem.

     

    Tag: cron, Rails, Ruby

    Nov 22 2007

    Rails Deployment and Installation - Nginx + Mongrel Cluster + Subversion + Capistrano

    etienne @ 7:17 pm

    Ok so I’ve finished my new Rails application and it’s now time to deploy, which has turned out to be somewhat more challenging than I thought. I’m deploying to a dedicated host running SuSe 9.3.

    Installation check list:

    1. Ruby
    2. RubyGems
    3. Rails
    4. Mongrel + Mongrel Cluster
    5. Nginx
    6. Subversion
    7. Capistrano

    So roll up your sleeves and lets get into it.

    1. Ruby

    We will build Ruby using the latest source that we will download from the Ruby web site. Before we can do this we need to do some initial setup which includes making sure the required packages are installed that will allow us to build Ruby from source.

    1. Start YAST, Software, Install and Remove Software

    2. Check if Ruby is already installed, remove it if it is

    3. Install the following packages if they have not already been installed

    • zlib-devel
    • make
    • gcc 
    • openssl
    • openssl-devel

    We then need to download the latest version (1.8.6-p110 at time of writing) of the Ruby source which can be downloaded from the official Ruby web site. Once you have downloaded the source we can start the build process.

    Create a temporary directory where we can save the source and do all of our builds:
    mkdir /tmp/build

    Save the Ruby source file to the new directory and extract all of the files using 
    tar zxvf ruby-1.8.6-p110.tar.gz

    Change into the new source directory created by the extraction process 
    cd ruby-1.8.6-p110

    Setup our make files
    ./configure –prefix=/usr –with–openssl

    Build Ruby
    make

    Install 
    sudo make install

    Check installation worked
    ruby -v
    which should return the Ruby version, e.g. ruby 1.8.6 (2007-09-23 patchlevel 110) [i686-linux]

    Remove the source files
    rm -rf  ruby-1.8.6-p110

    2. RubyGems

    As we do not want to run Rails without installing RubyGems we need to download the source code and build it. Go to RubyForge and download the latest source code (0.9.4 at time of writing).

    Save the source code in the temporary directory we created for Ruby above, i.e.
    /tmp/build

    Extract the source code using
    tar zxvf rubygems-0.9.4.tgz

    Change to the source directory created by the extraction process
    cd rubygems-0.9.4

    Perform the installation
    sudo ruby ./setup.rb 

    Remove source files
    rm -rf rubygems-0.9.4

    3. Rails

    We can now install Rails which is something you may already have done in your development environment. Use the include-dependencies option to make sure all required files are installed.

    Install Rails
    sudo gem install rails –include-dependencies 

    Verify the installation worked
    rails -v
    which returns the Rails version number for example Rails 1.2.5

    4. Mongrel + Mongrel Cluster

    To install Mongrel you will need to have Ruby 1.8.4 (or later) and RubyGems installed.

    Install Mongrel
    sudo gem install mongrel –include-dependencies

    We will be using a cluster of three mongrel processes, you need to decide how many will be suitable for your application. To run a cluster of mongrel processes we need to install the mongrel_cluster gem, for details on this gem see the Mongrel web site.

    Install the Mongrel Cluster gem
    sudo gem install mongrel_cluster

    Now that we have Mongrel and Mongrel Cluster installed we can continue with the configuration. I created a test Rails application to test my setup and make sure everything is working correctly. Later when I install my application I will make the required changes to these configs to work correctly with my "real" application.

    Create the Rails application we will use for testing, I’m placing it in the apache root directory under a new apps directory. To find the document root go to /etc/apache2 and do a grep on documentroot, i.e. grep -i documentroot
    cd /srv/www/htdocs/apps/
    rails testapp

    Create a user that will be used by the cluster
    useradd -system mongrel

    Create a cluster configuration file with three processes using ports 8001, 8002, and 8003. This will create a file in the config directory called mongrel_cluster.yml
    cd /srv/www/htdocs/apps/testapp
    mongrel_rails cluster::configure -p 8001 -e production -a 127.0.0.1 -N 3

    I then modified the default configuration file to look like this:

    user: mongrel
    cwd: /srv/www/htdocs/apps/testapp
    log_file: /var/log/mongrel.log
    port: "8001"
    environment: production
    group: www
    address: 127.0.0.1
    pid_file: tmp/pids/mongrel.pid
    servers: 3

    Change permissions and owner of the Rails application. I am using the same group as used by my Apache system (see /etc/apache2/uid.conf) in my case it’s www
    chown -R mongrel:www /srv/www/htdocs/apps/testapp

    That takes care of the installation and configuration. We can test to make sure the cluster is working by starting it and connecting to it via a browser.

    Start the cluster
    cd /srv/www/htdocs/apps/testapp
    mongrel_rails cluster::start

    Check status of the cluster
    cd /srv/www/htdocs/apps/testapp
    mongrel_rails cluster::status

    Run browser and connect to mongrel ports, you should see the standard Rails startup page
    127.0.0.1:8001
    127.0.0.1:8002
    127.0.0.1:8003

    To stop the cluster
    cd /srv/www/htdocs/apps/testapp
    mongrel_rails cluster::stop

    To restart after a code change for example
    cd /srv/www/htdocs/apps/testapp
    mongrel_rails cluster::restart 

    We need to setup the cluster so that it will be restarted when the server is rebooted. 

    Create a new directory and create a new link to the mongrel cluster configuration file we created in the previous step
    mkdir /etc/mongrel_cluster
    ln -s /srv/www/htdocs/apps/testapp/config/mongrel_cluster.yml \
    /etc/mongrel_cluster/testapp.yml

    Copy the shell script supplied when the mongrel cluster gem was installed to the init.d directory. The shell script allows us to control the cluster including start, stop, status, and restart.
    cp /usr/lib/ruby/gems/1.8/gems/mongrel_cluster_1.0.5/resources/mongrel_cluster \
    /etc/init.d/
    chmod +x /etc/init.d/mongrel_cluster
    Usage: /etc/init.d/mongrel_cluster <status|start|stop|restart>

    Add the mongrel cluster as a service
    Add the service:
    chkconfig mongrel_cluster 35
    To check the service:
    chkconfig –list mongrel_cluster
    To delete the service: chkconfig –del mongrel_cluster

    There was one modification I had to make to the mongrel_cluster script to get it to work without any errors: 
    Edit: /etc/init.d/mongrel_cluster
    Change:
    chown $USER:$USER $PID_DIR
    To: chown $USER:$GROUP $PID_DIR 
     

    5. Nginx

    Now that we have a working mongrel cluster running three mongrel processes how are we going to do the load balancing for these processes? I initially looked at using Apache 2.2 + mod_proxy_balancer but as SuSe 9.3 does not have any pre-built packages for 2.2 and I did not want to face the daunting task of building Apache myself I had to look at some alternatives.

    I initially considered Pen and Pound but then came across Nginx which seems to have a lot of positive feedback from users. From what I have read it seems that it only requires a small amount memory and is as fast if not faster than mod_proxy_balancer. So I decided to give it a try and so far I’ve been very happy with it.

    Before we can install nginx we need to install some packages required during the build process.

    1. Start YAST, Software, Install and Remove Software

    2. Install the following packages if they have not already been installed

    • pcre
    • pcre-devel
    • pcre++-devel
    • openssl
    • openssl-devel

    I found that these packages where not available on my installation CD so I used packages I found using rpmfind. I connected to this site from the SuSe machine using Konqueror, which when I clicked on the required package, the package information window was displayed allowing installation via YAST.

    We can now build and install nginx, one point to note is that I’m using the latest snapshot which includes the upstream_fair changes as discussed in Ezra Zygmuntowicz article A Fair Balancer for Nginx and Mongrel 

    Download snapshot from the link below, save the file in the /tmp/build directory
    http://git.localdomain.pl/?p=nginx.git;a=shortlog;h=upstream_fair

    Extract all files 
    tar xzvf nginx-x.x.xx.tar.gz

    Change to the extracted source directory
    cd nginx-x.x.xx 

    Setup our make files
    ./configure –with-http_ssl_module –prefix=/usr

    Build nginx
    make

    Install
    sudo make install

    Check installation worked
    nginx -v
    which should return the nginx version, e.g. nginx version: nginx/0.5.32

    Remove the source files
    rm -rf  nginx-x.x.xx

    Now we need to configure nginx to work with our Rails application, to do this I used a modified version of  Ezra Zygmuntowicz configuration file as described in the Nginx, my new favorite front end for mongrel cluster article. The other change I made to the configuration file was to include the performance improvement tip described in another of Ezra’s articles New Nginx conf with optimization.

    Create the nginx.conf file in the test Rails application config directory

    I changed the ownership and rights of this file to be the same as the other config files, i.e.
    chmod 664 nginx.conf
    chown mongrel:www nginx.conf

    Test nginx config file
    cd /srv/www/htdocs/apps/testapp
    nginx -t -c config/nginx.conf 

    Start nginx using the new config file
    cd /srv/www/htdocs/apps/testapp
    nginx -c config/nginx.conf

    Stop nginx
    ps aux | grep nginx
    kill -15 <pid>
    where <pid> is the pid of the master process

    Test nginx is working correctly, start a browser and connect to nginx using the port you specified in the config file listen directive, i.e. listen 80;. You should see the standard Rails welcome page.

    We now need to setup nginx to run as a service so that it will be restarted when the server is rebooted. To do this I modified a script that I found in a slicehost article Ubuntu LTS - adding an ngnix init script, here is my modified version which I’m sure can be improved on but for now it does what I need it to do. Create this script in the /etc/init.d directory in a file called nginx.

    #! /bin/sh

    PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
    DAEMON=/usr/sbin/nginx
    NAME=nginx
    DESC=nginx
    PIDFILE=/usr/logs/$NAME.pid
    DAEMON_CONFIG=/etc/nginx/nginx.conf

    test -x $DAEMON || exit 0

    set -e

    case "$1" in
      start)
            echo -n "Starting $DESC: "       
            start-stop-daemon –start –quiet –pidfile $PIDFILE \
                    –exec $DAEMON — -c $DAEMON_CONFIG
            echo " started"
            ;;
      stop)
            echo -n "Stopping $DESC: "
     if [ -f $PIDFILE ]; then
                    kill -15 `cat $PIDFILE 2>/dev/null`
            fi
            echo " stopped"
            ;;
      restart|force-reload)
            echo -n "Restarting $DESC: "
     if [ -f $PIDFILE ]; then
                    kill -15 `cat $PIDFILE 2>/dev/null`
            fi
            sleep 1
            start-stop-daemon –start –quiet –pidfile $PIDFILE \
                    –exec $DAEMON — -c $DAEMON_CONFIG
            echo " restarted"
            ;;
      status)
            echo "Status $DESC: "
            ps aux | grep -v grep | grep -v /bin/sh | grep $NAME
            ;;
      *)
            N=/etc/init.d/$NAME
            echo "Usage: $N {start|stop|status|restart}" >&2
            exit 1
            ;;
    esac

    exit 0

    Steps to configure nginx as a service.

    Create a new directory
    mkdir /etc/nginx

    Create a link to the nginx.conf file in the test Rails application config directory
    ln -s /srv/www/htdocs/apps/testapp/config/nginx.conf \
    /etc/nginx/nginx.conf

    Make the script executable
    chmod +x /etc/init.d/nginx
    Usage: /etc/init.d/nginx <status|start|stop|restart>

    Add it as a service
    Add the service: chkconfig nginx 35
    To check the service: chkconfig –list nginx 
    To delete the service: chkconfig –del nginx

     

     


    Next Page »