IT.Signals

Last year I decided to build a new development server running a number of Virtual Machines. After spending two days hitting myself over the head with OpenSuse + Xen and coming away bloodied and bruised, I decided to give Ubuntu + KVM a go. So far I have been very impressed, everything worked as advertised and I was able to get up and running in no time at all, I think I am an Ubuntu convert…

I originally created my VM with 20GB of disk space under the false impression that my image file would grow automatically as more space was required. After using the VM for sometime I realised that 20GB was not going to be enough and that I was going to need more space. The question was how do I increase the size without having to create the VM from scratch, not a task I relished as I’d spent many hours using the VM and had installed a lot of software and really did not want to repeat the process.

I used Google to search for how I could achieve this thinking that a lot of people would strike this problem and that a solution would be but a few clicks away…wrong! After a lot of searching I found a few articles on how this could be achieved, a lot of them where complicated including having to hack the MBR. After making a backup of my image file, I tried a couple of solutions but they ended up damaging my image file to the point where I could no longer boot the VM.

It’s at this point I started to sweat, I had spent hours installing and setting up software within my VM, I really did not want to have to do it all over again! After a lot of experimentation I found something that works reliably for me, I’m sure there are better ways of doing this (is’nt there always!) but this works for me every time.

DISCLAIMER: Take care in what you do, make sure you have a reliable backup of the VM image before you start. You do this at your own risk, I accept no responsibility if it all goes wrong.

So let’s dive in…

Step 1: Stop the VM

Step 2: Change to the directory where your VM image files are stored, mine is in /kvm/ubuntu-dev/

cd /kvm/ubuntu-dev

Here is what my 20GB file looks like.

Step 3: Backup the image file, my image file is called ubuntu-dev.img. I created a sub-directory called bak where I store my image backup file

sudo cp ubuntu-dev.img bak/ubuntu-dev.img.bak

Step 4: Create a blank qemu file, this is the file we will be adding to our VM image. Create it with a size equal to the amount of extra disk space you want added to your VM. In my case I want to add an additional 30GB of space.

sudo qemu-img create -f raw addon.raw 30G

This is what the addon file looks like on my system after it is created.

 

Step 5: Rename the original image file to .save as is shown in the image above. We never actually modify the original image file so we end up with two backup files, the .save as well as the backup file we created in Step 3, just in case something goes wrong. When you are sure everything has worked you can always delete these files.

Step 6: We are now ready to combine the image file with the new blank qemu file we created earlier. You need to be logged in as the super user to perform this step.

su

cat ubuntu-dev.img.save addon.raw >> ubuntu-dev.img

Step 7: Once this completes you can check the new file size, my new file size is now 50GB which is my original 20GB + 30GB of extra space.

Step 8: We have successfully increased the size of our image file by 30GB to 50GB, now we need to fix our partitions. To do this you need to downloaded GParted and create a live CD. Once you have created the CD and loaded into your machine run the following command to start GParted using the new image file.

sudo qemu -hda ubuntu-dev.img -cdrom /dev/cdrom -boot d

The following window should be displayed.

 

Select the "GParted Live (Default settings)" option or the one appropriate for you. You will then be prompted with a number of setup options, I normally accept the defaults. Once GParted has been loaded you should see your current VM image partitions, here is mine:

The first thing we need to do is delete the swap partition so that the primary ext3 partition is next to the new unallocated space we added, this will allow us to expand the primary partition. Note that the correct device is selected in the top right hand corner, it shows our newly expanded 50GB device.

So select the swap partition and select "Delete", the command will be added to the pending operations list shown in the bottom panel. We can undo all operations until we select "Apply".

Now we can resize the primary partition, select primary partition and select "Rezise/Move" and resize the partition as required. I’m reserving 2GB for the swap partition and allocate the rest to the primary partition.

The last step is to create a new swap partition.

We have now completed all the steps required to re-partition our image. We deleted the old swap partition, resized the primary partition and created a new swap partition. All these steps are listed as pending actions, we can still undo anything at this stage but I’m happy with what I’ve done and select "Apply" to get GParted to make the changes.

 GParted will then start making the changes.

Once completed we can see our new partitions, the primary partition is now 48GB in size and we have a 2GB swap partition.

Step 9: Exit GParted and restart your system, my VM automatically starts when the server is restarted. Login to the VM (hopefully everything starts ok). We need to activate the swap file in the new swap partition we created, so once you have logged in start GParted, your new partitions should be displayed.

 

 To activate the swap file, select the swap partition and press the right mouse button. The following popup menu should be displayed, select the "Swapon" option to turn the swap file on.

 

Conclusion

That’s it we have successfully added and extra 30GB of space to our VM image file. Once you are happy that everything is working you can remove your backup files. I normally keep mine for a little while until I’m 100% sure everything is working as it should.

I’ve been using rspec for my testing for some now and have played around the edges with rspec user stories. When I started working on a new application a month or two ago I thought it would be a great opportunity to revisit my testing approach and my testing toolkit. After reading and researching the current trends I have settled on the following:

• machinist - After trying many fixture replacement plugins I’m happy with machinist, so far it does everything I need.
• forgery - Machinist includes Sham which allows me to create dummy data for my fixtures, I use forgery which is a very nice fake data generator to supply the data to Sham,
• cucumber - Is the replacement for rspec user stories, it allows you to write plain text stories (features) and execute them as functional tests.
• webrat - It lets you write expressive and robust acceptance tests for a Ruby web application.
• rspec - Is a Behaviour Driven Development (BDD) framework for writing executable code examples.

After reading lot’s of articles and looking at lots of example code I was finding it all very daunting, I found myself suffering from a severe case of coders block. I really did not know where to start, how to structure my code, how to write the features and so on. I was hoping that the Rspec book might have been released before I had to start development to give me some direction but the beta release has been delayed and I had to face my fears and dive in.

Once I started, I was pleasantly surprised how natural the whole process felt, after writing my first half dozen features or so I was really starting to enjoy it! In this article I want to give a brief overview of what I did to get started, show some example features and steps, as well as demonstrating how all the tools in my toolkit mesh together to form a very nice testing framework. I’m still very new to the process of writing features and steps and am still becoming familiar with all the tools, so I’m sure that I will make adjustments as I become more experienced in using the framework.

I will not go into how to install the plugins as this is well documented for each plugin.

Creating my fixtures

I create my fixtures using machinist. To use machinist you use a class method called blueprint which is an extension of ActiveRecord::Base which means you can use it on any of your ActiveRecord models. In my application I have an "Account" link that allows the user to modify their account details including the password and email address which is the first feature I wanted to write. So the first thing I did was to create some blueprints for my user models in the blueprint.rb file which resides in the spec directory. The first thing I do is define some standard sham’s that I will use in my blueprints, I use forgery to populate these shams with some good dummy data.

require ‘forgery’

# Shams
# We use forgery to make up some test data

Sham.name  { NameForgery.full_name }
Sham.login  { InternetForgery.user_name }
Sham.email  { InternetForgery.email_address }
Sham.password  { BasicForgery.password }
Sham.string { BasicForgery.text }
Sham.text { LoremIpsumForgery.text }

# Blueprints

Role.blueprint do
  name { ‘guest’ }
end

SiteUser.blueprint do
  user_type { ‘SiteUser’ }
  login { Sham.login }
  name { Sham.name }
  email = Sham.email
  email { email }
  email_confirmation { email }
  pwd = Sham.password
  password { pwd }
  password_confirmation { pwd }
  accept_terms { ‘true’ }
  time_zone { ‘Melbourne’ }
end

OpenidUser.blueprint do
  user_type { ‘OpenidUser’ }
  time_zone { ‘Melbourne’ }
  email { Sham.email }
end

These blueprints allow me to easily create some objects using make, i.e.

user = SiteUser.make

Where to put my test files

There are any number of ways to structure your files and directories, after reading this article I decided to create a new sub-directory for each model that I wanted to test, so my directory structure looks something like this:

 

So for model User I created a users directory which contains a user.feature file and a steps_definitions sub-directory where the step files will be located, in this case we have user-steps.rb.

Setting up my env.rb

Before we start writing features we need to tweak the cucumber env.rb file. I’ve added code to load webrat and machinist, my file looks like this:

# Sets up the Rails environment for Cucumber
ENV["RAILS_ENV"] = "test"
require File.expand_path(File.dirname(__FILE__) + ‘/../../config/environment’)
require ‘cucumber/rails/world’
Cucumber::Rails.use_transactional_fixtures

# Add webrat
require "webrat"
Webrat.configure do |config|
  config.mode = :rails
end

# Comment out the next line if you’re not using RSpec’s matchers (should / should_not) in your steps.
require ‘cucumber/rails/rspec’

# Add machinist
require File.join(RAILS_ROOT, ’spec’, ‘blueprints’)

Writing my first feature

The first feature I wanted to write was the ability for the user to display their account details once they had logged in. I initially started writing the feature using my own words to describe it, for example:

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I go to the "Account" page
  Then I should see my account details
    And I should see a "Change Password" link
    And I should see "Change Email" link

I then implemented each of these in my steps file using webrat. It’s only after some time that I realized that cucumber comes with a webrat steps definition file, i.e. webrat_steps.rb. Which contains a lot of standard webrat steps which I could directly call from my feature definition. So I could rewrite my scenario using the webrat steps:

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

The steps in bold are all webrat steps that I do not have to implement in my own steps file as they are implemented in the webrat steps file! There are only two steps that I have to define myself.

Step 1: Given I am logged in as a SiteUser

This step is quite involved as I have to figure out how to login a user using a user name/password, I also need to cater for those users that use an OpenID. I decided to create a step_helper.rb file which is located in the /features/step-definitions sub-directory and will be automatically loaded by cucumber, which will contain any shared steps. It currently contains only one step that handles the login process for a user and looks like this:

# Login
# e.g.
# Given I login as a SiteUser in the guest role
# Given I login as a OpenidUser in the admin role
#
Given /I login as a (.*) in the (.*) role/i do |user_type, role|
  role = Role.make
  @user = Object::const_get(user_type).make_unsaved(:roles => role) 
  if (!@user.openid)
    @user.activate!
    visit login_path
    fill_in("login", :with => @user.login)
    fill_in("password", :with => @user.password)
  else
    # identity_url not allowed to be set via mass assignment in blueprint
    @user.identity_url = ‘https://good.openid.url/’
    @user.activate!
    visit login_with_openid_path
    fill_in("openid_identifier", :with => @user.identity_url)   
  end 
  click_button("Log in")
  Then ‘I should see "Dashboard"’
end

 To use it I create a separate step within my steps file. One important point to note is that you can reuse steps by calling them directly from within other steps. In the example below we are calling the shared login step from within the step by calling Given "I login as a #{user_type} in the guest role".

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

This step can then be used in my feature like this:

Given I am logged in as a SiteUser

Step 2: And I should see my account details

The last step we need to implement is the one that checks to make sure that the users account details are actually being displayed. The step do this looks like this:

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

Running my features

Before we run the feature here is the content of our feature and step file so far.

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

And my user-steps.rb looks like this:

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

When I run the the feature assuming I’ve implemented the code I get the following output:

 

More example features

Here are a few more example features and the steps.

Feature: User functions

In order to maintain the correct account information
As a logged in user
I want to maintain my account information

Scenario: Display my account information if I am an SiteUser
  Given I am logged in as a SiteUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should see "Change Password"
    And I should see "Change Email"

Scenario: Display my account information if I am an OpenidUser
  Given I am logged in as a OpenidUser
  When I follow "Account"
  Then I should see "Account"
    And I should see my account details
    And I should not see "Change Password"
    And I should not see "Change Email"

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in my new password details
    And I press "Change your password"
  Then my password should be changed
    And I should see "Account"
    And I should see "Password successfully updated"

Scenario: Not allow me to change my password if the old password incorrect
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in an incorrect old password
    And I press "Change your password"
  Then my password should not be changed
    And I should see "You password was not changed, your old password is incorrect."

Scenario: Not allow me to change my password when the password and confirmation is not the same
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in an incorrect password confirmation
    And I press "Change your password"
  Then my password should not be changed
    And I should see "New password does not match the password confirmation."

Scenario: Allow me to change my account details
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I fill in my new account details
    And I press "Save"
  Then my account details should be changed
    And I should see "Account details updated."

Scenario: Allow me to change my email
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Email"
    And I fill in my new email address
    And I press "Change your email"
  Then my email should be changed
    And I should see "Account"
    And I should see "Email successfully updated"

Scenario: Not allow me to change my email when the email and confirmation is not the same
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Email"
    And I fill in an incorrect email confirmation
    And I press "Change your email"
  Then my email should not be changed
    And I should see "New email does not match the email confirmation."

Steps in user-steps.rb

Given /I am logged in as a (.*)/i do |user_type|
   Given "I login as a #{user_type} in the guest role"
end

Then /I should see my account details/ do
  Then "I should see \"#{@user.name}\""
  Then "I should see \"#{@user.company}\""
end

When /I fill in my new password details/ do
  When "I fill in \"old_password\" with \"#{@user.password}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_123"’
end

When /I fill in an incorrect old password/ do
  When "I fill in \"old_password\" with \"#{@user.password + ‘xyz’}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_123"’
end

When /I fill in an incorrect password confirmation/ do
  When "I fill in \"old_password\" with \"#{@user.password}\""
  When ‘I fill in "password" with "test_123"’
  When ‘I fill in "password_confirmation" with "test_1234"’
end

When /I fill in my new account details/ do
  When ‘I fill in "user_name" with "Fred Flintstone"’
  When ‘I fill in "user_company" with "Acme Rocks"’
  When ‘I select "(GMT+10:00) Canberra" from "user_time_zone"’
end

Then /my password should be changed/ do
  SiteUser.find_by_id(@user.id).authenticated?(’test_123′).should == true
end

Then /my password should not be changed/ do
  SiteUser.find_by_id(@user.id).authenticated?(’test_123′).should == false
end

Then /my account details should be changed/ do
  u = SiteUser.find_by_id(@user.id)
  u.name.should == ‘Fred Flintstone’
  u.company.should == ‘Acme Rocks’
  u.time_zone.should == ‘Canberra’
end

When /I fill in my new email address/ do
  When ‘I fill in "email" with "test@isp.com"’
  When ‘I fill in "email_confirmation" with "test@isp.com"’
end

When /I fill in an incorrect email confirmation/ do
  When ‘I fill in "email" with "test@isp.com"’
  When ‘I fill in "email_confirmation" with "test123@isp.com"’
end

Then /my email should be changed/ do
  SiteUser.find_by_id(@user.id).email.should == ‘test@isp.com’
end

Then /my email should not be changed/ do
  SiteUser.find_by_id(@user.id).email.should_not == ‘test@isp.com’
end 

Feature Writing Style

There seems to be two styles that you can use when writing your features, "Imperative" and "Declarative" which is described in detail in the article Imperative vs Declarative Scenarios in User Stories written by Ben Mabey. I understand the extreme cases as demonstrated by the examples given by Ben, but what about a feature such as this one, can this be classed as being written using the Imperative style?

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I follow "Account"
    And I follow "Change Password"
    And I fill in my new password details
    And I press "Change your password"
  Then my password should be changed
    And I should see "Account"
    And I should see "Password successfully updated"

Does the Imperative style dictate that all steps appear in the feature, the example above do have some steps that are defined in the steps file, but there are a few other details that could be moved to the steps file as well. So if the one above is Imperative does this modified version make it declarative?

Scenario: Allow me to change my password
  Given I am logged in as a SiteUser
  When I go to the "Change Password" page
    And I fill in my new password details
  Then my password details should be changed

At this stage I’m not sure, for now I will stay with the first style as it provides details that may be important to someone reading the feature but hides the non-essential detail in the steps file.

For anyone struggling to get SQL Anywhere working with Rails there is some good news, I’ve received and email from Eric Farrar from Sybase to let me know that he has been doing a lot of work on developing a new adaptor for SQL Anywhere. I’ve included the content of his email below for anyone interested.

There is now a ActiveRecord adapter specifically for SQL Anywhere. It is available on RubyForge at sqlanywhere.rubyforge.org/. The source is hosted on GitHub and is available at github.com/sqlanywhere. This driver does not go through ODBC, but instead uses the new C API library to directly interface with SQL Anywhere databases. Please note that this driver can only be used with SQL Anywhere 10 and greater.