What’s the [5.0] in Rails 5’s ActiveRecord::Migration

The migration examples in the Rails Edge Guide are all like this

class MyMigration < ActiveRecord::Migration[5.0]
  ...
end

What’s that [5.0]? It can’t possibly be Ruby. Or is it?

With a little experimenting in rails c it’s easy to understand that [ is a class method of ActiveRecord::Migration. It’s defined in activerecord-5.0.0.beta1/lib/active_record/migration.rb and it accepts the 4.2 and 5.0 arguments. It allows us to select which version of the migrations we want to use. Production ready versions of ActiveRecord don’t have that method so it should go away as soon as Rails 5 goes out of beta.

 

Advertisement

Running Ruby on Rails tests with a ramdisk backed PostgreSQL

I’m not a fan of mocking objects in my Ruby On Rails tests so my tests always hit the database, which is PostgreSQL anytime I can make the choice and a customer doesn’t dictate MySQL.

Hitting the DB means that the test suite eventually slows down as tests pile up as the application grows.
It’s been a long time since I wanted to check what happens if I run my tests on a database backed on RAM instead of by a spinning disk. Are they going to be much faster?

Tl;dr No, they run at the same speed.

Creating the DB on the ramdisk

The setup is based on information provided at http://stackoverflow.com/questions/11764487/unit-testing-with-in-memory-database and http://jayant7k.blogspot.com.au/2006/08/ram-disk-in-linux.html
Thanks!

The test system is my laptop, an Ubuntu 12.04, i7-4700MQ CPU @ 2.40GHz, 16 GB RAM, SDD for the OS, HDD for my home and databases. The DB is PostgreSQL 9.3.

Linux has 16 ramdisks already created as /dev/ram* at boot time. Let’s take one and mount it.

mkdir ~/tmp/ram
sudo mkfs.ext4 -m 0 /dev/ram0
sudo mount /dev/ram0 ~/tmp/ram/
df -h ~/tmp/ram/
Filesystem Size Used Avail Use% Mounted on
/dev/ram0 58M 1.3M 56M 3% /home/me/tmp/ram

It’s a tiny disk and it turned out to be barely enough to accommodate my tests but it’s OK for experimenting.
You can make it larger if you need to. http://jayant7k.blogspot.com.au/2006/08/ram-disk-in-linux.html explains how.

We create a DB there now.

cd ~/tmp/ram
sudo bash
mkdir postgresql
chown postgres.postgres postgresql/
su - postgres
/usr/lib/postgresql/9.3/bin/initdb --locale=en_US.UTF-8 -D ~/tmp/ram/postgresql/
exit
mkdir postgresql/log
chown postgres.postgres postgresql/log/

We make it use a different port from the one used by the default PostgreSQL DB on the laptop.

vi postgresql/postgresql.conf

port = 5433

We start the DB and connect to it

sudo -u postgres /usr/lib/postgresql/9.3/bin/pg_ctl \
  -D ~/tmp/ram/postgresql/
  -l ~/tmp/ram/postgresql/log/postgresql-9.3-main.log start

psql -p 5433 -U postgres
 \l

Great!

Running tests

We edit config/database.yml to use the ramdisk DB

port: 5433

We create the test user and the test db

psql -p 5433 -U postgres
create role testuser login password 'password';
alter user testuser with createdb;
create database myapp_test owner testuser encoding='UTF8' lc_collate='en_US.UTF-8' lc_ctype='en_US.UTF-8';
\q
exit

We create the DB schema

cd the/rails/directory
RAILS_ENV=test rake db:migrate

And finally we benchmark the tests over the two databases.

rake spec:controllers

RAMDISK

Finished in 1 minute 5.34 seconds (files took 1.76 seconds to load)
Finished in 1 minute 4.26 seconds (files took 1.75 seconds to load)
Finished in 1 minute 2.07 seconds (files took 1.75 seconds to load)

HDD

Finished in 1 minute 7.09 seconds (files took 1.76 seconds to load)
Finished in 1 minute 6.01 seconds (files took 1.72 seconds to load)
Finished in 1 minute 4.68 seconds (files took 1.74 seconds to load)

2 seconds are not worth the trouble. Let’s benchmark the models.

rake spec:models

RAMDISK

Finished in 1 minute 36.8 seconds (files took 1.69 seconds to load)
Finished in 1 minute 38.08 seconds (files took 1.72 seconds to load)
Finished in 1 minute 37.9 seconds (files took 1.73 seconds to load)

HDD

Finished in 1 minute 38.64 seconds (files took 1.79 seconds to load)
Finished in 1 minute 32.73 seconds (files took 1.69 seconds to load)
Finished in 1 minute 41.89 seconds (files took 1.71 seconds to load)

No difference at all, only a bit more variance in the durations of the HDD tests.

This is my conjecture. The data go first into the OS file buffer, then are synced to the disks. Syncing to ramdisk is faster but if there is enough RAM data is staying in RAM anyway and it doesn’t matter if we’re using a ramdisk or a HDD. Remember: this is a test DB with a handful of data, not a large production DB with high I/O loads.

Let’s stop the DB and change the configuration to do without syncing. If there is no speedup my conjecture should be confirmed.

sudo -u postgres /usr/lib/postgresql/9.3/bin/pg_ctl \
  -D ~/tmp/ram/postgresql/ \
  -l ~/tmp/ram/postgresql/log/postgresql-9.3-main.log stop

sudo vi ~/tmp/ram/postgresql/postgresql.conf

fsync=off

sudo -u postgres /usr/lib/postgresql/9.3/bin/pg_ctl \
  -D ~/tmp/ram/postgresql/ \
  -l ~/tmp/ram/postgresql/log/postgresql-9.3-main.log start

Run the test on the ramdisk again.

rake spec:models

RAMDISK

Finished in 1 minute 36.52 seconds (files took 1.71 seconds to load)
Finished in 1 minute 35.45 seconds (files took 1.7 seconds to load)
Finished in 1 minute 36.59 seconds (files took 1.68 seconds to load)

No difference, so my tests didn’t move enough data to make the syncing operations relevant.

Conclusions

You can keep your test DB on a spinning disk and the OS buffering will make it fast.
If you want quick tests you probably have to mock everything and do without the DB.

Alternative: run tests in parallel with the parallel_tests gem.

Questions

Could I have done something better to make the ramdisk based DB run faster?

Upgrade a Rails 4 app to Rspec 3

I have a Rails 4 application with Rspec 2. I’m using a mix of should and expect assertions. I wanted to upgrade to Rspec 3 without changing the specs for now. I updated the Gemfile, run bundle install, rake spec and got many errors. Basically most helpers went missing (undefined methods visit, sign_in, root_path, etc., plus anything defined inside app/helpers). Googling around I found a solution for everything but the keys to restore the old behaviour are two.

1) The new rspec doesn’t include helpers based on the directory the specs are stored into. You either define the spec type with stuff like type: :feature or type: :controller or you add

config.infer_spec_type_from_file_location!


to the Rspec.config block.

2) The should syntax has been deprecated and doesn’t work any more by default. You must enable it with

config.expect_with :rspec do |c|
c.syntax = [:should, :expect]
end


Minor quirks:

• You must remove require ‘rspec/autorun’
• example doesn’t exist anymore. It has been replaced by RSpec.current_example

FactoryGirl and Paperclip: testing content types

I used Paparclip to add a picture to a model, something I did for years. This time I also added a validation for content types, and this might be a first time for me (I don’t want to grep all the models of all the past projects). The validation is

validates_attachment :picture,
  content_type: { content_type: ["image/jpg","image/png"] }

Now I want to test it. I was loading real image files in the objects created with FactoryGirl. This is the code

picture  { File.open("#{Rails.root}/#{%x[ls test-images/*jpg].split("\n").sample}") }

Note that I’m using %x[].sample to randomly pick an image from a directory, but that’s not important.

The code above doesn’t set a mime type and the validation fails. I had to google quite a lot to find the right hints (some solutions have been obsoleted by newer versions of Paperclip and maybe other parts of the toolchain). The solution is

Rack::Test::UploadedFile.new("#{Rails.root}/#{%x[ls test-images/*jpg].split("\n").sample}"), "image/jpg")

which loads the image and sets it’s content type.