Technology and Software

What’s the [5.0] in Rails 5’s ActiveRecord::Migration

The migration examples in the Rails Edge Guide are all like this

class MyMigration < ActiveRecord::Migration[5.0]
  ...
end

What’s that [5.0]? It can’t possibly be Ruby. Or is it?

With a little experimenting in rails c it’s easy to understand that [ is a class method of ActiveRecord::Migration. It’s defined in activerecord-5.0.0.beta1/lib/active_record/migration.rb and it accepts the 4.2 and 5.0 arguments. It allows us to select which version of the migrations we want to use. Production ready versions of ActiveRecord don’t have that method so it should go away as soon as Rails 5 goes out of beta.

 

Advertisements
Standard
Technology and Software

Ruby 2.3.0 InstructionSequence

Ruby 2.3 has been released on Christmas day as every Ruby version. It comes with a nice present: the RubyVM::InstructionSequence class with methods to compile scripts, save them and load them later. A quick example:

rvm install ruby-2.3.0
rvm use ruby-2.3.0
cat > test.rb
class Christmas
  def self.day
    25
  end
end
p Christmas.day

cat > compile.rb
instruction_sequence = 
  RubyVM::InstructionSequence.compile_file("test.rb")
File.open("test.iseq", "wb") do |file|
  file.write(instruction_sequence.to_binary)
end

cat > instruction_sequence = nil
File.open("test.iseq", "rb") do |file|
  instruction_sequence =
    RubyVM::InstructionSequence.load_from_binary(file.read)
end
instruction_sequence.eval

ruby compile.rb
ls
compile.rb  execute.rb  test.iseq  test.rb
ruby execute.rb
25

It works!

InstructionSequence comes with a caveat:

The goal of this project is to provide “machine dependent” binary file to achieve:

  • fast bootstrap time for big applications
  • reduce memory consumption with several techniques

“Machine dependent” means you can’t migrate compiled binaries to other machines.

Does it means that the compiled code won’t work on another machine? I generated the .iseq file on a Ubuntu 12.04 machine and uploaded it to a Ubuntu 14.04 one, both 64 bit. It keeps working and the directory structure of the two machines can be different, despite the presence in the compiled code of metadata about the source file.

I invite the readers to check the other methods of the class. They let allow for the compilation of strings of text and procs, setting compilation options, disassembling iseq code plus several instance methods that operate on an instruction sequence.

Standard
Technology and Software

Ruby performances with PostgreSQL and MySQL

(original post in the Italian Ruby Forum)

I had to convert a database seeding script from PostgreSQL 9.4 to MariaDB 10 (customer’s choice and with little enthusiasm I had to comply). This lead to a number of interesting discoveries about the pg and mysql2 Ruby drivers. Apart a few minor issues [1] [2] [3] [4] I immediately noticed that the script with MariaDB run 20 times (twenty) more slowly than the  PostgreSQL one: 21 minutes vs 1 minute and 3 seconds. Unusable and inexplicable.

Such a big difference can not be due to the database, so I started to investigate the configuration. Even the MySQL coming with Ubuntu 12.04 was too slow and I can expect that the distributors set it up reasonably well. At this point the suspect becomes the driver. I opened this issue https://github.com/brianmario/mysql2/issues/623 and they gave me two valuable tips: use a profiler ( https://github.com/ruby-prof/ruby-prof ) and the gem-import activerecord ( https://github.com/zdennis/activerecord-import ). I knew both of them but sometimes you have to be reminded about tools you don’t use often. Ops.

The profiler show that the driver uses pg prepared statements that give obvious benefits with the number of records created by my script (a little over 32,000). The version of mysql2 I had to use (0.3.x) does not have prepared statement (but the newer version does) and that seems to make the difference. I rewrote the script to use activerecord-import, which  inserts a whole array of objects at once. The script looks a bit unnatural, because I repeatedly needed the ids of the record I created to pass them along the associations, but the execution times for mysql2 dropped from 21 minutes to 1 minute and 33″. It was worth it. There are only 1,045 calls to the db and yet is always slower than 32k calls made by the original script with pg. The script with pg and activerecord-import dropped to 47 seconds.

Despite all the enhancements introduced in the import-activerecord calls my script’s calls to PostgreSQL add up to 9.4 seconds. The calls to MariaDB are  49.8 seconds. Ruby accounts for 40 seconds, regardless of the database used.

TL;DR

1) Work on PostgreSQL has performance advantages with Ruby due to drivers.

2) mysql2 0.4.0+ has prepared statements but if you’re working with Rails you must be careful. There are issues [A] [B] and it seems you need Rails 4.2.5+ to use it. I didn’t test the combination yet.

3) For details of my profiling research (tables, times, calls) read https://github.com/brianmario/mysql2/issues/623 

 

Finally, the issues I run into:

[1] https: //mariadb.com/kb/en/mariadb/installing-maria …

[2] For MariaDB install the gem mysql2 with

bundle config \
build.mysql2 --with-mysql-config=/path/to/mariadb/bin/mysql_config

Careful: this is globals so use –with-mysql-config=/usr/bin/mysql_config when you need to connect to MySQL.

[3] My script would clear the db before seeding using TRUNCATE CASCADE, but MySQL and MariaDB don’t have it. This  is the workaround

 connection = ActiveRecord :: Base.connection
 Connection.Execute ("SET foreign_key_checks = 0;")
 [all models] .each do | model |
   Connection.Execute ("TRUNCATE model.table_name # {}")
 end
 Connection.Execute ("SET foreign_key_checks = 1;")

[4] But neither ActiveRecord has TRUNCATE, so either you use some gems that add it to AR or even for PostgreSQL you need a loop like that, but you don’t need the SET foreign_ley_checks statements.

Standard
Technology and Software

Compile your own Ruby and use it with RVM

Prompted by the news about how gcc 4.9 makes Ruby 2.1 faster I decided to compile my own Ruby 2.2.0 and pit it against the one coming with rvm. I also want to keep switching between Rubies using RVM. I had to google a little to learn how to do it so I want to share.

rvm install 2.2.0
rvm use ruby-2.2.0
# find out the compilation options
ruby -r rbconfig -e 'puts RbConfig::CONFIG["configure_args"]'
 'optflags=-O2' '--enable-load-relative' '--sysconfdir=/etc'
 '--disable-install-doc' '--enable-shared'
wget http://cache.ruby-lang.org/pub/ruby/2.2/ruby-2.2.0.tar.gz
# important, always compare to the hash advertised at
# https://www.ruby-lang.org/en/downloads/
md5sum ruby-2.2.0.tar.gz
tar xzf ruby-2.2.0.tar.gz
cd ruby-2.2.0
mkdir -p /home/me/compiled-rubies/2.2.0p0
# configure with the same compilation options
# of the standard binary
CFLAGS=-O2 ./configure --enable-load-relative \
  --sysconfdir=/etc \
  --disable-install-doc --enable-shared \
  --prefix=/home/me/compiled-rubies/2.2.0p0
make
make test
make install
# make it available to rvm as ext-ruby-2.2.0-gcc4.9_O2
rvm mount /home/me/compiled-rubies/2.2.0p0 \
  -n ruby-2.2.0-gcc4.9_O2
rvm list
...
    ext-ruby-2.2.0-gcc4.9_O2 [ x86_64 ]
 => ruby-2.2.0 [ x86_64 ]
...
rvm use ext-ruby-2.2.0-gcc4.9_O2

The files in ~/.rvm/rubies/ext-ruby-2.2.0-gcc4.9_O2 will be symlinks to the ones in compiled-rubies/2.2.0p0 so don’t remove that directory.

The point of this post is already made but as a bonus here are the benchmarks of the two Rubies using Antonio Cangiano’s tests.

git clone git://github.com/acangiano/ruby-benchmark-suite.git
cd ruby-benchmark-suite
rvm use ruby-2.2.0 # for the standard one
rake
rvm use ext-ruby-2.2.0-gcc4.9_O2 # for the compiled one
rake # This might fail, see the note at the end

Here are the results: ruby-2.2.0 and ruby-2.2.0-gcc4.9_O2 (YAML), summary (CSV). TL;DR: the compiled Ruby is a little bit faster overall. It’s much faster in a few tests, a bit slower in some others. It’s a difficult choice and it probably depends on what you do.  Please notice all those tests that ended with errors (look at the YAML files). They could make a difference for the overall assessment of which version is faster but I didn’t dig into that issue yet.

In case of failure

Rake could end with a weird syntax error for the compiled Ruby. There are two possible fixes. One is to replace `which rake` with the version from the 2.2.0 binary distribution. The other is to really understand what’s going on. The key is: that rake is a bash script which execs a Ruby interpreter on itself using ruby’s -x switch which strips away the bash script at the beginning. But Ruby doesn’t seem to honour that. No time to investigate any further now…

Standard
Technology and Software

Ruby’s Influence over the Elixir Language

This pictures shows exactly what you’d expect a Ruby conference to be. Don’t you? Wait, it’s not what it looks. I can explain.

Ruby Day 2014 / Lunch Break

Ruby Day 2014 / Lunch Break

That was the lunch break and we had a wonderful sun and a wonderful lawn :-)

That was in Roncade, Italy at the premises of H-FARM.

I was there for the usual stuff: learn new things, meet people I knew and know new people. Plus, my first time at a Ruby conference, to give a talk. A talk about Ruby. No, a talk about a language designed to look like Ruby regardless of the huge differences beneath. This language is Elixir and this is my presentation on Slideshare (sorry for the fonts, some of them didn’t survive the conversion after the upload).

2014-10-23 – Update: we’ve got the video!

The original presentation files (odt, ppt, pdf) with the speaker notes and some tutorials are at http://connettiva.eu/rubyday

Visit also my GitHub repository for a demo Phoenix application (a RoR-like web framework for Elixir) at https://github.com/pmontrasio/phoenix-demo-app

Standard
Technology and Software, Tips

Upgrade a Rails 4 app to Rspec 3

I have a Rails 4 application with Rspec 2. I’m using a mix of should and expect assertions. I wanted to upgrade to Rspec 3 without changing the specs for now. I updated the Gemfile, run bundle install, rake spec and got many errors. Basically most helpers went missing (undefined methods visit, sign_in, root_path, etc., plus anything defined inside app/helpers). Googling around I found a solution for everything but the keys to restore the old behaviour are two.

1) The new rspec doesn’t include helpers based on the directory the specs are stored into. You either define the spec type with stuff like type: :feature or type: :controller or you add

config.infer_spec_type_from_file_location!

to the Rspec.config block.

2) The should syntax has been deprecated and doesn’t work any more by default. You must enable it with

config.expect_with :rspec do |c|
c.syntax = [:should, :expect]
end

Minor quirks:

  • You must remove require ‘rspec/autorun’
  • example doesn’t exist anymore. It has been replaced by RSpec.current_example
Standard
Technology and Software

FactoryGirl and Paperclip: testing content types

I used Paparclip to add a picture to a model, something I did for years. This time I also added a validation for content types, and this might be a first time for me (I don’t want to grep all the models of all the past projects). The validation is

validates_attachment :picture,
content_type: { content_type: ["image/jpg","image/png"] }

Now I want to test it. I was loading real image files in the objects created with FactoryGirl. This is the code

picture  { File.open("#{Rails.root}/#{%x[ls test-images/*jpg].split("\n").sample}") }

Note that I’m using %x[].sample to randomly pick an image from a directory, but that’s not important.

The code above doesn’t set a mime type and the validation fails. I had to google quite a lot to find the right hints (some solutions have been obsoleted by newer versions of Paperclip and maybe other parts of the toolchain). The solution is

Rack::Test::UploadedFile.new("#{Rails.root}/#{%x[ls test-images/*jpg].split("\n").sample}"), "image/jpg")

which loads the image and sets it’s content type.

Standard