I was using the geokit gem so I got the undefined method merge_conditions error as soon as I used a geokit extension into a ActiveRecord method. That’s true, merge_conditions has been removed from ActiveRecord::Base version 2.3.9 and geokit hasn’t been updated to work around that. Somebody monkey patched merge_conditions back into it (see this post on stackoverflow) but that’s not the right way to go with Rails 3.

Luckily there is a new version of the gem for Rails 3, a fork from the original gem. I tried to use it in the way I would have used the old one but I had ArgumentError instead of undefined method merge_conditions. Not a big improvement but after googling around and some trial and error I found a solution. This is the recipe:

Add gem “geokit-rails3″ into the Gemfile. Do a bundle install.

Use geokit like this

def index
  bounds = Geokit::Bounds.new(
             Geokit::LatLng.new(params[:swlat],
                                params[:swlng]),
             Geokit::LatLng.new(params[:nelat],
                                params[:nelng]))
  @points = MyModel.geo_scope(:bounds => bounds)
end

The combination of Geokit::Bounds.new, Geokit::LatLng.new and MyModel.geo_scope was the solution. Passing the params directly to geo_scope didn’t work.

It’s a project we inherited from another development team. We had two models in a N:N relationship (has_and_belongs_to_many) and a join table. We’re still learning what the code does and how it is organized so we didn’t realized that the join table has an ActiveRecord model too. Actually that’s unusual but it is also the way to go to have attributes on the relationship (Rails calls it a join model). Let’s say those models are Post, Author and the join table is authors_posts.

So, without paying attention to the real meaning of that table we grepped _id in the migration files and semi-automatically created a migration that added the foreign keys to all the tables. We had to inspect some tables to be sure of what we we’re doing but we didn’t pay attention to authors_posts. By the way, did I mention we are under time pressure and the previous developers didn’t properly cover the code with tests? Ok, you get the picture.

So, we’ve got our foreign keys. The few tests we have pass and the application keeps running. Good. That was the morning. A developer deleted a Post object  in the afternoon and got an error: the database complained that deleting a record from posts would break a foreign key in authors_posts. Strange… We started to look carefully at the code and realized that they used the authors_posts table (actually, it was named in the wrong order, posts_authors, that should have told us something was wrong) to manually store the association without letting ActiveRecord do it. They had has_many :authors_posts in the two models.

We spent time to fix that through all the code and the database (attribute names…). If it were not for the foreign keys:

1. the database would be soon full of author_posts records about deleted authors and posts (remember those are not the two models we’re using)
2. we would eventually find the bug in some more painful way, with more pressure to fix it, more code to fix, more data to change

Summing up, foreign keys are a kind of firewall in front of the data: you don’t run your network without a firewall, you don’t run your database without foreign keys. They catch all the bugs that escape your tests and the flaws in your design.

PS: to delete a object in that kind of association follow the instructions in this post. In short: post.authors.clear (which deletes the association) and post.delete

After a lot of googling and after learning a lot of things that nevertheless didn’t solve the problem I finally found this thread with the solution: add gem “rspec-rails” to the development group in the Gemfile.

I don’t why I should need the rspec gem in the development environment when rake spec runs in the test one (add “puts Rails.env” in one of your spec files and see) but that’s it. I tried to outsmart the framework and lost half an hour. I hope this will help somebody to google the solution faster!

We are developing an iPhone app to access a web service from mobile (can’t link it yet, sorry) and we run into a problem: we always got a “Data not found” error from the Engage server no matter what we did. We checked the code of the app and of the server again and again, read the documentation, looked into the forums and got stuck over there for way too long.

Finally we run into this:

My problem is resolved just by changing
One-time use auth_info tokens to Disabled
in Setting page in the dashboard.
I’m not sure what it’s for, but at least the problem goes away.

That was illuminating. We changed “One-time use auth_info tokens” to “Disabled” in the Engage dashboard and the problem was solved for us too.

We contacted Engage and they acknowledged that our new setup is correct and told us that

Tokens that are not “one-time use” are still short lived (10 mins)

That means that there should not be big security problems. Engage said they “are in the process of updating the documentation”, so hopefully other developers will be spared from all these troubles soon.

My setup is:

• Host system: Ubuntu 10.10 with VirtualBox 4.0.4.
• Guest systems: two headless Debian 4 and Debian 6 servers, three Windows XP clients to run tests with IE6, IE7 and IE8 (all the other browsers run directly on Ubuntu with the exception of Safari.)

The desired network configuration is:

• The host eth0 interface must be the only access to the Internet
• A virtual interface inside the host system should NAT the guest systems
• The guest systems should be able to access the Internet using the internal NAT
• The guest systems should be able to communicate with the host system using the virtual interface
• The guest systems should be able to communicate with each other using the virtual interface

Internet <--- eth0 ---> HOST <--- virt/if (NAT) ---+---> guest 1 (server)
                   (server & client)               |
                                                   +---> guest 2 (server)
                                                   |
                                                   +---> guest 3 (client)
                                                   |
                                                   +---> ...

With this configuration I can attach application servers to the address of the virtual interface and make them available to all the guest systems for testing, as well as to any application I run on the host system. I also want to access application servers on the guests so I’m going to use static addresses because I don’t want to setup a dynamic DNS for the internal network.

VirtualBox gives us a virtual interface called vboxnet0 which starts with the 198.162.56.1 IP address. I take advantage of that.

The problems to solve are

• Start vboxnet0 at boot time, because that address must be always available even if no guest OS is running.
• Setup the NAT.
• Assign addresses to the guest systems.

The procedure is based on https://help.ubuntu.com/community/Internet/ConnectionSharing but there are some differences.
The most important one is that connection sharing must not be enabled with the procedure shown there because it will change the address of your eth0 after a reboot and you won’t be able to access the Internet.

Configure the host

First, we bring up the virtual interface

sudo ifconfig vboxnet0 192.168.56.1

Then we configure NAT.

sudo iptables -A FORWARD -o eth0 -i vboxnet0 -s 192.168.56.0/24 -m conntrack --ctstate NEW -j ACCEPT
sudo iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -A POSTROUTING -t nat -j MASQUERADE

Update: You lose the configurations on the vboxnet0 device if to update VirtualBox to a newer release you need to remove the old one . You have two options: 1) reboot and get them back from the configuration file I’ll make you generate in the next steps of the procedure or 2) just repeat the two steps above. I tested option 2 with the upgrade from 4.0 to 4.1.

We want to make those changes permanent.

sudo iptables-save | sudo tee /etc/iptables.sav
sudo vi /etc/rc.local

We add these lines

iptables-restore < /etc/iptables.sav
ifconfig vboxnet0 192.168.56.1

and we enable IP forwarding

sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward"
sudo vi /etc/sysctl.conf

with these lines

# Internet connection sharing for VirtualBox VMs
# https://help.ubuntu.com/community/Internet/ConnectionSharing
net.ipv4.conf.default.forwarding=1
net.ipv4.conf.all.forwarding=1

Update: VirtualBox still works with that configuration after updating to Ubuntu 11.04 so you can safely chose to keep that file when the update process asks you whether to keep it or overwrite it with the standard one. By the way, it seems that IP forwarding is done with net.ipv4.ip_forward=1 in 11.04 but I didn’t tried it out.

We can restart the networking or reboot (it’s fast).

netstat -r should give something like this now:

Kernel IP routing table
Destination     Gateway              Genmask         Flags   MSS Window  irtt Iface
your.host.ip.addr *                  255.255.255.192 U         0 0          0 eth0
192.168.56.0      *                  255.255.255.0   U         0 0          0 vboxnet0
link-local        *                  255.255.0.0     U         0 0          0 eth0
default           your.gw.ip.addr    0.0.0.0         UG        0 0          0 eth0

Configure the guests

Start VirtualBox. Change the network of every guest OS to be Host Only Adapter, vboxnet0

Startup every VM in turn and change it’s network configuration.

DEBIAN

The configuration on the two Debian servers is a little complicated because they’re headless and everything must be done by command line.

sudo vi /etc/dhcp/dhclient.conf

Uncomment

prepend domain-name-servers comma_separated_list_of_DNS_ip_addresses;

Restart networking. That will update /etc/resolv.conf

sudo vi /etc/network/interfaces

Add a static address for eth0.

allow-hotplug eth0
iface eth0 inet static
address 192.168.56.x # x is any free address in the 192.161.56 network
netmask 255.255.255.0
gateway 192.168.56.1
up route add -net 192.168.56.0 netmask 255.255.255.0 gw 192.168.56.1
down route del -net 192.168.56.0 netmask 255.255.255.0 gw 192.168.56.1

Restart the networking or reboot.
You should be able to ping 192.168.56.1 and access the Internet. Check it with
telnet www.google.com 80
From the host system you should be able to ping and ssh this server.

WINDOWS XP

The configuration on the three Windows clients is easier, about as easy as on Ubuntu

Click Start, Control Panel, Network Connections.
Right click on the connection, Properties.
Select Internet Protocol (TCP/IP), click Properties.
Select Use the following IP address and enter
IP address: 192.168.56.x (x = an address you didn’t use yet)
Subnet mask: 255.255.255.0
Default gateway: 192.168.56.1
Preferred DNS server and Alternate DNS server: the addresses of the DNS of your provider.
Click OK and again OK.

You should be able to ping 192.168.56.1 and access the Internet. Start up a browser and check it.
From the host system you should be able to ping this machine.

Epilog

I can start my application servers on the host system and bind them to 192.168.56.1 even if VirtualBox is not started (remember to update the addresses in their configurations, probably from 127.0.0.1 to 192.168.56.1).
I can access the Internet from my guest systems, which is handy for downloading updates, but the Internet cannot access them thank to the internal NAT. Too bad VirtualBox doesn’t give us this setup out of the box. The closest option is Bridged Network but the guest systems are exposed on the Internet.

I’m familiar with C, shell, Perl, Java, JavaScript and Ruby. I’m also a strong believer that languages are made for people and not for computers. They must be easy to read and to write. They must not be cluttered with signs: the noise to signal ratio must be low. Luckily this is where modern languages are heading to as they’re getting rid of braces and compilers and interpreters are using the available CPU power to understand where statement end without forcing us to tell it them with ; characters.

This is a “first impact” report, so I’m sure that there are lots of things I didn’t understand and I might be very wrong on some points. Nevertheless, there is only one chance of making a good first impression and I think that languages should strive for making a good one by adopting the kind of syntax that developers are demonstrating to like.

Here we go!

• All variable names in Erlang must start with an uppercase letter. It looks very old style. It tastes of BASIC interpreters of the ’80s. I understand that variables are actually constants (they are “single assignment variables”) and atoms (much like Ruby’s symbols) start with a lowercase but it’s unsatisfactory.

• The _ anonymous variable is good. It looks like a hack but writing this code in Perl is a waste of time

($skip1, $var, $skip2) = split(/ /)

and Ruby’s

var = str.split(/ /)[1]

is less descriptive than Erlang’s

(_, Var, _) = split

• Ending each statement with . looks redundant. Why can’t one design a parser that understands when a statement ends? Ruby demonstrates that’s possible. Furthermore using a . is a bad choice because expressions like R = 5. (an integer) and R = 5.0. (a float) look very strange with the same character doing two different things (ok, Ruby suffers from the same problem with 5.0.to_i). It could be habit but ; is a more sensible choice for an expression terminator.

• Atoms are good for the very same reason Ruby’s symbols are good. No more having to assign values to the constants which we only care about for their name. That’s the job of the compiler.

• Nested tuples look like JSON, which is good. The example

1> F = {firstName, joe}.
{firstName,joe}
2> L = {lastName, armstrong}.
{lastName,armstrong}
3> P = {person, F, L}.

yields

{person,{firstName,joe},{lastName,armstrong}}

which is strikingly similar to

{person: {firstName: joe , lastName: armstrong }}

By the way

{_,{_,Name},_} = {person,{firstName,joe},{lastName,armstrong}}

extracts joe from the tuple.

• Strings as array of integers are OK as this is what they really are but the shorthand $a for the integer which represents the character a looks like a hack. How do you apply that to arbitrary characters regardless of their encoding?

• The – character as in -module or -export is ridiculous. I understand the reason # for C’s #define and #include (the preprocessor) but no modern language should do that.

• Having to declare the number of arguments of a function as in -export([factorial/1)] is also ridiculous. OK, there might be other functions with the same name and different number of arguments in the same module but I’m doing the job of the compiler and this is very wrong. There must be very good reasons for this choice, but this is not the way to design languages nowadays. We’ve got enough CPU and enough practice to let us write simpler code. This code

-module(math2).
-export([double/1]).
double(X) ->
  times(X, 2).
times(X, N) ->
 X * N.

should have been

module math
export double(X) ->
  times(X, 2)
times(X, N) ->
  X * N

No -, no useless () and [], no .

• The -> sign is an improvement over mainstream languages. They use things like these to define methods and functions

function times(x, n) { ... } // JavaScript
function times($x, $n) { ... } // PHP
int times(int n, int x) { ... } // C
def times(x, n) do ... end # Ruby
public int times(int x, int n) { ... } // Java
sub times { my $x = shift; my $n = shift; ... } # Perl
times () { x = $1; n = $2; ... } # bash

which are all more verbose than Erlang. However -> could be improved. Practice demostrated that two characters combinations are worse than single characters. PHPs -> is painful compared to other languages’ . and Ruby is switching to JavaScript’s : from its own => sign for defining key, value pairs in hash tables.

• Pattern matching with atoms is good. It can be used to implement object orientation in a different and maybe more descriptive way, as in

area({square, Side}) ->
  Side * Side;
area({rectangle, X, Y}) ->
  X * Y;

1> area({square, 5}).
25

• Message passing to processes is extremely good. The echo demo program

-module(echo).
-export([start/0, loop/0]).
start() ->
  spawn(echo, loop, []).
loop() ->
  receive
    {From, Message} ->
      From ! Message,
      loop()
    end.

is very small and clear: From is a process, Message is the message and From ! Message sends message to the from process just after receive {From, Message} has received it from the caller process (From), which is something like this.

import(echo).
P = echo:start().
P ! "hi there!".

Very easy to do.

That was the end of my first hand on session with Erlang. It was based on the example in the freely available excerpts of Concurrent Programming in Erlang and Programming Erlang.
My first impression is that the language has some very good points but lacks some polish. To be fair, Erlang first appeared in 1986, a time when developers where used to all those signs we’re looking at as clutter now. It probably looked like a terse language back then.

I learned some lessons in this month on the move and I want to share them. Most of them are about keeping your data safe. That means you can keep using your pc for all the time you’re away from home, which is important: if you don’t you’ll regret that extra weight in your luggage and the things you won’t be able to do.

1. Have a backup of all programs and data you need on a server on the Internet. If something goes very wrong you can download them and start over. I lost a program I only had on a USB stick and couldn’t reach my pc and my backup disks at home to get it again ;-)
2. Keep a local copy your mail because you’ll have to spend some time offline. Furthermore if you have local copies you won’t be forced to go online when it’s inconvenient or expensive to do it. I’m using Thunderbird and downloading every message, even from the gmail accounts I have to use for part of my work.
3. Backup everything on USB sticks or to the Internet. I was rsyncing my mail to the flash drive and pushing code to git remote repositories. I had too many photos for uploading them over the average Australian Internet connection but I had three copies of them on SD cards, the pc’s SSD and the sticks.
4. USB keys get corrupted sometimes and can’t be read anymore. You’re in trouble if you have something only there. Bring two sticks, maybe 8 GB or more. I had one and had to reformat it.
5. Internet connections can be very slow and very expensive so don’t count only on the Internet to sync and backup data. Luckily I didn’t but read more about this below.
6. Don’t plan to rely on 3G unless you know it’s available everywhere you’re planning to go. If it is, check the local data plans before you leave and buy a local data SIM card. I didn’t but I saw many people using them.
7. Encrypt your file systems just in case your pc gets stolen or lost.
8. The firewalls of some hotels let through only very few ports, sometimes only port 80. If the ssh port is open you can setup a ssh tunnel to smtp and pop servers (you need your own server on the Internet). It’s a bit painful but you can keep use your mail client. If it doesn’t, you need to use webmail until you get into a friendlier place or, if you planned in advance, you have a server on the Internet accepting ssh connections on port 80 and you tunnel over that.

Internet connections and phones deserve some more thoughts.

In my experience Internet was slow everywhere in Polinesia (tested in 2006) but they only have satellite links. Internet was slow everywhere in Mongolia but in the capital (2008). Australia fares better than them but worse than anywhere in Mexico (2005). That must be excused because bits have to go along cables under the ocean to reach every destination outside the country. Very remote areas such as Cape Tribulation might have incredibly slow connections reminding me of 56k modems. Those connections tend to be very expensive, maybe even more than 1 AUD per 10 minutes. Large cities have cheap connections which sometimes approached 1 Mb/s of real bandwidth. Not much actually but very fast compared to what you can get used to. Must locations in Australia have Internet access with Global Gossip. If WIFI is slow where you’re staying find out if they have a local shop and go there.

Always buy a local SIM card for your phone if you can. This is not possible in every country of the world but I got mine shipped from Australia to my home after making a VIP Backpackers card. I redirected my Skype account to it using call forwarding (you must pay for that but their rates are cheap) so customers and friends could talk to me without paying anything. I also created a Skype online number with my home town area code so people not using Skype could call me at their usual rate (you pay for that too).

When going to Australia keep in mind that phones often don’t work outside cities, even on the highways. The country is as large as Europe or the USA and there are not many people living there (20 millions) so a complete coverage it’s not economically feasible. Combining my experience with what people told me,Telstra has the best coverage followed by Optus. Vodafone has a small network (apparently merged with the one of 3) and it’s worth using it only if you’re staying in large cities. There are some virtual operators so check which network they use.

The solution? If you’re a content producer: create RGB jpegs. If you let people upload pictures on your site: convert pictures to RGB. This is how I did it in my latest Ruby on  Rails application which uses Paperclip to handle uploads.

Put this code into lib/paperclip_processors/colorspace.rb (edit: that was for Paperclip 2.3.1.1, check the comments for how to fix than in newer versions)

module Paperclip
  class Colorspace < Thumbnail
    def transformation_command
      " -colorspace RGB " + super
    end
  end
end

Add this line to your model, as an argument of has_attached_file

:processors => [:colorspace],

That’s it. For a discussion about CMYK images and Internet Explorer read this.

A little digging in the error trace got me to the method identify_quotation in the file /usr/lib/ruby/1.8/irb/ruby-lex.rb

I added these lines before RubyLex.fail SyntaxError, "unknown type of %string"

puts "Error parsing Ruby source."
puts "Here are the 200 characters following the error point.\n"
200.times do
  putc getc
end
puts ""

Running rake again let me see the code immediately following the point where the parser failed. As it turned out the problem was the %m format argument of the calls to strtime embedded inside %() or in multi line strings. Rephrasing those strings let rake updatepo succeed.