The scary part is that I have seen some of these in practice.

• Efficiency
• Correctness
• Maintainability
• Understandability
• Personal Expression
• Adherence to Standards
• Scalability
• Reusability
• Testability
• Portability

I’ll give our results here in a minute, but go ahead and pick your top five before reading on.

My personal picks were: Correctness, Maintainability, Understandability, Reusability and Testability.

Once everyone had picked their top five, we got together in groups of four and determine the top three items within our group of four. The top two was easy: Correctness and Maintainability. We had a tie for third place between Understandability and Reusability, and broke the tie in favor of Reusability because we felt that Understandability was intimately tied to Maintainability anyways.

Finally we surveyed the entire group and totalled the votes from each of the subgroups. Heres the results:

• 8: Correctness
• 8: Maintainability
• 3: Reusability
• 2: Efficiency
• 2: Scalability
• 1: Adherence to Standards

We see two strong winners here: Correctness and Maintainability. Every group voted for them. Beyond that, there is no clear winner. One thing I do find interesting is that Testability didn’t make the final cut in any of our groups. Even though our team is developing a strong test driven design culture, it hasn’t (yet) spread to the rest of the department. It seems we have our work cut out for us.

Here’s an example. Suppose you had a list of person objects, and you wanted to extract everybody under the age of 21. You might write some code like this.

  # ARRAY VERSION
  youngsters = people.select { |p| p.age < 21 }

This works great with small in-memeory collections. But what happens when you start storing your objects in a database. Fetching every row from the database and running the comparison on the is not only slow, it defeats the purpose of using a database. So suddenly your code becomes …

  # SQL VERSION
  youngsters = people.select "person.age < 21"

We pass in a string (instead of a block) so we can use the string to build a SQL query. Somewhere buried inside of the select method is a statement that looks something like this:

   def select(query_string)
     sql = "SELECT * FROM person WHERE #{query_string}"
     # Use the SQL string to query the database
   end

We have to switch to string encoded queries because we have a leaky abstraction.

Ryan Pavlik published an interesting Ruby library, called Criteria, that helps to plug this particular leak. Ryan’s Criteria library provides table objects that work like this …

   require 'criteria/sql'
   table = Criteria::SQLTable.new("person")
   query = table.age < 21
   puts query.select   # => "SELECT * FROM person WHERE (person.age < 21)"

Wow. Did you see what just happened? We took an ordinary Ruby expression (table.age < 21) and somehow captured it in data — data that we used to generate an SQL statment. Lets skip how this works for just a moment and consider what we can do with this.

Using Ryan’s criteria, we can now write a database backed collection that doesn’t require us to pass in SQL fragments to do arbitrary queries. Instead we can express our queries in natural Ruby syntax and let the library handle the conversion.

A collection that takes advantage of Criteria might look something like this (in part):

  class People
    def initialize(db)
      @db = db   # DBI database handle
    end

    def select(&block)
      table = Criteria::SQLTable.new("person")
      query = block.call(table)
      @db.select_all(query.select).collect { |row|
        Person.new(row['name'], row['age'])
      }
    end
  end

How it Works

The mechanism behind Criteria is surprisingly simple. It parses the expression by executing it. Sending any message to a table object will cause it to remember that message in a special criterion object, which is returned as the result of the message. Futher messages to the criterion object are also recorded and new criterion objects are returned. The end result is a network of criterion objects that resemble the parse tree for the expression being evaluated. Once you have that parse tree, the rest is easy.

Remaining Leaks

The Ruby code to generate the parse tree is about 50 lines of code, a real tribute to the flexibility of Ruby. However, it is not perfect. Since the library depends on recording messages, anything that is not a message will be lost. Since almost everything in Ruby (including operators) send messages, this is not a problem — except for the short circuit logic operators && and ||. So there’s one leak, Criteria expressions need to use & and | instead of the more natural && and || operators.

The second leak deals with how types are coerced in Ruby. The expression t.age < 21 will work because the less than message is sent to the table object and it knows how to handle it. However, the expression 21 > t.age will send the message to the integer object and it doesn’t know how to handle tables.

Fortunately the restrictions are fairly mild. The Criteria library represents some wonderful "outside the box" thinking to attack a particularly difficult problem.

Charles writes: "Make your actions undo-able. Make your deletions un-deletable." Then the user always has the ability to undo any accidents that may have slipped past his fingers.

This is a great idea. I know that when I started using Emacs as my main editor, it changed the way I editted text. Emacs has an unlimited undo feature, where you can undo the file all the way back to the beginning of the edit session. Suddenly I was no longer afraid to try out new features in the editor. Sure, it might screw up the file, but savlation was only a few keystrokes away.

Let’s start a campaign to rid the world of those annoying confirmation queries!

You can find it here.

On Line Noise

First of all, let me remind everyone that a line noise score is for entertainment purposes only, and trying to derive something deep or serious about a language from its line noise score is counterproductive.

Brian Marick reminded me of a signature survey that Ward Cunningham had done. Ward throws out all but a few characters to generate a signature for a Java file. Surprisingly, you can tell a great deal about a file from the signature. Check it out here.

James Robertson wonders how Smalltalk would fare against the line noise filter. There is some question how to exactly count Smalltalk code, since there is no standard text representation for Smalltalk. I tried the fileout format from Squeak and removed all the extra puctuation that is added during fileout. Here is my results for Smalltalk …

  animal.st (45): :#:'':'':'':'':''.:#:'':'':'':'':''.:=:.:[:|]

James came up with a score of 35, but on his own source code running on a different version of Smalltalk. A lot of the Smalltalk score came from the class declarations, and if VisualWorks differs from Squeak significantly in the way classes are declared, that could easily account for the difference.

Ted Leung would like to see a larger sample. He suspects that the Ruby line noise ratio would be higher. Perhaps, but I haven’t seen it. From the very little research I’ve done, Ruby and Python get roughly the same line noise score on larger files.

On Python and Ruby Mindsets

Richard comments on the PythonAndRuby posting and complains about my characterization of the Python "." operator. He says

"x[y]" is a dictionary lookup. "x.y" is an attribute lookup that can have nothing at all to do with a dictionary.

He mentions that the __ getattr __ and __ getattribute __ hooks in Python may redirect the "." operator so that no actual dictionary is used. He suggests "attribute lookup" would be more accurate.

His terminology is probably more accurate, and I was delibrately glossing over the details. But the main point that Python programmers view the "." as some type of lookup is still very pertinent.

The Right Attitude

In the midst of all these Python and Ruby comparisons, I would like to wrap up with a posting from Mauricio Fernandez’s blog. Mauricio recounts a conversation with a stranger on a plane where they discuss design patterns and dynamic languages.

It was only much later that I remembered that he told me he was a Pythonist. This means (obviously) that we should have fought to death. Too bad we forgot it ;)

I like that attitude!

Thanks to everyone for the feedback.

However, one thing I have noticed is that Rubyists and Pythonistas bring different assumptions to the table. What is intuitive to one group seems ad hoc and downright weird to the other.

Here’s one example that I noticed.

In Python, the construct "x.y" means lookup the name "y" in the dictionary of "x". (Ok, that’s simplified, but you get the picture). The result of that lookup may be a value (instance data), or may be a function that can be called (member functions). In other words, "." is basically a lookup operation.

In Ruby, the construct "x.y" means send a message named "y" to the object represented by "x". All operations on an object must be done through the sending of a message.

This leads to all kinds of non-intuitive behavior if you are expecting the wrong thing. For example, one Pythonista commenting on the fact that "x.y" and "x.y()" are equivalent asks the question: "How do you get access to the y function inside x?".

A Rubyist’s response to the question is "Huh?". You see, there are no "functions" in Ruby. At least not in the way the question assumes. Ruby’s unit of execution is a method, and methods are not first class objects in Ruby. If you wish to have a callable object that references a method on a particular object, then that is easily accomplished by asking the object for a Method object.

  o = Object.new
  m = o.method(:to_s)
  m.class              # => Method

Note the difference between a method (an internal Ruby construct) and a Method object (an object that represents the internal method bound to an instance). To "call" a method object (or any callable object in Ruby), you send the object the "call" message. So, the result of calling m should be identical to the result of sending to_s to o directly. And we see that it is.

  m.call               # => "#<Object:0x401c8618>"
  o.to_s               # => "#<Object:0x401c8618>"

Although Python and Ruby are similar in many ways, their basis for computation is quite different in some fundamental ways. Python follows C++ and Java in using "." as a structure selector operation, while Ruby is more closely attuned to Smalltalk’s message sending paradigm. Both approaches work and are interally consistent, but be prepared for surprises if you try to interpret one using the assumptions of the other.

Then I remembered the C/C++ assert macro. We wrote a a series of small functions that looked something like this:

  void TestCanRun () {
      assert(   CanRun( ... ) );
      assert( ! CanRun( ... ) );
      // and so on ...
   }

I made a TestAll() function that called all my little test functions and arranged to have TestAll() called as the first thing in main(). If all the assertions worked, the program completed normally. If an assertion failed, the program would abort with an informative error message.

It worked great. I wouldn’t leave production code like that, but it was a great way to introduce Test Driven Design to someone who had not seen it before, without getting into all the nitty-gritty details of setting up a full blown test suite.

I guess my motto is that friends shouldn’t let friends program without tests.

I did something similar a while back. A Java programmer had taken a look at Ruby and declare that he didn’t like all that "line noise" in the language. He was refering to the "@" and "$" characters used to mark instance variables and globals. I pointed out that Ruby actually uses quite a bit less punctuation than Java, and wrote the following linenoise program do demonstrate.

    #!/usr/bin/env ruby
    ARGV.each { |fn|
      noise = open(fn) { |file| file.read }.gsub(/[A-Za-z0-9_ \t\n]/m, "")
      puts "#{fn} (#{noise.size}): #{noise}"
    }

Linenoise will strip out all alphanumeric characters and white space, leaving only the "line noise" behind. Running linenoise on a series of small programs written in different languages produces this (edited slightly for line breaks) …

  animal.cc   (83):  #<>{:()=;};:{:();};::(){::<<"\";}:{:();};::(){::
                     <<"\";}(){*[]={,};(=;<;++)[]->();;}
  Animal.java (67):  {{();}{(){..("");}}{(){..("");}}([]){[]=[]{(),()}
                     ;(=;<.;++)[].();}}
  animal.pl   (41):  ;{{};}{"\";};{{};}{"\";};$(->,->){$->();}
  animal.py   (23):  :():"":():""[(),()]:.()
  animal.rb   (10):  """"[.,.].

The number in the paranthesis is the number of line noise characters in the file.

What I find interesting is the amount of semantic information that still comes through the "line noise". For example, the "#<>" sequence in the C++ code is obviously an include statement for something in the standard library and the "<<" are output statements using "cout".

It would be interesting to see if you could determine the language given only the line noise. You could tell Java from C++ by the ";}" vs ";};" punctuation. Python is pretty clear from the ’:():"":():’ style patterns.

Before I go, here is the source code to the Animal programs I used in my examples…

Language: C++

  #include <iostream>

  class Animal {
  public:
      virtual void talk() = 0;
  };

  class Dog : public Animal {
  public:
      virtual void talk();
  };

  void Dog::talk() {
      std::cout << "WOOF\n";
  }

  class Cat : public Animal {
  public:
      virtual void talk();
  };

  void Cat::talk() {
      std::cout << "MEOW\n";
  }

  int main() {
      Animal * (a[]) = { new Dog, new Cat };
      for (int i=0; i<2; i++)
          a[i]->talk();
      return 0;
  }

Language: Java

  public class Animal {
      interface IAnimal {
          void talk();
      }

      static class Dog implements IAnimal {
          public void talk () {
              System.out.println("WOOF");
          }
      }

      static class Cat implements IAnimal {
          public void talk() {
              System.out.println ("MEOW");
          }
      }

      public static void main (String args[]) {
          IAnimal[] zoo = new IAnimal[] { new Dog(), new Cat() };
          for (int i=0; i<zoo.length; i++)
             zoo[i].talk();
      }
  }

Language: Perl

  package Dog;

  sub new {
      bless {};
  }

  sub talk {
      print "WOOF\n";
  }

  package Cat;

  sub new {
      bless {};
  }

  sub talk {
      print "MEOW\n";
  }

  package main;

  for $a (Dog->new, Cat->new) {
      $a->talk();
  }

Language: Python

  class Dog:
      def talk(self):
          print "WOOF"

  class Cat:
      def talk(self):
          print "MEOW"

  for a in [Dog(), Cat()]:
      a.talk()

Language: Ruby

  class Dog
    def talk
      puts "WOOF"
    end
  end

  class Cat
    def talk
      puts "MEOW"
    end
  end

  for a in [Dog.new, Cat.new]
    a.talk
  end