I’ll be talking about Rake at the upcoming Ruby Conference. I’m in the process of cleaning up the code, writing more docs and getting my presentation ready. So if there is any feedback you would like to provide, now would be a good time.

What Is UML?

User Mode Linux is a version of Linux that sits on top of a running linux system and runs entirely in the users unprotected process space. At first glance, this seems like a really useless idea. Why run a virtual Linux when you have a "real" version already running on your hardware. It turns out there are a number of useful things

So why would you want to run a User Mode Linux? Here’s some ideas…

• Kernel Development. Running your test kernels in user space makes it much easier to debug your kernels. And crashing a kernel just brings down your local copy, nothing else.
• Experimenting with New Kernels or Distributions. Again, because the UML is isolated from the real system, you can try out new kernels or even whole distributions that you would not normally want to deal with.
• Secure Sandbox. Since the UML has no access (not explicitly granted) to the host Linux system, a UML is a great place to run potentially malicious applications.
• Experimentation. Ever wonder what would happen if you typed "rm -f /" as root. Do it in a UML without fear of breaking anything else.
• Networking. UMLs each act as their own host, so you can setup a network of UMLs on a single box to experiment with networking solutions.
• UML Farm. Put a bunch of UMLs on a single server and let individual users have an entire (virtual) linux system to themselves. We’ll come back to this idea in a moment.

You get the idea. So why is this blog hosted on a UML? First, we need some history.

The UML Coop

Over the years, the local ISP that I use went from being a great ISP to being a very mediocre company. Changing hands several times, it was finally bought by Nuvox who offered to "upgrade" our 100 Mbs of web storage to a generous 5 Mb. Several members of CLUG began looking for alternative web hosting solutions.

In January of this year (2003), David Coulson from the Cleveland Linux User Group came to give a presentation on UML. David has done extensive work with UML and mentioned that he was interested in starting a COOP that would purchase a server on the internet. Each member of the COOP would get their own UML instance to do whatever they wanted.

It took several months before the COOP came together, but by May we had purchased a server and hosted it at N2Net in Cleveland. The system is currently supporting around 20 UML instances with little problem (except for the one day we got slashdotted).

Using UMLs

How CLUG members are using their UMLs?

• Kip
  • Web hosting for shafferhouse.org, a family picture galary.
  • Web hosting for exams4pilots.org, a practice exam site for pilots.
  • TMDA Mail Filtering (TMDA is worth a posting in its own right).
• Jim
  • Web hosting for onestepback.org, this blog site.
  • Mail filtering (someday)
• Jeff
  • Jeff has applied to join the UML COOP, but hasn’t got his UML instance setup yet.
• Monty
  • Shell access from an internet host
• Ryan
  • Mirroring for picture galleries. Ryan hosts a web site on his home system, dlugosz.net. To avoid high bandwidth on his home box, he redirects graphic requests to his UML box.

Some UML Links

• user-mode-linux.sourceforge.net/
• usermodelinux.org/

The scary part is that I have seen some of these in practice.

• Efficiency
• Correctness
• Maintainability
• Understandability
• Personal Expression
• Adherence to Standards
• Scalability
• Reusability
• Testability
• Portability

I’ll give our results here in a minute, but go ahead and pick your top five before reading on.

My personal picks were: Correctness, Maintainability, Understandability, Reusability and Testability.

Once everyone had picked their top five, we got together in groups of four and determine the top three items within our group of four. The top two was easy: Correctness and Maintainability. We had a tie for third place between Understandability and Reusability, and broke the tie in favor of Reusability because we felt that Understandability was intimately tied to Maintainability anyways.

Finally we surveyed the entire group and totalled the votes from each of the subgroups. Heres the results:

• 8: Correctness
• 8: Maintainability
• 3: Reusability
• 2: Efficiency
• 2: Scalability
• 1: Adherence to Standards

We see two strong winners here: Correctness and Maintainability. Every group voted for them. Beyond that, there is no clear winner. One thing I do find interesting is that Testability didn’t make the final cut in any of our groups. Even though our team is developing a strong test driven design culture, it hasn’t (yet) spread to the rest of the department. It seems we have our work cut out for us.

Here’s an example. Suppose you had a list of person objects, and you wanted to extract everybody under the age of 21. You might write some code like this.

  # ARRAY VERSION
  youngsters = people.select { |p| p.age < 21 }

This works great with small in-memeory collections. But what happens when you start storing your objects in a database. Fetching every row from the database and running the comparison on the is not only slow, it defeats the purpose of using a database. So suddenly your code becomes …

  # SQL VERSION
  youngsters = people.select "person.age < 21"

We pass in a string (instead of a block) so we can use the string to build a SQL query. Somewhere buried inside of the select method is a statement that looks something like this:

   def select(query_string)
     sql = "SELECT * FROM person WHERE #{query_string}"
     # Use the SQL string to query the database
   end

We have to switch to string encoded queries because we have a leaky abstraction.

Ryan Pavlik published an interesting Ruby library, called Criteria, that helps to plug this particular leak. Ryan’s Criteria library provides table objects that work like this …

   require 'criteria/sql'
   table = Criteria::SQLTable.new("person")
   query = table.age < 21
   puts query.select   # => "SELECT * FROM person WHERE (person.age < 21)"

Wow. Did you see what just happened? We took an ordinary Ruby expression (table.age < 21) and somehow captured it in data — data that we used to generate an SQL statment. Lets skip how this works for just a moment and consider what we can do with this.

Using Ryan’s criteria, we can now write a database backed collection that doesn’t require us to pass in SQL fragments to do arbitrary queries. Instead we can express our queries in natural Ruby syntax and let the library handle the conversion.

A collection that takes advantage of Criteria might look something like this (in part):

  class People
    def initialize(db)
      @db = db   # DBI database handle
    end

    def select(&block)
      table = Criteria::SQLTable.new("person")
      query = block.call(table)
      @db.select_all(query.select).collect { |row|
        Person.new(row['name'], row['age'])
      }
    end
  end

How it Works

The mechanism behind Criteria is surprisingly simple. It parses the expression by executing it. Sending any message to a table object will cause it to remember that message in a special criterion object, which is returned as the result of the message. Futher messages to the criterion object are also recorded and new criterion objects are returned. The end result is a network of criterion objects that resemble the parse tree for the expression being evaluated. Once you have that parse tree, the rest is easy.

Remaining Leaks

The Ruby code to generate the parse tree is about 50 lines of code, a real tribute to the flexibility of Ruby. However, it is not perfect. Since the library depends on recording messages, anything that is not a message will be lost. Since almost everything in Ruby (including operators) send messages, this is not a problem — except for the short circuit logic operators && and ||. So there’s one leak, Criteria expressions need to use & and | instead of the more natural && and || operators.

The second leak deals with how types are coerced in Ruby. The expression t.age < 21 will work because the less than message is sent to the table object and it knows how to handle it. However, the expression 21 > t.age will send the message to the integer object and it doesn’t know how to handle tables.

Fortunately the restrictions are fairly mild. The Criteria library represents some wonderful "outside the box" thinking to attack a particularly difficult problem.

Charles writes: "Make your actions undo-able. Make your deletions un-deletable." Then the user always has the ability to undo any accidents that may have slipped past his fingers.

This is a great idea. I know that when I started using Emacs as my main editor, it changed the way I editted text. Emacs has an unlimited undo feature, where you can undo the file all the way back to the beginning of the edit session. Suddenly I was no longer afraid to try out new features in the editor. Sure, it might screw up the file, but savlation was only a few keystrokes away.

Let’s start a campaign to rid the world of those annoying confirmation queries!

You can find it here.

On Line Noise

First of all, let me remind everyone that a line noise score is for entertainment purposes only, and trying to derive something deep or serious about a language from its line noise score is counterproductive.

Brian Marick reminded me of a signature survey that Ward Cunningham had done. Ward throws out all but a few characters to generate a signature for a Java file. Surprisingly, you can tell a great deal about a file from the signature. Check it out here.

James Robertson wonders how Smalltalk would fare against the line noise filter. There is some question how to exactly count Smalltalk code, since there is no standard text representation for Smalltalk. I tried the fileout format from Squeak and removed all the extra puctuation that is added during fileout. Here is my results for Smalltalk …

  animal.st (45): :#:'':'':'':'':''.:#:'':'':'':'':''.:=:.:[:|]

James came up with a score of 35, but on his own source code running on a different version of Smalltalk. A lot of the Smalltalk score came from the class declarations, and if VisualWorks differs from Squeak significantly in the way classes are declared, that could easily account for the difference.

Ted Leung would like to see a larger sample. He suspects that the Ruby line noise ratio would be higher. Perhaps, but I haven’t seen it. From the very little research I’ve done, Ruby and Python get roughly the same line noise score on larger files.

On Python and Ruby Mindsets

Richard comments on the PythonAndRuby posting and complains about my characterization of the Python "." operator. He says

"x[y]" is a dictionary lookup. "x.y" is an attribute lookup that can have nothing at all to do with a dictionary.

He mentions that the __ getattr __ and __ getattribute __ hooks in Python may redirect the "." operator so that no actual dictionary is used. He suggests "attribute lookup" would be more accurate.

His terminology is probably more accurate, and I was delibrately glossing over the details. But the main point that Python programmers view the "." as some type of lookup is still very pertinent.

The Right Attitude

In the midst of all these Python and Ruby comparisons, I would like to wrap up with a posting from Mauricio Fernandez’s blog. Mauricio recounts a conversation with a stranger on a plane where they discuss design patterns and dynamic languages.

It was only much later that I remembered that he told me he was a Pythonist. This means (obviously) that we should have fought to death. Too bad we forgot it ;)

I like that attitude!

Thanks to everyone for the feedback.

However, one thing I have noticed is that Rubyists and Pythonistas bring different assumptions to the table. What is intuitive to one group seems ad hoc and downright weird to the other.

Here’s one example that I noticed.

In Python, the construct "x.y" means lookup the name "y" in the dictionary of "x". (Ok, that’s simplified, but you get the picture). The result of that lookup may be a value (instance data), or may be a function that can be called (member functions). In other words, "." is basically a lookup operation.

In Ruby, the construct "x.y" means send a message named "y" to the object represented by "x". All operations on an object must be done through the sending of a message.

This leads to all kinds of non-intuitive behavior if you are expecting the wrong thing. For example, one Pythonista commenting on the fact that "x.y" and "x.y()" are equivalent asks the question: "How do you get access to the y function inside x?".

A Rubyist’s response to the question is "Huh?". You see, there are no "functions" in Ruby. At least not in the way the question assumes. Ruby’s unit of execution is a method, and methods are not first class objects in Ruby. If you wish to have a callable object that references a method on a particular object, then that is easily accomplished by asking the object for a Method object.

  o = Object.new
  m = o.method(:to_s)
  m.class              # => Method

Note the difference between a method (an internal Ruby construct) and a Method object (an object that represents the internal method bound to an instance). To "call" a method object (or any callable object in Ruby), you send the object the "call" message. So, the result of calling m should be identical to the result of sending to_s to o directly. And we see that it is.

  m.call               # => "#<Object:0x401c8618>"
  o.to_s               # => "#<Object:0x401c8618>"

Although Python and Ruby are similar in many ways, their basis for computation is quite different in some fundamental ways. Python follows C++ and Java in using "." as a structure selector operation, while Ruby is more closely attuned to Smalltalk’s message sending paradigm. Both approaches work and are interally consistent, but be prepared for surprises if you try to interpret one using the assumptions of the other.