raganwald
(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)

Sunday, September 17, 2006
  Ruby Metaprogramming: A generic controller for REST-ful services on Rails


Disclaimer:

This is very raw code, and I've only been using it for a few days. It isn't in production. It hasn't been peer-reviewed. There are far fewer lines of test code than lines of functional code.

In short, this is really a peek over my shoulder at what's running on my MacBook.

Dale on the Approach

It has shortcomings. I recently read a blog post where the author complained that many of the naked people at Burning Man weren't pleasing to his eyes. Fair enough. But... is nudism always about exhibitionism? I think in many cases, it's about being perfectly aware of one's lack of physical beauty but being comfortable enough with oneself that you don't need to hide inside your clothing.

I am posting this code in that spirit. Warts and all, I am trying to get my head out of the "don't show it until it has been polished to perfection" mind set.

If you don't find it attractive, I hope you at least find it interesting.

What I needed:

I'm doing some work with bridging multiple Rails applications. In the back of my mind I thought I'd want to use ActiveWebServices in the production code, but when I got to the point of integrating my databases, I followed the principle of YAGNI and decided I would use the simplest thing that could possibly work.

So I figured I would build one of the Rails applications around a REST-ful interface. The other applications could wrestle with REXML and Net::HTTP to talk to it. The joke on me is that this isn't the simplest thing that could possibly work. In my ignorance of all things Rails, I didn't know that I could have one Rails application use different databases for different models.

Rails and REST:

Rails 1.1 makes REST controllers as easy as:
class WardersController < ApplicationController

def index
@warders = Warder.find :all
respond_to do |wants|
wants.html
wants.xml { render :xml => @warders.to_xml }
end
end

# ...

end
I also needed to implement the show action. Once again, it’s fairly easy to handle the default case:
def show
begin
@warder = Warder.find params[:id]
respond_to do |wants|
wants.html
wants.xml { render :xml => @warder.to_xml }
end
rescue ActiveRecord::RecordNotFound
render :nothing => true, :status => 404
end
end
Shortly after I had this working, it was time to implement filters. My first need was to be able to find the first Inmate that exactly matched something other than an id. I believe that it is far better to extract infrastructure through refactoring common code than to over-engineer up front. But it seemed obvious that I needed a generic “find on any column” feature.

I did cobble something together, and it worked: you could search for an inmate using a GET such as http://thevillage.org/inmates/show?number=6. Shortly after I had it working for the show action I discovered that I needed the exact same functionality for the index action as well. And it was obvious it ought to work for any model controller without resorting to copy and paste.

Rails Controllers and Filters

This is a common concern, and Rails provides Aspect-Oriented Programming hooks for controllers called filters just for this kind of purpose (they’re also handy for security). I decided to write a filter. Since I wanted to use it for all of my models, I placed the filter in application.rb, the common superclass of all controllers.

Let’s start with a look at what a typical controller now looks like:

class InmatesController < ApplicationController

before_filter :first_inmate, :only => [ :show, :update, :edit, :destroy ]
before_filter :all_inmates, :only => [ :index ]

def index
respond_to do |wants|
wants.html
wants.xml { render :xml => @inmates.to_xml }
end
end

def show
if @event.nil?
render :nothing => true, :status => 404
else
respond_to do |wants|
wants.html
wants.xml { render :xml => @inmate.to_xml }
end
end
end

# ...

end
As you can see, I’ve added before_filters to our actions. These filters set the instance variables @inmate or @inmates for the action’s use. You can also see that the filter methods are customized for each model class. As you’ll see in a moment, if you add a new model class, say, Rover, you’ll get first_rover and all_rovers methods for free.

Here’s the code for the methods:

class ApplicationController < ActionController::Base

class << self
def before_filter *args
method_name = args[0].to_s
if method_name =~ /^first_(.*)$/
class_name = Inflector.singularize((Inflector.camelize $1))
instance_name = '@' + Inflector.singularize((Inflector.underscore $1))
load "#{Inflector.singularize(Inflector.underscore $1)}.rb"
clazz = Kernel.const_get(class_name)
define_method method_name do
self.instance_variable_set instance_name, (find_filter :class => clazz, :find => :first)
end
elsif method_name =~ /^all_(.*)$/
class_name = Inflector.singularize((Inflector.camelize $1))
instance_name = '@' + Inflector.pluralize((Inflector.underscore $1))
load "#{Inflector.singularize(Inflector.underscore $1)}.rb"
clazz = Kernel.const_get(class_name)
define_method method_name do
self.instance_variable_set instance_name, (find_filter :class => clazz, :find => :all)
end
end
super
end
end

private

def find_filter options
options[:find] ||= :first
if params[:id]
begin
(options[:find] == :first && (Event.find params[:id])) || [ (Event.find params[:id]) ]
rescue ActiveRecord::RecordNotFound
(options[:find] == :all && []) || nil
end
else
column_names = options[:class].column_names
clauses = (column_names.inject([]) { |cond_clauses, column_name| params[column_name] && cond_clauses << "#{column_name} = ?"; cond_clauses })
full_conditions = (!clauses.empty? && (column_names.collect { |column_name| params[column_name] }).compact.unshift((clauses.join ' AND '))) || nil
if params[:order_by]
words = params[:order_by].split
order_column_name = (column_names.include? words.first) && words.first
if order_column_name
direction = (words.last && (words.last.upcase == 'DESC') && ' DESC') || ''
(full_conditions && (options[:class].find options[:find], :conditions => full_conditions, :order => "#{order_column_name}#{direction}")) || (options[:class].find options[:find], :order => "#{order_column_name}#{direction}")
elsif full_conditions
options[:class].find options[:find], :conditions => full_conditions
else
options[:class].find options[:find]
end
elsif full_conditions
options[:class].find options[:find], :conditions => full_conditions
else
options[:class].find options[:find]
end
end
end
end
As you can see, I intercept calls to before_filter and define a small wrapper method around find_filter. I was going to introspect on the controller name to get the class, however I wanted to allow the posibility that I would create controllers with arbitrary names. And after looking at the way the filters look, the method names seem to have documentation value.

It also seemed possible to eliminate the filter calls in the controller, but they are so short that it didn't seem like a big code win to abstract those up into ApplicationController.

So... there you have it. In summary, all of the controllers in this rails application now support simple filtering on column names like http://thevillage.org/escapes/outcome=failure or http://thevillage.org/butlers/stature=short&umbrella=yes.

My thoughts on metaprogramming:

I had to stop learning Lisp. I realized that at the rate I was going I would quickly automate myself right out of a job.
Senzei

I like dynamic metaprogramming. Obviously. On the other hand, it’s hard to write an IDE that knows where to find the definition for all_rovers. I think that as Ruby matures, people will look for ways to assist tool vendors, perhaps by extending reflection.

OO theorists sometimes complain that even this mild form of dynamic metaprogramming violates the purity of defining all of the verbs for a noun up front. At the risk of arguing with a straw man, I feel that this is well within the spirit of OOP. What we’re really saying is all objects that extend ApplicationController have two extra methods, first_something and all_somethings, we just don’t know the name of something yet.

Defining the methods using method_missing:

My first cut at the implementation used method_missing to handle the new methods instead of define_method. Stefan Tilkov's comment (below) inspired me to try defining the methods dynamically. Here's the original code:
 def method_missing method
method_name = method.id2name
if method_name =~ /^first_(.*)$/
class_name = Inflector.singularize((Inflector.camelize $1))
instance_name = '@' + Inflector.singularize((Inflector.underscore $1))
load "#{Inflector.singularize(Inflector.underscore $1)}.rb"
clazz = Kernel.const_get(class_name)
self.instance_variable_set instance_name, (find_filter :class => clazz, :find => :first)
elsif method_name =~ /^all_(.*)$/
class_name = Inflector.singularize((Inflector.camelize $1))
instance_name = '@' + Inflector.pluralize((Inflector.underscore $1))
load "#{Inflector.singularize(Inflector.underscore $1)}.rb"
clazz = Kernel.const_get(class_name)
self.instance_variable_set instance_name, (find_filter :class => clazz, :find => :all)
else
super
end
end
While I embrace the spirit of dynamic metaprogramming, I don’t care for the aesthetics of implementing methods with method_missing. In my ideal world, we’d combine pattern matching with our existing syntactic forms of defining a method so that it is much more obvious that ApplicationController defines those two methods. That would be a win for programmers and for tools. And we could do that today as syntactic sugar by rewriting pattern matching definitions using method_missing.
 

Comments on “Ruby Metaprogramming: A generic controller for REST-ful services on Rails:
Without knowing any Ruby I don't know how well I can comment on this, but when has that stopped anybody?

I like the "do the simplest thing that could possibly work" concept, simply because it is so good at biting people in the ass. I think most programmers get a little too wrapped up in specifics of the current problem. (generalizing myself to be "most programmers", feel free to shoot me down) Statements like "do the simplest thing that could possibly work" are the progenitors of monsters that would make Dr. Frankenstein squeamish when we allow ourselves to say "X will just be an implementation detail".

In your case, it looks like X was "connecting other rails applications to your RESTful interface". Keeping everything in balance and looking at the problem from multiple levels at once is hard. At the least I suck at it, and it appears to be the case that everyone imitates me on occassion.

Re: the metaprogramming statement that my quote prefaced, I actually don't like most popular language's support for metaprogramming. My programming "career" started out as maintenance work on a bunch of VB scripts left by a former employee. It quickly taught me that decipherability and consistency were awesome traits in a programming language.

Given all of that, metaprogramming scares me. It lets any old moron open up a class and break everyone else's code in new and painfully interesting ways. When this happens during a project the solution is simple: Go grab a clue-by-four and exorcise the problem. Trying to pick up after this six months later is another story.

Until I am convinced that a language has something to help me figure out when in the flow of execution something gets modified, to what, by whom, etc; I will be avoiding metaprogramming like the plague. From a maintenance perspective it is a great way to shoot yourself (or any hapless successors) in the foot a few months down the line.
 
Cool. I'm not yet confident enough with Ruby metaprogramming to fully understand this, though: If I understand the code correctly, you don't actually add the methods, do you? Instead, your "magic" seems to run any time one of the missing methods is called. Wouldn't it be easier/nicer to really add the methods, which would mean that method_missing would only be invoked the first time? Or am I missing something (quite likely)?
 
Wouldn't it be easier/nicer to really add the methods, which would mean that method_missing would only be invoked the first time?

Ahhh, memoization. That has several advantages. First, performance. Second, run-time reflection. If and when Ruby catches up with Smalltalk-80, we'll have inspectors that will display a class's methods, and your suggestion will be a vast improvement.

Thanks.
 
Doesn't irb allow for method inspection? What's the difference to Smalltalk's inspection (another language I'm much too ignorant of)?
 
Doesn't irb allow for method inspection?

Yes, of course it does. But I don't know: can you attach irb to an existing Ruby process?

If you can, irb has a command line interface. It isn't a lot of fun to browse objects and classes manually and it doesn't "make me happy."

Smalltalk provides a GUI you can use to browse objects and classes in a running image.

Ruby will benefit from tools like this. It just needs people who are passionate about IDEs.
 
Thanks. FYI to anyone trying to incorporate this code, here is my newby experience:

In the find_filter method, I had to change "Event" to "options[:class]". For the url's, I changed the last '/' to '?' to prevent routing errors.
 




<< Home
Reg Braithwaite


Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

Books
What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

Share
rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

Beauty
IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

Work
The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Management
Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

Notation
The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Opinion
Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

Whimsey
The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

History
06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /