raganwald
(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)

Friday, October 20, 2006
  Is your IDE tilting at the dynamic language windmill?


Quite a few programmers love having powerful IDEs. These IDEs perform certain simple refactorings somewhat automatically, they can take a compiled language and make it seem a little more like an interpreted language, they can find certain types of errors as you type…

In short, they do a lot of drudge work on the programmer’s behalf. What’s not to like about this? Nothing. These are useful tools. I have been working with Eclipse lately when editing Java, although I really don’t care one way or another. I like what it does but I’m fundamentally indifferent to its existence.

Of course, there are more than a few people who espouse the idea that the most powerful features of these IDEs are only possible with statically typed languages. And that this productivity is so important that it—ummm—eclipses all other considerations such as which language allows you to express a program with the least redundancy.

How interesting.

Do all powerful IDEs require static typing?

The idea that IDEs make you more productive is nothing particularly specific to statically typed languages. Have you ever heard of an editor called Emacs? Did you know that some of the hairiest, most dynamic, self-modifying code ever written was once part of Emacs? And that the author of that dynamic, self-modifying code, some fellow named Jim, went on to develop a popular statically typed language?

Anyways, lots of people use Emacs for Lisp (don’t you love the symmetry? you need to use Lisp for Emacs). They invest a lot of time making themselves more productive.

So here’s what I see from my chair: There’s this group of star-bellied sneetches who say “the important thing about being a productive programmer is having a tool that makes you productive.” And there’s this group of sneetches who say the same dammned thing. The only difference between them is that the star-bellied sneetches say that the productivity of the tool is all about the special things the tool can do when the programs are written in statically typed languages. The plain sneetches love the tool that they can customize so heavily it feels like a personal extension of their mind.

Hmmm.

Who deserves a title bout?

The thing that strikes me as obvious is that I’m the wrong person to argue with a star-bellied sneetch. I use a fairly nice text editor, TextMate, but it doesn’t do 1% of what Emacs does and while it does a lot more than I ever figured out how to make Eclipse do, it also doesn’t even try to do a lot of the star-bellied stuff like move methods around.

The right thing to do is to get two people who both agree that productivity from IDEs is important to debate how to do it. That way there’s one, clear point of debate: does an IDE + static typing beat a programmable editor by an order of magnitude?

Debating with me is going to be entertaining but inconclusive, because we’ll be mixing several arguments up: are IDEs important, and if they are, what’s the best way to make an IDE powerful, and if that is the best way, is it so good that it outweighs other considerations for choosing a programming language? If you stop by my house, that’s a conversation that is going to put a serious dent in my collection of single malts.

The third perspective

I’m not a sneetch in this matter. I spend far, far more time thinking about code than I do editing code. Therefore, I invest more energy looking for ways to improve the way I think about code than I do looking for ways to improve the way I edit code.

Kinda something not entirely unlike what happens when you sit down to improve the performance of a program. First you profile. Then you choose what to optimize. I’ve profiled myself, and what I need is more help deciding what to do than I do doing it.

I am also holding on to a somewhat axiomatic (and irrepressibly optimistic) belief that some languages have orders of magnitude higher signal-to-noise ratio than others. This is important for understanding why I am reluctant to pick a language because my IDE likes it.

For starters, one needs to understand the goal of all this refactoring: it’s to have better code. Better languages also achieve this goal. let’s look at a simple example and work backwards from that to the importance of languages.

Delegation and dynamic programming

Delegation in Ruby is easy because you have send and you have method_missing, so even if you don’t feel like using the built-in SimpleDelegator or Forwardable, you can forward messages whenever and however you like.

Whereas in Java, you need to define an Interface, then make sure that person_doing_the_actual_work implements the interface, then you can use a DynamicProxy to implement delegation. Note that you can’t delegate to a built-in class unless it implements an interface, so you can delegate to ArrayList (because it implements the List interface), but you can’t delegate to String.

(Well, you can forward calls to classes that don't implement interfaces, but you can't make the delegator look like a delegatee. So what you really have is a kind of private inheritence. You cannot, for example, decorate a StringBuffer so that you can set a flag for whether it has been encoded for display in HTML or not, and decorate the append method so that you can throw an exception when an unencoded String is appended to an encoded StringBuffer).

Why can’t you just generate the delegation using the IDE’s handy-dandy feature?

The problem is that code generation produces a static change. It’s a one-time thing, like one of those push-button programming wizards from MSFT. If you change interfaces, perhaps you are refactoring something else, your delegation breaks. So you have to write something relatively dynamic.

In Ruby, it looks things up at run time. I’ll settle for something that can reflect at compile time. But either way, you can’t just do it once. You have to have dynamic code handle the forwarding.

(There are exactly similar problems in all sorts of code. If you write some serialization code by hand or with an IDE, you have a problem when you expand an object. The old code doesn’t know how to serialize the new members. You have to make sure your serialization code reflects on the object so it can keep up with all changes.)

I find that IDEs are nice, but even the most sophisticated IDEs are… static. They make one-time changes to code. I’m far more interested in languages that let me write code that writes code, in languages that let me write code that doesn't need a lot of rewriting and fixing across the board when I make a change. (This is just my perspective. I’m still learning a lot about the intersection between theory and practice in programming languages.)

But this is why you’re tilting at windmills when you ask me why I don’t eschew Ruby for Java just so I can “rename class.”
 

Comments on “Is your IDE tilting at the dynamic language windmill?:
So, where do you live? I'd love the chance to deplete your stash of single malts.
 
One thing I find is often overlooked in the perennial IDE wars is code navigation features. After all, it has become commonplace to say that code is much more often read than written. IDEs can help, specially when facing an entirely new code base. A few helpful features are: jump to declaration (F3 in Eclipse); method call hierarchy (CTRL+ALT+H); find occurrences (CTRL+SHIFT+G); quick hierarchy (CTRL+T) - to find where the current method was declared and where it was overwritten in the inheritance tree; browser-like navigation (back, forward); open type (CTRL+SHIFT+T). That's about all I could remember now...
I'm not arguing that all these features are only possible in statically typed languages, my point is simply that language aware IDEs can help with much more than just refactorings.
 
One thing I find is often overlooked in the perennial IDE wars is code navigation features.

Fair enough, but it's probably better to compare notes with an Emacs user. Or a Smalltalk refactoring browser user. Or the author of BicycleRepairMan, an intelligent editor for Python.

I agree these features are useful. And I use them, for example I never use F3, but CTRL-click seems to do the same thing.

I'm just not prepared to give up an entire programming language for one of these features.
 
I confess, I find it difficult to take the Pub Landlord of development seriously on any issue, let alone the greatness of IDEs. But if the IDE is the revolution, surely what you want is an editor so tightly integrated with the language it's editing that there's no observable difference between the two - that the extension language for the editor is the language it's editing...? At what point does that cease to be an IDE and become "the editor is now part of the language", so that it can take advantage not only of static declaration, but of dynamic information too?

But then, it always seemed to me that the only reason there was even a debate about this was because of the silly insistence on writing code in batches because that's the way it was done 50 years ago when you left your stack of punched cards to compile overnight... Honestly, if the work that has been put into optimising batch-mode compilers and duplicating the functionality in the tools to be used with them over the past 40 years had instead been channelled into figuring out how to extract standalone binary lumps from, and enable effective collaboration between, interactive dynamic environments... well, for a start we'd all be using Lisp and Smalltalk machines. ;)
 
Reg :
I did use the refactoring browser (yes, Smalltalk is my first OO language), and it is far less comfortable than eclipse and intellij. I'm not arguing in the static types debates, only on this point. Renaming methods without having to go for each use (even if the browser brings me there automagically) is easier for instance.

Moreover, I think your delegation point is not quiet right, because even if Proxy on concrete classes would have been possible, it wouldn't have reduced the signal-to-noise ratio.

In language design, you have some forces: the user's desire (I want to write common stuff concisely, I want my mistakes to be outlined as soon as possible ...), the implemetor's desire (I don't want the compiler to be rocket science), the linguist desire (I want the language's construction to be sound). Then since implementor and designer did some compromises on user's back; come tools (IDE, JDO enhancers, parsers generators, xdoclet ...)

For the time being, I will only speak about sound type systems : "static type system" means "type are checked statically and dynamically" while "dynamic type system" means "types are checked only at runtime"

In addressing user's ease of use, there are various trends:
- meta-programming (with a spectrum going from C++ to smalltalk)
- simple syntax + tools (java) where the syntax is kept simple so that tools vendors have less pain at building their tools
- huge syntax that addresses by adhoc constructions the most common needs of developers (C#, old-school O'caml) where there is less space for tools.

There is another problem: semantic weakening. Each time you transform a sentence from one language to another, you lower the semantic. This is simply because it is easier to split a graph in various nodes than collapsing various nodes representing a pattern in one node. Imagine your delegation problem in english: "I want to delegate all methods but getName to delegatee". It will probably be transformed in a if + equation over the message selector in smalltalk, and tranformed in InvocationHandler + string equation in java.
Each time a tool does some work, semantic weakens, eventually everything end in PE or ELF executable format (or interpreted in some now rare cases).

Ok, so the problem for users is keeping as close as possible of the idea in their head while writing *and* reading code.
Basically, there are 2 ways for the delegation problem: a tool (like a refactoring menu) but it will weaken your semantic and produce much code of weak semantic, or a special language construct.
This is where static and dynamic typing split. In dynamic types systems, checking types at compile time is not an objective so you can produce a generic construction for various things, semantic is not very high (the word "delegation" isn't envolved) but not so low that code size will explode. Since static types soundness is not an objective, they emphasise on code and grammar size. In static type systems, the only way is creating a special construct, (imagine some sort of "language of design patterns"), semantic level and code size are fine but the grammar size explodes and moves very fast under user pressure.

Of course, the truth is that java introduced some generic construction, and no language actually put the "ad hoc construction" knob up to eleven. Compromises were made, but my point was showing that are the forces driving the choices of the various actors.
 
I'm just not prepared to give up an entire programming language for one of these features.

Well, of course. Again, my only point was that language aware IDEs offer help in more ways than just refactoring. Regarding your examples, the Smalltalk Browser is IMO a language aware IDE. Personally, I didn't quite got the hang of the Squeak Smalltalk Refactoring Browser when I last tried it, and would actually love to take up on your hypothetical suggestion of comparing notes with an experienced user. (don't take me wrong, I really liked Smalltalk - no other language that I know of has a syntax close to being so perfectly expressive). Now, EMACS is a different story, and I find hard to believe that one can be more productive coding in it than on a state of the art IDE (modulo experience and talent issues...).

Regarding the actual issue, I quite like statically typed languages - even pedestrian type systems like what we encounter in Java or C#. Right now, for instance, I'm coding in Scala, and loving it. My hope is that static metaprogramming can bring the expressibility of a statically typed language closer to that of, say, Ruby. I say "hope" because I don't have any real experience with macro systems.
 
- simple syntax + tools (java) where the syntax is kept simple so that tools vendors have less pain at building their tools

Thank you for giving me a laugh. Sure, I had a couple of beers at the bar before reading this, but clearly not as many as you did.
 
I think your delegation point is not quiet right, because even if Proxy on concrete classes would have been possible, it wouldn't have reduced the signal-to-noise ratio.

If you're saying that using InvocationHandlers and DynamicProxies don't reduce the signal-to-noise ratio, I have to agree.

They impose a lot of overhead that only pays off because there's no other way to do certain kind sof things in Java.
 
OFF-TOPIC

EMACS is to IDE
as
DRILL is to HOLE

I hate EMACS mostly because I am too dumb to use it. As a product manager and hobbyist programmer that is an immediate killer for me. It doesn't help that I met RMS once and he completely turned me off to everything opensource including gcc and EMACS.
 




<< Home
Reg Braithwaite


Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

Books
What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

Share
rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

Beauty
IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

Work
The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Management
Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

Notation
The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Opinion
Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

Whimsey
The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

History
06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /