(This is a snapshot of my old weblog. New posts and selected republished essays can be found at raganwald.com.)

Thursday, March 06, 2008
  Tool Time

‘Tool’ is a popular metaphor for a programming language but it’s very limited. ‘Rule’ might be a better metaphor. A language defines, quite literally, what you can and can’t do. People who argue for languages with more power are actually arguing for less rules.
Azubuko Obele

Is this really about “more power?” Or is this about whether a language has one obvious way to do things vs. more than one way to do things? Python is a considerably more powerful language than Java, but I would consider both languages to be firmly in the “It’s a good thing to impose rules on the programmer” camp.

It’s true that languages like Ruby permit programmers more freedom. But is it because such languages give programmers the tools to turn boilerplate into abstractions? Or is it because these languages provide multiple programming paradigms and because they have cultures encouraging programmer responsibility?

Python Programming for the Absolute Beginner is a must read for the programmer who hasn’t dived into Python yet. There’s a reason Google endorses Python. Find out why with this essential book: it introduces the language gently yet takes you as far as you want to go.

Java and Python are good languages to contrast. Java began with the idea that removing programmer freedom would produce better programs. Java was built by removing features from C++, and the result is a language with little power (whatever that means) and little freedom.

Python was also built with the philosophy that removing certain programmer freedoms would produce better programs, however Guido has been careful not to reject features simply because programmers might misuse them. Instead, he has been careful to design Python such that there are few features he deems unnecessary. His definition of unnecessary is a feature where there already exists a perfectly serviceable way to do it in Python.

Guido has also tried to remove choice in things that don’t matter, like indentation, while growing the language with features that fit within Python’s style and design æsthetic. The result is a language where programs are easy to read, where you do not have little dialects everywhere.

Yet, Python programmers rarely chafe that they are constrained from doing what they need to do. Clearly, it is possible to impose rules and to gently guide programmers towards a standard way to express ideas without hobbling them.

Haskell is quite possibly the most extremely limiting language in common use: there is one way to do everything, mathematically pure functional programming. You have strong, expressive static typing. Would anyone argue that switching from Java to Haskell is an attempt to deal with fewer rules and fewer restrictions, an attempt to gain more freedom as a programmer? Likewise, the Factor programming language imposes a single, constraining set of rules on the programmer: programmers switching to Factor must relinquish their local variables to gain Factor’s higher-order programming power.

Power and freedom are related ideas. So it is easy to see why many languages that provide more power also provide more freedom than hobbled languages. The trouble is that some languages are built as collections of unrelated features smooshed together, just as some programs are hotch potches of checkbox features. Good design in every field accomplishes the “antigravity” trick of accomplishing more with less. With poor design, adding new power requires adding more check boxes, more features, more ways to do things that interact with each other unexpectedly. It is not surprising that some reasonable people resist growing languages that are not designed well, that cannot grow in power without throwing more features on top of the pile.

But languages can provide more power while also providing more rules and, paradoxically, less freedom. It’s just that they provide less freedom to do things that are unimportant to the language designer (like fully unconstrained lambdas in Python) or less freedom to do things that make it difficult to reason about a program’s behavior (like dynamic typing or state changes in Haskell). This is accomplished through careful design from the outset, through having a guiding philosophy, and through building on sound foundations.

The issue is not that more power equals more freedom, just that more power (whatever that means) equals a certain kind of freedom denied to those who use less powerful languages. It does not mean more freedom overall, it may mean less freedom overall.

The Binford 6100 Man’s Kitchen is for the chef who is not afraid of power: it features a Binford Macrowave with 3 settings—High, Really High and Split-your-own-atoms. Photo courtesy Binford Tools.

I conclude that it is possible to argue for and against language “power” quite separately from arguing for and against constraining programmers with rules. “Collection of Rules” is a decent metaphor for programming languages, and some languages have more rules than others. But it is orthogonal to the metaphor of languages as tools and the discussion of to what extent providing programmers with more powerful tools leads to better programs.

Comments on “Tool Time:
I suspect python programmers don't chafe largely because those who do chafe switch languages. I found some features of python attractive for awhile but ultimately Guido's (sp?) removal of features drove me crazy and encouraged me to move over to ruby.

Now maybe in theory taking out of features and providing one way to do things is reasonable but the way choosen in python just seems too unfriendly towards a more functional/abstract approach to code. It is still allowed but seems disfavored in the deciscions.
@truepath: What features did he remove that bothered you? I've only seen Python gain features in the last 7 years.

He moved reduce() from __builtins__ to functools? Continued to shoot down tail recursion?
The premise is wrong. A computer language is not akin to rules, but akin to mathematics. Hell, it is mathematics!

So, compare it to algebra. The more theorems you have, the more powerful you are. Are theorems freedom? Not, of course not. Theorems are fundamental truths that can be combined to more easily achieve further results.

So it is with languages. In the absence of more powerful theorems, the solution to your problems will take longer to write. Or, you could first "prove" these theorems and then use them. Which would be like writing an interpreter or a compiler. In fact, that happens more often than you might think, as every DFA is also an interpreter.

So, what's this about freedom and the quality (for some values of quality...) of the programs written in a language?

Well, it's actually quite simple. We, humans, think naturally in some ways and not in others, logic notwithstanding. We are not computers.

Let me give an example here, because most people tend to disregard the above as obvious and irrelevant. The sun rose today, and it rose yesterday, and, in fact, I don't recall a single day of my life in which the sun didn't rise. I'm told this has been going on for a long time, and I'm inclined to believe so. The instinctual reaction of most people to this fact is to believe that it follows that the sun will rise tomorrow. And it is instinct. It's the basis of conditioning, that which makes the scalded cat afraid of cold water.

Yet, as most of the readers here will know, this has absolutely no basis in logic. If you wrote a program depending on that, it would be incorrect. And you can bet your ass it would crash within the first day of production usage. :-)

So, we are good at some stuff, but not at others. The pumping lemma is powerful, but is it easy to use? What's easier, deduction or reductio ad absurdum? Personally, I have always been an absurdum man.

And that's where the "rules" come in. It's not about rules. It's about providing theorems which we are not naturally disposed to misuse instead of those that we are. And, guys, some of those theorems we might misuse are actually more powerful.

Which, IMHO, is the case of deduction. In my own experience, people tend to screw up deduction much more easily than R.A.A., but deduction is certainly the more powerful of the two. Q.O.D. ;-)
#daniel "A computer language is not akin to rules, but akin to mathematics. Hell, it is mathematics!"

Nope, they(most of them) aren't.
"Power" and "Freedom" are marketing terms; they are positive-sounding words describing a trade-off between concerns. Stan Lee teaches us that the flip side of Power is responsibility: Java was much better than C++ not because it gave programmers less power, but because it also required a lot less responsibility from them: They no longer had to free every object they allocated, they no longer had to be careful of accidentally tripping over unintended compiler-generated copy constructors or dangling pointers.

One of my favorite things about Perl (and valid for most other scripting/dynamic/high-level languages, I guess) is that I can avoid a lot of arithmetic, incrementing loop counters and messing with array indexes, because I can work with lists, which are supported by the language with operations like foreach, push, shift, map, etc.

I guess my feeling is that Power and Freedom, while imprecise terms, are not really traded off against each other in all languages.

I'm not stating that they are always orthogonal. For example, I don't know how to add power to Java without adding a lot of freedom in the sense of supporting different dialects of Java.

But when switching or deigning languages, I do think there is some opportunity to choose power with either more or less freedom according to what you feel is most appropriate.
njharman said most languages are not mathematics.

Well, they are. They are all turing machines.
Saying that programming languages are mathematics because mathematics describes their properties is like saying that human beings are physics because physics describes our properties.

Sure, everything is Quantum Electrodynamics. However, QED is not helpful when trying to hire a programmer or woo a mate. Likewise, many important properties of programming languages can be precisely defined by mathematics.

However, many other important properties cannot. For example, we have not arrived at a consensus on how short or long programs ought to be. Too short, and we complain about them being unreadable. Too long, and we complain about them being unreadable.

They may all be Turing-equivalent (they are not Turing machines), however mathematics is silent on subject like this, which are sometimes likened to HCI and Human Factors, Psychology, Design, Writing, and many other fields.
Well, ok, turing-equivalents. My point is that no matter how humans look upon a language, the language itself is turing machine-like.

A computer language is _not_ a natural language. It doesn't convey emotion, it doesn't have subtexts. You won't find expletives in them. :-)

There are a number of ways in which language is used, but computer languages are much more limited that that. A computer language deals with commanding the CPU, and nothing else.

Whether you use // or /* */ is irrelevant. It doesn't matter if a javadoc can extract API-describing remarks. You can completely change these things and the execution of a program won't change a bit.

Whether a program is short of long won't change a bit what the language is doing.

It's like saying birds are more than bird because some of them are more pretty than others. A bird isn't pretty, a bird simply is. The prettyness an attribute of your relationship with the bird.

Same thing with languages. Languages are CPU commands. The relationship might not seem obvious, just like the pumping lemma doesn't seem to follow from basic group theory, but there is a clearly defined path from one to the other.

Humans relationship with computer languages doesn't change the nature of computer languages. A computer language is not a set of rules of what you can do or cannot rule. All computer languages can do the same things, because they are all equivalent. I can add pointers right back to Java by the simple expedient of allocating a huge array and handling all my data in this array.

So I stand by what I said. Computer languages are mathematics. They are a set of theorems and lemmas which you use to write instructions to a virtual turing machine, whether you understand this or not. And if you don't understand this, you are prone to misuse them. :-)

Our relationship with the languages is certainly affected by what the language has available. Some things are easier for humans to deal with than others. Some can greatly reduce the size of a program, by providing theorems to you instead of making you write them. But their nature is all the same.

To deny it, is like to deny that a set of equations used for civil engineering is not mathematics because it's being used in a different context than pure mathematics.

I agree with half of what you say, in that programming languages are equivalent in the Turing-equivalent sense and that programming languages are not equivalent in the ease-of-programming sense.

Now if you think the Turing-equivalent sense is the most important thing about programming languages, please be my guest. That is one of the most important things we have discovered about computer science but, IMO, one of the most irrelevant things we have discovered about software development.

You can jeer all you like, but knowing that I can Greenspun anything I like in Java doesn't help me much. As a matter of fact, I wrote a Scheme interpreter in Java back in 1999, so I am keenly aware that if you can do it in Scheme, you can (in a certain sense) do it in Java.

But does that help me ship software? Not really. Can I walk into a BigCo Java environment, embed my Scheme interpreter, and write code in Scheme that is really Java? No, although it is Java in a certain theoretical sense, it is Scheme in a practical sense that will arouse a certain heated objection from my colleagues that want Java code written in a Java style.

What does this matter? It matters in that although we can make a Java program produce the same result as a Scheme program, we cannot write a Java program in any arbitrary "style" permitted by the compiler. We have cultural limits that place strong bound son how easy or difficult it is to write certain programs in each of the various languages.

And why does that matter if the results are the same? Because software development is not free. In theory, programs always work, as we can produce any result we like.

In practice, programming is a probabalistic exercise. We have imperfect understanding of our requirements, insufficient testing, and never enough time to write the code as well as we are able. Therefore, ease of use of a programming language has a strong influence on the likelihood of a desirable outcome of a software development effort.
Where does language 'expressiveness' fit into this discussion? For example, it could be argued that C++ is a very powerful language, and affords the programmer a great deal of freedom, but it is not very expressive in the sense that you can't do a lot with a small amount of code. I guess languages like Lisp, Python and Ruby are the opposite to this in the sense that you can achieve a lot more with less code. I would say that this made them more expressive. But does it make them more powerful? You couldn't write a device driver in Python!

I'm guess I'm asking, when you talk about language 'power', is this a synonym for expressiveness?
Reginald, I'm not dismissing the differences between languages. I'm disagreeing with the assumption that they are differences between "rules" -- particularly in the sense of "limits".

IMO (being as emphatic as I am, I can't really put the H there... :), they are differences between "theorems" we use to help us achieve our goals.

For instance, in machine code there is no "for" loop as most of us would understand them. What you have are conditional jumps based on CPU flags.

But from that we developed a very useful construct, which is the "for" loop.

Now, even though the "for" loop is a very powerful construct, it is often prone to misuse. Few people can really truly simulate in their minds a "for" loop, so errors in that construct are not uncommon.

Now, a "foreach" (list iterator or mapper) construct might not be as flexible as a "for" construct, but it is easier for us to understand.

But that's not because I'm not disallowing people from walking a linked list themselves, or whatever, but because I'm using a construct human minds deal with more ease.

I'm not preaching that all computer languages are applied mathematics and, therefore, they are all equal. What I'm saying is that it is not rules that make a language powerful or prone to correct programs, but the selection of computer language "theorems". And those theorems may be powerful and easy to use at the same time.

Well... I'll try to leave it off now. Either I'm being misunderstood, or I'm misunderstanding you all.
Java began with the idea that removing programmer freedom would produce better programs. Java was built by removing features from C++, and the result is a language with little power (whatever that means) and little freedom.

Egads! That is so misleading!

Java began with the idea that removing unnecessary complexity would make for a more productive language. And frankly, on that count, it succeeded! Java is a far more productive language to code in than C++ - which is exactly what they were trying to achieve!

By abstracting away implementation details that programmers were being forced to waste time dealing with that were at best a huge chore and at worst a huge source of bugs (like manual memory management, array overflow checking, etc.) Java developers could spend far more time than C++ developers on *what* they were trying code, rather than *how* to code it.

Yes, since then languages like Ruby have taken this further, and have further abstracted away irritating implementation details that Java developers have to deal with.

But phrasing this as "Java began with the idea that removing programmer freedom would produce better programs" is absurdly misleading! You can't honestly believe that that was Jim Gosling's motivation back then, can you?!?!? It's been pretty well recorded by now exactly why Gosling made some of the design decisions he did with Java, and I think it's pretty clear that most of them stem from the desire to remove unnecessary complexity, not to arbitrarily remove freedom.

Yes, I have been following Java closely for a decade, and I have read Gosling and Steele on the subject of its design.


Your statement argues two things which are not mutually exclusive. language designers do not remove freedoms out of a sadistic desire to hobble programmers.

Language designers remove freedoms precisely to optimize the language for the cases that the designer feels involve necessary complexity at the expense of the cases that the designers feel involve unnecessary complexity.

Most languages these days remove the ability to directly manipulate CPU registers. It's a clear case of somebody deciding that register allocation is unnecessary complexity, and frankly I am quite content not using the register keyword any more.

However, that is also the removal of a freedom. The two things go together in many cases.

Or what about Python's enforced indentation? Guido obviously removed the freedom to express your program in the visual style you prefer. And while he was at it, he was forced to remove your freedom to compose arbitrary lambdas.

This latter point is a major constraint in comparison to languages like Lisp, Smalltalk, and Ruby. But of course, Guido's motivation was to remove the accidental complexity faced by programmers trying to read code written in an unfamiliar style.

So the two statements are compatible: Java began with the idea that removing programmer freedom would produce better programs can be true while the statement Java's design choices stem from the desire to remove unnecessary complexity is also true. the two are both true and intrinsically related when the design choices in question were deliberate choices to remove flexibility in order to remove what the designer felt was unnecessary complexity.

So... no need for us to argue.
So the two statements are compatible: "Java began with the idea that removing programmer freedom would produce better programs" can be true while the statement "Java's design choices stem from the desire to remove unnecessary complexity" is also true. the two are both true and intrinsically related
So... no need for us to argue.

Agreed, but the former carries a far more negative connotation, IMO, which is why I felt the need to chime in. I just thought you could have worded it in a much more positive way. This way it kinda just comes across as yet another Java-bashing blog post.

I do not find Java-as-a-language interesting enough to go out of my way to write about it very often. That is neither a positive nor a negative statement: languages intended to run the back ends for large banks are not supposed to be particularly interesting. If they are, it is as a side-effect.

The post in question is about programmer freedoms. I thought that the line about Java was appropriate in context. I hardly see the entire post as "Java bashing," not do I see that line as bashing Java one way or the other.

Quite frankly, I have written an awful lot of Java code, and at this point I don't lose too much sleep over Java's shortcomings for the simple reason that I think it has reached an evolutionary dead end. Or an optimal state for its chosen nice. I do not forsee much evolution from a language with such a large installed base and such a large, diverse user base.

So why complain if it will not change much?

OTOH, Ruby has many faults, but the language is actively evolving. Not only is the language itself changing, but people are spinning new languages off it.

Carping about Ruby is far more likely to produce change than carping about Java, thus you find me devoting entire articles to my opinion about Ruby's shortcomings:

Yurtles all the way down, please, or perhaps The significance of the meta-circular interpreter for example.
Oh dear, my spelling has produced a malapropism: "Yertle the Turtle" comes to mind. Perhaps I have unintentionally Godwinned myself?
Regarding your question: "Would anyone argue that switching from Java to Haskell is an attempt to deal with fewer rules and fewer restrictions, an attempt to gain more freedom as a programmer?"

I would. (Or at least toward Haskell-like programming by way of Scala.)

Type inference, structural typing, and higher-order functions conspire to free me from the redundancy and boilerplate, free me to write expressions and functions that state more directly what I mean, and free me to create functions when and where I need them.

Let me suggest tht "freedom" is a very overloaded term. Simplicity in some aspects of the language (e.g. smaller syntax) provides more freedom in others. There isn't a single numerical "freedom score" that compares one language to another on a simple linear scale.
First of all I think there is a bit of confusion about what way computer languages ARE mathematics. This statement is actually true in two ways.

It is true in the sense of computer languages all being reducable to mathematics in a formal sense but that's the uninteresting sense. It's also true in the sense that the practice of mathematics, particularly the development of notation and syntax to guide problem solving is really the same thing going on during the creation of productive programming languages.

For instance the difference between programming is say python from assembely language lets one do tasks trivially that would have taken weeks previously because the abstractions give you the appropriate options. Similarly the reason the greeks got stumped at even the simplest algebra problems was because they lacked the appropriate abstraction of writing things like x*x and so forth.

Note that I agree that the strength of a particular kind of abstraction lies as much in as what it makes hard to express as in what it makes easy. The reason notation like x*y*x is appropriate in algebra is as much because it makes it inconvenient to distinguish x*(y*x) from (x*y)*x. In fact I would go so far as to say the reason math is able to build on itself is through the adoption of abstractions that make it difficult to do the wrong things and easy to do the right ones (here difficult get confused by associativity easy to apply it).

The question is all about WHICH choices you make and whether the options you take away guide the user in the correct or incorrect direction. More on this in a moment
As a mathematician (well if I finish this thesis rather than posting on the internet) let me assure you it's not true that the more theorems the better. In fact this is a big reason why people outperform automated theorem provers. Rather than merely generate long lists of theorems good mathematicians create an elegant abstraction that makes it easy to build further abstractions above it so rather than continuing to muck about with long lists of theorems they prove several key theorems and then introduce a further layer of abstraction. Programmers do the same when they choose not to merely build huge lists of functions but define APIs, use MVC etc..

Guido's design philosophy misses this critical point. What makes abstractions work isn't that they let you solve 90% of problems at that level but that they support clean meta-abstractions. Guido tends to reject simple elegant building blocks that might make it slightly tougher to write that 90% of code but would offer intellectual coherency and elegance to support further meta-abstraction.

I mean suppose in calculus we adopted the same idea. 90% of the time we want to take the derivative of functions written in terms of +,*,/ and exponentiation (e.g. y=(x*x+x)/x^x ) and we could make the lives of all intro calc students infinitely easier by dropping the epsilon-delta limit definition of the derivative and just defining the derivative of a product, sum etc.. 90% of the cases are covered but what happens when we want to talk about tangent lines to y=sin(x). We wouldn't be unable to deal with these questions, we could define it in terms of limits of derivatives of approximating polynomials and get the same answer but now it becomes impossible to elegantly build the next layer of abstraction. Suddenly every time we want to prove a theorem about the derivative of f(x) we have to break it into cases.

To give a programming example where the more complex initial solution is the better one consider the notation in lisp. Sure, the parenthesises and infix(?) notation make it harder to write 95% of code but the regularity and elegance allow macros to be defined elegantly more than making up for the loss in that 95%.


Still, having said this as a practical matter python may be making the right decisions to be a productive language for most users. It's quite plausible that the average python user is never going to graduate to the next level of abstraction anyway (many declarative programmers never figure out currying or functionals) so it might be more productive for the average user but it's not for me.

Ruby came close to doing it right but it has a far too messy syntax and not enough thought into a unifying data model to be quite right.
One quick concrete example.

Generators, iterators etc.. in python are all special cases thus occupy valuable mental real estate. These are all instances of a more fundamental underlying abstraction in ruby that may be more difficult to use at first but ultimately is more elegant.
Factor has a library called locals that implements local variables.

<< Home
Reg Braithwaite

Recent Writing
Homoiconic Technical Writing / raganwald.posterous.com

What I‘ve Learned From Failure / Kestrels, Quirky Birds, and Hopeless Egocentricity

rewrite_rails / andand / unfold.rb / string_to_proc.rb / dsl_and_let.rb / comprehension.rb / lazy_lists.rb

IS-STRICTLY-EQUIVALENT-TO-A / Spaghetti-Western Coding / Golf is a good program spoiled / Programming conventions as signals / Not all functions should be object methods

The Not So Big Software Design / Writing programs for people to read / Why Why Functional Programming Matters Matters / But Y would I want to do a thing like this?

The single most important thing you must do to improve your programming career / The Naïve Approach to Hiring People / No Disrespect / Take control of your interview / Three tips for getting a job through a recruiter / My favourite interview question

Exception Handling in Software Development / What if powerful languages and idioms only work for small teams? / Bricks / Which theory fits the evidence? / Still failing, still learning / What I’ve learned from failure

The unary ampersand in Ruby / (1..100).inject(&:+) / The challenge of teaching yourself a programming language / The significance of the meta-circular interpreter / Block-Structured Javascript / Haskell, Ruby and Infinity / Closures and Higher-Order Functions

Why Apple is more expensive than Amazon / Why we are the biggest obstacles to our own growth / Is software the documentation of business process mistakes? / We have lost control of the apparatus / What I’ve Learned From Sales I, II, III

The Narcissism of Small Code Differences / Billy Martin’s Technique for Managing his Manager / Three stories about The Tao / Programming Language Stories / Why You Need a Degree to Work For BigCo

06/04 / 07/04 / 08/04 / 09/04 / 10/04 / 11/04 / 12/04 / 01/05 / 02/05 / 03/05 / 04/05 / 06/05 / 07/05 / 08/05 / 09/05 / 10/05 / 11/05 / 01/06 / 02/06 / 03/06 / 04/06 / 05/06 / 06/06 / 07/06 / 08/06 / 09/06 / 10/06 / 11/06 / 12/06 / 01/07 / 02/07 / 03/07 / 04/07 / 05/07 / 06/07 / 07/07 / 08/07 / 09/07 / 10/07 / 11/07 / 12/07 / 01/08 / 02/08 / 03/08 / 04/08 / 05/08 / 06/08 / 07/08 /