We don't need no stinkin' Scientific Method

Update I:

This post has been removed.

Originally, it was a critique of Does Functional Programming Really Matter?. However, since I wrote the original post, the authors have added a disclaimer—this is not a real attempt at a scientific paper—it's just a class project. Speaking for myself, I didn't believe the conclusions of this paper even when I wrote it, and doubly so now—to their document.

Under the circumstances, there is nothing left to dispute and I have removed my words.

I’ve been asked why I didn’t leave the critique on line so that people can “see what the fuss was all about.” Here’s the thing: now that the document has that disclaimer front and centre, the most interesting things to talk about have to do with Universities and grades and intellectual rigor and a bunch of other things that are not about programming.

Since this is a programming blog... this was an easy decision. If the original document was not written in earnest, it’s no longer of interest to me.

Update II:

All right, all right, I give. Here’s what I originally wrote, more or less. Remember, when I wrote this there was no disclaimer on the original article, so I was under the impression that the authors actually believed what they were saying (imagine that), and that the authors were writing a paper to academic standards.

It turns out that’s not the case, so now I’m the one making disclaimers. The following represented my view given the information available at the time. Now that I have new information, my belief is that the paper is a NOOP: It says, in effect, “the following is not true: X, Y, Z.” Such a statement is neither supportable nor refutable.

Anyhoo, here’s what I still have of the original post:

---

We don’t need no stinkin’ Scientific Method

Here’s a paper worth reading: “Does Functional Programming Really Matter? Two Approaches to Comparing Functional and Object-Oriented Programming.”

The authors show how the important ideas in Why Functional Programming Matters can be reproduced in Java programs.

Let me summarize the paper:

Java is Turing Complete. Therefore, all of the ideas in FP can be reproduced in Java. We show this by Greenspunning the important ideas in WhyFP. In essence, we show how to write a poorly specified, buggy implementation, of half of Haskell in Java.

We discover that Java is a cumbersome language for implementing functional programming. Instead of stopping right there, we show our bias by saying that writing programs in an object-oriented language is superior to a purely functional approach, but excuse our problems by saying that Java is at fault, not the premise of embedding FP in OOP.

We do not give examples of embedding FP in other OOP languages that might have radically different models, such as Ruby, SmallTalk, Self, Dylan, Ocaml, or CLOS, leaving the reader to perform their own experimentation. But we claim that a mixed approach is obviously superior to a pure FP approach.

Despite the fact that the document is formulated in a pseudo-academic style, I personally conclude this is junk science. I don’t argue with the authors’ conclusions: I enjoy writing mixed programs in Ruby, and on this very blog I showed how to emulate lazy evaluation using objects.

I do think it’s valuable to show techniques like this. I think it reaches out to the Blub community and shows them that you can expand your horizons without having to throw away everything you’ve learned about your current programming language.

If it gets even one person to read WhyFP for themselves, it’s a net win to share these thoughts with the world.

But conducting research with a bias, performing an experiment that disproves your conjecture, then hand waving the experiment’s results away is not science. It’s just an opinion, nothing more, no different than the stuff I write here in raganwald.

The value of the authors’ academic experience ought to be to teach them how to do basic scientific research.

Where is the study conducted on carefully balanced groups, some who complete a task in pure idiomatic Java, some who complete the same task in Haskell, and some who mix techniques?

Where are the tables showing comparative program lengths, running times, average number of bugs needing correction, or other hard metrics comparing programs to each other?

This “paper” should not be confused with science.  ¶ 8:54 AM