The Next Lisp:
Back to the Future
An
Invited Talk
©
Dr Mark Tarver, 2009
OK; from the
title we're obviously going to talk about Lisp.
Actually it’s a bit tricky because I’m
also going to talk about open source and open
education later too. It might seem even trickier
if I start off by debating exactly what we mean
by ‘Lisp’. But I'm going to do just
that because it will shed a light on what I'm
going to say. And I'm going to start off with a
biological metaphor, and there will be a few of
them in this talk. I hope any biologists out
there will forgive me.
In biology there is a distinction between
genotype and phenotype. The genotype of a person
is the genetic legacy that the person carries.
The phenotype is the way that genetic legacy
expresses itself. So, for example, you might find
that you carry a gene from your father for blue
eyes and one from your mother for brown eyes.
Your genotype is blue-brown, but your phenotype
is the way your eyes appear which is brown.
Applied to languages the genotype of the language
is the DNA of the language. It is the essential
ideas of the language. The phenotype is the
actualization of those ideas on a platform.
If we ask ourselves, ‘What is the genotype
of Lisp?’, what is its DNA, its genetic
legacy, I'd suggest that it is composed of the
following five ideas.
1.
Recursion as the primary mean of expressing
procedure calling.
2. Heterogenous lists as the way of composing
and dissecting complex data.
3. Programming by defining functions.
4. Garbage collection.
5. Programs as data.
So having
defined Lisp as a genotype, we can now ask what
we mean by ‘Lisp’. Are we talking about
a genotype - a particular set of ideas, or a
phenotype, the way those ideas are represented on
the computer? If we're talking about a genotype
then Lisp is doing very well. Python, Ruby,
NewLisp, Clojure and even TCL/tk are all
inheritors of this genetic legacy. But I guess if
I stood here and talked about Python for an hour
nobody would be too pleased. So I have to talk
about phenotypes, and the main one is Common Lisp
and that is not doing so well.
What Went
Wrong with Common Lisp?
To understand Common Lisp and what has gone wrong
with it, we have to begin with Lisp 1.5. Lisp 1.5
was the brainchild of John McCarthy and his group
and it was the first practical Lisp to
incorporate all the ideas of the Lisp genotype.
Lisp 1.5 was maybe 30 years ahead of the game in
terms of programming language design. A lot of
very bright guys ran with the idea and it mutated
into a polyphony of Lisp dialects - Zeta Lisp,
DEC-10 Lisp, InterLisp-D and so on. So Common
Lisp was devised as an acceptable political
solution for unifying the many tongues of Lisp.
But now, 25 years on, it is not the dominant
phenotype of the Lisp genotype. Why?
The answer is that quite a few mistakes were made
with Common Lisp. The first was that it was too
big. The drive was to make the language
compatible with the previous dialects and the
easiest way was to union these features together.
This is generally what happens when you get a
committee of guys each with their own input. The
language specification explodes.
But for all the size of the CL spec, there are
things that are missing from it. The foreign
language interface is not defined. Calling the OS
is possible in CL but every platform does it
differently. Threads and concurrency are nowhere
in sight (remember this is 1984).
The Common Lisp community has tried to deal with
this through libraries. But these libraries are
unsupported, and either undocumented or poorly
so. I regard this as partly a weakness of the
open source model, but of this later. But the
result is that CL has become unpopular with
developers who want a good solution which is free
and well supported. For that they turn to Python.
On the academic front CL is not popular as an
introduction to functional programming and is the
victim of early standardization. To a degree, CL
is a snapshot of functional programming in the
’70s. Pattern-matching and static typing are
not there. There is much in CL that is heavily
procedural, too much so for teachers trying to
convey the art of functional programming. The
syntax is viewed as unattractive, Larry
Wall’s comment that ‘Lisp has the
visual appeal of oatmeal with finger nail
clippings mixed in.’ is funny and
irreverent, but largely true. It’s a great
language but it doesn’t look great.
I guess a review of past mistakes would be
incomplete without a look at Lisp machines. In
‘The Bipolar Lisp Programmer’ I talked
about brilliant failures and the Lisp machines
were exactly that. The basic idea was simple:
make Lisp run fast by making it run on special
hardware. The problem was that in the mid-80s
standard hardware was doubling in performance
every 18 months. So a Lisp machine that promised
at the design stage to be 10x faster than
conventional hardware might be 5x faster by the
time it appeared and that advantage would last
only 2 years or so.
Symbolics woke up far too late that the real
strength of the company lay in its ideas - in
Genera. In our biological terms, they had got
stuck on obsessing over the phenotype, and missed
the significance of the genotype. So the DNA of
Genera got trapped on a dying phenotype and today
all that work is a lost investment. Imagine what
the Lisp community would be like today if
Symbolics had done what Torvalds did later and
moved their ideas onto a PC!
When the CL community is exposed to criticism of
CL their attitude is confrontational and
conservative. For example, nearly every other
newbie who hits CL for the first time complains
about the brackets. And when they do they are
told to shut up because they don't understand
Lisp. Actually that response shows that the
shushers don’t understand Lisp, because the
Lisp genotype is not hung on the idea of
brackets. Guido van Rossum realised that in
creating Python. Python is not new; its just a
sugaring on the Lisp genotype in favour of a
syntax that people feel comfortable with. And
look how far that idea went.
Along with all these ills went another - macro
laziness. CL programmers think that
macros are absolutely central whereas they are
not. In Qi, the defmacro construction could be
eliminated from the source without remainder by
using the programmable reader. I've left them in
because of laziness. But macro laziness arises
when a challenge to CL is met by ‘I could
always write a macro to do that’. It's the
same response of the slob on the sofa watching TV
and saying ‘I can always get into
shape’. Actually being in shape and being
able to get in shape are not the same. And since
CL has stayed on the same sofa for 25 years,
getting this fat specification from the supine
position and into shape is not easy.
|