--- a/content/blog/2018/08/a-road-to-common-lisp.markdown Wed Aug 15 16:24:26 2018 +0000
+++ b/content/blog/2018/08/a-road-to-common-lisp.markdown Sat Aug 18 21:14:55 2018 +0000
@@ -1,6 +1,6 @@
+++
title = "A Road to Common Lisp"
-snip = "One way to learn this old language."
+snip = "How and why you can and should learn this old language."
date = 2018-08-15T16:00:00Z
draft = false
@@ -11,6 +11,11 @@
through email and social media posts in the hopes that someone might find it
useful.
+One disclaimer up front: this is *a* road to Common Lisp, not *the* road to
+Common Lisp. It's what I followed (without some of the dead ends) and has
+a *lot* of my personal opinions baked in, but it is by no means the only way to
+learn the language.
+
<div id="toc"></div>
## Context
@@ -131,9 +136,9 @@
a small(ish) number of stable libraries, and library writers often try to
minimize dependencies by utilizing as much of the core language as possible.
I personally try to stick to fewer than ten or so dependencies for my
-applications and no more than two or three for my libraries (preferably
-zero, if possible), but I'm probably a bit more conservative here than most
-folks. I *really* don't like the Hamster Wheel.
+applications and no more than two or three for my libraries (preferably zero, if
+possible), but I'm probably a bit more conservative than most folks. I *really*
+don't like the Hamster Wheel.
It's also worth noting that since Common Lisp has been around and stable for so
long, it has *libraries* older and more stable than many programming languages.
@@ -209,21 +214,21 @@
In C you have a layer of macros on top, written in a preprocessor macro
language. The macro layer and the language layer are separate from each other,
-with the macro layer providing one one extra level of abstractive power (which
-is certainly useful).
+with the macro layer providing one one extra level of abstractive power (which,
+don't get me wrong, is *certainly* useful).
In Common Lisp, you write macros *in Common Lisp itself*. You can then use
those macros to write functions, and use those functions to write more macros.
Instead of two stratified layers it's a *feedback loop* of abstractive power.
-But macros aren't the only thing about Common Lisp that make it so practical
-and extensible. Something people often don't realize is that while Common Lisp
-is an extremely high-level language thanks to macros, it also has plenty of
+But macros aren't the only thing about Common Lisp that make it so practical and
+extensible. Something people often don't realize is that while Common Lisp is
+an extremely high-level language thanks to macros, it also has plenty of
low-level facilities as part of the language. It's never going to be as
-low-level as something like C or Rust, but you might be surprised at some of
-the things that the ANSI spec includes.
+low-level as something like C, Rust, or Forth, but you might be surprised at
+some of the things that the ANSI spec includes.
-Want to see the assembly a particular function compiles down to?
+Want to see the assembly code a particular function compiles down to?
[`DISASSEMBLE`][dis] it!
Want to stack-allocate something to avoid some garbage collection? X3J13
@@ -307,11 +312,6 @@
better than others. I'll recommend the ones I personally think are the best,
but don't hesitate to browse around and find others.
-One final disclaimer: this is *a* road to Common Lisp, not *the* road to Common
-Lisp. It's what I followed (without some of the dead ends) and has a lot of my
-personal opinions baked in, but is by no means the only way to learn the
-language. Ask around and get some more opinions — more options won't hurt!
-
### Get a Lisp
To get started with Common Lisp you'll need to install a Common Lisp
@@ -319,9 +319,9 @@
implementations of it, which gives you choices. There are a bunch of options,
but I'll make it simple for you:
-* If you're comfortable with the command line, installing packages with
- a package manager, and already have a text editor you like, choose [SBCL][].
-* Otherwise, choose [ClozureCL][CCL] (often called "CCL").
+* If you're using MacOS and want a single GUI app you can download from the App
+ Store, choose [ClozureCL][CCL] (often abbreviated "CCL").
+* Otherwise, choose [SBCL][].
That's Clozure with a Z. Clojure is something entirely different that just
happens to have a confusingly similar name.
@@ -382,8 +382,9 @@
NIL
*
-Or this if you chose CCL (the program might be annoyingly named `ccl64` if
-you're on a 64-bit system):
+Or if you chose CCL but still want to use the command line, rather than the
+MacOS app (the command line program might be annoyingly named `ccl64` if you're
+on a 64-bit system):
$ ccl64
Clozure Common Lisp Version ...
@@ -414,10 +415,9 @@
which fixes some minor errors in the 1990 version. If you can afford it I'd
recommend buying the 2013 edition, but the 1990 version will also do fine.
-Go through the book and *do all the exercises*. This will take a while.
-
-This is mainly meant to get you started overcoming some of the main obstacles to
-being comfortable in Common Lisp, like:
+Go through the book and *do all the exercises*. This will take a while, and
+is mainly meant to get you started overcoming some of the main obstacles to
+being comfortable in Common Lisp, such as:
* How am I ever going to remember all these weird function names?
* Why do people use strings so rarely?
@@ -431,10 +431,10 @@
you're too terse. Creating hours of newbie misery and confusion to save a few
flicks of an expert's scroll wheel is a poor tradeoff to make.
-You should also join the `#clschool` channel on Freenode so you can ask
-questions if you get stuck. For the most part people there are friendly and
-helpful, though I'll warn you in advance that there's at least one person who
-can sometimes be abrasive. There's also a `#clnoobs` channel, but that was
+You should also join the `#clschool` channel on the Freenode IRC network so you
+can ask questions if you get stuck. For the most part people there are friendly
+and helpful, though I'll warn you in advance that there's at least one person
+who can sometimes be abrasive. There's also a `#clnoobs` channel, but that was
mostly abandoned during the latest wave of Freenode spam because no one had ops
to help combat the spam.
@@ -459,19 +459,22 @@
Unfortunately the book doesn't include exercises. If you *really* want to get
the most out of it you can type in all the code as you're reading it and poke at
it, but if you've already done the exercises in the previous book it's probably
-safe to just sit down and read the book carefully.
+safe to just sit down and read the book carefully. Don't read more than
+a chapter or two a day. It will take a while for your brain to digest all the
+information.
Make sure you understand everything as you go through the book. Don't be afraid
to ask questions on IRC or Discord (or email me if you want, I don't mind) if
something's not clear.
You should also begin to get comfortable looking up things in [the Common Lisp
-language specification][clhs] itself. It's the ultimate manual for Common
-Lisp. It can be pretty dense at points, but can answer many questions you
-might have if you read it slowly and carefully. You can either use the index
-page to find what you're looking for or just search on Google for "clhs
-whatever". CLHS stands for "Common Lisp HyperSpec", which is the hyperlinked,
-HTML version of the spec.
+language specification][clhs] itself. It's the ultimate manual for Common Lisp.
+It can be pretty dense at points, but can answer many questions you might have
+if you read it slowly and carefully. You can either use the index page to find
+what you're looking for or just search on Google for "clhs whatever" (CLHS
+stands for "Common Lisp HyperSpec", which is the hyperlinked, HTML version of
+the spec). If you already use the Dash app for MacOS, it has the Common Lisp
+spec available.
(Some people will tell you to learn the language by just reading the spec. That's
ridiculous — it's like trying to learn French by reading a dictionary. It's
@@ -520,7 +523,7 @@
3. Edit some code with your editor.
4. Tell the running process to compile *only the code you edited*.
5. Interact with the changed code in the process via the REPL, an HTTP request, etc.
-6. Observe the output (either in the console, a browser, etc).
+6. Observe the output (in the console, a browser, etc).
7. Go to 3.
When you embrace the Lisp way of working you'll rarely recompile and reload an
@@ -567,10 +570,10 @@
When you work in this style with Common Lisp I think you'll really grow to love
it. Writing in other languages will begin to feel like shipping your code off
-to the DMV and getting it back a week with a page full of red ink somewhere in
-the hundred forms you filled out. Writing in Common Lisp feels like interacting
-with a living, breathing organism, or like [teaching things to an eager
-assistant][eager].
+to the DMV and getting it back a week later with a page full of red ink
+somewhere in the hundred forms you filled out. Writing in Common Lisp feels
+like interacting with a living, breathing organism, or like [teaching things to
+an eager assistant][eager].
This philosophy of Lisp being not just a programming *language* but a living,
breathing programming [*system*][gabriel-system] goes beyond just the short
@@ -611,8 +614,8 @@
Once you figure out the problem ("Oh, I see, the `calculate-armor-percentage`
function returns `0` if a shielding spell ran out during the same frame") you
can fix the code, recompile the problematic function, and *restart the execution
-of that function in the call stack*! Your warrior's sword lands, and you move
-back to what you were doing before.
+of that function (or any other one!) in the call stack*! Your warrior's sword
+lands, and you move back to what you were doing before.
You don't have to track down the bug from just a stack trace, like a detective
trying to piece together what happened by the [blood stains][] on
@@ -622,13 +625,14 @@
[blood stains]: https://www.usenix.org/system/files/1311_05-08_mickens.pdf
-Maybe you don't make video games. But this process can be useful in all kinds
-of contexts. Maybe you're writing a web app that talks to an API somewhere, and
-are debugging a request that fails somewhere between two calls to the API, e.g.
-between "create widget `foo`" and "add `foo` to widget list `bar`". Instead of
-just aborting the request, logging a stack trace, and now leaving things in
-a possibly weird state (`foo` having been created without being in the expected
-`bar` list), you can fix the problem and allow the request to finish properly.
+Maybe you don't make video games, sure, but this process can be useful in all
+kinds of contexts. Maybe you're writing a web app that talks to an API
+somewhere, and are debugging a request that fails between two calls to the API,
+e.g. between "create widget `foo`" and "add `foo` to widget list `bar`".
+Instead of just aborting the request, logging a stack trace, and now leaving
+things in a possibly weird state (`foo` having been created without being in the
+expected `bar` list), you can fix the problem and allow the request to finish
+properly.
Of course this won't always work. If you've got a big function that does some
side effects and then crashes, restarting execution of the function would make
@@ -641,7 +645,12 @@
fancy editor plugins — it's baked into the bones of the language. For example:
the standard specifies a method named
[`update-instance-for-redefined-class`](http://clhs.lisp.se/Body/f_upda_1.htm)
-that lets you customize what happens to objects when their class is redefined! This isn't something you'll use all the time, but something like Sketch (a Common Lisp equivalent of Java's Processing library) [uses it][sketch-uifrc] to make working with
+that lets you customize what happens to objects when their class is redefined!
+This isn't something you'll use all the time, but something like Sketch (a
+Common Lisp equivalent of Java's Processing library) [uses it][sketch-uifrc] to
+automatically update the running sketch when you redefine its class.
+Dynamically updating running code in a safe, consistent way doesn't require any
+dark magic in Common Lisp because it's the expected, usual way to work.
[sketch-uifrc]: https://github.com/vydd/sketch/blob/19fe20502d26fc75752e710dfe5106ed13345c5a/src/sketch.lisp#L118-L121
[eager]: https://www.reddit.com/r/lisp/comments/4oo1cp/common_lisp_for_clojure_programmer/d4eec68/
@@ -678,13 +687,15 @@
language server, I think it would be a hugely useful contribution to the
community. Having an LSP server would mean you could get a much nicer
programming experience in many editors out of the box, which would help new
-people quite a lot. I think you could piggyback on top of Swank to do a lot of
-the language-side stuff, and it would mostly be a matter of implementing the
-LSP interface. If this sounds interesting to you, please let me know — I'd be
-willing to help. I've done a bit of work at my day job making a Scala LSP
-language server that uses IntelliJ as a backend, so I have at least some idea
-of how that sausage gets made. I just don't have the time or motivation to do
-an entire LSP server for Common Lisp all by myself.
+people quite a lot.
+
+I think you could piggyback on top of Swank to do a lot of the language-side
+stuff, and it would mostly be a matter of implementing the LSP interface. If
+this sounds interesting to you, please let me know — I'd be willing to help.
+I've done some work at my day job making a Scala LSP language server that uses
+IntelliJ as a backend, so I have at least some idea of how that sausage gets
+made. I just don't have the time or motivation to do an entire LSP server for
+Common Lisp all by myself.
[LSP]: https://langserver.org/
@@ -703,10 +714,9 @@
[PAIP]: https://github.com/norvig/paip-lisp
This book was written in 1992 so it's not about the hyped up AI fields you've
-been hearing about in the news like machine learning or deep learning — instead
-it's a tour of [Good Old-Fashioned AI][gofai]. Even if you're not particularly
-interested in this kind of AI, the book is a great example of how to write
-Common Lisp code.
+been hearing about in the news like machine learning — instead it's a tour of
+[Good Old-Fashioned AI][gofai]. Even if you're not particularly interested in
+this kind of AI, the book is a great example of how to write Common Lisp code.
One thing I really love about this book is that almost all the functions in it
have docstrings. If you look at most other programming books they omit the
@@ -731,7 +741,7 @@
[gofai]: https://en.wikipedia.org/wiki/Symbolic_artificial_intelligence
-### Switch Implementations
+### Switch Things Up
Now that you're comfortable in Common Lisp and your programming environment,
it's time to push yourself out of your comfort zone again. At the beginning I
@@ -740,13 +750,18 @@
in it.
This may seem a bit like running in place, but making sure your code runs in
-more than one implementation will keep you honest and force you to write
+more than one implementation will keep you honest. I will force you to write
portable code that doesn't rely on anything implementation-specific that might
change in the next decade or two. And you might even discover that you like
this other implementation better than the original — maybe CCL's super-fast
compile times make you smile, or SBCL's strong type inference catches more of
your bugs.
+Go through all the code you've written so far and make sure it all runs in the
+new implementation. You might also want to take this opportunity to refactor or
+rewrite some of it — you've learned a lot since you first started, so your
+earliest Common Lisp code will probably look pretty rough to you now.
+
### Recipes for Success
The final technical book I'll recommend to every aspiring Lisp programmer is
@@ -768,7 +783,8 @@
If you've gotten this far you're pretty invested in Common Lisp, and I want to
recommend one not-strictly-technical book that I think you'll really enjoy:
Patterns of Software by Richard Gabriel. It's available as a PDF on [the
-author's site][], and you can still find used print copies online if you prefer.
+author's site][pos], and you can still find used print copies online if you
+prefer.
This is *not* the "Gang of Four"/"Design Patterns" book that you might have
already read or heard about, but is a set of essays on a variety of
@@ -888,7 +904,7 @@
If you're running Linux and like tinkering with your desktop environment,
[StumpWM][] is an X window manager written in Common Lisp. I've just recently
-switched back to Linux so I've only been using it for a month or so, but it's
+switched back to Linux so I've only been using it for about two months, but it's
really pleasant to be able to customize my working environment with Common Lisp.
StumpWM has a small but friendly community — if you're looking for a non-trivial
@@ -919,20 +935,41 @@
### More Implementations
-todo
+I had you use SBCL and CCL because those are the most popular free Common Lisp
+implementations today, but they aren't the only actively-developed ones out
+there. There's plenty of others you might want to explore:
+
+* [ABCL][TODO] runs on the JVM.
+* [ECL][TODO] can be embedded in a C program.
+* [CLASP][TODO] is still under development, but is an implementation designed to
+ be easy to interface with C++.
+* [Lispworks][TODO] and [AllegroCL][TODO] are commercial implementations with
+ a lot of extra features.
+
+TODO more?
+
+(I omitted CLISP because I'm sore about them choosing a name that confuses the
+heck out of new people. Hey, I warned you this post would contain Opinions™.)
+
+I personally tend to use SBCL for my personal projects, but I also make sure the
+units tests for all my libraries run in CCL, ABCL, and ECL. This keeps me
+honest and gives me a reasonable degree of confidence that I'm writing portable
+code.
## Modern Common Lisp
Common Lisp is old and stable, but that doesn't mean it's stagnant. The
language gives you plenty of power to build on, and before I wrap this up I want
-go over a couple of things that often trip up new people.
+go over a couple of recent developments in the Common Lisp world that the older
+books you've been learning from don't talk about. I also want to clarify some
+things that often trip up new people.
### Structure
Common Lisp's terminology for various parts of projects is often confusing to
new people because it's old and uses a lot of words that we use now (like
-“package”) to mean different things than people mean today. Things get easier
-once you internalize what Common Lisp means by the terms.
+“package”) to mean subtly different things than people mean today. Things get
+easier once you internalize what Common Lisp means by the terms.
(Side note: I posted a quick-and-dirty version of this section as a [comment][]
on Lobste.rs while I was waiting for a plane — this section of the post is an
@@ -963,9 +1000,9 @@
In many languages like Python, Java, or Clojure, a file's package and its
location on the hard drive are tied together. For example: when you say `import
-foo.bar.baz` in Python, Python will look for a `baz.py` inside the `foo/bar/`
-folder (it's a little more complicated than this, but that doesn't matter for
-this example).
+foo.bar.baz` in Python, Python will look for a `baz.py` file inside the
+`foo/bar/` directory (it's a little more complicated than this, but that doesn't
+matter for this example).
In Common Lisp, this is not the case. **Files and packages are completely
unrelated in Common Lisp.** You can have many files that all work in the same
@@ -976,9 +1013,9 @@
procedural art library [Flax][] most of the packages are each used in one
specific file, much like you would do in modern languages. But the
`flax.drawing` package contains not only a drawing protocol but also several
-implementations of it (PNG, SVG, etc), and so I split the code into [a series
-of separate files][flax-drawing], each one dealing with how to draw a single
-format (plus one for the protocol itself).
+implementations of that protocol (PNG, SVG, etc), and so I split the code into
+[a series of separate files][flax-drawing], each one dealing with how to draw
+a single format (plus one for the protocol itself).
I could have created separate packages for each implementation and set up the
imports/exports between them, but I didn't feel like the extra boilerplate was
@@ -1063,8 +1100,8 @@
load the files in the correct order so that each package is defined before it
is ever used.
-So to recap: a system is a collection of code and a description of how to load
-it, a list of its dependencies, and some metadata. Now let's move up one level
+To review: a system is a collection of code and a description of how to load it,
+a list of its dependencies, and some metadata. Now let's move up one level
higher to the final layer of structure you need to know about.
[cl-digraph]: https://github.com/sjl/cl-digraph
@@ -1084,14 +1121,14 @@
[Bobbin]: https://github.com/sjl/bobbin
-The `bobbin` system contains the actual data structure and API. It has no
-dependencies.
+* The `bobbin` system contains the actual data structure and API. It has no
+ dependencies.
-The `bobbin/test` system contains the unit tests. It depends on the
-`bobbin` system (because that's the code it's going to test) and the `1am`
-system (a unit test framework). I made this a separate system because it allows
-users to load the main code without also having to load the unit testing
-framework if they're not going to be running the tests.
+* The `bobbin/test` system contains the unit tests. It depends on the `bobbin`
+ system (because that's the code it's going to test) and the `1am` system (a
+ unit test framework). I made this a separate system because it allows users
+ to load the main code without also having to load the unit testing framework
+ if they're not going to be running the tests.
[Graphviz]: https://www.graphviz.org/
@@ -1114,10 +1151,11 @@
that people expect in the modern world. So when you say `(ql:quickload
:bobbin)` you’re asking Quicklisp to download Bobbin (and any dependencies) if
necessary, and then hand it off to ASDF to actually load the code of the
-`bobbin` system. Unlike ASDF, Quicklisp is relatively new in the Common Lisp
-world (it's only about eight years old) and so is not bundled with any modern
-Lisp implementations that I know of, which is why you need to install it
-separately.
+`bobbin` system.
+
+Unlike ASDF, Quicklisp is relatively new in the Common Lisp world (it's only
+about eight years old) and so is not bundled with any modern Lisp
+implementations that I know of, which is why you need to install it separately.
[Quicklisp]: https://www.quicklisp.org/beta/
@@ -1140,12 +1178,12 @@
### Common Libraries
-Common Lisp doesn't have a *large* of a community as some newer languages, but
+Common Lisp doesn't have as *large* of a community as some newer languages, but
it still has a lot of libraries because it's had a community for a longer time.
-The stability of the core language means that many libraries that were written
-in portable Common Lisp ten or twenty years ago can still run just fine.
+The stability of the core language means that many libraries written in portable
+Common Lisp ten or fifteen years ago can still run just fine today.
-In this section I'll give you a quick overview of some of the more popular
+In this final section I'll give you a quick overview of some of the more popular
libraries you might run into as you learn the language. You don't have to use
all of them, but it's helpful to have some idea of what's available.
@@ -1259,7 +1297,7 @@
using the latest version of the implementations you care about and writing
portable code. For the compiling-into-binaries functionality I'd recommend
using your implementation's built-in support for this, or using UIOP's wrapper
-around that, or using a separate library like Shinmera's [Deploy][].
+around that, or using a separate library like [Deploy][].
Of course your mileage might vary. If you find yourself *really* needing to run
specific versions of specific Common Lisp implementations in rapid succession,