--- a/content/blog/2018/07/a-road-to-common-lisp.markdown Wed Jul 25 22:32:23 2018 +0000
+++ b/content/blog/2018/07/a-road-to-common-lisp.markdown Fri Aug 03 18:17:15 2018 -0700
@@ -1,6 +1,6 @@
+++
title = "A Road to Common Lisp"
-snip = "How I learned Lisp, and you can too."
+snip = "One way to learn this old, deep language."
date = 2018-07-03T16:00:00Z
draft = false
@@ -142,7 +142,7 @@
sometimes libraries can just be *done*, not *abandoned*, so don't dismiss them
out of hand.
-#### Power
+#### Extensibility and Power
Part of Common Lisp's practicality comes from its extensibility. No one has
been clamoring for a new version of the specification that adds features because
@@ -611,16 +611,149 @@
## Where to Go From Here
+If you made it through all the books and activities in the previous section:
+congratulations, you're off to a great start! Now that you've got a decent
+handle on the core language you can explore in many different directions,
+depending on your interests.
+
### Macros
+
+If you want to learn the secrets of macros, you'll probably want to read and
+work through [On Lisp][] and [Let Over Lambda][] (in that order).
+
+I'll say that you should take both books (*especially* the latter) with a large
+grain of salt. A lot of Common Lisp users don't agree with all of the arguments
+and style in these books, but I think they can still provide plenty of value if
+you read them with a critical mind.
+
+[On Lisp]: http://www.paulgraham.com/onlisp.html
+[Let Over Lambda]: https://letoverlambda.com/
+
### Object-Oriented Programming with CLOS
+
+Common Lisp has some very sophisticated support for Object-Oriented Programming
+through CLOS. If you have bad memories of OOP from working in a Java cube farm
+like I did, I'd urge you to give CLOS a fair chance to change your mind.
+
+Start with [Object-Oriented Programming in COMMON LISP: A Programmer's Guide to
+CLOS][Keene] — it's a wonderfully-written, short and to-the-point book that will
+give you a good overview of how CLOS is intended to be used.
+
+If you really want to bend your mind, try [The Art of the Metaobject
+Protocol][amop] (usually abbreviated as AMOP). This book will probably take you
+a couple of tries to get through. Read it until you hit a mental wall, go work
+on other things for a couple of months, and come back and try again. Repeat
+that process as many times as is necessary.
+
+[Keene]: https://www.amazon.com/Object-Oriented-Programming-COMMON-LISP-Programmers/dp/0201175894
+[amop]: https://www.amazon.com/Art-Metaobject-Protocol-Gregor-Kiczales/dp/0262610744
+
### Low-Level Programming
+
+Low-level programming can mean a lot of different things.
+
+If you're interested in writing emulators for old computers, I wrote [a series
+of posts][chip8posts] on making a [CHIP-8][] emulator in Common Lisp.
+[cl-6502][] is an emulator for the processor used in the NES (and lots of other
+things) and has a really nice [literate programming][cl-6502-book] version
+that's wonderful to read through.
+
+If you want to interface with C libraries, [CFFI][] is what you want to use.
+The manual is worth reading, though I wish it were a little gentler to start
+with.
+
+[chip8posts]: http://stevelosh.com/blog/2016/12/chip8-cpu/
+[CHIP-8]: https://en.wikipedia.org/wiki/CHIP-8
+[cl-6502]: https://github.com/kingcons/cl-6502
+[cl-6502-book]: http://redlinernotes.com/docs/cl-6502.pdf
+[CFFI]: https://common-lisp.net/project/cffi/
+
### Web Development
+
+Unfortunately I don't have too many suggestions for web development in Common
+Lisp. I've made a conscious effort to avoid web development in the past five or
+so years, because it seems like the Hamster Wheel of Backwards Incompatibility
+has become more of a Hamster Centrifuge in that field.
+
+There is a `#lispweb` channel on Freenode and a `#webdev` channel in the Lisp
+Discord, so if you have questions you could start by asking there. Those
+channels are a bit less populated than the other Lisp channels, so don't expect
+an answer immediately.
+
### Game Development
+
+Common Lisp has a small but enthusiastic community of people who like making
+games. There's a `#lispgames` channel on Freenode and a `#gamedev` channel on
+the Lisp Discord that you should join if you're interested.
+
+[Land of Lisp](http://landoflisp.com/) is a fun little book to go through. The
+coding style in the book has some... "eccentricities", which is why I don't
+recommend it as a first book on Lisp (e.g. using `ash` instead of `truncate` or
+`floor` for integer division), but if you know the language and just want to get
+started making some simple games I think you'll enjoy working through it.
+
+If you want an excuse to make a game in Lisp in a week, the Lisp Game Jam is
+something you can join. It's usually held once or twice each year, so you'll
+have to search around (or ask in `#lispgames`) to find out when the next one is.
+
+Lisp doesn't have any engine as full-featured as Unity, but several people are
+currently working on making 3D game engines. Ask around to see what people are
+using these days. Unfortunately a 3D game engine will generally need to
+interface with the OS to render images and produce audio, and so can't be
+written in pure Common Lisp. This means that some running on the Hamster Wheel
+of Backwards Incompatibility will be necessary to keep up with OS changes (e.g.
+[Apple deprecating OpenGL][eat-shit-apple]).
+
+If you're interested in old-school ASCII/tile-based games, I've personally done
+some work with using [ncurses][] and [bearlibterminal][] in Common Lisp.
+There's something really fun about making a game people can play over telnet!
+Feel free to get in touch with me if you're interested in that kind of stuff and
+want to know more.
+
+[eat-shit-apple]: https://www.macrumors.com/2018/06/05/apple-deprecates-opengl-opencl-gaming/
+[ncurses]: https://github.com/HiTECNOLOGYs/cl-charms
+[bearlibterminal]: http://foo.wyrd.name/en:bearlibterminal
+
### Window Management
+
+If you're running Linux and like tinkering with your desktop environment,
+[StumpWM][] is a X window manager written in Common Lisp. I've just recently
+switched to Linux so I've only been using it for a month or so, but it's really
+pleasant to be able to customize my working environment with Common Lisp.
+
+StumpWM has a small but friendly community — if you're looking for a non-trivial
+open source Common Lisp project to contribute to, StumpWM would be a great
+choice.
+
+[StumpWM]: https://stumpwm.github.io/
+
### Unit Testing
+If you're coming from a modern language, especially one with a lot of
+"test-driven development" advocates, you might be surprised at the lack of an
+emphasis on unit testing in Common Lisp. I think one reason for this is that in
+some languages a unit test is the simplest way to actually *run* a function,
+but Lisp's interactive style of development gives you an even easier
+alternative: just run the function in the REPL!
+
+Despite the lack of heavy unit testing in the community, there are almost as
+many unit testing *frameworks* as there are Common Lisp programmers! This is
+probably because making a unit testing framework is so easy with a few macros.
+I personally love [1am][], but there are *plenty* more to choose from.
+
+Whichever one you choose, please make sure to be a good citizen and create
+a separate ASDF system for your unit tests, so people can use your library
+without having to load Yet Another Testing Framework.
+
+[1am]: https://github.com/lmj/1am
+
## Modern Common Lisp
+Common Lisp is old and stable, but that doesn't mean it's stagnant. The
+language gives you plenty of power to build on, and before I wrap this up I want
+to mention a few things that have become "defacto standards" (or at least
+"pretty commonly seen").
+
### Structure and Building
#### Packages
@@ -628,24 +761,177 @@
#### Projects
#### Naming Conventions
-### Common/Important Libraries
+### Common Libraries
+
+Common Lisp doesn't have a *large* of a community as some newer languages, but
+it still has a lot of libraries because it's had a community for a longer time.
+The stability of the core language means that libraries that were written in
+portable Common Lisp ten or twenty years ago can still run just fine.
+
+In this section I'll give you a quick overview of some of the more popular
+libraries you might encounter as you learn the language. You don't have to use
+all of them, of course, but it's helpful to have some idea of what's available.
+
+#### Alexandria
+
+[Alexandria][alexandria] is one of the most popular Common Lisp libraries (the
+name is a pun on the [Library of Alexandria][library-of-alex]), and it's
+a collection of all kinds of useful little utility functions like
+`read-file-into-byte-vector` and `map-permutations`.
+
+There are a *lot* of utility libraries for Common Lisp around — one rite of
+passage is building up your own personal utility library over time — but
+Alexandria is the most popular one. Most projects with any dependencies at all
+will eventually end up with Alexandria in the dependency graph somewhere.
+
+#### Bordeaux Threads
+
+[Bordeaux Threads][bt] was mentioned earlier. Threads aren't part of the Common
+Lisp standard, but most implementations provide their own custom interface for
+working with them. Bordeaux Threads wraps all these implementation-specific
+interfaces and provides an API so you can write threaded code that will work
+portably.
+
+If you're looking for something like Java's `new Thread(() -> foo()).start();`,
+this is what you want.
+
+#### CFFI
+
+[CFFI][] is a foreign-function interface library that lets you load
+C libraries (e.g. `foo.dylib` or `foo.so`) and call the functions in them. It
+works by wrapping implementation-specific interfaces, because this isn't part of
+the Common Lisp standard.
+
+Unfortunately it has the same name as Python's FFI library, so if you're
+searching for documentation make sure you're looking at the right version.
+
+#### CL-PPCRE
+
+[CL-PPCRE][ppcre] is an implementation of Perl-compatible regular expressions.
+If you're looking to use regular expressions in Common Lisp, this is what you
+want.
+
+#### Drakma
+
+[Drakma][] is an HTTP client. If you need to make an HTTP request, this is what
+you want. There are other HTTP clients around, but Drakma is commonly used and
+is fine for almost anything you might need.
+
+#### Iterate
+
+[Iterate][iterate] is a replacement for the `loop` macro. It works similarly,
+but has a more Lispy syntax and a well-defined API for extending it with new
+iteration constructs. I personally really like it, but beware: if you get used
+to `iterate` going back to vanilla `loop` will feel painful.
+
+#### local-time
+
+[local-time][] is a library for working with time and dates in Common Lisp. The
+standard has some basic support for times built in, but if you want to do much
+calculation with times (including timezones) this is probably what you want. If
+you're looking for something like [Joda Time][joda-time] in Common Lisp, this is
+as close as you're going to get.
+
+[joda-time]: http://www.joda.org/joda-time/
+
+#### lparallel
+
+[lparallel][] is a library that builds on top of Bordeaux Threads to make common
+parallel processing operations much easier. Think of it as [GNU Parallel][] for
+Lisp, with a few extra features (e.g. channels and tasks).
+
+For example: if you've got a big vector you're mapping over with
+`(map 'vector #'work some-vector)` you can split it into chunks and
+run in multiple threads by changing it to
+`(lparallel:pmap 'vector #'work some-vector)`.
+
+[GNU Parallel]: https://www.gnu.org/software/parallel/
-[Alexandria][alexandria]
-[Drakma][]
-[Bordeaux Threads][bt]
-[Flexi Streams][flexi]
-[Gray Streams][gray]
-[Iterate][iterate]
-[local-time][]
-[lparallel][]
-[named-readtables][]
-[CL-PPCRE][ppcre]
-[Roswell][]
-[SERIES][series]
-Trivial-whatever
-[uiop][]
-[usocket][]
+#### Named Readtables
+
+[Named readtables][named-readtables] is a library that adds namespaces for
+readtables.
+
+One painful part of the standard is that reader macros are added and removed to
+the global readtable on the fly, so if you load multiple systems that define the
+same reader macros things can get messy. Named readtables adds some much-needed
+hygiene to that process. If you're working with reader macros at all you
+absolutely want to use this.
+
+#### Roswell
+
+[Roswell][] is a couple of things rolled into one. It's a C program that
+handles installing and running multiple different Common Lisp implementations
+(kind of like [NVM](https://github.com/creationix/nvm) or
+[rvm](https://rvm.io/)), and it also provides a unified way to write small shell
+scripts in Common Lisp and compile them into binaries.
+
+I used Roswell for a little over a year, but I eventually stopped and now
+I personally don't think it's worth the trouble, for a couple of reasons.
+
+First: if you write portable code you generally don't need to worry running
+a particular version of an implementation, because Common Lisp is quite stable.
+I usually just install the latest version of each implementation I use with
+a package manager or by building from source.
+
+Second: after using it for a while I found that Roswell was always quite brittle
+to upgrade, and whenever things broke it would spew an almost JVM-sized stack
+trace without a decent error message.
+
+For me, the negatives outweighed the positives. I'd personally recommend simply
+using the latest version of the implementations you care about and writing
+portable code. For the compiling-into-binaries functionality I'd recommend
+using your implementation's built-in support for this, or using UIOP's wrapper
+around that, or using a separate library like Shinmera's [Deploy][].
+
+Of course your mileage might vary. If you find yourself *really* needing to run
+specific versions of specific implementations in rapid succession, you should
+look into Roswell.
+
+[Deploy]: https://shinmera.github.io/deploy/
+#### SERIES
+
+[SERIES][series] was almost included in Common Lisp (it's in [Appendix A of
+CLtL2][series-cltl2]), but didn't quite make it. It's a library for writing
+functional code that looks like the traditional `map` and `filter` and `reduce`
+operations but which compiles down to efficient loops.
+
+If you're looking for Clojure's transducers in Common Lisp, this is what you
+want.
+
+[series-cltl2]: https://www.cs.cmu.edu/Groups/AI/html/cltl/clm/node347.html
+
+#### st-json
+
+JSON support in Common Lisp is a god damn mess. There are [an absurd number of
+JSON libraries][sabra] and I don't really *like* any of them.
+
+For me, the most important quality I need in a JSON library is an unambiguous,
+one-to-one mapping of types. For example: some libraries will deserialize JSON
+arrays as Lisp lists, and JSON `true`/`false` as `t`/`nil`. But this means `[]` and
+`false` both deserialize to `nil`, so you can't reliably round trip anything!
+
+I've settled on using [st-json][] and wrapping it up to be a little more
+ergonomic with some glue code. It's not the fastest solution out there, but
+it's fine for my needs. There are plenty of other options out there, so if you
+have different needs than me you should look into them.
+
+[sabra]: https://sites.google.com/site/sabraonthehill/home/json-libraries
+[st-json]: https://marijnhaverbeke.nl/st-json/
+
+#### usocket
+
+[usocket][] is a library for sockets. Sockets and networking aren't part of the
+Common Lisp standard, but most implementations provide a custom interface for
+working with them. usocket wraps the implementation-specific interfaces and
+provides an API so you can write networking code portably.
+
+If you want to make Lisp listen on a port and read streams of bytes from
+clients, or want to connect to a port and send raw bytes to it, this is what you
+want.
+
+[library-of-alex]: https://en.wikipedia.org/wiki/Library_of_Alexandria
[alexandria]: https://common-lisp.net/project/alexandria/
[iterate]: https://common-lisp.net/project/iterate/
[series]: https://www.cliki.net/Series
@@ -662,3 +948,11 @@
[Roswell]: https://github.com/roswell/roswell
+## Good Luck!
+
+I hope this whirlwind tour was useful. Common Lisp is an old, deep language.
+It's not something you can learn in a month, but if you're willing to spend the
+time it will reward careful study.
+
+Feel free to email me or pop into IRC or Discord if you have questions. Good
+luck!