Experience of Learning Lisp

[mrove]

I am in the process of learning Lisp, and am interested in the experience other people had while learning it.

What drew me to Lisp - as a programmer of 15 years (10 professionally), I always like to learn new things and expand my horizons. Sadly, as a LotusScript (if you're not familiar with it, think Visual Basic of 1995, and if you're not familiar with that, just think crap) programmer of 10 years, Javascript, Java, and C# have been the extent of the new things that I've had at least a little experience with to expand my horizons. I've recently come across Paul Graham's writings and others, and the idea that learning Lisp would change my thinking (it already has) and the idea of code as data strongly appealed to me.

I'm currently still in the very early stages (Ch 13 of Practical Common Lisp) of learning, but have a lot of thoughts going through my head.

How long did it take you to get started? I've been going at a fairly slow pace (only up to Ch 13 after 2-3 weeks), but Lisp is still almost completely foreign to me. It's like looking at Perl. Although I have limited experience using languages, I read a lot of code in various languages, and it's generally pretty easy to get an idea of what's going on. Not so with Lisp. So many functions have cryptic names that are on par with assembly for understandability. And how many functions there are! I'm getting the impression that Lisp offers the functionality of Java plus a good chunk of Java's libraries all within the language spec. So... that's great, but a troublesome downside for a newcomer is the sheer quantity of functions, many with cryptic names with no organization - everything's just there, like Java keywords. I appreciate the logical hierarchy and organization imposed on methods in an OO design. (Before I get too far, I am aware that Lisp supports objects, but what I'm addressing here is the large quantity of functions in the language spec that would be much more comprehensible and easier to learn if they had some sort of organization.)

I feel like my first act after learning enough Lisp to get started would be to rewrite Lisp. (And WOW is it pleasing that you can actually do that in Lisp!) Create macros to give functions names that are meaningful to English speakers. Learning Lisp is like learning a whole new foreign language - knowing English is not sufficient to deduce the purpose and behavior of many Lisp functions simply from their name. And organization - if a group of functions only operates on lists, then perhaps name them list-add(), list-remove()? For the similar functions that operate on trees, or plists, or whatever, give them similarly clear names.

While I'm at it, I might hide some of the inconsistencies. It seems like Lisp's destructive functions are wantonly destructive! Why no be productively destructive? I was completely confused when I discovered that you still had to use the return value of destructive functions:
(setf *list-3* (delete 4 *list-3*))
If delete is destructive, why?! must I use setf I railed. It was a good bit more reading before I realized that destructive functions destroy with no (apparent) rhyme or reason, and the destroyed parameter is different from the value returned. I would be inclined to replace all these functions with functions that modify the parameter to produce the desired result, rather than requiring the caller to use the function's return value for the desired results and trashing the parameter!

As early as I am in my process of learning Lisp, I am very excited by it, and dumbfounded that we ended up with popular languages we've suffered with for so many decades, while Lisp sat there quietly in the shadows. It didn't take too much experience programming before I was frustrated by the fact that I couldn't tell my code to modify code or write code! I mean seriously, why won't they let me do that?! When reflection finally became popular, that was at least some small relief, but the full power of Lisp's macros is sorely needed!

All these thoughts lead me to wonder... has someone tried to "clean up" Lisp? Is there some reason it's a bad idea? It seems cruel to me that the amazing power of Lisp is hidden away in a cryptic language riddled with idiosyncrasies and inconsistencies. Surely someone else has had a go at improving the situation...? It seems like with some macros and code organization, Lisp could become far more accessible to the large numbers of C/C++/C#, Java, and even Basic programmers out there.

Given the current business environment I suspect I'll be fortunate if I'm even able to use the JVM-friendly Clojure, with Common Lisp being a distant pipe-dream (unless I get a new job, which Lisp just might encourage me to attempt). How similar is Clojure to Common Lisp? Have they attempted to smooth out some of the rough edges?

[findinglisp]

I am in the process of learning Lisp, and am interested in the experience other people had while learning it.

Congrats! You'll thank yourself later.

How long did it take you to get started? I've been going at a fairly slow pace (only up to Ch 13 after 2-3 weeks), but Lisp is still almost completely foreign to me. It's like looking at Perl. Although I have limited experience using languages, I read a lot of code in various languages, and it's generally pretty easy to get an idea of what's going on. Not so with Lisp.

You can learn basic Lisp syntax fairly quickly, but the advanced features will definitely take you a while. IMO, PCL is a good book, but I actually like starting with Graham's ANSI Common Lisp first, followed by PCL.

So many functions have cryptic names that are on par with assembly for understandability. And how many functions there are! I'm getting the impression that Lisp offers the functionality of Java plus a good chunk of Java's libraries all within the language spec. So... that's great, but a troublesome downside for a newcomer is the sheer quantity of functions, many with cryptic names with no organization - everything's just there, like Java keywords. I appreciate the logical hierarchy and organization imposed on methods in an OO design. (Before I get too far, I am aware that Lisp supports objects, but what I'm addressing here is the large quantity of functions in the language spec that would be much more comprehensible and easier to learn if they had some sort of organization.)

The good news is that the actual core of Lisp is relatively small by the standards of almost any other language. Most of the CL spec is really library functions. The downside is that because Lisp is old, many of the names of library functions are archaic and crufty. More than once, I have found myself reimplementing something already in the standard library simply because I didn't realize it was there because it lived under a name I wasn't expecting. I don't know a way to prevent that other than working with the language for a while.

I feel like my first act after learning enough Lisp to get started would be to rewrite Lisp. (And WOW is it pleasing that you can actually do that in Lisp!) Create macros to give functions names that are meaningful to English speakers. Learning Lisp is like learning a whole new foreign language - knowing English is not sufficient to deduce the purpose and behavior of many Lisp functions simply from their name. And organization - if a group of functions only operates on lists, then perhaps name them list-add(), list-remove()? For the similar functions that operate on trees, or plists, or whatever, give them similarly clear names.

You can do this, but I'd warn you away from using it as a long-term crutch. As I have suggested before, when you learn French or Japanese, it's better to try to speak them with a native accent, not your foreign accent. It might seem more rational to you to do something one way or another, but the native speakers will be very annoyed at you. As a short-term crutch while you are learning, go ahead. As you said, Lisp makes it possible, so have fun. But realize that you'll be trading off some long-term items.

While I'm at it, I might hide some of the inconsistencies. It seems like Lisp's destructive functions are wantonly destructive! Why no be productively destructive? I was completely confused when I discovered that you still had to use the return value of destructive functions:
(setf *list-3* (delete 4 *list-3*))
If delete is destructive, why?! must I use setf I railed. It was a good bit more reading before I realized that destructive functions destroy with no (apparent) rhyme or reason, and the destroyed parameter is different from the value returned. I would be inclined to replace all these functions with functions that modify the parameter to produce the desired result, rather than requiring the caller to use the function's return value for the desired results and trashing the parameter!

I originally thought the same thing. The nice thing is that you'll find that Lisp is actually more consistent this way: you should use SETF with all those functions, whether they are destructive or not. This also makes it easier sometimes to go in and optimize code, moving from a more functional style, without destructive functions, to using the destructive versions. You simply have to change a function name, after checking that using a destructive function is actually safe, and you're golden. That said, I agree that this confused me several times at first. Graham's ANSI Common Lisp actually does a good job with describing this.

As early as I am in my process of learning Lisp, I am very excited by it, and dumbfounded that we ended up with popular languages we've suffered with for so many decades, while Lisp sat there quietly in the shadows. It didn't take too much experience programming before I was frustrated by the fact that I couldn't tell my code to modify code or write code! I mean seriously, why won't they let me do that?! When reflection finally became popular, that was at least some small relief, but the full power of Lisp's macros is sorely needed!

Yup. Your eyes will be progressively opened as you delve in deeper. I think every Lisper reaches this point where the lightbulb goes on and you say, "Wow, I can't believe I missed this language all this time. Where has it been hiding?"

All these thoughts lead me to wonder... has someone tried to "clean up" Lisp? Is there some reason it's a bad idea? It seems cruel to me that the amazing power of Lisp is hidden away in a cryptic language riddled with idiosyncrasies and inconsistencies. Surely someone else has had a go at improving the situation...? It seems like with some macros and code organization, Lisp could become far more accessible to the large numbers of C/C++/C#, Java, and even Basic programmers out there.

Well, yes, sort of. Scheme is more clean than Common Lisp. Clojure is a modern Lisp.

That said, Common Lisp is sort of like English: it has a long history and as a result, lot of inconsistencies and exceptions to the rule, but everybody knows it and in practice it's very powerful. A young Finnish kid on a train outside Helsinki once told me, "English is the easiest language in the world to learn to speak poorly. It's the most difficult language in the world to learn to speak well." The same might be said of Common Lisp, I suppose. While people have talked about cleaning up Common Lisp for a while, and there have been several serious attempts at it (Google for ISLISP, for instance), in practice the challenge would be similar to trying to "clean up" English. You're better of making a clean break and starting fresh, as did Scheme and Clojure, using the general ideas that go along with Lisp and reimplementing the standard library to suit.

Given the current business environment I suspect I'll be fortunate if I'm even able to use the JVM-friendly Clojure, with Common Lisp being a distant pipe-dream (unless I get a new job, which Lisp just might encourage me to attempt). How similar is Clojure to Common Lisp? Have they attempted to smooth out some of the rough edges?

From what I can tell (not having done a lot in Clojure), it smooths out a lot of rough edges, but introduces some idiosyncrasies of its own. That said, there is a version of CL that runs on the JVM (Google for "Armed Bear Common Lisp," ABCL).

[Harleqin]

All these thoughts lead me to wonder... has someone tried to "clean up" Lisp?

Yes. Every single newbie, before they began to see a significant part of the whole thing.

[nuntius]

If you do go on a cleanup expedition, here are a few suggestions.
- Keep the parens.
- Start by splitting things into smaller packages. e.g. ansi.alist, ansi.plist, ansi.cons, ...
- Keep names like CAR and CDR, but rename things like MAPCAN.
- Use symbol macros in these subpackages to redirect back to the symbols in CL; this way you don't have to worry about recreating argument lists, avoid any inefficiencies, and don't lose docstrings.
- Pick a reader (such as on http://www.informatimago.com/develop/lisp/), and make it part of your standard. Document places to make it more extensible.

I first saw the word "peloria" when Kenny Tilton used it to describe many attempts at reworking CL by regularizing argument lists.

[gugamilare]

While I'm at it, I might hide some of the inconsistencies. It seems like Lisp's destructive functions are wantonly destructive! Why no be productively destructive? I was completely confused when I discovered that you still had to use the return value of destructive functions:
(setf *list-3* (delete 4 *list-3*))
If delete is destructive, why?! must I use setf I railed.

Because in lisp there is no "pass variable by reference" to a function. If you need to modify the value of some variable, you need either use setf or a macro. This is a characteristic of functional style, and, probably, in a few months, you will think, like me, that passing variables "by reference" is strongly associated with headaches

Note that, in order to create a function like delete that changes the second variable (so you can avoid using setf), the function would need to modify the car and the cdr of *list-3*. And, if the result was nil, that wouldn't work as well.

But, if you think harder, you will see that, if you have macros, you don't need to complicate things. Just use them:

Code: Select all

(defmacro deletef (list item)
  `(setf ,list (delete ,item ,list)))

I swapped the order of the arguments because it is more or less of a convention that modifying arguments be the first argument of you macro (the terminating 'f' is another convention for modifying macros like this one).

Some utilities libraries (like alexandria) provide some macros that does what you want. It also swaps the order of the arguments, like I did:

Code: Select all

cl-user> (defvar *list* (list 1 2 3 4 5))
*list*
cl-user> (deletef *list* 1)
(2 3 4 5)
cl-user> *list*
(2 3 4 5)

Hum... By the way, talking about Alexandria:

"As a project Alexandria's goal is to reduce duplication of effort and improve portability of Common Lisp code according to its own idiosyncratic and rather conservative aesthetic."

Maybe Alexandria is more or less what you are looking for?

There is also CDR which tries to establish new standards, but nothing was achieved by CDR up to now (that I know of).

All these thoughts lead me to wonder... has someone tried to "clean up" Lisp? Is there some reason it's a bad idea? It seems cruel to me that the amazing power of Lisp is hidden away in a cryptic language riddled with idiosyncrasies and inconsistencies. Surely someone else has had a go at improving the situation...? It seems like with some macros and code organization, Lisp could become far more accessible to the large numbers of C/C++/C#, Java, and even Basic programmers out there.

Well, many people try to "clean up" Lisp in several ways. Most of them are just newbies that know very little and end up doing a terrible, useless job. I believe that it isn't Lisp what needs to be cleaned up, but people need to get used to it. Just give yourself time to try and you will see Lisp as a very natural language, and it doesn't have as much riddles, idiosyncrasies and inconsistencies as you think. It surely is not perfect, but it is far away from a bad language. IMO, the only really bad thing about Common Lisp is the lack of modern standardization (to standardize things like threads, sockets, gray-streams and other stuff).

[skypher]

Most of your questions have already been answered well, so I'm just throwing in some bits:

I'm currently still in the very early stages (Ch 13 of Practical Common Lisp) of learning, but have a lot of thoughts going through my head.

Use Lisp to give your thoughts form. Write software in Lisp. This will greatly accelerate your learning.

How long did it take you to get started? I've been going at a fairly slow pace (only up to Ch 13 after 2-3 weeks), but Lisp is still almost completely foreign to me.

After almost two years I've grown familiar with Lisp.

It's like looking at Perl.

Only with Lisp there are great patterns behind it, and at some point you will discover that the core principles of Lisp are surprisingly simple and uniform compared to other languages.

but what I'm addressing here is the large quantity of functions in the language spec that would be much more comprehensible and easier to learn if they had some sort of organization.)

But they are organized. Browsing the HyperSpec you'll find everything neatly organized into categories.

It seems like with some macros and code organization, Lisp could become far more accessible to the large numbers of C/C++/C#, Java, and even Basic programmers out there.

Lisp requires a different mindset, but there are projects like NewLISP (targeting PHP programmers) and Clojure (targeting Java programmers).

Given the current business environment I suspect I'll be fortunate if I'm even able to use the JVM-friendly Clojure, with Common Lisp being a distant pipe-dream (unless I get a new job, which Lisp just might encourage me to attempt).

Web applications can be written in any Lisp you like.

Leslie

[Paul Donnelly]

I appreciate the logical hierarchy and organization imposed on methods in an OO design. (Before I get too far, I am aware that Lisp supports objects, but what I'm addressing here is the large quantity of functions in the language spec that would be much more comprehensible and easier to learn if they had some sort of organization.)

On the plus side, it means that they're just there — you don't have to obtain or know about some particular sort of object to use them. And they're fairly well organized in the Hyperspec. Just check the relevant “dictionary” section. For example, maybe you look in the Strings Dictionary for string-related functions.

I feel like my first act after learning enough Lisp to get started would be to rewrite Lisp. (And WOW is it pleasing that you can actually do that in Lisp!) Create macros to give functions names that are meaningful to English speakers.

...and lose the ability to pass them as functions. If it's a function to begin with, turning it into a macro is just going to make things awkward.

Learning Lisp is like learning a whole new foreign language - knowing English is not sufficient to deduce the purpose and behavior of many Lisp functions simply from their name. And organization - if a group of functions only operates on lists, then perhaps name them list-add(), list-remove()? For the similar functions that operate on trees, or plists, or whatever, give them similarly clear names.

And then your clean, readable code turns into list- list- list- list-? Seems like a net lose.

While I'm at it, I might hide some of the inconsistencies. It seems like Lisp's destructive functions are wantonly destructive! Why no be productively destructive? I was completely confused when I discovered that you still had to use the return value of destructive functions:
(setf *list-3* (delete 4 *list-3*))
If delete is destructive, why?! must I use setf I railed. It was a good bit more reading before I realized that destructive functions destroy with no (apparent) rhyme or reason, and the destroyed parameter is different from the value returned. I would be inclined to replace all these functions with functions that modify the parameter to produce the desired result, rather than requiring the caller to use the function's return value for the desired results and trashing the parameter!

Since they are functions they're not able to modify the parameter even if they want to. You'd have to use macros for that, but then you wouldn't be able to pass them as parameters to higher-order functions. And most of the time the input to these functions isn't stored in a variable anyway. So what are we going to do, check every call to see if the argument is a variable, or make it incompatible with the most common use case?

The destructive functions are efficiency hacks. They don't exist for side effects; they are allowed to have side effects in the name of efficiency. So why do extra work to set the original variable to the return value when the programmer probably just wants the return value anyway? Because in Lisp you're probably either nesting function calls, establishing a new binding with “let”, or using some looping construct, right? How often are you putting your values into variables then modifying them there?

All these thoughts lead me to wonder... has someone tried to "clean up" Lisp? Is there some reason it's a bad idea? It seems cruel to me that the amazing power of Lisp is hidden away in a cryptic language riddled with idiosyncrasies and inconsistencies.

“A cryptic language riddled with idiosyncrasies and inconsistencies”? You make it sound like C++ or something. No doubt Lisp has its idiosyncrasies, but it's far from cryptic and inconsistent overall. Every language has its quirks, and I don't think Common Lisp has more than its share.

There are lots of things about Common Lisp that could stand to be cleaned up (such as combining semi-redundant functions, making the built-in functions generic, and reducing the complexity of filenames since file systems seem to have settled down), but updating the standard is expensive and there are still plenty of people with PTSD from creating the Common Lisp standard in the first place. Common Lisp the standard isn't changing any time soon, and I think most long-time Lispers feel that the difficulty of making up and implementing a new Lisp that functions as well as Common Lisp is much greater than the benefit derived from doing so. Of course, some people might say that Clojure or Scheme is just that Lisp, but personally I think CL's stability, maturity, and the synergy between its features are big assets that I want in the Lisp I'm using.

[tayssir]

How long did it take you to get started?

Complex question, given that I was learning a big gestalt of stuff at that point. (I found computing beyond boring, so I looked around for its possiblities. Why can't programming be as entertaining as any other computer game?) Also, I then considered improved skill with emacs and linux to be a prerequisite for seriously using Common Lisp.

I've been going at a fairly slow pace (only up to Ch 13 after 2-3 weeks), but Lisp is still almost completely foreign to me.

A chapter a day or so isn't slow. Also, ideally you'd have some sort of mentoring.

It's like looking at Perl. Although I have limited experience using languages, I read a lot of code in various languages, and it's generally pretty easy to get an idea of what's going on.

Readability's an interesting topic. One fortunate thing is that you can ask the Lisp environment questions about what you read, using (say) the REPL. This resource is not captured in most discussions of readability. With computers and programming languages, the notion of readability is expanded to include being able to ask certain questions (and expect a response), as well as change the context which things execute under.

There are parts of Lisp which are unnecessarily difficult to read. For instance, forms like 'cond' are garnished with too many parentheses. Evolutions of Lisp like Clojure attempt to reduce this problem.

Also that there's a different reading style, since people using Lisp often just pass results from one operator to another, rather than accumulate stuff in intermediate variables all the time. So instead of vertical towers of:

thing = this
thing = that

you see strings of:

<- <- <- <-

Though there are exceptions; my uses of CL's 'loop' look more like the former.

I'm curious how it'd be if Lisps used postfix. (Forth doesn't look so bad.)

And how many functions there are! I'm getting the impression that Lisp offers the functionality of Java plus a good chunk of Java's libraries all within the language spec. So... that's great, but a troublesome downside for a newcomer is the sheer quantity of functions, many with cryptic names with no organization - everything's just there, like Java keywords. I appreciate the logical hierarchy and organization imposed on methods in an OO design. (Before I get too far, I am aware that Lisp supports objects, but what I'm addressing here is the large quantity of functions in the language spec that would be much more comprehensible and easier to learn if they had some sort of organization.)

Yes, dumping everything in the same namespace is a bug. I remember hearing it was considered a known flaw during Common Lisp's standardization, but they didn't have the resources then to do anything about it. By that time, the participants were taking an economic battering.

That said, the back of Graham's _ANSI Common Lisp_ contains a very good reference for me, solving that problem to a great extent. I haven't really read the other parts of the book, but that quickref put everything in perspective. Maybe there's similar cheatsheets to download out there.

It reminds me of Patrick Chan's Java series, which I had similar views on.

It's true that many names are baroque; it's a bit like an old city where you can see artifacts from different generations and movements. Parts of which are definitely rough; others more well-developed.

If delete is destructive, why?! must I use setf I railed. It was a good bit more reading before I realized that destructive functions destroy with no (apparent) rhyme or reason, and the destroyed parameter is different from the value returned. I would be inclined to replace all these functions with functions that modify the parameter to produce the desired result, rather than requiring the caller to use the function's return value for the desired results and trashing the parameter!

Yeah, as I understand, the spec's semantics were loose here in order to give implementors leeway to make optimizations for their users. I just use the non-destructive versions; and consider the destructive versions as possible resources in weird situations. (Which I haven't come across.)

As early as I am in my process of learning Lisp, I am very excited by it, and dumbfounded that we ended up with popular languages we've suffered with for so many decades, while Lisp sat there quietly in the shadows. It didn't take too much experience programming before I was frustrated by the fact that I couldn't tell my code to modify code or write code! I mean seriously, why won't they let me do that?!

Yeah, it's still a bit of a bizarre feeling. We use these tools to automate. Why not try to complete the circle and let the tools have some ability to be used on themselves?

Given the current business environment I suspect I'll be fortunate if I'm even able to use the JVM-friendly Clojure, with Common Lisp being a distant pipe-dream (unless I get a new job, which Lisp just might encourage me to attempt). How similar is Clojure to Common Lisp? Have they attempted to smooth out some of the rough edges?

Oh, definitely. Clojure has evolved CL in a number of daring ways. And they've put an effort into cleanup. If you're ever jaded by CL, you might want to try Clojure. (And given that I've until recently used CL nearly every single day for.. longer than is healthy.. being jaded is an issue. I mean, part of programming is dead if you somehow can't improve on general-purpose languages.)

(I don't think I'm too biased in favor of cleanliness; Scheme is cleaner than CL, but I'd probably rather program in Python than any Scheme implementation I've tried.)

Dunno which Lisp you'll find more fun to use. And you can have a couple in your toolbox.

[Paul]

While I'm at it, I might hide some of the inconsistencies.

Like what? The only thing that bugs me is (file-position s n) instead of (setf (file-position s) n)

[mrove]

Thanks a lot for all of your replies. I certainly realize that going into something new one's perspectives lack experience, so I'm grateful to get insights from people who have not only had the same (or similar) thoughts I am having, but have the benefit of experience and hindsight to balance things.