SideEffects wrote:Usually compiling, loading and file dependencies are dealt with using a system definition facility like ASDF. There is a short introduction of Common Lisp project organization here.
I guess ASDF is the piece I'm missing here. Groan, why does this have to be so complicated?
It doesn't necessarily have to be complicated. Worst case, just create a file that loads all your other files. I do this all the time when I don't want to deal with ASDF. If I was creating a large system or a library for release that had lots of other dependencies, I would use ASDF, but it doesn't have to be any more complex than:
Code: Select all
(load "a/A.lisp")
(load "c/C.lisp")
...etc...
Just make sure you load them in the right order. Then you'd have:
a/A.lisp
Code: Select all
(defpackage :a (:use :cl))
(in-package :a)
(defun foo ...)
(defun bar ...)
c/C.lisp
Code: Select all
(defpackage :c (:use :cl)) ;; could also (:use :cl :a) if you wanted to call FOO and BAR directly, without A:FOO and A:BAR
(in-package :c)
(a:foo 1 2 3)
(a:bar 1 3 3)
(defun baz ...) ;; BAZ is now interned in package C
Note that all these forms are also independent of files. The reason is that while the reader is reading a file (when you load it), it's simply building up state as it goes. It reads a form and then evaluates it. It then reads the next form and evaluates that. Etc., until it reaches end-of-file. Thus, you need to execute a DEFPACKAGE before you execute an IN-PACKAGE that references it, but there is no reason that those have to be in the same file. For instance, many people put a bunch of DEFPACKAGE forms into a single file called something like packages.lisp that gets loaded first. This defines all the packages up front. Then every subsequent file has an IN-PACKAGE form that tells the reader that everything after that point is to be read into the specified package.
Also, note that you could also literally concatenate all these files together in whatever load order was required to make it all work and load just that single file, like so:
bighonkingfile.lisp:
Code: Select all
(defpackage :a (:use :cl))
(in-package :a)
(defun foo ...)
(defun bar ...)
(defpackage :c (:use :cl)) ;; could also (:use :cl :a) if you wanted to call FOO and BAR directly, without A:FOO and A:BAR
(in-package :c)
(a:foo 1 2 3)
(a:bar 1 3 3)
(defun baz ...) ;; BAZ is now interned in package C
Then just (LOAD "bighonkingfile.lisp").
The reader would then process the DEFPACKAGE forms, then various IN-PACKAGE forms, etc. In other words, LOAD is simply like pointing the REPL at a file and everything in the file is processed effectively as if you had typed it at the REPL (this ignores compilation and EVAL-WHEN and some other nuances; that's the advanced course).
So, remember, DEFPACKAGE creates an in-memory data structure into which symbols are interned. The IN-PACKAGE form sets the *PACKAGE* variable which tells the reader which package is the current package. As the reader encounters unprefixed symbols, it interns them into the current package specified by *PACKAGE*. It's really that simple. The reader is independent of files; the reader operates at the REPL and with LOAD (LOAD is basically the equivalent of typing all the forms in the file into the REPL by hand, only automated). Forget files completely other than as blobs of program text which can be LOADed. For what it's worth, ASDF (and any other defsystem) is simply a complex way of specifying which files depend on other files so that the dependencies can be handled automatically to create the correct sequence of file loads. Think of ASDF as being sort of like apt-get or yum on a Linux box, but for file loading instead of handling package dependencies.
If all this seems pretty byzantine, it's only because it's not automated like it has been in many other languages. There is just some more machinery exposed here with Lisp that is otherwise handled automatically in Ruby, Java, etc. The mechanisms are all pretty simple, however.
I know that when I was going through this EXACT SAME confusion a few years ago, it didn't really make sense to me until I realized that a symbol isn't just a source-code thing--it's a first-class object that persists in the running image. So is a package. These are data structures at runtime, not just data structures in the compiler. When you write out a symbol name in a file, the reader basically converts that to a reference (a pointer) to a symbol data structure in the running image. But when you type "FOO" it has to know which "FOO" symbol you want to reference. To do that, it looks in a package data structure. You can think of a package data structure as a big hashtable that can map a symbol name (a string) to a symbol data structure (that's what INTERN does). Within a package, there is only one "FOO" symbol, but there can be another "FOO" symbol in another package. When the reader is reading along, it finds a block of source text that names a symbol, it then calls INTERN on it, using the current *PACKAGE* and gets back a reference to a symbol data structure. DEFPACKAGE creates the package (the hashtable). IN-PACKAGE simply sets the value of *PACKAGE*.