define-compiler-macro

[Harnon]

Does anyone know what define-compiler-macro is useful for? It seems pretty much like a macro, except,according to documentation, "Unlike an ordinary macro, a compiler macro can decline to provide an expansion merely by returning a form that is the same as the original (which can be obtained by using &whole)." In what way is this useful??

[findinglisp]

It's useful for optimization of certain pieces of code. For instance, perhaps an algorithm becomes really, really fast when some of its arguments are constants or something like that. In that case the macro hands back a really fast version. If the arguments aren't constant, it returns the same general function call. You might ask, "But can't a standard macro do that too?" And the answer is yes. The difference is that you can define a compiler macro after you have written your code. In other words, the macro name can shadow another function name. That's very different than a typical macro where the name either represents a function or it represents a macro, but not both. This behavior allows you to return the same call to the old function if it doesn't find an interesting case worth optimizing. If you do that with a standard macro, the compiler loops, repeatedly calling the macro over and over again.

This is definitely a niche thing. You probably won't use a compiler macro unless you're really working hard to optimize a particular piece of code. I'm not a great Lisp hacker by any means, but I haven't had a need to use one ever.

[Harnon]

Thx
But i still don't understand exactly how to shadow the function name with the compiler-macro.
Looking at the documentation, it appears you have to (funcall (compiler-macro-function 'name) (name args) nil)
where name is the name of the compiler-macro. If i define a function with the same name, the function will always be used I also tried (setf (symbol-function 'name) (compiler-macro-function 'name)), but this doesn't work...
it appears to always want 2 args even if i only define it with one.

[qbg]

Thx
But i still don't understand exactly how to shadow the function name with the compiler-macro.
Looking at the documentation, it appears you have to (funcall (compiler-macro-function 'name) (name args) nil)
where name is the name of the compiler-macro. If i define a function with the same name, the function will always be used I also tried (setf (symbol-function 'name) (compiler-macro-function 'name)), but this doesn't work...
it appears to always want 2 args even if i only define it with one.

Say you have a FIB function, you then can do something like this

Code: Select all

(define-compiler-macro fib (n &whole w)
  (if (integerp n)
      ;Compute the nth fib. number
      w))


Then when you compile functions that call FIB with an integer literal the compiler can use this compiler macro to replace the call with its actual value (in this case).

Compiler macros are there to trade compile time speed for run time speed.

[makia]

hmm, but how you have such info at compile time ?

[Kompottkin]

hmm, but how you have such info at compile time ?

Recall what qbg said (emphasis mine):

Then when you compile functions that call FIB with an integer literal ...

Of course, a call like (fib x) (which the compiler macro will simply see as the list (fib x) -- note that the symbol x is not integerp!) can not be optimized in this way. But if you literally write something like (fib 10) somewhere in your code, the compiler macro for fib will get 10 as its argument and is able to precompute the value and insert it into the code instead of the function call.

[Kompottkin]

By the way, as a real-world example, I have actually used DEFINE-COMPILER-MACRO with quite satisfying effects in Objective-CL, an Objective-C bridge for Common Lisp.

Background: In order to invoke the method insertObject:atIndex: on the NSArray array with the arguments thing and position, I'd write:

Code: Select all

(invoke array :insert-object thing :at-index position)

Note that the method name has been split into two parts that have to be reassembled when actually calling the method. Now, calling an Objective-C method involves a number of steps (a bit simplified here):

1. Convert arguments to Objective-C values
2. Find method selector (an object that identifies the method name; it's a bit like an interned symbol in Lisp)
3. Find method by selector
4. Call method
5. Convert return value to a Lisp object

When profiling, I noticed that finding method selectors by first converting a list such as (:INSERT-OBJECT :AT-INDEX) (the one in the above example) into a string and then asking the Objective-C runtime for a selector of that name was by far the most expensive of all of the above operations. So what I did was define a compiler macro that first expanded the above call into the simpler call:

Code: Select all

(invoke-by-name array (selector '(:insert-object :at-index)) thing position)

and then another one on SELECTOR:

Code: Select all

(define-compiler-macro selector (&whole form method-name)
  (if (constantp method-name)
      (selector-load-time-form (eval method-name))
      form))

so that the original call would be replaced with something that looked for selectors at load-time rather than run-time.

(If you're really curious, SELECTOR-LOAD-TIME-FORM looks like this:

Code: Select all

(defun selector-load-time-form (method-name)
  `(load-time-value (handler-case
                        (find-selector ',method-name)
                      (serious-condition ()
                        (warn
                         (make-condition 'simple-style-warning
                                         :format-control
                                         "~S designates an unknown ~
                                          method selector."
                                         :format-arguments
                                         (list ',method-name)))
                        ',method-name))))

which, modulo error handling, is about equivalent to:

Code: Select all

(defun selector-load-time-form (method-name)
  `(load-time-value (find-selector ',method-name)))

It has the added benefit that mistyped method names are caught and cause warnings at load-time! So this compiler macro does two of the things that the static typing crowd always brags about: improved performance and some pre-run-time safety.)

[makia]

yes, i know that ... but then there is no huge impact in real programs if you can only optimize literal arguments ?

[Kompottkin]

... but then there is no huge impact in real programs ...?

Sure, compiler macros are a niche tool. findinglisp already remarked that. Most programs will probably not benefit from compiler macros at all. On the other hand, Objective-CL benefits immensely, and I hear that Edi Weitz’ CL-PPCRE does so as well, so there certainly can be a “huge impact” in “real programs”.

[bsdfish]

Alternatively, lets say you are working on a matrix library.

Code: Select all

(defun m* (matrix-1 matrix-2 &optional target)
   (if (null target)
       (setf target (make-matrix-of-right-size))
   ; blas call to multiply matrix-1 by matrix-2 and save to target
)

(defun m+ (matrix-1 matrix-2 &optional target)
   (if (null target)
       (setf target (make-matrix-of-right-size))
   ; blas call to add matrix-1 by matrix-2 and save to target
)


Now, you notice that a lot of times, you do something like (m+ mat1 (m* mat2 mat3)) When done naively, m* allocates a new matrix, and m+ also allocates a new matrix. However, in this case, m+ could just reuse the matrix returned by m* ...

Code: Select all

(define-compiler-macro m+ (matrix-1 matrix-2 &optional target)
   (when (and (null target) (listp matrix-2) (eq (car matrix-2) m*))
         `(let ((m2 ,matrix-2))
                  (m+ ,matrix-1 m2 m2))))


Or, even better, you realize there's a BLAS function call which performs m1 + m2*m3 and use the compiler-macro to use that call when (m+ mat1 (m* mat2 mat3)) is called.