I dont get Macros

Discussion of Common Lisp
Destruct1
Posts: 20
Joined: Wed Jan 20, 2010 1:40 am

I dont get Macros

Post by Destruct1 » Wed Jan 20, 2010 2:22 am

I am new to lisp and I am already glad that I learned the language.

When people complain about the syntax and/or lisp in general I think they are are confused by the concepts and not the language itsself. For example I hardly needed a day for recursion, conditionals and the parenthesis structure, but I needed half a week just to get the applicative operators and to learn why programming with them is a good idea. I think many people are confused by the complex concepts like functional programming, recursion, functions as primary objects, the extended list navigation functions (like first, cdar, rest) and the not always intuitive list building functions (list, append, cons). After I "got" those they are easy to use and effective.

Here is my problem though: Apparently macros are one of the few thing which makes Lisp different and better than other modern high-level languages like Python or Ruby (I am not talking about low-level languages like C). And Macros are "responsible" for the typical lisp code with numerous parens and the compact not line oriented structure. So, all in all, macros seem important.

But I can't figure out why they are a powerful tool. The behave like functions: If lisp finds an object after the opening parens it is assumed to be a function/macro. The symbol-cell and the corresponding call is triggered, the arguments evaluated (functions) or taken (macro) and some list is returned. Why are macros such a good idea?

My teaching book defines two differences between macros and functions:
1) Macros dont eval their arguments
2) Macros return List-Code which is immeaditly evaluated

I dont see any possibilities for the first point to have a effect: Suppose you have some babble input and want them to communicate with your system. So you build a parser which morphs a string input into useful code. Considering the high number of sequence operations I think macros are not really needed. You just take string or list as input, build tokens and modify them. The point here is: You can evaluate your input everytime, because either your input makes sense from a lisp point-of-view (and you want them to be evaluated) or you take a string/list as a selfevaluating object and return them after you parsed them (which is needed anyway).

And the second point makes no sense too: A function returns data. If a macro returns code which is evaluated it must, at the end, produce data.

Last point: Macros can define functions or variables at the top level. I dont think that is not that useful either because functions can return functions and set variables. While it might be nice to write a function defining macro a function which returns a function does a similar thing.

Code: Select all

;not tested
(defmacro adder1 (name arg)
   `(defun ,name (input)
        (+ input ,arg))

;function does just as well
(defun adder (arg)
      (lambda (d) (+ d arg)))
Can somebody show me either
a) examples where macros are desperatly needed
b) some patterns where macros are used
c) explain how to build an embedded language on top of lisp like I heard so many times

ramarren
Posts: 613
Joined: Sun Jun 29, 2008 4:02 am
Location: Warsaw, Poland
Contact:

Re: I dont get Macros

Post by ramarren » Wed Jan 20, 2010 3:36 am

One thing you don't seem to be aware of: Common Lisp is a compiled language. Most implementation will compile to native machine code applying many optimizations. See the output of DISASSEMBLE function, although some implementations, like Clisp do emit bytecode. Some implementations have direct interpreter mode, but that is not really relevant to the purpose of macros.
Destruct1 wrote:1) Macros dont eval their arguments
2) Macros return List-Code which is immeaditly evaluated
This is not exactly true. A macro is a hook into the compiler. The argument to a macro is a source form, and the returned code is not immediately evaluated, but returned to the compiler for further processing. The point is to avoid processing the macro-input every time and just execute the optimized returned code.

Canonical examples of basic macros are definers, like the standard DEFCLASS and dynamic context establishers, like WITH-OPEN-FILE. For that matter, SETF is a macro.

Obviously, everything that a macro does can be achieved by manually typing in its expansion, but the same is true about compilers, the same thing could be achieved by typing in machine code directly.

One problem with explaining macros is that simple examples are so simple that it is not obvious that they are worth it, but complex examples are incomprehensible to someone who doesn't already understand the system the example is taken from. That said, the most recent macro I had written was in my parser combinator library, which contains nineteen macro definitions in total.

gugamilare
Posts: 406
Joined: Sat Mar 07, 2009 6:17 pm
Location: Brazil
Contact:

Re: I dont get Macros

Post by gugamilare » Wed Jan 20, 2010 6:18 am

Macros are useful for many purposes. They can make your code faster in many cases by transporting some of the computation from runtime to compile-time, for instance, simplifying the computation when some of the arguments are known. For that matter, there exist compiler-macros, you can learn about them later.

Another example is convenience. 99% of CL libraries come with macros to simplify the use of the library. The macro with-open-file is a canonical example: instead of forcing the programmer to close every file that was open, create a macro that does it for the programmer. Instead of teaching the programmer what functions are needed to initialize and shutdown a library (for instance, SDL or opengl), you can create a macro that accepts some options and does the dirty job for the programmer (e.g. with-sdl or with-opengl).

Those are just the simplest examples, there are many situations where you can create your own syntax that still looks like lisp, but which wouldn't be possible without creating macros or modifying the compiler to accept them. The canonical example is the loop macro, e.g.:

Code: Select all

(loop for i from 0 to 200 collect i)
This code would not work if loop was a function: the environment would complain that the symbols for, i, from, to and collect are not defined.

In any case, just give it some time and you will understand and create very useful macros :)

Paul Donnelly
Posts: 148
Joined: Wed Jul 30, 2008 11:26 pm

Re: I dont get Macros

Post by Paul Donnelly » Wed Jan 20, 2010 10:59 pm

Destruct1 wrote:1) Macros dont eval their arguments
2) Macros return List-Code which is immeaditly evaluated

I dont see any possibilities for the first point to have a effect
No, this is great. You can transform your input however you like to automate writing basically any code you can dream up. If the arguments were evaluated, everything would have to be quoted. It's always hard to give good macro examples that won't be trivial or too complicated to understand, but (besides the many examples in the standard) metabang-bind (http://common-lisp.net/project/metabang-bind/) might make sense to you. Common Lisp has a lot of forms that establish bindings, and when you need to use a bunch of them it can get hairy. Thanks to macros, it was possible to write a wrapper that can be much simpler to use.

The loop macro is another good example.
Destruct1 wrote:And the second point makes no sense too: A function returns data. If a macro returns code which is evaluated it must, at the end, produce data.
It's not necessarily immediately evaluated. It's substituted into your code in place of the macro call. This means you can do things like creating lexical variable bindings that you couldn't do otherwise except by writing all the code yourself every time. Some people like to say about Lisp that "code is data". Yeah, functions take and return data, but what about when that data is code? You need a macro so you can type the input data verbatim, and you need need a macro so the returned data can actually end up in your code before compilation or evaluation.

krz
Posts: 2
Joined: Sun Jan 24, 2010 10:57 am
Location: Rochester, New York, USA
Contact:

Re: I dont get Macros

Post by krz » Sun Jan 24, 2010 12:20 pm

(My apologies if I misunderstood your question.)

Actually, the bit about evaluation is a very important distinction. Imagine you wanted to express something like

Code: Select all

(unless (zerop x)
  (/ foo x))
Obviously, we don't want to do the division unless we know x is not zero. So, we need something that will not always evaluate (/ foo x). This is why this must be a macro, and cannot be a function. If it were the latter, the division would always get evaluated regardless of the value returned by zerop. (Strictly speaking, that's not true; you could quote the expression, and then call eval when you deem it safe, but eval in application-level software is rare (and often (though not always) bad style).

What's more, many lisp systems only provide one conditional special operator under the hood. Usually, this is cond or if. In those systems, the one is written in terms of the other as a macro. Then, in turn, the (slightly higher) level constructs like when are written in terms of if and progn. Then, finally, the nonstandard-but-you-find-them-everywhere idioms like awhen or let-when are written as macros in terms of those.

Have you played much with macroexpand-1 and macroexpand? Also, nonstandard but very common is macroexpand-all (if your system lacks it, there are variants of it all over the 'net).

wentbackward
Posts: 2
Joined: Sat Jan 30, 2010 4:08 am

Re: I dont get Macros

Post by wentbackward » Sat Jan 30, 2010 4:34 am

I think a practical example would help. Look up the macro aif. There's one in Paul Graham's on-lisp book. It's simple and easy to understand. Once you've got it, try to write something similar, like awhen. Switch to a language you're familiar with and try to implement aif. This will give you a taste of macro power in albeit a simple example.

You're right, the concepts behind lisp are hard. Peter Norvig wrote a great paper called Learn Programming in 10 years (or something like that). I think the concepts behind programming in general are hard, which is why there aren't that many good programmers (IMO there aren't even many mediocre programmers). People think that because some guy can put together a UI or web page that they're a programmer. However I'm often ashamed by the mess that I so often come across and that my profession for more than 20 years is currently in such disarray. One good programmer is an order of magnitude more valuable that most people I come across day to day in my business.

Your questions are intelligent and clearly you're on the good path, ignore what the majority think, persevere, you're needed!

- Paul

dlweinreb
Posts: 41
Joined: Tue Jul 01, 2008 5:11 am
Location: Lexington, MA
Contact:

Re: I dont get Macros

Post by dlweinreb » Sun Jan 31, 2010 8:54 am

The best explanation of macros that I know of is from Practical Common Lisp. See the chapter on writing your own macros, at http://gigamonkeys.com/book/macros-defi ... r-own.html.

Macros are very different from functions. They are a way to write language extensions and domain-specific languages in Lisp.

JamesF
Posts: 98
Joined: Thu Jul 10, 2008 7:14 pm

Re: I dont get Macros

Post by JamesF » Sun Jan 31, 2010 6:30 pm

I'll pitch in with my bit of input, because you haven't already been deluged enough :)

Macros are evaluated at compile-time, and are never seen at runtime (unless the application itself writes and compiles code that calls on them).
When the compiler is reading the source-code and encounters a call to a macro, it immediately executes the macro, replaces that section of source-code with the result, then compiles the altered source.
To use a really trivial example that shows none of the actual power of a macro, if you write this:

Code: Select all

(defmacro foo (zot)
 (list 'format 't "~a" zot))

(defun bar ()
 (foo "boing"))
...the effect will be the same as if you'd written:

Code: Select all

(defun bar ()
 (format t "~a" "boing"))
It may help to explain the intent of macros, to make sense of what they do. You know how sometimes you find yourself writing the same pattern of code for the nth time, and think, "man, I should really write a programme that would write this code for me"? That's what they do. Macros are mini-programmes that the compiler runs to generate the source-code; they let you define the pattern, then invoke that pattern and just feed it the interesting bits. Don't let the above example fool you into thinking that you can only use substitution; you can do anything to the input that can be expressed in Lisp.

Where the power and confusion both enter the picture is that you get to call on the full power of Lisp to generate Lisp source-code. On the one hand, you don't have to switch contexts to think in a different language when writing your code-generating programmes, but on the other hand there's not much to remind you that it's the output of the macro call that will be compiled and not the literal source that you're writing.

Destruct1
Posts: 20
Joined: Wed Jan 20, 2010 1:40 am

Re: I dont get Macros

Post by Destruct1 » Mon Feb 01, 2010 2:31 am

JamesF wrote: You know how sometimes you find yourself writing the same pattern of code for the nth time, and think, "man, I should really write a programme that would write this code for me"? That's what they do.
The main point is: What is the difference between a function and a macro. What is so special about a macro that cannot be done otherwise?

Lets go through the examples here and compare them with Python code.
I keep the Python Code simple and readable to avoid confusion, but all my examples can be written shorter.

1)

Code: Select all

(unless (zerop x)
  (/ foo x))
Thats easy:

Code: Select all

def safediv (x, y):
  if y!=0:
    return (x/y)
  else:
    return (False)
2) The DoPrimes Macro from Practical Common Lisp
Its like dotimes but only outputs prime numbers. It uses primep to check if something is a primenumber

Code: Select all

def primegen (until_number_reached):
  primegen = (e for e in range (2, until_number_reached) if primep (e) == True)
  return (primegen) # return an generator expression which can be iterated over

In action:

for x in primegen (30):
  # do something


3) The anphoric Macro in OnLisp:
It’s not uncommon in a Lisp program to want to test whether an expression
returns a non-nil value, and if so, to do something with the value. If the expression
is costly to evaluate, then one must normally do something like this:
(let ((result (big-long-calculation)))
(if result
(foo result)))
Wouldn’t it be easier if we could just say, as we would in English:
(if (big-long-calculation)
(foo it))
By taking advantage of variable capture, we can write a version of if which works
just this way.
[..snip..]
and used as in the previous example:
(aif (big-long-calculation)
(foo it))
I probably dont get that, but it is very easy:

Code: Select all

def aif (long_calculation, further_func):
  if (long_calucaltion) != 0:
    return (further_func (long_calculation))
  else:
    return (0) # ??
So here is the point: Macros can be easily implemented by other things

ramarren
Posts: 613
Joined: Sun Jun 29, 2008 4:02 am
Location: Warsaw, Poland
Contact:

Re: I dont get Macros

Post by ramarren » Mon Feb 01, 2010 4:10 am

Destruct1 wrote:So here is the point: Macros can be easily implemented by other things
To reiterate: macros are hooks for a compiler. They allow you to extend the compiler. Macros can be implemented as other things in the same sense that everything a compiler does can be by writing it manually in assembly. The point of macros is the same point as that of compilers: it allows automation and syntax/semantic extension.

Post Reply