Programming Style & (eval ...)

Discussion of Common Lisp
taylor_venable
Posts: 10
Joined: Tue Jul 15, 2008 4:50 am
Location: Fort Wayne, Indiana, United States
Contact:

Programming Style & (eval ...)

Post by taylor_venable » Tue Jul 22, 2008 8:23 am

Hello all. I've been using Lisp in various forms for the last few years, but I've only recently started to really get into it. I find that for me to really get to know a language it helps to write some common libraries in it, so I decided to start porting some of Haskell's list library functions into Lisp. I ran into a problem right away though when trying to write all which takes a predicate and a list, returning true when all the elements in the list satisfy the predicate. So, for example

Code: Select all

CL-USER> (all #'(lambda (x) (= (mod x 2) 0)) '(2 2 4 8 6 0))
T
CL-USER> (all #'(lambda (x) (= (mod x 2) 0)) '(2 2 4 8 6 1))
NIL
My first thought was that this is just a fold of the and operation into the result of mapping the predicate over the list. That means basically applying and to all the list elements, so in my ignorance I tried to do:

Code: Select all

(apply 'and (mapcar #'evenp '(2 4 6 8 9))) => nil
But of course that doesn't work because and isn't a function, it's a macro. So the solution I eventually discovered was:

Code: Select all

(defun all (f l)
  (let ((r (mapcar f l)))
    (eval `(and ,@r))))
Which leads me to my question: is it considered acceptable style to use eval to do something like this? I typically shun the use of eval in other languages, because it's usually just a cheap way of getting something done which could be handled in more elegant ways, but with the data-is-code orientation of Lisp, is this a common tactic to employ?

I'd also be interested to see any alternate implementations of and; I thought it might be cool to have a macro that expanded into [using the example above] (and t t t t nil) but I can't seem to make it happen. My best guess was

Code: Select all

(defmacro all (f l)
  `(let ((r (mapcar ,f ,l)))
     (and ,@r)))
But that doesn't work because r is unbound -- which I think is because it doesn't exist in the environment into which the macro is expanded. However, even if I define an r using let before using the macro, it still doesn't work, not that that would be what I wanted anyway.

Thanks for any advice!
"I have never let my schooling interfere with my education." -- Mark Twain

ramarren
Posts: 613
Joined: Sun Jun 29, 2008 4:02 am
Location: Warsaw, Poland
Contact:

Re: Programming Style & (eval ...)

Post by ramarren » Tue Jul 22, 2008 9:09 am

taylor_venable wrote:I ran into a problem right away though when trying to write all which takes a predicate and a list, returning true when all the elements in the list satisfy the predicate.
First, note that such function already exists: every.
taylor_venable wrote:My first thought was that this is just a fold of the and operation into the result of mapping the predicate over the list. That means basically applying and to all the list elements, so in my ignorance I tried to do:

Code: Select all

(apply 'and (mapcar #'evenp '(2 4 6 8 9))) => nil
This is not a fold. Apply calls a function with arguments given in a list. A fold is done by reduce function, and in this case, since and is a macro, would look:

Code: Select all

(reduce #'(lambda (lv rv) (and lv rv)) (mapcar #'evenp '(2 4 5 8 9)))
Note that this is not short-circuiting, as mapcar will obviously evaluate the predicate on all arguments. It can be done trivially by iterating on the list:

Code: Select all

(dolist (e '(2 4 5 8 9) t)
  (unless (evenp e) (return  nil)))


If you really want to use a macro as a function you can always wrap it in lambda. It is different from eval in that the function is constructed at compile-time (or maybe even read-time?), not run-time. But this is rarely needed in practice, because for short-circuiting operators functional versions already exist (every, some, notevery, notany).
taylor_venable wrote:Which leads me to my question: is it considered acceptable style to use eval to do something like this? I typically shun the use of eval in other languages, because it's usually just a cheap way of getting something done which could be handled in more elegant ways, but with the data-is-code orientation of Lisp, is this a common tactic to employ?
In my (not very extensive, mind you) experience using eval is very rare, and it's use can be almost always replaced with something better.
taylor_venable wrote:I'd also be interested to see any alternate implementations of and;
I'm just making this up, so this is not even remotely efficient and may even not work, but:

Code: Select all

(defmacro my-and (&body body)
  (let ((step-fun (gensym))
        (thunk-list (gensym))
        (thunks (gensym)))
    `(let ((,thunk-list (list ,@(loop
                                   for b in body
                                   collect `(lambda ()
                                              ,b)))))
       (labels ((,step-fun (,thunks)
                  (cond ((null ,thunks)
                         t)
                        ((not (funcall (car ,thunks)))
                         nil)
                        (t (,step-fun (cdr ,thunks))))))
         (,step-fun ,thunk-list)))))
taylor_venable wrote: I thought it might be cool to have a macro that expanded into [using the example above] (and t t t t nil) but I can't seem to make it happen. My best guess was (...)
Expanding to and is not a good idea for the same reason apply isn't when used on arbitrary sequences, there is an argument number limit. In any case, evaluating the predicate at compile time would require the arguments to be known at compile time, and then why would you expand to (and t t t nil) at all, if you could just compute nil right there?

I hope that helped in some way.

reuben.cornel
Posts: 7
Joined: Sun Jun 29, 2008 9:48 am
Location: Sterling, Virginia, USA
Contact:

Re: Programming Style & (eval ...)

Post by reuben.cornel » Tue Jul 22, 2008 5:12 pm

taylor_venable wrote: I'd also be interested to see any alternate implementations of and;
Here's my two cents...

Code: Select all

(defun my-and(&rest bool-list)
  (null (remove t bool-list)))

reuben.cornel
Posts: 7
Joined: Sun Jun 29, 2008 9:48 am
Location: Sterling, Virginia, USA
Contact:

Re: Programming Style & (eval ...)

Post by reuben.cornel » Tue Jul 22, 2008 5:17 pm

Nice reply Ramarren, liked your method of explanation.

taylor_venable
Posts: 10
Joined: Tue Jul 15, 2008 4:50 am
Location: Fort Wayne, Indiana, United States
Contact:

Re: Programming Style & (eval ...)

Post by taylor_venable » Wed Jul 23, 2008 9:57 am

Ramarren wrote:I hope that helped in some way.
It did, and your explanation was enlightening. I had forgotten about argument length limits as well; in my mind I was mistakenly mapping (and x y z ...) [Lisp] to something like (x AND y AND z AND ...) [non-Lisp] and I see that in point of the fact that there are length limits, this would not be entirely the case, when the argument list is sufficiently long as to cause an error. But one question for clarification around that: since (and x y ...) just expands to (if x (and y ...) nil) then the limit of the argument list length would be hit in the function which does the expansion, right?

Thanks very much!
"I have never let my schooling interfere with my education." -- Mark Twain

ramarren
Posts: 613
Joined: Sun Jun 29, 2008 4:02 am
Location: Warsaw, Poland
Contact:

Re: Programming Style & (eval ...)

Post by ramarren » Wed Jul 23, 2008 10:15 am

Actually, I am not sure how this works really, but I think so. Macros are normal functions after all, they are just called by the compiler on the source tree... of course, in my SBCL:

Code: Select all

CALL-ARGUMENTS-LIMIT is an external symbol in #<PACKAGE "COMMON-LISP">.
It is a constant; its value is 536870911.
And quite possibly on 64 bit systems this would be even higher, so this is unlikely to be a problem in practice unless operating on really big datasets. Still, apply on arbitrary lists seems like bad style to me.

theclapp
Posts: 17
Joined: Wed Jul 16, 2008 11:05 am

Re: Programming Style & (eval ...)

Post by theclapp » Wed Jul 23, 2008 10:47 am

Ramarren wrote:Still, apply on arbitrary lists seems like bad style to me.
Agree. Especially since on some Lisps it's much lower. Lispworks for Linux Professional v5.2: 2047. Big enough for any hand-written function call you care to write (probably -- how masochistic are you? :) ), but too small to not worry about your APPLY arguments.

(I run into the shell version of this problem occasionally at work (AIX 5.2). The command line limit is fairly low (~20 kb). I like Linux, where it's on the order of 2 mb. Fortunately I'm not an idiot and am fully aware of xargs. :) )

I'm not sure if it's surprising or not that the Open Source tools have much higher limits than the commercial tools.

metageek
Posts: 10
Joined: Fri Jul 25, 2008 8:01 am

Re: Programming Style & (eval ...)

Post by metageek » Fri Jul 25, 2008 8:43 am

Ramarren wrote:And quite possibly on 64 bit systems this would be even higher
Yes: it's 1152921504606846975 (still in SBCL). That's (2^60)-1, a little over 1 quintillion. I suppose a 64-bit machine could have enough memory to have that many arguments...but not this year.

(Let's see, assume each argument takes one cons cell and one integer, or 12 bytes, for a total of 12 exabytes of RAM. RAM is currently around $25/GB, so 12 EB would cost 300 billion dollars. Anybody who can spend that much on RAM can afford to have the compiler extended to handle longer argument lists. Or I suppose it could be virtual memory; with terabyte hard drives selling for around $200, 12 EB of disk would only cost 2.4 billion dollars...but it'd take over 100 years to read it all in over a 16x PCIe card. Again, anybody who can wait 100 years for an answer is not living within the same limitations I am.)
theclapp wrote:I'm not sure if it's surprising or not that the Open Source tools have much higher limits than the commercial tools.
I'd say not. Smaller limits can generally be implemented more efficiently, and commercial Common Lisp implementations have to distinguish themselves on performance.

taylor_venable
Posts: 10
Joined: Tue Jul 15, 2008 4:50 am
Location: Fort Wayne, Indiana, United States
Contact:

Re: Programming Style & (eval ...)

Post by taylor_venable » Fri Jul 25, 2008 9:32 am

Ramarren wrote:
taylor_venable wrote:Which leads me to my question: is it considered acceptable style to use eval to do something like this? I typically shun the use of eval in other languages, because it's usually just a cheap way of getting something done which could be handled in more elegant ways, but with the data-is-code orientation of Lisp, is this a common tactic to employ?
In my (not very extensive, mind you) experience using eval is very rare, and it's use can be almost always replaced with something better.
I have another related question. Since macros don't evaluate their arguments, is there a way to get an argument to a macro to evaluate without using eval? The specific instance which makes me ask this is that I've been working with cl-who, a package that takes lists and turns them into SGML. There's a macro with-html-output-to-string which takes an argument like (:p "hello") and turns it into <p>hello</p>. But what I need is to evaluate some code, then pass the result to with-html-output-to-string, like so:

Code: Select all

(cl-who:with-html-output-to-string ... (car '((:p "hello")))) => "<p>hello</p>"
But since the argument isn't evaluated, cl-who doesn't know what to do with (car ...) so it outputs nothing. What technique is commonly employed to do this? By using eval I can write a macro like:

Code: Select all

(defmacro test (form)
  `(cl-who:with-html-output-to-string (*standard-output*) ,(eval form)))
This works, but along the lines of avoiding eval I wondered if there was another way to escape from macro non-evaluation?
"I have never let my schooling interfere with my education." -- Mark Twain

findinglisp
Posts: 447
Joined: Sat Jun 28, 2008 7:49 am
Location: Austin, TX
Contact:

Re: Programming Style & (eval ...)

Post by findinglisp » Fri Jul 25, 2008 10:21 am

taylor_venable wrote: This works, but along the lines of avoiding eval I wondered if there was another way to escape from macro non-evaluation?
Typically, there is an "escape symbol" of some sort that looks like a function call that tells the macro to insert that part of the form such that it will be evaluated in the macro-expansion. The macro must then process its parameters such that it looks for that tag and does the right thing. I think Peter Siebel had an example of that in Practical Common Lisp. See Chapter 31: Practical: An HTML Generation Library, the Compiler:
http://www.gigamonkeys.com/book/practic ... piler.html
Cheers, Dave
Slowly but surely the world is finding Lisp. http://www.findinglisp.com/blog/

Post Reply