Set-dispatch-macro-character - don't eval the result

[wvxvw]

Sorry if the title is vague. It's just the function name, which is a bit too long
The problem: I'm trying to understand how set-dispatch-macro-character works, the code below doesn't have any other practical purpose, only for learning:

Code: Select all

(set-dispatch-macro-character
 #\# #\{
 #'(lambda (first-char second-char stream)
     (declare (ignore first-char))
     (declare (ignore second-char))
     (labels ((read-white (stream)
		(loop for white-char = (read-char stream nil)
		   do (format t "reading white~&")
		   do (cond
			((char= white-char +closing-curly+)
			 (return t))
			((not white-char)
			 (return))
			((not (find white-char +white+))
			 (unread-char white-char stream)
			 (return)))))
	      (read-key (stream)
		(when (not (read-white stream))
		  (let ((result
			 (loop for key-char = (read-char stream nil)
			    do (when (or (find key-char +white+)
					 (char= key-char +colon+)
					 (not key-char))
				 (return (coerce token 'string)))
			    collect key-char into token)))
		    (read-white stream)
					; This character must be column
					; We should signal syntax error here, but
					; for the purpose of this demo 
					; assume the input is always correct
		    (read-char stream nil)
		    (read-white stream)
		    result))))
       (loop for key = (read-key stream)
	  do (format t "reading key: ~a~&" key)
	  do (when (not key) (return result-list)) ; This is what is being returned
	  collect (cons key (read stream))
	  into result-list))))

Now, what I wanted it to do:

Code: Select all

#{ key : 42 }
; would create:
((key . 42))

I.e. I wanted it to create a list with sub-lists of two elements, instead, what happens is that if I later use this macro handler, it tries to evaluate the list as if it was a function call. So, for my example it would tell that I'm making illegal function call.

EDIT:
I actually did it like this:

Code: Select all

(set-dispatch-macro-character
 #\# #\{
 #'(lambda (first-char second-char stream)
     (declare (ignore first-char))
     (declare (ignore second-char))
     (labels ((read-white (stream)
		(loop for white-char = (read-char stream nil)
		   do (format t "reading white~&")
		   do (cond
			((char= white-char +closing-curly+)
			 (return t))
			((not white-char)
			 (return))
			((not (find white-char +white+))
			 (unread-char white-char stream)
			 (return)))))
	      (read-key (stream)
		(when (not (read-white stream))
		  (let ((result
			 (loop for key-char = (read-char stream nil)
			    do (when (or (find key-char +white+)
					 (char= key-char +colon+)
					 (not key-char))
				 (return (coerce token 'string)))
			    collect key-char into token)))
		    (read-white stream)
					; This character must be column
					; If it's not, we should signal syntax error here, but
					; for the purpose of this demo 
					; assume the input is always correct
		    (read-char stream nil)
		    (read-white stream)
		    result))))
       (loop for key = (read-key stream)
	  do (format t "reading key: ~a~&" key)
	  do (when (not key)
	       (format t "about to return: ~a~&" result-list)
	       (return (list 'quote result-list)))
	  collect (cons key (read stream))
	  into result-list))))

But it doesn't feel right / I would be happy to know why it didn't work the first time.

[edgar-rft]

Sorry but I can't reproduce the problem because Lisp complains about several undefined variables: +CLOSING-CURLY+, +COLON+, +WHITE+.

I assume the missing definitions are:

Code: Select all

(defconstant +closing-curly+ #\})
(defconstant +colon+ #\:)
(defconstant +white+ (list #\Space #\Tab))

Could you please try to give complete code examples? This will save a lot of work for people who try to help you.

I would be happy to know why it didn't work the first time.

Because Lisp works in a READ-EVAL-PRINT loop where the expansion of a read-macro happens at READ time, so if the read-macro returns an unquoted list, then the subsequent EVAL will try to evaluate the list as a function call.

It's the same game like:

Code: Select all

CL-USER> ((key . 42))
error: illegal function call

CL-USER> '((key . 42))
((KEY . 42))

CL-USER> (quote ((key . 42)))
((KEY . 42))

Be careful with the second example, many Lisp compilers will treat '((key . 42)) like a constant expression and will signal errors if you try to modify elements of the list afterwards. The most fool-proofed version is the last example, like you already did it in your code.

This is what the second version of your macro returns to EVAL:

Code: Select all

;; (return (list 'quote result-list)) => (quote ((key . 42)))

CL-USER> (quote ((key . 42)))
((KEY . 42))

Just a simple (return (quote result-list)) will not work:

Code: Select all

;; (return (quote result-list)) => result-list

CL-USER> result-list
error: unbound variable RESULT-LIST

This is because the binding of RESULT-LIST only exists inside the LOOP.

[edgar-rft]

Another solution would be using backquote and comma:

Code: Select all

(return `(quote ,result-list)) => (quote ((key . 42)))

[wvxvw]

Thank you for explanation, it just didn't occur to me, that evaluating a list (which I thought to be already evaluated), wouldn't evaluate to itself, but instead will try to "execute" it.
Ah, and I had forgotten that part where the constants were defined. Sorry. But yes, it was just like you have it + newline and return characters in the +white+.
Oh, and thank you for the improved solution!