Page 1 of 1

Something wrong with imagemagick and cl-base64

Posted: Thu Apr 02, 2009 6:43 am
by schoppenhauer
With the following code

Code: Select all

(defun stretched-base64-image (img)
  "Call ImageMagick to resize that file to 32x32. This is made for sbcl."
  (let*
      ((imagemagick (sb-ext:run-program "/usr/bin/convert" (list "-geometry" "32x32!" "-" "-")
					:input :stream :output :stream :error :stream :wait nil))
       (thread (sb-thread:make-thread #'(lambda ()
					  (labels ((beginread (cbyte cinput stream)
						     (if (eql cbyte 'EOF)
							 (make-array (list (length cinput))
								     :initial-contents (nreverse cinput)
								     :adjustable nil)
							 (progn
							   (beginread
							    (read-byte stream nil 'EOF)
							    (push cbyte cinput) stream)))))
					    (beginread (read-byte
							(sb-ext:process-output imagemagick)
							nil 'EOF)
						       nil
						       (sb-ext:process-output imagemagick)))))))
    (write-sequence img (sb-ext:process-input imagemagick))
    (finish-output (sb-ext:process-input imagemagick))
    (close (sb-ext:process-input imagemagick))
    (cl-base64:usb8-array-to-base64-string (sb-thread:join-thread thread))))
I want to convert an Image, which I already loaded into a byte-vector, using

Code: Select all

(with-open-file (in file :element-type '(unsigned-byte 8)) 
	(let* ((length (file-length in))
	       (content (make-array (list length)
				    :element-type '(unsigned-byte 8)
				    :adjustable nil)))
	  (read-sequence content in)
	  content))
into a byte-vector with a smaller image and return this byte-vectore as a base64-string. I am working with SBCL. SBCL complains about the line

Code: Select all

 (read-byte stream nil 'EOF)
with

Code: Select all

; file: /tmp/fileAJJi3Q.lisp
; in: DEFUN STRETCHED-BASE64-IMAGE
;     (READ-BYTE STREAM NIL 'MY-PACKAGE::EOF)
; --> BLOCK IF LET IF SB-IMPL::EOF-OR-LOSE IF ERROR 
; ==>
;   STREAM
; 
; note: deleting unreachable code
; 
; compilation unit finished
;   printed 1 note
but this shouldnt be a problem - as far as I read the documentation http://www.ai.mit.edu/projects/iiip/doc ... -byte.html.

I know there is cl-magick but I couldnt find a function to convert byte-vectors into byte-vectors, and I have my reasons why I load the files into byte-vectors, and dont access the files directly (and as written below, this shouldnt be the problem here).

The whole code seems to work, but returns strange output, when running convert manually, and then encoding it with uuencode --base64, something completely different results.

To find the error, I replaced 32x32! by 1x1! and removed the base64-encoding. The resulting vector is correct - when I convert the file manually and look at the bytes written, the result is the same. Only the base64-encoded result is different.

So there must be something wrong with cl-base64. Any suggestions?

EDIT:Ok, I have made my own Base64-Implementation, and with it, it works:

Code: Select all

(defun get-base64-char-for-number (i)
  (declare (type (integer 0 63) i))
  (elt "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/" i))

(defun base64-encode-threebytes (byte1 byte2 byte3)
  (declare (type (unsigned-byte 8) byte1 byte2 byte3))
  (coerce
   (list
    (get-base64-char-for-number (logand #b111111 (ash byte1 -2)))
    (get-base64-char-for-number (logand #b111111 (+ (ash (ash byte1 6) -2) (ash byte2 -4))))
    (get-base64-char-for-number (logand #b111111 (+ (ash (ash byte2 4) -2) (ash byte3 -6))))
    (get-base64-char-for-number (logand #b111111 (ash (ash byte3 2) -2)))) 'string))  


(defun base64-encode-bytelist (bytelist &optional (ret ""))
  (if bytelist
      (if (cdr bytelist)
	  (if (cddr bytelist)
	      (base64-encode-bytelist
	       (cdddr bytelist)
	       (concatenate 'string
			    ret
			    (base64-encode-threebytes
			     (car bytelist)
			     (cadr bytelist)
			     (caddr bytelist))))
	      ;;else (genau zwei elemente)
	      (concatenate 'string ret			   
			   (base64-encode-threebytes
			    (car bytelist)
			    (cadr bytelist)
			    0)
			   "="))
	  ;;else (genau ein element)
	  (concatenate 'string ret			   
		       (base64-encode-threebytes
			(car bytelist) 0 0)
		       "=="))
      ;;else (kein element)
      ret))
I can live with that, but ... how can this be? Why does cl-base64 return something else?

Re: Something wrong with imagemagick and cl-base64

Posted: Thu Apr 02, 2009 9:34 am
by ramarren
You are passing a general array while cl-base64 is expecting (unsigned-byte 8) array, and it has declared (safety 0) so it doesn't notice. Change your make-array in stretched-base64-image to:

Code: Select all

(make-array (list (length cinput))
            :initial-contents (nreverse cinput)
            :adjustable nil
            :element-type '(unsigned-byte 8))
and it should work.

Re: Something wrong with imagemagick and cl-base64

Posted: Thu Apr 02, 2009 1:21 pm
by schoppenhauer
Ah. Ok thank you.

I wonder if it is better to just keep my own little implementation of base64. Its certainly a little slower, but on the other hand I do not have to convert the List into an Array first, and i will have one dependency less.

I would like to implement the whole stuff implementation-independent, but actually I cannot find any library for command-execution which works on multiple implementations.

Re: Something wrong with imagemagick and cl-base64

Posted: Thu Apr 02, 2009 9:51 pm
by ramarren
There is trivial-shell, but I don't know how good it is for this purpose.

Re: Something wrong with imagemagick and cl-base64

Posted: Fri Apr 03, 2009 5:26 am
by schoppenhauer
Thank you. No, I dont think that it is good for this purpose, as it works only with strings. I cannot pass a byte-vector as input and grab a byte-vector as output. Actually I cannot even see how to easily patch the code for this purpose.

I could try to use flexi-streams and convert the byte-vector to a string, but this would be only a hack. I could use uudecode and uuencode around the convert-command, but this would also be a hack.

So I maybe better concentrate on SBCL for this purpose.

Re: Something wrong with imagemagick and cl-base64

Posted: Fri Apr 03, 2009 7:27 am
by ramarren
Regarding the original post, the function to read an image from memory into an ImageMagick wand seems to be MagickReadImageBlob, so it is possible to use the API to do this.

Another somewhat portable way would be to go through a temporary file. It should be also possible to do the resize using only Lisp code. I have even written some time ago a pure-Lisp png reader, although it is rather slow and not tested much. There are some libpng bindings, but that would obviously require foreign code. And there is ZPNG for writing. This would leave only the resizing algorithm to be implemented.

Re: Something wrong with imagemagick and cl-base64

Posted: Sun Apr 05, 2009 1:59 pm
by schoppenhauer
Thank you. I used lisp-magick. Modified my code:

Code: Select all

(defun stretched-base64-image (img)
  "Call ImageMagick to resize that file to 32x32."
  (lisp-magick:with-magick-wand (mywand)
    (lisp-magick::magick-read-image-blob mywand img)
    (lisp-magick::magick-resize-image mywand 32 32 #x00000000 1d0)
    (base64-encode-byteseq (lisp-magick::magick-get-image-blob mywand))))
It now also works under CLISP.

Lisp-Magick has a strange license: http://www.nil.at/software/lisp-magick.html

Re: Something wrong with imagemagick and cl-base64

Posted: Sun Apr 12, 2009 1:29 pm
by Wodin
schoppenhauer wrote:[...]Lisp-Magick has a strange license: http://www.nil.at/software/lisp-magick.html
This just looks like the normal BSD License (without the advertising clause). Although I do think "his contributors" sounds a bit funny.

See also the MIT License and the ISC License, which are similar.