Something wrong with imagemagick and cl-base64

Discussion of Common Lisp

Something wrong with imagemagick and cl-base64

Postby schoppenhauer » Thu Apr 02, 2009 6:43 am

With the following code
Code: Select all
(defun stretched-base64-image (img)
  "Call ImageMagick to resize that file to 32x32. This is made for sbcl."
  (let*
      ((imagemagick (sb-ext:run-program "/usr/bin/convert" (list "-geometry" "32x32!" "-" "-")
               :input :stream :output :stream :error :stream :wait nil))
       (thread (sb-thread:make-thread #'(lambda ()
                 (labels ((beginread (cbyte cinput stream)
                       (if (eql cbyte 'EOF)
                      (make-array (list (length cinput))
                             :initial-contents (nreverse cinput)
                             :adjustable nil)
                      (progn
                        (beginread
                         (read-byte stream nil 'EOF)
                         (push cbyte cinput) stream)))))
                   (beginread (read-byte
                     (sb-ext:process-output imagemagick)
                     nil 'EOF)
                         nil
                         (sb-ext:process-output imagemagick)))))))
    (write-sequence img (sb-ext:process-input imagemagick))
    (finish-output (sb-ext:process-input imagemagick))
    (close (sb-ext:process-input imagemagick))
    (cl-base64:usb8-array-to-base64-string (sb-thread:join-thread thread))))

I want to convert an Image, which I already loaded into a byte-vector, using
Code: Select all
(with-open-file (in file :element-type '(unsigned-byte 8))
   (let* ((length (file-length in))
          (content (make-array (list length)
                :element-type '(unsigned-byte 8)
                :adjustable nil)))
     (read-sequence content in)
     content))

into a byte-vector with a smaller image and return this byte-vectore as a base64-string. I am working with SBCL. SBCL complains about the line
Code: Select all
 (read-byte stream nil 'EOF)

with
Code: Select all
; file: /tmp/fileAJJi3Q.lisp
; in: DEFUN STRETCHED-BASE64-IMAGE
;     (READ-BYTE STREAM NIL 'MY-PACKAGE::EOF)
; --> BLOCK IF LET IF SB-IMPL::EOF-OR-LOSE IF ERROR
; ==>
;   STREAM
;
; note: deleting unreachable code
;
; compilation unit finished
;   printed 1 note

but this shouldnt be a problem - as far as I read the documentation http://www.ai.mit.edu/projects/iiip/doc ... -byte.html.

I know there is cl-magick but I couldnt find a function to convert byte-vectors into byte-vectors, and I have my reasons why I load the files into byte-vectors, and dont access the files directly (and as written below, this shouldnt be the problem here).

The whole code seems to work, but returns strange output, when running convert manually, and then encoding it with uuencode --base64, something completely different results.

To find the error, I replaced 32x32! by 1x1! and removed the base64-encoding. The resulting vector is correct - when I convert the file manually and look at the bytes written, the result is the same. Only the base64-encoded result is different.

So there must be something wrong with cl-base64. Any suggestions?

EDIT:Ok, I have made my own Base64-Implementation, and with it, it works:
Code: Select all
(defun get-base64-char-for-number (i)
  (declare (type (integer 0 63) i))
  (elt "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/" i))

(defun base64-encode-threebytes (byte1 byte2 byte3)
  (declare (type (unsigned-byte 8) byte1 byte2 byte3))
  (coerce
   (list
    (get-base64-char-for-number (logand #b111111 (ash byte1 -2)))
    (get-base64-char-for-number (logand #b111111 (+ (ash (ash byte1 6) -2) (ash byte2 -4))))
    (get-base64-char-for-number (logand #b111111 (+ (ash (ash byte2 4) -2) (ash byte3 -6))))
    (get-base64-char-for-number (logand #b111111 (ash (ash byte3 2) -2)))) 'string)) 


(defun base64-encode-bytelist (bytelist &optional (ret ""))
  (if bytelist
      (if (cdr bytelist)
     (if (cddr bytelist)
         (base64-encode-bytelist
          (cdddr bytelist)
          (concatenate 'string
             ret
             (base64-encode-threebytes
              (car bytelist)
              (cadr bytelist)
              (caddr bytelist))))
         ;;else (genau zwei elemente)
         (concatenate 'string ret            
            (base64-encode-threebytes
             (car bytelist)
             (cadr bytelist)
             0)
            "="))
     ;;else (genau ein element)
     (concatenate 'string ret            
             (base64-encode-threebytes
         (car bytelist) 0 0)
             "=="))
      ;;else (kein element)
      ret))

I can live with that, but ... how can this be? Why does cl-base64 return something else?
Sorry for my bad english.
Visit my blog http://blog.uxul.de/
schoppenhauer
 
Posts: 99
Joined: Sat Jul 26, 2008 2:30 pm
Location: Germany

Re: Something wrong with imagemagick and cl-base64

Postby Ramarren » Thu Apr 02, 2009 9:34 am

You are passing a general array while cl-base64 is expecting (unsigned-byte 8) array, and it has declared (safety 0) so it doesn't notice. Change your make-array in stretched-base64-image to:

Code: Select all
(make-array (list (length cinput))
            :initial-contents (nreverse cinput)
            :adjustable nil
            :element-type '(unsigned-byte 8))


and it should work.
Ramarren
 
Posts: 613
Joined: Sun Jun 29, 2008 4:02 am
Location: Warsaw, Poland

Re: Something wrong with imagemagick and cl-base64

Postby schoppenhauer » Thu Apr 02, 2009 1:21 pm

Ah. Ok thank you.

I wonder if it is better to just keep my own little implementation of base64. Its certainly a little slower, but on the other hand I do not have to convert the List into an Array first, and i will have one dependency less.

I would like to implement the whole stuff implementation-independent, but actually I cannot find any library for command-execution which works on multiple implementations.
Sorry for my bad english.
Visit my blog http://blog.uxul.de/
schoppenhauer
 
Posts: 99
Joined: Sat Jul 26, 2008 2:30 pm
Location: Germany

Re: Something wrong with imagemagick and cl-base64

Postby Ramarren » Thu Apr 02, 2009 9:51 pm

There is trivial-shell, but I don't know how good it is for this purpose.
Ramarren
 
Posts: 613
Joined: Sun Jun 29, 2008 4:02 am
Location: Warsaw, Poland

Re: Something wrong with imagemagick and cl-base64

Postby schoppenhauer » Fri Apr 03, 2009 5:26 am

Thank you. No, I dont think that it is good for this purpose, as it works only with strings. I cannot pass a byte-vector as input and grab a byte-vector as output. Actually I cannot even see how to easily patch the code for this purpose.

I could try to use flexi-streams and convert the byte-vector to a string, but this would be only a hack. I could use uudecode and uuencode around the convert-command, but this would also be a hack.

So I maybe better concentrate on SBCL for this purpose.
Sorry for my bad english.
Visit my blog http://blog.uxul.de/
schoppenhauer
 
Posts: 99
Joined: Sat Jul 26, 2008 2:30 pm
Location: Germany

Re: Something wrong with imagemagick and cl-base64

Postby Ramarren » Fri Apr 03, 2009 7:27 am

Regarding the original post, the function to read an image from memory into an ImageMagick wand seems to be MagickReadImageBlob, so it is possible to use the API to do this.

Another somewhat portable way would be to go through a temporary file. It should be also possible to do the resize using only Lisp code. I have even written some time ago a pure-Lisp png reader, although it is rather slow and not tested much. There are some libpng bindings, but that would obviously require foreign code. And there is ZPNG for writing. This would leave only the resizing algorithm to be implemented.
Ramarren
 
Posts: 613
Joined: Sun Jun 29, 2008 4:02 am
Location: Warsaw, Poland

Re: Something wrong with imagemagick and cl-base64

Postby schoppenhauer » Sun Apr 05, 2009 1:59 pm

Thank you. I used lisp-magick. Modified my code:

Code: Select all
(defun stretched-base64-image (img)
  "Call ImageMagick to resize that file to 32x32."
  (lisp-magick:with-magick-wand (mywand)
    (lisp-magick::magick-read-image-blob mywand img)
    (lisp-magick::magick-resize-image mywand 32 32 #x00000000 1d0)
    (base64-encode-byteseq (lisp-magick::magick-get-image-blob mywand))))


It now also works under CLISP.

Lisp-Magick has a strange license: http://www.nil.at/software/lisp-magick.html
Sorry for my bad english.
Visit my blog http://blog.uxul.de/
schoppenhauer
 
Posts: 99
Joined: Sat Jul 26, 2008 2:30 pm
Location: Germany

Re: Something wrong with imagemagick and cl-base64

Postby Wodin » Sun Apr 12, 2009 1:29 pm

schoppenhauer wrote:[...]Lisp-Magick has a strange license: http://www.nil.at/software/lisp-magick.html

This just looks like the normal BSD License (without the advertising clause). Although I do think "his contributors" sounds a bit funny.

See also the MIT License and the ISC License, which are similar.
Wodin
 
Posts: 56
Joined: Sun Jun 29, 2008 8:16 am


Return to Common Lisp

Who is online

Users browsing this forum: Yahoo [Bot] and 3 guests

cron