LispForum

Posted: **Mon Aug 18, 2008 12:44 pm**

Has anyone had any experience with persistent object libraries? I've tried cl-prevalence, elephant, and rucksack, but none work on clisp 2.45, running on windows xp

. Is there any out there that actually work?

Oh, and i do realize that elephant uses uffi, which doesn't support clisp. I used the cffi-uffi-compat package that came with cffi. I eventually ran up against an error i couldn't understand, namely 'can't redefine standardized method. use call-next-method instead.' hmmm.
Also, rucksack doesn't support clisp either, but i modified the file package.lisp so as to not signal an error when clisp is used. Then i told it to use the clos package for clisp (the file package.lisp required each implementation to specify location of mop protocol). Then, i modified rucksack.lisp in order to make dummy with-lock, process-lock, and make-lock functions/macros, since clisp doesn't support multithreads. This time, the error that i ran up against was that file-position was attempting to position a stream past then end of the file. Argg...
And yes, it works on sbcl, but i want to use clisp!

Posted: **Mon Aug 18, 2008 1:56 pm**

A hah! I found an annoying implementation difference! On clisp, if file-position is given a number beyond it's eof, it signals an error, but if sbcl receives a number beyond it's eof, it just fills the non-used positions with a #\null character or 0 or whatever

I redefined file-position in the rucksack package. Wow, now the macro with-rucksack works

, now i have to see what's else is wrong, still getting that pesky end of file error!

Posted: **Mon Aug 18, 2008 2:43 pm**

Ok, so i found the problem, but am a little clueless. Here's the new file-position function, which assumes the stream is of type '(unsigned-byte 8), since it always is in rucksack.

Code: Select all

(defun file-position (stream &optional position)
	#-clisp (cl:file-positions stream position)
        #+clisp (if (not position)
                          (cl:file-position stream)
	                  (let ((eof-position (file-length stream)))
	                      (if (>= position eof-position)
                                   (progn 
                                          (format t "Before: stream-element-type is ~A~%" (stream-element-type stream))
	                                  (format t "EOF-Position is ~A current-file-position is ~A wanted-position is ~A~%" eof-position
	                                         (file-position stream) position)
	                              (loop repeat (1+ (- position eof-position))
	                                    for index = 1 then (1+ index)
	                                    do(format t "current-rep is ~A cur-file-length ~A~%" index
	                                              (file-length stream))
	                                    do(progn (print "writing") (print (write-byte 0 stream)))) 
	                               (format t "right-before-positioning: file-length is ~A~%" (file-length stream))
                                       (cl:file-position stream position) (format t "After~2%"))
                                   (cl:file-position stream position)))))

It appears that for one call tof ile-position, the file-length isn't increasing even though write-byte is called. The output for this particular case is (minus the "writing" 0 part, which indicates that the write has succeded, according to write-byte) :

Code: Select all

Before: stream-element-type is (unsigned-byte 8)
EOF-Position is 10 current-file-position is 3 wanted-position is 24

current-rep is 1 cur-file-length 10
current-rep is 2 cur-file-length 10
current-rep is 3 cur-file-length 10
current-rep is 4 cur-file-length 10
current-rep is 5 cur-file-length 10
current-rep is 6 cur-file-length 10
current-rep is 7 cur-file-length 10
current-rep is 8 cur-file-length 10
current-rep is 9 cur-file-length 11
current-rep is 10 cur-file-length 12
current-rep is 11 cur-file-length 13
current-rep is 12 cur-file-length 14
current-rep is 13 cur-file-length 15
current-rep is 14 cur-file-length 16
current-rep is 15 cur-file-length 17

right-before-position: file-length is 18

As you can see, i need an extra length of (24-10) +1 = 15 added, so the end file-length should be 10+15 = 25.
But, since for some reason it stays at 10 for a while, it doesn't work.
Anyone knows what's happening? Thx!

EDIT:I even tried restarting clisp and re-loading, i got the same error, which is something of a relief

Posted: **Mon Aug 18, 2008 5:51 pm**

Woops. I forgot to set the file-position to the end of the stream for this particular case

Modified Code:

Code: Select all

(defun file-position (stream &optional position)
  #-clisp  (cl:file-position stream position)
  #+clisp  (if (not position)
                          (cl:file-position stream)
	                  (let ((eof-position (file-length stream)))
	                      (if (>= position eof-position)
                                   (progn 
                                      (cl:file-position stream eof-position)
	                              (loop repeat (1+ (- position eof-position))
                                               do(write-byte 0 stream))
                                      (cl:file-position stream position)) 
                                   (cl:file-position stream position)))))

Hmm... It seems sbcl is way faster than the same test program in clisp. Wonder why?

Posted: **Mon Aug 18, 2008 10:50 pm**

OK, so i think i finally got rucksack working with clisp. Just a quick question, though. Is the above file-position function the most efficient implementation or can i do better. In addition, just wondering, is the file-length property figured out by counting through the file or is this property stored somehow in the stream?

Posted: **Mon Sep 01, 2008 7:55 am**

I don't know the answer to your question, but I'm impressed by your diligence in making this work with CLisp

Posted: **Mon Sep 01, 2008 9:55 am**

Thx! The changes were actually very very minor, its just finding them thats really annoying. Which reminds me, i should probably relate what i did to get it to work, for others.

Locate the package.lisp file.
At the top should be something like #-(...bunch of lisp implementations) (error "...")
Include clisp in the bunch of lisp implementations.

Then, go to the :use statement. Add #+clisp :clos to the list
Now, we need to add a new file-position function. In sbcl, if we try to position
the file beyond the eof, it will add #\Null characters or 0s for padding depending on
stream type.

Add following statement to defpackage statement
(:shadow :file-position)

Then, after the (in-package :rucksack) at the end of the file, add the following.

Code: Select all

(defun file-position (stream &optional position)
  #-clisp  (cl:file-position stream position)
  #+clisp  (if (not position)
               (cl:file-position stream)
	         (let ((eof-position (file-length stream)))
	            (if (>= position eof-position)
                      (progn 
                          (cl:file-position stream eof-position)
	                    (loop repeat (1+ (- position eof-position))
                                do(write-byte 0 stream))
                                      (cl:file-position stream position)) 
                      (cl:file-position stream position)))))

Locate the rucksack.lisp file
Rucksack uses locks. Since clisp doesn't contain these, you need to add dummy
lock macros. Should end up looking like this

Code: Select all

(defun make-lock (&key (name "lock"))
  #+allegro
  (mp:make-process-lock :name name)
  #+lispworks
  (mp:make-lock :name name)
  #+sbcl
  (sb-thread:make-mutex :name name)
  #+openmcl
  (ccl:make-lock name)
  #-(or allegro lispworks sbcl openmcl)
  nil)


(defmacro with-lock ((lock) &body body)
  #+allegro
  `(mp:with-process-lock (,lock) ,@body)
  #+lispworks
  `(mp:with-lock (,lock) ,@body)
  #+sbcl
  `(sb-thread:with-mutex (,lock) ,@body)
  #+openmcl
  `(ccl:with-lock-grabbed (,lock) ,@body)
  #-(or allegro lispworks sbcl openmcl) `(progn ,@body))


(defun process-lock (lock)
  #+lispworks
  (mp:process-lock lock)
  #+sbcl
  (sb-thread:get-mutex lock)
  #-(or sbcl lispworks)
  nil)


(defun process-unlock (lock)
  #+lispworks
  (mp:process-unlock lock)
  #+sbcl
  (sb-thread:release-mutex lock)
  #-(or sbcl lispworks)
  nil)

Locate the serialize.lisp file. Change the max-character-code function to the following.
The problem here is that in sbcl, it will return 0 if the string is empty, but clisp will return nil.
(defun max-character-code (string)
"Returns the highest character code in string."
#+clisp (if (string-equal string "") (return-from max-character-code 0))
(loop for char across string
maximize (char-code char)))

Finally, and this is a hack, we need to suppress clisp warnings. The thing is, when we have nested (with-rucksacks) for the same database, it tries to have more than one open stream to a file. If we don't suppress it, it will crash our program. I guess you could also try try catch blocks, but whatever.

In windows:create a shortcut to clisp by right clicking on clisp.exe and clicking create shortcut. Rename it as "click-appease" or something. Right click shortcut and click on properties. Under shortcut tab, rename target to something like following, replacing thing between parantheses by the pathname of your shortcut.
"C:\Program Files\clisp-2.45\clisp.exe" -on-error appease
Click OK, and your done. Use this shortcut when you use clisp and rucksack together.
No clue how to do it on other systems.

A much better way would probably be to create a global hashtable which has
the pathname as a key and an open stream or nil if there is none as the item. Something like that, anyways. Actually, you shouldn't have nested with-rucksacks for the same database in the first place, but you can figure out why the example signals a warning.

LispForum

Persistent Objects

Persistent Objects

Re: Persistent Objects

Re: Persistent Objects

Re: Persistent Objects

Re: Persistent Objects

Re: Persistent Objects

Re: Persistent Objects