help with the heap

[beren]

Hi again,

I have been using GSL (Gnu Scientific Library) inside lisp (GSLL) to do some mathematical calculations, and so far has been a real charm!
But now I am trying to make some serious stuff and I need to invert matrices of about 1500x1500 elements.
At this point when I try to do the (make-marray...) ;;<- is like make-array, but in the GSL way
with those dimensions I get my heap exhausted (using SBCL and CLISP at least)

I tryed changing the dynamical memory (--dynamic-space-size and -m respectively) with no success.
So, I can think of 2 ways you could really help me:
1) Is there a way I could increase the heap space as much as I need?
2) Does GSL(L) have support for sparse matrix? From the GSL documentation I will think that it does not, but maybe some one of you have a clue about.

Thank you for the help, I would really hate to have to go back to C!! I am enjoying Lisp so much!
Cheers!!

[dmitry_vk]

Is it Lisp's heap of C's heap being exhausted?

[beren]

Uhm... I do not know.
I think that with SBCL the GSL callings are made without copying the memory block, so I think It must be just the Lisp's one. But I am not sure. The actual error I get is:

* Heap exhausted during allocation: 76496896 bytes available, 100446392 requested.

With no further references. How do I bactrace that?

TY!

[dmitry_vk]

Uhm... I do not know.
I think that with SBCL the GSL callings are made without copying the memory block, so I think It must be just the Lisp's one. But I am not sure. The actual error I get is:

* Heap exhausted during allocation: 76496896 bytes available, 100446392 requested.

With no further references. How do I bactrace that?

TY!

Type (room) at REPL, it will give details about memory usage. You might have a memory leak somewhere or high memory usage or lisp has too less memory (for SBCL, --dynamic-space-size command-line option controls the size of the heap) or there is something else. If there is a memory leak or high memory usage, SBCL's allocation profiler might help determine the function that causes it.

[beren]

Uhm... I do not know.
I think that with SBCL the GSL callings are made without copying the memory block, so I think It must be just the Lisp's one. But I am not sure. The actual error I get is:

* Heap exhausted during allocation: 76496896 bytes available, 100446392 requested.

With no further references. How do I bactrace that?

TY!

Type (room) at REPL, it will give details about memory usage. You might have a memory leak somewhere or high memory usage or lisp has too less memory (for SBCL, --dynamic-space-size command-line option controls the size of the heap) or there is something else. If there is a memory leak or high memory usage, SBCL's allocation profiler might help determine the function that causes it.

I have tryed that, but the funny thing is that, for example,
sbcl --dynamic-space-size 100 and
sbcl --dynamic-space-size 1000
gives me the exact same result:

Code: Select all

* (room)

Dynamic space usage is:   25,399,288 bytes.
Read-only space usage is:      3,512 bytes.
Static space usage is:         2,160 bytes.
Control stack usage is:        1,400 bytes.
Binding stack usage is:          344 bytes.
Control and binding stack usage is for the current thread only.
Garbage collection is currently enabled.

Breakdown for dynamic space:
   9,572,392 bytes for    11,840 code objects.
   3,675,040 bytes for    84,674 instance objects.
   3,188,000 bytes for    57,723 simple-vector objects.
   3,150,952 bytes for   393,869 cons objects.
   5,833,392 bytes for   141,156 other objects.
  25,419,776 bytes for   689,262 dynamic objects (space total.)


I have not tryied the allocation profiler yet, but I am pretty sure that the fault is from my big marray. I dont think there is any leakage (hope GSL handles that well!!) but still Lisp does not let me allocate all the memory I want. Why?

Any other clues??
TY

[Wodin]

If you don't get an answer here, try the sbcl-help list:

https://lists.sourceforge.net/lists/listinfo/sbcl-help

[Liam]

I am the author of GSLL.

In SBCL, the arrays are not duplicated between the Lisp side and the C side; the native Lisp array is used in GSL. In all other implementations, it is duplicated. I have successfully used GSLL on a 3000x3000 matrix, so it is not a GSLL/GSL/SBCL problem that you are experiencing. I don't think GSL has the ability to handle sparse matrices (and thus of course GSLL doesn't) but I know people are interested in it and working on ideas for Lisp. If you want to pursue a C solution, post to the GSL mailing list for advice (which may also spur some development there if enough people are interested). You might also want to joint the gsll-devel mailing list, because there are subscribers there who have used large matrices successfully.