Copy-on-Write Mapping of Memory - is it possible?

[schoppenhauer]

Whjat I was wondering since a long time now is whether it is possible to map a part of the current heap memory onto another one in a Copy-On-Write-Manner, like it is done when fork(2)-ing under Linux, just that the same process can request it explicitly.

In theory, using some magic with libsigsegv should make it possible to do something like "copy on read" - apply libsigsegv on a part of the memory which is allocated but not written yet, and copy the pages when they are accessed. With even more magic, one may redirect this access to the previous memory part. But that sounds ... slow.

It is not that I need something like that. It is just that it sounds very useful, especially when you have a lot of objects that only differ insignificantly, or when you want to copy some large object quickly. And it sounds as if all the parts needed for this were there. So, is something like this in Linux? Or in any other Unix? Or any other System?

[findinglisp]

Linux has shared memory with copy on write (see the MAP_SHARED flag to mmap, I think). I'm not an expert on shared memory programming.

That said, sharing the heap like this would likely be fruitless. Fork uses shared memory because C programs that fork are either spawning versions of themselves to handle things like sockets or are about to exec another program on top of themselves. Copy-on-write optimizes those cases fairly nicely.

With Lisp, if you have a heap that's being GC'd, all those pages that the GC touches will probably be written to (mark bits, copying collectors, etc.), thus destroying the advantage of the shared memory. SBCL has a heap that is not GC'd, and that might be a candidate for what you want. Note that lack of GC doesn't necessarily mean lack of writing to a page at all, so I'm not sure if it would actually fulfill the requirement. Most other Lisps have simpler GC schemes without a permanent heap, however, so they would probably not work.

Finally, note that there is no way to configure SBCL to use this (from what I know at this point). You'd have to hack it.

[schoppenhauer]

This was not an SBCL question, I am not sure how come that you talk about it. And as I pointed out, I just wondered whether this was possible with some system. Trivially, a Copying GC cannot make an advantage of copy-on-write pages directly (however, indirectly, it could optimize some parts, like the parts that stay constant).

MAP_SHARED is not really copy-on-write in the sense that it just keeps the changings of the files in the file cache - which is also not guaranteed, according to the manpage. After all I do not think that this has anything to with what I want. And actually, I do not know any option to mmap doing this, though looking at the manpage carefully.

And I also do not think that it is fruitless. Linux uses it to optimize copying of large, similar structures. Such structures occur as well in programming.

[findinglisp]

I understand it wasn't an SBCL question. I assumed that it was a Lisp question, though, right?

My only point was that, in general, GC's fight with copy-on-write because they typically store things like mark bits close to the objects they identify, and thus they have a tendency to write something to just about every page whenever they do a GC pass. That can be avoided by storing mark bits outside the heap, etc., but you have to be deliberate about it.

If you were looking for an existing Lisp that might help you do this, the only one I could think of that might have a GC that would place nice would be SBCL (and even then there's probably a lot of work). That was how I came to suggest it. If that's not the case, then forget I said it.

If you're just looking to use a CoW memory region from a Lisp, you can access the Linux API through FFI (assuming you can find a Linux API that does what you want, which it sounds like MAP_SHARED doesn't). Any memory you allocated in that way would have to be managed manually, however. It wouldn't be part of the Lisp heap.