I don't know if it is pretty or not, but calling gcc to create a shared object and linking to it is what ECL does. A friend of one friend of mine also used to do this in a program written in C# (but nothing to do with Lisp).makia wrote:I'm doing small lisp like compiler (to x86 asm) and what is bothering me is "runtime compilation", it's simple enough to do simple static C-like compiler with basic Lisp stuff (C runtime/gc and Lisp compiler emiting x86 asm) but i'm not sure how to do runtime compilation and all the problems that goes with it.
I assume that I can start extern gcc assembler each time from Lisp REPL and use C lib dynamic linker/loader (not pretty ?),
yes, i know that, but for example in ECL when you type function in REPL i doubt that you get compiled function (it does not invoke GCC), so if I want something like that I need to invoke gcc on every REPL input or implement interpreter beside compilerfindinglisp wrote:Yup, take a look at ECL. When you call COMPILE-FILE in ECL, it effectively spits out a temporary .c file, invokes GCC on it, and then when you LOAD it, it dynamically loads the resulting dynamic lib.
Most other Lisp compilers don't work this way, BTW. Others, like SBCL, have their own compiler/assembler internal to the Lisp image and can generate assembly code directly into memory or to a file without the need of gcc.
I played only a little with assembly, but as far as I know you just mark a heap region as executable if necessary, depending on your operating system (see NX bit on Wikipedia, mprotect syscall on Linux) and jump to it. Of course you need an assembler to generate the machine code which can be placed in memory directly. Other than that you don't need "runtime support to push code into heap" other than normal memory manipulation tools.makia wrote:I'm more interested in SBCL/CCL way, to assemble things directly into heap but i'm not sure how to implement this, obvious compiler need to have runtime support to push code into heap ... so it looks like that i dont know how to connect runtime and compiler to have working environment
You'll also need garbage collector support for moving and reclaiming the code. Moving code in memory might (or might not, depending on linkage strategy) cause text relocations of code that references the generated code. If the generated code closes over some variables, then GC also should support walking over closures.Ramarren wrote: I played only a little with assembly, but as far as I know you just mark a heap region as executable if necessary, depending on your operating system (see NX bit on Wikipedia, mprotect syscall on Linux) and jump to it. Of course you need an assembler to generate the machine code which can be placed in memory directly. Other than that you don't need "runtime support to push code into heap" other than normal memory manipulation tools.
Indeed, when you type a function in the REPL (or just loads a file without compiling it) ECL transforms your code into bytecode, which it interprets later when necessary. ANSI Common Lisp does not require implementations to compile every function, that is why the functions compile-function and compile-file exist. But there is also nothing preventing you to compile every function using gcc either.makia wrote:yes, i know that, but for example in ECL when you type function in REPL i doubt that you get compiled function (it does not invoke GCC)
Re: Runtime compilation strategy
and things in that project 
It mentions some type of run-time compilation too.