cffi-clutter

[ramarren]

Hello, forum

In the last few days I have been fighting the "no way to do GUI in CL" problem by creating the bindings to Clutter Toolkit. It is a rather neat base for creating OpenGL based user interfaces. It is really a base though, as it provides no complex widgets, but it gives a signal-based event system and rendering management. It also quite intensively uses the GObject system, but it turned out to be much easier to manipulate from CL than I thought. There are examples how to subclass a GObject from Lisp, including implementing interfaces. It requires about as much boilerplate as in C, unfortunately, and I can't think of a good way to get rid of it.

The bindings are on github. There are two problems remaining that I am not sure how to solve. First, there are absolute paths in grovel/bindings file, and I am not sure how the get rid of those. Especially include directories, cffi-grovel seems not to allow `pkg-config` calls.

The second issue is more stylistic. I have semi-autogenerated the bindings (ie. autogenerated from a heavily hand-edited input files using a minimal subset of C), and given all generated function names a '%' prefix. But I don't think that there is much point in wrapping foreign Clutter objects in Lisp-side objects, so the public interface right now requires using those '%' functions, which is probably bad style, since those are usually used to signify functions internal to implementation. On the other hand, I don't want to get rid of '%' globally, because it makes wrapping those functions which use out-pointer C idiom annoying. On the other other hand as I written before, Clutter is low level enough that those function can be, in a sense, considered internal anyway.

Oh, well. I would appreciate any input on those issues, and if someone would test the bindings and check if I haven't somehow make them work on my machine only (other than the paths issues, which have to be modified for every installation). Note that the bindings are for recently released version 0.9.6, which breaks API compatibility with 0.8 series. I have no idea what versions are in package managers. In portage there is none, so I compiled Clutter from sources, which only compounds the paths issue.

[dmitry_vk]

That sounds interesting.
I've been coding my cl-gtk2 binding for some time and have some experience working with GObject.
It looks like that you do not use the GObject system properly. First, it has a first-class support for using closures as callbacks (no need to defcallback every callback type and generic type-safe way to invoke/emit signals using GValue and GClosure). This lets you pass a closure as a signal handler and have it properly garbage collected.
Also, it does not seem that you use the GObject memory management to properly garbage-collect GObjects.
Then, you do not need to grovel for enums and flags, objects, properties and signals — GObject has it all exposed. The only things that require grovelling are:
1) methods/functions
2) vtable structures for interfaces/objects (needed for subclassing GObjects and implementing GInterfaces)
3) some properties that are for some reason not exposed through GObject type system
4) structures
Btw, I have a pretty good (as far as I think ) GObject binding at cl-gtk2. I think it is worth looking at it (however, subclassing objects is not quite supported yet, but implementing GInterfaces works.

[ramarren]

It looks like that you do not use the GObject system properly.

That is more than likely, since I haven't really looked into it before, and only read about its features in minimal way necessary to implement Clutter features one after another.

First, it has a first-class support for using closures as callbacks (no need to defcallback every callback type and generic type-safe way to invoke/emit signals using GValue and GClosure). This lets you pass a closure as a signal handler and have it properly garbage collected.

I noticed that, but I have an irrational hate and fear of GValue , and don't want to write a proper marshaller. And before I understood how that works I had all those callbacks defined, and then it didn't seem worth bothering. And I am not sure about performance issues, although there probably is no point in worrying at this point with all the other things flying around already in GObject alone... not to mention I trampoline to Lisp functions anyway. At least those do get garbage collected.

Now that I think of it, I am not sure how much would moving parts of that mechanism into C-land would gain me anyway. I would have to write a marshaller in Lisp, and get hit from all those calls to g_value_get/set_something, and this way I just leave type translation to FFI with defcallback'ed thing serving as a marshaller of sorts.

Also, it does not seem that you use the GObject memory management to properly garbage-collect GObjects.

I actually checked for that, and GObjects do get collected. Clutter does g_object_ref_sink() when an actor is parented and g_object_unref() when they are unparented, so actors are killed when they are dropped of stage, and other things are cleaned up explicitly in main wrapper. I chose not to even try doing something with Lisp-side proxy objects and finalization.

Then, you do not need to grovel for enums and flags, objects, properties and signals — GObject has it all exposed. The only things that require grovelling are:
1) methods/functions
2) vtable structures for interfaces/objects (needed for subclassing GObjects and implementing GInterfaces)
3) some properties that are for some reason not exposed through GObject type system
4) structures

Right, I wrote that property-typemap.lisp monstrosity before I realized that the entire point of GObject properties is introspection, got to rewrite that. Enums/flags need mapping from C-names to Lisp-names anyway, even if I used GObject to map to them from values, so I might just as well grovel for them, saving a FFI call per translation.

Btw, I have a pretty good (as far as I think ) GObject binding at cl-gtk2. I think it is worth looking at it (however, subclassing objects is not quite supported yet, but implementing GInterfaces works.

Subclassing is not hard, but a bit messy, since you can't really avoid grovelling (unless vtable offsets are hardcoded, and that never ends well) and defcallback'ing a lot, since virtual function translations need outright function pointers, not GClosures. I think.

Anyway, it seems that every GTK binding has its own GObject, or at least its subset, binding. Someone should create a stand alone complete one to be used for binding of systems using it. Someone who is not me, because I don't really understand that well how it works and I am not sure if I really want to, the entire thing is a poster child for Greenspun's Tenth Rule. I'm just happy to see rotating triangles which don't leak memory all over the place. I hope they don't, anyway.

[dmitry_vk]

I actually checked for that, and GObjects do get collected

Actually, what I meant is that you do not use GC, but manually specify what objects should be collected.

Enums/flags need mapping from C-names to Lisp-names anyway, even if I used GObject to map to them from values, so I might just as well grovel for them, saving a FFI call per translation.

Enums/Flags have "nicknames" which are good as Lisp names (they even use dashes as word separators ). E.g., GTK_TYPE_TOPLEVEL has nickname "toplevel". Properties names are good for Lisp names.

Subclassing is not hard, but a bit messy, since you can't really avoid grovelling (unless vtable offsets are hardcoded, and that never ends well) and defcallback'ing a lot, since virtual function translations need outright function pointers, not GClosures. I think.

Yes, it's not hard. All the mess with defcallbacks and vtables can and should be hidden be macros. I actually have the vtable and corresponding functions generated by a macro. A good way to automatically obtain vtable offsets and signatures is to use GObjectIntrospection.

[gugamilare]

Neat! One more Lisp toy to play with

The second issue is more stylistic. I have semi-autogenerated the bindings (ie. autogenerated from a heavily hand-edited input files using a minimal subset of C), and given all generated function names a '%' prefix. But I don't think that there is much point in wrapping foreign Clutter objects in Lisp-side objects, so the public interface right now requires using those '%' functions, which is probably bad style, since those are usually used to signify functions internal to implementation. On the other hand, I don't want to get rid of '%' globally, because it makes wrapping those functions which use out-pointer C idiom annoying. On the other other hand as I written before, Clutter is low level enough that those function can be, in a sense, considered internal anyway.

Let me stick my nose and give my opinion. You should keep the preffix '%' for generated functions, but always create a function without it and only export those such functions. If you need to add type-checks and do some wrapping or even generalizing these functions a little bit, you won't need to remanufacture everything.

It's not hard to let the REPL do this job for you. You can use do-symbols, or fetch the function definitions from the file in a list (place " '( " in the REPL, copy the contents of the file to the REPL, put a " ) " to terminate and evaluate) and loop into that list to extract the function names and return a list of function definitions.

I have no idea what versions are in package managers. In portage there is none, so I compiled Clutter from sources, which only compounds the paths issue.

Ubuntu Jaunty (the last, and the one I use) have the Clutter version 0.9 and a quick search reveals that Debian have the version 0.8. The fact that gentoo doesn't have any version is, at least, uncommon; I have the felling that gentoo gets almost all packages first, specially small and unknown ones (no compilation is needed, and many developers use it).

[dmitry_vk]

Note that the bindings are for recently released version 0.9.6, which breaks API compatibility with 0.8 series. I have no idea what versions are in package managers. In portage there is none, so I compiled Clutter from sources, which only compounds the paths issue.

According to the blog post you mentioned, «This version is fully API and ABI incompatible with the previous 0.8 releases.» So API should not break.
I've cleaned up my GObject binding and I'm trying to use it to create binding for clutter. But I noticed that the clutter is rather picky about threading: if I compile a binding and call init in one thread, then start the main function from the REPL (compilation and evalution from REPL creates threads in SLIME), clutter prints messages "(<unknown>:20977): Clutter-CRITICAL **: clutter_id_pool_lookup: assertion `id < id_pool->array->len' failed". But if I call init from the same thread that I call main, it works.
It seems that you have experience with clutter; have you experienced this issue? If yes, how did you solve it (I didn't look very much at your sources)?

[ramarren]

Note that the bindings are for recently released version 0.9.6, which breaks API compatibility with 0.8 series. I have no idea what versions are in package managers. In portage there is none, so I compiled Clutter from sources, which only compounds the paths issue.

According to the blog post you mentioned, «This version is fully API and ABI incompatible with the previous 0.8 releases.» So API should not break.

Note: fully incompatible. It changes types of most functions and many are replaced outright with other interfaces.

I've cleaned up my GObject binding and I'm trying to use it to create binding for clutter. But I noticed that the clutter is rather picky about threading: if I compile a binding and call init in one thread, then start the main function from the REPL (compilation and evalution from REPL creates threads in SLIME), clutter prints messages "(<unknown>:20977): Clutter-CRITICAL **: clutter_id_pool_lookup: assertion `id < id_pool->array->len' failed". But if I call init from the same thread that I call main, it works.
It seems that you have experience with clutter; have you experienced this issue? If yes, how did you solve it (I didn't look very much at your sources)?

You have to call g_thread_init first, then clutter_threads_init, and then all Clutter calls need to be locked with clutter_threads_enter/leave.

[Harnon]

I was just wandering, are you using windows with clutter? Because i'm trying to build clutter but not managing it....
It appears that it includes glx.h (from mesa libgl) which then includes some x11 headers which means i'm going to have to build this on cygwin...

[ramarren]

I was just wandering, are you using windows with clutter? Because i'm trying to build clutter but not managing it....
It appears that it includes glx.h (from mesa libgl) which then includes some x11 headers which means i'm going to have to build this on cygwin...

I only tried building Clutter on Gentoo Linux. It seems rather Linux centric, but there are some blog posts which claim it is possible to build on Windows.

[Harnon]

lol. i followed the link and tried it, downloaded codeblocks and everything. 8 errors, syntax for that matter. Sigh...