Page 1 of 1

Would a truly 'Open' Genera succeed?

Posted: Sun Oct 04, 2009 11:32 pm
by TheGZeus
Granted, there are degrees of success, but with all the potential in that bundle of code, how much would come to fruition in the following scenario:
All of the Genera code is released under a BSD or GPL license.

To bring it to its full potential would probably be monumental, after all, there's a C compiler in there that's been undergoing bit-rot for a couple decades, but let's start small, with the straight-up lisp portions of it.

The current implementation, off of a lisp machine, runs on Alpha hardware, and is pretty intensive to set up on anything else as a result. Furthermore, it's running a virtual lisp machine and all the lisp code is running on _that_. So the first task is to get that virtual lisp machine running on x86, other chips can wait for now.
The development environment is legendary, especially for lisp. However, many emacs users are rather accustomed to various extensions and methodologies, and Zmacs might be a bit frustrating at first. Would an effort be made to port what needs to be ported? How much should be forgotten for what reason? Would it fragment the Emacs community? How long until someone has a howto for setting up SLIME/Genera?

Would the fact that you're running on a VM and a large amount of code comes with Genera give a performance/resource hit? Considering the cost of RAM at the time Symbolics was going, I doubt it, but that's yet to be tested in this imaginary world.

What would the first misguided development move be, once it was ported? Optimisation before updating the system? Separate it out into separate applications? Throw out the other programming languages?


Looking ahead further, what happens when you update that C compiler to the latest standard and GCC extensions? At what point does the OS it's running on become something close to a hindrance? If all your C applications/libraries run under Genera, even at a likely performance hit, what do you run outside it? I become confused thinking about this, as I understand that you can make applications written in different languages talk to each other in interesting and useful ways, but I don't know what they are...

The fact that I have all these questions, and others think that anyone using a truly open Genera is a pipe dream casts some doubt on it for me as well...

The fact that it's fully compliant with the CL standard, and includes an interface to X(not CLX, but it's there), CLim, and numerous libraries and extensions, it's already ahead of CLisp in many ways. It lacks threads, and we don't know how it performs, so it's behind SBCL on the pure speed front, at the least on modern processors. This is just looking at it as a runtime. Factor in the environment and I think we'd have a real contender on our hands if it was made to interface with modern *nix systems(I'm sure some things have changed a bit in the past 16+ years). It might even attract new users that wouldn't otherwise consider lisp. Emacs is one thing, but figuring out SLIME, a piece of software with no releases, the somewhat cryptic, limited, and always terse SBCL debugger(or in my experience less helpful CLisp debugger) and how to set all that up is too much for many people starting out.
Zmacs wouldn't help with the Emacs keyboard controls barrier, in fact the Super, Hyper, Meta, Control, Symbol system would increase that, really. Not to mention a number of other keys having totally different functions and names (Triangle??)... but the gains would be more obvious and instantanious.


This is pretty much all questions, right now. Symbolics Inc. still holds the copyright to Genera, though that company is really just one guy, selling and possibly fixing old lisp machines and selling Maxima.
Apparently an original Symbolics keyboard is 125usd.

Re: Would a truly 'Open' Genera succeed?

Posted: Mon Oct 05, 2009 11:39 am
by findinglisp
I think it would be pretty interesting, but I think making this really work well would be a large task. As I understand it, the current Alpha port is basically a C program emulating the original Symbolics CPU all the way down to the microcode. In other words, it's bit-for-bit identical in operation to the actual Symbolics hardware. Given how old the Symbolics hardware is, you can emulate it pretty fast on newer hardware and still get speeds that are as good or better than the original (sort of like emulating a complete Apple ][ 6502 processor on a modern Pentium 4).

If you really wanted to make this fly, you'd want to:
  1. Rewrite the lowest levels of the system such that you weren't doing any of the emulation any more. This would give you a big speed boost, but it would be a huge rewrite of the "OS" layer of the system (anything that had a hardware dependency).
  2. You'd want to bring the upper layers of the system up to modern times. While a lot of the application code (I assume zmacs, etc.) should be written in CL or zlisp, this is still a big task. I'd probably handle this by starting with a working system and just gradually replacing it over time. There is nothing "wrong" with the Genera UI, it's just been frozen in time since Symbolics closed down.
The big task there is the first. It would be interesting to see what it would take to do it, however.

As an aside, it might be easier to create a Genera-like system using Movitz and/or McCLIM/Climacs. These projects are very Genera-like and they have been written in modern CL from the start.

Re: Would a truly 'Open' Genera succeed?

Posted: Mon Oct 05, 2009 2:51 pm
by TheGZeus
findinglisp wrote: As an aside, it might be easier to create a Genera-like system using Movitz and/or McCLIM/Climacs. These projects are very Genera-like and they have been written in modern CL from the start.
Do you think these tasks would be easier if the code was available?
I've heard much woe about McCLIM from people who work with thinks like LispWorks regularly and find McCLIM to be a disaster in comparison. That is to say, incomplete, non-standard, and buggy.
How do you mean 'modern' CL? I realise alot of what's in Genera is Zetalisp at a certain level, but it implements the CL standard fully, if their white-papers are to be believed.
The standard has resisted modernisation at every turn, the only thing 'added' is threads, and each implementation handles those differently.
but your hypothetical plans are a crysalisation of what I had a vague idea of a good plan(is that a sentence?).
Rip out the lower C level, get it running on commodity hardware without emulation.
However, I think making the C level portable first would be a good plan.
Then anyone with access to a C compiler and an X server would be able to work with/on it.
Considering what small parts I've read on what they went through to get it running on Alpha were the words of one who'd experienced more than a modicum of pain as a result, it might hurt alot to get it running even in userspace.
This might get Movitz some new devs, and there's nothing wrong with Genera on commodity hardware with nothing between it and the user.
That makes the monumental task of updating that C compiler all the more important. More applications at the user's fingertips.
Porting GTK+ et al to CLIM would be... interesting(read: probably really F****** hard).
Then someone has to writ a C++ compiler, and that's... not gonna be fun.
template in your template

Re: Would a truly 'Open' Genera succeed?

Posted: Mon Oct 05, 2009 5:17 pm
by findinglisp
TheGZeus wrote:
findinglisp wrote: As an aside, it might be easier to create a Genera-like system using Movitz and/or McCLIM/Climacs. These projects are very Genera-like and they have been written in modern CL from the start.
Do you think these tasks would be easier if the code was available?
I'm not sure I understand the question. Which code? Genera? Yea, I'm assuming that Genera code would be available if you were to try to do any of this with Genera. Without the code, the only option would be to start with Movitz or McCLIM/Climacs. I'm suggesting that even with code available it might be better to start with a more modern codebase. That's purely speculation on my part, however, and not based on me knowing anything about the Genera code. I'm simply assuming that it's old and crufty.
I've heard much woe about McCLIM from people who work with thinks like LispWorks regularly and find McCLIM to be a disaster in comparison. That is to say, incomplete, non-standard, and buggy.
That might be, also. I have never hacked McCLIM, so I don't know how the code looks. That said, if you're going to do work, would you rather do it there, or on 20-year-old Genera code?
How do you mean 'modern' CL? I realise alot of what's in Genera is Zetalisp at a certain level, but it implements the CL standard fully, if their white-papers are to be believed.
Surely, Genera implements standard CL. The question is, how much of the Genera code is written in it? My hunch is that most of the OS-level code and standard utilities is written in Zetalisp with the option of writing user programs in CL. I'm guessing that if you were to start hacking Genera at the OS level, you'd be writing a lot of Zetalisp and there would be almost no way around that.
The standard has resisted modernisation at every turn, the only thing 'added' is threads, and each implementation handles those differently.
IMO, if you were going to do a modern version, you'd target a specific CL implementation first (say SBCL), and you'd use whatever facilities it offered without worrying about portability first.
but your hypothetical plans are a crysalisation of what I had a vague idea of a good plan(is that a sentence?).
Rip out the lower C level, get it running on commodity hardware without emulation.
However, I think making the C level portable first would be a good plan.
Then anyone with access to a C compiler and an X server would be able to work with/on it.
Considering what small parts I've read on what they went through to get it running on Alpha were the words of one who'd experienced more than a modicum of pain as a result, it might hurt alot to get it running even in userspace.
I think you're in for a world of pain using the Genera code base at all. That isn't to suggest the pain isn't worth it, but realize that you have a huge codebase here with 20 years of bitrot.
This might get Movitz some new devs, and there's nothing wrong with Genera on commodity hardware with nothing between it and the user.
That makes the monumental task of updating that C compiler all the more important. More applications at the user's fingertips.
Porting GTK+ et al to CLIM would be... interesting(read: probably really F****** hard).
Then someone has to writ a C++ compiler, and that's... not gonna be fun.
IMO, there is little reason to run anything but Lisp applications on this. IMO, I would actually prefer that this not run "on the metal" but rather on top of a standard OS. I'd be happy if I could get a Lisp machine "application" running in a Window on Linux with all the development tools, etc. (Think Squeak Smalltalk here.)

Re: Would a truly 'Open' Genera succeed?

Posted: Tue Oct 06, 2009 11:34 am
by rjs
The part of OpenGenera that emulates the Ivory processor has already been ported to x86_64, there is a description of how to use it here. You still need the OpenGenera image though.

Another route to a more modern lisp machine could be to use the CADR emulator. The LMI source tree has been posted on the web which adds a TCP/IP stack to the MIT sources and looks as if it should still build for the CADR as well as for the LMI Lambda.

If you really want to run directly on modern hardware then I think Movitz is a better starting point. A halfway house could be to extend it to run on top of Xen, you would then need fewer device drivers in lisp.

Re: Would a truly 'Open' Genera succeed?

Posted: Tue Oct 06, 2009 6:02 pm
by findinglisp
rjs wrote:If you really want to run directly on modern hardware then I think Movitz is a better starting point. A halfway house could be to extend it to run on top of Xen, you would then need fewer device drivers in lisp.
Ah, good point. Yes, IMO, I really can't imagine completely throwing away my standard OS (Linux or Windows or whatnot) for a Lisp Machine OS of some sort. As you point out, the driver support is the Achilles heel of any operating system. Virtualization, whether Xen, KVM, or VMware, allows you to get away with simple mass storage and Ethernet support, however. In that sense, Movitz becomes much more interesting.

Re: Would a truly 'Open' Genera succeed?

Posted: Wed Oct 07, 2009 12:36 am
by TheGZeus
Yeah, I was more interested if it would succeed as a lisp implementation/development environment.

Even with that virtualisation involved, how's the performance? Has anyone done any tests? I'd do some, but I pulled a boner and installed i386 instead of amd64 on my desktop.
I'm gonna reinstall when I get any new hardware, as a 'while I'm at it' afternoon project.

Re: Would a truly 'Open' Genera succeed?

Posted: Wed Oct 07, 2009 10:28 am
by findinglisp
Good question. I'm still running a 32-bit x86 environment, too. I need to update it to 64-bit but haven't gotten there yet.