Meta Lisp?

[xibalba]

Cross-posting from another forum:

And, does this idea make any sense, or am I talking out of my butt?

basic lisp = most basic features of a language
-extension a: operating system - low level (Forth?)
-extension b: advanced math - mid level (?) (Lush?)
-extension c: graphics - high level
-extension d: scripting - intended for embedded apps, or web pages (very fast, and hard to mess up, I guess? Can play nice with other languages?) (Shen?)

I think that a lot of people don't use Common Lisp because it's so freaking huge. The syntax is simple, but the (I think the word is semantics) semantics are absurd - there are so many hidden rules and hidden effects, and very little is obvious to someone reading a book (like me), entering the example programs into an interpreter, and then screwing around with those examples to try to break them (to determine what they can and can't do). It's like PEMDAS - we only need PEMDAS because math doesn't force you to write your math problems in such a way that the order of operations is obvious to a newb.

So, the smartest thing to do, I think, is start from the beginning of the Lisp language family, and then implement extensions to the most basic features, to do different things. The benefit to this is that someone who's learning to use HTML Scripting Lisp, can very easily learn to use the other Lisp extensions. So you only have to learn one thing, and then you can move on to working on another part of an operating system or application without having to learn a whole new language, because everything in the operating system is derived from Lisp, and every extension to Lisp is derived from a very small group of basic commands.

[nuntius]

This will probably come off as overly snarky, but "its called Scheme".

By my understanding, Scheme was and largely still is designed to embody principles like those you describe.
The Scheme community strives for a simple, clean, elegant language.

The problem is that large systems are inherently complicated. Elegance is difficult. Together, progress is often slower than in more "pragmatic" languages.

CL is a more pragmatic language. They had a bunch of code and decided to preserve compatibility rather than starting over.

Would a clean start be good? Theoretically, yes; but it might also set development back another decade or two.

I think that the designer of new languages today should take an approach similar to that of CL or C++. Add new features, and provide mechanisms to easily integrate legacy code.
It is not a novel idea, but one that is very hard to pull off right. While designers of old merely thought of new ideas, designers of today must also have experience and wisdom to fill a need within the existing ecosystem.

So don't throw away the CL package. Instead, create useful extensions that provide improved functionality. Or develop applications and libraries. Or develop new techniques for low-level language binding. etc.

There's an old saying: "Necessity is the mother of invention"

In the programming context, I think that means "start writing useful code, and that will focus you on the real problems and their solutions".

[xibalba]

I'm actually not a programmer (he said, as he edged towards the door). The closest I get to programming is using spreadsheets to speed up hex editing, when modifying old PS1 games, or using what little I can understand about MIPS R3000A to facilitate save state corruption.

I come across a lot of crazy stuff that was apparentally created in the throes of an acid binge, so I spend a lot of time wondering how they could've done it better. Stuff like: sprite sheets that are only compressed in their bottom half; data tables full of booleans that use a whole byte for true/false, instead of using bit toggles; files where the palettes and images for non-player characters were lumped in with the scripting to control interactions with these characters, which is further lumped together with the event scripts for each field screen; and constantly finding related data stored in multiple, far-flung, differntally-organized data tables.

I don't have anything against Common Lisp, nor it's philosophy; it's just that I (admittedly lacking in real world experience on the matter) question how it's implemented, how it was organized when it was created. It's been 15 years since Common Lisp was standardized. Saying it's not popular is an understatement, and yet languages derived from it are constantly popping up.

http://en.wikipedia.org/wiki/Forth_%28p ... anguage%29
http://lush.sourceforge.net/
http://en.wikipedia.org/wiki/Shen_%28pr ... anguage%29 //Derived from Qi
http://en.wikipedia.org/wiki/Qi_%28prog ... anguage%29

Other languages have, famously, borrowed from the predecessors of Common Lisp, and they became enormously popular while Common Lisp did not.

Real motivation: I'm starting school in the fall, and I want to know if attempting what amounts to a re-organization of Common Lisp into a more modular, orthogonal form is a worthwhile goal.

[nuntius]

The crazy stuff you are finding sounds par for the course, especially in the context of old console games. My first computers were a Vic 20 and later a C64. Their programs showed the same sort of crazy optimizations. Here's a fun writeup on Crash Bandicoot that gives some insight into the culture.

Is reorganizing CL worthwhile? I had a similar project called cl-sanity when I was first learning. Reading the spec, sorting things into different packages, etc. It was quite educational, but I wouldn't obsess with it. Writing programs that you can use and share with others is much more satisfying and worthwhile.

I suspect as you gain experience, you'll keep revisiting "what would make programming more effective?", try various languages, and maybe even try implementing some.

FYI, when they were forming the standard, there was a proposal to split CL into a smaller core and optional extensions. Unfortunately, the proposal came in a bit late and there wasn't enough time for it to be properly evaluated. I forget where I found it documented. Perhaps in some of the hyperspec notes.

[nuntius]

By random coincidence (or my strange taste in blogs), I just ran across this.
http://shape-of-code.coding-guidelines. ... h-mistake/

[nuntius]

and a couple more language design links.
http://www.coyotos.org/pipermail/bitc-d ... 03300.html
http://lambda-the-ultimate.org/

[xibalba]

The crazy stuff you are finding sounds par for the course, especially in the context of old console games. My first computers were a Vic 20 and later a C64. Their programs showed the same sort of crazy optimizations. Here's a fun writeup on Crash Bandicoot that gives some insight into the culture.

Thanks, the other guys will like that! It explains why we end up with such insane data tables. Though to be fair, it isn't always their fault.

Gah. I keep hearing about how Square does things this way or that, when you're dealing with the specifics of how things like multiplication are done in the ASM. Almost every single thing you're attributing to Square is actually being done by their compiler. I'm not saying there aren't legit things to point out about Square's code (hardcoding is a big offender, and some of the stuff you mentioned about rounding errors is relevant), but a lot of this stuff... they probably didn't have anything to do with. As far as I know, for example, they probably did something like:

return (fix)(1/3) * number; // No FPU, using fixed point

and the compiler created something like:

lui r3,0x5555
ori r3,r3,0x5556
mult r3,r4
mfhi r2
jr r31
nop

It's not as if they went "HMMM... 0x55555556 is how to represent 1/3". Same with the percents. The 32 is probably because it's 2^5 and the compiler wanted to do a bit shift.

I came across another case of compiler stupidity on http://board.byuu.org/, but I don't remember where it is. Byuu was complaining about GCC doing something insane when compiling bsnes, and he submitted a bug report for gcc 4.6 or gcc 4.7, don't remember which.

Is reorganizing CL worthwhile? I had a similar project called cl-sanity when I was first learning. Reading the spec, sorting things into different packages, etc. It was quite educational, but I wouldn't obsess with it. Writing programs that you can use and share with others is much more satisfying and worthwhile.

I suspect as you gain experience, you'll keep revisiting "what would make programming more effective?", try various languages, and maybe even try implementing some.

FYI, when they were forming the standard, there was a proposal to split CL into a smaller core and optional extensions. Unfortunately, the proposal came in a bit late and there wasn't enough time for it to be properly evaluated. I forget where I found it documented. Perhaps in some of the hyperspec notes.

So I'm basically talking about doing what they thought of too late, when they had too much time invested in the current version of CL to start over again. Good to know that someone else thought of it too.

I'm also curious about what kind of community would grow around, say, a Lisp based OS, built to take advantage of emerging hardware: tablet devices; multiprocessing/massively parallel computers; distrubuted computing; cloud computing; faster and more efficient compression algorithms for video, sound, and images; algorithms that will improve connection speed and bandwidth availablity for commonplace connections; and networking/scripting languages that are better at the job than the existing favorites.

So I'll need a language that can do an awful lot, with a very small set of commands, but which can be extended, with ease, to do anything. It makes no sense to use as massive a language as Common Lisp as a replacement for PHP, but it does make sense to use the most basic part of a Lisp, with extensions specific to do the work that PHP does. Would it take up less space, and be faster than, PHP? Would it be easier to create modifications to, say, forum software, using a Lisp?

My reason for going to school for Computer Science is to create an entity (a community; a set of tools that will enable and facilitate the creation, expansion, integration (into existing media), and independence (from existing media) of this community; and a standard for interaction and contribution in this community) that will allow anyone in the world with an internet connected device to get a free education, K through College.

Others are having the same ideas - India in particular is making strides in the field of cheap tablets, which makes sense since it has about 175 million children that need an education; MIT is offering accrediation of some sort for free online classes - but there is nothing to bring them together, and to attract others, likeminded.

I'm disgusted by how much not-for-profit schools charge their students, when they know they can get away with it because most of the students will be forced to get a Pell Grant, or a loan. They are forcing you to get into debt, because they know you will if you want to get out of poverty, because they have a monopoly on a Bachelors degree. I hope to break their monopoly, and to bankrupt as many of them as I can, with a superior business model. Namely, a free online college, that is ad-supported, with videos, podcasts, GPL'd textbooks, IRC, internet telephony, and forums, in which students, student-aids, and teachers will interact.

If I know anything, I know that social media is a force mulitplier for intelligence and effort. It is directly proportional to the number of people in the network, and the amount of effort and thought they put into their projects in the community.