Implement a form of LISP for the web?

[Mangala]

I have a question/idea, and it may or may not have already been thought of/discussed on this forum.

I want to allow some of my users to be able to mix and match data online from other web pages using simple set functions similar to those found in LISP.

For instance, if a user finds a website with a list of U.S. presidents, I'd like them to be able to add the URL for that list on my website, my website would then screen-scrape the president list, and now have a permanent "functional" list of U.S. presidents which they can then link to other lists of relevant information.

If a user then finds a table with columns for "U.S. President Name" and "U.S. President Birthplace", my website could screen-scrape that table as well, "JOIN" the data, and now, whenever someone looks at my website's "U.S. President" list, they'll also be able to find a mapping for every president's birthplace. And so on for other data.

Eventually, I would want to get closer and closer to allowing users to play with these data lists in the way that LISP is able to manipulate this data - providing a full graphical interface for programming with LISP on a website.

Essentially, this would be like implementing LISP for the internet using some clever screen scraping scripts. LISP is great, but what it seems to lack is a large amount of data to fully show off its expressive power. The internet could provide that data.

Do you think this would work? Has it already been tried? I'd like your thoughts.

Thanks.

[Jasper]

I am not sure what you mean. You seem to want to make some form of database where a lot of information can be organized, with a lot of automation. And let that database be shared through some interface in lisp. That sounds like a cool idea, but i don't any reason why the users should use lisp because you just serve the information. What would this 'screen scraping' entail? How does it know where the useful data lies?

With the name 'lisp for the web' i think a lisp programming language for the web. Something that replaces javascript, or flash. That also be cool cooler. But i do not see any reason that language would have be very different then lisps that, say compile to binary. In a perfect world(imo) the language of the internet would be lisp, restricted to different extents. For instance, a basic page would just be limited to functions specifying the page, a slightly less basic page can run code before the final show, and an interactive one can do a lot more, but be limited with its access to disks/data it can break your computer with.

[schoppenhauer]

If I understand you and the many buzzwords I have heard right, what you are talking about is something which is similar to some goals for the semantic web. I.e. creating a big database with semantic connections, etc.

I think this is definitily a direction the web will walk into, but it will - as almost anything in the web - be much more quick&dirty than anything one could create using lisp.

Something I miss - when talking about "the web" - is some lisp-similar embedded in xml-forms. I.e. you have lisp-constructs and the lisp-philosophy, but in xml-syntax, with all the "benefits", like you can use DOM and the whole stuff (which is comparably useless but "standard"). Maybe as an alternative to XSLT. I dont like XML, but it has spread, we have to accept it, and it uses S-Expressions like any lisp, so it shouldnt be too hard to map a lisp onto it, why not trying to bring as much lisp-philosophy as possible into it?

[Jasper]

Something I miss - when talking about "the web" - is some lisp-similar embedded in xml-forms. I.e. you have lisp-constructs and the lisp-philosophy, but in xml-syntax, with all the "benefits", like you can use DOM and the whole stuff (which is comparably useless but "standard"). Maybe as an alternative to XSLT. I dont like XML, but it has spread, we have to accept it, and it uses S-Expressions like any lisp, so it shouldnt be too hard to map a lisp onto it, why not trying to bring as much lisp-philosophy as possible into it?

I don't like XML either; afaik it is just nested lists with a crappy notation. To be clear, you mean you want a function that takes a nested list, then expands where it finds keywords, and then expands that into valid XML? I mean, writing the stuff entirely in actual XML seems silly. Anyway, that shouldn't be too hard, i can add that to my project as a little sideline project.

Edit: it reminds me of this.

[schoppenhauer]

afaik it is just nested lists with a crappy notation.

Mostly. But - to be fair - there are a few quite good ideas like Datatype Definitions in it - which may also be crappy in notation and implementation, but exist - and it is still better than nothing. At least it is an abstract notation for Trees with possibilities to transform them, with some kind of useable namespace-mechanism, with comparably well API-Specifications to access them... I mean thats more than you can expect when considering that it is really used in "practical" projects - where you otherwise find crap like c++...

To be clear, you mean you want a function that takes a nested list, then expands where it finds keywords, and then expands that into valid XML? I mean, writing the stuff entirely in actual XML seems silly. Anyway, that shouldn't be too hard, i can add that to my project as a little sideline project.

Yes. It may sound stupid - and in fact, it is - as all the stuff common ITiods are using... But:

Edit: it reminds me of this.

I dont know X#, but ever heard of XSLT? (See http://en.wikipedia.org/wiki/XSLT if not). I mean in general, there is XML - which is far away from being good, but it is there - a notation for trees... Then in general it is a good Idea to create a Scripting-Language with this syntax, so if you already have a Parser for XML, you dont need an additional parser for some other language. I mean thats sort of lisp's Idea to unify data and code - as soon as one accepts XML . But when you look at XSLT you'll find out that it is even more junk than XML. So what I mean is: XML is there, and will not go away, and if it goes away, then something worse will follow... XML is crap, but it could be worse... But XSLT is the last crap on earth... Why not at least trying to bring a few lisp-ideas into the XML-World?

And yes, to me, it looks silly to write Code in entire XML by hand - but there are a lot of people doing so, or using XML-Editors. I myself mostly generate it using Lisp, where I can use Lisp-S-Expressions

[Jasper]

At least it is an abstract notation for Trees with possibilities to transform them, with some kind of useable namespace-mechanism

I guess we got to give them that, yes..

I dont know X#, but ever heard of XSLT?

I only checked it a millisecond on wikipedia when you mentioned it before.

I added an nested-list -> xml to my project. (Well, not entirely sure, maybe still subset.) Also added a simpler macro-only resolver, when they are combined it should do what you are suggesting.

The xml writer is in convert/xml.lisp, requiring other/generic.lisp Note that attr is defaultly the symbol indicating a tag has attributes; for any list in the nested lists (when (listp (second list)) (eql (car (second list)) 'attr)) then (cdr (second list)) is an assoc-list with attributes. You can also do <lala \> these, when the second (or third if attributed) is 'no-close.

The simple macro resolver is in other/simple-macexpand.lisp, requiring other/generic.lisp, other/argument-positioner.lisp and namespace.lisp. Add macros with (add-mac name (arguments) (..some optional keyword stuff..) ..code that produces new list.. if use values, and second is :stop, it will not macro-expand again at that point, but it will do it with lower-lying things. (maybe i should make a :full-stop)
Then (produce-xml-stream (resolve code) t) should execute the macros and then write the xml. (tour/ has more.)

The project also has a reader that can read nested lists from files, with an extra method of writing s-expressions, but unfortunately i haven't made it such that it can read strings yet.

I guess that stuff might still have some bugs, though. You think it is useful or do you think just regular writing directly to files might be better? Seems unlikely that there isn't a library doing it out there already.

Edit: I should add that i myself did html manually. I guess i'll be using said thingy.

[Jasper]

I hope my earlier reply isn't overkill. Maybe i should make a little package with only the xml bit, rather then the whole project Lang-lisp. I guess to be more complete it should also include a reader. Even have an idea of an icon for it
Edit: there is already stuff for it out there. 'sudo aptitude search cl- |grep ' cl-' |grep -i xml' already gives me cl-s-xml and cl-xmls, and there is cl-yaclml for writing html.

[schoppenhauer]

Lol! That Icon is good! Anyway. At least I begin to like JavaScript at the moment.

[duncan]

[quote="Jasper"]I guess we got to give them that, yes..[quote="Jasper"]

No, actually we don't. But to understand why you need to understand a bit of history about XML that has been swept under the rug.

Before there was XML there was SGML. It was meant as a meta-markup language for documents that had to be processed by machines. Just to give them a bit of structure, you see. Boeing was a major player in this because- well you wouldn't believe how many documents a company like Boeing generates. Lots...

Anyway, though the guys that invented SGML are pretty proud of having done so it was definitely a case of reinventing the wheel. Structured data- who would have thunk it. They came to their senses around about the time they realized that they had no idea how to process structured data. But they had to process it because.. well in all the confusion around structuring data they had failed to think about how to present it. And presenting it meant processing it. Luckily Lisp came to the rescue in the form of scheme. Of course they couldn't call it scheme, since they couldn't claim to have invented scheme, so.. they called it DSSSL. XSLT is DSSSL's direct descendant.

Thus, XSLT is a scheme. It's not a very good scheme, but it has a signal virtue- its invention can be attributed to certain people to whom the invention of scheme cannot be attributed. Too bad for all the XSLT "programmers" out there. Or maybe good for them. Hard to say.