Search found 9 matches

by sdp
Sat Nov 13, 2010 3:29 pm
Forum: Common Lisp
Topic: utf-8 in SBCL
Replies: 15
Views: 20806

Re: utf-8 in SBCL

But is the length of individual messages known? It is part of the message headers (not the same header I'm reading at the opening of the socket). For serious network programming I belive iolib is currently best, since it handles nonblocking communication and multiplexing. After thinking about it, I...
by sdp
Sat Nov 13, 2010 1:57 pm
Forum: Common Lisp
Topic: utf-8 in SBCL
Replies: 15
Views: 20806

Re: utf-8 in SBCL

...if the end of the message is not signalled by closing the socket or the length is known. Although I suppose that is included in the header?
The length is not known, after the header it's a potentially infinite two-way conversation until either server or client sends a control sequence.
by sdp
Sat Nov 13, 2010 1:16 pm
Forum: Common Lisp
Topic: utf-8 in SBCL
Replies: 15
Views: 20806

Re: utf-8 in SBCL

It was more subtly broken, the default eol-style for my mac is just :lf, but the input spec uses :crlf, so I was getting #\Return at the end of every line. I fixed the eol-style, so this works properly: (defun read-header (stream) (with-output-to-string (header) (loop for line = (read-line stream ni...
by sdp
Sat Nov 13, 2010 11:41 am
Forum: Common Lisp
Topic: utf-8 in SBCL
Replies: 15
Views: 20806

Re: utf-8 in SBCL

Ah, I wasn't properly finishing my loop at the end of the header and it was continuing to read into a binary section. Here's the modified code that exits properly: (defun read-header (stream) (let ((header "") (line (read-line stream nil))) (loop while (not (string= line (string #\Return))...
by sdp
Sat Nov 13, 2010 10:57 am
Forum: Common Lisp
Topic: utf-8 in SBCL
Replies: 15
Views: 20806

Re: utf-8 in SBCL

If you are not transmission speed constrained and control the encoding used by the server a UTF-32 might be better. That is what SBCL uses internally, at least on my system. I do control the encoding used by the server, but not the client and the specs say utf-8 so I'll just have to make it work. I...
by sdp
Sat Nov 13, 2010 10:26 am
Forum: Common Lisp
Topic: utf-8 in SBCL
Replies: 15
Views: 20806

Re: utf-8 in SBCL

UTF-8 is a binary format, in which byte 159 by itself does not denote anything. After a quick inspection of the UTF-8 standard, I can see that for characters between U+0080 and U+07FF, UTF-8 uses 16 bits. Remember that unicode and a particular translation of it into a byte sequence are not the same...
by sdp
Sat Nov 13, 2010 9:40 am
Forum: Common Lisp
Topic: utf-8 in SBCL
Replies: 15
Views: 20806

Re: utf-8 in SBCL

I explored the characters a little further, I can store such character to a string, create a stream from that string and read from it: COMMON-LISP-USER> (setf *my-string* (make-string 1)) COMMON-LISP-USER> (setf (char *my-string* 0) #\u009F) #\Application-Program-Command COMMON-LISP-USER> *my-string...
by sdp
Sat Nov 13, 2010 9:05 am
Forum: Common Lisp
Topic: utf-8 in SBCL
Replies: 15
Views: 20806

Re: utf-8 in SBCL

Thanks for your quick reply! =) Shouldn't it still be read as a character like #\u009F? COMMON-LISP-USER> (char-code #\u009F) 159 COMMON-LISP-USER> (code-char 159) #\Application-Program-Command 159 isn't the only code that fails, for example: decoding error on stream #<SB-SYS:FD-STREAM for "soc...
by sdp
Sat Nov 13, 2010 12:00 am
Forum: Common Lisp
Topic: utf-8 in SBCL
Replies: 15
Views: 20806

utf-8 in SBCL

I am trying to read a utf-8 stream: (defun read-header (stream) (let ((header "") (line (read-line stream nil))) (loop while (not (eq line "")) do (setf header (concatenate 'string header line)) (setf line (read-line stream nil))) header)) but I keep running into this error: debu...