convert a string of numbers to array of numbers?

Discussion of Common Lisp
Post Reply
speech impediment
Posts: 36
Joined: Mon May 04, 2009 5:19 pm

convert a string of numbers to array of numbers?

Post by speech impediment » Sat Oct 16, 2010 11:56 pm

After about a year and a half of Lisp study, I've been starting to feel comfortable about starting a small project. Perhaps I've been hasty... I am trying to convert a comma separated values file to a 2d array. I looked at other people's code and I can't understand why their code is so long just to parse a csv file. Perhaps my approach is naive, but this is what I did:

Code: Select all

(defvar *stream* (open "C:\\yada\\table.csv"))
	(substitute #\space #\, (read-line *stream*))

;Sample result:
"2010-09-30 115.05 115.79 113.59 114.13 287106700 114.13"
I probably should use WITH-OPEN-FILE instead of OPEN, but I couldn't read the next line in the csv file. I know I don't have an iterative array making part yet, but I am stuck at reading the entire string separated by spaces. I used READ-FROM-STRING (and a few combination of its keywords), but it seems to stop at the first space:

Code: Select all

(read-from-string (substitute #\space #\, (read-line *stream*))

;Sample result:
|2010-09-27|
10
I understand the first element isn't really a number, but I forgot what those bars mean: |...|
I also tried to substitute the quotation marks but it seems that Lisp doesn't understand this: #\"

ramarren
Posts: 613
Joined: Sun Jun 29, 2008 4:02 am
Location: Warsaw, Poland
Contact:

Re: convert a string of numbers to array of numbers?

Post by ramarren » Sun Oct 17, 2010 12:19 am

speech impediment wrote:I looked at other people's code and I can't understand why their code is so long just to parse a csv file.
The CSV format is described by RFC4180. It is not quite that trivial in general case. Also, you have to spend some code on parsing the values themselves. Lisp reader, which is what READ-FROM-STRING uses, is specialized for reading Lisp forms and reading general data with it is not a good idea.

You should probably use already existing CSV-reading library, or at least a parser generator. CLiki has a page with a list of parsers and generators. I have written a parser combinator library which also can be used for this purpose.
speech impediment wrote:I understand the first element isn't really a number, but I forgot what those bars mean: |...|
Bar syntax is for quoting symbol names. A name of a symbol in Common Lisp can be an arbitrary string, and bars are used when that string could conflict with some other reader syntax.

nuntius
Posts: 538
Joined: Sat Aug 09, 2008 10:44 am
Location: Newton, MA

Re: convert a string of numbers to array of numbers?

Post by nuntius » Sun Oct 17, 2010 12:21 am

speech impediment wrote:

Code: Select all

(defvar *stream* (open "C:\\yada\\table.csv"))
(substitute #\space #\, (read-line *stream*))
(edited for clarity -- the initial indentation fooled me for a while.
speech impediment wrote:I probably should use WITH-OPEN-FILE instead of OPEN, but I couldn't read the next line in the csv file.
With-open-file is just a macro that closes the file when the macro body completes (try macroexpanding it to see how it works). It is perfectly fine to close the file manually by yourself.
speech impediment wrote: I know I don't have an iterative array making part yet, but I am stuck at reading the entire string separated by spaces. I used READ-FROM-STRING (and a few combination of its keywords), but it seems to stop at the first space:
Yes. Read-from-string just reads one form (symbol, number, string, parenthesized expression, etc.) at a time. You will have to use it in a loop to get everything out of the line.
speech impediment wrote:

Code: Select all

(read-from-string (substitute #\space #\, (read-line *stream*))

;Sample result:
|2010-09-27|
10
I understand the first element isn't really a number, but I forgot what those bars mean: |...|
I also tried to substitute the quotation marks but it seems that Lisp doesn't understand this: #\"
This "substitute #\space #\," should work as long as your values don't contain commas or spaces (i.e. no string values). The vertical bars mark symbol names containing nonstandard characters (mixed upper and lower case, spaces, etc.). The quotation marks are added by the printer to make strings usable by READ -- they don't actually exist in your sample data.

speech impediment
Posts: 36
Joined: Mon May 04, 2009 5:19 pm

Re: convert a string of numbers to array of numbers?

Post by speech impediment » Sun Oct 17, 2010 1:59 am

Thanks again Ramarren and nuntius. It seems like I have a lot more work to do before I do any useful programming. :P

findinglisp
Posts: 447
Joined: Sat Jun 28, 2008 7:49 am
Location: Austin, TX
Contact:

Re: convert a string of numbers to array of numbers?

Post by findinglisp » Tue Oct 19, 2010 7:17 pm

Ramarren wrote:
speech impediment wrote:I looked at other people's code and I can't understand why their code is so long just to parse a csv file.
The CSV format is described by RFC4180. It is not quite that trivial in general case. Also, you have to spend some code on parsing the values themselves. Lisp reader, which is what READ-FROM-STRING uses, is specialized for reading Lisp forms and reading general data with it is not a good idea.
Indeed. The CSV format is one of those deceptive formats that seems easy until you discover all the corner cases, protocols for escaping various special characters, etc. In short, it's harder than it looks, which results in the complexity of the code.
Cheers, Dave
Slowly but surely the world is finding Lisp. http://www.findinglisp.com/blog/

Post Reply