python - Using csv module to read ascii delimited text?

Question

You may or may not be aware of ASCII delimited text, which has the nice advantage of using non-keyboard characters for separating fields and lines.

Writing this out is pretty easy:

import csv

with open('ascii_delim.adt', 'w') as f:
    writer = csv.writer(f, delimiter=chr(31), lineterminator=chr(30))
    writer.writerow(('Sir Lancelot of Camelot', 'To seek the Holy Grail', 'blue'))
    writer.writerow(('Sir Galahad of Camelot', 'I seek the Grail', 'blue... no yellow!'))

And, sure enough, you get things dumped out properly. However, on reading, lineterminator does nothing, and if I try to do:

open('ascii_delim.adt', newline=chr(30))

It throws a ValueError: illegal newline value:

So how can I read in my ASCII delimited file? Am I relegated to doing line.split(chr(30))?

Answer 1

You can do it by effectively translating the end-of-line characters in the file into the newline characters csv.reader is hardcoded to recognize:

import csv

with open('ascii_delim.adt', 'w') as f:
    writer = csv.writer(f, delimiter=chr(31), lineterminator=chr(30))
    writer.writerow(('Sir Lancelot of Camelot', 'To seek the Holy Grail', 'blue'))
    writer.writerow(('Sir Galahad of Camelot', 'I seek the Grail', 'blue... no yellow!'))

def readlines(f, newline='
'):
    while True:
        line = []
        while True:
            ch = f.read(1)
            if ch == '':  # end of file?
                return
            elif ch == newline:  # end of line?
                line.append('
')
                break
            line.append(ch)
        yield ''.join(line)

with open('ascii_delim.adt', 'rb') as f:
    reader = csv.reader(readlines(f, newline=chr(30)), delimiter=chr(31))
    for row in reader:
        print row

Output:

['Sir Lancelot of Camelot', 'To seek the Holy Grail', 'blue']
['Sir Galahad of Camelot', 'I seek the Grail', 'blue... no yellow!']