python - Line buffered serial input

Question

I have a serial device that I'm trying to read input from. I sent it a string "ID ", and it returns "ID XX " (where is an ASCII carriage return, hex 0x0d).

Since the eol option on serial.readline is no longer supported, I'm using TextIOWrapper to read from the serial port and return a line at a time.

My problem is that instead of returning my string as soon as it sees the carriage return, it's waiting until the twice the timeout I set when I opened the serial port. I'd like it to return the string immediately as soon as it reads an entire line since I may have hundreds of these commands to send to the device and I don't want to wait for the timeout each time. If I set timeout to 0, then I get no output at all (presumably because my script stops waiting before the device has a chance to output anything), and if I set the timeout to None, the script blocks forever.

Here's a simple test script:

import serial
import io
import time

ser = serial.Serial("/dev/ttyUSB0", baudrate=9600,
                    bytesize=8, parity='N', stopbits=1,
                    xonxoff=0, rtscts=1, timeout=5)

sio = io.TextIOWrapper(io.BufferedRWPair(ser, ser),
                       newline=None)


sio.write(unicode("ID
"))
sio.flush()

print "reading..."

x = sio.readline()

print len(x)
print x

The script always takes 10 seconds from the time it says "reading" until it prints the "ID XX" string that it read from the serial port.

I'm certain that the device is outputting the carriage return, as I've used strace to watch the reads:

select(4, [3], [], [], {5, 0})          = 1 (in [3], left {4, 991704})
read(3, "I", 8192)                      = 1
select(4, [3], [], [], {5, 0})          = 1 (in [3], left {4, 999267})
read(3, "D", 8191)                      = 1
select(4, [3], [], [], {5, 0})          = 1 (in [3], left {4, 999420})
read(3, " ", 8190)                      = 1
select(4, [3], [], [], {5, 0})          = 1 (in [3], left {4, 999321})
read(3, "X", 8189)                      = 1
select(4, [3], [], [], {5, 0})          = 1 (in [3], left {4, 999355})
read(3, "X", 8188)                      = 1
select(4, [3], [], [], {5, 0})          = 1 (in [3], left {4, 999171})
read(3, "
", 8187)                     = 1
select(4, [3], [], [], {5, 0})          = 0 (Timeout)
select(4, [3], [], [], {5, 0})          = 0 (Timeout)

You can see the 2 select() timeouts that give the 10 second delay, but you can also clearly see the carriage return being read. I've tried setting the newline parameter to 'None' and '' (which should automatically allow , , and ), and to ' ', but with the same result each time.

I've also tried setting the buffer_size in the BufferedRWPair() call to '1' to keep it from buffering input, but that made no difference.

Any idea what I'm doing wrong?

If I can't get this working, my next step will be to use serial.read() to read a character at a time and do my own line buffering, but I wanted to try to do it the "right" way with textiowrapper first.

Answer 1

Wasted a few hours on this today. It turned out that io.BufferedReader reads until it has filled its buffer and then passes the buffer to io.TextIOWrapper. The default buffer size is 8192, so depending on your device this might take a while.

The correct example code should be:

# buffer size is 1 byte, so directly passed to TextIOWrapper
sio = io.TextIOWrapper(io.BufferedRWPair(ser, ser, 1), encoding='ascii')
print sio.readline()[:-1]