java - Process OutputStream hangs when reading from python sub-process

Question

This code works as expected.

public static void main(String[] args) {
        char[] buffer=new char[16];
        try {
            Process process = Runtime.getRuntime().exec("bc");
            Writer w = new OutputStreamWriter(process.getOutputStream());
            BufferedReader output = new BufferedReader(new InputStreamReader(process.getInputStream()));
            for (int i = 0; i < 10; i++) {
                w.write("1+" + i + "
");
                w.flush();
                output.read(buffer);
                System.out.println(new String(buffer).trim());
                buffer=new char[16];
            }   
        } catch (IOException ioe) {
            ioe.printStackTrace();
        }
    }

In the for loop the commands are sent to bc and the correct results are obtained and printed via the BufferedReader output.

1
2
3
4
5
6
7
8
9
10

If I change the code to

public static void main(String[] args) {
        char[] buffer=new char[16];
        try {
            Process process = Runtime.getRuntime().exec("python");
            Writer w = new OutputStreamWriter(process.getOutputStream());
            BufferedReader output = new BufferedReader(new InputStreamReader(process.getInputStream()));
            Scanner s = new Scanner(process.getInputStream());
            for (int i = 0; i < 10; i++) {
                w.write("1+" + i + "
");
                w.flush();
                output.read(buffer);
                System.out.println(new String(buffer).trim());
                buffer=new char[16];
            }   
        } catch (IOException ioe) {
            ioe.printStackTrace();
        }

    }

Then the code hangs at the output.read(buffer); line.

Why is that? I would assume that the python and bc interactive REPLs would be read in the same way. What is the difference and what is the suggested work around?

I am using OS X, java 13, and the python interpreter is Python 3.8.6.

question from:https://stackoverflow.com/questions/65929760/process-outputstream-hangs-when-reading-from-python-sub-process

Answer 1

Python buffers output by default so you won't be able to read it's output until it flushes the stream. This is probably why your loop is not reading the response to each write. To fix, you could either split your read/write into loops on different threads, or launch python with python -u to make Python output unbuffered.