multithreading - Having a console in a single-threaded Python script

Question

I would like to have an interactive console in a single-threaded script that has several TCP connections open. This means I can't just have a standard input blocking the thread.

Is there an easy way to do this? Or should I just put the console in its own thread and be done with it?

Answer 1

You can subclass InteractiveConsole (from the builtin 'code' module) and override the push() method with a wrapper that redirects stdout/stderr to a StringIO instance before forwarding to the base InteractiveConsole's push() method. Your wrapper can return a 2-tuple (more, result) where 'more' indicates whether InteractiveConsole expects more input, and 'result' is whatever InteractiveConsole.push() wrote to your StringIO instance.

It sounds harder than it is. Here's the basic premise:

import sys
from cStringIO import StringIO
from code import InteractiveConsole
from contextlib import contextmanager

__all__ = ['Interpreter']


@contextmanager
def std_redirector(stdin=sys.stdin, stdout=sys.stdin, stderr=sys.stderr):
    """Temporarily redirect stdin/stdout/stderr"""

    tmp_fds = stdin, stdout, stderr
    orig_fds = sys.stdin, sys.stdout, sys.stderr
    sys.stdin, sys.stdout, sys.stderr = tmp_fds
    yield
    sys.stdin, sys.stdout, sys.stderr = orig_fds


class Interpreter(InteractiveConsole):
    """Remote-friendly InteractiveConsole subclass

    This class behaves just like InteractiveConsole, except that it
    returns all output as a string rather than emitting to stdout/stderr

    """
    banner = ("Python %s
%s
" % (sys.version, sys.platform) +
              'Type "help", "copyright", "credits" or "license" '
              'for more information.
')

    ps1 = getattr(sys, "ps1", ">>> ")
    ps2 = getattr(sys, "ps2", "... ")


    def __init__(self, locals=None):
        InteractiveConsole.__init__(self, locals=locals)
        self.output = StringIO()
        self.output = StringIO()

    def push(self, command):
        """Return the result of executing `command`

        This function temporarily redirects stdout/stderr and then simply
        forwards to the base class's push() method.  It returns a 2-tuple
        (more, result) where `more` is a boolean indicating whether the
        interpreter expects more input [similar to the base class push()], and
        `result` is the captured output (if any) from running `command`.

        """
        self.output.reset()
        self.output.truncate()
        with std_redirector(stdout=self.output, stderr=self.output):
            try:
                more = InteractiveConsole.push(self, command)
                result = self.output.getvalue()
            except (SyntaxError, OverflowError):
                pass
            return more, result

Check out this complete example, which accepts input from a UDP socket:

• http://files.evadeflow.com/pyrrepl.zip

Start two consoles and run server.py in one, client.py in the other. What you see in client.py should be indistinguishable from python's regular interactive interpreter, even though all commands are being round-tripped to server.py for evaluation.

Of course, using sockets like this is terribly insecure, but it illustrates how to evaluate an external input asynchronously. You should be able to adapt it to your situation, as long as you trust the input source. Things get 'interesting' when someone types:

while True: continue

But that's another problem entirely... :-)