Ctrl-C crashes Python after importing scipy.stats

Question

I'm running 64-bit Python 2.7.3 on Win7 64-bit. I can reliably crash the Python interpreter by doing this:

>>> from scipy import stats
>>> import time
>>> time.sleep(3)

and pressing Control-C during the sleep. A KeyboardInterrupt is not raised; the interpreter crashes. The following is printed:

forrtl: error (200): program aborting due to control-C event
Image              PC                Routine            Line        Source

libifcoremd.dll    00000000045031F8  Unknown               Unknown  Unknown
libifcoremd.dll    00000000044FC789  Unknown               Unknown  Unknown
libifcoremd.dll    00000000044E8583  Unknown               Unknown  Unknown
libifcoremd.dll    000000000445725D  Unknown               Unknown  Unknown
libifcoremd.dll    00000000044672A6  Unknown               Unknown  Unknown
kernel32.dll       0000000077B74AF3  Unknown               Unknown  Unknown
kernel32.dll       0000000077B3F56D  Unknown               Unknown  Unknown
ntdll.dll          0000000077C73281  Unknown               Unknown  Unknown

This makes it impossible to interrupt long-running scipy calculations.

Googling for "forrtl" and the like, I see suggestions that this kind of problem is due to use of a Fortran library that overrides Ctrl-C handling. I don't see a bug on the Scipy trackerbut given that Scipy is a library for use with Python, I would consider this a bug. It breaks Python's handling of Ctrl-C. Is there any workaround for this?

Edit: Following @cgohlke's suggestion I tried to add my own handler after importing scipy. This question about a related issue shows that adding a signal handler doesn't work. I tried using the Windows API SetConsoleCtrlHandler function via pywin32:

from scipy import stats
import win32api
def doSaneThing(sig, func=None):
    print "Here I am"
    raise KeyboardInterrupt
win32api.SetConsoleCtrlHandler(doSaneThing, 1)

After this, hitting Ctrl-C prints "Here I am", but Python still crashes with the forrtl error. Sometimes I also get a message saying "ConsoleCtrlHandler function failed", which quickly disappears.

If I run this in IPython, I can see a normal Python KeyboardInterrupt traceback before the forrtl error. I also see a normal Python traceback followed by the forrtl error if I raise some other error instead of KeyboardInterrupt (e.g., ValueError):

ValueError                                Traceback (most recent call last)
<ipython-input-1-08defde66fcb> in doSaneThing(sig, func)
      3 def doSaneThing(sig, func=None):
      4     print "Here I am"
----> 5     raise ValueError
      6 win32api.SetConsoleCtrlHandler(doSaneThing, 1)

ValueError:
forrtl: error (200): program aborting due to control-C event
[etc.]

It seems that whatever the underlying handler is doing, it's not just trapping Ctrl-C directly, but is reacting to the error condition (ValueError) and crashing itself. Is there any way to eliminate this?

question from:https://stackoverflow.com/questions/15457786/ctrl-c-crashes-python-after-importing-scipy-stats

Answer 1

Here's a variation on your posted solution that may work. Maybe there's a better way to solve this problem -- or maybe even avoid it all together by setting an environment variable that tells the DLL to skip installing a handler. Hopefully this helps until you find a better way.

Both the time module (lines 868-876) and _multiprocessing module (lines 312-321) call SetConsoleCtrlHandler. In the case of the time module, its console control handler sets a Windows event, hInterruptEvent. For the main thread, time.sleep waits on this event via WaitForSingleObject(hInterruptEvent, ul_millis), where ul_millis is the number of milliseconds to sleep unless interrupted by Ctrl+C. Since the handler that you've installed returns True, the time module's handler never gets called to set hInterruptEvent, which means sleep cannot be interrupted.

I tried using imp.init_builtin('time') to reinitialize the time module, but apparently SetConsoleCtrlHandler ignores the 2nd call. It seems the handler has to be removed and then reinserted. Unfortunately, the time module doesn't export a function for that. So, as a kludge, just make sure you import the time module after you install your handler. Since importing scipy also imports time, you need to pre-load libifcoremd.dll using ctypes to get the handlers in the right order. Finally, add a call to thread.interrupt_main to make sure Python's SIGINT handler gets called^[1].

For example:

import os
import imp
import ctypes
import thread
import win32api

# Load the DLL manually to ensure its handler gets
# set before our handler.
basepath = imp.find_module('numpy')[1]
ctypes.CDLL(os.path.join(basepath, 'core', 'libmmd.dll'))
ctypes.CDLL(os.path.join(basepath, 'core', 'libifcoremd.dll'))

# Now set our handler for CTRL_C_EVENT. Other control event 
# types will chain to the next handler.
def handler(dwCtrlType, hook_sigint=thread.interrupt_main):
    if dwCtrlType == 0: # CTRL_C_EVENT
        hook_sigint()
        return 1 # don't chain to the next handler
    return 0 # chain to the next handler

win32api.SetConsoleCtrlHandler(handler, 1)

>>> import time
>>> from scipy import stats
>>> time.sleep(10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyboardInterrupt

---

[1] interrupt_main calls PyErr_SetInterrupt. This trips Handlers[SIGINT] and calls Py_AddPendingCall to add checksignals_witharg. In turn this calls PyErr_CheckSignals. Since Handlers[SIGINT] is tripped, this calls Handlers[SIGINT].func. Finally, if func is signal.default_int_handler, you'll get a KeyboardInterrupt exception.