Two Python modules require each other's contents - can that work?

Question

I have a Bottle webserver module with the following line:

from foobar.formtools import auto_process_form_insert

And the foobar.formtools module contains this line:

from foobar.webserver import redirect, redirect_back

Of course, both result in the following errors (respectively):

ImportError: cannot import name auto_process_form_insert
ImportError: cannot import name redirect

Is it simply a fact that in Python two modules can't import each other and all module imports must be hierarchical in nature, or am I doing something wrong? Alternatively, is there a workaround short of placing all these nice functions in new modules?

Answer 1

Modules can import each other cyclically, but there's a catch. In the simple case, it should work by moving the import statements to the bottom of the file or not using the from syntax.

Here's why that works:

When you import a module, Python first checks sys.modules. If it's in there, it just imports from there. If it's not there, it tries to import it in the normal way; basically, it finds the file and runs the stuff in it.

Running a module populates the module's contents. For example, say we have this module, creatively named example_opener:

import webbrowser

def open_example():
    webbrowser.open('http://www.example.com/')

At the start, the module is empty. Then Python executes:

import webbrowser

After that, the module only contains webbrowser. Then Python executes this:

def open_example():
    webbrowser.open('http://www.example.com/')

Python creates open_example. Now the module contains webbrowser and open_example.

Say webbrowser contained this code:

from example_opener import open_example

def open(url):
    print url

Say example_opener is imported first. This code is executed:

import webbrowser

webbrowser has not yet been imported, so Python executes the contents of webbrowser:

from example_opener import open_example

example_opener has been imported, but not yet fully executed. Python doesn't care. Python pulls the module out of sys.modules. At this point, example_opener is still empty. It hasn't defined open_example yet, nor even completed importing webbrowser. Python can't find open_example in example_opener, so it fails.

What if we imported open_example from the end of webbrowser and webbrowser from the end of example_opener? Python would start by executing this code:

def open_example():
    webbrowser.open('http://www.example.com/')

webbrowser does not exist yet, but it doesn't matter until open_example is called. Now example_opener contains only open_example. It then executes:

import webbrowser

It has not been imported yet, so Python executes webbrowser. It starts:

def open(url):
    print url

It defines open. Then it executes:

from example_opener import open_example

example_opener is in sys.modules, so it uses that. example_opener contains open_example, so it succeeds. Python finishes importing webbrowser. That concludes importing webbrowser from example_opener. That's the last thing in example_opener, so the import of example_opener finishes, successful, as well.