pytest - In Python, how can I patch a function from another file that's not imported locally?

Question

I'm working on learning Pythonic test development, and stumbled across this seemingly counterintuitive issue. When I'm patching a function that is defined in the same file as the code under test, the patch works correctly. But when I import a function from a different file, the only way to get the patch working correctly is to make the import local instead of defining it at the top of the file.

Minimal reproduction:

a/b.py:

from x.y import z


def c():
    print("In a.b.c")


class D:
    def do_stuff_with_a_b_c(self):
        print("In do_stuff_with_a_b_c")
        c()

    def do_stuff_with_x_y_z(self):
        from x.y import z
        print("In do_stuff_with_x_y_z")
        z()

x/y.py:

def z():
    print("In x.y.z")

tests/d_tests.py:

import inspect
import unittest
from unittest.mock import patch

from x import y
from a.b import D


class DTests(unittest.TestCase):
    def test_do_stuff_with_a_b_c(self):
        print(f"In {inspect.stack()[0][3]}")
        D().do_stuff_with_a_b_c()

    @patch("a.b.c")
    def test_do_stuff_with_patched_a_b_c(self, a_b_c_method):
        print(f"In {inspect.stack()[0][3]}")
        D().do_stuff_with_a_b_c()

    def test_do_stuff_with_x_y_z(self):
        print(f"In {inspect.stack()[0][3]}")
        D().do_stuff_with_x_y_z()

    @patch("x.y.z")
    def test_do_stuff_with_patched_x_y_z(self, x_y_z_method):
        print(f"In {inspect.stack()[0][3]}")
        D().do_stuff_with_x_y_z()

    def test_do_stuff_with_patch_object_x_y_z(self):
        print(f"In {inspect.stack()[0][3]}")
        with patch.object(y, "z"):
            D().do_stuff_with_x_y_z()


if __name__ == '__main__':
    unittest.main()

When I run the tests with the code as above, I get the following output:

In test_do_stuff_with_a_b_c
In do_stuff_with_a_b_c
In a.b.c
In test_do_stuff_with_patch_object_x_y_z
In do_stuff_with_x_y_z
In test_do_stuff_with_patched_a_b_c
In do_stuff_with_a_b_c
In test_do_stuff_with_patched_x_y_z
In do_stuff_with_x_y_z
In test_do_stuff_with_x_y_z
In do_stuff_with_x_y_z
In x.y.z

However, when I comment out the local import of x.y.z in do_stuff_with_x_y_z, I get the following output:

In test_do_stuff_with_a_b_c
In do_stuff_with_a_b_c
In a.b.c
In test_do_stuff_with_patch_object_x_y_z
In do_stuff_with_x_y_z
In x.y.z
In test_do_stuff_with_patched_a_b_c
In do_stuff_with_a_b_c
In test_do_stuff_with_patched_x_y_z
In do_stuff_with_x_y_z
In x.y.z

What is the the difference between the two forms that causes patch to work as expected in one scenario but not the other?

Answer 1

The two cases - top-level versus local import - have to be handled differently while patching. You have to patch an object as used in the module, as described in the documentation, or in this blog post by Ned Batchelder, who describes the problem in more detail.

In your first case, you import the module z at runtime after it has been patched, so the patched module is imported. In your second case (the more standard case of top-level import) you import the module before it is patched, and have now a reference to the unpatched z in your module b, so for the patching to work you have to patch this reference:

@patch('a.b.z')
def test_do_stuff_with_patched_x_y_z(self, mocked_z):
   ... 

To sum this up: you always have to check how the object to use is imported in the module to be tested. There are basically two cases:

• the object is imported like import x or import x.y (and used like x.y.z) - in this case it can be patched as x.y.z. The same is true if you import the module locally inside a function, as in your initial case.
• the object is imported like from x.y import z (or from .y import z) - in this case a local reference is created that refers to the object, and the patching shall be done for that reference (e.g. a.b.z in your case)