Navigating Mocks in Python: Strategies for Safe and Efficient Mock Usage

Do you spend a lot of time on maintaining your mocks? Has your test suite passed and your code failed to run in production? This could be a result of using mocks too aggressively or not utilising safety features provided by Python’s mock library. In this post, we will explore the benefits and potential risks of using mocks, discuss strategies for designing high-level interfaces that are safe to mock, and examine a linter tool that ensures safe mock usage. Let’s begin by examining the potential risks associated with using mocks.

Mocks can be a valuable tool in testing, but they also come with certain drawbacks. One major issue is that mocks may not accurately reflect the conditions of the production environment, resulting in a test passing but failures when your code runs in production. There are also many benefits to using mocks. For example, a test suit can run faster by removing slow interactions, such as HTTP requests, using mocks. Additionally, mocks can also facilitate testing in situations where certain interactions may be impossible or costly to perform frequently, such as interactions with cloud providers. Mocks also enable tests to be run when you otherwise might not be able to, such as when cut-off from the Internet as you travel. In the following paragraphs, we will explore strategies for using mocks responsibly to avoid the issues of divergence between testing and production environments, while still reaping the benefits of mock usage. First, we will consider alternative options to using mocks.

Before implementing a mock, it’s important to consider whether there are other ways to run your tests without using mocks. For example, when working with database interactions, an alternative to using a mock is to use a local, in-memory database like SQLite. While interactions with SQLite may not perfectly match other SQL flavors, it can still provide a close approximation that can be further supplemented with tests against the production SQL flavor during CI/CD. Similarly, when dealing with interactions with an HTTP API, it may be worth starting without mocks and only introducing them later if tests become too slow or the interactions become too costly. Keep in mind that creating and maintaining complex mocks in your tests can consume a lot of time and resources, which can be more impactful to your team or company than the cost savings from avoiding a few hundred dollars in bills.

So when is it appropriate to use mocks? The answer we will come to is that mocks should typically be employed on high-level interfaces to dependencies. A dependency interface is an abstraction of a dependency that simplifies or enhances its API to better suit the needs of the project at hand. (It’s important to keep in mind that what may be considered a valuable abstraction in one project may not be relevant in another. The abstraction should not seek to improve the dependency API for general use, it should seek to simplify it given how the project at hand needs to use it.) For instance, when our code relies on interactions with a database, we could write SQL statements alongside our business logic and send them to the database for processing. This would constitute a form of an interface (in this case, the SQL statements), but it’s considered a low-level abstraction of the database since the business logic would still need to know how to translate its needs into raw SQL, including joins, indexes, and how to handle communication errors with the database. Using mocks in this scenario would require replicating much of the behavior of SQL, which can be both costly and time-consuming to create and maintain.

A higher-level interface could involve using an Object-Relational Mapping (ORM) library, which enables interactions with the database to resemble interacting with objects, hiding much of the complexity of generating and executing SQL. This constitutes a mid-level abstraction, as working with ORMs still requires an understanding of relationships, indexing, and database session management. Writing mocks at this level of abstraction is generally simpler than at the raw SQL level, but the data model is still likely to be fairly complex. Mocking an ORM will require replicating some of the ORM’s logic, which can be complex.

The level of interface that we will consider for mocking is at an even higher level than the previous examples discussed. One way to do this is to think about how to explain to someone who doesn’t understand the inner workings of a database (or the external dependency in general) what is needed from the dependency to fulfill a specific business need. For example, to place a merchandise order, the system needs to record the customer’s information, the product(s) being ordered, and the quantity of each product. This information can inform the design of the interface, which should consist of a method, such as place_order, that takes in a customer identifier, such as a email_address, and a list of products with unique identifiers, such as product_code, and quantities. The method should return a unique identifier for the order, such as an order_id, and raise a DatabaseError if any issues occur.

To illustrate why this interface is preferable, consider an alternative of using the ORM directly in your business logic. In that case, we would need to take the email and lookup the customer record, look up the product code for the product record, handle the cases where either of those records are not in the database, and then create a new record in the orders table and retrieve the unique identifier of that record. This may well be how to implement the place_order interface, although using this high-level interface should abstract away the complex details of interacting with the database. This makes the mock of this interface simpler and less prone to diverging from the underlying implementation that will be used in the production environment. The place_order interface still needs to be tested, although it is probably best to test it in isolation as a part of your integration tests. The integration tests can be used to guarantee the interface works, the rest of the test suite can then assume that it works and mock it out as required.

Determining the appropriate abstraction level for an interface can be challenging. If the interface is too high-level, changes to the code may require changes to the interface, reducing the benefits of being able to mock it out. On the other hand, if the interface is too low-level, the mock of the interface will be complex and the risk of divergence between the test environment and production environment increases. One way to determine the appropriate level for an interface is to consider the consequences of having too high or low level of abstraction. If most of the logic is found within the interface, it may be considered too high-level. Conversely, if complex mocking logic is required, it may indicate that the interface is too low-level.

Now that we have established the kind of interfaces that are safe to mock, let’s look at some practicalities. Even high level interfaces risk divergence between the mocks in your test suit and the real interface. Consider the following example where the place_order interface has changed to accept customer_id instead of email_address:

#source.py
def place_order(customer_id, products):
    ...
    order_id = ...
    ...
    return order_id


def handle_order_request():
    ...
    customer_email = ...
    purchased_products = ...
    ...
    return place_order(email_address=customer_email, products=purchased_products)


if __name__ == "__main__":
    handle_order_request()

This is an example test with place_order mocked:

# test_source.py
from unittest import mock

import source


def test_handle_order_request():
    with mock.patch("source.place_order") as mocked_place_order:
        order_id = source.handle_order_request()

        assert order_id == mocked_place_order.return_value

These tests pass when they are run:

pytest test_source.py
========================================================================================== test session starts ==========================================================================================
collected 1 item                                                                                                                                                                                        

test_source.py .                                                                                                                                                                                  [100%]

=========================================================================================== 1 passed in 0.02s ===========================================================================================

When you actually run this code, however, you will get the following error:

python source.py 
Traceback (most recent call last):
  File "source.py", line 17, in <module>
    handle_order_request()
  File "source.py", line 13, in handle_order_request
    return place_order(email_address=customer_email, products=purchased_products)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: place_order() got an unexpected keyword argument 'email_address'

One way to ensure responsible mock usage is by utilizing the safety feature of mocks that allows them to verify compliance with the interface definition. This can be achieved by using the autospec feature, which creates a mock object that has the same methods and attributes as the object being specified. This mock object will raise an error if any method is called that is not defined in the original object, helping to prevent divergence between the test environment and production environments. By using this feature, you can ensure that your mock usage adheres to the defined interface, promoting a safer and more efficient testing process:

# test_source.py
from unittest import mock

import source


def test_handle_order_request():
    with mock.patch("source.place_order", autospec=True) as mocked_place_order:
        order_id = source.handle_order_request()

        assert order_id == mocked_place_order.return_value

When the test suite is run with the autospec argument included, it raises an error to indicate that the use of place_order does not comply with the interface:

pytest test_source.py
========================================================================================== test session starts ==========================================================================================
collected 1 item                                                                                                                                                                                        

test_source.py F                                                                                                                                                                                  [100%]

=============================================================================================== FAILURES ================================================================================================
_______________________________________________________________________________________ test_handle_order_request _______________________________________________________________________________________

    def test_handle_order_request():
        with mock.patch("source.place_order", autospec=True) as mocked_place_order:
>           order_id = source.handle_order_request()

test_source.py:8: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
source.py:13: in handle_order_request
...
>                           raise TypeError(msg) from None
E                           TypeError: missing a required argument: 'customer_id'

inspect.py:3125: TypeError

After correcting the argument, the test suite passes again:

# source.py
def place_order(customer_id, products):
    ...
    order_id = ...
    ...
    return order_id


def handle_order_request():
    ...
    customer_id = ...
    purchased_products = ...
    ...
    return place_order(customer_id=customer_id, products=purchased_products)


if __name__ == "__main__":
    handle_order_request()

pytest test_source.py
========================================================================================== test session starts ==========================================================================================
collected 1 item                                                                                                                                                                                        

test_source.py .                                                                                                                                                                                  [100%]

=========================================================================================== 1 passed in 0.02s ===========================================================================================

The flake8-mock-spec linter is a helpful tool for ensuring responsible mock usage by enforcing compliance with the interface of the mocked object. For example, when run on the earlier test suite that doesn’t use the autospec argument, it reminds the us to make use of this feature, promoting safer and more efficient testing. By using this linter, you can ensure that your mock usage adheres to the defined interface, reducing the risk of divergence between the test and production environments, and making it easier to maintain the test suite.

flake8 test_source.py 
test_source.py:7:10: TMS020 unittest.mock.patch should be called with any of the spec_set, autospec, spec, new, new_callable arguments, more information: https://github.com/jdkandersson/flake8-mock-spec#fix-tms020

In summary, we discussed that mocking can speed up tests and avoid potentially costly interactions with external dependencies whilst running the risk of divergence between the mocked and production environment. We considered alternatives to mocking, such as using in-memory databases in your test suit. If mocking is required, we considered how to mock responsibly by defining high level interfaces with simple mocks that minimise the risk of divergence between the mocked and production environment. Finally, we explored the Python mocking feature that enforces mock interactions comply with the interfaces of the mocked object. You can ensure usage of this feature in your code base using the flake8-mock-spec linter. By following these guidelines, you can ensure that your tests run efficiently while minimising the risk of divergence between the test and production environments.