There is some overlap between what could go into the setup phase of a test or into a fixture. There are a few reason why certain pieces of setup code are better suited to a fixture and also why other code might be better off in the test itself. This post explores a few examples to help illustrate how you can decide where to put the setup code to minimise the maintenance cost and maximise the value of the test suit.
Continue reading “Should this code be in the test or a fixture?”Tag: pytest
Reduce Duplication in Pytest Parametrised Tests using the Walrus Operator
Do you find yourself having to repeat literal values (like strings and integers) in parametrised tests? I often find myself in this situation and have been looking for ways of reducing this duplication. To show an example, consider this trivial function:
def get_second_word(text: str) -> str | None:
"""Get the second word from text."""
words = text.split()
return words[1] if len(words) >= 2 else None
Testing Python Connexion Optional Query Parameter Names
connexion is a great tool for building APIs in Python. It allows you to make the openAPI specification define input validation that is automatically enforced, maps API endpoints to a named function and handles JSON serialising and de-serialising for you.
One of the things to keep in mind when using connexion is that, if you have defined query parameters, only the query parameters that you have define can every get passed to the functions handling the endpoint. For example, consider the following API specification for retrieving employees from a database:
specification.yml
# specification.yml
openapi: "3.0.0"
info:
title: Sample API
description: API demonstrating how to check for optional parameters.
version: "0.1"
paths:
/employee:
get:
summary: Gets Employees that match query parameters.
operationId: library.employee.search
parameters:
- in: query
name: join_date
schema:
type: string
format: date
required: false
description: Filters employees for a particular date that they joined the company.
- in: query
name: business_unit
schema:
type: string
required: false
description: Filters employees for a particular business unit.
- in: query
name: city
schema:
type: string
required: false
description: Filters employees for which city they are working in.
responses:
200:
description: Returns all the Employees in the database that match the query parameters.
content:
application/json:
schema:
type: object
properties:
first_name:
description: The first name of the Employee.
type: string
last_name:
description: The last name of the Employee.
type: string
If you call /employee with some_parameter, you don’t receive an error because connexion will not call your function withsome_parameter as it is not defined as a query parameter in the specification and hence gets ignored. To help prove that, consider the following project setup.
Project Setup
The above specification.yml is included in the root directory. The library folder is also in the root directory. Inside the library folder is the following __init__.py file:
library/__init__.py
# library.__init__.py
from . import employee
The library folder contains the employee folder which has the following __init__.py and controller.py files:
library/employee/__init__.py
library/employee/controller.py
# library.employee.__init__.py
from .controller import search
# library.data.employee.controller.py
def search(join_date = None, business_unit = None, city = None):
return [{'first_name': 'David', 'last_name': 'Andersson'}]
Test Setup
pytest is used to test the API. To help with the API tests, pytest-flask is used. To show that calling the /employee endpoint with parameters that aren’t in the specification does not result in an error, the following test configuration is placed in the root directory:
conftest.py
# conftest.py
from unittest import mock
import pytest
import connexion
import library
@pytest.fixture(scope='session')
def setup_mocked_search():
"""Sets up spy fr search function."""
with mock.patch.object(library.employee, 'search', wraps=library.employee.search) as mock_search:
yield mock_search
@pytest.fixture(scope='function')
def mocked_search(setup_mocked_search):
"""Resets search spy."""
setup_mocked_search.reset_mock()
return setup_mocked_search
@pytest.fixture(scope='session')
def app(setup_mocked_search):
"""Flask app for testing."""
# Adding swagger file
test_app = connexion.FlaskApp(__name__, specification_dir='.')
test_app.add_api('specification.yml')
return test_app.app
For now, focus on the app fixture which uses the standard connexion API setup and then yields the flask application. Through the magic of purest-flask, we can then define the following test to demonstrate that the API can be called with parameters not named in the openAPI specification:
def test_some_parameter(client):
"""
GIVEN a value for some parameter
WHEN GET /employee is called with the some_parameter set to the some parameter value
THEN no exception is raised.
"""
client.get('/employee?some_parameter=value 1')
Verifying Endpoint Parameter Names
This means that making a mistake with the query parameter names in the openAPI specification and the arguments for the function that handles the endpoint can go undetected. To check this is not the case, you could come up with a number of tests that only pass if the query parameter was correctly passed to the search function. This can be tedious without mocking the search function as you would have to create tests with just the right setup so that the effect of a particular query parameter is noticed.
There is an easier way with mocks. Unfortunately, it is not as easy as monkey patching the search function in the body of a test function. The problem is that, after the app fixture has been called, the reference to the search function has been hard coded into the flask application and will not get affected by mocking the search function. It is possible to retrieve the reference to the search function in the flask application instance, but you will have to dig fairly deep and may have to use some private member variables, which is not advisable.
Instead, let’s take another look at the contest.py file above. The trick is to apply a spy to the search function before the app fixture is invoked. The reason a spy rather than a mock is used is because the spy will be in p0lace for all tests and we may want to write a test where the real search function is called! With a spy the underlying function still gets called, which it does not if a mock is used instead of the search function. The fixture on lines 8-12 adds a spy to the search function. To ensure that it gets called before the app fixture, the app fixture takes it as an argument, although it doesn’t make use of the fixture. Because the app fixture is a session level fixture, we also want the fixture that sets up the spy to be a session level fixture, otherwise all the tests will be slowed down. However, this means that the spy state will leak into other tests. This can be solved with a fixture similar to the fixture on lines 15-19 which resets the spy state before each test.
Now we can use the test client to define tests that check that each of the query parameters is passed to the search function. The tests that perform these checks may look like the following:
test.py
# test.py
"""Endpoint tests for /employee"""
def test_some_parameter(client, mocked_search):
"""
GIVEN library.data.employee.search has a spy and a value for some parameter
WHEN GET /employee is called with the some_parameter set to the some parameter value
THEN library.data.employee.search is called with no parameters.
"""
client.get('/employee?some_parameter=value 1')
# Checking search call
mocked_search.assert_called_once_with()
def test_join_date(client, mocked_search):
"""
GIVEN library.data.employee.search has a spy and a date
WHEN GET /employee is called with the join_date set to the date
THEN library.data.employee.search is called with the join date.
"""
client.get('/employee?join_date=2000-01-01')
# Checking search call
mocked_search.assert_called_once_with(join_date='2000-01-01')
def test_business_unit(client, mocked_search):
"""
GIVEN library.data.employee.search has a spy and a business unit
WHEN GET /employee is called with the business_unit set to the business unit
THEN library.data.employee.search is called with the business unit.
"""
client.get('/employee?business_unit=business unit 1')
# Checking search call
mocked_search.assert_called_once_with(business_unit='business unit 1')
def test_city(client, mocked_search):
"""
GIVEN library.data.employee.search has a spy and a city
WHEN GET /employee is called with the city set to the city
THEN library.data.employee.search is called with the city.
"""
client.get('/employee?city=city 1')
# Checking search call
mocked_search.assert_called_once_with(city='city 1')
You now have a starting point for how to test that the function arguments and query parameter names in the specification match. You may not want to create these tests for each query parameter for every endpoint, especially if a query parameter is required, as an exception will be raised because the underlying function will be called with a query parameter which is not listed in the function signature.
Testing Decorated Python Functions
Decorators are a great way of adding functionality to a function with minimal impact on the function itself. On top of that, decorator logic can be re-used on other functions that also require the new functionality. For example the following decorator prints a message to standard output every time a function is called.
# plain_main.py
"""Demonstrates a simple decorator."""
def decorator(func):
"""
A simple decorator that adds printing a message on a function call.
Args:
func: The function to decorate.
Returns:
The decorated function.
"""
def inner(*args, **kwargs):
"""Function that is called instead of original function."""
print('The decorator was called.')
return func(*args, **kwargs)
return inner
@decorator
def main():
print('The main function was called.')
if __name__ == '__main__':
print('Calling the main function.')
main()
$ python3 plain_main.py
Calling the main function.
The decorator was called.
The main function was called.
The drawback of decorators is that the decorator is applied as soon as the interpreter reaches the function definition and it is hard to access the original function without the decorator applied. This might be desirable during testing where testing of the function and the decorator should be separated.
Adding Testing Guard Logic to a Decorator
The solution is to optionally skip the decorator logic if a certain condition is met that is only true during testing. For example, skip the decorator logic if the TESTING environment variable is set.
# check_main.py
"""Demonstrates a decorator with a testing guard."""
import os
def decorator(func):
"""
A simple decorator that adds printing a message on a function call unless
the TESTING environment variable is set.
Args:
func: The function to decorate.
Returns:
The decorated function.
"""
def inner(*args, **kwargs):
"""Function that is called instead of original function."""
# Checking for TESTING environment variable
if os.getenv('TESTING') is not None:
# Skipping decortor logic
return func(*args, **kwargs)
# Running decorator logic
print('The decorator was called.')
return func(*args, **kwargs)
return inner
@decorator
def main():
print('The main function was called.')
if __name__ == '__main__':
print('Calling the main function without TESTING set.')
main()
print('Calling the main function with TESTING set.')
os.environ['TESTING'] = ''
main()
$ python3 check_main.py
Calling the main function without TESTING set.
The decorator was called.
The main function was called.
Calling the main function with TESTING set.
The main function was called.
As you can see, the decorator logic was executed under normal circumstances (main call on line 39) and was skipped when the TESTING environment variable was set (main call on line 43). The reason was because of the guard statement on line 21 that checks for the TESTING environment variable.
Guard Decorator
You might now say: “great, thank you David. Now I have to rewrite all of my decorator functions!” Ah, but you don’t. If you stay with me through a little more complex decorator code, you won’t have to! The idea is to write a decorator that modifies another decorator’s behaviour.
# guard_main.py
"""Demonstrates a guard decorator."""
import os
def testing_guard(decorator_func):
"""
Decorator that only applies another decorator if the TESTING environment
variable is not set.
Args:
decorator_func: The decorator function.
Returns:
Function that calls a function after applying the decorator if TESTING
environment variable is not set and calls the plain function if it is set.
"""
def replacement(original_func):
"""Function that is called instead of original function."""
def apply_guard(*args, **kwargs):
"""Decides whether to use decorator on function call."""
if os.getenv('TESTING') is not None:
return original_func(*args, **kwargs)
return decorator_func(original_func)(*args, **kwargs)
return apply_guard
return replacement
@testing_guard
def decorator(func):
"""
A simple decorator that adds printing a message on a function call.
Args:
func: The function to decorate.
Returns:
The decorated function.
"""
def replacement(*args, **kwargs):
"""Function that is called instead of original function."""
print('The decorator was called.')
return func(*args, **kwargs)
return replacement
@decorator
def main():
print('The main function was called.')
if __name__ == '__main__':
print('Calling the main function without TESTING set.')
main()
print('Calling the main function with TESTING set.')
os.environ['TESTING'] = ''
main()
$ python3 guard_main.py
Calling the main function without TESTING set.
The decorator was called.
The main function was called.
Calling the main function with TESTING set.
The main function was called.
As you can see, the behaviour of the code is exactly the same but the decorator is in its original form. The reason this works is because the guard decorator gets to intercept each function call and can then decide whether to first apply the decorator or call the plain function on lines 23-25.
On top of not having to re-write decorator functions which you don’t want to execute during testing, you also get to separate the logic that determines whether the decorator is applied from the decorator logic which will reduce the chances of accidentally executing decorator logic as you make changes to the decorator. It also helps you write clear unit tests for both the decorator and the guard decorator.
Working with Pytest
The last consideration is how do you apply this in practice. I would argue that, unless you are testing the decorator or guard decorator, you should always have the TESTING environment variable set. This ensures that you are only testing function logic and not decorator logic. You can achieve this by putting a fixture in your root conftest.py file with autouse set to True.
@pytest.fixture(scope='function', autouse=True)
def set_testing(monkeypatch):
"""Sets the TESTING environment variable."""
monkeypatch.setenv('TESTING', '')
When you are testing the decorator you would always ensure that the TESTING environment variable is not set. You can achieve that using a fixture that clears the TESTING environment variable that also has autouse set to True as a part of the file that tests the decorator.
@pytest.fixture(scope='function', autouse=True)
def delete_testing(monkeypatch):
"""Deletes the TESTING environment variable."""
monkeypatch.delenv('TESTING', raising=False)
Finally, for testing the guard decorator, make whether the TESTING environment variable is set part of the tests themselves. Don’t be shy about overriding the functionality of any autouse fixtures as a part of the test function to demonstrate what the intended state of the test is clearly.
def test_guard_decorator_testing_set(monkeypatch):
"""
GIVEN TESTING environment variable set and ...
WHEN ...
THEN ...
"""
# Setting TESTING environment variable
monkeypatch.setenv('TESTING', '')
# Other test code
def test_guard_decorator_testing_not_set(monkeypatch):
"""
GIVEN TESTING environment variable is not set and ...
WHEN ...
THEN ...
"""
# Setting TESTING environment variable
monkeypatch.delenv('TESTING', raising=False)
# Other test code
Thats it! Consider whether environment variables is the best way of indicating that decorator logic should be skipped during testing and also what the best name of the environment variable is for you. I hope this was useful to you!
David Andersson 