Help your Users fix your Errors

Have you ever encountered an error when using a package and then gone to Google to find out how to solve the error only not to find any clear answers? Wouldn’t you have preferred to go directly to documentation that tells you exactly what went wrong and how to resolve that error? A lot of us can tell similar stories, especially when we try something new. I have also abandoned an otherwise promising package after encountering an error that wasn’t clear or when I didn’t know how to solve the error.

Many of us create tools, web applications, packages, you name it, that are used by other people. So we may well have the opportunity to turn what would have been a frustrating experience for our users into a reason to keep using that tool, web application or package by improving the way we communicate errors to our users.

Before getting into the specifics, first let’s define an error to highlight the areas where we can improve our user’s experience. Errors come in different forms, it may be an exception raised by a package as we are developing or testing an application (hopefully before we deploy to production). It could be a web API returning a 4XX series error or it could be a error message on a form a user is filling. It could also be an entry our users see in the production logs at 4 am on Sunday after a long night partying. In all of these cases, our users want the quickest path to resolve the error so they can go back to sleep, finish filling in that form before logging off or complete that great new feature they have been working on using your package. You can see that this topic is applicable whenever you are creating something for someone else, and sometimes that someone else is you using your own tool a few months later and you have forgotten exactly how to use it.

Next, let’s discuss what we can do to improve communicating errors to our users. There are three techniques we will discuss, the first is when to raise an error, the second is explaining what went wrong and the third topic, which is often missed yet is immensely valuable to our users, is helping our users resolve the error.

Starting with when to raise an error. The obvious answer is when something has gone fatally wrong and your code can’t do anything to recover. There are two concepts in that statement worth exploring. The first is “when something has gone fatally wrong” with the key part being when and the question being how do you know when as your are coding. As a user, I have encountered many cases where I have used a package and an error occurs like the package encountered a HTTP 4XX error when communicating with a server. As a user, I have no idea why that error occurred or how to solve it. Presumably, the 4XX error is avoidable so it is likely that an argument or configuration I have passed to the package was wrong or I have not provided some essential input. So it would be better if the package looked at the input provided to it, validate it and raise an error during the validation to make it clear to the user that the input is wrong, rather than raising it later when it encounters an internal problem. Users will understand errors raised in response to input as they will likely have seen them before. The second part is “your code can’t do anything to recover.” Being proactive with trying to recover from errors is more advanced, although it is sometimes possible. It is like, rather than a code formatter giving you an error, it fixes the formatting issue for you. Another example is rather than telling your users not to have trailing whitespace in input, removing that trailing whitespace for them. Some judgement is required when doing this as it can be expensive to write this cleanup code and also could introduce other interesting errors, although your users will likely benefit from your error recovery efforts.

Moving on to explaining what went wrong. It is easy to describe a 4XX error in your code as “encountered an error communicating with the server.” Whilst that is true, it isn’t helpful to your users. The user will need information about exactly what they did wrong to cause the error. It is like a teacher telling a student that the essay is wrong without pointing our, for example, spelling mistakes or logical flaws in an argument. Just telling the student that something generic is wrong makes it harder for the student to know what to do to remedy the situation, and may lead to many random attempts by the user to figure out exactly what the problem is. So always link any error that you raise back to something the user did to cause that error, or be clear that the user cannot do anything to resolve the error (like 5XX series HTTP errors).

Finally, let’s consider the topic that many developers miss, telling users how to fix an error. This part is like the teacher giving the student some examples of how they can structure their arguments to improve the reasoning. Or a word processor suggesting spelling corrections. An example in the context of a package might be explaining that the user needs to supply an API token and providing a link to documentation for how to generate that API token directly in the error message. In some cases, fixes can be provided directly in the error message, such as a linter telling the user that the variable name is too short and giving them the minimum number of characters variable names should have. More often, fixing an error benefits from more detailed documentation, such as the example earlier on how to generate API tokens. Even in the case of the linter, it would be valuable to point to documentation that discusses why it is important not to use a single character for variable names and some general principles around what makes a good variable name.

Partially to learn how to write a linter, also to provide a tool to people to help remember to include links to documentation in errors and primarily to help improve the way errors are raised in the Python community, I created a linter for checking that any exceptions raised in your code include a link for more information about the error. It is a flake8 plugin available through pip, you can find a getting started guide here: https://github.com/jdkandersson/flake8-error-link

To demonstrate how it works, let’s say I write the following command line utility that greets users:

# test.py
def main():
    """Say hello to users."""
    parser = argparse.ArgumentParser(prog="SayHello", description="Greets users.")
    parser.add_argument("name", help="Your name")

    name = parser.parse_args().name

    if name.startswith(" ") or name.endswith(" "):
        raise ValueError("Surely your name doesn't have leading or trailing whitespace!")

    print(f"Hi there, {name}!")


if __name__ == "__main__":
    main()

This is how the utility behaves when called from the command line:

$ python test.py ghost 
Hi there, ghost!
$ python test.py "ghost "
Traceback (most recent call last):
  File "/workspaces/flake8-error-link/test.py", line 18, in <module>
    main()
  File "/workspaces/flake8-error-link/test.py", line 12, in main
    raise ValueError("Surely your name doesn't have leading or trailing whitespace!")
ValueError: Surely your name doesn't have leading or trailing whitespace!

When the flake8-error-link linter is run on this code:

$ flake8 test.py
test.py:12:9: ELI001 builtin exceptions should be raised with a link to more information: https://github.com/jdkandersson/flake8-error-link#fix-eli001

You can see that the ELI001 error is raised reminding us to include a link to more information. (As a side note, see how the linter message actually includes a ink to more information on how to fix the problem!) When we comply by changing the code to:

# test.py
def main():
    """Say hello to users."""
    parser = argparse.ArgumentParser(prog="SayHello", description="Greets users.")
    parser.add_argument("name", help="Your name")

    name = parser.parse_args().name

    if name.startswith(" ") or name.endswith(" "):
        raise ValueError(
            "Surely your name doesn't have leading or trailing whitespace!",
            "more information: https://jdkandersson.com/2022/12/20/help-your-users-fix-your-errors/",
        )

    print(f"Hi there, {name}!")


if __name__ == "__main__":
    main()

The linting problem is resolved:

$ flake8 test.py

Some notes on using the linter:

• It will look for any string constant with a link to more information in the arguments to the exception constructor. It knows how to handle things like f-strings. If you find any cases that are not handled correctly, please leave an issue on the repository: https://github.com/jdkandersson/flake8-error-link/issues!
• There are a few error codes that target different scenarios. For example, ELI001 is only used for builtin exceptions and ELI002 is used for custom exceptions. This is because you may chose to include the link to more information directly on your definition of the exception. In that case, you can disable ELI002 so that the linter knows not to check custom exceptions. (If you re-define builtin exceptions, you are on your own 😛). I would recommend to provide a custom link whenever you raise an exception rather than using a generic link so that you point your users to exactly the right place in the documentation explaining how to solve their specific error.
• Similarly, ELI003 is used in cases where the more information link is not found and the linter detects you are passing a variable as an argument. ELI004 is used for re-raising exceptions.
• You can configure the linter with a custom regular expression for detecting the link to more information using the --error-link-regex argument. By default, it looks for more information: <link> in any of the string constants in the arguments of the exception constructor.

In summary, depending on how your raise an error, it can drive away your users or turn them into advocates for what you created for them. When you have to raise an error, raise it as early as possible and after trying to recover from it. When you write the error message, explain what went wrong in terms of what the user did wrong to cause the error, or be clear that the user cannot do anything to resolve the error. Finally, include information on how to fix the error and it is generally useful to include a link to your documentation describing the error and possible resolutions in more detail. The flake8-error-link flake8 plugin can help you ensure that you always include a link to more information whenever you raise an exception.