Pages

Topics

Tools

↑↓navigate↵openescclose
All guides

Python Hypothesis Property-Based Testing Basics

Learn how to write property-based tests in Python with Hypothesis: describe invariants, generate inputs with strategies, build composite data, and debug failures through shrinking.

~6 min readPython
#property-based testing#hypothesis#python testing#test strategies
Illustration of property-based testing with Python Hypothesis showing generated inputs and a shrinking failure.

Writing unit tests for a function that accepts a wide range of inputs often means manually listing edge cases. For a simple function like reverse, you might test an empty list, a single element, a list with duplicates, and a large list. Property-based testing with Python Hypothesis flips this process: you describe a property that should always hold, and Hypothesis generates many inputs to verify it.

The Core Idea: Properties Instead of Examples

In example-based testing, you write a test that calls the function with a specific input and asserts a specific output. Property-based testing instead asserts an invariant that must hold for a whole class of inputs. For reverse, a natural property is that reversing a list twice returns the original list. Another property is that the first element of the reversed list is the last element of the original list.

Here is how that looks with Hypothesis:

python
from hypothesis import given, strategies as st @given(st.lists(st.integers())) def test_reverse_twice_is_identity(lst): assert lst[::-1][::-1] == lst

Hypothesis will generate many lists of integers and run this test with each one. If any input causes the assertion to fail, Hypothesis reports the minimal failing input.

Minimal Setup: Installing Hypothesis and Writing a First Test

Install Hypothesis with pip:

bash
pip install hypothesis

Then create a test file, for example test_reverse.py. The test above can be run by pytest directly. Hypothesis also integrates with unittest if you place the test in a subclass of unittest.TestCase. For a standalone script, you can call test_reverse_twice_is_identity() directly, but the typical workflow is to run it through a test runner.

The @given decorator takes one or more strategies as arguments. Each argument corresponds to a parameter of the test function. In the example, lst is generated by st.lists(st.integers()). Hypothesis runs the test with a default of 100 examples, though you can change that with max_examples.

Using Strategies to Define Input Space

Strategies describe the set of values that Hypothesis can generate. The strategies module provides a wide range of built-in strategies:

  • st.integers() generates arbitrary integers.
  • st.text() generates Unicode strings.
  • st.floats() generates floating-point numbers.
  • st.lists(elements) generates lists of values produced by elements.
  • st.dictionaries(keys, values) generates dictionaries.
  • st.booleans() generates True and False.

You can constrain these strategies to focus on realistic inputs. For example, st.integers(min_value=1, max_value=100) only generates integers from 1 to 100. st.lists(st.integers(), min_size=1, max_size=10) generates lists with between 1 and 10 elements. This is useful when your function has preconditions.

Strategies can also be transformed. The .map() method applies a function to each generated value, and .filter() restricts the values to those that satisfy a predicate. For example:

python
positive_ints = st.integers().filter(lambda x: x > 0)

However, be careful with .filter() when the predicate is too restrictive, because Hypothesis may struggle to find valid inputs.

Combining Strategies for Complex Inputs

For data structures that have multiple fields, you can use the @composite decorator to combine strategies. This is cleaner than chaining .map() and .flatmap().

Suppose you need to generate a dictionary representing a user with a name, age, and email. You can define a composite strategy:

python
from hypothesis import given, strategies as st @st.composite def user_strategy(draw): name = draw(st.text(min_size=1, max_size=20)) age = draw(st.integers(min_value=0, max_value=120)) email = draw(st.text(min_size=5, max_size=50)) return {"name": name, "age": age, "email": email} @given(user_strategy()) def test_user(user): assert user["age"] >= 0 assert len(user["name"]) > 0

The draw function inside the composite strategy pulls a value from the given strategy. This lets you build complex inputs that depend on other generated values. You can also use assume inside a composite to discard invalid combinations, but use it sparingly because it can reduce the efficiency of the search.

How Hypothesis Handles Failures: Shrinking and Reproducing

When a property fails, Hypothesis does not simply report the first failing input. It attempts to shrink the input to a minimal failing example, which makes debugging much easier. For example, if a list of integers fails a property, Hypothesis will try to remove elements and reduce the values to find the smallest list that still fails.

You can see this in action by intentionally writing a buggy function. Suppose you have a function that incorrectly handles negative numbers:

python
def abs_all(numbers): return [abs(n) for n in numbers if n >= 0]

A property test that asserts abs_all(numbers) == [abs(n) for n in numbers] would fail for inputs containing negative numbers. Hypothesis would shrink the failing input to something like [-1] instead of a long random list.

To reproduce a specific failure, Hypothesis prints a @reproduce_failure decorator with the version and blob. You can paste that decorator above the test to rerun the exact failing input. This is useful when you want to debug without rerunning the full search.

Property-based tests generate and execute many examples, so they are typically slower than a few hand-written example tests. The default max_examples is 100, which is enough for many cases but can be increased to 1000 if you need more thorough coverage. Conversely, you can lower it to 10 for a quick smoke test during development.

Hypothesis also supports a deadline setting to enforce a time limit per example. If an example takes longer than the deadline, the test fails. This is useful for catching performance regressions, but it can be flaky on slow CI machines, so set it carefully.

The runtime cost is usually acceptable because Hypothesis uses a database of previously found failures to focus on edge cases. It also uses a heuristic search that adapts to the input space. For most unit tests, the added confidence outweighs the extra seconds.

Common Mistakes When Writing Property Tests

One common mistake is writing a property that is too weak. For example, testing that a function returns a list of the same length does not verify that the elements are correct. The property should capture the essential behavior of the function.

Another mistake is using strategies that are too narrow. If you restrict inputs to a small range, you may miss the edge cases that property-based testing is meant to find. Let the strategy cover the full domain unless the function has documented preconditions.

Overusing assume can also hurt. assume filters out generated inputs, and if the filter is too strict, Hypothesis may spend many examples rejecting inputs. Prefer to design strategies that only generate valid inputs in the first place.

Finally, remember that property-based tests are not a replacement for example-based tests. They complement each other. Use property tests for invariants and example tests for specific regressions.

When to Choose Property-Based Testing Over Example-Based Tests

Property-based testing shines when you have a function with a large or complex input space, and you can state a clear invariant that must hold. Sorting functions, serializers, parsers, and data transformations are good candidates. If you find yourself writing many example-based tests to cover edge cases, a property test might reduce that burden.

On the other hand, example-based tests are better when the expected behavior is known for a specific input, such as a known bug fix or a documented API contract. They are also easier to read for non-technical stakeholders because they show concrete input-output pairs.

A practical approach is to start with a few example-based tests to lock in known behavior, then add property-based tests to explore the input space. When a property test finds a bug, add that specific case as an example-based regression test.

Try matching browser tools

All tools

Small utilities that fit Python work — they run entirely in this browser tab, so nothing you paste leaves your device.

All Python guides

More in Python

All Python guides

The guides filed next to this one in Python, in library order.