Skip to content

python-proptest

python-proptest is a property-based testing (PBT) framework ported from for cppproptest, drawing inspiration from libraries such as Haskell's QuickCheck and Python's Hypothesis. Property-based testing shifts the focus from example-based verification to defining universal properties or invariants that must hold true for an input domain.

python-proptest provides seamless integration with unittest or pytest through the convenient decorators like @for_all, automatically detecting pytest context and adapting behavior accordingly.

Instead of manually crafting test cases for specific inputs, PBT allows you to describe the domain of inputs your function expects and the general characteristics of the output (e.g., add(a, b) should always be greater than or equal to a and b if they are non-negative). PBT then generates hundreds or thousands of varied inputs, searching for edge cases or unexpected behaviors that violate your defined properties. This approach significantly increases test coverage and the likelihood of finding subtle bugs.

The core workflow involves:

  1. Defining a property: A function that takes generated inputs and asserts an expected invariant. See Properties.
  2. Specifying generators: Mechanisms for creating random data conforming to certain types or constraints, often built by composing simpler generators using combinators. See Generators and Combinators.
  3. Execution: python-proptest automatically runs the property function against numerous generated inputs (typically 100+).
  4. Shrinking: If a test case fails (the property returns false or throws), python-proptest attempts to find a minimal counterexample by simplifying the failing input. See Shrinking.
  5. Enhanced testing: Use decorators like @example, @settings, and @matrix for specific test cases, configuration, and exhaustive testing. See Decorators.

Consider verifying a round-trip property for a custom parser/serializer:

import json
from python_proptest import run_for_all, Gen

"""Test that serializing and parsing preserves data."""
# Generator for keys (non-empty strings without special characters)
key_gen = Gen.str(min_length=1, max_length=10).filter(
    lambda s: s and '&' not in s and '=' not in s
)
# Generator for arbitrary string values
value_gen = Gen.str(min_length=0, max_length=10)
# Generator for dictionaries with our specific keys and values
data_object_gen = Gen.dict(key_gen, value_gen, min_size=0, max_size=10)

# Simple lambda-based property is suitable for run_for_all
result = run_for_all(
    lambda original_data: json.loads(json.dumps(original_data)) == original_data,
    data_object_gen
)
assert result is True

This PBT approach facilitates the discovery of edge cases and intricate bugs that might be neglected by traditional, example-driven testing methodologies.

Getting Started

Installation

To add python-proptest to your project, run the following command:

pip install python-proptest

For development dependencies:

pip install python-proptest[dev]

Core Concepts and Features

Understanding these key components will help you use python-proptest effectively:

  • Generators: Produce random data of various types (primitives, containers) according to specified constraints (e.g., Gen.int(), Gen.list(...)). Learn how to create the basic building blocks of your test data.

  • Combinators: Modify or combine existing generators to create new ones. Discover techniques to constraint, combine, and transform generators for complex data structures.

  • Properties (Property, run_for_all): Express conditions or invariants that should hold true for generated data. python-proptest runs these properties against many generated examples using the run_for_all function or Property class methods. Understand how to define the invariants your code should satisfy and how to run tests.

  • Shrinking: When a property fails, python-proptest attempts to find a minimal counterexample by simplifying the failing input using logic associated with the generated value (often via a Shrinkable structure). See how python-proptest helps pinpoint failures.

  • Stateful Testing: Go beyond simple input-output functions and test systems with internal state by generating sequences of operations or commands. Learn how to model and verify stateful behaviors.

API Overview

python-proptest provides two main approaches for property-based testing:

Available Generators

Primitive Generators:

Container Generators:

Special Generators:

Dependent Generation Combinators:

Selection Combinators:

Transformation Combinators:

Decorators:

1. Function-based Approach (Works with both pytest and unittest)

from python_proptest import run_for_all, Gen

def test_addition_commutativity():
    def property_func(x: int, y: int):
        return x + y == y + x

    run_for_all(property_func, Gen.int(), Gen.int())

2. Decorator-based Approach

from python_proptest import for_all, Gen, example, settings, matrix

@for_all(Gen.int(), Gen.int())   # Test domain with random generated values
@matrix(x=[0, 1], y=[0, 1])      # Test combination of edge cases exhaustively
@example(42, 24)                 # Test specific known values
@settings(num_runs=50, seed=42)  # Configure test parameters
def test_addition_commutativity(x: int, y: int):
    assert x + y == y + x

# Run the test
test_addition_commutativity()

3. Framework Integration

The @for_all decorator integrates with both unittest and pytest using direct decoration:

Unittest Integration: 

import unittest
from python_proptest import for_all, Gen

class TestMathProperties(unittest.TestCase):
    @for_all(Gen.int(), Gen.int())
    def test_addition_commutativity(self, x: int, y: int):
        """Test that addition is commutative - direct decoration."""
        self.assertEqual(x + y, y + x)

Pytest Integration: 

import pytest
from python_proptest import for_all, Gen

class TestMathProperties:
    @for_all(Gen.int(), Gen.int())
    def test_addition_commutativity(self, x: int, y: int):
        """Test that addition is commutative - direct decoration."""
        assert x + y == y + x

Choosing the Right Approach

python-proptest provides multiple approaches for defining property tests. Choose based on your needs:

Use run_for_all Function for Simple Lambda-Based Tests

Suitable for simple property checks that can be expressed as lambdas:

from python_proptest import run_for_all, Gen

# Type checks
result = run_for_all(
    lambda x: isinstance(x, int),
    Gen.int(min_value=0, max_value=100)
)

# Range validations
result = run_for_all(
    lambda x: 0 <= x <= 100,
    Gen.int(min_value=0, max_value=100)
)

# Simple assertions
result = run_for_all(
    lambda lst: all(isinstance(x, int) for x in lst),
    Gen.list(Gen.int())
)

Use @run_for_all Decorator for Named Functions

While standalone run_for_all function can only work on lambdas, @run_for_all decorator can work on a named function.

from python_proptest import run_for_all, Gen

gen = Gen.chain(Gen.int(1, 10), lambda x: Gen.int(x, x + 10))

@run_for_all(gen, num_runs=50)
def check_dependency(pair):
    base, dependent = pair
    self.assertGreaterEqual(dependent, base)
    self.assertLessEqual(dependent, base + 10)
# Auto-executes when decorated!


gen = Gen.aggregate(
    Gen.int(0, 10),
    lambda n: Gen.int(n, n + 5),
    min_size=5, max_size=10
)

@run_for_all(gen, num_runs=30)
def check_increasing(values):
    for i in range(1, len(values)):
        self.assertGreaterEqual(values[i], values[i - 1])

Use @for_all for Test Frameworks

@for_all decorator can be used to create a property-based test in a test frameworks like unittest or pytest. Method parameters are randomized according to the specified generators and the method body is wrapped within a property-based test loop. As a result, the method becomes a conforming test case of the test framework.

import unittest
from python_proptest import for_all, Gen

class TestStringProperties(unittest.TestCase):
    @for_all(Gen.str(), Gen.str())
    def test_string_concatenation(self, s1: str, s2: str):
        """Test string concatenation properties."""
        result = s1 + s2
        self.assertEqual(len(result), len(s1) + len(s2))
        self.assertTrue(result.startswith(s1))
        self.assertTrue(result.endswith(s2))

Note that @for_all decorator does not itself executes the test and let it execute by the containing framework whereas @run_for_all does the execution as well.

from python_proptest import for_all, Gen

@for_all(Gen.int(), Gen.int())
def test_complex_math_property(x: int, y: int):
    """Test complex mathematical property with multiple conditions."""
    result = x * y + x + y
    assert result >= x
    assert result >= y
    assert result % 2 == (x + y) % 2

# need to call the function explicitly to execute the property test
test_complex_math_property()

# this one does not need explicit call
@run_for_all(Gen.str(), Gen.str())
def test_string_operations(s1: str, s2: str):
    """Test string operations with multiple assertions."""
    combined = s1 + s2
    assert len(combined) == len(s1) + len(s2)
    assert combined.startswith(s1)
    assert combined.endswith(s2)

Guidelines

  • Use run_for_all function for immediately testing a property with a lambda with single statement
  • Use @run_for_all decorator for immediately testing a property with a named functions with longer body
  • Use @for_all for use in test frameworks like unittest or pytest.

All approaches reach the same goal - choose based on your testing framework and preferences. For more details on decorators, see Decorators. For framework integration, see Pytest Integration, Unittest Integration, and Pytest Best Practices.