Property-Based Testing

Overview

Core principle: Instead of testing specific examples, test properties that should hold for all inputs.

Rule: Property-based tests generate hundreds of inputs automatically. One property test replaces dozens of example tests.

Property-Based vs Example-Based Testing

Aspect	Example-Based	Property-Based
Test input	Hardcoded examples	Generated inputs
Coverage	Few specific cases	Hundreds of random cases
Maintenance	Add new examples manually	Properties automatically tested
Edge cases	Must think of them	Automatically discovered

Example:

# Example-based: Test 3 specific inputs
def test_reverse():
    assert reverse([1, 2, 3]) == [3, 2, 1]
    assert reverse([]) == []
    assert reverse([1]) == [1]

# Property-based: Test ALL inputs
@given(lists(integers()))
def test_reverse_property(lst):
    # Property: Reversing twice returns original
    assert reverse(reverse(lst)) == lst

---

Tool Selection

Language	Tool	Why
Python	Hypothesis	Most mature, excellent shrinking
JavaScript/TypeScript	fast-check	TypeScript support, good integration
Java	jqwik	JUnit 5 integration
Haskell	QuickCheck	Original property-based testing library

First choice: Hypothesis (Python) or fast-check (JavaScript)

---

Basic Property Test (Python + Hypothesis)

Installation

pip install hypothesis

---

Example

from hypothesis import given
from hypothesis.strategies import integers, lists

def reverse(lst):
    """Reverse a list."""
    return lst[::-1]

@given(lists(integers()))
def test_reverse_twice(lst):
    """Property: Reversing twice returns original."""
    assert reverse(reverse(lst)) == lst

Run:

pytest test_reverse.py

Output:

Trying example: lst=[]
Trying example: lst=[0]
Trying example: lst=[1, -2, 3]
... (100 examples tested)
PASSED

If property fails:

Falsifying example: lst=[0, 0, 1]

---

Common Properties

1. Inverse Functions

Property: f(g(x)) == x

from hypothesis import given
from hypothesis.strategies import text

@given(text())
def test_encode_decode(s):
    """Property: Decoding encoded string returns original."""
    assert decode(encode(s)) == s

---

2. Idempotence

Property: f(f(x)) == f(x)

@given(lists(integers()))
def test_sort_idempotent(lst):
    """Property: Sorting twice gives same result as sorting once."""
    assert sorted(sorted(lst)) == sorted(lst)

---

3. Invariants

Property: Some fact remains true after operation

@given(lists(integers()))
def test_reverse_length(lst):
    """Property: Reversing doesn't change length."""
    assert len(reverse(lst)) == len(lst)

@given(lists(integers()))
def test_reverse_elements(lst):
    """Property: Reversing doesn't change elements."""
    assert set(reverse(lst)) == set(lst)

---

4. Commutativity

Property: f(x, y) == f(y, x)

@given(integers(), integers())
def test_addition_commutative(a, b):
    """Property: Addition is commutative."""
    assert a + b == b + a

---

5. Associativity

Property: f(f(x, y), z) == f(x, f(y, z))

@given(integers(), integers(), integers())
def test_addition_associative(a, b, c):
    """Property: Addition is associative."""
    assert (a + b) + c == a + (b + c)

---

Test Strategies (Generating Inputs)

Built-In Strategies

from hypothesis.strategies import (
    integers,
    floats,
    text,
    lists,
    dictionaries,
    booleans,
)

@given(integers())
def test_with_int(x):
    pass

@given(integers(min_value=0, max_value=100))
def test_with_bounded_int(x):
    pass

@given(text(min_size=1, max_size=10))
def test_with_short_string(s):
    pass

@given(lists(integers(), min_size=1))
def test_with_nonempty_list(lst):
    pass

---

Composite Strategies

Generate complex objects:

from hypothesis import strategies as st
from hypothesis.strategies import composite

@composite
def users(draw):
    """Generate user objects."""
    return {
        "name": draw(st.text(min_size=1, max_size=50)),
        "age": draw(st.integers(min_value=0, max_value=120)),
        "email": draw(st.emails()),
    }

@given(users())
def test_user_validation(user):
    assert validate_user(user) is True

---

Filtering Strategies

Exclude invalid inputs:

@given(integers().filter(lambda x: x != 0))
def test_division(x):
    """Test division (x != 0)."""
    assert 10 / x == 10 / x

# Better: Use assume
from hypothesis import assume

@given(integers())
def test_division_better(x):
    assume(x != 0)
    assert 10 / x == 10 / x

---

Shrinking (Finding Minimal Failing Example)

When a property fails, Hypothesis automatically shrinks the input to the smallest failing case.

Example:

@given(lists(integers()))
def test_all_positive(lst):
    """Fails if any negative number."""
    assert all(x > 0 for x in lst)

Initial failure:

Falsifying example: lst=[-5, 3, -2, 0, 1, 7, -9]

After shrinking:

Falsifying example: lst=[-1]

Why it matters: Minimal examples are easier to debug

---

Integration with pytest

# test_properties.py
from hypothesis import given, settings
from hypothesis.strategies import integers

@settings(max_examples=1000)  # Run 1000 examples (default: 100)
@given(integers(min_value=1))
def test_factorial_positive(n):
    """Property: Factorial of positive number is positive."""
    assert factorial(n) > 0

Run:

pytest test_properties.py -v

---

JavaScript Example (fast-check)

Installation

npm install --save-dev fast-check

---

Example

import fc from 'fast-check';

function reverse(arr) {
  return arr.slice().reverse();
}

// Property: Reversing twice returns original
test('reverse twice', () => {
  fc.assert(
    fc.property(fc.array(fc.integer()), (arr) => {
      expect(reverse(reverse(arr))).toEqual(arr);
    })
  );
});

---

Advanced Patterns

Stateful Testing

Test state machines:

from hypothesis.stateful import RuleBasedStateMachine, rule, initialize

class QueueMachine(RuleBasedStateMachine):
    def __init__(self):
        super().__init__()
        self.queue = []
        self.model = []

    @rule(value=integers())
    def enqueue(self, value):
        self.queue.append(value)
        self.model.append(value)

    @rule()
    def dequeue(self):
        if self.queue:
            actual = self.queue.pop(0)
            expected = self.model.pop(0)
            assert actual == expected

TestQueue = QueueMachine.TestCase

Finds: Race conditions, state corruption, invalid state transitions

---

Anti-Patterns Catalog

❌ Testing Examples, Not Properties

Symptom: Property test with hardcoded checks

# ❌ BAD: Not a property
@given(integers())
def test_double(x):
    if x == 2:
        assert double(x) == 4
    elif x == 3:
        assert double(x) == 6
    # This is just example testing!

Fix: Test actual property

# ✅ GOOD: Real property
@given(integers())
def test_double(x):
    assert double(x) == x * 2

---

❌ Overly Restrictive Assumptions

Symptom: Filtering out most generated inputs

# ❌ BAD: Rejects 99% of inputs
@given(integers())
def test_specific_range(x):
    assume(x > 1000 and x < 1001)  # Only accepts 1 value!
    assert process(x) is not None

Fix: Use strategy bounds

# ✅ GOOD
@given(integers(min_value=1000, max_value=1001))
def test_specific_range(x):
    assert process(x) is not None

---

❌ No Assertions

Symptom: Property test that doesn't assert anything

# ❌ BAD: No assertion
@given(integers())
def test_no_crash(x):
    calculate(x)  # Just checks it doesn't crash

Fix: Assert a property

# ✅ GOOD
@given(integers())
def test_output_type(x):
    result = calculate(x)
    assert isinstance(result, int)

---

CI/CD Integration

# .github/workflows/property-tests.yml
name: Property Tests

on: [pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.11'

      - name: Install dependencies
        run: pip install hypothesis pytest

      - name: Run property tests
        run: pytest tests/properties/ -v --hypothesis-show-statistics

---

Quick Reference: Property Patterns

Pattern	Example Property
Inverse	decode(encode(x)) == x
Idempotence	f(f(x)) == f(x)
Invariant	len(filter(lst, f)) <= len(lst)
Commutativity	add(a, b) == add(b, a)
Associativity	(a + b) + c == a + (b + c)
Identity	x + 0 == x
Consistency	sort(lst)[0] <= sort(lst)[-1]

---

Bottom Line

Property-based testing generates hundreds of inputs automatically to test properties that should hold for all inputs. One property test replaces dozens of example tests.

Use for:

• Pure functions (no side effects)
• Data transformations
• Invariants (sorting, reversing, encoding/decoding)
• State machines

Tools:

• Hypothesis (Python) - most mature
• fast-check (JavaScript) - TypeScript support

Process:

1. Identify property (e.g., "reversing twice returns original")
2. Write property test with generator
3. Run test (generates 100-1000 examples)
4. If failure, Hypothesis shrinks to minimal example
5. Fix bug, add regression test

If you're writing tests like "assert reverse([1,2,3]) == [3,2,1]" for every possible input, use property-based testing instead. Test the property, not examples.