Testing & Code Quality
What you’ll learn. What tests are for and how to spend your testing budget; the test pyramid and its modern variants; how to write unit, integration, property-based and contract tests; test-driven development as a design technique; the difference between mocks, stubs, fakes and spies (and why fakes usually win); mutation testing as a test of your tests; safe refactoring; and how to give and receive code review. Examples are Python (pytest, Hypothesis) and TypeScript (Vitest/Jest style), labelled.
What tests are for
Section titled “What tests are for”Tests serve four purposes: regression protection (catch breakage when code changes), design feedback (hard-to-test code is usually badly coupled; see Core Principles), executable documentation (examples of intended behavior), and confidence to change (the ability to refactor and deploy frequently). Tests cannot prove the absence of bugs (Dijkstra), only reveal their presence; the aim is economically justified confidence.
A good test suite is:
- Fast: seconds for the inner loop; a slow suite is run less often.
- Deterministic: no flakiness; a flaky test teaches people to ignore failures.
- Isolated: tests do not depend on each other’s order or state.
- Readable: a test is a specification; the intent should be clear.
- Resilient to refactoring: tests assert behavior, not implementation. Tests that break when you rename a private method cost more than they give.
- Specific on failure: the failure message points to the cause.
The test pyramid and its variants
Section titled “The test pyramid and its variants”Mike Cohn’s test pyramid: many fast unit tests at the base, fewer integration (service) tests in the middle, and few slow end-to-end (UI) tests at the top.
/\ E2E / UI tests: few, slow, brittle, high realism / \ /----\ Integration tests: moderate number, real dependencies (DB, queue) / \ /--------\ Unit tests: many, milliseconds, isolatedRationale: cost (speed, maintenance, flakiness) and diagnosability rise with scope; fault localization falls. Variants: the testing trophy (Kent C. Dodds; emphasizes integration tests, with static analysis at the base) and the testing honeycomb (Spotify; for microservices, mostly integration tests of a service in isolation with few unit and few end-to-end tests). Which shape fits depends on where bugs live: logic-heavy code benefits from unit tests; glue-heavy code (CRUD, microservices) from integration tests. Do not treat the shape as dogma; ask “where do our defects come from, and what is the cheapest test that would catch them?”
Static checks form the real base: type checkers (mypy, pyright, TypeScript), linters, formatters. Strong types eliminate whole classes of tests.
Unit tests
Section titled “Unit tests”A unit test exercises a small piece of behavior (a function, a class) in isolation, fast, with no I/O.
Structure each test as Arrange-Act-Assert (or Given-When-Then), one logical behavior per test, descriptive names:
# Python (pytest)import pytestfrom decimal import Decimal
def apply_discount(total: Decimal, code: str) -> Decimal: rates = {"SAVE10": Decimal("0.10"), "SAVE20": Decimal("0.20")} if code not in rates: raise ValueError(f"unknown code: {code}") return (total * (1 - rates[code])).quantize(Decimal("0.01"))
def test_valid_code_reduces_total(): assert apply_discount(Decimal("100.00"), "SAVE10") == Decimal("90.00")
def test_unknown_code_is_rejected(): with pytest.raises(ValueError, match="unknown code"): apply_discount(Decimal("100.00"), "BOGUS")
@pytest.mark.parametrize("total,code,expected", [ ("100.00", "SAVE10", "90.00"), ("59.99", "SAVE20", "47.99"),])def test_discounts(total, code, expected): assert apply_discount(Decimal(total), code) == Decimal(expected)// TypeScript (Vitest/Jest)import { describe, it, expect } from "vitest";import { slugify } from "./slugify";
describe("slugify", () => { it("lowercases and hyphenates", () => { expect(slugify("Hello World")).toBe("hello-world"); }); it.each([[" Trim me ", "trim-me"], ["A--B", "a-b"]])("handles %j", (input, out) => { expect(slugify(input)).toBe(out); });});What to test: behavior at boundaries (zero, one, many, empty, max), error paths, invariants. Do not test the framework, trivial getters, or private methods (test them through the public interface).
Test doubles and dependencies. Make code testable by injecting dependencies (clock, randomness, I/O) rather than hard-coding them (Creational Patterns: DI). Functional core, imperative shell lets you test most logic as pure functions with no doubles at all.
Mocks, stubs, fakes and spies
Section titled “Mocks, stubs, fakes and spies”Gerard Meszaros’s xUnit Test Patterns distinguishes:
- Dummy: placeholder passed but never used.
- Stub: returns canned answers to calls.
- Spy: records how it was called, for later assertion.
- Mock: pre-programmed with expectations about calls; fails if they are not met (verifies interactions).
- Fake: a lightweight working implementation (in-memory repository, fake SMTP server, SQLite instead of Postgres).
# Python: stub, mock, and fake for the same dependencyfrom unittest.mock import Mock
class FakeUserRepo: # FAKE: real behavior, in memory def __init__(self): self.rows = {} def add(self, u): self.rows[u["id"]] = u def get(self, id): return self.rows.get(id)
def test_welcome_email_sent_with_fake_and_spy(): repo, mailer = FakeUserRepo(), Mock() # Mock used as a spy SignupService(repo, mailer).register({"id": 1, "email": "a@x.com"}) assert repo.get(1) is not None # state assertion on the fake mailer.send.assert_called_once_with("a@x.com", "Welcome!") # interaction assertionMocks vs fakes: a practical stance. Heavy use of interaction-verifying mocks couples tests to how code works (which methods call which), so refactoring breaks tests without any behavior change (“over-mocking”). Prefer state-based testing with fakes: assert on observable outcomes (what’s in the repository, what the response is). Use mocks/spies sparingly for outbound side effects with no observable state (an email was sent) and at architectural boundaries you own. Guidelines:
- Don’t mock what you don’t own: wrap third-party libraries in an adapter you own (Structural Patterns) and fake that; mocking a vendor SDK encodes your assumptions about it, not its real behavior.
- Don’t mock value objects or simple collaborators; use the real ones.
- Verify fakes against the real thing with a shared contract test suite run against both implementations (see below).
- London (mockist) vs Chicago/Detroit (classicist) schools of TDD differ here; the classicist style (real objects, state assertions) is generally more refactoring-friendly.
Integration tests
Section titled “Integration tests”Integration tests check that components work together with real dependencies: your code against an actual database, queue, or HTTP boundary. They find what unit tests cannot: SQL errors, ORM mapping, serialization, config, migrations, transaction behavior.
Modern practice makes them cheap: Testcontainers (Java origin; Python and Node ports) spins up real Postgres/Redis/Kafka in Docker per test run; embedded servers; transactions rolled back per test. Test the HTTP API in-process (FastAPI TestClient, supertest).
# Python: integration test against a real Postgres (testcontainers) with per-test rollbackimport pytest, psycopgfrom testcontainers.postgres import PostgresContainer
@pytest.fixture(scope="session")def pg(): with PostgresContainer("postgres:16") as c: with psycopg.connect(c.get_connection_url().replace("+psycopg2", "")) as conn: conn.execute("CREATE TABLE users(id serial PRIMARY KEY, email text UNIQUE NOT NULL)") conn.commit() yield c
def test_unique_email_enforced(pg): url = pg.get_connection_url().replace("+psycopg2", "") with psycopg.connect(url) as conn: conn.execute("INSERT INTO users(email) VALUES ('a@x.com')") with pytest.raises(psycopg.errors.UniqueViolation): conn.execute("INSERT INTO users(email) VALUES ('a@x.com')") conn.rollback()(Check the container library’s current connection API; it changes between versions.) Keep integration tests focused: test the seam, not business logic combinatorics, which belongs in unit tests.
End-to-end (E2E) tests drive the whole system through the UI (Playwright, Cypress) or public API. Keep a small set for critical user journeys; they are slow and flaky when overused. Techniques to reduce flakiness: stable selectors, auto-waiting, hermetic test data, no sleeps.
Property-based testing
Section titled “Property-based testing”Example-based tests check the cases you thought of. Property-based testing (QuickCheck, John Hughes and Koen Claessen, 2000) states a property that must hold for all inputs and lets the framework generate hundreds of random inputs, then shrinks failures to a minimal counterexample.
# Python (Hypothesis)from hypothesis import given, strategies as st
def my_sort(xs): return sorted(xs)
@given(st.lists(st.integers()))def test_sort_properties(xs): out = my_sort(xs) assert len(out) == len(xs) # same length assert all(a <= b for a, b in zip(out, out[1:])) # ordered assert sorted(out) == sorted(xs) # same multiset assert my_sort(out) == out # idempotent
@given(st.text())def test_roundtrip(s): assert decode(encode(s)) == s # encode/decode round trip// TypeScript (fast-check)import fc from "fast-check";test("reverse twice is identity", () => { fc.assert(fc.property(fc.array(fc.integer()), a => JSON.stringify([...a].reverse().reverse()) === JSON.stringify(a)));});Useful property patterns: round trips (parse/serialize), idempotence (f(f(x)) == f(x)), invariants (output sorted, balance never negative), oracle (compare with a simple, slow reference implementation), metamorphic relations (f(x + 0) == f(x), adding items never decreases total), and commutativity. Stateful property testing generates sequences of operations against a model (finding bugs in data structures, databases, state machines). Hypothesis stores failing examples to replay. Cost: harder to write good properties; payoff is large for parsers, serializers, algorithms and concurrency-sensitive code. Use it alongside examples, not instead.
Related: fuzzing (coverage-guided random inputs for security and robustness; AFL, libFuzzer, Go’s built-in fuzzing, Atheris) and snapshot/golden-file tests (store expected output; review diffs carefully so you don’t rubber-stamp changes).
Contract testing
Section titled “Contract testing”In a system of services, end-to-end integration tests are slow and brittle. Contract testing verifies that a consumer’s expectations match a provider’s behavior without deploying both.
Consumer-driven contracts (Pact being the best-known tool): the consumer’s tests record the requests it makes and the responses it needs, producing a pact file; the provider’s CI replays those requests against the real provider and checks the responses satisfy them. Breakage is caught at the provider’s build before deploy; the can-i-deploy check gates releases. Related: schema-based contracts (OpenAPI diffing, protobuf buf breaking, API Design), and shared contract test suites that run the same tests against a fake and the real implementation to keep the fake honest:
# One behavioral contract, run against every implementationimport pytest
@pytest.fixture(params=[InMemoryUserRepo, PostgresUserRepo])def repo(request): return request.param()
def test_get_returns_none_for_missing(repo): assert repo.get("nope") is None
def test_add_then_get_roundtrip(repo): repo.add({"id": "1", "email": "a@x.com"}) assert repo.get("1")["email"] == "a@x.com"Test-driven development
Section titled “Test-driven development”TDD (Kent Beck) is a design discipline: Red (write a failing test for the next small behavior), Green (write the simplest code that passes), Refactor (clean up with tests as a safety net); repeat in cycles of minutes.
Benefits: forces thinking about the interface and requirements first; produces testable, decoupled designs; yields a regression suite as a byproduct; keeps the code always working; small steps reduce debugging. Limits: awkward for exploratory/spike work (explore first, then rewrite test-first), for UI visuals, and for problems where you don’t yet know the design; dogmatic TDD can lead to test-induced design damage (excess indirection solely for mocking). DHH’s “TDD is dead” debate (2014) and the Beck/Fowler/DHH discussions remain a worthwhile read on trade-offs. Pragmatic adoption: use TDD for logic with clear specifications and for bug fixes (write a failing test that reproduces the bug first, then fix it).
Related practices: BDD (behavior-driven development: Given/When/Then scenarios with stakeholders, tools such as Cucumber/behave); ATDD (acceptance tests first).
Coverage and mutation testing
Section titled “Coverage and mutation testing”Code coverage (line, branch) shows which code ran during tests, not whether it was checked. 100% coverage with no assertions proves nothing. Coverage is useful in the negative: uncovered code is definitely untested. Treat it as a diagnostic, not a target (Goodhart’s law); mandating 100% yields assertion-free tests.
Mutation testing evaluates test quality directly. A tool automatically introduces small bugs (mutants) into the code: flip < to <=, replace + with -, delete a statement, return a constant. It runs the tests against each mutant. If tests fail, the mutant is killed (good); if tests still pass, the mutant survived, revealing behavior no test checks. The mutation score = killed / (total - equivalent). Tools: PIT (Java), Stryker (JavaScript/TypeScript, C#), mutmut and Cosmic Ray (Python), cargo-mutants (Rust).
def is_adult(age): return age >= 18
def test_is_adult(): assert is_adult(30) # also passes for mutant `age > 18` and `age >= 19`: they survive!# Add boundary tests: assert is_adult(18) and not is_adult(17), which kills those mutants.Cost: runs the suite many times, so apply to critical modules or changed code (incremental mutation). Equivalent mutants (changes that don’t alter behavior) cause noise. It is the most direct answer to “do my tests actually detect bugs?”
Flaky tests and test hygiene
Section titled “Flaky tests and test hygiene”Common causes of flakiness: shared state between tests, order dependence, time (sleep, current time, time zones), randomness without seeds, concurrency races, network calls to real services, unstable external data, and resource limits. Fixes: inject clocks and seeds, use hermetic fixtures, wait on conditions instead of sleeping, isolate state per test, quarantine and track flaky tests with a deadline to fix or delete. Treat flakiness as a bug in the test or, often, a real race in the code (Concurrency).
Organize tests to mirror behavior, keep test data builders (aUser().admin().build(); see the Builder pattern in Creational Patterns) to avoid duplicated setup, and treat test code as production code: refactor it, review it.
Refactoring
Section titled “Refactoring”Refactoring (Martin Fowler, 1999; 2nd ed. 2018) is changing the internal structure of code without changing its observable behavior, in small, safe steps, to make it easier to understand and modify. It is distinct from rewriting or adding features: do not mix refactoring and behavior changes in the same commit (the “two hats” rule).
Precondition: tests that pin the current behavior. For legacy code without tests, Michael Feathers’ Working Effectively with Legacy Code offers characterization tests (capture what the code currently does), finding seams to break dependencies, and the Sprout/Wrap techniques.
Catalog of “code smells” that signal refactoring opportunities: long function, large class, duplicated code, long parameter list, feature envy (a method uses another class’s data more than its own), shotgun surgery (one change touches many classes), divergent change, primitive obsession, data clumps, switch statements on type codes, speculative generality, comments compensating for unclear code, mutable global state.
Core refactorings: Extract Function/Method, Inline, Rename, Extract Variable, Move Function, Replace Conditional with Polymorphism, Introduce Parameter Object, Replace Magic Number with Constant, Encapsulate Collection, Replace Temp with Query, Extract Class.
# Before: long function mixing concernsdef process(order): total = 0 for l in order["lines"]: total += l["qty"] * l["price"] if order["customer"]["tier"] == "gold": total *= 0.9 print(f"Order {order['id']}: {total}") return total
# After: Extract Function, intention-revealing names; behavior unchangeddef subtotal(lines): return sum(l["qty"] * l["price"] for l in lines)def tier_discount(tier): return 0.9 if tier == "gold" else 1.0
def process(order): total = subtotal(order["lines"]) * tier_discount(order["customer"]["tier"]) print(f"Order {order['id']}: {total}") return totalMake steps small (minutes), run tests after each, commit often, use IDE automated refactorings (they are behavior-preserving by construction), and apply the Boy Scout Rule (leave the code a little cleaner than you found it) in small increments rather than “big-bang refactoring projects”. The strangler fig pattern replaces legacy systems incrementally. See technical debt in Engineering Practice.
Code review
Section titled “Code review”Code review catches defects, spreads knowledge, maintains consistency and mentors. Google’s engineering practices documentation sets a durable standard: approve a change once it definitely improves the overall code health of the system, even if it is not perfect. Evidence (e.g. Microsoft and Google studies) suggests reviews are more effective for maintainability, knowledge sharing and design feedback than for finding low-level bugs; automated tooling should handle style and basic defects.
As an author:
- Keep changes small (a few hundred lines at most; review quality collapses beyond that); one logical change per PR.
- Write a clear description: what, why, how tested, risks, screenshots; link the issue.
- Self-review the diff before requesting review; make CI green first.
- Separate refactors from behavior changes.
- Respond to every comment; ask questions rather than defend.
As a reviewer:
- Review the design first (does this belong here, right abstractions?), then correctness, tests, readability, security, performance, operability (logging, metrics, rollback).
- Be fast: delays block teammates; aim for response within a working day.
- Distinguish blocking issues from nits/suggestions; prefix (“nit:”, “optional:”, “question:”).
- Critique the code, not the person; ask questions; explain why; offer alternatives.
- Check the tests: do they actually test the behavior? Are failure cases covered?
- Automate style: formatters, linters, type checks, security scanners, so humans discuss substance.
- Review checklist prompts: Could this break existing behavior? What happens on failure/timeouts/retries? Is it idempotent? What happens at 10x scale? Are secrets, injection or authorization concerns handled (Security)? Is the change reversible (feature flag, migration compatibility)?
Pairing and mob programming are synchronous alternatives; trunk-based teams often combine small PRs with pre-merge CI and post-merge checks. AI-assisted review is now common as a first pass; see Engineering Practice for how to use it responsibly.
Pitfalls
Section titled “Pitfalls”- Testing implementation details; heavy mocking; tests that break on every refactor.
- Chasing a coverage number rather than confidence.
- Slow, flaky E2E suites that everyone ignores.
- Shared mutable fixtures and order-dependent tests.
- Mocking third-party libraries directly.
- Giant PRs that get rubber-stamped.
- Refactoring without a safety net, or mixed with feature work.
- Treating tests as write-once: unmaintained tests rot into noise.
Key takeaways
Section titled “Key takeaways”- Tests buy confidence to change; value comes from speed, determinism and behavioral focus.
- Choose a test mix by where your bugs come from: pyramid for logic, trophy/honeycomb for integration-heavy systems.
- Prefer fakes and state assertions to mocks; do not mock what you do not own.
- Property-based tests find edge cases you did not imagine; contract tests keep services compatible without full E2E.
- TDD is a design aid, strongest for well-specified logic and bug reproduction.
- Mutation testing checks whether tests would detect bugs; coverage alone cannot.
- Refactor in small steps under test; keep refactoring separate from behavior changes.
- Review small changes, design first, with kind and specific feedback.
Exercises
Section titled “Exercises”- Hands-on (Python). Write a
parse_duration("1h30m")function using TDD, three cycles minimum. Then write a Hypothesis round-trip property with aformat_durationfunction. - Conceptual. A test asserts
mock_repo.save.assert_called_once()andmock_repo.get.assert_called_with(id). A refactor merges the two calls into oneupsert. What happens, and how would a fake-based test differ? - Hands-on. Take a function with an
if x >= 10branch and a test suite with 100% line coverage. Identify mutants that would survive and add tests to kill them. - Conceptual. Your microservice team’s E2E suite takes 90 minutes and fails 15% of the time randomly. Propose a restructuring.
- Hands-on (TypeScript). Use fast-check to test an
encode/decodepair for URL slugs, including a property about idempotence.
Solutions (sketch)
- The mock-based test fails even though behavior is unchanged; a fake-based test asserts the resulting stored state (“after the operation the repository contains the record”), so it still passes.
- Mutants
>= 10to> 10or>= 11survive unless you test the boundary at exactly 10 (and 9, 11). - Replace most E2E with per-service integration tests plus consumer-driven contract tests; keep a handful of critical-path E2E smoke tests; run in parallel with hermetic data; quarantine and fix flaky tests; use staging probes and canary deployments for production verification.
Further reading
Section titled “Further reading”- Kent Beck, Test-Driven Development: By Example (2002).
- Martin Fowler, Refactoring: Improving the Design of Existing Code (2nd ed., 2018).
- Michael Feathers, Working Effectively with Legacy Code (2004).
- Gerard Meszaros, xUnit Test Patterns (2007).
- Steve Freeman and Nat Pryce, Growing Object-Oriented Software, Guided by Tests (2009).
- Vladimir Khorikov, Unit Testing: Principles, Practices, and Patterns (2020).
- Titus Winters, Tom Manshreck, Hyrum Wright, Software Engineering at Google (2020), chapters on testing and code review.
- Google Engineering Practices documentation: “How to do a code review”.
- Koen Claessen and John Hughes, “QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs” (2000).
- Hypothesis documentation (hypothesis.readthedocs.io) and Pact documentation (docs.pact.io).