Pytest in Practice

A developer learning Python by building a video clipping CLI application shared practical lessons on pytest, covering test discovery conventions, the use of Python's built-in assert with pytest's assertion rewriting, and pytest.approx for floating-point comparisons. The writeup illustrates these concepts with examples from the CLI's FFmpeg, video metadata, audio extraction, and transcription code, including golden-output regression tests.

I've been learning Python by building a video clipping CLI application, and along the way I've ended up learning a lot more than I expected. Some of it was straightforward. Some of it made me stop and think, especially when I first started writing tests. Things like: conftest.py ? patch and monkeypatch ? These are the kinds of things that can feel like arbitrary rules when you're first encountering them. Once I understood the reasoning behind them, though, pytest started making a lot more sense. So this isn't meant to be a complete pytest reference or an explanation of every feature pytest has. It's more like a collection of the things that finally "clicked" for me while testing a real Python project. The examples are based on my video clipping CLI, so you'll see things like FFmpeg, video metadata, audio extraction, transcription, temporary files, external dependencies, and slow tests. Hopefully, if you're also learning Python by building something rather than just following tutorials, some of these lessons save you a few of the "why does it work like that ?" moments I had. If you're using uv , installing pytest and coverage support is straightforward: uv add --dev pytest pytest-cov uv run pytest --version You now have everything needed to start writing and running tests. A pytest test can be nothing more than a function: python tests/core/test video inspector.py from clipper.core.video inspector import get video metadata def test get video metadata returns none on missing file : result = get video metadata "does not exist.mp4" assert result is None Run it with: uv run pytest That's it. There is no base class, no registration, and no decorator required. pytest finds the test because: test That naming convention is the mechanism. pytest automatically walks the current directory and its subdirectories looking for test files. By default, it looks for files matching patterns such as: test .py test.py Inside those files, it looks for functions matching: test and, inside classes whose names start with Test , methods that also match test . So this: tests/ └── core/ └── test video inspector.py is enough for pytest to discover the file automatically. You don't need to register the test anywhere. This is also why commands such as: uv run pytest --cov=src work without explicitly listing every test file. pytest's discovery mechanism does the work. assert One of the nicest things about pytest is that you use Python's built-in assert . python def test fps math : assert round 29.970029970029969, 2 == 29.97 You don't need a collection of methods like: assertEqual ... assertTrue ... assertFalse ... assertIn ... Instead, pytest rewrites assert statements when running tests so that failures contain useful information about what each side evaluated to. That makes a failed assertion much more informative than simply: Assertion failed pytest.approx Don't compare floating-point calculations using exact equality. Instead: python import pytest def test duration seconds : assert 125.5 == pytest.approx float "125.500001" Floating-point calculations rarely land exactly where you expect. This matters especially when dealing with things like frame rates, durations, timestamps, or calculations involving Fraction . Without pytest.approx , you might end up writing this everywhere: assert abs actual - expected < 0.001 pytest.approx gives you the tolerance without repeating that boilerplate. pytest.approx isn't limited to individual numbers. You can use it when comparing nested structures: python def test transcribe matches golden output sample speech path, sample transcript, : result = transcribe str sample speech path assert result == pytest.approx sample transcript This is particularly useful for regression or "golden file" tests. Imagine a transcript like: { "segments": { "start": 0.123456, "end": 1.987654, "words": {"word": "hello", "start": 0.123}, {"word": "world", "start": 0.456}, , } } You don't want to write a loop comparing every timestamp manually. pytest.approx can recursively compare numeric values with tolerance while still using normal equality for strings and other values — so it's practical to compare an entire expected transcript while allowing for harmless floating-point differences. When the expected behavior is for a function to raise an exception, use pytest.raises . python import pytest def test extract audio raises on bad codec : with pytest.raises ffmpeg.Error : extract audio "in.mp4", "out.wav", codec="not a real codec", This test verifies two things: The test fails if the function doesn't raise anything. It also fails if it raises a different exception type. Both outcomes are useful information. pytest.raises Only Works When the Exception Escapes There's an important distinction here. Suppose your function catches its own exception: python def extract audio ... : try: ... except ffmpeg.Error: logger.error ... return None The exception never reaches the caller. Therefore, this won't work: with pytest.raises ffmpeg.Error : extract audio ... There's nothing left for pytest.raises to catch. Instead, test the function's actual contract: result = extract audio ... assert result is None And, if logging is part of the behavior you care about, use caplog to verify that the expected error was logged. A useful rule of thumb: If the function returns None from its except block, test the return value rather than expecting an exception. Suppose you need to test several timestamp formats. You could write three tests: python def test timestamp one : ... def test timestamp two : ... def test timestamp three : ... But the test logic is identical. That's exactly what pytest.mark.parametrize is for: @pytest.mark.parametrize "timestamp,expected seconds", "00:00:01", 1.0 , "00:01:00", 60.0 , "01:00:00", 3600.0 , , def test validate timestamp parses valid formats timestamp, expected seconds, : assert validate timestamp timestamp == pytest.approx expected seconds Think of it like a form letter: the first argument names the blanks to fill in timestamp , expected seconds , and the second argument is a stack of filled-in versions. pytest prints one out and runs the test for each one: test validate timestamp parses valid formats 00:00:01-1.0 test validate timestamp parses valid formats 00:01:00-60.0 test validate timestamp parses valid formats 01:00:00-3600.0 Each case can pass or fail independently. You could achieve something similar with a loop: python def test timestamps : for timestamp, expected in cases: assert validate timestamp timestamp == expected But there's a major difference: a normal loop stops at the first failure. With parametrization, pytest runs every case. So one test run tells you exactly which inputs are broken instead of merely telling you: test timestamps failed The signal is simple: If you're about to write several nearly identical test functions that differ only in their input values, consider parametrize . It works just as well for valid inputs as it does for malformed or invalid ones. pytest supports test classes: python class TestGetVideoMetadata: def test returns none for missing file self : assert get video metadata "missing.mp4" is None def test has audio true when audio stream present self : ... But classes aren't mandatory. Use them when they provide something useful, such as: Don't use a class purely because it feels more organized. A well-named test file already provides plenty of organization. Each test method gets a fresh instance of the class. So this: python class TestSomething: def test one self : self.value = 1 def test two self : assert self.value == 1 is not valid. test two doesn't receive the instance used by test one . That's intentional — tests should be isolated from one another. Depending on mutable state from another test creates order-dependent and difficult-to-debug failures. Fixtures are one of pytest's most useful features. A fixture is a function that provides something a test needs — think of it like an ingredient a recipe asks for by name. You don't prep it inside the recipe itself; you just say "I need sample video path " and pytest hands it to you already made. python tests/conftest.py import pytest from pathlib import Path @pytest.fixture def sample video path - Path: return Path file .parent / "fixtures" / "sample.mp4" A test can then request it simply by putting the fixture name in its parameters: python tests/core/test video inspector.py def test metadata against real file sample video path : result = get video metadata str sample video path assert result is not None pytest sees sample video path in the test signature, finds the fixture with that name, runs it, and passes the result to the test. No import is required. conftest.py is automatically discovered by pytest. Fixtures defined there are available to test files in the same directory and its subdirectories — it works like a shared pantry that any test file in that part of the tree can pull from without asking for a delivery. For example: tests/ ├── conftest.py ├── core/ │ └── test video inspector.py └── api/ └── test transcription.py A fixture defined in tests/conftest.py is available to both tests/core/ and tests/api/ . You can also have a more specific conftest.py deeper in the tree: tests/ ├── conftest.py └── core/ ├── conftest.py └── test video inspector.py The deeper fixture configuration can then be used specifically by that subtree. A useful convention is: root conftest.py for shared fixtures, nested conftest.py files for fixtures that are genuinely local to a particular test area. Fixtures don't necessarily have to run for every test. You can control their lifetime: @pytest.fixture scope="function" default @pytest.fixture scope="module" once per test file @pytest.fixture scope="session" once per entire test run The default is scope="function" , which means a fresh fixture is created for every test. Prefer the default unless there is a good reason to reuse the fixture. A broader scope can make sense when setup is genuinely expensive and safe to share. For example, generating a synthetic sample video once per session may be much cheaper than generating it once for every test. pytest provides several useful fixtures out of the box. tmp path Provides a unique temporary directory for each test. Great for anything that needs to write files. caplog Captures logging output. Useful when you want to verify that something logged an error or warning. capsys Captures stdout and stderr. Useful for testing CLI output. Temporarily changes attributes, environment variables, dictionary entries, and other values. pytest automatically restores the changes after the test. You can inspect available fixtures with: uv run pytest --fixtures Suppose you want to verify that audio extraction creates an actual file: python def test extract audio writes real file sample video path, tmp path, : output path = tmp path / "output.wav" result = extract audio str sample video path , str output path , assert result == str output path assert output path.exists This is an integration-style test. It actually runs FFmpeg and checks that a real file was created. But because tmp path is temporary, the test doesn't leave generated files scattered around your project — pytest creates the directory and handles its cleanup. Python gives you unittest.mock . pytest gives you monkeypatch . They overlap, but each one shines in a different situation. unittest.mock.patch Use patch when you want a more capable mock, particularly when you need to inspect how something was called. python from unittest.mock import patch def test metadata no audio : fake probe = { "format": { "filename": "silent.mp4", "duration": "10.0", }, "streams": { "codec type": "video", "r frame rate": "25/1", }, , } with patch "clipper.core.video inspector.ffmpeg.probe", return value=fake probe, : result = get video metadata "silent.mp4" assert result "has audio" is False patch temporarily replaces the target with a mock. It is particularly useful when you want to assert things like: mock input.assert called once with "in.mp4" monkeypatch is pytest's own fixture. It's particularly convenient for simple temporary substitutions. python def test uses default bitrate env var monkeypatch : monkeypatch.setenv "CLIPPER DEFAULT BITRATE", "256k", assert get default bitrate == "256k" There's no context manager and no manual cleanup — pytest automatically restores the environment variable after the test. You can also replace attributes with monkeypatch.setattr ... or dictionary entries with monkeypatch.setitem ... . Imagine your code is a toy robot and you want to test what happens when one of its components behaves differently. You don't want to actually break the component — you just want to temporarily pretend that it behaves differently. monkeypatch is like putting a temporary sticky note over the component: "For this test, pretend this value is different." pytest removes the note when the test finishes. patch is another version of the same idea, but with a much more powerful mock object behind it. patch can easily track call count, arguments, return values, exceptions, and chained calls. Both clean themselves up reliably when used correctly. With patch , the common form in this project is: with patch ... : ... The mock is restored automatically when the block ends, even if the test fails. A practical rule: | Need | Use | |---|---| | Assert how something was called | patch | | Check call count or exact arguments | patch | | Mock a chained/fluent API | patch | | Replace an environment variable | monkeypatch | | Temporarily change one attribute | monkeypatch.setattr ... | | Temporarily change a dictionary entry | monkeypatch.setitem ... | | Replace an external dependency and inspect the mock | patch | The important thing isn't to memorize a strict rule. It's to recognize the distinction: If you need to inspect how the dependency was used, patch is usually the natural choice. If you just need to temporarily replace a value or attribute, monkeypatch is often simpler. side effect : Make a Mock Fail Mocks aren't only useful for returning fake successful results. They can also simulate failures. return value means "always return this." side effect means "raise this exception instead." with patch "clipper.core.video inspector.ffmpeg.probe", side effect=ffmpeg.Error cmd="ffprobe", stdout=b"", stderr=b"bad file", , : result = get video metadata "bad.mp4" This lets you exercise failure paths without needing to create a genuinely broken video file. This is probably the most important mocking rule in the entire guide: Mock the boundary, not the logic you're trying to test. Suppose your code does this: probe = ffmpeg.probe path The interesting logic is what your application does with the result. So mock ffmpeg.probe and provide realistic fake data — don't mock the parsing logic itself. A test that mocks everything, including the thing you're trying to test, can end up proving nothing more than: "my mocks returned what I told them to return." Put another way — you're testing your recipe, not reinventing the oven. External boundaries are good mocking candidates: Your own business logic is usually what you want to execute for real. This catches a lot of people. Suppose your module contains: python video inspector.py import ffmpeg def get video metadata path : return ffmpeg.probe path You might think you should patch ffmpeg.probe . But that's not necessarily the correct target — the code under test looks up ffmpeg in its own module namespace. Therefore, patch where the name is looked up: patch "clipper.core.video inspector.ffmpeg.probe" The general rule is: Patch where the object is looked up, not where it was originally defined. If you patch the wrong path, the real dependency may still run. That can make a test unexpectedly hit FFmpeg, the network, a database, or some other external system. Some APIs aren't a single function call. For example, ffmpeg-python can use a fluent chain: ffmpeg.input ... .output ... .overwrite output .run ... Mocking that requires walking the chain. MagicMock automatically creates mocks for attributes you access, so mock input.return value represents the result of ffmpeg.input ... , .output.return value represents the next link in the chain, and so on until you reach .run — the actual operation you want to make fail. python def test extract audio sad path caplog : fake error = ffmpeg.Error cmd="ffmpeg", stdout=b"", stderr=b"encoding failed", with patch "clipper.core.video inspector.ffmpeg.input" as mock input: mock run = mock input .return value .output .return value .overwrite output .return value .run mock run.side effect = fake error result = extract audio "in.mp4", "out.wav", assert result is None assert "error" in caplog.text.lower Walking the chain into its own variable mock run before setting side effect keeps the assignment itself simple and unambiguous — trying to set an attribute in the middle of a long parenthesized chain is a common source of syntax mistakes. A simple and predictable test layout goes a long way. If your source looks like: src/ └── clipper/ └── core/ └── video inspector.py mirror it under tests/ : tests/ └── core/ └── test video inspector.py This makes it immediately obvious where the tests for a source module live. tests/ Outside src/ Use: project/ ├── src/ ├── tests/ └── pyproject.toml rather than: src/ ├── clipper/ └── tests/ Keeping tests outside the package helps avoid accidentally shipping them with your built distribution and keeps the distinction between application code and test code clear. init .py pytest doesn't require your test directories to be Python packages. So this is perfectly fine: tests/ ├── core/ │ └── test video inspector.py └── api/ └── test transcription.py No init .py required. There can be edge cases where you intentionally want package semantics, but don't add init .py to test directories just because you think pytest requires it. It doesn't. tests/ ├── conftest.py ├── fixtures/ │ └── sample.mp4 └── core/ └── test video inspector.py This makes the distinction obvious: test .py contains test logic, fixtures/ contains test data. Not every test costs the same amount to run. A test that calls real FFmpeg is different from a test that checks a pure Python function. You can mark integration tests: python @pytest.mark.integration def test extract audio against real file ... : ... Then register the marker in pyproject.toml : tool.pytest.ini options markers = "integration: calls real ffmpeg/ffprobe against a fixture file", Now you can run the fast suite without integration tests: pytest -m "not integration" And run everything with: pytest This gives you a useful distinction between fast unit tests and slower integration tests. print Everywhere pytest captures output by default. That can be surprising when you're debugging — a print result that produces nothing on screen. Here are the options you'll reach for most often. -s : Show stdout/stderr uv run pytest -s This disables output capturing, so your print statements become visible. It's useful, although there are usually better debugging tools than adding prints everywhere. -k : Run Tests by Name If you're working on audio extraction, there's no reason to run 200 unrelated tests. uv run pytest -k "extract audio" pytest runs tests whose names match the expression. This is one of the most useful options during active development. -x : Stop at the First Failure uv run pytest -x Instead of getting a wall of failures, pytest stops after the first one. This is particularly useful when fixing a cascading failure. --pdb : Drop Into the Debugger uv run pytest --pdb When a test fails, pytest drops you into an interactive debugger at the point of failure. Inside the debugger, you can inspect variables and control execution — c continues, q quits. This can be much faster than trying to predict where a bug is before running the test. -vv : More Verbose Output uv run pytest -vv Useful when assertions involve large dictionaries, long strings, or parametrized tests and you want more detail in the output. --lf : Run Only the Last Failures uv run pytest --lf --lf means "last failed." It reruns tests that failed during the previous run instead of running the entire suite. This is particularly useful when you're fixing several failures one by one. These options can be combined: uv run pytest -k "extract audio" -x --pdb This means: run only tests matching extract audio , stop at the first failure, and open the debugger at the failure. That's a very practical development loop. pytest doesn't natively use line numbers to select a test. This isn't standard pytest test selection: pytest test file.py:29 Instead, select by node ID: pytest tests/core/test video inspector.py::test metadata no audio Or use -k : pytest -k "metadata no audio" Editors such as VS Code and PyCharm can provide "run this test" buttons. They translate your selection into pytest's name-based test selection behind the scenes. There are plugins that provide literal line-number selection, but you generally don't need one. Sometimes you just want to see what a function actually returned. One quick technique is to deliberately make an assertion fail: python def test scratch sample video path : result = get video metadata str sample video path assert result == {} If {} isn't the actual result, pytest's failure output will show you the value. Delete the temporary test afterward. Another option is: uv run pytest --showlocals or the shorthand -l , which shows local variables when a test fails. Don't guess which tests are slow. Measure them. uv run pytest --durations=10 This reports the ten slowest tests after the run. You may discover that one or two tests account for most of the delay. Optimizing everything before measuring can waste the very time you're trying to save. Tests that load a real model or perform real inference are good candidates for a slow marker: python @pytest.mark.slow def test transcribe real audio produces valid shape sample speech path, : ... Register the marker: tool.pytest.ini options markers = "integration: calls real ffmpeg/ffprobe against a fixture file", "slow: tests that load a real model or run real transcription", Then your normal development loop can skip them: pytest -m "not slow" And your complete suite can still run with pytest . Suppose your application loads a model once at module level, and subsequent calls reuse it. Then only the first test that needs the model may pay the expensive loading cost — the others can reuse the already-loaded instance. You can verify whether this is actually happening with uv run pytest --durations=10 . If the first model-related test is slow and subsequent tests are much faster, your existing caching may already be doing the job. If every test is slow, something may be reloading the model when it shouldn't. Measure first. If the entire suite is slow, rather than just a handful of expensive tests, parallel execution may help. uv add --dev pytest-xdist uv run pytest -n auto -n auto lets xdist determine a useful number of workers. But don't reach for parallelization automatically — for many projects, simply excluding slow tests during development pytest -m "not slow" is enough. Coverage is useful. It tells you which lines of code weren't exercised by your tests. uv run pytest --cov=src This can help you find code paths you haven't thought about. But coverage doesn't tell you whether your tests are good. You can achieve 100% coverage with tests that assert almost nothing meaningful: python def test everything : my function The code executed. Coverage increases. But the test may tell you almost nothing about whether the behavior is correct. So treat coverage as a flashlight — use it to illuminate parts of the code you haven't tested. Don't treat it as a scoreboard you have to maximize. When adding tests for a new function or feature, ask: What should happen when everything is valid? Consider: empty input, missing fields, boundary values, unexpected but valid combinations. What should happen when input is invalid, an external dependency fails, a file doesn't exist, or an API returns an unexpected result? Usually mock things that are external, slow, non-deterministic, or side-effecting. But let your own logic run for real. If the test name contains the word "and," that's sometimes a sign that you're testing two behaviors at once. A test should ideally answer one clear question. pytest can look like a large collection of features when you're first learning it. But after working with it for a while, I found that most of what I actually needed came down to a few ideas. Tests are ordinary Python functions. pytest discovers them through naming conventions. Assertions are ordinary Python assert statements. pytest makes their failures informative. Fixtures provide reusable things that tests need. conftest.py makes shared fixtures available without imports. Parametrization lets you run the same test logic against multiple inputs. Mocks let you replace boundaries you don't want to exercise for real. Things like FFmpeg, APIs, databases, and other external systems are often good candidates — while the logic you're actually testing should usually run normally. Markers let you separate tests by purpose or cost. And the debugging tools make it much easier to figure out why something failed without resorting to print everywhere. Most importantly, I stopped thinking of testing as "how do I get my coverage number up?" and started thinking more in terms of "what question am I trying to answer with this test?" For me, that shift made pytest much easier to understand. The syntax is the easy part. The useful part is learning what behavior is worth testing, what should be isolated, what should run for real, and how to make a failure tell you something useful.