The Easy Part Was the GitHub Action

I thought I was adding CI. I ended up refactoring the test suite.

TLDR: I sat down to add GitHub Actions to a side project and discovered the tests had fallen behind the code. The workflow took twenty lines. Fixing the fixtures, isolating the database, and splitting the test suite took the rest of the work.

It started as a cleanup

I sat down to clear out long-standing issues on my F1 race-analytics app (code, live). The tests had fallen behind the code. I'd refactored the app a few months earlier and never refactored the tests to match. I knew it. It was on the list, waiting for me to come back to it.

One of the open issues was a review asking me to automate the suite with GitHub Actions: run the tests on every push so nothing broken lands on main. Good place to start. Before writing any workflow, I ran the suite to see what CI would actually be running.

uv run pytest

Red. No surprise. The CI request and the test debt turned out to be the same problem. There's no point automating a suite that doesn't pass. So the workflow could wait. First the tests had to be real.

Step one: make it pass

The errors were all the same kind. My fixtures built model objects that were missing required fields, so the objects couldn't be created at all.

The fix was mechanical: add the fields. Doing it first mattered, though. On a red suite you can't tell a new break from one that was already there, so every change after is a guess.

A review comment had suggested instrumenting pytest to always run coverage, so I added that too. Then the workflow I'd come for:

name: Tests
on:
  push:
    branches: [main]
  pull_request:
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v5
      - run: uv sync
      - run: uv run pytest

With the suite finally green, I could have opened the PR and moved on. Instead I started actually reading the tests I'd been ignoring. Two bigger problems showed up.

The tests were using my real database

A green checkmark only tells you that the tests passed. If the tests aren't isolated or aren't covering the right things, it doesn't mean much. Here is what the green suite was doing. A fixture dropped and recreated the real SQLite file before each run, and the functions under test opened their own session on a module-level engine. There was no way to point them anywhere else.

engine = create_engine("sqlite:///database.db")

def create_races(year, races_data):
    with Session(engine) as session:   # its own session, on the real DB
        ...

Tests that hit your real database are slow. They leak state between runs. They depend on order. They can wipe real data. Mine were passing and doing all four. That shared engine is the root of it, and the trap reaches well beyond tests. Bob Belderbos covers this same problem, caused by picking the wrong scope for a stateful object, in Two Python Scoping Bugs.

The fix is dependency injection. The function stops reaching for a global and takes a session:

def create_races(session, year, races_data):
    ...

Production passes the real session. Tests pass an in-memory one, built fresh per test and gone when the test ends:

@pytest.fixture
def session():
    engine = create_engine(
        "sqlite://",
        connect_args={"check_same_thread": False},
        poolclass=StaticPool,
    )
    SQLModel.metadata.create_all(engine)
    with Session(engine) as session:
        yield session
    engine.dispose()

sqlite:// with no path lives in memory. Each test gets its own, and the real database.db is never touched. The StaticPool and check_same_thread bits matter in a second.

Database tests and route tests in one file

The second problem was structural, and coverage is what flagged it. Everything lived in test_app.py, but every test in it hit the database layer. The routes came back at zero coverage, including a standings page I'd just shipped.

I split it three ways: test_database.py for the database layer, test_app.py for the routes, conftest.py for the shared fixtures.

Route tests drive the app through FastAPI's TestClient. The trick is overriding get_session so the routes use the test database, and building the client without its lifespan so startup doesn't fire real network calls or rebuild the real database:

@pytest.fixture
def client(session):
    def get_session_override():
        return session

    app.dependency_overrides[get_session] = get_session_override
    client = TestClient(app)        # no context manager, lifespan stays off
    yield client
    app.dependency_overrides.clear()

TestClient runs the app on another thread, and SQLite blocks cross-thread access to a connection by default. StaticPool keeps one shared connection alive so both threads see the same in-memory database.

What CI was actually for

I'd added the Action the moment the suite went green. But a workflow only matters as much as the tests under it, and at that point mine still weren't isolated and barely touched the routes. It became a gate worth trusting only after the rest of the work. Now every push runs an isolated, split suite against a clean in-memory database, on a machine that isn't my laptop.

Going in, I thought the task was adding CI. Looking back, that was the easy part. Most of the work was fixing the assumptions that had built up in the test suite over time.

A green checkmark is a claim about your tests. What the claim is worth gets decided long before the workflow ever runs.