Built without an API key

The decision

The paid dependency goes behind an interface on day one

Costs to run

Nothing. That's rather the point

What you get

A free, offline test suite, and a feature you can demo anywhere

A real moment: a demo on Tuesday, tests on every push

Paid dependencies tend to arrive late, and tests can't afford to call them. Both are solved by the same small decision.

The binding that paid for itself four times

The research stage sits behind a single-method contract:

interface Researcher:
    research(organisation, profile, onProgress?) -> AiResult

There are two implementations. One makes the live, streamed call with server-side web tools. The other builds fixture research from the organisation's own database record, and reports fake tool activity through the same callback, at a pace a human can actually watch.

The dependency container picks one whenever something asks for a researcher:

when something asks for a Researcher:
    useFixtures = config.fake is on, or config.api_key is blank

    give it FixtureResearcher if useFixtures
    otherwise LiveResearcher

That's about fifteen lines. Here is what they bought:

Development before the key existed. The entire interface — every screen, every state, every edge case — was built and demoed for weeks with no API account at all. Waiting on procurement didn't block a single day of work.

Degrading instead of failing. If the key is missing in some environment, the feature serves sample data, clearly labelled as a demonstration, rather than an error page. A missing configuration value becomes a degraded feature, not an outage.

Tests with no network and no spend. The full pipeline runs in CI without touching the internet.

Demos that cost nothing. Anyone can show the feature working without spending money, and with the same polished result every time.

One binding, four payoffs.

Keeping sample data and live data apart

Research is cached for 30 days and reused, so the same research is only ever paid for once a month.

Fixtures and live research share a table, and an organisation that was demoed before the key was added has sample research in its history. Sample research must never stand in for the real thing.

So every run records was_faked, and reuse only ever matches like with like:

fixturesInUse = config.fake is on, or config.api_key is blank

previous = latest run where
    organisation = this one
    and research is present
    and was_faked = fixturesInUse

Live research is never replaced by fixtures. Fixtures are never presented as real. A test runs a demo first and a live run after it, and asserts the fixture research isn't reused.

A related rule, in the other direction: research from a run that failed at a later step is still reused. The research succeeded and was paid for; only the generation failed. So freshness is dated from completed_at, or updated_at when there isn't one, rather than requiring a completed run. Research that was paid for is always put to use.

Testing a non-deterministic feature deterministically

The suite has 33 feature tests and makes no network calls. It works because of the seams already described.

Four things make it possible:

• The researcher interface swaps in fixtures.
• The generation fake activates under the same condition, so the whole pipeline runs end to end rather than stopping at the seam.
• The fixture pacing is set to zero in tests. The fixture researcher normally pauses between fake tool calls so a human can watch the progress page fill in. In CI that's just waiting.
• A fake queue for the controller tests, and synchronous execution for the pipeline tests.

What's striking is what the tests are about. Almost none of them assert anything about the quality of model output:

Prompt quality is reviewed by hand, against real organisations, because that's a judgement call and people are the right tool for it. The tests cover what tests can prove: that a run is paid for exactly once.

Two commands make up the operator's toolkit

A preflight check verifies everything a live run needs, and prints a verdict for each:

  ✔ Module enabled            yes
  ✔ Model                     (from config)
  ✔ API key                   set (••••)
  ✔ Fake mode forced          no
  ✔ Calendar dates seeded     74
  Calling the API…
  Connected. Model replied: ready

It makes one tiny real API call, costing a fraction of a penny, so "the key is set" and "the key works" are both confirmed before a customer ever tries the feature. Those are different facts and only one of them is free to check.

The seeded-dates line is there because the seeder runs separately from the migrations, so the check confirms it on every new environment.

A reset command gives an organisation back its allowance. By default it sets reset_at on the runs rather than deleting them, so the 30-day research cache survives. Freeing an allowance should never discard research that's already been paid for. A --purge flag really deletes, warns about what it's discarding, and leaves alone any drafts already created from those ideas.

Both commands mean support can help a customer in seconds, with a tool that has tests behind it.

What carries over to the next slow, paid job

None of this was AI-specific. Strip out the model and the same nine lessons apply to a video encode, a nightly import, or anything else that takes minutes and costs money.

1. 1Make one database row the source of truth. The worker writes it, the browser polls it, the limits count it. Everything becomes inspectable, and recovery becomes an UPDATE.
2. 2Give long jobs their own queue connection. The visibility timeout belongs to the connection, and the default one was tuned for two-second jobs.
3. 3Write the timeouts down, and test the one you can. The visibility timeout and the process manager's stop grace period must both outlast the job's timeout, so every run finishes exactly once.
4. 4Use a check and a lock. The check handles the slow duplicate click. Only an atomic lock handles two that arrive together.
5. 5Check for stale runs wherever they cause harm — including at the very start of the job itself.
6. 6Put the paid dependency behind an interface on day one. Development without a key, tests without spend, demos without a bill, and a graceful fallback, all from one binding.
7. 7Make cost a property of each result, priced per model, and total it in one place.
8. 8Treat model output like form input. Parse it, clamp it, whitelist it, and only then save it.
9. 9Hand off to the screens people already use. The feature's job ends at "draft created".

The model was never the hard part. The craft is in everything that makes one press of a button dependable, every single time.