One row tells the whole story

The decision

One database row, or something cleverer

Costs to run

Nothing — it's a row you already needed

What you get

Progress that survives restarts, deploys and everything in between

A real moment: a deploy at minute two

Deploys don't wait for users. This is the situation the whole part is built around, and the reason the answer is a database row rather than anything cleverer.

The problem with a job you can't see

The work doesn't happen in the request. The user presses a button, the request returns immediately, and the real work starts somewhere else entirely — in a worker process, on a different core, possibly on a different machine.

So the page now has a genuine problem. It has to show a user what's happening inside a process it cannot see, cannot ask, and has no connection to.

There are glamorous answers to this. A websocket channel. An in-memory progress registry. An event bus. Every one of them has the same flaw: the state lives in a process, and processes restart. A job that is running perfectly well should still be visible after a deploy.

The unglamorous answer is one row in one table, and it holds up better than any of them.

Four columns run the whole interface

Every run is one row in idea_runs. The browser, the queue worker, the usage limits and the support commands all read and write that row and nothing else.

A status alone isn't enough to draw a useful page, though. A four-minute stage deserves more than one word. Four columns do the real work:

Two small details in those writers matter more than they look.

The step counter can't go backwards. markStep() uses max(current_step, number). Stages can report slightly out of order, and a bar that only ever moves forward is one a user can trust.

The activity column is written constantly. reportActivity() fires on every single tool call the model makes — dozens of times in a run. So it writes exactly one column and nothing else: one small UPDATE, no other columns touched. Keeping that write tiny is what lets the page narrate every search and every page read.

How the page watches: it just asks, repeatedly

The progress page calls a JSON endpoint every two seconds. The first call goes out after 1.2 seconds, so the first tick lands almost immediately and the page feels alive from the first second.

One request returns everything the page needs:

{
  "status": "researching",
  "percentage": 38,
  "steps": [ { "number": 3, "state": "active", "detail": null } ],
  "activity": "Searching: [organisation] events 2026",
  "waiting": false,
  "seconds_on_step": 41,
  "verified": [ ],
  "sources": [ ],
  "finished": false,
  "redirect": null
}

Note what's in that payload: verified and sources. The findings from the research stage are sent as they're confirmed, so the "Verified information" and "Sources" panels fill in while the user watches. They are not staring at a spinner for three minutes waiting for everything at once. They're watching something get built.

"Why not websockets?" — the honest answer

This is the part people argue about, so here is the actual reasoning rather than a principle.

The app is a conventional request-and-response web app with a database-backed queue. Adding a websocket server for one feature means a new process to run, a new thing to monitor, and a new thing to explain to whoever is on call. Server-sent events are lighter and would work — but they hold a web worker open for the whole run, and that worker should be free to serve everyone else.

Against that: a poll every two seconds, hitting an indexed primary key, costs almost nothing. And on a three-minute job, nobody can tell the difference between instant and two seconds late.

The calculation flips completely if you have many concurrent viewers, sub-second updates, or a real-time feature already in the product. Many frameworks now ship a first-party websocket server, so the cost of that choice is lower than it used to be. It just wasn't lower than zero, which is what polling cost here.

The row is addressable, so guard it

There is a catch to making one row the centre of everything. Routes bind to it by ID:

GET /ideas/{run}

A route that looks a record up by its ID will happily resolve any run, including one belonging to a different organisation. A run's research can contain an organisation's plans and its performance history. A guessable ID must never hand that over.

So every action checks ownership explicitly, before it does anything else:

function authoriseRun(run, organisation):
    if run.organisation_id != organisation.id:
        respond 403 Forbidden

That covers the progress page, the status endpoint, and the approve, dismiss and refine actions. Two tests attempt cross-organisation access and expect a 403, so the guarantee is proven on every build rather than assumed.

What this bought

By the end of this decision, every later problem in the series has somewhere to be solved:

• An interrupted run is a row with an old timestamp, so it's found and closed automatically.
• Usage limits are a count over rows, not a counter that can drift.
• Support can unblock a customer by updating a row, with no deploy.
• Every run leaves a complete record behind, which is why the cost reporting in Part 4 is a SQL query rather than a trip to a provider's dashboard.

All of that comes free once the truth is a row.