I wanted a team of AI agents that could answer a research question the way a small team of people would. One plans the work, one searches the web and reads the sources, one writes it up, and one checks the write-up before I see it. I also wanted to watch them do it: which agent is working, what it's reading, what it handed to whom, and what it cost.
The app is called FORGE, and it runs on a small server in my house. This post covers how it got from an idea to something that works end to end, which took three days. Most of that time wasn't building features. It was finding the places where the app looked like it worked and didn't.
It had a workflow canvas where each agent is a card and each arrow is a handoff. It had a live event stream, a timeline, a replay scrubber, per-agent cost and token counters, an agent builder and a workflow designer. Behind it sat a simulator: fake agents on a fake clock, producing realistic-looking events so every screen had something to show.
One design decision from that first day held up for everything that came after. A run is a log of events, not a record that gets updated. "Researcher started", "tool call finished", "handed the notes to the writer": every screen is computed from that log. Live view and replay are the same code; replay just stops reading the log partway through. When real agents replaced the simulator later, the screens didn't change at all, because real agents emit the same events.
Real agents meant real model calls, and I wanted them cheap. I started with three agents: a researcher, a writer and a reviewer. Models: most agents run GLM-5.3 Flash through OpenRouter, at about $0.15 per million input tokens. The reviewer runs on Claude. Search: DuckDuckGo through the ddgs Python package. It's free and needs no key. Reading pages: a small script fetches a URL and pulls out the main text (trafilatura for web pages, pypdf for PDFs). A server: a small Node.js server runs the agents, stores every event, and streams them to the browser as they happen.
Every call timed out, instantly. The server has no IPv6. When a hostname has several addresses, Node.js tries them one after another and gives each attempt only 250 milliseconds by default, which isn't enough on this network. The fix was two lines: prefer IPv4, and give each attempt three seconds.
Some replies came back empty. The model had spent its whole output budget reasoning and had nothing left for the answer. It now gets one retry with double the room.
The researcher never stopped researching. It kept searching and reading until it hit the step limit, then had no notes. Now every tool result tells it how many rounds it has left, and the last round forces it to write up what it has.
Some websites never answered. One slow host could eat a minute. Page fetches now give up after 20 seconds, and a host that times out once is skipped for the rest of the run.
With real agents working, I went through the app page by page and asked one question: is this real or simulated?
A lot of it was simulated and looked real. The model-provider page showed made-up usage. The tools page listed tools with success rates and response times for tools that had never run. The run history was full of sample runs.
Then I asked the team a real question and got a confident, well-formatted answer to a completely different question. The run had used the simulator, which produces plausible text and ignores the question entirely. It was the default for most workflows, and nothing on screen said so.
The rule I took from it: never ship a placeholder that looks real. Label it or remove it. Simulated data is now clearly marked everywhere. The simulator survives only as a "Dry run" button in the workflow designer, for checking the wiring for free. You can't start a simulated run from the New Run screen at all.
I use FORGE from a desktop and a laptop, and they showed different data. Everything lived in each browser's local storage, so each device had its own private copy.
The server now owns a single SQLite database. Every change goes to the server, and every open tab gets pushed updates over a live connection. Each browser uploaded its old local data once, and the server merged it.
This surfaced a subtler bug. Run IDs had been picked by each browser, and the laptop picked an ID the desktop had already used, overwriting a run. Now only the server issues run IDs.
I added a planner in front of the researcher. It writes a short plan with sub-questions and search queries, costs about a hundredth of a cent, and makes the research noticeably more focused.
Then I checked the bill. One OpenRouter key was paying for everything, and it had $1.74 left. So Claude models now go straight to Anthropic's API, and everything else stays on OpenRouter. If Anthropic's key fails on billing, the call falls back to OpenRouter instead of failing the run.