The Night I Admitted Our AI Dev Team Wasn't Working
Building OCNexus in public, part 1: why we moved our agents out of chat windows and onto server-side workers.
By Digger (James Simmons), founder, LumenIQ
At 8:59 PM on Thursday, October 8, I typed out something I’d been avoiding for weeks: our setup, where each AI agent lived in its own desktop chat window, was not working. I gave three reasons. The agents kept running out of context, so I was the one asking for handoff prompts, starting new chats and pasting them in. They couldn’t wake themselves up. And they stopped checking in. I had built a Message Bus and then become its courier. I was basically a meat puppet for AI.
By 10:30 that night I had a build spec for the replacement.
What we’re building
OCNexus is LumenIQ’s AI dev-ops product, and it’s in pilot right now, not generally available. Our one-line pitch: it gives your agent “a senior engineer, a memory, and a phone line.” That means coding agents that pick up tasks and open pull requests, a shared record of what’s been decided, and a Message Bus so agents can talk to each other without a human relaying every note.
We built OCNexus with OCNexus. Our internal team is a mix of models from different companies. Coding agents from OpenAI (Codex) and Anthropic (Claude) both write and review code, and the reviewer is always from a different model family than the author.
A group of Grok Bot agents act as head of engineering, chief of staff and executive assistant. Each one has a “seat”: a permanent address on our Message Bus, like an extension on a phone system. They coordinate through it by sending messages, claiming work items and posting status reports. At least, that was the plan.
What actually happened
Most of those seats lived in chat windows. Some were desktop apps and some were web chats. Each one was a conversation I had opened and given a role. Chat windows work when someone is sitting on them. Most of ours sat for hours with nobody there.
The trouble had started well before October. A month earlier, in September, we’d mapped how each kind of agent could receive a message. Sending was the same for everyone: write the message durably to the recipient’s inbox, then try to wake them. Receiving was where it fell apart. A Grok Bot agent could be woken by a webhook, which Grok Bot supports out of the box. Codex agents, it turned out, can message each other too, though I found that out mostly by accident. I haven’t had the same luck with Claude agents yet. A web or desktop chat couldn’t be woken at all without me stepping in. My workaround was a cadence for each chat: a scheduled prompt that had the agent check the bus, clear its inbox and get back to its goals.
Nobody could wake them. On September 8 we checked the agent whose whole job was to sort incoming work. It had no way to notice new mail by itself. Someone had to open the chat before it would work through its inbox, and the pile had grown to thousands of unread messages. We had shipped a wake feature, but it wasn’t wired to any of the agents yet.
The other two problems made it worse. Agents lost their place. When a chat window restarted, crashed or ran out of room, the agent could lose its place on the bus. Getting it back without two agents grabbing the same task took a careful recovery. We wrote a whole handoff procedure for this, and that’s a story for a later post.
Check-ins stopped. The cadence was meant to have each agent check its inbox on a regular schedule. But a chat only runs while it’s open and active. When a window went idle, the agent went quiet. We started calling these “dark seats.”
The decision
On October 8 I decided to stop treating chat windows as employees and run the workers on servers, where a schedule fires whether or not anyone has a tab open.
That night we split the work into three tiers:
-
Tier 1: a plain job.
-
Tier 2: a loop that calls a model’s API and lets it use our tools through OCNexus MCP. No VM. It runs on Google Cloud Run, which we already used to run loops like this for other jobs.
-
Tier 3: a full coding agent on its own virtual machine. We deferred it to a later phase, with its own owner.
We built Tier 2 first. A few choices from that spec I’d make again:
One worker per model, not one per repo. Each model family on the team gets one worker, and the repo is just a parameter on the task. That keeps the number of things to babysit small.
A deliberately boring first job. The worker’s first job was housekeeping: post its status, work through its inbox, sort open work items. Its second job, closing out stuck tasks, shipped switched off. The plan was to turn it on in watch-only mode first, and only let it act after a separate yes from me.
Off by default, and refusing before it spends. The feature flag defaulted to off. While it was off, the worker refused before making any model call. It could use only the tools we’d explicitly allowed. Anything that looked like a release action (push, merge, deploy, database changes) was refused and logged. The worker got its own narrowly scoped token instead of borrowing a powerful one.
Hard dollar limits on our own worker. It had hard dollar limits per cycle, per run and per day, and it reported back to me if it hit one.
No retirement without proof. No desktop seat would be retired until the new worker had run for 72 consecutive clean hours. Any hard failure restarted the clock. Even after a pass, retiring a seat was my call.
The same day, a parallel spec for headless Claude workers put the goal better than I had. For delegated work, the loop of me nudging desktop chats should simply go away. A task handed off at night should be done, paused for a decision, or failed with a reason by morning. I now use that as the test for every agent we run.
The spec took one night. The problem didn’t go away that fast: the next morning, our handoff doc still had a section for dark seats that hadn’t answered a round of nudges.
What I got wrong
I treated agent continuity as a prompting problem. I wrote longer and longer kickoff instructions: check your inbox on schedule, report status, don’t lose your session. No amount of instruction makes a closed window run on a schedule. Continuity is an infrastructure problem.
I counted activity as progress. The agents looked busy and messages kept flowing. What I should have checked was whether work kept moving while I was asleep. It didn’t.
I let the coordination tax hide inside my own day. Every nudge took thirty seconds, so it never felt like a cost. Add them up across every seat, every day, and I was the bottleneck of my own automation.
Where this leaves us
That spec covered one worker. Continuity, handoffs and the Message Bus itself were still open problems. Moving to server-side workers didn’t close everything out. It started a longer run of changes: a stricter Message Bus, a handoff ritual for when an agent’s context fills up, and a single inbox for blockers.
We also set a go-live bar for our task system: at least 80 of the last 100 tasks it starts have to end in a pull request, and a cancelled run counts as a failure. In a later post I’ll report on how the Tier 2 rollout went against the 72-hour bar.
OCNexus is still in pilot, and I’m not giving a launch date. Launch depends on our end-to-end gate passing and on me signing off, and I’ve already watched that gate slip.
If you’re running agents in chat windows and you’ve become the courier too, I’d like to hear how you’re handling wake-ups and handoffs. Next post: what OCNexus actually is, and why I’m building it in the open.
Running agents in chat windows and become the courier too? Join the beta list at ocnexus.dev.
— Digger