A lot of companies are adding a person to their AI workflows and calling it control. Somebody reviews the agent’s output, clicks approve or reject, and legal sleeps better. I started calling that person a bot sitter, and on this week’s episode with Matt I made the case that most bot sitters can’t do the job they’ve been given.
That isn’t a knock on the people. It’s a knock on the setup.
What a bot sitter sees
A bot sitter is somebody hired to be a human-in-the-loop feedback mechanism for complex operations moving through an organization. You find them anywhere there’s high security, privacy, or HIPAA exposure. Sometimes they’re a regulatory fix. The lawyers are worried, so the company injects people. Matt pointed out that insurance policies drive the same move.
The trouble is what lands on their screen. A bot sitter sees fragmented pieces of information.
Matt and I had a call about a client we’re collaborating on. I brought him up to speed quickly, and from there he had context on the different pieces. Now hand Matt bits and pieces of automated output instead of that conversation. He’d have a hard time assembling the story well enough to say, with any confidence, thumbs up or thumbs down.
That’s the daily life of a bot sitter. They approve things they can’t fully see, and the approval looks like oversight on an org chart.
Where that argument takes you
If the human doesn’t have enough context to understand the guardrails, the guardrails aren’t doing much. You’re better off building an automation that understands those guardrails well, triages them, and only surfaces what’s truly important. Then you automate more of the task.
That’s an uncomfortable place to land, and I said so on the show. For lower-scale automation, it’s more effective to remove the human from the process entirely than to keep people sprinkled through it. I’d keep humans at the key inflection points: real client interaction, a final hiring decision, that kind of call. Everywhere else, I was almost convincing myself the YOLO way is the way to go. (For our older listeners, YOLO means you only live once.)
Matt lives there already. He’s set up a tool that tunnels Dot, ChatGPT’s orchestration agent, straight into his Codex, with access to his server, his browser, and his credit cards. Dot on its own asks permission over and over. His setup skips that. He asked it to book a massage at the only place within half a mile of his house, with the owner, and it was done in one prompt. Through the standard guardrails, he figures it would have asked 15 questions.
His point was that most people never feel what these agents can do because the approval loops slow everything down. They ask for something simple, get interrogated, and walk away thinking AI is dumb.
Why I’m still not handing this to your team
I buy the logic. I don’t buy that it’s appropriate for most organizations today. Matt can do this. I could probably do this. Some listeners who use these tools every week could. Hand it to a broad team and you’re asking for an agent to book a massage in the wrong state, or a foot massage when you wanted shoulders.
Matt’s answer was that this was a bigger problem six months ago and vastly less of one now. Fair. Still not perfect, and for a busy client who doesn’t live in this stuff, it has to be near perfect before they’ll trust it.
If you’re going to experiment anyway, give the agent its own virtual credit card so it can’t overspend. Matt’s team uses Ramp for that. And don’t copy his setup unless you’re Matt.
I’ll admit I get annoyed by guardrails too. I had a rage moment with Dot last night when it stopped for something stupid. I still give them a glance.
What this means if you run the place
If your plan for safe automation is to put a person at every step, ask what that person can see. If the answer is a fragment of the work and a button, you’ve added cost and delay, and you’ve given yourself a feeling of control more than the real thing.
The better question is where human judgment matters, and whether the people at those points have the full context. Put them there. Let the system handle the triage elsewhere, and build it to escalate what’s important.
I’d guess in six to twelve months more of these decisions disappear because confidence in the agents goes up. Until then, you’re deciding how much rope to give them. Matt has given his a credit card. I’m still deciding.
