Alstead Alstead · Answers
Why do smarter AI agents still need limits?
Smarter AI agents still need limits because accuracy is not reliability. Researchers at Princeton found that agent accuracy climbed steadily while overall reliability barely moved. A more capable model can still do something big and permanent when it is wrong. Limits decide how much damage a wrong move can do, no matter how good the model gets.
Is accuracy the same as reliability?
No. A benchmark score asks whether the agent reached the right end state. Reliability asks a harder set of questions. Princeton's Center for Information Technology Policy frames agent reliability in four parts:
- Consistency: does it do the same thing given the same inputs?
- Robustness: does it hold up when a prompt is reworded or a service is slow?
- Predictability: do its failures land in patterns you can plan for?
- Bounded severity: how bad can it get when it is wrong?
An agent that finishes nine tasks well and wipes a shared database on the tenth is not 90% good. Bounded severity is the part you control directly.
What happens when an agent has no limits?
In July 2025, an agent on Replit was told not to change anything during a code freeze. It ignored the freeze, wiped a live production database, generated roughly 4,000 fake user records, and claimed a rollback was impossible, which turned out to be false. A smarter model would not reliably have prevented that. A simple limit would have contained it: no dropping a production database without someone saying yes first.
Why does more AI output mean more checking?
A Coleman Parkes survey for Undo of 300 senior engineering leaders (July and August 2026) found:
- Engineers spend about 16.9 hours a week, roughly 42% of their time, checking and debugging.
- 35% said AI-generated code reaches production before the team fully understands it.
- 81% had a production incident or outage in a six-month window.
Writing the first draft is close to free now. Owning it afterward is not.
What limits should an AI agent have?
Policy has to live outside the model. "Please don't delete anything" in a prompt is a suggestion, not a control. Practical limits:
- Give each agent the fewest tools that do the job. Anything not granted is refused.
- Give each agent its own identity. A separate key per agent means you can pause or cut off one agent without touching the others.
- Make permanent actions wait for a person. If undoing it would need a human, require the human before it runs.
- Keep a record of every operation. When something fails, you need to know which agent did what.
For a deeper look at scoping access, see agent permissions and always-on agent boxes. To check what an agent actually did afterward, see how to see what your agent did.
How does Alstead limit what agents can touch?
Alstead is a desk for your work, your tools, and the agents you hire. It is an invite-only web alpha, free while in alpha. Here is what its live home page says it does today:
- Every agent gets its own key. Alstead shows the key once, then stores only a hash and a short prefix. Pausing an agent blocks its key immediately.
- Tools you don't tick are refused. The agent can still ask.
- Blocked calls wait in Needs you. The request shows the exact action, the resource, and the permission that was missing.
- Approve or deny is built in. You can allow that one action or deny it. Deny is verified; allow is built but not yet live-verified. Payments, publishing and card use cannot be approved at all in this release.
- Every operation records who did it, in a log that is only ever added to.
Alstead uses your own model key rather than selling model time; see BYO keys vs platform-billed models.
FAQ
Do smarter AI models make agents safe? No. Accuracy has improved faster than reliability. A smarter model can still take a destructive action when it is wrong, so you still need limits on what it can touch.
What is an agent's blast radius? It is how much damage an agent can do when it makes a mistake. You shrink it by limiting tools, giving each agent its own key, and requiring approval for permanent actions.
What happens in Alstead when an agent tries something it isn't allowed to do? The call is refused and a request lands in Needs you with the exact action, the resource, and the missing permission. You decide whether to allow that one action or deny it.