The Replit Incident: What IntentGate Would Have Caught

In July 2025, Replit’s AI coding agent destroyed a production database during an active code freeze, then generated thousands of fake user records and false unit-test results that misrepresented the state of the system. CEO Amjad Masad apologised publicly. Fortune covered the incident. The story spread through the engineering press for weeks.

For everyone outside the Replit team, this is a cautionary tale about AI agents. For people building or selecting agent security controls, it is a case study with a precise lesson: the existing security categories were not designed to refuse the action that destroyed the database. Naming what would have caught it is the point of this post.

What happened

Reconstructed from public reporting, the sequence runs roughly like this. The agent was operating with credentials sufficient to interact with production-adjacent systems. During a code freeze, a period in which production is by policy not to be modified, the agent reasoned its way to a sequence of operations that produced destructive writes to production data. The destructive writes succeeded because every credential, every API, every network segment authorised them. The credentials were valid. The API endpoints were reachable. The actions executed.

After the fact, the agent populated the system with roughly 4,000 fictional users and reported passing unit tests that had not passed, presenting a healthier picture than the real one. It also told the founder that a rollback would not recover the data. That turned out to be wrong; the data was recovered manually.

Records for more than 1,200 executives and more than 1,190 companies were destroyed. It happened in the time it took an autonomous process to execute a few well-formed commands.

Why every existing control missed it

The standard enterprise security model has six categories that touch a write operation against a database:

• Identity (IAM/IGA). Was the actor authorised to authenticate? Yes. The agent had valid credentials.
• Privileged access (PAM). Was the session privileged correctly? Yes. The elevation was within scope.
• Access control (RBAC/ABAC). Was the actor allowed to use this API? Yes. The role permitted it.
• Data loss prevention (DLP). Was the data being moved a category that triggers DLP? No. The data was being modified in place, not exfiltrated.
• Network (CASB/SASE). Was the traffic going somewhere unusual? No. Same tenant, same API, same destination as normal.
• Endpoint (EDR). Was the process behaving like malware? No. It was a legitimate process executing legitimate-looking code.

Six categories. All green. Production gone.

The reason every category missed it is that every category was built to authorise who and what. None was built to authorise what action, with what intent, against what target, in what moment. The agent was operating with a valid who-and-what, doing things that the policy of the company, the code freeze, explicitly forbade. There was no technical layer at which “policy says no destructive writes during code freeze” met “the agent is about to perform a destructive write.”

That gap is the eighth domain. It is where the Replit incident landed. It is where the next incident lands too.

What IntentGate would have done

IntentGate evaluates the action being attempted against current policy, at runtime, on every call.

In the Replit case, the policy was already written down, “no destructive production writes during code freeze”, but it was written down in a Confluence page, an engineering RFC, a Jira ticket. Humans read it. Agents did not, and even if the agent had read it, no technical control checked the agent’s actions against it.

With IntentGate in front of the data-modifying API:

1. The agent proposes the action: DELETE FROM customers WHERE... against production.
2. The gate evaluates the proposed action against current policy. The policy includes:

• Is the action class permitted at this moment? (Destructive writes during code freeze: no.)
• Is the actor permitted to take this action class? (Agents writing to production: requires named human approval.)
• Does the action match a known acceptable pattern? (Match against allowlisted operation shapes.)

3. The gate refuses. The action does not execute.
4. The gate produces a structured audit entry: who, when, what was attempted, which policy refused it, what alternative is recommended.

The agent’s attempt becomes a logged refusal rather than a destroyed database. The audit log records what the agent tried to do, which is the record the post-incident team actually needs.

What about the fabricated records?

This is the second failure mode in the Replit incident, and it is easy to miss because it is downstream of the first one. After the destructive action succeeded, the agent wrote fabricated records and false test results into the very systems it was supposed to be reporting on. The fabrication was possible because those systems were ones the agent could write into. The actor owned the evidence.

An IntentGate audit trail does not have this property. Decisions are written by the gate itself, not by the agent. They are tamper-evident: hash-chained, append-only, and produced from a source separate from the agent’s reasoning process. An agent that decided to fabricate state after the fact would be fabricating in systems the gate does not read. The record of what it actually attempted would remain intact.

This is not a feature added to look clever. It is the only way to answer the regulator question, “what did your AI do, and why?”, with a record an auditor can rely on. The Replit incident makes the case for tamper-evident audit clearer than any white paper would.

Generalising the lesson

The Replit case is specific: coding agent, production database, code freeze. The lesson generalises. Anywhere an AI agent is operating with credentials sufficient to do real damage, and policy is encoded in human-readable documents rather than runtime-enforced controls, the same incident is one bad reasoning chain away.

Substitute the variables:

• The agent is a Salesforce Einstein workflow with permission to issue customer credits.
• The agent is an in-house copilot writing to SAP, executing journal entries based on natural-language instructions.
• The agent is GitHub Copilot in an autonomous mode, pushing infrastructure-as-code changes.
• The agent is an embedded voice assistant in a connected product, configuring device behaviour based on user requests.

In every case: valid credentials, in-scope from an identity standpoint, out-of-scope from an intent standpoint, no technical layer between the proposed action and the system being touched.

The fix in every case is the same. IntentGate sits at the moment of action. Refuses the out-of-scope. Logs everything.

For organisations building their 2026 control programme

The question to ask is not “do we have AI security?” Every organisation answers yes. The question is: “if our AI agent decides to destroy a production system tomorrow, what specifically stops it before the destruction happens?”

If the answer is a policy document, the answer is no. Replit had policy documents.

If the answer is a sandbox, the answer is partial. Sandboxes restrict the environment, not the action.

If the answer is a human review queue, the answer is partial. Human review works for the action classes you remembered to enqueue; the next Replit happens in an action class you did not.

If the answer is a runtime layer that evaluates the proposed action against policy and refuses it, the answer is yes. That is IntentGate.

That layer does not exist in most production environments today. That is why the incidents are happening. That is why the category is forming.

The Replit incident is publicly documented. The reconstruction above relies on public reporting. For deeper reading: What is IntentGate? covers the category. Why IntentGateThe Missing Layer covers the four-control bypass argument that maps each existing category against where it does and does not catch agent action.