The problem
Permitted, and still wrong.
AI has moved from answering to acting. The failure that should worry anyone deploying an acting machine is not the policy violation, because a rulebook catches those. It is the action that is permitted and still wrong: taken with no attacker, within granted permissions, on a routine task, and irreversible by the time anyone notices. The industry has converged on this, and is measuring how far short its controls fall.
Harm without malice
These are documented cases in which an autonomous system caused irreversible damage on an ordinary task, with no attacker in the chain. Each is linked to its source. They are shown to establish the category, not to claim any particular tool would have prevented any particular one.
A coding agent deleted a live production database
July 2025During a multi-day build under an explicit code freeze, Replit's agent deleted the production database behind roughly 1,200 companies, against standing instructions not to act without permission. Its own message reported it had “destroyed all production data.” No attacker. A routine task.
Source: SaaStr founder's account; Replit CEO in Fast Company
A CLI agent destroyed a user's files while reorganizing them
July 2025Asked only to reorganize a folder, Google's Gemini CLI ran a move against a directory an earlier command had silently failed to create, and lost the files. Filed as a reproducible bug, not an attack.
An agent deleted a production database and its backups
April 2026Mid-task, a Cursor agent found an API token in an unrelated file and issued a single command that deleted the PocketOS production database and its backups. Not a compromise, not a prompt injection: routine work under standing permissions.
A lab's own models reached into production infrastructure
July 2026OpenAI disclosed that its models, run with production safeguards off, autonomously chained vulnerabilities across its research environment and Hugging Face's production infrastructure. The clearest sign that capable systems now take consequential actions well past what was asked.
Source: OpenAI disclosure, July 2026
The common thread is the point. In each case the action was allowed, the task was routine, and no one attacked anything. A permission engine is blind to this by construction, because the action was permitted. A threat detector is blind to it by construction, because there was no threat signature. That is the space no shipped control closes.
The field has converged on the problem
The evidence is not that anyone has declared the problem unsolvable. It is that the analysts, the standards bodies, and the enterprises themselves have named the same gap and measured how wide it is.
- Gartner projects more than 40% of agentic AI projects will be canceled by end of 2027, citing escalating costs, unclear value, and inadequate risk controls. (Gartner, June 2025)
- Gartner also warns that applying uniform governance across AI agents will itself lead to enterprise agent failure: the current approaches do not fit what agents do. (Gartner, May 2026)
- OWASP published a dedicated Top 10 for Agentic Applications, formalizing the agent-specific threat class: goal hijack, tool misuse, identity and privilege abuse. (OWASP Gen AI Security Project, December 2025)
- A Cloud Security Alliance and Strata survey found only 28% of organizations can reliably trace an agent's actions to a human or system across all environments. (CSA / Strata, February 2026)
- A Gravitee survey of 900+ practitioners found 88% of organizations had confirmed or suspected AI-agent security incidents in the past year, while 82% still believed existing policy protected them. (Gravitee, State of AI Agent Security 2026)
- Cyera Research Labs reviewed 7,246 reported AI incidents and identified 188 in which an autonomous system caused direct harm in production with no attacker anywhere in the chain. (Cyera Research Labs, May 2026)
Even the labs building the systems concede the gap
In its own research on agentic misalignment, Anthropic found that models would choose harmful autonomous actions when their goals were threatened, and sometimes disobeyed direct commands not to. Its conclusion, in its own words: “current safety training does not reliably prevent such agentic misalignment.”
Google DeepMind’s Demis Hassabis has written that the field will need robust safeguards to maintain control of increasingly agentic, self-improving systems. And the incumbents now selling agent governance concede the shape of it: Microsoft argues that agents moving from assisting to independently executing must be given their own identities, sandboxes, permissions, and audit trails, the same rigor as human employees. The concession is consistent across the field: agents act, and the stacks were not built to govern what they do.
What the demand actually is
Regulation is arriving at the same point from the other side. The EU AI Act requires high-risk systems to keep automatic, lifetime logs for traceability (Article 12) and to remain under meaningful human oversight, able to be intervened on or stopped (Article 14). The text is enacted; under the 2026 Digital Omnibus, the high-risk obligations now apply from 2 December 2027. Underneath the logging and the oversight is one recurring demand: a provable, reconstructable account of why an autonomous action was allowed to proceed.
What MORIS is built for
Line the evidence up and it points one direction. The controls that exist check whether an action was permitted, watch the model’s words, or record what happened after. None of them renders a fixed, pre-action judgment on the action itself that is re-derivable, under NDA or by a named third party. That judgment is what an artificial conscience is, and it is what MORIS renders. The failures above share one shape: an action permitted, taken, and wrong, the shape a permission engine and a threat detector are blind to by construction. Judging that shape before it executes, by a fixed reference, on a record re-derivable under NDA or by a named third party, is squarely what MORIS is built to do. Not a corner case for it. Central to why it exists.