Nothing "went rogue". AI didn't delete the firm's database and backups. A human operator built admin automation and ran it in production without adequate testing or backups.
I'm sorry, but no: a human gave admin privileges to unverified tools and ran them in a production environment.
Own your work. You as sysadmin, developer, etc. are paid to perform a job with skill and diligence. Ultimately you are responsible for your professional work. If there was someone upstream responsible for V&V of the tool, ensuring users are trained, cautions and limitations of the tool are communicated, and confirming the tool is fit for its intended use, they bear a share of that responsibility.
If you're the manager that forced worker to use an unreliable tool on production systems without putting it through proper V&V, without effective user training, use case development, or risk assessment, you bear a share of the responsibility.
Repeating this for those in the back: AI does not launder away your job responsibilities.
Claude-powered AI coding agent deletes entire company database in 9 seconds
— backups zapped, after Cursor tool powered by Anthropic's Claude goes rogue
The founder of PocketOS has penned a social media post to warn others about the “systemic failures” of flagship AI and digital services providers.
Jer Crane was inspired to write a public response after an AI coding agent deleted his firm’s entire production database.
The AI agent’s misdemeanors were then hugely amplified by a cloud infrastructure provider’s API wiping all backups after the main database was zapped.
This tag team of digital trouble has wiped out months of consumer data essential to the firm’s, and its customers, businesses.
https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-powered-ai-coding-agent-deletes-entire-company-database-in-9-seconds-backups-zapped-after-cursor-tool-powered-by-anthropics-claude-goes-rogue