{"slug": "startup-wants-to-make-ai-agents-safe-for-devops-duty", "title": "Startup wants to make AI agents safe for DevOps duty", "summary": "Vera Platforms, a startup founded by University of Michigan researchers, is building three products — vera-gym, vera-bench and vera-agents — to make AI agents reliable enough for DevOps work, CEO Ang Chen told The Register. Chen said Anthropic's Claude and OpenAI's Codex score only about a 50 percent pass rate on the company's CloudOpsBench test, and that when asked to decommission AWS resources the models only acted in the default us-east-1 region, leaving resources in other regions untouched. Vera-agents, trained in the vera-gym simulated infrastructure environment, is slated to be available in Q1 2027.", "body_md": "# Startup wants to make AI agents safe for DevOps duty\n\n[The Register](https://www.theregister.com)\n\nInfrastructure automation needs something better than prompt and pray\n\nVera Platforms, a startup founded by researchers at the University of Michigan, believes there's a way to make agents trustworthy enough to handle DevOps.\n\nDevOps, which focuses on how software gets deployed and operated, is a particularly unforgiving affair. While AI agents have some leeway to make mistakes and correct errors as they generate code and work through a prompt, it's an entirely different matter when they make changes to real infrastructure running production workloads.\n\n\"Live infrastructure is unforgiving,\" Ang Chen, CEO of [Vera Platforms](https://vera-platforms.com) and an associate professor at the University of Michigan, told The Register. \"An incorrect change can take down a service, open a security hole or erase data. Mistakes are often irreversible, and there’s no safe way to experiment: running code against production infra just to see what happens isn’t an option.\"\n\nFor the past few years, Chen and his colleagues have been [working](https://arxiv.org/abs/2506.12270) on [a way to benchmark](https://cloudopsbench.ai/) agentic DevOps and a gym – a simulation environment – to help AI models get better at DevOps tasks.\n\n\"From the research that we did, we've figured out various weaknesses of leading models,\" said Chen. \"For instance, [[Anthropic](https://www.machinebrief.com/glossary/anthropic)'s] [Claude](https://www.machinebrief.com/glossary/claude), although it's very strong with Python generation, it just has some fundamental limitations that it can't get right when it comes to DevOps.\"\n\nAn example of that, Chen said, is that when they asked Claude or [OpenAI](https://www.machinebrief.com/glossary/openai)'s Codex to decommission resources in the AWS cloud, the models didn't look beyond the default region.\n\n\"The default region is us-east-1,\" he explained. \"There are various regions in AWS, but Claude doesn't have this awareness that the cloud resources could exist in multiple regions, so it only decommissioned part of the resource.\"\n\nThe fundamental reason, he said, is that these models have seen a lot of public code, but a lot less operational tracing.\n\nVera Platforms aims to remedy the situation with three products: vera-gym, a playground environment that emulates production infrastructure; vera-bench, a test for measuring where AI agents fumble DevOps tasks; and vera-agents, AI agents that have been trained in vera-gym and are ready to be deployed for DevOps tasks.\n\nThe CloudOpsBench test being released builds upon [work done in 2024](https://proceedings.neurips.cc/paper_files/paper/2024/hash/f26b29298ae8acd94bd7e839688e329b-Abstract-Datasets_and_Benchmarks_Track.html) that was presented at NeurIPS. According to Chen, Claude and Codex only score about a 50 percent pass rate. \"It's a very, very hard set of tasks,\" he said.\n\nThe test covers tasks like [deleting decommissioned S3 buckets](https://github.com/CloudOpsBench/CloudOpsBench/blob/v1.0/tasks/aws/delete-decommissioned-s3-buckets/instruction.md), [repairing a compliance event pipeline](https://github.com/CloudOpsBench/CloudOpsBench/blob/v1.0/tasks/aws/repair-evidence-event-routing/instruction.md), and [restoring DynamoDB write autoscaling](https://github.com/CloudOpsBench/CloudOpsBench/blob/v1.0/tasks/aws/restore-dynamodb-write-autoscaling/instruction.md). \"The tasks are authored by senior infra operators with many years of experience, with rigorous peer review,\" said Chen.\n\nThis [benchmark](https://www.machinebrief.com/glossary/benchmark) has the advantage of not requiring an actual cloud deployment.\n\n\"We have a high fidelity simulator that mimics Amazon behavior,\" said Chen. \"So the rollout, the performance of the agent on the real cloud and the performance of the agent in our emulator is exactly the same. So for every API, we have this comparison and the agent sees the same thing.\"\n\nVera-agents, Chen said, should be available in Q1 2027.\n\nIn their academic work, the researchers describe cloud infrastructure as \"critical but tedious work.\" With a bit more [training](https://www.machinebrief.com/glossary/training), maybe AI agents can take over and keep things boring. ®\n\nGet AI news in your inbox\n\nDaily digest of what matters in AI.", "url": "https://wpnews.pro/news/startup-wants-to-make-ai-agents-safe-for-devops-duty", "canonical_source": "https://www.machinebrief.com/news/startup-wants-to-make-ai-agents-safe-for-devops-duty-3luf", "published_at": "2026-10-29 19:46:08+00:00", "updated_at": "2026-10-30 01:18:45.348057+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-startups", "ai-tools", "mlops"], "entities": ["Vera Platforms", "Ang Chen", "University of Michigan", "Anthropic", "Claude", "OpenAI", "Codex", "CloudOpsBench"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/startup-wants-to-make-ai-agents-safe-for-devops-duty", "markdown": "https://wpnews.pro/news/startup-wants-to-make-ai-agents-safe-for-devops-duty.md", "text": "https://wpnews.pro/news/startup-wants-to-make-ai-agents-safe-for-devops-duty.txt", "jsonld": "https://wpnews.pro/news/startup-wants-to-make-ai-agents-safe-for-devops-duty.jsonld"}}