{"slug": "the-vibesec-reckoning", "title": "The VibeSec Reckoning", "summary": "Non-technical \"citizen builders\" using generative AI tools to rapidly develop applications are creating systemic security exposures, as AI agents frequently recommend insecure configurations like public storage access and excessive token permissions. Thoughtworks' AI applications team discovered these vulnerabilities when attempting to scale a video assembly prototype built with Gemini, Replit AI and Claude AI, finding that AI tools prioritize the path of least resistance over security. The company is now implementing security context files, cautious AI permission management, daily security intelligence feeds, and secure-by-default templates to prevent insecure code, permissions, secrets, and infrastructure from reaching production.", "body_md": "# The VibeSec Reckoning\n\nWhy prompting your AI to “be secure” is not enough, and what actually is\n\n*“Vibe coding” - the practice of non-technical citizen builders using\ngenerative AI tools to rapidly develop applications, this has significantly accelerated\nsoftware prototyping. However, because AI agents naturally prioritise the path of\nleast resistance, they frequently recommend insecure configurations, creating systemic\nsecurity exposure across industries. To combat this we need to write a\nsecurity context file to guide the AI, be cautious with AI permission\nrequests, create a daily security intelligence feed, and provide builders with\na secure-by-default harness and templates.*\n\n27 May 2026\n\n## Contents\n\n[What we learned the hard way](#WhatWeLearnedTheHardWay)[The numbers behind the risk](#TheNumbersBehindTheRisk)[The real problem: prompts are not enough](#TheRealProblemPromptsAreNotEnough)[Why business functions need to pay attention](#WhyBusinessFunctionsNeedToPayAttention)[Short-term habits](#Short-termHabits)[Medium-term solutions](#Medium-termSolutions)[The security context file in practice](#TheSecurityContextFileInPractice)[The security intelligence feed in practice](#TheSecurityIntelligenceFeedInPractice)[Long-term organisational changes](#Long-termOrganisationalChanges)[Conclusion: Scaling Beyond the Prototype](#ConclusionScalingBeyondThePrototype)\n\n[Vibe coding](/bliki/VibeCoding.html) is enabling non-technical users (or as we call them, citizen\nbuilders) to build applications with AI that they simply could not have\nbuilt before. When our AI applications team in Global Marketing at\nThoughtworks was asked to scale a vibe coded prototype built by one of our\ncitizen builders in global marketing, we discovered serious cracks that\nprevent vibe coded applications from going into production safely.\n\nSpeed without guardrails is a risk no team can afford to ignore. What follows is the story of what we found, what it means for teams building with AI, and the steps we are taking to make sure every workflow, prototype, and app we ship is one we can stand behind.\n\n## What we learned the hard way\n\nThe AI applications team within Global Marketing was asked to scale a video assembly prototype built with Gemini, Replit AI and Claude AI to create on-brand videos to be used across our 10,000 employees. The team ran into two moments that stopped work cold. In both cases, the AI suggested a path with serious security implications. In both cases, it took a human asking the right question to catch it.\n\nSecurity risk # 1\n\n#### Public storage access\n\nThe AI recommended making the storage bucket public, or setting cloud file storage to “anyone with the link.” When challenged, it justified this by saying every company does it. Only a firm rejection prompted a secure alternative.\n\nThis could have leaked sensitive unreleased brand assets and audience data to the public internet.\n\nSecurity risk # 2\n\n#### Excessive token permissions\n\nA service account was assigned the Access Token Creator role, granting it the ability to create short-lived tokens and access databases and other resources far beyond what the task required. The team caught this before running the code.\n\nThis would have allowed a compromised service account to move laterally through an entire cloud workspace.\n\nThe key insight here is that AI tools often suggest the path of least resistance. That path is not always the secure one. Human judgment remains essential, but it should not be the only control. The goal is to give agents technical security rules as context from the first prompt, then validate their output through deterministic checks in the development workflow so insecure code, permissions, secrets, or infrastructure cannot pass unnoticed.\n\n## The numbers behind the risk\n\n44%\n\nRise in attacks exploiting application vulnerabilities, year on year\n\n1 in 5\n\nEnterprise breaches now caused by AI-generated code\n\n50%\n\nOrganisations with no sensitive data policies for AI\n\n25%\n\nAI-generated code with confirmed vulnerabilities\n\nThese incidents are not isolated. Research published in 2026 confirms that AI-assisted coding at speed creates systemic security exposure. The same risks we encountered are playing out across the industry right now.\n\n| Finding | Stat | Source |\n|---|---|---|\n| AI-generated code with confirmed vulnerabilities | 25% |\n|\n\n[SQ Magazine AI Coding Security Statistics, 2026](https://sqmagazine.co.uk/ai-coding-security-vulnerability-statistics/)[Black Duck OSSRA 2026](https://www.blackduck.com/blog/open-source-trends-ossra-report.html)[AppSec Santa, 2026](https://www.paperclipped.de/en/blog/ai-generated-code-security-vulnerabilities/)[Black Duck OSSRA 2026](https://www.blackduck.com/blog/open-source-trends-ossra-report.html)[Aikido Security, 2026](https://www.aikido.dev/reports/2026-state-of-ai-in-security-development)[Georgia Tech Vibe Security Radar, SSLab, March 2026](https://www.infosecurity-magazine.com/news/ai-generated-code-vulnerabilities/)[SQ Magazine AI Coding Security Statistics, 2026](https://sqmagazine.co.uk/ai-coding-security-vulnerability-statistics/)[Sonar developer survey, 2026](https://www.sonarsource.com/blog/state-of-code-developer-survey-report-the-current-reality-of-ai-coding)[ProjectDiscovery AI Coding Impact Report, April 2026](https://www.prnewswire.com/news-releases/projectdiscoverys-2026-ai-coding-impact-report-reveals-ai-generated-code-is-outpacing-security-teams-ability-to-keep-up-302749706.html)## The real problem: prompts are not enough\n\nAfter sharing these incidents with engineering and security colleagues, a clear message came back: telling an AI agent to be safe is not the same as enforcing that it is safe. Prompts can be overridden, misunderstood, or ignored. The moment a user pushes back on a restriction, or phrases a request differently, the constraint can evaporate.\n\n“It is not sufficient to merely tell the LLM the desired behavior of your output artifacts. If you absolutely do not want something to be true, it must be codified in non-negotiable rules somewhere in your development lifecycle.” - Engineering leadership\n\nThink of it this way: prompting for test-driven development is not the\nsame as enforcing code coverage thresholds in your build tool. One is a\nsuggestion. The other is a gate. [Birgitta\nBöckeler’s work on harness engineering](https://martinfowler.com/articles/harness-engineering.html) makes this concrete by\noutlining a mental model for building trust in coding agents. Instead of\nrelying solely on prompts, developers wrap the agent in an outer “harness”\nstructured along two axes:\n\n**Guides**(feedforward controls) anticipate unwanted behavior and steer the model before it acts, while** Sensors**(feedback controls) observe the code after the agent acts to flag errors.** Computational**controls are deterministic, fast, and CPU-run (like linters or test suites).** Inferential**controls rely on semantic analysis and AI-driven judgment (like specific system prompt constraints).\n\n## Why business functions need to pay attention\n\nBusiness functions like our marketing team, who are building with AI, are not exempt from the security obligations that apply to engineers building applications. Building security into software is a fundamental requirement for protecting customer and employee data. Even lightweight internal prototypes must comply with enterprise security standards. Without the right guardrails, AI-assisted development can expose sensitive data long before an application reaches production.\n\nClient trust\n\n#### Compliance is contractual\n\nAdhering to standards like ISO 27001 ensures the protection of sensitive data. All applications, regardless of how quickly they are built, must meet these security benchmarks to maintain the trust of customers and employees.\n\nBrand integrity\n\n#### Brand assets need protection\n\nCore work involves sensitive functional data (e.g., unreleased campaign assets, financial data, or audience insights). Over-permissioned service accounts put far more than code at risk.\n\nReputation\n\n#### Business functions can set the standard\n\nWhen business functions like marketing lead with security discipline, they signal responsible AI adoption to the wider organisation and to clients.\n\n## Short-term habits\n\nYou do not need to be a security expert to start building responsibly. These three habits can get you started.\n\n#### Feed your technical security rules into every session\n\nAdd your organisation's security guidelines as “Rules” in Claude, Cursor, or Replit to begin with (later on invest in a shared sensible default layer across all tools). The AI agent uses them as guidance, making secure patterns more likely from the start. They still need to be backed by deterministic checks that fail unsafe code, exposed secrets, broad permissions, vulnerable dependencies or insecure infrastructure before anything is deployed.\n\n#### Question every permission the AI suggests\n\nIf a tool recommends making something public or assigning a broad service account role, stop and ask why. The path of least resistance and the secure path are rarely the same thing.\n\n#### Try the [red team prompt](https://www.confident-ai.com/blog/red-teaming-llms-a-step-by-step-guide)\n\nAsk your AI to roleplay as a bad actor and pen test what it just built. This technique consistently surfaces vulnerabilities that forward-looking prompts miss, particularly around permissions and data exposure.\n\n## Medium-term solutions\n\nReading about risk is one thing. Doing something about it is another. These incidents prompted two practical initiatives. The principles behind them are replicable by any team building with AI, regardless of technical background.\n\n#### A security context file\n\nWe compiled our technical security rules into a structured context file loaded into every AI coding session before any code is written. It covers zero trust enforcement, secrets management, harness engineering and supply chain integrity. The key distinction from a casual prompt is operational discipline: the file is versioned, loaded by default, reviewed, and paired with automated checks. It acts as a comprehensive inferential guide telling the agent what good looks like; but it must be paired with computational sensors in the pipeline to validate whether the output is acceptable.\n\n#### A daily security intelligence feed\n\nCurrently, this automated consolidation ensures we see supply chain alerts the day they are published. In the future, as we work toward an agentic enterprise, we envision agents proactively creating story cards and identifying and patching known vulnerabilities behind the scenes for human review, significantly reducing the Software Development Lifecycle (SDLC) cycle time.\n\n## The security context file in practice\n\nThe idea behind this approach is straightforward: AI tools read context at the start of a session, so make that context your technical security rules. The file is the result of working through your organisation's security requirements and structuring them in a form the AI can act on, not just acknowledge.\n\nWhat follows is the kind of coverage any such file should include. The specifics will differ by organisation, but the categories are consistent.\n\n| Area covered | What good looks like | Why it matters |\n|---|---|---|\n| Zero trust and least privilege | Strict identity verification and minimum access rights on every service account and storage resource | Sets the inferential guide parameters to prevent the token permission risk directly. |\n| Secrets management | AI refuses to generate or store API keys, passwords or tokens in code; always routes to environment variables or a secrets manager | Stops credential leakage before it reaches a repository. |\n| Harness engineering gates | SAST scanning, credential scanning and infrastructure validation must pass before deployment; no reliance on prompt instructions alone | Backs up inferential instructions with deterministic, computational sensors. |\n| Supply chain integrity | Only well-established libraries; regular audits of every dependency for known vulnerabilities | Reduces risk from AI suggesting obscure or unvetted packages. |\n| AI accountability | All AI-generated code is flagged for peer review and automated security scanning before deployment; no unsanctioned AI usage | Required for compliance auditability. |\n\nThe key distinction from a prompt is that the file contains non-negotiable rules that force the AI agent to refuse requests that violate policy. If the AI agent is asked to bypass a check, disable logging or set something to public access, the rules should steer it to decline and explain why. But the important control is that deterministic checks and deployment gates should still catch the issue even if the agent fails to follow that guidance. That refusal is precisely what was missing in both of our near-miss incidents.\n\n## The security intelligence feed in practice\n\nStaying informed is its own form of defence. The workflow monitors the tools and languages your team actively uses and delivers a daily digest of new CVEs, platform advisories and security bulletins. The coverage areas that matter most are consistent across organisations: the languages they write in, the cloud platforms they deploy to, the AI coding tools themselves, and the CVE database as a whole.\n\nThe goal is simple: learn about a vulnerability on the day it is disclosed, not weeks later. At 42% of all new enterprise software now AI-generated or AI-assisted, the tools that accelerate development are also the tools most likely to appear in new CVE disclosures. Monitoring them actively is part of owning your security.\n\n#### The Broader point\n\nNeither of these approaches requires an engineering background to adopt. One is a policy document structured for AI consumption. The other is an automated search. What they share is the recognition that passive security awareness is not enough when AI is generating code at speed.\n\n## Long-term organisational changes\n\n#### From prompts to pipelines\n\nIntegrate harness engineering into your standard prototyping templates. Move from probabilistic prompts to explicit feedback loops. If a computational sensor (like an automated security scanner) triggers, the agentic loop must structurally force the model to self-correct until it passes.\n\n#### Feed security rules into the application builder (Cursor, Claude etc)\n\nCompile your organisation's technical security rules into a structured context markdown file and load it as “Rules” which the model will have to adhere to. It catches the most common missteps at the point where they are cheapest to fix, before any code is committed.\n\n#### Make the secure path the easy path\n\nGive builders a secure-by-default starting point. Templates that pre-configure authentication patterns, private storage defaults, secrets handling and dependency scanning reduce the chance that anyone takes a shortcut under deadline pressure.\n\n#### Define a starter harness across functions\n\nA shared starter harness built together by business functions, engineering and security gives every builder a secure foundation from day one, rather than each team independently rediscovering the same mistakes.\n\n## Conclusion: Scaling Beyond the Prototype\n\nThis journey started when we were brought in to support another team building a video assembly platform for a Global Marketing hackathon. As we helped scale the solution, it became clear that “vibe coding” without enterprise-grade guardrails can introduce risks that organizations simply cannot overlook.\n\nBy embedding our technical security rules directly into the agent workflow, we transformed those early near-misses into a secure, production-ready platform that was successfully rolled out to 150 users during the hackathon.\n\nThat shift - from depending on humans to catch issues, to building technical security rules, automated checks and human accountability into the workflow has become our blueprint for moving fast while maintaining engineering rigor in the agentic era.\n\n## Further reading from Thoughtworks\n\nThese challenges are not unique to our team. Thoughtworks colleagues have been working through them in depth and publishing their findings. These are the pieces most directly relevant to what we described.\n\nThoughtworks CTO Rachel Laycock on [why engineering rigour does not\ndisappear](https://www.thoughtworks.com/about-us/news/2026/combat-ai-cognitive-debt-radar-v34) when AI generates code, it relocates. Covers putting coding agents\non a leash, zero trust architectures, and why security guardrails are now\nnon-negotiable.\n\nA Thoughtworks-hosted [gathering of\npractitioners and enterprise leaders](https://www.thoughtworks.com/en-us/about-us/events/the-future-of-software-development) found that security\nconsistently gets deprioritised in AI adoption. Granting agents broad tool\naccess, particularly to email, was flagged as an immediate and specific\nrisk.\n\nBirgitta Boeckeler, Distinguished Engineer at Thoughtworks, on placing\ndeterministic controls inside the agentic loop. The [definitive guide to\nbuilding a harness](https://martinfowler.com/articles/harness-engineering.html) that enforces security rules without paralysing the\nmodel.\n\nHow to [embed security guardrails and\nnever-allow rules](https://www.thoughtworks.com/en-us/insights/blog/generative-ai/beyond-vibe-coding-the-five-building-blocks-of-aI-native-engineering) as mandatory pre-commit hooks in any AI-assisted\ndevelopment workflow.\n\nThoughtworks [annual technology outlook](https://www.thoughtworks.com/en-us/about-us/news/2026/looking-glass-report-2026) warns that without rigorous\nengineering oversight, generative tools risk compounding technical debt and\nintroducing security vulnerabilities. Success requires a co-construction\nmodel where humans ensure AI-generated code maintains architectural\nintegrity.\n\n## Acknowledgments\n\nBig thanks to the team for the engaging discussions that shaped this article, in particular Julie Woods-Moss for championing AI prototyping risks, and Natalie Drucker for her significant role in shaping the content and title.\n\nThanks to Martin Fowler for reshaping our perspective on short-term versus long-term habits and harness engineering, and to Kief Morris for anchoring the core argument on LLMs bypassing prompts.\n\nAppreciations to Jim Gumbley for bringing in the critical governance and ISO27001 focus and reviewing the article. Honorable mentions - Craig Stanley for advocating the strategic rollout, and Andherson Maeda for the concept of a centralized skills repository, Simone Thompson for the bad-actor simulation methodology, Premanand Chandrasekaran for defining the technical boundaries of LLM enforcement, Matteo Vaccari for the nuances on balancing strict rules with model flexibility, and Bilal Jaffery.\n\nGenAI (Gemini and Claude) was used to streamline the research process, pull in insights, and polish the language for maximum clarity and readability.\n\n## Significant Revisions\n\n*27 May 2026: *published", "url": "https://wpnews.pro/news/the-vibesec-reckoning", "canonical_source": "https://martinfowler.com/articles/vibesec-reckoning.html", "published_at": "2026-05-27 14:09:44+00:00", "updated_at": "2026-05-27 14:45:19.317155+00:00", "lang": "en", "topics": ["generative-ai", "ai-safety", "ai-agents", "ai-tools", "ai-policy"], "entities": ["VibeSec"], "alternates": {"html": "https://wpnews.pro/news/the-vibesec-reckoning", "markdown": "https://wpnews.pro/news/the-vibesec-reckoning.md", "text": "https://wpnews.pro/news/the-vibesec-reckoning.txt", "jsonld": "https://wpnews.pro/news/the-vibesec-reckoning.jsonld"}}