{"slug": "ai-cobol-to-java-migration-bugs-included", "title": "AI COBOL-to-Java Migration: Bugs Included", "summary": "A team successfully migrated COBOL programs to Java using an AI LLM, preserving legacy bugs as intentional behavior because existing behavior is the specification. The team treated old COBOL as ground truth, built a golden-test harness with thousands of production input-output pairs, and used a file-by-file conversion pipeline with human review for diffs. The AI hallucinated two off-by-one errors in a date-conversion routine, but static analysis caught them, and the team emphasizes that the old code is the spec and the test suite is the safety net.", "body_md": "# AI COBOL-to-Java Migration: Bugs Included\n\n`COMPUTE`\n\nstatement since 1994.That's not a failure. That's the whole point of legacy migration.\n\nWhen you're moving a system that has been running for decades, the existing behavior *is* the specification. The accounting department has built workflows around that rounding error. The CRM team has a workaround for the leap-year glitch. If the AI had \"fixed\" those quirks, the migrated system would break downstream processes in ways that have nothing to do with the code itself. So preserving bugs is the correct outcome — it means the LLM understood the contract, not just the syntax.\n\nHere's how the team actually pulled it off:\n\n1. **They treated the old COBOL as ground truth.** No prompts like \"modernize this\" or \"improve performance.\" Every instruction was scoped to a specific program, with the output format defined and the runtime environment pinned.\n\n2. **They built a golden-test harness first.** Instead of relying on the AI to verify its own work, they fed it thousands of input-output pairs from the production logs. The AI was told to match those outputs exactly, down to the decimal point.\n\n3. **They used a file-by-file conversion pipeline.** A small agent orchestrated the migration — reading the COBOL section, generating the Java class, compiling it, running the tests, and reporting diffs. The human reviewer only got involved when the diff count crossed a threshold.\n\nThat's where the prompt engineering mattered. The key wasn't a clever system prompt. It was embedding the test harness and the error history into the context so the model knew *why* the code looked the way it did. The AI still hallucinated two new off-by-one errors in a date-conversion routine — but the static analysis step caught them before anyone noticed.\n\nIf you're thinking about doing this yourself, here's the real lesson:\n\n**The old code is the spec.** Don't ask the LLM to improve, refactor, or clean up anything during migration. You'll lose behavior you didn't even know you depended on.**The test suite is your safety net.** The AI reproducing the old bugs is proof it learned the system's quirks. The test suite is what tells you it didn't invent new ones.**You still need a human to review the diffs.** The LLM's migration was impressive, but it flagged two suspicious loops that turned out to be its own mistakes. A human who knows the domain had to confirm those were hallucinations, not legacy behavior.\n\nThe real risk with LLM-based migration isn't that it copies bugs. It's that it silently produces new ones while making the old code look clean. The team's approach — treat the output with suspicion, compare it against a large corpus of real transactions, and keep the migration pipeline reversible — handled that risk well.\n\nIf you're planning something similar, start small. Pick one program, build a golden-test harness, run the migration, and diff the behavior. Once you see the AI reproduce\n\n[Next How an AI wrote and illustrated fables about threads and packets →](/en/news/4809/)", "url": "https://wpnews.pro/news/ai-cobol-to-java-migration-bugs-included", "canonical_source": "https://promptcube3.com/en/news/4811/", "published_at": "2026-08-03 05:41:25+00:00", "updated_at": "2026-08-03 05:54:03.104498+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "ai-agents", "developer-tools"], "entities": ["COBOL", "Java", "LLM"], "alternates": {"html": "https://wpnews.pro/news/ai-cobol-to-java-migration-bugs-included", "markdown": "https://wpnews.pro/news/ai-cobol-to-java-migration-bugs-included.md", "text": "https://wpnews.pro/news/ai-cobol-to-java-migration-bugs-included.txt", "jsonld": "https://wpnews.pro/news/ai-cobol-to-java-migration-bugs-included.jsonld"}}