How to prove an AI-converted COBOL program still behaves the same

A developer demonstrated a golden-file parity testing approach for AI-converted COBOL, showing that a Java rewrite of a monthly interest calculation compiles cleanly but diverges from the legacy program on negative balances because COBOL's signed-to-unsigned MOVE stores absolute value. The proposed JUnit 5 test, driven by expected values recorded from the original COBOL program at boundary inputs like the 10,000 threshold and rounding-sensitive amounts, catches the discrepancy that a test generated from the Java would miss.

AI can turn a COBOL program into Java that compiles on the first try. A clean compile tells you the syntax is valid. It says nothing about whether the Java produces the same numbers as the COBOL, and in financial code the numbers are the whole job. This post walks through one small interest calculation, shows where a faithful-looking conversion goes wrong, and builds the test that catches it. The expected values in that test come from the legacy system itself, which is the step most teams skip. Here is the rule in COBOL, with declarations trimmed for space: 01 ACCT-BALANCE PIC S9 7 V99. 01 WS-BAL PIC 9 7 V99. 01 WS-RATE PIC 9V999. 01 WS-INTEREST PIC 9 7 V99. COMPUTE-INTEREST. MOVE ACCT-BALANCE TO WS-BAL. IF WS-BAL 10000 COMPUTE WS-RATE = 0.035 ELSE COMPUTE WS-RATE = 0.025 END-IF. COMPUTE WS-INTEREST ROUNDED = WS-BAL WS-RATE / 12. Balances above 10,000 earn 3.5% a year and everything else earns 2.5%. The monthly interest is rounded to two decimals. Two details sit in the declarations instead of the logic. ACCT-BALANCE is signed the S in the picture clause and WS-BAL is not. Moving a signed value into an unsigned field stores its absolute value, so a balance of -50.00 becomes 50.00. Also, ROUNDED rounds halves away from zero. Without it, COBOL truncates the result. Here is a reasonable Java version: public static BigDecimal monthlyInterest BigDecimal balance { BigDecimal rate = balance.compareTo new BigDecimal "10000" 0 ? new BigDecimal "0.035" : new BigDecimal "0.025" ; return balance.multiply rate .divide BigDecimal.valueOf 12 , 2, RoundingMode.HALF UP ; } It uses BigDecimal instead of double, keeps the threshold and both rates, and rounds to two places. A reviewer would approve it. For a balance of -50.00 it returns -0.10, while the COBOL program returns 0.10. The old behavior may be a bug, or it may be a rule that production depends on. The converted code cannot tell you, and neither can a test generated from the Java. The person who owns the business rule has to decide, and the decision should be written down before cutover. Run the original program and record its output for a set of inputs. Those recordings become the expected values. Choose inputs around every boundary: the threshold, zero, a negative balance, the largest value the field can hold, and values where rounding and truncation give different answers. The values below were worked out from the rules above to keep the post short. On a real project, record them by running the legacy program on the same inputs. balance,expected 500.00,1.04 800.00,1.67 10000.00,20.83 10000.01,29.17 0.00,0.00 -50.00,0.10 Some of these rows are there for a reason. 800.00 gives 1.67 rounded and 1.66 truncated, so it catches a missing rounding rule. 10000.00 is not above the threshold, so it stays on the lower rate. 10000.01 is the first balance above it and gives 29.17 29.16 if truncated . The test reads that file with JUnit 5: class MonthlyInterestParityTest { @ParameterizedTest @CsvFileSource resources = "/golden/interest.csv", numLinesToSkip = 1 void matchesLegacyOutput String balance, String expected { BigDecimal actual = InterestCalculator.monthlyInterest new BigDecimal balance ; assertEquals new BigDecimal expected , actual ; } } assertEquals on a BigDecimal compares the scale as well as the value, which is what you want here, because 0.1 and 0.10 are different in a fixed-width record. Run it against the first Java version and one row fails: -50.00 expects 0.10 and gets -0.10. After the business owner confirms the old behavior has to stay, the fix is small: public static BigDecimal monthlyInterest BigDecimal signedBalance { // Legacy behavior, kept on purpose: the sign is dropped. BigDecimal balance = signedBalance.abs ; BigDecimal rate = balance.compareTo new BigDecimal "10000" 0 ? new BigDecimal "0.035" : new BigDecimal "0.025" ; return balance.multiply rate .divide BigDecimal.valueOf 12 , 2, RoundingMode.HALF UP ; } The sign rule is one case. These are others worth a golden row: AI is useful for most of this work. It drafts the Java, lists boundary inputs by reading the code, and writes the test harness. It cannot supply the expected values. A test generated from the converted code will happily expect -0.10. Google's published account of an AI-assisted migration points the same way. About 80% of the code changes in it were fully AI-authored, and engineers still reviewed each change before it landed. The model handled volume, while people and tests handled correctness. The paper is How is Google using AI for internal code migrations? https://arxiv.org/abs/2501.06972 . We wrote up the wider process, from code analysis through a cutover readiness checklist, in our guide to legacy application modernization with AI https://www.xccelera.ai/blogs/legacy-application-modernization-ai . The parity test above is the validate stage of that process. If a conversion has surprised you with a quirk like these, add it in the comments. I would like to collect more of them.