AI refactor suggestions are hypotheses. Not facts. A free coding model rewrites your messy legacy function. The diff looks clean. CI stays green. Then a customer hits an edge case you forgot.
This article shows a small workflow. Characterize legacy behavior first. Let the model propose a refactor. Run the same tests against both versions. The verdict: safe or not safe.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
Legacy code has no spec. The only reliable spec is current behavior. Even bugs are behavior. If your refactor changes a bug, you need to know.
A characterization test records inputs and outputs. It does not judge right or wrong. It freezes the current contract. After freezing, every difference becomes visible.
Pick one messy function. I used a shipping calculator. Nested conditionals, magic numbers, zero tests.
Write a probe script. Call the function with realistic cases. Save outputs as JSON.
import json
from legacy import calculate_shipping
cases = [
{'items': [{'weight': 2.0, 'qty': 3}], 'region': 'US'},
{'items': [{'weight': 0.5, 'qty': 10}], 'region': 'EU'},
{'items': [{'weight': 0.2, 'qty': 1}], 'region': 'US'},
{'items': [{'weight': 5.0, 'qty': 2}], 'region': 'JP'},
]
for c in cases:
result = calculate_shipping(c['items'], c['region'])
print(json.dumps({'input': c, 'output': result}))
Save output to captured.json
. That becomes ground truth.
MonkeyCode's free model access lets me prompt from the CLI. I gave the model one strict instruction: keep behavior identical.
Refactor calculate_shipping into smaller functions.
Do NOT change edge cases. Do NOT change rounding.
Extract private helpers only.
The model returned a diff. It split the function into three helpers. The logic looked clean. Clean looks are not evidence.
Save the refactored version as shipping_refactored.py
. Leave the original untouched.
Load captured.json
. Write a characterization test that runs against either version. The assertion is simple: same input, same output.
import json
import pytest
from legacy import calculate_shipping
from shipping_refactored import calculate_shipping as refactored
with open('captured.json') as f:
cases = json.load(f)
@pytest.mark.parametrize('case', cases, ids=range(len(cases)))
def test_characterization(case):
expected = case['output']
assert calculate_shipping(case['input']['items'], case['input']['region']) == expected
Run the test on the original. It passes. Of course it does. It measured the original.
Point the same test at the refactored version. Now the diff matters.
Parametrized tests only compare to captured values. A differential test compares old and new directly. It finds divergence.
@pytest.mark.parametrize('case', cases)
def test_differential(case):
args = case['input']
old = calculate_shipping(args['items'], args['region'])
new = refactored(args['items'], args['region'])
assert old == new, f'mismatch for {case}'
My first run showed one mismatch. A single item under 0.3 kg in the US region. The model had changed weight <= 0.5
to weight < 0.5
. One byte flipped a shipping charge from $0 to $4.99.
The test caught it. The model's diff looked perfect. The evidence said otherwise.
Recorded cases cover what you saw. They do not cover what you missed. Property-based testing generates new inputs and searches for contradictions.
from hypothesis import given, strategies as st
Item = st.fixed_dictionaries({
'weight': st.floats(min_value=0.01, max_value=50.0),
'qty': st.integers(min_value=1, max_value=20),
})
@given(items=st.lists(Item, min_size=1, max_size=5),
region=st.sampled_from(['US', 'EU', 'JP']))
def test_property_differential(items, region):
old = calculate_shipping(items, region)
new = refactored(items, region)
assert old == new
I ran 1000 generated cases. The differential test found 17 failures. All sat near the same weight boundary. The model introduced a systematic off-by-one error.
Property tests do not prove equivalence. They prove divergence under search. That is enough to reject a refactor.
I used MonkeyCode's free server option to run the harness. It kept the experiment isolated from my local environment. No CI machine needed for a 20-test suite.
Do not treat the free server as a production runner. It is scratch space. Free tiers change. Quotas are not guarantees.
Characterization tests freeze behavior on collected samples. Missing samples mean missing boundaries. Add property-based guessing. Review the diff by hand.
Tests ignore performance. Identical values can still be 10x slower. Add a benchmark if latency matters.
Exception semantics matter. Two functions can return the same values but raise different exceptions. My tests did not cover exception paths. Yours should.
Skip one-off scripts. Characterization adds ceremony you do not need.
Skip fresh code with a real spec. Write behavior tests from the spec.
Skip it if you cannot review the model's diff. The workflow only finds differences. It does not judge which behavior is correct. You still have to think.
The model made refactoring cheap. Tests made it safe. AI reviews your code. Nobody tested the reviewer. Characterization tests are how you do it.
Capture behavior. Write tests. Run both versions. Let the diff prove itself.
The free model gives you speed. The free server gives you isolation. The tests give you the truth. Use all three.