{"slug": "show-hn-shadowdiff-shadow-testing-and-sql-diffing-for-ai-migrated-back-ends", "title": "Show HN: ShadowDiff – Shadow testing and SQL diffing for AI-migrated back ends", "summary": "A developer released ShadowDiff, a network proxy and analysis engine that mirrors live HTTP traffic to both a legacy backend and its AI-refactored replacement to isolate regressions before production deployment. ShadowDiff targets the roughly 5% of hidden edge cases that AI code migration engines miss, including serialization discrepancies such as Jackson omitting empty lists versus Go serializing null, timezone truncations, float precision drift, and database state regressions like missing audit writes and N+1 query loops. The tool integrates with Envoy Proxy and Istio service mesh via request_mirror_policies, filters non-deterministic noise by comparing twin baseline instances, and uses Google Gemini to generate Go and Jest tests from detected differences.", "body_md": "**Eliminating the 5% risk that stops enterprise CTOs from deploying AI-modernized backends to production.**\n\n🌐 **Interactive Live Cloud Sandbox**: [https://shadowdiff-demo.fly.dev](https://shadowdiff-demo.fly.dev/) (Self-driving dual-traffic simulator active)\n\nEnterprises are actively modernizing mission-critical legacy backends (Java 7/8 Spring monoliths, Python 2 Django, PHP 5.6 backends) into modern cloud-native services (Go, TypeScript/Node, FastAPI) using AI code migration engines.\n\nWhile AI generates ~95% of refactored code accurately, **the remaining 5% of hidden edge cases make production deployment an unacceptable risk**:\n\n- **Subtle Serialization Discrepancies** : Jackson omitting empty lists vs Go serializing`null` , datetime timezone truncations, float precision drift.\n- **Implicit Framework Quirks** : Legacy servlet filter quirks, undocumented fallback headers, HTTP 200 responses with embedded error payloads.\n- **Database State Regressions** : Silent differences in database transaction boundaries, missing audit writes, or N+1 query loop explosions.\n\n**ShadowDiff** is a high-performance network proxy and analysis engine that duplicates real staging or production HTTP traffic, sending identical payloads to both the legacy service and the AI-refactored service simultaneously.\n\n```\n                           [ Live Inbound Traffic ]\n                                      │\n                                      ▼\n                       ┌─────────────────────────────┐\n                       │    Envoy Gateway / Mesh     │\n                       │  (request_mirror_policies)  │\n                       └──────────────┬──────────────┘\n                                      │\n                  ┌───────────────────┴───────────────────┐\n                  ▼                                       ▼\n        ┌───────────────────┐                   ┌───────────────────┐\n        │  Legacy Service   │                   │  ShadowDiff Core  │\n        │     (Target)      │                   │  (Analysis Engine)│\n        └─────────┬─────────┘                   └─────────┬─────────┘\n                  │                                       │\n                  │ (Live Response to Client)             ▼\n                  │                             ┌───────────────────┐\n                  │                             │   AI-Refactored   │\n                  │                             │   Candidate (S)   │\n                  │                             └─────────┬─────────┘\n                  │                                       │\n                  └───────────────────┬───────────────────┘\n                                      ▼\n                       ┌─────────────────────────────┐\n                       │ 3-Way Diff & SQL Comparator │\n                       │ - P1 ⊕ P2 Noise Filter      │\n                       │ - SQL Mutation Interceptor  │\n                       │ - N+1 Query Loop Detector   │\n                       └──────────────┬──────────────┘\n                                      │\n                         (Pure Regressions Isolated)\n                                      ▼\n                       ┌─────────────────────────────┐\n                       │ Automated AI Test Generator │\n                       │ - Powered by Google Gemini  │\n                       │ - Synthesizes Go/Jest Tests │\n                       │ - Root Cause Code Fixes     │\n                       └─────────────────────────────┘\n```\n\nShadowDiff integrates directly with **Envoy Proxy** and **Istio Service Mesh** using `request_mirror_policies`. Traffic is cloned asynchronously at line rate in the network layer with **zero application latency penalty**.\n\nAutomatically filters out non-deterministic runtime noise (timestamps, UUIDs, session nonces) by evaluating twin baseline instances:\n\nCaptures and correlates SQL statements executed by both services for each HTTP trace:\n\n- \n**Missing Writes** : Flags when legacy wrote to`audit_logs` or ledger tables that modern code omitted.\n- \n**Transaction Leaks** : Warns if legacy used`BEGIN...COMMIT` and modern ran autocommit statements.\n- \n**N+1 Query Explosions** : Detects when modern ORMs loop and execute$\\ge 3\\times$ more queries than legacy.\n\nEvery isolated edge case is analyzed by **Google Gemini** (`gemini-2.0-flash`), which writes a turnkey integration test (Go `httptest` or Vitest) and diagnoses the code fix.\n\nEliminates manual triaging by directly integrating with the GitHub API:\n\n- **Automatic Branching** : Spins up`shadowdiff/fix-<trace_id>` from the repository's default branch.\n- **Test File Commitment** : Automatically commits the synthesized regression test directly to the repository (e.g.`tests/regression/test_drift_<trace_id>.go` ).\n- **Rich Pull Request Generation** : Opens a detailed PR with side-by-side JSON diff tables, SQL mutation waterfall analysis, N+1 query warnings, and local verification commands.\n- **1-Click Triage in Console** : Developers can inspect the incident in the web console and click**\"🚀 Open GitHub PR\"** to immediately open the PR on GitHub.\n\nA native deployment blocker that evaluates traffic quality during pre-release soak tests or integration pipelines:\n\n- \n**Automated Evaluation** : Assesses semantic parity, database mutations, and N+1 query loops over a specified evaluation window.\n- \n**Zero False Failure** : Honors Diffy noise cancellation ($\\mathcal{N} = P_1 \\oplus P_2$ ), only failing on genuine regressions.\n- \n**Pipeline Integration** : Returns standard exit codes (`0` for PASS,`1` for FAIL/violation,`2` for error) with high-impact ANSI colored terminal summaries and optional JSON format.\n\n```\n# Evaluate quality gate in your CI/CD pipeline:\nshadowdiff gate \\\n  -dashboard-url=http://localhost:8083 \\\n  -duration=15s \\\n  -min-parity=100.0 \\\n  -min-requests=50 \\\n  -max-drifts=0 \\\n  -allow-n-plus-one=false \\\n  -allow-db-drift=false\n```\n\nAllows safe shadow mirroring of mutating HTTP methods (`POST`, `PUT`, `DELETE`, `PATCH`) with **zero database pollution**:\n\n- **Transactional Savepoint Rollbacks** : Injects boundary headers to ensure shadow writes execute within an isolated savepoint and automatically roll back.\n- **Ephemeral Schema Sandboxing** : Dynamically creates disposable schemas per test run and drops them upon completion.\n- **Serverless CoW Branching** : Leverages Neon / Supabase serverless branching to clone database state instantly via API for mutating candidate runs.\n\nIncorporate ShadowDiff directly into your automated deployment pipeline (`.github/workflows/shadowdiff.yml`):\n\n```\nname: Dual-Traffic Shadow Gating\non: [pull_request, push]\n\njobs:\n  shadow-gate:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: actions/setup-go@v5\n        with:\n          go-version: '1.22'\n\n      - name: Build & Start Services\n        run: |\n          go build -o bin/shadowdiff ./cmd/shadowdiff/main.go\n          ./bin/legacy-service -port=:8081 &\n          ./bin/refactored-service -port=:8082 &\n          ./bin/shadowdiff -proxy-port=:8080 -primary-url=http://localhost:8081 -shadow-url=http://localhost:8082 -sandbox-mode=rollback &\n          sleep 2\n\n      - name: Replay Traffic\n        run: |\n          for i in {1..25}; do\n            curl -s \"http://localhost:8080/api/users?id=$i\" > /dev/null\n            curl -s -X POST \"http://localhost:8080/api/orders\" -d '{\"amount\":99.50}' > /dev/null\n          done\n\n      - name: Run ShadowDiff Quality Gate\n        run: |\n          ./bin/shadowdiff gate \\\n            -dashboard-url=http://localhost:8083 \\\n            -min-parity=100.0 \\\n            -min-requests=20 \\\n            -allow-n-plus-one=false \\\n            -allow-db-drift=false\n```\n\nLaunch the complete 5-container service mesh testbed (Envoy Gateway + ShadowDiff + Legacy Target + Twin Replica + AI Candidate):\n\n```\n# 1. Clone the repository\ngit clone https://github.com/vishalrajofficial/ShadowDiff.git\ncd ShadowDiff\n\n# 2. Launch the Envoy Service Mesh topology\ndocker compose -f docker-compose.envoy.yml up -d\n# Send test traffic to the Envoy Gateway (mirrored to ShadowDiff automatically):\ncurl http://localhost:10000/api/users/1\ncurl -X POST http://localhost:10000/api/orders\n```\n\nOpen the telemetry dashboard at **`http://localhost:8083`** to inspect the live 3-way diffs, SQL mutation timeline, and Gemini-generated tests!\n\nFor production Kubernetes clusters running Istio or native Envoy, deploy the included `EnvoyFilter` to mirror traffic dynamically:\n\n```\nkubectl apply -f deploy/k8s/envoy-shadow-sidecar.yaml\n```\n\nSee [docs/ARCHITECTURE.md](https://github.com/vishalrajofficial/ShadowDiff/blob/main/docs/ARCHITECTURE.md) for full architectural specifications.\n\nThis project is licensed under the [MIT License](https://github.com/vishalrajofficial/ShadowDiff/blob/main/LICENSE).", "url": "https://wpnews.pro/news/show-hn-shadowdiff-shadow-testing-and-sql-diffing-for-ai-migrated-back-ends", "canonical_source": "https://github.com/vishalrajofficial/ShadowDiff", "published_at": "2026-11-15 06:54:38+00:00", "updated_at": "2026-11-15 07:24:53.150550+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-agents", "ai-infrastructure"], "entities": ["ShadowDiff", "Envoy Proxy", "Istio", "Google Gemini", "Jackson", "Go", "Java", "Python"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-shadowdiff-shadow-testing-and-sql-diffing-for-ai-migrated-back-ends", "markdown": "https://wpnews.pro/news/show-hn-shadowdiff-shadow-testing-and-sql-diffing-for-ai-migrated-back-ends.md", "text": "https://wpnews.pro/news/show-hn-shadowdiff-shadow-testing-and-sql-diffing-for-ai-migrated-back-ends.txt", "jsonld": "https://wpnews.pro/news/show-hn-shadowdiff-shadow-testing-and-sql-diffing-for-ai-migrated-back-ends.jsonld"}}