{"slug": "show-hn-vinv-ties-every-runtime-trace-to-code-segment-prevents-reward-hacking", "title": "Show HN: Vinv-Ties every runtime trace to code segment, prevents reward hacking", "summary": "Vinv, an open-source tool released under Apache License 2.0, records runtime traces of Python backends and independently verifies coding agents' fixes by replaying starts, live ports, and acceptance tests the agent never sees, preventing reward hacking. The tool, which runs entirely on the user's machine with no accounts or telemetry, ties every request to the exact code line that served it and allows one-click reversion of all changes from an episode. Vinv currently supports Python backends with plans for TypeScript and Go next, and works with agents including Claude Code, Codex CLI, and Cursor over MCP.", "body_md": "**Vinv records a real run of your Python backend and checks your coding agent's \"done\" — replayed start, live port, acceptance tests the agent never sees.**\n\n[ vinv.ai](https://vinv.ai) ·\n\n**Free & open source · Apache License 2.0**· Python first · TS & Go next\n\n**No account. No API keys. No telemetry. Everything runs on your machine.**\n\nYour agent has never watched your code run — it edits the wrong handler, invents return shapes, then grades its own homework: *\"Done — all tests pass\"* while the server won't even start.\n\nVinv records the run, ties every request to the exact line that served it, hands that evidence to your agent — and when the agent claims a fix, **Vinv checks it independently**: replayed start, live port, acceptance tests the agent never sees. One click reverts everything an episode touched — untracked files included.\n\n`// the loop, for real — installed, discovered, traced, broken, fixed, verified — on this very repo`\n\nEight real captures of vinv running on its own repository: ① ./install.sh — one command, everything builds from source · ② Auto-Pilot discovers all 4 services (HTTP sidecar, dev server, two stdio MCP servers) and sets them up unaided · ③ the Code Graph — 4,036 symbols, live runtime overlay · ④ real traced traffic with per-endpoint health reports · ⑤ served to your agent over MCP — semantic search and fault-ranked suspects, real round-trips · ⑥ a real production bug (BrokenPipeError in live runs) detected and auto-dispatched — \"Fix sent, agent working\" · ⑦ input/output probes re-verify the fix against golden baselines · ⑧ everything green, zero clicks. Runtime tracing is Python-first, so trace/flamegraph frames show the Python service; the JS services get discovery, verified starts, and stdio/port probes.\n\n**Watches your code run**— a zero-edit Python tracer records your code's calls as they happen: timing, memory, arguments, returns, errors, and the request that triggered each one.**Maps your codebase**— a persistent semantic index + interactive Code Graph, built with a** local embedding model**([CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed), MIT, 137M params — downloaded once, runs on your GPU/CPU), updated incrementally on save.**Answers with evidence**— Ask Vinv cites the exact symbols behind every claim and marks runtime facts that went stale when the code changed.** Closes the loop**— hand an issue to your own coding agent; Vinv composes the evidence pack, dispatches it, and** verifies the result itself**.** Works with the agent you already use**— Claude Code, Codex CLI, Cursor (CLI & chat), Gemini CLI, Copilot Chat, Windsurf Cascade — over MCP and headless dispatch.**Your agent is also Vinv's only LLM**: every analysis step routes through the coding-agent CLI you already pay for. No provider keys, no model picker.\n\nEverything runs from this repository — no marketplace, no downloads, no accounts:\n\n```\ngit clone https://github.com/VinvAI/VinvAI ~/.vinv/engines\ncd ~/.vinv/engines && ./install.sh\n```\n\n`install.sh`\n\nbuilds the Python engines ([uv](https://docs.astral.sh/uv/)), the Rust index ([rustup](https://rustup.rs)), packages the editor extension (node), and installs it into every editor CLI it finds (VS Code, Cursor, Windsurf, VSCodium, Trae). Then **open your repo** — Auto-Pilot takes it from there, zero clicks: the embedding model downloads (~500 MB once) and auto-tunes to your hardware, the index builds, every service is discovered, set up through your coding agent, started under tracing, and verified serving. Problems found along the way are dispatched to your agent and re-verified automatically.\n\nHonest scope:Python backends first — other stacks get the index, graph, and QnA, but no runtime evidence (TS & Go next) · v0.0.x · works with six agents over MCP and headless dispatch · Vinv doesnotdo cost metering, security scanning of generated code, or production deployment.\n\nVinv's release gate is Vinv: the loop above ran against this repository, and these numbers come from that run —\n\n**Zero-click Auto-Pilot**— discover → set up every service via your agent → start under tracing → build insights → probe → fix → re-verify, until green or budget. All 4 of this repo's services (an HTTP sidecar, a dev server, two stdio MCP servers) came up unaided; run-to-completion CLIs are recognized and skipped, never retry-looped.**Measured, not assumed**— the embedder benchmarks your actual hardware once and persists the winner (`~/.vinv/embedder_tuned.json`\n\n). On an M-series Mac that verdict was CPU at 15.3 texts/s vs 0.2 on the GPU — a 70× trap the old \"prefer GPU\" default walked straight into.**Crash-proof by proof**— we killed the indexer mid-run (resumed at row 448 exactly), killed the embedder mid-build (indexer rode it out), and killed a traced service with SIGKILL (supervisor exits in 0s, trace intact).**Finds what nobody filed**— running its own analyzers on its own traces surfaced five unfiled issues, including 83% duplicate compute (now cached) and an 80%-noise LLM adjudication queue (now culled).**Optimizes without regressing**— every endpoint gets golden input/output baselines; a \"faster\" change that drops a response field is flagged** degraded**. Retrieval config changes only on off-policy-evaluation wins (doubly-robust, bootstrap CI, 5% canary, auto-rollback).** Search that names the line**— hybrid local retrieval: file hit@10 0.90, symbol MRR 0.51, 81ms median — and every answer is a real symbol with its`def`\n\nbody and line numbers, joined to what actually ran.\n\nFour artefacts, one join. Vinv indexes **the code** (every symbol and call edge) and generates — from your own run — **the traces** (one span per call), **the logs** (timestamped call events from that same trace), and **the metrics** (per-symbol latency & memory), then ties all four to the *exact function that handled each request*. The artefacts are commodities; **the join is not**. One graph, served to your agent, instead of four tools it has to guess between.\n\n**Trace**— run your Python service under the bundled tracer: no SDK, no code changes, no dashboards. Calls are recorded with real values (summarized and redacted — see[Data & privacy](#-data--privacy)).**Index**— every function and class is embedded locally and ranked into a semantic index with a call/inheritance graph; incremental updates on save keep it current. Embeddings come from a local sidecar (`vinv-embedder`\n\n) that auto-sizes batches to your device (CUDA → Apple Silicon → CPU) and resumes cleanly after a crash.**Serve**— two MCP servers hand the context graph to your agent: semantic code search, fault-localization rankings, observed values, call slices, coverage, blast radius.**Verify**— after a dispatched fix, Vinv replays the recorded start command in a fresh process, requires the port to actually serve, and runs acceptance tests generated*before*the fix that the agent never sees. Failures feed the next attempt; inconclusive evidence escalates to your judgment card.**Learn**— every decision and outcome is propensity-logged; retrieval and context-pack composition update only when off-policy evaluation shows a real win over your baseline (local, never uploaded).\n\n| Directory | What it is |\n|---|---|\n`extension/` |\n\n`index/`\n\n`embedder/`\n\n`tracelens/`\n\n`identification/`\n\n`handbook/`\n\n`.vinv/vinv.md`\n\n) via your harness`bringup/`\n\n`goal/`\n\n[,](/VinvAI/VinvAI/blob/main/core)`core/`\n\n`contracts/`\n\n`tests/e2e/`\n\nPython engines are a single [uv](https://docs.astral.sh/uv/) workspace — `uv sync`\n\nat the repo root sets up everything.\n\n** vinv-index** — the codebase and the session:\n\n| Tool | Returns | Ask it when |\n|---|---|---|\n`vinv_query` |\nRanked symbols with paths + a decision id | Any by-meaning code search — before grep |\n`vinv_feedback` |\nack | Once after acting on results (reward −1..1) — trains retrieval |\n`vinv_session` |\n`trajectory` · `status` · `issues` · `hotspots` · `memory_trends` · `cache_candidates` ; actions `fix` · `run_sweep` · `set_goal` · `set_budget` |\nRead Vinv's live state — or queue fixes/sweeps — from chat |\n\n** vinv-runtime** — the captured runs (read-only, provenance-stamped):\n\n| Tool | Returns | Ask it when |\n|---|---|---|\n`rank_suspects` |\nSymbols ranked by fault-localization score over pass/fail requests, real error messages attached | First, on any failure — before reading source |\n`values_of` |\nObserved argument/return types, null-rates, ranges, top values | \"What does this function actually receive?\" |\n`slice` |\nThe observed caller chain from request root to the symbol, values at each frame | \"How did this bad value get here?\" |\n`coverage_of` |\nWhat ran, how often, ok/error, timing | \"Did my change even execute?\" |\n`callers_of` / `blast_radius` / `why_did_this_run` |\nObserved callers · transitive impact · entry-point paths | \"Who calls this / what breaks if it's wrong / why did it run?\" |\n\n**Everything lives on your machine**: per-repo state in`.vinv/`\n\n(auto-gitignored), per-machine state in`~/.vinv/`\n\n. Learning ledgers are local and never uploaded.**No telemetry. None.** The extension makes no network calls of its own; the only download is the embedding model (Hugging Face, once). Everything else builds from this repository.- Traces store bounded\n**summaries**, not raw values: strings become length + hash prefix; parameters with sensitive names (`password`\n\n,`token`\n\n,`api_key`\n\n,`secret`\n\n,`authorization`\n\n,`credential`\n\n) are flagged redacted and never have contents captured; everything is capped at 256 bytes by default. - Your code never leaves your machine — the only LLM Vinv talks to is the coding-agent CLI\n**you** configured, through its own auth.\n\nSee [CONTRIBUTING.md](/VinvAI/VinvAI/blob/main/CONTRIBUTING.md). Quick version: `uv sync`\n\nfor the Python engines, `cargo build`\n\nin `index/`\n\n, `npm install && npm run check`\n\nin `extension/`\n\n, and `python tests/e2e/planted_bug_golden/run.py`\n\nmust stay green. Good first issues are labeled.\n\n[Apache License 2.0](/VinvAI/VinvAI/blob/main/LICENSE) © 2026 VinvAI.\n\nPython first, TS & Go next ·\n\n**Your agent says it's done. Vinv says prove it.**", "url": "https://wpnews.pro/news/show-hn-vinv-ties-every-runtime-trace-to-code-segment-prevents-reward-hacking", "canonical_source": "https://github.com/VinvAI/VinvAI", "published_at": "2026-07-24 10:44:13+00:00", "updated_at": "2026-07-24 11:22:28.109800+00:00", "lang": "en", "topics": ["developer-tools", "ai-agents", "ai-safety", "artificial-intelligence"], "entities": ["Vinv", "Apache License 2.0", "Claude Code", "Codex CLI", "Cursor", "Gemini CLI", "Copilot Chat", "Windsurf Cascade"], "alternates": {"html": "https://wpnews.pro/news/show-hn-vinv-ties-every-runtime-trace-to-code-segment-prevents-reward-hacking", "markdown": "https://wpnews.pro/news/show-hn-vinv-ties-every-runtime-trace-to-code-segment-prevents-reward-hacking.md", "text": "https://wpnews.pro/news/show-hn-vinv-ties-every-runtime-trace-to-code-segment-prevents-reward-hacking.txt", "jsonld": "https://wpnews.pro/news/show-hn-vinv-ties-every-runtime-trace-to-code-segment-prevents-reward-hacking.jsonld"}}