{"slug": "building-an-ai-forensic-investigator-for-vehicle-failures", "title": "Building an AI Forensic Investigator for Vehicle Failures", "summary": "A developer built FaultTrace, an autonomous vehicle-forensics agent for the TrueForge Agent Harness Hackathon, designed to investigate vehicle failures by gathering evidence, testing hypotheses, and running Bayesian analysis. The agent stops for human approval before physical actions, and the project aims to generalize to other safety-sensitive systems.", "body_md": "*Built for the TrueForge Agent Harness Hackathon (Aug 24–30, 2026).*\n\n\"Something expensive broke. Figure out why and prove it. Ask a human before you touch anything.\"\n\nThat was basically the idea behind FaultTrace.\n\nI wanted to build something that felt more like an actual investigation than another chatbot with a few tools attached. The result is an autonomous vehicle-forensics agent that can gather evidence, test competing explanations, run actual analysis, and stop when it reaches a physical-world action that needs a human.\n\nAnd honestly, the interesting part wasn't getting the first version working.\n\nIt was getting the whole thing to **keep working reliably**.\n\nThis is how I built it, and what went wrong along the way.\n\nA car throws:\n\n```\nP0171 — System Too Lean\n```\n\nA chatbot can explain what P0171 means in a few seconds.\n\nBut that's not really the hard part.\n\nA technician still has to figure out *why* the car thinks it's running lean. It could be a vacuum leak, a dirty MAF sensor, a fuel-delivery problem, or even an O2 sensor that's giving misleading information.\n\nSo the real problem isn't:\n\n\"What does P0171 mean?\"\n\nIt's:\n\n\"Which of several possible causes actually explains the evidence?\"\n\nThat's an investigation.\n\nThat distinction is what led me to build **FaultTrace**.\n\nGiven a vehicle failure event, FaultTrace:\n\nFor the hackathon, I kept the scope deliberately concrete: **vehicle diagnostic forensics**.\n\nThe hero scenario is a cracked brake-booster vacuum hose on a 2003 Honda Accord, resulting in `P0171`\n\n+ `P0300`\n\n.\n\nIt's a small enough problem to demonstrate end-to-end, but complicated enough to make the agent actually investigate rather than just look up a DTC.\n\nThe vehicle domain is the implemented MVP. The underlying investigation pattern is intended to generalize to other safety-sensitive physical systems later.\n\nHere's the high-level architecture:\n\n``` php\nflowchart TB\n\n    U[\"User / Technician\"] --> AG\n\n    subgraph TF[\"TrueForge Harness\"]\n\n        direction TB\n\n        AG[\"Investigator Agent<br/>(faulttrace-investigator)\"]\n\n        SUB[\"Dynamic Subagents<br/>(per-hypothesis fan-out)\"]\n\n        SBX[\"Harness sandbox<br/>(optional agent-generated checks)\"]\n\n        RANK[\"Bayesian ranking<br/>prior × likelihood → posterior\"]\n\n        SES[\"Persistent session\"]\n\n    end\n\n    subgraph MCP[\"faulttrace-vehicle MCP server\"]\n\n        R1[\"get_dtcs · get_freeze_frame\"]\n\n        R2[\"get_sensor_log · get_compact_telemetry\"]\n\n        R3[\"lookup_dtc_knowledge · get_vehicle_info\"]\n\n        RA[\"run_analysis\"]\n\n        G2[\"request_measurement — Tier 2\"]\n\n        G3[\"clear_codes · order_part — Tier 3\"]\n\n    end\n\n    AG --> R1\n    AG --> R2\n    AG --> R3\n    AG --> SES\n\n    AG -- \"hypothesis fan-out\" --> SUB\n    SUB -- \"supporting / contradictory evidence\" --> AG\n\n    AG -- \"run analysis\" --> RA\n    RA --> FIXED[\"fixed analyze.py (server-side, deterministic)\"]\n    FIXED --> RANK\n    RANK --> AG\n\n    SUB -- \"optional custom checks\" --> SBX\n\n    AG -- \"propose physical action\" --> AP[\"Human approval gate\"]\n    AP -- \"approved → invoke\" --> G2\n    AP -- \"approved → invoke\" --> G3\n    AP -- \"rejected → cancel\" --> XL[\"no tool call\"]\n```\n\nThe important thing here isn't the number of boxes.\n\nIt's the loop.\n\n```\nFailure event\n(DTC + freeze-frame + sensor conditions)\n        ↓\nObserve\n(read-only evidence via MCP)\n        ↓\nForm competing root-cause hypotheses\n(each with a predicted signature)\n        ↓\nRun computed analysis\n(real computation, not LLM text math)\n        ↓\nEvaluate supporting vs contradictory evidence\n        ↓\nBayesian update\nprior × likelihood → posterior differential\n        ↓\nIdentify remaining uncertainty\n        ↓\nChoose the next diagnostic\n(expected information gain + cost)\n        ↓\nSTOP for human approval\nbefore Tier 2 / Tier 3 actions\n        ↓\nNew evidence arrives\n        ↓\nContinue investigation\n        ↓\nDefensible root-cause conclusion\n```\n\nThat is the part I wanted to get right.\n\nThe model isn't supposed to just produce a diagnosis and call it a day. It needs to figure out what it knows, what it doesn't know, and what it should do next.\n\nThis was one of the design questions I kept coming back to.\n\nIf I could replace the whole system with:\n\n\"Paste your DTC into ChatGPT\"\n\nthen I hadn't really built an agent.\n\nFaultTrace needs to:\n\nSo the distinction is pretty simple:\n\n**A chatbot gives you an answer. FaultTrace runs an investigation.**\n\nThis was important to me.\n\nI didn't want to build a normal chatbot and then bolt TrueForge onto it just so I could say I used the sponsor's technology.\n\nThe harness is actually doing a lot of the work.\n\nFaultTrace runs as a TrueForge agent, defined by a manifest containing the model, instructions, and MCP configuration.\n\nThe agent talks to a real vehicle MCP server over HTTP.\n\nThe tools aren't simulated function descriptions sitting inside the prompt. The agent actually reaches the server and gets data back.\n\nFaultTrace can fan out the investigation by hypothesis.\n\nFor example:\n\n```\n                    FaultTrace\n                        │\n              Bayesian differential\n                        │\n          ┌─────────────┼─────────────┐\n          ↓             ↓             ↓\n     Vacuum leak     Misfire /      Sensor\n     investigator    ignition     plausibility\n                     investigator   investigator\n```\n\nEach subagent gets its own thread and sandbox and reports back supporting and contradictory evidence.\n\nThe model can propose analysis, but the important computation happens in code.\n\nThe fixed `analyze.py`\n\nlibrary performs the deterministic diagnostic calculations, including the Bayesian differential and expected information gain.\n\nAn investigation isn't just one request/response.\n\nThe TrueForge session can pause for an approval, reconnect, and continue the same investigation. In the demo, the investigation actually resumes mid-flow.\n\nThis is probably my favorite part.\n\nIf the agent decides it needs a physical measurement or wants to clear codes/order a part, the harness pauses the action and puts the decision in front of a human.\n\nNothing executes until it's approved.\n\nThe vehicle MCP server currently exposes 13 tools across the investigation and safety workflow.\n\nSome of the important ones are:\n\n| Tool | What it does |\n|---|---|\n`list_vehicles` |\nDiscover available vehicles |\n`get_vehicle_info` |\nRetrieve vehicle metadata |\n`get_dtcs` |\nRetrieve diagnostic trouble codes |\n`get_freeze_frame` |\nRetrieve the fault-state snapshot |\n`get_pid_list` |\nDiscover available telemetry PIDs |\n`get_compact_telemetry` |\nRetrieve bounded current telemetry |\n`get_sensor_log` |\nRetrieve historical sensor telemetry |\n`lookup_dtc_knowledge` |\nRetrieve scenario-specific diagnostic knowledge |\n`run_analysis` |\nRun the deterministic computed differential |\n`request_measurement` |\nRequest an additional diagnostic measurement |\n`clear_codes` |\nClear diagnostic codes |\n`order_part` |\nRequest a replacement part |\n\nThe last two are intentionally gated.\n\nThat distinction matters because the agent can be autonomous without being allowed to do whatever it wants.\n\nOne thing I didn't want was a model looking at a bunch of numbers and then casually saying:\n\n\"I'm 91% confident this is a vacuum leak.\"\n\nThat's not very convincing.\n\nSo FaultTrace has a deterministic analysis layer.\n\nThe model supplies the investigation context, but the actual analysis code calculates the differential.\n\nConceptually:\n\n```\nP(hypothesis | evidence)\n        ∝\nP(evidence | hypothesis) × P(hypothesis)\n```\n\nThe analyzer takes scenario-specific priors and telemetry-derived likelihoods, performs the Bayesian update, and returns a normalized posterior ranking.\n\nIt also calculates **expected information gain** for the available diagnostic tests.\n\nThat gives the agent something more useful than \"try another test\":\n\n```\nCurrent uncertainty\n        ↓\nEvaluate available tests\n        ↓\nCalculate expected information gain\n        ↓\nConsider test cost\n        ↓\nSelect the most useful next test\n```\n\nThe result is reproducible because the computation is deterministic and seeded.\n\nThat separation is important to the architecture:\n\nThe LLM decides what to investigate. The analysis code calculates the numbers.\n\nEvery hypothesis isn't just a label.\n\nIt comes with a prediction:\n\n\"If this hypothesis is actually true, what should I see in the data?\"\n\nFor example, a vacuum leak should produce a different pattern from a MAF fault or an ignition problem.\n\nThat gives the agent something concrete to test.\n\nInstead of:\n\n```\nHypothesis: Vacuum leak\n```\n\nwe have:\n\n```\nHypothesis:\nVacuum leak\n\nPredicted signature:\n- elevated fuel trims\n- stronger effect at idle\n- abnormal airflow relationship\n- correlation with misfire behavior\n```\n\nThe analysis then checks the telemetry against those signatures.\n\nThis also makes the final explanation much more useful because we can show both:\n\n**why a hypothesis fits**\n\nand\n\n**why another hypothesis doesn't.**\n\nI wanted the agent to actively look for evidence *against* its own hypotheses too.\n\nSo a final differential isn't just:\n\n```\nVacuum leak\n✓ Fuel trims support this\n✓ MAF relationship supports this\n```\n\nIt should look more like:\n\n```\nVACUUM LEAK\n\nSupporting evidence\n✓ Positive fuel trim at idle\n✓ Airflow relationship matches predicted behavior\n✓ Misfire pattern is consistent\n\nContradictory evidence\n⚠ Idle instability is weaker than expected\n\nMissing evidence\n? Fuel pressure under load\n```\n\nAnd for another hypothesis:\n\n```\nMAF FAULT\n\nSupporting evidence\n✓ Some airflow irregularity\n\nContradictory evidence\n✕ Fuel-trim behavior is more consistent with\n  unmetered air\n\nMissing evidence\n? Independent airflow measurement\n```\n\nThat \"why not?\" reasoning is a big part of making the result feel forensic rather than classificatory.\n\nThis is another part I really wanted to avoid making into a hard-coded flowchart.\n\nSuppose the agent has narrowed the problem down to two plausible causes:\n\n```\n1. Vacuum leak\n2. Weak fuel delivery\n```\n\nThe agent shouldn't just say:\n\n\"More data is needed.\"\n\nIt should ask:\n\n\"What measurement would actually separate these two explanations?\"\n\nThat's where expected information gain comes in.\n\nThe analysis evaluates the available tests and returns something like:\n\n```\nRecommended test:\nfuel_pressure_under_load\n\nExpected information gain:\nX.XX bits\n\nCost:\nLow\n\nReason:\nThe result is expected to distinguish the two leading hypotheses.\n```\n\nAnd then the agent stops.\n\nIt doesn't execute the physical test automatically.\n\nThis is deliberately simple.\n\nThe agent can do these autonomously:\n\nHuman approval required:\n\nHuman approval required:\n\nThe rule is:\n\nInvestigate freely. Act carefully.\n\nThere are two layers of protection here. TrueForge provides the actual approval gate, and the MCP server independently refuses gated calls that don't contain the required approval.\n\nSo even if something goes wrong in the agent layer, the server has another line of defense.\n\nThis was probably the biggest practical lesson I got from the build.\n\nI initially assumed that if a model was good enough at reasoning, I could just swap it into the same agent and everything would behave roughly the same.\n\nNope.\n\nThe behavior around tools and subagents can be dramatically different.\n\nI initially used:\n\n```\nopenrouter/z-ai-glm-5.3-flash\n```\n\nbecause it was cheap and fast.\n\nIt did something interesting: it actually spawned real `create_sub_agent`\n\nthreads.\n\nI could see three separate `thread.created`\n\nevents with different thread IDs.\n\nSo, great?\n\nNot quite.\n\nThe subagents then hit a wall because the harness's local sandbox is macOS/Linux only, while I was running the project on Windows. They ended up trying to execute their Python in a cloud sandbox and timing out.\n\nSo I had:\n\n```\nbeautiful fan-out\n       ↓\nthree real subagents\n       ↓\nsandbox timeout\n       ↓\nzero useful results\n```\n\nThat wasn't exactly the demo I wanted.\n\nI switched the primary model to Gemini 2.5 Flash.\n\nNow I had the opposite problem.\n\nGemini would talk about subagents without actually creating them.\n\nI'd see things like:\n\n```\nSub-agent: investigating vacuum leak...\nSub-agent: checking ignition...\n```\n\nbut there were no real `thread.created`\n\nevents.\n\nIt was essentially role-playing the delegation.\n\nThat's a surprisingly important distinction when you're building an agent system.\n\nI eventually had to make delegation explicit and give the subagents very concrete instructions.\n\nThings like:\n\nOne small example caused a ridiculous amount of pain:\n\n```\nrpm\n```\n\nwas the actual PID.\n\nThe subagent would sometimes assume:\n\n```\nengine_rpm\n```\n\nand the whole analysis would fail.\n\nAnother lesson: when running Python through the shell, I had much better results writing it to a file and executing the file than trying to pipe complex Python through `echo`\n\n.\n\nSometimes the \"AI problem\" is just shell quoting.\n\nThe other thing that bit me was the approval flow.\n\nI needed the model to produce the actual gated tool call so TrueForge could surface the approval UI.\n\nInstead, Gemini would sometimes do this:\n\n```\nThe next step would be to request\na fuel-pressure measurement.\n\nWould you like me to proceed?\n```\n\nLooks reasonable to a human.\n\nBut it completely bypasses the actual approval mechanism.\n\nThere was no approval button because there was no tool call.\n\nSo I added an explicit rule:\n\nIf the agent has decided on a gated action, the turn must end with the gated tool call rather than a prose question.\n\nThat was one of those tiny prompt changes that made a huge difference.\n\nI also ran Qodo code review on the project's pull requests.\n\nIt wasn't just a checkbox for the hackathon. A few findings actually changed the implementation.\n\nQodo flagged that `run_analysis`\n\ncould be treated as one source of posterior probabilities while the orchestrator had already derived another view from subagent evidence.\n\nThat was a dangerous design.\n\nI changed the architecture so:\n\n`run_analysis`\n\nis the single authoritative source of posterior probabilities.\n\nSubagents contribute evidence and narrative analysis, but they don't modify the posterior.\n\nQodo also caught a mismatch where a tool description exposed scenario information that the response itself intentionally kept hidden.\n\nI aligned the MCP contract with the actual allow-listed response.\n\nThis one was particularly important.\n\nA subagent recipe had accidentally hard-coded Scenario A's VIN.\n\nThat meant Scenario B or C could have caused the subagent to investigate the wrong vehicle.\n\nQodo caught it.\n\nThe fix was simple: every subagent now receives and reuses the VIN from the actual failure event.\n\nQodo also suggested converting a smoke test to CommonJS.\n\nI pushed back on that one.\n\nThe repository uses ESM with `\"type\": \"module\"`\n\n, and the actual tests follow that convention. So I dismissed the suggestion with the reasoning recorded in the PR.\n\nThat's a useful lesson too:\n\nCode review tools are extremely useful, but they aren't infallible. You still need to understand the code you're reviewing.\n\nIt's the fact that FaultTrace **knows when to stop**.\n\nThe agent can investigate a problem for as long as the work is read-only.\n\nBut eventually it might conclude:\n\n```\nThe next useful step is a physical measurement.\n```\n\nAt that point:\n\n```\n┌─────────────────────────────────────────────┐\n│         HUMAN APPROVAL REQUIRED             │\n│                                             │\n│  Request fuel-pressure measurement?         │\n│                                             │\n│       [ Approve ]       [ Reject ]          │\n└─────────────────────────────────────────────┘\n```\n\nThe agent waits.\n\nIf the human rejects it, nothing happens.\n\nIf the human approves it, the action executes and the same investigation can continue with the new evidence.\n\nThat's the boundary I wanted:\n\n**Autonomous investigation. Human-controlled action.**\n\nA hackathon project should be honest about this.\n\nScenarios B (dirty MAF) and C (stuck O2) exist for regression coverage.\n\nScenario A (vacuum leak) is the hero.\n\nThe broader architecture could eventually apply to industrial machinery, robotics, energy systems, and other physical systems, but those are **future applications**, not claims about what this MVP already implements.\n\nThe basic setup is:\n\n```\nnpm install\nnpm start\n\ncd mcp-server\nnpm install\nnpm run start:http\n```\n\nThe TrueForge harness runs on:\n\n```\nhttp://localhost:8790\n```\n\nThe repository contains the remaining configuration and environment setup required to run the demo.\n\nI've kept the main demo focused on one investigation rather than trying to show every feature.\n\nThe flow is:\n\n```\nDTC event\n   ↓\nMCP evidence collection\n   ↓\nCompeting hypotheses\n   ↓\nDynamic subagents\n   ↓\nSandbox analysis\n   ↓\nBayesian differential\n   ↓\nExpected information gain\n   ↓\nRecommended diagnostic\n   ↓\nHuman approval\n   ↓\nInvestigation resumes\n   ↓\nRoot-cause report\n```\n\nIf I had to summarize the whole project in one sentence:\n\nThe hard part of an agent isn't getting a model to reason — it's making the entire loop reliable.\n\nGetting an LLM to say:\n\n\"I think this is a vacuum leak\"\n\nis easy.\n\nGetting it to:\n\nis a very different problem.\n\nThat's what made FaultTrace interesting to build.\n\nAnd that's probably the biggest thing I took away from the hackathon:\n\n**The model is only one component of an agent. The orchestration around it is where the real engineering starts.**\n\n*Built for the TrueForge Agent Harness Hackathon. AI coding assistants were used during development, and the code was reviewed throughout the project, including with Qodo.*", "url": "https://wpnews.pro/news/building-an-ai-forensic-investigator-for-vehicle-failures", "canonical_source": "https://dev.to/harshuldwivedi/building-an-ai-forensic-investigator-for-vehicle-failures-cjp", "published_at": "2026-08-29 10:25:43+00:00", "updated_at": "2026-08-29 10:49:27.001113+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-agents", "ai-tools", "ai-research"], "entities": ["FaultTrace", "TrueForge Agent Harness", "Honda Accord"], "alternates": {"html": "https://wpnews.pro/news/building-an-ai-forensic-investigator-for-vehicle-failures", "markdown": "https://wpnews.pro/news/building-an-ai-forensic-investigator-for-vehicle-failures.md", "text": "https://wpnews.pro/news/building-an-ai-forensic-investigator-for-vehicle-failures.txt", "jsonld": "https://wpnews.pro/news/building-an-ai-forensic-investigator-for-vehicle-failures.jsonld"}}