{"slug": "don-t-start-with-rag-lessons-from-building-an-automotive-ai-pipeline", "title": "Don't Start With RAG: Lessons From Building an Automotive AI Pipeline", "summary": "Inspecly's engineering team built an automotive AI pipeline that prioritizes structured data and provenance over a RAG-first approach. The team normalizes inputs like OBD codes, photos, and voice transcriptions into a structured object before applying LLMs, using exact lookups for validated data and tool-using agents only when structured knowledge is missing. This design improves reliability and traceability of AI-generated diagnoses.", "body_md": "When building an AI product, it's tempting to start with the fashionable pieces.\n\nVector database.\n\nRAG.\n\nAgents.\n\nMultimodal models.\n\nThen connect everything to an LLM and hope the final prompt makes sense of it.\n\nWhile building the automotive AI pipeline behind Inspecly, we ended up taking almost the opposite approach.\n\nThe first question wasn't:\n\nWhich LLM should we use?\n\nIt was:\n\nWhat information do we actually have, how reliable is it, and which system should process it?\n\nThat distinction changed the architecture.\n\n**The input is messy by default**\n\nA driver rarely describes a vehicle problem like a mechanic.\n\nThey might say:\n\n\"My car makes a strange noise when I start it.\"\n\nBut a request can also contain:\n\nThese inputs do not have the same reliability.\n\nAn OBD code is structured information.\n\nA photo is visual evidence.\n\nA voice message represents what the driver observed.\n\nVehicle metadata may require exact lookup.\n\nTreating all of them as equivalent pieces of text would be a mistake.\n\nSo before asking an LLM to reason about the problem, we normalize the available evidence.\n\nConceptually:\n\n```\n{ \"vehicle\": { \"make\": \"...\", \"model\": \"...\", \"vin\": \"optional\" }, \"description\": \"The engine loses power when accelerating.\", \"voice_transcription\": null, \"obd_codes\": [\"...\"], \"images\": [...] }\n```\n\nThis object doesn't contain a diagnosis.\n\nIt describes what we actually know.\n\n**Why not send everything to one multimodal LLM?**\n\nYou absolutely could.\n\nDescription + images + OBD + vehicle information → one model → final answer.\n\nIt is very attractive for a prototype.\n\nIt's also difficult to control.\n\nConsider these inputs:\n\nOBD code\n\n→ deterministic lookup\n\nPhoto\n\n→ visual analysis\n\nVoice\n\n→ transcription\n\nVehicle information\n\n→ exact lookup/API\n\nUnknown technical information\n\n→ tool-based retrieval\n\nSafety constraint\n\n→ explicit business rule\n\nThese are fundamentally different operations.\n\nPutting everything into one giant prompt hides those differences.\n\nIt also becomes much harder to answer:\n\nWhere did this conclusion come from?\n\nThat's a serious problem once the system moves beyond a demo.\n\n**Structured data comes first**\n\nOne design decision became particularly important for us:\n\nIf reliable structured information already exists, use it directly.\n\nFor example, when an OBD diagnostic trouble code is available, we first query our curated OBD database.\n\nA record can contain information such as:\n\n```\n{ \"code\": \"...\", \"explanation\": \"...\", \"possible_causes\": [], \"possible_actions\": [], \"validation_status\": \"reviewed\" }\n```\n\nWhy retrieve semantically similar paragraphs from documents when the system can perform an exact lookup against validated fields?\n\nThis is an important distinction.\n\n**Structured data and RAG solve different problems.**\n\nStructured data is excellent for:\n\nexact identifiers,\n\nvalidated fields,\n\ncontrolled records,\n\ndeterministic queries.\n\nRAG becomes useful when knowledge primarily lives inside documents.\n\n**What if the structured knowledge is missing?**\n\nThat's where agents become useful.\n\nOur internal database cannot contain every code, every manufacturer-specific interpretation and every vehicle configuration.\n\nWhen structured knowledge is missing, a tool-using agent can search for additional information.\n\nBut there is an important rule:\n\nRetrieved information should not silently become equivalent to validated information.\n\nInstead, preserve provenance.\n\nFor example:\n\n```\n{ \"code\": \"...\", \"source_type\": \"tool_agent\", \"sources\": [], \"validation_status\": \"unverified\" }\n```\n\nThe final system should know whether a piece of information came from:\n\na reviewed internal database,\n\nan external technical source,\n\na tool-using agent,\n\nimage analysis,\n\nor the driver themselves.\n\nThe model doesn't only need context.\n\nIt needs context with provenance.\n\n**Images are evidence, not diagnosis**\n\nVision models are another useful component.\n\nA photo might reveal:\n\nThe output should therefore look closer to:\n\n```\n{ \"observation\": \"Possible fluid trace\", \"confidence\": \"medium\", \"limitations\": [ \"The source is not visible\" ], \"requires_physical_inspection\": true }\n```\n\nrather than:\n\nYour vehicle has an oil leak.\n\nThat difference matters.\n\nAI systems often sound more certain than the evidence actually allows.\n\n**Voice is just another source of context**\n\nVoice is valuable because describing a mechanical problem through a form can be difficult.\n\nThe message gets transcribed, then becomes another input to the evidence layer.\n\nBut again:\n\nis not necessarily a confirmed technical fact.\n\nIt's something reported by the driver.\n\nThat distinction should survive the entire pipeline.\n\n**The next step: a structured evidence layer**\n\nToday, multiple processing paths can eventually contribute information to the generation context.\n\nAs the number of sources increases, simple concatenation becomes harder to control.\n\nYou eventually need something closer to:\n\n```\n{\n  \"reported_symptoms\": [],\n  \"obd_findings\": [],\n  \"visual_findings\": [],\n  \"retrieved_information\": [],\n  \"missing_information\": [],\n  \"conflicts\": [],\n  \"safety_flags\": []\n}\n```\n\nThe final LLM can then generate from this normalized evidence instead of receiving an unstructured wall of text.\n\nThis makes several things easier:\n\nIt also makes the architecture much easier to evolve.\n\n**So where is RAG?**\n\nWe aren't starting with a large automotive RAG pipeline.\n\nAnd that's deliberate.\n\nAdding PDFs to a vector database isn't the difficult part.\n\nThe difficult part is knowing whether a retrieved procedure applies to the correct:\n\nA perfectly retrieved technical procedure for the wrong engine generation can still be completely wrong for the vehicle in front of you.\n\nSo our current priority is:\n\nValidated structured data\n\n↓\n\nExplicit rules / APIs\n\n↓\n\nTool-based retrieval when needed\n\n↓\n\nLLM generation\n\nRAG becomes much more valuable when we have a controlled corpus of manufacturer manuals and validated technical documentation.\n\nEventually, the source router could look roughly like:\n\nThis isn't a universal hierarchy.\n\nThe broader point is more important:\n\nA generated or retrieved paragraph should not silently override a reviewed fact.\n\n**Two audiences, same evidence**\n\nAnother interesting problem is that the same evidence needs different outputs.\n\nA driver needs:\n\nA garage needs:\n\nWe don't need two independent AI analyses.\n\nWe need two representations of the same evidence.\n\nThat distinction has become an important part of the product architecture.\n\n**What I'm learning from building this**\n\nThe quality of an AI response doesn't start with the final prompt.\n\nIt starts much earlier:\n\nInput collection\n\n↓\n\nNormalization\n\n↓\n\nSource routing\n\n↓\n\nEvidence + provenance\n\n↓\n\nConflict / uncertainty handling\n\n↓\n\nGeneration\n\nA better model can improve generation.\n\nIt cannot fix an architecture where every source is mixed into one untraceable context.\n\nMy current takeaway is simple:\n\nUse deterministic systems where knowledge is deterministic.\n\nUse agents where flexibility is useful.\n\nUse RAG when document retrieval is actually the problem.\n\nAnd preserve uncertainty instead of asking the LLM to hide it behind a confident answer.\n\nI wrote a more detailed version of this architecture — including the current pipeline, future evidence layer and planned RAG architecture — here:\n\n**Original deep dive:**\n\n[https://younes.hashnode.dev/inside-inspecly-s-automotive-ai-pipeline-from-driver-symptoms-to-actionable-garage-requests](https://younes.hashnode.dev/inside-inspecly-s-automotive-ai-pipeline-from-driver-symptoms-to-actionable-garage-requests)\n\nI'm also building AI DevList, where I curate useful resources about agents, LLM engineering, MCP, RAG, evals and production AI — with a short explanation of why each resource matters.\n\nI'm curious how other teams handle this:\n\nDo you start from the LLM and build outward, or from the evidence and build inward?", "url": "https://wpnews.pro/news/don-t-start-with-rag-lessons-from-building-an-automotive-ai-pipeline", "canonical_source": "https://dev.to/younes_bentlili_9480340f/dont-start-with-rag-lessons-from-building-an-automotive-ai-pipeline-2igc", "published_at": "2026-08-16 18:03:12+00:00", "updated_at": "2026-08-16 18:12:16.116960+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-products", "developer-tools"], "entities": ["Inspecly"], "alternates": {"html": "https://wpnews.pro/news/don-t-start-with-rag-lessons-from-building-an-automotive-ai-pipeline", "markdown": "https://wpnews.pro/news/don-t-start-with-rag-lessons-from-building-an-automotive-ai-pipeline.md", "text": "https://wpnews.pro/news/don-t-start-with-rag-lessons-from-building-an-automotive-ai-pipeline.txt", "jsonld": "https://wpnews.pro/news/don-t-start-with-rag-lessons-from-building-an-automotive-ai-pipeline.jsonld"}}