{"slug": "building-bis-smartassist-manakai-zero-hallucination-ai-for-23800-indian-with", "title": "Building BIS SmartAssist (MANAKAI): Zero-Hallucination AI for 23,800+ Indian Standards with OpenAI Codex", "summary": "A developer built BIS SmartAssist (MANAKAI), a retrieval-augmented conversational system that answers questions about India's 23,866+ published Indian Standards with strict citation traceability and a safe-refusal fallback. The system combines BM25 lexical search with 384-dimensional dense embeddings, reranks pooled candidates using a domain-tuned MiniLM cross-encoder, and gates answers on a closed-form confidence formula (rerank score, entity match, dual-channel consensus, and top-score margin) with a 0.70 threshold. The developer reports Recall@5 of 98.00% and MRR of 0.9323, and describes pair-programming the pipeline with OpenAI Codex.", "body_md": "When an MSME manufacturer, startup founder, or laboratory engineer asks an AI:\n\n*\"What are the mandatory testing parameters under Scheme-I ISI Mark for IS 1786 (High Strength Deformed Steel Bars)?\"*\n\nStandard commercial LLMs almost always fail catastrophically. They invent fictional standard numbers, cite revoked clauses, confuse voluntary Indian Standards with mandatory Quality Control Orders (QCOs), and hallucinate chemical tolerance limits. In regulatory engineering, a single fabricated number can lead to legal penalties or factory shutdowns.\n\nThe **Bureau of Indian Standards (BIS)** administers over **23,866 published Indian Standards (IS)**, hundreds of mandatory QCOs, and complex certification schemes (ISI Mark, CRS, Hallmarking, ECO Mark). \n\nTo solve this regulatory discovery bottleneck, we engineered **BIS SmartAssist (MANAKAI)** — a production-grade, domain-grounded conversational system with strict citation traceability and zero hallucination.\n\nHere is the story of how we built it, and our step-by-step pair-programming walkthrough with **OpenAI Codex**.\n\nTo architect our dual-channel retrieval pipeline, confidence calibration formula, and API handlers, we paired up with **OpenAI Codex**. \n\nBelow is the interactive session trace recorded and scrubbed in the browser. You can explore the exact prompts, internal reasoning, tool calls, and test runs:\n\nOur foundational rule: **Every factual claim must be strictly traceable to its primary document: Standard Number, Clause, Page Number, and official BIS portal reference.** If confidence falls below calibrated thresholds, the system safely refuses to answer instead of guessing.\n\n``` php\nflowchart TD\n    User([User Query / Product / Standard]) --> Intent[Intent Classifier & Exact Regex Extractor]\n\n    subgraph Candidate_Pooling [\"1. Dual-Channel Candidate Pooling (50 Candidates)\"]\n        Intent -->|Lexical Term Frequency| BM25[\"Lexical BM25 Okapi Index<br/>(k1=1.5, b=0.75)\"]\n        Intent -->|Semantic Similarity| Dense[\"Dense Semantic Embeddings<br/>(384-d Cosine Sim)\"]\n        BM25 --> Pool[Candidate Union & Min-Max Normalization]\n        Dense --> Pool\n    end\n\n    subgraph Neural_Reranking [\"2. Cross-Encoder Neural Reranker\"]\n        Pool --> Rerank[\"Domain-Tuned Cross-Encoder<br/>(MiniLM-L-6-v2 + Hard Negatives)\"]\n        Rerank --> TopK[\"Ranked Top-5 Context Windows\"]\n    end\n\n    subgraph Calibration [\"3. Closed-Form Confidence Calibration\"]\n        TopK --> ConfCalc[\"Confidence Calibration Formula<br/>(Entity match + Dual Consensus + Margin)\"]\n    end\n\n    ConfCalc --> Decision{Confidence >= 0.70?}\n    Decision -->|Yes| Grounded[Strict Evidence Synthesis + Primary Citations]\n    Decision -->|No| SafeRefusal[Safe Refusal: Request Specific Standard/Product]\n```\n\nLexical search excels at exact standard numbers (`IS 1786`, `IS 1061`), while semantic search captures natural descriptions (` rebar testing`, `safety boots`). We pool candidates from both channels:\n\nTop candidates are evaluated simultaneously by a cross-encoder (`ms-marco-MiniLM-L-6-v2`) fine-tuned with adversarial hard negatives. This lifts our **Recall@5 to 98.00%** and **MRR to 0.9323**.\n\nConfidence is computed in closed form using multi-signal telemetry:\n\n$$\\text{Confidence} = 0.40 \\cdot s_{\\text{rerank}} + 0.30 \\cdot m_{\\text{entity}} + 0.15 \\cdot c_{\\text{dual}} + 0.15 \\cdot (s_1 - s_2)$$\n\nWhere:\n\n🔗 **Live Web App:** [https://bis-lake.vercel.app/#main-content](https://bis-lake.vercel.app/#main-content) • **Source on GitHub:** `Master66999/BIS.git`\n\n```\n// 🔗 Live App: https://bis-lake.vercel.app/#main-content\n// 🔗 GitHub: https://github.com/Master66999/BIS.git\n// src/services/retrieval.ts\n\nexport interface Candidate {\n  standardNumber: string;\n  title: string;\n  division: string;\n  bm25Score: number;\n  denseScore: number;\n  rerankScore?: number;\n}\n\nexport async function poolCandidates(query: string, limit = 50): Promise<Candidate[]> {\n  const [lexicalMatches, semanticMatches] = await Promise.all([\n    bm25Index.search(query, limit),\n    vectorIndex.similaritySearch(query, limit)\n  ]);\n\n  const candidateMap = new Map<string, Candidate>();\n\n  lexicalMatches.forEach(item => {\n    candidateMap.set(item.id, {\n      ...item,\n      bm25Score: normalize(item.score),\n      denseScore: 0\n    });\n  });\n\n  semanticMatches.forEach(item => {\n    const existing = candidateMap.get(item.id);\n    if (existing) {\n      existing.denseScore = normalize(item.score);\n    } else {\n      candidateMap.set(item.id, {\n        ...item,\n        bm25Score: 0,\n        denseScore: normalize(item.score)\n      });\n    }\n  });\n\n  return Array.from(candidateMap.values());\n}\n```\n\nWe evaluated our pipeline against 500 domain-specific queries covering QCO mandates, lab testing thresholds, and product certification:\n\n| Metric | Baseline RAG (Vector Only) | BM25 Only | BIS SmartAssist (Hybrid + Reranker) | \n|---|---|---|---|\n| **Recall@1** | 68.4% | 72.1% | **91.4%** | \n| **Recall@5** | 81.2% | 84.6% | **98.0%** | \n| **Mean Reciprocal Rank (MRR)** | 0.742 | 0.778 | **0.932** | \n| **Factual Hallucination Rate** | 14.8% | 6.2% | **0.0% (Zero)** | \n| **Average Query Latency** | 340ms | 45ms | **180ms** | \n\n`DATABASE_URL`, API keys, local home directories) before sharing agent traces publicly.`{% agent_session ... %}`) gives readers and judges verifiable proof of how the solution was designed and coded.`codex-session-ktsrzd`) to see the complete pair-programming walkthrough!\n*What techniques are you using to prevent hallucination in mission-critical RAG applications? Let's discuss in the comments below!*", "url": "https://wpnews.pro/news/building-bis-smartassist-manakai-zero-hallucination-ai-for-23800-indian-with", "canonical_source": "https://dev.to/athlawange/building-bis-smartassist-manakai-zero-hallucination-ai-for-23800-indian-standards-with-openai-20b8", "published_at": "2026-10-17 08:13:30+00:00", "updated_at": "2026-10-17 08:31:55.054017+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "ai-agents", "mlops"], "entities": ["Bureau of Indian Standards", "BIS SmartAssist", "MANAKAI", "OpenAI Codex", "MiniLM-L-6-v2", "GitHub", "Vercel"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/building-bis-smartassist-manakai-zero-hallucination-ai-for-23800-indian-with", "markdown": "https://wpnews.pro/news/building-bis-smartassist-manakai-zero-hallucination-ai-for-23800-indian-with.md", "text": "https://wpnews.pro/news/building-bis-smartassist-manakai-zero-hallucination-ai-for-23800-indian-with.txt", "jsonld": "https://wpnews.pro/news/building-bis-smartassist-manakai-zero-hallucination-ai-for-23800-indian-with.jsonld"}}