Building BIS SmartAssist (MANAKAI): Zero-Hallucination AI for 23,800+ Indian Standards with OpenAI Codex

When an MSME manufacturer, startup founder, or laboratory engineer asks an AI:

"What are the mandatory testing parameters under Scheme-I ISI Mark for IS 1786 (High Strength Deformed Steel Bars)?"

Standard commercial LLMs almost always fail catastrophically. They invent fictional standard numbers, cite revoked clauses, confuse voluntary Indian Standards with mandatory Quality Control Orders (QCOs), and hallucinate chemical tolerance limits. In regulatory engineering, a single fabricated number can lead to legal penalties or factory shutdowns.

The Bureau of Indian Standards (BIS) administers over 23,866 published Indian Standards (IS), hundreds of mandatory QCOs, and complex certification schemes (ISI Mark, CRS, Hallmarking, ECO Mark).

To solve this regulatory discovery bottleneck, we engineered BIS SmartAssist (MANAKAI) — a production-grade, domain-grounded conversational system with strict citation traceability and zero hallucination.

Here is the story of how we built it, and our step-by-step pair-programming walkthrough with OpenAI Codex.

To architect our dual-channel retrieval pipeline, confidence calibration formula, and API handlers, we paired up with OpenAI Codex.

Below is the interactive session trace recorded and scrubbed in the browser. You can explore the exact prompts, internal reasoning, tool calls, and test runs:

Our foundational rule: Every factual claim must be strictly traceable to its primary document: Standard Number, Clause, Page Number, and official BIS portal reference. If confidence falls below calibrated thresholds, the system safely refuses to answer instead of guessing.

flowchart TD
    User([User Query / Product / Standard]) --> Intent[Intent Classifier & Exact Regex Extractor]

    subgraph Candidate_Pooling ["1. Dual-Channel Candidate Pooling (50 Candidates)"]
        Intent -->|Lexical Term Frequency| BM25["Lexical BM25 Okapi Index<br/>(k1=1.5, b=0.75)"]
        Intent -->|Semantic Similarity| Dense["Dense Semantic Embeddings<br/>(384-d Cosine Sim)"]
        BM25 --> Pool[Candidate Union & Min-Max Normalization]
        Dense --> Pool
    end

    subgraph Neural_Reranking ["2. Cross-Encoder Neural Reranker"]
        Pool --> Rerank["Domain-Tuned Cross-Encoder<br/>(MiniLM-L-6-v2 + Hard Negatives)"]
        Rerank --> TopK["Ranked Top-5 Context Windows"]
    end

    subgraph Calibration ["3. Closed-Form Confidence Calibration"]
        TopK --> ConfCalc["Confidence Calibration Formula<br/>(Entity match + Dual Consensus + Margin)"]
    end

    ConfCalc --> Decision{Confidence >= 0.70?}
    Decision -->|Yes| Grounded[Strict Evidence Synthesis + Primary Citations]
    Decision -->|No| SafeRefusal[Safe Refusal: Request Specific Standard/Product]

Lexical search excels at exact standard numbers (IS 1786, IS 1061), while semantic search captures natural descriptions ( rebar testing, safety boots). We pool candidates from both channels:

Top candidates are evaluated simultaneously by a cross-encoder (ms-marco-MiniLM-L-6-v2) fine-tuned with adversarial hard negatives. This lifts our Recall@5 to 98.00% and MRR to 0.9323.

Confidence is computed in closed form using multi-signal telemetry:

$$\text{Confidence} = 0.40 \cdot s_{\text{rerank}} + 0.30 \cdot m_{\text{entity}} + 0.15 \cdot c_{\text{dual}} + 0.15 \cdot (s_1 - s_2)$$

Where:

🔗 Live Web App: https://bis-lake.vercel.app/#main-content • Source on GitHub: Master66999/BIS.git

// 🔗 Live App: https://bis-lake.vercel.app/#main-content
// 🔗 GitHub: https://github.com/Master66999/BIS.git
// src/services/retrieval.ts

export interface Candidate {
  standardNumber: string;
  title: string;
  division: string;
  bm25Score: number;
  denseScore: number;
  rerankScore?: number;
}

export async function poolCandidates(query: string, limit = 50): Promise<Candidate[]> {
  const [lexicalMatches, semanticMatches] = await Promise.all([
    bm25Index.search(query, limit),
    vectorIndex.similaritySearch(query, limit)
  ]);

  const candidateMap = new Map<string, Candidate>();

  lexicalMatches.forEach(item => {
    candidateMap.set(item.id, {
      ...item,
      bm25Score: normalize(item.score),
      denseScore: 0
    });
  });

  semanticMatches.forEach(item => {
    const existing = candidateMap.get(item.id);
    if (existing) {
      existing.denseScore = normalize(item.score);
    } else {
      candidateMap.set(item.id, {
        ...item,
        bm25Score: 0,
        denseScore: normalize(item.score)
      });
    }
  });

  return Array.from(candidateMap.values());
}

We evaluated our pipeline against 500 domain-specific queries covering QCO mandates, lab testing thresholds, and product certification:

DATABASE_URL, API keys, local home directories) before sharing agent traces publicly.{% agent_session ... %}) gives readers and judges verifiable proof of how the solution was designed and coded.codex-session-ktsrzd) to see the complete pair-programming walkthrough! What techniques are you using to prevent hallucination in mission-critical RAG applications? Let's discuss in the comments below!