# Building Production-Grade Semantic Search with GPT-5 and Microsoft Foundry, From Scratch

> Source: <https://dev.to/jubinsoni/building-production-grade-semantic-search-with-gpt-5-and-microsoft-foundry-from-scratch-2he>
> Published: 2026-07-19 03:40:11+00:00

Most "RAG tutorials" stop at a single embedding query against a single index. That works for a demo and falls over the moment a real user asks something like "compare our Q3 and Q4 vendor contracts and flag anything that changed" — a question that needs multiple sub-queries, reasoning about what's still missing, and synthesis across documents.

Microsoft Foundry's answer to this is **Foundry IQ**: an agentic retrieval layer built on Azure AI Search that treats retrieval as a reasoning task rather than a single keyword or vector lookup. This is a from-scratch build of a semantic search pipeline using GPT-5 for query planning/synthesis and Foundry IQ for retrieval.

Instead of one query hitting one index once, Foundry IQ's knowledge base plans sub-queries, executes them in parallel against one or more knowledge sources, evaluates whether it has enough signal, and iterates before synthesizing a final, cited answer.

The knowledge base sits between your agent and the underlying content. Your Foundry agent doesn't talk to Azure AI Search directly — it calls the knowledge base's MCP endpoint, which handles planning, retrieval, and synthesis behind a single tool call.

A knowledge source is a reusable reference to your underlying content — in this example, a Blob Storage container of documents. Creating it also triggers Azure AI Search to generate the index, skillset, and indexer needed to chunk and vectorize the content automatically.

``` python
from azure.search.documents.indexes import SearchIndexClient
from azure.search.documents.indexes.models import (
    SearchIndexKnowledgeSource,
    SearchIndexKnowledgeSourceParameters,
)
from azure.identity import DefaultAzureCredential

index_client = SearchIndexClient(
    endpoint="https://<your-search-service>.search.windows.net",
    credential=DefaultAzureCredential(),
)

knowledge_source = SearchIndexKnowledgeSource(
    name="vendor-contracts-ks",
    search_index_parameters=SearchIndexKnowledgeSourceParameters(
        search_index_name="vendor-contracts-index",
    ),
)
index_client.create_or_update_knowledge_source(knowledge_source=knowledge_source)
```

The knowledge base ties one or more knowledge sources together with an LLM deployment that handles query planning and answer synthesis.

```
from azure.search.documents.indexes.models import (
    KnowledgeBase,
    KnowledgeBaseAzureOpenAIModel,
    AzureOpenAIVectorizerParameters,
)

knowledge_base = KnowledgeBase(
    name="vendor-contracts-kb",
    knowledge_sources=[{"name": "vendor-contracts-ks"}],
    models=[
        KnowledgeBaseAzureOpenAIModel(
            azure_open_ai_parameters=AzureOpenAIVectorizerParameters(
                resource_url="https://<your-foundry-resource>.openai.azure.com",
                deployment_name="gpt-5-mini",
                model_name="gpt-5-mini",
            )
        )
    ],
    output_configuration={
        "modality": "answerSynthesis",  # verbatim extractive data is the alternative
    },
)
index_client.create_or_update_knowledge_base(knowledge_base=knowledge_base)
```

`output_configuration`

is the key lever here: `answerSynthesis`

returns a pre-generated, cited answer, while extractive mode returns verbatim source chunks and leaves reasoning entirely to your agent's own model. Extractive mode costs less and gives your agent more control; synthesis mode does more work up front at the retrieval layer.

Each knowledge base exposes a standalone MCP endpoint. Any MCP-compatible client — including Foundry Agent Service, but also GitHub Copilot or other MCP clients — can call its `knowledge_base_retrieve`

tool directly.

``` python
from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

project = AIProjectClient(
    endpoint="https://<your-foundry-project-endpoint>",
    credential=DefaultAzureCredential(),
)

agent = project.agents.create_agent(
    model="gpt-5-mini",
    name="contract-search-agent",
    instructions="Answer questions about vendor contracts using the knowledge base tool. Always cite sources.",
    tools=[{
        "type": "mcp",
        "server_url": "https://<your-search-service>.search.windows.net/knowledgebases/vendor-contracts-kb/mcp?api-version=2026-05-01-preview",
        "allowed_tools": ["knowledge_base_retrieve"],
    }],
)
```

The knowledge base's `retrieval_reasoning_effort`

setting controls how much LLM-driven planning happens before retrieval, and it's the main dial for balancing latency, cost, and answer quality.

| Reasoning effort | What happens | Best for |
|---|---|---|
| Minimal | Bypasses LLM query planning entirely; direct hybrid search | Simple factual lookups, latency-sensitive paths |
| Medium | LLM reformulates and may decompose the query into sub-queries | Multi-part or ambiguous questions |
| High | Full iterative planning, evaluates sufficiency, re-queries as needed | Complex, multi-hop questions across many documents |

Lowering reasoning effort is also the primary way to control the number of GPT-5 tokens consumed per query — fewer planning passes and less iteration directly reduce both latency and inference cost, without needing to touch the underlying index.

The reason this matters versus classic RAG: a query like "which vendors had payment terms that changed between the Q3 and Q4 renewals" can't be answered by a single embedding lookup. With `medium`

or `high`

reasoning effort, the knowledge base decomposes it into sub-queries (find Q3 renewals, find Q4 renewals, compare payment terms fields), runs them in parallel against the knowledge source, and only synthesizes a final answer once it judges the retrieved context sufficient — re-querying automatically if it isn't.