{"slug": "building-production-grade-semantic-search-with-gpt-5-and-microsoft-foundry-from", "title": "Building Production-Grade Semantic Search with GPT-5 and Microsoft Foundry, From Scratch", "summary": "A developer built a production-grade semantic search pipeline using GPT-5 and Microsoft Foundry's Foundry IQ, an agentic retrieval layer on Azure AI Search. The system treats retrieval as a reasoning task, planning sub-queries, executing them in parallel, and synthesizing cited answers. The knowledge base exposes an MCP endpoint for integration with Foundry Agent Service or other MCP-compatible clients.", "body_md": "Most \"RAG tutorials\" stop at a single embedding query against a single index. That works for a demo and falls over the moment a real user asks something like \"compare our Q3 and Q4 vendor contracts and flag anything that changed\" — a question that needs multiple sub-queries, reasoning about what's still missing, and synthesis across documents.\n\nMicrosoft Foundry's answer to this is **Foundry IQ**: an agentic retrieval layer built on Azure AI Search that treats retrieval as a reasoning task rather than a single keyword or vector lookup. This is a from-scratch build of a semantic search pipeline using GPT-5 for query planning/synthesis and Foundry IQ for retrieval.\n\nInstead of one query hitting one index once, Foundry IQ's knowledge base plans sub-queries, executes them in parallel against one or more knowledge sources, evaluates whether it has enough signal, and iterates before synthesizing a final, cited answer.\n\nThe knowledge base sits between your agent and the underlying content. Your Foundry agent doesn't talk to Azure AI Search directly — it calls the knowledge base's MCP endpoint, which handles planning, retrieval, and synthesis behind a single tool call.\n\nA knowledge source is a reusable reference to your underlying content — in this example, a Blob Storage container of documents. Creating it also triggers Azure AI Search to generate the index, skillset, and indexer needed to chunk and vectorize the content automatically.\n\n``` python\nfrom azure.search.documents.indexes import SearchIndexClient\nfrom azure.search.documents.indexes.models import (\n    SearchIndexKnowledgeSource,\n    SearchIndexKnowledgeSourceParameters,\n)\nfrom azure.identity import DefaultAzureCredential\n\nindex_client = SearchIndexClient(\n    endpoint=\"https://<your-search-service>.search.windows.net\",\n    credential=DefaultAzureCredential(),\n)\n\nknowledge_source = SearchIndexKnowledgeSource(\n    name=\"vendor-contracts-ks\",\n    search_index_parameters=SearchIndexKnowledgeSourceParameters(\n        search_index_name=\"vendor-contracts-index\",\n    ),\n)\nindex_client.create_or_update_knowledge_source(knowledge_source=knowledge_source)\n```\n\nThe knowledge base ties one or more knowledge sources together with an LLM deployment that handles query planning and answer synthesis.\n\n```\nfrom azure.search.documents.indexes.models import (\n    KnowledgeBase,\n    KnowledgeBaseAzureOpenAIModel,\n    AzureOpenAIVectorizerParameters,\n)\n\nknowledge_base = KnowledgeBase(\n    name=\"vendor-contracts-kb\",\n    knowledge_sources=[{\"name\": \"vendor-contracts-ks\"}],\n    models=[\n        KnowledgeBaseAzureOpenAIModel(\n            azure_open_ai_parameters=AzureOpenAIVectorizerParameters(\n                resource_url=\"https://<your-foundry-resource>.openai.azure.com\",\n                deployment_name=\"gpt-5-mini\",\n                model_name=\"gpt-5-mini\",\n            )\n        )\n    ],\n    output_configuration={\n        \"modality\": \"answerSynthesis\",  # verbatim extractive data is the alternative\n    },\n)\nindex_client.create_or_update_knowledge_base(knowledge_base=knowledge_base)\n```\n\n`output_configuration`\n\nis the key lever here: `answerSynthesis`\n\nreturns a pre-generated, cited answer, while extractive mode returns verbatim source chunks and leaves reasoning entirely to your agent's own model. Extractive mode costs less and gives your agent more control; synthesis mode does more work up front at the retrieval layer.\n\nEach knowledge base exposes a standalone MCP endpoint. Any MCP-compatible client — including Foundry Agent Service, but also GitHub Copilot or other MCP clients — can call its `knowledge_base_retrieve`\n\ntool directly.\n\n``` python\nfrom azure.ai.projects import AIProjectClient\nfrom azure.identity import DefaultAzureCredential\n\nproject = AIProjectClient(\n    endpoint=\"https://<your-foundry-project-endpoint>\",\n    credential=DefaultAzureCredential(),\n)\n\nagent = project.agents.create_agent(\n    model=\"gpt-5-mini\",\n    name=\"contract-search-agent\",\n    instructions=\"Answer questions about vendor contracts using the knowledge base tool. Always cite sources.\",\n    tools=[{\n        \"type\": \"mcp\",\n        \"server_url\": \"https://<your-search-service>.search.windows.net/knowledgebases/vendor-contracts-kb/mcp?api-version=2026-05-01-preview\",\n        \"allowed_tools\": [\"knowledge_base_retrieve\"],\n    }],\n)\n```\n\nThe knowledge base's `retrieval_reasoning_effort`\n\nsetting controls how much LLM-driven planning happens before retrieval, and it's the main dial for balancing latency, cost, and answer quality.\n\n| Reasoning effort | What happens | Best for |\n|---|---|---|\n| Minimal | Bypasses LLM query planning entirely; direct hybrid search | Simple factual lookups, latency-sensitive paths |\n| Medium | LLM reformulates and may decompose the query into sub-queries | Multi-part or ambiguous questions |\n| High | Full iterative planning, evaluates sufficiency, re-queries as needed | Complex, multi-hop questions across many documents |\n\nLowering reasoning effort is also the primary way to control the number of GPT-5 tokens consumed per query — fewer planning passes and less iteration directly reduce both latency and inference cost, without needing to touch the underlying index.\n\nThe reason this matters versus classic RAG: a query like \"which vendors had payment terms that changed between the Q3 and Q4 renewals\" can't be answered by a single embedding lookup. With `medium`\n\nor `high`\n\nreasoning effort, the knowledge base decomposes it into sub-queries (find Q3 renewals, find Q4 renewals, compare payment terms fields), runs them in parallel against the knowledge source, and only synthesizes a final answer once it judges the retrieved context sufficient — re-querying automatically if it isn't.", "url": "https://wpnews.pro/news/building-production-grade-semantic-search-with-gpt-5-and-microsoft-foundry-from", "canonical_source": "https://dev.to/jubinsoni/building-production-grade-semantic-search-with-gpt-5-and-microsoft-foundry-from-scratch-2he", "published_at": "2026-07-19 03:40:11+00:00", "updated_at": "2026-07-19 04:27:49.105948+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure", "ai-tools", "developer-tools"], "entities": ["Microsoft Foundry", "Foundry IQ", "Azure AI Search", "GPT-5", "MCP"], "alternates": {"html": "https://wpnews.pro/news/building-production-grade-semantic-search-with-gpt-5-and-microsoft-foundry-from", "markdown": "https://wpnews.pro/news/building-production-grade-semantic-search-with-gpt-5-and-microsoft-foundry-from.md", "text": "https://wpnews.pro/news/building-production-grade-semantic-search-with-gpt-5-and-microsoft-foundry-from.txt", "jsonld": "https://wpnews.pro/news/building-production-grade-semantic-search-with-gpt-5-and-microsoft-foundry-from.jsonld"}}