{"slug": "grounded-search-agents-microsoft-foundry-azure-ai-search-cosmos-db-function", "title": "Grounded Search Agents: Microsoft Foundry + Azure AI Search + Cosmos DB + Function Calling", "summary": "Microsoft Foundry, Azure AI Search, and Azure Cosmos DB are combined to build a grounded search agent that uses function calling to route between unstructured document retrieval and structured data queries, always returning citations. The system outperforms plain RAG pipelines by integrating vector search with semantic ranking and maintaining conversation state in Cosmos DB.", "body_md": "A plain RAG pipeline — embed a query, hit a vector index, stuff the top-k chunks into a prompt — works fine for a single-turn FAQ bot. It falls apart the moment you need:\n\nThis post builds a small but complete **grounded search agent**: Azure OpenAI (via Microsoft Foundry) decides when to call a search tool versus a structured data tool, Azure AI Search handles retrieval and semantic ranking over unstructured documents, and Azure Cosmos DB stores conversation state and structured order/customer metadata. The agent uses OpenAI function calling to route between the two data sources and always returns citations.\n\nWe'll build this from the ground up in Python.\n\nThe system has two phases: **ingestion** (getting documents into a searchable index) and **runtime** (the agent loop that answers questions).\n\nAt runtime, the agent sits between the user and two tools, using function calling to decide which one (or both) to invoke:\n\nCreate the index with both a vector field (for semantic similarity) and a semantic configuration (for ranking), since combining the two consistently outperforms either alone.\n\n``` python\n# create_index.py\nfrom azure.search.documents.indexes import SearchIndexClient\nfrom azure.search.documents.indexes.models import (\n    SearchIndex, SimpleField, SearchableField, SearchField,\n    SearchFieldDataType, VectorSearch, HnswAlgorithmConfiguration,\n    VectorSearchProfile, SemanticConfiguration, SemanticPrioritizedFields,\n    SemanticField, SemanticSearch\n)\nfrom azure.core.credentials import AzureKeyCredential\n\nendpoint = \"https://<your-search-service>.search.windows.net\"\ncredential = AzureKeyCredential(\"<admin-key>\")\nindex_client = SearchIndexClient(endpoint, credential)\n\nfields = [\n    SimpleField(name=\"id\", type=SearchFieldDataType.String, key=True),\n    SearchableField(name=\"content\", type=SearchFieldDataType.String),\n    SimpleField(name=\"source_url\", type=SearchFieldDataType.String),\n    SearchField(\n        name=\"content_vector\",\n        type=SearchFieldDataType.Collection(SearchFieldDataType.Single),\n        searchable=True,\n        vector_search_dimensions=1536,\n        vector_search_profile_name=\"default-profile\",\n    ),\n]\n\nvector_search = VectorSearch(\n    profiles=[VectorSearchProfile(name=\"default-profile\", algorithm_configuration_name=\"hnsw-config\")],\n    algorithms=[HnswAlgorithmConfiguration(name=\"hnsw-config\")],\n)\n\nsemantic_config = SemanticConfiguration(\n    name=\"default-semantic\",\n    prioritized_fields=SemanticPrioritizedFields(\n        content_fields=[SemanticField(field_name=\"content\")]\n    ),\n)\n\nindex = SearchIndex(\n    name=\"support-docs-index\",\n    fields=fields,\n    vector_search=vector_search,\n    semantic_search=SemanticSearch(configurations=[semantic_config]),\n)\n\nindex_client.create_or_update_index(index)\nprint(\"Index created.\")\n```\n\nNow embed and upload chunks. Chunking matters more than most tutorials admit — 512-token chunks with ~15% overlap is a reasonable default for policy/support documents.\n\n``` python\n# ingest.py\nfrom openai import AzureOpenAI\nfrom azure.search.documents import SearchClient\nfrom azure.core.credentials import AzureKeyCredential\nimport uuid\n\naoai = AzureOpenAI(\n    azure_endpoint=\"https://<your-foundry-resource>.openai.azure.com\",\n    api_key=\"<key>\",\n    api_version=\"2024-10-21\",\n)\nsearch_client = SearchClient(\n    endpoint=endpoint,\n    index_name=\"support-docs-index\",\n    credential=AzureKeyCredential(\"<admin-key>\"),\n)\n\ndef chunk_text(text: str, max_tokens: int = 512, overlap: int = 80):\n    words = text.split()\n    step = max_tokens - overlap\n    return [\" \".join(words[i:i + max_tokens]) for i in range(0, len(words), step)]\n\ndef embed(text: str) -> list[float]:\n    resp = aoai.embeddings.create(model=\"text-embedding-3-small\", input=text)\n    return resp.data[0].embedding\n\ndef ingest_document(text: str, source_url: str):\n    docs = []\n    for chunk in chunk_text(text):\n        docs.append({\n            \"id\": str(uuid.uuid4()),\n            \"content\": chunk,\n            \"source_url\": source_url,\n            \"content_vector\": embed(chunk),\n        })\n    search_client.upload_documents(documents=docs)\n\nwith open(\"return_policy.txt\") as f:\n    ingest_document(f.read(), source_url=\"policies/return_policy.txt\")\n```\n\nNot every question is a document-retrieval problem. \"Where's my order?\" is a point lookup. Store operational data in Cosmos DB, partitioned by `customer_id`\n\nfor predictable query performance.\n\n``` python\n# cosmos_setup.py\nfrom azure.cosmos import CosmosClient, PartitionKey\n\nclient = CosmosClient(\"<cosmos-endpoint>\", credential=\"<cosmos-key>\")\ndb = client.create_database_if_not_exists(\"retail\")\norders = db.create_container_if_not_exists(\n    id=\"orders\",\n    partition_key=PartitionKey(path=\"/customer_id\"),\n    offer_throughput=400,\n)\n\norders.upsert_item({\n    \"id\": \"4471\",\n    \"customer_id\": \"cust_9001\",\n    \"status\": \"delayed\",\n    \"delay_reason\": \"carrier weather disruption\",\n    \"eta\": \"2026-08-02\",\n})\n```\n\nThis is the core piece: Azure OpenAI decides, per turn, whether to call `search_docs`\n\n, `get_order`\n\n, both, or neither. We define the tools as JSON schemas and let the model pick.\n\n``` python\n# agent.py\nimport json\nfrom openai import AzureOpenAI\nfrom azure.search.documents import SearchClient\nfrom azure.cosmos import CosmosClient\n\naoai = AzureOpenAI(\n    azure_endpoint=\"https://<your-foundry-resource>.openai.azure.com\",\n    api_key=\"<key>\",\n    api_version=\"2024-10-21\",\n)\nsearch_client = SearchClient(endpoint, \"support-docs-index\", AzureKeyCredential(\"<admin-key>\"))\ncosmos = CosmosClient(\"<cosmos-endpoint>\", credential=\"<cosmos-key>\")\norders_container = cosmos.get_database_client(\"retail\").get_container_client(\"orders\")\n\ntools = [\n    {\n        \"type\": \"function\",\n        \"function\": {\n            \"name\": \"search_docs\",\n            \"description\": \"Semantic search over policy and support documents. Returns passages with citations.\",\n            \"parameters\": {\n                \"type\": \"object\",\n                \"properties\": {\"query\": {\"type\": \"string\"}},\n                \"required\": [\"query\"],\n            },\n        },\n    },\n    {\n        \"type\": \"function\",\n        \"function\": {\n            \"name\": \"get_order\",\n            \"description\": \"Look up a customer order by ID for status, ETA, and delay reason.\",\n            \"parameters\": {\n                \"type\": \"object\",\n                \"properties\": {\n                    \"order_id\": {\"type\": \"string\"},\n                    \"customer_id\": {\"type\": \"string\"},\n                },\n                \"required\": [\"order_id\", \"customer_id\"],\n            },\n        },\n    },\n]\n\ndef search_docs(query: str):\n    results = search_client.search(\n        search_text=query,\n        query_type=\"semantic\",\n        semantic_configuration_name=\"default-semantic\",\n        top=3,\n    )\n    return [{\"content\": r[\"content\"], \"source\": r[\"source_url\"]} for r in results]\n\ndef get_order(order_id: str, customer_id: str):\n    item = orders_container.read_item(item=order_id, partition_key=customer_id)\n    return {\"status\": item[\"status\"], \"reason\": item.get(\"delay_reason\"), \"eta\": item.get(\"eta\")}\n\ndef run_agent(user_message: str, customer_id: str, history: list):\n    messages = history + [{\"role\": \"user\", \"content\": user_message}]\n    response = aoai.chat.completions.create(\n        model=\"gpt-4o\",\n        messages=messages,\n        tools=tools,\n        tool_choice=\"auto\",\n    )\n    msg = response.choices[0].message\n\n    if msg.tool_calls:\n        messages.append(msg)\n        for call in msg.tool_calls:\n            args = json.loads(call.function.arguments)\n            if call.function.name == \"search_docs\":\n                result = search_docs(**args)\n            elif call.function.name == \"get_order\":\n                args[\"customer_id\"] = customer_id  # enforce, don't trust model-provided id\n                result = get_order(**args)\n            messages.append({\n                \"role\": \"tool\",\n                \"tool_call_id\": call.id,\n                \"content\": json.dumps(result),\n            })\n        final = aoai.chat.completions.create(model=\"gpt-4o\", messages=messages)\n        return final.choices[0].message.content\n\n    return msg.content\n```\n\nA few things worth calling out:\n\n`customer_id`\n\nis overwritten server-side before hitting Cosmos DB, even though the model could technically supply one — this prevents cross-tenant data leaks via prompt injection.`tool_choice=\"auto\"`\n\n`chat.completions.create`\n\ncall (after tool results are appended) is what produces the grounded, cited final answer.Conversation history should not live in application memory — store it in Cosmos DB so any instance of the service can resume a session.\n\n``` php\n# session_store.py\ndef load_history(session_id: str) -> list:\n    container = cosmos.get_database_client(\"retail\").get_container_client(\"sessions\")\n    try:\n        item = container.read_item(item=session_id, partition_key=session_id)\n        return item[\"messages\"]\n    except Exception:\n        return []\n\ndef save_history(session_id: str, messages: list):\n    container = cosmos.get_database_client(\"retail\").get_container_client(\"sessions\")\n    container.upsert_item({\"id\": session_id, \"messages\": messages})\n```\n\nChoosing between search modes is a real architectural decision, not just a config toggle:\n\n| Retrieval Mode | Best for | Weakness |\n|---|---|---|\n| Keyword (BM25) | Exact terms, SKUs, error codes | Misses paraphrased queries |\n| Pure vector search | Conceptual / paraphrased questions | Can surface plausible-but-wrong matches |\n| Hybrid (keyword + vector) | General-purpose document Q&A | Slightly higher latency than either alone |\n| Semantic ranking on top of hybrid | Conversational, nuanced queries | Adds a reranking pass; not needed for simple lookups |\n| Structured query (Cosmos DB point read) | Known-ID lookups (orders, accounts) | Not applicable to unstructured content |\n\nFor most support/document agents, hybrid retrieval with semantic ranking is the right default, reserving structured lookups for anything with a stable ID.", "url": "https://wpnews.pro/news/grounded-search-agents-microsoft-foundry-azure-ai-search-cosmos-db-function", "canonical_source": "https://dev.to/jubinsoni/grounded-search-agents-microsoft-foundry-azure-ai-search-cosmos-db-function-calling-4l7h", "published_at": "2026-07-27 01:58:15+00:00", "updated_at": "2026-07-27 03:02:03.700782+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "ai-agents", "ai-infrastructure"], "entities": ["Microsoft Foundry", "Azure AI Search", "Azure Cosmos DB", "Azure OpenAI", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/grounded-search-agents-microsoft-foundry-azure-ai-search-cosmos-db-function", "markdown": "https://wpnews.pro/news/grounded-search-agents-microsoft-foundry-azure-ai-search-cosmos-db-function.md", "text": "https://wpnews.pro/news/grounded-search-agents-microsoft-foundry-azure-ai-search-cosmos-db-function.txt", "jsonld": "https://wpnews.pro/news/grounded-search-agents-microsoft-foundry-azure-ai-search-cosmos-db-function.jsonld"}}