{"slug": "building-ai-applications-with-ogx-the-open-source-agentic-api-server", "title": "Building AI Applications with OGX, the Open-Source Agentic API Server", "summary": "OGX, the open-source agentic API server formerly known as Meta's Llama Stack, has been rebranded with a model-agnostic, multi-SDK, production-grade mission, according to its GitHub page. The MIT-licensed project has over 8,400 GitHub stars and 1,400 forks and serves as a drop-in, self-hosted replacement for the OpenAI API that supports the OpenAI, Anthropic, and Google GenAI SDKs against the same server, with its Responses API handling server-side agentic orchestration, tool calling, MCP server integration, and built-in file search for RAG. A tutorial builds a RAG application called DocBrain using uv, Ollama with llama3.2:3b, and the OpenAI Python client, starting the server at https://localhost:8321.", "body_md": "# Building AI Applications with OGX, the Open-Source Agentic API Server\n\nOGX is an open-source agentic API server for building OpenAI-compatible AI applications, using any model and any infrastructure.\n\nHere's the fact worth knowing before anything else about this topic: **OGX is the rebrand of Meta's Llama Stack**. Its own GitHub page states it directly, [Llama Stack is now OGX](https://github.com/ogx-ai/ogx), with the name change accompanying a real shift in mission toward being model-agnostic, multi-SDK, and production-grade — not a new, unrelated project with a confusingly similar architecture. That single fact resolves a lot of confusion for anyone researching agentic API servers, OpenAI-compatible APIs, or local AI application development right now, since a good amount of what's written about Llama Stack still applies directly to OGX under its new name.\n\n## What Is OGX (and Why It Used to Be Called Llama Stack)\n\nOGX is plainly described as: [an open-source agentic API server for building AI applications, OpenAI-compatible, any model, any infrastructure](https://github.com/ogx-ai/ogx). Released under the MIT license, it's a genuinely established project — over 8,400 GitHub stars and 1,400 forks as of this writing — with regular community calls and active development.\n\nThe core pitch is a drop-in replacement for the OpenAI API that you run yourself, on a laptop, in a datacenter, or in the cloud, and point any OpenAI-compatible client or agentic framework at it. Behind that single API sits a pluggable provider architecture: develop locally against Ollama, deploy to production against vLLM, or connect to a managed cloud service — all through the exact same API surface, so swapping the backend later doesn't mean rewriting the application on top of it. It goes further than just chat completions too: the Responses API handles server-side agentic orchestration, tool calling, MCP server integration, and built-in file search for retrieval-augmented generation (RAG) in a single API call, and OGX natively supports the OpenAI, Anthropic, and Google GenAI SDKs against the same server, not just one vendor's client library.\n\nIn this article, we will build an app — let's call it DocBrain — that does two things. First, real RAG: upload documents, and it answers questions grounded in their actual content, with retrieval happening automatically, not through a manually wired vector database and a separate retrieval script. Second, a genuinely agentic capability: while reviewing your documents, it can call a custom tool to flag and save a real action item — a lease renewal deadline, a follow-up task — to an actual file on disk, so it doesn't just describe what you should do, it does the bookkeeping.\n\n## Prerequisites and Installing OGX\n\nYou need **[uv](https://docs.astral.sh/uv/getting-started/installation/)**, Python's fast package manager, and a local model backend. This tutorial uses **[Ollama](https://ollama.com/)**, though the same code works unchanged against vLLM or a cloud provider later — that's the entire point of OGX's provider architecture.\n\n```\n# Install Ollama, then pull a model\nollama pull llama3.2:3b\n\n# Set up a real project (not the throwaway uvx command from a quick trial)\nuv init docbrain && cd docbrain\nuv add ogx openai\n```\n\n**Code explanation:** `uv init` scaffolds a proper project rather than relying on a one-off uvx invocation, which matters the moment this stops being a five-minute experiment and becomes something you actually keep using. `uv add ogx openai` installs both the OGX server package and the standard OpenAI Python client, since that client is literally all DocBrain needs to talk to OGX.\n\n## Starting the Server and Verifying It Works\n\n```\nuv run ogx go\n```\n\nWith Ollama running and a model already pulled, this starts the OGX server at `https://localhost:8321`. Confirm it's actually healthy before writing any application code:\n\n```\ncurl -k -s https://localhost:8321/v1/models | python -m json.tool\n```\n\n**Code explanation:** [OGX listens on HTTPS by default and is secure out of the box](https://ogx-ai.github.io/docs/getting_started/quickstart), so local testing needs the `-k` flag to skip certificate verification against its self-signed local cert. A healthy response lists your registered model under `/v1/models`, confirming the server started and Ollama connected correctly, before you've written a single line of the actual application.\n\n## Basic Chat Through the Responses API\n\nThe smallest real call, establishing the pattern everything else in this tutorial builds on:\n\n``` python\nimport httpx\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"https://localhost:8321/v1\",\n    api_key=\"fake\",\n    http_client=httpx.Client(verify=False),\n)\n\nresponse = client.responses.create(\n    model=\"ollama/llama3.2:3b\",\n    input=\"What is OGX?\",\n)\nprint(response.output_text)\n```\n\n**Code explanation:** `api_key=\"fake\"` isn't a placeholder you're meant to fill in — OGX doesn't check it at all for a local server, any string works. `http_client=httpx.Client(verify=False)` is the Python equivalent of curl's `-k` flag from the previous step, skipping certificate verification for local development. The model string follows a `provider/model-name` format — `ollama/llama3.2:3b` — which is how OGX knows which of its configured providers to route the request to. `response.output_text` is the Responses API's convenience property for the final text answer, with no manual parsing of a choices array required.\n\n## Adding Real RAG: Uploading Documents and Creating a Vector Store\n\nThis is where OGX's built-in retrieval actually shows up, and it's genuinely compact:\n\n```\n# Upload a document\nfile = client.files.create(\n    file=open(\"lease_agreement.pdf\", \"rb\"),\n    purpose=\"assistants\",\n)\n\n# Create a vector store and index the file\nvector_store = client.vector_stores.create(\n    name=\"docbrain-kb\",\n    file_ids=[file.id],\n)\n\n# Ask questions with file search\nresponse = client.responses.create(\n    model=\"ollama/llama3.2:3b\",\n    input=\"Summarize the key points\",\n    tools=[{\n        \"type\": \"file_search\",\n        \"vector_store_ids\": [vector_store.id],\n    }],\n)\nprint(response.output_text)\n```\n\n**Code explanation:** `client.files.create` uploads the raw document; `purpose=\"assistants\"` marks it as content meant to be retrievable rather than, say, a fine-tuning dataset. `client.vector_stores.create` is the step that actually does the work most RAG tutorials spend dozens of lines on manually — chunking the document, embedding those chunks, and indexing them into a searchable store — all handled server-side by OGX. The `tools` list on the `responses.create` call is what activates retrieval for this specific question: `file_search` combined with the vector store's ID tells OGX to search that store for relevant content and ground the model's answer in what it finds, automatically, before generating a response.\n\n## Adding a Custom Tool: Flagging and Saving Action Items\n\nThis is the genuinely agentic half of DocBrain — a real Python function the model can choose to call.\n\n``` python\n# tools/action_items.py\nimport json\nimport os\nfrom datetime import datetime, timezone\n\nACTION_ITEMS_FILE = \"action_items.jsonl\"\n\ndef save_action_item(description: str, source_document: str, priority: str = \"medium\") -> dict:\n    \"\"\"Appends one action item as a JSON line to a local file.\"\"\"\n    if not description.strip():\n        raise ValueError(\"description cannot be empty\")\n    if priority not in (\"low\", \"medium\", \"high\"):\n        raise ValueError(f\"priority must be low, medium, or high, got '{priority}'\")\n\n    entry = {\n        \"description\": description.strip(),\n        \"source_document\": source_document.strip(),\n        \"priority\": priority,\n        \"flagged_at\": datetime.now(timezone.utc).isoformat(),\n    }\n    with open(ACTION_ITEMS_FILE, \"a\") as f:\n        f.write(json.dumps(entry) + \"\\n\")\n    return {\"status\": \"saved\", \"entry\": entry}\n\ndef list_action_items() -> list[dict]:\n    \"\"\"Reads back every saved action item, an empty list if none exist yet.\"\"\"\n    if not os.path.exists(ACTION_ITEMS_FILE):\n        return []\n    with open(ACTION_ITEMS_FILE) as f:\n        return [json.loads(line) for line in f if line.strip()]\n\nACTION_ITEM_TOOL_SCHEMA = {\n    \"type\": \"function\",\n    \"name\": \"save_action_item\",\n    \"description\": (\n        \"Save a genuine, concrete action item found while reviewing a document. \"\n        \"Only call this for real, actionable tasks, not general summary points.\"\n    ),\n    \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n            \"description\": {\"type\": \"string\", \"description\": \"The concrete action item, in one sentence\"},\n            \"source_document\": {\"type\": \"string\", \"description\": \"Which uploaded document this came from\"},\n            \"priority\": {\"type\": \"string\", \"enum\": [\"low\", \"medium\", \"high\"]},\n        },\n        \"required\": [\"description\", \"source_document\"],\n    },\n}\n```\n\n**Code explanation:** `save_action_item` validates its own input before writing anything — an empty description or an invalid priority value raises immediately rather than silently corrupting the saved file. Storing entries as JSON Lines (one JSON object per line, appended) rather than one large JSON array is a deliberate, practical choice: appending a line is trivial and safe even if the process is interrupted mid-write, while appending to a JSON array correctly requires reading, modifying, and rewriting the whole file every time. `ACTION_ITEM_TOOL_SCHEMA` is the description of this function the model actually sees — a tight, specific description (\"only call this for real, actionable tasks\") matters here, since a vague tool description tends to get over-triggered on things that aren't genuinely actionable.\n\n## Putting It Together: The Full Assistant\n\n``` python\n# docbrain.py\nimport httpx\nimport json\nfrom openai import OpenAI\nfrom tools.action_items import save_action_item, ACTION_ITEM_TOOL_SCHEMA\n\nclient = OpenAI(\n    base_url=\"https://localhost:8321/v1\",\n    api_key=\"fake\",\n    http_client=httpx.Client(verify=False),\n)\nMODEL = \"ollama/llama3.2:3b\"\n\ndef setup_knowledge_base(file_paths: list[str]) -> str:\n    file_ids = []\n    for path in file_paths:\n        uploaded = client.files.create(file=open(path, \"rb\"), purpose=\"assistants\")\n        file_ids.append(uploaded.id)\n    vector_store = client.vector_stores.create(name=\"docbrain-kb\", file_ids=file_ids)\n    return vector_store.id\n\ndef ask(question: str, vector_store_id: str) -> str:\n    response = client.responses.create(\n        model=MODEL,\n        input=question,\n        tools=[\n            {\"type\": \"file_search\", \"vector_store_ids\": [vector_store_id]},\n            ACTION_ITEM_TOOL_SCHEMA,\n        ],\n    )\n    for item in response.output:\n        if getattr(item, \"type\", None) == \"function_call\" and item.name == \"save_action_item\":\n            args = json.loads(item.arguments)\n            save_action_item(**args)\n    return response.output_text\n\nif __name__ == \"__main__\":\n    vector_store_id = setup_knowledge_base([\"lease_agreement.pdf\", \"vendor_contract.pdf\"])\n    answer = ask(\n        \"Summarize the key terms of these documents, and flag anything \"\n        \"that needs action before a deadline.\",\n        vector_store_id,\n    )\n    print(answer)\n```\n\n**Code explanation:** Both tools — `file_search` and the custom `save_action_item` schema — sit in the same `tools` list on one API call, which is the actual point of the Responses API's design: retrieval and custom tool calling aren't two separate systems bolted together, they're two entries in one list the model reasons over together. The loop over `response.output` is what makes the tool genuinely functional rather than decorative: it inspects every item the model produced, and any `function_call` matching `save_action_item` gets its arguments parsed and actually executed, writing a real line to `action_items.jsonl` on disk.\n\n## Running the Full Project\n\nWith Ollama running and a model pulled, start the server, then run the assistant against your own documents:\n\n```\nuv run ogx go          # in one terminal\nuv run python docbrain.py    # in another, once the server is healthy\n```\n\nReplace `lease_agreement.pdf` and `vendor_contract.pdf` with your own files. Ask it something real. If it finds a genuine deadline or task while answering, check `action_items.jsonl` afterward — a real, saved record will be sitting there, not just text that scrolled past in a terminal.\n\n## Wrapping Up\n\nThe actual reason OGX's pitch is worth taking seriously isn't any single feature in this tutorial — it's what happens later. The application we build here runs entirely on a local Ollama model with zero cost. The day you need production-grade reliability instead, moving to vLLM or a hosted provider means changing the model string and provider configuration, not rewriting the application. The vector store logic, the tool, the dispatch loop — all of it stays exactly as it is.\n\nThat's the real difference between an API wrapper and an actual agentic platform, and it's the reason this project is worth building on rather than throwing away the moment it needs to grow up.\n\n \n\n \n\n[**\\[Shittu Olumide\\](https://www.linkedin.com/in/olumide-shittu/)**](https://www.linkedin.com/in/olumide-shittu) is a software engineer and technical writer passionate about leveraging cutting-edge technologies to craft compelling narratives, with a keen eye for detail and a knack for simplifying complex concepts. You can also find Shittu on [Twitter](https://twitter.com/Shittu_Olumide_).", "url": "https://wpnews.pro/news/building-ai-applications-with-ogx-the-open-source-agentic-api-server", "canonical_source": "https://www.kdnuggets.com/building-ai-applications-with-open-source-agentic-api-server-ogx", "published_at": "2026-10-15 14:00:00+00:00", "updated_at": "2026-10-15 14:18:48.566641+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "agent-protocols", "ai-infrastructure"], "entities": ["OGX", "Llama Stack", "Meta", "Ollama", "vLLM", "OpenAI", "Anthropic", "Google GenAI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/building-ai-applications-with-ogx-the-open-source-agentic-api-server", "markdown": "https://wpnews.pro/news/building-ai-applications-with-ogx-the-open-source-agentic-api-server.md", "text": "https://wpnews.pro/news/building-ai-applications-with-ogx-the-open-source-agentic-api-server.txt", "jsonld": "https://wpnews.pro/news/building-ai-applications-with-ogx-the-open-source-agentic-api-server.jsonld"}}