{"slug": "build-a-chat-api-with-asp-net-core-and-an-llm", "title": "Build a Chat API with ASP.NET Core and an LLM", "summary": "A developer outlined a step-by-step approach to building a chat API with ASP.NET Core, C#, and Microsoft.Extensions.AI, showing how to wrap an LLM behind the IChatClient abstraction so the application service stays independent of the model provider. The guide starts with a minimal POST endpoint, then adds a scoped ChatService layer, custom request/response DTOs to keep the API contract stable, and plans for conversation history, ownership, persistence, and failure handling as the prototype scales to multiple users. The author stresses that the frontend never talks to the model provider directly and that no agent framework or vector database is needed for a working chat endpoint.", "body_md": "Build a Chat API with ASP.NET Core and an LLM\n\nLearn how to build a chat API with ASP.NET Core and an LLM using Microsoft.Extensions.AI: conversation history, ownership, failure handling, and testing.\n\n`build-chat-api-aspnet-core-llm`\n\nchat API with ASP.NET Core\n\nIn the previous guide, we built the foundation of an AI application with ASP.NET Core and kept the LLM as a dependency inside the application, not the owner of authentication, business logic, and data access.\n\nNow we'll build something users can interact with: a chat API with ASP.NET Core. A chat interface is usually the first feature people associate with an AI application, but sending a message to a model is only one part of the workflow. The API has to accept input, maintain a conversation, call the model, handle failures, and return a response the frontend can consume. Once several users arrive, conversation ownership and persistence matter too.\n\nWe'll use ASP.NET Core, C#, and Microsoft.Extensions.AI, starting with a basic endpoint and then adding conversation history, a separate service layer, and fixes for problems that appear when a prototype becomes a real application. The architecture stays small: no agent framework or vector database is needed to make a chat endpoint work.\n\nImagine an AI assistant in a developer support platform. A user asks, \"What is dependency injection in ASP.NET Core?\", then \"Explain scoped versus singleton services,\" then \"Show me a small example using an interface.\" The assistant should understand these belong to one conversation.\n\nThe API exposes two operations: create a conversation, and send a message to an existing one.\n\n```\nFrontend → ASP.NET Core API → Chat Service → IChatClient → LLM\n```\n\nOne boundary doesn't change: the frontend never talks to the model provider directly. All model requests go through our backend.\n\n```\ndotnet new webapi -n FoundryChat\ncd FoundryChat\n```\n\nInstall `Microsoft.Extensions.AI` and the provider package for your model (Ollama, Azure OpenAI, or another supported integration). The examples use the `IChatClient` abstraction, so the application service stays independent of the provider, which helps if you later move from a local model to a hosted one.\n\nDecide what the API accepts and returns first:\n\n```\npublic sealed record SendMessageRequest(string Message);\n\npublic sealed record ChatResponse(Guid ConversationId, string Message);\n```\n\nReturning your own DTO instead of the model's raw response keeps the API contract stable and leaves room to add metadata later.\n\nThe client belongs in dependency injection:\n\n``` js\nbuilder.Services.AddSingleton<IChatClient>(sp =>\n{\n    // Create the provider-specific chat client here.\n    // Configure the model and endpoint.\n    // Return the client as IChatClient.\n});\n```\n\nThat's an architectural sketch, not a complete provider registration, because the constructor and package setup depend on your provider. The chat service receives `IChatClient` through its constructor, and a controller never needs to know which model runs or how the provider authenticates.\n\n```\nusing Microsoft.Extensions.AI;\n\npublic sealed class ChatService(IChatClient chatClient)\n{\n    public async Task<string> SendAsync(string message, CancellationToken cancellationToken)\n    {\n        var response = await chatClient.GetResponseAsync(message, cancellationToken: cancellationToken);\n        return response.Text ?? string.Empty;\n    }\n}\n\nbuilder.Services.AddScoped<ChatService>();\n\n[ApiController]\n[Route(\"api/chat\")]\npublic class ChatController(ChatService chatService) : ControllerBase\n{\n    [HttpPost]\n    public async Task<IActionResult> Send(SendMessageRequest request, CancellationToken cancellationToken)\n    {\n        var response = await chatService.SendAsync(request.Message, cancellationToken);\n        return Ok(new { message = response });\n    }\n}\n```\n\nThe API now returns an LLM response, with an obvious limitation: every request is independent. The model has no idea what the user asked before.\n\n**User:** What is dependency injection? **Assistant:** A technique for supplying dependencies to a class instead of having it create them. **User:** Show me an example in C#.\n\nThe last message means nothing on its own, and the model may produce an unrelated example. The application has to send the previous messages along with the new one. Microsoft.Extensions.AI represents them as `ChatMessage`:\n\n``` js\nvar messages = new List<ChatMessage>\n{\n    new(ChatRole.User, \"What is dependency injection?\"),\n    new(ChatRole.Assistant, \"Dependency injection supplies dependencies to a class.\"),\n    new(ChatRole.User, \"Show me an example in C#.\")\n};\n\nvar response = await _chatClient.GetResponseAsync(messages, cancellationToken: cancellationToken);\n```\n\nThis is the essential difference between a basic LLM endpoint and a conversational API: the application owns the history, and the model uses it to generate the next response.\n\nSending the whole conversation from the frontend on every request creates trust and consistency problems, so the backend should own it. A new conversation gets an ID:\n\n```\npublic sealed record CreateConversationResponse(Guid ConversationId);\n\n[HttpPost(\"conversations\")]\npublic IActionResult CreateConversation() =>\n    Ok(new CreateConversationResponse(Guid.NewGuid()));\n```\n\nThe frontend then calls `POST /api/chat/conversations/{id}/messages` with `{\"message\": \"Explain dependency injection with an example.\"}`. But generating an ID isn't storing a conversation, so we need somewhere to keep the messages. For a first implementation, an in-memory store is enough to understand the workflow:\n\n```\npublic sealed class ChatConversation\n{\n    public Guid Id { get; init; }\n    public List<ChatMessage> Messages { get; } = [];\n}\n\npublic sealed class ConversationStore\n{\n    private readonly ConcurrentDictionary<Guid, ChatConversation> _conversations = new();\n\n    public ChatConversation Create()\n    {\n        var conversation = new ChatConversation { Id = Guid.NewGuid() };\n        _conversations[conversation.Id] = conversation;\n        return conversation;\n    }\n\n    public bool TryGet(Guid id, out ChatConversation conversation) =>\n        _conversations.TryGetValue(id, out conversation!);\n}\n\nbuilder.Services.AddSingleton<ConversationStore>();\n```\n\nThat's for a prototype or single-process demo only. An in-memory store loses data on restart and doesn't make concurrent updates to one conversation safe. For a real application, persist conversations and messages in a database such as SQL Server.\n\n```\npublic sealed class ChatService(IChatClient chatClient, ConversationStore store)\n{\n    public async Task<string> SendAsync(Guid conversationId, string message, CancellationToken cancellationToken)\n    {\n        if (!store.TryGet(conversationId, out var conversation))\n            throw new KeyNotFoundException(\"Conversation was not found.\");\n\n        conversation.Messages.Add(new ChatMessage(ChatRole.User, message));\n\n        var response = await chatClient.GetResponseAsync(\n            conversation.Messages, cancellationToken: cancellationToken);\n\n        var assistantMessage = response.Text ?? string.Empty;\n        conversation.Messages.Add(new ChatMessage(ChatRole.Assistant, assistantMessage));\n\n        return assistantMessage;\n    }\n}\n```\n\nThe flow: find the conversation, append the user's message, send the history to the model, append the assistant's reply, and return it. The model stores nothing. The application controls the history and decides what goes into each request. The controller endpoint:\n\n```\n[HttpPost(\"conversations/{conversationId:guid}/messages\")]\npublic async Task<IActionResult> SendMessage(\n    Guid conversationId, SendMessageRequest request, CancellationToken cancellationToken)\n{\n    if (string.IsNullOrWhiteSpace(request.Message))\n        return BadRequest(\"Message cannot be empty.\");\n\n    try\n    {\n        var response = await _chatService.SendAsync(conversationId, request.Message, cancellationToken);\n        return Ok(new ChatResponse(conversationId, response));\n    }\n    catch (KeyNotFoundException)\n    {\n        return NotFound(\"Conversation was not found.\");\n    }\n}\n```\n\nAn AI endpoint is still an HTTP endpoint. Validate empty messages, maximum length, conversation identifiers, request rate, and authentication:\n\n```\npublic sealed record SendMessageRequest(\n    [Required]\n    [StringLength(4000, MinimumLength = 1)]\n    string Message);\n```\n\nThe right limit depends on your application and the model's context window. Treat user messages as untrusted input: they should never override authorization or expose secrets from the system prompt.\n\n**A conversation ID is not an authorization mechanism.** If one user can reach another's conversation by guessing or obtaining its ID, you have a serious data exposure problem. Associate each conversation with an authenticated user (`ConversationId`, `UserId`, `CreatedAt`, `UpdatedAt`), and verify ownership on every message request:\n\n``` js\nvar userId = User.FindFirst(ClaimTypes.NameIdentifier)?.Value;\n```\n\nQuery by both the conversation ID and the user ID. Never fetch by ID alone and assume the caller is allowed to use it. This matters most when conversations hold private customer data or internal business information.\n\nThe model call is a remote dependency. The provider can be unavailable, overloaded, rate-limiting you, or slow, and the connection can drop. Pass the cancellation token through the whole request path, as in `SendAsync` above, and distinguish client cancellation, provider failure, and application errors. Don't return raw provider exception details to the frontend: return a meaningful error and log the technical detail internally. Retry deliberately, because retrying every failed model request adds latency and cost without fixing the cause.\n\nRight now every request sends the entire conversation. That's fine for a short chat, but with hundreds of messages it raises token use, latency, and context-window pressure, and earlier information gets lost among newer messages. Introduce a context strategy:\n\nStored history and model context don't have to be identical: the database can keep the full conversation while the model receives only a subset.\n\nUnit tests shouldn't need a real model. Because the service depends on `IChatClient`, you can substitute a fake that returns a predictable response, and verify that a conversation is created, user messages are stored, the assistant reply is appended, missing conversations return an error, empty messages are rejected, and cancellation propagates. Answer quality needs a separate evaluation process: a successful HTTP response doesn't prove the assistant answered correctly, so use representative questions and expected behavior.\n\nI wouldn't deploy the in-memory version for a multi-user application. A production chat API needs more around the same core design:\n\n| Component | Responsibility | \n|---|---|\n| Authentication | Identify the requesting user | \n| Authorization | Enforce conversation ownership | \n| SQL Server | Persist conversations and messages | \n| Chat Service | Coordinate the conversational workflow | \n| IChatClient | Communicate with the selected model | \n| Context Manager | Control message history and token limits | \n| Rate Limiting | Protect the API and model budget | \n| Logging | Diagnose requests without exposing sensitive content | \n| Error Handling | Return controlled failures | \n| Evaluation | Measure answer quality and regressions | \n\nNone of this requires turning the chat endpoint into an autonomous agent. The final request path is:\n\n```\nFrontend → ASP.NET Core API → Authentication → Conversation Service\n        → Context Preparation → IChatClient → LLM\n```\n\nThe response comes back through the application, where the assistant message is stored before it's returned. SQL Server can replace the in-memory store without changing the chat service's responsibility. That's the advantage of keeping the model integration behind an application boundary.\n\nA chat API with ASP.NET Core is relatively simple once the model integration is separated from the application. The interesting decisions come later: maintaining conversations, protecting user data, managing context, handling model failures, and supporting multiple users.\n\nStart with a small chat service and a clear conversation model, persist history when the application needs it, and add context management before conversations get large. The same foundation can support a knowledge assistant, a customer-support chatbot, or an AI feature inside an existing business application. A natural next step is streaming, so users see the answer as it's generated instead of waiting for the whole response.\n\nMore practical AI and .NET engineering guides: [AI Engineering Foundry](https://aiengfoundry.com)", "url": "https://wpnews.pro/news/build-a-chat-api-with-asp-net-core-and-an-llm", "canonical_source": "https://dev.to/aiengfoundry/build-a-chat-api-with-aspnet-core-and-an-llm-5b04", "published_at": "2026-11-09 12:10:20+00:00", "updated_at": "2026-11-09 12:23:47.536276+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "large-language-models", "ai-agents"], "entities": ["ASP.NET Core", "Microsoft.Extensions.AI", "C#", "IChatClient", "Ollama", "Azure OpenAI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/build-a-chat-api-with-asp-net-core-and-an-llm", "markdown": "https://wpnews.pro/news/build-a-chat-api-with-asp-net-core-and-an-llm.md", "text": "https://wpnews.pro/news/build-a-chat-api-with-asp-net-core-and-an-llm.txt", "jsonld": "https://wpnews.pro/news/build-a-chat-api-with-asp-net-core-and-an-llm.jsonld"}}