Build a Chat API with ASP.NET Core and an LLM

A developer outlined a step-by-step approach to building a chat API with ASP.NET Core, C#, and Microsoft.Extensions.AI, showing how to wrap an LLM behind the IChatClient abstraction so the application service stays independent of the model provider. The guide starts with a minimal POST endpoint, then adds a scoped ChatService layer, custom request/response DTOs to keep the API contract stable, and plans for conversation history, ownership, persistence, and failure handling as the prototype scales to multiple users. The author stresses that the frontend never talks to the model provider directly and that no agent framework or vector database is needed for a working chat endpoint.

Build a Chat API with ASP.NET Core and an LLM Learn how to build a chat API with ASP.NET Core and an LLM using Microsoft.Extensions.AI: conversation history, ownership, failure handling, and testing. build-chat-api-aspnet-core-llm chat API with ASP.NET Core In the previous guide, we built the foundation of an AI application with ASP.NET Core and kept the LLM as a dependency inside the application, not the owner of authentication, business logic, and data access. Now we'll build something users can interact with: a chat API with ASP.NET Core. A chat interface is usually the first feature people associate with an AI application, but sending a message to a model is only one part of the workflow. The API has to accept input, maintain a conversation, call the model, handle failures, and return a response the frontend can consume. Once several users arrive, conversation ownership and persistence matter too. We'll use ASP.NET Core, C , and Microsoft.Extensions.AI, starting with a basic endpoint and then adding conversation history, a separate service layer, and fixes for problems that appear when a prototype becomes a real application. The architecture stays small: no agent framework or vector database is needed to make a chat endpoint work. Imagine an AI assistant in a developer support platform. A user asks, "What is dependency injection in ASP.NET Core?", then "Explain scoped versus singleton services," then "Show me a small example using an interface." The assistant should understand these belong to one conversation. The API exposes two operations: create a conversation, and send a message to an existing one. Frontend → ASP.NET Core API → Chat Service → IChatClient → LLM One boundary doesn't change: the frontend never talks to the model provider directly. All model requests go through our backend. dotnet new webapi -n FoundryChat cd FoundryChat Install Microsoft.Extensions.AI and the provider package for your model Ollama, Azure OpenAI, or another supported integration . The examples use the IChatClient abstraction, so the application service stays independent of the provider, which helps if you later move from a local model to a hosted one. Decide what the API accepts and returns first: public sealed record SendMessageRequest string Message ; public sealed record ChatResponse Guid ConversationId, string Message ; Returning your own DTO instead of the model's raw response keeps the API contract stable and leaves room to add metadata later. The client belongs in dependency injection: js builder.Services.AddSingleton<IChatClient sp = { // Create the provider-specific chat client here. // Configure the model and endpoint. // Return the client as IChatClient. } ; That's an architectural sketch, not a complete provider registration, because the constructor and package setup depend on your provider. The chat service receives IChatClient through its constructor, and a controller never needs to know which model runs or how the provider authenticates. using Microsoft.Extensions.AI; public sealed class ChatService IChatClient chatClient { public async Task<string SendAsync string message, CancellationToken cancellationToken { var response = await chatClient.GetResponseAsync message, cancellationToken: cancellationToken ; return response.Text ?? string.Empty; } } builder.Services.AddScoped<ChatService ; ApiController Route "api/chat" public class ChatController ChatService chatService : ControllerBase { HttpPost public async Task<IActionResult Send SendMessageRequest request, CancellationToken cancellationToken { var response = await chatService.SendAsync request.Message, cancellationToken ; return Ok new { message = response } ; } } The API now returns an LLM response, with an obvious limitation: every request is independent. The model has no idea what the user asked before. User: What is dependency injection? Assistant: A technique for supplying dependencies to a class instead of having it create them. User: Show me an example in C . The last message means nothing on its own, and the model may produce an unrelated example. The application has to send the previous messages along with the new one. Microsoft.Extensions.AI represents them as ChatMessage : js var messages = new List<ChatMessage { new ChatRole.User, "What is dependency injection?" , new ChatRole.Assistant, "Dependency injection supplies dependencies to a class." , new ChatRole.User, "Show me an example in C ." }; var response = await chatClient.GetResponseAsync messages, cancellationToken: cancellationToken ; This is the essential difference between a basic LLM endpoint and a conversational API: the application owns the history, and the model uses it to generate the next response. Sending the whole conversation from the frontend on every request creates trust and consistency problems, so the backend should own it. A new conversation gets an ID: public sealed record CreateConversationResponse Guid ConversationId ; HttpPost "conversations" public IActionResult CreateConversation = Ok new CreateConversationResponse Guid.NewGuid ; The frontend then calls POST /api/chat/conversations/{id}/messages with {"message": "Explain dependency injection with an example."} . But generating an ID isn't storing a conversation, so we need somewhere to keep the messages. For a first implementation, an in-memory store is enough to understand the workflow: public sealed class ChatConversation { public Guid Id { get; init; } public List<ChatMessage Messages { get; } = ; } public sealed class ConversationStore { private readonly ConcurrentDictionary<Guid, ChatConversation conversations = new ; public ChatConversation Create { var conversation = new ChatConversation { Id = Guid.NewGuid }; conversations conversation.Id = conversation; return conversation; } public bool TryGet Guid id, out ChatConversation conversation = conversations.TryGetValue id, out conversation ; } builder.Services.AddSingleton<ConversationStore ; That's for a prototype or single-process demo only. An in-memory store loses data on restart and doesn't make concurrent updates to one conversation safe. For a real application, persist conversations and messages in a database such as SQL Server. public sealed class ChatService IChatClient chatClient, ConversationStore store { public async Task<string SendAsync Guid conversationId, string message, CancellationToken cancellationToken { if store.TryGet conversationId, out var conversation throw new KeyNotFoundException "Conversation was not found." ; conversation.Messages.Add new ChatMessage ChatRole.User, message ; var response = await chatClient.GetResponseAsync conversation.Messages, cancellationToken: cancellationToken ; var assistantMessage = response.Text ?? string.Empty; conversation.Messages.Add new ChatMessage ChatRole.Assistant, assistantMessage ; return assistantMessage; } } The flow: find the conversation, append the user's message, send the history to the model, append the assistant's reply, and return it. The model stores nothing. The application controls the history and decides what goes into each request. The controller endpoint: HttpPost "conversations/{conversationId:guid}/messages" public async Task<IActionResult SendMessage Guid conversationId, SendMessageRequest request, CancellationToken cancellationToken { if string.IsNullOrWhiteSpace request.Message return BadRequest "Message cannot be empty." ; try { var response = await chatService.SendAsync conversationId, request.Message, cancellationToken ; return Ok new ChatResponse conversationId, response ; } catch KeyNotFoundException { return NotFound "Conversation was not found." ; } } An AI endpoint is still an HTTP endpoint. Validate empty messages, maximum length, conversation identifiers, request rate, and authentication: public sealed record SendMessageRequest Required StringLength 4000, MinimumLength = 1 string Message ; The right limit depends on your application and the model's context window. Treat user messages as untrusted input: they should never override authorization or expose secrets from the system prompt. A conversation ID is not an authorization mechanism. If one user can reach another's conversation by guessing or obtaining its ID, you have a serious data exposure problem. Associate each conversation with an authenticated user ConversationId , UserId , CreatedAt , UpdatedAt , and verify ownership on every message request: js var userId = User.FindFirst ClaimTypes.NameIdentifier ?.Value; Query by both the conversation ID and the user ID. Never fetch by ID alone and assume the caller is allowed to use it. This matters most when conversations hold private customer data or internal business information. The model call is a remote dependency. The provider can be unavailable, overloaded, rate-limiting you, or slow, and the connection can drop. Pass the cancellation token through the whole request path, as in SendAsync above, and distinguish client cancellation, provider failure, and application errors. Don't return raw provider exception details to the frontend: return a meaningful error and log the technical detail internally. Retry deliberately, because retrying every failed model request adds latency and cost without fixing the cause. Right now every request sends the entire conversation. That's fine for a short chat, but with hundreds of messages it raises token use, latency, and context-window pressure, and earlier information gets lost among newer messages. Introduce a context strategy: Stored history and model context don't have to be identical: the database can keep the full conversation while the model receives only a subset. Unit tests shouldn't need a real model. Because the service depends on IChatClient , you can substitute a fake that returns a predictable response, and verify that a conversation is created, user messages are stored, the assistant reply is appended, missing conversations return an error, empty messages are rejected, and cancellation propagates. Answer quality needs a separate evaluation process: a successful HTTP response doesn't prove the assistant answered correctly, so use representative questions and expected behavior. I wouldn't deploy the in-memory version for a multi-user application. A production chat API needs more around the same core design: | Component | Responsibility | |---|---| | Authentication | Identify the requesting user | | Authorization | Enforce conversation ownership | | SQL Server | Persist conversations and messages | | Chat Service | Coordinate the conversational workflow | | IChatClient | Communicate with the selected model | | Context Manager | Control message history and token limits | | Rate Limiting | Protect the API and model budget | | Logging | Diagnose requests without exposing sensitive content | | Error Handling | Return controlled failures | | Evaluation | Measure answer quality and regressions | None of this requires turning the chat endpoint into an autonomous agent. The final request path is: Frontend → ASP.NET Core API → Authentication → Conversation Service → Context Preparation → IChatClient → LLM The response comes back through the application, where the assistant message is stored before it's returned. SQL Server can replace the in-memory store without changing the chat service's responsibility. That's the advantage of keeping the model integration behind an application boundary. A chat API with ASP.NET Core is relatively simple once the model integration is separated from the application. The interesting decisions come later: maintaining conversations, protecting user data, managing context, handling model failures, and supporting multiple users. Start with a small chat service and a clear conversation model, persist history when the application needs it, and add context management before conversations get large. The same foundation can support a knowledge assistant, a customer-support chatbot, or an AI feature inside an existing business application. A natural next step is streaming, so users see the answer as it's generated instead of waiting for the whole response. More practical AI and .NET engineering guides: AI Engineering Foundry https://aiengfoundry.com