# Event-Driven AI Orchestration with Kafka in .NET: A Production Blueprint > Source: > Published: 2026-10-21 03:42:14+00:00 Event-Driven AI Orchestration with Kafka in .NET: Learn how to build a production‑grade AI workflow in .NET using Kafka, covering outbox, transactional producers, deterministic keys, DLQ, and observability for high‑throughput, low‑cost orchestration. In a multi‑tenant scheduling platform, a team glued [Azure]()-ai-foundry-vs-azure-openai-service-observability-in-production-20261007" class="internal-link">Azure OpenAI calls into a handful of background services without a solid event backbone. The result was missed appointments, duplicated prompts, and a Kafka topic that grew until the broker crashed. The LLM was fine; the orchestration was broken. The core issue: an event‑driven architecture that ignored the plumbing that keeps AI pipelines reliable at scale. Our production system processes ~8 k RPS of appointment requests. Each request triggers a language‑model inference, persists a summary, and pushes a notification. The flow looks like: `Appointment` + `AiRequestOutbox` in a single EF Core transaction.`ai.requests` to Kafka.`ai.results`, updates Queues (Service Bus, RabbitMQ) offer at‑most‑once delivery and limited replay. Kafka gives you a durable log, partitioned streams, and built‑in replayability. For AI, replaying a failed inference without re‑invoking the model saves tokens and cost. However, Kafka’s higher memory footprint and the need for a cluster add operational overhead. Transactional producers (Kafka `transactional.id`) guarantee atomicity across writes and publishes but require idempotent producers and a dedicated transaction manager. The outbox pattern keeps the transaction in the DB, which is simpler and works well with existing relational workloads. The trade‑off is a slight latency increase due to polling. Blocking the consumer on a 200 ms to 2 s inference keeps the code simple but caps throughput. Offloading to a worker pool and emitting a `model.completed` event frees the consumer but adds complexity in state management. Managed services give you elasticity, compliance, and reduced operational toil. Self‑hosted models provide predictable cost and full control over scaling, but you must manage GPUs, patching, and data residency. The choice hinges on data sensitivity, latency budgets, and cost models. | Scenario | Recommendation | Key Trade‑off | |---|---|---| | Regulated data, strict compliance | Kafka + Azure OpenAI with VNet integration | Higher operational cost vs. compliance | | Low latency, high volume | Kafka with transactional producer and async inference workers | Complexity in idempotency vs. throughput | | Budget‑constrained, predictable usage | Self‑hosted LLM on AKS + Kafka outbox | Operational overhead vs. cost savings | | Rapid prototyping | Azure Service Bus + synchronous inference | Limited replayability vs. simplicity | `appointmentId`) floods one partition, causing a backlog.` auto.commit` – Commits before successful inference, causing lost work on failure.`max.poll.records` to throttle; instead, back‑pressure must be handled in the consumer. In production we adopt the following pattern: `ProcessedRequests` table before calling the model.`ai.dlq`. We expose `dlq_message_total` as a Prometheus counter and trigger alerts on spikes.`transactional.id` and use `producer.sendOffsetsToTransaction` to commit offsets only after the result has been persisted.`ContextId` to avoid re‑fetching the same data for each request.`consumer.poll` returns quickly and by using `max.poll.interval.ms` tuned to the longest inference time.`consumer.commitAsync` to reduce blocking.`linger.ms` and `batch.size` settings to batch messages when the outbox worker is idle.`kafka.server.KafkaServer#totalMemory` and scale brokers accordingly. For a 3‑node cluster handling 10 k RPS, 32 GB per broker is a safe baseline.`partition reassignment` during low traffic windows to avoid data skew.`topic.replication.factor` to 3 for durability.`kafka_consumer_lag` metrics. Use Kafka transactional producers with a unique "transactional.id". In the consumer, call producer.sendOffsetsToTransaction after persisting the result. This guarantees that the offset commit and the result publication happen atomically, eliminating duplicate or lost events. Choose deterministic keys that correlate with the natural partition key, such as appointmentId or a composite of userId+timestamp. Avoid random UUIDs or null keys, which scatter events across partitions and cause uneven load. Maintain a processedRequests lookup (e.g., a table or Redis set) keyed by the message key. Before invoking the model, check this store; if the request has already been processed, skip the call. This prevents token waste and duplicate notifications. Persist the outbox entity in the same DbContext transaction that creates the domain entity. After commit, a lightweight worker polls the outbox table, publishes the event to Kafka, and marks the row as dispatched. This keeps the write and publish atomic without a dedicated transaction manager. Expose Prometheus counters like dlq_message_total and kafka_consumer_lag. Configure alerts for spikes in DLQ volume or lag thresholds, and surface metrics in Grafana dashboards to detect back‑pressure or processing failures early. `EnableIdempotence = true` and use transactional writes so that an inference request and its response are committed atomically. Event‑driven AI orchestration is not a “plug‑and‑play” feature of .NET or Kafka. It requires a disciplined approach to message design, idempotency, transactional guarantees, and observability. By following the patterns above – deterministic keys, outbox + DLQ, transactional producers, and a robust monitoring stack – you can build a production‑grade AI workflow that scales from a handful of requests to tens of thousands per second while keeping costs under control and ensuring data integrity.