{"slug": "what-buns-rust-rewrite-tells-us-about-rebuilding-the-ai-infrastructure-layer-in", "title": "What Bun’s Rust Rewrite Tells Us About Rebuilding the AI Infrastructure Layer in C#", "summary": "A developer argues that C# is well-positioned for AI infrastructure, citing TensorSharp, a native C# inference engine that outperforms stable-diffusion.cpp in benchmark tests. The article draws parallels between Bun's Rust rewrite and the need for stronger compile-time guarantees in production AI systems, suggesting that as AI moves from research to deployment, languages like C# offer both performance and robust ecosystem support.", "body_md": "**Original Chinese article:**\n\n[https://www.cnblogs.com/shanyou/p/21309486](https://www.cnblogs.com/shanyou/p/21309486)\n\nBun’s migration from Zig to Rust demonstrates a broader infrastructure trend: as software moves from experimentation into production, compiler-enforced correctness becomes more valuable than conventions that depend on developers always being careful.\n\nThe same transition may now be happening in AI infrastructure.\n\nPython remains excellent for research, training and rapid prototyping. However, production AI systems also need lifecycle management, API contracts, observability, dependency injection, database integration, deployment tooling, concurrency and predictable resource usage.\n\nThe article argues that C# is unusually well positioned for this layer.\n\nIts central piece of evidence is [TensorSharp](https://github.com/zhongkaifu/TensorSharp), a native C# inference engine whose reported Qwen Image Edit 2511 benchmark results outperform `stable-diffusion.cpp`\n\nin several pipeline stages.\n\nThe broader thesis is not simply that C# can run AI workloads. It is that C# can combine near-C++ inference performance with the application and infrastructure capabilities of the .NET ecosystem.\n\nThe article then extends this technical argument into a philosophical one:\n\n**Builder → AI Agent Leader → Taste**\n\nAs AI makes implementation increasingly accessible, human value shifts from writing every line of code toward defining problems, coordinating agents, evaluating results and deciding what is worth building.\n\nAt the end of 2025, the Bun team described migrating approximately 535,000 lines of Zig code to Rust using 64 Claude instances over an 11-day period.\n\nBun is a JavaScript runtime, which creates an inherently difficult boundary:\n\nThe article highlights examples such as use-after-free failures, invalidated hash maps, out-of-bounds writes and reference-counting problems.\n\nThese were not presented as isolated coding mistakes. They were symptoms of a structural problem: when garbage-collected code and manually managed memory interact, lifecycle correctness may depend heavily on conventions, testing, fuzzing and developer discipline.\n\nRust changes the feedback loop.\n\nInstead of discovering a lifetime problem after a crash, the compiler can reject an invalid ownership relationship before the program runs. In that model, rules that would otherwise live in a style guide become enforceable properties of the type system.\n\nThe article argues that production AI systems are encountering a similar transition.\n\n| Runtime-infrastructure problem | Comparable AI-infrastructure problem |\n|---|---|\n| Manual memory combined with JavaScript GC | Python’s dynamic runtime, GIL and native-library boundaries |\n| Large codebases that depend on conventions | Growing collections of difficult-to-maintain AI “glue code” |\n| Memory and concurrency failures discovered at runtime | Production crashes, leaks and concurrency bottlenecks |\n| Rapid AI-assisted rewrites | Increasing maintenance costs as infrastructure expands |\n\nThe conclusion is not that Python should disappear. Python remains highly valuable for algorithms, research and training.\n\nThe claim is narrower: **AI inference services are becoming production infrastructure rather than laboratory scripts, and the infrastructure layer increasingly benefits from compiled languages and stronger contracts.**\n\nBefore arguing that C# is a good infrastructure language, the article asks a more fundamental question:\n\n**Can C# compete with C++ at the inference-engine level?**\n\nIts answer is based on reported results from [TensorSharp](https://github.com/zhongkaifu/TensorSharp), a deep-learning inference engine implemented in C#.\n\nThe benchmark compared its Qwen Image Edit 2511 pipeline with `stable-diffusion.cpp`\n\n.\n\n`544 × 1184`\n\n| Metric |\n|\n|---|\n\nThe data is attributed to [TensorSharp](https://github.com/zhongkaifu/TensorSharp) PR #81 and its author, Zhongkai Fu.\n\nThe article’s argument is not merely that one C# implementation won one benchmark.\n\nIts more important claim is that C# can reach C++-class inference performance while remaining integrated with a managed production stack.\n\nA C++ inference engine may provide excellent low-level performance, but a complete production system still needs capabilities such as:\n\nWith C#, these capabilities can exist in the same runtime and programming model as the inference engine.\n\nThis is why the article describes [TensorSharp](https://github.com/zhongkaifu/TensorSharp) not as “C# glue around a native engine,” but as evidence that C# can be used to build the engine itself.\n\nThe article does not argue that C# is universally superior.\n\nDifferent languages occupy different optimization points.\n\nRust is a strong choice when the system requires:\n\nBun’s choice of Rust therefore makes sense.\n\nGo is exceptionally strong for:\n\nThe article characterizes Go as the native language of cloud infrastructure.\n\nC# occupies a different position. It combines managed memory and high-level application development with increasingly capable low-level primitives:\n\n`Span<T>`\n\n`Memory<T>`\n\n`ref struct`\n\n`unsafe`\n\ncode where necessaryIts central advantage is described as **full-lifecycle coverage**.\n\nC# can be used for:\n\n| Area | Go | Rust | C# |\n|---|---|---|---|\n| Memory model | Simple GC | Ownership and borrow checking | GC plus low-level memory APIs |\n| Concurrency | Goroutines | Tokio and async ecosystems |\n`async` /`await` , TPL and runtime integration |\n| Compilation | Extremely fast | Generally slower | Moderate and practical |\n| Binary footprint | Usually very small | Potentially very small | Larger, but still compact with NativeAOT |\n| Kubernetes | Excellent | Improving | Strong, especially with Aspire |\n| Observability | Usually configured manually | Usually configured manually | Strong OpenTelemetry integration |\n| ORM and migrations | Multiple external options | Several emerging options | EF Core and Code First |\n| Dependency injection | Usually external or manual | Usually manual | Native framework integration |\n| API development | Lightweight frameworks | Strong modern frameworks |\n|\n\nThe article summarizes the trade-off this way:\n\nThe article provides several additional benchmarks to support the broader C# infrastructure argument.\n\nThese numbers should be treated as the article’s reported comparisons rather than universal results for every workload.\n\n| Language | Reported AWS Lambda cold start, 1,024 MB |\n|---|---|\n| Python | 325 ms |\n| Go | 45 ms |\n| Rust | 30 ms |\n| C# NativeAOT | 35 ms |\n\n| Deployment | Reported image size |\n|---|---|\n| Python AI inference stack | 1,200 MB |\n| Minimal Go service | 15 MB |\n| C# NativeAOT service | 45 MB |\n\nThe article argues that Go’s smaller binary is impressive, while the C# deployment includes a much broader application stack, potentially including dependency injection, observability and production-service infrastructure.\n\nThe article also cites the following throughput figures on an RTX 4090:\n\n| Model | PyTorch | ONNX Runtime through C# | Reported advantage |\n|---|---|---|---|\n| DeepSeek 1.5B Int4 | 49.7 tok/s | 313.3 tok/s | 6.3× |\n| DeepSeek 7B Int4 | 43.5 tok/s | 161.0 tok/s | 3.7× |\n\n| Concurrent users | Python RPS | C# RPS |\n|---|---|---|\n| 100 | 3,200 | 9,500 |\n| 500 | 4,200 | 42,000 |\n| 1,000 | 4,500 | 78,000 |\n\nFor 1,000 concurrent users, the article reports approximately:\n\nFor a one-gigabyte JSON-processing workload on AWS Lambda, it lists:\n\n| Language | Reported processing time |\n|---|---|\n| Python | 12,000 ms |\n| Go | 3,200 ms |\n| Rust | 2,050 ms |\n| C# NativeAOT | 2,050 ms |\n\nAgain, these results are workload-specific. The intended point is that modern C# should not automatically be treated as a slow enterprise runtime.\n\nThe Bun discussion returns here.\n\nDynamic languages frequently discover certain classes of errors only when a code path is executed:\n\nC# cannot eliminate every runtime failure, but it can move many problems earlier through:\n\nThis matters because production infrastructure becomes expensive when errors appear only after deployment.\n\nGo also catches many type errors at compile time, but the article emphasizes that C# combines these checks with a richer application framework and lifecycle model.\n\nThe article presents C# as a recurring first-class language across Microsoft’s AI and agent stack.\n\nIts timeline includes:\n\nIt also states that more than 10,000 organizations use Azure AI Foundry Agent Service, citing examples such as KPMG, BMW and Fujitsu.\n\nThe larger point is that C# developers are not accessing the Microsoft AI ecosystem through an afterthought or secondary binding. They are participating through one of the stack’s primary languages.\n\nThe article defines total inference cost as more than model computation:\n\nA system that generates tokens quickly may still be expensive if it requires:\n\n| Cost area | Python | Go | C# |\n|---|---|---|---|\n| Container image | About 1.2 GB | About 15 MB | About 45 MB |\n| Cold start | 3–10 seconds in larger stacks | Under 100 ms | Under 100 ms |\n| Concurrency | Often uses multiple processes around the GIL | Goroutines | Async runtime and thread pool |\n| Runtime errors | Frequently discovered in production | Explicit error handling | More opportunities for compile-time detection |\n| Observability | Often assembled from third-party components | Usually configured manually | OpenTelemetry and Aspire integration |\n| Kubernetes deployment | Commonly hand-maintained YAML | Commonly hand-maintained YAML | Aspire can generate deployment resources |\n\nThe article argues that [TensorSharp](https://github.com/zhongkaifu/TensorSharp) changes the image-generation cost model by placing inference inside a smaller and more manageable C# service stack.\n\nIt specifically contrasts:\n\nThis is presented as the economic foundation for a proposed component called TokenHub, which would track and manage the cost of AI operations.\n\nThe article proposes a layered architecture rather than rewriting every AI algorithm in C#.\n\n```\nPython algorithm layer\n- PyTorch training\n- Jupyter experimentation\n- Existing research ecosystem\n\n             ↓\n\nMCP protocol boundary\n- Cross-language service interface\n\n             ↓\n\nC# AI-native infrastructure layer\n- TensorSharp for image and text inference\n- MetaSkill DAG for workflow orchestration\n- Harness runtime for execution\n- TokenHub for cost tracking\n- AxonHub for data collection and CDC\n- Semantic Kernel for LLM orchestration\n- Microsoft Agent Framework for agent lifecycle\n- ONNX Runtime C# APIs for general inference\n\n             ↓\n\n.NET runtime\n- NativeAOT\n- Managed memory\n- Low-level performance APIs\n\n             ↓\n\nLifecycle-management layer\n- .NET Aspire\n- OpenTelemetry\n- EF Core\n```\n\nThe architecture follows three principles.\n\nThe proposal does not attempt to rewrite PyTorch training, research notebooks or every scientific package.\n\nInstead, Python capabilities can be exposed as services across an MCP boundary.\n\nThe C# layer handles orchestration, persistence, observability, deployment, lifecycle management and selected inference engines.\n\n[TensorSharp](https://github.com/zhongkaifu/TensorSharp) is used as the primary example of C# implementing a performance-critical engine rather than merely calling a separate C++ executable.\n\nThe second half of the article moves beyond language selection.\n\nIt asks what happens when AI and modern frameworks make engine construction accessible to many more developers.\n\nThe proposed progression is:\n\n```\nBuilder → AI Agent Leader → Taste\n```\n\nHistorically, building an inference engine required knowledge of:\n\nThe article argues that projects such as [TensorSharp](https://github.com/zhongkaifu/TensorSharp), combined with Aspire, Semantic Kernel and Microsoft Agent Framework, reduce the amount of specialized knowledge required to turn an idea into a working AI service.\n\nThe important shift is not that engineering disappears.\n\nIt is that writing code becomes a means rather than the defining identity of the role.\n\nAs AI generates more implementation code, humans increasingly focus on:\n\nFor example, an AI marketing-image system might use:\n\nThe human role is not merely to fix generated code.\n\nThe human decides whether the system solves the correct business problem, follows the intended brand style and remains within acceptable cost and risk boundaries.\n\nThe article defines Taste as more than personal preference.\n\nTaste is structured judgment about quality, value and boundaries.\n\nWhen an AI system can propose many architectures, human judgment selects the design that balances:\n\nThe article uses [TensorSharp](https://github.com/zhongkaifu/TensorSharp) PR #81 as an example: decisions about DiT reconstruction and CUDA Graph Capture are not simply binary matters of right and wrong. They involve trade-offs among speed, memory and complexity.\n\nWhen AI can generate unlimited features, someone still has to decide:\n\nWhen AI can generate almost any content or action, humans must define boundaries around:\n\nThe article’s position is that automation can free humans from repetitive execution, but it cannot eliminate the need to decide what should exist.\n\nThis is one of the article’s most important disclaimers:\n\n**The Taste-gate system described below is a design proposal. It has not yet been implemented in the** **OpenClaw.NET****repository.**\n\nAccording to the article, [OpenClaw.NET](http://OpenClaw.NET) already contains passive or safety-oriented governance capabilities such as:\n\n`user_input`\n\npause pointsThese mechanisms can expose plans, evidence, risks and approval records for inspection.\n\nHowever, most of them do not actively stop an agent workflow based on product quality, aesthetics or broader value judgments.\n\nThe article proposes adding concepts such as:\n\n`TasteGate`\n\n`ITasteGate<TInput, TOutput>`\n\ninterface`TasteDecision`\n\nresult`BrandTaste`\n\n, `EthicalTaste`\n\nand `TechnicalTaste`\n\nThe gate would produce one of three outcomes:\n\nThis is more useful than a simple approve/reject model because many AI outputs are not fundamentally invalid; they merely need another iteration.\n\nThe Agent Leader translates business intent into explicit constraints.\n\nPossible outputs include:\n\nThe AI system performs the work through an orchestrated workflow:\n\nKey outputs are evaluated against:\n\nThe final result is Pass, Retry or Abort.\n\nThe article presents a conceptual workflow like this:\n\n```\nNatural-language user request\n        ↓\nBrand-Taste constraints\n- Technology-oriented blue palette\n- Minimalist visual language\n- No human figures\n        ↓\nCost constraint\n- No more than $0.50 per generation\n        ↓\nAgent workflow\n- Prompt optimization through Semantic Kernel\n- Image generation through TensorSharp and CUDA\n- CLIP or aesthetic-quality evaluation\n- TokenHub cost calculation\n        ↓\nTaste gate\n- Technical review\n- Product and brand review\n- Ethical and copyright review\n        ↓\nPass  → return the image and cost report\nRetry → revise the prompt and regenerate, up to a fixed limit\nAbort → record the failure, raise an alert and request human review\n```\n\nThe proposal suggests placing gates according to two factors:\n\nLow-impact, low-uncertainty decisions can remain autonomous.\n\nHigh-impact, high-uncertainty decisions should require direct human involvement.\n\nThe article proposes expressing some constraints as C# types rather than keeping everything in prompts or informal documentation.\n\nA conceptual `BrandTaste`\n\nrecord could contain fields such as:\n\nA generic Taste-gate interface could require both its input and output to implement an auditable contract.\n\nThis would not make aesthetic judgment fully compile-time enforceable. A compiler cannot objectively determine whether an image is beautiful.\n\nHowever, the type system can enforce that:\n\nThe article describes this philosophy as **“Taste as types.”**\n\nThe goal is to move as much governance as possible away from undocumented runtime behavior and into explicit, inspectable contracts.\n\nThe article presents the following illustrative comparison:\n\n| Capability | Current AI agent | Human Agent Leader | Expected relationship |\n|---|---|---|---|\n| Technical execution | 9/10 | 7/10 | AI executes; human decides |\n| Product insight | 7/10 | 9/10 | AI assists; human leads |\n| Ethical sensitivity | 4/10 | 9/10 | AI assists; human leads |\n| Systems thinking | 8/10 | 9/10 | AI assists; human leads |\n| Aesthetic intuition | 3/10 | 9/10 | AI assists; human leads |\n| Risk awareness | 6/10 | 9/10 | AI assists; human leads |\n| Ability to anticipate evolution | 5/10 | 8/10 | AI assists; human leads |\n\nThese numbers are conceptual rather than scientific measurements.\n\nThey express the article’s belief that AI may exceed humans at implementation while remaining weaker at value judgments that depend on culture, responsibility, long-term context and lived experience.\n\nThe proposed progression is:\n\n| Level | Role | Primary capability | Typical tools | Main output |\n|---|---|---|---|---|\n| Level 1 | Builder | Coding, debugging and optimization | IDE, Git and CI/CD | Features |\n| Level 2 | Agent Operator | Prompting and agent configuration | Semantic Kernel and AutoGen | Agent efficiency |\n| Level 3 | Agent Leader | Problem definition, tool selection, orchestration and review | MetaSkill DAG, Harness and TokenHub | System-level value |\n| Level 4 | Taste Architect | Domain modeling, values, ethics and evolutionary direction | DDD, ontologies and typed Taste constraints | Organizational judgment |\n\nThe transition is described as:\n\nThe article concludes with a simple division of responsibilities.\n\nBun chose Rust because a JavaScript runtime requires strict memory control and deep native interoperability.\n\nGo remains an excellent language for cloud-native infrastructure.\n\nPython remains indispensable for AI research and training.\n\nThe article’s argument is that C# is increasingly occupying another high-value layer: the productionization, servicing, orchestration and operation of AI systems.\n\n[TensorSharp](https://github.com/zhongkaifu/TensorSharp) is presented as evidence that C# can also move downward into the inference-engine layer without giving up the broader lifecycle capabilities of .NET.\n\nBut the most important argument is ultimately not about language performance.\n\nAs implementation becomes easier, the human role changes:\n\nThe future is therefore not simply about replacing Python, Go, Rust or C++ with C#.\n\nIt is about using each language where it provides the most leverage—and using C# to build an integrated AI infrastructure layer that allows people to spend less time assembling operational plumbing and more time exercising judgment.\n\n**The long-term human advantage is not merely the ability to build. It is the ability to decide what is worth building.**\n\n**Original Chinese article:**\n\n[https://www.cnblogs.com/shanyou/p/21309486](https://www.cnblogs.com/shanyou/p/21309486)\n\nBun’s migration from Zig to Rust demonstrates a broader infrastructure trend: as software moves from experimentation into production, compiler-enforced correctness becomes more valuable than conventions that depend on developers always being careful.\n\nThe same transition may now be happening in AI infrastructure.\n\nPython remains excellent for research, training and rapid prototyping. However, production AI systems also need lifecycle management, API contracts, observability, dependency injection, database integration, deployment tooling, concurrency and predictable resource usage.\n\nThe article argues that C# is unusually well positioned for this layer.\n\nIts central piece of evidence is [TensorSharp](https://github.com/zhongkaifu/TensorSharp), a native C# inference engine whose reported Qwen Image Edit 2511 benchmark results outperform `stable-diffusion.cpp`\n\nin several pipeline stages.\n\nThe broader thesis is not simply that C# can run AI workloads. It is that C# can combine near-C++ inference performance with the application and infrastructure capabilities of the .NET ecosystem.\n\nThe article then extends this technical argument into a philosophical one:\n\n**Builder → AI Agent Leader → Taste**\n\nAs AI makes implementation increasingly accessible, human value shifts from writing every line of code toward defining problems, coordinating agents, evaluating results and deciding what is worth building.\n\nAt the end of 2025, the Bun team described migrating approximately 535,000 lines of Zig code to Rust using 64 Claude instances over an 11-day period.\n\nBun is a JavaScript runtime, which creates an inherently difficult boundary:\n\nThe article highlights examples such as use-after-free failures, invalidated hash maps, out-of-bounds writes and reference-counting problems.\n\nThese were not presented as isolated coding mistakes. They were symptoms of a structural problem: when garbage-collected code and manually managed memory interact, lifecycle correctness may depend heavily on conventions, testing, fuzzing and developer discipline.\n\nRust changes the feedback loop.\n\nInstead of discovering a lifetime problem after a crash, the compiler can reject an invalid ownership relationship before the program runs. In that model, rules that would otherwise live in a style guide become enforceable properties of the type system.\n\nThe article argues that production AI systems are encountering a similar transition.\n\n| Runtime-infrastructure problem | Comparable AI-infrastructure problem |\n|---|---|\n| Manual memory combined with JavaScript GC | Python’s dynamic runtime, GIL and native-library boundaries |\n| Large codebases that depend on conventions | Growing collections of difficult-to-maintain AI “glue code” |\n| Memory and concurrency failures discovered at runtime | Production crashes, leaks and concurrency bottlenecks |\n| Rapid AI-assisted rewrites | Increasing maintenance costs as infrastructure expands |\n\nThe conclusion is not that Python should disappear. Python remains highly valuable for algorithms, research and training.\n\nThe claim is narrower: **AI inference services are becoming production infrastructure rather than laboratory scripts, and the infrastructure layer increasingly benefits from compiled languages and stronger contracts.**\n\nBefore arguing that C# is a good infrastructure language, the article asks a more fundamental question:\n\n**Can C# compete with C++ at the inference-engine level?**\n\nIts answer is based on reported results from [TensorSharp](https://github.com/zhongkaifu/TensorSharp), a deep-learning inference engine implemented in C#.\n\nThe benchmark compared its Qwen Image Edit 2511 pipeline with `stable-diffusion.cpp`\n\n.\n\n`544 × 1184`\n\n| Metric |\n|\n|---|\n\nThe data is attributed to [TensorSharp](https://github.com/zhongkaifu/TensorSharp) PR #81 and its author, Zhongkai Fu.\n\nThe article’s argument is not merely that one C# implementation won one benchmark.\n\nIts more important claim is that C# can reach C++-class inference performance while remaining integrated with a managed production stack.\n\nA C++ inference engine may provide excellent low-level performance, but a complete production system still needs capabilities such as:\n\nWith C#, these capabilities can exist in the same runtime and programming model as the inference engine.\n\nThis is why the article describes [TensorSharp](https://github.com/zhongkaifu/TensorSharp) not as “C# glue around a native engine,” but as evidence that C# can be used to build the engine itself.\n\nThe article does not argue that C# is universally superior.\n\nDifferent languages occupy different optimization points.\n\nRust is a strong choice when the system requires:\n\nBun’s choice of Rust therefore makes sense.\n\nGo is exceptionally strong for:\n\nThe article characterizes Go as the native language of cloud infrastructure.\n\nC# occupies a different position. It combines managed memory and high-level application development with increasingly capable low-level primitives:\n\n`Span<T>`\n\n`Memory<T>`\n\n`ref struct`\n\n`unsafe`\n\ncode where necessaryIts central advantage is described as **full-lifecycle coverage**.\n\nC# can be used for:\n\n| Area | Go | Rust | C# |\n|---|---|---|---|\n| Memory model | Simple GC | Ownership and borrow checking | GC plus low-level memory APIs |\n| Concurrency | Goroutines | Tokio and async ecosystems |\n`async` /`await` , TPL and runtime integration |\n| Compilation | Extremely fast | Generally slower | Moderate and practical |\n| Binary footprint | Usually very small | Potentially very small | Larger, but still compact with NativeAOT |\n| Kubernetes | Excellent | Improving | Strong, especially with Aspire |\n| Observability | Usually configured manually | Usually configured manually | Strong OpenTelemetry integration |\n| ORM and migrations | Multiple external options | Several emerging options | EF Core and Code First |\n| Dependency injection | Usually external or manual | Usually manual | Native framework integration |\n| API development | Lightweight frameworks | Strong modern frameworks |\n|\n\nThe article summarizes the trade-off this way:\n\nThe article provides several additional benchmarks to support the broader C# infrastructure argument.\n\nThese numbers should be treated as the article’s reported comparisons rather than universal results for every workload.\n\n| Language | Reported AWS Lambda cold start, 1,024 MB |\n|---|---|\n| Python | 325 ms |\n| Go | 45 ms |\n| Rust | 30 ms |\n| C# NativeAOT | 35 ms |\n\n| Deployment | Reported image size |\n|---|---|\n| Python AI inference stack | 1,200 MB |\n| Minimal Go service | 15 MB |\n| C# NativeAOT service | 45 MB |\n\nThe article argues that Go’s smaller binary is impressive, while the C# deployment includes a much broader application stack, potentially including dependency injection, observability and production-service infrastructure.\n\nThe article also cites the following throughput figures on an RTX 4090:\n\n| Model | PyTorch | ONNX Runtime through C# | Reported advantage |\n|---|---|---|---|\n| DeepSeek 1.5B Int4 | 49.7 tok/s | 313.3 tok/s | 6.3× |\n| DeepSeek 7B Int4 | 43.5 tok/s | 161.0 tok/s | 3.7× |\n\n| Concurrent users | Python RPS | C# RPS |\n|---|---|---|\n| 100 | 3,200 | 9,500 |\n| 500 | 4,200 | 42,000 |\n| 1,000 | 4,500 | 78,000 |\n\nFor 1,000 concurrent users, the article reports approximately:\n\nFor a one-gigabyte JSON-processing workload on AWS Lambda, it lists:\n\n| Language | Reported processing time |\n|---|---|\n| Python | 12,000 ms |\n| Go | 3,200 ms |\n| Rust | 2,050 ms |\n| C# NativeAOT | 2,050 ms |\n\nAgain, these results are workload-specific. The intended point is that modern C# should not automatically be treated as a slow enterprise runtime.\n\nThe Bun discussion returns here.\n\nDynamic languages frequently discover certain classes of errors only when a code path is executed:\n\nC# cannot eliminate every runtime failure, but it can move many problems earlier through:\n\nThis matters because production infrastructure becomes expensive when errors appear only after deployment.\n\nGo also catches many type errors at compile time, but the article emphasizes that C# combines these checks with a richer application framework and lifecycle model.\n\nThe article presents C# as a recurring first-class language across Microsoft’s AI and agent stack.\n\nIts timeline includes:\n\nIt also states that more than 10,000 organizations use Azure AI Foundry Agent Service, citing examples such as KPMG, BMW and Fujitsu.\n\nThe larger point is that C# developers are not accessing the Microsoft AI ecosystem through an afterthought or secondary binding. They are participating through one of the stack’s primary languages.\n\nThe article defines total inference cost as more than model computation:\n\nA system that generates tokens quickly may still be expensive if it requires:\n\n| Cost area | Python | Go | C# |\n|---|---|---|---|\n| Container image | About 1.2 GB | About 15 MB | About 45 MB |\n| Cold start | 3–10 seconds in larger stacks | Under 100 ms | Under 100 ms |\n| Concurrency | Often uses multiple processes around the GIL | Goroutines | Async runtime and thread pool |\n| Runtime errors | Frequently discovered in production | Explicit error handling | More opportunities for compile-time detection |\n| Observability | Often assembled from third-party components | Usually configured manually | OpenTelemetry and Aspire integration |\n| Kubernetes deployment | Commonly hand-maintained YAML | Commonly hand-maintained YAML | Aspire can generate deployment resources |\n\nThe article argues that [TensorSharp](https://github.com/zhongkaifu/TensorSharp) changes the image-generation cost model by placing inference inside a smaller and more manageable C# service stack.\n\nIt specifically contrasts:\n\nThis is presented as the economic foundation for a proposed component called TokenHub, which would track and manage the cost of AI operations.\n\nThe article proposes a layered architecture rather than rewriting every AI algorithm in C#.\n\n```\nPython algorithm layer\n- PyTorch training\n- Jupyter experimentation\n- Existing research ecosystem\n\n             ↓\n\nMCP protocol boundary\n- Cross-language service interface\n\n             ↓\n\nC# AI-native infrastructure layer\n- TensorSharp for image and text inference\n- MetaSkill DAG for workflow orchestration\n- Harness runtime for execution\n- TokenHub for cost tracking\n- AxonHub for data collection and CDC\n- Semantic Kernel for LLM orchestration\n- Microsoft Agent Framework for agent lifecycle\n- ONNX Runtime C# APIs for general inference\n\n             ↓\n\n.NET runtime\n- NativeAOT\n- Managed memory\n- Low-level performance APIs\n\n             ↓\n\nLifecycle-management layer\n- .NET Aspire\n- OpenTelemetry\n- EF Core\n```\n\nThe architecture follows three principles.\n\nThe proposal does not attempt to rewrite PyTorch training, research notebooks or every scientific package.\n\nInstead, Python capabilities can be exposed as services across an MCP boundary.\n\nThe C# layer handles orchestration, persistence, observability, deployment, lifecycle management and selected inference engines.\n\n[TensorSharp](https://github.com/zhongkaifu/TensorSharp) is used as the primary example of C# implementing a performance-critical engine rather than merely calling a separate C++ executable.\n\nThe second half of the article moves beyond language selection.\n\nIt asks what happens when AI and modern frameworks make engine construction accessible to many more developers.\n\nThe proposed progression is:\n\n```\nBuilder → AI Agent Leader → Taste\n```\n\nHistorically, building an inference engine required knowledge of:\n\nThe article argues that projects such as [TensorSharp](https://github.com/zhongkaifu/TensorSharp), combined with Aspire, Semantic Kernel and Microsoft Agent Framework, reduce the amount of specialized knowledge required to turn an idea into a working AI service.\n\nThe important shift is not that engineering disappears.\n\nIt is that writing code becomes a means rather than the defining identity of the role.\n\nAs AI generates more implementation code, humans increasingly focus on:\n\nFor example, an AI marketing-image system might use:\n\nThe human role is not merely to fix generated code.\n\nThe human decides whether the system solves the correct business problem, follows the intended brand style and remains within acceptable cost and risk boundaries.\n\nThe article defines Taste as more than personal preference.\n\nTaste is structured judgment about quality, value and boundaries.\n\nWhen an AI system can propose many architectures, human judgment selects the design that balances:\n\nThe article uses [TensorSharp](https://github.com/zhongkaifu/TensorSharp) PR #81 as an example: decisions about DiT reconstruction and CUDA Graph Capture are not simply binary matters of right and wrong. They involve trade-offs among speed, memory and complexity.\n\nWhen AI can generate unlimited features, someone still has to decide:\n\nWhen AI can generate almost any content or action, humans must define boundaries around:\n\nThe article’s position is that automation can free humans from repetitive execution, but it cannot eliminate the need to decide what should exist.\n\nThis is one of the article’s most important disclaimers:\n\n**The Taste-gate system described below is a design proposal. It has not yet been implemented in the** **OpenClaw.NET****repository.**\n\nAccording to the article, [OpenClaw.NET](http://OpenClaw.NET) already contains passive or safety-oriented governance capabilities such as:\n\n`user_input`\n\npause pointsThese mechanisms can expose plans, evidence, risks and approval records for inspection.\n\nHowever, most of them do not actively stop an agent workflow based on product quality, aesthetics or broader value judgments.\n\nThe article proposes adding concepts such as:\n\n`TasteGate`\n\n`ITasteGate<TInput, TOutput>`\n\ninterface`TasteDecision`\n\nresult`BrandTaste`\n\n, `EthicalTaste`\n\nand `TechnicalTaste`\n\nThe gate would produce one of three outcomes:\n\nThis is more useful than a simple approve/reject model because many AI outputs are not fundamentally invalid; they merely need another iteration.\n\nThe Agent Leader translates business intent into explicit constraints.\n\nPossible outputs include:\n\nThe AI system performs the work through an orchestrated workflow:\n\nKey outputs are evaluated against:\n\nThe final result is Pass, Retry or Abort.\n\nThe article presents a conceptual workflow like this:\n\n```\nNatural-language user request\n        ↓\nBrand-Taste constraints\n- Technology-oriented blue palette\n- Minimalist visual language\n- No human figures\n        ↓\nCost constraint\n- No more than $0.50 per generation\n        ↓\nAgent workflow\n- Prompt optimization through Semantic Kernel\n- Image generation through TensorSharp and CUDA\n- CLIP or aesthetic-quality evaluation\n- TokenHub cost calculation\n        ↓\nTaste gate\n- Technical review\n- Product and brand review\n- Ethical and copyright review\n        ↓\nPass  → return the image and cost report\nRetry → revise the prompt and regenerate, up to a fixed limit\nAbort → record the failure, raise an alert and request human review\n```\n\nThe proposal suggests placing gates according to two factors:\n\nLow-impact, low-uncertainty decisions can remain autonomous.\n\nHigh-impact, high-uncertainty decisions should require direct human involvement.\n\nThe article proposes expressing some constraints as C# types rather than keeping everything in prompts or informal documentation.\n\nA conceptual `BrandTaste`\n\nrecord could contain fields such as:\n\nA generic Taste-gate interface could require both its input and output to implement an auditable contract.\n\nThis would not make aesthetic judgment fully compile-time enforceable. A compiler cannot objectively determine whether an image is beautiful.\n\nHowever, the type system can enforce that:\n\nThe article describes this philosophy as **“Taste as types.”**\n\nThe goal is to move as much governance as possible away from undocumented runtime behavior and into explicit, inspectable contracts.\n\nThe article presents the following illustrative comparison:\n\n| Capability | Current AI agent | Human Agent Leader | Expected relationship |\n|---|---|---|---|\n| Technical execution | 9/10 | 7/10 | AI executes; human decides |\n| Product insight | 7/10 | 9/10 | AI assists; human leads |\n| Ethical sensitivity | 4/10 | 9/10 | AI assists; human leads |\n| Systems thinking | 8/10 | 9/10 | AI assists; human leads |\n| Aesthetic intuition | 3/10 | 9/10 | AI assists; human leads |\n| Risk awareness | 6/10 | 9/10 | AI assists; human leads |\n| Ability to anticipate evolution | 5/10 | 8/10 | AI assists; human leads |\n\nThese numbers are conceptual rather than scientific measurements.\n\nThey express the article’s belief that AI may exceed humans at implementation while remaining weaker at value judgments that depend on culture, responsibility, long-term context and lived experience.\n\nThe proposed progression is:\n\n| Level | Role | Primary capability | Typical tools | Main output |\n|---|---|---|---|---|\n| Level 1 | Builder | Coding, debugging and optimization | IDE, Git and CI/CD | Features |\n| Level 2 | Agent Operator | Prompting and agent configuration | Semantic Kernel and AutoGen | Agent efficiency |\n| Level 3 | Agent Leader | Problem definition, tool selection, orchestration and review | MetaSkill DAG, Harness and TokenHub | System-level value |\n| Level 4 | Taste Architect | Domain modeling, values, ethics and evolutionary direction | DDD, ontologies and typed Taste constraints | Organizational judgment |\n\nThe transition is described as:\n\nThe article concludes with a simple division of responsibilities.\n\nBun chose Rust because a JavaScript runtime requires strict memory control and deep native interoperability.\n\nGo remains an excellent language for cloud-native infrastructure.\n\nPython remains indispensable for AI research and training.\n\nThe article’s argument is that C# is increasingly occupying another high-value layer: the productionization, servicing, orchestration and operation of AI systems.\n\n[TensorSharp](https://github.com/zhongkaifu/TensorSharp) is presented as evidence that C# can also move downward into the inference-engine layer without giving up the broader lifecycle capabilities of .NET.\n\nBut the most important argument is ultimately not about language performance.\n\nAs implementation becomes easier, the human role changes:\n\nThe future is therefore not simply about replacing Python, Go, Rust or C++ with C#.\n\nIt is about using each language where it provides the most leverage—and using C# to build an integrated AI infrastructure layer that allows people to spend less time assembling operational plumbing and more time exercising judgment.\n\n**The long-term human advantage is not merely the ability to build. It is the ability to decide what is worth building.**", "url": "https://wpnews.pro/news/what-buns-rust-rewrite-tells-us-about-rebuilding-the-ai-infrastructure-layer-in", "canonical_source": "https://dev.to/zhongkaifu/what-buns-rust-rewrite-tells-us-about-rebuilding-the-ai-infrastructure-layer-in-c-14ch", "published_at": "2026-07-11 06:49:06+00:00", "updated_at": "2026-07-11 07:13:38.612995+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-infrastructure", "developer-tools", "machine-learning", "ai-products"], "entities": ["Bun", "Rust", "C#", "TensorSharp", "Zhongkai Fu", "Qwen", "stable-diffusion.cpp", ".NET"], "alternates": {"html": "https://wpnews.pro/news/what-buns-rust-rewrite-tells-us-about-rebuilding-the-ai-infrastructure-layer-in", "markdown": "https://wpnews.pro/news/what-buns-rust-rewrite-tells-us-about-rebuilding-the-ai-infrastructure-layer-in.md", "text": "https://wpnews.pro/news/what-buns-rust-rewrite-tells-us-about-rebuilding-the-ai-infrastructure-layer-in.txt", "jsonld": "https://wpnews.pro/news/what-buns-rust-rewrite-tells-us-about-rebuilding-the-ai-infrastructure-layer-in.jsonld"}}