# Show HN: TensorSharp: Open-Source Local LLM Inference Engine

> Source: <https://tensorsharp.ai/>
> Published: 2026-07-10 02:42:07+00:00

# TensorSharp

A **native .NET LLM inference engine for GGUF models** — with a command-line tool, a browser chat server, and **Ollama- & OpenAI-compatible APIs** for programmatic access.

Everything runs on **your own hardware**: your laptop, workstation, or server. No data leaves the machine, there are no per-token fees, and the same engine powers a quick command-line test, a shared internal chatbot, and a production REST endpoint. This wiki is the complete guide — pick a starting point below or use `/` to search.

## Explore the wiki

[🚀](getting-started.html)

### Getting Started

Prerequisites, build, download a model, and stream your first reply.

[⌨️](cli.html)

### Command Line

Run prompts, images, audio, batches, and benchmarks from the CLI.

[🌐](server.html)

### Server & Web UI

Host a browser chatbot and HTTP endpoints on localhost.

[🔌](http-api.html)

### HTTP API

Call it from curl, Python, or any Ollama/OpenAI client.

[🧩](code-api.html)

### C# Library

Embed the engine directly in your .NET application.

[📚](api-reference.html)

### API Reference

Searchable tables of flags, env vars, endpoints, and types.

[🧠](models.html)

### Models

Supported architectures, downloads, multimodal, and reasoning.

[📖](glossary.html)

### Glossary & FAQ

New to LLMs? Plain-language definitions and common questions.

## Quick start in ~30 seconds

After installing the [.NET 10 SDK](getting-started.html#prerequisites), you are four commands away from a streaming reply (model download aside).

-
### Clone & build

The native GGML library compiles automatically on the first build.

```
git clone https://github.com/zhongkaifu/TensorSharp.git
cd TensorSharp
dotnet build TensorSharp.slnx -c Release
```

-
### Download a model

A small, well-tested starting point is

**Gemma-4-E4B (Q8_0)** from Hugging Face. More in[Model downloads](models.html#downloads). -
### Run it

Pick the

for your hardware.`--backend`

```
echo "Explain mixture-of-experts in one sentence." > prompt.txt

# macOS (Apple Silicon)
./TensorSharp.Cli --model gemma-4-E4B-it-Q8_0.gguf --input prompt.txt --backend ggml_metal

# Windows / Linux + NVIDIA
./TensorSharp.Cli --model gemma-4-E4B-it-Q8_0.gguf --input prompt.txt --backend ggml_cuda
```

-
### Prefer a UI + API?

Start the server and open the browser chat — it also serves the compatibility endpoints.

```
./TensorSharp.Server --model gemma-4-E4B-it-Q8_0.gguf --backend ggml_metal
# open http://localhost:5000
```

## Why TensorSharp?

### Private by default

Inference happens on your hardware. Prompts, documents, and images never leave the machine.

### No per-token bill

Run as much as your hardware allows — predictable cost, no metered API.

### Drop-in compatible

Speaks the Ollama and OpenAI wire formats, so existing tools and SDKs just work.

### Runs anywhere

NVIDIA (CUDA), AMD / Intel / NVIDIA (Vulkan), Apple Silicon (Metal/MLX), or pure CPU — with automatic fallbacks.

### Modern model support

Gemma, Qwen, GPT-OSS, Nemotron-H, Mistral, plus vision, audio, reasoning & tools.

### Built in .NET

A native C# engine you can embed in your apps, not just a black-box binary.

[🏁](benchmarks.html#head-to-head)

### Benchmarked vs llama.cpp

On identical GGUF files and the same GPU it trades wins with the C++ engine: the 26B-A4B MoE prefills 1.32× faster with first tokens 1.30× sooner, 12B wins or ties every decode scenario (1.17×), and JSON-mode decode streams 7.7× faster on E4B.

## Who is this for?

TensorSharp serves a wide range of visitors. Here is the fastest path for each.

#### Beginners & students

Start with the [Glossary & FAQ](glossary.html), then [Getting Started](getting-started.html).

#### Developers

Jump to the [HTTP API](http-api.html), [C# Library](code-api.html), and [API Reference](api-reference.html).

#### Senior / principal engineers

Read [Advanced Features](advanced.html) — paged KV, continuous batching, speculative decoding.

#### Managers, CTOs & CEOs

See the [business value](overview.html#business) and [capability matrix](overview.html#status).

#### Sales & marketing

Use the [feature catalog](features.html) and [benchmarks](benchmarks.html) for positioning.

#### Researchers & professors

Explore [model architectures](models.html) and the [head-to-head benchmarks](benchmarks.html#head-to-head).