Ollama local LLM: running models and monitoring them (metrics, logs, traces) A developer walkthrough shows how to install Ollama, serve local LLMs through its REST and OpenAI-compatible APIs, and monitor the runtime like a production service. The guide covers native timing fields in /api/generate responses (load_duration, eval_count, eval_duration for tokens-per-second throughput), availability probes via /api/tags and /api/ps, log collection, and client-side OpenTelemetry traces feeding into Dynatrace. It warns that Ollama has no built-in authentication and should sit behind an access-controlled reverse proxy, and notes vLLM is often a better fit for many concurrent users on datacentre GPUs. Running Ollama locally has become the simplest way into self-hosted LLMs: one command to install it, another to download a model, and an HTTP API ready to respond. It is one of the most followed open source projects on GitHub, used everywhere from a developer’s laptop to a team GPU server. As soon as an Ollama local LLM serves several people, it deserves to be monitored like any other service: metrics, logs and traces. This simplicity has a downside. As long as Ollama remains a personal tool, nobody wonders how long a model takes to load or why an answer takes twelve seconds to arrive. As soon as it becomes a service that other applications depend on, it has to be treated like any other production component: you need to know whether it responds, how fast, with what GPU memory consumption, and for which callers. This article introduces the project, installs it step by step, then shows how to monitor it: what the API exposes natively, how to collect the logs, how to produce OpenTelemetry traces on the client side and how all of this flows into an observability tool such as Dynatrace. Ollama is a language model runtime that wraps model download, quantisation and serving behind a command-line interface and a REST API. Internally it relies on llama.cpp and the GGUF format, which lets it run on CPU, on NVIDIA or AMD GPUs, and on Apple Silicon chips. Its strengths for a technical team: /api/generate , /api/chat , /api/tags … and an OpenAI-compatible API under /v1 , so existing SDKs can be reused; Its limits are just as clear: Ollama aims for simplicity more than maximum throughput. To serve many concurrent users on datacentre GPUs, a server such as vLLM in production https://span.observer/en/vllm-production-metrics/ is often a better fit. On Linux, the official script installs the binary and creates a systemd service: curl -fsSL https://ollama.com/install.sh | sh systemctl status ollama On macOS and Windows, a graphical installer is available. In a container, the official image is used like this add --gpus=all if the NVIDIA Container Toolkit is installed : docker run -d --name ollama \ -v ollama:/root/.ollama \ -p 11434:11434 \ ollama/ollama ollama pull llama3.2 ollama run llama3.2 "Explain what an SLO is in two sentences." ollama ps models loaded in memory, CPU or GPU ollama list models present on disk Available names and tags change constantly: check the model library on the project’s website before hard-coding a name in a script. By default, the server listens on port 11434 of the local machine. A non-streamed call returns, in addition to the text, timing information that is very useful for monitoring: curl -s http://localhost:11434/api/generate -d '{ "model": "llama3.2", "prompt": "Why is the sky blue?", "stream": false }' | jq '{total duration, load duration, prompt eval count, eval count, eval duration}' Durations are expressed in nanoseconds. load duration reveals the cost of a cold load; eval count divided by eval duration gives the generation throughput in tokens per second. To expose Ollama to other machines, the OLLAMA HOST variable sets the listening address. Other variables control how long a model stays in memory or the degree of parallelism. Their list and exact behaviour change from one version to the next: refer to the FAQ in the official Ollama documentation https://github.com/ollama/ollama/tree/main/docs . The API has no built-in authentication: put a reverse proxy with access control in front of it, and never expose it directly on the internet. Monitoring a local LLM relies on four families of signals. Ollama provides only some of them natively; the rest is built around it. A simple call to /api/tags or to the server root is enough as an availability probe. The /api/ps endpoint lists the loaded models, their size and the share placed in GPU memory: it is the API equivalent of ollama ps , easy to query from a synthetic monitor. To our knowledge, Ollama does not expose a native Prometheus endpoint at the time of writing; check the release notes, as the topic is regularly discussed. Useful metrics are therefore calculated from API responses total latency, load time, input and output tokens, throughput and from infrastructure metrics. On the GPU side, NVIDIA’s DCGM exporter or a host agent provides card utilisation, memory and temperature. Installed as a service, Ollama writes to the systemd journal; in a container, to standard output: journalctl -u ollama -f docker logs -f ollama These logs show, among other things, GPU detection, model loading and memory errors. Verbose mode is enabled through a debug variable described in the documentation. A collection agent OneAgent, or an OpenTelemetry Collector with the filelog or journald receiver centralises them. The richest data lives with the caller. The OpenLLMetry project publishes OpenTelemetry instrumentations for several providers, including Ollama, which produce one span per call with the model, the tokens consumed and the duration, following the GenAI semantic conventions. Example in Python: pip install ollama opentelemetry-sdk opentelemetry-exporter-otlp \ opentelemetry-instrumentation-ollama python from opentelemetry import trace from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import BatchSpanProcessor from opentelemetry.exporter.otlp.proto.http.trace exporter import OTLPSpanExporter from opentelemetry.instrumentation.ollama import OllamaInstrumentor import ollama provider = TracerProvider provider.add span processor BatchSpanProcessor OTLPSpanExporter trace.set tracer provider provider OllamaInstrumentor .instrument reponse = ollama.chat model="llama3.2", messages= {"role": "user", "content": "Hello"} print reponse "message" "content" Check the exact package and class names in the OpenLLMetry repository: these instrumentations change quickly. We compare this approach with Langfuse in our article on Langfuse and OpenLLMetry https://span.observer/en/langfuse-openllmetry-llm-observability/ . The OTLP exporter is configured through the standard OpenTelemetry variables. For direct sending to Dynatrace SaaS, the environment’s OTLP endpoint and an API token with the appropriate ingest permissions traces, metrics, logs are enough: export OTEL SERVICE NAME="assistant-interne" export OTEL EXPORTER OTLP ENDPOINT="https://