{"slug": "ollama-local-llm-running-models-and-monitoring-them-metrics-logs-traces", "title": "Ollama local LLM: running models and monitoring them (metrics, logs, traces)", "summary": "A developer walkthrough shows how to install Ollama, serve local LLMs through its REST and OpenAI-compatible APIs, and monitor the runtime like a production service. The guide covers native timing fields in /api/generate responses (load_duration, eval_count, eval_duration for tokens-per-second throughput), availability probes via /api/tags and /api/ps, log collection, and client-side OpenTelemetry traces feeding into Dynatrace. It warns that Ollama has no built-in authentication and should sit behind an access-controlled reverse proxy, and notes vLLM is often a better fit for many concurrent users on datacentre GPUs.", "body_md": "Running Ollama locally has become the simplest way into self-hosted LLMs: one command to install it, another to download a model, and an HTTP API ready to respond. It is one of the most followed open source projects on GitHub, used everywhere from a developer’s laptop to a team GPU server. As soon as an Ollama local LLM serves several people, it deserves to be monitored like any other service: metrics, logs and traces.\n\nThis simplicity has a downside. As long as Ollama remains a personal tool, nobody wonders how long a model takes to load or why an answer takes twelve seconds to arrive. As soon as it becomes a service that other applications depend on, it has to be treated like any other production component: you need to know whether it responds, how fast, with what GPU memory consumption, and for which callers.\n\nThis article introduces the project, installs it step by step, then shows how to monitor it: what the API exposes natively, how to collect the logs, how to produce OpenTelemetry traces on the client side and how all of this flows into an observability tool such as Dynatrace.\n\nOllama is a language model runtime that wraps model download, quantisation and serving behind a command-line interface and a REST API. Internally it relies on llama.cpp and the GGUF format, which lets it run on CPU, on NVIDIA or AMD GPUs, and on Apple Silicon chips.\n\nIts strengths for a technical team:\n\n`/api/generate`, `/api/chat`, `/api/tags`…) and an OpenAI-compatible API under `/v1`, so existing SDKs can be reused;\nIts limits are just as clear: Ollama aims for simplicity more than maximum throughput. To serve many concurrent users on datacentre GPUs, a server such as [vLLM in production](https://span.observer/en/vllm-production-metrics/) is often a better fit.\n\nOn Linux, the official script installs the binary and creates a systemd service:\n\n```\ncurl -fsSL https://ollama.com/install.sh | sh\nsystemctl status ollama\n```\n\nOn macOS and Windows, a graphical installer is available. In a container, the official image is used like this (add `--gpus=all` if the NVIDIA Container Toolkit is installed):\n\n```\ndocker run -d --name ollama \\\n  -v ollama:/root/.ollama \\\n  -p 11434:11434 \\\n  ollama/ollama\nollama pull llama3.2\nollama run llama3.2 \"Explain what an SLO is in two sentences.\"\nollama ps     # models loaded in memory, CPU or GPU\nollama list   # models present on disk\n```\n\nAvailable names and tags change constantly: check the model library on the project’s website before hard-coding a name in a script.\n\nBy default, the server listens on port 11434 of the local machine. A non-streamed call returns, in addition to the text, timing information that is very useful for monitoring:\n\n```\ncurl -s http://localhost:11434/api/generate -d '{\n  \"model\": \"llama3.2\",\n  \"prompt\": \"Why is the sky blue?\",\n  \"stream\": false\n}' | jq '{total_duration, load_duration, prompt_eval_count, eval_count, eval_duration}'\n```\n\nDurations are expressed in nanoseconds. `load_duration` reveals the cost of a cold load; `eval_count` divided by `eval_duration` gives the generation throughput in tokens per second.\n\nTo expose Ollama to other machines, the `OLLAMA_HOST` variable sets the listening address. Other variables control how long a model stays in memory or the degree of parallelism. Their list and exact behaviour change from one version to the next: refer to the FAQ in the [official Ollama documentation](https://github.com/ollama/ollama/tree/main/docs). The API has no built-in authentication: put a reverse proxy with access control in front of it, and never expose it directly on the internet.\n\nMonitoring a local LLM relies on four families of signals. Ollama provides only some of them natively; the rest is built around it.\n\nA simple call to `/api/tags` or to the server root is enough as an availability probe. The `/api/ps` endpoint lists the loaded models, their size and the share placed in GPU memory: it is the API equivalent of `ollama ps`, easy to query from a synthetic monitor.\n\nTo our knowledge, Ollama does not expose a native Prometheus endpoint at the time of writing; check the release notes, as the topic is regularly discussed. Useful metrics are therefore calculated from API responses (total latency, load time, input and output tokens, throughput) and from infrastructure metrics. On the GPU side, NVIDIA’s DCGM exporter or a host agent provides card utilisation, memory and temperature.\n\nInstalled as a service, Ollama writes to the systemd journal; in a container, to standard output:\n\n```\njournalctl -u ollama -f\ndocker logs -f ollama\n```\n\nThese logs show, among other things, GPU detection, model loading and memory errors. Verbose mode is enabled through a debug variable described in the documentation. A collection agent (OneAgent, or an OpenTelemetry Collector with the `filelog` or `journald` receiver) centralises them.\n\nThe richest data lives with the caller. The OpenLLMetry project publishes OpenTelemetry instrumentations for several providers, including Ollama, which produce one span per call with the model, the tokens consumed and the duration, following the GenAI semantic conventions. Example in Python:\n\n```\npip install ollama opentelemetry-sdk opentelemetry-exporter-otlp \\\n  opentelemetry-instrumentation-ollama\npython\nfrom opentelemetry import trace\nfrom opentelemetry.sdk.trace import TracerProvider\nfrom opentelemetry.sdk.trace.export import BatchSpanProcessor\nfrom opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter\nfrom opentelemetry.instrumentation.ollama import OllamaInstrumentor\nimport ollama\n\nprovider = TracerProvider()\nprovider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))\ntrace.set_tracer_provider(provider)\nOllamaInstrumentor().instrument()\n\nreponse = ollama.chat(model=\"llama3.2\",\n                      messages=[{\"role\": \"user\", \"content\": \"Hello\"}])\nprint(reponse[\"message\"][\"content\"])\n```\n\nCheck the exact package and class names in the OpenLLMetry repository: these instrumentations change quickly. We compare this approach with Langfuse in [our article on Langfuse and OpenLLMetry](https://span.observer/en/langfuse-openllmetry-llm-observability/).\n\nThe OTLP exporter is configured through the standard OpenTelemetry variables. For direct sending to Dynatrace SaaS, the environment’s OTLP endpoint and an API token with the appropriate ingest permissions (traces, metrics, logs) are enough:\n\n```\nexport OTEL_SERVICE_NAME=\"assistant-interne\"\nexport OTEL_EXPORTER_OTLP_ENDPOINT=\"https://<environment>.live.dynatrace.com/api/v2/otlp\"\nexport OTEL_EXPORTER_OTLP_HEADERS=\"Authorization=Api-Token%20<token>\"\nexport OTEL_EXPORTER_OTLP_PROTOCOL=\"http/protobuf\"\n```\n\nIn a more serious deployment, it is better to go through an OpenTelemetry Collector, which receives traces and logs, adds host and Kubernetes attributes, then exports to the backend. The spans then appear in the distributed traces view, attached to the calling service, and the host running Ollama is correlated with its GPU metrics. The principle is the same as the one described in [end-to-end OpenTelemetry](https://span.observer/en/opentelemetry-end-to-end-dynatrace/): the value comes from the link between the model call, the service and the infrastructure.\n\nFew dashboards are really useful: p50 and p95 latency per model, tokens generated per minute, share of calls with a cold load, GPU memory used, error rate. One alert on p95 latency and another on memory saturation cover the essentials.\n\nFor internal use, prototypes or sensitive data, yes, provided you accept smaller models and size the hardware appropriately. For heavy concurrent traffic, a throughput-oriented server is preferable.\n\nNot natively, to our knowledge. You combine the durations returned by the API, the application’s OpenTelemetry traces and the host’s GPU metrics. Check the latest releases, as this may change.\n\nThe `ollama ps` command or the `/api/ps` endpoint shows the CPU/GPU split for each loaded model; the startup logs also show the GPUs detected.\n\nOllama makes local inference almost trivial, but a shared LLM is still a production service. By combining API probes, centralised logs, GPU metrics and application-side OpenTelemetry traces, you get a complete picture, ready to be used in Dynatrace or another backend. To see where Ollama fits among the other building blocks, read our [comparison of self-hosted open source AI stacks](https://span.observer/en/self-hosted-open-source-ai-stack/), and to add a chat interface to it, our [Open WebUI](https://span.observer/en/self-hosted-open-webui-chatgpt/) guide.\n\n*span.observer is an independent community blog, not affiliated with Dynatrace LLC or the projects mentioned. Trademarks belong to their respective owners. Tools change quickly: check commands and options in their official documentation.*\n\n*Originally published on [span.observer](https://span.observer/en/ollama-local-llm-monitoring/). French version: [https://span.observer/fr/ollama-llm-local-supervision/](https://span.observer/fr/ollama-llm-local-supervision/).*", "url": "https://wpnews.pro/news/ollama-local-llm-running-models-and-monitoring-them-metrics-logs-traces", "canonical_source": "https://dev.to/spanobserver/ollama-local-llm-running-models-and-monitoring-them-metrics-logs-traces-3no9", "published_at": "2026-11-22 21:30:20+00:00", "updated_at": "2026-11-22 21:42:05.308575+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "mlops", "ai-tools"], "entities": ["Ollama", "llama.cpp", "Dynatrace", "vLLM", "OpenTelemetry", "Llama 3.2", "NVIDIA", "AMD"], "also_reported_by": [], "takeaway": "Ollama's non-streamed /api/generate response returns nanosecond timings — load_duration for cold-load cost and eval_count/eval_duration for tokens-per-second throughput — that can feed availability probes on /api/tags and /api/ps plus client-side OpenTelemetry traces.", "recommendation": {"advice": "Monitor self-hosted Ollama with /api/tags and /api/ps probes and the nanosecond timing fields from non-streamed /api/generate calls, and put an access-controlled reverse proxy in front since the API has no built-in authentication.", "playbook": "production-ops", "playbook_url": "https://wpnews.pro/playbooks/production-ops"}, "alternates": {"html": "https://wpnews.pro/news/ollama-local-llm-running-models-and-monitoring-them-metrics-logs-traces", "markdown": "https://wpnews.pro/news/ollama-local-llm-running-models-and-monitoring-them-metrics-logs-traces.md", "text": "https://wpnews.pro/news/ollama-local-llm-running-models-and-monitoring-them-metrics-logs-traces.txt", "jsonld": "https://wpnews.pro/news/ollama-local-llm-running-models-and-monitoring-them-metrics-logs-traces.jsonld"}}