{"slug": "llm-manager-orchestrating-ollama-and-llama-cpp-with-pure-bash", "title": "LLM-Manager: Orchestrating Ollama and Llama.cpp with Pure Bash", "summary": "A developer built LLM-Manager, a lightweight Bash-based orchestration suite for managing local and remote LLM inference engines like Ollama and Llama.cpp across Linux and WSL2. The tool uses a modular plug-and-play architecture with a JSON/interactive dual interface, eliminating the need for complex Python scripts or Docker setups. It provides unified commands for starting, stopping, loading, and monitoring models while outputting both human-readable messages and machine-parsable JSON for automation.", "body_md": "**LLM-Manager** is a lightweight, modular Bash suite with a dual JSON/Interactive interface designed to manage local and remote inference engines across Linux and WSL2.\n\nWhen I started experimenting with Large Language Models (LLMs) to build an On-Premise RAG (Retrieval-Augmented Generation) application, I hit a massive roadblock: **environment fragmentation**.\n\nManaging multiple inference engines like Ollama and Llama.cpp meant memorizing different command-line flags, environment variables, and configurations. Once my frontend and backend prototypes were ready for testing, I realized I was spending too much time manually starting, stopping, loading, and unloading models.\n\nI looked online for solutions. Most people suggested complex Python scripts, heavy Docker setups, n8n workflows, or complicated web dashboards.\n\nI didn't want the bloat. I wanted something lightweight that executed commands as if I were doing them manually, but with zero cognitive load.\n\nThat is why I built **LLM-Manager**: a modular orchestration suite written entirely in pure Bash.\n\nChoosing Bash wasn't about being old-school; it was a pragmatic engineering decision:\n\n`npm install`\n\n, no runtime dependencies. It’s native and lightning-fast.The system is designed with a strict plug-and-play modular layout. At the center sits a single entry-point orchestrator (`engine-run.sh`\n\n) that validates arguments against whitelists and routes actions to engine-specific scripts.\n\n```\n.\n├── engine.conf               # Global configuration constants\n├── engine-models.json        # Model registry with per-engine metadata\n├── engine-templates.json     # Prompt/Model templates by family\n├── engine-run.sh             # Main orchestrator & entry-point\n├── engine-common.sh          # Shared utilities (OS detection, JSON formatting)\n├── engine-status.sh          # Cross-engine status aggregation\n├── engine-system.sh          # Hardware metric probing\n├── logs/                     # Centralized logs\n├── llama/                    # Llama.cpp backend scripts\n└── ollama/                   # Ollama backend scripts\n```\n\nEvery engine directory implements a consistent interface (`start.sh`\n\n, `stop.sh`\n\n, `status.sh`\n\n, `load.sh`\n\n, `unload.sh`\n\n, `show.sh`\n\n, `remove.sh`\n\n). If an engine doesn't support a specific action, a simple stub script that exits with `0`\n\nkeeps the pipeline happy.\n\nOne of the core features of LLM-Manager is how it handles output.\n\n`stderr`\n\n.`stdout`\n\n.This dual nature makes it perfect for local interactive use, but also means it acts as a local proxy. You can run it over **Remote SSH** and pipe the clean JSON straight into another monitoring script, custom Web UI, or automation tool.\n\nRunning `./engine-run.sh status`\n\nprobes the system metrics and queries active network ports, spitting out a comprehensive payload:\n\n```\n{\n  \"timestamp\": \"2026-05-29T06:38:30Z\",\n  \"status\": \"success\",\n  \"action\": \"status\",\n  \"engine\": \"all\",\n  \"data\": {\n    \"system\": {\n      \"os_type\": \"wsl\",\n      \"memory\": { \"total_mb\": 5927, \"available_mb\": 4555 },\n      \"gpu\": { \"detected\": true, \"name\": \"AMD Radeon(TM) Graphics\", \"vram_total_mb\": 512 },\n      \"cpu\": { \"cores\": 4, \"load_1m\": 1.11 }\n    },\n    \"engines\": {\n      \"ollama\": { \"state\": \"stopped\", \"port\": 1234 },\n      \"llama\": { \"state\": \"stopped\", \"port\": 12345 }\n    }\n  }\n}\n```\n\nIf a command fails or is called without parameters, the machine gets the JSON error contract, and the human operator gets a clean, human-readable usage menu:\n\n```\nError: LLM Manager\nUsage: engine-run.sh <action> [engine] [args...]\nactions:\n    config                           Global config\n    models [-h]                      List available models (-h human readable)\n    status <engine>                  Show global or engine status\n    start <engine> [model] [users]   Start an engine\n    stop <engine>                    Stop an engine\n...\n```\n\nManaging raw `.gguf`\n\nfiles on Ollama can be a chore since it requires a `Modelfile`\n\n. LLM-Manager abstracts this entirely in the backend via model loading strategies in `engine-models.json`\n\n.\n\nIf a model is configured with a `gguf`\n\nloader, the `load.sh`\n\nscript **dynamically generates the required Modelfile on the fly**, injecting correct prompt templates based on the model family, and loading it into Ollama seamlessly. It also supports `native`\n\nstrategies to pull directly from the official Ollama registry, or `auto`\n\nto fallback if the local file is missing.\n\nThe project is fully open-source. If you want to see how the WSL2/PowerShell bridges are handled, how the dynamic Modelfiles are generated, or if you want to use it to clean up your own local LLM testing environment, check out the repository:\n\nA lightweight, modular Bash orchestration suite to manage, start, stop, and monitor local and remote LLM inference engines (Ollama, Llama.cpp) with a dual interactive/JSON interface.\n\nDeveloped primarily to solve the complexity of managing ibrid environments (like Windows hosts from WSL2) and remote deployments via SSH without the overhead of heavy Python or dashboard solutions.\n\nBefore running the orchestrator, ensure your environment has the following tools installed:\n\n`jq`\n\n`sudo apt install jq`\n\non Debian/Ubuntu).`curl`\n\n`ollama`\n\nand `llama.cpp`\n\n(with `vLLM`\n\nplanned).I am currently working on completing the `vLLM`\n\nengine integration and refining the startup health-checks into proactive retry loops.\n\nLet me know what you think or if you've built similar lightweight alternatives for your AI workflows!", "url": "https://wpnews.pro/news/llm-manager-orchestrating-ollama-and-llama-cpp-with-pure-bash", "canonical_source": "https://dev.to/bumbulik0/llm-manager-orchestrating-ollama-and-llamacpp-with-pure-bash-181p", "published_at": "2026-05-29 08:16:54+00:00", "updated_at": "2026-05-29 08:42:35.847629+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "ai-infrastructure", "ai-products", "ai-startups"], "entities": ["LLM-Manager", "Ollama", "Llama.cpp", "Bash", "Linux", "WSL2", "RAG", "n8n"], "alternates": {"html": "https://wpnews.pro/news/llm-manager-orchestrating-ollama-and-llama-cpp-with-pure-bash", "markdown": "https://wpnews.pro/news/llm-manager-orchestrating-ollama-and-llama-cpp-with-pure-bash.md", "text": "https://wpnews.pro/news/llm-manager-orchestrating-ollama-and-llama-cpp-with-pure-bash.txt", "jsonld": "https://wpnews.pro/news/llm-manager-orchestrating-ollama-and-llama-cpp-with-pure-bash.jsonld"}}