I built a tiny proxy that gives GLM 5.2 vision (or any text LLM) – MIT

Developer EekoSystems released VisionBridge, an open-source proxy that gives text-only large language models vision capabilities by routing image analysis to a separate vision model. The MIT-licensed tool sits between chat interfaces and reasoning models, enabling targeted visual inspection, OCR, and document scanning without retraining. It supports multiple backends including LM Studio, Ollama, and vLLM, and is available via Docker or Python.

Give text-only LLMs vision through a tiny OpenAI-compatible proxy. VisionBridge sits between your chat UI and your models. It lets a reasoning model from LM Studio, Ollama, vLLM, Z.ai, OpenRouter, or any OpenAI-compatible backend inspect images by asking a separate vision model targeted questions. No training. No weights. Bring your own reasoning model and your own vision model. OpenWebUI / LibreChat / app | v VisionBridge at /v1/chat/completions | +-- reasoning model: LM Studio / Ollama / vLLM / GLM / DeepSeek | +-- vision model: LLaVA / Qwen-VL / GLM-V / GPT-4o-mini / any VLM VisionBridge receives an OpenAI-style chat request. If the request contains images, it stores the images, rewrites the prompt for the reasoning model, and lets the reasoning model call tools: look image id, question for targeted visual inspection ocr image id for text extraction scan image id for a tiled full-document text sweep of dense pages crop and look image id, box, question for zooming into a region small crops are automatically upscaled before the VLM sees them compare image id a, image id b, question for judging two images together The vision model does the seeing. The reasoning model does the thinking. Before the loop starts, every image gets a one-shot scene description from the vision model in parallel, cached by content , so the reasoning model begins with global context and spends its tool budget on details. Multiple tool calls in one turn run concurrently , and repeated images across a conversation are served from a content-addressed cache instead of being re-fetched and re-decoded. Tool calling uses the backend's native OpenAI function calling when available. In the default TOOL MODE=auto , VisionBridge probes the reasoning backend once and transparently falls back to a prompt-JSON protocol for backends and models that don't support tools — so it works with strong hosted models and weak local ones alike. The loop is built to survive imperfect models: hallucinated image ids, bad crop boxes, malformed JSON, and vision-backend hiccups are fed back to the reasoning model as observations so it can correct itself, and hitting the inspection limit triggers a final best-effort answer instead of an error. docker run -p 8080:8080 \ -e REASONING BASE URL=http://host.docker.internal:1234/v1 \ -e REASONING API KEY=lm-studio \ -e REASONING MODEL=local-model \ -e VISION BASE URL=http://host.docker.internal:11434/v1 \ -e VISION API KEY=ollama \ -e VISION MODEL=llava:13b \ ghcr.io/eekosystems/visionbridge:latest Then point your OpenAI-compatible client at: Base URL: http://localhost:8080/v1 API key: anything Model: visionbridge cd visionbridge python -m venv .venv source .venv/bin/activate Windows: .venv\Scripts\activate pip install -e ". dev " cp .env.example .env Windows: copy .env.example .env visionbridge serve --host 0.0.0.0 --port 8080 Check your backend configuration any time with: visionbridge doctor It verifies both backends are reachable, that the configured models exist, and that a test completion succeeds. All settings come from environment variables or .env : | Variable | Default | Purpose | |---|---|---| REASONING BASE URL | http://localhost:1234/v1 | Reasoning backend | REASONING API KEY | local | Reasoning API key | REASONING MODEL | local-model | Reasoning model id | VISION BASE URL | http://localhost:11434/v1 | Vision backend | VISION API KEY | local | Vision API key | VISION MODEL | llava:13b | Vision model id | BRIDGE MODEL NAME | visionbridge | Model name the proxy advertises | TOOL MODE | auto | auto , native , or prompt tool calling | MAX VISION CALLS | 8 | Reasoning turns before a forced final answer | REQUEST TIMEOUT SECONDS | 120 | Per-backend-call timeout | BACKEND MAX RETRIES | 2 | Retries on 429/5xx/transport errors | SCENE PRIMING | true | Describe each image once before the loop starts | ALLOW FILE URLS | false | Allow file:// image URLs | ALLOW PRIVATE URLS | false | Allow image fetches from private/internal hosts | MAX IMAGE BYTES | 20971520 | Max bytes per image | MAX IMAGES PER REQUEST | 16 | Max images per request | IMAGE CACHE ENTRIES | 64 | Content-addressed image cache size | ENABLE TRACES | true | Keep tool-loop traces for /v1/traces/{id} | BRIDGE API KEYS | empty | Comma-separated keys; when set, /v1/ requires one | EXTRA MODELS | empty | JSON map of extra model names to backend pairs | LOG JSON | false | Emit JSON log lines | One instance can serve several reasoning+vision pairs under different model names. Unspecified fields inherit from the default pair: EXTRA MODELS={"visionbridge-fast": {"reasoning": {"model": "qwen3:8b"}, "vision": {"model": "llava:7b"}}} or point EXTRA MODELS FILE at a JSON file . All names appear in GET /v1/models , and clients select a pair with the model field. - Open LM Studio. - Load your reasoning model. - Go to the Developer tab and start the local server. - Use the default base URL: REASONING BASE URL=http://localhost:1234/v1 REASONING API KEY=lm-studio REASONING MODEL=local-model LM Studio can also be used as the vision backend if you load a vision-capable model and expose it through the same OpenAI-compatible server. REASONING BASE URL=http://localhost:11434/v1 REASONING API KEY=ollama REASONING MODEL=qwen3:32b VISION BASE URL=http://localhost:11434/v1 VISION API KEY=ollama VISION MODEL=llava:13b REASONING BASE URL=http://localhost:8000/v1 REASONING API KEY=token-abc123 REASONING MODEL=Qwen/Qwen3-32B VISION BASE URL=http://localhost:8001/v1 VISION API KEY=token-abc123 VISION MODEL=Qwen/Qwen3-VL-8B-Instruct In OpenWebUI: php Admin Panel - Settings - Connections - OpenAI-compatible Base URL: http://host.docker.internal:8080/v1 API Key: anything Model: visionbridge VisionBridge intentionally exposes the small subset most clients need: GET /health add ?probe=true to verify both backends are reachable GET /v1/models POST /v1/chat/completions GET /v1/traces/{completion id} — the full tool-loop trace for a recent request: every tool call, its arguments, the vision model's observation, and timings. Invaluable for answering "why did it say that?" Responses include a usage block aggregated across every reasoning and vision call made for the request, and an X-Request-ID header for correlating logs. Streaming : image-free requests are streamed token-by-token straight from the reasoning backend. Requests with images stream too: SSE heartbeats keep the connection alive while tools run, and in native tool-calling mode the final answer is forwarded token-by-token as the reasoning model produces it prompt-JSON mode buffers the loop and streams the answer in chunks . Errors : backend failures return HTTP 502, invalid inputs 400. Auth : set BRIDGE API KEYS=key1,key2 to require a Bearer key on /v1/ endpoints /health stays open . For anything internet-facing, still prefer a reverse proxy with TLS in front. Image URL fetching is hardened by default: hosts resolving to private, loopback, or link-local addresses are rejected SSRF guard , redirects are validated hop-by-hop, downloads are size-capped, and payloads must decode as real images. Run VisionBridge behind your own reverse proxy / auth layer if you expose it beyond localhost — it deliberately ships without built-in auth. benchmarks/ contains a harness that scores VisionBridge against a direct VLM relaxed accuracy + ANLS on DocVQA-style JSONL task sets. See benchmarks/README.md /thomasunise/visionbridge/blob/main/benchmarks/README.md . Actively developed. See CHANGELOG.md /thomasunise/visionbridge/blob/main/CHANGELOG.md for what's new in 0.2.0. Next up: - published benchmark numbers against caption-only prompting - persistent on-disk image/primer cache - OpenWebUI and LibreChat screenshots Contributions welcome — see CONTRIBUTING.md /thomasunise/visionbridge/blob/main/CONTRIBUTING.md . MIT