Run Amazon Bedrock locally, with real completions from Ollama

MiniStack 1.4.0 introduces four new services that emulate Amazon Bedrock end-to-end, providing deterministic mock responses for Converse and InvokeModel APIs. A new environment variable allows the mock to be replaced with a real LLM backend via an OpenAI-compatible endpoint, enabling local development with actual completions from models like Ollama.

MiniStack 1.4.0 ships four new services that emulate Amazon Bedrock end to end, and one env var that turns the mock into a real LLM backend. Out of the box, Converse and InvokeModel return deterministic mock responses whose wire shape matches the model family you asked for. Anthropic, Titan, Nova, Llama, Mistral, Cohere and AI21 each have distinct response schemas, and MiniStack selects the right one by model ID prefix. Streaming operations ConverseStream, InvokeModelWithResponseStream use the real eventstream wire format, so your SDK's streaming parser runs exactly as against AWS. That means your test suite exercises the real code paths: request building, response parsing, stream handling. No network call, no API bill. One honest caveat: token counts are a chars/4 heuristic. Shape-correct, but don't assert exact values against a mock. docker run -p 4566:4566 \ -e MINISTACK BEDROCK PROXY URL=http://host.docker.internal:11434/v1 \ ministackorg/ministack With the proxy set to any OpenAI-compatible /chat/completions endpoint Ollama, llama.cpp, vLLM , MiniStack translates your Bedrock prompt to OpenAI shape, forwards it, and translates the response back into Bedrock shape: bedrock = boto3.client "bedrock-runtime", endpoint url="http://localhost:4566", region name="us-east-1", aws access key id="test", aws secret access key="test" resp = bedrock.converse modelId="anthropic.claude-3-5-sonnet-20240620-v1:0", messages= {"role": "user", "content": {"text": "Hello from MiniStack "} } , print resp "output" "message" "content" 0 "text" a real completion Unchanged AWS SDK. Real model. Zero cloud. If the proxy endpoint is unreachable, MiniStack falls back to the mock silently. Your CI doesn't care whether Ollama is running. All four services are account and region scoped 1.4.0 also shipped multi-region support, separate post . The Bedrock work was led by dcabib. docker run -p 4566:4566 ministackorg/ministack MIT, free forever. Changelog: https://ministack.org/blog/changelog-v1-4-0.html https://ministack.org/blog/changelog-v1-4-0.html