{"slug": "show-hn-modelship-an-open-source-ray-backed-model-cluster", "title": "Show HN: Modelship – an open source ray backed model cluster", "summary": "Modelship, an open source Ray-backed model cluster, was released on GitHub, scoring 17/17 on the Open Responses conformance suite according to its production readiness documentation. The project spins up a Ray cluster, deploys models via loaders including vllm, llamacpp, diffusers and sherpa_onnx, and serves an OpenAI responses-ready API on top of them, supporting CPU, Metal and CUDA with Prometheus metrics, a Grafana dashboard, Docker, Kubernetes or native installs, automatic model context sizing and MCP support.", "body_md": "Hey all, just wanted to share how my spare time has been spent for quite some time now.\n\nOk, this one started oddly. My aim initially was just to serve multiple models on one GPU and fire it up in my unraid machine.\n\nBy the time that first bit was done, I realised I could just add one more thing... and another... and well, from beginning of this year when, imho, AI became useful, the whole thing it kind of blew up.\n\nSo here's what it does:\n\n- spins a ray cluster\n\n- deploys models using various loaders (vllm, llamacpp, diffusers, sherpa_onnx, etc.)\n\n- serves an openai responses ready api on top of them\n\nNow some the details, already listed in my production readiness ([https://docs.model-ship.ai/production-readiness/#production-...](https://docs.model-ship.ai/production-readiness/#production-readiness)) plan:\n\n- scores 17/17 on the Open Responses conformance suite and it's got its own responses redis backed store\n\n- currently supports cpu, metal and cuda\n\n- it has prometheus metrics and grafana dashboard\n\n- can run under docker, k8s or simply native install\n\n- automatically sizes model context\n\n- has MCP support\n\nFeedback would be much appreciated.\n\n[https://github.com/modelship-ai/modelship](https://github.com/modelship-ai/modelship)\n\nComments URL: [https://news.ycombinator.com/item?id=50040528](https://news.ycombinator.com/item?id=50040528)\n\nPoints: 1\n\n# Comments: 0", "url": "https://wpnews.pro/news/show-hn-modelship-an-open-source-ray-backed-model-cluster", "canonical_source": "https://github.com/modelship-ai/modelship", "published_at": "2026-10-11 07:28:20+00:00", "updated_at": "2026-10-11 07:53:18.070490+00:00", "lang": "en", "topics": ["ai-infrastructure", "large-language-models", "ai-tools", "mlops", "agent-protocols"], "entities": ["Modelship", "Ray", "vllm", "llamacpp", "diffusers", "sherpa_onnx", "GitHub", "OpenAI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-modelship-an-open-source-ray-backed-model-cluster", "markdown": "https://wpnews.pro/news/show-hn-modelship-an-open-source-ray-backed-model-cluster.md", "text": "https://wpnews.pro/news/show-hn-modelship-an-open-source-ray-backed-model-cluster.txt", "jsonld": "https://wpnews.pro/news/show-hn-modelship-an-open-source-ray-backed-model-cluster.jsonld"}}