{"slug": "show-hn-jinfer-ai-inference-engine-for-the-jvm-ai-in-a-jar", "title": "Show HN: Jinfer – AI inference engine for the JVM. AI in a jar", "summary": "Quixotic AI released jinfer, an early CPU-only inference engine for the JVM that supports chat, vision, audio, embeddings, reranking, and TTS without a Python runtime, ONNX, Docker containers, sidecar processes, or IPC. The stack includes toknroll (pure-Java tokenizers), gguf/safetensors model-format readers, jam quantized matmul kernels the developer says are competitive with llama.cpp on CPU, and jota, a Tensor API targeting Java, C, CUDA, HIP, Metal, OpenCL, and Mojo. The release ships integrations for Spring AI and LangChain4j and is compatible with GraalVM Native Image, and the developer is seeking feedback on the API surface.", "body_md": "Hi HN, Some would call AI on the JVM quixotic. And so, Quixotic AI was born.\n\njinfer is an inference engine for the JVM: chat, vision, audio, embeddings, reranking, and TTS. No Python runtime, no ONNX, no Docker containers, no sidecar process, no IPC. Finally, AI in a jar.\n\nThe stack underneath is built for the JVM rather than bolted onto it:\n\n```\n    toknroll: pure-Java tokenizers, zero dependencies\n    gguf / safetensors: read and write llama.cpp and HuggingFace model formats\n    jam: quantized matmul kernels, competitive with llama.cpp on CPU\n    jota: Tensor API targeting Java, C, CUDA, HIP, Metal, OpenCL, and Mojo\n```\n\nIt ships with integrations for Spring AI and LangChain4j, and is compatible with GraalVM Native Image for low-overhead, self-contained binaries with millisecond startup.\nThis is an early release (CPU only): I'd especially like feedback on the API surface.\n\nSite: [https://qxotic.ai](https://qxotic.ai)\nRepo: [https://github.com/qxoticai/qxotic](https://github.com/qxoticai/qxotic)\n\nComments URL: [https://news.ycombinator.com/item?id=49708628](https://news.ycombinator.com/item?id=49708628)\n\nPoints: 1\n\n# Comments: 0", "url": "https://wpnews.pro/news/show-hn-jinfer-ai-inference-engine-for-the-jvm-ai-in-a-jar", "canonical_source": "https://qxotic.ai/", "published_at": "2026-09-15 06:42:32+00:00", "updated_at": "2026-09-15 07:02:23.928229+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "developer-tools", "large-language-models", "ai-products"], "entities": ["Quixotic AI", "jinfer", "JVM", "toknroll", "jam", "jota", "Spring AI", "LangChain4j"], "alternates": {"html": "https://wpnews.pro/news/show-hn-jinfer-ai-inference-engine-for-the-jvm-ai-in-a-jar", "markdown": "https://wpnews.pro/news/show-hn-jinfer-ai-inference-engine-for-the-jvm-ai-in-a-jar.md", "text": "https://wpnews.pro/news/show-hn-jinfer-ai-inference-engine-for-the-jvm-ai-in-a-jar.txt", "jsonld": "https://wpnews.pro/news/show-hn-jinfer-ai-inference-engine-for-the-jvm-ai-in-a-jar.jsonld"}}