{"slug": "qvac-building-local-first-peer-to-peer-ai-applications-and-systems", "title": "QVAC: Building local-first, peer-to-peer AI applications and systems", "summary": "QVAC, an open-source ecosystem for building local-first, peer-to-peer AI applications, has been released. It enables running AI tasks like LLMs, speech, and RAG locally across platforms or via P2P inference, with an OpenAI-compatible API and full cross-platform support.", "body_md": "**QVAC** is an open-source, cross-platform ecosystem for building local-first, peer-to-peer **AI** applications and systems. With QVAC, you can run AI tasks like LLMs, speech, RAG, and more locally across Linux, macOS, Windows, Android, and iOS — or delegate inference to peers using its built-in P2P capabilities.\n\n**Local-first:** load AI models and perform inference on your own machine. No third-party APIs, SaaS, or cloud involved.**P2P:** build unstoppable internet systems — like BitTorrent, IPFS, and blockchain networks, but for AI.**Cross-platform:** consistent developer experience across hardware, operating systems, and JS runtime environments — write code once, run it everywhere.**OpenAI-compatible API:** integrate with the broader AI ecosystem.**Open source:** 100% free to use and modify — build on top, contribute back, be part of our community.\n\nQVAC is composed of JavaScript libraries and tools that converge in the JS SDK. *The SDK is the main entry point for using QVAC*. It is type-safe and exposes all QVAC capabilities through a unified interface. It runs on Node.js, [Bare runtime](https://bare.pears.com), and [Expo](https://expo.dev).\n\nAdditionally, QVAC provides a CLI with tools and an HTTP server that exposes an [ OpenAI-compatible API](https://platform.openai.com/docs/api-reference).\n\n*By implementing the OpenAI API format, QVAC can integrate with the broader AI ecosystem.*\n\nInstall the `@qvac/sdk`\n\nnpm package in your project. Then load models and run AI inference locally, or delegate inference to peers using the built-in P2P features.\n\n- Create the examples workspace:\n\n```\nmkdir qvac-examples\ncd qvac-examples\nnpm init -y && npm pkg set type=module\n```\n\n- Install the SDK:\n\n```\nnpm install @qvac/sdk\n```\n\n- Create the quickstart script:\n\n``` js\nimport { loadModel, LLAMA_3_2_1B_INST_Q4_0, completion, unloadModel, } from \"@qvac/sdk\";\ntry {\n    // Load a model into memory\n    const modelId = await loadModel({\n        modelSrc: LLAMA_3_2_1B_INST_Q4_0,\n        modelType: \"llm\",\n        onProgress: (progress) => {\n            console.log(progress);\n        },\n    });\n    // You can use the loaded model multiple times\n    const history = [\n        {\n            role: \"user\",\n            content: \"Explain quantum computing in one sentence\",\n        },\n    ];\n    const result = completion({ modelId, history, stream: true });\n    for await (const token of result.tokenStream) {\n        process.stdout.write(token);\n    }\n    // Unload model to free up system resources\n    await unloadModel({ modelId });\n}\ncatch (error) {\n    console.error(\"❌ Error:\", error);\n    process.exit(1);\n}\n```\n\n- Run the quickstart script:\n\n```\nnode quickstart.js\n```\n\n**Completion:** LLM inference for text generation and chat via.`qvac-fabric-llm.cpp`\n\n**Text embeddings:** vector embedding generation for semantic search, clustering, and retrieval, via`qvac-fabric-llm.cpp`\n\n.**Translation:** text-to-text neural machine translation (NMT), via`qvac-fabric-llm.cpp`\n\nand[Bergamot](https://browser.mt).**Transcription:** automatic speech recognition (ASR) for speech-to-text viaor`qvac-ext-lib-whisper.cpp`\n\n[NVIDIA Parakeet](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2).**Text-to-Speech:** speech synthesis for text-to-speech (TTS) via[ONNX Runtime](https://onnxruntime.ai).**OCR:** optical character recognition (OCR) for extracting text from images via ONNX runtime.**Image generation:** text-to-image generation via.`qvac-ext-stable-diffusion.cpp`\n\n**Fine-tuning:** adapting LLMs to domain-specific tasks via LoRA.**Multimodal:** LLM inference over text, images, and other media within a single conversation context.**RAG:** out-of-the-box retrieval-augmented generation workflow.\n\n**Delegated inference:** delegate inference to peers via the[Holepunch stack](https://holepunch.to), enabling resource sharing.**Fetch models:** download AI models from peers via the distributed model registry.**Blind relays:** connect peers across NATs/firewalls by routing traffic through relay nodes.\n\n**Plugin system**: build lean apps by including only required AI capabilities, and extend the SDK by plugging in custom capabilities.** Logging:**visibility into what's happening during loading, inference, and other operations.** Download Lifecycle:**pause and resume model downloads.** Sharded models:**download a model that is sharded into multiple parts.\n\nTip\n\nFor comprehensive QVAC documentation, see [https://docs.qvac.tether.io](https://docs.qvac.tether.io).\nThere, you'll find [the compatibility matrix, installation instructions per environment/platform](https://docs.qvac.tether.io/sdk/getting-started/installation/), [reference with code examples for using each functionality](https://docs.qvac.tether.io/sdk/getting-started/), and much more.\n\nMonorepo structure overview. All QVAC components live under `/packages`\n\n, including the SDK, libraries, and tooling. Not every component is published to npm.\n\nLegend:\n\n**Core:** foundational building blocks shared across the ecosystem.**Addon:** capability packages — each QVAC capability is implemented by one or more addons.**SDK:** primary entry point for consumers.**Tool:** user-facing tools and services that support the ecosystem.\n\n| Package | Description | Category |\n|---|---|---|\n| sdk | Main entry point to develop AI applications with QVAC | SDK |\n| lib-decoder-audio | Audio decoder library leveraging FFmpeg for efficient audio decoding as preprocessing step for other addons | Addon |\n| lib-infer-llamacpp-embed | Native C++ addon for running text embedding models to generate high-quality contextual embeddings via `qvac-fabric-llm.cpp` |\nAddon |\n| lib-infer-llamacpp-llm | Native C++ addon for running Large Language Models (LLMs) via `qvac-fabric-llm.cpp` |\nAddon |\n| diffusion-cpp | Native C++ addon for text-to-image generation via `qvac-ext-stable-diffusion.cpp` |\nAddon |\n| lib-infer-nmtcpp | Native C++ addon for translation using either `qvac-fabric-llm.cpp` or\n|\nAddon |\n| lib-infer-onnx | Bare addon for ONNX Runtime session management | Addon |\n| lib-infer-onnx-tts | Text-to-Speech (TTS) library using Chatterbox and Supertonic neural TTS model via ONNX Runtime | Addon |\n| lib-infer-parakeet | High-performance speech-to-text inference addon using via NVIDIA/Parakeet | Addon |\n| transcription-whispercpp | Library for running Whisper transcription model for audio transcription via `qvac-ext-lib-whisper.cpp` |\nAddon |\n| inference-addon-cpp | Header-only C++ library providing common abstractions and infrastructure for building high-performance inference addons | Addon |\n| langdetect-text | Language detection library providing interface for detecting language of given text | Addon |\n| langdetect-text-cld2 | Language detection using CLD2 with same API as @qvac/langdetect-text | Addon |\n| ocr-onnx | Optical Character Recognition (OCR) addon using ONNX Runtime | Addon |\n| rag | JavaScript library for Retrieval-Augmented Generation (RAG) with document ingestion, vector search, and LLM integration | Addon |\n| dl-base | Base class for QVAC dataloader libraries providing common interface for loading data from various sources | Core |\n| dl-filesystem | Data loading library for loading model weights and resources from local filesystem | Core |\n| dl-hyperdrive | Data loading library for loading model weights and resources from Hyperdrive distributed file system | Core |\n| error | Standardized error handling capabilities for all QVAC libraries | Core |\n| infer-base | Base class for inference addon clients defining common lifecycle and generic methods for model interaction | Core |\n| logging | Logger wrapper that normalizes logging interface across QVAC libraries | Core |\n| cli | Command-line interface for the QVAC ecosystem with tooling for building, bundling, and managing QVAC-powered applications | Tool |\n| diagnostics | Diagnostic report generation library for QVAC | Tool |\n| lib-registry-server | Distributed model registry for downloading AI models for local inference and contributing new models | Tool |\n| lint-cpp | Configuration files for formatting and linting C++ source files with pre-commit hooks | Tool |\n\n- For the standard development workflow used in this monorepo, see\n.`/docs/gitflow.md`\n\n- For development specifics of each QVAC component, refer to the documentation in the respective subdirectory under\n`/packages`\n\n. - For the QVAC architecture as a whole, see\n`/docs/architecture`\n\n.\n\nBuilt something with QVAC? Add a badge or banner to your README, website, or app. It is a simple way to highlight your project, help others discover QVAC, and strengthen our community.\n\nBy using these badges and banners, you help foster the QVAC ecosystem!\n\nChoose a banner or badge below and copy its Markdown snippet, or copy its image URL and use the hosted SVG asset directly.\n\nLarge format badges (240x60) for prominent placement in your README header.\n\n**Banner usage**\n\n```\n[![Built with QVAC](https://raw.githubusercontent.com/tetherto/qvac/refs/heads/main/docs/branding/qvac-banner-dark-glow.svg)](https://github.com/tetherto/qvac)\n```\n\nCompact badges for use alongside other shields/badges in your README.\n\n**Compact**\n\n| Variant | Dark bg | Light bg |\n|---|---|---|\n| Green logo | ||\n| Monochrome |\n\n**Inline**\n\n| Variant | Dark bg | Light bg |\n|---|---|---|\n| Green logo | ||\n| Monochrome |\n\n**Badge usage**\n\n```\n[![Built with QVAC](https://raw.githubusercontent.com/tetherto/qvac/refs/heads/main/docs/branding/qvac-badge-green-dark.svg)](https://github.com/tetherto/qvac)\n```\n\n", "url": "https://wpnews.pro/news/qvac-building-local-first-peer-to-peer-ai-applications-and-systems", "canonical_source": "https://github.com/tetherto/qvac", "published_at": "2026-06-26 15:26:38+00:00", "updated_at": "2026-06-26 15:35:55.194983+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "ai-infrastructure", "ai-agents"], "entities": ["QVAC", "Holepunch", "Bare", "Expo", "OpenAI", "NVIDIA", "ONNX Runtime", "Bergamot"], "alternates": {"html": "https://wpnews.pro/news/qvac-building-local-first-peer-to-peer-ai-applications-and-systems", "markdown": "https://wpnews.pro/news/qvac-building-local-first-peer-to-peer-ai-applications-and-systems.md", "text": "https://wpnews.pro/news/qvac-building-local-first-peer-to-peer-ai-applications-and-systems.txt", "jsonld": "https://wpnews.pro/news/qvac-building-local-first-peer-to-peer-ai-applications-and-systems.jsonld"}}