# Unusual parallel inference using consumer RTX rig > Source: > Published: 2026-06-15 18:37:48+00:00 I had a very brief look into it but it looks promising at first glance, Technical Report: Project Aegis (The Sentinel Module) **Subject:** Asymmetric Monitoring via Dedicated iGPU Micro-Inference **Scope:** iGPU Inference Stability, Model Selection for 8GB Constraints, and Logic Guardrail Definition. I. Scope of Observation: The “Small” Task Analysis To maximize the utility of a small model (under 3B parameters), we must move away from “creative” tasks and toward **Deterministic Monitoring**. The Sentinel does not need to think; it needs to *verify*. The following are specific, high-value functions for the iGPU-bound model: 1. Schema & Syntax Validation (The Gatekeeper) **JSON Integrity:** Ensuring the primary model’s output is valid JSON before it hits the system’s parser. If the 3090 misses a closing bracket or quote, the Sentinel catches it and requests a “Fix_Syntax” correction. **Regex Matching:** Verifying that specific strings (e.g., URLs, file paths, or email formats) are correctly structured. 2. Loop & Stutter Detection (The Pulse Check) **Token Repetition Monitoring:** Detecting when the primary model gets stuck in a “loop” (repeating the same phrase or sentence). **Stall Detection:** If the 3090 stops producing tokens for more than X seconds while in an active state, the Sentinel triggers a system heartbeat check. 3. Logic & Constraint Adherence (The Rulebook) **Instruction Drift:** Checking if the agent is still following the “System Prompt.” If the user asks for code and the agent begins to provide long-winded conversational filler, the Sentinel flags a “Context_Drift” warning. **Constraint Verification:** Ensuring the model hasn’t violated specific constraints (e.g., “Don’t use library X,” or “Keep response under 200 words”). 4. Safety & Content Filtering **Out-of-Bounds Detection:** A fast, local check to ensure the primary model isn’t hallucinating dangerous instructions or leaking system information into the user interface. II. Model Selection: The “Goldilocks” Zone (8GB / iGPU) Given the 8GB RAM allocation and the use of an Intel iGPU via Vulkan/OpenCL, we need models that are highly optimized for **quantized inference**. We want a model with high “Reasoning Density”—meaning it stays smart while being small enough to run at high speeds on system memory. Recommendation 1: Phi-3 Mini (3.8B) - *The Logic Powerhouse* **Why:** Microsoft’s Phi-3 is arguably the best-performing model under 4B parameters. It punches far above its weight in logical reasoning and instruction following. **Quantization:** A **Q4_K_M or Q5_K_M GGUF** version would sit comfortably in ~2.5GB to 3GB of VRAM/System RAM, leaving plenty of “breathing room” for the system’s overhead and a large context window. **Suitability:** Perfect for complex logic checks like “Is this code valid?” Recommendation 2: Qwen-2 (1.8B or 7B) - *The Efficiency King* **Why:** The Qwen series is exceptionally well-optimized for small-scale inference. The **1.8B version** is incredibly fast and can be used as a “High-Speed Filter.” **Quantization:** A **Q8_0 GGUF** of the 1.8B model would use less than 2GB of space, making it lightning-fast on an iGPU. **Suitability:** Ideal if you want near-instantaneous “Gatekeeper” feedback for simple tasks like JSON verification. Recommendation 3: Gemma-2 (2B) - *The Balanced Choice* **Why:** Google’s Gemma-2 2B is highly polished and handles multi-step reasoning better than most other models in its weight class. **Suitability:** Excellent for “Sense Checking” the tone and intent of the primary model’s output. III. System Robustness & iGPU Technicalities Running a secondary inference engine on an iGPU introduces unique challenges regarding hardware stability and driver interaction. To ensure 99.9% uptime, the following technical requirements must be met: 1. The “Headless” Isolation Strategy To prevent conflicts between the Intel Graphics drivers and the NVIDIA CUDA drivers, the Sentinel should run in a **headless state**. This means it does not use a display buffer; it communicates solely via a local API (like `llama_server` or `Ollama` ). By isolating the process, we ensure that an error in the iGPU’1s Vulkan stack does not crash the desktop environment or the 3090’s CUDA context. 2. The Vulkan/OpenCL Pipeline Since Intel iGPUs don’t support CUDA, the Sentinel must use **llama.cpp with Vulkan or OpenCL support**. This allows the model to run on the integrated graphics chip using its own dedicated execution path, completely independent of the NVIDIA driver stack. 3. Memory Partitioning (The 8GB Buffer) By dedicating a specific portion of system RAM for the iGPU, we create a “Safe Zone.” Even if the 3090 pushes the system to the limit of its VRAM, the Sentinel remains in a stable memory pool. This prevents **Out-of-Memory (OOM)** errors from cascading from one GPU to the other. 4. Asynchronous Communication (The Bridge) The most critical component for robustness is the **Asynchronous Pipeline**. The 3090 should not “wait” for a response from the iGPU in a blocking manner. Instead, it should stream its output to a buffer; the Sentinel reads this buffer and sends a “Pass/Fail” signal back via a message broker (like **Redis** or a lightweight **FastAPI** endpoint). This ensures that if the iGPU is slightly slower than the 3099, the user’s experience isn’t affected. IV. Summary of the Sentinel Logic Flow **Primary Model (3090):** Generates high-quality content \rightarrow Outputs to a Buffer. **Sentinel Model (iGPU):** Scans the buffer for: **Critical Errors:** (e.g., “Broken JSON”, “Infinite Loop”). **Minor Errors:** (e.g., “Grammar Slip,” “Instruction Drift”). **Decision Engine:** - If **No Error**: The content is pushed to the user’s UI immediately. - If **Minor Error**: A hidden “Correction Request” is sent to the 3090. - If **Critical Error**: The system halts and logs a specific error code for the developer. Final Conclusion By implementing the **Sentinel Module**, you are essentially building a **Dual-Core Intelligence System**. You aren’t just running two models; you are creating a “Validator” that allows the main model to be more creative, while the iGPU ensures the output is technically sound. This significantly increases the reliability of the Hermes agent and provides a professional-grade architecture for local AI development.