# IntelShed — open-source system combining hybrid RAG, GNN entity resolution, federated learning, and LLM-compiled multi-agent orchestration > Source: > Published: 2026-08-03 00:50:57+00:00 I am building IntelShed, an open-source system that combines hybrid RAG, GNN-based entity resolution, federated learning with differential privacy, and LLM-compiled multi-agent orchestration. The application domain is open-source intelligence (OSINT) — the system aggregates 50 public data feeds and produces automated 24h security briefings — but the ML architecture is general. I would value feedback and contributions from this community. **### Architecture overview** - Hybrid retrieval: pgvector HNSW (cosine) + tsvector GIN (BM25) + reciprocal rank fusion (k=60) + BGE cross-encoder reranker (ONNX int8 quantized, CUDA). ~2,000 indexed chunks on PostgreSQL 16. - Entity graph: FollowTheMoney schema, ~142k entities, 173k edges (77k exact match, 7k Splink probabilistic, 6.5k GNN-predicted). Splink dual-pipeline with LLM-in-the-loop adjudication for ambiguous sameAs pairs. - GNN pipeline: GAT for correlation (static + temporal edge features), T-GCN forecaster (GCN+GRU, 24h prediction), GNNExplainer for edge importance attribution. - Federated learning: FedAvg gradient exchange between GPU nodes with DP-SGD. Tested but single-trainer only so far. - Trust layer: Two-sided Bayesian source adjudication (Beta sensitivity + specificity per source), constraint caps on corroboration (geographic >2000km, temporal >7d, event-type incompatibility). Corroboration rate calibrated from 100% to 16.8%. - Multi-agent: 8-phase orchestrator with loop detection (state_hash MD5, threshold=3), per-phase evaluators, red-team veto gate. Plus a “Vibe Graphing” module where an LLM dynamically compiles a workflow graph from a natural-language query (3-stage: role assignment → topology design → semantic completion). - LLM: NVIDIA NIM cascade (5 models, 3 tried per call with 2s retry on 429), Ollama offline fallback. Central dispatch via llm_provider.py. - Runs entirely on free and open-source resources. No commercial API is a hard dependency. **### Where I could use help** ****1. RAG and retrieval quality**** Evaluation of retrieval quality (recall@k, precision@k) against labeled eval sets. Tuning of HNSW parameters (ef_search currently 200, ef_construction). Comparison against alternative reranker models or quantization strategies. The corpus is small (~2,000 chunks) but growing — I am interested in scaling behavior. ****2. Entity resolution**** Blocking rule optimization for Splink (currently blocking on name_last). Matching accuracy evaluation. Approaches to handling schema heterogeneity across 50+ feed sources that map to FollowTheMoney. The LLM adjudication path uses NVIDIA NIM for ambiguous pairs — feedback on prompt design and calibration welcome. ****3. Federated learning**** FedAvg implementation with DP-SGD is code-complete (23 tests, 6 E2E checks pass) but validated with a single trainer only. I am deploying a second GPU node (identical RTX 3080 Ti 16GB) for real two-node gradient exchange. Feedback on aggregation strategy (currently weighted by val_auc), privacy budget, and convergence behavior welcome. ****4. Bayesian trust scoring**** Two-sided Beta-Binomial adjudication per source with online posterior updates. Corroboration caps applied via geographic, temporal, and event-type constraints. I would welcome independent evaluation of the methodology from a statistics perspective, particularly the calibration approach and the choice of priors. Contradiction detection and narrative clustering (UMAP + HDBSCAN) are open and unimplemented. ****5. LLM-compiled workflow graphs (Vibe Graphing)**** An LLM generates a multi-agent workflow specification from a natural-language query in 3 stages: role assignment (which agents), topology design (execution order, parallelism), semantic completion (prompts + tools per agent). The spec is compiled to a graph_runtime.Graph and executed with topological scheduling. I would welcome feedback on this approach vs. fixed pipelines, and evaluation of compilation quality across query types. **### What is not needed** - Feature requests without implementation discussion. **### How to contribute** Repository: [GitHub - sookoothaii/IntelShed: Spatial intelligence workstation — 47 live OSINT feeds, FollowTheMoney entity graph on PostgreSQL+pgvector, 8-phase multi-agent orchestrator with 4-layer anti-hallucination guards, Bayesian source adjudication, W3C PROV evidence DAG, GNN pipeline (GAT+T-GCN+FedAvg+DP-SGD), 3-node mesh with Ed25519 auth, automated 24h briefing via NVIDIA NIM 7-model · GitHub](https://github.com/sookoothaii/IntelShed) (MIT license). 6,300+ tests, ruff + tsc + prettier pre-commit, 8 CI jobs. See CONTRIBUTING.md for setup. Issues labeled `help wanted` and `evaluation` are open. I am particularly interested in: - Independent benchmarks of retrieval quality against labeled eval sets - Security review of the MCP tool surface (81 tools, optional 9 meta-tool progressive discovery mode) - Federated learning validation with real multi-node gradient exchange - Statistical evaluation of the Bayesian trust scoring methodology If you find bugs, design flaws, or areas for improvement, please open an issue with specifics. I prefer verifiable claims over speculative suggestions. **### Technical stack** Python 3.12, FastAPI, PostgreSQL 16 + pgvector + postgis, DuckDB, PyTorch + PyG (GAT + T-GCN), ONNX Runtime (CUDA), Ollama, NVIDIA NIM, Docker Compose. Edge node: Raspberry Pi 4 with offline autonomy (13 deterministic rule categories, no LLM required).