IntelShed — open-source system combining hybrid RAG, GNN entity resolution, federated learning, and LLM-compiled multi-agent orchestration

Developer Alex K. is building IntelShed, an open-source OSINT system that aggregates 50 public data feeds into automated 24-hour security briefings, combining hybrid RAG, GNN entity resolution, federated learning with DP-SGD, and LLM-compiled multi-agent orchestration. The system indexes ~2,000 chunks on PostgreSQL 16, maintains a FollowTheMoney entity graph with ~142k entities and 173k edges, and runs entirely on free and open-source resources with no commercial API dependency. The developer is seeking community feedback on retrieval quality, entity resolution, federated learning, Bayesian trust scoring, and LLM-compiled workflow graphs.

I am building IntelShed, an open-source system that combines hybrid RAG, GNN-based entity resolution, federated learning with differential privacy, and LLM-compiled multi-agent orchestration. The application domain is open-source intelligence OSINT — the system aggregates 50 public data feeds and produces automated 24h security briefings — but the ML architecture is general. I would value feedback and contributions from this community. Architecture overview - Hybrid retrieval: pgvector HNSW cosine + tsvector GIN BM25 + reciprocal rank fusion k=60 + BGE cross-encoder reranker ONNX int8 quantized, CUDA . ~2,000 indexed chunks on PostgreSQL 16. - Entity graph: FollowTheMoney schema, ~142k entities, 173k edges 77k exact match, 7k Splink probabilistic, 6.5k GNN-predicted . Splink dual-pipeline with LLM-in-the-loop adjudication for ambiguous sameAs pairs. - GNN pipeline: GAT for correlation static + temporal edge features , T-GCN forecaster GCN+GRU, 24h prediction , GNNExplainer for edge importance attribution. - Federated learning: FedAvg gradient exchange between GPU nodes with DP-SGD. Tested but single-trainer only so far. - Trust layer: Two-sided Bayesian source adjudication Beta sensitivity + specificity per source , constraint caps on corroboration geographic 2000km, temporal 7d, event-type incompatibility . Corroboration rate calibrated from 100% to 16.8%. - Multi-agent: 8-phase orchestrator with loop detection state hash MD5, threshold=3 , per-phase evaluators, red-team veto gate. Plus a “Vibe Graphing” module where an LLM dynamically compiles a workflow graph from a natural-language query 3-stage: role assignment → topology design → semantic completion . - LLM: NVIDIA NIM cascade 5 models, 3 tried per call with 2s retry on 429 , Ollama offline fallback. Central dispatch via llm provider.py. - Runs entirely on free and open-source resources. No commercial API is a hard dependency. Where I could use help 1. RAG and retrieval quality Evaluation of retrieval quality recall@k, precision@k against labeled eval sets. Tuning of HNSW parameters ef search currently 200, ef construction . Comparison against alternative reranker models or quantization strategies. The corpus is small ~2,000 chunks but growing — I am interested in scaling behavior. 2. Entity resolution Blocking rule optimization for Splink currently blocking on name last . Matching accuracy evaluation. Approaches to handling schema heterogeneity across 50+ feed sources that map to FollowTheMoney. The LLM adjudication path uses NVIDIA NIM for ambiguous pairs — feedback on prompt design and calibration welcome. 3. Federated learning FedAvg implementation with DP-SGD is code-complete 23 tests, 6 E2E checks pass but validated with a single trainer only. I am deploying a second GPU node identical RTX 3080 Ti 16GB for real two-node gradient exchange. Feedback on aggregation strategy currently weighted by val auc , privacy budget, and convergence behavior welcome. 4. Bayesian trust scoring Two-sided Beta-Binomial adjudication per source with online posterior updates. Corroboration caps applied via geographic, temporal, and event-type constraints. I would welcome independent evaluation of the methodology from a statistics perspective, particularly the calibration approach and the choice of priors. Contradiction detection and narrative clustering UMAP + HDBSCAN are open and unimplemented. 5. LLM-compiled workflow graphs Vibe Graphing An LLM generates a multi-agent workflow specification from a natural-language query in 3 stages: role assignment which agents , topology design execution order, parallelism , semantic completion prompts + tools per agent . The spec is compiled to a graph runtime.Graph and executed with topological scheduling. I would welcome feedback on this approach vs. fixed pipelines, and evaluation of compilation quality across query types. What is not needed - Feature requests without implementation discussion. How to contribute Repository: GitHub - sookoothaii/IntelShed: Spatial intelligence workstation — 47 live OSINT feeds, FollowTheMoney entity graph on PostgreSQL+pgvector, 8-phase multi-agent orchestrator with 4-layer anti-hallucination guards, Bayesian source adjudication, W3C PROV evidence DAG, GNN pipeline GAT+T-GCN+FedAvg+DP-SGD , 3-node mesh with Ed25519 auth, automated 24h briefing via NVIDIA NIM 7-model · GitHub https://github.com/sookoothaii/IntelShed MIT license . 6,300+ tests, ruff + tsc + prettier pre-commit, 8 CI jobs. See CONTRIBUTING.md for setup. Issues labeled help wanted and evaluation are open. I am particularly interested in: - Independent benchmarks of retrieval quality against labeled eval sets - Security review of the MCP tool surface 81 tools, optional 9 meta-tool progressive discovery mode - Federated learning validation with real multi-node gradient exchange - Statistical evaluation of the Bayesian trust scoring methodology If you find bugs, design flaws, or areas for improvement, please open an issue with specifics. I prefer verifiable claims over speculative suggestions. Technical stack Python 3.12, FastAPI, PostgreSQL 16 + pgvector + postgis, DuckDB, PyTorch + PyG GAT + T-GCN , ONNX Runtime CUDA , Ollama, NVIDIA NIM, Docker Compose. Edge node: Raspberry Pi 4 with offline autonomy 13 deterministic rule categories, no LLM required .