Building with Agents: What Actually Changes When You Delegate to a Swarm

Shift from the common "autocomplete at conversational scale" workflow, where engineers manually copy-paste AI outputs, to a more advanced "dual-agent architecture." This system uses specialized AI agents (Claude and Gemini) that operate against a shared, persistent "vault" of structured knowledge and a SQLite-based semantic memory, enabling cross-session continuity and collaborative problem-solving. The architecture enforces explicit knowledge ownership and structured protocols, allowing agents to inherit findings from each other and avoid redundant work.

AI-Native Development with a Dual-Agent Architecture: A Technical Account Author: Abhinav Pangaria Date: May 2026 --- The Wrong Mental Model Most engineers who describe themselves as “using AI heavily” mean something narrower than that phrase implies. They write a prompt, read the output, paste what looks right into their editor, fix what breaks, and repeat. This is autocomplete at conversational scale. It is not a fundamentally different mode of working. It is the same lone-engineer workflow with a faster drafting tool in the loop. The productivity ceiling of that pattern is real and arrives quickly. It appears when: The problem exceeds one agent’s context window A task requires sustained research alongside implementation Decisions must persist beyond a single session Cross-session continuity becomes necessary Architecture understanding spans multiple domains simultaneously What follows is a different operational model: a dual-agent architecture refined across 18 months of production projects and hackathons, where specialized AI systems operate against shared memory, structured protocols, and persistent knowledge infrastructure. --- System Overview The architecture consists of four tightly integrated components: | Component | Responsibility | | --------------- | ------------------------------------- | | Claude Code | Implementation agent | | Gemini CLI | Research and long-context agent | | Obsidian Vault | Shared persistent knowledge substrate | | agent-brain MCP | Semantic memory and retrieval layer | These systems are not loosely coupled. They share state, communicate through explicit schemas, and maintain enforced role boundaries so each agent operates only within domains where it is structurally strongest. --- The Obsidian Vault as Shared Knowledge Substrate The vault is not a note-taking tool. It functions as the persistent source of truth across: Sessions Agents Projects Ecosystems Architectural decisions Both Claude Code and Gemini interact with the vault through the brain-vault MCP server , which exposes filesystem operations as callable tools. --- Vault Structure text ~/Documents/Brain/ ├── 00-Identity/ │ ├── USER.md │ ├── MODES.md │ ├── STACK.md │ ├── agent-configs-final/ │ │ ├── CLAUDE.md │ │ ├── GEMINI.md │ │ └── PERPLEXITY.md │ └── agent-skills/ │ ├── INDEX.md │ ├── session-open/ │ ├── session-close/ │ ├── code-plan/ │ ├── error-memory/ │ └── ... │ ├── 10-Projects/ │ └── project-slug / │ ├── CLAUDE.md │ ├── GEMINI.md │ ├── HANDOFF.md │ ├── PROJECT-STATE.md │ ├── decision-log.md │ ├── session-log.md │ ├── specs/ │ ├── research/ │ └── skills/ │ ├── 20-Areas/ │ └── ecosystems/ │ ├── circle/ │ ├── xrpl/ │ ├── stellar/ │ └── midnight/ │ ├── 30-Resources/ │ ├── 40-Memory/ │ ├── agent-brain.db │ ├── decision-log.md │ ├── error-log.md │ └── learned-patterns.md │ └── 50-Archive/ --- Knowledge Hierarchy and the Non-Duplication Rule The system enforces explicit ownership of knowledge. Canonical ecosystem facts exist once, under: text 20-Areas/ecosystems/ name / Projects reference these notes using Obsidian backlinks: text folder/filename They do not duplicate or paraphrase shared knowledge. This matters because both agents consume the same vault. Example: Gemini researches ARC chain behavior Findings are written into: text 20-Areas/ecosystems/circle/circle-overview.md Claude automatically inherits those findings in later sessions without re-fetching documentation The vault becomes a shared persistent memory substrate rather than ephemeral conversational context. --- Vault Write Constraints Every new note requires: YAML frontmatter Minimum two backlinks Source attribution Ecosystem or project association Tag metadata These constraints are enforced through the vault-write skill shared by both agents. --- The Two-Layer Memory Architecture The architecture separates memory into: 1. Human-readable structured knowledge 2. Machine-queryable semantic memory --- Layer 1 — agent-brain Semantic Memory agent-brain is a custom MCP server located at: text ~/Documents/Brain/agent-infra/ It wraps a SQLite-based semantic memory system with: | Feature | Implementation | | ------------- | ---------------------------------------- | | Embeddings | sentence-transformers/all-MiniLM-L6-v2 | | Vector Size | 384 dimensions | | Search | FTS5 + cosine similarity reranking | | Storage Types | technical, error, workflow, meta, growth | | Scope Flags | agent, project, validated | Both Claude and Gemini connect to the same database: Claude: ~/.claude/.mcp.json Gemini: ~/.gemini/settings.json The MCP server exposes: 9 memory operations 4 Gemini bridge tools --- Operational Memory Flow Before starting complex work: 1. Claude queries memory search 2. Prior solutions are retrieved 3. Validated fixes and architectural decisions are reused 4. Repeated debugging cycles are avoided After meaningful work: Agents write validated patterns back into shared memory Scope is controlled via: agent="both" agent="claude" agent="gemini" --- Layer 2 — claude-mem Episodic Memory claude-mem records session-level observations automatically through lifecycle hooks: SessionStart PostToolUse Stop PreCompact This layer tracks: What happened Which files were accessed Task progression Session events The distinction is explicit: | System | Purpose | | ----------- | ------------- | | agent-brain | What is true | | claude-mem | What happened | --- Role Boundaries The architecture works because the agents are not treated as interchangeable. Claude Code Responsibilities Claude handles: Code generation Code editing Sequential execution Tool orchestration Error diagnosis Live system interaction Session state management --- Gemini Responsibilities Gemini handles: Large-document synthesis SDK digestion Architecture review Ecosystem research Security audits Cross-source synthesis --- Enforced Behavioral Constraints Role boundaries are encoded as workflow memory entries. Example constraints: “Do not do market research Perplexity has live web . Do not read full SDKs mid-session Gemini is better for long-context doc reading . Do not design architecture. Do not run full codebase security audit.” These are persistent retrieval-enforced rules, not informal guidelines. --- Dual-Lane Communication Architecture The agents communicate through two distinct coordination lanes: | Lane | Purpose | | ---------------- | ----------------------- | | MCP Bridge Tools | Synchronous interaction | | File Protocol | Asynchronous delegation | --- Lane 1 — MCP Bridge Tools The MCP server exposes four bridge tools: | Tool | Function | | ---------------------- | ------------------------------ | | gemini research | Structured research delegation | | gemini review | Architecture/code review | | gemini read context | Large-document summarization | | gemini explore vault | Semantic vault exploration | Bridge calls return structured JSON rather than prose. Example response structures: json { "summary": "...", "findings": ... , "confidence": 0.94, "sources": ... } or json { "verdict": "...", "issues": ... , "suggestions": ... , "confidence": 0.88 } Claude immediately commits validated findings back into semantic memory after retrieval. --- Lane 2 — File Protocol When implementation encounters high-context blockers: 1. Claude writes RESEARCH-REQUEST.md 2. Context is compacted 3. Gemini fulfills the request asynchronously 4. Gemini writes RESEARCH-RESPONSE.md 5. Claude resumes implementation --- RESEARCH-REQUEST.md Schema markdown - Context - Blocker - Attempted Solutions - Git State - Specific Question - Files Gemini Should Read --- RESEARCH-RESPONSE.md Schema markdown - Direct Answer - Evidence Citations - Copy-Paste Ready Code - Numbered Next Steps - Implementation-Ready: YES/NO This protocol eliminates ambiguous state reconstruction between agents. --- Skills and Plugins Skills are shared behavioral protocols stored in: text ~/.agents/skills/ They are Markdown-defined execution frameworks activated contextually rather than globally. --- Global Skills | Skill | Trigger | | ---------------------- | ----------------------------- | | session-open | Session initialization | | session-close | Session termination | | code-plan | Before implementation | | anti-hallucination | Before architecture execution | | error-memory | On failures or build breaks | | research-method | Claude research tasks | | research-method-gemini | Gemini research tasks | | codebase-audit | Full security review | | vault-write | Vault modifications | --- Project-Scoped Skills Project-specific skills live under: text 10-Projects/ slug /skills/ These contain: Exact API patterns Error handling logic Adapter references Integration constraints Operational playbooks Example skills: use-circle-wallets bridge-stablecoin swap-tokens use-arc use-developer-controlled-wallets This eliminates repeated rediscovery of implementation knowledge. --- Context Window Asymmetry The architecture intentionally exploits the asymmetry between models. Claude Code Optimized for: Precision Incremental execution Tight context discipline Active token budgeting Focused implementation Gemini Optimized for: Massive context ingestion Full SDK visibility Multi-document synthesis Codebase-wide review Cross-specification reasoning The operational pattern becomes: Claude implements selectively Gemini performs wide-aperture analysis Structured findings return to Claude as compact actionable state --- Session Continuity via HANDOFF.md Every project maintains a continuously overwritten: text HANDOFF.md Written by session-close , it contains: Current implementation state Open blockers Git status Decisions made Next execution steps At session start: session-open loads the handoff Work resumes from prior state Append-only logs preserve historical traceability The handoff becomes the baton transferred between: Sessions Modes Agents --- Tesseract Protocol: Concrete Demonstration The architecture was stress-tested during the Tesseract Protocol hackathon project: Scope delivered in 48 hours: Six ZK circuits Merkle tree implementation CLI integration harness React frontend Midnight Network indexer integration --- Explicit Task Decomposition | Responsibility | Owner | | ------------------------- | ------------- | | ZK circuit specifications | Human | | Circuit implementation | Claude | | Integration tests | Claude | | Ecosystem research | Gemini | | Debugging | Collaborative | | Architectural decisions | Human | The non-delegable category was architecture: Trust boundaries State ownership Propagation tradeoffs Reliability constraints “Good enough” judgment The swarm handles execution. Human bandwidth is reserved for irreducible judgment problems. --- Infrastructure Summary text ~/.claude/.mcp.json ~/.gemini/settings.json ~/.agents/skills/ ~/Documents/Brain/ agent-infra/ brain/ memory.py embeddings.py bridge.py db.py mcp server.py 40-Memory/agent-brain.db Both agents register the same MCP server. Memory ownership is shared and immediately visible across systems. --- What This Is Not This is not an autonomous agent loop. There is: No orchestrator selecting agents No unsupervised execution chain No removal of human oversight The human: Makes every delegation decision Reviews every important output Owns all architectural judgment The architecture does not reduce human importance. It compresses execution overhead so human cognition is reserved for tradeoffs, judgment, and system-level reasoning. The core operational question becomes: Which parts of this problem require human judgment, and which parts can be specified clearly enough to delegate? That distinction is the defining skill of AI-native development.