cd /news/artificial-intelligence/building-with-agents-what-actually-c… Β· home β€Ί topics β€Ί artificial-intelligence β€Ί article
[ARTICLE Β· art-7651] src=gist.github.com β†— pub= topic=artificial-intelligence verified=true sentiment=Β· neutral

Building with Agents: What Actually Changes When You Delegate to a Swarm

Shift from the common "autocomplete at conversational scale" workflow, where engineers manually copy-paste AI outputs, to a more advanced "dual-agent architecture." This system uses specialized AI agents (Claude and Gemini) that operate against a shared, persistent "vault" of structured knowledge and a SQLite-based semantic memory, enabling cross-session continuity and collaborative problem-solving. The architecture enforces explicit knowledge ownership and structured protocols, allowing agents to inherit findings from each other and avoid redundant work.

read9 min views14 publishedMay 22, 2026

Author: Abhinav Pangaria Date: May 2026


Most engineers who describe themselves as β€œusing AI heavily” mean something narrower than that phrase implies.

They write a prompt, read the output, paste what looks right into their editor, fix what breaks, and repeat.

This is autocomplete at conversational scale. It is not a fundamentally different mode of working. It is the same lone-engineer workflow with a faster drafting tool in the loop.

The productivity ceiling of that pattern is real and arrives quickly. It appears when:

  • The problem exceeds one agent’s context window
  • A task requires sustained research alongside implementation
  • Decisions must persist beyond a single session
  • Cross-session continuity becomes necessary
  • Architecture understanding spans multiple domains simultaneously

What follows is a different operational model: a dual-agent architecture refined across 18 months of production projects and hackathons, where specialized AI systems operate against shared memory, structured protocols, and persistent knowledge infrastructure.


The architecture consists of four tightly integrated components:

Component Responsibility
Claude Code Implementation agent
Gemini CLI Research and long-context agent
Obsidian Vault Shared persistent knowledge substrate
agent-brain MCP Semantic memory and retrieval layer

These systems are not loosely coupled. They share state, communicate through explicit schemas, and maintain enforced role boundaries so each agent operates only within domains where it is structurally strongest.


The vault is not a note-taking tool. It functions as the persistent source of truth across:

  • Sessions
  • Agents
  • Projects
  • Ecosystems
  • Architectural decisions

Both Claude Code and Gemini interact with the vault through the brain-vault MCP server, which exposes filesystem operations as callable tools.


~/Documents/Brain/
β”œβ”€β”€ 00-Identity/
β”‚   β”œβ”€β”€ USER.md
β”‚   β”œβ”€β”€ MODES.md
β”‚   β”œβ”€β”€ STACK.md
β”‚   β”œβ”€β”€ agent-configs-final/
β”‚   β”‚   β”œβ”€β”€ CLAUDE.md
β”‚   β”‚   β”œβ”€β”€ GEMINI.md
β”‚   β”‚   └── PERPLEXITY.md
β”‚   └── agent-skills/
β”‚       β”œβ”€β”€ INDEX.md
β”‚       β”œβ”€β”€ session-open/
β”‚       β”œβ”€β”€ session-close/
β”‚       β”œβ”€β”€ code-plan/
β”‚       β”œβ”€β”€ error-memory/
β”‚       └── ...
β”‚
β”œβ”€β”€ 10-Projects/
β”‚   └── [project-slug]/
β”‚       β”œβ”€β”€ CLAUDE.md
β”‚       β”œβ”€β”€ GEMINI.md
β”‚       β”œβ”€β”€ HANDOFF.md
β”‚       β”œβ”€β”€ PROJECT-STATE.md
β”‚       β”œβ”€β”€ decision-log.md
β”‚       β”œβ”€β”€ session-log.md
β”‚       β”œβ”€β”€ specs/
β”‚       β”œβ”€β”€ research/
β”‚       └── skills/
β”‚
β”œβ”€β”€ 20-Areas/
β”‚   └── ecosystems/
β”‚       β”œβ”€β”€ circle/
β”‚       β”œβ”€β”€ xrpl/
β”‚       β”œβ”€β”€ stellar/
β”‚       └── midnight/
β”‚
β”œβ”€β”€ 30-Resources/
β”‚
β”œβ”€β”€ 40-Memory/
β”‚   β”œβ”€β”€ agent-brain.db
β”‚   β”œβ”€β”€ decision-log.md
β”‚   β”œβ”€β”€ error-log.md
β”‚   └── learned-patterns.md
β”‚
└── 50-Archive/

The system enforces explicit ownership of knowledge.

Canonical ecosystem facts exist once, under:

20-Areas/ecosystems/[name]/

Projects reference these notes using Obsidian backlinks:

[[folder/filename]]

They do not duplicate or paraphrase shared knowledge.

This matters because both agents consume the same vault.

Example:

  • Gemini researches ARC chain behavior
  • Findings are written into:
20-Areas/ecosystems/circle/circle-overview.md
  • Claude automatically inherits those findings in later sessions without re-fetching documentation

The vault becomes a shared persistent memory substrate rather than ephemeral conversational context.


Every new note requires:

  • YAML frontmatter
  • Minimum two backlinks
  • Source attribution
  • Ecosystem or project association
  • Tag metadata

These constraints are enforced through the vault-write skill shared by both agents.


The architecture separates memory into:

  1. Human-readable structured knowledge
  2. Machine-queryable semantic memory

agent-brain is a custom MCP server located at:

~/Documents/Brain/agent-infra/

It wraps a SQLite-based semantic memory system with:

Feature Implementation
Embeddings sentence-transformers/all-MiniLM-L6-v2
Vector Size 384 dimensions
Search FTS5 + cosine similarity reranking
Storage Types technical, error, workflow, meta, growth
Scope Flags agent, project, validated

Both Claude and Gemini connect to the same database:

  • Claude: ~/.claude/.mcp.json
  • Gemini: ~/.gemini/settings.json

The MCP server exposes:

  • 9 memory operations
  • 4 Gemini bridge tools

Before starting complex work:

  1. Claude queries memory_search
  2. Prior solutions are retrieved
  3. Validated fixes and architectural decisions are reused
  4. Repeated debugging cycles are avoided

After meaningful work:

  • Agents write validated patterns back into shared memory

  • Scope is controlled via:

    • agent="both"
    • agent="claude"
    • agent="gemini"

claude-mem records session-level observations automatically through lifecycle hooks:

  • SessionStart
  • PostToolUse
  • Stop
  • PreCompact

This layer tracks:

  • What happened
  • Which files were accessed
  • Task progression
  • Session events

The distinction is explicit:

System Purpose
agent-brain What is true
claude-mem What happened

The architecture works because the agents are not treated as interchangeable.

Claude Code Responsibilities #

Claude handles:

  • Code generation
  • Code editing
  • Sequential execution
  • Tool orchestration
  • Error diagnosis
  • Live system interaction
  • Session state management

Gemini Responsibilities #

Gemini handles:

  • Large-document synthesis
  • SDK digestion
  • Architecture review
  • Ecosystem research
  • Security audits
  • Cross-source synthesis

Role boundaries are encoded as workflow memory entries.

Example constraints:

β€œDo not do market research (Perplexity has live web). Do not read full SDKs mid-session (Gemini is better for long-context doc reading). Do not design architecture. Do not run full codebase security audit.”

These are persistent retrieval-enforced rules, not informal guidelines.


The agents communicate through two distinct coordination lanes:

Lane Purpose
MCP Bridge Tools Synchronous interaction
File Protocol Asynchronous delegation

The MCP server exposes four bridge tools:

Tool Function
gemini_research Structured research delegation
gemini_review Architecture/code review
gemini_read_context Large-document summarization
gemini_explore_vault Semantic vault exploration

Bridge calls return structured JSON rather than prose.

Example response structures:

{
  "summary": "...",
  "findings": [...],
  "confidence": 0.94,
  "sources": [...]
}

or

{
  "verdict": "...",
  "issues": [...],
  "suggestions": [...],
  "confidence": 0.88
}

Claude immediately commits validated findings back into semantic memory after retrieval.


When implementation encounters high-context blockers:

  1. Claude writes RESEARCH-REQUEST.md
  2. Context is compacted
  3. Gemini fulfills the request asynchronously
  4. Gemini writes RESEARCH-RESPONSE.md
  5. Claude resumes implementation

- Context
- Blocker
- Attempted Solutions
- Git State
- Specific Question
- Files Gemini Should Read

- Direct Answer
- Evidence Citations
- Copy-Paste Ready Code
- Numbered Next Steps
- Implementation-Ready: YES/NO

This protocol eliminates ambiguous state reconstruction between agents.


Skills are shared behavioral protocols stored in:

~/.agents/skills/

They are Markdown-defined execution frameworks activated contextually rather than globally.


Skill Trigger
session-open Session initialization
session-close Session termination
code-plan Before implementation
anti-hallucination Before architecture execution
error-memory On failures or build breaks
research-method Claude research tasks
research-method-gemini Gemini research tasks
codebase-audit Full security review
vault-write Vault modifications

Project-specific skills live under:

10-Projects/[slug]/skills/

These contain:

  • Exact API patterns
  • Error handling logic
  • Adapter references
  • Integration constraints
  • Operational playbooks

Example skills:

  • use-circle-wallets
  • bridge-stablecoin
  • swap-tokens
  • use-arc
  • use-developer-controlled-wallets

This eliminates repeated rediscovery of implementation knowledge.


The architecture intentionally exploits the asymmetry between models.

Claude Code #

Optimized for:

  • Precision
  • Incremental execution
  • Tight context discipline
  • Active token budgeting
  • Focused implementation

Gemini #

Optimized for:

  • Massive context ingestion
  • Full SDK visibility
  • Multi-document synthesis
  • Codebase-wide review
  • Cross-specification reasoning

The operational pattern becomes:

  • Claude implements selectively
  • Gemini performs wide-aperture analysis
  • Structured findings return to Claude as compact actionable state

Every project maintains a continuously overwritten:

HANDOFF.md

Written by session-close, it contains:

  • Current implementation state
  • Open blockers
  • Git status
  • Decisions made
  • Next execution steps

At session start:

  • session-open loads the handoff
  • Work resumes from prior state
  • Append-only logs preserve historical traceability

The handoff becomes the baton transferred between:

  • Sessions
  • Modes
  • Agents

The architecture was stress-tested during the Tesseract Protocol hackathon project:

Scope delivered in 48 hours:

  • Six ZK circuits
  • Merkle tree implementation
  • CLI integration harness
  • React frontend
  • Midnight Network indexer integration

Responsibility Owner
ZK circuit specifications Human
Circuit implementation Claude
Integration tests Claude
Ecosystem research Gemini
Debugging Collaborative
Architectural decisions Human

The non-delegable category was architecture:

  • Trust boundaries
  • State ownership
  • Propagation tradeoffs
  • Reliability constraints
  • β€œGood enough” judgment

The swarm handles execution. Human bandwidth is reserved for irreducible judgment problems.


~/.claude/.mcp.json
~/.gemini/settings.json
~/.agents/skills/

~/Documents/Brain/
  agent-infra/
    brain/
      memory.py
      embeddings.py
      bridge.py
      db.py
    mcp_server.py

40-Memory/agent-brain.db

Both agents register the same MCP server. Memory ownership is shared and immediately visible across systems.


This is not an autonomous agent loop.

There is:

  • No orchestrator selecting agents
  • No unsupervised execution chain
  • No removal of human oversight

The human:

  • Makes every delegation decision
  • Reviews every important output
  • Owns all architectural judgment

The architecture does not reduce human importance. It compresses execution overhead so human cognition is reserved for tradeoffs, judgment, and system-level reasoning.

The core operational question becomes:

Which parts of this problem require human judgment, and which parts can be specified clearly enough to delegate?

That distinction is the defining skill of AI-native development.

── more in #artificial-intelligence 4 stories Β· sorted by recency
── more on @abhinav pangaria 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain β€” perfect for shipping the agent you just read about.

$git push zahid main
β†’ Live at https://your-agent.zahid.host βœ“
Get free account β†’ Pricing
from €0/mo Β· no card required
LIVE [news/building-with-agents…] indexed:0 read:9min 2026-05-22 Β· β€”