Building Self-Evolving AI Agents with OpenSpace Using Skills, MCP, Lineage, and Low-Cost Reuse

A tutorial demonstrates building self-evolving AI agents using the OpenSpace framework, which stores evolved capabilities in SQLite with versioning and lineage metadata and supports lower-cost, reusable agent behavior through FIX, DERIVED, and CAPTURED skills. The workflow covers environment setup, sparse repository cloning, live task execution, skill evolution, MCP-based agent integration, and warm-task reuse.

In this tutorial, https://github.com/MARKTECHPOST-AI-MEDIA-INC/AI-Agents-Projects-Tutorials/blob/main/Agentic%20AI%20Codes/HKUDS OpenSpace Self Evolving AI Agents Marktechpost.ipynb we build and examine an OpenSpace https://github.com/HKUDS/OpenSpace workflow, progressing from environment setup and sparse repository cloning to live task execution, skill evolution, and MCP-based agent integration. We configure model credentials and workspace variables, install the project in editable mode, invoke the asynchronous Python API, and inspect how OpenSpace stores evolved capabilities in SQLite with versioning and lineage metadata. We also create a custom SKILL.md, connect host-agent skills, test warm-task reuse, launch the streamable HTTP MCP server, and analyze the showcase evolution database to understand how FIX, DERIVED, and CAPTURED skills support lower-cost, reusable agent behavior. python import os, sys, subprocess, sqlite3, json, textwrap, shutil, time, pathlib ANTHROPIC API KEY = "" OPENAI API KEY = "" OPENSPACE MODEL = "anthropic/claude-sonnet-4-5" OPENSPACE CLOUD KEY = "" assert sys.version info = 3, 12 , f"OpenSpace needs Python 3.12+, Colab has {sys.version}. " "Runtime Change runtime type, or use a fallback py312 venv." print "✅ Python:", sys.version.split 0 REPO DIR = "/content/OpenSpace" if not os.path.exists REPO DIR : subprocess.run "git", "clone", "--filter=blob:none", "--sparse", "https://github.com/HKUDS/OpenSpace.git", REPO DIR , check=True, subprocess.run "git", "sparse-checkout", "set", "--no-cone", "/ ", " /assets/" , cwd=REPO DIR, check=True, print "✅ Cloned to", REPO DIR print "Top-level:", sorted os.listdir REPO DIR subprocess.run sys.executable, "-m", "pip", "install", "-q", "-e", REPO DIR , check=True subprocess.run sys.executable, "-m", "pip", "install", "-q", "nest asyncio" , check=True for cli in "openspace-mcp", "openspace-dashboard" : path = shutil.which cli print f"{'✅' if path else '⚠️ '} {cli}: {path}" subprocess.run "openspace-mcp", "--help" , check=False WORKSPACE = "/content/openspace workspace" SKILLS DIR = "/content/my agent skills" os.makedirs WORKSPACE, exist ok=True os.makedirs SKILLS DIR, exist ok=True env lines = f"OPENSPACE MODEL={OPENSPACE MODEL}", f"OPENSPACE WORKSPACE={WORKSPACE}", f"OPENSPACE HOST SKILL DIRS={SKILLS DIR}", if ANTHROPIC API KEY: env lines.append f"ANTHROPIC API KEY={ANTHROPIC API KEY}" if OPENAI API KEY: env lines.append f"OPENAI API KEY={OPENAI API KEY}" if OPENSPACE CLOUD KEY: env lines.append f"OPENSPACE API KEY={OPENSPACE CLOUD KEY}" env path = os.path.join REPO DIR, "openspace", ".env" pathlib.Path env path .write text "\n".join env lines + "\n" pathlib.Path "/content/.env" .write text "\n".join env lines + "\n" for line in env lines: k, , v = line.partition "=" os.environ k = v print "✅ .env written:\n" + "\n".join l.split "=" 0 + "= " if "KEY" in l else l for l in env lines HAS LLM KEY = bool ANTHROPIC API KEY or OPENAI API KEY if not HAS LLM KEY: print "⚠️ No LLM key set — Steps 4/6 live execution will be skipped." We verify the Python runtime, define the required API credentials, and configure the OpenSpace model and optional cloud access settings. We clone the repository with sparse checkout, install the package in editable mode, and confirm that the OpenSpace command-line tools are available. We then create the workspace and skill directories, write the environment configuration files, export the required variables, and detect whether live LLM execution is enabled. python import asyncio, nest asyncio nest asyncio.apply async def run task query: str : from openspace import OpenSpace async with OpenSpace as cs: result = await cs.execute query print "── RESPONSE " + "─" 50 print result "response" :3000 for skill in result.get "evolved skills", : print f" 🧬 Evolved: {skill 'name' } origin={skill 'origin' } " return result if HAS LLM KEY: result 1 = asyncio.run run task "Write a Python function that parses a CSV of employee hours and " "computes weekly payroll with overtime 1.5x beyond 40h . Test it " "on a small synthetic example and show the output." else: print "⏭️ Skipped live task no key ." def dump db db path, max rows=8 : if not os.path.exists db path : print "No DB at", db path ; return con = sqlite3.connect db path cur = con.cursor tables = r 0 for r in cur.execute "SELECT name FROM sqlite master WHERE type='table'" .fetchall print f"📀 {db path}\n tables: {tables}" for t in tables: try: cols = c 1 for c in cur.execute f"PRAGMA table info {t} " .fetchall rows = cur.execute f"SELECT FROM {t} LIMIT {max rows}" .fetchall print f"\n▶ {t} {len rows } shown cols={cols :8 }{'…' if len cols 8 else ''}" for r in rows: print " ", str r :160 except Exception as e: print f" skip {t}: {e} " con.close runtime db = os.path.join WORKSPACE, ".openspace", "openspace.db" alt db = os.path.join REPO DIR, ".openspace", "openspace.db" dump db runtime db if os.path.exists runtime db else alt db try: from openspace.skill engine.registry import SkillRegistry from openspace.skill engine import types as sk types print "\n✅ skill engine importable:", n for n in dir sk types if n 0 .isupper :6 except Exception as e: print "ℹ️ registry import note:", e We initialize asynchronous execution in Google Colab and define a reusable function to submit tasks via the OpenSpace Python API. We run an initial payroll-generation task and inspect any skills that OpenSpace evolves during post-execution analysis. We also examine the SQLite database structure, display stored records, and verify that the skill registry and type definitions are accessible programmatically. if HAS LLM KEY: result 2 = asyncio.run run task "Extend the payroll logic: add a second CSV of tax withholding rates " "per employee and produce net pay. Reuse any prior payroll skill." dump db runtime db if os.path.exists runtime db else alt db, max rows=12 else: print "⏭️ Skipped warm-rerun demo no key ." custom = pathlib.Path SKILLS DIR / "colab-csv-report" custom.mkdir parents=True, exist ok=True custom / "SKILL.md" .write text textwrap.dedent """\ --- name: colab-csv-report description: Turn any CSV into a short markdown report with summary stats, null counts, dtypes, and 3 key observations. Use pandas; never plot. --- colab-csv-report 1. Load the CSV with pandas on bad lines="skip" fallback . 2. Emit: shape, dtypes table, describe , null counts. 3. Write 3 bullet observations in plain markdown. 4. If parsing fails, retry with sep=None, engine="python" . """ print "✅ Custom skill written:", custom / "SKILL.md" for host skill in "delegate-task", "skill-discovery" : src = os.path.join REPO DIR, "openspace", "host skills", host skill dst = os.path.join SKILLS DIR, host skill if os.path.isdir src and not os.path.isdir dst : shutil.copytree src, dst print "✅ Host skills installed into agent dir:", sorted os.listdir SKILLS DIR if HAS LLM KEY: df csv = "/content/demo.csv" pathlib.Path df csv .write text "name,dept,hours,rate\nAda,Eng,45,90\nGrace,Eng,38,95\nAlan,Math,50,80\n" asyncio.run run task f"Using the colab-csv-report skill, produce a markdown report for {df csv}" We submit a related payroll task to observe how OpenSpace reuses or derives capabilities from previously generated skills. We create a custom SKILL.md that instructs the agent to analyze CSV files and produce structured Markdown reports. We then install the OpenSpace host skills, generate a demonstration dataset, and execute the custom capability through the same evolving agent workflow. mcp proc = subprocess.Popen "openspace-mcp", "--transport", "streamable-http", "--host", "127.0.0.1", "--port", "8081" , stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, env={ os.environ}, time.sleep 8 try: import urllib.request req = urllib.request.Request "http://127.0.0.1:8081/mcp", method="GET" try: urllib.request.urlopen req, timeout=5 print "✅ MCP streamable-HTTP endpoint is up at http://127.0.0.1:8081/mcp" except urllib.error.HTTPError as e: print f"✅ MCP server alive HTTP {e.code} on bare GET, expected for MCP " except Exception as e: print "⚠️ MCP probe failed:", e print json.dumps { "mcpServers": { "openspace": { "command": "openspace-mcp", "toolTimeout": 600, "env": { "OPENSPACE HOST SKILL DIRS": SKILLS DIR, "OPENSPACE WORKSPACE": WORKSPACE, "OPENSPACE API KEY": "sk-xxx optional, for cloud ", }, } } }, indent=2 mcp proc.terminate We start the OpenSpace MCP server using the streamable HTTP transport and bind it to a local Colab endpoint. We probe the endpoint to confirm that the server process is running, even when a basic HTTP request returns an MCP-specific response status. We also generate an example MCP host configuration that external agents can use to access the OpenSpace workspace and skill directories. if OPENSPACE CLOUD KEY: subprocess.run "openspace-upload-skill", str custom , check=False print "✅ Cloud CLI demo executed upload ." else: print "ℹ️ Cloud skipped — set OPENSPACE CLOUD KEY to enable " "openspace-upload-skill / openspace-download-skill." showcase db = os.path.join REPO DIR, "showcase", ".openspace", "openspace.db" dump db showcase db, max rows=10 if os.path.exists showcase db : con = sqlite3.connect showcase db cur = con.cursor for t in r 0 for r in cur.execute "SELECT name FROM sqlite master WHERE type='table'" : cols = c 1 .lower for c in cur.execute f"PRAGMA table info {t} " if "origin" in cols: print f"\n📊 Evolution-mode breakdown in table '{t}':" for origin, n in cur.execute f"SELECT origin, COUNT FROM {t} GROUP BY origin ORDER BY 2 DESC" : print f" {origin: 10}: {n}" con.close print """ ══════════════════════════════════════════════════════════════════ 🎓 TUTORIAL COMPLETE — what you now have: • OpenSpace installed + configured in Colab • Live task execution via the Python API if key set • A custom SKILL.md your agent auto-discovers • Host skills delegate-task, skill-discovery staged for any SKILL.md-capable agent Claude Code / Codex / OpenClaw / nanobot • An MCP server you booted over streamable HTTP • Full lineage inspection of a 60+ skill evolution DB Next: run more related tasks and watch token usage drop as skills FIX / DERIVE / CAPTURE themselves. Dashboard needs Node ≥ 20 : openspace-dashboard --port 7788 + cd frontend && npm i && npm run dev ══════════════════════════════════════════════════════════════════ """ We conditionally upload the custom skill to the OpenSpace cloud community when a valid cloud API key is available. We inspect the repository’s showcase SQLite database to study the stored skills, metadata, quality information, and complete evolution lineage. We finally aggregate skills by origin type and summarize the Colab environment, custom capability, MCP integration, and self-evolving workflow that we establish throughout the tutorial. In conclusion, we established a practical OpenSpace environment that demonstrates how agent capabilities are executed, persisted, reused, and progressively improved. We worked directly with the Python API, local skill directories, SQLite-backed registries, MCP transports, and optional cloud skill-sharing commands, giving us visibility into both the user-facing workflow and the underlying skill-engine architecture. We also verified how related tasks can reuse previously evolved knowledge, how custom skills become discoverable at runtime, and how lineage records expose the development history of each capability, leaving us with a strong foundation for building self-evolving, cost-efficient agent systems in Colab. Check out the Full Code here. Also, feel free to follow us on and don’t forget to join our Twitter https://x.com/intent/follow?screen name=marktechpost and Subscribe to 150k+ML SubReddit https://www.reddit.com/r/machinelearningnews/ . Wait are you on telegram? our Newsletter https://www.aidevsignals.com/ now you can join us on telegram as well. https://t.me/machinelearningresearchnews Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us https://forms.gle/wbash1wF6efRj8G58 Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.