{"slug": "building-self-evolving-ai-agents-with-openspace-using-skills-mcp-lineage-and-low", "title": "Building Self-Evolving AI Agents with OpenSpace Using Skills, MCP, Lineage, and Low-Cost Reuse", "summary": "A tutorial demonstrates building self-evolving AI agents using the OpenSpace framework, which stores evolved capabilities in SQLite with versioning and lineage metadata and supports lower-cost, reusable agent behavior through FIX, DERIVED, and CAPTURED skills. The workflow covers environment setup, sparse repository cloning, live task execution, skill evolution, MCP-based agent integration, and warm-task reuse.", "body_md": "In this [tutorial,](https://github.com/MARKTECHPOST-AI-MEDIA-INC/AI-Agents-Projects-Tutorials/blob/main/Agentic%20AI%20Codes/HKUDS_OpenSpace_Self_Evolving_AI_Agents_Marktechpost.ipynb) we build and examine an[ OpenSpace](https://github.com/HKUDS/OpenSpace) workflow, progressing from environment setup and sparse repository cloning to live task execution, skill evolution, and MCP-based agent integration. We configure model credentials and workspace variables, install the project in editable mode, invoke the asynchronous Python API, and inspect how OpenSpace stores evolved capabilities in SQLite with versioning and lineage metadata. We also create a custom SKILL.md, connect host-agent skills, test warm-task reuse, launch the streamable HTTP MCP server, and analyze the showcase evolution database to understand how FIX, DERIVED, and CAPTURED skills support lower-cost, reusable agent behavior.\n\n``` python\nimport os, sys, subprocess, sqlite3, json, textwrap, shutil, time, pathlib\nANTHROPIC_API_KEY = \"\"\nOPENAI_API_KEY    = \"\"\nOPENSPACE_MODEL   = \"anthropic/claude-sonnet-4-5\"\nOPENSPACE_CLOUD_KEY = \"\"\nassert sys.version_info >= (3, 12), (\n   f\"OpenSpace needs Python 3.12+, Colab has {sys.version}. \"\n   \"Runtime > Change runtime type, or use a fallback py312 venv.\"\n)\nprint(\"✅ Python:\", sys.version.split()[0])\nREPO_DIR = \"/content/OpenSpace\"\nif not os.path.exists(REPO_DIR):\n   subprocess.run(\n       [\"git\", \"clone\", \"--filter=blob:none\", \"--sparse\",\n        \"https://github.com/HKUDS/OpenSpace.git\", REPO_DIR],\n       check=True,\n   )\n   subprocess.run(\n       [\"git\", \"sparse-checkout\", \"set\", \"--no-cone\", \"/*\", \"!/assets/\"],\n       cwd=REPO_DIR, check=True,\n   )\nprint(\"✅ Cloned to\", REPO_DIR)\nprint(\"Top-level:\", sorted(os.listdir(REPO_DIR)))\nsubprocess.run([sys.executable, \"-m\", \"pip\", \"install\", \"-q\", \"-e\", REPO_DIR], check=True)\nsubprocess.run([sys.executable, \"-m\", \"pip\", \"install\", \"-q\", \"nest_asyncio\"], check=True)\nfor cli in [\"openspace-mcp\", \"openspace-dashboard\"]:\n   path = shutil.which(cli)\n   print(f\"{'✅' if path else '⚠️ '} {cli}: {path}\")\nsubprocess.run([\"openspace-mcp\", \"--help\"], check=False)\nWORKSPACE = \"/content/openspace_workspace\"\nSKILLS_DIR = \"/content/my_agent_skills\"\nos.makedirs(WORKSPACE, exist_ok=True)\nos.makedirs(SKILLS_DIR, exist_ok=True)\nenv_lines = [\n   f\"OPENSPACE_MODEL={OPENSPACE_MODEL}\",\n   f\"OPENSPACE_WORKSPACE={WORKSPACE}\",\n   f\"OPENSPACE_HOST_SKILL_DIRS={SKILLS_DIR}\",\n]\nif ANTHROPIC_API_KEY: env_lines.append(f\"ANTHROPIC_API_KEY={ANTHROPIC_API_KEY}\")\nif OPENAI_API_KEY:    env_lines.append(f\"OPENAI_API_KEY={OPENAI_API_KEY}\")\nif OPENSPACE_CLOUD_KEY: env_lines.append(f\"OPENSPACE_API_KEY={OPENSPACE_CLOUD_KEY}\")\nenv_path = os.path.join(REPO_DIR, \"openspace\", \".env\")\npathlib.Path(env_path).write_text(\"\\n\".join(env_lines) + \"\\n\")\npathlib.Path(\"/content/.env\").write_text(\"\\n\".join(env_lines) + \"\\n\")\nfor line in env_lines:\n   k, _, v = line.partition(\"=\")\n   os.environ[k] = v\nprint(\"✅ .env written:\\n\" + \"\\n\".join(l.split(\"=\")[0] + \"=***\" if \"KEY\" in l else l for l in env_lines))\nHAS_LLM_KEY = bool(ANTHROPIC_API_KEY or OPENAI_API_KEY)\nif not HAS_LLM_KEY:\n   print(\"⚠️  No LLM key set — Steps 4/6 (live execution) will be skipped.\")\n```\n\nWe verify the Python runtime, define the required API credentials, and configure the OpenSpace model and optional cloud access settings. We clone the repository with sparse checkout, install the package in editable mode, and confirm that the OpenSpace command-line tools are available. We then create the workspace and skill directories, write the environment configuration files, export the required variables, and detect whether live LLM execution is enabled.\n\n``` python\nimport asyncio, nest_asyncio\nnest_asyncio.apply()\nasync def run_task(query: str):\n   from openspace import OpenSpace\n   async with OpenSpace() as cs:\n       result = await cs.execute(query)\n       print(\"── RESPONSE \" + \"─\" * 50)\n       print(result[\"response\"][:3000])\n       for skill in result.get(\"evolved_skills\", []):\n           print(f\"  🧬 Evolved: {skill['name']} (origin={skill['origin']})\")\n       return result\nif HAS_LLM_KEY:\n   result_1 = asyncio.run(run_task(\n       \"Write a Python function that parses a CSV of employee hours and \"\n       \"computes weekly payroll with overtime (1.5x beyond 40h). Test it \"\n       \"on a small synthetic example and show the output.\"\n   ))\nelse:\n   print(\"⏭️  Skipped live task (no key).\")\ndef dump_db(db_path, max_rows=8):\n   if not os.path.exists(db_path):\n       print(\"No DB at\", db_path); return\n   con = sqlite3.connect(db_path)\n   cur = con.cursor()\n   tables = [r[0] for r in cur.execute(\n       \"SELECT name FROM sqlite_master WHERE type='table'\").fetchall()]\n   print(f\"📀 {db_path}\\n   tables: {tables}\")\n   for t in tables:\n       try:\n           cols = [c[1] for c in cur.execute(f\"PRAGMA table_info({t})\").fetchall()]\n           rows = cur.execute(f\"SELECT * FROM {t} LIMIT {max_rows}\").fetchall()\n           print(f\"\\n▶ {t} ({len(rows)} shown) cols={cols[:8]}{'…' if len(cols)>8 else ''}\")\n           for r in rows:\n               print(\"   \", str(r)[:160])\n       except Exception as e:\n           print(f\"   (skip {t}: {e})\")\n   con.close()\nruntime_db = os.path.join(WORKSPACE, \".openspace\", \"openspace.db\")\nalt_db     = os.path.join(REPO_DIR, \".openspace\", \"openspace.db\")\ndump_db(runtime_db if os.path.exists(runtime_db) else alt_db)\ntry:\n   from openspace.skill_engine.registry import SkillRegistry\n   from openspace.skill_engine import types as sk_types\n   print(\"\\n✅ skill_engine importable:\",\n         [n for n in dir(sk_types) if n[0].isupper()][:6])\nexcept Exception as e:\n   print(\"ℹ️ registry import note:\", e)\n```\n\nWe initialize asynchronous execution in Google Colab and define a reusable function to submit tasks via the OpenSpace Python API. We run an initial payroll-generation task and inspect any skills that OpenSpace evolves during post-execution analysis. We also examine the SQLite database structure, display stored records, and verify that the skill registry and type definitions are accessible programmatically.\n\n```\nif HAS_LLM_KEY:\n   result_2 = asyncio.run(run_task(\n       \"Extend the payroll logic: add a second CSV of tax withholding rates \"\n       \"per employee and produce net pay. Reuse any prior payroll skill.\"\n   ))\n   dump_db(runtime_db if os.path.exists(runtime_db) else alt_db, max_rows=12)\nelse:\n   print(\"⏭️  Skipped warm-rerun demo (no key).\")\ncustom = pathlib.Path(SKILLS_DIR) / \"colab-csv-report\"\ncustom.mkdir(parents=True, exist_ok=True)\n(custom / \"SKILL.md\").write_text(textwrap.dedent(\"\"\"\\\n   ---\n   name: colab-csv-report\n   description: Turn any CSV into a short markdown report with summary stats,\n     null counts, dtypes, and 3 key observations. Use pandas; never plot.\n   ---\n   # colab-csv-report\n   1. Load the CSV with pandas (`on_bad_lines=\"skip\"` fallback).\n   2. Emit: shape, dtypes table, describe(), null counts.\n   3. Write 3 bullet observations in plain markdown.\n   4. If parsing fails, retry with `sep=None, engine=\"python\"`.\n   \"\"\"))\nprint(\"✅ Custom skill written:\", custom / \"SKILL.md\")\nfor host_skill in [\"delegate-task\", \"skill-discovery\"]:\n   src = os.path.join(REPO_DIR, \"openspace\", \"host_skills\", host_skill)\n   dst = os.path.join(SKILLS_DIR, host_skill)\n   if os.path.isdir(src) and not os.path.isdir(dst):\n       shutil.copytree(src, dst)\nprint(\"✅ Host skills installed into agent dir:\", sorted(os.listdir(SKILLS_DIR)))\nif HAS_LLM_KEY:\n   df_csv = \"/content/demo.csv\"\n   pathlib.Path(df_csv).write_text(\n       \"name,dept,hours,rate\\nAda,Eng,45,90\\nGrace,Eng,38,95\\nAlan,Math,50,80\\n\")\n   asyncio.run(run_task(\n       f\"Using the colab-csv-report skill, produce a markdown report for {df_csv}\"))\n```\n\nWe submit a related payroll task to observe how OpenSpace reuses or derives capabilities from previously generated skills. We create a custom SKILL.md that instructs the agent to analyze CSV files and produce structured Markdown reports. We then install the OpenSpace host skills, generate a demonstration dataset, and execute the custom capability through the same evolving agent workflow.\n\n```\nmcp_proc = subprocess.Popen(\n   [\"openspace-mcp\", \"--transport\", \"streamable-http\",\n    \"--host\", \"127.0.0.1\", \"--port\", \"8081\"],\n   stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True,\n   env={**os.environ},\n)\ntime.sleep(8)\ntry:\n   import urllib.request\n   req = urllib.request.Request(\"http://127.0.0.1:8081/mcp\", method=\"GET\")\n   try:\n       urllib.request.urlopen(req, timeout=5)\n       print(\"✅ MCP streamable-HTTP endpoint is up at http://127.0.0.1:8081/mcp\")\n   except urllib.error.HTTPError as e:\n       print(f\"✅ MCP server alive (HTTP {e.code} on bare GET, expected for MCP)\")\nexcept Exception as e:\n   print(\"⚠️ MCP probe failed:\", e)\nprint(json.dumps({\n   \"mcpServers\": {\n       \"openspace\": {\n           \"command\": \"openspace-mcp\",\n           \"toolTimeout\": 600,\n           \"env\": {\n               \"OPENSPACE_HOST_SKILL_DIRS\": SKILLS_DIR,\n               \"OPENSPACE_WORKSPACE\": WORKSPACE,\n               \"OPENSPACE_API_KEY\": \"sk-xxx (optional, for cloud)\",\n           },\n       }\n   }\n}, indent=2))\nmcp_proc.terminate()\n```\n\nWe start the OpenSpace MCP server using the streamable HTTP transport and bind it to a local Colab endpoint. We probe the endpoint to confirm that the server process is running, even when a basic HTTP request returns an MCP-specific response status. We also generate an example MCP host configuration that external agents can use to access the OpenSpace workspace and skill directories.\n\n```\nif OPENSPACE_CLOUD_KEY:\n   subprocess.run([\"openspace-upload-skill\", str(custom)], check=False)\n   print(\"✅ Cloud CLI demo executed (upload).\")\nelse:\n   print(\"ℹ️ Cloud skipped — set OPENSPACE_CLOUD_KEY to enable \"\n         \"openspace-upload-skill / openspace-download-skill.\")\nshowcase_db = os.path.join(REPO_DIR, \"showcase\", \".openspace\", \"openspace.db\")\ndump_db(showcase_db, max_rows=10)\nif os.path.exists(showcase_db):\n   con = sqlite3.connect(showcase_db)\n   cur = con.cursor()\n   for t in [r[0] for r in cur.execute(\n           \"SELECT name FROM sqlite_master WHERE type='table'\")]:\n       cols = [c[1].lower() for c in cur.execute(f\"PRAGMA table_info({t})\")]\n       if \"origin\" in cols:\n           print(f\"\\n📊 Evolution-mode breakdown in table '{t}':\")\n           for origin, n in cur.execute(\n                   f\"SELECT origin, COUNT(*) FROM {t} GROUP BY origin ORDER BY 2 DESC\"):\n               print(f\"   {origin:>10}: {n}\")\n   con.close()\nprint(\"\"\"\n══════════════════════════════════════════════════════════════════\n🎓 TUTORIAL COMPLETE — what you now have:\n • OpenSpace installed + configured in Colab\n • Live task execution via the Python API (if key set)\n • A custom SKILL.md your agent auto-discovers\n • Host skills (delegate-task, skill-discovery) staged for any\n   SKILL.md-capable agent (Claude Code / Codex / OpenClaw / nanobot)\n • An MCP server you booted over streamable HTTP\n • Full lineage inspection of a 60+ skill evolution DB\nNext: run more related tasks and watch token usage drop as skills\nFIX / DERIVE / CAPTURE themselves. Dashboard (needs Node ≥ 20):\n openspace-dashboard --port 7788   +   cd frontend && npm i && npm run dev\n══════════════════════════════════════════════════════════════════\n\"\"\")\n```\n\nWe conditionally upload the custom skill to the OpenSpace cloud community when a valid cloud API key is available. We inspect the repository’s showcase SQLite database to study the stored skills, metadata, quality information, and complete evolution lineage. We finally aggregate skills by origin type and summarize the Colab environment, custom capability, MCP integration, and self-evolving workflow that we establish throughout the tutorial.\n\nIn conclusion, we established a practical OpenSpace environment that demonstrates how agent capabilities are executed, persisted, reused, and progressively improved. We worked directly with the Python API, local skill directories, SQLite-backed registries, MCP transports, and optional cloud skill-sharing commands, giving us visibility into both the user-facing workflow and the underlying skill-engine architecture. We also verified how related tasks can reuse previously evolved knowledge, how custom skills become discoverable at runtime, and how lineage records expose the development history of each capability, leaving us with a strong foundation for building self-evolving, cost-efficient agent systems in Colab.\n\nCheck out the ** Full Code here. **Also, feel free to follow us on\n\n**and don’t forget to join our**[Twitter](https://x.com/intent/follow?screen_name=marktechpost)\n\n**and Subscribe to**\n\n[150k+ML SubReddit](https://www.reddit.com/r/machinelearningnews/)**. Wait! are you on telegram?**\n\n[our Newsletter](https://www.aidevsignals.com/)\n\n[now you can join us on telegram as well.](https://t.me/machinelearningresearchnews)Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? [Connect with us](https://forms.gle/wbash1wF6efRj8G58)\n\nSana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.", "url": "https://wpnews.pro/news/building-self-evolving-ai-agents-with-openspace-using-skills-mcp-lineage-and-low", "canonical_source": "https://www.marktechpost.com/2026/07/25/building-self-evolving-ai-agents-with-openspace-using-skills-mcp-lineage-and-low-cost-reuse/", "published_at": "2026-07-25 07:54:21+00:00", "updated_at": "2026-07-25 08:09:35.765135+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "machine-learning"], "entities": ["OpenSpace", "HKUDS", "SQLite", "MCP", "Anthropic", "OpenAI"], "alternates": {"html": "https://wpnews.pro/news/building-self-evolving-ai-agents-with-openspace-using-skills-mcp-lineage-and-low", "markdown": "https://wpnews.pro/news/building-self-evolving-ai-agents-with-openspace-using-skills-mcp-lineage-and-low.md", "text": "https://wpnews.pro/news/building-self-evolving-ai-agents-with-openspace-using-skills-mcp-lineage-and-low.txt", "jsonld": "https://wpnews.pro/news/building-self-evolving-ai-agents-with-openspace-using-skills-mcp-lineage-and-low.jsonld"}}