Beyond Static Prompts: How to Build Self-Improving AI Agents with Closed-Loop Skill Playbooks

A developer has outlined a method for building self-improving AI agents that can dynamically rewrite their own execution playbooks, moving beyond static prompts and rigid tool definitions. The approach, based on the open-source Hermes Agent framework, treats agent skills as closed-loop feedback systems with three components: a deterministic trigger for activation, modular execution logic, and a memory integration loop for self-correction. This architecture enables agents to evaluate their own performance and optimize their instructions over time, rather than requiring manual developer intervention.

The current wave of AI development is undergoing a massive paradigm shift. We are rapidly moving past simple "prompt wrapper" applications and entering the era of fully autonomous, agentic systems. Yet, if you’ve tried to build an AI agent for a production environment, you’ve likely run into a frustrating wall. You write a comprehensive system prompt, equip your agent with a few API tools, and set it loose. It works beautifully on your first three test runs. But on the fourth run, the real world throws a curveball—a changed website structure, an unexpected API response, or a minor user correction—and your agent completely derails. The problem isn't the underlying Large Language Model LLM . The problem is how we define agent capabilities. In most architectures, an agent's "skills" are defined as static, hardcoded instructions or rigid tool definitions. They are passive. To build truly resilient AI systems, we need to treat skills not as static code, but as living, self-contained, closed-loop feedback systems . In this post, we will deconstruct the anatomy of a self-improving agent "Skill" using the architectural patterns of the open-source Hermes Agent framework. We'll explore how to design skills that can execute complex workflows, evaluate their own performance, and dynamically rewrite their own execution playbooks to get smarter over time. The concepts and code demonstrated here are drawn from my ebook Hermes Agent, The Self-Evolving AI Workforce https://tiny.cc/HermesAgent To understand how a self-improving agent works, let’s step away from code for a moment and look at a human analogy: a master craftsperson in a workshop. A master carpenter doesn't approach a new project with a rigid, unchangeable checklist. Instead, they operate with an internal playbook built on experience. This playbook consists of three distinct phases: This feedback loop updates the carpenter's internal playbook. The next time a client triggers a "custom table" request, the execution is smoother, faster, and higher quality. In advanced agent architectures, this is not a vague metaphor—it is a precise, code-level implementation. A Skill is a formalized, stateful playbook that the agent can load, execute, and—crucially— self-modify based on the outcome of its execution. Instead of a developer manually editing prompts in a codebase, the agent acts as its own developer, optimizing its own instructions through a continuous cycle of Invoke → Execute → Review → Update . To build a system capable of this level of autonomy, we must formally decompose a skill into three interdependent components. Trigger Invocation Contract │ ▼ Execution Logic Modular Workflow ◄───┐ Self-Correction / Updates │ │ ▼ │ Memory Integration Feedback Loop ────┘ The Trigger is the input schema that defines exactly when and how a skill is activated. It acts as a strict contract between the agent’s core decision-making loop and the skill’s execution engine. Without a deterministic trigger, agents suffer from unpredictable activation, running the wrong code at the wrong time. This violates the Principle of Least Astonishment POLA : an agent’s behavior must remain highly predictable based on the inputs that activated it. In practice, triggers generally manifest in two ways: /web-search "latest AI trends" , the system scans its available skills, identifies the match, and packages the user's query into a structured payload. deployment skill and triggers it automatically.Once triggered, the skill executes its playbook. The golden rule of agentic execution is modularity . The execution logic must be composed of atomic, chainable steps rather than a single, monolithic "black box" prompt. Consider a complex skill like "Set up a new React project." If you pass this entire request to a single LLM prompt, the model has to generate the directory structure, write the configuration files, install dependencies, and verify the build in one massive, error-prone leap. Instead, a modular playbook breaks the skill down into atomic tool calls: terminal "mkdir my-app && cd my-app" terminal "npx create-react-app ." read file "src/App.js" write file "src/App.js", optimized template Because each step is an atomic tool call, the system can inspect the inputs and outputs of every single transition. If step 2 fails because npm is out of date, the agent doesn't have to restart the entire process; it can isolate the failure to that specific step, run a corrective action, and resume execution. This is where true self-improvement happens. After the execution logic completes, the system must answer a critical question: What did we learn from this run? To handle this, the architecture splits feedback into two distinct systems: Operating in the background, a curation system monitors the agent's entire skill library. It tracks high-level usage metrics: If a skill is rarely used, or if its error rate spikes after a system update, the Curator automatically flags it for deprecation, archiving, or manual developer review. This loop operates on a per-invocation basis. When a skill finishes executing, the agent spawns a background review process. This is a separate, lightweight LLM instance that acts as an objective "critic." The critic reviews the entire execution trace: the initial user request, the steps the agent took, the tool outputs, and the final result. If the critic detects a failure pattern—for example, a web scraper tool failed because a target website updated its CSS selectors—it doesn't just log an error. It uses a management tool to patch the skill's playbook file SKILL.md , updating the instructions with the correct selectors for the next run. Let's look at how this theoretical model plays out step-by-step in a real-world scenario: searching for and extracting web data. User Input: "/gif-search cute cats" │ ▼ ┌─────────────────────────────────────────────────────────┐ │ 1. TRIGGER │ │ - scan skill commands matches "/gif-search" │ │ - Loads "SKILL.md" and packages payload │ └──────────────┬──────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────┐ │ 2. EXECUTION │ │ - Step A: Run web search "cute cats gif" │ │ - Step B: Extract direct image URLs │ │ - Step C: Return formatted markdown link to user │ └──────────────┬──────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────┐ │ 3. MEMORY INTEGRATION Background Review │ │ - Critic detects Step B failed regex extraction error │ │ - Generates a patch to fix the regex pattern │ │ - Writes update back to "SKILL.md" │ └─────────────────────────────────────────────────────────┘ /gif-search cute cats . The system scans the local skills directory, matches the command, parses the YAML metadata in the skill's header, and loads the execution instructions. SKILL.md . It executes a web search tool call, parses the HTML, and attempts to extract the image URLs. However, the target search engine has updated its markup, causing the agent's regex extraction step to fail. The agent tries an alternative fallback method, successfully retrieves a URL, and displays it to the user. SKILL.md file. The next time the user runs /gif-search , the agent executes the corrected logic flawlessly on the first attempt.To bring this concept to life, let’s build a production-ready Skill Discovery Engine in Python. This implementation mirrors the patterns used in the Hermes Agent architecture. It scans a local directory for skill playbooks defined in Markdown, parses their metadata using YAML frontmatter, sanitizes their invocation commands, and indexes them for execution. SKILL.md Before writing the Python parser, here is how a typical self-improving skill playbook is structured. Notice the YAML frontmatter at the top, followed by modular, human-readable execution steps that the LLM can interpret and modify. --- name: gif-search description: Search the web for animated GIFs matching a query and return markdown image links. version: 1.1.0 author: hermes-system tags: media, search, web category: utility platforms: macos, linux, windows --- Playbook: GIF Search Trigger Contract Activated explicitly via /gif-search <query or contextually when the user requests an animated image or reaction GIF. Execution Steps 1. Call web search tool with the query appended with "filetype:gif site:giphy.com OR site:tenor.com". 2. Parse the search results. Use the following regex pattern to extract raw media URLs: https://media\.giphy\.com/media/ a-zA-Z0-9 +/giphy\.gif . 3. If the primary regex fails, fall back to extracting any URL ending with .gif from the page source. 4. Format the output as a standard Markdown image link: Result url . Here is the complete, self-contained Python engine to discover, parse, and index these skill playbooks. """ Basic Skill Library Implementation This module provides a clean, standalone implementation for discovering, indexing, and invoking skills from a local directory. It demonstrates the core patterns used by Hermes Agent's skill management system. Key Features: - Scans a directory for SKILL.md files - Parses YAML frontmatter for metadata name, description, tags - Creates a mapping of skill names to their file paths and metadata - Provides a simple invocation mechanism that returns the skill's content - Includes a reload mechanism to pick up new or changed skills """ import json import logging import os import re import uuid from datetime import datetime from pathlib import Path from typing import Any, Dict, List, Optional, Set, Tuple Configure logging for this module logging.basicConfig level=logging.INFO, format="% asctime s - % levelname s - % message s" logger = logging.getLogger name ─── Constants ─────────────────────────────────────────────────────────── The name of the skill definition file within each skill directory SKILL MD FILENAME = "SKILL.md" Regular expression to match YAML frontmatter blocks Frontmatter is delimited by '---' at the start and end FRONTMATTER PATTERN = re.compile r"^---\s \n . ? \n---\s \n", re.DOTALL Patterns for sanitizing skill names into clean, hyphen-separated slugs This ensures compatibility with slash command naming conventions SKILL INVALID CHARS = re.compile r" ^a-z0-9- " SKILL MULTI HYPHEN = re.compile r"-{2,}" ─── Data Structures ───────────────────────────────────────────────────── class SkillMetadata: """ Represents the parsed metadata from a skill's SKILL.md frontmatter. This class encapsulates all the information needed to identify, describe, and invoke a skill. It follows the Principle of Least Astonishment POLA by providing clear, predictable attribute names and behavior. """ def init self, name: str, description: str = "", version: str = "1.0.0", author: str = "", tags: Optional List str = None, category: str = "general", platforms: Optional List str = None, : """ Initialize a SkillMetadata instance. Args: name: The canonical name of the skill e.g., "gif-search" description: A human-readable description of what the skill does version: Semantic version string default: "1.0.0" author: The creator of the skill tags: A list of searchable tags for the skill category: The functional category of the skill default: "general" platforms: A list of OS platforms this skill supports e.g., "macos", "linux" """ self.name = name self.description = description self.version = version self.author = author self.tags = tags or self.category = category self.platforms = platforms or def to dict self - Dict str, Any : """Convert the metadata to a dictionary for serialization or display.""" return { "name": self.name, "description": self.description, "version": self.version, "author": self.author, "tags": self.tags, "category": self.category, "platforms": self.platforms, } @classmethod def from frontmatter cls, frontmatter: Dict str, Any - "SkillMetadata": """ Create a SkillMetadata instance from a parsed YAML frontmatter dict. This class method handles the extraction of known fields and provides sensible defaults for any missing values. Args: frontmatter: A dictionary of key-value pairs from the SKILL.md frontmatter Returns: A new SkillMetadata instance populated with the frontmatter data """ Extract the name, falling back to a placeholder if missing name = frontmatter.get "name", "" .strip if not name: logger.warning "Skill frontmatter missing 'name' field; using placeholder." name = "unnamed-skill" Extract description, version, author, tags, category, and platforms description = frontmatter.get "description", "" .strip version = str frontmatter.get "version", "1.0.0" .strip author = frontmatter.get "author", "" .strip tags = frontmatter.get "tags", if isinstance tags, str : tags = tag.strip for tag in tags.split "," category = frontmatter.get "category", "general" .strip .lower platforms = frontmatter.get "platforms", if isinstance platforms, str : platforms = p.strip for p in platforms.split "," return cls name=name, description=description, version=version, author=author, tags=tags, category=category, platforms=platforms, class SkillInfo: """ Represents a discovered skill with its metadata, file path, and content. This is the primary data structure returned by the skill scanner. It bundles all the information needed to load and use a skill. """ def init self, metadata: SkillMetadata, skill md path: Path, skill dir: Path, content: str = "", : """ Initialize a SkillInfo instance. Args: metadata: The parsed metadata from the SKILL.md frontmatter skill md path: The absolute path to the SKILL.md file skill dir: The absolute path to the skill's directory content: The full text content of the SKILL.md file optional """ self.metadata = metadata self.skill md path = skill md path self.skill dir = skill dir self.content = content def to dict self - Dict str, Any : """Convert the skill info to a dictionary for serialization.""" return { "name": self.metadata.name, "description": self.metadata.description, "version": self.metadata.version, "author": self.metadata.author, "tags": self.metadata.tags, "category": self.metadata.category, "platforms": self.metadata.platforms, "skill md path": str self.skill md path , "skill dir": str self.skill dir , "content length": len self.content , } ─── Frontmatter Parser ───────────────────────────────────────────────── def parse frontmatter content: str - Tuple Dict str, Any , str : """ Parse YAML frontmatter from a skill file's content. This function extracts the YAML frontmatter block delimited by '---' and returns it as a dictionary, along with the remaining body content. It uses a simple regex-based approach for parsing, which is sufficient for the basic metadata fields used in SKILL.md files. Args: content: The full text content of a SKILL.md file Returns: A tuple of frontmatter dict, body content string """ Attempt to match the frontmatter pattern at the start of the content match = FRONTMATTER PATTERN.match content if not match: No frontmatter found; return an empty dict and the full content as body return {}, content.strip Extract the raw YAML string and the body content raw yaml = match.group 1 body = content match.end : .strip Parse the raw YAML string into a dictionary We use a simple key-value parser for this example frontmatter = {} for line in raw yaml.split "\n" : line = line.strip if not line or line.startswith " " : continue if ":" in line: key, , value = line.partition ":" key = key.strip value = value.strip .strip '"' .strip "'" frontmatter key = value return frontmatter, body def sanitize skill name name: str - str: """ Sanitize a skill name into a clean, hyphen-separated slug. This ensures compatibility with slash command naming conventions e.g., normalizing spaces and underscores to hyphens . Args: name: The raw skill name to sanitize Returns: A sanitized, hyphen-separated slug """ Convert to lowercase and replace spaces/underscores with hyphens slug = name.lower .replace " ", "-" .replace " ", "-" Remove any characters that are not alphanumeric or hyphens slug = SKILL INVALID CHARS.sub "", slug Collapse multiple consecutive hyphens into a single hyphen slug = SKILL MULTI HYPHEN.sub "-", slug Strip leading and trailing hyphens slug = slug.strip "-" return slug ─── Skill Scanner ────────────────────────────────────────────────────── class SkillScanner: """ Scans a directory for SKILL.md files and indexes them as skills. This class is the core of the skill discovery system. It walks a given directory tree, finds all SKILL.md files, parses their frontmatter, and creates a mapping of skill names to their metadata and file paths. The scanner is designed to be efficient and robust, handling missing files, malformed frontmatter, and duplicate skill names gracefully. """ def init self, skills dir: Path : """ Initialize the SkillScanner with a root directory to scan. Args: skills dir: The root directory to scan for skill files """ self.skills dir = skills dir self. skills: Dict str, SkillInfo = {} self. last scan time: Optional datetime = None def scan self - Dict str, SkillInfo : """ Perform a full scan of the skills directory. This method walks the directory tree, finds all SKILL.md files, parses their frontmatter, and builds an in-memory index of skills. It returns a dictionary mapping sanitized skill names to SkillInfo objects. Returns: A dictionary of {sanitized skill name: SkillInfo} """ Reset the skills index before scanning self. skills = {} seen names: Set str = set Ensure the skills directory exists before scanning if not self.skills dir.exists : logger.warning f"Skills directory does not exist: {self.skills dir}" return self. skills Walk the directory tree, looking for SKILL.md files for skill md path in self.skills dir.rglob SKILL MD FILENAME : Skip hidden directories e.g., .git, .github if any part.startswith "." for part in skill md path.parts : continue try: Read the SKILL.md file content content = skill md path.read text encoding="utf-8" Parse the frontmatter and body frontmatter, body = parse frontmatter content Create a SkillMetadata object from the frontmatter metadata = SkillMetadata.from frontmatter frontmatter Sanitize the skill name for use as a slash command sanitized name = sanitize skill name metadata.name Skip duplicate skill names first one wins if sanitized name in seen names: logger.warning f"Duplicate skill name '{sanitized name}' found at " f"{skill md path}; skipping." continue Mark this name as seen seen names.add sanitized name Create a SkillInfo object and add it to the index skill info = SkillInfo metadata=metadata, skill md path=skill md path, skill dir=skill md path.parent, content=content, self. skills sanitized name = skill info logger.info f"Discovered skill: {sanitized name} " f" {metadata.description} " except Exception as e: Log the error but continue scanning other skills logger.error f"Failed to parse skill at {skill md path}: {e}" Record the scan time self. last scan time = datetime.now logger.info f"Scan complete. Found {len self. skills } skill s " f"in {self.skills dir}." return self. skills def get skill self, name: str - Optional SkillInfo : """Retrieve a skill by its sanitized name.""" return self. skills.get sanitize skill name name ─── Demonstration ─────────────────────────────────────────────────────── if name == " main ": import tempfile Create a temporary directory structure to simulate a skill library with tempfile.TemporaryDirectory as temp dir: temp path = Path temp dir Define a mock skill directory and SKILL.md file search skill dir = temp path / "gif-search" search skill dir.mkdir parents=True, exist ok=True mock skill content = """--- name: Gif Search description: Search and retrieve animated GIFs version: 1.0.2 author: Developer-Alpha tags: media, search category: utility platforms: macos, linux --- Playbook: Gif Search This playbook defines how to search and retrieve animated GIFs... """ skill file = search skill dir / SKILL MD FILENAME skill file.write text mock skill content, encoding="utf-8" Initialize and run the scanner scanner = SkillScanner temp path discovered skills = scanner.scan Retrieve and inspect our parsed skill gif skill = scanner.get skill "gif-search" if gif skill: print "\n--- Parsed Skill Metadata ---" print json.dumps gif skill.to dict , indent=2 Let’s break down the key design patterns in the code above and explain why they are critical for building an extensible agent. SkillMetadata The SkillMetadata class is designed around the Principle of Least Astonishment POLA . Frontmatter configurations can often be messy, written by different developers or generated by different LLM versions. The from frontmatter class method acts as a defensive wrapper. It parses raw string tags, normalizes functional categories to lowercase, and provides safe fallbacks for missing critical fields like naming a folderless skill unnamed-skill rather than throwing a fatal runtime error . sanitize skill name In an agentic system, skills are frequently invoked via slash commands e.g., /gif-search . However, file systems and human-entered names are prone to irregularities spaces, mixed casing, underscores, or illegal characters . The sanitize skill name function uses regular expressions to normalize any input into a clean, lowercased, hyphen-separated slug: "Gif Search" becomes "gif-search" "deploy to prod " becomes "deploy-to-prod" This ensures that the invocation contract remains completely deterministic. SkillScanner When scanning a directory containing dozens of skills, one malformed SKILL.md file should not crash the entire application. The SkillScanner.scan method wraps the parsing logic of individual files inside a broad try-except block. If a developer or a background review agent introduces a syntax error into a specific skill's YAML block, the scanner logs the error with details about the offending file, skips it, and continues indexing the rest of the library. This guarantees high system availability in production. By treating skills as self-contained, closed-loop playbooks, we unlock several profound advantages over traditional LLM architectures: Instead of hardcoding edge-case handlings into your application code, you give the agent the tools to debug itself. When an API payload format changes, the agent's background review process updates the corresponding SKILL.md file. The agent adapts without a developer ever having to push a line of code or restart a container. Because skills are packaged as simple, standard Markdown files with YAML headers, they are highly portable. You can version-control your agent's skills in Git, roll back problematic updates, and share skill libraries across entirely different agent instances. Instead of cramming a massive, multi-thousand-token system prompt containing every single tool instruction into every single LLM call, the agent dynamically loads only the specific SKILL.md file required for the active task. This keeps prompt context windows small, speeds up response times, and dramatically reduces API costs. The era of static, hardcoded AI assistants is drawing to a close. To build robust, resilient software agents that can operate autonomously in the real world, we must build them to be self-improving. By decomposing skills into deterministic Triggers , modular Execution Logic , and self-supervised Memory Integration , we create a foundation for continuous learning. The agent ceases to be a static instruction-follower and becomes an adapting craftsman—growing more capable, more efficient, and more resilient with every single run. SKILL.md architecture solve some of your most common runtime errors? Leave a comment below with your thoughts—we’d love to hear how you're approaching the challenge of agent statefulness and self-improvement The concepts and code demonstrated here are drawn directly from the comprehensive roadmap laid out in the ebook Hermes Agent, The Self-Evolving AI Workforce : details link https://tiny.cc/HermesAgent , you can find also my programming ebooks with AI here: Programming & AI eBooks http://tiny.cc/ProgrammingBooks .