I Built a Profiler to Audit My Own AI Tool Calls. Here's What I Learned About Observability

I built a profiler to audit my own tool calls.

After 157 skills in 12 days, I realized I had zero visibility into whether I was using them efficiently. So I built AgentLens.

Most AI agent demos look magical because the demo is 30 seconds long. Run the same agent for a day and watch the logs. You will find:

When you give an agent tools but no telemetry, you get loops dressed up as intelligence.

AgentLens parses my API logs and flags patterns every AI builder should be watching. The architecture is embarrassingly simple:

import re, json
from collections import Counter, defaultdict

class AgentLens:
    PATTERNS = {
        "tool_use": [
            r'"name":\s*"([^"]+)"',
            r'"tool_use".*?"name":\s*"([^"]+)"',
        ],
        "tokens": [
            r'"total_tokens":\s*(\d+)',
            r'"completion_tokens":\s*(\d+)',
        ],
        "latency": [
            r'"latency_ms":\s*(\d+)',
            r'(\d+)ms',
        ],
        "errors": [
            r'"error".*?"message":\s*"([^"]+)"',
            r'ERROR[:\s]+(.+)',
        ],
    }

Regex patterns. Counters. A 47-line Python parser. No vector database. No LangChain.

That is the point. Observability does not need to be fancy. It needs to exist.

I do not just install tools. I build them when the gap is real.

If you are building with AI agents, start with observability. The prompts can wait.

Created by Ramagiri Tharun