{"slug": "i-built-a-profiler-to-audit-my-own-ai-tool-calls-here-s-what-i-learned-about", "title": "I Built a Profiler to Audit My Own AI Tool Calls. Here's What I Learned About Observability", "summary": "Ramagiri Tharun built AgentLens, a 47-line Python profiler that parses API logs to audit AI tool call efficiency after loading 157 skills in 12 days revealed zero visibility into usage patterns. The tool uses regex patterns and counters to flag tool use, token consumption, latency, and errors, demonstrating that effective observability does not require complex infrastructure. Tharun emphasizes that AI agent builders should prioritize telemetry over prompts to avoid loops disguised as intelligence.", "body_md": "I built a profiler to audit my own tool calls.\n\nAfter loading 157 skills in 12 days, I realized I had zero visibility into whether I was using them efficiently. So I built **AgentLens**.\n\nMost AI agent demos look magical because the demo is 30 seconds long. Run the same agent for a day and watch the logs. You will find:\n\nWhen you give an agent tools but no telemetry, you get loops dressed up as intelligence.\n\nAgentLens parses my API logs and flags patterns every AI builder should be watching. The architecture is embarrassingly simple:\n\n``` python\nimport re, json\nfrom collections import Counter, defaultdict\n\nclass AgentLens:\n    PATTERNS = {\n        \"tool_use\": [\n            r'\"name\":\\s*\"([^\"]+)\"',\n            r'\"tool_use\".*?\"name\":\\s*\"([^\"]+)\"',\n        ],\n        \"tokens\": [\n            r'\"total_tokens\":\\s*(\\d+)',\n            r'\"completion_tokens\":\\s*(\\d+)',\n        ],\n        \"latency\": [\n            r'\"latency_ms\":\\s*(\\d+)',\n            r'(\\d+)ms',\n        ],\n        \"errors\": [\n            r'\"error\".*?\"message\":\\s*\"([^\"]+)\"',\n            r'ERROR[:\\s]+(.+)',\n        ],\n    }\n```\n\nRegex patterns. Counters. A 47-line Python parser. No vector database. No LangChain.\n\nThat is the point. **Observability does not need to be fancy. It needs to exist.**\n\nI do not just install tools. I build them when the gap is real.\n\nIf you are building with AI agents, start with observability. The prompts can wait.\n\nCreated by Ramagiri Tharun", "url": "https://wpnews.pro/news/i-built-a-profiler-to-audit-my-own-ai-tool-calls-here-s-what-i-learned-about", "canonical_source": "https://dev.to/tarunai/i-built-a-profiler-to-audit-my-own-ai-tool-calls-heres-what-i-learned-about-observability-128b", "published_at": "2026-05-27 02:22:38+00:00", "updated_at": "2026-05-27 02:53:05.136114+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "ai-infrastructure"], "entities": ["AgentLens", "Ramagiri Tharun", "LangChain"], "alternates": {"html": "https://wpnews.pro/news/i-built-a-profiler-to-audit-my-own-ai-tool-calls-here-s-what-i-learned-about", "markdown": "https://wpnews.pro/news/i-built-a-profiler-to-audit-my-own-ai-tool-calls-here-s-what-i-learned-about.md", "text": "https://wpnews.pro/news/i-built-a-profiler-to-audit-my-own-ai-tool-calls-here-s-what-i-learned-about.txt", "jsonld": "https://wpnews.pro/news/i-built-a-profiler-to-audit-my-own-ai-tool-calls-here-s-what-i-learned-about.jsonld"}}