I Built a Profiler to Audit My Own AI Tool Calls. Here's What I Learned About Observability

Ramagiri Tharun built AgentLens, a 47-line Python profiler that parses API logs to audit AI tool call efficiency after loading 157 skills in 12 days revealed zero visibility into usage patterns. The tool uses regex patterns and counters to flag tool use, token consumption, latency, and errors, demonstrating that effective observability does not require complex infrastructure. Tharun emphasizes that AI agent builders should prioritize telemetry over prompts to avoid loops disguised as intelligence.

I built a profiler to audit my own tool calls. After loading 157 skills in 12 days, I realized I had zero visibility into whether I was using them efficiently. So I built AgentLens . Most AI agent demos look magical because the demo is 30 seconds long. Run the same agent for a day and watch the logs. You will find: When you give an agent tools but no telemetry, you get loops dressed up as intelligence. AgentLens parses my API logs and flags patterns every AI builder should be watching. The architecture is embarrassingly simple: python import re, json from collections import Counter, defaultdict class AgentLens: PATTERNS = { "tool use": r'"name":\s " ^" + "', r'"tool use". ?"name":\s " ^" + "', , "tokens": r'"total tokens":\s \d+ ', r'"completion tokens":\s \d+ ', , "latency": r'"latency ms":\s \d+ ', r' \d+ ms', , "errors": r'"error". ?"message":\s " ^" + "', r'ERROR :\s + .+ ', , } Regex patterns. Counters. A 47-line Python parser. No vector database. No LangChain. That is the point. Observability does not need to be fancy. It needs to exist. I do not just install tools. I build them when the gap is real. If you are building with AI agents, start with observability. The prompts can wait. Created by Ramagiri Tharun