{"slug": "different-agent-frameworks-log-input-tokens-differently-here-s-the-fix", "title": "Different agent frameworks log 'input_tokens' differently. Here's the fix.", "summary": "A developer created trace-field-normalize, an open-source Python library that standardizes inconsistent field names across different agent frameworks. The tool maps variants like `input_tokens`, `prompt_tokens`, and `tokens_prompt` to a single canonical `tokens_in` field, and supports custom field mappings with optional preservation of original names. Built with only Python standard library dependencies, the library reads and normalizes trace data line by line from JSONL files without modifying source events.", "body_md": "*This is a submission for the Hermes Agent Challenge.*\n\nI run agents on three different frameworks. Each one logs differently. One uses `input_tokens`\n\n. Another uses `prompt_tokens`\n\n. A third uses `tokens_prompt`\n\n. My trace analysis code had to handle all three. Every new framework meant another alias to add everywhere.\n\nThat's `trace-field-normalize`\n\n.\n\n``` python\nfrom trace_field_normalize import normalize_event\n\nevent = {\"input_tokens\": 42, \"latency_ms\": 350, \"model_id\": \"claude-sonnet-4-5\"}\nresult = normalize_event(event)\n# {\"tokens_in\": 42, \"duration_ms\": 350, \"model\": \"claude-sonnet-4-5\"}\n```\n\nUnknown fields pass through unchanged. Nothing is dropped.\n\n``` python\nfrom trace_field_normalize import normalize_file\n\nevents = normalize_file(\"raw_traces.jsonl\", \"normalized.jsonl\")\n```\n\nReads the source line by line, normalizes each event, writes to dest. Blank lines are skipped.\n\n| Canonical | Variants |\n|---|---|\n`tokens_in` |\n`input_tokens` , `prompt_tokens` , `tokens_prompt`\n|\n`tokens_out` |\n`output_tokens` , `completion_tokens` , `tokens_completion`\n|\n`cost_usd` |\n`cost` , `price_usd` , `usd` , `price`\n|\n`duration_ms` |\n`latency_ms` , `elapsed_ms` , `duration` , `latency`\n|\n`kind` |\n`event_type` , `type` , `event_kind`\n|\n`name` |\n`step` , `tool` , `tool_name` , `function_name`\n|\n`error` |\n`err` , `exception` , `error_message`\n|\n`model` |\n`model_id` , `model_name`\n|\n`lane` |\n`worker` , `agent_id` , `thread`\n|\n`timestamp` |\n`ts` , `time` , `created_at` , `event_time`\n|\n\n``` python\nfrom trace_field_normalize import FieldMap, normalize_event\n\nfm = FieldMap(\n    {\"score\": [\"rating\", \"confidence\"]},\n    include_defaults=True,  # keep the built-in map too\n)\n\nevent = {\"rating\": 0.95, \"input_tokens\": 100}\nnormalize_event(event, fm)\n# {\"score\": 0.95, \"tokens_in\": 100}\n```\n\n`include_defaults=False`\n\ngives you a clean slate.\n\nIf the event already has the canonical name, variants are left alone:\n\n```\nevent = {\"tokens_in\": 99, \"input_tokens\": 42}\nnormalize_event(event)\n# {\"tokens_in\": 99, \"input_tokens\": 42}\nnormalize_event({\"ts\": 1700000000}, keep_original=True)\n# {\"timestamp\": 1700000000, \"ts\": 1700000000}\n```\n\nUseful when downstream code still uses the old name.\n\n``` python\nfrom trace_field_normalize import normalize_events\n\nevents = normalize_events(raw_event_list)\n```\n\nReturns a new list. Originals are not modified.\n\nStandard library only: `json`\n\n, `dataclasses`\n\n, `pathlib`\n\n. Nothing else.\n\n```\ngit clone https://github.com/MukundaKatta/trace-field-normalize\ncd trace-field-normalize && pip install -e .\n```\n\n", "url": "https://wpnews.pro/news/different-agent-frameworks-log-input-tokens-differently-here-s-the-fix", "canonical_source": "https://dev.to/mukundakatta/different-agent-frameworks-log-inputtokens-differently-heres-the-fix-igk", "published_at": "2026-05-28 21:06:18+00:00", "updated_at": "2026-05-28 21:25:52.455685+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "ai-infrastructure", "mlops", "large-language-models"], "entities": ["Hermes Agent", "Claude Sonnet"], "alternates": {"html": "https://wpnews.pro/news/different-agent-frameworks-log-input-tokens-differently-here-s-the-fix", "markdown": "https://wpnews.pro/news/different-agent-frameworks-log-input-tokens-differently-here-s-the-fix.md", "text": "https://wpnews.pro/news/different-agent-frameworks-log-input-tokens-differently-here-s-the-fix.txt", "jsonld": "https://wpnews.pro/news/different-agent-frameworks-log-input-tokens-differently-here-s-the-fix.jsonld"}}