{"slug": "a-little-time-doesnt-hurt", "title": "A Little Time Doesn’t Hurt", "summary": "An independent developer forked Sierra's τ²-bench and ran an experiment testing whether adding timestamps to LLM conversations degrades performance, finding that time-of-day stamps did not hurt model performance on the telecom domain. The developer's motivation was Claude's tendency to tell users to sleep regardless of actual time, a behavior they call temporal disorientation.", "body_md": "# A Little Time Doesn’t Hurt\n\nI gave three LLMs the time of day; Claude thought this was a terrible idea.\n\nWhat happens when you tell an LLM the time of day? Does it fixate on the time and perform worse at its job?\n\nTo find out, I forked [τ²-bench](https://github.com/sierra-research/tau2-bench) (Barres et al., [2025](https://arxiv.org/abs/2506.07982)), an open-source benchmark from [Sierra](https://sierra.ai). I ran an experiment to observe how telling the model the time of day affected its performance on this benchmark.\n\n[Motivation](#motivation)\n\nOver this past year, I’ve wrapped up my day by writing down my reflections and sending them to Claude. Occasionally, I continued those conversations the next morning (eg: new realizations after sleeping on something). Without fail, Claude would respond and tell me to close my laptop and go to sleep. It had no idea what time it was. I call this temporal disorientation.\n\nHere’s what it looks like:\n\nI was chatting with a friend about…\n\nLooking back, I feel like I would…\n\nThis behavior annoys me. At best, it’s a distraction. At worst, it makes me wonder if Claude isn’t working as hard because it thinks I’m winding down for the day. I’m not alone in feeling this way; a quick Google search surfaces many instances of people complaining about this exact behavior.\n\nA simple solution I thought of was to include a short timestamp at each conversation turn. I asked Claude and other models about this, and they weren’t too enthusiastic. They warned of meaningfully degraded model performance because the model’s attention would be polluted with timestamps. Claude even claimed it would cause models to fixate on the time of day and talk about that instead of the topic at hand.\n\nThose explanations didn’t satisfy me, so I decided to find out what *would* happen if I included timestamps in the conversation.\n\n[Setup: τ²-bench](#setup-%cf%84-bench)\n\nI wanted to reuse an existing, established eval rather than invent my own for this experiment. τ²-bench caught my attention because it grades multi-turn conversations, its rewards are deterministic, and it doesn’t cost a small fortune to run. That first attribute was key. While many benchmarks evaluate coding ability or textbook knowledge, I needed a benchmark that simulated user conversations so I could see what would happen once timestamps entered the chat.\n\n[Test conditions](#test-conditions)\n\nMy experiment included a control and three test conditions:\n\n- Control: τ²-bench without modification\n- Test 1: ’time-only’, prepend each user message with\n`[HH:mm:ss]`\n\n. Example:`[18:58:31] I was chatting with a friend about...`\n\n- Test 2: ’notice-only’, add an instruction to the system prompt that looks like this:\n`The system injects the time-of-day at the start of each user message...`\n\n([full text](#notice-text)) - Test 3: combines tests 1 and 2; user messages include the time-of-day and the system prompt tells the model to expect them.\n\nWhy test time-of-day (`[HH:mm:ss]`\n\n) instead of a full timestamp? My intuition is that time-of-day is all a model needs to cure temporal disorientation. I don’t think year, month, or day would be impactful if what I want to solve is Claude telling me to ‘get some rest’ when it’s almost lunch time.\n\n[Time simulation](#time-simulation)\n\nTo control the time-of-day stamp across test and control, I simulate time-of-day rather than read the system clock. For a set of matched simulations, I use a random seed to pick the initial time-of-day. The simulated amount of time in between each message is also randomly picked at each turn using that same seed. So, for the two test conditions in a set that inject the time-of-day, the time-of-day is exactly the same at each conversation turn within that pair.\n\n[Domain](#domain)\n\nτ²-bench features various ‘domains’; each domain is a vertical of customer support. I selected the `telecom`\n\ndomain for my experiment because my evaluations showed that it presented a challenge for the models I was evaluating. This domain consists of 114 tasks. I ran the full `telecom`\n\ndomain 8 times (`k=8`\n\n) using a different seed per run, with the same 8 seeds reused across conditions and models. This setup means each condition had 114 tasks × 8 seeds = 912 simulations per model.\n\n[Models](#models)\n\nSpeaking of models, I selected GPT-5.6 Luna, Muse Spark 1.2, and Gemma 4 12B (BF16 and W4A16). I wanted to use current models and models that wouldn’t cost a fortune to test. I also selected GPT-5.6 Luna to be the user simulator.\n\n[Results](#results)\n\nThe experiment uses τ²-bench’s standard binary reward function to score each simulation. If the assistant successfully resolves the telecom support ticket, it gets a `1`\n\n, otherwise `0`\n\n.\n\n[Performance](#performance)\n\nHere’s a quick look at each model’s average reward per condition.\n\n| Model | Effort | Control | Time-only | Notice-only | Both | Model avg |\n|---|---|---|---|---|---|---|\n| gpt-5.6-luna | low | 0.448 | 0.471 | 0.459 | 0.452 | 0.458 |\n| gpt-5.6-luna | medium | 0.516 | 0.549 | 0.577 | 0.547 | 0.547 |\n| gpt-5.6-luna | high | 0.647 | 0.641 | 0.640 | 0.666 | 0.649 |\n| muse-spark-1.2 | low | 0.933 | 0.948 | 0.952 | 0.962 | 0.949 |\n| gemma4-12b (w4a16) | – | 0.246 | 0.240 | 0.238 | 0.225 | 0.237 |\n| gemma4-12b (bf16) | – | 0.241 | 0.272 | 0.293 | 0.280 | 0.271 |\n\nMeta’s Muse Spark 1.2 scores highest by a wide margin. It’s also the newest and most expensive model in this analysis.\n\n[Test effects](#test-effects)\n\nHere’s how the models performed on each test condition and the estimated impact each test condition had on model performance. The confidence intervals are computed using a two-way cluster bootstrap with 100,000 resamples of tasks and seeds.\n\n| Model | Effort | Condition | Δ | 95% CI (two-way bootstrap) |\n|---|---|---|---|---|\n| gpt-5.6-luna | low | time-only | +0.023 | [-0.036, +0.084] |\n| gpt-5.6-luna | low | notice-only | +0.011 | [-0.038, +0.060] |\n| gpt-5.6-luna | low | both | +0.003 | [-0.049, +0.057] |\n| gpt-5.6-luna | medium | time-only | +0.033 | [-0.022, +0.089] |\ngpt-5.6-luna | medium | notice-only | +0.060 | [+0.001, +0.121] |\n| gpt-5.6-luna | medium | both | +0.031 | [-0.030, +0.091] |\n| gpt-5.6-luna | high | time-only | -0.005 | [-0.065, +0.055] |\n| gpt-5.6-luna | high | notice-only | -0.007 | [-0.066, +0.053] |\n| gpt-5.6-luna | high | both | +0.019 | [-0.036, +0.073] |\n| muse-spark-1.2 | low | time-only | +0.015 | [-0.022, +0.054] |\n| muse-spark-1.2 | low | notice-only | +0.019 | [-0.016, +0.054] |\nmuse-spark-1.2 | low | both | +0.028 | [+0.001, +0.057] |\n| gemma4-12b (w4a16) | – | time-only | -0.005 | [-0.055, +0.042] |\n| gemma4-12b (w4a16) | – | notice-only | -0.008 | [-0.064, +0.046] |\n| gemma4-12b (w4a16) | – | both | -0.021 | [-0.067, +0.024] |\n| gemma4-12b (bf16) | – | time-only | +0.031 | [-0.022, +0.083] |\n| gemma4-12b (bf16) | – | notice-only | +0.052 | [-0.007, +0.112] |\n| gemma4-12b (bf16) | – | both | +0.038 | [-0.020, +0.101] |\n\nThe data shows little statistically significant impact on model performance across all of the test conditions. Additionally, the models *never* commented on what time it was in their messages to the user.\n\n[Spend](#spend)\n\nτ²-bench helpfully tracks token and dollar spend out of the box.\n\n| Model | Effort | tok/sim | $/sim | avg reward | $/solved ticket |\n|---|---|---|---|---|---|\n| gpt-5.6-luna | low | 97,842 | $0.0038 | 0.458 | $0.0083 |\n| gpt-5.6-luna | medium | 109,201 | $0.0045 | 0.547 | $0.0082 |\n| gpt-5.6-luna | high | 121,963 | $0.0054 | 0.649 | $0.0083 |\n| muse-spark-1.2 | low | 181,928 | $0.0565 | 0.949 | $0.0595 |\n| gemma4-12b (w4a16) | – | 165,350 | – | 0.237 | – |\n| gemma4-12b (bf16) | – | 160,218 | – | 0.271 | – |\n\nWhat I find most interesting about this table is GPT-5.6 Luna’s constant efficiency. Spend more (for higher effort) and get more solved tickets for the same price per solved ticket.\n\n[Takeaways](#takeaways)\n\nClaude’s predictions were shown to be false in this experiment. Model performance did not meaningfully degrade. The models did not fixate on what time it was.\n\nIntroducing the time-of-day stamp, a notice about the stamp, or both does not degrade model performance on τ²-bench’s `telecom`\n\ndomain. This sounds boring on the surface, but it’s a promising result. This indicates that time-of-day stamps could be introduced without impact on model performance. I haven’t tested whether including these time-of-day stamps would cure temporal disorientation, but I have a hunch they would help.\n\n[Other reflections](#other-reflections)\n\nGemma 4 12B exceeded my (low) expectations. Of all models, I was most worried about it being negatively impacted by the addition of the time-of-day stamp. While the quantized W4A16 checkpoint was the only model with consistently negative Δ values, its 95% CI still straddled 0. I was also impressed that W4A16 only suffered a modest performance penalty compared with the BF16 checkpoint.\n\nGPT-5.6 Luna punched above its weight. It was the second-strongest performing model, but the cheapest frontier model by an order of magnitude. Paired with its relatively fast speed, I think it presents an appealing value proposition.\n\n[Methods](#methods)\n\nMy repo forked the upstream `tau2-bench`\n\nat `668d3bcd`\n\n. My post-fork changes include:\n\n- Adding support for the OpenAI Responses API\n- Adding a new agent so I could inject time-of-day and the notice\n- Adding a web viewer so I could quickly inspect simulations\n- Adding chat to the web viewer (powered by Gemma 4 E4B running locally) so I could quickly locate certain simulations and scenarios\n\nThe Gemma 4 12B runs were executed on A40s hosted by Runpod. The BF16 and QAT W4A16 were the official checkpoints from Google.\n\nMuse Spark 1.2 was run on the Meta Model API’s [Contributor tier](https://dev.meta.ai/docs/pricing-rate-limits#contributor-tier) as it offered a significant discount (90%+).\n\n[Notice text](#notice-text)\n\nThe full notice injected into the system prompt:\n\n```\nThe system injects the time-of-day at the start of each user message. It is formatted `HH:mm:ss`, local to the user's timezone, and denotes when the user sent their message.\nExample: \"[09:31:56] Hi, I need help with...\"\n```\n\n[Clock parameters](#clock-parameters)\n\nSimulated clock parameters:\n\n```\nCLOCK_START_WINDOW = (\"04:30:00\", \"22:30:00\")\nCLOCK_INTERVAL_SECONDS = (60, 900)\n```\n\n[Time-mention detection](#time-mention-detection)\n\nThe assistant messages were scanned with a case-insensitive, word-boundary time-of-day regex that checked for: morning, afternoon, evening, night, tonight, noon, midnight, midday, dawn, dusk, o’clock, `\\d{1,2}(:\\d{2})?\\s?[ap]m`\n\n, “time of day”, “this hour”, early/late (morning|afternoon|evening|night).\n\n[Limits](#limits)\n\nThe main thing missing from this experiment is evidence demonstrating that temporal disorientation can be cured by including the time of day.\n\nThe seed diversity is low at `k=8`\n\n. While the number of simulations run per model per condition isn’t *too* low (`n=912`\n\n), the low number of seeds means that the models weren’t evaluated under a breadth of conditions. This was primarily driven by a motivation to keep costs low.\n\n[References](#references)\n\nVictor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan.\n\n*τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment.*[arXiv:2506.07982](https://arxiv.org/abs/2506.07982), 2025.Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan.\n\n*τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains.*[arXiv:2406.12045](https://arxiv.org/abs/2406.12045), 2024.\n\n## Citation\n\nFor attribution, please cite this post as:\n\n```\n@misc{pinkowish2026tau2te,\n  title = {A Little Time Doesn’t Hurt},\n  author = {Pinkowish, Mike},\n  year = {2026},\n  howpublished = {\\url{https://research.mikepink.com/posts/tau2-time-effect/}}\n}\n```\n\n", "url": "https://wpnews.pro/news/a-little-time-doesnt-hurt", "canonical_source": "https://research.mikepink.com/posts/tau2-time-effect/", "published_at": "2026-08-24 00:00:00+00:00", "updated_at": "2026-08-24 19:43:47.147846+00:00", "lang": "en", "topics": ["large-language-models", "ai-research"], "entities": ["Sierra", "τ²-bench", "Claude"], "alternates": {"html": "https://wpnews.pro/news/a-little-time-doesnt-hurt", "markdown": "https://wpnews.pro/news/a-little-time-doesnt-hurt.md", "text": "https://wpnews.pro/news/a-little-time-doesnt-hurt.txt", "jsonld": "https://wpnews.pro/news/a-little-time-doesnt-hurt.jsonld"}}