{"slug": "ask-hn-is-anyone-giving-out-tokens-for-benchmarking-llms", "title": "Ask HN: Is anyone giving out tokens for benchmarking LLMs?", "summary": "A Hacker News user is seeking tokens for benchmarking LLMs' performance playing Nethack, with a comment suggesting LMarena (arena.ai) as a possible source, though API access for automation is uncertain.", "body_md": "Hacker News\nnew\n|\npast\n|\ncomments\n|\nask\n|\nshow\n|\njobs\n|\nsubmit\nlogin\nAsk HN: Is anyone giving out tokens for benchmarking LLMs?\n1 point\nby\nemosenkis\n13 minutes ago\n|\nhide\n|\npast\n|\nfavorite\n|\n1 comment\nI'm thinking of benchmarking different models performance playing Nethack and learning the game under different constraints.\nhelp\ncodersfocus\n8 minutes ago\n[–]\nLMarena, now\nhttps://arena.ai\n. Not sure if they have API access to automate it though.\nreply\nConsider applying for YC's Fall 2026 batch!\nApplications\nare open till July 27.\nGuidelines\n|\nFAQ\n|\nLists\n|\nAPI\n|\nSecurity\n|\nLegal\n|\nApply to YC\n|\nContact\nSearch:", "url": "https://wpnews.pro/news/ask-hn-is-anyone-giving-out-tokens-for-benchmarking-llms", "canonical_source": "https://news.ycombinator.com/item?id=49038150", "published_at": "2026-07-24 16:38:17+00:00", "updated_at": "2026-07-24 16:52:26.647820+00:00", "lang": "en", "topics": ["large-language-models"], "entities": ["LMarena", "arena.ai", "Nethack"], "alternates": {"html": "https://wpnews.pro/news/ask-hn-is-anyone-giving-out-tokens-for-benchmarking-llms", "markdown": "https://wpnews.pro/news/ask-hn-is-anyone-giving-out-tokens-for-benchmarking-llms.md", "text": "https://wpnews.pro/news/ask-hn-is-anyone-giving-out-tokens-for-benchmarking-llms.txt", "jsonld": "https://wpnews.pro/news/ask-hn-is-anyone-giving-out-tokens-for-benchmarking-llms.jsonld"}}