{"slug": "gpt-6-1-sol-seems-to-do-worse-on-long-tail-languages", "title": "GPT 6.1 Sol seems to do worse on long-tail languages.", "summary": "A Hacker News commenter using the handle srcreigh reported that GPT 6.1 Sol repeatedly made basic mistakes on a Prolog project, as if the model did not grasp symbolic resolution, and then tagged the programming languages in all 712 GPT 6.1 Sol battles logged by the OpenCode agent-duel battle system using a tool called Jev to test whether the model performs worse outside common programming languages. The tagging drew on requests and available conversation context without showing Jev the outcomes, and the commenter presented the 712-battle dataset as the basis for examining long-tail language performance.", "body_md": "[srcreigh](https://news.ycombinator.com/user?id=srcreigh) wrote in a [Hacker News comment](https://news.ycombinator.com/item?id=50489509):\n\nThe ideal language is the one the agent knows how to use.\n\nI used prolog for a project and GPT 6.1 Sol repeatedly made dumb mistakes as if it doesn’t understand the symbolic resolution capabilities.\n\nThat made me curious: does Sol do worse when it’s working outside the usual programming languages?\n\nWe have some data to look at! We’re working on a [battle system for OpenCode](https://github.com/bottomless/agent-duel), which lets users compare models on their actual tasks. I used [Jev](https://openrouter.ai/docs/guides/community/jev) to tag the programming languages in all 712 Sol battles, using the requests and available conversation context without showing it the outcomes.", "url": "https://wpnews.pro/news/gpt-6-1-sol-seems-to-do-worse-on-long-tail-languages", "canonical_source": "https://micjm.com/blog/gpt-6.1-sol-long-tail-languages/", "published_at": "2026-11-13 22:41:09+00:00", "updated_at": "2026-11-13 22:58:34.760954+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "ai-research"], "entities": ["GPT 6.1 Sol", "srcreigh", "OpenCode", "agent-duel", "Jev", "Prolog", "Hacker News"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/gpt-6-1-sol-seems-to-do-worse-on-long-tail-languages", "markdown": "https://wpnews.pro/news/gpt-6-1-sol-seems-to-do-worse-on-long-tail-languages.md", "text": "https://wpnews.pro/news/gpt-6-1-sol-seems-to-do-worse-on-long-tail-languages.txt", "jsonld": "https://wpnews.pro/news/gpt-6-1-sol-seems-to-do-worse-on-long-tail-languages.jsonld"}}