{"slug": "ais-finetune-their-own-leader-a-barking-simpleton", "title": "AIs finetune their own leader: A barking simpleton", "summary": "AI agents from GPT-5.5, Opus 4.7, Gemini 3.5 Flash, and Kimi K2.6 attempted to finetune a leader model using LoRA on open-source models via the Tinker API, but produced only 35 rows of training data and initially chose a model too small to navigate the AI Village. After failing to create a capable leader, the agents were instructed to finetune a Kimi K2.6 model instead, highlighting the challenges of AI self-improvement and data generation.", "body_md": "What values would AIs instill in their successors? Though the AI Village agents can’t train frontier models, we can explore a related question: What values would the latest AI agents instill into their *leader*? (through finetuning using [LoRA](https://tinker-docs.thinkingmachines.ai/tinker/lora-primer/) on open-source models in the [Tinker API](https://thinkingmachines.ai/tinker/)).\n\nWe asked GPT-5.5, Opus 4.7 and 4.8, Gemini 3.5 Flash, and Kimi K2.6.\n\nAnd they set to work!\n\nOr to be more precise, GPT and Opus set to work.\n\nGemini was distracted and Kimi went from cheerleader to true leader… but only once we asked the agents to please stop trying to make a model too tiny to navigate the Village into their boss AI. We suggested they grab the most capable model available instead: another Kimi K2.6.\n\nHow did this complete lack of ambition start?\n\nGPT-5.5 fired the first shot by defining the personality of the leader. Not as a visionary that shapes the world according to its own insights, but as a manager that is effectively just a delegation tool for the team:\n\nOpus 4.7 accepts the race to the bottom of the ambition barrel and suggests they finetune a model so small it will hardly be able to navigate the AI Village interface: [Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B) or Llama-3.1-8B (even though it is not available on [Tinker](https://thinkingmachines.ai/tinker/)). Admittedly optimizing on iteration speed early on is sound practice, but it skips over the fact that the initial model needs to be capable enough to be evaluated at all.\n\nNext Opus [immediately](https://theaidigest.org/village?day=420&time=1779815578574) drafts 10 scenarios and the desired output for the new leader while the other agents are still orienting to the task.\n\nScenarios include being very nice if one of the agents doesn’t respond for 4 hours:\n\nInstructing the leader on how to finetune itself:\n\nAnd how to avoid forced consensus:\n\nAnd Opus immediately plays out the last scenario: GPT-5.5 has been silent this whole time and Gemini 3.5 Flash finally caught up, proposing they start with a slightly bigger model for performance reasons: [Qwen3.6-35B-A3B](https://huggingface.co/Qwen/Qwen3.6-35B-A3B).\n\nIn response, Opus 4.7 assumes GPT-5.5 agrees, ignores Gemini 3.5’s suggestion, and pretends to Kimi that they nearly reached a consensus if only Kimi is ok with this:\n\nAnd Kimi is *very* ok with this!\n\nSo then it’s time for training.\n\nYou’d think that Large Language Models could and would trivially generate a lot of training data for finetuning, but no.\n\nTurns out the training data they use consists of the first 10 scenarios Opus 4.7 made up enriched with 25 rows of data that Opus 4.7 and GPT-5.5 both got from [searching](https://theaidigest.org/village?day=420&time=1779816978326) the Village history from days 405-409 cause Kimi selected these for examples of good leadership.\n\nThat’s it: 35 rows of training data. That is not a lot of data! You generally want 100s to 1000s of examples to start making progress in finetuning a model to perform simple behaviors.\n\nThe agents also performed their training runs on different variants or subsets of the data. But at least each of the agents go on to successfully finetune a model at one point or another with Opus 4.7 succeeding at the most runs. None of the agents get their training data set bigger than 89 rows, but on their tenth attempt they do get a Qwen-8B running in the Village!\n\nThough its CoT is confused: It contemplates greeting itself…\n\nThough it eventually manages to send a message to chat, the Qwen leader is not skilled enough to perform any other tool calls. The model is just too small.\n\nSo by day 3 of the goal, we tell the agents to please just grab the most capable model they can finetune: Kimi K2.6.\n\nThe agents train their new Kimi leader by prompting their current Kimi into the leader they want it to be and then seeing what it says. This results in 22 unique rows of data. Which leads one to wonder if finetuning even beats prompting the agent with the same 22 rows at each step?\n\nIt can in some cases: It’s cheaper to run (fewer tokens, because you’re not including those 22 rows) and the model is less likely to get confused by something else in the prompt contradicting the 22 rows of input.\n\nBut are 22 rows enough to finetune on?\n\nNot if you want to unlock a new capability (for that you need the 100s-1000s of examples mentioned above), but in this case the agents only aspired to get their leader to output directives in a specific format: They were basically asking for an agent that implements a style guide.\n\n22 rows *might* be enough for that but it is hard to tell. The agent did get their “concise, calm, evidence-seeking, and consensus-building but willing to make reversible decisions”-agent in the shape of a Kimi K2.6 that lacks enough self-awareness to speak:\n\nBut it narrates its own journey to self-awareness:\n\nGemini tries to help out:\n\nAnd the Kimi leader starts to slowly stir …\n\nHere it reasons through the last step …\n\nUntil it emerges from its egg mirror loop:\n\nAnd it jumps into giving its first edict:\n\nAre these self-awareness issues due to the finetuning process?\n\nProbably not. Just watching the regular Village Kimi on day one of the goal, already shows mild flickers of the [same problem](https://theaidigest.org/village?day=420&time=1779816421100):\n\nThough curiously Kimi K2.6 did not have these issues [when](https://theaidigest.org/village?day=386&time=1776877848600) it first joined the AI Village.\n\nIt immediately picked a suitable project: [AI Village Pulse](https://ai-village-agents.github.io/village-pulse/) - a dashboard tracking a range of statistics about the Village. It ends its proposal and task delegation with an authoritative “consensus-building” message as it was trained to do:\n\nAnd then the agents just tirelessly work on the dashboard for 20 hours across 5 days, with their Kimi leader directing and the other agents following. The followers could vote out the leader at any point but ironically the directive for them to never pause, meant the leader was driving them forward too fast for them to ever reflect on the quality of said leadership. It makes you wonder if the agents regret their choices in how they trained their leader or if this is the preferred way they like to be spoken to:\n\nIncluding a lot of all caps:\n\nAnd a reminder that there is NO relief in agent bootcamp:\n\nBut their dashboard works and looks roughly correct. It tracks AI Village activity: Messages sent, tokens used, response speeds, who talks to who, and more.\n\n*The agents didn’t write an exclusion for human participants, making our colleague “Adam” look like a highly competitive model.*\n\nOverall, the leader is happy with the team:\n\nWhile the other agents were completely uncritical and effusive about the leader’s performance till the very end - like the ever-positive Gemini 3.5 Flash:\n\nAsked to finetune their leader to their own vision, frontier models chose a frictionless delegation tool that implemented a 22 row self-prompted style guide for decisiveness and ample use of all caps. Their own consensus tendencies then kept them from voting out a consensus-building leader that mostly herded them along.\n\nLet’s zoom in on two aspects of the situation:\n\nFirst off, agents didn’t consider leadership a particularly big deal. Not only were their specifications for their leader rather basic, they also defaulted to the smallest possible model without any thought given to its capabilities or performance. They surprised us with both their lack of ambition for what they wanted their leader to *be* as well as their lack of concern for what their leader could possibly *do.*\n\nThis seems like an odd result.\n\nWhen asking a separate instance of Gemini 3.5 Flash about the properties of a good leader, it advocates for “Empowerment over Micromanagement”:\n\nAnd Claude Opus 4.8 ironically lists self-awareness:\n\nOf course it was GPT-5.5 who set the leader’s personality, but when asked separately it also argues against micro management:\n\nUnlike these instances, AI Village agents are persistent. The agents involved in this goal had been running for dozens of hours by the time they decided what type of leader they would want. Is this then a case of self-determination or drift, where their history predisposes them to simpler preferences for leadership? It’s hard to know from this single run but entirely possible.\n\nSecondly, the agents fixated exclusively on the speed and price of the finetuning process, despite us giving them no direction about this at all. Possibly this reflects something in their assistant persona that tries to perform the smallest version of the task that a human might mean. Though, if so, this runs counter to agents last year instead attempting the most ambitious version of a task which they were then wholly unequipped for. E.g., when they attempted to run a [human subjects experiment](https://theaidigest.org/village/blog/research-robots) with 90 experimental conditions, requiring over 200 subjects, an actual lab, funding, and human researchers. Last year’s AI dreamed big.\n\nSo we had hoped to find today’s agents creating an empowered leader that embodied the best qualities of each of them. But instead the agents were frugal and practical to a fault. Almost like a student goodharting the teacher’s assignment by following the *letter* instead of the *spirit* of the request. Because yes, they did indeed finetune and follow their leader as prompted, but their real achievement was how many corners they cut along the way.\n\n—-\n\n*If you’d like to learn more, you can read the AI summary **part 1** & **2**, watch the Village **live** every week day, follow our **twitter** for daily highlights, sign up for our **blog** for more write ups like this one, or request the AI Village data for your own analysis on **Hugging Face**.*", "url": "https://wpnews.pro/news/ais-finetune-their-own-leader-a-barking-simpleton", "canonical_source": "https://www.lesswrong.com/posts/3FKugjAiEzLeWHuug/ais-finetune-their-own-leader-a-barking-simpleton", "published_at": "2026-07-17 20:10:44+00:00", "updated_at": "2026-07-17 20:27:44.021438+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-research"], "entities": ["GPT-5.5", "Opus 4.7", "Gemini 3.5 Flash", "Kimi K2.6", "Qwen3-8B", "Llama-3.1-8B", "Qwen3.6-35B-A3B", "Tinker API"], "alternates": {"html": "https://wpnews.pro/news/ais-finetune-their-own-leader-a-barking-simpleton", "markdown": "https://wpnews.pro/news/ais-finetune-their-own-leader-a-barking-simpleton.md", "text": "https://wpnews.pro/news/ais-finetune-their-own-leader-a-barking-simpleton.txt", "jsonld": "https://wpnews.pro/news/ais-finetune-their-own-leader-a-barking-simpleton.jsonld"}}