# Taobao Live Built AI Hosts That Adapt Without Retraining > Source: > Published: 2026-09-27 10:00:00+00:00 8 min read A human livestream host can change the pitch when a promotion changes. Taobao Live’s AI hosts had a harder time. Update a campaign rule, rename a tool or rewrite a sales instruction and the model could keep behaving as if the old version were still running. That created a real growth problem inside the sales system. The hosts needed to answer product questions quickly enough for a live room, follow the latest merchant and compliance rules, and still move viewers towards product pages and purchases. In an [Alibaba-authored technical report updated on 26 August 2026](https://arxiv.org/abs/2608.15763), the TaoLive AIGC LLM Team describes how it trained a smaller model to respond to changing instructions without retraining for every update. The subsequent Taobao Live A/B test compared the complete system with the platform’s existing ReAct agent across nearly two million participating users. Reliability is the open question with any AI agent making decisions in front of a customer. We examined that risk directly in [our analysis of why AI agents produce false order confirmations](https://industrycontents.com/ai-agent-journey-hallucinations/), a failure mode any digital avatar streamer has to guard against. ## The host had to sell and change at the same time [Taobao Live](https://live.taobao.com/) is Alibaba’s livestream shopping platform. Its digital avatar streamers answer viewer questions, discuss products, follow the live room and use commerce tools while their speech is converted into an animated broadcast. The service has two competing needs. Large general models can interpret unfamiliar instructions, but the paper reports median end-to-end latency above 11 seconds for one strong model the team tested. Compact models respond faster, yet domain training can make them memorize specific skill names, tool descriptions and prompt layouts. That brittleness matters in live commerce. Operators add campaign instructions, update product rules, change tools and revise compliance boundaries. The report says skills are added weekly. Retraining the policy model after every change would slow the commercial team and increase operating cost. The documented hypothesis was that varying the surrounding agent configuration during training would make the compact model read the current instructions instead of relying on memorized labels. The researchers call the method Harness-Aware Training, or HAT. ## The team separated the model from the sales playbook The system split the AI livestream shopping host into two layers. A relatively stable model handled reasoning and response generation. An editable harness held the sales and operating instructions around it. That harness contained four parts. Skills described reply strategies and tool choices. Hooks checked inputs and outputs and could trigger a correction. A prompt pipeline assembled product information, live-room status and active rules. A tool registry connected the agent to product, inventory and commerce services. Operators could change those parts without updating the model weights. The paper describes a human-reviewed loop in which AI clustered failures and proposed edits, while a developer approved the plan, ran an evaluation and decided whether the change should advance. The difficulty was teaching the model to respect a harness that would not stay fixed. During training, the team renamed skills and tools, rewrote descriptions, moved prompt sections, adjusted valid numerical constraints and changed retry behaviour. The underlying task stayed the same while the surface form moved. The compact 35 billion parameter model then went through supervised fine-tuning, on-policy distillation and reinforcement learning in a production-informed live-room simulator. The goal was not simply a stronger AI sales agent. It was an agent that could keep working when its operating environment changed. ## Later edits exposed an early stopping point Before the production experiment, the team tested how far it could improve behaviour by editing the harness while keeping a strong model fixed. On a 482-example development set, the first revision increased accuracy from 82.40 to 92.13 but reduced effectiveness to 84.16. The second revision reached 92.55 for accuracy and 92.75 for effectiveness. Two later rounds added more long-tail rules and made one or both measures worse, so the team stopped at the second version. This failed progression is part of the useful experiment. More rules did not produce a steadily better sales agent. The team needed an explicit checkpoint where the system handled the important cases without accumulating instructions that damaged broader performance. The paper also compared ordinary fixed-harness fine-tuning with HAT. Fixed-harness training improved the live-commerce tasks but reduced performance on a general instruction-following benchmark by 7.7 points. Training across varied harness states largely preserved that ability while improving task performance and robustness to the tested changes. ## A blind test preceded the traffic test The researchers next compared the complete harness system with the existing ReAct-style baseline on 100 real livestream requests. Both systems received the same input, and annotators saw the responses in random order without knowing which system produced them. Sixty-four comparisons were ties. The harness system was preferred in 35 cases and ReAct in one. Among the 36 cases with a preference, that produced a 97.2% reported win rate, with a two-sided binomial test below 0.001. The annotators most often attributed the harness wins to better input understanding, more reliable output and behaviour that suited the scenario. The one ReAct win came from a short ambiguous comment where the harness system returned an irrelevant FAQ-style answer. The blind test gave the team a controlled quality check, but it could not show whether the responses changed shopping behaviour. That required the live comparison. ## Nearly two million users entered the live comparison The Taobao Live A/B test used stable user-level assignment. The ReAct control received 1,581,494 participating unique visitors, or 80% of traffic. The harness treatment received 395,276, or 20%. The total experiment population was 1,976,770. The comparison covered the full production systems. It did not isolate the training method, editable harness, policy model or routing configuration. Any measured difference belongs to the combined treatment. Taobao Live chose two business outcomes. Confirmed-receipt gross merchandise value measured merchandise transactions after the buyer confirmed receipt. Item page views measured visits to product detail pages. Because the traffic split was unequal, the platform divided each outcome by the number of participating users before calculating the relative difference. The company reported confirmed-receipt GMV per participant 4.3340% higher for the harness system than for ReAct. Item page views per participant increased by 0.9107%. The experiment platform classified both as significantly positive. The strongest caveat sits inside the paper. The supplied dashboard snapshot showed one positively significant day for each metric and did not expose the statistical test, threshold, p-values or confidence intervals. The report therefore cannot support a claim about a durable revenue effect. It also withholds the absolute GMV and item-view baselines. ## The test changed the unit of growth work The experiment matters beyond digital avatar streamers because it changes what a commercial team can update. Instead of treating every campaign change as a model-training project, Taobao Live moved more behaviour into reviewed skills, prompts, hooks and tools. That can shorten the distance between finding a sales failure and testing a correction. The paper says behaviour could be adjusted within hours, although it does not provide a before-and-after measure of release speed or engineering cost. A comparable business could test the same operating idea with a narrower system. A customer-service agent could receive changing refund rules. A sales assistant could receive new qualification criteria. A marketplace agent could receive campaign-specific promotion instructions. In each case, the comparison should hold the user population and commercial outcome constant while changing the complete agent system. The conditions for reproduction are demanding. Taobao Live trained a specialised 35 billion parameter model, used real interaction data, built a simulator, calibrated automated judging against humans and served the result on Nvidia H20 hardware. Smaller companies may be able to test modular instructions without reproducing that training stack, but they should not assume the reported revenue lift transfers with the architecture. The next test should separate the sources of the gain. The production comparison establishes that the combined harness treatment outperformed ReAct on the reported snapshot. It does not show whether the lift came from harness-aware training, the modular runtime, better routing or another configuration difference. A factorial test or carefully staged rollout would turn that system-level result into a more reusable growth decision. Two other Alibaba-affiliated platforms ran comparable production tests around the same time. [Kuaishou let an AI agent design and revise its own recommendation experiments](https://industrycontents.com/kuaishou-a-b-agent-growth-tests/), and [AMap replaced part of its homepage ranking with a generative model](https://industrycontents.com/amap-a-b-test-gwhere-homepage-recommendations/). All three tests share the same underlying wager: that a system trained to adapt on its own can outperform a fixed pipeline without slowing the product down. ## Sources - [TaoLive Digital Avatar Agent Technical Report](https://arxiv.org/abs/2608.15763) , TaoLive AIGC LLM Team and co-authors, version 3 dated 26 August 2026 - [Full paper with the production A/B test and caveats](https://arxiv.org/html/2608.15763v3) - [Taobao Live](https://live.taobao.com/) - [Alibaba Group](https://www.alibabagroup.com/)