{"slug": "accelerating-language-model-workflows-with-prompt-choreography", "title": "Accelerating Language Model Workflows with Prompt Choreography", "summary": "TJ Bai and Jason Eisner published \"Prompt Choreography\" in Transactions of the Association for Computational Linguistics Volume 14 (pages 253–270), a framework that maintains a dynamic global KV cache so each LLM call can attend to an arbitrary, reordered subset of previously encoded messages, including parallel calls. The authors report 2.0–6.2× faster time-to-first-token per message and end-to-end speedups above 2.2× in workflows dominated by redundant computation, and show that fine-tuning the LLM to work with the cache helps it mimic results produced by re-encoding messages in a new context.", "body_md": "##### Abstract\n\nLarge language models are increasingly deployed in multi-agent workflows. We introduce Prompt Choreography, a framework that efficiently executes LLM workflows by maintaining a dynamic, global KV cache. Each LLM call can attend to an arbitrary, reordered subset of previously encoded messages. Parallel calls are supported. Though caching messages’ encodings sometimes gives different results from re-encoding them in a new context, we show in diverse settings that fine-tuning the LLM to work with the cache can help it mimic the original results. Prompt Choreography significantly reduces per-message latency (2.0–6.2× faster time-to-first-token) and achieves substantial end-to-end speedups (>2.2×) in some workflows dominated by redundant computation.\n- Anthology ID:\n- 2026.tacl-1.13\n- Volume:\n- [Transactions of the Association for Computational Linguistics, Volume 14](https://aclanthology.org/volumes/2026.tacl-1/)\n- Month:\n- Year:\n- 2026\n- Address:\n- Cambridge, MA\n- Venue:\n- [TACL](https://aclanthology.org/venues/tacl/)\n- SIG:\n- Publisher:\n- MIT Press\n- Note:\n- Pages:\n- 253–270\n- Language:\n- URL:\n- [https://aclanthology.org/2026.tacl-1.13/](https://aclanthology.org/2026.tacl-1.13/)\n- DOI:\n- [10.1162/tacl.a.643](https://doi.org/10.1162/tacl.a.643)\n- Cite (ACL):\n- TJ Bai and Jason Eisner. 2026. [Accelerating Language Model Workflows with Prompt Choreography](https://aclanthology.org/2026.tacl-1.13/) .*Transactions of the Association for Computational Linguistics* , 14:253–270.\n- Cite (Informal):\n- [Accelerating Language Model Workflows with Prompt Choreography](https://aclanthology.org/2026.tacl-1.13/) (Bai & Eisner, TACL 2026)\n- PDF:\n- [https://aclanthology.org/2026.tacl-1.13.pdf](https://aclanthology.org/2026.tacl-1.13.pdf)", "url": "https://wpnews.pro/news/accelerating-language-model-workflows-with-prompt-choreography", "canonical_source": "https://aclanthology.org/2026.tacl-1.13/", "published_at": "2026-10-07 00:00:00+00:00", "updated_at": "2026-10-08 00:17:44.580177+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "ai-agents", "ai-infrastructure", "mlops"], "entities": ["TJ Bai", "Jason Eisner", "Prompt Choreography", "Transactions of the Association for Computational Linguistics", "MIT Press"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/accelerating-language-model-workflows-with-prompt-choreography", "markdown": "https://wpnews.pro/news/accelerating-language-model-workflows-with-prompt-choreography.md", "text": "https://wpnews.pro/news/accelerating-language-model-workflows-with-prompt-choreography.txt", "jsonld": "https://wpnews.pro/news/accelerating-language-model-workflows-with-prompt-choreography.jsonld"}}