Air Traffic Control Using Large Language Models: Prompt Engineering, Architecture, and Evaluation

A new arXiv preprint (2608.19299v1) reports that large language models can generate operationally realistic air traffic control transmissions, but only with lightly constrained prompts and in-context examples. In experiments across nine open- and closed-source LLMs, the lightest prompts performed best, while the most heavily scripted prompt collapsed as its own errors accumulated, a problem fixed by injecting correct history. The study used a hand-transcribed general-aviation flight over San Francisco's 'Bay Tour' route as ground truth and validated an LLM-as-judge (GPT-5.5) against human expert annotation.

arXiv:2608.19299v1 Announce Type: new Abstract: Air traffic control ATC communication is a safety-critical dialogue that remains largely human-driven even as other parts of air traffic management have been semi-automated. In this article, we experimentally evaluate whether large language models LLMs can generate operationally realistic ATC transmissions. An experimental general-aviation flight flying over the San Francisco "Bay Tour" route is hand-transcribed and used as ground truth P0 . Through a pilot-in-the-loop process we design five prompt structures P1-P5 of increasing constraint and embed them in a stateful multi-turn pipeline, where the model plays ATC to a fixed pilot transcript while conditioning on the accumulating dialogue history. Across nine open- and closed-source LLMs we vary the prompt, the presence of a worked transcript from a different experimental flight as an in-context example, and whether the model conditions on its own prior replies or on injected ground-truth history. Turns are scored with lexical, structural, and semantic similarity metrics and by an LLM-as-judge GPT-5.5 validated against human expert annotation. Supplying a worked example improves similarity, but tightening the prompt does not: the lightest prompts perform best and the most heavily scripted one collapses as its own errors accumulate through the dialogue, which injecting correct history repairs. These results outline a concrete path and its current limits toward LLM-assisted ATC.