Building a speech editing API on AuK-Flash: the knobs it ignores and the wording it doesn't

A developer wrapping Tencent's AuK-Flash speech editing model as an API found that a working install needs about 18.75 GB of memory rather than the 6.8 GB listed on the model page, because instruction understanding runs through Alibaba's Qwen2.5-Omni-3B encoder, pushing peak VRAM to about 20.3 GB and requiring a 24 GB card. The writeup also reports that the inference code ignores step-count and guidance parameters passed to Flash, that fixed prompt templates matter far more than free-form text (one template misread the description aloud in 8 of 8 tries versus 7 of 8 correct with another), and that the Qwen Research Licence on the encoder blocks commercial use, with a collaborator on GitHub issue #13 saying a permissively licensed encoder is planned.

Speech pipelines usually chain separate tools: text to speech, cloning, noise removal, speaker separation. AuK-Flash https://huggingface.co/tencent/AuK-Flash replaces the chain with one model driven by a sentence. You send Replace 'twelve percent' with 'eighteen percent'. and a recording, and the same voice comes back saying eighteen. On an NVIDIA RTX A5000 that edit kept the graphics card GPU busy for 0.73 seconds. AuK and AuK-Flash come from Ziyang Ma and 32 co-authors at Shanghai Jiao Tong University, Tencent Hunyuan and the Shanghai Innovation Institute. Our AuK-Flash test on a 24 GB card https://inferencekey.com/news/auk-speech-generation-editing-one-model has the measurements. This piece covers the decisions you face wrapping it as an API. The AuK-Flash page lists about 6.8 GB. That is not what you deploy. AuK understands your instruction through a second model, Alibaba's Qwen2.5-Omni-3B the "encoder" , which weighs about 12 GB more. A working install pulls 18.75 GB. The memory on the card VRAM follows. We saw about 15.6 GB in use once loaded and about 20.3 GB at peak. Our own example's .env.example says "about 17 GB of free VRAM", which invites a 16 GB card. Plan for 24 GB. A CPU offload setting, which parks part of the model in ordinary system RAM, cut the peak to about 11.3 GB. Each request then took 13 to 15 seconds, longer than the audio it produced. It might fit a 16 GB card, untested. The encoder also carries the licence question. AuK's code and weights are MIT, but Qwen2.5-Omni-3B is under the Qwen Research Licence, which does not allow commercial use. On issue 13 https://github.com/Tencent-Hunyuan/AuK/issues/13 , project collaborator ZhikangNiu replied that the team plans a permissively licensed encoder in a future release. Settle this before a product depends on it. AuK sharpens audio from a blurry sketch in repeated passes. The base model does 32, each also computing an unguided version to push the result toward your instruction that push is "guidance" . AuK-Flash was trained to need 4 passes and no extra version: 4.5 times faster, by the authors' measurement. The inference code what runs the trained model on each request hard-codes that recipe and ignores any step count or guidance strength you pass Flash. A source comment says re-enabling guidance "blows up the amplitude". If your wrapper exposes those parameters, hide them or label them base-model only. In voice design you describe a voice in text and give it a line. Two phrasings that look interchangeable behave very differently: Based on the following description: "...", generate speech content "...". Generate speech based on the following description: "...". The content to speak is: "...". The first comes from the project's command-line example: 7 of 8 English voice designs came out exactly right. The second is the cookbook's English template, and in 8 of 8 tries the model spoke part of the description out loud. The technical report recommends a separate language model in front to rewrite requests into the trained phrasing. We ran without one. So never forward free-form text. Keep fixed templates on the server, fill the slots, and test each template like an endpoint. A single model is one artifact to ship and patch, and adding a task means writing an instruction, not integrating a tool. The price is concurrency requests running at once . Our adapter, the code that loads the model and answers requests, allows one GPU job at a time: four edits fired together came back in turn, the last after 5.8 seconds. Warm calls model loaded, first request done used 0.4 to 0.8 seconds of GPU, so short edits queue fine. A long job blocks the next caller. The cold start, the time from asking for the model to it being ready to serve, was 71.6 seconds on a GPU machine already connected to InferenceKey with no model files on it. The cold start and per-task timings for AuK-Flash https://inferencekey.com/news/auk-speech-generation-editing-one-model break that down step by step. We transcribed every output with OpenAI's Whisper large-v3-turbo and diffed the text. No edit touched the words around the change: a cheap regression test for any speech editing service. Text cannot tell you whether the pitch went up, the whisper sounds like one, or the swapped word has an audible join. That needs an ear or a pitch measurement, and we used neither. Our inputs were synthetic voices from Kyutai's Pocket TTS see our Pocket TTS Spanish voice on CPU https://inferencekey.com/research/pocket-tts-spanish-cpu-voice , the friendliest possible case. Noisy rooms and overlapping speakers remain untested. Training covered Chinese and English only; our Spanish attempt was partly garbled. Benchmark scores 87.5 correct edits in 100 on SpeechEditBench are the authors', not reproduced. Our run was around 60 requests, nearly all single attempts, none crashed. No ready-made InferenceKey backend handles AuK yet, so we wrote that small adapter ourselves. It pins Tencent's code, caps outputs at 30 seconds and answers on the chat address, because our text-to-speech address cannot yet return audio from a custom adapter, a gap on our side. The AuK team built both models and wrote the report and the cookbook in the AuK repository https://github.com/Tencent-Hunyuan/AuK . SGLang-Omni, vLLM-Omni and 0xShug0's audio.cpp each added ways to run AuK. Alibaba's Qwen team made the encoder. Issue reporters raised the licence, Polish and shouting questions. We spotted the release in AI Search's news video. The full AuK-Flash write-up on inferencekey.com https://inferencekey.com/news/auk-speech-generation-editing-one-model has the per-task table, every audio sample before and after, the platform issues we hit and the SDK example that chains a design, clone, edit and speed-up.