Parakeet.java: Automatic Speech Recognition in Pure Java

A pure-Java implementation of NVIDIA's Parakeet automatic speech recognition model, called Parakeet.java, matches the reference parakeet.cpp engine word-for-word at F16 quantization with 0.00% WER difference on LibriSpeech test-clean, according to benchmarks published by developer qxoticai. On an AMD Ryzen 7 PRO 8840U laptop with 8 threads, the tdt-0.6b-v3 Q4_K model reached 25.1x real-time factor versus parakeet.cpp's 12.6, while the smaller tdt_ctc-110m Q4_K model hit 81.2x versus 45.8x, transcribing an hour of speech in 44 seconds. The implementation runs without ONNX Runtime, whisper.cpp or a Python runtime, supports streaming transcription, word timings and confidence scores, and offers GraalVM Native Image support that loads a 600M model in about 0.2 seconds.

Automatic Speech Recognition for the JVM An implementation of NVIDIA Parakeet https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3 for the JVM: blazing fast on ordinary CPUs, competitive with the native implementations. No ONNX Runtime, no whisper.cpp, no Python runtime. Just the JVM. <sub Live transcription of JFK's 1961 inaugural address. Play it with sound. https://cdn.jsdelivr.net/gh/qxoticai/assets@e1fe3eacf469cf2962083a924d6daf260759e926/qxotic/parakeet-live-transcription.mp4 </sub - Competitive with the reference engines. Keeps pace with parakeet.cpp https://github.com/mudler/parakeet.cpp and sherpa-onnx https://github.com/k2-fsa/sherpa-onnx , word for word identical. - Matches the reference with 100% accuracy. At F16 the transcript matches parakeet.cpp https://github.com/mudler/parakeet.cpp exactly; the rest is quantization noise, not drift. - Streaming support. Audio in as it arrives, text out in pieces, with a draft tail that keeps up with the speaker. - Word timings and confidence. Every token carries its audio span and how sure the decoder was. - Multilingual. Parakeet v3 transcribes and punctuates without a language flag. - First-class support for GraalVM's Native Image. Low memory footprint, standalone images, fast startup e.g. 600M model is loaded and ready to transcribe in about 0.2 s. LibriSpeech test-clean, first 100 utterances 901 s , 8 threads on an AMD Ryzen 7 PRO 8840U laptop, median of 3 runs. The last column scores each run against parakeet.cpp's transcript rather than against the reference, so 0% means both engines heard exactly the same words. How to reproduce this, step by step https://github.com/qxoticai/qxotic/blob/main/jinfer/jinfer-parakeet/WER.md . | Model | Engine | WER | RTFx | WER vs. parakeet.cpp | |---|---|---|---|---| | tdt-0.6b-v3 Q4 K | jinfer | 2.04% | 25.1 | 0.17% | | | parakeet.cpp | 2.04% | 12.6 | - | | tdt-0.6b-v3 Q8 0 | jinfer | 2.04% | 21.6 | 0.04% | | | parakeet.cpp | 2.04% | 14.9 | - | | tdt-0.6b-v3 F16 | jinfer | 2.04% | 13.6 | 0.00% | | | parakeet.cpp | 2.04% | 15.3 | - | | tdt ctc-110m Q4 K | jinfer | 1.99% | 81.2 | 0.34% | | | parakeet.cpp | 2.12% | 45.8 | - | | v3 int8 ONNX | sherpa-onnx | 2.16% | 18.6 | 1.27% | RTFx is seconds of audio transcribed per wall second, model load excluded. The small model transcribes an hour of speech in 44 seconds. For the full matrix - both models, three engines, five quantizations, 1 to 16 threads and three JVMs, on a second machine - see BENCHMARKS.md https://github.com/qxoticai/qxotic/blob/main/jinfer/jinfer-parakeet/BENCHMARKS.md . | Checkpoint | Languages | Parameters | GGUF Q8 0 | |---|---|---|---| | parakeet-tdt-0.6b-v3 https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3 | multilingual | 600M | tdt-0.6b-v3-q8 0.gguf https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/tdt-0.6b-v3-q8 0.gguf?download=true | | parakeet-tdt-0.6b-v2 https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2 | English | 600M | tdt-0.6b-v2-q8 0.gguf https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/tdt-0.6b-v2-q8 0.gguf?download=true | | parakeet-tdt-1.1b https://huggingface.co/nvidia/parakeet-tdt-1.1b | English | 1.1B | tdt-1.1b-q8 0.gguf https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/tdt-1.1b-q8 0.gguf?download=true | | parakeet-tdt ctc-1.1b https://huggingface.co/nvidia/parakeet-tdt ctc-1.1b | English | 1.1B | tdt ctc-1.1b-q8 0.gguf https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/tdt ctc-1.1b-q8 0.gguf?download=true | | parakeet-tdt ctc-110m https://huggingface.co/nvidia/parakeet-tdt ctc-110m | English | 110M | tdt ctc-110m-q8 0.gguf https://huggingface.co/mudler/parakeet-cpp-gguf/resolve/main/tdt ctc-110m-q8 0.gguf?download=true | Every checkpoint also ships Q4 K , Q5 K , Q6 K and F16 in mudler/parakeet-cpp-gguf https://huggingface.co/mudler/parakeet-cpp-gguf , named the same way: swap the quantization in the file name. Q8 0 balances quality and size, Q4 K runs quickest. The CLI downloads and caches them by reference, so no manual download is needed: jinfer pull mudler/parakeet-cpp-gguf/tdt-0.6b-v3-q8 0.gguf jinfer -m mudler/parakeet-cpp-gguf/tdt-0.6b-v3-q8 0.gguf --transcribe speech.wav Any format ffmpeg reads, resampled by jinfer-codecs . The transcript goes to stdout, so it pipes. Raw 16 kHz mono PCM on stdin, transcribed as it arrives. The microphone comes from ffmpeg, so the capture flags are the only part that differs: Linux PulseAudio or PipeWire ffmpeg -nostats -loglevel error -f pulse -i default -ar 16000 -ac 1 -f s16le - \ | jinfer -m mudler/parakeet-cpp-gguf/tdt-0.6b-v3-q4 k.gguf --transcribe - macOS AVFoundation; ffmpeg -f avfoundation -list devices true -i "" names the inputs ffmpeg -nostats -loglevel error -f avfoundation -i ":0" -ar 16000 -ac 1 -f s16le - \ | jinfer -m mudler/parakeet-cpp-gguf/tdt-0.6b-v3-q4 k.gguf --transcribe - Windows DirectShow; ffmpeg -list devices true -f dshow -i dummy names the inputs ffmpeg -nostats -loglevel error -f dshow -i audio="Microphone" -ar 16000 -ac 1 -f s16le - ^ | jinfer -m mudler/parakeet-cpp-gguf/tdt-0.6b-v3-q4 k.gguf --transcribe - On a terminal, stderr shows the live view: final words settle into the scrollback, each committed piece lands in color and fades into the text, the provisional tail follows in grey italics with a band of light sweeping through it, and a waveform dances with the voice. Doubtful words are flagged, so a name the model is unsure of stands out. --theme mint|nord|catppuccin|ember|frost|mono picks the palette; colors degrade to 256, to 16, and to plain attributes under NO COLOR . Redirected, the same run logs final text and partials as plain lines, so it scripts. Through LangChain4j https://github.com/qxoticai/qxotic/blob/main/jinfer/jinfer-langchain4j/README.md , which resolves the model ref and downloads it on first use: js try var transcriber = JinferTranscriptionModel.builder .model "mudler/parakeet-cpp-gguf/tdt-0.6b-v3-q8 0.gguf" .build { Transcription transcription = transcriber.transcribe Path.of "speech.wav" ; System.out.println transcription.text ; for Transcription.Word word : transcription.words System.out.printf "%s %.2f %s%n", word.start , word.confidence , word.text ; } Or through Spring AI https://github.com/qxoticai/qxotic/blob/main/jinfer/jinfer-spring-ai/README.md , with jinfer-spring-ai on the classpath: js try var transcriber = JinferTranscriptionModel.builder .model "mudler/parakeet-cpp-gguf/tdt-0.6b-v3-q8 0.gguf" .build { AudioTranscriptionResponse response = transcriber.call new AudioTranscriptionPrompt new FileSystemResource "speech.wav" ; System.out.println response.getResult .getOutput ; } Live audio needs jinfer-parakeet directly: feed samples as they arrive, print the final pieces, and poll the provisional tail for a responsive UI. static <S extends RuntimeState void live TranscriptionModel<?, ?, S model, float pcm { try S state = model.newState ; TranscriptionStream stream = model.stream state { int chunk = model.sampleRate / 2; // feed half a second at a time for int at = 0; at < pcm.length; at += chunk { Transcription piece = stream.feed pcm, at, Math.min chunk, pcm.length - at ; System.out.print piece.text ; // final text, often empty, never revised System.err.print "\r" + stream.partial .text ; // provisional, replaced next time } System.out.println stream.finish .text ; // the rest, final } } Final pieces never change and join, in order, into the whole transcript. Token times are offsets from the start of the stream. Audio is decoded in chunks with context on both sides, as NeMo streams Parakeet: 10 s of left context, a 2 s chunk, 2 s of right context. Only the chunk's tokens become final, while the decoder carries its state across chunks, so words continue across boundaries. Final text trails the speaker by 2 to 4 s; the provisional tail, decoded over a shorter window, refreshes twice a second while speech comes in. Offline transcription runs the same path with 46 s chunks, so a two hour recording never holds a two hour attention matrix. Parakeet v3 emits cased, punctuated text and one timing per token. It consumes mono 16 kHz audio; anything else is refused rather than silently resampled, since a rate mismatch quietly degrades recognition. Short windows around digital silence can blank NVIDIA-NeMo/Speech 15757 https://github.com/NVIDIA-NeMo/Speech/issues/15757 , so live recording with a room noise floor transcribes better than padded silence.