JetSpec

JetSpec, a new speculative decoding method, trains a causal parallel draft head over fused hidden states from a frozen target model, enabling lossless verification of candidate trees in one forward pa…

Researchers introduced JetSpec, a speculative decoding method that trains a causal parallel draft head on a frozen target model to draft entire speculative trees in one pass while preserving autoregre…