How Lossless Is Lossless Speculative Decoding? The Role of Numerical Precision in Orthrus

Orthrus, a hybrid autoregressive-diffusion architecture for accelerating autoregressive language-model inference, claims its intra-model consensus mechanism enables lossless speculative decoding, according to the source. The source examines how numerical precision affects whether that losslessness claim holds in practice.

Orthrus is a hybrid autoregressive-diffusion architecture that accelerates autoregressive language-model inference by generating multiple tokens in parallel while using a frozen autoregressive backbone. Its central claim is that an intra-model consensus mechanism enables lossless speculative decodin