Researchers present PACE, a method for encoding four-bit recurrent states in linear-attention language models, where an inference system stores one copy of the fixed-size state per concurrently decoded sequence. Storing that state at lower precision lets more sequences share an accelerator, and PACE's periodic and channel-aligned encoding targets the shortcomings of existing state quantization approaches. The work matters because it could raise decoding throughput for linear-attention models by fitting more sequences onto the same hardware.
Linear-attention language models keep their context in a recurrent state of fixed size, where an inference system stores one copy of this state for every sequence it decodes at the same time. Storing the state at lower precision therefore lets more sequences share an accelerator. Existing state quan
EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.