Retrofitting language models to operate over bytes
A Nature paper introduces 'byteification', a two-stage conversion procedure that retrofits existing subword-tokenized large language models into byte-level models with minimal extra training. The resulting byte-level mod…