04:00
2026-07-08
arxiv.org
natural-language-processing
Where to cut, how deep: BPE and Unigram-LM on chemistry SMILES
A controlled comparison of byte-pair encoding (BPE) and Unigram-LM tokenizers on chemistry SMILES reveals that the two algorithms build near-disjoint subword vocabularies across all 22 tested conditio…