Arbitrage: Efficient Reasoning via Advantage-Aware Speculation
UC Berkeley, ICSI, and LBNL researchers introduced ARBITRAGE, a step-level speculative decoding framework that uses a lightweight router to dynamically choose between draft and target model steps, red…