Parallelize speculative decoding with P-EAGLE on Amazon SageMaker AI
AWS invented Parallel-EAGLE (P-EAGLE), a speculative decoding method that parallelizes draft token generation, achieving up to 1.69x throughput speedup over vanilla EAGLE frameworks. Amazon SageMaker …