05:25
2026-09-24
dev.to
machine-learning
In 2018 I hand-wrote a C++ deep learning framework so I'd never pad a batch. In 2023 LLM serving landed on the same structure.
A developer who built InsNet, a ~21,000-line C++14 deep learning library with 4,828 lines of hand-written CUDA kernels, starting in 2018, describes how its padding-free dynamic batching design anticip…