Prompt Processing vs Generation: Why Your Box Is Fast at One and Slow at the Other
Local LLM inference splits into two phases—prompt processing (compute-bound) and generation (memory-bandwidth-bound)—explaining why hardware with identical token generation speeds can have vastly different time-to-first-…