T4

An engineer's analysis reveals that LLM inference throughput on NVIDIA GPUs is bottlenecked by memory bandwidth, not compute. A 7B model achieves ~15 tok/s on a T4 but over 3,500 tok/s on an H100, a 1…

A developer built nocando, a pre-flight environment-semantics linter for ML workloads that checks whether code is coherent with the declared hardware environment before occupying resources. The tool a…

A developer compared the costs of AI inference on Replicate and deAPI across four common tasks, finding that deAPI's task-based billing is often cheaper than Replicate's time-based or per-unit pricing…