AIPerf is the only way I've found to get real LLM inference benchmarks
AIPerf is a dedicated benchmarking tool that measures LLM inference performance by generating concurrent request loads, according to a first-person account of using it to test deployments. The account states that manual …