Eradicating AWS OOM Kills: 10M LLM Inferences on a $0 Cloud-Cost O(1) Memory Clamped Architecture

An engineer ran a 10-million-record batch LLM inference pipeline locally on an HP Z4 G4 workstation, completing the workload in 10.9 hours at roughly 254.9 requests per second with a 100% success rate and zero cloud compute cost. The pipeline held memory flat between 6.72GiB and 9.4GB (O(1) constant space) by substituting libjemalloc2 via LD_PRELOAD to bypass glibc heap fragmentation, and finished with a 3.4GB SQLite WAL that passed PRAGMA integrity_check.

High cloud API costs and Out-Of-Memory OOM failures in large-scale data pipelines are architectural defects, not hardware constraints. Standard cloud approaches force teams to continually scale up high-memory instances e.g., AWS r6i/r7i to survive memory leaks and heap fragmentation. This report documents the performance and execution of a 10,000,000-record batch LLM inference pipeline executed locally on an HP Z4 G4 workstation Intel Xeon W-2155, 128GB ECC DDR4 RAM, NVIDIA RTX 5060 Ti 16GB VRAM, NVMe PCIe Gen4 SSD . Verified Benchmark Telemetry - Total Executed Workload: 10,000,000 10M Async LLM / Batch Requests - Success Rate: 100.00% 0.00% error rate, 0 dropped requests - Total Runtime: 10.9 hours continuous execution - Average Throughput: ~254.9 req/s - Memory Bounds: Clamped flatline between 6.72GiB and 9.4GB RAM O 1 constant space - Database Integrity: 3.4GB SQLite WAL PRAGMA integrity check: ok - Cloud Compute Cost: $0.00 Technical Architecture & Memory Enforcement 1. Memory Allocator Substitution libjemalloc2 Standard glibc malloc exhibits severe heap fragmentation under sustained Python allocation and deallocation cycles, causing process RSS growth and eventual OOM terminations. Injecting libjemalloc2 via LD PRELOAD bypasses glibc allocation delays.