Eradicating AWS OOM Kills: 10M LLM Inferences on a $0 Cloud-Cost O(1) Memory Clamped Architecture

High cloud API costs and Out-Of-Memory (OOM) failures in large-scale data pipelines are architectural defects, not hardware constraints. Standard cloud approaches force teams to continually scale up high-memory instances (e.g., AWS r6i/r7i) to survive memory leaks and heap fragmentation.

This report documents the performance and execution of a 10,000,000-record batch LLM inference pipeline executed locally on an HP Z4 G4 workstation (Intel Xeon W-2155, 128GB ECC DDR4 RAM, NVIDIA RTX 5060 Ti 16GB VRAM, NVMe PCIe Gen4 SSD).

Verified Benchmark Telemetry

- Total Executed Workload: 10,000,000 (10M) Async LLM / Batch Requests
- Success Rate: 100.00% (0.00% error rate, 0 dropped requests)

• Total Runtime: 10.9 hours continuous execution

- Average Throughput: ~254.9 req/s
- Memory Bounds: Clamped flatline between 6.72GiB and 9.4GB RAM (O(1) constant space)
- Database Integrity: 3.4GB SQLite WAL (PRAGMA integrity_check: ok)
- Cloud Compute Cost: $0.00

Technical Architecture & Memory Enforcement

1. Memory Allocator Substitution (libjemalloc2) Standard glibc malloc exhibits severe heap fragmentation under sustained Python allocation and deallocation cycles, causing process RSS growth and eventual OOM terminations. Injecting libjemalloc2 via LD_PRELOAD bypasses glibc allocation delays.