LLM inference vs. the OOM killer
The open-source Rust inference library behind the NobodyWho Chat test app can register for OS memory-warning signals to automatically unload LLM models before Android and iOS Out Of Memory killers terminate the app, acco…