Should Every Application Really Run Its Own AI for Resource Management?

A developer argues against embedding AI models in every application for resource management, noting that the OS already handles scheduling and that inference overhead could outweigh savings. The developer suggests AI should only assist in uncertain cases, with deterministic rollback conditions, while the OS remains the authority.

Imagine five applications running on a computer, and each has a small AI model trying to optimize how that application uses the system. Capable hardware can easily carry that extra weight. But those models are still spending the user's computation. That is what makes me question the architecture. If every application starts carrying a model to observe its resource usage, predict what it will need and make its own optimization decisions, we haven't removed resource management from the operating system. The OS is still there doing the actual scheduling. We have just added several smaller decision-making systems around it. It knows memory pressure. It knows which processes are waiting for CPU time, which ones are active, what is being allocated and where contention is happening. More importantly, it sees all applications at once. An application optimizing itself doesn't have that same view. Still, the OS has a blind spot. It can know what an application needs without completely knowing what the user wants from it. It can know that a program wants 4 GB of RAM and several busy threads. That doesn't mean it knows what the user actually wants from that program. Maybe an application doing almost nothing in the background is the one I expect to respond instantly in the next few seconds. Meanwhile another process can be consuming a large amount of CPU for something I don't currently care about. Looking at resource demand alone doesn't completely reveal that difference. This is where I think AI becomes interesting—but not as a replacement for the scheduler. If a deterministic rule already gives the OS an answer, use it. There is little reason to run an inference just to rediscover a decision we could calculate directly. Sometimes there isn't even time to ask. Some OS operations happen on extremely short timescales. Put model inference in that path and the thing supposedly optimizing the system can become the reason it is slower. Input preparation costs something. Inference costs something. Monitoring the result costs something too. You could eventually spend more computation deciding how to save computation than you actually save. That would be a strange optimization. So I wouldn't put AI in every decision path. I see it being useful in the less obvious cases, where several system signals exist but they don't translate cleanly into a deterministic decision. A model trained specifically for this domain could receive things such as RAM pressure, CPU utilization, queue states and process behaviour. Historical user behaviour might be useful too, but I would treat that as secondary data rather than letting the whole system reshape itself around one person's habits. The model's job wouldn't be to control the machine. It would take a situation containing uncertainty and turn it into useful information that the OS can evaluate. The operating system remains the authority because, in the end, it is the component actually controlling the resources. I would put another restriction on it. Before the OS performs an AI-influenced optimization, there should already be a deterministic condition for reversing that action. If the change pushes the system somewhere it shouldn't, recovery shouldn't depend on asking the same model whether its previous prediction was a mistake. Rollback is already defined. This doesn't mean every application-level AI should disappear. If an IDE needs a model to understand code, that model belongs there. The same applies to an image editor using a model for image processing. Those are features of the application itself. I am talking specifically about AI being added to applications to manage shared system resources. Hardware resources don't belong to individual applications. Every application ultimately asks the same operating system for those resources anyway. Having several independent models reason about their individual pieces of the machine while the OS remains responsible for the whole machine creates unnecessary system friction. Let applications keep the AI they need for their actual work. For resource optimization, I would rather give the intelligence to the component that already holds the authority—and make sure that intelligence knows when not to get involved.