Koog is nice, but it does have some limitations. Chief among those is a near-complete focus on hosted models, or at least their APIs.
You can use Koog with local models, but only if those models run inside some container that has an API that Koog can speak. That is unrealistic for an on-phone model, and even for a desktop or server, that kind of container might add unnecessary overhead.
There is a LiteRT module supporting Google's LiteRT-LM on-Android model engine. However, this appears to be undocumented, other than one GitHub page. While the official library only supports Android, the module's code is small enough that porting it to other desired platforms seems reasonable.
koog-ondevice and koog-edge are efforts to have Koog support other on-device models, both device-supplied or developer-supplied. Strangely, there does not seem to be an active effort to get Koog to integrate with llama.cpp, let alone anything MLX-based for Apple Silicon.
So, if your focus is on local models, you might need to consider using other APIs.
If your interest lies in [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM), but beyond Android, in addition to porting the Koog library, you could consider:
If you want broader model support, [Llamatik](https://www.llamatik.com/) supports GGUF models on all major Kotlin Multiplatform targets. Writing a Koog prompt executor that delegates to Llamatik would seem to be practical.
If you want deeper support for desktop targets, [Langchain4j](https://docs.langchain4j.dev/) is a Java SDK that has [a `-ktx`-style extension library](https://docs.langchain4j.dev/tutorials/kotlin/) to help ease its use in Kotlin.
My guess is that there is more to come, especially if [hosted models become ruinously expensive](https://commonsware.com/blog/2026/03/06/what-if-ai-compute-costs-soar.html) in the future.
This concludes the "Thursdays with Koog" series. I might write about Koog again from time to time, but it will no longer be every Thursday, as it has been for the past 11 weeks. 😅