{"slug": "thursdays-with-koog-choosing-local", "title": "Thursdays with Koog: Choosing Local", "summary": "The \"Thursdays with Koog\" series concluded after 11 weeks with a post examining Koog's near-complete focus on hosted models and its limits for local, on-device inference. The post notes Koog's only on-device path is an undocumented LiteRT module supporting Google's LiteRT-LM on Android, while koog-ondevice and koog-edge pursue other on-device models and no active effort exists to integrate llama.cpp or MLX for Apple Silicon. Alternatives cited for local models include Llamatik, which supports GGUF models across all major Kotlin Multiplatform targets, and Langchain4j, a Java SDK with a Kotlin extension library.", "body_md": "[Koog](https://docs.koog.ai) is nice, but it does have some limitations. Chief among those is a near-complete focus on hosted models, or at least their APIs.\n\nYou can use Koog with local models, but only if those models run inside some container that has an API that Koog can speak. That is unrealistic for an on-phone model, and even for a desktop or server, that kind of container might add unnecessary overhead.\n\nThere is [a LiteRT module](https://mvnrepository.com/artifact/ai.koog/prompt-executor-litert-client-android) supporting [Google's LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM) on-Android model engine. However, this appears to be undocumented, other than [one GitHub page](https://github.com/JetBrains/koog/blob/develop/prompt/prompt-executor/prompt-executor-clients/prompt-executor-litert-client/Module.md). While the official library only supports Android, the module's code is small enough that porting it to other desired platforms seems reasonable.\n\n[`koog-ondevice`](https://github.com/uny/koog-ondevice) and [` koog-edge`](https://github.com/lemcoder/koog-edge) are efforts to have Koog support other on-device models, both device-supplied or developer-supplied. Strangely, there does not seem to be an active effort to get Koog to integrate with [`llama.cpp`](https://github.com/ggml-org/llama.cpp), let alone anything MLX-based for Apple Silicon.\n\nSo, if your focus is on local models, you might need to consider using other APIs.\n\nIf your interest lies in [LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM), but beyond Android, in addition to porting the Koog library, you could consider:\n\nIf you want broader model support, [Llamatik](https://www.llamatik.com/) supports GGUF models on all major Kotlin Multiplatform targets. Writing a Koog prompt executor that delegates to Llamatik would seem to be practical.\n\nIf you want deeper support for desktop targets, [Langchain4j](https://docs.langchain4j.dev/) is a Java SDK that has [a `-ktx`-style extension library](https://docs.langchain4j.dev/tutorials/kotlin/) to help ease its use in Kotlin.\n\nMy guess is that there is more to come, especially if [hosted models become ruinously expensive](https://commonsware.com/blog/2026/03/06/what-if-ai-compute-costs-soar.html) in the future.\n\nThis concludes the \"Thursdays with Koog\" series. I might write about Koog again from time to time, but it will no longer be *every* Thursday, as it has been for the past 11 weeks. 😅", "url": "https://wpnews.pro/news/thursdays-with-koog-choosing-local", "canonical_source": "https://pac.commonsware.com/archive/thursdays-with-koog-choosing-local/", "published_at": "2026-09-10 13:00:00+00:00", "updated_at": "2026-09-10 13:03:26.158843+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "developer-tools", "ai-products"], "entities": ["Koog", "JetBrains", "LiteRT-LM", "Google", "koog-ondevice", "koog-edge", "llama.cpp", "Llamatik"], "alternates": {"html": "https://wpnews.pro/news/thursdays-with-koog-choosing-local", "markdown": "https://wpnews.pro/news/thursdays-with-koog-choosing-local.md", "text": "https://wpnews.pro/news/thursdays-with-koog-choosing-local.txt", "jsonld": "https://wpnews.pro/news/thursdays-with-koog-choosing-local.jsonld"}}