Last week, I posted the inaugural "Thursdays with Koog", exploring the basics of how Knosh (my one-shot coding agent) interacts with Koog (JetBrains' LLM interaction framework).
In that post, I showed that:
knosh prompt --agentId=general "What can you do for me?"
...causes Knosh to look up the definition of the general
agent, which is encoded in Markdown:
---
model: "ollama/qwen3.6:35b-a3b-coding-nvfp4"
description: a general-purpose agent with full tool access
---
You are a helpful assistant to an experienced software developer.
But, somewhere along the line, we need to take that model
and teach Koog that it represents what LLM we want to use. The left side (ollama
) represents Koog's provider ID, while the right side (qwen3.6:35b-a3b-coding-nvfp4
) represents the model to use with that provider... but we need to map those strings to Koog objects.
Knosh parses that Markdown into an AgentConfig, with
modelProvider
and modelName
holding those two pieces of the model
frontmatter property from the Markdown. We use the modelProvider
to find our provider in a descriptors
roster that is based on which Koog providers Knosh supports:
private val descriptors: List<ProviderDescriptor> =
listOf(
ProviderDescriptor(
configName = "ollama",
provider = LLMProvider.Ollama,
modelDefinitions = null,
envVarName = null,
configKeyName = null,
readKey = { null },
createClient = { _, knoshConfig ->
OllamaClient(httpClientFactory = this.ollamaHttpClientFactory, baseUrl = knoshConfig.ollamaURL)
},
),
ProviderDescriptor(
configName = "anthropic",
provider = LLMProvider.Anthropic,
modelDefinitions = AnthropicModels,
envVarName = "ANTHROPIC_API_KEY",
configKeyName = "anthropicApiKey",
readKey = { it.anthropicApiKey },
createClient = { apiKey, _ -> AnthropicLLMClient(apiKey) },
),
ProviderDescriptor(
configName = "mistral",
provider = LLMProvider.MistralAI,
modelDefinitions = MistralAIModels,
envVarName = "MISTRAL_API_KEY",
configKeyName = "mistralApiKey",
readKey = { it.mistralApiKey },
createClient = { apiKey, _ -> MistralAILLMClient(apiKey) },
),
ProviderDescriptor(
configName = "openai",
provider = LLMProvider.OpenAI,
modelDefinitions = OpenAIModels,
envVarName = "OPENAI_API_KEY",
configKeyName = "openAiApiKey",
readKey = { it.openAiApiKey },
createClient = { apiKey, _ -> OpenAILLMClient(apiKey) },
),
)
LLMProvider is Koog's representation of an LLM provider (sometimes, naming is actually easy!). Koog ships with a series of providers, a mix of dedicated model houses (OpenAI, Anthropic, etc.) and service providers (e.g., Amazon Bedrock, OpenRouter). However,
LLMProvider
is just a identifier and a display name — it houses no real business logic.That lands in LLMClient implementations, one per provider. These know how to talk the specific Web service (or whatever) API to the provider and use that to send prompts and get responses. These get supplied by specific Koog dependencies, so you can load in just the provider(s) that your app intends to support. Many of these clients require an API key; some of the properties on Knosh's
ProviderDescriptor
say where and how to look up the API key to use.Those give us information about a provider. We also need to get a Koog object representing the model. That is an LLModel, which ties a model name (e.g.,
claude-haiku-4-5
) and a provider together, along with some operational information about that model:Many providers will have an LLModelDefinitions, which amounts to a list of
LLModel
instances, representing the known models for that provider at the time that particular version of Koog shipped. For example, 1.0.0
of the Anthropic Koog library does not know about Sonnet 5, Fable, etc., as those were released by Anthropic after Koog released 1.0.0
.To fill in the gaps, you are welcome to construct your own LLModel
definitions. That is also needed for providers where there is no canonical roster of supported models. In the case of Knosh, that's important for Ollama support, as what models Ollama has depends on what you had Ollama download and install.
So, Knosh will try to find a matching LLModel
from the LLModelDefinitions
for the provider, and if that fails, it builds its own, limiting the capabilities to what Knosh needs:
private fun resolveModel(
provider: LLMProvider,
modelId: String,
fallbackContextLength: Long,
modelDefinitions: LLModelDefinitions?,
): LLModel =
modelDefinitions?.models?.find { it.id == modelId }
?: LLModel(
provider = provider,
id = modelId,
capabilities =
listOf(
LLMCapability.Completion,
LLMCapability.Temperature,
LLMCapability.Schema.JSON.Basic,
LLMCapability.Tools,
),
contextLength = fallbackContextLength,
)
Given the LLMProvider
, the LLModel
, and the LLMClient
, you are in position to start executing prompts... which we will explore in next week's "Thursdays with Koog" post.