Knosh 0.3.0
uses this code to create a Koog AIAgent
:
return AIAgent(
promptExecutor = promptExecutor,
llmModel =
resolveModel(
descriptor.provider,
agentConfig.modelName,
agentConfig.contextLength,
descriptor.modelDefinitions,
),
toolRegistry =
toolSet.build(
agentConfig,
config.commandPermissions,
config.commandExternalDirectories,
config.logFullToolCalls,
config.allowedToolIds,
knoshConfig.toolLimits,
knoshConfig.webFetch,
),
temperature = config.temperature ?: agentConfig.temperature,
maxIterations = config.maxIterations,
systemPrompt =
assembleSystemPrompt(
listOf(agentConfig.systemPrompt) +
loadAgentsMarkdown(
configDir = context.userHome / ".config" / "knosh",
workingDir = context.workingDir,
fileSystem = fileSystem,
)
),
) {
if (onMetrics != null) {
install(MetricsFeature.Feature) { callback = onMetrics }
}
if (onStatus != null) {
install(StatusFeature.Feature) { callback = onStatus }
}
}
}
We have discussed most of those parameters in earlier issues of this newsletter. Two remain: temperature
and maxIterations
.
Temperature is fun... where it is available. Temperature sometimes is described as controlling how "creative" and LLM is. In truth, it just controls how random numbers get applied.
LLMs are stochastic parrots. Given a prompt, they generate additional words, a token at a time. The tokens are chosen based on probabilities based on the prompt text.
For example, suppose our prompt is purely the following, without any sort of system prompt from the agent harness:
Where does the line "Four score and seven years ago" appear?
Most Americans hopefully recognize that as the opening line of Lincoln's Gettysburg Address. LLMs do not "know" anything, but they will have ingested lots of documents that refer to that line.
When it comes to the first word to generate for a response to the prompt, high-probability words includes "Lincoln's" and "That" (to begin a sentence like "That is Lincoln's Gettysburg Address"). In this case, "Lincoln's" is high probability based purely on context, and "That" is high probability because it is a common first word of a sentence in a response to a question. Other words that are contextually relevant (e.g., "Gettysburg", "Address") are lower in likelihood, just because fewer sentences will start with them. And there are plenty of words that are rather unlikely to be chosen (e.g., "rutabaga"). The LLM "chooses" a word based on this sort of weighted probability distribution, making a random selection.
Temperature affects how random that selection can be. A low temperature steers the LLM towards the highest-probability words. Higher temperatures allow the LLM potentially to pull in words from deeper in the "long tail" of candidates.
Temperature is a capability of an LLM. Some have dropped it, while others still use it. We supply a value to Koog, and Koog decides whether to pass it along to the LLM based on the model's capabilities. Koog knows capabilities for many popular models, and if you create your own (e.g., for a local model), you need to indicate if you think that it supports temperature or not.
maxIterations
helps you prevent the agent from wandering around aimlessly for extended periods, burning tokens as it goes.
We've talked about how AIAgent
handles prompts and tool calls. It does all that without your involvement.
maxIterations effectively caps how much of that happens. It definitely caps tool calls. It might cap "thinking" messages — the precise scope of
maxIterations
is unclear. But, the idea is that you can control how long the LLM goes before your agent harness "pulls the plug".Both of those parameters are driven by agent frontmatter, with built-in defaults. You can have temperature
and/or maxIterations
frontmatter values in an agent definition Markdown file, and those will flow through to the AIAgent
constructor call. So you can fine-tune these things if you want or ignore them if the defaults work well enough for you.
Speaking of frontmatter, next week's "Thursdays with Koog" will explore another facet of the Knosh frontmatter: the tool security system. This determines which tools Knosh will make available to an agent or command and what those tools can do.