Thursdays with Koog: Other Agent Settings

Knosh 0.3.0, an open-source tool, now uses Koog's AIAgent with configurable temperature and maxIterations parameters. Temperature controls the randomness of token selection in LLMs, while maxIterations limits the agent's iterations to prevent excessive token usage. Koog supports temperature for many models and allows custom models to specify support.

Knosh https://knosh.commonsware.com 0.3.0 uses this code to create a Koog https://docs.koog.ai AIAgent : return AIAgent promptExecutor = promptExecutor, llmModel = resolveModel descriptor.provider, agentConfig.modelName, agentConfig.contextLength, descriptor.modelDefinitions, , toolRegistry = toolSet.build agentConfig, config.commandPermissions, config.commandExternalDirectories, config.logFullToolCalls, config.allowedToolIds, knoshConfig.toolLimits, knoshConfig.webFetch, , temperature = config.temperature ?: agentConfig.temperature, maxIterations = config.maxIterations, systemPrompt = assembleSystemPrompt listOf agentConfig.systemPrompt + loadAgentsMarkdown configDir = context.userHome / ".config" / "knosh", workingDir = context.workingDir, fileSystem = fileSystem, , { if onMetrics = null { install MetricsFeature.Feature { callback = onMetrics } } if onStatus = null { install StatusFeature.Feature { callback = onStatus } } } } We have discussed most of those parameters in earlier issues of this newsletter. Two remain: temperature and maxIterations . Temperature is fun... where it is available. Temperature sometimes is described as controlling how "creative" and LLM is. In truth, it just controls how random numbers get applied. LLMs are stochastic parrots. Given a prompt, they generate additional words, a token at a time. The tokens are chosen based on probabilities based on the prompt text. For example, suppose our prompt is purely the following, without any sort of system prompt from the agent harness: Where does the line "Four score and seven years ago" appear? Most Americans hopefully recognize that as the opening line of Lincoln's Gettysburg Address https://en.wikipedia.org/wiki/Gettysburg Address . LLMs do not "know" anything, but they will have ingested lots of documents that refer to that line. When it comes to the first word to generate for a response to the prompt, high-probability words includes "Lincoln's" and "That" to begin a sentence like "That is Lincoln's Gettysburg Address" . In this case, "Lincoln's" is high probability based purely on context, and "That" is high probability because it is a common first word of a sentence in a response to a question. Other words that are contextually relevant e.g., "Gettysburg", "Address" are lower in likelihood, just because fewer sentences will start with them. And there are plenty of words that are rather unlikely to be chosen e.g., "rutabaga" . The LLM "chooses" a word based on this sort of weighted probability distribution, making a random selection. Temperature affects how random that selection can be. A low temperature steers the LLM towards the highest-probability words. Higher temperatures allow the LLM potentially to pull in words from deeper in the "long tail" of candidates. Temperature is a capability of an LLM. Some have dropped it, while others still use it. We supply a value to Koog, and Koog decides whether to pass it along to the LLM based on the model's capabilities. Koog knows capabilities for many popular models, and if you create your own e.g., for a local model , you need to indicate if you think that it supports temperature or not. maxIterations helps you prevent the agent from wandering around aimlessly for extended periods, burning tokens as it goes. We've talked about how AIAgent handles prompts https://pac.commonsware.com/archive/thursdays-with-koog-prompts-everywhere/ and tool calls https://pac.commonsware.com/archive/thursdays-with-koog-tools/ . It does all that without your involvement. maxIterations https://docs.koog.ai/agents/basic-agents/ adjust-agent-iterations effectively caps how much of that happens. It definitely caps tool calls. It might cap "thinking" messages — the precise scope of maxIterations is unclear. But, the idea is that you can control how long the LLM goes before your agent harness "pulls the plug".Both of those parameters are driven by agent frontmatter, with built-in defaults. You can have temperature and/or maxIterations frontmatter values in an agent definition Markdown file, and those will flow through to the AIAgent constructor call. So you can fine-tune these things if you want or ignore them if the defaults work well enough for you. Speaking of frontmatter, next week's "Thursdays with Koog" will explore another facet of the Knosh frontmatter: the tool security system. This determines which tools Knosh will make available to an agent or command and what those tools can do.