In a past "Thursdays with Koog" issue, we saw that Koog's AIAgent
knows how to handle tool calls for us. We can provide a prompt and get the final response from the LLM, and any tool calls the LLM requests in between those are handled by the agent.
In truth, the AIAgent
is involved in that logic, but the real work is handled by a strategy implementation.
Koog has an AIAgentStrategy
interface with a bunch of sub-interfaces and implementations. In addition to a name
, AIAgentStrategy
defines execute()
, which AIAgent
itself wraps via its run()
function, with a bit of additional indirection. execute()
does the work. So, in the end, the strategy is what orchestrates the work with the LLM, and since the strategy is defined by an interface, we can plug in other strategies as needed.
The default strategy is built by the singleRunStrategy()
factory function. Its KDoc shows the flow that Knosh uses:
- Start the agent.
- Call the LLM with the input.
- Execute a tool based on the LLM's response.
- Send the tool result back to the LLM.
- Repeat until LLM indicates no further tool calls are needed or the agent finishes.
Basically, singleRunStrategy()
is your "one-shot" pattern.
Under the covers, singleRunStrategy()
is a one-off implementation of a graph strategy:
@JvmOverloads
public fun singleRunStrategy(parallelTools: Boolean = false): AIAgentGraphStrategy<String, String> = strategy<String, String>("single_run") {
val nodeCallLLM by nodeLLMRequest()
val nodeExecuteTool by nodeExecuteTools()
val nodeSendToolResult by nodeLLMSendToolResults()
edge(nodeStart forwardTo nodeCallLLM)
edge(nodeCallLLM forwardTo nodeExecuteTool onToolCalls { true })
edge(nodeCallLLM forwardTo nodeFinish onTextMessage { true })
edge(nodeExecuteTool forwardTo nodeSendToolResult)
edge(nodeSendToolResult forwardTo nodeFinish onTextMessage { true })
edge(nodeSendToolResult forwardTo nodeExecuteTool onToolCalls { true })
}
(the above is from Koog 1.1.1)
This defines an object graph, with edge()
connecting two processing nodes. The interaction begins at nodeStart
and flows through the graph, eventually terminating at nodeFinish
. The strategy()
builder offers a DSL for defining the graph, designed to make the code read almost like plain English:
Strategies can get a lot more elaborate:
The bad news is that nodes are synchronous, insofar as they implement a simple execute()
function (often defined by a DSL-supplied lambda expression). execute()
is a suspend fun
, so you can do I/O and stuff, but in the end, execute()
needs to return whatever the graph needs to continue.
A side effect of this approach is that creating a multi-turn agent, with several rounds of user input, gets weird. Effectively, you build a graph where you have a node that takes the latest message from the LLM and synchronously returns the next input from the user. That node can leverage coroutines, such as posting to a Flow
or Channel
and observing a Flow
or Channel
to get the response, but it needs to be able to return the user's message, which your graph can then process. Your UI needs to be conducive to such an arrangement, and that might take a while to "get your head wrapped around".
You don't have to use custom graphs. Knosh doesn't. You can create a multi-turn agent just by iteratively calling run()
on your AIAgent
set up to use singleRunStrategy()
. Managing the overall prompt becomes your job: for your second turn, you need to determine how to combine your original prompt with the result from the LLM and (presumably) fresh input from the user, but without all of the tool responses (which were "consumed" in the process of giving you the LLM result). Personally, and off the cuff, I would not mind this and might prefer it to trying to twist my overall app to be visible just via a graph node. But, you have options.
Next week's "Thursdays with Koog" will explore some other Koog "knobs we can turn and switches we can flip", such as the maximum number of iterations to use, the temperature, and more.