Day 2/30: LLM Tool Calls

I still remember the frustration of trying to integrate a large language model (LLM) into our support bot. The model could understand natural language, generate human-like responses, and even learn from feedback. However, it lacked the ability to interact with external tools and services, making it feel like a disembodied brain - incredibly smart, but unable to act on its knowledge. Our bot would often respond with phrases like "I can help you with that," but then fail to actually perform the task. It was as if the LLM had no "hands" to manipulate the world.

The problem was that our LLM was not designed to make tool calls from scratch. It relied on pre-existing frameworks and libraries to interact with external services, which limited its flexibility and autonomy. We needed a way to give our LLM the ability to call tools and services without relying on these frameworks, essentially giving it "hands" to manipulate the world.

To solve this problem, we turned to the Model Context Protocol (MCP) and LangGraph. MCP provides a standardized way to represent and interact with external tools and services, while LangGraph allows us to model complex workflows and decision-making processes. By combining these two technologies, we could create a system that enables our LLM to make tool calls from scratch, without relying on pre-existing frameworks.

Here's an example of how we used LangGraph and MCP to give our LLM "hands":

import langgraph
from langgraph import StateGraph, add_node, add_conditional_edges
from mcp import Tool, ToolCall

class SendMessageTool(Tool):
    def __init__(self, message):
        self.message = message

    def execute(self):
        print(f"Sending message: {self.message}")

graph = StateGraph()

start_node = add_node(graph, "start")
message_node = add_node(graph, "send_message")
end_node = add_node(graph, "end")

add_conditional_edges(graph, start_node, message_node, lambda x: x["action"] == "send_message")
add_conditional_edges(graph, message_node, end_node, lambda x: x["message"] is not None)

tool_call = ToolCall(SendMessageTool, "Hello, world!")

class LLM:
    def __init__(self, graph):
        self.graph = graph

    def make_tool_call(self, tool_call):
        current_node = self.graph.get_node("start")
        while current_node != self.graph.get_node("end"):
            next_node = self.graph.get_next_node(current_node, {"action": "send_message", "message": tool_call.tool.message})
            if next_node == self.graph.get_node("send_message"):
                tool_call.tool.execute()
            current_node = next_node

llm = LLM(graph)
llm.make_tool_call(tool_call)

This code defines a simple tool that sends a message, and an LLM that uses a LangGraph state graph to determine when to make a tool call. The LLM can make tool calls from scratch, without relying on pre-existing frameworks.

One practical gotcha we learned from this experience is that it's essential to carefully design the state graph and tool calls to avoid infinite loops. If the graph is not properly structured, the LLM may get stuck in an infinite loop, making repeated tool calls without achieving any meaningful outcome.

As we continue to develop our agentic AI system, we'll explore more advanced topics, such as integrating multiple tools and services, and using MCP to handle errors and exceptions. Tomorrow, we'll dive deeper into the world of agentic AI and explore new ways to give our LLM "hands" to manipulate the world.