Day 2/30: LLM Tool Calls

A developer integrated the Model Context Protocol (MCP) and LangGraph to enable a large language model (LLM) to make tool calls from scratch, giving it the ability to interact with external services without relying on pre-existing frameworks. The approach uses a state graph to model workflows and conditional edges to determine when to execute tool calls. A key lesson learned is the need to carefully design the graph to avoid infinite loops.

I still remember the frustration of trying to integrate a large language model LLM into our support bot. The model could understand natural language, generate human-like responses, and even learn from feedback. However, it lacked the ability to interact with external tools and services, making it feel like a disembodied brain - incredibly smart, but unable to act on its knowledge. Our bot would often respond with phrases like "I can help you with that," but then fail to actually perform the task. It was as if the LLM had no "hands" to manipulate the world. The problem was that our LLM was not designed to make tool calls from scratch. It relied on pre-existing frameworks and libraries to interact with external services, which limited its flexibility and autonomy. We needed a way to give our LLM the ability to call tools and services without relying on these frameworks, essentially giving it "hands" to manipulate the world. To solve this problem, we turned to the Model Context Protocol MCP and LangGraph. MCP provides a standardized way to represent and interact with external tools and services, while LangGraph allows us to model complex workflows and decision-making processes. By combining these two technologies, we could create a system that enables our LLM to make tool calls from scratch, without relying on pre-existing frameworks. Here's an example of how we used LangGraph and MCP to give our LLM "hands": python import langgraph from langgraph import StateGraph, add node, add conditional edges from mcp import Tool, ToolCall Define a tool that our LLM can call class SendMessageTool Tool : def init self, message : self.message = message def execute self : Simulate sending a message print f"Sending message: {self.message}" Create a LangGraph state graph graph = StateGraph Add nodes to the graph start node = add node graph, "start" message node = add node graph, "send message" end node = add node graph, "end" Add conditional edges to the graph add conditional edges graph, start node, message node, lambda x: x "action" == "send message" add conditional edges graph, message node, end node, lambda x: x "message" is not None Define a tool call tool call = ToolCall SendMessageTool, "Hello, world " Create an LLM that can make tool calls class LLM: def init self, graph : self.graph = graph def make tool call self, tool call : Use the graph to determine the next action current node = self.graph.get node "start" while current node = self.graph.get node "end" : Get the next node based on the current state next node = self.graph.get next node current node, {"action": "send message", "message": tool call.tool.message} Execute the tool call if next node == self.graph.get node "send message" : tool call.tool.execute current node = next node Create an instance of the LLM and make a tool call llm = LLM graph llm.make tool call tool call This code defines a simple tool that sends a message, and an LLM that uses a LangGraph state graph to determine when to make a tool call. The LLM can make tool calls from scratch, without relying on pre-existing frameworks. One practical gotcha we learned from this experience is that it's essential to carefully design the state graph and tool calls to avoid infinite loops. If the graph is not properly structured, the LLM may get stuck in an infinite loop, making repeated tool calls without achieving any meaningful outcome. As we continue to develop our agentic AI system, we'll explore more advanced topics, such as integrating multiple tools and services, and using MCP to handle errors and exceptions. Tomorrow, we'll dive deeper into the world of agentic AI and explore new ways to give our LLM "hands" to manipulate the world.