Three months into building AI applications, I hit a wall.
My AI assistant could answer questions beautifully. It could summarize documents, explain concepts, draft responses, and hold intelligent conversations.
But it couldn’t do anything.
A user would ask: “Can you book me a meeting with the team for Thursday at 3 PM?”
Learning Generative AI From Scratch: The Complete Roadmap (120+ Articles)
The AI would respond: “Of course! I’d be happy to help you schedule that meeting. You’ll want to open your calendar application and create a new event for Thursday at 3 PM…”
It was giving instructions for how to do a task instead of actually doing it. Like asking a colleague to send an email and they respond by explaining how email works.
The gap between talking about actions and performing actions is exactly what Function Calling solves.
And understanding it properly — not just as a concept but as something you architect deliberately — changes what becomes possible in your AI applications.
This is Part 33 of Learning Generative AI From Scratch. In Part 32 we covered Prompt Templates and Prompt Chaining — how to structure AI workflows. Now we’re going one step further: how AI models trigger actual functions in your application to perform real-world actions.
Prompt Templates and Prompt Chaining: How to Build AI Workflows That Actually Scale — Part 32
Function Calling is a capability that allows an LLM to request the execution of specific functions defined in your application — passing structured parameters, receiving structured results, and using those results to complete a task.
The critical distinction from everything we’ve built so far:
When Function Calling is available, the model doesn’t just generate text — it can generate structured requests for your application to execute real operations: database queries, API calls, file operations, calendar updates, email sends, payment processing — anything your application can do.
The model decides what to call and with what parameters. Your application decides whether to execute it and how. That separation of concerns is the architectural foundation that makes Function Calling safe and powerful at the same time.
Before writing any code, understanding the exact flow matters:
User: "What's the weather in Bengaluru and book a cab if it's raining?"↓LLM receives message + available function definitions↓LLM decides: "I need weather data first"LLM generates structured function call request:{ "function": "getWeather", "parameters": { "city": "Bengaluru" }}↓Your application executes getWeather("Bengaluru")Returns: { "condition": "Heavy Rain", "temp": 22 }↓LLM receives result, decides: "It's raining — book cab"LLM generates second function call:{ "function": "bookCab", "parameters": { "pickup": "user location", "destination": "office", "type": "standard" }}↓Your application executes bookCab(...)Returns: { "bookingId": "CAB-4829", "eta": "8 minutes" }↓LLM generates final natural language response:"It's raining heavily in Bengaluru (22°C). I've booked a cab for you — it arrives in 8 minutes.Booking reference: CAB-4829."
Two things to notice:
First — the LLM made two sequential decisions, each building on the previous result. It didn’t know it needed to book a cab until it knew the weather.
Second — your application executed both functions. The LLM never directly called any API. It requested, your code executed, results flowed back.
If you read Part 28 of this series, you might be wondering: isn’t this exactly what Tool Calling does?
Yes — and no. Here’s the precise distinction:
Function Calling is the underlying capability — the model generating structured requests for specific functions with typed parameters. It’s the mechanism.
Tool Calling (Part 28) is a higher-level abstraction — Spring AI’s @Tool annotation pattern that wraps Function Calling into a simpler developer experience.
Think of it this way:
In Part 28, we used @Tool annotations and let Spring AI handle the function registration, parameter parsing, and result injection automatically.
In this article, we’re going one layer deeper — understanding how the raw Function Calling mechanism works, how to define function schemas explicitly, how to handle multi-step function execution, and how to build production-grade patterns that @Tool alone doesn't cover.
Both matter. Understanding the engine helps you use the steering wheel better.
The model knows which functions are available because you describe them in a structured schema — typically JSON Schema format — that gets included in the API request.
@Configurationpublic class FunctionSchemaConfig { // Define function schemas as Spring AI FunctionCallback objects @Bean public FunctionCallback getWeatherFunction() { return FunctionCallbackWrapper.builder(new WeatherFunction()) .withName("getWeather") .withDescription( "Get current weather conditions for a specific city. " + "Returns temperature in Celsius and weather condition. " + "Use this when the user asks about weather or before " + "making outdoor activity recommendations." ) .withInputType(WeatherFunction.WeatherRequest.class) .build(); } @Bean public FunctionCallback bookCabFunction() { return FunctionCallbackWrapper.builder(new CabBookingFunction()) .withName("bookCab") .withDescription( "Book a cab/taxi for the user. " + "Use this when the user explicitly asks to book " + "transportation or when weather conditions suggest " + "they need a ride. Always confirm with user before booking." ) .withInputType(CabBookingFunction.BookingRequest.class) .build(); } @Bean public FunctionCallback sendEmailFunction() { return FunctionCallbackWrapper.builder(new EmailFunction()) .withName("sendEmail") .withDescription( "Send an email on behalf of the user. " + "Use ONLY when the user explicitly requests sending an email. " + "Never send emails without explicit user instruction." ) .withInputType(EmailFunction.EmailRequest.class) .build(); }}
Notice the descriptions are detailed and include when to use each function — not just what it does. This is critical. The model reads these descriptions to decide which function to call. Vague descriptions lead to wrong function selection.
Each function is a Java class that implements the actual logic:
// Weather function@Componentpublic class WeatherFunction implements Function<WeatherFunction.WeatherRequest, WeatherFunction.WeatherResponse> { @Override public WeatherResponse apply(WeatherRequest request) { // In production: call real weather API // For demo: return realistic mock data log.info("Executing getWeather for city: {}", request.city()); return switch (request.city().toLowerCase()) { case "bengaluru", "bangalore" -> new WeatherResponse("Bengaluru", 22, "Heavy Rain", 85); case "mumbai" -> new WeatherResponse("Mumbai", 31, "Humid", 90); case "delhi" -> new WeatherResponse("Delhi", 38, "Sunny", 20); default -> new WeatherResponse(request.city(), 28, "Partly Cloudy", 60); }; } public record WeatherRequest( @JsonProperty(required = true, value = "city") String city ) {} public record WeatherResponse( String city, int temperatureCelsius, String condition, int humidityPercent ) {}}// Cab booking function@Componentpublic class CabBookingFunction implements Function<CabBookingFunction.BookingRequest, CabBookingFunction.BookingResponse> { @Override public BookingResponse apply(BookingRequest request) { log.info("Executing bookCab: pickup={}, destination={}", request.pickup(), request.destination()); // In production: call actual cab booking API // Simulate booking confirmation String bookingId = "CAB-" + String.format("%04d", (int)(Math.random() * 9999)); return new BookingResponse( bookingId, request.pickup(), request.destination(), "8 minutes", "₹180 - ₹220", "CONFIRMED" ); } public record BookingRequest( @JsonProperty(required = true) String pickup, @JsonProperty(required = true) String destination, @JsonProperty(defaultValue = "standard") String cabType ) {} public record BookingResponse( String bookingId, String pickup, String destination, String estimatedArrival, String estimatedFare, String status ) {}}// Email function@Componentpublic class EmailFunction implements Function<EmailFunction.EmailRequest, EmailFunction.EmailResponse> { private final JavaMailSender mailSender; public EmailFunction(JavaMailSender mailSender) { this.mailSender = mailSender; } @Override public EmailResponse apply(EmailRequest request) { log.info("Executing sendEmail to: {}", request.to()); try { SimpleMailMessage message = new SimpleMailMessage(); message.setTo(request.to()); message.setSubject(request.subject()); message.setText(request.body()); mailSender.send(message); return new EmailResponse( true, "Email sent successfully to " + request.to() ); } catch (Exception e) { log.error("Email sending failed: {}", e.getMessage()); return new EmailResponse( false, "Failed to send email: " + e.getMessage() ); } } public record EmailRequest( @JsonProperty(required = true) String to, @JsonProperty(required = true) String subject, @JsonProperty(required = true) String body ) {} public record EmailResponse( boolean success, String message ) {}}
Wiring it all together with Spring AI:
@Servicepublic class FunctionCallingAssistant { private final ChatClient chatClient; public FunctionCallingAssistant( ChatClient.Builder builder, FunctionCallback getWeatherFunction, FunctionCallback bookCabFunction, FunctionCallback sendEmailFunction) { this.chatClient = builder .defaultSystem(""" You are a helpful personal assistant. You have access to functions that can perform real actions. Important guidelines: - Use functions when the user needs real data or wants an action performed - Always inform the user what action you're about to take before taking it - For irreversible actions (booking, sending emails), confirm with the user first unless they've been explicit - Report function results naturally in your response - If a function fails, explain what happened clearly """) .defaultFunctions( getWeatherFunction, bookCabFunction, sendEmailFunction ) .build(); } public String chat(String userMessage) { return chatClient.prompt() .user(userMessage) .call() .content(); } // Session-aware version with conversation history public String chat(String sessionId, String userMessage, ChatMemory memory) { return chatClient.prompt() .user(userMessage) .advisors(MessageChatMemoryAdvisor.builder(memory) .conversationId(sessionId) .build()) .call() .content(); }}
The real power of Function Calling emerges when the model needs to call multiple functions — some in sequence, some conditionally based on previous results.
Spring AI handles this automatically in many cases, but for complex workflows you want explicit control:
@Servicepublic class MultiStepFunctionService { private final ChatClient chatClient; private final WeatherFunction weatherFunction; private final CabBookingFunction cabFunction; public AssistantResponse handleComplexRequest(String userRequest) { List<FunctionCall> executedFunctions = new ArrayList<>(); List<String> functionResults = new ArrayList<>(); // First pass — might trigger function calls ChatResponse response = chatClient.prompt() .user(userRequest) .call() .chatResponse(); // Process any function calls in the response while (containsFunctionCalls(response)) { List<FunctionCall> calls = extractFunctionCalls(response); for (FunctionCall call : calls) { // Execute the function Object result = executeFunction(call); executedFunctions.add(call); functionResults.add( call.functionName() + " returned: " + toJson(result) ); // Feed result back to model response = chatClient.prompt() .user(userRequest) .system("Previous function results:\n" + String.join("\n", functionResults)) .call() .chatResponse(); } } return new AssistantResponse( extractTextContent(response), executedFunctions, functionResults ); } private Object executeFunction(FunctionCall call) { return switch (call.functionName()) { case "getWeather" -> weatherFunction.apply( fromJson(call.parameters(), WeatherFunction.WeatherRequest.class) ); case "bookCab" -> cabFunction.apply( fromJson(call.parameters(), CabBookingFunction.BookingRequest.class) ); default -> throw new UnknownFunctionException( call.functionName() ); }; } public record FunctionCall(String functionName, String parameters) {} public record AssistantResponse( String textResponse, List<FunctionCall> executedFunctions, List<String> functionResults ) {}}
Building a working demo is one thing. Running Function Calling in production adds four concerns that most tutorials don’t cover.
Some functions are reversible (get weather, look up data). Some aren’t (send email, process payment, delete record). Never let the model trigger irreversible actions without explicit user confirmation:
@Servicepublic class ConfirmationGateService { private final ChatClient chatClient; private final Map<String, PendingAction> pendingActions = new ConcurrentHashMap<>(); public GatedResponse handleWithConfirmation( String sessionId, String userMessage) { // Check if this is a confirmation response if (isConfirmation(userMessage)) { return executePendingAction(sessionId, userMessage); } // Classify if the request needs an irreversible action ActionClassification classification = classifyAction(userMessage); if (classification.requiresConfirmation()) { // Store the pending action pendingActions.put(sessionId, new PendingAction( userMessage, classification.actionType(), classification.actionSummary() ) ); // Ask for confirmation instead of executing return GatedResponse.requiresConfirmation( "I'm about to " + classification.actionSummary() + ". Shall I proceed? (yes/no)" ); } // Safe action — execute immediately String result = chatClient.prompt() .user(userMessage) .call() .content(); return GatedResponse.completed(result); } private GatedResponse executePendingAction( String sessionId, String confirmation) { PendingAction pending = pendingActions.get(sessionId); if (pending == null) { return GatedResponse.completed( "I don't have a pending action to confirm." ); } if (confirmation.toLowerCase().contains("yes")) { pendingActions.remove(sessionId); String result = chatClient.prompt() .user(pending.originalRequest()) .system("The user has confirmed this action. Proceed.") .call() .content(); return GatedResponse.completed(result); } pendingActions.remove(sessionId); return GatedResponse.completed( "Understood — I've cancelled that action." ); } public record PendingAction( String originalRequest, String actionType, String actionSummary ) {} public record GatedResponse( String message, boolean requiresConfirmation, boolean completed ) { public static GatedResponse requiresConfirmation(String msg) { return new GatedResponse(msg, true, false); } public static GatedResponse completed(String msg) { return new GatedResponse(msg, false, true); } }}
Every function call should be logged — what was called, with what parameters, by which user, at what time, and what the result was:
@Entity@Table(name = "function_call_audit")public class FunctionCallAudit { @Id @GeneratedValue(strategy = GenerationType.IDENTITY) private Long id; private String sessionId; private String userId; private String functionName; @Column(columnDefinition = "TEXT") private String inputParameters; @Column(columnDefinition = "TEXT") private String outputResult; private boolean success; private Long executionTimeMs; private LocalDateTime executedAt; private String errorMessage;}@Aspect@Componentpublic class FunctionCallAuditAspect { private final FunctionCallAuditRepository auditRepository; @Around("@annotation(auditFunctionCall)") public Object auditCall(ProceedingJoinPoint joinPoint, AuditFunctionCall auditFunctionCall) throws Throwable { long start = System.currentTimeMillis(); FunctionCallAudit audit = new FunctionCallAudit(); audit.setFunctionName(auditFunctionCall.value()); audit.setExecutedAt(LocalDateTime.now()); audit.setInputParameters(toJson(joinPoint.getArgs())); try { Object result = joinPoint.proceed(); audit.setOutputResult(toJson(result)); audit.setSuccess(true); audit.setExecutionTimeMs(System.currentTimeMillis() - start); auditRepository.save(audit); return result; } catch (Exception e) { audit.setSuccess(false); audit.setErrorMessage(e.getMessage()); audit.setExecutionTimeMs(System.currentTimeMillis() - start); auditRepository.save(audit); throw e; } }}
Some functions are expensive — API calls with per-request costs, database operations that impact performance. Rate limit them per user:
@Servicepublic class RateLimitedFunctionService { private final Map<String, RateLimiter> limiters = new ConcurrentHashMap<>(); // 10 weather lookups per user per minute private static final int WEATHER_LIMIT = 10; // 3 cab bookings per user per hour private static final int CAB_BOOKING_LIMIT = 3; public <T> T executeWithRateLimit(String userId, String functionName, Supplier<T> functionCall) { String key = userId + ":" + functionName; RateLimiter limiter = limiters.computeIfAbsent( key, k -> buildLimiter(functionName) ); if (!limiter.tryAcquire()) { throw new RateLimitExceededException( "Rate limit exceeded for " + functionName + ". Please try again later." ); } return functionCall.get(); } private RateLimiter buildLimiter(String functionName) { return switch (functionName) { case "getWeather" -> RateLimiter.create(WEATHER_LIMIT / 60.0); case "bookCab" -> RateLimiter.create(CAB_BOOKING_LIMIT / 3600.0); default -> RateLimiter.create(30 / 60.0); // default: 30/min }; }}
Functions fail. APIs go down. Databases timeout. Your AI application should degrade gracefully, not crash:
@Servicepublic class ResilientFunctionWrapper { public <T> FunctionResult<T> executeWithFallback( String functionName, Supplier<T> primaryFunction, Supplier<T> fallbackFunction, String fallbackReason) { try { T result = primaryFunction.get(); return FunctionResult.success(result); } catch (ExternalApiException e) { log.warn("Function {} failed, using fallback: {}", functionName, e.getMessage()); try { T fallbackResult = fallbackFunction.get(); return FunctionResult.fallback( fallbackResult, fallbackReason ); } catch (Exception fallbackEx) { log.error("Both primary and fallback failed for {}", functionName); return FunctionResult.failed( "Service temporarily unavailable" ); } } } public record FunctionResult<T>( T data, boolean success, boolean usedFallback, String message ) { public static <T> FunctionResult<T> success(T data) { return new FunctionResult<>(data, true, false, null); } public static <T> FunctionResult<T> fallback(T data, String reason) { return new FunctionResult<>(data, true, true, reason); } public static <T> FunctionResult<T> failed(String message) { return new FunctionResult<>(null, false, false, message); } }}
Bringing everything together into a production-ready endpoint:
@RestController@RequestMapping("/api/assistant")public class FunctionCallingController { private final FunctionCallingAssistant assistant; private final RateLimitedFunctionService rateLimiter; private final ChatMemory chatMemory; @PostMapping("/chat") public ResponseEntity<ChatResponse> chat( @RequestBody ChatRequest request, @RequestHeader("X-User-Id") String userId) { try { // Rate limit check rateLimiter.executeWithRateLimit( userId, "chat", () -> null ); // Process with function calling enabled String response = assistant.chat( request.sessionId(), request.message(), chatMemory ); return ResponseEntity.ok( new ChatResponse( response, request.sessionId(), LocalDateTime.now() ) ); } catch (RateLimitExceededException e) { return ResponseEntity .status(HttpStatus.TOO_MANY_REQUESTS) .body(new ChatResponse( "You're sending requests too quickly. " + "Please wait a moment.", request.sessionId(), LocalDateTime.now() )); } } public record ChatRequest(String sessionId, String message) {} public record ChatResponse( String response, String sessionId, LocalDateTime timestamp ) {}}
A question that confused me for weeks: when do I use Function Calling, and when do I use RAG?
| Situation | Use ||---------------------------------------------------------------|-------------------|| User asks about information in your documents | RAG || User asks for real-time data (weather, stocks, live status) | Function Calling || User wants to perform an action (book, send, create, update) | Function Calling || User asks questions about your product knowledge base | RAG || User needs calculations or data processing | Function Calling || User asks, "What does my contract say about X?" | RAG || User says, "Update my profile with this information." | Function Calling |
They’re complementary, not competing. Production AI applications typically use both — RAG for knowledge retrieval, Function Calling for actions and real-time data.
When I first got Function Calling working properly, I showed a colleague a demo — the assistant checking weather and booking a cab in one natural conversation.
His reaction surprised me: “This is just if-else logic with extra steps.”
He wasn’t entirely wrong. At a certain level of abstraction, Function Calling is a dynamic dispatch mechanism — the model selects which function to call based on natural language intent instead of explicit code conditions.
But that’s exactly what makes it powerful.
Traditional if-else dispatch requires you to anticipate every possible user intent and write explicit code for each one. Function Calling lets natural language be the dispatch mechanism — the model infers intent from how users actually express themselves, not how you anticipated they would.
That shift — from you anticipating intent to the model understanding intent — is what makes AI applications feel qualitatively different from rule-based systems, even when the underlying execution is similar.
The intelligence isn’t in the functions. It’s in knowing which function to call, when, and with what — and that’s what the model contributes.
Build a personal task manager assistant with three functions:
Test with these natural language requests:
Observe how the model maps natural language to structured function calls without you writing any intent-parsing logic. Then add a fourth function — deleteTask(taskId) — with a confirmation gate, and test what happens when a user says "Delete all my completed tasks."
Function Calling bridges the gap between AI that talks about actions and AI that performs them.
The model contributes intelligence — understanding what the user wants, selecting the right function, passing the right parameters, interpreting results. Your application contributes capability — the actual connections to real systems, with proper validation, security, and error handling.
Together they create something neither could build alone: an AI application that understands natural language and does something real with that understanding.
In Part 34, we’ll cover Tool Calling Explained — building on everything in this article to show how Spring AI’s higher-level @Tool abstraction makes Function Calling more ergonomic for common use cases, and when to use the raw function calling approach instead.
The pieces are coming together. Keep building.
Function Calling Explained: How AI Models Trigger Real Actions in Your Application — Part 33 was originally published in Stackademic on Medium, where people are continuing the conversation by highlighting and responding to this story.