Function calling vs free-form generation

A developer building a fintech receipt-scanning app found that free-form LLM generation produced non-deterministic, unauditable results — odd rounding, invented discounts, and unfounded fraud flags — and switched to function calling to constrain the model to narrow, typed endpoints. The approach, using tools such as extract_amount(image) → float, record_transaction(user_id, amount, category) → bool, and check_fraud(user_id, amount) → bool, keeps business logic, auth checks, retries, and logging on the server side; the same pattern was applied in an edutech tutoring bot via a solve_linear(a, b) → float function to prevent hallucinated formulas.

Last month I was knee‑deep in a fintech app that let users snap a receipt and instantly see a spending summary. The LLM was supposed to read the image, pull out numbers, and dump them into our ledger. Instead it started “doing its own business logic”: rounding oddly, applying a grandma‑style discount, even flagging a transaction as fraud on a hunch. It felt like handing a chef a pan and watching them secretly add mystery spices. The issue was clear: I was using free‑form generation as a Swiss‑army knife and letting the model decide how to do the work. The result? Non‑deterministic output, audit nightmares, and a pile of “why did the AI do that?” tickets. Enter function calling. Think of it as a polite waiter: you tell the model, “If you need a number, call the extract amount image function and I’ll give you a float.” The model can’t magically compute inside its brain; it must ask your API for the exact piece you typed. Your service handles the heavy lifting, has auth checks, logs every request, retries on failure, and returns a deterministic result. This is exactly what OpenAI’s Function Calling and the “Tools” feature do – expose narrow, typed endpoints and keep the business logic on the server side. A quick analogy: free‑form generation is like giving a kid a 3‑D printer and saying “make me a toy.” They might print a dinosaur, a spaceship, or a banana. Function calling is like handing them a LEGO set with a clear instruction sheet – they can only build what the instructions allow, and you can count each piece as it’s added. In our fintech case we defined three tools: extract amount image bytes → float record transaction user id, amount, category → bool check fraud user id, amount → bool Now the LLM says, “I need the amount from this receipt – call extract amount ,” and we get a clean number. If the API hiccups we retry, everything is logged, and auditors can see exactly what was asked and what was returned. No hidden discounts. In edutech we built a tutoring bot that could answer math questions, but we didn’t want it to hallucinate formulas. We exposed a solve linear a, b → float function. The model now asks, “What’s x in 2x+3=7? Call solve linear with a=2, b=-4 .” The answer is always correct, traceable, and teachers can audit the logs. Key take‑aways So the next time you’re tempted to let the LLM write the whole recipe, remember: let it be the clever waiter, not the secret chef. What narrow tool would you love to add to your stack? Drop a comment and let’s swap ideas If you are someone who loves to know the technical work and architecture design I have shared more details based on my experience on this here: https://github.com/SalmonJoy/My guide for building AI systems/blob/main/Function calling vs free-form generation.md https://github.com/SalmonJoy/My guide for building AI systems/blob/main/Function calling vs free-form generation.md