Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3

A technical tutorial demonstrates an end-to-end supervised fine-tuning pipeline for the XYZ-Aquila-SFT dataset using Hugging Face Transformers, PyTorch, and PEFT, culminating in LoRA fine-tuning of Qwen3-0.6B for tool-calling tasks. The guide covers dataset streaming, parsing multi-turn tool-use trajectories, converting tool schemas, and evaluating tool-call prediction before and after training, with configuration parameters including 400 streamed examples, 30 training steps, and a learning rate of 1e-4.

In this tutorial, we implement an end-to-end supervised fine-tuning pipeline for the XYZ-Aquila-SFT https://huggingface.co/datasets/XYZAILab/XYZ-Aquila-SFT dataset, Hugging Face Transformers, PyTorch, and PEFT. We stream and inspect the dataset, parse multi-turn tool-use trajectories, extract structured tool calls, analyze corpus characteristics, and preserve embedded reasoning and observation patterns. We then convert tool schemas between message-embedded and structured formats, render Qwen-compatible ChatML with assistant-only loss masking, prepare a custom PyTorch dataset and collator, and fine-tune Qwen3-0.6B with LoRA. Finally, we evaluate tool-call prediction before and after training and export both the transformed dataset and corpus statistics for further experimentation. python import os, sys, subprocess CFG = dict REPO = "XYZAILab/XYZ-Aquila-SFT", LANG = "en", N STREAM = 400, N EVAL = 40, MODEL ID = "Qwen/Qwen3-0.6B", MAX SEQ LEN = 2048, LENGTH POLICY = "truncate", RUN TRAINING = True, MAX STEPS = 30, GRAD ACCUM = 8, LR = 1e-4, LORA R = 16, RUN EVAL = True, N EVAL PROBES = 24, OUT DIR = "/content/aquila out", SEED = 0, os.makedirs CFG "OUT DIR" , exist ok=True def pip pkgs : subprocess.run sys.executable, "-m", "pip", "install", "-q", "-U", pkgs , check=False pip "datasets =3.0.0", "transformers =4.51.0", "peft =0.13.0", "accelerate =1.0.0" import json, re, math, random, statistics as stats from collections import Counter, defaultdict from dataclasses import dataclass, field from typing import Any, Dict, List, Optional import torch import matplotlib.pyplot as plt from datasets import load dataset from transformers import AutoTokenizer, AutoModelForCausalLM, get cosine schedule with warmup random.seed CFG "SEED" ; torch.manual seed CFG "SEED" DEV = "cuda" if torch.cuda.is available else "cpu" BF16 = DEV == "cuda" and torch.cuda.is bf16 supported print f"device={DEV} bf16={BF16} torch={torch. version }" print f"\n 1 streaming {CFG 'REPO' }:{CFG 'LANG' } ..." stream = load dataset CFG "REPO" , CFG "LANG" , split="train", streaming=True RAW: List Dict str, Any = list stream.take CFG "N STREAM" print f" pulled {len RAW } rows; keys = {list RAW 0 .keys }" r = RAW 0 print f" question :110 : { r 'question' :110 }..." print f" answer : { r 'answer' :80 }" print f" number of tool calls : { r 'number of tool calls' }" print f" trajectory len : {len r 'trajectory' } msgs" print f" role sequence first8 : { m 'role' for m in r 'trajectory' :8 }" We configure the dataset, model, training parameters, output directory, and reproducibility settings for the complete workflow. We install the required Hugging Face, PEFT, Accelerate, and PyTorch-related dependencies and detect whether a CUDA GPU and BF16 support are available. We then stream a limited number of XYZ-Aquila-SFT examples, inspect the dataset schema, and examine the structure of the first tool-use trajectory. TOOLS BLOCK RE = re.compile r"<tools \s . ? \s </tools ", re.S THINK RE = re.compile r"<think . ? </think ", re.S TOOL RESP RE = re.compile r"<tool response \s . ? \s </tool response ", re.S TOOLS HDR RE = re.compile r"\n\n Tools\n\n" def iter json objects text: str, limit: int = 1 : """Nesting-safe JSON scanner. Regex like r'\\{. ?\\}' breaks on nested arguments objects, which every real tool call has.""" dec, i, n, out = json.JSONDecoder , 0, len text , while i < n and len out < limit: while i < n and text i not in "{ ": i += 1 if i = n: break try: obj, end = dec.raw decode text, i except json.JSONDecodeError: i += 1 continue out.append obj ; i = end return out def parse tool calls content: str - List Dict str, Any : calls = for m in re.finditer r"<tool call ", content : got = iter json objects content m.end : , limit=1 if got: calls.append got 0 return calls @dataclass class Trajectory: question: str answer: str declared calls: int messages: List Dict str, str system core: str = "" tools: List Dict str, Any = field default factory=list tools suffix: str = "" calls: List Dict str, Any = field default factory=list n observations: int = 0 n think: int = 0 @property def tool names self : return c.get "name", "?" for c in self.calls @property def depth self : return len self.messages def parse row row: Dict str, Any - Trajectory: msgs = {"role": m "role" , "content": m "content" } for m in row "trajectory" t = Trajectory row "question" , row "answer" , row "number of tool calls" , msgs if msgs and msgs 0 "role" == "system": sysmsg = msgs 0 "content" split = TOOLS HDR RE.search sysmsg if split: t.system core = sysmsg :split.start t.tools suffix = sysmsg split.start : else: t.system core = sysmsg blk = TOOLS BLOCK RE.search sysmsg if blk: t.tools = iter json objects blk.group 1 , limit=64 for m in msgs: if m "role" == "assistant": t.calls += parse tool calls m "content" t.n think += len THINK RE.findall m "content" else: t.n observations += len TOOL RESP RE.findall m "content" return t TRAJ = parse row r for r in RAW t0 = TRAJ 0 print f"\n 2 parsed {len TRAJ } trajectories" print f" tool schemas found : { fn.get 'function', fn .get 'name' for fn in t0.tools }" print f" parsed calls : {len t0.calls } declared {t0.declared calls} " print f" observations : {t0.n observations} think blocks: {t0.n think}" if t0.calls: print f" sample call : {json.dumps t0.calls 0 , ensure ascii=False :200 }" agree = sum len t.calls == t.declared calls for t in TRAJ print f" parser vs 'number of tool calls': {agree}/{len TRAJ } exact match" calls per = len t.calls for t in TRAJ depth per = t.depth for t in TRAJ chars per = sum len m "content" for m in t.messages for t in TRAJ name freq = Counter n for t in TRAJ for n in t.tool names argkey freq = defaultdict Counter for t in TRAJ: for c in t.calls: args = c.get "arguments", {} if isinstance args, dict : for k in args: argkey freq c.get "name", "?" k += 1 def q xs, p : xs = sorted xs ; return xs min len xs - 1, int p len xs print "\n 3 corpus statistics" print f" tool calls / traj : mean {stats.mean calls per :.1f} p50 {q calls per,.5 } " f"p90 {q calls per,.9 } max {max calls per }" print f" messages / traj : mean {stats.mean depth per :.1f} p90 {q depth per,.9 } max {max depth per }" print f" chars / traj : mean {stats.mean chars per :,.0f} p90 {q chars per,.9 :,}" print f" tool distribution : {dict name freq }" for k, v in argkey freq.items : print f" {k:<24} arg keys - {dict v.most common 6 }" tot = sum chars per ; top = sum sorted chars per -max 1, len chars per //10 : print f" top-10% longest trajectories hold {100 top/tot:.1f}% of all characters" fig, ax = plt.subplots 1, 3, figsize= 15, 3.6 ax 0 .hist calls per, bins=40 ; ax 0 .set yscale "log" ; ax 0 .set title "tool calls / trajectory" ax 1 .hist depth per, bins=40 ; ax 1 .set yscale "log" ; ax 1 .set title "messages / trajectory" ax 2 .bar list name freq , list name freq.values ; ax 2 .set title "tool usage" ; ax 2 .tick params axis="x", rotation=20 plt.tight layout ; plt.show We define nesting-safe utilities for extracting JSON tool calls, reasoning blocks, observations, and embedded tool schemas from each conversation. We convert every raw dataset row into a structured trajectory object and verify that the parsed tool-call counts match the values declared by the dataset. We then calculate corpus-level statistics and visualize the distributions of tool calls, message depth, trajectory size, and tool usage frequency. QWEN3 TOOLS TMPL = "You are provided with function signatures within <tools </tools XML tags:\n<tools \n" "{lines}\n</tools \n\nFor each function call, return a json object with function name " "and arguments within <tool call </tool call XML tags:\n<tool call \n" '{{"name": <function-name , "arguments": <args-json-object }}\n</tool call ' def extract tools t: Trajectory - Dict str, Any : """message-embedded schemas - {'messages': ... , 'tools': ... }""" msgs = dict m for m in t.messages if msgs and msgs 0 "role" == "system": msgs 0 "content" = t.system core return {"messages": msgs, "tools": t.tools, "question": t.question, "answer": t.answer} def render tools rec: Dict str, Any - List Dict str, str : """inverse: structured tools - schemas re-embedded in the system message""" msgs = dict m for m in rec "messages" if rec "tools" and msgs and msgs 0 "role" == "system": lines = "\n".join json.dumps x, ensure ascii=False for x in rec "tools" msgs 0 "content" = msgs 0 "content" + QWEN3 TOOLS TMPL.format lines=lines return msgs rt = render tools extract tools t0 exact = rt 0 "content" == t0.messages 0 "content" print f"\n 4 extract- render byte-exact: {exact}" if not exact: print " template drift detected - using verbatim tools suffix for render " a, b = t0.messages 0 "content" , rt 0 "content" i = next i for i in range min len a , len b if a i = b i , min len a , len b print f" first divergence @{i}: {a i:i+70 r} vs {b i:i+70 r}" tok = AutoTokenizer.from pretrained CFG "MODEL ID" if tok.pad token is None: tok.pad token = tok.eos token IM START, IM END, NL = "<|im start| ", "<|im end| ", "\n" def render and mask t: Trajectory, max len: int, policy: str : """Manual ChatML so we control masking token-exactly. WHY NOT apply chat template : Qwen3's template deletes <think ...</think from every assistant turn except the last. On this dataset that silently destroys most of the reasoning supervision you are paying to train on. """ ids, labels = , for m in t.messages: head = tok f"{IM START}{m 'role' }{NL}", add special tokens=False .input ids body = tok m "content" , add special tokens=False .input ids tail = tok f"{IM END}{NL}", add special tokens=False .input ids seg = head + body + tail if m "role" == "assistant": lab = -100 len head + body + tail else: lab = -100 len seg ids += seg; labels += lab if len ids max len: if policy == "drop": return None ids, labels = ids :max len , labels :max len if all l == -100 for l in labels : return None return {"input ids": ids, "labels": labels} probe = {"role": "system", "content": "S"}, {"role": "user", "content": "U"}, {"role": "assistant", "content": "A"} mine = "".join f"{IM START}{m 'role' }{NL}{m 'content' }{IM END}{NL}" for m in probe theirs = tok.apply chat template probe, tokenize=False, add generation prompt=False print f"\n 5 manual ChatML == chat template on tool-free probe: { mine == theirs}" if mine = theirs: print f" mine : { mine r}\n theirs: { theirs r} informational only " ENC = e for e in render and mask t, CFG "MAX SEQ LEN" , CFG "LENGTH POLICY" for t in TRAJ if e sup = sum 1 for x in e "labels" if x = -100 / len e "labels" for e in ENC print f" encoded {len ENC }/{len TRAJ } examples" print f" supervised-token ratio: mean {stats.mean sup :.3f} p10 {q sup,.1 :.3f} p90 {q sup,.9 :.3f}" over = sum 1 for t in TRAJ if sum len tok m 'content' , add special tokens=False .input ids for m in t.messages :3 CFG "MAX SEQ LEN" print f" trajectories whose first 3 msgs alone exceed MAX SEQ LEN: {over}" SPLIT = len ENC - min CFG "N EVAL" , len ENC //5 TRAIN ENC, EVAL TRAJ = ENC :SPLIT , TRAJ SPLIT: class SFTSet torch.utils.data.Dataset : def init self, rows : self.rows = rows def len self : return len self.rows def getitem self, i : return self.rows i def collate batch : L = max len b "input ids" for b in batch pad = tok.pad token id return { "input ids": torch.tensor b "input ids" + pad L-len b "input ids" for b in batch , "labels": torch.tensor b "labels" + -100 L-len b "labels" for b in batch , "attention mask": torch.tensor 1 len b "input ids" + 0 L-len b "input ids" for b in batch , } loader = torch.utils.data.DataLoader SFTSet TRAIN ENC , batch size=1, shuffle=True, collate fn=collate print f"\n 6 train={len TRAIN ENC } eval trajectories={len EVAL TRAJ }" We extract embedded tool definitions into a structured format and reconstruct them to test whether the conversion preserves the original system message. We manually render each trajectory in ChatML format to retain all reasoning content and apply loss only to assistant-generated tokens. We also tokenize the examples, enforce the selected sequence-length policy, create the training and evaluation split, and prepare a padded PyTorch DataLoader. python def build probes trajs, n : """Teacher-forced probes: cut the trajectory right before an assistant turn that issues a tool call; the gold label is that call.""" probes = for t in trajs: for i, m in enumerate t.messages : if m "role" = "assistant": continue gold = parse tool calls m "content" if not gold: continue prefix = "".join f"{IM START}x 'role' {NL}" for x in prefix = "".join f"{IM START}{p 'role' }{NL}{p 'content' }{IM END}{NL}" for p in t.messages :i + f"{IM START}assistant{NL}" if len tok prefix, add special tokens=False .input ids CFG "MAX SEQ LEN" - 160: continue probes.append {"prefix": prefix, "gold": gold 0 } break if len probes = n: break return probes @torch.no grad def eval tool calls model, probes, tag : model.eval name hit = arg f1 = parsed = 0 for p in probes: enc = tok p "prefix" , return tensors="pt", add special tokens=False .to model.device out = model.generate enc, max new tokens=160, do sample=False, pad token id=tok.pad token id gen = tok.decode out 0 enc.input ids.shape 1 : , skip special tokens=True pred = parse tool calls gen or iter json objects gen, limit=1 or None 0 if not isinstance pred, dict : continue parsed += 1 g = p "gold" name hit += int pred.get "name" == g.get "name" pk = set pred.get "arguments" or {} .keys if isinstance pred.get "arguments" , dict else set gk = set g.get "arguments" or {} .keys if isinstance g.get "arguments" , dict else set if pk or gk: inter = len pk & gk arg f1 += 0.0 if inter == 0 else 2 inter/ len pk +len gk n = max 1, len probes print f" {tag} parseable {parsed}/{n} | tool-name acc {name hit/n:.3f} | arg-key F1 {arg f1/n:.3f}" return dict parsed=parsed/n, name acc=name hit/n, arg f1=arg f1/n PROBES = build probes EVAL TRAJ, CFG "N EVAL PROBES" print f" built {len PROBES } teacher-forced probes" results = {} if CFG "RUN TRAINING" : from peft import LoraConfig, get peft model dtype = torch.bfloat16 if BF16 else torch.float32 model = AutoModelForCausalLM.from pretrained CFG "MODEL ID" , torch dtype=dtype, attn implementation="sdpa" .to DEV model.config.use cache = False model.gradient checkpointing enable model.enable input require grads if CFG "RUN EVAL" and PROBES and DEV == "cuda": print "\n 8 baseline eval" results "before" = eval tool calls model, PROBES, "base" model = get peft model model, LoraConfig r=CFG "LORA R" , lora alpha=2 CFG "LORA R" , lora dropout=0.05, bias="none", task type="CAUSAL LM", model.print trainable parameters opt = torch.optim.AdamW p for p in model.parameters if p.requires grad , lr=CFG "LR" , weight decay=0.0, betas= 0.9, 0.95 sched = get cosine schedule with warmup opt, 5, CFG "MAX STEPS" scaler = torch.amp.GradScaler "cuda", enabled= DEV == "cuda" and not BF16 amp dt = torch.bfloat16 if BF16 else torch.float16 print f"\n 7 training {CFG 'MAX STEPS' } steps " f" bs1 x accum{CFG 'GRAD ACCUM' } = {CFG 'GRAD ACCUM' } traj/step " model.train ; step = 0; run = None; it = iter loader while step < CFG "MAX STEPS" : opt.zero grad set to none=True ; acc = 0.0 for in range CFG "GRAD ACCUM" : try: batch = next it except StopIteration: it = iter loader ; batch = next it batch = {k: v.to DEV for k, v in batch.items } with torch.autocast DEV, dtype=amp dt, enabled= DEV == "cuda" : loss = model batch .loss / CFG "GRAD ACCUM" scaler.scale loss .backward if scaler.is enabled else loss.backward acc += loss.item if scaler.is enabled : scaler.unscale opt scaler.step opt , scaler.update if scaler.is enabled else opt.step sched.step ; step += 1 run = acc if run is None else 0.9 run + 0.1 acc if step % 5 == 0 or step == 1: print f" step {step: 3}/{CFG 'MAX STEPS' } loss {acc:.4f} ema {run:.4f} " f"lr {sched.get last lr 0 :.2e} ppl {math.exp min 20, acc :.1f}" model.save pretrained f"{CFG 'OUT DIR' }/lora adapter" ; tok.save pretrained f"{CFG 'OUT DIR' }/lora adapter" print f" adapter - {CFG 'OUT DIR' }/lora adapter" if CFG "RUN EVAL" and PROBES and DEV == "cuda": print "\n 8 post-training eval" model.config.use cache = True results "after" = eval tool calls model, PROBES, "lora" model.config.use cache = False if "before" in results and "after" in results: print "\n delta:", {k: round results 'after' k - results 'before' k , 3 for k in results 'after' } print " 30 steps on ~350 trajectories is a smoke test, not a result — " "expect noise, and scale N STREAM/MAX STEPS for anything real. " We build teacher-forced evaluation probes by cutting trajectories immediately before assistant turns that contain tool calls. We load Qwen3-0.6B, measure its baseline tool-call performance, attach LoRA adapters, and fine-tune the model using gradient accumulation, mixed precision, checkpointing, clipping, and cosine learning-rate scheduling. We then evaluate the adapted model, compare its metrics with the baseline, and save the trained LoRA adapter and tokenizer. struct path = f"{CFG 'OUT DIR' }/aquila {CFG 'LANG' } structured tools.jsonl" with open struct path, "w", encoding="utf-8" as f: for t in TRAJ: f.write json.dumps extract tools t , ensure ascii=False + "\n" stats path = f"{CFG 'OUT DIR' }/corpus stats.json" with open stats path, "w" as f: json.dump {"n": len TRAJ , "tool freq": dict name freq , "calls mean": stats.mean calls per , "calls max": max calls per , "depth p90": q depth per, .9 , "encoded": len ENC , "supervised ratio mean": stats.mean sup , "eval": results}, f, indent=2 print f"\n 9 wrote:\n {struct path}\n {stats path}" print "done." We export every parsed trajectory as a structured JSONL record containing messages, tool schemas, questions, and answers. We also save a JSON report containing corpus size, tool frequencies, trajectory statistics, supervised-token ratios, and available evaluation results. We finish the workflow with reusable dataset artifacts, analytical outputs, and model files stored in the configured output directory. In conclusion, we completed a practical pipeline for analyzing, transforming, fine-tuning, and evaluating complex tool-use trajectories from the XYZ-Aquila-SFT dataset. We preserved the original conversational structure, applied token-level supervision only to assistant responses, and used LoRA to adapt Qwen3-0.6B efficiently on a Colab-compatible GPU. We also compared baseline and post-training tool-call performance through teacher-forced evaluation and exported reusable structured records, model adapters, and analytical statistics. This workflow gives us a strong foundation for scaling tool-aware supervised fine-tuning, testing alternative sequence-length policies, and training more capable agentic language models. Check out the FULL CODES here . Also, feel free to follow us on and don’t forget to join our Twitter https://x.com/intent/follow?screen name=marktechpost and Subscribe to 150k+ML SubReddit https://www.reddit.com/r/machinelearningnews/ . Wait are you on telegram? our Newsletter https://magic.beehiiv.com/v1/f5e63dd4-5653-4f09-83e2-321a8b1ba526?email={{email}} now you can join us on telegram as well. https://t.me/machinelearningresearchnews Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us https://forms.gle/wbash1wF6efRj8G58 Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.