Teaching an LLM a little Jiu Jitsu

A practitioner built a domain-expert Brazilian Jiu Jitsu coaching assistant on a 7-billion-parameter open-weights model, drawing on roughly 30,000 instructional video segments, 14,000 competition match clips and 79 written technical guides. The seven-step pipeline cleans phonetic transcription errors and prepends contextual retrieval headers before embedding, builds an explicit knowledge graph of canonical tags and cross-school synonyms, audits video keyframes with CLIP plus a vision-language model, and fine-tunes an ONNX bge-reranker-base cross-encoder on timestamped links mined from existing documents. Training runs 2.5x decision-weighted QLoRA SFT followed by zero-extra-VRAM PEFT DPO on a single GPU, gated by an automated linter that rejects datasets above a 0.35% 7-gram threshold, with deployment on volume-backed vLLM serverless.

Earlier this year, my colleagues https://siliconangle.com/2026/03/05/specialized-telecom-ai-reshapes-network-operations-mwc26/ at Relational AI https://www.relational.ai/ helped AT&T https://www.linkedin.com/posts/farbodtavakkoli earlier-this-year-we-trained-and-open-sourced-share-7513354677996965889-jUke/ build LLM models focused on Telecom called Otel https://arxiv.org/html/2610.07766v1 and push it to the top of the leaderboard https://huggingface.co/spaces/GSMA/open-telco-leaderboard . I wish I could tell you all about it, but I wasn’t on the team. No worries though, you can read all about on the links above. Instead I’m going to tell you about training an LLM focused on Brazilian Jiu Jitsu. The idea is to ask it all kinds of dumb beginner questions and have it provides great answers. All without getting frustrated or telling you to focus on the fundamentals before trying flying armbars. Now, I wish I could just sign up Claude to our local school, fast forward 10 years and be done, but LLMs can’t learn that way. Large Language Models need Language, they need Text. Luckily there are thousands of “Instructionals” video classes teaching a topic or technique of usually high quality and thousands of YouTube videos of varying degrees of quality we can use as our starting point. While kid classes pay the bills for local schools, it turns out middle aged men are the target audience of the instructional market. I had already purchased over 50 instructionals on BJJ Fanatics https://bjjfanatics.com/ , but that wasn’t nearly enough, I had to use the Jozef Chen method https://www.youtube.com/shorts/KDgwxidXdRw and got a lot more. I also got some competition match clips that were narrated from youtube and downloaded them using yt-dlp https://github.com/yt-dlp/yt-dlp . Everyone is short on attention these days, so this is the AI:DR version for you to replicate this for your own video or document library: 1. Clean phonetic transcription errors and prepend Contextual Retrieval headers Context: Author — Source | Category | Tags before embedding. 2. Build an explicit Knowledge Graph mapping canonical tags, cross-school synonyms, parent-child tag hierarchy, and mutual-exclusivity rules. 3. Audit video keyframes with CLIP + a Vision-Language Model so silent physical demonstrations become searchable Visual Demonstration chunks. 4. Mine timestamped links from your existing docs to fine-tune an ONNX Cross-Encoder Re-Ranker bge-reranker-base with 5-way listwise loss. 5. Synthesize a 3-Track SFT dataset 6-excerpt RAFT with distractors + diagnostic Go/No-Go gates + Knowledge Graph lineage/rules and gate it with an automated <0.35% 7-gram linter . 6. Run 2.5x Decision-Weighted QLoRA SFT followed by Zero-Extra-VRAM PEFT DPO with model.disable adapter : + 0.15 NLL anchor on a single GPU. 7. Verify against a rubric-scored domain benchmark and deploy on volume-backed vLLM serverless . A Recipe for Building a Domain-Expert AI Coach Suppose you have a massive library of specialized domain knowledge—in our case, about 30,000 instructional video segments , 14,000 competition match clips , and 79 written technical guides on Brazilian Jiu-Jitsu. Your domain might be surgical procedures, aviation maintenance, industrial robotics, or tax law, but the problem is the same. When a user asks a question, you want an AI assistant that: 1. Speaks the real language of the domain knowing when five different terms from different schools mean the exact same thing . 2. Retrieves the exact moments that matter across both spoken explanations and silent visual demonstrations. 3. Writes a clear, multi-paragraph answer with inline 1 – 6 citations that jump the video player to the exact timestamp, while ignoring near-miss search results. 4. Runs fast and cheap on a 7-billion-parameter open-weights model without frontier API bills. If you just wire up basic vector search to an off-the-shelf 7B model, you will get vague fortune-cookie advice, hallucinated rules, and backwards physical mechanics like telling a student to spin the wrong way out of a heel hook . We are going to walk through the complete, step-by-step recipe we used to build and taking our domain benchmark BJJ-Bench-250 to 5.00 / 5.0 on Kinematic Precision with a 0.0% distractor citation rate . Let’s jump in. The Ingredients Architecture Overview Before we start cooking, here is what goes into the system: You only need a single GPU an NVIDIA L40S, A40, or A100 on RunPod for ~30 minutes to train both the cross-encoder re-ranker and the 7B language model. Step 1: Clean the “Slightly Wrong” Data and Add Contextual Headers Real-world transcripts are messy. Instructors are out of breath on a mat, using domain loanwords from Japanese and Portuguese. Speech-to-text models like Whisper hear “kimura” as "camera" , “De La Riva” as "dealer river" , “Danaher” as "donna hair" , “ashi garami” as "oshi garami" , and “inside sankaku” as "inside sinkhole" . 1a. Repair Phonetic Mishearings First Don’t waste model capacity teaching an LLM to guess phonetic typos if you can fix them deterministically at ingest time. Build a domain replacement dictionary and run it over your raw transcripts before you embed or train on them: js const PHONETIC REPAIRS = /\bcamera\b/gi, 'kimura' , /\bdealer river\b/gi, 'de la riva' , /\bdonna ?:her|hair \b/gi, 'Danaher' , /\boshi garami\b/gi, 'ashi garami' , /\binside sinkhole\b/gi, 'inside sankaku' , /\bhonor roll\b/gi, 'honey hole' , /\bMarcelo team\b/gi, 'marcelotine' , /\b ?:uma|homo plata\b/gi, 'omoplata' , ; 1b. Prepend Contextual Retrieval Headers Before Embedding A 60-second video transcript window is full of pronouns: “When he pushes here, I move my foot there and lock this up.” By itself, that chunk has no idea what position it is in. Before computing dense embeddings BAAI/bge-small-en-v1.5 or full-text search vectors, prepend a structured Contextual Retrieval Header to every chunk: BJJ Context: Gordon Ryan — Systematically Attacking The Guard NOGI | Chapter: Half Butterfly Sweeps | Tags: half-butterfly, butterfly-hook, half-guard <spoken transcript body Now every single chunk carries the instructor, course title, ruleset/attire GI vs. NOGI , chapter, and canonical tags right inside its vector representation. Step 2: Build a Domain Knowledge Graph Vector similarity is great at fuzzy semantics, but it is terrible at strict domain logic. Before training your model, encode your domain’s taxonomy into an explicit Knowledge Graph . Our graph has three jobs: 1. Cross-Lineage Synonym Equivalence 153 canonical tags, 388 synonyms : Different schools name the same mechanic differently. Inside Sankaku , The Saddle , 4/11 , Honey Hole , and Top Rock all point to the canonical saddle node. When a user asks about "honey hole" , the graph expands the search query across all synonyms and passes a lineage note into the LLM prompt so it knows they are the exact same position. 2. Broad-to-Specific Tag Pruning : If a video segment or competition clip is tagged with a specific guard half-butterfly or de-la-riva-guard or a specific leg entanglement saddle or outside-ashi , the graph automatically suppresses broad parent tags open-guard , ashi-garami during auto-tagging. That keeps Inverse Document Frequency IDF tag weights sharp during hybrid search. 3. Kinematic Mutual Exclusivity scoreKinematicCompatibility : Two athletes cannot be in closed-guard and in the saddle at the same time. Encoding mutually exclusive positions in the graph lets our re-ranker apply a deterministic -0.25 penalty to near-miss search results that start from a physically incompatible position, and a -0.20 penalty when GI vs. NOGI attire mismatches. Step 3: Give Your Retriever “Eyes” Multimodal Visual Grounding In any video-heavy domain, experts show things with their hands that they never say out loud: “Look at my grip right here—if I angle my knee like this, he can’t flatten me.” If you only index audio transcripts, those physical details are invisible. Here is how to turn silent visual demonstrations into searchable text chunks: 1. Extract Candidate Keyframes : Pull video frames at the timestamps where your guides or chapters reference a demonstration. 2. Filter Talking Heads with Contrastive CLIP Scoring : Run a 512-dim CLIP vision model Qdrant/clip-ViT-B-32-vision over every frame and score it against positive domain prompts “two grapplers actively demonstrating a jiu-jitsu technique on the mat” minus negative prompts “a static title card or a single person talking to the camera” . 3. Caption the Physical Mechanics with a Vision-Language Model : For verified demonstration keyframes 861 in our library , fuse the spoken quote, the guide’s biomechanical claim, and a Video-VLM Qwen2.5-VL-7B kinematic description of the grips, hooks, and limb angles visible on screen. 4. Index into chunk embeddings : Insert those descriptions into your vector table as Visual Demonstration — <Topic ... chunks pinned to the exact demonstration timestamp timestamp sec . Now, when a user asks how a position is set up, hybrid search retrieves both the spoken explanation and the Visual Demonstration chunk. Step 4: Train a Custom Cross-Encoder Re-Ranker from Internal Citations A bi-encoder embeds the user’s question and the video chunk separately into two 384-dimensional vectors and takes their dot product. That makes it fast enough to scan 265,000 chunks in PostgreSQL via HNSW, but it compresses an entire 60-second video transcript into a single vector. Subtle mechanical differences “defending the knee cut from bottom half guard” vs. “finishing the knee cut from top headquarters” blur together and that’s a real problem. First-stage hybrid retrieval combining pgvector HNSW dense search https://github.com/pgvector/pgvector/blob/master/README.md , PostgreSQL tsvector BM25 search, and IDF-weighted tag matching via Reciprocal Rank Fusion https://medium.com/@devalshah1619/mathematical-intuition-behind-reciprocal-rank-fusion-rrf-explained-in-2-mins-002df0cc5e2a gives you a fast pool of about 30 candidate moments. To pick the best 6 moments for the LLM, you want a cross-encoder re-ranker that reads question, candidate passage together with full bidirectional attention. Off-the-shelf re-rankers BAAI/bge-reranker-base https://huggingface.co/BAAI/bge-reranker-base Mine Your Existing Documentation Links Our 79 markdown guides contained about 3500 timestamped video links /watch/<segment uuid ?t=<seconds . Every link is a verified ground-truth pair connecting a technical claim to an exact video chunk We combined those guide citations with our Visual Demonstration chunks and newly tagged taxonomy chunks to build about 4000 training triplets , where each row contains: - query : The technical question or biomechanical claim from the guide section. - positive : The exact transcript or visual-demo chunk at timestamp t with its Contextual Retrieval header . - hard negatives 4 per query : 1. Two chunks from different segments in the same instructional volume same instructor, same general topic, wrong specific detail . 2. One chunk from the same guide topic with high lexical word overlap. 3. One cross-instructional chunk sharing a canonical tag. - Two chunks from Fine-Tune with 5-Way Listwise Loss and Export to ONNX We fine-tuned BAAI/bge-reranker-base for 3 epochs lr=2e-5 , ~2.5 minutes on a single GPU using a combined 5-way listwise Cross-Entropy + pairwise margin loss , then exported the model directly to ONNX so it runs locally inside our embedding microservice with zero GPU overhead: A cross-encoder BAAI/bge-reranker-base , a 278M-parameter RoBERTa model takes a query, passage pair concatenated together— <s query </s </s passage </s —and outputs a single scalar logit s q , p representing relevance. If you train a re-ranker with simple binary cross-entropy scoring one passage at a time as 0 or 1 , the model never directly compares the gold chunk against its near-miss sibling chunks from the same volume. Instead, for every query in a batch, we interleave 5 passages pos, neg1, neg2, neg3, neg4 , run them through the cross-encoder, reshape the output logits to batch size, 5 where column 0 is the gold positive and columns 1..4 are the four hard negatives, and optimize two losses together: 1. 5-Way Listwise InfoNCE Temperature-Scaled Cross-Entropy : Treats the 5 passages as a 5-class classification problem where class 0 is the winner, scaled by temperature : 2. Pairwise Margin Ranking Loss margin = 1.25 : Forces the positive logit to beat every individual hard negative by at least 1.25 logit units so scores stay well-separated when we later combine them with Knowledge Graph compatibility adjustments: Here is the core training loop: for step, qs, pos list, neg1 list, neg2 list, neg3 list, neg4 list in enumerate loader, 1 : bsz = len qs Interleave pos, neg1, neg2, neg3, neg4 per query - bsz 5 pairs flat q, flat p = , for i in range bsz : flat q.extend qs i 5 flat p.extend pos list i , neg1 list i , neg2 list i , neg3 list i , neg4 list i enc = tokenizer flat q, flat p, padding=True, truncation=True, max length=384, return tensors="pt" .to device optimizer.zero grad with torch.cuda.amp.autocast enabled=use amp : logits = model enc .logits.view bsz, 5 col 0 = positive, cols 1..4 = hard negatives targets = torch.zeros bsz, dtype=torch.long, device=device 1. 5-Way Listwise InfoNCE Loss temperature = 0.7 loss ce = ce loss fn logits / 0.7, targets 2. Pairwise Margin Ranking Loss margin = 1.25 margins = torch.clamp 1.25 - logits :, 0 - logits :, j , min=0.0 .mean for j in range 1, 5 loss = loss ce + 0.25 sum margins scaler.scale loss .backward scaler.step optimizer scaler.update scheduler.step Exporting to ONNX for Zero-GPU Production Serving In production, you don’t want to spin up a PyTorch GPU worker just to re-rank 20 candidate chunks. By exporting the fine-tuned cross-encoder to an ONNX graph with dynamic batch size and sequence axes, our lightweight Python embedding service runs it on CPU via onnxruntime + Rust tokenizers in about 40ms per query: 1. Export PyTorch cross-encoder to ONNXtorch.onnx.export model, dummy "input ids" , dummy "attention mask" , "models/bjj-reranker-v3-onnx/model.onnx", input names= "input ids", "attention mask" , output names= "logits" , dynamic axes={ "input ids": {0: "batch size", 1: "sequence"}, "attention mask": {0: "batch size", 1: "sequence"}, "logits": {0: "batch size"}, }, opset version=14, do constant folding=True, And at request time, OnnxCrossEncoder loads model.onnx and tokenizer.json once at boot with zero PyTorch dependency: python class OnnxCrossEncoder: def init self, model dir: Path : self.tokenizer = Tokenizer.from file str model dir / "tokenizer.json" self.tokenizer.enable truncation max length=384 self.tokenizer.enable padding pad id=1, pad token="<pad " self.session = ort.InferenceSession str model dir / "model.onnx" , providers= "CPUExecutionProvider" , def rerank self, query: str, documents: list str - list float : encodings = self.tokenizer.encode batch query, doc for doc in documents feed = { "input ids": np.array e.ids for e in encodings , dtype=np.int64 , "attention mask": np.array e.attention mask for e in encodings , dtype=np.int64 , } logits = self.session.run "logits" , feed 0 return float x for x in logits.reshape -1 .tolist Fine-tuning for 3 epochs lr=2e-5 , ~2.5 minutes on a single GPU jumped Recall@1 by +10.0% and MRR@10 to 0.8749 : | Re-Ranker Model 250 Held-Out Queries, 10 Candidates Each | MRR@10 | Recall@1 | Recall@3 | Mean Gold Rank | |---|---|---|---|---| | Off-the-shelf BAAI/bge-reranker-base | 0.8224 | 68.0% | 96.0% | 1.48 | | Fine-Tuned bjj-reranker-v3 ONNX | 0.8749 | 78.0% +10.0% | 97.6% | 1.37 | Step 5: Build a Three-Track SFT Dataset and Lint It Automatically Now let’s prepare the training data for the 7B coach model. If you want a model that works both as a RAG synthesizer when video excerpts are provided and as a direct domain expert answering biomechanics, lineage, and ruleset questions , build a Three-Track ChatML Dataset about 3,000 conversations : Track A: Multi-Document RAFT — Retrieval-Augmented Fine-Tuning 1,200 conversations 1,200 conversations In production, /ask sends 6 retrieved excerpts 1 – 6 from multiple instructors. Usually 4 of those excerpts directly answer the question “oracles” and 2 are near-miss neighbors “distractors” . If you only train your model on clean, single-excerpt prompts, it will either summarize 1 and ignore 2 – 6 , or hallucinate citations to the distractors. Using RAFT , every Track A training prompt packs 4 oracle chunks + 2 hard distractor chunks shuffled into 1 – 6 . The assistant completion is a 3-to-4 paragraph synthesis that weaves the multiple instructors together, places 1 – 6 citations at the end of supported sentences, and never cites the distractor indices . Track B: Deep Biomechanics & Diagnostic “Go / No-Go” Gates 1,680 conversations Real domain expertise is built around diagnostic physical checks —binary tests that tell you whether to advance a technique or bail out: - Outside Heel Hook Escape : “Can you point your toes plantarflexion ?” If yes, you still have rotational slack—clear your knee line. If your foot is pinned in dorsiflexion, spinning will tear your knee—tap immediately. - Knee Cut Pass : “Can your heel touch your backside?” If yes, the space behind your knee wedge is closed—slice down-and-back toward their hip fulcrum. - Half-Butterfly vs. Butterfly Guard : Distinguishing half-butterfly butterfly half guard , where your bottom leg traps one of the passer’s legs while your inside leg inserts a single butterfly-hook wedge from butterfly-guard double butterfly , seated open guard with both hooks inside . Make sure the concrete mechanical levers elbow , knee , hip , frame , wedge , fulcrum , vector , angle , wrist , heel , hook , post appear in the first paragraph of your training answers so the model leads with actionable mechanics rather than throat-clearing preamble. Track C: Combinatorial Knowledge Graph Lineage & Rulesets 210 conversations Generate Q&A pairs directly from your Knowledge Graph and competition rulebooks IBJJF, ADCC, EBI covering every synonym family, belt-level legality rule heel hooks, knee reaping, slams, jumping guard, sleeve cuffs , and tactical match strategy. Gate Your Dataset with an Automated Linter & Critic Never launch a training run without linting the dataset first. Our linter enforces three hard gates: 1. The <0.35% 7-Gram Diversity Gate : Counts every 7-word sequence across all 3,090 assistant responses. If any 7-word phrase repeats in more than 0.35% of responses, the linter rejects the dataset so the LoRA adapter can never memorize a repetitive boilerplate suffix. 2. The Biomechanical & Role-Inversion Critic : Scans every response for domain contradictions e.g., equating pointing toes with dorsiflexion, telling the top passer to tap, or claiming two synonymous positions are different . 3. Citation & Lever Density Gate : Verifies that Track A responses cite only oracle indices and that responses average 8.7+ concrete anatomical levers . Step 6: Train the 7B Model — Decision-Weighted QLoRA SFT + Zero-Extra-VRAM DPO With our 3,090-sample SFT dataset and 680-pair DPO dataset ready, training Qwen/Qwen2.5-7B-Instruct happens in two fast stages on a single GPU. Stage 6a: QLoRA SFT with 2.5x Decision-Token Weighting We load Qwen/Qwen2.5-7B-Instruct in 4-bit NF4 and attach a LoRA adapter across all 7 linear projections q proj , k proj , v proj , o proj , gate proj , up proj , down proj . Two implementation details make a huge difference here: 1. Contiguous Completion-Only Loss Masking with 2.5x Decision Weights : We mask the system and user prompt tokens labels = -100 so cross-entropy loss is only computed on the assistant’s coaching response. Within that response, we apply a 2.5x loss weight to high-leverage decision tokens: citation brackets 1 – 6 , diagnostic gates plantarflexion , dorsiflexion , tap , and ruleset/strategy terms IBJJF , ADCC , EBI , legal , illegal , reaping , strategy , position , lineage . 2. Slice Active Tokens Before Softmax 15x Less VRAM : Because Qwen2.5 has a large vocabulary 152,064 tokens , computing CrossEntropyLoss reduction="none" across a full padded batch allocates gigabytes of intermediate logits for prompt and padding tokens that just get masked out anyway. Slicing only the unmasked assistant tokens mask = seq labels = -100 per sequence before calling cross entropy is mathematically identical and cuts logit VRAM by more than 15x : python def compute weighted causal lm loss logits, labels, token weights : loss fct = torch.nn.CrossEntropyLoss reduction="none" total weighted loss = logits.new zeros , dtype=torch.float32 total weight = logits.new zeros , dtype=torch.float32 for i in range logits.size 0 : seq labels = labels i, 1: mask = seq labels = -100 if not mask.any : continue seq logits = logits i, :-1, : mask active labels = seq labels mask active weights = token weights i, 1: mask .float per tok loss = loss fct seq logits.float , active labels total weighted loss += per tok loss active weights .sum total weight += active weights.sum return total weighted loss / total weight.clamp min=1.0 One epoch over our stratified priority pool 1,314 sequences, oversampling flagship diagnostic gates, rulesets, tactics, and lineage finishes in 10.1 minutes , bringing completion loss down to 1.7644 1.4705 validation loss . Stage 6b: Zero-Extra-VRAM PEFT DPO + +0.15 NLL Anchor SFT teaches the model how to write like a coach . Direct Preference Optimization DPO teaches it what mistakes never to make . We built 680 hard-negative preference pairs chosen vs. rejected across five categories: 1. Diagnostic Go/No-Go Gates e.g., plantarflexion vs. dorsiflexion . 2. Kinematic Vectors e.g., driving the knee cut down-and-back vs. forward-and-out . 3. Competition Rulesets & Tactical Strategy e.g., ADCC vs. IBJJF heel hook/slam rules; choosing a chest-to-chest body lock pass over a loose toreando pass against a flexible guard player . 4. Lineage & New Taxonomy Equivalence half-butterfly vs. butterfly-guard , saddle vs. inside sankaku . 5. Multi-Excerpt RAFT Anti-Distractor Pairs chosen cites 3–4 oracle excerpts across 3 paragraphs; rejected cites a distractor excerpt . Standard DPO trainers load a second frozen 7B reference model into GPU memory. When you are training a PEFT LoRA adapter, you don’t need a second model at all—just wrap the reference forward pass in with model.disable adapter : and add a +0.15 chosen nll anchor so DPO sharpens decision boundaries without hurting prose fluency: Exact reference log-probabilities with zero extra VRAMwith model.disable adapter , torch.no grad , torch.cuda.amp.autocast dtype=compute dtype : ref c logp, = compute completion logprob and nll model, c ids, c lbls ref r logp, = compute completion logprob and nll model, r ids, r lbls with torch.cuda.amp.autocast dtype=compute dtype : pol c logp, c nll = compute completion logprob and nll model, c ids, c lbls pol r logp, = compute completion logprob and nll model, r ids, r lbls logits diff = beta pol c logp - pol r logp - ref c logp - ref r logp dpo loss = -F.logsigmoid logits diff .mean loss = dpo loss + 0.15 c nll / grad accum Running DPO beta=0.1, lr=5e-6 over 532 balanced pairs takes 8.8 minutes on a single GPU and reaches a 97.2% Preference Margin Win Rate dpo loss = 0.0297 . Step 7: Verify with a Domain Benchmark & Deploy with Volume-Backed Serverless Never ship a model on vibes. Build a deterministic domain test suite with explicit rubrics must mention domain concepts, penalize forbidden myths, concrete lever counts, and held-out multi-excerpt RAFT citation checks . At inference time, we also pair the model with a lightweight, reference-free Verifiable Reward RLVR & Kinematic Consensus Filter that checks tag consensus across the 6 retrieved excerpts so an outlier distractor chunk can never be cited. Here is how the finished recipe maxdemarzi/bjj-coach-7b-v3 scores across all 250 questions of : | Category Questions | Kinematic Precision 1–5 | Actionability Score 1–5 | Hallucination Pass Rate | |---|---|---|---| | Nomenclature & Lineage 50 | 5.00 / 5.0 | 5.00 / 5.0 | 100.0% | | Diagnostic Go/No-Go Gates 75 | 5.00 / 5.0 | 5.00 / 5.0 | 100.0% | | Kinematic Troubleshooting 75 | 5.00 / 5.0 | 4.92 / 5.0 | 100.0% | | Ruleset & Tactical Strategy 50 | 5.00 / 5.0 | 4.78 / 5.0 | 100.0% | | Overall BJJ-Bench-250 | 5.00 / 5.0 | 4.93 / 5.0 | 100.0% | And on our 50 Held-Out 6-Excerpt RAFT Prompts 4 oracle excerpts + 2 hard distractors per prompt : - 3-to-4 Paragraph Multi-Instructor Synthesis Rate : 100.0% - Multi-Oracle Citation Rate ≥2≥2 Oracles Cited : 100.0% - Distractor Citation Hallucination Rate : 0.0% Production Tip: Volume-Backed Serverless Deployment Once training finishes, merge the LoRA adapter into the 16-bit base model PeftModel.from pretrained ... .merge and unload and push the 16-bit safetensors to Hugging Face maxdemarzi/bjj-coach-7b-v3 . To serve it on RunPod https://www.runpod.io/ Serverless runpod/worker-v1-vllm:v2.30.1 with QUANTIZATION=bitsandbytes without paying for an idle 24/7 GPU—and without waiting 3 minutes on cold start for a worker to download 15 GB of weights from Hugging Face—attach a persistent RunPod Network Volume mounted at /runpod-volume and point both Hugging Face and vLLM caches to the volume: MODEL NAME=maxdemarzi/bjj-coach-7b-v3QUANTIZATION=bitsandbytesLOAD FORMAT=bitsandbytesBASE PATH=/runpod-volumeHF HOME=/runpod-volume/huggingface-cache/hubHUGGINGFACE HUB CACHE=/runpod-volume/huggingface-cache/hubVLLM CACHE ROOT=/runpod-volume/vllm-cache After the first warmup request populates /runpod-volume , every subsequent serverless worker boots straight from local NVMe storage in seconds.