Teaching an LLM a little Jiu Jitsu

Earlier this year, my colleagues at Relational AI helped AT&T build LLM models focused on Telecom called Otel and push it to the top of the leaderboard. I wish I could tell you all about it, but I wasn’t on the team. No worries though, you can read all about on the links above. Instead I’m going to tell you about training an LLM focused on Brazilian Jiu Jitsu. The idea is to ask it all kinds of dumb beginner questions and have it provides great answers. All without getting frustrated or telling you to focus on the fundamentals before trying flying armbars. Now, I wish I could just sign up Claude to our local school, fast forward 10 years and be done, but LLMs can’t learn that way. Large Language Models need Language, they need Text.

Luckily there are thousands of “Instructionals” (video classes teaching a topic or technique) of usually high quality and thousands of YouTube videos of varying degrees of quality we can use as our starting point. While kid classes pay the bills for local schools, it turns out middle aged men are the target audience of the instructional market. I had already purchased over 50 instructionals on BJJ Fanatics, but that wasn’t nearly enough, I had to use the Jozef Chen method and got a lot more. I also got some competition match clips that were narrated from youtube and downloaded them using yt-dlp.

Everyone is short on attention these days, so this is the AI:DR version for you to replicate this for your own video or document library:

1. Clean phonetic transcription errors and prependContextual Retrieval headers ([Context: Author — Source | Category | Tags] ) before embedding.
2. Build an explicit Knowledge Graph mapping canonical tags, cross-school synonyms, parent-child tag hierarchy, and mutual-exclusivity rules.
3. Audit video keyframes with CLIP + a Vision-Language Model so silent physical demonstrations become searchable[Visual Demonstration] chunks.
4. Mine timestamped links from your existing docs to fine-tune anONNX Cross-Encoder Re-Ranker (bge-reranker-base ) with 5-way listwise loss.
5. Synthesize a 3-Track SFT dataset (6-excerpt RAFT with distractors + diagnostic Go/No-Go gates + Knowledge Graph lineage/rules) and gate it with an automated**<0.35% 7-gram linter** .
6. Run 2.5x Decision-Weighted QLoRA SFT followed by Zero-Extra-VRAM PEFT DPO (with model.disable_adapter(): +0.15 NLL anchor) on a single GPU.
7. Verify against a rubric-scored domain benchmark and deploy onvolume-backed vLLM serverless .

Suppose you have a massive library of specialized domain knowledge—in our case, about 30,000 instructional video segments, 14,000 competition match clips, and 79 written technical guides on Brazilian Jiu-Jitsu. Your domain might be surgical procedures, aviation maintenance, industrial robotics, or tax law, but the problem is the same.

When a user asks a question, you want an AI assistant that:

1. Speaks the real language of the domain (knowing when five different terms from different schools mean the exact same thing).
2. Retrieves the exact moments that matter across both spoken explanations and silent visual demonstrations.
3. Writes a clear, multi-paragraph answer with inline[1] –[6] citations that jump the video player to the exact timestamp, while ignoring near-miss search results.
4. Runs fast and cheap on a 7-billion-parameter open-weights model without frontier API bills.

If you just wire up basic vector search to an off-the-shelf 7B model, you will get vague fortune-cookie advice, hallucinated rules, and backwards physical mechanics (like telling a student to spin the wrong way out of a heel hook).

We are going to walk through the complete, step-by-step recipe we used to build and taking our domain benchmark (BJJ-Bench-250) to 5.00 / 5.0 on Kinematic Precision with a 0.0% distractor citation rate.

Let’s jump in.

The Ingredients (Architecture Overview) #

Before we start cooking, here is what goes into the system:

You only need a single GPU (an NVIDIA L40S, A40, or A100 on RunPod for ~30 minutes) to train both the cross-encoder re-ranker and the 7B language model.

Step 1: Clean the “Slightly Wrong” Data and Add Contextual Headers #

Real-world transcripts are messy. Instructors are out of breath on a mat, using domain loanwords from Japanese and Portuguese. Speech-to-text models like Whisper hear “kimura” as "camera", “De La Riva” as "dealer river", “Danaher” as "donna hair", “ashi garami” as "oshi garami", and “inside sankaku” as "inside sinkhole".

1a. Repair Phonetic Mishearings First

Don’t waste model capacity teaching an LLM to guess phonetic typos if you can fix them deterministically at ingest time. Build a domain replacement dictionary and run it over your raw transcripts before you embed or train on them:

const PHONETIC_REPAIRS = [  [/\bcamera\b/gi, 'kimura'],  [/\bdealer river\b/gi, 'de la riva'],  [/\bdonna (?:her|hair)\b/gi, 'Danaher'],  [/\boshi garami\b/gi, 'ashi garami'],  [/\binside sinkhole\b/gi, 'inside sankaku'],  [/\bhonor roll\b/gi, 'honey hole'],  [/\bMarcelo team\b/gi, 'marcelotine'],  [/\b(?:uma|homo) plata\b/gi, 'omoplata'],];

1b. Prepend Contextual Retrieval Headers Before Embedding

A 60-second video transcript window is full of pronouns: “When he pushes here, I move my foot there and lock this up.” By itself, that chunk has no idea what position it is in.

Before computing dense embeddings (BAAI/bge-small-en-v1.5) or full-text search vectors, prepend a structured Contextual Retrieval Header to every chunk:

[BJJ Context: Gordon Ryan — Systematically Attacking The Guard (NOGI) | Chapter: Half Butterfly Sweeps | Tags: half-butterfly, butterfly-hook, half-guard]<spoken transcript body>

Now every single chunk carries the instructor, course title, ruleset/attire (GI vs. NOGI), chapter, and canonical tags right inside its vector representation.

Step 2: Build a Domain Knowledge Graph #

Vector similarity is great at fuzzy semantics, but it is terrible at strict domain logic. Before training your model, encode your domain’s taxonomy into an explicit Knowledge Graph.

Our graph has three jobs:

1. Cross-Lineage Synonym Equivalence (153 canonical tags, 388 synonyms) : Different schools name the same mechanic differently.Inside Sankaku ,The Saddle ,4/11 ,Honey Hole , andTop Rock all point to the canonicalsaddle node. When a user asks about"honey hole" , the graph expands the search query across all synonyms and passes a lineage note into the LLM prompt so it knows they are the exact same position.
2. Broad-to-Specific Tag Pruning : If a video segment or competition clip is tagged with a specific guard (half-butterfly orde-la-riva-guard ) or a specific leg entanglement (saddle oroutside-ashi ), the graph automatically suppresses broad parent tags (open-guard ,ashi-garami ) during auto-tagging. That keeps Inverse Document Frequency (IDF) tag weights sharp during hybrid search.
3. Kinematic Mutual Exclusivity (scoreKinematicCompatibility) : Two athletes cannot be in closed-guard and in thesaddle at the same time. Encoding mutually exclusive positions in the graph lets our re-ranker apply a deterministic-0.25 penalty to near-miss search results that start from a physically incompatible position, and a-0.20 penalty when[GI] vs.[NOGI] attire mismatches.

Step 3: Give Your Retriever “Eyes” (Multimodal Visual Grounding) #

In any video-heavy domain, experts show things with their hands that they never say out loud: “Look at my grip right here—if I angle my knee like this, he can’t flatten me.” If you only index audio transcripts, those physical details are invisible.

Here is how to turn silent visual demonstrations into searchable text chunks:

1. Extract Candidate Keyframes : Pull video frames at the timestamps where your guides or chapters reference a demonstration.
2. Filter Talking Heads with Contrastive CLIP Scoring : Run a 512-dim CLIP vision model (Qdrant/clip-ViT-B-32-vision ) over every frame and score it against positive domain prompts (“two grapplers actively demonstrating a jiu-jitsu technique on the mat” ) minus negative prompts (“a static title card or a single person talking to the camera” ).
3. Caption the Physical Mechanics with a Vision-Language Model : For verified demonstration keyframes (861 in our library), fuse the spoken quote, the guide’s biomechanical claim, and aVideo-VLM (Qwen2.5-VL-7B) kinematic description of the grips, hooks, and limb angles visible on screen.
4. Index into chunk_embeddings : Insert those descriptions into your vector table as[Visual Demonstration — <Topic>] ... chunks pinned to the exact demonstration timestamp (timestamp_sec ).

Now, when a user asks how a position is set up, hybrid search retrieves both the spoken explanation and the [Visual Demonstration] chunk.

Step 4: Train a Custom Cross-Encoder Re-Ranker from Internal Citations #

A bi-encoder embeds the user’s question and the video chunk separately into two 384-dimensional vectors and takes their dot product. That makes it fast enough to scan 265,000 chunks in PostgreSQL via HNSW, but it compresses an entire 60-second video transcript into a single vector. Subtle mechanical differences (“defending the knee cut from bottom half guard” vs. “finishing the knee cut from top headquarters”) blur together and that’s a real problem.

First-stage hybrid retrieval (combining pgvector HNSW dense search, PostgreSQL tsvector BM25 search, and IDF-weighted tag matching via Reciprocal Rank Fusion) gives you a fast pool of about 30 candidate moments. To pick the best 6 moments for the LLM, you want a cross-encoder re-ranker that reads (question, candidate_passage) together with full bidirectional attention.

Off-the-shelf re-rankers (BAAI/bge-reranker-base

Mine Your Existing Documentation Links

Our 79 markdown guides contained about 3500 timestamped video links (/watch/<segment_uuid>?t=<seconds>). Every link is a verified ground-truth pair connecting a technical claim to an exact video chunk!

We combined those guide citations with our [Visual Demonstration] chunks and newly tagged taxonomy chunks to build about 4000 training triplets, where each row contains:

• query : The technical question or biomechanical claim from the guide section.
• positive : The exact transcript or visual-demo chunk at timestamp t (with its Contextual Retrieval header).
• hard_negatives (4 per query) :
  1. Two chunks from different segments in the same instructional volume (same instructor, same general topic, wrong specific detail).
  2. One chunk from the same guide topic with high lexical word overlap.
  3. One cross-instructional chunk sharing a canonical tag.
• Two chunks from

Fine-Tune with 5-Way Listwise Loss and Export to ONNX

We fine-tuned BAAI/bge-reranker-base for 3 epochs (lr=2e-5, ~2.5 minutes on a single GPU) using a combined 5-way listwise Cross-Entropy + pairwise margin loss, then exported the model directly to ONNX so it runs locally inside our embedding microservice with zero GPU overhead:

A cross-encoder (BAAI/bge-reranker-base, a 278M-parameter RoBERTa model) takes a (query, passage) pair concatenated together—<s> query </s></s> passage </s>—and outputs a single scalar logit s(q,p) representing relevance.

If you train a re-ranker with simple binary cross-entropy (scoring one passage at a time as 0 or 1), the model never directly compares the gold chunk against its near-miss sibling chunks from the same volume. Instead, for every query in a batch, we interleave 5 passages ([pos, neg1, neg2, neg3, neg4]), run them through the cross-encoder, reshape the output logits to (batch_size, 5) where column 0 is the gold positive and columns 1..4 are the four hard negatives, and optimize two losses together:

1. 5-Way Listwise InfoNCE (Temperature-Scaled Cross-Entropy) : Treats the 5 passages as a 5-class classification problem where class0 is the winner, scaled by temperature :
2. Pairwise Margin Ranking Loss (margin = 1.25) : Forces the positive logit to beatevery individual hard negative by at least1.25 logit units so scores stay well-separated when we later combine them with Knowledge Graph compatibility adjustments:

Here is the core training loop:

for step, (qs, pos_list, neg1_list, neg2_list, neg3_list, neg4_list) in enumerate(, 1):    bsz = len(qs)    # Interleave [pos, neg1, neg2, neg3, neg4] per query -> (bsz * 5) pairs    flat_q, flat_p = [], []    for i in range(bsz):        flat_q.extend([qs[i]] * 5)        flat_p.extend([pos_list[i], neg1_list[i], neg2_list[i], neg3_list[i], neg4_list[i]])    enc = tokenizer(flat_q, flat_p, padding=True, truncation=True, max_length=384, return_tensors="pt").to(device)    optimizer.zero_grad()    with torch.cuda.amp.autocast(enabled=use_amp):        logits = model(**enc).logits.view(bsz, 5)  # col 0 = positive, cols 1..4 = hard negatives        targets = torch.zeros(bsz, dtype=torch.long, device=device)        # 1. 5-Way Listwise InfoNCE Loss (temperature = 0.7)        loss_ce = ce_loss_fn(logits / 0.7, targets)        # 2. Pairwise Margin Ranking Loss (margin = 1.25)        margins = [torch.clamp(1.25 - (logits[:, 0] - logits[:, j]), min=0.0).mean() for j in range(1, 5)]        loss = loss_ce + 0.25 * sum(margins)    scaler.scale(loss).backward()    scaler.step(optimizer)    scaler.update()    scheduler.step()

Exporting to ONNX for Zero-GPU Production Serving

In production, you don’t want to spin up a PyTorch GPU worker just to re-rank 20 candidate chunks. By exporting the fine-tuned cross-encoder to an ONNX graph with dynamic batch_size and sequence axes, our lightweight Python embedding service runs it on CPU via onnxruntime + Rust tokenizers in about 40ms per query:

And at request time, OnnxCrossEncoder loads model.onnx and tokenizer.json once at boot with zero PyTorch dependency:

class OnnxCrossEncoder:    def __init__(self, model_dir: Path):        self.tokenizer = Tokenizer.from_file(str(model_dir / "tokenizer.json"))        self.tokenizer.enable_truncation(max_length=384)        self.tokenizer.enable_padding(pad_id=1, pad_token="<pad>")        self.session = ort.InferenceSession(            str(model_dir / "model.onnx"),            providers=["CPUExecutionProvider"],        )    def rerank(self, query: str, documents: list[str]) -> list[float]:        encodings = self.tokenizer.encode_batch([(query, doc) for doc in documents])        feed = {            "input_ids": np.array([e.ids for e in encodings], dtype=np.int64),            "attention_mask": np.array([e.attention_mask for e in encodings], dtype=np.int64),        }        logits = self.session.run(["logits"], feed)[0]        return [float(x) for x in logits.reshape(-1).tolist()]

Fine-tuning for 3 epochs (lr=2e-5, ~2.5 minutes on a single GPU) jumped Recall@1 by +10.0% and MRR@10 to 0.8749:

Re-Ranker Model (250 Held-Out Queries, 10 Candidates Each)	MRR@10	Recall@1	Recall@3	Mean Gold Rank
Off-the-shelf BAAI/bge-reranker-base	0.8224	68.0%	96.0%	1.48
Fine-Tuned bjj-reranker-v3 (ONNX)	0.8749	78.0% (+10.0% )	97.6%	1.37

Step 5: Build a Three-Track SFT Dataset (and Lint It Automatically) #

Now let’s prepare the training data for the 7B coach model.

If you want a model that works both as a RAG synthesizer (when video excerpts are provided) and as a direct domain expert (answering biomechanics, lineage, and ruleset questions), build a Three-Track ChatML Dataset (about 3,000 conversations):

Track A: Multi-Document RAFT — Retrieval-Augmented Fine-Tuning (1,200 conversations)

1,200 conversations) In production, /ask sends 6 retrieved excerpts ([1]–[6]) from multiple instructors. Usually 4 of those excerpts directly answer the question (“oracles”) and 2 are near-miss neighbors (“distractors”).

If you only train your model on clean, single-excerpt prompts, it will either summarize [1] and ignore [2]–[6], or hallucinate citations to the distractors.

Using RAFT, every Track A training prompt packs 4 oracle chunks + 2 hard distractor chunks shuffled into [1]–[6]. The assistant completion is a 3-to-4 paragraph synthesis that weaves the multiple instructors together, places [1]–[6] citations at the end of supported sentences, and never cites the distractor indices.

Track B: Deep Biomechanics & Diagnostic “Go / No-Go” Gates (1,680 conversations)

Real domain expertise is built around diagnostic physical checks—binary tests that tell you whether to advance a technique or bail out:

• Outside Heel Hook Escape :“Can you point your toes (plantarflexion)?” If yes, you still have rotational slack—clear your knee line. If your foot is pinned in dorsiflexion, spinning will tear your knee—tap immediately.
• Knee Cut Pass :“Can your heel touch your backside?” If yes, the space behind your knee wedge is closed—slice down-and-back toward their hip fulcrum.
• Half-Butterfly vs. Butterfly Guard : Distinguishinghalf-butterfly (butterfly half guard , where your bottom leg traps one of the passer’s legs while your inside leg inserts a singlebutterfly-hook wedge) frombutterfly-guard (double butterfly , seated open guard with both hooks inside).

Make sure the concrete mechanical levers (elbow, knee, hip, frame, wedge, fulcrum, vector, angle, wrist, heel, hook, post) appear in the first paragraph of your training answers so the model leads with actionable mechanics rather than throat-clearing preamble.

Track C: Combinatorial Knowledge Graph Lineage & Rulesets (210 conversations)

Generate Q&A pairs directly from your Knowledge Graph and competition rulebooks (IBJJF, ADCC, EBI) covering every synonym family, belt-level legality rule (heel hooks, knee reaping, slams, jumping guard, sleeve cuffs), and tactical match strategy.

Gate Your Dataset with an Automated Linter & Critic

Never launch a training run without linting the dataset first. Our linter enforces three hard gates:

1. The <0.35% 7-Gram Diversity Gate : Counts every 7-word sequence across all 3,090 assistant responses. If any 7-word phrase repeats in more than0.35% of responses, the linter rejects the dataset so the LoRA adapter can never memorize a repetitive boilerplate suffix.
2. The Biomechanical & Role-Inversion Critic : Scans every response for domain contradictions (e.g., equating pointing toes with dorsiflexion, telling the top passer to tap, or claiming two synonymous positions are different).
3. Citation & Lever Density Gate : Verifies that Track A responses cite only oracle indices and that responses average8.7+ concrete anatomical levers .

Step 6: Train the 7B Model — Decision-Weighted QLoRA SFT + Zero-Extra-VRAM DPO #

With our 3,090-sample SFT dataset and 680-pair DPO dataset ready, training Qwen/Qwen2.5-7B-Instruct happens in two fast stages on a single GPU.

Stage 6a: QLoRA SFT with 2.5x Decision-Token Weighting

We load Qwen/Qwen2.5-7B-Instruct in 4-bit NF4 and attach a LoRA adapter () across all 7 linear projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj).

Two implementation details make a huge difference here:

1. Contiguous Completion-Only Loss Masking with 2.5x Decision Weights : We mask the system and user prompt tokens (labels = -100 ) so cross-entropy loss is only computed on the assistant’s coaching response. Within that response, we apply a**2.5x loss weight** to high-leverage decision tokens: citation brackets ([1] –[6] ), diagnostic gates (plantarflexion ,dorsiflexion ,tap ), and ruleset/strategy terms (IBJJF ,ADCC ,EBI ,legal ,illegal ,reaping ,strategy ,position ,lineage ).
2. Slice Active Tokens Before Softmax (>15x Less VRAM) : BecauseQwen2.5 has a large vocabulary (152,064 tokens), computingCrossEntropyLoss(reduction="none") across a full padded batch allocates gigabytes of intermediate logits for prompt and padding tokens that just get masked out anyway. Slicing only the unmasked assistant tokens (mask = seq_labels != -100 ) per sequence before callingcross_entropy is mathematically identical and cuts logit VRAM by more than15x :

def compute_weighted_causal_lm_loss(logits, labels, token_weights):    loss_fct = torch.nn.CrossEntropyLoss(reduction="none")    total_weighted_loss = logits.new_zeros((), dtype=torch.float32)    total_weight = logits.new_zeros((), dtype=torch.float32)    for i in range(logits.size(0)):        seq_labels = labels[i, 1:]        mask = seq_labels != -100        if not mask.any():            continue        seq_logits = logits[i, :-1, :][mask]        active_labels = seq_labels[mask]        active_weights = token_weights[i, 1:][mask].float()        per_tok_loss = loss_fct(seq_logits.float(), active_labels)        total_weighted_loss += (per_tok_loss * active_weights).sum()        total_weight += active_weights.sum()    return total_weighted_loss / total_weight.clamp(min=1.0)

One epoch over our stratified priority pool (1,314 sequences, oversampling flagship diagnostic gates, rulesets, tactics, and lineage) finishes in 10.1 minutes, bringing completion loss down to 1.7644 (1.4705 validation loss).

Stage 6b: Zero-Extra-VRAM PEFT DPO + +0.15 NLL Anchor

SFT teaches the model how to write like a coach. Direct Preference Optimization (DPO) teaches it what mistakes never to make.

We built 680 hard-negative preference pairs (chosen vs. rejected) across five categories:

1. Diagnostic Go/No-Go Gates (e.g., plantarflexion vs. dorsiflexion).
2. Kinematic Vectors (e.g., driving the knee cut down-and-back vs.forward-and-out ).
3. Competition Rulesets & Tactical Strategy (e.g., ADCC vs. IBJJF heel hook/slam rules; choosing a chest-to-chest body lock pass over a loose toreando pass against a flexible guard player).
4. Lineage & New Taxonomy Equivalence (half-butterfly vs.butterfly-guard ,saddle vs.inside sankaku ).
5. Multi-Excerpt RAFT Anti-Distractor Pairs (chosen cites 3–4 oracle excerpts across 3 paragraphs;rejected cites a distractor excerpt).

Standard DPO trainers load a second frozen 7B reference model into GPU memory. When you are training a PEFT LoRA adapter, you don’t need a second model at all—just wrap the reference forward pass in with model.disable_adapter(): and add a +0.15 * chosen_nll anchor so DPO sharpens decision boundaries without hurting prose fluency:

Running DPO (beta=0.1, lr=5e-6) over 532 balanced pairs takes 8.8 minutes on a single GPU and reaches a 97.2% Preference Margin Win Rate (dpo_loss = 0.0297).

Step 7: Verify with a Domain Benchmark & Deploy with Volume-Backed Serverless #

Never ship a model on vibes. Build a deterministic domain test suite with explicit rubrics (must_mention domain concepts, penalize forbidden myths, concrete lever counts, and held-out multi-excerpt RAFT citation checks).

At inference time, we also pair the model with a lightweight, reference-free Verifiable Reward (RLVR) & Kinematic Consensus Filter that checks tag consensus across the 6 retrieved excerpts so an outlier distractor chunk can never be cited.

Here is how the finished recipe (maxdemarzi/bjj-coach-7b-v3) scores across all 250 questions of :

Category (Questions)	Kinematic Precision (1–5)	Actionability Score (1–5)	Hallucination Pass Rate
Nomenclature & Lineage (50)	5.00 / 5.0	5.00 / 5.0	100.0%
Diagnostic Go/No-Go Gates (75)	5.00 / 5.0	5.00 / 5.0	100.0%
Kinematic Troubleshooting (75)	5.00 / 5.0	4.92 / 5.0	100.0%
Ruleset & Tactical Strategy (50)	5.00 / 5.0	4.78 / 5.0	100.0%
Overall (BJJ-Bench-250)	5.00 / 5.0	4.93 / 5.0	100.0%

And on our 50 Held-Out 6-Excerpt RAFT Prompts (4 oracle excerpts + 2 hard distractors per prompt):

• 3-to-4 Paragraph Multi-Instructor Synthesis Rate :100.0%
• Multi-Oracle Citation Rate (≥2≥2 Oracles Cited) :100.0%
• Distractor Citation Hallucination Rate :0.0%

Production Tip: Volume-Backed Serverless Deployment

Once training finishes, merge the LoRA adapter into the 16-bit base model (PeftModel.from_pretrained(...).merge_and_unload()) and push the 16-bit safetensors to Hugging Face ( maxdemarzi/bjj-coach-7b-v3).

To serve it on RunPod Serverless (runpod/worker-v1-vllm:v2.30.1 with QUANTIZATION=bitsandbytes) without paying for an idle 24/7 GPU—and without waiting 3 minutes on cold start for a worker to download 15 GB of weights from Hugging Face—attach a persistent RunPod Network Volume mounted at /runpod-volume and point both Hugging Face and vLLM caches to the volume:

MODEL_NAME=maxdemarzi/bjj-coach-7b-v3QUANTIZATION=bitsandbytesLOAD_FORMAT=bitsandbytesBASE_PATH=/runpod-volumeHF_HOME=/runpod-volume/huggingface-cache/hubHUGGINGFACE_HUB_CACHE=/runpod-volume/huggingface-cache/hubVLLM_CACHE_ROOT=/runpod-volume/vllm-cache

After the first warmup request populates /runpod-volume, every subsequent serverless worker boots straight from local NVMe storage in seconds.

source & further reading

maxdemarzi.com — original article How I use Claude Eating our own tails You’re working like a caveman she said.