{"slug": "fine-tuning-microsoft-harrier-oss-v1-270m-with-sentencetransformertrainer-is-it", "title": "Fine-tuning microsoft/harrier-oss-v1-270m with SentenceTransformerTrainer — is it supported?", "summary": "Microsoft's harrier-oss-v1-270m sentence-transformers model can be fine-tuned with SentenceTransformerTrainer and MultipleNegativesRankingLoss for Portuguese QA retrieval, though no public example combines that exact stack, according to a technical analysis of the model card. The model card states Harrier uses a decoder-only architecture, last-token pooling and L2-normalized embeddings, and that query instructions are part of training so omitting them degrades performance while document-side instructions are not needed. The recommended setup applies the instruction prefix to the query/anchor side during training and keeps documents unprompted to avoid a train/inference mismatch.", "body_md": "Seems practically supported?\n\n## `microsoft/harrier-oss-v1-270m` with `SentenceTransformerTrainer`\n\nYes — **this should be a supported and reasonable setup**, with one important caveat: I have not found a public example that exactly combines:\n\n```\nmicrosoft/harrier-oss-v1-270m\n+ SentenceTransformerTrainer\n+ MultipleNegativesRankingLoss\n+ Portuguese QA retrieval\n```\n\nHowever, the evidence strongly points to this being a valid path:\n\n- [`microsoft/harrier-oss-v1-270m`](https://huggingface.co/microsoft/harrier-oss-v1-270m) is packaged as a`sentence-transformers` model and can be loaded with`SentenceTransformer(\"microsoft/harrier-oss-v1-270m\")` .\n- The model card explicitly shows Sentence Transformers usage and encodes **queries with a prompt** while encoding**documents without a prompt** .\n- The model card says Harrier uses a **decoder-only architecture** ,**last-token pooling** , and**L2-normalized embeddings** .\n- The model card also says query instructions are how the model is trained and that omitting them can degrade performance; document-side instructions are not needed.\n- [`SentenceTransformerTrainingArguments`](https://sbert.net/docs/package_reference/sentence_transformer/training_args.html) supports training-time`prompts` , including column-specific prompt mappings.\n- [`MultipleNegativesRankingLoss`](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiple-negativesrankingloss) is the standard Sentence Transformers loss for positive`(query, document)` /`(anchor, positive)` retrieval pairs.\n- Nearby public examples exist, especially a Harrier-family Vietnamese legal retrieval model using Sentence Transformers + MNRL, and SkillRet-style decoder-embedding fine-tuning using query instructions and unprompted documents.\n\nMy recommendation is:\n\n```\ntraining query / anchor:      Instruct: ...\\nQuery: <Portuguese question>\ntraining document / positive: <raw Portuguese passage>\n\ninference query:              Instruct: ...\\nQuery: <Portuguese question>\ninference document:           <raw Portuguese passage>\n```\n\nSo: **apply the instruction prefix to the query/anchor side during training, not only at inference.** Keep documents/passages unprompted.\n\n## \n\nYes. For Harrier, the instruction is not just an inference-time decoration. It is part of the expected query-side input format.\n\nThe [Harrier-270M model card](https://huggingface.co/microsoft/harrier-oss-v1-270m) shows this Sentence Transformers pattern:\n\n```\nquery_embeddings = model.encode(queries, prompt_name=\"web_search_query\")\ndocument_embeddings = model.encode(documents)\n```\n\nThe same model card also shows the raw Transformers pattern:\n\n``` php\ndef get_detailed_instruct(task_description: str, query: str) -> str:\n    return f\"Instruct: {task_description}\\nQuery: {query}\"\n\n# Each query must come with a one-sentence instruction that describes the task.\n# No need to add instruction for retrieval documents.\n```\n\nThe FAQ is especially relevant: it says query instructions are how the model is trained, omitting them causes degradation, and document-side instructions are not needed.\n\nSo for Portuguese QA retrieval, I would train with:\n\n```\nquery / anchor:\nInstruct: Given a Portuguese question, retrieve relevant Portuguese passages that answer the question\nQuery: <question>\n\ndocument / positive:\n<passage>\n```\n\nand infer with exactly the same policy:\n\n```\nquery:\nInstruct: Given a Portuguese question, retrieve relevant Portuguese passages that answer the question\nQuery: <question>\n\ndocument:\n<passage>\n```\n\nThis avoids a train/inference mismatch.\n\n### \n\n```\ntraining query:\nQual é o prazo para interpor recurso administrativo?\n\ninference query:\nInstruct: Given a Portuguese question, retrieve relevant Portuguese passages that answer the question\nQuery: Qual é o prazo para interpor recurso administrativo?\n```\n\nThis fine-tunes the model on raw questions but deploys it on prompted questions. For Harrier, that is probably the wrong distribution.\n\n### \n\n```\ntraining query:\nInstruct: Given a Portuguese question, retrieve relevant Portuguese passages that answer the question\nQuery: Qual é o prazo para interpor recurso administrativo?\n\ninference query:\nInstruct: Given a Portuguese question, retrieve relevant Portuguese passages that answer the question\nQuery: Qual é o prazo para interpor recurso administrativo?\n```\n\nDocuments should remain raw in both training and inference:\n\n```\ntraining document:\nO prazo para interposição de recurso administrativo é de 10 dias úteis...\n\nindexed document:\nO prazo para interposição de recurso administrativo é de 10 dias úteis...\n```\n\n## `SentenceTransformerTrainer`\n\nIf your dataset columns are named `query` and `document`, use column-specific prompts:\n\n``` python\nfrom sentence_transformers import (\n    SentenceTransformer,\n    SentenceTransformerTrainer,\n    SentenceTransformerTrainingArguments,\n    losses,\n)\nfrom sentence_transformers.training_args import BatchSamplers\n\nmodel = SentenceTransformer(\n    \"microsoft/harrier-oss-v1-270m\",\n    model_kwargs={\"dtype\": \"auto\"},\n)\n\nquery_prompt = (\n    \"Instruct: Given a Portuguese question, \"\n    \"retrieve relevant Portuguese passages that answer the question\\n\"\n    \"Query: \"\n)\n\nargs = SentenceTransformerTrainingArguments(\n    output_dir=\"harrier-270m-pt-qa-mnrl\",\n\n    per_device_train_batch_size=8,\n    gradient_accumulation_steps=16,  # effective batch size 128 on 1 GPU\n\n    learning_rate=5e-6,\n    num_train_epochs=1,\n    warmup_ratio=0.10,\n    lr_scheduler_type=\"cosine\",\n\n    bf16=True,\n    gradient_checkpointing=True,\n\n    batch_sampler=BatchSamplers.NO_DUPLICATES,\n\n    prompts={\n        \"query\": query_prompt,\n        \"document\": \"\",\n    },\n\n    logging_steps=50,\n    save_strategy=\"steps\",\n    save_steps=500,\n    save_total_limit=2,\n)\n\nloss = losses.MultipleNegativesRankingLoss(\n    model,\n    directions=(\"query_to_doc\",),\n)\n\ntrainer = SentenceTransformerTrainer(\n    model=model,\n    args=args,\n    train_dataset=train_dataset,  # columns: query, document\n    loss=loss,\n)\n\ntrainer.train()\ntrainer.save_model(\"harrier-270m-pt-qa-mnrl/final\")\n```\n\nIf your dataset columns are named `anchor` and `positive`, change only the prompt mapping:\n\n```\nprompts={\n    \"anchor\": query_prompt,\n    \"positive\": \"\",\n}\n```\n\nThe important rule is simple:\n\n```\nquery-like column:    prompt\ndocument-like column: no prompt\n```\n\n## \n\nI would start with the English instruction format because it matches Harrier’s public prompt style:\n\n```\nInstruct: Given a Portuguese question, retrieve relevant Portuguese passages that answer the question\nQuery:\n```\n\nThe query and document texts themselves should remain Portuguese.\n\nAfter you have a baseline, test a Portuguese instruction as an ablation:\n\n```\nInstruct: Dada uma pergunta em português, recupere passagens em português relevantes que respondam à pergunta\nQuery:\n```\n\nI would not start by translating the structural markers to `Instrução:` and `Consulta:`. Keep `Instruct:` and `Query:` first, because that matches the Harrier format shown in the model card and [`config_sentence_transformers.json`](https://huggingface.co/microsoft/harrier-oss-v1-270m/blob/main/config_sentence_transformers.json).\n\nRecommended first prompt:\n\n```\nquery_prompt = (\n    \"Instruct: Given a Portuguese question, \"\n    \"retrieve relevant Portuguese passages that answer the question\\n\"\n    \"Query: \"\n)\n```\n\n## `MultipleNegativesRankingLoss` appropriate?\n\nYes. For QA retrieval, your data usually has the form:\n\n```\nquery:    Portuguese question\npositive: passage that answers the question\n```\n\nThat is a natural fit for [`MultipleNegativesRankingLoss`](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiple-negativesrankingloss), which is designed for positive pairs such as `(query, response)` or `(anchor, positive)`.\n\nA basic version is:\n\n```\nloss = losses.MultipleNegativesRankingLoss(model)\n```\n\nFor clarity in retrieval, I would write:\n\n```\nloss = losses.MultipleNegativesRankingLoss(\n    model,\n    directions=(\"query_to_doc\",),\n)\n```\n\nThis trains the model so that each query is closer to its matching passage than to other passages in the batch.\n\n## \n\nMNRL uses other positives in the batch as negatives. That is efficient, but it can be harmful if some of those “negatives” are actually relevant.\n\nExample:\n\n```\nquery:\nComo solicitar a segunda via da fatura?\n\npositive A:\nA segunda via da fatura pode ser solicitada no portal do cliente.\n\npositive B for another query:\nPara emitir uma cópia da fatura, acesse Minha Conta e clique em Segunda Via.\n```\n\nFor the first query, positive B is not really negative. It probably answers the same question. If it appears in the same batch, MNRL may incorrectly push it away.\n\nThis is common in QA retrieval, FAQ retrieval, legal retrieval, policy retrieval, support retrieval, and any corpus with repeated answer templates.\n\nUse:\n\n```\nbatch_sampler=BatchSamplers.NO_DUPLICATES\n```\n\nThe [Sentence Transformers training overview](https://sbert.net/docs/sentence_transformer/training_overview.html) specifically notes that losses using in-batch negatives benefit from no duplicate samples in a batch. The [loss docs](https://sbert.net/docs/package_reference/sentence_transformer/losses.html) also discuss cached / larger-batch variants of MNRL.\n\nAlso deduplicate aggressively before training:\n\n- exact duplicate passages;\n- near-duplicate chunks;\n- boilerplate-heavy passages;\n- repeated FAQ answers;\n- multiple chunks from the same source document;\n- multiple positives that answer the same query.\n\n## \n\nHarrier and BGE-M3 should not be treated as interchangeable SBERT-style encoders.\n\n### \n\n[`microsoft/harrier-oss-v1-270m`](https://huggingface.co/microsoft/harrier-oss-v1-270m) is:\n\n- decoder-only;\n- multilingual;\n- 270M parameters;\n- 640-dimensional embeddings;\n- up to 32,768 tokens;\n- last-token pooled;\n- L2-normalized;\n- instruction-sensitive on the query side.\n\nWhen used through `SentenceTransformer`, last-token pooling and normalization are handled automatically.\n\nIf using raw `AutoModel`, you must reproduce the model-card pooling behavior yourself:\n\n``` python\ndef last_token_pool(last_hidden_states, attention_mask):\n    left_padding = attention_mask[:, -1].sum() == attention_mask.shape[0]\n    if left_padding:\n        return last_hidden_states[:, -1]\n    sequence_lengths = attention_mask.sum(dim=1) - 1\n    batch_size = last_hidden_states.shape[0]\n    return last_hidden_states[torch.arange(batch_size), sequence_lengths]\n```\n\nFor this use case, I would stay with `SentenceTransformer` unless there is a strong reason not to.\n\n### \n\n[`BAAI/bge-m3`](https://huggingface.co/BAAI/bge-m3) is not just a dense embedding model. Its model card describes it as multi-functional, multilingual, and multi-granular:\n\n- dense retrieval;\n- sparse retrieval;\n- multi-vector retrieval;\n- more than 100 languages;\n- up to 8192 tokens.\n\nThis matters for a fair comparison. Do not compare:\n\n```\nBGE-M3 hybrid/sparse/multi-vector system\nvs\nHarrier dense-only system\n```\n\nand call that a model-only comparison.\n\nFairer comparisons are:\n\n```\nBGE-M3 dense vs Harrier dense\nBGE-M3 hybrid vs Harrier dense + BM25\nBGE-M3 + reranker vs Harrier + reranker\n```\n\n## \n\nFor a first full-model Harrier-270M MNRL run, I would start conservatively.\n\n| Parameter | Recommended first value | \n| Base model | `microsoft/harrier-oss-v1-270m` | \n| Loss | `MultipleNegativesRankingLoss` | \n| Direction | `(\"query_to_doc\",)` | \n| Query prompt | yes | \n| Document prompt | no | \n| Learning rate | `5e-6` | \n| LR candidates | `3e-6` ,`5e-6` ,`1e-5` | \n| Epochs | `1` | \n| Warmup ratio | `0.10` | \n| Scheduler | `cosine` | \n| Precision | `bf16` if supported | \n| Physical batch size | `4–16` , depending on GPU | \n| Effective batch size | `128–256` | \n| Batch sampler | `BatchSamplers.NO_DUPLICATES` | \n| Gradient checkpointing | yes if memory-bound | \n| Max sequence length | `512` or`1024` first | \n\n I would not start with `5e-5` for full-model MNRL fine-tuning. Harrier is already a strong embedding model; the goal is adaptation, not overwriting its embedding geometry.\n\nA useful nearby reference is [`mainguyen9/vietlegal-harrier-0.6b`](https://huggingface.co/mainguyen9/vietlegal-harrier-0.6b), a Harrier-family Vietnamese legal retrieval model that reports Sentence Transformers training, MNRL, hard-negative mining, LR `3e-6`, batch size `256`, one epoch, warmup `10%`, cosine scheduler, and bf16. It is not the same model size or language, but it is a closer reference than generic BERT/SBERT defaults.\n\n## `CachedMultipleNegativesRankingLoss`?\n\nUse it if your GPU memory prevents a useful effective batch size.\n\nMNRL benefits from larger batches because larger batches provide more in-batch negatives. If normal MNRL is memory-bound, try:\n\n```\nloss = losses.CachedMultipleNegativesRankingLoss(\n    model,\n    mini_batch_size=32,\n)\n```\n\nThen test effective batch sizes like:\n\n```\n256\n512\n1024\n```\n\nBut I would not make cached MNRL the first experiment. First establish that the simple MNRL setup works.\n\n## \n\nRun these in order.\n\n| Run | Model | Training | Query prompt | Doc prompt | LR | Effective batch | Purpose | \n| A | BGE-M3 | existing fine-tune | current | current | current | current | incumbent baseline | \n| B | Harrier-270M | none | yes | no | — | — | zero-shot baseline | \n| C | Harrier-270M | MNRL | yes | no | `5e-6` | 128 | main first run | \n| D | Harrier-270M | MNRL | yes | no | `3e-6` | 128–256 | lower-LR check | \n| E | Harrier-270M | MNRL | yes | no | `1e-5` | 128–256 | upper-LR check | \n| F | Harrier-270M | MNRL | no | no | `5e-6` | 128 | prompt ablation | \n| G | Harrier-270M | Cached MNRL | yes | no | `5e-6` | 256–1024 | batch-size check | \n| H | Harrier-270M | hard-negative stage | yes | no | `3e-6–5e-6` | task-dependent | ranking refinement | \n\n The most important comparison is:\n\n```\nfine-tuned BGE-M3\nvs\nzero-shot Harrier with query instruction\nvs\nfine-tuned Harrier with query instruction\nvs\nfine-tuned Harrier without query instruction\n```\n\nLeaderboard scores are useful for model shortlisting, but the final decision should be based on your own Portuguese QA retrieval benchmark.\n\nFor broader benchmark context, see [MTEB](https://huggingface.co/spaces/mteb/leaderboard), [MMTEB](https://arxiv.org/abs/2502.13595), and the original [MTEB paper](https://arxiv.org/abs/2210.07316). MTEB-style scores are useful, but they do not replace task-specific evaluation.\n\n## \n\nUse the same evaluation pipeline for BGE-M3 and Harrier.\n\nMinimum retrieval metrics:\n\n```\nnDCG@10\nMRR@10\nRecall@5\nRecall@10\nRecall@50\nRecall@100\n```\n\nWhy these metrics matter:\n\n| Metric | What it tells you | \n| `Recall@50` /`Recall@100` | Whether the retriever can put the right passage somewhere in the candidate pool | \n| `Recall@5` /`Recall@10` | Whether the retriever is good enough for direct RAG context selection | \n| `MRR@10` | Whether the first relevant passage appears early | \n| `nDCG@10` | Ranking quality when there are multiple relevant passages | \n\n Also track operational metrics:\n\n```\nembedding throughput\nquery latency\nindex size\nGPU memory\nembedding dimension\nchunk length\nmax sequence length\n```\n\nFor Portuguese-specific external sanity checks, useful resources include:\n\n## \n\nDo not start with hard negatives. Start with clean query-positive MNRL.\n\nAfter the first baseline is stable:\n\n```\n1. Embed the full corpus.\n2. Retrieve top 100 candidates per training query.\n3. Remove known positives.\n4. Skip the top few candidates if they may be unlabeled positives.\n5. Sample negatives from ranks 20–100 or 50–100.\n6. Train a second stage with explicit negatives or a hard-negative-aware setup.\n```\n\nThe reason to avoid the top retrieved “negative” is that it may actually be a valid answer that was not labeled.\n\nThe [SkillRet paper](https://arxiv.org/html/2605.05726v1) is a useful related reference. It fine-tunes decoder-style embedding models using `MultipleNegativesRankingLoss`, applies the same task-specific query instruction to anchor queries during training, uses no document prompt for Harrier/Qwen-style embedding models, and mines hard negatives for the reranker stage. It also reports that fine-tuning Harrier-OSS-0.6B and Qwen3-Embedding-0.6B gives nearly identical performance in that task, suggesting that the training recipe matters at least as much as the exact decoder-embedding base.\n\n## \n\n### \n\nBad:\n\n```\ndataset query already contains:\nInstruct: ...\nQuery: ...\n\nand TrainingArguments also uses:\nprompts={\"query\": \"Instruct: ...\\nQuery: \"}\n```\n\nThis produces:\n\n```\nInstruct: ...\nQuery: Instruct: ...\nQuery: <question>\n```\n\nUse one method:\n\n```\nEither store raw queries and use prompts=...\nor store prompted queries and do not use prompts=...\n```\n\nI recommend storing raw queries and using `prompts=...`.\n\n### \n\nBad:\n\n```\ndocument:\nInstruct: Given a Portuguese question, retrieve relevant Portuguese passages that answer the question\nQuery: <passage>\n```\n\nFor Harrier retrieval, documents should be raw passages.\n\n### \n\nBad:\n\n```\ntraining query:  raw question\ninference query: prompted question\n```\n\nBetter:\n\n```\ntraining query:  prompted question\ninference query: prompted question\n```\n\n### \n\nBad:\n\n```\nBGE-M3 hybrid vs Harrier dense-only\n```\n\nBetter:\n\n```\nBGE-M3 dense vs Harrier dense\nBGE-M3 hybrid vs Harrier dense + BM25\nBGE-M3 + reranker vs Harrier + reranker\n```\n\n### \n\nHarrier supports long context, but that does not mean a first fine-tune should use 32k tokens.\n\nStart with:\n\n```\n512 or 1024 tokens\n```\n\nThen test:\n\n```\n2048\n4096\n8192\n```\n\nonly if your evaluation set shows that longer passages help.\n\nIn retrieval, better chunking is often more useful than simply increasing max length.\n\n## \n\nUse the same query prompt and raw documents:\n\n``` python\nfrom sentence_transformers import SentenceTransformer\n\nmodel = SentenceTransformer(\"harrier-270m-pt-qa-mnrl/final\")\n\nquery_prompt = (\n    \"Instruct: Given a Portuguese question, \"\n    \"retrieve relevant Portuguese passages that answer the question\\n\"\n    \"Query: \"\n)\n\nqueries = [\n    \"Qual é o prazo para interpor recurso administrativo?\",\n]\n\ndocuments = [\n    \"O prazo para interposição de recurso administrativo é de 10 dias úteis...\",\n    \"A segunda via da fatura pode ser solicitada no portal do cliente...\",\n]\n\nquery_embeddings = model.encode(\n    queries,\n    prompt=query_prompt,\n    normalize_embeddings=True,\n)\n\ndocument_embeddings = model.encode(\n    documents,\n    normalize_embeddings=True,\n)\n\nscores = query_embeddings @ document_embeddings.T\nprint(scores)\n```\n\nAvoid passing both `prompt` and `prompt_name` unless you intentionally want one to override the other. A related [Qwen3 Embedding discussion](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B/discussions/5) notes that explicit `prompt` takes priority over `prompt_name` in Sentence Transformers-style usage.\n\n## \n\nFor this Portuguese QA retrieval use case, I would proceed like this:\n\n1. Keep your fine-tuned BGE-M3 model as the incumbent baseline.\n2. Evaluate Harrier-270M zero-shot with the correct query instruction and raw documents.\n3. Fine-tune Harrier with MNRL using query-side instruction during training.\n4. Do not prompt documents.\n5. Start with `lr=5e-6` , one epoch, warmup`0.10` , cosine scheduler, bf16, effective batch size around`128` .\n6. Run LR ablations at `3e-6` ,`5e-6` , and`1e-5` .\n7. Use `BatchSamplers.NO_DUPLICATES` .\n8. Deduplicate query/document pairs aggressively.\n9. Try `CachedMultipleNegativesRankingLoss` if memory prevents larger effective batches.\n10. Add hard negatives only after the clean first-stage baseline works.\n11. Compare systems fairly: dense vs dense, hybrid vs hybrid, reranked vs reranked.\n12. Decide based on your own held-out Portuguese QA retrieval set, not only Multilingual MTEB v2.\n\n### \n\n- **Supported?** Yes, practically. Harrier-270M is a Sentence Transformers model and should work with`SentenceTransformerTrainer` .\n- **Exact public recipe?** I have not found an exact Harrier-270M + STTrainer + MNRL + Portuguese QA recipe.\n- **Instruction during training?** Yes. Apply it to the query/anchor side during training and inference.\n- **Documents?** Keep documents/passages unprompted.\n- **Loss?**`MultipleNegativesRankingLoss` is appropriate for`(query, positive passage)` pairs.\n- **Main risks?** Prompt mismatch, false negatives, duplicates, too-high LR, too-small effective batch, and incorrect pooling if using raw`AutoModel` .\n- **Starting hyperparameters?**`lr=3e-6` to`1e-5` , one epoch, warmup`0.10` , cosine scheduler, bf16, effective batch`128–256` ,`BatchSamplers.NO_DUPLICATES` .\n- **Best next experiment?** Harrier zero-shot prompted vs Harrier MNRL prompted vs Harrier no-prompt ablation vs your fine-tuned BGE-M3 baseline.", "url": "https://wpnews.pro/news/fine-tuning-microsoft-harrier-oss-v1-270m-with-sentencetransformertrainer-is-it", "canonical_source": "https://discuss.huggingface.co/t/fine-tuning-microsoft-harrier-oss-v1-270m-with-sentencetransformertrainer-is-it-supported/175947#post_3", "published_at": "2026-09-24 19:00:09+00:00", "updated_at": "2026-09-24 19:02:33.734357+00:00", "lang": "en", "topics": ["natural-language-processing", "machine-learning", "ai-research"], "entities": ["Microsoft", "harrier-oss-v1-270m", "SentenceTransformerTrainer", "MultipleNegativesRankingLoss", "Sentence Transformers", "Hugging Face"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/fine-tuning-microsoft-harrier-oss-v1-270m-with-sentencetransformertrainer-is-it", "markdown": "https://wpnews.pro/news/fine-tuning-microsoft-harrier-oss-v1-270m-with-sentencetransformertrainer-is-it.md", "text": "https://wpnews.pro/news/fine-tuning-microsoft-harrier-oss-v1-270m-with-sentencetransformertrainer-is-it.txt", "jsonld": "https://wpnews.pro/news/fine-tuning-microsoft-harrier-oss-v1-270m-with-sentencetransformertrainer-is-it.jsonld"}}