{"slug": "oncollm-adapting-medgemma-4b-and-27b-to-oncology-tasks-with-unsloth", "title": "OncoLLM: adapting MedGemma 4B and 27B to oncology tasks with Unsloth", "summary": "A team fine-tuned MedGemma 4B with LoRA and MedGemma 27B with QLoRA for oncology tasks using Unsloth 2026.9.12 on a single DGX Spark, publishing the report with a DOI on Zenodo and the models and dataset publicly. The team reports that the 4B adapter (r=64, alpha=16) behaved as if scaled by roughly 0.5 instead of the nominal alpha/r = 0.25: loaded with Transformers plus PEFT at nominal scale, 0 of 4 identity probes passed, while at 2x scale 4 of 4 passed, and merging with save_pretrained_merged at nominal alpha lost most of the tuning. The team states it did not identify the cause in the code and asks whether this is known behavior.", "body_md": "We fine-tuned MedGemma 4B (LoRA) and 27B (QLoRA) for oncology with Unsloth on a single DGX Spark. Report with DOI, models and dataset are public.\n\nOne finding you may want to look at (section 5): with Unsloth 2026.9.12, the 4B adapter (r=64, alpha=16) behaves as if scaled by ~0.5 instead of alpha/r = 0.25. Loaded with Transformers + PEFT at nominal scale, 0/4 identity probes pass; at 2x, 4/4. Merging with save_pretrained_merged at nominal alpha lost most of the tuning. We did not identify the cause in the code. Is this a known behavior?\n\nUnsloth is credited in the model cards. Feel free to share it if useful. Thanks for the tooling.\n\nSee the repo : [Grujowmi (ANTOINE Pierre)](https://huggingface.co/Grujowmi)\n\nSee the report : [OncoLLM: adapting MedGemma 4B and 27B to oncology tasks | Zenodo](https://zenodo.org/records/23134374)\n\nHave fun to give us a feedback", "url": "https://wpnews.pro/news/oncollm-adapting-medgemma-4b-and-27b-to-oncology-tasks-with-unsloth", "canonical_source": "https://discuss.huggingface.co/t/oncollm-adapting-medgemma-4b-and-27b-to-oncology-tasks-with-unsloth/182908#post_1", "published_at": "2026-10-05 06:40:42+00:00", "updated_at": "2026-10-05 06:41:17.257410+00:00", "lang": "en", "topics": ["large-language-models", "ai-research", "machine-learning", "ai-safety"], "entities": ["MedGemma 4B", "MedGemma 27B", "Unsloth", "DGX Spark", "Transformers", "PEFT", "Zenodo", "Grujowmi"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/oncollm-adapting-medgemma-4b-and-27b-to-oncology-tasks-with-unsloth", "markdown": "https://wpnews.pro/news/oncollm-adapting-medgemma-4b-and-27b-to-oncology-tasks-with-unsloth.md", "text": "https://wpnews.pro/news/oncollm-adapting-medgemma-4b-and-27b-to-oncology-tasks-with-unsloth.txt", "jsonld": "https://wpnews.pro/news/oncollm-adapting-medgemma-4b-and-27b-to-oncology-tasks-with-unsloth.jsonld"}}