OncoLLM: adapting MedGemma 4B and 27B to oncology tasks with Unsloth

A team fine-tuned MedGemma 4B with LoRA and MedGemma 27B with QLoRA for oncology tasks using Unsloth 2026.9.12 on a single DGX Spark, publishing the report with a DOI on Zenodo and the models and dataset publicly. The team reports that the 4B adapter (r=64, alpha=16) behaved as if scaled by roughly 0.5 instead of the nominal alpha/r = 0.25: loaded with Transformers plus PEFT at nominal scale, 0 of 4 identity probes passed, while at 2x scale 4 of 4 passed, and merging with save_pretrained_merged at nominal alpha lost most of the tuning. The team states it did not identify the cause in the code and asks whether this is known behavior.

We fine-tuned MedGemma 4B LoRA and 27B QLoRA for oncology with Unsloth on a single DGX Spark. Report with DOI, models and dataset are public. One finding you may want to look at section 5 : with Unsloth 2026.9.12, the 4B adapter r=64, alpha=16 behaves as if scaled by ~0.5 instead of alpha/r = 0.25. Loaded with Transformers + PEFT at nominal scale, 0/4 identity probes pass; at 2x, 4/4. Merging with save pretrained merged at nominal alpha lost most of the tuning. We did not identify the cause in the code. Is this a known behavior? Unsloth is credited in the model cards. Feel free to share it if useful. Thanks for the tooling. See the repo : Grujowmi ANTOINE Pierre https://huggingface.co/Grujowmi See the report : OncoLLM: adapting MedGemma 4B and 27B to oncology tasks | Zenodo https://zenodo.org/records/23134374 Have fun to give us a feedback