OncoLLM: adapting MedGemma 4B and 27B to oncology tasks with Unsloth

We fine-tuned MedGemma 4B (LoRA) and 27B (QLoRA) for oncology with Unsloth on a single DGX Spark. Report with DOI, models and dataset are public.

One finding you may want to look at (section 5): with Unsloth 2026.9.12, the 4B adapter (r=64, alpha=16) behaves as if scaled by ~0.5 instead of alpha/r = 0.25. Loaded with Transformers + PEFT at nominal scale, 0/4 identity probes pass; at 2x, 4/4. Merging with save_pretrained_merged at nominal alpha lost most of the tuning. We did not identify the cause in the code. Is this a known behavior?

Unsloth is credited in the model cards. Feel free to share it if useful. Thanks for the tooling.

See the repo : [Grujowmi (ANTOINE Pierre)](https://huggingface.co/Grujowmi)

See the report : [OncoLLM: adapting MedGemma 4B and 27B to oncology tasks | Zenodo](https://zenodo.org/records/23134374)

Have fun to give us a feedback