{"slug": "lora-vs-dora-why-dora-underperformed-in-my-tests", "title": "LoRA vs DoRA: Why DoRA Underperformed in My Tests", "summary": "In a head-to-head test, standard LoRA outperformed Weight-decomposed Low-Rank Adaptation (DoRA) by 2.1% in accuracy while taking 15% less training time per epoch, according to a developer's experiments on a consumer-grade GPU. The author found DoRA's additional magnitude component added computational overhead and caused more volatile loss curves, making standard LoRA the safer choice for smaller, niche datasets.", "body_md": "# LoRA vs DoRA: Why DoRA Underperformed in My Tests\n\nWeight-decomposed Low-Rank Adaptation (DoRA) is marketed as a way to bridge the gap between LoRA and full fine-tuning, but in my recent head-to-head test, it actually lagged behind standard LoRA in both training speed and final accuracy.\n\nI kept the hyperparameters identical for both runs to ensure a fair comparison:\n\nThe issue became apparent during the first two epochs. DoRA introduces an additional magnitude component that requires more computation per update. While the overhead seems negligible on paper, it added a noticeable drag to my iterations.\n\nFor this specific project, the added complexity of DoRA wasn't worth the trade-off. It seems the \"better learning capacity\" of weight decomposition doesn't always trigger on smaller, niche datasets where standard LoRA is already highly efficient. If you're constrained by hardware, sticking to a basic LoRA AI workflow is likely the safer bet for faster deployment.\n\nI ran these experiments on a consumer-grade GPU to see if the theoretical benefits of decomposing magnitude and direction actually translate to real-world performance on smaller datasets.\n\n## The Setup\n\nI kept the hyperparameters identical for both runs to ensure a fair comparison:\n\n**Rank (r):** 8**Alpha:** 16**Learning Rate:** 2e-4**Epochs:** 3\n\n## The Diagnosis\n\nThe issue became apparent during the first two epochs. DoRA introduces an additional magnitude component that requires more computation per update. While the overhead seems negligible on paper, it added a noticeable drag to my iterations.\n\nMore frustrating was the convergence. I noticed the loss curve for DoRA was significantly more volatile than LoRA's. Instead of the smoother descent I expected, DoRA struggled to stabilize, leading to a lower validation score.\n\n**Training Time:** DoRA took ~15% longer per epoch.**Accuracy:** LoRA outperformed DoRA by 2.1% on my specific evaluation set.**VRAM Usage:** Roughly the same, though DoRA's overhead slightly increased peak memory.\n\n## Conclusion\n\nFor this specific project, the added complexity of DoRA wasn't worth the trade-off. It seems the \"better learning capacity\" of weight decomposition doesn't always trigger on smaller, niche datasets where standard LoRA is already highly efficient. If you're constrained by hardware, sticking to a basic LoRA AI workflow is likely the safer bet for faster deployment.\n\n[Next Train-Test Split: A Practical Guide for Data Science →](/en/threads/3867/)\n\n## All Replies （3）\n\nJ\n\nI've noticed DoRA takes way longer to converge on smaller datasets compared to standard LoRA.\n\n0\n\nL\n\nP\n\nDid you try adjusting the learning rate or sticking with the default settings for both?\n\n0", "url": "https://wpnews.pro/news/lora-vs-dora-why-dora-underperformed-in-my-tests", "canonical_source": "https://promptcube3.com/en/threads/3877/", "published_at": "2026-07-26 22:47:00+00:00", "updated_at": "2026-07-26 23:08:40.249049+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "ai-tools"], "entities": ["LoRA", "DoRA"], "alternates": {"html": "https://wpnews.pro/news/lora-vs-dora-why-dora-underperformed-in-my-tests", "markdown": "https://wpnews.pro/news/lora-vs-dora-why-dora-underperformed-in-my-tests.md", "text": "https://wpnews.pro/news/lora-vs-dora-why-dora-underperformed-in-my-tests.txt", "jsonld": "https://wpnews.pro/news/lora-vs-dora-why-dora-underperformed-in-my-tests.jsonld"}}