-
How to Regularize your LoRA Training Better
Six ways to add dropout-style noise to LoRA fine-tuning, compared across three datasets, two base models and five seeds. One of them, Drop-LoRA-Path, beats standard LoRA dropout and weight decay in every comparison.