LoRA & DoRA in TinyGrad

From-scratch implementations of LoRA (Low-Rank Adaptation) and DoRA (Weight-Decomposed LoRA) in TinyGrad, showing how injecting low-rank adapters into linear layers reduces trainable parameter count while maintaining fine-tuning performance.

🔗 GitHub