LoRA
LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning (PEFT) technique that freezes the pre-trained model weights and injects trainable rank decomposition matrices into each layer of the Transformer architecture, reducing training VRAM requirements.
Frequently Asked Questions
Why is LoRA so popular?▼
It reduces the number of trainable parameters by up to 99%, allowing developers to fine-tune 7B or 13B models on consumer-grade GPUs.
What is a LoRA Adapter?▼
A tiny file containing the trained rank weights. These adapters can be dynamically swapped or merged onto the base model at runtime.
Quick Facts
- CategoryModel Training
- Key ApplicationResource-limited model fine-tuning, adapter model customization, and specialized behavior updates
Coverage Trend12 Weeks
Related AI Terms
LoRA Media Coverage & Intelligence
AI4SE and SE4AI Exploration: A Decade Looking Back and Forward
The March 2020 INCOSE INSIGHT special issue on AI and Systems Engineering (SE) became the most downloaded issue in the publication's history and launched a rese
After raising record $75B in IPO, Elon Musk's SpaceX pops 20% in trading debut
Updated with Friday's trading: Shares of Elon Musk's Space Exploration Technologies Corp. jumped 19% over their $135 asking price Friday morning after one of the most hotly anticipated initial public stock offerings in living memory. The stock closed at about $161, valuing the company at $2.1 trilli
How courts are coping with a flood of AI-generated lawsuits
Most days in her chambers, Judge Maritza Braswell, a federal magistrate judge in Colorado, sifts through stacks of documents written by people without a lawyer.