# What is QLoRA? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Model Training  
> **Canonical Citation:** [https://spidits.com/ai-glossary/qlora](https://spidits.com/ai-glossary/qlora)

## Definition
Quantized Low-Rank Adaptation (QLoRA) is an advanced parameter-efficient fine-tuning (PEFT) technique that runs LoRA over a base model quantized to 4-bit precision. It uses special formats like NormalFloat4 to maintain model accuracy while drastically reducing VRAM overhead.

## Architectural Mechanics & Deep Dive
QLoRA (Quantized Low-Rank Adaptation) is an advanced parameter-efficient fine-tuning technique that allows fine-tuning massive models on a single GPU. It works by quantizing the base model weights to 4-bit NormalFloat (NF4) to minimize memory usage, and adding a small set of trainable 16-bit LoRA weight matrices, maintaining full fine-tuning accuracy.

## Industry Applications & Real-World Use Cases
Fine-tuning large models (e.g. 70B parameters) on consumer GPUs, edge device training, and cost-effective cloud updates.

## Frequently Asked Questions

### How does QLoRA save memory compared to standard LoRA?
Standard LoRA loads the base model in 16-bit or 8-bit. QLoRA loads it in 4-bit, compressing base weight memory by up to 75%.

### Does QLoRA degrade fine-tuning quality?
No, QLoRA introduces techniques like double quantization and page optimizers to match the accuracy of standard 16-bit fine-tuning.

## Related Terminology
- [lora](https://spidits.com/ai-glossary/lora)
- [quantization](https://spidits.com/ai-glossary/quantization)
- [peft](https://spidits.com/ai-glossary/peft)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*