# What is Double Quantization? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Model Operations  
> **Canonical Citation:** [https://spidits.com/ai-glossary/double-quantization](https://spidits.com/ai-glossary/double-quantization)

## Definition
Double Quantization is a memory-saving process introduced in QLoRA that quantizes the quantization constants themselves, reducing the memory footprint of fine-tuning runs with zero accuracy loss.

## Architectural Mechanics & Deep Dive
Double quantization is an advanced model optimization technique pioneered in QLoRA to reduce memory overhead. It quantizes the quantization constants themselves, converting 32-bit floating-point scales into 8-bit representations. This nested quantization step saves a significant amount of active memory when loading massive Large Language Models onto consumer-grade GPUs, without degrading the model's accuracy.

## Industry Applications & Real-World Use Cases
Parameter-efficient fine-tuning, VRAM budget optimization, and local model updates.

## Frequently Asked Questions

### How does double quantization save memory?
It compresses the block-level quantization scale factors from 32-bit floats to 8-bit floats, saving around 0.37 bits per parameter on average.

### Which PEFT method uses double quantization?
QLoRA, which utilizes this method to fine-tune 70B parameter models on a single 48GB GPU.

## Related Terminology
- [quantization](https://spidits.com/ai-glossary/quantization)
- [qlora](https://spidits.com/ai-glossary/qlora)
- [peft](https://spidits.com/ai-glossary/peft)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*