AI Glossary: Letter "Q"

Quantized Low-Rank Adaptation (QLoRA) is an advanced parameter-efficient fine-tuning (PEFT) technique that runs LoRA over a base model quantized to 4-bit precision. It uses special formats like NormalFloat4 to maintain model accuracy while drastically reducing VRAM overhead.

Quantization is the process of compressing neural network parameters by reducing the numerical precision of its weights (e.g. converting 16-bit floating points to 4-bit integers), lowering VRAM requirements and accelerating inference.

Quiet-STaR (Self-Taught Reasoner) is a training methodology where language models learn to generate variable-length internal rationale tokens (hidden thoughts) at every token position in parallel, evaluated via REINFORCE based on how much the rationale improves next-token prediction accuracy.