# What is Quantization? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Model Operations  
> **Canonical Citation:** [https://spidits.com/ai-glossary/quantization](https://spidits.com/ai-glossary/quantization)

## Definition
Quantization is the process of compressing neural network parameters by reducing the numerical precision of its weights (e.g. converting 16-bit floating points to 4-bit integers), lowering VRAM requirements and accelerating inference.

## Architectural Mechanics & Deep Dive
Quantization is a model optimization technique that reduces the numerical precision of model weights (e.g., converting 32-bit floating-point numbers into 8-bit or 4-bit integers). This significantly reduces the model's memory footprint and accelerates inference speed, enabling large models to run on local hardware with minimal accuracy loss.

## Industry Applications & Real-World Use Cases
Mobile AI execution, local model hosting (GGUF, AWQ), and consumer GPU model loading

## Frequently Asked Questions

### Does quantization degrade model performance?
It can cause minor degradation in accuracy, but modern quantization algorithms (like GPTQ or AWQ) minimize this loss while reducing model size by up to 75%.

### What is the difference between GGUF, AWQ, and GPTQ quantization frameworks?
GGUF is designed for CPU and unified Apple Silicon RAM execution, AWQ (Activation-aware Weight Quantization) protects critical activation channels for GPU inference, and GPTQ performs post-training quantization optimized for high-speed Nvidia VRAM loading.

### What are GGUF and EXL2?
Lightweight quantized file formats optimized to run LLMs locally on CPUs/Macs (GGUF) or GPUs (EXL2).

## Related Terminology
- [models](https://spidits.com/ai-glossary/models)
- [llm](https://spidits.com/ai-glossary/llm)
- [gpu](https://spidits.com/ai-glossary/gpu)

---
*From the SPIDITS AI Knowledge Base (336+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*