# What is GRPO? Definition & Technical Mechanics

> **Source:** [SPIDITS AI AI Glossary](https://spidits.com/ai-glossary)  
> **Category:** Model Training  
> **Canonical Citation:** [https://spidits.com/ai-glossary/grpo](https://spidits.com/ai-glossary/grpo)

## Definition
Group Relative Policy Optimization (GRPO) is a parameter-efficient reinforcement learning algorithm used to align language models. Rather than relying on a separate reward model, GRPO evaluates model responses relative to a group of generated answers, reducing GPU overhead.

## Architectural Mechanics & Deep Dive
Group Relative Policy Optimization (GRPO) is a reinforcement learning algorithm designed to align language models with human preferences without the extreme memory overhead of standard methods. Unlike PPO, which requires training and hosting a separate critic model to score states, GRPO samples a group of candidate outputs for a prompt and evaluates their rewards relative to the group average. This relative reward signal optimizes the model policy directly, saving substantial GPU memory.

## Industry Applications & Real-World Use Cases
Reasoning model alignment, RLHF pipeline scaling, and math/logic model training

## Frequently Asked Questions

### What is the difference between GRPO and PPO?
Proximal Policy Optimization (PPO) requires training a separate critic/reward model to score outputs. GRPO calculates relative rewards within a group of outputs, saving significant memory.

### What is GRPO (Group Relative Policy Optimization) and how does it optimize reasoning models?
GRPO is a reinforcement learning algorithm that evaluates responses relative to a group of answers. It eliminates the need for a separate critic/reward model, saving GPU memory and enabling the scaling of reasoning steps.

## Related Terminology
- [rlhf](https://spidits.com/ai-glossary/rlhf)
- [alignment](https://spidits.com/ai-glossary/alignment)
- [dpo](https://spidits.com/ai-glossary/dpo)

---
*From the SPIDITS AI Knowledge Base (300+ terms indexed). Explore full technical definitions: [https://spidits.com/ai-glossary](https://spidits.com/ai-glossary)*