What is Kahneman-Tversky Optimization?

Detailed Deep Dive

Kahneman-Tversky Optimization (KTO) is an alignment algorithm introduced by Ethayarajh et al. (2024) inspired by Daniel Kahneman and Amos Tversky's Nobel-winning Prospect Theory. Unlike DPO and RLHF, which require expensive preference pairs (comparing two completions for the same prompt), KTO aligns models using unpaired binary signals—knowing only whether a single response was desirable or undesirable. By weighting loss according to how humans value gains and losses relative to a reference status quo, KTO achieves performance on par with DPO while using far easier to collect real-world user feedback.