What is Mixture of Agents?

Detailed Deep Dive

Mixture of Agents (MoA) is an collaborative LLM architecture that routes queries through a multi-layered hierarchy of generator and aggregator models. In the initial layers, several diverse models generate independent candidate responses in parallel. An aggregator model then synthesizes these intermediate inputs to generate a unified final output. This cooperative reasoning exploits the collective intelligence of individual networks, consistently achieving higher quality, correctness, and reasoning depth than any single agent acting alone.