In Mixture-of-Agents, every model in a layer receives all outputs from the layer below before producing its own response; capability comes from the inference graph, not new training. The authors report that the stack outscored its stronger single-model comparator on AlpacaEval 2.0. As a Compositional Intelligence case, it puts attention on benchmarked arrangements of heterogeneous models while keeping the claim bounded to the reported comparison.
Models are stacked in layers. Every agent in a layer sees all the outputs of the layer beneath it as extra context, then writes its own response.
The observation behind it is that a model produces better answers when shown other models' attempts, including attempts weaker than what it would write unaided.
A stack of open-source models assembled this way outperforms a single stronger model on AlpacaEval 2.0. No fine-tuning is involved, only arrangement at inference time.