Research / Compositional Intelligence

Evolutionary Optimization of Model Merging Recipes

Demonstrates evolutionary discovery across both parameter and data-flow composition spaces.

Model merging usually depends on someone with good intuition picking what to combine. This runs an evolutionary search over the recipe instead, in two spaces at once.

Parameter space asks which weights to mix and how. Data-flow space asks which layers to keep, from which model, in what order. Searching both together finds combinations that neither space reaches alone.

The method produced a Japanese-language math model and a Japanese vision-language model, in both cases by rearranging existing open models rather than by training a new one.

Read the original