A unified framework for neural network recombination that achieves state-of-the-art fine-tuning with fewer than 200 parameters.
March 31, 2026
Original Paper
Decompose, Mix, Adapt: A Unified Framework for Parameter-Efficient Neural Network Recombination and Compression
arXiv · 2603.27383
The Takeaway
It allows practitioners to perform model compression and parameter-efficient fine-tuning (PEFT) simultaneously, making it possible to deploy and adapt large models on extremely resource-constrained edge devices.
From the abstract
Parameter Recombination (PR) methods aim to efficiently compose the weights of a neural network for applications like Parameter-Efficient FineTuning (PEFT) and Model Compression (MC), among others. Most methods typically focus on one application of PR, which can make composing them challenging. For example, when deploying a large model you may wish to compress the model and also quickly adapt to new settings. However, PEFT methods often can still contain millions of parameters. This may be small