📄 Recurrent Knowledge Identification and Fusion for Language Model Continual Learning

Summary

This paper tackles continual learning for large language models, where a model must absorb tasks one after another without losing earlier skills (catastrophic forgetting) while still benefiting from links between tasks (knowledge transfer). Earlier merging approaches decide which parameters matter using importance scores computed once per task. As training moves the weights, those fixed scores become less reliable. The authors propose Recurrent-KIF, inspired by the Complementary Learning Systems theory of hippocampus and neocortex.

Training is split into repeated cycles with two loops:

• Inner loop: quickly adapts to the new task and flags the parameters that matter for it.
• Outer loop: re-estimates importance for earlier tasks from a small replay buffer, using the current weights. It then fuses the inner and outer task vectors with binary masks that prune redundant updates and merge the key ones, including shared regions.

Because fusion happens many times during training rather than once at the end, the method can use intermediate states. On two CL benchmarks, with backbones from 770M to 13B parameters, it outperformed state-of-the-art baselines.

Key points

• Model scale: tested on backbones from 770M (T5-large) to 13B parameters, including LLaMA2-7B. Tasks include IMDB, AG News, Amazon and Copa. On IMDB and AG News, results approach multi-task learning.
• Ablation on the Long Sequence Benchmark (order 1): the full model reaches OP 77.9 and BWT −3.4.
  • With static importance (−DIE), scores drop to 74.8 / −4.8.
  • Merging without knowledge identification collapses to 52.3 / −21.5.
• Alternative fusion schemes: global merging gives 72.1 / −11.2 and soft adaptive fusion gives 76.1 / −4.1. Freezing task-shared regions gives 75.8 / −4.3.
• Multi-round fusion: it beats single-step fusion, but gains level off and then decline when fusion happens too often.
• Visualizations on T5-large: large update magnitude does not imply importance, with mostly encoder parameters being critical. Importance scores for earlier tasks shift as training goes on.