Adapter Merging with Centroid Prototype Mapping
for Scalable Class-Incremental Learning

Takuma Fukuda, Hiroshi Kera, Kazuhiko Kawamoto

https://arxiv.org/abs/2412.18219

Abstract

We propose Adapter Merging with Centroid Prototype Mapping (ACMap), an exemplar-free framework for classincremental learning (CIL) that addresses both catastrophic forgetting and scalability. While existing methods tradeoff between inference time and accuracy, ACMap consolidates task-specific adapters into a single adapter, ensuring constant inference time across tasks without compromising accuracy. The framework employs adapter merging to build a shared subspace that aligns task representations and mitigates forgetting, while centroid prototype mapping maintains high accuracy through consistent adaptation in the shared subspace. To further improve scalability, an early stopping strategy limits adapter merging as tasks increase. Extensive experiments on five benchmark datasets demonstrate that ACMap matches state-ofthe-art accuracy while maintaining inference time comparable to the fastest existing methods

Acknowledgement

We extend our gratitude to the authors of the following resources for their invaluable contributions to the field of class-incremental learning, which significantly informed and inspired our research:

• Deep Class-Incremental Learning: A Survey
• PILOT: A Pre-Trained Model-Based Continual Learning Toolbox
• Revisiting Class-Incremental Learning with Pre-Trained Models: Generalizability and Adaptivity are All You Need
• Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning

Installation

This repository supports two setup methods:

• A. Container setup with Docker (recommended)
• B. Local setup with Rye

Windows is not supported, so please refer to docker/Dockerfile and pyproject.toml to set up the environment manually.

A. Container setup with Docker (recommended)

Build the Docker container:

bash docker.sh build

Create a .env file by copying .env.example. Be sure to exclude .env from version control.

cp .env.example .env

Start the container shell:

bash docker.sh shell

B. Local setup with Rye

If you haven't installed Rye yet, run:

curl -sSf https://rye.astral.sh/get | bash

You can install Python and the necessary packages:

rye sync

Dataset Preparation

Download the datasets with cmd/download.py from the source, and extract its contents:

python3 cmd/download.py --help

    Usage: download.py [OPTIONS]

    Options:
        --name TEXT     Dataset name (CUB200|ImageNet-R|ImageNet-A|VTAB)  [required]
        --out_dir TEXT  Download destination  [required]
        --help          Show this message and exit

These datasets are referenced from the APER.

• CIFAR100: will be automatically downloaded by the code.
• CUB200: Google Drive: link or Onedrive: link
• ImageNet-R: Google Drive: link or Onedrive: link
• ImageNet-A: Google Drive: link or Onedrive: link
• VTAB: Google Drive: link or Onedrive: link

https://github.com/zhoudw-zdw/RevisitingCIL

Training

To start training, run the following command:

python3 src/acmap/train.py --config exps/cifar.yaml --dataset_dir ./dataset

The adapter checkpoints for each task will be saved in ./data/acmap/ckpts/. To skip saving checkpoints, include the --debug option.

To adjust the initial number of classes (init_cls) and the incremental step size (increment), run:

python3 src/acmap/train.py --init_cls 20 --increment 20

For more details, refer to src/acmap/utils/options.py.

(Optional) Logging with W&B

We support logging with W&B for tracking experiments and visualizing results. To enable this, obtain an API key from W&B and add it to .env, then run the following command:

python3 src/acmap/train.py --logger wandb