I-Hsien Kuo, NicholasHarandi, MehrtashFourrier, NicolasWalder, ChristianFerraro, GabrielaSuominen, Hanna2026-01-012026-01-01ORCID:/0000-0002-4195-1641/work/207330482ORCID:/0000-0003-3652-9689/work/162782447https://hdl.handle.net/1885/733799144Neural networks suffer from catastrophic forgetting and are unable to sequentially learn new tasks without guaranteed stationarity in data distribution. Continual learning could be achieved via replay { by concurrently training externally stored old data while learning a new task. However, replay becomes less effective when each past task is allocated with less memory. To overcome this difficulty, we supplemented replay mechanics with meta-learning for rapid knowledge acquisition. By employing a meta-learner, which learns a learn-ing rate per parameter per past task, we found that base learners produced strong results when less memory was available. Additionally, our approach inherited several meta-learning advantages for continual learning: it demonstrated strong robustness to continually learn under the presence of noises and yielded base learners to higher accuracy in less updates.This research was supported by the Australian Government Research Training Program (AGRTP) Scholarship. We also thank our reviewers for the constructive feedback.12enPublisher Copyright: Copyright © The authors and PMLR 2023.202185171442685