知识库｜湖南麓川信息科技有限公司

User Tools

Site Tools

Trace:
deep_learning:incremental_learning

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
deep_learning:incremental_learning [2021/07/27 23:28]
jordan [Papers] 		deep_learning:incremental_learning [2023/03/08 16:04] (current)
xujianglong ↷ Page moved from 内部资料:deep_learning:incremental_learning to deep_learning:incremental_learning
Line 5: Line 5:
 Incremental Learning(IL) is also referred to as: Continual Learning(CL), Life Long Learning(LLL), Online Learning(OL), Never Ending Learning(NEL). Incremental Learning(IL) is also referred to as: Continual Learning(CL), Life Long Learning(LLL), Online Learning(OL), Never Ending Learning(NEL).
  
-Incremental learning aims to develop artificially intelligent systems that can continuously learn to address new tasks from new data while preserving knowledge learned from previously learned tasks.+Incremental learning aims to develop artificially intelligent systems that can continuously learn to address new tasks from new data while preserving knowledge learned from previously learned tasks.(([[https://arxiv.org/abs/2010.15277|Class-incremental learning: survey and performance evaluation on image classification]]))
  
 +===== Key Challenge =====
 +
 +Catastrophic Forgetting(CF): Learning multiple tasks in sequence, however, remains a substantial challenge for deep learning. When trained on a new task, standard neural networks forget most of the information related to previously learned tasks, a phenomenon referred to as “catastrophic forgetting”(([[https://arxiv.org/abs/1904.07734v1|Three scenarios for continual learning]])).
 +
 +===== Scenarios =====
 +
 +There are 3 scenarios on Incremental Learning(([[https://arxiv.org/abs/1904.07734v1|Three scenarios for continual learning]])), including:
 +
 +  * Task Incremental Learning
 +  * Domain Incremental Learning
 +  * Class Incremental Learning
 +
 +===== Methods =====
 +
 +All methods of incremental learning can be concluded into 4 categories(([[https://arxiv.org/abs/1904.07734v1|Three scenarios for continual learning]])).
 +
 +  * Task specific components(sub-network per task): **XDG**(Context-dependent Gating)
 +  * regularized optimization(differently regularizing parameters): **[[deep_learning:incremental_learning:EWC]]**(Elastic Weight Consolidation), **SI**(Synaptic Intelligence)
 +  * Modifying Training Data(pseudo-data, generate samples): **[[deep_learning:incremental_learning:LwF]]**(Learning without Forgetting), **DGR**(Deep Generative Replay)
 +  * Using Exemplars(store data from previous tasks): **[[deep_learning:incremental_learning:iCaRL]]**
 ===== Resources ===== ===== Resources =====
  
Line 19: Line 39:
 [[https://github.com/xialeiliu/Awesome-Incremental-Learning | Awesome Incremental Learning Papers]] [[https://github.com/xialeiliu/Awesome-Incremental-Learning | Awesome Incremental Learning Papers]]
  
-[[Incremental Learning Papers]]+[[.incremental_learning:incremental_learning_papers | Incremental Learning Papers]]
deep_learning/incremental_learning.1627399691.txt.gz · Last modified: 2021/07/27 23:28 by jordan

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki