Continual Learning via Inter-Task Synaptic Mapping

Continual learning aims to emulate the underlying trait of natural learning to learn various tasks on the fly without losing competence for what has been achieved in the past [1, 2, 3]. Ideally, already owned skill should ease the learning process of different-but-related tasks in the future. This problem appears as an extension of data stream learning [4] where a learner must not only demonstrate adaptive and evolving aptitudes to handle non-stationary environments but also possess the knowledge retention property such that it gains more intelligence as the increase of tasks being learned [5].

The underlying challenge of continual learning lies in the catastrophic forgetting problem where learning a new task catastrophically replaces old knowledge with new one thereby losing its relevance to handle old tasks. The regularization approach [5, 6] is among several well-known methods to resolve the catastrophic forgetting problem of the continual learning  [5, 6]. The key idea is to prevent important parameters from being perturbed by learning a new task. Elastic Weight Consolidation (EWC) [6] is one pioneering work in this area where the unique trait is seen in the application of Fisher information matrix to construct the parameter importance matrix while integrating the L-2 norm like regularization approach [5, 6]. The synaptic intelligence (SI) method is proposed to address expensive computation of parameter importance matrix via the Fisher information matrix [7]. The accumulation of loss for every training sample is used to inform the significance of each synaptic. This work has been extended in [8] called onlineEWC using the Laplace approximation. The learning without forgetting (LWF) approach is slightly different where it uses a joint optimization procedure between the cross entropy loss of current task and the knowledge distillation loss [9]. It is later found that the knowledge distillation loss can be replaced by the cross-entropy loss [9]. Memory Aware Synapses (MAS) is a regularization-based approach where the importance of network parameters are calculated in an unsupervised and online manner. Another prominent work is proposed in [10] where the regularization mechanism is performed in the neuron level instead of the synaptic level by measuring neuron importance and governing learning rates.

The main limitation of these approaches lies in the absence of inter-task relationship where a new task might share some commonalities to previous tasks. That is, a neuron might still accept new information or even contribute to understand the current task. Since a new task is embraced by those which do not contribute too much to the previous tasks, one should guarantee enough network capacity to handle all tasks otherwise new synapses have to be grown. Another challenge also exists in scaling the parameter importance matrix which can quickly explode in the large-scale problems thus causing the unlearning effect since it has to be accumulated across all tasks [11] unless an independent parameter importance matrix has to be created from scratch for every task.

An Inter-Task-Synaptic-Mapping (ISYANA) is proposed here to address the catastrophic forgetting problem of the continual learning. ISYANA is built upon the task-to-synapses and task-to-task mappers. The task-to-synapses module provides a relational mapping between each hidden node and all tasks determining the acceptable type of information it can absorb. The task-to-task component exhibits the inter-task relationship thus revealing the complete problem structure and the commonalities of each task. This module is capable of providing a sort of neighborhood degree of a sample to all tasks which can be linked to the task-to-synapses module. The relationship of a sample to previous tasks is assessed by checking its proximity to previous tasks. A node is frozen by reducing its learning rate provided that it has low relevance to the current task and the current task characterizes low mutual information to the old tasks. In a nutshell, the parameter importance matrix is crafted from the combination of the task-to-task and task-to-synapses relationships. The catastrophic forgetting is handled in the neuron level due to the fact where a connective weight plays little role to a learning problem unless it is combined with other weights to construct a neuron,. i.e., hierarchical structure of a network structure.

The concept-to-concept or task-to-task module is devised by the cluster guided mechanism in the deep latent space. It puts forward a meta-network crafted from an extension of the deep clustering network [12, 13]. That is, a class-specific cluster is designed in the deep latent space. Note that the clustering approach is carried out fully in the per-class manners such that learning a new task should not disturb the representation of old tasks. The concept-to-concept module applies the growing and consolidation phases. The growing phase constructs a set of clusters describing the current task based on the class label of each task. The consolidation phase is undertaken after the growing phase by calculating the mean values of the clusters to bound the network complexity where the mean values of the clusters exhibit a complete summary of already seen data points. In other words, there does not exist any requirement to store representation layer of each task which does not sustain for large-scale problems. The inter-task similarity is calculated via the KL divergence approach among all clusters between tasks.

The task-to-synapses module complements the task-to-task module where it functions as the one-to-one mapping between a neuron and a target class. It measures the mutual information of a node to each task making possible for a neuron to still accept the current concept notwithstanding that it is highly relevant to previously seen tasks. In short, a node can be shared by different but related tasks. All of which can be done in the online fashion without being prone to the issue of exploding parameter importance matrix thereby improving its feasibility for a high number of tasks. The advantage of ISYANA has been numerically validated in benchmark continual learning problems. It is compared with the prominent continual learning algorithms where it demonstrates competitive performance compared to recently published algorithms. Codes of ISYANA is made available in https://github.com/ContinualAL/ISYANAKBS.

ISYANA is derived from recent works on regularization principle for continual learning where the important parameters of old tasks are precluded from accepting the new concept to overcome the catastrophic forgetting problem. In a nutshell, the regularization-based approach works in the synaptic level formulated as a joint optimization problem as follows:

where $\alpha$ is a regularization constant while $\theta,\theta^{*}$ are the existing network parameters and the previous optimal parameters trained at $T-1$ task. That is, all network parameters are embedded to $\theta$. The key component lies in the parameter importance matrix $Z$ measuring the contribution of each parameter in the previous task. $Z$ is computed from the Fisher information matrix concept in EWC while the synaptic intelligence (SI) utilizes the combination of network gradient and parameter movement to deduce the network significance. Nonetheless, the main bottleneck of this approach is its application to the large-scale problems since the parameter importance matrix $Z_{t}$ has to be calculated for each task and combined to previous task $Z_{t-1}+Z_{t}$ when being applied to the $t+1$ task [6, 7]. This issue causes the explosion of $Z$ in long run thereby leading to the unlearning effect of old parameters and hindering for the positive forward and backward transfer mechanism. It is almost impossible to be resolved without the introduction of new parameters to handle a new task. The practical solution is with the normalization technique [5, 6]. Another solution is by purposely insert the forgetting factor $\gamma_{t}Z_{t-1}+Z_{t}$ [5, 6].

Our approach, ISYANA, offers an alternative solution where the inter-task relationship is taken into account while considering the relevance between synaptic and a task. The goal is to allow a node to learn multiple relevant tasks thereby underpinning the positive forward and backward transfer. That is, the two mappings reveal the common feature of a task which can be shared across similar tasks while retaining private information of a particular task. Furthermore, the inter-task mapping also eliminates the accumulation of parameter importance mapping thus coping with the unlearning effect with the absence of controlled forgetting mechanism [5, 6].

This section discusses the experimental procedure of our algorithm in four problems: splitMNIST [3, 17], permuttedMNIST [3], rotatedMNIST [18] and omniglot  [8]. ISYANA is compared against eight recently published algorithms: Context-dependent Gating (XDG) [19], Elastic Weight Consolidation (EWC) [6], onlineEWC [3, 8], SI [7], LWF [9], Deep Generative Replay (DGR) [20], Deep Generative Replay distillation [1, 20] and Averaged Gradient Episodic Memory (A-GEM) [21].
Simulation Protocol: the standard evaluation protocol of continual learning [5, 6] is followed here where three evaluation metrics, namely average accuracy, positive forward transfer and positive backward transfer, are applied. It not only evaluates the ISYANA’s aptitude in dealing with catastrophic forgetting, but also assesses whether the past knowledge improves the network performance in learning new tasks or the newly observed knowledge in the current task undermines its capability in coping with the previous tasks. Suppose that $S\in\Re^{T\times T}$ stands for an evaluation matrix where $S_{i,j}$ denotes the test accuracy on the task $T_{j}$ after observing completely the task $T_{i}$ while $\hat{b}$ is the accuracy vector for each task. The three evaluation metrics [1, 2, 3] are mathematically defined as follows:

where $ACC,BWT,FWT$ stand for the average accuracy, the backward transfer and the forward transfer respectively. The $BWT$ and $FWT$ complement $ACC$ if the accuracy returns the same results.
Baseline: the characteristics of eight baseline algorithms are outlined as follows:
XDG [19] uses a randomly generated number of hidden nodes to accept new tasks while leaving the remainder of nodes unchanged.
EWC [6] is a regularization approach for handling the catastrophic forgetting problem where the parameter importance matrix $Z$ is derived from the Fisher information matrix.
OnlineEWC [8] is an online variant of EWC where it utilizes the Laplace approximation rather than the posterior approximation. It retains only the latest running sum on the Fisher’s information matrix.
SI [7] is akin to EWC but offers an alternative avenue in computing the parameter importance matrix $Z$. That is, it is calculated using the accumulated gradient of network parameters to reduce the computational burden.
LWF [9] prevents the catastrophic forgetting by formulating the joint optimization problem between the cross entropy loss of current task and the knowledge distillation loss.
DGR [20] utilizes the information replay mechanism to cope with the catastrophic forgetting problem. It makes use of the generative adversarial network (GAN) principle where there exists a generator and solver for each task.
DGR+distill [1, 20] presents an extension of DGR with the knowledge distillation concept. It reduces the computational issue on DGR with the integration of generative model into the main model through the backward connection.
A-GEM [21] is an extension of Generative Episodic Memory [3] where it applies some modification of loss function of GEM.
All algorithms are configured under the same network structure and utilizes the implementation [1, 2]. In a nutshell, the hyperparameters of consolidated algorithms are exhibited in Table 6 and 7. Additional details of the experiments can be found in the Appendix.
Datasets: the characteristics of the four baseline problems are detailed in the following.
Rotated MNIST is an extension of the original MNIST problem [3, 17, 18] having non-stationary traits. The changing characteristics are induced by dynamic rotations between $[0,180]$ degrees.
Permutted MNIST presents the non-stationary version of MNIST problem by applying the fixed pixel permutation. Both permutation and rotation of the permuttedMNIST and the rotatedMNIST are unrelated for each task and represent the concept drift while having fixed target classes across tasks.
Split MNIST features the incremental class problem where the full MNIST problem is divided into 5 subsets or tasks of disjoint digits: $T_{1}\in[0,1],T_{2}\in[2,3],T_{3}\in[4,5],T_{4}\in[6,7],T_{5}\in[8,9]$.
Omniglot is a popular benchmark problem of the few-shot learning modified for the continual learning. It presents the continual learning of handwritten characters of 50 alphabets [8]. Each alphabet is considered as a separate task. Because of its size having 500 classes in total, this problem is capable of testing the scalability of continual learner.

We implement our flow based on the work [1, 2] in the python programming language and perform our experiments on a Windows platform. We adopt the original 28x28 pixel grey-scale images without pre-processing [1, 2]. For SplitMNIST, the dataset is divided into five tasks where each task is a two-way classification. For permutedMNIST and rotatedMNIST, the dataset is split into ten tasks where each task is a ten-way classification respectively. For omniglot dataset, it consists of 50 tasks where each task encompasses 10 classes [8]. It is akin to the splitMNIST problem where each task is devoted to handle the incremental class problem. We run each simulation for 10 times with different random seeds and adopt the average value as the measurable criterion to evaluate the performance of each method. Table 1, 2, 3 and 4 corresponding to Fig. 2, Fig. 3, Fig. 4 and Fig. 5 show the numerical results, i.e., the ACC, BWT, and FWT among different approaches for splitMNIST, rotatedMNIST, permutedMNIST and omniglot respectively.

Table 2 corresponding to Fig. 3 demonstrate that ISYANA offers significant performance improvement over other algorithms in the permuttedMNIST problem while achieving the highest FWT. This observation demonstrates the advantage of neuron-level plasticity control to cope with the changing data distributions. The use of synaptic-to-task mapping allows flexible activation and deactivation of a node in respect to their relevance to a task. On the other hand, the task-to-task mapping measuring the inter-task relationship is capable of supporting positive forward and backward transfer. That is, an important node of previous tasks still accepts the incoming task provided high commonalities between these tasks. On the other hand, ISYANA attains comparable numerical results to SI and A-GEM in the rotatedMNIST problem. Notwithstanding that A-GEM delivers the highest performance here, A-GEM exploits past samples for sample replay mechanism. The advantage of ISYANA for large-scale continual learning problem is demonstrated in the Omniglot problem where its performance outperforms other algorithms except A-GEM with noticeable margin. It should be interpreted carefully since A-GEM utilizes external memory for sample replay whereas ISYANA is memory-free. ISYANA also has better BWT than A-GEM in this context. This finding is supported by the fact that ISYANA supports the positive forward and backward transfers across the task. Moreover, it is perceived that the explosion $Z$ is apparent here for other methods hindering for its deployment in the large-scale setting. Note that the omniglot problem comprises 50 tasks and 500 classes in total.

ISYANA is, however, inferior to A-GEM, EWC, o-EWC and SI in the splitMNIST problem. We argue that this is as a result of the class-dependent nature of ISYANA where incremental classes are presented here. This leads to inaccurate estimation of neuron importance and inter-task relationship.

This section aims to study the effect of learning components to the overall learning performance of ISYANA. The underlying interest is to assess the inter-task nature of ISYANA where shareable information of each task is exploited to enable positive forward and backward transfer. That is, the tast-to-task mapping is deactivated such that the learning process is solely guided by the synaptic-to-task mapping. This study is carried out using all four datasets: splitMNIST, rotatesMNIST, permuttedMNIST and omniglot. Table 5 reports our results.

From Table 5, the advantage of the task-to-task module is obvious in which its absence results in performance’s deterioration. It reduces the classification rate of ISYANA by around $0.4\%-0.5\%$ in the splitMNIST problem and the rotatedMNIST problem. On the other hand, there does not exist any performance difference in the splitMNIST problem. Dramatic performance degeneration exists in the omniglot problem where ISYANA’s accuracy decreases by around $25\%$. This finding demonstrates the influence of the task-to-task component for a large-scale continual learning problem. It aligns with the fact that the task-to-task component is capable of exploiting the inter-task relationship which can be shared across important neurons.

In the continual learning domain, there exists other approaches to combat the catastrophic forgetting approach encompassing the memory-based approach and the structure-based approach. It is outlined as follows:
Memory-based Approach: the catastrophic forgetting problem is solved here by utilizing external memory of past samples to be replayed such that the catastrophic forgetting problem can be overcome. iCaRL is a representative approach utilizing the external memory in addressing the catastrophic forgetting problem [22]. It creates exemplar sets representing each class and the classification decision is performed by checking the similarity of a sample to the most similar exemplar set. Gradient Episodic Memory [3] and its extension A-GEM [21] are categorized as the memory-based approaches. GEM utilizes an episodic memory [3] storing a subset of the observed samples where the forgetting case is indicated by measuring the angle of the gradient vector and the proposed update. A-GEM modifies the loss function of GEM expediting the model update. Deep Generative Replay (DGR) [20] does not store previous samples to alleviate the catastrophic forgetting problem rather utilizes generative adversarial network (GAN) to generate pseudo-data preventing the catastrophic forgetting problem. This work is extended in [1] with the knowledge distillation and feedback connection. Since the memory-based approach utilizes the experience-replay mechanism to address the catastrophic forgetting problem, it is capable of handling the single-head continual learning scenario where other approaches cannot deliver. Nevertheless, compared to regularization-based approach, the memory-based approach is computationally prohibitive. Furthermore, choosing which samples to retain remains an open question because of the dynamic learning environments.
Structure-based Approach: another approach in the continual learning uses the structure-based approach where it freezes some components of a model while introducing a new component to embrace new task. Progressive Neural Network [23] is a pioneering work in this domain where a new column is inserted for new task while freezing the old components to avoid the catastrophic forgetting problem. Dynamically Expandable Networks (DEN) [24] is another technique in this domain where it addresses some flaws of PNN. It adopts the selective retraining approach and the splitting/duplicating strategy. Recently, the so-called learn-to-grow framework is put forward in [25]. It adopts the neural architecture search to find a network structure that best handles each task. The underlying limitation of the structure-based approach for continual learning is seen in its high computational and memory burdens.

An inter-task synaptic mapping (ISYANA) is proposed here to perform the continual learning of data streams having the dynamic and evolving characteristics. ISYANA is underpinned by two components, the task-to-task and synaptic-to-task mapping components where the task-to-task mapping is developed to study the mutual information across tasks while the synaptic-to-task mapping focuses on the mapping between a node to a task. This trait enables exploitation of shareable information across task while retaining private information thereby enhancing the forward and backward transfers. The advantage of ISYANA has been numerically validated in the benchmark problems and recently published algorithms clearly showing improved performance. Our future work targets an extension of ISYANA for purely streaming environments where there exist uncertainties for task’s boundary.

This research is financially supported by National Research Foundation, Republic of Singapore under IAFPP in the AME domain (contract no.: A19C1A0018). This work was mainly done when the first author was a research fellow in NTU.