Unsupervised Competitive Hardware Learning Rule for Spintronic Clustering Architecture

Neuromorphic computing promises exceptional capabilities for artificial intelligence through highly-efficient circuit structures mimicking the structure and functionality of the human brain. While neural networks composed of artificial synapses and neurons have been demonstrated with conventional CMOS devices, it is expected that computing efficiency can be improved by more closely emulating neurobiological hardware. Thus, the potential use of non-volatile spintronic analog devices that can be interconnected in a manner that enables on-chip unsupervised learning is particularly promising. In particular, spintronic domain-wall (DW) devices have been proposed as capable synapses [1] and neurons [2] and achieve a remarkable energy efficiency and reliability relative to competing non-volatile candidate devices such as resitive/filamentary memristive and phase-change memory (PCM) candidate devices for on-chip learning [3]. Recently, DW synaptic devices have been co-integrated with transistor learning circuits to show basic learning [4].

Domain wall-magnetic tunnel junction devices (DW-MTJs) are spintronic devices that provide non-volatile memory. The position of the DW along its track determines the tunneling resistance of a DW-MTJ and can be modulated by applying a current through the track. Three-terminal DW-MTJs have been leveraged as analog artificial synapses with long tunnel barriers and as binary artificial neurons by a digital version with shorter tunnel barrriers [5]. Furthermore, a CMOS-free monolothic multi-layer perceptron system can be achieved solely with three-terminal DW-MTJ synapses and four-terminal DW-MTJ neurons intrinsically capable of emulating the leaky-integrate-and-fire (LIF) neuron model. However, as this perceptron system does not utilize the third terminal of the DW-MTJ synapses, there is an opportunity to use this third terminal for efficient on-chip unsupervised learning. We propose such an approach to clustering using DW-MTJ crossbars that minimize the use of CMOS.

A neuromorphic crossbar array is a hardware organization of circuit units that can perform brain-inspired functionalities, and has its origins in the use of crossbars for logic and memory. Like the crossbar array, the biological brain can be simplified and abstracted as a neural network consisting of neurons connected by synapses.

Neurons receive electrical or chemical stimuli through the dendrites, perform computations in the cell body (i.e., soma), and propagate the output through axons. Synapses are the junctions between neurons that modulate the signal propagation strength. The leaky integrate-and-fire (LIF) neuron is a popular artificial neuron model that mimics the biological neuron. An LIF neuron continuously integrates energy provided through the synapses connected to the input neuron.Throughout the integration process, the stored energy leaks if the energy received by the neuron is insufficient. When the stored energy reaches a particular threshold, the LIF neuron generates a spike that can be propagated to the next connected neuron or be utilized to perform inference.

The synaptic connections of a neuromorphic crossbar can be implemented with variable resistors and many such crossbar architectures have been proposed using memristors [6], 1-transistor 1-resistor RRAM [7], phase-change memories [8], and ferroelectric RAM [9], among others. These variable resistors define the weighted connections between neurons. The weight tuning is performed by changing the conductance of the synapses and is referred to as ‘learning’ when the tuning is conducive to the optimization of some objective function for which a cognitive task is defined (i.e. face recognition).

Fig. 1 shows a $5\times 5$ neuromorphic crossbar array where horizontal word lines are connected to the input neurons and vertical bit lines are connected to the output neurons. At the intersection of each word line and bit line, a synapse is placed that defines the connectivity between the input and the output neuron. In this fashion, a weight matrix is formed by the synapses, and the output neurons receive the product of vector-matrix multiplication between an input vector and the weight matrix. Cascading multiple crossbar array results in a multi-layer perceptron.

Fig. 2(a) shows a three-terminal (3T) DW-MTJ that acts as a synapse by providing an analog conductance element to the neural system [10]. Between terminals S1 and S2, it has a free ferromagnetic domain wall track divided into +z and -z directed magnetic domains separated by a DW. An analog MTJ is formed between the S1 and S3 terminals by the DW track, tunnel barrier and, the -z directed fixed ferromagnet. The position of the DW can be controlled by applying the S2-S1 spin-transfer torque inducing write current. The MTJ conductance can be represented as a parallel-resistance combination of parallel and anti-parallel (with respect to the fixed ferromagnet) portions of the DW-MTJ. A positive (negative) S2-S1 current grows the anti-parallel (parallel) +z (-z) domain and decreases (increases) the conductance of the DW-MTJ.

Fig. 2(b) shows a four-terminal (4T) version of DW-MTJ which performs the leaky integrate-and-fire neuronal functionalities [11]. Similar to the synapse, it has a DW track between terminals N1 and N2. The magnetic state of the DW track is coupled to a free ferromagnet of smaller dimension. The electrically-insulating magnetic coupling layer separates the N1-N2 write current path from the N3-N4 read path. The N3-N4 path has an MTJ of smaller dimension compared to the synapse MTJ. Spin-transfer torque inducing current through the N1-N2 path performs integration by moving the DW from right-to-left direction. Leaking causes the DW to move in left-to-right direction and can be performed by one of three approaches - dipolar coupling field [5], anisotropy gradient along the DW track [12] and shape variation of the DW track [13]. When the DW crosses beneath the dipolarly-coupled free ferromagnet during the integration process, the MTJ between the N3-N4 path switches from an anti-parallel low conductive state to a parallel high conductive state, thus performing the firing functionality.

The process of updating weights in a neural network is called learning. Depending on the presence of a teacher signal, learning can be classified into the categories of supervised, unsupervised and semi-supervised methods. In our hardware system, we have focused on optimizing the unsupervised learning primitive. This primitive can later be co-integrated with other learning sub-systems, as demonstrated in Section VIII. Our approach implements unsupervised learning as an electrical approximation of Hebbian (associative) learning, which can be efficiently implemented via the spike-timing-dependent plasticity (STDP) rule; this rule can efficiently extract information from real-time data, such as images, video, or other raw sensor data [14].

Fig. 3(a) shows the electrical equivalents of the DW-MTJ synapse and neuron discussed in section III. The synapse is represented by an analog resistor $R_{\text{MTJ}}$ in the S1-S3 path and a conducting wire in the S1-S2 path. The conducting wire represents the DW track which has a negligible resistance compared to the MTJ resistance. The neuron equivalent also consists of a conducting wire in the N1-N2 path representing the DW track and a binary resistor $R_{\text{MTJ}}$ in the N3-N4 path. The isolation between the N1-N2 and N3-N4 paths results from the electrically insulated magnetic coupling layer in Fig. 2 (b).

Fig. 3(b) illustrates the neuron-synapse connectivity underlying the inference and learning mechanisms. The synapse is connected between an input pattern source and a post-synaptic neuron. The image pixel can be converted into binary 0 and 1 input patterns corresponding to voltage levels $Gnd$ and $V_{\text{dd}}$, respectively. The input voltage is connected to the S1 terminal of the synapse and a feed-forward connection is made between the S3 and N1 terminals. The feed-forward current $I_{\text{FF}}$ flows through the S1-S3-N1-N2 path, is modulated by the analog resistor $R_{\text{MTJ}}$ corresponding to the synapse, and results in the integration process of the post-synaptic neuron.

The learning is performed by creating a feedback connection between the N4 and S2 terminals. The feedback current $I_{\text{FB}}$ flows through the N3-N4-S2-S1 path and is modulated by the binary resistor $R_{\text{MTJ}}$ in the N3-N4 path. The N3 terminal is connected to voltage level $V_{\text{dd}}$/2. Therefore, the feedback current $I_{\text{FB}}$ has a positive (negative) value if the input voltage connected to the N1 terminal is $Gnd$ ($V_{\text{dd}}$). The positive (negative) $I_{\text{FB}}$ current through the S2-S1 path decreases (increases) the synaptic analog conductance of $R_{\text{MTJ}}$ connected in the S1-S3 path, as mentioned in Section III. The conductance update summary is provided in Table I, where the synapse conductance changes only when the post-synaptic neuron fires. The binary resistor $R_{\text{MTJ}}$ connected between N3-N4 controls the rate of the learning mechanism. For a firing (non-firing) neuron, $R_{\text{MTJ}}$ connected between N3-N4 provides a high (low) conductance and results in a higher (lower) value of the $I_{\text{FB}}$ current and learning rate.

Fig. 4 shows the proposed DW-MTJ crossbar with a feedback connection from 4-terminal DW-MTJ neurons to 3-terminal DW-MTJ synapses in order to perform unsupervised learning. Input signals are provided to the left terminals of the synapses. Electrical signal flows across the tunnel barrier to the central terminal of each synapse, eventually reaching the connected output neuron. The current flowing across each synapse tunnel barrier is a function of the DW position, and therefore the synaptic weight given by the conductance of the DW-MTJ [15]. The weighted current fed into the N1 terminal of the output neuron performs LIF functionalities by modulating its DW position. The feedback operation to perform unsupervised learning is achieved by connecting the N4-terminal of the output neuron to the right terminals of the synapses connected to its N1 terminal.

The behavior of DW-MTJ synapses and neurons is given by equations that characterize tunnel barrier conductance as a function of DW position and the motion of DWs as a function of current flow between the left and right terminals. For the synapse $s$, its conductance $G_{s}$ at time $t$ is given by (1), where $x^{t}_{s}$ is the position of the synapse DW along the track, $w_{s}$ is the width of the tunnel barrier, $G_{\text{AP}}$ is the anti-parallel conductance, and $G_{\text{P}}$ is the parallel conductance. For consistency with the literature on neural networks, we use this conductance value to define the weight of the synapse $\omega^{t}_{s}$ as a normalized value between $0$ and $1$ in (2). In the neuron $n$, the short tunnel barrier produces only binary conductance states characterized by (3), where $b_{n}$ is the position of the neuron barrier along the track. In every synapse $s$, a current $I$ causes the DW to move from its current position $x^{t}_{s}$ to $x^{t+1}_{s}$ after a time $\Delta T$ (4). In the neuron $n$, a leaking force $L$ is included such that the DW dynamics are given by (5). Due to this leaking and the lateral inhibition among nearby neurons, the perceptron system can be designed to ensure that exactly one neuron fires in response to each set of inputs, and that the neurons are reset before the next set of inputs is provided [5].

Our approach relies on two phases: training and inference. In line (3): If $V^{d_{ij}}=\text{Gnd}$, then the feedback current $I^{\text{out}}_{ij}$ flows through synapse $s^{a}_{ij}$ in the right-to-left direction, leading to $\omega^{t+1}_{s^{a}_{ij}}<\omega^{t}_{s^{a}_{ij}}$. That is, the synaptic weight of $s^{a}_{ij}$ will be decreased because this synapse did not contribute to detecting pixel $d_{ij}$. If $V^{d_{ij}}=V_{\text{dd}}$, then the feedback current $I^{\text{out}}_{ij}$ flows in the left-to-right direction through the $s^{a}_{ij}$ synapse, leading to $\omega^{t+1}_{s^{a}_{ij}}\geq\omega^{t}_{s^{a}_{ij}}$. That is, the synaptic weight of $s^{a}_{ij}$ will be increased because this synapse contributed to detecting pixel $d_{ij}$. Since the neurons are reset after each input, we need not worry about the changes in $x^{t+1}_{n},G^{t+1}_{n}$ for output neurons.

In this work, we highlight how a set of DW-MTJ synapses connected to several competitively learning DW-MTJ neurons can implement an effective form of unsupervised learning (Clustering) and have demonstrated the fundamentals of the electrical circuit and algorithm implementation that make this possible. Through concrete use-cases on the MNIST, Fisher Iris, and Wisconsin Breast Cancer datasets, we have demonstrated the utility of this approach and its superiority to standard shallow neural network learning. In our next work, we plan to build upon these promising early results by incorporating additional bio-plausible neuronal effects such as homeostasis and resonant and fire, on the neuron level, and three-factor and other additional plasticity effects at the synapse level.