TMS-Crossbars with Tactile Sensing

Fig. 1 shows a single layer $m\times n$ neural network representation of sensor crossbar array with 1T1M1S cell configuration. The 1T1M1S configuration consists of a Transistor Selector, a memristor and a Sensor is shown in Fig. 1. The network consists of $m$ Horizontal Lines (HLs) and $n$ Vertical Lines (VLs); in Fig. 1 $m=3$ and $n=4$. The inputs $\left(v_{1},v_{2},v_{3}\right)$ are the supply voltage to the sensors in each HLs. The sensor will sense the external stimuli and convert it to corresponding electrical signals. The sensor output is then weighted using the memristive block. Similar to many prior arts, the transistor is used to control the accessibility of devices in a cell. Through the HLs and VLs, the synaptic weighting functions are executed in parallel. The output currents $\left(i_{1},i_{2},i_{3},i_{4}\right)$, are the weighted summation of inputs and can be read from the VLs.

The effective conductance of each cell $g_{kl}$, with $k\in\{1,m\},l\in\{1,n\}$, is the combined conductance of sensor, memristor and switching blocks. Thus the effective conductance of each cell becomes $g_{kl}=g_{S_{kl}}||g_{M_{kl}}||g_{T}$ where $g_{S_{kl}},g_{M_{kl}}$ and $g_{T}$ are the effective conductance of sensor block, memristor and transistor at ON state respectively. When transistor is at OFF state, $g_{T}=0$ and hence $g_{kl}=0$. The transistor can be turned ON/OFF using the Select Line (SL). The total current in each column is the sum of individual currents flowing through each cell in that particular column.

The VL readouts can provide only limited information regarding the sensed data for tactile applications. To increase the scalibility of system under implementation, more features need to extracted from the sensed data. Hence the HL read-out are also used in addition to VL readout. The weighted summation are taken along both VL and HL lines simultaneously. This increases the number of post-neurons in the neural network layer. The above mentioned 1T1M1S configuration can be used for VL-HL readout crossbar circuit. However, this results in sneak path current issues which degrades the detection accuracy [9]. The 1T1M1S configuration provides cell selection only in the VL lines of the crossbar. This will not ensure proper cell selection in the HL lines and hence results in leakage currents. Fig. 2(a) shows the leakage currents for VL-HL readouts in $2\times 2$ crossbars when $S_{11}$ is active, where $S_{11}$ denotes the sensor in first row and first column. Ideally the sensed data can be read from $i_{1},i_{3}$ whereas $i_{2},i_{4}$ should be zero. Since all nodes are short circuited, there will be current leakage and $i_{1},i_{2},i_{3}$ and $i_{4}$ will have the same currents. Thus, the core operation of a neuromorphic system, i.e, multiply and accumulate (MAC), cannot be implemented with the 1T1M1S crossbar array for VL-HL readouts.

For the VL-HL readouts in TMS crossbars, we propose 2 Transistors 1 Memristor 1 Sensor (2T1M1S) cell configuration as shown in Fig. 2(b). In 2T1M1S, the two transistor configuration is used to read the sensor data along the horizontal and vertical lines of the crossbars. A two transistors one memristor (2T1M) cell configuration was proposed in [10, 11] for online learning of memristor where each memristor is connected to two inputs, one is the original input signal and other is the inverted input signal, obtained with the two transistors configuration. In contrast for the proposed 2T1M1S, one transistor act as a select switch in the VL_k line and the other transistor act as switch in the HL_l line. The output currents in VL_k and HL_l lines are weighted summation of inputs and nodal conductance.

In Braille system, each character is represented by 6 dots (D1, D2, D3, D4, D5, D6), Fig. 3(a). Since some of the braille symbols have the same notations, we use 2 additional dots (D7, D8) to differentiate them. Each dots represents one cell of TMS crossbar array in the 2T1M1S configuration. The weighted summation of each dots are taken through HL and VL lines, Fig. 3(b). The first layer of our neural network consists of 4 preneurons and 6 postneurons. Of the 6 post neuron 2 neurons represents column out and 4 neurons represents row out in the crossbar array as shown in Fig. 3(b).

In the sensor block of 2T1M1S sensor crossbar array, the commercially available force sensing resistors (FSR) is used as unit elements for the Braille character inputs. The equivalent circuit model of the FSR, specifically Tekscan (Flexiforce sensors) [13], used in the proposed design is presented in Fig. 4(a). It consists of a variable resistance part which can vary the resistance according to the applied force as per the FLX-A505 datasheet values [13]. The force dependent variable resistance equation is $R_{1}(f)=1/((1.5\mu*f)+c)$. Here $f$ is the applied force and the constant $c$ gives the bias voltage. The output is calculated using the voltage divider rule. Thus, with increase in force, resistance reduces and output voltage increases. The supply voltage to the sensor is 0.5V. The resistance variation along with the corresponding conductance variation with respect to applied force is plotted in Fig. 4(b). The force taken as a reference for detection is 20 lb.

Except first layer, other layers are implemented with memristive crossbar arrays using 1T1M cell configuration [14, 4]. The braille system designed using one-hot encoding with the memristor crossbar arrays requires weight updating with a training algorithm. The training is done using custom made Python programs. The trained weights are programmed to the corresponding memristors in the crossbar array. The hidden layer have ReLU activation function and the output neuron uses softmax activation function to get one-hot encoded output. The voltage signal applied to the HL lines is multiplied by the memristor conductance. The currents are then converted by means of the summation amplifier.

The onehot encoding of the braille system is done using softmax activation function at the output layer. The circuit level implementation of the softmax is shown in Fig. 5. It consists of three main blocks: (a) exponential block, (2) Summation, and (3) Division block [15]. The novelty of our design is the softmax implementation using transistors amplifiers designed using 22nm COMS technology. The design is compatible with the preceding neural network layers designed using nanometer scale devices. The output of exponential block is $x_{z}=R_{f}I_{s}\exp\left({\frac{a_{z}}{V_{T}}}\right)$ [16]. Where $V_{T}$ is the thermal voltage and $I_{s}$ is the saturation current. The summation block sum up the values from all the exponential block. The conversion of column current to voltage is done with a 3 stage amplifier proposed in [17] using 22nm CMOS node. The output of the weighted summer will be $x_{tot}=\sum_{z}\left(\frac{R_{f}}{R}\right)x_{z}$. If $R=R_{f}$, $x_{tot}=\sum_{z}x_{z}$, i.e. summation of exponential operation. Substituting $x_{z}$, $x_{tot}=\sum_{z}R_{f}I_{s}\exp\left({\frac{a_{z}}{V_{T}}}\right)$. Next process is to divide $x_{z}$ with $x_{tot}$. Fig. 5 shows the analog division circuit whose output is proportional to the division of two input values. We can divide two numbers by first calculating their logarithm, then calculate the difference between the log values and calculate the exponent of the difference value. The output of division circuit is $R_{f}I_{s}\left({\frac{V_{1}}{V_{2}}}\right)$ [16]. Substituting $x_{z}$ and $x_{tot}$ as $V_{1}$ and $V_{2}$, the theoretical value of $y_{z,theory}$ can be calculated as [15]:

Fig 6(a) compares the effective cell conductance of a single cell $g$ with the chosen range of memristance ${R_{OFF}}/{R_{ON}}={100K\Omega}/{1K\Omega}$ values. The result shows the effective conductance reduces as resistance increases when the select transistor is ON. This property helps to take the weighted summation of the sensor out from different cells and hence MAC operation. When the transistor is turned OFF, the extremely small transistor conductance $g_{T}$ dominates the cell conductance. Therefore, the VL and HL currents comes only from those vertical and horizontal ON cells.

Fig 6(b) shows the variation of effective cell conductance $g$ with varying force in $lbs$. The effective conductance increases with increase in the applied force. The figure shows the variation of $g$ with the memristance $1K\Omega$, $10K\Omega$, $50K\Omega$ and $100K\Omega$. Fig 6(a) and Fig 6(b) shows that the proposed TMS crossbars can show significant variation in output with different memristor values and variation in applied force. Thus sensor conductance, $g_{S}$, play a significant role in TMS crossbar in effective cell conductance computations and neuromorphic computing. For the Braille character recognition system, it is considered that the tactile patch is detected with a force of $20lbs$.

125 distinct Braille characters including Grade 1 and Grade 2 Braille system [20] are implemented with the proposed TMS crossbar array. Grade 1 consists of alphabets (capital and small), numbers punctuation and symbols. The Grade 2 braille system represent a shortened form of common words and sounds. Depending on ON/OFF select line of (D7,D8), the 125 braille characters are divided into different groups. Group1: Alphabet Capital (Grade1, 27 symbols), Group2: Alphabet Small (Grade1, 26 symbols), Group3: Words (Grade2, 46 words) and Group4: Numbers, punctuation & symbols (Grade1, 26 symbols). The Fusion set is the combination of Grade 1, Grade 2, Grade 3 and Grade 4 braille characters with 125 braille symbols. (D1, D2, D3, D4, D5, D6, D7, D8) values (ON, OFF, OFF, OFF, OFF, OFF, OFF, OFF) and (ON, OFF, OFF, OFF, OFF, OFF, OFF, ON) represents A and a respectively.

The neural computation in the crossbars are prone to noise. This noise can result from thermal noise or resistance variability. To test the impact of signal noise on the sensor crossbar, we apply additive Gaussian noise with variances, $\sigma^{2}=0.02$, $\sigma^{2}=0.05$, $\sigma^{2}=0.1$ and $\sigma^{2}=0.5$. The dataset consists of 5 copies of the 125 different braille symbols. Table I shows the detection accuracy of implementing the Braille dataset with variation of noise. In the neural network model of the implementation, the first dense layer uses the ReLu activation function, while the last layer is the softmax function. The second neural network layer or first 1T1M crossbar size is taken as 6$\times$14 where as size of output neurons is dependent on the size of the selected group. The accuracy in detecting the symbols for each group is presented and compared with fusion dataset. Table I compares the recognition accuracy of analog TMS crossbar with binary TMS crossbars. The results conveys that analog system performs better in noisy environment whereas binary systems are more prone to errors and hence detection accuracy reduces. Table II shows the comparison of the proposed method with other Braille character recognition system in the literature. The proposed method shows improved performance in terms of detection accuracy for both 26 (Group 1) and 37 (37 characters from Group 3) number of Braille characters. The proposed implementation is scalable solution that can implement 125 Braille characters without much degradation in performance.

The trained weights are then mapped to the memristor crossbar array. In the softmax circuit, $I_{s}=1nA$ and $R_{f}=100K\Omega$. The output of the softmax activation function of the proposed system is shown in Fig 7(a) for Group1, alphabet ’t’ and Group4, number ’9’. The softmax activation function represents the output with probabilities and the output port with maximum probability is the detected symbol. According to the figure Group1, alphabet ’t’ refers to port 20 and Group4, number ’9’ refers to port 107. The output is validated with the theoretical value in eqn. (1) and the software simulated output with $\left(\frac{e^{a_{z}}}{\sum_{i}e^{a_{z}}}\right)$ using Python programming. For hardware out, softmax output calculated by taking the exponential of $\frac{V_{m}}{V_{T}}$. Hence the minimum probable symbol values approximates to zero and maximum probable symbol values will be enhanced.

Fig 7(b), Fig 7(c) and Fig 7(d) shows the variation in softmax output with noise. The output of the proposed design tested with Group1- alphabet ’P’, Group2 - alphabet ’p’, Group3 - word ’people’, Group3 - sound ’ch’, Group4 - symbol ’#’ is presented. Fig 7(b) shows the output without noise and the symbol is correctly identified at port 16, 42, 68, 90 and 120 respectively. If the same input is corrupted with noise of variance $\sigma^{2}=0.05$, ’P’, ’p’, ’people’ and ’#’ are detected correctly but ’ch’ will be falsely detected as ’us’. Whereas with noise of variance $\sigma^{2}=0.05$, ’p’ and ’ch’ will be falsely detected as ’f’ and ’it’ respectively. This shows that as noise increases false detection increases and reduces accuracy.

Table III provides the area and power requirements of the proposed TMS-based Braille system. The power and area requirements of crossbars and amplifiers are listed separately in Table III. The output of crossbars can be parallel processed or serial processed [23]. A detailed description of serial processing in TMS crossbars is presented in Supplementary Material. Since the power consumption of parallel processing increases to 0.2W, the serial processing of with an analog signal multiplexing and a single division circuit, with power consumption of 3.7mW, can be considered a power efficient way of implementing the TMS-based Braille system. With the sensor area of $50.8mm\times 50.8mm$ [13], the area of Layer 1 rises to 0.02$m^{2}$ whereas the other layers gives an area of 512.898E-6 $m^{2}$ for parallel processing and 129.6E-6 $m^{2}$ for serial processing respectively. Table III also presents the power and area calculation of the Binary TMS implementation of Braille system for parallel and series processing as in [24]. The sensors give analog output, and the tactile solutions implemented as binary systems require data converters, interface, and adder circuits that require higher area and power than analog circuits.