HEAL: Brain-inspired Hyperdimensional Efficient Active Learning

In this paper, we focus on a multi-class classification problem, with ${\bm{\mathcal{X}}}$ being the instance space and $\bm{\mathcal{Y}}$ being the label space. Suppose that there are $C$ different classes in $\bm{\mathcal{Y}}$: $\{1,\dots,C\}$. We are interested in a classifier that predicts the label of each instance, which can be regarded as a mapping function $\bm{F_{\omega}}:\bm{\mathcal{X}}\rightarrow\bm{\mathcal{Y}}$. $\bm{\omega}$ stands for the classifier parameters. For the case of AL, we assume that the classifier is trained in a supervised way; however, instead of having a large labeled dataset in the beginning, the classifier starts training with a small labeled training set $\bm{\mathcal{D}}_{tr}$ and the size of training set will gradually growing as the AL algorithm kicked in. This particular training procedure is known as pool-based AL since there is a relatively large pool of unlabeled data $\bm{\mathcal{D}}_{pool}$. As regular supervised learning, a testing or validation dataset $\bm{\mathcal{D}}_{te}$ is used for evaluation.

Fig. 1 presents the procedure of AL, which includes an oracle to provide extra labeled data points. The oracle can give the correct label for every data point in $\bm{\mathcal{D}}_{pool}$ as long as the annotation request is made from AL. The labeled data points are then removed from $\bm{\mathcal{D}}_{pool}$ and added to the training dataset $\bm{\mathcal{D}}_{tr}$. The goal of AL, in short, is to minimize the number of annotation requests (also the size of the labeled dataset) while ensuring satisfying prediction accuracy. Specifically, at each step, AL uses an acquisition function $\bm{\mathcal{A}}$ to rank samples in $\bm{\mathcal{D}}_{tr}$ and select extra data that maximizes this function, given the current classifier trained using $\bm{\mathcal{D}}_{tr}$:

In practice, AL algorithms usually acquire a batch of new data points instead of only one $\bm{x}_{to\_label}$ at each step to reduce the frequency and cost of model retraining. This can be achieved by simply acquiring top $K$ data points according to the acquisition function $\bm{\mathcal{A}}$. Sometimes, it is also required to slightly modify the acquisition function [19] or add extra metrics [16] to maintain the diversity in acquired batches. (more details in Section 4.2).

Bayesian inference plays a key role in uncertainty-based AL as it provides a principled way of understanding the uncertainty inherent in predictions [19, 15]. In Bayesian inference, the neural networks are no longer modeled in static weights $\bm{\omega}$ but stochastically in a distribution $p\left(\bm{\omega}|\bm{\mathcal{D}}_{tr}\right)$ [25]. It is known as the posterior distribution whose variance quantifies the belief in DNN weights given the seen data points in training. This also means a prior belief $p\left(\bm{\omega}\right)$ exists before training. Using the posterior, the predictive distribution is calculated as: $p\left({y|\bm{x},\bm{\mathcal{D}}_{tr}}\right)=\mathbb{E}_{\bm{\omega}\sim p\left({\bm{\omega}|\bm{\mathcal{D}}_{tr}}\right)}\left[p\left({y|\bm{x},\bm{\omega}}\right)\right]$. We can compute the posterior using the Bayes Theorem as follows:

with $p\left({\bm{\mathcal{D}}_{tr}|\bm{\omega}}\right)$ being the likelihood. From the equation above, it is clear that the exact posterior will be intractable in DNNs because the weight space is generally high-dimensional and takes exponential time to evaluate the integral in the denominator [23].

Therefore, approximating the posterior in a more tractable way is fundamental for achieving Bayesian inference in DNNs. Variational Inference (VI) has been a widely applied method that replaces the original $p\left({\bm{\omega}|\bm{\mathcal{D}}_{tr}}\right)$ with a simpler variational distribution with a tractable $q\left({\bm{\omega}|\bm{\omega}_{q}}\right)$, e.g., a family of Gaussian distributions [22, 21]. This is usually achieved by minimizing the Kullback-Leibler (KL) divergence that measures the differences between two probability distributions. By rearranging the terms in the KL divergence, we get Evidence Lower Bound (ELBO).

Early works that tried to apply VI for DNNs built up the foundation for what we know today as Bayesian Neural Networks (BNN) [26, 27, 28]. However, they faced several challenges such as the lack of support for modern DNNs and scalability to larger datasets [29]. More recent BNN works [21, 22, 23] make it practical to apply Bayesian inference for deep learning tasks, thanks to their compatibility with most DNN structures and mini-batch gradient descent optimization.

Apart from VI-based BNNs, researchers also develop methods based on ensemble learning for uncertainty estimation in DNNs. For example, MC-Dropout [24] proposes to approximate Bayesian inference by adding dropout layers. In its design, dropout layers are enabled also in network inference instead of just during training. The predictive distribution and uncertainty are derived from these multiple forward passes, assuming that the predictions follow a Gaussian distribution. On the other hand, Deep Ensemble [30] more explicitly utilizes the ensemble of DNNs with bootstrapping and adversarial training. For classification problems, the predictive probabilities are computed as the average of ensemble softmax probabilities. Prior works [31, 32] also show that Deep Ensemble generally achieves the best performance on uncertainty estimation, compared to the implicit ensemble in MC-Dropout and VI-based BNN.

As is pointed out in the introduction, one motivation of this paper is that current AL methods face various kinds of inefficiencies; these challenges are the consequences of using acquisition functions with high overhead as well as relying on computation-heavy DNNs (or BNNs) for training and inference.

However, the aforementioned methods generally have significant acquisition costs as they require BNN during uncertainty estimation. As for the BNNs mentioned above in 2.2, they all show overheads in different aspects. First, to obtain the predictive distribution, methods like MC-Dropout and Bayes-by-Backprop require more than 100 forward inferences on one single test sample. The inefficient BNN inference directly leads to a higher acquisition cost since every AL step requires scoring all data in $\bm{\mathcal{D}}_{pool}$. The same is true for Deep Ensemble where the inference is inefficiently carried out on multiple sub-models. Second, training BNNs is notably more costly than regular DNNs using similar architectures. For example, network parameters are doubled in BNNs that parameterize the variance of weights [23], and they can have poor scalability to dataset sizes [38].

In addition, AL methods with only the uncertainty metric are prone to sample duplicate data points in the batch acquisition, compromising the advantage in data efficiency. The next few categories of AL methods mitigate this problem by explicitly or implicitly considering the diversity in batch acquisition. However, they have also shown significantly higher overhead in acquisition.

As for the drawbacks, they still suffer from the overhead of Bayesian inference and posterior computation, since the information metric selected will only be helpful and properly calibrated with Bayesian inference. In addition, computing equation 3 in each step of batch acquisition is time-consuming and memory-heavy as the complexity grows exponentially with the batch sizes [19]. With a batch size larger than 10, the acquisition runtime can become inhabitable for deployment on CPUs or any resource-limited hardware.

The human brain is a very efficient and robust learning machine and inspires many different machine learning algorithms. HDC, as our focus in this paper, utilizes high-dimensional representations to emulate brain functionalities for ML tasks. Representations in HDC are considered holographic since the information is evenly distributed among all dimensions of the hypervector [3]; therefore, corruption or noises on some dimensions will not lead to catastrophic information loss [42].

As the background, we start with how HDC represents symbols and memorizes concepts in a structured way. Suppose there are three different symbols $a$, $b$, and $c$, we map them to three randomly-sampled hypervector representations $\bm{h}_{a}$, $\bm{h}_{b}$, and $\bm{h}_{c}$ with dimensionality $D$. More specifically, every element in those hypervectors is i.i.d. usually following a symmetric, zero-mean distribution [9]. For example, $\bm{h}$ can be bipolar high-dimensional vectors, whose elements are either +1 or -1 with equal probability [3]. Hypervectors can represent not only individual symbols but also a combination or association of multiple symbols, which is enabled by the following three fundamental hypervector operations.

• •

  Hypervector Similarity ($\delta$) represents how close two hypervectors are in the hyperspace, defined based on the normalized dot product: $\delta\left({\bm{h}_{a},\bm{h}_{b}}\right)=\bm{h}_{a}\cdot\bm{h}_{b}/(||\bm{h}_{a}||*||\bm{h}_{b}||)$. When $D$ is in the range of several hundred to around ten thousand, the similarity between any two random hypervectors is nearly zero (also known as near-orthogonal) because of the property of high-dimensional space.

• •

  Bundling ($\oplus$) in HDC means the element-wise addition of two or more hypervectors. This creates a new hypervector that represents the set of hypervectors and thus remains similar to all its constituents. Suppose we have $\bm{h}_{s}=\bm{h}_{a}\oplus\bm{h}_{b}$, and then we check the similarity between the bundled hypervector and each symbol hypervector: $\delta(\bm{h}_{s},\bm{h}_{a})\approx\delta(\bm{h}_{s},\bm{h}_{b})\gg 0$ while $\delta(\bm{h}_{s},\bm{h}_{c})\approx 0$.

• •

  Binding ($\odot$) stands for the element-wise multiplication of two hypervectors. It generates a dissimilar vector representing their association. To associate $a$ with $b$, we bind their corresponding hypervectors: $\bm{h}_{(a,b)}=\bm{h}_{a}\odot\bm{h}_{b}$. Unlike bundling, $\bm{h}_{(a,b)}$ is dissimilar to both constituents, e.g., $\delta(\bm{h}_{(a,b)},\bm{h}_{a})\approx 0$.

The bundling operation is usually used when creating HDC-based ML models such as the classifier hypervectors and regression model hypervectors. The binding operation is used when associating features and values. We will cover more details regarding the usage of hypervector operations in Section 3.1.

In the past few years, HDC has gained significant traction as an emerging computing paradigm, especially for its deployments in machine learning and reasoning tasks. Prior works have proposed HDC-based algorithms and learning frameworks for classification [43, 4, 44], clustering [45, 46], regression [47, 9], and reinforcement learning [14, 48, 49] problems, showing the benefit of fast convergence in learning, high power/energy efficiency, natural data reuse and acceleration on customized devices [12, 11, 50, 51], and robustness on error-prone emerging hardware [52, 53]. Particularly, HDC has been successfully applied to many supervised learning tasks. For biomedical applications, HDC-based algorithms with low power consumption are aimed at solving problems like DNA sequencing [50, 6], health monitoring [54, 55, 56], and subject intention recognition [57]. In work [58], the proposed method achieves one/few-shot learning on edge devices for the task of epileptic seizure detection. In addition, HDC has also shown significantly faster learning in classifying human faces [59], spam texts [60], texts[61], etc. More recently, researchers also proposed to incorporate uncertainty estimation into HDC-based regression via a customized HDC encoder that randomly drops dimensions [62]. However, this method is not suitable for our use case since dropping dimensions has little impact on classification results due to the robustness of HDC-based classification. However, this paper focuses on incorporating uncertainty estimation into HDC classifiers and designing an AL mechanism in an efficient way for enhancing HDC-based ML with better data efficiency.

The encoding process in HDC is essentially mapping a vector of features to the high-dimensional space or hyperspace. HDC encoders need to maintain the relationship between input features in the hyperspace, e.g., two similar inputs are mapped to hypervectors with relatively larger cosine similarity [63]. For $\mathsf{HEAL}$, we leverage the Fractional Power Encoding (FPE) to achieve a holographic reduced representation [8], i.e., symbols and their structured combinations can be represented uniformly in a hyperspace. In Fig. 2, we provide the outline of the HDC encoding process in $\mathsf{HEAL}$.

We start by assuming a feature vector input of length $n$: $\bm{x}=\left[x_{1},x_{2},\dots,x_{n}\right]^{T}$, and we view each feature in $\bm{x}$ as a symbol to encode. The HDC encoder is pre-loaded with a series of positional hypervectors $\{\bm{\rho}_{1},\bm{\rho}_{2},\dots,\bm{\rho}_{n}\}$, each corresponding to one feature/position of the input feature. Elements in positional hypervectors are randomly sampled unitary phasors, i.e., $\bm{\rho}_{n}=\{e^{i\theta_{\bm{\rho}_{n}}}\}^{D}$ where $\theta_{\bm{\rho}_{n}}\sim\mathcal{N}(0,1)$. Thus, the randomly sampled positional hypervectors are still near-orthogonal to each other in the hyperspace. Notice that although they are no longer bipolar hypervectors, binding and bundling operations mentioned in Section 2.4 still apply to them.

The difference in FPE lies in the way it associates features and their values. If we use regular binding defined in Section 2.4, two hypervectors, one for the feature and one for its value, will be combined using element-wise multiplication (e.g., $\bm{\rho}_{1}\odot\bm{v}_{x_{1}}$). This requires value quantization such that value hypervectors $\bm{v}$ represent different discrete steps of feature values, which inevitably introduces information loss and cannot scale to data with large value ranges. FPE allows this to happen in a much finer granularity with fractional power self-binding [8]. Recall that binding results in a dissimilar hypervector, and thus we can encode integer feature values by repeatedly self-binding the feature positional hypervector. For example, we can encode $x_{1}=3$ as: $\bm{\rho}_{1}^{3}=\bm{\rho}_{1}\odot\bm{\rho}_{1}\odot\bm{\rho}_{1}$. With FPE, a feature with floating number value is encoded similarly, e.g., $x_{2}=5.5$ is encoded by elementwise exponential $\bm{\rho}_{2}^{5.5}=e^{i\bm{\theta}_{\bm{\rho}_{2}}*5.5}$. More generally, for the feature vector input $\bm{x}$, we have the encoding process as follows:

where we bind all encoded features together and $\Theta$ stands for an $n\times D$ matrix with each row being the $\bm{\theta}_{\bm{\rho}_{n}}$ vector. Notice that $\bm{h}_{\bm{x}}\in\mathbb{C}^{D}$ and similarity is real-valued, the similarity computation for hypervectors with FPE is defined as:

where $\bm{h}_{\bm{x^{\prime}}}^{{\dagger}}$ is the complex conjugate and we normalize the real part of the dot product; this is also known as the Euclidean angle for complex vectors.

BNNs with ensemble-based design have been leveraged for uncertainty-based AL since methods like MC-Dropout can estimate the model confidence via the variance in prediction [64]. On the other hand, HDC is also compatible with ensemble learning; we can train multiple HDC sub-models, each with its own HDC encoder and bootstrapped training set, i.e., bagging. During the inference, all sub-models will contribute to the prediction in a consensus-based way.

Assume $E$ HDC sub-models $\{\bm{M}_{1},\bm{M}_{2},\cdots,\bm{M}_{E}\}$, and the first sub-model $\bm{M}_{1}$ will be trained with a data point $\bm{x}$ with label $l_{true}$. We first encode this sample to hypervector $\bm{h}_{\bm{x}}$ via equation 4 and then perform a similarity check with every class hypervector in model $\bm{M}_{1}$ using equation 5. The prediction process for $\bm{M}_{1}$ is shown below:

Then, class hypervectors will be updated according to how well the model predicts. More specifically, if the prediction is not correct ($l_{pred}\neq l_{true}$), the update process for sub-model $\bm{M}_{1}$ is as the following:

where $\lambda$ is the learning rate. The similarity values $\delta$ in the updates function as the feedback that dynamically controls the learning rate. For instance, a $\delta\left({\bm{h}_{\bm{x}},\bm{m}_{l_{true}}}\right)$ near 1 means that the class hypervector already contains the information in the query and only a slight update is needed as in equation 7; the wrong prediction may be the result of $\delta\left({\bm{h}_{\bm{x}},\bm{m}_{l_{pred}}}\right)$ being incorrectly high, and therefore equation 8 will try to rectify. In addition, if the prediction is correct, then the HDC model is not updated. The process above happens separately for every sub-model during training based on bootstrapped sampling, and they are trained iteratively with batched samples.

To qualitatively evaluate this estimation of model uncertainty, we apply HDC ensemble training using the MNIST dataset while testing on the NotMNIST [65] dataset, which is comprised of out-of-distribution (OOD) alphabet images with 10 classes and the same image size as MNIST. Since the ensemble model only knows handwritten digits from training, we expect the prediction probability to be closer to uniform during testing. In other words, each ensemble model will disagree with each other when evaluated on OOD samples, which results in a much larger predictive entropy.

In Fig. 3, we present the histogram of predictive entropy when testing the HDC naive ensemble with samples from both MNIST and NotMNIST datasets. For the in-distribution case, the histogram in Fig. 3(a) shows a high density around zero and indicates that ensemble sub-models agree with each other, i.e., lower uncertainty in prediction. As for the OOD case in Fig. 3(b), it is clear by comparison that the HDC naive ensemble now shows a higher level of uncertainty as larger entropy values show up in the histogram, indicating the disagreement between sub-models for some data points. However, the quality of this uncertainty estimation is lower than expected because the highest density still occurs at zero entropy values. This means that when deploying the naive ensemble design in practice, the HDC ensemble will frequently be over-confident about its prediction, and thereby unsuitable for uncertainty-based AL.

One possible reason why the HDC naive ensemble performs poorly is that HDC sub-models fail to understand the data space from diverse perspectives, which is crucial for fostering disagreement when facing unseen data points. In fact, due to the way model hypervectors are constructed, i.e., being the superposition of seen data, even with bootstrapping, these HDC sub-models are somewhat similar after training. Therefore, there is a high chance that sub-models will predict similar labels when they actually have low confidence.

In previous HDC classification works, the model hypervectors are always initialized as zero vectors. Since these hypervectors are considered as the memory and prototype of data points of different classes, starting with zero values indicates no prior knowledge about the task or classes. Notice that this is different from DNNs with back-propagation training, as the zero initialization for neurons leads to uninformative gradients and poor learning results.

However, in $\mathsf{HEAL}$, we enhance the naive design in Section 3.2 with HDC prior hypervectors that serve two purposes: first, they are non-zero initializations of HDC models; second, these hypervectors, as a whole, represent the prior model distribution or prior knowledge about a certain task. In Fig. 4, we give an overview of introducing prior hypervectors in HDC ensemble learning. We randomly sample these hypervectors with i.i.d. elements from the standard Gaussian distribution $\mathcal{N}(0,1)$, and as a way of model initialization, we enable prior hypervectors for every sub-model in the HDC ensemble. We refer to them as $\{\bm{M}_{1}^{p},\bm{M}_{2}^{p},\dots,\bm{M}_{E}^{p}\}$, in contrast to regular HDC models $\bm{M}$.

As shown in Fig. 4, we perform two separate similarity checks in each sub-model; one is between the encoded data $\bm{h}_{\bm{x}}$ and HDC prior hypervectors $\bm{M}^{p}$ and the other one is with model hypervectors $\bm{M}$. To make predictions, we combine these two sets of similarity values, e.g., for the first sub-model:

During the HDC model training, the prior hypervectors will not be updated and stay static and the model hypervectors follow slightly different update rules based on previous equations 7 and 8:

where $S\left({\bm{h}_{\bm{x}},\bm{m}_{l}}\right)=\delta\left({\bm{h}_{\bm{x}},\bm{m}_{l}}\right)+\delta\left({\bm{h}_{\bm{x}},\bm{m}_{l}^{p}}\right)$ is the sum of two similarity values. Introducing prior hypervectors to training and inference helps ensemble sub-models learn with diverse viewing angles and allows them to disagree with each other when uncertain.

We will name $\bm{M}^{p}$ in Fig. 4 as isolated prior hypervectors, meaning their similarity with queries is computed separately. However, there is an obvious alternative to equation 11: $l_{pred}=\operatorname*{argmax}_{l}\delta\left({\bm{h}_{\bm{x}},\bm{m}_{l}+\bm{m}_{l}^{p}}\right)$, where the prior hypervectors are combined/bundled with model hypervectors before calculating the similarity. We compare the two different ways below:

Fig. 5 compares the quality of uncertainty estimation between using isolated prior hypervectors and combining the prior with trainable models. With isolated prior hypervectors, in most cases, the HDC ensemble is able to reveal high uncertainty when predicting OOD test samples, showing an entropy value significantly larger than zero. However, when the prior hypervectors are combined into the model, the quality of uncertainty estimation is notably worse. The degradation may be attributed to the model hypervectors dominating the similarity computation. From equation 14 and 15, we observe that the difference is in the denominators: in the combined case, the norm is taken for the bundled hypervectors, whereas in the isolated case, the normalization happens separately, maintaining the influence of prior hypervectors on the overall similarity computation. More specifically, during HDC training, the model easily becomes much larger in norm ($||\bm{m}_{l}||\gg||\bm{m}_{l}^{p}||$), diminishing the effect of $\bm{m}_{l}^{p}$ and discouraging the diversity in sub-models.

In Algorithm 1, we present the pseudo-code for HDC ensemble learning with prior hypervectors, which is used in $\mathsf{HEAL}$ to aid the estimation of model uncertainty.

In this section, we explore the opportunities for further improving the efficiency of HDC uncertainty estimation in $\mathsf{HEAL}$. We will approach this via techniques such as a shared HDC encoder, encoded data reuse, similarity computation reuse, pre-normalization, and dynamic dimension regeneration.

It is common for AL techniques to leverage probabilities in the acquisition function because DNNs usually add a softmax layer to normalize logits and get probability values in the range of 0 to 1. However, HDC does not give outputs interpretable as probabilities nor does it learn based on cross-entropy loss. This means that adding an extra softmax layer will not give informative predictive probabilities. Instead of using softmax, in equation 9, we estimate the predictive probability from predictions of ensemble sub-models. Although we have shown that it is quite useful for qualitative analysis, this is inevitably a coarse approximation and will limit its usage in uncertainty-based AL.

With no access to predictive probabilities, prior AL techniques like confidence and entropy sampling are unsuitable for HDC. However, we noticed that the margin sampling, although based on probabilities in the original design, can be adapted for HDC. It captures model uncertainty through the margin between the top two predictions, which generalizes beyond probability values. In $\mathsf{HEAL}$, we propose to directly utilize the similarity values from equation 15 and compute the prediction margin in the AL acquisition function:

The value of $\bm{\mathcal{A}}\left({\bm{x}}\right)$ becomes larger when the similarity with $\bm{m}_{l^{\prime}}$ gets closer to the one with $\bm{m}_{l}$, i.e., higher uncertainty about $\bm{x}$. We apply ensemble model averaging for similarity values; $l$ is the predicted label with the highest average similarity values and $l^{\prime}$ is the one with the second highest values.

where $\bm{m}_{l}^{i}$ represents the model(prior) hypervectors of class $l$ corresponding to the $i$-th sub-model. In every acquisition step, we sample a batch of data points from $\bm{\mathcal{D}}_{pool}$, which rank top-$K$ in the values of $\bm{\mathcal{A}}\left({x}\right)$.

HDC ensemble with prior hypervectors plays a key role in the $\mathsf{HEAL}$ acquisition process. After training, each sub-model can be confident about its prediction, giving high similarity values for the predicted class. However, sub-models will predict different labels for data points that the ensemble model as a whole is uncertain about. With ensemble model averaging in equation 17, the similarity value for a particular class is much lower due to the disagreement, and the margin sampling in equation 16 will capture the narrowed gap between top-2 predictions.

One common challenge in uncertainty-based AL is that the algorithm tends to select similar samples in batch acquisition. Notice that during batch acquisition, the model is not updated immediately until a full batch of data points is annotated. Therefore, a top-$K$ ranking will repeatedly select helpful data points that contain duplicate information. As we mentioned in Section 2.3, DNN-based AL methods cope with this problem via joint mutual information, representative sampling, and data mining algorithms, although with significant acquisition overheads due to these added components.

In this section, we propose an efficient diversity metric that helps $\mathsf{HEAL}$ acquire not only informative but also diverse data points, without introducing costly computations as in prior DNN-based methods. We utilize lightweight HDC operations and leverage the intrinsic memorization capability of hypervectors to achieve diverse acquisition. We notice that the requirement for diversity can be achieved by checking the similarity between the candidate data point and existing points in the current batch. In other words, hypervector similarity checks can be a strong tool for filtering out duplicate data points. This diversity metric can be seamlessly included in $\mathsf{HEAL}$ since the dataset has already been encoded to hypervector representations.

As shown in Fig. 6, instead of computing pair-wise similarity, we consider the acquired data points altogether by constructing a memorization hypervector for each class $\{\bm{n}_{1},\bm{n}_{2},\dots,\bm{n}_{C}\}$. These hypervectors memorize and categorize the acquired data points according to the pseudo label $\hat{l}$, i.e., the label predicted by all HDC sub-models via voting. They are initialized with all-zero vectors. To begin with, we perform inference on the unlabeled dataset $\bm{\mathcal{D}}_{pool}$ and rank it via the acquisition function $\bm{\mathcal{A}}\left({\bm{x}}\right)$ to prioritize the data points for which the model has lower confidence. The first acquired data point $\bm{x}_{1}$ (i.e., the one with the largest value of $\bm{\mathcal{A}}\left({\bm{x}}\right)$) will be automatically added to the batch. We update the memory hypervector that corresponds to the pseudo label: $\bm{n}_{\hat{l}_{\bm{x}_{1}}}=\bm{n}_{\hat{l}_{\bm{x}_{1}}}\oplus\bm{h}_{\bm{x}_{1}}$. For the next candidate data point $\bm{x}_{2}$ that ranked second, if it has the same pseudo label as $\bm{x}_{1}$, we will check its similarity with the memorization hypervector: $\delta\left({\bm{h}_{\bm{x}_{2}},\bm{n}_{\hat{l}_{\bm{x}_{1}}}}\right)$. We only acquire this sample when it shows low similarity values and discard it if otherwise. In our implementation, we use a similarity threshold $\gamma=0.4$. If its prediction leads to a yet empty memory hypervector, it will be acquired directly. We will repeat this process until the batch has been filled with acquired data points. For the next step of batch acquisition, the memory hypervectors are re-initialized.

We present the pseudo-code for $\mathsf{HEAL}$ in Algorithm 2. Before the acquisition process, we first pre-encode $\bm{\mathcal{D}}_{pool}$ and pre-compute similarities with prior hypervectors as mentioned in Section 3.4. Then in each acquisition step $t$, we annotate the acquired data points $\bm{\mathcal{B}}$ to the training dataset, remove them from $\bm{\mathcal{D}}_{pool}$, and then train the ensemble model on the new training set.

To evaluate the performance of $\mathsf{HEAL}$, we compare it against several existing AL algorithms that are widely applied, including traditional methods that are based on simple measures, and modern methods that are either non-Bayesian or Bayesian in terms of how uncertainty is estimated. Our baselines also include AL algorithms that explicitly consider diversity in acquisition. The following is a list of the baseline algorithms used in comparison.

1. 1.

   Rand: A naive design (i.e., non-AL) that randomly acquires unlabeled samples for expert annotation.

2. 2.

   Conf (Confidence Sampling): uses simple model confidence as the sign for uncertainty [20]. The acquisition starts with the sample with the lowest predicted class probability.

3. 3.

   Marg (Margin Sampling): An uncertainty-based algorithm, whose uncertainty metric is built upon the probability difference between the top-2 predicted classes [35]. The sample with the smallest margin will be selected first for annotation. Margin sampling can be enhanced by BNNs for better uncertainty estimation.

   1. (a)

      Marg_de: It uses Deep Ensemble as the BNN backbone.

   2. (b)

      Marg_dropout: uses MC-Dropout as the BNN backbone.

4. 4.

   ALBL: A hybrid AL method that aims at the balance between uncertainty-based (Conf) and diversity-based (Coreset) AL algorithms via bandit-style selection [17].

5. 5.

   BADGE: An intelligent hybrid method that uses gradient information in the DNN classifier to incorporate both predictive uncertainty and sample diversity into acquisition [16].

6. 6.

   BatchBALD: An information-theoretic AL method that identifies the most informative samples based on estimated information gain, computed via Bayesian models [19]. The metric used considers both model uncertainty and diversity of the acquired batch.

As for our proposed $\mathsf{HEAL}$, since many of its components are optional during practical implementations, we evaluated it with the following different settings:

• •

  $\mathsf{HEAL}$: the bare bone of our uncertainty-based AL framework using HDC ensemble with prior hypervectors.

• •

  $\mathsf{HEALneural}$: enhanced by hyperdimension regeneration for a more lightweight model and faster acquisition.

• •

  $\mathsf{HEALdiverse}$: enhanced by HDC memory hypervectors for diversity-aware batch acquisition.

Our AL evaluation follows the batch-mode setup; for all algorithms, it starts with 20 initial labeled training samples ($|\mathcal{D}_{pool}|=20$), and the acquisition batch size $b$ for each AL step ranges from 20 to 200. As for the model backbone, similar to BatchBALD, we select the multilayer perception (MLP) model for all DNN/BNN-based AL on all datasets; it has two hidden layers and each layer has 256 neurons. The models are trained on Pytorch using cross-entropy loss and the Adam optimizer. For $\mathsf{HEAL}$ and $\mathsf{HEALdiverse}$, we use hypervectors with $D=2000$; and $\mathsf{HEALneural}$, as we mentioned before, uses $D=1000$. For all algorithms, we train the classifier from scratch at every step of acquisition until the training accuracy hits 99%. All experiments are repeated five times and averaged.

We showcase our proposed AL frameworks on four different open datasets, as shown in Table 1. The first dataset is a speech recognition dataset and the rest three are for human activity recognition tasks. In practice, human activity recognition mainly involves on-body multi-sensor data analysis, where collecting labeled data for diverse user groups takes a significant amount of effort. Our experiments aim to show the effectiveness of AL algorithms, especially $\mathsf{HEAL}$, on saving annotation costs in similar tasks. Notice that the DSADS and PAMAP datasets undergo a widely applied data preprocessing as described in [70].

The learning curve is an intuitive way to evaluate the effectiveness of AL algorithms, which record the testing accuracy at each step of acquisition with an increasing number of labeled samples. In Fig. 7, we show the averaged testing accuracy of five runs for all four datasets and different AL algorithms. The batch size of acquisition is $b=20$. In Fig. 8, we present learning curves with other batch sizes ($b={50,100,200}$). Firstly, we observe that all AL algorithms eventually outperform random acquisition by a noticeable gap. Most AL algorithms help reach model convergence much faster than the case without AL. As for the performance of $\mathsf{HEAL}$ methods with different configurations, they often achieve significantly higher testing accuracy than other baselines during the first half of the curve, i.e., with less than 1000 labeled samples. When closer to model convergence, $\mathsf{HEAL}$ enhanced HDC classifier is comparable to hybrid AL methods including BADGE and BatchBALD. Albeit using similar acquisition metrics as margin sampling, $\mathsf{HEAL}$ achieves better AL quality, thanks to the optimal combination of HDC’s intrinsic learning efficiency and our uniquely developed uncertainty estimation approach for HDC-based classifiers. For more comparison between different variants of margin sampling and $\mathsf{HEAL}$, please refer to Fig. 14. For illustrations of HDC inherent learning efficiency and its comparison with $\mathsf{HEAL}$, please refer to Fig. 15.

To illustrate the efficiency of our AL framework, we collect the acquisition runtime for most algorithms (including $\mathsf{HEAL}$) using Intel Core i7-12700 CPU; except for BatchBALD, which is not efficient and scalable on CPU platforms, we use NVIDIA RTX 4090 GPU instead. In Fig. 9(a), we compare the average acquisition runtime of each AL method with two different batch size settings. As highlighted in this figure, $\mathsf{HEAL}$ and its variants have notably faster acquisition compared to most baselines. When $b=200$, the speedups of $\mathsf{HEAL}$ over marginal sampling with dropout, ALBL, BADGE, and BatchBALD are 18$\times$, 11$\times$, 239$\times$, and more than 40000$\times$, respectively. Confidence sampling is the fastest in acquisition due to its non-Bayesian uncertainty estimation. However, this naive estimation leads to its sub-optimal performance in AL. With the help of NeuralHD, $\mathsf{HEALneural}$ is about 45% faster than regular $\mathsf{HEAL}$ in acquisition. Due to the diversity-aware acquisition module, there is a relatively small overhead in $\mathsf{HEALdiverse}$. In Section 5.3, we will present the benefits brought by this extra module. In Fig. 9 (b), we show the size of the labeled training dataset needed for each AL method to achieve 99% of the accuracy obtained with the full dataset. This metric represents how much annotation effort can be saved with AL. The figure shows that $\mathsf{HEAL}$ is on average better than most baselines except the information-theoretic BatchBALD which comes at a huge cost. Note that the acquisition in BatchBALD is significantly slower than others even if it is the only method run on a powerful GPU. This is because BatchBALD suffers from the combinatorial explosion in its estimation of joint distributions [19].

As we mentioned in Section 4.2, datasets with a large number of similar samples pose challenges to many existing AL algorithms. Therefore, in this section, we ramp up the difficulty and evaluate these AL methods on specially modified datasets. For each of the previously tested datasets, we copy the training dataset four times, meaning that each unique sample now has five duplicates. We then repeat the evaluations in Section 5.2 and record the learning curves in Fig. 10. As expected, AL methods without effective diversity metrics such as confidence sampling and marg_de suffer from significant degradation in AL performance. In addition, the performance of ALBL over confidence sampling hints at the benefit of filtering out duplicate samples during acquisition. Nevertheless, many AL algorithms are showing acquisition quality worse than random selection. In contrast, methods like $\mathsf{HEALdiverse}$ and BADGE still maintain high acquisition efficiency with $\mathsf{HEALdiverse}$ giving the highest testing accuracy in the first half of the learning curve. Methods including BADGE and BatchBALD are also among the best performing AL methods during the second half, however, their acquisition costs are orders of magnitude larger. Also interestingly, we observe that methods that rely solely on ensemble generally perform poorly, as their sub-models are prone to be similar due to training on an overflow of duplicated samples and thereby compromise their uncertainty estimation. Fig. 11(a) shows that $\mathsf{HEALdiverse}$ is 41$\times$ (1000$\times$) faster than BADGE (BatchBALD) in terms of the average acquisition runtime. Fig. 11(b) shows that $\mathsf{HEALdiverse}$ is among the best performing AL methods for all datasets and significantly outperforms the regular $\mathsf{HEAL}$ design.

To better compare different AL methods comprehensively, we apply a pair-wise comparison method proposed in [16]. Every time algorithm X beats algorithm Y in terms of testing accuracy, the latter accumulates a certain amount of penalty. We normalize the values to $[0,1]$. Fig. 12 is the pair-wise comparison averaged on datasets without duplicate samples, and Fig. 13 is averaged on datasets with duplicate samples. A better-performing AL method shows more small values (i.e., dark color) in a row, e.g., $\mathsf{HEAL}$ outperforms in Fig. 12 and $\mathsf{HEALdiverse}$ in Fig. 13.

In this section, we illustrate the benefits of designing non-trivial uncertainty estimation techniques for AL by comparing methods with or without these techniques. In Fig. 14, ’marg’ stands for the basic margin sampling without using any BNNs, and ’HDCmarg’ refers to simple HDC similarity-based margin sampling without HDC ensemble models with prior hypervectors. In general, $\mathsf{HEALdiverse}$ and margin sampling with MC-dropout significantly outperform their naive versions.

Prior HDC arts such as OnlineHD and NeuralHD are known for their better learning efficiency, which mainly comes from the brain-inspired hypervector representation and operations. In Fig. 15, we compare the $\mathsf{HEAL}$ and $\mathsf{HEALneural}$ to their classifier backbones in terms of the learning curve. As for OnlineHD and NeuralHD, due to the lack of an AL framework, they will select random samples for annotation. The clear gap in the figure highlights the efficacy of the $\mathsf{HEAL}$ and underscores the advantages of integrating the existing capabilities of HDC with a tailored AL framework to further enhance learning efficiency.