Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separation

Recent theoretical works [5, 6] tackle OSR with theoretical guarantees on the performance but with specific distributional modeling assumptions (e.g. Gaussian mixture). [7] conduct theoretical studies in a more general setting by extending the classical closed-set PAC framework [8] to open-set environments, deriving analytical bounds of the generalization error in the context of OSR. [9] relates OSR to transfer learning and interprets the unknown class samples as covariate shifts. This enables the substitution of theoretical bounds derived in the transfer learning setting [10] to open-set environments. On the other hand, [11] observes that for a model trained only with known class samples, the magnitudes of representation vectors tend to exhibit relatively larger values over the known class than over the unknown ones. [12] empirically proved that the standard discriminative models detect unknown classes mainly based on their unfamiliar features rather than based on the novelty of unknown category.

For a general, broad survey on OSR models, the readers are recommended to [4, 13]. Here, we focus on reviewing state-of-the-art OSR models, mainly focusing on discriminative ones.

The basic baseline model [14, 1, 15] trained by softmax cross-entropy loss is known to perform both closed-set classification and unknown class detection reasonably effectively. To enhance its unknown detection mechanism, OpenMax [16] applied probabilistic modification on the softmax activation based on extreme value theory. DOC [17] replaced the softmax cross entropy with the one-vs-rest logistic regression, finding its effectiveness on invalid topic rejection in natural language. RPL [18] proposed to maximize inter-class separation in the form of reciprocal, followed by a variant [19] that utilizes synthetic, adversarially generated unknown class. CPN [20] learns embedding metrics by modeling each known class as a group of multiple prototypes. PROSER [21] leverages latent mixup samples [22, 23] as a generated unknown class and places their representations near the known class representations. [24], on the other hand, proposed a collection of multiple one-vs-rest networks to mitigate the over-confidence and poor generalization issue, and utilizes a collective decision score for effective OSR.

Recently, [15] demonstrated that the basic SCE baseline could outperform all other OSR baselines if the SCE model is trained with strong data augmentation and utilizes state-of-the-art optimization techniques. On the other hand, [25] showed that a prior on well separated discriminative embedding is still critical for effective open-set recognition.

Within the domain of discriminative learning, though not explicitly in the context of OSR, the Jacobian of the representation function has been examined in closed-set settings within various contexts. [26, 27] demonstrate that the explicit minimization of the Frobenius norm of the Jacobian of classification prediction output, specifically the softmax output and logit, promotes a smoothness prior on it, subsequently enhancing the generalization of recognition in closed-set scenarios. Nevertheless, the explicit computation of the Jacobian demands substantial computational resources. To address this, [28] has introduced an efficient method of computing the Jacobian norm via its random projection, serving as an unbiased estimator of the raw Jacobian norm.

[27, 29, 30] have noted that the smoothness prior, as enforced by Jacobian norm penalization, reduces the sensitivity of the network output to minute input perturbations, thereby making the network robust against adversarial examples. On a theoretical level, [31] has identified a close connection between weight decay and the Jacobian norm, establishing that under ideal conditions, a gradient update with weight decay equates to penalizing the Frobenius norm of the Jacobian matrix of representation.

However, all preceding studies on Jacobian analysis have been confined to the context of closed-set learning. To the best of our knowledge, our research represents the first instance of analyzing the Jacobian norm within an open-set scenario, providing a rigorous examination of its relationship to the unknown class.

During closed-set metric learning, the representation embedding function $\boldsymbol{f}:\mathbb{R}^{d}\to\mathbb{R}^{d_{z}}$ of a discriminative model is trained to minimize intra-class distances $\mathcal{D}(\boldsymbol{f}(\boldsymbol{x}),\boldsymbol{w}_{y})$ and maximize inter-class distances $\mathcal{D}(\boldsymbol{f}(\boldsymbol{x}),\boldsymbol{f}(\boldsymbol{x}^{\prime}))$ for known class samples $\boldsymbol{x}$ and $\boldsymbol{x}^{\prime}$ paired with different class labels $y$ and $y^{\prime}$ ($y\neq y^{\prime}$). The prototype vector $\boldsymbol{w}_{y}{\in}\mathbb{R}^{d_{z}}$ is a proxy  for the $y$-th known class $C_{y}$, and is formulated as a learnable parameter. The known set $\mathcal{K}{=}\cup_{k=1}^{K}C_{k}$ consists of $K$ disjoint disconnected known classes $C_{k}$. The train samples $\boldsymbol{x}$ and $\boldsymbol{x}^{\prime}$ are sampled from the known set, while the labels $y$ and $y^{\prime}$ from the corresponding label space $\mathcal{Y}_{\mathcal{K}}=\{1,\dots,K\}$. The open set $\mathcal{O}:=\mathcal{X}\setminus\mathcal{K}$ is the complement of the known set in the (bounded) global space $\mathcal{X}=[-1,1]^{d}\subseteq\mathbb{R}^{d}$. The unknown class $\mathcal{U}$ we consider is a proper subset of the open set $\mathcal{O}$. Since our task is not to discriminate within the unknown class, we treat the unknown class as a single class, although it may consist of a diverse type of object.

During training, the model has no access to the unknown class $\mathcal{U}$, and is trained only with the $K$ number of known classes to discriminate them. After training, the OSR model should not only discriminate each class in the known set but also need to differentiate the unknown from the known. Hence, the unknown class should be separated from all known classes in the representations space such that $\boldsymbol{f}(\mathcal{K})\cap\boldsymbol{f}(\mathcal{K})=\varnothing$.

We prove our theoretical claims by observing how the embedding function $\boldsymbol{f}$ changes on a class interpolating path (i.e., a path $\boldsymbol{\gamma}{:}[0,1]{\to}\mathcal{X}$ that interpolates two different known classes $C_{i}$ and $C_{j}$ by traversing $t$ from $0$ to $1$ with $\boldsymbol{x}_{0}{\in}C_{i}$ and $\boldsymbol{x}_{1}{\in}C_{j}$ as depicted in Fig. 1). The detailed assumptions and full proofs to the theoretical statements are given in A and B, respectively.

Firstly, we show that, during the closed-set supervision, the intra-class distance minimization minimizes the length of the projected path over the known class:

On the other hand, the inter-class distance maximization is presumed to increase the length of any linear path between the known classes $C_{i}$ and $C_{j}$ in the representation space by Assumption 2b. In summary, intra-class distance minimization reduces the projected path length, while the inter-class distance maximization increases the projected path length.

Now, the increasing/decreasing trend of the projected path length due to the metric learning is transferred to the Jacobian norm $\lVert\frac{d\boldsymbol{f}(\boldsymbol{\gamma}(t))}{dt}\rVert_{2}$ via the path length equation

Accordingly, we expect that intra-class distance minimization minimizes the Jacobian norm over the known class intersecting path. In contrast, inter-class distance maximization increases the Jacobian norm over the open set intersecting path. This description, however, is constrained to the local paths. The following theorem assures that this phenomenon is extendible from the local path to the global region. In other words, the closed-set metric learning minimizes the Jacobian norm over the known classes and increases the Jacobian norm over the open set $\mathcal{O}$.

Theorem 2b indicates that the length of the projected path can be accessed from the global integral of the Jacobian norm. Thereby, we find that the strictly increasing trend of Jacobian norm integral is positively correlated to the strictly increasing trend of the projected inter-class path length. Based on our overall observations, we deduct the below corollaries:

Hence, maximizing the inter-class distances between the known classes access to the unknown class samples indirectly via the region $S$ of high Jacobian norm, and increases the Jacobian norm of unknown class representations.

Overall, by metric learning, the model increases the expected Jacobian norm difference between the known and unknown

The increased Jacobian norm difference then separates the known classes from the unknown class in the representation space:

We highlight that the Jacobian norm characteristic is only one of the many explanatory factors that demystifies how closed-set metric learning derives OSR; our analysis does not fully characterize all connections between closed-set metric learning and OSR. One apparent phenomenon our theory does not explain is that known and unknown representations can be separated in the metric space with having the same Jacobian norm value. Moreover, our theory is limited in characterizing the support set $S$. As the support set does not include the whole part of the open set, there would be some unknown class that is not included in the support. In this case, the Jacobian norm difference indicated in Eq. (4) would not be explanatory.

Training a discriminative model over the known classes reduces the Jacobian norm over known class samples, while increasing the volume of region of high Jacobian norm in the open set. Due to the increased volume of high Jcaobian norm region, the unknown class samples likely fall into this region, and thus involve high Jacobian norm values. Overall, the embedding representations of known classes are separated from those of unknown class because the Jacobian norms of known classes are low while the Jacobian norms of unknown class are high. Our theoretical finding is summarized in Fig. 2.

We empirically analyze the relationship between the Jacobian norm difference and the unknown class detection to evidence our theoretical analysis. To this end, we train our proposed model as described in Sec. 5 and 6, and evaluate over the standard OSR benchmark datasets [14]. To compute the degree of separation between known and unknown, we use the detection score provided in Sec. 5.3 and evaluate the area under the receiver-operating-characteristic curve (AUC) metric [32]. The discriminative (cluster) quality of known class representations is measured in Davies-Bouldin Index (DBI) [33], which measures the ratio of intra-class distance to inter-class distance. All experiments are conducted with one 12GB GPU RTX 2080-ti. Due to resource limitations, empirical observations are made on standard OSR datasets rather than recently proposed high-resolution OSR datasets [15].

In practice, however, the inter/intra-class distance optimizations conflict; thus, the overall gradient norm increases for both the known and unknown.

Moreover, on some datasets (SVHN and TinyImageNet), the inter-class separation may not be substantial due to innate data characteristics such as small inter-class data variance. Accordingly, based on Theorem 2b, the weak inter-class separation induces relatively smaller difference in Jacobian norm between the known and unknown, resulting a larger overlap between them.

Although the global trend has a simple correspondence between these metrics, a more careful look at the graphs of Fig. 5 shows that the metrics involve different phases during training. Specifically, the intra/inter ratio is stable at the early stage of training. On the other hand, the inter-class distance is still increasing even at a later stage. The Jacobian norm difference rises more gradually, and the rate of increase becomes large at the last stage. The separation between the known and unknown also increases largely at the early stage but continues to improve even later in training. These observations show that the known and unknown class representations are separated as the model makes their Jacobian norm different. Still, the Jacobian norm is not the only factor contributing to their separation.

Fig. 6(1st row) shows that the degree of separation between the known and unknown strongly correlates to the Jacobian norm difference. This observation evidences our theoretical claim that the closed-set metric learning separates the unknown by increasing their Jacobian norm difference during training. There is, however, nonlinearity between these two metrics, showing that the Jacobian norm difference is not the only factor contributing to the separation of unknown class representation.

Fig. 6(2nd row) shows a similar correlation trend between the intra/inter-class distance ratio (DBI) and the Jacobian norm difference. However, the nonlinearity between them is severe. The plot indicates that the Jacobian norm difference abruptly increases at a later stage of training where the intra/inter ratio is already small and stable.

Our analysis indicates that the powerful inter-class separation is the key to separate the known from the unknown in the Jacobian norm and therefore in the representation space. Motivated upon this theory, we devise a marginal one-vs-rest (m-OvR) loss that induces powerful inter-class separation by preventing the collapse between inter-class prototypes $\boldsymbol{w}_{k}$ and effective inter-class gradients. The m-OvR loss is given by

where $(\boldsymbol{x},y)$ is a labeled sample, and $1\{\cdot\}$ is an indicator function. The class probability $p(k|\boldsymbol{x})$ is given by $\sigma(Ts_{k})$ where $\sigma$ is the sigmoid activation, $s_{k}$ is the cosine similarity between the representation $\boldsymbol{f}(\boldsymbol{x})$ and the $k$-th class proxy prototype $\boldsymbol{w}_{k}$, and $T$ is a scale term to calibrate the sigmoid probability.

During training, the bare minimization of the loss in Eq. (5) involves a harmful behavior; particularly, minimizing the loss in Eq. (5) collapses the inter-class prototypes as observed by below proposition:

This inter-class collapse weakens the inter-class separation. We mitigate this situation by inserting a margin in the similarity computation; namely, during the training of the OvR metric-learning loss, the similarity is computed by

where $m>0$ is the margin. The margin ensures an angular gap of degree $2m$ between inter-class prototypes, thus preventing their collapse:

In addition, the proposed m-OvR induces more powerful inter-class separation than the standard softmax cross-entropy (SCE) loss:

Therefore, m-OvR is more effective at increasing the Jacobian norm difference and, hence, the unknown class detection performance accordingly.

The empirical observations given in Fig. 8 indicate the effectiveness of m-OvR compared to the SCE loss in terms of Jacobian norm difference, discriminative quality of known class representations, and the unknown class detection performance based on the detector in Sec. 5.3.

Using the Jacobian norm principle from Sec. 3, we explain how the standard techniques (weight decay, auxiliary self-supervision, and data augmentation) improve the separation between known and unknown class representations, thereby improving the OSR performance. Our final model is combined with these techniques.

Our final metric-learning objective is to minimize the combined loss $\mathcal{L}+\lambda_{self}\mathcal{L}_{self}$ with data augmentation and weight decay.

To effectively detect unknown class samples during the inference stage, we utilize the sample-wise loss function. Based on our theoretical finding, the loss function is aware of both the Jacobian norm difference and the closeness to the known class prototype:

Hence, the loss function (1) differentiates the the known class representations in the low Jacobian norm region from the unknown class representations residing in the region of high Jacobian norm, and (2) separates the known class close to the prototypes $w_{k}$ from the unknown class instances. The positive correlation indicated in Fig. 9 vindicates the property of loss function described in Eq. (8).

Our method is evaluated for unknown class detection performance (AUC) and closed-set accuracy (ACC). The protocol used in [14] is adopted with the following benchmark datasets:

• 1.

  CIFAR10 and SVHN: Among the total ten classes, $K{=}6$ classes are chosen as the known ones, regarding the rest as a single unknown class. CIFAR10 [42] consists of generic object images while SVHN [43] of street view numbers.

• 2.

  CIFAR10+ and CIFAR50+: To make CIFAR10 more challenging, CIFAR10+ and CIFAR50+ are considered, in which $K{=}4$ known classes are selected from CIFAR10 while 10 (or 50) classes from CIFAR100 [42] constitute a single unknown class.

• 3.

  TinyImageNet: In TinyImagenet (TIN) [44] with more diverse categories, $K{=}20$ classes constitutes the known, while the other 180 remaining ones form a single unknown class.

A significant attribute of our methodology lies in the employment of our margin-based loss, m-OvR, which not only optimizes intra-class compactness but also ensures inter-class separation by circumventing inter-class collapse, as detailed in Prop. 7. This aspect renders our work as an improvement over the prevailing techniques such as RPL, CPN, and OvRN-CD, which predominantly focus on the inter-class aspects alone. Furthermore, our methodology incorporates carefully chosen subsidiary techniques, including weight decay, representation unit-normalization, and self-supervision through rotation prediction, which can efficaciously enhance OSR.

We note that our approach, even without the use of complex training tricks but solely utilizing the m-OvR loss, is comparable to the state-of-the-art GoodOSR. The pivotal differentiation lies in that GoodOSR boosts the OSR performance by excessive hyperparameter tuning and various cutting-edge training tricks, while ours is simply based on the loss function design.

On the other hand, if the weight decay coefficient $\lambda_{wd}$ is overly large, then it collapses the embedding to a constant (i.e., zero vector). At the same time, overly small $\lambda_{wd}$ has no impact as a regularizer. Finally, we remark that selecting a proper coefficient for the weight decay is not tricky by observing the train loss dynamic during the early stage.

As already remarked in Sec. 5.2, the rotation-based self-supervision auxiliary loss contributes positively only when its coefficient $\lambda_{self}$ is small (i.e., smaller than $1$). The unknown class detection performance is robust for the small values of $\lambda_{self}$.

In the 2-dimensional visualization of Fig. 14 obtained by applying t-SNE on the embedding representations of data samples, the known classes exhibit small Jacobian norm values while the unknown samples have larger Jacobian norm values. Moreover, the degree of distance-wise separation becomes high when the Jacobian norm contrast between the known and unknown classes is more vivid.