Part-guided Relational Transformers for Fine-grained Visual Recognition

Handling subtle differences using part-level local features has been widely studied in the past decades. Pioneer works [29, 17, 30, 31, 32, 20] tend to discover the part-level features with manual labels and amplify these local representations when obtaining the final features. For example, Zhang et al. [29] proposed a part-based R-CNN network, using part detectors for building pose-normalized representations. Krause et al. [33] proposed to detect the object part with an unsupervised co-segmentation and alignment manner, enhancing the final classification abilities. Huang et al. [17] integrated a part detection network in the recognition network and developed two streams to encode the part-level and object-level cues simultaneously. In addition, Wei et al. [31] proposed to learn accurate object part masks using an additional FCN and then aggregated these masked features with the global ones. Although promising results have been achieved by using strongly supervised annotations, labeling segmentation masks and accurate object parts are still time-and labor-consuming.

On the other hand, deep neural networks have the natural ability in discovering object parts [34] with only weakly supervised labels. Object parts would emerge automatically in the feature maps during the gradient learning process. Benefiting from these observations, recent works [35, 36, 37, 21, 22, 23, 38, 39] proposed to learn the part-aware features with auto-discovered part masks. Leading by this motivation, Simon et al. [36] proposed a local region discovery model using deep neural activation maps, constructing part-level attentions for final representation. Lam et al. [37] proposed to search object parts using heuristic function and successor functions. Wei et al. [40] proposed a flood-fill method which aggregated these features maps as holistic object activations. However, this operation only considers the extraction of object features, which does not reflect the part localizations. Peng et al. [41] proposed to select image patches as candidates to select the informative regions and extract the feature of these regions for complementation. Wang et al. [39] proposed to regularize the learned feature maps in a sparse representation manner and adopted axillary part branches for feature representation regularization. Moreover, Ge et al. [18] proposed a bottom-up architecture to obtain instance detection and segmentation via weakly supervised object labels. Combing these part proposals, a context encoding LSTM is further proposed to fuze the sequential part information for image classification. However, these works only considered the localization of part regions, while the contextual relationship of these local regions is less explored. In this paper, we propose to discover the discriminative parts and investigate their spatial relations in one unified framework.

The other family for solving the fine-grained visual categorization task is to regularize the feature learning process or formulating rich feature relationships. Inspired by the bilinear pooling operation [5], high-order relationship matrices [13, 7, 8, 42, 43] has been widely used for feature representations. For example, Yu et al. [13] proposed to learn the heterogonous feature interaction from different levels of feature layers, building compact cross-layer relationships. However, simply adopting the second-order feature would lead to a dimensional disaster for optimization, thus subsequent works proposed to use low-rank presentations [14], feature factorization [15] or Grassmann constraints [16] for building compact homogenous features. These works [14, 15, 16] greatly reduce the computation cost of high order matrices but still show comparable results. Besides the efforts on the second-order relationships, the recent network tends to explore the third-order relationships of feature channels. Leading by the non-local [44] design in building global relationships of different channels, several works [8, 9] have been proposed to explore the channel-wise relationship by attention mechanisms. Gao et al. [9] proposed to learn the channel-wise relationship by image-level relations and cross-image relations by a contrastive constraint.

Thinking the feature representation problem from another perspective, other works tend to explore the cross-layer relationships [45, 46, 10, 47, 6, 48] or auxiliary information [49]. Sun et al. [10] proposed a multi-attention multi-constraint (MAMC) to regularize the same class focusing on the same attention region. Based on this MAMC module, Luo et al. [47] proposed to learn relationships between different images and different network layers. In addition, Chen et al. [50] proposed a destruction and reconstruction framework to learn the local feature transformation rules for obtaining robust feature representations. Moreover, an attentive pair relation network [51] is proposed to fine the group-wise relationship inter-and intra-classes. Beyond these designs on network architectures, adopting pair-wise confusions [52] and maximizing the entropy [11] also show effectiveness in improving the final representation. Different from the aforementioned methods, in this work, we propose to find the feature embedding by two steps, i.e., discovering local parts and transforming for relational embedding, which incorporates the advantages of part discovery but also constructs reliable feature regularization.

Transformer architectures [24, 53] have achieved great progress in the field of natural language processing. Benefiting from its ability for long-term relationship construction, several works [54, 55] proposed to introduce this architecture based on the convention CNN encoders. For example, Carion etal [54] proposed to localize different object regions with multiple attention regions in transformer layers. Beyond these works, recent ideas [56, 57, 58, 59] proposed to directly encode the images patches, e.g., $16\times 16$ as the basic pattern to understanding images. As the representative work, ViT [56] built a strong pretrained transformer backbone, which greatly surpass the performance of conventional CNNs. However, this encoding manner also introduces huge computation costs. Several works proposed to introduce the shifted windows [57] and local encoding [58] to build hierarchical encoders, which exploit the advantages of local understandings in CNNs and also reduce computation costs. Based on these successful backbone encoders, utilizing vision transformers has also shown great benefits in semantic segmentation [60] and fine-grained visual classification [61]. Different from the TransFG [61] with ViT [56] encoders, our model builds relationships between high-level CNN features. In this manner, the local patterns with detailed features can be well represented and the contextual relationships are also constructed simultaneously.

Our proposed PART approach for fine-grained visual recognition is illustrated in Fig. 2. Our first key idea in exploring fine-grained recognition is to build contextual dependencies by the conjunction of successful Transformers with conventional Convolutional Neural Networks (CNNs). The second motif is to regularize the network embedding be aware of the local discriminative regions, which is desired for distinguishing visually similar samples. Beyond these designs, one global relation transformation and three local relation transformations are proposed to learn the crucial attention features by building regionally dense connections during the gradient descent process.

Given an input image $\mathcal{I}$, the general procedure in optimizing fine-grained recognition task is to optimize the target $\mathbb{E}$ between prediction and groundtruth label $\bm{\mathrm{y}}$:

where $\Phi(\cdot;\bm{\mathrm{\theta}})$ represents feature extractor $\Phi$ with learnable parameters $\bm{\mathrm{\theta}}$. $\bm{\mathrm{w}}_{c}$ represents the final classification projection vectors. Different from the general classification optimizations, our proposed network first generates two different feature embedding, i.e., $\bm{\mathrm{X}}^{p},\bm{\mathrm{X}}^{g}\in\mathbb{R}^{W\times H\times C}$. Thus we could localize the object part proposals $\bm{\mathrm{P}}$ with part discovery module $\mathcal{M}_{part}$, i.e., $\mathcal{P}=\mathcal{M}_{part}(\bm{\mathrm{X}}^{p})$, and construct high-order correlations with Transformers $\mathcal{G},\mathcal{T}$ for global and local branches.

Beyond these foundations, we thus make a general assumption that the object part proposals $\mathcal{P}$ share the same groundtruth labels $\bm{\mathrm{y}}$ with the global image $\mathcal{I}$. Hence the overall optimization in our framework can be presented as:

where $\circledast$ indicates that the latter term extract features from the former one. $\bm{\mathrm{w}}^{\{g,l\}}$ represent the global and local classification projections and $\lambda$ is the balanced weight. Hence our network introduces the relational transformation constraints and part localization constraints in fine-grained classification tasks. We will elaborate the details of these two modules in Section III-B and Section III-C.

Excavating discriminative features by localizing object parts has been widely explored by previous works [29, 17, 30, 31, 32, 20] with bounding box annotations or segmentation mask annotations. In our proposed PART framework, we propose to explore the object parts in a weakly supervised manner, which takes advantage of the class activation maps generated during the gradient descent process. Recent research [34] shows that object parts emerge automatically during the learning of classification tasks, i.e., during the training process, network features can be automatically regularized for excavating discriminative features and distinguishing near-duplicated objects.

Based on these findings, several recent ideas, e.g., RA-CNN [21], MA-CNN [46], propose to constrain these activation regions into part groups by their similarities. Different from these works, our part discovery module aims to provide discriminative region priors $\mathcal{P}$ for the transformer module. With these priors, the transformer module is proposed to mine the contextual object clues within each region. To achieve this, our proposed part discovery module follows two meaningful rules: 1) the highly activated features maps are attached with high priorities during the back-propagation process; 2) different activated parts should have the minimum overlap for discriminative part finding. These rules ensure CAMs are not restricted to local regions and enhance the generalization ability of deep models. Besides that, our part discovery module can be trained end-to-end in one unified scheme and does not cost additional computation costs for inference.

Given the extracted backbone features $\bm{\mathrm{X}}^{p}\in\mathbb{R}^{W\times H\times C}$, we define the class activation scores $\bm{\mathrm{s}}_{c}$ of the $c$th channel:

where $\epsilon$ is set to keep nonzero of the denominator. With this score as the ranking indicator, we thus rearrange the $\bm{\mathrm{X}}^{*}=\bm{\texttt{ActivationSort}}(\bm{\mathrm{X}}$, $\bm{\mathrm{s}})$ and build a part proposal stack to select the most important one from the stack top.

Starting from this part proposal stack, we first sample a random rate for each part $\eta_{c}\sim{N}(\mu,\sigma)$, enhancing robustness for threshold selection. For each selected feature $\bm{\mathrm{x}}^{*}_{c}\in\bm{\mathrm{X}}$, the part proposals $\bm{\mathrm{p}}^{*}$ are localized with the ROICrop operation:

The final bounding box regions can be automatically obtained by calculating the range of coordinates in the x-axis and y-axis. However, as an unsupervised part discovery method, different feature maps usually localize similar part regions, which may restrict the learning of discriminative features. Inspired by the Non-Maximum Suppression operation [62] in edge detection, we remove the redundant part proposals by calculating the Intersection over Union (IoU) of the current bounding box with the existing boxes in the part selection set $\mathcal{P}$. By repeating this process, all the legal part proposals will be automatically added into the final selection set until $|\mathcal{P}|\geq N$. Specially, we further apply an iteration terminator if all this part selection module could not find appropriate object parts. The detailed process is elaborated in Algorithm 1. With this proposed part selection module, our approach has the potential to find discriminative regional features automatically.

As mentioned in Section III-A, one crucial problem in fine-grained recognition is to build contextual dependencies for different semantic parts. However, limited by the design of 2D Convolutional Neural Networks, each feature unit has a restricted receptive field which is fixed by the convolutional kernels. To solve this problem, we propose to adopt the Transformer [24], which has been established as a state-of-the-art transduction model to take advantage of long-term correlation in semantic sentence understanding. Motivated by its succusses in natural language processing, we make an attempt to introduce it into the fine-grained recognition task, which is presented as the transformation operation in Eqn. (2).

Fine-grained recognition with visual transformers. The conventional transformers tend to take the input of a whole language sentence and building correlations of different semantic words, which are now used to replace the Recurrent Neural Networks (RNNs) in many NLP tasks to build long-term dependencies. Besides its superior performance in relation to modeling, embedding this architecture into existing CNN backbones, e.g., ResNet, VGGNet, is thus essential. In our PART framework, CNN backbones and transformers play different roles in fine-grained visual understanding. Solely adopting transformer models may result in a larger receptive field in relational modeling, but leads to a high computation burden for network optimization. In addition, keeping the main body of CNNs can also benefit from the pretrained common knowledge, e.g., ImageNet [63]. Hence in our approach, we propose to make a conjunction of conventional CNNs and the successful Transformer model. The detailed architecture is illustrated in Fig. 3.

Positional encoding. Another drawback in attention-based modeling methods is that structural information would be forgotten, especially when transforming CNN feature maps $\bm{\mathrm{X}}\in\mathbb{R}^{W\times H\times C}$ to multiple vectors. When dense connections are constructed, their original structural information could be easily lost. Inspired by [24, 54] in keeping position information, here we adopt the positional encoding $\bm{\mathrm{E}}$ into the original backbone features $\bm{\mathrm{X}}$. With the positional encoded features regardless of part proposals in Fig. 3, the key idea is to build global relation matrices $\bm{\mathrm{A}}\in\mathbb{R}^{WH\times WH}$, indicating the dense dependencies of each pixel unit to all the other pixels. Let $i,j$ be the positions of query and key values, this spatial correlation can be formulated as:

where $\bm{\mathrm{W}}_{\{qry,key\}}$ denote the learnable weight of query and key projections. Here we adopt the relative positional encoding of Transformer-XL [64] to build different positional encoding for different pixels. We build the 1D learning for relative encoding and 2D positional learning for absolute encoding. By expanding Eqn. (5) and using learnable $\bm{\mathrm{u}},\bm{\mathrm{v}}$, this relative encoding correlations $\bm{\mathrm{A}}$ can be presented as:

where key weights are spilt into $\bm{\mathrm{W}}_{key}$ and relative position $\widehat{\bm{\mathrm{W}}}_{key}$ respectively. $\bm{\mathrm{R}}$ denotes the sinusoid encoding matrix in [24] without learnable weights.

Multi-head attention with transformers. The relational matrices $\bm{\mathrm{A}}_{i,j}$ are multiplied with the original feature $\bm{\mathrm{X}}$, resulting in the enhanced feature of one head $\bm{\mathrm{H}}$:

As illustrated in Fig. 3, each transformer unit is composed of multiple transformer heads, i.e., $\mathcal{H}=\{\bm{\mathrm{H}}_{1},\ldots,\bm{\mathrm{H}}_{m}\}$ and $m$ denotes the number of attention heads. Employing multiple attention heads encourages different projections of query, key and value triplets to construct relational embedding. Then these attention heads are concatenated and fused with the feed-forward network, and the final output $\bm{\mathrm{O}}$ can be formally presented as:

where $\bm{\mathrm{W}}^{o}\in\mathbb{R}^{mC_{h}\times C}$ denotes the learnable weight for linear transformations, and $C_{h}$ denotes the dimension of one attention head.

Relations with CNNs. Based on this designment of relational embedding transformers, networks have the potential to learn the long-term dependencies between any two pixels. Notably, this attentive transformer layer and convolutional layers process 2D features in similar ways. In other words, the transformer layers are attached with more relational embedding learning matrices compared to the convolutional layers and in extreme cases, the transformers can be degenerated to simulate CNNs. The main theorem from [65] is as follows:

Detailed proof can refer to [65], which is abstracted in Appendix References. Theorem III.1 shows that the transformer modules have the potential to represent existing CNN layers but have the strong potential to explore cross-pixel relations. It means that the CNNs are “restricted” transformers that have fixed attentional embedding or limited receptive fields.

Part-guided relation embedding. As aforementioned in Eqn. (2), our proposed optimization framework aims to construct global relations as well as discovering discriminative part relations simultaneously. Benefiting from the strong relation embedding ability of transformer models, we propose to learn the part-level relations by the auto-discovered part in Section III-B. We first adopt the learned part masks $\bm{\mathrm{p}}_{i}\in\mathcal{P}$ as attentive masks before feeding into the transformer layers. Thus the learned architecture of each attention head can be presented as:

where $\odot$ denotes the Hadamard product for mask region selection. We thus concatenate the multiple heads of each part following Eqn. (8), forming output $\bm{\mathrm{O}}^{\bm{\mathrm{p}}_{i}}$ of each part $\bm{\mathrm{p}}_{i}$. After passing the mask selection operation, a dense connection within each local part is constructed, which discovers the spatial relations while neglects the unnecessary ones.

Moreover, the part-guided transformer units can be stacked in sequence, passing each enhanced feature $\bm{\mathrm{O}}^{\bm{\mathrm{p}}_{i}}$ into the next layer. This operation enlarges the reception field of each pixel and constructs strong correlations by mixing connections hops:

where $S$ denotes the number of stacked layers of transformer model $\mathcal{T}$. Similarly, the global relation embedding transformer $\mathcal{G}$ is defined in similar manners.

In our training scheme, we conduct a collaborative learning scheme of the global and $N$ part transformation branches in Fig. 2. Each branch is supervised with an individual softmax cross-entropy loss for minimizing target $\mathbb{E}$ in Eqn. (2). Thus our overall learning is composed of one global loss for contextual relation learning and $N$ part losses for local feature discovery. Note that our proposed PART approach resort to the $N$ auxiliary part branches for feature regularization in the training stage, whilst not relying on these parts in the testing stage. Moreover, simply aggregating these features would also lead to overfitting issues and inferior results. Compared to the ResNet backbone, we only introduce limited learnable matrices for relational embedding and conduct an end-to-end training with the conjunction of part discovery module and relational embedding module. Our framework greatly alleviates the computation burden and retains the discriminative part simultaneously.

To evaluate the effectiveness of our proposed PART approach, we conduct experiments on three public popular benchmarks, namely Caltech-UCSD Birds (CUB-200-2011) [1], Stanford-Cars [3], and FGVC-Aircrafts [4]. CUB-200-2011 dataset [1] contains 11,788 images of 200 wild bird species, which is the most widely used benchmark. Stanford-Cars dataset [3] includes 16,185 images of 196 car subcategories. FGVC-Aircraft dataset [4] contains 10,000 images of 100 classes of fine-grained aircrafts. We follow the standard training/testing splits as in the original works [1, 3, 4]. For evaluations, we use the top-1 evaluation criterion following previous works and only adopt the classification label for supervised training without any additional part annotations.

For the implementation of baseline and our proposed PART framework, we adopt ResNet-50 and ResNet-101 [67] networks pretrained on ImageNet [63] as our backbone for fair comparisons. We remove the last bottleneck in stage 4 to directly get the feature of 512 dimensions. We use the SGD optimizer with an initial learning rate of $8e-4$ annealed by 0.1 for every 60 epochs. We adopt the commonly used data augmentation techniques, i.e., random cropping and erasing, left-right flipping, and color jittering for robust feature representations. Our model is relatively lightweight and is trained end-to-end on two NVIDIA 2080Ti GPUs for acceleration. We use the group-wise sampler with a batchsize of 16, sampling 4 samples for each class. For hyper-parameter fine-tuning, we randomly select 10% of the training set as validation. The balanced weights $\lambda_{\bm{\mathrm{p}}}$ for multiple part loss functions are empirically set as 0.1. The channel dimension for feeding into relation embedding module is $C=512$ and the part branch number is set as 4, the IoU threshold is $th=0.6$. The stacked layer number $S$ is set as 3 for the global branch and 1 for the local branches. The training and testing protocol follow the state-of-the-art works [47, 50, 51] using random cropping of $448\times 448$ in training and only use one-crop during inference, which does not rely on the multi-crop features like previous works [21, 46, 23]. Notably, in the inference stage, we only use the global feature for accuracy reports, and the local branches can be simply omitted for acceleration.

In this subsection, we conduct experimental comparisons with state-of-the-art models on three representative benchmarks, i.e., CUB-200-2011 [1], Stanford-Cars [3], and FGVC-Aircrafts [4].

Comparison on CUB-200-2011 dataset. CUB-200-2011 is the most widely-used dataset, consisting of 200 bird species with visually similar appearances. The classification accuracy can be found in Tab. I. Following the aforementioned group ways in Section I, we divide the state-of-the-art models into two groups, i.e., feature regularized learning and part-based learning. It can be found that the earlier works, e.g., [33, 66] relies on the accurate part localization information, while achieving only 84.7% base performance. Hence with the development of auto-discovered part localization methods [46, 23], the classification accuracy has been increased by over 3% without any additional annotations.

On the other hand, feature-learning-based algorithms also show advantages in extracting informative knowledge and forming efficient representations. Hence our proposed PART approach is a joint learning algorithm with collaborative feature relation learning and part feature discovery. Combining these merits, our proposed method generates state-of-the-art results of 89.6% accuracy, which demonstrates the effectiveness of our proposed learning framework. Following API-Net [51], we extend our model with the deeper ResNet101 backbones, which achieves 90.1% and is 1.5% higher than [51].

Comparison on FGVC-Aircraft dataset. Similar to the benchmarking on CUB-200-2011 dataset, here we make comparisons on sub-categories of aircrafts in Tab. II. Recent approaches, e.g., DCL [50] and Cross-X [47], achieve performance of $93.0\%$ and $92.7\%$, which is much higher than the previous work [5]. To make fair comparisons with these works, we adopt the lightweight ResNet-50 backbone to achieve the state-of-the-art performance of 94.4%, while previous works rely on deep backbones for feature extraction, e.g., [9] with ResNet-101 and [52, 11] with DenseNet-161. In addition, other works [21, 46, 23] adopt the multi-crop inference to obtain multi-scale features, introducing additional computation burdens. Different from these mentioned issues, our proposed PART approach presents robust features with ResNet-50 backbone and one-crop inference.

Comparison on Stanford-Cars dataset. Tab. III exhibits the results on the Stanford-Cars dataset. Stanford-Cars is a much easier dataset, in which previous methods achieve preliminary results of over 92.5% accuracy. It can be found that methods on this dataset perform very similar performance, e.g., TASN [8] and HBP [13] of 93.7% performance. While obtaining multi-level feature representations [47] and using separate layer initialization strategies [6], high performance as 94.5% can be achieved. Surprisingly, even on this dataset, our method can provide a steady improvement compared to state-of-the-art results, reaching 95.1% performance. With deep backbones i.e., ResNet101, the performance shows a slight improvement to 95.3% in top-1 accuracy.

In this subsection, we first conduct ablation studies of our proposed different modules, and study the manner of positional encoding in relation to transformation modules. Then we make a detailed analysis of the part effects in our framework and finally make explanations using visualized comparisons.

Ablation studies on different components. To evaluate the effectiveness of our proposed part discovery module and relation transformation module, we conduct detailed ablation studies on three representative benchmarks [1, 3, 4]. In Tab. IV, the first line shows the baseline performance on these three benchmarks, using the same data augmentation and training hyper-parameters as our final model. It can be found that our baseline model surpasses many earlier works and our proposed modules, i.e., relation transformation module $\mathcal{M}_{Relation}$ and part discovery module $\mathcal{M}_{Part}$ can improve the recognition ability steadily, e.g., from 85.4% to 89.6% (4 parts) on CUB-200-2011 dataset. In the second row, we only adopt the relation transformation module with stacked manners, and set the head number of multi-head attention as 2. It can be found that the performance has increased notably by incorporating the part-level features. This verifies that the effectiveness of our proposed module, even on the high-performance baseline. Moreover, when setting the number of attention heads as 4, performance on these three datasets can be slightly improved.

Effects of feature encoding. We first present the performance of the baseline method in the first two rows on the CUB benchmark dataset. The first row indicates that without the group sampler in Section IV-B of a class-wise balanced manner, baseline model would result in a performance drop of over $1\%$. As a natural drawback in building contextual dense relations, the spatial structure information is lost during the vectorization process. The result (with 2 attention heads) without positional information can be found in the third-row of Tab. V. By incorporating the absolute positional encoding operation (sinusoid encoding function in [24]), the recognition accuracy has been increased from 87.5% to 88.1%. In this paper, we adopt relative encoding which uses learned encoding parameters to encode the unique positional information for each pixel. The result can be found in the fifth row, which also has the potential to simulate any convolutional layers as in Theorem III.1. And finally our full model with the part discovery module achieves the best performance in the last row.

Effects of part discovery. To evaluate the effectiveness of the part discovery module, here we exhibit different hyperparameters in the part discovery module in Tab. VI. In our proposed algorithm, the part selection is mainly affected by two hyper-parameters, i.e., the capacity of the part stack $|\bm{\mathrm{X}}^{*}|$ and the final selected part number $|\mathcal{P}|$. The selected part number also affects the network architecture, deciding the number of regularized part-level branches. When applying no auxiliary part branches, in Tab. VI, our proposed approach with only transformer modules achieves the performance of 88.2%. With the increase of selected part number $|\mathcal{P}|$, auxiliary supervision is also automatically added into the overall framework. It can be found that our approach with more parts ($|\mathcal{P}|=4$) reaches a high performance of 89.6%. This indicates the effectiveness of using auxiliary part branches as regularization. However, with the part numbers $|\mathcal{P}|$ increasing to be 5, the final performance decreases to 88.8%. Discovering more parts would incorporate meaningless background regions into computation and also restrain the learning of global features, which jointly lead to an inferior response on class activation maps. Moreover, when squeezing the capacity of the part stack $|\bm{\mathrm{X}}^{*}|$, the part discovery module would also lead to similar performance, which verifies the robustness of our proposed method.

During the inference phase, we only rely on the global branches and drop the redundant part features. We also conduct experiments of direct concatenating these branches rather than using them as regularization. The performance of condition $|\mathcal{P}|=1$ in the first row of Tab. VI drops to 88.5%. Assuming the same classification task of 200 classes, the overall parameters of ResNet50 is 25.6M. Comparing to the full model with all 4 parts of 32.9M, our final model only requires 28.7M parameters for learning, which is implementation-friendly.

Visualized explanations. We further investigate the class activation maps using state-of-the-art Grad-CAMs [68]. As shown in Fig. 4, we present the summation of all channels of baseline model (ResNet50) in Fig. 4 a), and our proposed PART in c). Interestingly, the baseline model can easily fall into local attentions to distinguish the visually similar samples, which leads to inferior generalization capabilities. While our proposed PART can localize the full object, resulting in a robust recognition ability. Moreover, we visualize the referred channels of our model in Fig. 4 b). It can be found that different channels focus on different semantic parts, forming the final robust representations. Although these “parts” do not strictly meet the definition of natural semantics, while still responding to meaningful object regions.