Indirect-Instant Attention Optimization for
Crowd Counting in Dense Scenes

This strategy emphasizes that targets at different scales need to be perceived by perceptual fields of different sizes, and is generally implemented using a multi-column convolutional architecture.

MCNN [14] first uses a three-column network architecture to extract multi-scale features to accommodate scale variations due to different camera angles. Inspired by [14], Switching-CNN [15] retains the multi-column structure, but adds a classifier to select the best branch suitable for the current scale. In addition, for multi-scale feature fusion, it is locally adaptive rather than the global fixed strategy in [14]. CSRNet [16] adapts a dilated convolutional layer to increase the receptive field as an alternative to the pooling operations, but it tends to lead to grid effects, further leading to local information loss. DADNet [17] is dedicated to innovation in multi-scale feature fusion, which advocates the use of multi-column dilation convolution to effectively learn multi-scale visual contextual cues, containing both the multi-column idea in [14] and the essence of dilation convolution in [16]. Unlike the above methods, AMSNet [18] utilizes neural architecture search and introduces an end-to-end search encoder-decoder architecture to automatically design the network models.

Attention mechanism is activated by the sigmoid function, which directs the model to focus on regions where the signal response is obvious and suppresses background noise, thus acting as a top-level supervision.

SFANet [12] trains attention as a separate task pathway, similar to W-Net [19]. However, its attention label are generated by density map paired with artificial threshold, which is not robust enough to generalize to scenes with different densities. ASNet [20] considers the density of different regions in an image varies widely, which leads to different counting performance, and thus proposes a density attention network. This method provides attention masks with different density levels for the convolutional extraction unit, which is an aid and also a supervision. RANet [21] emphasizes the optimization of attention, using two modules to handle global attention and local attention separately, and then finally fusing them according to the interdependencies between features. PCC-Net [22] encodes the global, local and pixel-level features of the crowd scene separately and processes them in separate tasks. One of the FBS modules is responsible for segmenting the head region and background to further eliminate erroneous estimates. This fuels the accuracy of the prediction,but the training cost is high. DADNet [17] innovatively proposes scale-aware dilated attention, which uses multi-column dilated convolution to obtain attention maps responsible for different receptive fields, and thus focuses on heads with different scales, reproducing scale-awareness in a new way.Also, considering the adaptability to complex issues of scale variations, feature fusion is handled by direct summation instead of concatenation. Recognizing that it is often difficult to generate accurate attention maps directly, CFANet [23] turns to a coarse-to-fine progressive attention mechanism through two branches, the crowd region recognizer (CRR) and the density level estimator (DLE). This mode both suppresses the influence of background to reduce misidentification and adaptively assigns attention weights to different density regions. SGANet [24], the underlying structure is similar to SFANet [12]: the attention map is homologous to the initial feature map, and they both belong to the dual path task. However, the former improves the quality of attention due to the introduction of the tuned inception-v3 [25] as Encoder, which improves the model effect tremendously. Attention determines the authority of top-level supervision, and this method improves the decision-making ability of attention from the root.

SASNet [26] applies softmax to the traditional convolutional neural network, using it to rescale the attention weights, multiply them with the high-dimensional feature maps, and select the highest scoring hierarchical feature map among the resulting feature fusion maps. Transformer [27] uses softmax to guarantee the non-negativity of the matrix and local attention amplification. However, its time complexity is the square of the sequence length, resulting in excessive overhead. cosformer[28] replaces non-decomposable nonlinear softmax operations with linear operations with a decomposable nonlinear reweighting mechanism, which not only achieves comparable or better performance than softmax-attention across a range of tasks, but also has linear space and time complexity. [29] introduces softmax into network pruning. Softmax-attention channel pruning consists of training, pruning, and fine-tuning steps. In the training step, it trains the network to be pruned; in the pruning step, it uses softmax to determine the importance of each channel and remove the relatively unimportant channels; in the fine-tuning step, the pruned model is trained with the same epochs used in the training step above [29].

Similar to previous work [12], we place the front thirteen convolutional layers of VGG-16 [30] with four pooling layers in an encoder for extracting low-level features, such as edges and textures. Its output map is then upsampled spatially by a factor of $2$ using bilinear interpolation. Immediately afterwards, the upsampled map is merged with the feature map obtained by the third convolution via channel-wise concatenation. Finally, the merged feature map is convolved through a $1\times 1$ layer to obtain $F_{in}$ as the input of the first IIAO module, where convolution is used to reduce the aliasing effect due to upsampling [31]. Hence, the generated $F_{in}$ spatial size is $8$ times smaller than the original input. Note that the resolution of $F_{in}$ no longer changes during all subsequent processing before the last single convolutional layer.

As shown in Figure 3, the Indirect-Instant Attention Optimization (IIAO) module consists of two main components: the Adaptive Scale Pyramid (ASP) submodule and the SoftMax-Attention (SMA) submodule. As mentioned in Section III-A, $F_{in}\in\mathbb{R}^{C\times H\times W}$ is an input to IIAO module, where $C$ denotes the number of channels, and $H$ and $W$ represent the height and width, respectively, which are 8 times smaller than those of the original image. $F_{in}$ will produce two different types of feature maps each time it passes through the IIAO module: $F_{out}$ continues to pass backward; while $F_{wei}$ fuses attention directly with the ground-truth map for loss calculation.

Adaptive Scale Pyramid (ASP) submodule utilizes a multi-column architecture to acquire multi-scale features, while automatically completing the multi-scale feature fusion task at the channel-level along with the generation of $F_{wei}$ (see the next submodule for details), alleviating the limitation where the receptive field of each branch is fixed within a certain range. In detail, to reduce the parameter overhead, we conduct a $1\times 1$ convolution filter at the beginning of ASP to compress the dimensionality of $F_{in}$ to $1/4$, and then expand it into four branches, each containing two of the three sizes of convolution filters $1\times 1$, $3\times 3$, and $5\times 5$. In accordance with the design pattern of feature pyramid [13], the further down the branch, the larger the perceptual field, so as to build a multi-scale perceptron to obtain contextual information. In each branch, the first convolution filter reduces the dimension by another factor of $4$ , and after collating the feature information, the second convolution filter recovers it, at which point the dimension of $F_{in}$ in each branch is $[C/4,H,W]$. Along these lines, the $F_{mul}$ generated by the ASP submodule has exactly the same dimensions as the original $F_{in}$. Since there is no message passing or parameter sharing in ASP , $F_{mul}$ at this time just incorporates features from multiple scales without fusion or selection, which is essentially an immature aggregate. We defer this work to the SMA submodule, whose process of generating $F_{wei}$ also involves the model learning the dynamic weights of each location in $F_{mul}$ online for all its channels, and finally using this weight, which represents the critical degree of information, to complete the adaptive multi-scale feature fusion task.

SoftMax-Attention (SMA) submodule receives $F_{in}$ from the residual connection, and the Transitive Attention Unit provides it contextual attention to obtain $F_{att}$. Specifically, to facilitate arithmetic and extract feature information of diverse levels, the Transitive Attention Unit first uses $1\times 1$ convolution filter with ReLU activation function to reduce the channels of $F_{in}$, and obtain $F_{in}\in\mathbb{R}^{C/r\times H\times W}$, where $r$ is the hyperparameter, specifying the reduction ratio. Then use another $1\times 1$ convolution filter to restore it and modulate with the sigmoid function to get the global context attention, denoted by $F_{att}\in\mathbb{R}^{C\times H\times W}$. At this node, $F_{att}$ splits into two paths, the first path replicates the traditional attention mechanism [12], applying element-wise multiplication to $F_{att}$ and $F_{mul}$, thus playing a supervisory role to enhance the key information in $F_{mul}$ and suppress its background noise. The result of multiplying the two is represented by $F_{out}$, which is still identical to the shape of $F_{in}$, and is then transmitted to the subsequent network sections.

After the last IIAO module, $F_{out}$ is regressed by a $1\times 1$ convolution filter to generate the final prediction map $F_{pre}$.

Crucially, if we pursue timely and reliable attentional supervision, we should start the loss and gradient computation immediately after multiplying $F_{att}$ by $F_{mul}$, while keeping all its parameters unchanged. However, at this time, $F_{att}$ is generally in a high-dimensional state, and the corresponding pseudo-label can only be one-dimensional. Therefore, before calculating the loss, we must first reduce the dimensionality of $F_{att}$. To average it would mean that all feature layers have the same weight, the key information and environmental noise would be neutralized, which is contrary to the original intention of the attention mechanism, whereas if convolutional dimensionality reduction is used, additional learnable parameters will be introduced, which makes it impossible to know whether the effect of the model stems from the attention supervision or the convolution learning during the process of dimensionality reduction.

Based on the above analysis, we propose a SoftMax-Attention strategy to optimize the loss calculation process of attention in the second path. This approach removes pseudo-labeling and does not introduce additional learnable parameters, enabling loss and gradient computation to correct the kernel parameters immediately after acting as a supervisor. We place this device halfway through the network multiple times to ensure that the convergence of the network is always on track. To expand, after each $F_{att}$ is taken from Transitive Attention Unit, it is transformed into a probability distribution between $[0,1]$ in the channel direction using the softmax function, with the aim of learning the dynamic weight of the feature expressed by each pixel in $F_{mul}$ across all channels at that location. Then, $F_{mul}$ is multiplied by $F_{att}$, noting that the result is different from $F_{out}$. All channels are summed vertically to obtain the weighted density map of fused attention to features, denoted by $F_{wei}\in\mathbb{R}^{1\times H\times W}$.

Up to this point, the body of the attention loss calculation is mathematically transformed from a high-dimensional attention map to an ordinary one-dimensional feature map, $F_{wei}$, because what each of its pixel values represents is, again, the number of heads at that location. We know that 1D feature map labels are easy to produce and relatively reliable, so this special transformation is feasible and meaningful. This is an indirect way of correcting parameters in high-dimensional attention that are difficult to optimize; at the same time, this correction occurs just after the attention map has completed its supervisory role, so it is also an instant way. Combined, this is called Indirect-Instant attention.

Another thing to note is that this device allows the network to learn the weights of each channel under each branch, so the SoftMax-Attention acts on different feature branches. And different branches have different perceptual capabilities for features of different scales, so the process to get $F_{wei}$ also completes the task of multi-scale feature fusion for the ASP submodule.

To summarize, the SoftMax-Attention strategy enables the attention map to pre-emptively complete loss and gradient computation in an indirect way midway through the network, thus addressing two defects in the attention loss calculation, while it provides adaptive multi-scale feature fusion service for the ASP submodule. So this is a simple but efficient approach. Ablation experiments demonstrate that the SoftMax-Attention strategy achieves better results than averaging or convolutional dimensionality reduction methods.

SASNet [26] also borrows the softmax algorithm, which utilizes VGG-16 [30] network regression to obtain 5 layers of Confidence Head and Density Head. The two are then upsampled to the same size and each is concatenated to obtain Confidence Maps and Density Maps. Finally, the former is softmaxed and multiplied with the latter, and then summed by dimensional direction to get the final prediction map. It can be seen that the basic action is similar, but we note that the features obtained by this treatment may not be smooth enough, and saturated training on it is not the only choice, so instead of using it as the final estimates , we place it in the middle of the network and target the optimization by introducing RCLoss that focus on error-prone regions. In addition, the concatenation operation requires undifferentiated upsampling of multilayer feature maps with a maximum ratio of up to 16, which severely blurs the information at deeper levels and thus puts it at a disadvantage in the scale selection stage. In contrast to rough processing, we focus on the particular advantages of softmax dimensionality reduction. Stacking IIAO modules to progressively improve attentional reliability and guiding feature pyramid regression to learn prediction maps for high ratings. Also, feeding high-resolution patches alleviates the problem of feature extinction in overly deep network.

Euclidean Distance. The vast majority of researchers tend to choose the euclidean distance to measure the pixel error between the final prediction map and the ground-truth density map, which is defined as follows:

Where $N$ is the number of images in a training batch, $X_{i}$ denotes the current training image, $\varTheta$ is a set of learnable parameters in the network, so $P\left(X_{i};\varTheta\right)$ represents the prediction map for it, and $G_{i}^{GT}$ refers to its ground-truth density map.

Regional Correlation Loss. Realistic scenes with different densities of people are unevenly distributed, and there may be multiple spatially uncorrelated error-prone regions in a single image, but MSELoss assumes that the pixels are isolated and independent from each other, and spatial correlation cannot be guaranteed. In addition, although the $F_{wei}$ generated by Soft Block is mathematically equivalent to the ordinary density map, the internal data are coarser due to the specificity of the transformation method. To cope with above problem, we innovatively propose the Regional Correlation Loss function (RCLoss) and apply it to the IIAO module, the pairing of the two can optimize attention more effective. At the end of the network, the more general MSELoss is used to act on the final prediction map, which is actually a review of the effectiveness of the IIAO module paired with RCLoss. The computational flow of RCLoss is shown in Figure 4.

RCLoss focuses on using the $F_{wei}$ generated by the SMA submodule, subtracts it from the ground-truth map and obtains the absolute value, where the result is represented by the Error map. Set the sliding window to traverse the Error map, find the pixel point with a large error in each of the obtained sub-windows, and apply an incremental penalty based on the error value of the position itself, i.e. , the strength of the penalty is determined by the degree of error in that position. To ensure information continuity between sub-windows, we set the sliding stride slightly smaller than the sliding window size. In this way, the resulting overlapping regions will be analyzed multiple times, but only extremely error-prone positions will be repeatedly penalized, and low-sensitivity areas are simply ignored.

In detail, the Error map, denoted by $E\in\mathbb{R}^{1\times H\times W}$, is obtained first.

Assuming that the size and stride of the sliding window are $k$ and $s$, respectively, then the maximum sliding times of the window in the horizontal and vertical directions can be calculated, which are represented by $R_{max}$ and $D_{max}$, respectively.

Each sub-window has its pixel point with the largest value of its error, called the hard point, whose value is $MAX_{r,d},\{1\!\leq\!r\!\leq\!R_{max},1\!\leq\!d\!\leq\!D_{max}\}$. By analyzing each sub-window in turn, penalizing positions where the error value is close to $MAX_{r,d}$, the amount of penalty is determined by the constant $threshold$, and the strength of the penalty is related to the degree of deviation from its own prediction. For those points where the error value is within the tolerance range, MSELoss is used as their loss . The specific RCLoss algorithm is shown below:

$Loss_{r,d}$ represents the total loss of each sub-window, and $i$ and $j$ are iterators of its width and height, respectively. Within each sub-window, first determine the hard point and calculate its error value $MAX_{r,d}$; then, search for error-prone points whose error value is greater than the product of $threshold$ and $MAX_{r,d}$; and finally calculate the loss together with the rest of the ordinary positions according to Equation (6) . $EP$ and $ET$ represent error-prone and error-tolerant points , respectively.

All reachable sub-window losses in a training batch are accumulated to give the final RCLoss:

Final Loss Function. The entire network produces three density maps of the same size, including two $F_{wei}$s output from the IIAO module and a final prediction map $F_{pre}$. By weighting the different tasks, the unified objective function required by training can be formulated as:

where $\lambda$ and $\gamma$ are the weight terms of the two loss functions. Both can be set to fixed values for experiments with all datasets, showing an excellent generalization ability.

SASNet[26] also mentions the concept of hard pixel, but it is quite different from this paper, which can be summarized in three aspects. 1) It cuts the prediction map into four parts, selects the one with the largest total error, and then recursively cuts and compares until the pixel with the largest error in the entire prediction map is determined, which is called hard pixel. By contrast, we consider that the probability of large-scale human heads appearing on the dividing line is high, and forcible cutting will cause irreversible damage to potential targets and affect the prediction of the network. We therefore propose sliding windows with overlapping regions and perform ablation experiments for the overlap length, which is finally determined to be $8$. Since the final prediction map has the same 8X perceptual field, the mapping back to the original map is a $64\times 64$ region, which is close to the size of a larger human head. 2) The number of penalty objects is different. Our hard point is not the only object to be punished, but plays a more important role as a reference. Each of the four sub-windows has its own hard point, which indicates the position with the largest prediction deviation in the current sub-window. Suppose its value is MAX, search for points in this sub-window whose error value is close to MAX, treat them as penalty objects as well, and whether they are close or not is evaluated by threshold. The purpose of this move is to take into account the uneven distribution of head density, where errors may be concentrated in a certain area. Restricting to hard pixel may have limited effect. 3) The intensity of penalty varies among different penalty objects; after all, their degree of prediction deviation is different. The hard point, as the biggest troublemaker, undoubtedly receives the most attention; for its affiliated penalty points, the penalty intensity is softened with the decrease of the error . See Equation (6) for specific rules.

In the training phase, we randomly crop a $400\times 400$ patch and flip it horizontally with a probability of $0.5$. For the ShanghaiTech Part_A[14] and UCF-QNRF[32] datasets containing grey images, we change the color images to grey with a probability of $0.1$. During the test phase, for the datasets containing extremely large resolution, i.e., UCF-QNRF[32], JHU-Crowd++[33] and NWPU-Crowd[34],we scale images with side lengths greater than $5000$ to $4/5$ of their original size.

As in previous work [12], we blur each head annotation with a Gaussian kernel to generate training labels. In detail, for crowd-sparse datasets, such as ShanghaiTech Part_B [14], we use fixed-size kernels to generate ground-truth, while for other datasets with relatively dense scenes, geometric adaptive kernel based on the nearest neighbor algorithm are utilized.

Except for Primary Feature Extractor, the parameters of the subsequent layers are randomly initialized by a Gaussian distribution with a mean of $0$ and a standard deviation of $0.01$. The $r$ in Transitive Attention Unit is set to $16$. For training details, we choose the Adam [35] optimizer to retrain the model, with an initial learning rate of 1E-4, halved every 100 rounds. Weight items $\lambda$ and $\gamma$ are set to $1.5$ and $0.5$, respectively, in the training of all datasets.

There are two mainstream metrics for evaluating the performance in crowd counting task: Mean Absolute Error (MAE) and Mean Squared Error (MSE). They are defined as follows:

where $N$ is the number of images in the test set, and $P_{i}$ and $g_{i}$ are the predicted number of targets in the $i$-th image and its corresponding ground-truth number, respectively.

ShanghaiTech dataset is composed of two parts. Part_A contains 482 images, sourced from the web, with rich scenes and variable scales, and the number of images used for training and test are 300 and 182, respectively; Part_B contains 716 images, collected from Shanghai streets, with relatively sparse distribution and a fixed image size of $1024\times 768$, further divided into a training set containing 400 images and a test set containing 316 images. The experimental results are shown in Table I, which indicates that our method achieves the desired level in both dense and sparse scenes.

UCF_CC_50 dataset is a small but challenging dataset, containing 50 images collected from the internet. The number of people annotated in each image varies widely from 94 to 4,543. Due to the limited samples, there is no official division between the training and test set, but rather a 5-fold cross-validation is suggested as a unified test approach. We strictly follow this rule for our experiments, and the results are also shown in Table I. It can be seen that our method is 8.33% ahead of the SOTA level in terms of MAE metric, and it is 13.8% ahead in terms of MSE. Thus, our method still performs well in extremely dense scenes.

UCF-QNRF dataset has 1535 images containing 1,251,642 annotation points. The training and test sets are composed of 1201 and 334 images, respectively. The density of annotated targets ranges from 49 to 12,865, and the average image size is $2013\times 2902$, which poses an enormous challenge for counting task. As shown in Table I, we achieve improvements of 4.26% and 2% in terms of the MAE and MSE , respectively, compared to the SOTA methods.

JHU-Crowd++ dataset has a more complex context than the above datasets, with 4372 images and 1.51 million annotations. All images are divided into 2722 training images, 500 val images and 1600 test images. We follow the official description, classify val and test by density level, and compare them one by one. The results of both are shown in Tables II and III. In the val set, both metrics in our Overall and Medium parts outperform the previous SOTA method by 6.11%, 7.19%, 9.13%, and 6.88%, respectively. In the High part, the MAE trails that of the optimal method by 0.97%, and MSE leads by 4.84%. More notably, for the test set, we have made progress across the board in both High and Medium parts, leading the second place by 9.63%, 5.82%, 6.13%, and 1.57% in the two metrics, respectively. Also, for the Low part, MSE surpasses the previous SOTA method by 27.58%, which is a great improvement. Unfortunately, however, in the Overall part, both metrics lag behind the MAN [46].

NWPU-Crowd is currently the most challenging dataset in the field of crowd counting. It includes 5109 images and 2.13 million annotation points. The training set, val set and test set contain 3109, 500 and 1500 images, respectively. This dataset introduces 351 negative sample images, i.e., unoccupied scenes. The maximum number of targets for a single image up to $20033$, the average size is $2191\times 3209$, and the maximum size reaches $4028\times 19044$, which requires high computing power. As can be seen from Table IV, we rank first in the val set of the published methods. More critically, for the test set, we have surpassed this year’s SOTA method by 3.28% on the MSE metric, but are 7.3% behind in MAE compared to it. Detailed, for different scene $(S0\sim S4)$ and luminance $(L0\sim L2)$ categories, we are 18.94% and 7.5% ahead of the previou SOTA method in $S4$ and $L2$, respectively; in $S0$, we also reach the optimal level, outperforming the second place by 14.89% .

To explain the excellent performance of our network in the above benchmark datasets, we dissect in detail the contribution of its basic components and hyperparameters to the overall . In this section, we first observe the huge gains brought by the SoftMax-Attention strategy to the network and the change in gains when stacking different numbers of IIAO modules, then discuss the effectiveness of RCLoss and $threshold$ , and finally explore the necessity of sliding windows with overlap for RCLoss. To be fair, all the following experiments are conducted on the JHU-Crowd++ dataset (val set).

Superiority of the SoftMax-Attention Strategy: We promote SoftMax-Attention and experimentally verify its advantages. The first variant is the baseline that discards the Soft Block, which can also be expressed as: baseline= Primary Feature Extractor + Adaptive Scale Pyramid + Transitive Attention Unit , so the baseline does not compute losses on $F_{att}$, but allows it to learn itself during gradient back-propagated. Next, three sets of control experiments are then set up according to the different methods of dimensionality reduction for $F_{att}$, in the following order:

1. 1.

   Averaging $F_{att}$ by channel direction.

2. 2.

   Setting $1\times 1$ convolution to reduce the channel to 1.

3. 3.

   Using the SoftMax-Attention strategy.

To control variables consistently, all experiments up to now, including baseline, only use MSELoss. As shown in Table V, all indicators in the three control trials showed a decreasing trend. However, Group 1 is less effective than baseline, which is to be expected since it defies the will of the attention mechanism. Both Group 2 and Group 3 are ahead of baseline, while the lower counting error of the latter confirms our hypothesis that there is indeed room for optimizing the loss calculation process , and our proposed IIAO module based on SoftMax-Attention plays a key role. Not negligibly, Group $\ell$ optimized for $F_{wei}$ using RCLoss achieves better results, suggesting that RCLoss is the icing on the cake. To intuitively demonstrate the improvement achieved by SoftMax-Attention relative to the baseline, we select typical samples from each dataset, see Figure 5. Further, to observe the overall prediction performance of each experimental group on JHU-Crowd++ dataset(val set) , we plot PRE-GT scatter diagram with regression line containing all samples in this dataset, and the auxiliary line $y=x$ represents the ideal case where the counting prediction accuracy is 100%. The aggregated results are shown in Figure 7. Qualitatively, the tightness of the regression line to $y=x$ symbolizes the quality of prediction (positive correlation); quantitatively, the closer $R^{2}$ is to $1$, the smaller the overall error fluctuation.

Influence of stacking number of IIAO modules: As aforementioned, IIAO can optimize attention and improve model performance, but the following experiments illustrate that more stacks may not be better. For a more comprehensive consideration, in addition to the necessary MAE and MSE , we also add the monitoring of three indicators Param size, GFLOPs and Inference time, which are closely related to the practical value. Table VI shows the performance of the network when stacking different numbers of IIAO modules: the MAE and MSE have converged when the number is $2$, and the other metrics are within acceptable zone at this point. Moving on, excessive intervention leads to over-smoothing of features, while the redundant parameters put a great strain on the computer hardware. Note that to avoid the impact of inconsistent image sizes or other accidental factors on the inference process, we perform $1000$ consecutive forward inferences on a $3\times 400\times 400$ patch, and finally take the average time as the Inference time.

Effectiveness of the RCLoss and $\bm{threshold}$: Regional error-proneness varies by density distribution, while the proposed RCLoss can provide region-dependent loss penalties. To verify its validity, we replace RCLoss with MSELoss, consider as the benchmark; immediately after, revert to RCLoss and adjust the value of $threshold$ several times as a multi-group control trial. Figure 6 visualizes the performance fluctuations caused by RCLoss on the model. To expand on this, if the $threshold$ is too small, more regions are penalized and even adjacent regions are linked, making it difficult to highlight high-frequency error locations and thus converging slowly. Conversely, if too large, the penalty condition becomes so harsh that the RCLoss will infinitely degenerate into MSELoss. Furthermore, the precision requirements for floating-point numbers become excessively stringent. From the curve, for the JHU-Crowd++ dataset (val set), it is optimal when using RCLoss and setting the $threshold$ to $0.95$. Notably, the density level varies from one dataset to another, so we add experiments to the respective optimal values of $threshold$ under different datasets, refer to Table VII.

Necessity of the Sliding Window in RCLoss: The crowd density in the real scene is unevenly distributed, and there may be multiple error-prone regions in a single image, which results in the points with the top $N$ error values being cross-regional and losing spatial correlation. Hence, we divide and conquer the image, using a sliding window to provide more continuity candidates for the model, where the key element is the setting of the sliding stride and the window size. For the $50\times 50$ $F_{wei}$, we have three requirements for the selection of these two parameters.

1. 1.

   Padding = $0$

2. 2.

   The sliding window traverses all positions in $E$

3. 3.

   The maximum overlap length between subwindows is $8$

With the above premise while keeping $threshold$ at $0.95$, we have experimented with several possible scenarios. The results are shown in Figure 8, which indicates that a size of $27$ and a stride of $23$ is the optimal combination. This is a window with an overlap length of $8$, and the known receptive field is also $8$; thus, the area mapped to the original image is $64\times 64$, which is close to the size of a larger human head. While it is more likely that the larger head appears on the dividing line, in this case, this approach enables cross-image block analysis of features on the dividing line, thus avoiding possible targets from being corrupted.