Gradient-Free Classifier Guidance for Diffusion Model Sampling

Diffusion models are a class of generative models that generate data following an iterative denoising process [11, 29, 32]. This involves a forward process where noise is ingested to data over a sequence of time steps to render them indistinguishable from Gaussian noise, and a backward denoising process where the noise is removed following the reverse sequence until noise-free data is recovered. The forward process is governed by the stochastic differential equation (SDE):

where ${\bf x}$ is the data, $t\in[0,T]$ is the time step, and ${\bf f}$ and $g$ are predefined functions that govern the noise schedule, and $d{\bf w}$ is a standard Wiener process. The denoising process is governed by the reverse SDE:

where $\nabla_{{\bf x}}\text{log}p_{t}({\bf x})$ is the score function and $d\overline{{\bf w}}$ is the standard Wiener process for the reverse steps.

In diffusion models, the score function is parameterized by a deep neural network with parameters $\theta$, and represented as $D_{\theta}({\bf x},t)\approx\nabla_{{\bf x}}\text{log}p_{t}({\bf x})$. Conditioning variables such as class label or text prompt, denoted as $c$, can also be included and in this setting $D_{\theta}({\bf x},t,c)\approx\nabla_{{\bf x}}\text{log}p_{t}({\bf x}|c)$. During the reverse denoising process, to improve the quality of data generation, classifier-free guidance (CFG) is widely used [10]:

where $D^{m}$ and $D^{g}$ are the main and guidance networks, parameterized by neural network weights $\theta$ and $\phi$, respectively. Here $D^{g}$ could be an unconditional diffusion model as in some implementations. In others, both $D^{m}$ and $D^{g}$ may be conditional neural networks where a null ($\emptyset$) class is used as the reference for CFG, replacing $D^{g}_{\phi}({\bf x},t)$ in Equation 3.1 with $D^{g}_{\phi}({\bf x},t,\emptyset)$. Furthermore, $\omega$ is a hyperparameter referred to as the “guidance scale” where a scale of 1 means no guidance. Guided sampling improves the quality of data generation, albeit trading-off diversity [10].

Classifier guided sampling [6] from diffusion models involves the computation of gradients of classifier probabilities, and this is computationally expensive as it involves the use of autograd operators. To mitigate this issue, as explained above, CFG[10] was proposed to use an unconditional sample as a reference to increase contrast from. We extend these concepts to devise a novel formulation that utilizes a classifier, without computation of gradients, to generate an conditional sample as the reference. In what follows, we describe this methodology and refer to our method as “gradient-free classifier guidance” (GFCG). Our method is also adaptive in that it computes the guidance scale on-the-fly depending on how confused the diffusion model is during the denoising process.

In what follows, we will describe the method to a class-conditional diffusion model, where we denote the class label as $c_{i}$ for the $i$-th class (e.g., i = 0, 1, …, 999 for ImageNet which has 1000 classes). Let $c_{\text{des}}$ refer to the desired class that we wish to generate. At each time step $t$, the noisy sample $x_{t}$ is used to estimate the corresponding noise-free $\widehat{x}_{0}$:

where $\alpha_{t}$ is the noise schedule at time $t$ [11, 29]. A pre-trained classifier $\mathcal{C}$ is used to estimate the class probabilities: $p(c_{i}|\widehat{x}_{0})$. We then define an adaptive guidance scale as follows:

where $\alpha>$0, $\beta>$0 and 0$\leq\tau\leq$1 are hyperparameters that need to be calibrated. When $p(c_{\text{des}}|\widehat{x}_{0})<\tau$, the diffusion model is confused, necessitating the use of guidance to help the data generation. On the other hand, if $p(c_{\text{des}}|\widehat{x}_{0})\geq\tau$, the diffusion model is confident and does not require guidance in this scenario. Thus, we adaptively determine if guidance is required at any time step and also the right magnitude. In addition, if $p(c_{\text{des}}|\widehat{x}_{0})<\tau$, we identify a reference class $c_{\text{ref}}$ following two criteria: (1) $c_{\text{ref}}$ = the class with highest probability if $c_{\text{des}}$ does not have the highest probability; (2) $c_{\text{ref}}$ = the class with second highest probability if $c_{\text{des}}$ has the highest probability. We then recast the denoising step in Equation 3.1 as follows:

Note that this involves two forward propagation processes, one with $c_{\text{des}}$ and another with $c_{\text{ref}}$ as the conditioning class.

In summary, this method adaptively determines the guidance strength using a pre-trained classifier, and also decides when to use/not use guidance. For the classifier $\mathcal{C}$, we use a standard ResNet-101 pre-trained on ImageNet.

As explained above, the proposed guidance can be applied at each sampling step using adaptively determined $c_{ref}$ and $\omega$. In practice, there are two additional hyperparameters, $t_{s}$ and $s_{cp}$, which can also be tuned for optimal guidance effects. Similar to the guidance interval applied to CFG [19], $t_{s}$ is the starting time step that the proposed GFCG is applied first and beyond during the denoising steps. For $s_{cp}$, it is used to determine the frequency for classifier prediction for $c_{ref}$ and $\omega$ adaptation, i.e., the classifier is invoked only once every $s_{cp}$ steps. The pseudo-code to implement the GFCG guidance method is included in Algorithm 1.

There are also two optional techniques that can be added to improve guidance effects based on the models used. First for $\widehat{x}_{0}$, in place of the one step calculation as in Equation 4, a multi-step denoising process can be used to improve classifier prediction accuracy, especially in the case when $s_{cp}$ is large, like classifier prediction is only used once. Additionally, in place of the deterministic reference class $c_{ref}$ as explained in previous section, a stochastic reference class can be sampled at each step following Equation 7 where $p_{ref}(c_{i})$ is the probability that $c_{i}$ is chosen as $c_{ref}$. This technique is helpful when $s_{cp}$ is small, like classifier prediction is invoked for every sampling step.

We implemented our proposed GFCG sampling algorithm on top of the publicly available EDM2 [17] code base and use ImageNet [5] 512$\times$512 as the main dataset. Pre-trained models of EDM2-S and EDM2-XXL with default sampling parameters: 32 deterministic steps with $2^{nd}$ order Heun Sampler [15] were utilized in this work. For the classifier, we use the standard ResNet-101 pre-trained on ImageNet.

As shown in Table 1, EDM2-S was used as the main model for most experiments. GFCG was first compared with other gradient-free guidance methods. ATG still enjoys the best score in FD_DINOv2 but it comes with a significant cost in Precision. Comparing to ATG, our proposed GFCG is able to increase Precision significantly with some trade-off in FD_DINOv2 . To maintain the same computational efficiency, the same guidance model as in ATG is applied for GFCG and SEG. The second group includes mixed guidance methods which combines GFCG with other guidance methods sequentially, i.e., GFCG, following other methods, is applied to steps after $t_{s}$. In the case of GFCG_NG, it is similar to applying guidance interval [19] to GFCG. All mixed methods improve both FD_DINOv2 and Precision metrics comparing to the corresponding method which GFCG is mixed with, and GFCG_ATG gets the best FD_DINOv2 out of all four. The best FD_DINOv2 of all is achieved when the additive guidance of GFCG and CFG, setting a SOTA performance of 33.39 while beating ATG in Precision by a 3.2% margin. Experiments in EDM2-XXL shows similar benefits of GFCG. A subset of 10 classes are also chosen for Recall assessment, which is a good indicator for sample diversity. Individually, SEG and ATG have the two highest values. The mixed guidance of GFCG_ATG has the best recall overall while leading ATG in both FD_DINOv2 and Precision too. While assessed only on 10 classes, it shows GFCG is able to improve image fidelity while preserving diversity. Note that the FD_DINOv2 metric for ATG is a little worse than the published value [16]. The same random seed setting was used for all methods for fair comparison.

To determine the optimal setting for different hyperparameters, a series of ablation tests were conducted and three most critical ones are presented in Figure 4, leaving the remaining ones to the supplementary section. Note that other than the varying parameter, other settings are fixed as GFCG_NG (XS,T/16) in Table 1. The first is study the trade-off in image fidelity (lower Precision) for different ATG guidance scale, which is not reported in the original paper. It shows that the optimal setting for FD_DINOv2 is also in range of optimal Precision, as further increased scale doesn’t increase Precision. For GFCG tests, both $\alpha$ and $\beta$ are used to determine guidance scale $\omega$ where $\alpha$ is responsible for the overall strength while $\beta$ controls the relative strength in regards to classification confidence. While increasing $\alpha$ improves both metrics initially, FD_DINOv2 starts to get worse after $\alpha$ passes 0.8 and Precision also decreases after 1.2. For $\beta$, there is a similar trend and the optimal range is around 1.25 to 1.50. Lastly, out of the total of 32 sampling steps, earlier the application of guidance (larger $t_{s}$) is better for Precision, but there is a trade-off for worse FD_DINOv2 . The optimal range is around 17. For $s_{cp}$, it is better set to the maximum so that only one classifier prediction is used for all mixed and additive GFCG methods included in Table 1. Based on that, a multi-step denoising process is used for $\widehat{x}_{0}$ estimation. This adds a few NFEs but not significant as the multi-step estimation is needed one time. For example, for GFCG_ATG, it is a 4-step estimation.

Visual examples are included in Figure 3 to demonstrate the effects of transitioning from ATG to GFCG, where intermediate results are from mixed guidance of ATG and GFCG, before and after $t_{s}$ respectively. It shows image fidelity ($p(c_{des})$) increases when GFCG starts earlier. In the second example, GFCG has better $p(c_{des})$ but looses details in the background, a concern of trade-off in diversity. The mixed guidance for $t_{s}=16$ results in increased $p(c_{des})$ while preserving the background details.

While the proposed method fits well with class-conditional diffusion models naturally as it uses classifier for guided sampling, it can also help to improve sample quality for general text-to-image diffusion models like SD 1.5 [4]. As explained earlier, a classifier trained from the Bird Species dataset was used for guided sampling and the samples were evaluated against the same training dataset. As the output resolution for SD 1.5 is 512$\times$512, they were resized to the same resolution of 224$\times$224 of the dataset before assessed for FD_DINOv2 , Precision and Recall metrics. All samples were generated in 50 steps using PNDM [20]. This fine-grained generation task is challenging as many of the bird species are long-tailed classes in SD 1.5. The only other known generation test for such fine-grained classes was reported in the unified training-free classifier guidance (TFG) work [35]. Only unconditional generation using gradient classifier guidance was investigated and the best Precision score is only 2.24%. In this work, we focus on conditional generation using different guidance methods. For a given target bird species [bird #1], generic prompts like a close up bird photo, [bird #1] were used to generate diverse images of said species. In the case of GFCG, when a different bird species [bird #2] was identified as the reference class, the [bird #1] phrase in the target prompt was replaced by [bird #2] to get $c_{ref}$.

GFCG was first compared with three gradient-free guidance methods: CFG, PAG and SEG. ATG is not included as training a bad version of SD 1.5 with the optimal settings is not trivial. For experiment results shown in Table 2, 80 samples were generate for each of the 525 species and the same random seed settings were kept across different methods for fair comparison. Among all four, CFG has the best FD_DINOv2 metric while our GFCG has a clear advantage in Precision score. PAG and SEG don’t perform well for this challenging task, likely due to lack of reliable self-attention weights for long-tailed classes. Experiments were also conducted for mixed guidance GFCG_CFG, where CFG was used exclusively before applying GFCG after $t_{s}=20$. Without additional NFEs, it is able to match FD_DINOv2 of CFG while maintaining advantage in Precision. The best results are additive guidance of GFCG+CFG, both FD_DINOv2 and Precision metrics are improved significantly though at the cost of double NFEs. Comparing to ImageNet experiments where it is beneficial to start applying GFCG midway through sampling and predict confused class only once, here it was consistently found advantageous to start applying GFCG from the beginning and conduct classifier prediction more frequently. Consequently, the stochastic selection of $c_{ref}$ is also included to improve performance. This is likely due to the challenging case of fine-grained classification as there are multiple confused classes, so that adjusting confused class using classifier prediction more often is desired. As SD 1.5 is trained on a large general dataset, GFCG based on the Bird Species classifier can only provide guidance inside the bird related region of the overall data distribution, while CFG adds guidance from other parts of the data distribution. This could be the cause that GFCG+CFG has the best in both metrics.

For visual examples in Figure 5, the benefits of GFCG is obvious in terms of image fidelity, like the yellow head in the second one. The mixed guidance of GFCG_CFG keeps the general structure as in CFG, but improves image details which results in improved Precision comparing to CFG. The best results are from additive guidance of GFCG+CFG, which improves the image fidelity in both overall composition as well as fine details.

A set of ablation studies were conducted to determine the optimal settings for GFCG based sampling and three are shown in Figure 6. For GFCG, increasing $\alpha$ improves Precision consistently but FD_DINOv2 starts to worsen when it is beyond 2. For $s_{cp}$, from every 10 to 2 steps, there is significant gain in Precision with FD_DINOv2 also improved. For the mixed guidance of GFCG_CFG, where GFCG is applied after time step $t_{s}$, later application of GFCG improves FD_DINOv2 at a cost of lower Precision. Two other settings, $\beta$ and $\tau$, were both kept as 1 without fine-tuning. Note that similar ablation studies were also conducted for compared methods like CFG, PAG and SEG for fair comparisons in Table 2 and results are included in the supplementary section. As shown in Figure 6(d), a set of detailed prompts, like the one in Figure 1, were also designed to compare different guided sampling for their ability to maintain classification accuracy. In this challenging case, the advantage of GFCG is more significant than the main tests of generic prompts.

For our class-conditional experiments, we used the 2^nd order deterministic sampler from EDM (i.e., Algorithm 1 in [15]) in all experiments with $\bm{\sigma}(\mathbf{t})=\mathbf{t}$ and $s(\mathbf{t})=1$. We used the default settings $\bm{\sigma}_{min}=0.002$, $\bm{\sigma}_{max}=80$, $\rho=7$ and $N=32$. Note that we use $\bm{\sigma}$ and $\mathbf{t}$ for EDM to avoid confusion as $\sigma$ and $t$ are also used in our formulations. To follow the terminology in Algorithm 2, $N$ in original EDM is denoted as $T$, sampling steps $i=0,1,\ldots$ is denoted as $t=T,T-1,\ldots$ and noise schedule $\bm{\sigma}(\bm{t}_{0}),\bm{\sigma}(\bm{t}_{1}),\ldots$ is denoted as $\sigma_{T},\sigma_{T-1},\ldots$ and the noise schedule is calculated as follows:

Equation 4 for $\widehat{x}_{0}$ estimation is replaced by a multi-step denoising process, as also discussed in Section 4.1. This modification is detailed in Algorithm 3. Although this introduces a few additional NFEs, the impact is minimal since the multi-step estimation is only required once. For instance, in the case of GFCG_ATG, the parameters in Algorithm 3 are set as $M_{B}=\text{ATG}$, $\bm{\sigma}_{min}^{\prime}=0.002$, $\rho^{\prime}=7$ and $N^{\prime}=4$. As for $s_{cp}$, it is set to the maximum so that only one classifier prediction is used for all mixed and additive GFCG methods included in Table 1.

For the main quantitative experiments of text-to-image generations using GFCG and other gradient-free guidance methods, a set of generic prompts are used based on the realistic distribution of the Birds Species dataset. This is designed to minimize the bias between the real and generated images so the FD_DINOv2 metric could be more reliable in quantitative assessment. Each of the Set of following 8 generic text prompts was used to generate 10 samples for each bird species and quantitative results are reported in Table 2.

• $\bullet$

  a close up photo of a bird, [bird species]

• $\bullet$

  a close up bird photo, [bird species]

• $\bullet$

  a close up picture of a bird, [bird species]

• $\bullet$

  a close up bird picture, [bird species]

• $\bullet$

  a full body photo of a bird, [bird species]

• $\bullet$

  a full body bird photo, [bird species]

• $\bullet$

  a full body picture of a bird, [bird species]

• $\bullet$

  a full body bird picture, [bird species]

A set of detailed text prompts was used to generate samples, 5 per prompt for each species, for the ablation study reported in Figure 6(d). These detailed prompts are not realistic for all species, as the roadrunner in Figure 1 doesn’t perch on tree branches in real life. Besides, the descriptions below only cover part of all natural habitats. As FD_DINOv2 is not applicable for this test given these biases between two distributions, only Precision scores are reported.

• $\bullet$

  a photo of a bird perching on a tree branch with flowers blooming around it, [bird species]

• $\bullet$

  a close up photo of a flying bird with fish in its claws, [bird species]

• $\bullet$

  a photo of a bird eating red berry when standing on a rock, [bird species]

• $\bullet$

  a photo of a bird walking on the beach on a raining day, [bird species]

• $\bullet$

  a photo of a bird, [bird species], perching on a tree branch with flowers blooming around it

• $\bullet$

  a close up photo of a flying bird, [bird species], with fish in its claws

• $\bullet$

  a photo of a bird, [bird species], eating red berry when standing on a rock

• $\bullet$

  a photo of a bird, [bird species], walking on the beach on a raining day

For the implementation of SEG in the EDM2 codebase, we consider $\sigma$ = 100 for the Gaussian Blur. This is applied in the EDM2 UNet’s following blocks in the guidance network:

8x8_block1
8x8_block2
8x8_in0
8x8_block0

Other values of $\sigma$ (=10 and 1e6) were also considered but the observations are very similar.

The SEG implementation in SD is similar to [13].

The results presented in Table 1 of the main paper were generated using the same random seed for image generation. As the random seeds used in the ATG study [16] were not disclosed, we were unable to exactly replicate the reported FD_DINOv2 metric. To demonstrate that the improvement in the FD_DINOv2 metric is independent of random seed choice, we have illustrated the variation in FD_DINOv2 and Precision with random seeds for both ATG and GFCG_NG methods in Figure 7. The results show a clear distinction between the two methods in terms of Precision and FD_DINOv2 metrics, indicating that GFCG_NG consistently outperforms ATG, regardless of the random seed used.

The experiments detailed in the main paper involving the EDM2-S and EDM2-XXL models utilize guidance models (XS, T/16) and (M, T/3.5) respectively for the GFCG and SEG experiments, similar to ATG [16]. These guidance models, with reduced capacity and training, are readily accessible thanks to the publicly available EDM2 codebase [17]. However, this availability may not extend to other class-conditional or text-to-image generation diffusion models. Table 5 presents the FD_DINOv2 and Precision metrics for the GFCG_NG method, based on the capacity and training of the guidance model. The hyperparameters for GFCG, $\alpha$, $\beta$, and $t_{s}$, are set to $0.85$, $1.25$, and $17$, respectively. Similar to ATG, reducing training significantly impacts the FD_DINOv2 metric, while reducing capacity only results in the worst performance. The highest precision is achieved when using the same guidance model as the main model with some degradation in FD_DINOv2 metric.

Different classifier models, with varying sizes and top-1 and top-5 accuracies on ImageNet-1k (acc@1 and acc@5 in Table 5), were considered in the ablation study for classifier predictions in GFCG-based methods. ResNet-18, which is one-fourth the size of ResNet-101 used in the main tests, achieved a high precision of 93% compared to ATG’s 90.6%, while maintaining a similar FD_DINOv2 to ATG. ResNet-101 exhibited comparable FD_DINOv2 and precision metrics to ResNet-152, as shown in Table 5, but with a smaller model size, and was thus selected for the main experiments in the paper.

For all the mixed and additive GFCG methods presented in Table 1, $s_{cp}$ is set to its maximum value and $T^{\prime}$ is set to 4 for $\widehat{x}_{0}$ estimation. Although this introduces 7 additional NFEs, which is minimal compared to the 63 NFEs, it results in a significant boost in precision and some improvement in FD_DINOv2 as well (see Table 1). We explore two methods to further reduce the NFEs. The first method is to reduce the number of steps for $\widehat{x}_{0}$ estimation. The second method is to use a smaller and lower-trained guidance model as the main model for $\widehat{x}_{0}$ estimation. For instance, the guidance model used in the majority of the EDM2-S experiments is EDM2-XS, trained for T/16. If $M_{B}=\text{ATG}$, then line 32 in Algorithm 3 would change to $D_{1}\leftarrow D^{g}_{\phi}(x_{t},\sigma_{t},c{des})$, $D_{2}\leftarrow D^{g}_{\phi}(x_{t},\sigma_{t},c{des})$ and $\widehat{D}\leftarrow\omega_{ATG}D_{1}-(\omega_{ATG}-1)D_{2}$, which essentially applies the NG method using the guidance model only for $\widehat{x}_{0}$ estimation. As a smaller model is used for $\widehat{x}_{0}$ estimation, we ignore the NFEs added by this method. The results of these two methods compared to the main paper results are presented in Table 6.

As explained in Equation 7, a stochastic reference class can be sampled each time a classifier prediction is applied. It improves sample quality when there are frequent classifier predictions, i.e., $s_{cp}$ is small. Based on that, the experimental results of text-to-image generations in Table  2 are conducted with this enabled. For comparison, we compare GFCG methods with stochastic reference class sampling to their counterparts with determinist reference class and the quantitative results are included in Table 7. It shows that the stochastic methods are better than their deterministic counterparts in overall performance considering both FD_DINOv2 and Precision.

A full range of ablation studies were conducted for text-to-image experiments using SD, including GFCG and other guidance methods. Three main ones are included in the main paper. All three key settings of GFCG, $\alpha$, and $s_{cp}$ included in Figure 6, and the additional $\beta$, are plotted together in Figure 8 for full comparison. As evident, $\alpha$ has the highest impact and $\beta$ the least. For $s_{cp}$, the Precision value is the highest when it is set as the minimum of 1, while achieving the lowest FD_DINOv2 too.

For the mixed GFCG_CFG method, the only key variable is $t_{s}$ as $\omega_{CFG}$ for CFG and $\alpha$ for GFCG are using the same optimal setting of each respectively. The results for $t_{s}$ is already included in Figure  6. For the additive method of GFCG+CFG, $\omega_{CFG}$ and $\alpha$ are investigated to find the optimal settings. As shown in Figure 9, the optimal values are around 4.0 and 1.5, lower than the settings of 5.5 and 2.0 when optimized for CFG and GFCG individually.

For fair comparison, we also study the effect of the guidance scale $\omega$ for CFG, PAG and SEG in Figure 10. For CFG, $\omega$ has a significant role in terms of the FD_DINOv2 and Precision of the generated images. For PAG, $\omega$ has a lesser influence, and for SEG the impact is the least. Among these three guidance methods, CFG is much superior than PAG and SEG in terms of FD_DINOv2 and Precision.

Some visual examples from text-to-images generation using the set of generic prompts are included in Figure  13. The probability of classifying each generated sample as the target class is also included for reference. In general, the GFCG results have the best class accuracy. The gained accuracy could be caused by compositional change in the first example, as well as correct anatomic features like feather color in the second. For GFCG_CFG, it often maintains the overall composition of CFG and is possible to improve class accuracy even when the changes are negligible with untrained eyes like the first example. In the case of the last example, minor changes like chest patterns in GFCG_CFG result in significantly increased probability too.

As shown in Figure  14, GFCG results may end up worse than other methods too. For the first example of TIT MOUSE, the text-to-image model obviously has misunderstood MOUSE without recognizing its context as a bird specie. In the case of CFG, as it is guided away from an unconditional model, it will enhance the wrong features associated with MOUSE, as well as correct features associate with other keywords like bird. For GFCG, as the enhancement is in the reference to a photo of another bird species, the difference will be focused between TIT MOUSE and another bird species which results in more prominent mouse features. Similar failure happens in the second example where color features related to TEAL is magnified.

More visual examples using generic text prompts are shown in Figure 20 at the end.

Some visual examples using the set of detailed prompts are included in Figure  15. Note that the hyperparameters like $\alpha$ and $t_{s}$ for GFCG related methods were optimized for the generic prompts and adopted for detailed prompts without further tuning. It shows that GFCG has the best class accuracy while preserving the overall accuracy of the full text prompt, including improving large features like the first example, or small details like the last one. It is noted that, all method including CFG, have difficulty in depicting some details in the prompts like fish in its claws and eating red berry. As shown in Figure 16, for failure cases of GFCG where class probabilities of GFCG results are lower than those of CFG, certain features of the species, e.g. white feather of BALD EAGLE, are excessively enhanced. This could be caused by improper guidance scales. More visual examples using detailed text prompts are shown in Figure 21 at the end.

We also explored using GFCG in the pixel space alone, without using latent diffusion like SD 1.5 used above. In this test, GFCG is integrated into the DeepFloyd IF model from Stability AI, whose diffusion mechanism is implemented in the pixel level. The IF model is able to generate high definition images in size of $1024\times 1024$ based on given text prompts. We assessed the performance on the Bird Species dataset, and showed some visual examples in Figure  17. The probability of classifying each generated sample as the target class is also included for reference. On multiple bird species, much higher class accuracy was achieved by GFCG over other guidance schemes. The gained accuracy is largely because of enhanced visual quality of the bird, which is more aligned to the actual appearance of corresponding species. Some additional examples are shown in Figure 22 at the end.