Free Lunch in Pathology Foundation Model: Task-specific Model Adaptation with Concept-Guided Feature Enhancement

To further evaluate the generalization ability of the proposed CATE method, we conduct additional experiments on TCGA-BRCA dataset with different settings of in-domain (IND) and out-of-domain (OOD) sites. The detailed experimental settings are shown below:

• •

  BRCA ($N_{\text{IND}}$=1): One site as the in-domain data, and the other sites as the out-of-domain data. The details of the in-domain site are shown below:

  • –

    AR: 44 slides of IDC and 15 slides of ILC.

• •

  BRCA ($N_{\text{IND}}$=2): Two sites as the in-domain data, and the other sites as the out-of-domain data. The details of the in-domain sites are shown below:

  • –

    AR: 44 slides of IDC and 15 slides of ILC.

  • –

    B6: 39 slides of IDC and 6 slides of ILC.

The additional results are presented in Table 4. It is evident that the proposed CATE-MIL consistently delivers superior performance in both in-domain and out-of-domain settings, demonstrating its effectiveness in enhancing task-specific information and minimizing the impact of irrelevant data.

In the main paper, we divided the dataset into in-domain and out-of-domain sites to evaluate the generalization ability of the proposed CATE method. To further evaluate its effectiveness, we conducted additional experiments under traditional settings, randomly splitting the dataset into training, validation, and testing sets. We report the performance metrics, including means and standard deviations over 10 Monte-Carlo Cross-Validation runs, in Table 5. Additionally, we provide comparisons of training times and parameter sizes for various methods.

It is evident that the proposed CATE-MIL consistently outperforms others in both AUC and ACC metrics, underscoring its superiority. Furthermore, CATE-MIL benefits from shorter training times and smaller parameter sizes compared to the R²T-MIL method.

CATE is a general framework that can be applied to more complex tasks beyond cancer subtyping, such as Gleason grading in prostate cancer. We have conducted conducted experiments on the TCGA-PRAD dataset to evaluate the performance of CATE-MIL in Gleason grading. Specifically, the samples in PRAD dataset are divided into Gleason pattern 3, 4, and 5, and the task is to classify the samples into these three categories. The results are shown in Table 6. It is evident that CATE-MIL consistently outperforms the base model ABMIL in both in-domain and out-of-domain settings, demonstrating its effectiveness in enhancing task-specific information and minimizing the impact of irrelevant data. In the future, as more studies reveal the connection between morphological features and molecular biomarkers and more powerful pathology VLMs are developed, our framework has the potential to benefit more complex tasks.

To provide a more comprehensive evaluation of the proposed CATE-MIL, we conduct additional experiments using site-preserved cross-validation [11], where the samples from the same site are preserved in the same fold. For each fold, we split the data into training and testing sets, and these testing sets are regarded as in-domain testing data. And the other sites are used as out-of-domain testing data. The results are shown in Table 7. It is evident that CATE-MIL consistently outperforms ABMIL in both in-domain and out-of-domain settings.

The overall objective of the proposed CATE-MIL is a weighted sum of the PIM and SIM losses, along with the classification loss:

$$\mathcal{L}=\mathcal{L}_{CE}+\lambda_{P}\mathcal{L}_{PIM}+\lambda_{S}\mathcal{L}_{SIM}.$$

(12)

We note that the hyperparameters for the weights of the PIM and SIM losses were not specifically tuned in the main paper. To investigate the effect of the weight of PIM and SIM losses on the model performance, we conduct an ablation study on BRCA ($N_{\text{IND}}$=2) with CATE-MIL (ABMIL integrated with CATE), varying the weights of PIM and SIM losses. The results are shown in Figure 4.

From Figure 4, it is evident that CATE enhances the performance of the base model ABMIL in most scenarios, particularly in out-of-domain settings. When the weight of PIM loss $\lambda_{P}$ is too low, model performance suffers, underscoring the significant role of PIM loss in enhancing task-specific information. Conversely, an excessively high $\lambda_{P}$ also diminishes performance, as the model overly prioritizes maximizing mutual information between the original and calibrated features at the expense of optimizing classification loss. Regarding the weight of SIM loss $\lambda_{S}$, optimal performance is achieved when $\lambda_{S}$ is approximately 30. If the $\lambda_{S}$ is too low, the model fails to effectively eliminate irrelevant information, thereby impairing performance. Conversely, if $\lambda_{S}$ is too high, the model risks discarding task-relevant information, leading to performance degradation.

In this section, we conduct an ablation study to investigate the impact of the number of representative patches, $k$, on model performance, as discussed in Section 3.2. In practice, the representative patches are selected based on the similarities between the original image feature and the corresponding class-specific concept anchor.

As shown in Figure 5, it is evident that CATE generally enhances the performance of the base model, ABMIL. When the number of representative patches $k$ is too small, model performance degrades due to insufficient capture of task-specific information. Conversely, an excessively large $k$ also leads to performance degradation in out-of-domain scenarios, as it introduces noise from irrelevant information. Consequently, we have set the number of representative patches $k$ to 10 in the main paper.

We further conducted experiments on CATE-MIL without concept alignment (discarding PIM loss and SIM loss of the CIB module) and replaced the CIB module with an MLP to investigate the effect of concept alignment and the increased number of parameters. The results are shown in Table 8. The performance of CATE-MIL significantly decreases in both cases, demonstrating the importance of concept alignment in the CIB module and that the improvements of CATE are not due to the increased number of parameters.

• •

  TCGA-BRCA: The dataset contains nine disease subtypes, and this study focuses on the classification of Invasive ductal carcinoma (IDC, 726 slides from 694 cases) and invasive lobular carcinoma (ILC, 149 slides from 143 cases). The dataset is collected from 36 sites with 20 of them having both IDC and ILC slides, and the other 16 sites only have IDC slides or ILC slides.

• •

  TCGA-NSCLC: The dataset contains two disease subtypes, including lung adenocarcinoma (LUAD, 492 slides from 430 cases) and lung squamous cell carcinoma (LUSC, 466 slides from 432 cases). The dataset is collected from 66 sites. Different from TCGA-BRCA, each site only contains one disease subtype.

• •

  TCGA-RCC: The dataset contains three disease subtypes, including clear cell renal cell carcinoma (CCRCC, 498 slides from 492 cases), papillary renal cell carcinoma (PRCC, 289 slides from 267 cases), and chromophobe renal cell carcinoma (CHRCC, 118 slides from 107 cases). The dataset is collected from 55 sites. Similar to TCGA-NSCLC, each site only contains one disease subtype.

To validate that the proposed CATE effectively improves MIL model performance by enhancing the task-specific information and eliminating the disturbance of task-irrelevant information, we conduct extensive experiments across three datasets under various in-domain and out-of-domain settings.

Specifically, for TCGA-BRCA dataset, we conduct the following experiments:

• •

  BRCA ($N_{\text{IND}}$=1): One site as the in-domain data, and the other sites as the out-of-domain data. The details of the in-domain site are shown below:

  • –

    D8: 59 slides of IDC and 8 slides of ILC.

• •

  BRCA ($N_{\text{IND}}$=2): Two sites as the in-domain data, and the other sites as the out-of-domain data. The details of the in-domain sites are shown below:

  • –

    A8: 48 slides of IDC and 5 slides of ILC.

  • –

    D8: 59 slides of IDC and 8 slides of ILC.

For TCGA-NSCLC dataset, we conduct the following experiments:

• •

  NSCLC ($N_{\text{IND}}$=2): Two sites as the in-domain data, and the other sites as the out-of-domain data. The details of the in-domain sites are shown below:

  • –

    44: 43 slides of LUAD and 0 slides of LUSC.

  • –

    22: 0 slides of LUAD and 36 slides of LUSC.

• •

  NSCLC ($N_{\text{IND}}$=4): Four sites as the in-domain data, and the other sites as the out-of-domain data. The details of the in-domain sites are shown below:

  • –

    44: 43 slides of LUAD and 0 slides of LUSC.

  • –

    50: 20 slides of LUAD and 0 slides of LUSC.

  • –

    22: 0 slides of LUAD and 36 slides of LUSC.

  • –

    56: 0 slides of LUAD and 35 slides of LUSC.

For TCGA-RCC dataset, we conduct the following experiments:

• •

  RCC ($N_{\text{IND}}$=3): Three sites as the in-domain data, and the other sites as the out-of-domain data. The details of the in-domain sites are shown below:

  • –

    A3: 48 slides of CCRCC, 0 slides of CHRCC, and 0 slides of PRCC.

  • –

    KL: 0 slides of CCRCC, 24 slides of CHRCC, and 0 slides of PRCC.

  • –

    2Z: 0 slides of CCRCC, 0 slides of CHRCC, and 23 slides of PRCC.

• •

  RCC ($N_{\text{IND}}$=6): Six sites as the in-domain data, and the other sites as the out-of-domain data. The details of the in-domain sites are shown below:

  • –

    A3: 48 slides of CCRCC, 0 slides of CHRCC, and 0 slides of PRCC.

  • –

    AK: 15 slides of CCRCC, 0 slides of CHRCC, and 0 slides of PRCC.

  • –

    KL: 0 slides of CCRCC, 24 slides of CHRCC, and 0 slides of PRCC.

  • –

    KM: 0 slides of CCRCC, 20 slides of CHRCC, and 0 slides of PRCC.

  • –

    2Z: 0 slides of CCRCC, 0 slides of CHRCC, and 23 slides of PRCC.

  • –

    5P: 0 slides of CCRCC, 0 slides of CHRCC, and 15 slides of PRCC.

In this part, we will define the sufficiency and consistency in the context of the concept information bottleneck (CIB) module.

First, we define the mutual information between two random variables $X$ and $Y$ as:

$$I(X;Y)=\sum p(x,y)\log\frac{p(x,y)}{p(x)p(y)}.$$

(13)

Sufficiency. $\hat{\boldsymbol{\alpha}}$ is sufficient for $\boldsymbol{\rm{y}}\iff I(\hat{\boldsymbol{x}};\boldsymbol{\rm{y}}|\hat{\boldsymbol{\alpha}})=0\iff I(\hat{\boldsymbol{x}};\boldsymbol{\rm{y}})=I(\hat{\boldsymbol{\alpha}};\boldsymbol{\rm{y}})$.

The sufficiency definition requires that the calibrated features $\hat{\boldsymbol{\alpha}}$ encapsulate all information about the label information $\boldsymbol{\rm{y}}$ that is accessible from the original features $\hat{\boldsymbol{x}}$. In essence, the calibrated feature $\hat{\boldsymbol{\alpha}}$ of original feature $\hat{\boldsymbol{x}}$ is sufficient for determining label $\boldsymbol{\rm{y}}$ if and only if the amount of information regarding the specific task is unchanged after the transformation.

Consistency. $\boldsymbol{\rm{c}}$ is consistent with $\hat{\boldsymbol{x}}$ for $\boldsymbol{\rm{y}}\iff I(\boldsymbol{\rm{y}};\hat{\boldsymbol{x}}|\boldsymbol{\rm{c}})=0$.

The consistency definition requires that the concept anchor $\boldsymbol{\rm{c}}$ and the original feature $\hat{\boldsymbol{x}}$ should be consistent with the label information $\boldsymbol{\rm{y}}$.

In this part, we will prove that the predictive information for the concept anchor can be maximized by maximizing the InfoNCE mutual information lower bound, which is defined in Equation 4. As in [27], the optimal value for $f\left(\boldsymbol{c},\hat{\boldsymbol{\alpha}}_{i}\right)$ should be proportional to the density ratio:

$$f\left(\boldsymbol{c},\hat{\boldsymbol{\alpha}}_{i}\right)\propto\frac{p\left(\boldsymbol{c},\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}.$$

(14)

And we can get:

$$\begin{split}&I_{NCE}^{opt}\left(\hat{\boldsymbol{\rm{\alpha}}};\boldsymbol{\rm{c}}\right)=\operatornamewithlimits{\mathbb{E}}_{\hat{\boldsymbol{\rm{\alpha}}},\boldsymbol{c}^{\text{cs}}_{\text{pos}}}\left[\sum_{i=1}^{k}\log\frac{\frac{p\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}},\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}}{\sum_{j=1}^{m}\frac{p\left(\boldsymbol{c}^{\text{cs}}_{j},\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}^{\text{cs}}_{j}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}+\sum_{j=1}^{n}\frac{p\left(\boldsymbol{c}^{\text{ca}}_{j},\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}^{\text{ca}}_{j}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}}\right]\\ &=-\operatornamewithlimits{\mathbb{E}}_{\hat{\boldsymbol{\rm{\alpha}}},\boldsymbol{c}^{\text{cs}}_{\text{pos}}}\left[\sum_{i=1}^{k}\log\left(1+\frac{p\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}},\hat{\boldsymbol{\alpha}}_{i}\right)}\left(\sum_{j\neq\text{pos}}\frac{p\left(\boldsymbol{c}_{j}^{\text{cs}},\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}_{j}^{\text{cs}}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}+\sum_{j=1}^{n}\frac{p\left(\boldsymbol{c}_{j}^{\text{ca}},\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}_{j}^{\text{ca}}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}\right)\right)\right]\\ &\approx-\operatornamewithlimits{\mathbb{E}}_{\hat{\boldsymbol{\rm{\alpha}}},\boldsymbol{c}^{\text{cs}}_{\text{pos}}}\left[\sum_{i=1}^{k}\log\left(1+\frac{p\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}},\hat{\boldsymbol{\alpha}}_{i}\right)}\left(\left(m-1\right)\operatornamewithlimits{\mathbb{E}}_{\hat{\boldsymbol{\alpha}}_{i}}\frac{p\left(\boldsymbol{c}_{j}^{\text{cs}},\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}_{j}^{\text{cs}}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}+n\operatornamewithlimits{\mathbb{E}}_{\hat{\boldsymbol{\alpha}}_{i}}\frac{p\left(\boldsymbol{c}_{j}^{\text{ca}},\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}_{j}^{\text{ca}}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}\right)\right)\right]\\ &=-\operatornamewithlimits{\mathbb{E}}_{\hat{\boldsymbol{\rm{\alpha}}},\boldsymbol{c}^{\text{cs}}_{\text{pos}}}\left[\sum_{i=1}^{k}\log\left(1+\frac{p\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}},\hat{\boldsymbol{\alpha}}_{i}\right)}\left(m+n-1\right)\right)\right]\\ &\leq-\operatornamewithlimits{\mathbb{E}}_{\hat{\boldsymbol{\rm{\alpha}}},\boldsymbol{c}^{\text{cs}}_{\text{pos}}}\left[\sum_{i=1}^{k}\log\left(\frac{p\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}}\right)p\left(\hat{\boldsymbol{\alpha}}_{i}\right)}{p\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}},\hat{\boldsymbol{\alpha}}_{i}\right)}\left(m+n\right)\right)\right]\\ &=\sum_{i=1}^{k}I\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}};\hat{\boldsymbol{\alpha}}_{i}\right)-k\log\left(m+n\right)\\ &=I\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}};\hat{\boldsymbol{\rm{\alpha}}}\right)-k\log\left(m+n\right),\end{split}$$

(15)

where $\boldsymbol{C}_{\text{neg}}$ denotes the negative concepts, including both negative class concepts and other type concepts. We can see that the InfoNCE estimation is a lower bound of the mutual information between the concept anchor and the calibrated features:

$$I\left(\boldsymbol{c}^{\text{cs}}_{\text{pos}};\hat{\boldsymbol{\rm{\alpha}}}\right)\geq I_{NCE}^{opt}\left(\hat{\boldsymbol{\rm{\alpha}}};\boldsymbol{\rm{c}}\right)+k\log\left(m+n\right)\geq I_{NCE}\left(\hat{\boldsymbol{\rm{\alpha}}};\boldsymbol{\rm{c}}\right).$$

(16)

Thus, the predictive information of calibrated features for the concept anchor can be maximized by maximizing the InfoNCE mutual information lower bound.

As shown in Equation 3, the mutual information between the original features and the calibrated features $I(\hat{\boldsymbol{x}};\hat{\boldsymbol{\alpha}})$ can be decomposed into the mutual information $I(\hat{\boldsymbol{\alpha}};\boldsymbol{\rm{c}})$ between the calibrated features and the concept anchor and the mutual information $I(\hat{\boldsymbol{x}};\hat{\boldsymbol{\alpha}}|\boldsymbol{\rm{c}})$ between the concept anchor and the calibrated features:

$$I(\hat{\boldsymbol{x}};\hat{\boldsymbol{\alpha}})=\underbrace{I(\hat{\boldsymbol{\alpha}};\boldsymbol{\rm{c}})}_{\text{Predictive information for }\boldsymbol{\rm{c}}}+\underbrace{I(\hat{\boldsymbol{x}};\hat{\boldsymbol{\alpha}}|\boldsymbol{\rm{c}})}_{\text{Superfluous information}}.$$

(17)

As introduced in section 3.2, the first item is the predictive information for the concept anchor, which can be maximized by maximizing the InfoNCE mutual information lower bound. Thus, the second superfluous information item can be minimized by minimizing $I(\hat{\boldsymbol{x}};\hat{\boldsymbol{\alpha}})$. In practice, the superfluous information minimization is conducted on all patches in the subset $\boldsymbol{\rm{x}}$, since the task-irrelevant information is distributed across all patches. Thus, the superfluous information can be minimized by minimizing $I({\boldsymbol{x}};{\boldsymbol{\alpha}})$. As in [1], this mutual information can be represented as:

$$\begin{split}I({\boldsymbol{x}};{\boldsymbol{\alpha}})&=\int p\left({\boldsymbol{x}},{\boldsymbol{\alpha}}\right)\log\frac{p\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)}{p\left({\boldsymbol{\alpha}}\right)}d{\boldsymbol{x}}d{\boldsymbol{\alpha}}\\ &=\int p\left({\boldsymbol{x}},{\boldsymbol{\alpha}}\right)\log p\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)d{\boldsymbol{x}}d{\boldsymbol{\alpha}}-\int p\left({\boldsymbol{\alpha}}\right)\log p\left({\boldsymbol{\alpha}}\right)d{\boldsymbol{\alpha}}.\end{split}$$

(18)

However, the computing of the marginal distribution $p\left({\boldsymbol{\alpha}}\right)=\int p\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)p\left({\boldsymbol{x}}\right)d{\boldsymbol{x}}$ is intractable. Thus, we can let the Gaussian distribution $r\left({\boldsymbol{\alpha}}\right)$ approximate the marginal distribution $p\left({\boldsymbol{\alpha}}\right)$. Since the Kullback-Leibler divergence between two distributions is non-negative, we can get:

$$\int p\left({\boldsymbol{\alpha}}\right)\log p\left({\boldsymbol{\alpha}}\right)d{\boldsymbol{\alpha}}\geq\int p\left({\boldsymbol{\alpha}}\right)\log r\left({\boldsymbol{\alpha}}\right)d{\boldsymbol{\alpha}}.$$

(19)

Thus, the mutual information $I({\boldsymbol{x}};{\boldsymbol{\alpha}})$ have the upper bound:

$$\begin{split}I({\boldsymbol{x}};{\boldsymbol{\alpha}})&\leq\int p\left({\boldsymbol{x}},{\boldsymbol{\alpha}}\right)\log p\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)d{\boldsymbol{x}}d{\boldsymbol{\alpha}}-\int p\left({\boldsymbol{\alpha}}\right)\log r\left({\boldsymbol{\alpha}}\right)d{\boldsymbol{\alpha}}\\ &=\int p\left({\boldsymbol{x}}\right)p\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)\log\frac{p\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)}{r\left({\boldsymbol{\alpha}}\right)}d{\boldsymbol{x}}d{\boldsymbol{\alpha}}.\end{split}$$

(20)

In practice, to compute this upper bound, we approximate distribution $p\left({\boldsymbol{x}}\right)$ with the empirical distribution:

$$p\left({\boldsymbol{x}}\right)=\frac{1}{N}\sum^{N}_{i=1}\delta_{{\boldsymbol{x}}_{i}}\left({\boldsymbol{x}}\right).$$

(21)

Thus, the upper bound of the mutual information $I({\boldsymbol{x}};{\boldsymbol{\alpha}})$ can be computed as:

$$\int p\left({\boldsymbol{x}}\right)p\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)\log\frac{p\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)}{r\left({\boldsymbol{\alpha}}\right)}d{\boldsymbol{x}}d{\boldsymbol{\alpha}}\approx\frac{1}{N}\sum^{N}_{i=1}q_{\theta}\left({\boldsymbol{\alpha}}_{i}|{\boldsymbol{x}}_{i}\right)\log\frac{q_{\theta}\left({\boldsymbol{\alpha}}_{i}|{\boldsymbol{x}}_{i}\right)}{r\left({\boldsymbol{\alpha}}_{i}\right)},$$

(22)

where $q_{\theta}\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)$ is a variational distribution with parameter $\theta$ to approximate $p\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)$, and we implement this variational distribution with MLP:

$$q_{\theta}\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)=\mathcal{N}\left({\boldsymbol{\alpha}}|MLP^{\mu}\left({\boldsymbol{x}}\right),MLP^{\Sigma}\left({\boldsymbol{x}}\right)\right),$$

(23)

where $MLP^{\mu}$ and $MLP^{\Sigma}$ are implemented with MLPs which output the mean $\mu$ and covariance matrix $\Sigma$ of the Gaussian distribution. In practice, we utilize the reparameterization trick [18] to sample from the Gaussian distribution to get an unbiased estimate of the gradient to optimize the opjective. Thus, the upper bound can be optimized by minimizing the KL divergence between the variational distribution $q_{\theta}\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)$ and the Gaussian distribution $r\left({\boldsymbol{\alpha}}\right)$:

$$\min_{\theta}\mathbb{E}_{{\boldsymbol{x}}}\left[D_{KL}\left(q_{\theta}\left({\boldsymbol{\alpha}}|{\boldsymbol{x}}\right)||r\left({\boldsymbol{\alpha}}\right)\right)\right].$$

(24)