Unsupervised Cross-Domain Soft Sensor Modelling via Deep Physics-Inspired Particle Flow Bayes

UDA is a special case of transfer learning, where the source and target domains are from the same space and the labels are missing in the target domain [6]. Consider a sequence of data and label subspace pairs $\{\mathcal{X}_{n}^{\mathcal{S}}\times\mathcal{Y}_{n}^{\mathcal{S}}\}_{n=0}^{L}$ and $\{\mathcal{X}_{n}^{\mathcal{T}}\times\mathcal{Y}_{n}^{\mathcal{T}}\}_{n=0}^{L}$ of length $L$, known respectively as the source and target domains with distinctive marginal data distributions $p(\mathcal{X}_{n}^{\mathcal{S}})\neq p(\mathcal{X}_{n}^{\mathcal{T}})$. The objective of time series UDA is to predict the missing target labels ${y}^{\mathcal{T}}_{0:L}\equiv\{{y}_{n}\,|\,{y}_{n}\in\mathcal{Y}_{n}^{\mathcal{T}}\}_{n=0}^{L}$, using the cross-domain data ${x}_{0:L}\equiv\{{x}_{n}\,|\,{x}_{n}\in\mathcal{X}_{n}^{\mathcal{S}}\cup\mathcal{X}_{n}^{\mathcal{T}}\}_{n=0}^{L}$ and source labels ${y}^{\mathcal{S}}_{0:L}\equiv\{{y}_{n}\,|\,{y}_{n}\in\mathcal{Y}_{n}^{\mathcal{S}}\}_{n=0}^{L}$ [5]. In general, deep UDA models achieve this by learning, at each sample index ${n}$, a domain-invariant latent feature space $\mathcal{Z}_{n}$ that generalizes well across domains, so that it accurately predicts the labels in both the label spaces $\{\mathcal{Y}_{n}^{\mathcal{S}},\mathcal{Y}_{n}^{\mathcal{T}}\}$. On one hand, the DAT-based models [11, 12, 13, 14] incorporate a feature-level domain discriminator, where its adversarial gradients are backpropagated to train the feature extractors in generating domain-invariant feature representations. On the other hand, the VB-based models [15, 16, 17, 18] implement variational inference to learn domain-invariant probabilistic latent feature representations that generalize the knowledge learned from the supervised source domain to the unsupervised target domain. Nevertheless, these deep UDA approaches hinge on the assumption of domain invariance, which could impede the generalization and expressive abilities of feature models in characterizing complex cross-domain system dynamics [27].

The AEVB is a class of probabilistic deep learning framework that aims to maximize the lower bound of the joint data log-likelihood $\log p(X)$. By introducing a latent state $Z$, a variational evidence lower bound (ELBO) of the data log-likelihood can be obtained via the importance decomposition $\mathcal{L}^{\text{ELBO}}(\theta,\vartheta)=\mathbb{E}_{q_{\vartheta}(Z|X)}\left[\log p_{\theta}(X|Z)\right]+\mathcal{D}^{\text{KL}}\big{[}q_{\vartheta}(Z|X)\|p_{\theta}(Z)\big{]}$, where $\mathcal{D}^{\text{KL}}$ denotes the (positive-valued) Kullback-Leibler divergence (KLD). The data likelihood $p_{\theta}(X|Z)$, the latent approximate posterior distribution $q_{\vartheta}(Z|X)$, and the latent prior distribution $p_{\theta}(Z)$ are mean-field Gaussians parameterized respectively by the fully-connected neural network (FCNN) models: decoder, prior, encoder, known collectively as the variational AE (VAE). The parameters $\theta$ and $\vartheta$ denote the set of NN parameters governing these models. A significant drawback of VAE is the assumption that the time series data are independent and identically distributed (i.i.d.), which falls short at capturing the underlying temporal characterization. To address this, the sequential AEVB framework [28] introduces a sequential ELBO as follows:

where $a_{0:k}$ denotes a partial sequence up to the ${k}^{\text{th}}$ sample. Similarly, the conditional data likelihood $p_{\theta}(x_{n}|z_{0:n},x_{0:n-1})$, the latent prior distribution $p_{\theta}(z_{n}|z_{0:n-1},x_{0:n-1})$, and the approximate posterior $q_{\vartheta}(z_{n}|x_{0:n},z_{0:n-1})$ are mean-field Gaussians parameterized by RNN models, known collectively as the variational RNN (VRNN). However, existing VRNN-based approach rely on the variational approximation, which significantly compromises their generalization capability to perform effective domain adaptation. Moreover, the RNN hidden states are not explicitly subjected to posterior update, which could result in poorly conditioned hidden states that generate latent features divergent from the actual system representation.

Bayesian inference, which is a more general approach than variational inference [29], constructs an exact approximation of the true posterior recursively via the Bayes rule $p(z_{n}|x_{0:n})\propto p(z_{n}|x_{0:n-1})\,p(x_{n}|z_{n},x_{0:n-1})$. Particle flow, first introduced in [30], achieves this by subjecting prior samples to a series of infinitesimal transformations that follow a linear ordinary differential equation (ODE) $\frac{dz}{d\tau}=u$ in continuous pseudo-time $\tau\in[{0},{1}]$. The function $f$, known as the velocity, is formulated such that the flow inherently performs the Bayes update, and the transformed samples represent an empirical (Monte Carlo) approximation of the true posterior. While particle flow is closely related to normalizing flows [31, 25] for variational inference, the latter do not exhibit a Bayes update. Pal et al. [32] recently combined graph representation learning with particle flow for spatiotemporal time series forecasting, but relied on restrictive local linearization for nonlinear models. Nonetheless, existing flow-based methods do not consider the issues of domain shift and missing labels.

From a probabilistic perspective, the unsupervised cross-domain soft sensor modelling problem can be viewed as maximizing the joint likelihood $p_{\theta}(x_{0:L},y^{\mathcal{S}}_{0:L})$ of the cross-domain time series data and the source labels. The maximum likelihood problem can be formulated as follows:

Taking into account the difficulty of directly optimizing (2), we introduce a sequence of stochastic latent state space $\{\mathcal{Z}_{n}\}_{n=0}^{L}$ and consider an importance decomposition of the conditional data log-likelihood that holds for any choice of approximate posterior $q_{\vartheta}(z_{0:n})$ as follows:

where

Here, $p^{+}(z_{0:n})$ denotes the intractable true joint latent posterior $p(z_{0:n}|x_{0:n})$. Also, we assumed that $p(z_{0:n}|x_{0:n})=p(z_{0:n}|x_{0:n},y^{\mathcal{S}}_{0:n})$, i.e., the true posterior is self-sufficient and gains no further information from observing the labels.

Subsequently, we consider factorizing the conditional data likelihood in (3) as follows:

where we assumed that $p_{\theta}(x_{n},y^{\mathcal{S}}_{n},z_{n}|z_{0:n-1},x_{0:n-1})=p_{\theta}(x_{n},y^{\mathcal{S}}_{n},z_{n}|z_{0:n-1})$, i.e., the approximate posterior latent sequence $z_{0:n-1}$ accurately characterizes the preceding time series $x_{0:n-1}$. Finally, substituting (5) into (3), we obtain the sequential Bayes objective (SBO) as follows:

As a result, we reformulated the cross-domain soft sensing problem as an end-to-end expectation-maximization algorithm (6). In the maximization step, the first two objectives identify a pair of generative data likelihood $p_{\theta}(x_{n}|z_{0:n})$ and label likelihood $p_{\theta}(y^{\mathcal{S}}_{n}|z_{0:n-1})$ that represent the cross-domain data and source labels. In the expectation step, the last two objectives aim to find the optimal marginal approximate posterior $q_{\vartheta}(z_{n}|z_{0:n-1})$ and joint approximate posterior $q_{\vartheta}(z_{0:n})$ that resemble the conditional prior $p_{\theta}(z_{n}|z_{0:n-1})$ and the intractable true posterior $p^{+}(z_{0:n})$, respectively.

In particular, the first three objectives in (6) are the same as those in the VRNN’s sequential ELBO (1). However, we have added an additional (third) semi-supervised objective to account for the source labels. In addition to these objectives, our SBO formulation includes the final KLD objective, which inherently performs a Bayesian smoothing [29] of the partial state sequence $z_{0:n}$. This Bayesian smoothing encourages the latent state sequence to adopt an approximate posterior feature representation that facilitates a comprehensive characterization of the cross-domain time series data.

In this subsection, we introduce a RNN parameterization for the proposed SBO. First, we parameterize the generative data and label likelihoods, and the latent prior in (6) as Gaussian distributions, given by:

where $\Sigma^{\text{dec}}_{n}=\text{diag}(\sigma_{n}^{\text{prior}^{2}})$ and $\Sigma^{\text{prior}}_{n}=\text{diag}(\sigma_{n}^{\text{prior}^{2}})$ are isotropic covariances, and $\text{diag}(\cdot)$ denotes the diagonal function. In particular, we allow the latent prior (7c) to coincide with the label likelihood (7b), allowing the models to learn to predict the source labels. This supervised knowledge in the source domain can then be adapted to the unsupervised target domain during the Bayes update of the proposed particle flow.

Subsequently, the mean and standard deviation pairs in (7) are obtained using a stochastic RNN as follows:

where $\varphi_{\theta}^{\text{rnn}}$ is the RNN model, and the prior model $\varphi_{\theta}^{prior}$, decoder model $\varphi_{\theta}^{dec}$ and state encoding model $\varphi_{\theta}^{z}$ are FCNNs. Here, the memory-encoding RNN hidden states $h_{n}\in\mathbb{R}^{n_{h}}$ serve as an embedding of the preceding time series $z_{0:n-1}$ to retain the intrinsic temporal information; this allows us to work with a reduced-dimension state space $\mathbb{R}^{n_{z}+n_{h}}$ instead of the exponentially growing $\mathbb{R}^{n_{z}\times n}$ for the entire latent sequence, thus avoiding curse of dimensionality [33].

To sample from the latent prior $p_{\theta}(z_{n}|z_{0:n-1})=p_{\theta}(z_{n}|h_{n})$, we compute its mean and standard deviation using equations (8a) to (8b), and then apply the reparameterization trick [23]. It is worth noting that although the hidden states have a degenerate conditional distribution $\delta_{\theta}(h_{n}|{z}_{n-1},{h}_{n-1})$, where $\delta(\cdot)$ is the Dirac delta function, they remain stochastic due to the recurrence and the variability of latent states after integrating out the conditioned variables. Additionally, unlike in VRNN [28], data are not fed into the RNN model to minimize the inheritance of data noise. Fig. 2 outlines the (features) prior and (data) decoder operations in DPFB.

In this subsection, we introduce particle flow as a Bayesian inference technique to parameterize the approximate posterior. More precisely, a set of sampled prior latent and hidden state particles $(z_{n}^{i},h_{n}^{i})$ is subjected to a series of infinitesimal transformations, collectively mapping them to $(\bar{z}_{n}^{i},\bar{h}_{n}^{i})$. These FCNN-parameterized transformations, denoted individually as $\alpha_{\vartheta}:\mathbb{R}^{n_{x}+n_{h}}\rightarrow\mathbb{R}^{n_{x}+n_{h}}$, are governed by an ODE $\frac{d(z,h)}{d\tau}=u$ on a continuous pseudo-time interval $\tau\in[{0},{1}]$. The flow velocity function $f$ is designed to induce a Bayes update, such that the transformed particles empirically constitute an approximate posterior $q^{\alpha_{\vartheta}}(\bar{z}_{n},\bar{h}_{n})$ that statistically resembles the true posterior $p^{+}(\bar{z}_{n},\bar{h}_{n})$.

In particular, we sample the prior particles $(z_{n}^{i},h_{n}^{i})\sim p_{\theta}(z_{n}|{h}_{n})\,\delta_{\theta}(h_{n}|\bar{z}_{n-1}^{i},\bar{h}_{n-1}^{i})$ using the RNN and prior models (8a)-(8b), given transformed particles from the previous sample. Incorporating the RNN parameterization (8) followed by the particle flow into our SBO (6), we obtain

where we obtain the third objective using the density transformation $q^{\alpha_{\vartheta}}(\bar{z}_{n}|{h}_{n})=p_{\theta}({z}_{n}|{h}_{n})/|{D}_{z}\alpha_{\vartheta}|$. Here, $|\cdot|$ denotes the determinant operator, and $D\alpha_{\vartheta}$ denotes the Jacobian of $\alpha_{\vartheta}$ with respect to its input variables.

Integrating particle flow into our framework provides two key advantages. First, it results in an end-to-end learning framework that eliminates the disjointed encoder model in VRNN. This prevents unstable KLD gradients that arises from divergent prior and encoder distributions [34]. Second and more importantly, it preserves the expressive and generalization powers of the approximate posterior $q^{\alpha_{\vartheta}}(\bar{z}_{n}|\bar{h}_{n})$ by overcoming the restrictive variational approximation. Nevertheless, it is not possible to directly optimize the KLD in (9) due to the intractable true posterior $p^{+}(\bar{z}_{n},\bar{h}_{n})$. Taking this into account, we propose the particle flow in the following section.

In this subsection, we propose a physics-inspired particle flow that draws inspiration from the control-oriented approaches [35, 36] to Bayesian inference. The proposed particle flow simulates fluid advection, where the flow (transport) of fluid is driven by an irrotational potential-generated velocity field. We provide a planar illustration of the particle flow in Fig. 3. Subsequently, we derive a tractable flow objective that solves the minimum KLD problem in the SBO (9). Consequently, the particle flow learns to inherently perform a Bayes update on the RNN-extracted features towards constructing a representative approximate posterior.

Our proposed particle flow takes the following form:

where $\nabla$ denotes the Del operator (gradient) with respect to $(z,h)$. Here, we parameterize the normalized (mean-subtracted) velocity potential $\phi_{\vartheta}:\mathbb{R}^{n_{h}+n_{y}}\rightarrow\mathbb{R}$ as FCNN. By discretizing (10) using a forward Euler scheme with step size $\Delta_{\tau}$, we obtain the infinitesimal piecewise-linear transformation given by:

Here, $({z}^{i}_{\tau},{h}^{i}_{\tau})$ represent the trajectories traced out by the feature particles, as they undergo the particle flow over the defined pseudo-time interval $\tau\in[0,1]$. Therefore, the particle trajectories begin at the initial coordinates $({z}_{n}^{i},{h}_{n}^{i})=({z}_{0}^{i},{h}_{0}^{i})$ and end at final coordinates $(\bar{z}_{n}^{i},\bar{h}_{n}^{i})=({z}_{1}^{i},{h}_{1}^{i})$.

As a direct consequence, the proposed particle flow (10) induces an evolution in the resulting empirical distribution $q^{\alpha_{\vartheta}}({z}_{\tau},{h}_{\tau})$, which can be described by a Kolmogorov forward equation as follows:

where $\operatorname{\mathrm{Div}}$ denotes the divergence operator $(\nabla\cdot)$. Upon closer examination, it becomes apparent that this forward equation is in fact a continuity equation which governs the advection of fluid, uncovering its fundamental connection to physics. In particular, $q^{\alpha_{\vartheta}}$ corresponds to the fluid density, and $\phi_{\vartheta}$ represents the driving velocity potential. The resulting gradient $\nabla\phi_{\vartheta}$ acts as the irrotational (conservative) velocity field, as illustrated by the (coloured) arrows in Fig. 3.

Next, we derive a tractable flow objective for the proposed particle flow (10)-(11) using the calculus of variations in two stages, so that it induce a Bayes update of the RNN-extracted prior feature particles. In the first stage, Proposition 1 shows that solving the minimum KLD problem is equivalent to solving a partial differential equation (PDE). In the second stage, Proposition 2 yields a tractable particle flow objective that provides a weak solution to the aforementioned PDE.

To summarize, Propositions 1 and 2 have allowed us to reformulate the intractable minimum KLD problem (13) as a tractable particle flow objective (16). As a result, by optimizing the velocity potential $\phi_{\vartheta}$ with respect to (16) and transporting the prior samples through the corresponding infinitesimal transformations (11), we are able to perform exact, non-variational Bayesian update of the RNN-extracted feature particles to obtain an empirical approximate posterior that accurately resembles the true posterior in a statistical sense.

The minimum covariance objective in (16) plays a crucial role as it ensures that a decrease in the NIS (15) corresponds to an increase in the velocity potential. The gradient $\nabla\phi_{\vartheta}$ consistently points in the direction of greatest potential ascent, and by minimizing the covariance, we guarantee that the potential-generated velocity field drives the flow of latent particles towards the high probability regions of true posterior distribution, as illustrated in Fig. 3. These regions, characterized by low NIS values, are where the true posterior is concentrated. As a result, the proposed particle flow inherently performs a Bayes update of the RNN-extracted features to facilitate effective learning and adaptation.

Given that the sampling rate is high enough to accurately represent the underlying system dynamics with negligible sampling error, we can set $\Delta_{\tau}=1$ and perform the particle flow (10)-(11) in a single step. Additionally, to improve data characterization in the velocity potential, we incorporate a measurement encoding FCNN model $\varphi_{\vartheta}^{x}(x_{n})$ to extract useful features from the data. After incorporating these considerations and replacing the minimum KLD problem in (9) with the particle flow objective (16), the final SBO for our DPFB framework is given by:

Here, we opted to exclude the third objective in (9) for our implementation to avoid the computation of the expensive Jacobian determinant term.

On that note, we conclude the development of our proposed DPFB framework. Overall, the framework intrinsically performs an end-to-end expectation-maximization algorithm. In the maximization step, the stochastic RNN learns to model a generative (features) prior and a (data) decoder likelihood that captures the distributional uncertainties of the cross-domain data and source labels. Using the pair of feature prior and data likelihood, the expectation step construct a cohesive approximate (features) posterior that is representative of the unsupervised data across domains. These steps performed together, allow the framework to learn a domain-adaptive probabilistic feature representation that accurately characterizes complex cross-domain system dynamics and effectively facilitates cross-domain soft sensor modelling. Fig. 2 outlines the particle flow operation in DPFB, where it bridges the (features) prior and the (data) decoder.

The Cranfield Multiphase Flow Process (MFP) [38] is a real industrial process that employs advanced condition monitoring techniques based on heterogeneous sources. The Cranfield MFP is designed to provide a controlled and measured flow rate of water, oil, and air to a pressurized system. The process flow diagram is illustrated in Fig. 4, and a detailed description of the process can be found in [38].

During the data collection process, the air and water flow rates were deliberately varied between the set points (operating conditions) listed in Table I to obtain a good variety of process changes and capture a wide range of distinct dynamics. For this case study, the variables that require specialized measurement sensors, including pressure, flow rate, and density, are identified as the states that need to be predicted using the proposed soft sensor model. The remaining variables are considered as measurement data, which the model utilizes for state prediction. Table II provides a list of all MFP variables. All variables are sampled at 1 Hz.

To evaluate the cross-domain soft sensing performance of our proposed framework, we compare it against several state-of-the-art UDA approaches based on deep autoencoder: DLSN [7], domain adversarial: DARNN [11], variational Bayes: MVI [15] and a combination of these concepts: VRADA [19], InfoVDANN [20] and DPTR [21]. Note that we have extended DSLN and MVI based on their respective sequential counterparts [39, 40] to allow for time series regression. The RNNs in these baseline approaches are designed to be single-layered Long Short-Term Memory (LSTM) networks for capturing complex temporal patterns. Additionally, we include a stacked (three-layered) LSTM [26] trained on complete source and target labels (termed LSTM-C) to benchmark the overall UDA performances. Furthermore, to outline the importance of the physic-inspired particle flow in our proposed framework, we replace it with two widely recognized normalizing flows: IAF [31] and iResNet [41]. To demonstrate the resilience of the proposed framework to particle degeneracy, we also compare it with a deep probabilistic model based on particle filtering: FIVO [42]. All these baselines are trained with missing target labels, and their architectures are designed to match the number of model parameters in DPFB.

The MFP dataset contains three data sequences with 13200, 10372 and 9825 data points, respectively. The first two are used for training and the last one is used for testing. The target (state) labels in the testing dataset serve only as ground-truths for result validation. For the purpose of unsupervised domain adaptation, the data region in which the air flow rate variables range from 0.0278 - 0.0347 m³/s is considered as the source domain, which constitutes 45% of the entire training data. The region where the air flow rate is outside of this range is considered as the target domain, where the state labels are assumed to be missing. Taking into account the conditions, the time series data (and labels) consist of time windows that switch between the two domains, as shown in Fig. 5.

In terms of model architecture, our proposed DPFB framework employs a single-layered Gated Recurrent Unit (GRU) with a hidden state $h_{n}\in\mathbb{R}^{512}$ of size 512 as the RNN model $\varphi_{\theta}^{\text{rnn}}$. The prior model $\varphi_{\theta}^{prior}$ and the decoder model $\varphi_{\theta}^{dec}$ implemented as FCNNs with hidden layers {256, 128} and {512, 256, 128}, respectively. Here, each entry of the curly brackets is a hidden layer and the value denotes layer width. The state and measurement encoding model $\varphi_{\theta}^{x}$, $\varphi_{\theta}^{y}$ are also implemented as FCNNs with hidden layers {256} and {128}, respectively. The velocity potential $\phi_{\vartheta}$ is modeled as a bottleneck FCNN with hidden layers {512, 256, 128}. These model hyperparameters are chosen based on training loss.

All proposed and baseline models are trained on batches of data and label sequences with a fixed length of 100. We optimize the models using the Adam optimizer with annealing and L2 regularization of 0.01, for 300 epochs. The batch sizes and initial learning rates are selected from 8, 16, 32 and 5$\times\text{10}^{\text{-4}}$, 1$\times\text{10}^{\text{-4}}$, 5$\times\text{10}^{\text{-5}}$, respectively. We use 8 sample particles during training, and a single particle without reparameterization during inference.

In this section, we evaluate the performances of DPFB and baselines using the metrics: State Prediction Error: the normalized root-mean-square error (NRMSE) between the predicted states and the ground-truth labels, to assess the accuracy of point state estimates based on the approximate posterior. Coefficient of Determination: the proportion of variation (R${}^{\text{2}}$) between the predicted states and the ground-truths, to assess the unbiasedness and sharpness of the approximate posterior.

Table III shows the comparison results. Overall, our proposed DPFB outperforms the state-of-the-art VB and DAT methods, achieving the lowest state NRMSE and the highest R${}^{\text{2}}$ among the baselines. Furthermore, DPFB performs better with particle flow than the normalizing flows, IAF and i-ResNet, due to the Bayes update performed via the proposed physics-inspired particle flow. Unlike FIVO which attains low R² scores due to repeated importance weighting and resampling, DPFB exhibits robustness to the particle degeneracy issue as it does not involve these particle filtering operations. Notably, the difference between the DPFB and the baselines grows larger on the unsupervised target domains consisting of unseen system dynamics. Moreover, the DPFB achieves scores that are closest to the fully-supervised LSTM-C benchmark. These results demonstrate the outstanding ability of DPFB in producing accurate state predictions amidst the missing target labels. Additionally, they show the DPFB’s capability to perform effective UDA on time series data with switching domains. By virtue of this capability, our framework addresses the cross-domain soft sensing problem in dynamic systems with varying operating conditions.

Fig. 5 shows the validation results of the state predictions against the ground truth labels. Notably, during the unsupervised time windows where the system operates in the target domains, the predictions of our proposed DPFB are able to track the ground truths more closely than the second-best performing VRADA. This is particularly evident in the air flow rate predictions in the third row of Fig. 5. Unlike VRADA, the air flow rate prediction of DPFB is not restricted within the predetermined source domain range of 0.0278 - 0.0347 m³/s. This capability of DPFB to generalize predictions beyond the source domains is due to its competence in accurately performing soft sensing (state inference) using unsupervised data. This is achieved through the proposed particle flow, which circumvents the variational approximation of variational Bayes and facilitates exact (non-variational) Bayesian inference of the state labels across both supervised and unsupervised domains.

Fig. 6 compares the PCA embeddings (using cosine kernel) of the extracted latent state features and the PCA embeddings of the ground truth labels. The latent state embeddings generated by our proposed DPFB model are found to be most consistent with the ground truth embeddings. Additionally, the PCA embeddings of the source and target domains exhibit less overlap in both the DPFB and ground truth embeddings compared to the other baseline methods. This can be attributed to the dispensing of domain-invariant enforcing techniques such as DAT in our DPFB framework, which helps in retaining the rich representation of the extracted posterior feature space. Consequently, the models can construct an inclusive joint (time) feature space capable of adopting complex cross-domain state representations of dynamic systems. These results illustrate the DPFB is more effective in generating accurate state predictions and constructing a cohesive probabilistic feature representation that better captures the underlying characteristics of the unsupervised data.

Fig. 7 presents the t-SNE embeddings of the extracted hidden state features from the RNN-based models. The hidden state embeddings of our proposed DPFB exhibit a structured temporal arrangement that closely resembles that of those of the fully-supervised LSTM-C benchmark. This similarity demonstrates the ability of the proposed particle flow to endow the RNN hidden states with temporal characteristics that are inherent the dynamic system. As a result, the endowed temporal coherence facilitates a smooth transition between the source and target time windows, enabling effective cross-domain soft sensor modelling. In contrast, baseline models that lack this temporal coherence experience difficulty in transitioning between the domains and leads to poor performance. These results highlight the importance of subjecting the hidden states to a Bayes update via the proposed particle flow.