Asymptotic Security using Bayesian Defense Mechanism with Application to Cyber Deception

Model-based security analysis helps the system designer to prioritize security investments [11]. Attack graphs [12] and attack trees [13] are basic models of vulnerabilities, attacks, and consequences. Incorporating defensive actions into the graphical representation induces defense trees [14]. For dynamic models, attack countermeasure trees, partially observable MDP, and Bayesian network model have been used [15, 16, 17]. Those probabilistic models naturally lead to Bayesian defense mechanisms, such as Bayesian intrusion detection [18, 19], Bayesian intrusion response [20], and Bayesian security risk management [21]. Meanwhile, the model of the dynamical system to be protected is also used for control system security [22, 23]. For example, identifying existence of stealthy attacks and removing the vulnerability require the dynamical model [24, 25], and attack detection performance can be enhanced by model knowledge [26]. Our Bayesian defense mechanisms can be interpreted as a generalization of those approaches. This work reveals a fundamental property of such commonly used model-based defense schemes.

Game theory is a standard approach to modeling the decision making in cyber security, where there inevitably arises a need to address strategic interactions between the attacker and the defender [27, 28]. In particular, games with incomplete information play a crucial role in deceptive situations [29, 30, 31, 32]. The modeling in this study follows the signaling game framework in [33, 34]. Our main concern is especially on asymptotic phenomena in the dynamic deception and effectiveness of model knowledge.

Our finding is based on analysis of an asymptotic behavior of Bayesian inference. The convergence property of Bayesian inference on the true parameter is referred to as Bayesian consistency, which has been investigated mainly in the context of statistics [35, 36]. However, those existing results are basically applicable only to independent and identically distributed (i.i.d.) samples because the discussion mostly relies on the strong law of large numbers (SLLN). Although there is an extension to Markov chains [37], the observable variable in our work is not Markov. Indeed, sophisticated attackers can choose strategies such that the states at all steps are correlated with the entire previous trajectory. Thus, existing results for Bayesian consistency cannot be applied to our problem in a straightforward manner.

Preliminary versions of this work have been presented in [38, 39], but they made the claim of Theorem 2 as an assumption rather than proving it. Moreover, they did not include rigorous proofs of the claims in Section 3 and analysis of the bluffing proposed in Section 4.

In Section 2, we present a motivating example of water supply networks, and subsequently, formulate the decision making as a stochastic signaling game. Section 3 analyzes the consequence of the formulated game and shows that Bayesian defense mechanisms can achieve asymptotic security of the system to be protected. In Section 4, we analyze a defensive deception that utilizes asymmetric recognition as an application of the finding of Section 3. The game of interest is reformulated using the Mertens-Zamir model. It is shown that the attacker possibly stops the execution even if the defender is unaware of the exploited vulnerabilities, as long as the defender’s belief is concealed. Section 5 verifies the theoretical results through numerical simulation. Finally, Section 6 concludes and summarizes the paper.

Let $\mathbb{N}$, ${\mathbb{Z}_{+}}$, and $\mathbb{R}$ be the sets of natural numbers, nonnegative integers, and real numbers, respectively. The $k$-ary Cartesian power of the set $\mathcal{X}$ is denoted by $\mathcal{X}^{k}.$ The tuple $(x_{0},\ldots,x_{k})$ is denoted by $x_{0:k}$. The cardinality of a set $\mathcal{X}$ is denoted by $|\mathcal{X}|$. For a set $\mathcal{X},$ the Kronecker delta denoted by $\delta:\mathcal{X}\times\mathcal{X}\to\{0,1\}$ is defined by $\delta(x,y)=1$ if $x=y$ and $\delta(x,y)=0$ otherwise. The $\sigma$-algebra generated by a random variable $X$ is denoted by $\sigma(X)$. For a sequence of events $E_{k}$ for $k\in\mathbb{N}$, the supremum set $\cap_{N=1}^{\infty}\cup_{k=N}^{\infty}E_{k}$, namely, the event where $E_{k}$ occurs infinitely often, is denoted by $\{E_{k}\ {\rm i.o.}\}$. Jensen’s inequality, which is often applied in this paper, is given as follows: For a real convex function $\varphi$ and a finite set $\mathcal{X}$, the inequality

$$\textstyle{\sum_{x\in\mathcal{X}}p(x)\varphi(a(x))\geq\varphi\left(\sum_{x\in\mathcal{X}}p(x)a(x)\right)}$$

(1)

holds where $a:\mathcal{X}\to\mathbb{R}$ and $p:\mathcal{X}\to[0,1]$ that satisfies the equation $\sum_{x\in\mathcal{X}}p(x)=1$. The inequality is reversed if $\varphi$ is concave. The generalized Borel-Cantelli’s second lemma is given as follows [40, Theorem 4.3.4]: Let $\mathcal{F}_{k}$ for $k\in{\mathbb{Z}_{+}}$ be a filtration of a probability space $(\Omega,\mathcal{F},\mathbb{P})$ with $\mathcal{F}_{0}:=\{\emptyset,\Omega\}$ and let $E_{k}$ for $k\in{\mathbb{Z}_{+}}$ be a sequence of events with $E_{k}\in\mathcal{F}_{k+1}$. Then

$$\textstyle{\{E_{k}\ {\rm i.o.}\}=\left\{\omega\in\Omega:\sum_{k=0}^{\infty}\mathbb{P}(E_{k}|\mathcal{F}_{k})(\omega)=\infty\right\}.}$$

(2)

The appendix contains the proofs of the claims made in the paper.

As a motivating example, we consider water distribution networks (WDNs), which supply drinking water of suitable quality to customers. Because of their indispensability to our life, WDNs are an attractive target for adversaries and expose their architecture to cyber-physical attacks [41]. In particular, we treat the water tank system illustrated by Fig. 1, where a tank is connected to a reservoir within a WDN. The amount of the water in the tank varies due to usage for drinking and flow between the external network. Thus the tank system is needed to be properly controlled through actuation of the pump and the valve to keep the water amount within a desired range [42]. A programmable logic controller (PLC) transmits on/off control signals to the pump and the valve monitoring the state, namely, the water level of the tank. The dynamics is modeled as a MDP, where the state space and the action space are given by quantized water levels and finite control actions. Interaction to the external network is modeled as the randomness in the process.

We here suppose an attack scenario considered in [43]. The adversary succeeds to hijack the PLC and can directly manipulate its control logic. Such an intrusion can be carried out by stealthy and evasive maneuvers in advanced persistent threats [44]. The objective of the attack is to damage the system by causing water overflow through inappropriate control signals without being detected. To deal with this attack, we consider a Bayesian defense mechanism, which utilizes the data of the monitored state and forms her belief on existence of the attacker based on the system model. The Bayesian defense mechanism chooses a proper reaction by identifying if the system is under attack through an observation of the state. If the system’s behavior is highly suspicious, for example, the defense mechanism takes an aggressive reaction such as log analysis, dispatch of operators, or emergency shutdown.

The defender’s belief on the existence of an attacker plays a key role to analyze the consequence of the threat. When the attacker naively executes an attack, the system’s behavior becomes different from the one of the normal operation and accordingly the belief increases. On the other hand, if the attacker chooses sophisticated attacks that deceive the defender, the belief may decrease. Our main interest in this study is to investigate the defense capability achieved by the Bayesian defense mechanism.

We introduce the general description based on dynamic games with incomplete information. In particular, we refer to the game as a stochastic signaling game where the system’s dynamics is given as an MDP and the type of a player is unknown to the opponent.

The system to be protected with a Bayesian defense mechanism is depicted in Fig. 2. The system is modeled by a finite MDP governed by two players as in standard stochastic games. Formally, the MDP considered in this paper is given by the tuple $\mathcal{M}:=(\mathcal{X},\mathcal{A},\mathcal{R},P,P_{0})$ where $\mathcal{X}$ is a finite state space, $\mathcal{A}$ and $\mathcal{R}$ are finite action spaces, $P:\mathcal{X}\times\mathcal{X}\times\mathcal{A}\times\mathcal{R}\to[0,1]$ is a transition probability, and $P_{0}:\mathcal{X}\to[0,1]$ is the probability distribution of the initial state. The state at the $k$th step is denoted by $x_{k}\in\mathcal{X}$. There is an agent who can alter the system through an action $a_{k}\in\mathcal{A}$ for $k\in{\mathbb{Z}_{+}}$. We refer to the agent as sender as in standard signaling games. Based on the measured output, the Bayesian defense mechanism, called a receiver, chooses an action $r_{k}\in\mathcal{R}$ at each time step. We henceforth refer to $r_{k}$ as a reaction for emphasizing that $r_{k}$ denotes a counteraction against potentially malicious attacks. The system dynamics is given by $P$, where the transition probability from $x$ to $x^{\prime}$ with $a$ and $r$ is denoted by $P(x^{\prime}|x,a,r)$. To eliminate the possibility of trivial stealthy attacks, we assume that the system’s behavior varies in a stochastic sense when different actions are taken.

Next, we determine the class of the decision rules. Let $\theta\in{\it\Theta}$ denote the type of the sender. For simplicity, the type is assumed to be binary, i.e., ${\it\Theta}=\{\theta_{\rm b},\theta_{\rm m}\},$ where $\theta_{\rm b}$ and $\theta_{\rm m}$ correspond to benign and malicious senders, respectively. The types $\theta_{\rm b}$ and $\theta_{\rm m}$ describe the situations where there does not and does exist an adversary, respectively. The true type $\theta$ is known to the sender, but unknown to the receiver. Let $\bar{s}^{\rm s}:=(\bar{s}^{\rm s}_{k})_{k\in{\mathbb{Z}_{+}}}$ and $\bar{s}^{\rm r}:=(\bar{s}^{\rm r}_{k})_{k\in{\mathbb{Z}_{+}}}$ denote the sender’s and receiver’s pure strategy, respectively. It is assumed that the receiver’s available information about the sender type is only the state, i.e., she cannot observe her instantaneous utility, defined below, nor the sender’s action. Similarly, it is assumed that the sender can observe only the state and her action. The strategies at the $k$th step with the available information are given by $\bar{s}^{\rm s}_{k}:{\it\Theta}\times\mathcal{H}^{\rm s}_{k}\to\mathcal{A},$ and $\bar{s}^{\rm r}_{k}:\mathcal{H}^{\rm r}_{k}\to\mathcal{R}$ where $h^{\rm s}_{k}\in\mathcal{H}^{\rm s}_{k}$ and $h^{\rm r}_{k}\in\mathcal{H}^{\rm r}_{k}$ are histories at the $k$th step given by $h^{\rm s}_{k}=(x_{0:k},a_{0:k-1})$ and $h^{\rm r}_{k}=(x_{0:k},r_{0:k-1}).$ Note that the resulting state trajectory is not Markov since the strategies depend on the entire history. Because we consider pure strategies, it suffices to consider the state-history dependent strategies $s^{\rm s}_{k}:{\it\Theta}\times\mathcal{X}^{k+1}\to\mathcal{A}$ and $s^{\rm r}_{k}:\mathcal{X}^{k+1}\to\mathcal{R}$, recursively defined by

$$\begin{array}[]{l}s^{\rm s}_{k}(\theta,x_{0:k}):=\bar{s}^{\rm s}_{k}(\theta,x_{0:k},s^{\rm s}_{0:k-1}(x_{0:k-1})),\\ s^{\rm r}_{k}(x_{0:k}):=\bar{s}^{\rm r}_{k}(x_{0:k},s^{\rm r}_{0:k-1}(x_{0:k-1})).\end{array}$$

(4)

The strategy profile is denoted by $s:=(s^{\rm s},s^{\rm r}).$ The sender’s and receiver’s admissible strategy sets are denoted by $\mathcal{S}^{\rm s}$ and $\mathcal{S}^{\rm r}$, respectively. The set of admissible strategy profiles is denoted by $\mathcal{S}:=\mathcal{S}^{\rm s}\times\mathcal{S}^{\rm r}$. Note that, although we do not specify $\mathcal{S}$ here, it can be taken to be any set of state-history dependent strategies. While we consider a general strategy set in Sec. 3, we impose a constraint on $\mathcal{S}$ in Sec. 4.

Once a strategy profile is fixed, the stochastic property of the system is induced. Construct the canonical measurable space $(\Omega,\mathcal{F})$ of the MDP with the sender type where $\Omega:={\it\Theta}\times\Pi_{k=0}^{\infty}(\mathcal{X}\times\mathcal{A}\times\mathcal{R})$ and $\mathcal{F}$ is its product $\sigma$-algebra [45, Chapter 2]. We denote $\omega=(\theta,(x_{0},a_{0},r_{0}),(x_{1},a_{1},r_{1}),\ldots)\in\Omega$. The random variables $\Theta,X_{k},A_{k},$ and $R_{k}$ are defined on the measurable space $(\Omega,\mathcal{F})$ by the projections of $\omega$ such that $\Theta(\omega):=\theta,X_{k}(\omega):=x_{k},$ $A_{k}(\omega):=a_{k},$ $R_{k}(\omega):=r_{k}.$ The probability measure on $(\Omega,\mathcal{F})$, induced by $s$, is denoted by $\mathbb{P}^{s}$, which satisfies

$$\left\{\begin{array}[]{l}\mathbb{P}^{s}(X_{0}=x_{0})=P_{0}(x_{0}),\\ \mathbb{P}^{s}(A_{k}=a_{k}|\Theta=\theta,X_{0:k}=x_{0:k})=\delta(a_{k},s^{\rm s}_{k}(\theta,x_{0:k})),\\ \mathbb{P}^{s}(R_{k}=r_{k}|X_{0:k}=x_{0:k})=\delta(r_{k},s^{\rm r}_{k}(x_{0:k})),\\ \mathbb{P}^{s}(X_{k+1}=x_{k+1}|X_{0:k}=x_{0:k},A_{k}=a_{k},R_{k}=r_{k})\\ \quad=P(x_{k+1}|x_{k},a_{k},r_{k}),\\ \mathbb{P}^{s}(\Theta=\theta)=\pi_{0}(\theta)\end{array}\right.$$

(5)

for any $k\in{\mathbb{Z}_{+}}$ with the initial distribution of the sender type $\pi_{0}:{\it\Theta}\to[0,1]$. We denote the conditional probability $\mathbb{P}^{s}(\cdot|\Theta=\theta)$ by $\mathbb{P}^{s}_{\theta}$. To simplify the notation, we denote the conditional probability mass function with type $\theta$ by

$$p^{s}_{\theta}(x_{k+1}|x_{0:k}):=\mathbb{P}^{s}_{\theta}(X_{k+1}=x_{k+1}|X_{0}=x_{0},\ldots,X_{k}=x_{k}).$$

(6)

The expectation with respect to $\mathbb{P}^{s}$ is denoted by $\mathbb{E}^{s}$.

We introduce each player’s belief on the uncertain variables next. The receiver’s belief at the $k$th step is given by

$$\begin{array}[]{l}\pi^{\rm r}_{k}(\theta,a_{0:k-1}|x_{0:k},r_{0:k-1})\\ :=\mathbb{P}^{s}(\Theta=\theta,A_{0:k-1}=a_{0:k-1}|X_{0:k}=x_{0:k},R_{0:k-1}=r_{0:k-1})\end{array}$$

(7)

for $k\in{\mathbb{Z}_{+}}$. The belief can be recursively computed by Bayes’ rule

$$\begin{array}[]{l}\pi^{\rm r}_{k+1}(\theta,a_{0:k}|x_{0:k+1},r_{0:k})=\delta(a_{k},s^{\rm s}_{k}(\theta,x_{0:k}))\\ \times\dfrac{P(x_{k+1}|x_{k},s^{\rm s}_{k}(\theta,x_{0:k}),r_{k})\pi^{\rm r}_{k}(\theta,a_{0:k-1}|x_{0:k},r_{0:k-1})}{\sum_{\phi\in{\it\Theta}}P(x_{k+1}|x_{k},s^{\rm s}_{k}(\phi,x_{0:k}),r_{k})\pi^{\rm r}_{k}(\phi,a_{0:k-1}|x_{0:k},r_{0:k-1})}\end{array}$$

(8)

when the denominator is nonzero. To simplify notation, we introduce the receiver’s belief only of the sender type:

$$\pi^{\rm r}_{k}(\theta|x_{0:k}):=\pi^{\rm r}_{k}(\theta,s^{\rm s}_{0:k-1}(\theta,x_{0:k-1})|x_{0:k},s^{\rm r}_{0:k-1}(x_{0:k-1})),$$

(9)

which follows Bayes’ rule

$$\begin{array}[]{l}\pi^{\rm r}_{k+1}(\theta|x_{0:k+1})\\ =\dfrac{P(x_{k+1}|x_{k},s^{\rm s}_{k}(\theta,x_{0:k}),s^{\rm r}_{k}(x_{0:k}))\pi^{\rm r}_{k}(\theta|x_{0:k})}{\sum_{\phi\in{\it\Theta}}P(x_{k+1}|x_{k},s^{\rm s}_{k}(\phi,x_{0:k}),s^{\rm r}_{k}(x_{0:k}))\pi^{\rm r}_{k}(\phi|x_{0:k})}.\end{array}$$

(10)

The sender’s belief can similarly be defined and is denoted by $\pi^{\rm s}_{k}(r_{0:k-1}|\theta,x_{0:k},a_{0:k-1})$.

In Sec. 3, the initial beliefs are assumed to be known to both players, i.e., we make the common prior assumption. Since we consider pure strategies, $r_{0:k-1}$ is uniquely determined by $x_{0:k-1}$ once the strategy is fixed. Hence, the sender’s belief does not appear explicitly in Sec. 3. On the other hand, in Sec. 4, we consider the case where the initial belief is unknown to the sender, modeling the possibility of bluffing.

Let $U^{\rm s}:{\it\Theta}\times\mathcal{X}\times\mathcal{A}\times\mathcal{R}\to\mathbb{R}$ be the sender’s instantaneous utility. For a given strategy profile $s\in\mathcal{S}$ and type $\theta\in{\it\Theta}$, the sender’s expected average utility at the $k$th step with the horizon length $T$ is given by

$$\begin{array}[]{l}\bar{U}^{\rm s}_{k,T}(s_{k:k+T}|\theta,x_{0:k})\\ \displaystyle{:=\mathbb{E}^{s}\left[\dfrac{1}{T+1}\left.\sum_{\tau=k}^{k+T}U^{\rm s}(\Theta,X_{\tau},s^{\rm s}_{\tau}(\Theta,X_{0:\tau}),s^{\rm r}_{\tau}(X_{0:\tau}))\right|\theta,x_{0:k}\right].}\end{array}$$

(11)

Similarly, with the receiver’s instantaneous utility given by $U^{\rm r}:{\it\Theta}\times\mathcal{X}\times\mathcal{A}\times\mathcal{R}\to\mathbb{R}$, the receiver’s expected average utility at the $k$th step with the horizon length $T$ is given by

$$\begin{array}[]{l}\bar{U}^{{\rm r}}_{k,T}(s_{k:k+T}|x_{0:k})\\ \displaystyle{:=\mathbb{E}^{s}\left[\dfrac{1}{T+1}\left.\sum_{\tau=k}^{k+T}U^{\rm r}(\Theta,X_{\tau},s^{\rm s}_{\tau}(\Theta,X_{0:\tau}),s^{\rm r}_{\tau}(X_{0:\tau}))\right|x_{0:k}\right].}\end{array}$$

(12)

We denote the limits by $(\bar{U}^{{\rm s}}_{k},\bar{U}^{{\rm r}}_{k}):=\lim_{T\to\infty}(\bar{U}^{\rm s}_{k,T},\bar{U}^{{\rm r}}_{k,T})$ assuming they exist. Under this notation, the strategy profile $s=(s^{\rm s},s^{\rm r})$ is said to be a perfect Bayesian equilibrium (PBE) if

$$\left\{\begin{array}[]{l}s^{\rm s}_{k:\infty}\in{\rm BR}^{\rm s}_{k}(s^{\rm r}_{r:\infty}|\theta,x_{0:k}),\quad\forall\theta\in{\it\Theta},\\ s^{\rm r}_{k:\infty}\in{\rm BR}^{\rm r}_{k}(s^{\rm s}_{k:\infty}|x_{0:k})\end{array}\right.$$

(13)

for any $k\in{\mathbb{Z}_{+}}$ and $x_{0:k}\in\mathcal{X}^{k+1}$ where ${\rm BR}^{\rm s}_{k}$ and ${\rm BR}^{\rm r}_{k}$ are best responses defined by

$$\begin{array}[]{l}{\rm BR}^{\rm s}_{k}(s^{\rm r}_{k:\infty}|\theta,x_{0:k}):=\displaystyle{\operatorname*{arg\,max}_{\tilde{s}^{\rm s}_{k:\infty}\in\mathcal{S}^{\rm s}_{k:\infty}}\ \bar{U}^{{\rm s}}_{k}((\tilde{s}^{\rm s}_{k:\infty},s^{\rm r}_{k:\infty})|\theta,x_{0:k})},\\ {\rm BR}^{\rm r}_{k}(s^{\rm s}_{k:\infty}|x_{0:k}):=\displaystyle{\operatorname*{arg\,max}_{\tilde{s}^{\rm r}_{k:\infty}\in\mathcal{S}^{\rm r}_{k:\infty}}\ \bar{U}^{{\rm r}}_{k}((s^{\rm s}_{k:\infty},\tilde{s}^{\rm r}_{k:\infty})|x_{0:k})}.\end{array}$$

(14)

Note that, our analysis can be extended to the case of general objective functions rather than expected average utilities as long as the adversary with the utilities avoids being detected, which is formally stated in Definition 1 below.

We define the game formulated above by

$$\mathcal{G}_{1}:=(\mathcal{M},\mathcal{S},U,{\it\Theta},\pi_{0}),$$

(15)

where the initial belief is common information. This game belongs to the class of incomplete, imperfect, and asymmetric information stochastic games. Owing to the existence of the type $\theta,$ which is unknown to the receiver, the information is incomplete. Because the actions taken by each player are unobservable to the opponent, the information is imperfect and asymmetric. Although investigating existence and computing equilibria of the game are challenging, we discuss properties of equilibria on the premise that they exist and are given because our interest here lies in the consequences for the threat.

The random variable of the belief on the type $\theta\in{\it\Theta}$ at the $k$th step $\pi^{\theta}_{k}:\Omega\to[0,1]$ is given by

$$\pi^{\theta}_{k}(\omega):=\pi^{\rm r}_{k}(\theta|X_{0:k}(\omega)).$$

(16)

Recall that $\pi^{\theta}_{k}$ represents the defender’s confidence on existence of an attacker. If the belief is low in spite of existence of malicious signals, this means that the Bayesian defense mechanism is deceived. Because we are interested in whether the Bayesian defense mechanism is permanently deceived, or not, we examine asymptotic behavior of the belief.

We first investigate increment of the belief sequence. The following lemma is key to our analysis.

Lemma 1 roughly implies that the expectation of the belief on the true type is non-decreasing. As a direct conclusion of this lemma, the following theorem holds.

Theorem 1 implies that the belief has a limit even if an intermittent attack is executed. Fig. 3 depicts the distributions of the belief sequence when there exists an attacker. Owing to the model knowledge, if the adversary stops the attack at some time step then the belief is invariant, which is illustrated as the transition of the belief at $k=1$ in Fig. 3. Moreover, the expectation of the belief is non-decreasing over time as claimed by Lemma 1. Thus, there exists a limit $\pi^{\theta_{\rm m}}_{\infty}$ as shown at the right of Fig. 3.

We next investigate the limit. An undesirable limit is $\pi^{\theta}_{\infty}=0$, which means that the defender is completely deceived. We show that this does not happen as long as the initial belief is nonzero. The following lemma holds.

Lemma 2 leads to the following theorem.

Theorem 2 implies that the complete deception described by $\pi^{\theta}_{\infty}=0$ does not occur.

Remark: Theorems 1 and 2 can heuristically be justified from an information-theoretic perspective as follows. Suppose that the state sequence $x_{0:k}$ is observed at the $k$th step. Then the belief is given by

$$\begin{array}[]{cl}\pi_{k}(\theta|x_{0:k})&=\dfrac{\pi_{0}(\theta)}{\sum_{\phi\neq\theta}\frac{p^{s}_{\phi}(x_{0:k})}{p^{s}_{\theta}(x_{0:k})}\pi_{0}(\phi)+\pi_{0}(\theta)}\\ &=\dfrac{\pi_{0}(\theta)}{\sum_{\phi\neq\theta}{\rm exp}(kS^{\phi}_{k}(x_{0:k}))\pi_{0}(\phi)+\pi_{0}(\theta)}\end{array}$$

(19)

where $p^{s}_{\theta}(x_{0:k})$ and $p^{s}_{\phi}(x_{0:k})$ are the joint probability mass functions of $x_{0:k}$ with respect to $\mathbb{P}^{s}_{\theta}$ and $\mathbb{P}^{s}_{\phi}$, respectively, and

$$S^{\phi}_{k}(x_{0:k}):=\dfrac{1}{k}\sum_{i=1}^{k}\log\dfrac{p^{s}_{\phi}(x_{i}|x_{0:i-1})}{p^{s}_{\theta}(x_{i}|x_{0:i-1})}.$$

(20)

Assuming that $p^{s}_{\theta}(x_{k}|x_{0:k-1})$ approaches a stationary distribution $p^{s}_{\theta}(x)$ on $\mathcal{X}$ and SLLN can be applied, we have

$$\lim_{k\to\infty}S^{\phi}_{k}=\mathbb{E}_{x\sim p^{s}_{\theta}}\left[\log p^{s}_{\phi}(x)/p^{s}_{\theta}(x)\right]=-D_{\rm KL}(p^{s}_{\theta}||p^{s}_{\phi})$$

(21)

where $D_{\rm KL}$ denotes the Kullback-Leibler divergence. Since $D_{\rm KL}$ is nonnegative for any pair of distributions, $S^{\phi}_{k}$ converges to a nonpositive number, which results in convergence of $\pi^{\theta}_{k}$. If $p^{s}_{\theta}\neq p^{s}_{\phi}$ for any $\phi\in{\it\Theta}\setminus\{\theta\}$, the limit of $S^{\phi}_{k}$ becomes negative, and hence $\lim_{k\to\infty}\exp(kS^{\phi}_{k}(x_{0:k}))=0,$ which leads to

$$\begin{array}[]{cl}\displaystyle{\lim_{k\to\infty}\pi_{k}(\theta|x_{0:k})}&=\dfrac{\pi_{0}(\theta)}{\displaystyle{\sum_{\phi\neq\theta}\lim_{k\to\infty}{\rm exp}(kS^{\phi}_{k}(x_{0:k}))\pi_{0}(\phi)+\pi_{0}(\theta)}}\\ &=1.\end{array}$$

(22)

Thus, the belief of the true type converges to one. Such a convergence property of the Bayesian estimator on the true parameter, referred to as Bayesian consistency, has been investigated mainly in the context of statistics [35, 36]. In this sense, Theorems 1 and 2 can be regarded as another representation of Bayesian consistency. However, note again that this discussion is not a rigorous proof but a heuristic justification because the state is essentially non-i.i.d. and even non-ergodic in our game-theoretic formulation.

It has turned out that the belief has a positive limit. To clarify our interest, we define the notion of detection-averse utilities.

Definition 1 characterizes utilities where the malicious sender avoids having the defender form a firm belief on the existence of an attacker. An example of detection-averse utilities is given in Appendix 7. Naturally, strategies reasonable for the attacker should be detection-averse as long as the defender possesses an effective counteraction. If the utilities of interest are not detection-averse, this means that the defense mechanism cannot cope with the attack because the attacker is not afraid to reveal herself. For protecting such systems, appropriate counteractions should be implemented beforehand.

Suppose that there is an effective countermeasure, and hence the utilities are detection-averse. A simple malicious sender’s strategy that satisfies (23) is to imitate the benign sender’s strategy after a finite number of time steps. We give a formal definition of such strategies.

The objective of this subsection is to show that Bayesian defense mechanisms can restrict all reasonable strategies to be asymptotically benign as long as an effective countermeasure is implemented.

As a preparation for proving our main claim, we investigate the asymptotic behavior of state transition. From Theorems 1 and 2, we can expect that the state eventually loses information on the type, which is justified by the following lemma.

Under Assumption 1, which eliminates the possibility of stealthy attacks, Lemma 3 implies that the actions themselves must be identical. This fact yields the following theorem, one of the main results in this paper.

Theorem 3 implies that the malicious sender’s action converges to the benign action. Equivalently, an attacker necessarily behaves as a benign sender after a finite number time steps. Therefore, the system is guaranteed to be secure in an asymptotic manner, i.e., Bayesian defense mechanisms can prevent deception in an asymptotic sense. This result indicates the powerful defense capability achieved by model knowledge.

The result in Section 3 claims that the defender, namely, the Bayesian defense mechanism, always wins in an asymptotic manner when the stochastic model of the system is available and the vulnerability to be exploited for intrusion is known and modeled. The latter condition is quantitatively described by the condition $\pi_{0}(\theta_{\rm m})>0$. Although the derived result proves a quite powerful defense capability, it is also true that it is almost impossible to be aware of all possible vulnerabilities in advance. Moreover, it is also challenging to implement effective countermeasures for all scenarios and to compute the equilibrium of the dynamic game.

In this section, as an application of the finding in the previous section, we consider defensive deception using bluffing that utilizes asymmetric recognitions between the attacker and the defender. Suppose that an attacker exploits a vulnerability of which the defender is unaware but the attacker is unaware of the defender’s unawareness. Then their recognition becomes asymmetric in the sense that the attacker does not correctly recognize the defender’s recognition of the vulnerability. This situation naturally arises in practice because the defender’s recognition is private information. By utilizing the asymmetric recognition, the defender can possibly deceive the attacker such that the attacker believes that the defender might be aware of the vulnerability and carrying out effective counteractions. Specifically, we consider the bluffing strategies where the system’s state does not possess information about the defender’s belief. For instance, if the defender chooses the reactions that affect only the players’ utilities without influence to the system, the state is independent of the reaction. By concealing the defender’s unawareness, the defender’s recognition, which is quantified by her belief, is completely unknown to the attacker over time.

The defensive deception is possibly able to force the attacker to withdraw even if the defender is actually unaware of the exploited vulnerability. For instance, consider the example in Sec. 2.1 and suppose that emergency shutdown of the system can be carried out by the defender. Suppose also that the attacker wants to keep administrative privileges of the PLC. In this case, the attacker may rationally terminate her evasive maneuvers after a finite number of time steps due to the risk of sudden shutdown. The objective of this section is to show that the hypothesis is true in a formal manner.

The situation of interest in this section is that the defender is unaware of the vulnerability to be exploited but the attacker is not necessarily aware of this unawareness. To address the uncertainty on defender’s recognition, the attacker forms her belief on the defender’s belief. Fig. 4 illustrates the attacker’s belief on the defender’s belief with the common prior assumption, i.e., the initial defender’s belief is known to the attacker, which has been made in the previous section. In this case, the attacker has a firm belief that the defender is unaware of the vulnerability. On the other hand, Fig. 5 illustrates the attacker’s belief without the common prior assumption. Then the attacker’s belief is no longer firm as depicted by the figure. In addition, because of the lack of the common prior assumption, the defender also forms another belief on the attacker’s belief on the defender’s belief on the existence of an attacker. This procedure repeats indefinitely and induces infinitely many beliefs.

The notion of belief hierarchy has been proposed to handle the infinitely many beliefs [9, 10, 46]. A belief hierarchy is formed as follows. Let $\Delta(\cdot)$ denote the set of probability measures over a set. The first-order initial belief is given as $\pi^{1}_{0}\in\Delta({\it\Theta})$, which describes the defender’s initial belief on existence of the attacker. The second-order initial belief is given as $\pi^{2}_{0}\in\Delta(\Delta({\it\Theta}))$, which describes the attacker’s initial belief on the defender’s first-order belief. In a similar manner, the belief at any level is given, and the tuple of beliefs at all levels is referred to as a belief hierarchy.

To handle belief hierarchies, the Mertens-Zamir model has been introduced [9, 10, 46]. The model considers type structure, in which a belief hierarchy is embedded. A type structure consists of players, sets of types, and initial beliefs. In particular, a type structure for our situation of interest can be given by

$$\mathcal{T}=(({\rm s},{\rm r}),({\it\Theta}^{\rm s},{\it\Theta}^{\rm r}),(\pi^{\rm s}_{0},\pi^{\rm r}_{0}))$$

(26)

where $({\rm s},{\rm r})$ represents the sender and the receiver, ${\it\Theta}^{\rm s}$ and ${\it\Theta}^{\rm r}$ represent the sets of player types, and $\pi^{\rm s}_{0}:{\it\Theta}^{\rm r}\times{\it\Theta}^{\rm s}\to[0,1]$ and $\pi^{\rm r}_{0}:{\it\Theta}^{\rm s}\times{\it\Theta}^{\rm r}\to[0,1]$ represent the initial beliefs. The value $\pi^{\rm s}_{0}(\theta^{\rm r}|\theta^{\rm s})$ denotes the sender’s initial belief of the receiver type $\theta^{\rm r}$ when the sender type is $\theta^{\rm s}$, and $\pi^{\rm r}_{0}(\theta^{\rm s}|\theta^{\rm r})$ denotes the corresponding receiver’s initial belief. The first-order initial belief is given by $\pi^{1}_{0}(\theta^{\rm s})=\pi^{\rm r}_{0}(\theta^{\rm s}|\theta^{\rm r})$ for the true receiver type $\theta^{\rm r}\in{\it\Theta}^{\rm r}$, and the second-order initial belief is given by $\pi^{2}_{0}(\pi^{\rm r}(\cdot|\theta^{\rm r})|\theta^{\rm s})=\pi^{\rm s}_{0}(\theta^{\rm r}|\theta^{\rm s})$ for the true sender type $\theta^{\rm s}\in{\it\Theta}^{\rm s}$. By repeating it, the belief at any level of the belief hierarchy can be derived from the type structure. Importantly, for any reasonable belief hierarchy there exists a type structure that can generate the belief hierarchy of interest. For a formal discussion, see [9, 10, 46].

We model the situation of interest by using the binary type sets:

$${\it\Theta}^{\rm s}=\{\theta^{\rm s}_{\rm b},\theta^{\rm s}_{\rm m}\},\quad{\it\Theta}^{\rm r}=\{\theta^{\rm r}_{\rm u},\theta^{\rm r}_{\rm a}\}.$$

(27)

While $\theta^{\rm s}_{\rm b}$ and $\theta^{\rm s}_{\rm m}$ represent benign and malicious senders, respectively, $\theta^{\rm r}_{\rm u}$ and $\theta^{\rm r}_{\rm a}$ represent receivers being unaware and aware of the vulnerability, respectively. The receiver’s initial beliefs are set to

$$\pi^{\rm r}_{0}(\theta^{\rm s}_{\rm b}|\theta^{\rm r}_{\rm u})=1,\quad\pi^{\rm r}_{0}(\theta^{\rm s}_{\rm m}|\theta^{\rm r}_{\rm u})=0$$

(28)

and

$$\pi^{\rm r}_{0}(\theta^{\rm s}_{\rm b}|\theta^{\rm r}_{\rm a})=\alpha,\quad\pi^{\rm r}_{0}(\theta^{\rm s}_{\rm m}|\theta^{\rm r}_{\rm a})=1-\alpha$$

(29)

with $\alpha\in[0,1)$. The initial beliefs mean that, the receiver $\theta^{\rm r}_{\rm u}$ is unaware of the vulnerability and firmly believes that the system is normally operated, while the receiver $\theta^{\rm r}_{\rm a}$ is aware of the vulnerability and suspects existence of an attacker with probability $1-\alpha$. The sender’s initial beliefs are assumed to be given by

$$\pi^{\rm s}_{0}(\theta^{\rm r}_{\rm u}|\theta^{\rm s}_{\rm b})=1,\quad\pi^{\rm s}_{0}(\theta^{\rm r}_{\rm a}|\theta^{\rm s}_{\rm b})=0,$$

(30)

and

$$\pi^{\rm s}_{0}(\theta^{\rm r}_{\rm u}|\theta^{\rm s}_{\rm m})=\beta,\quad\pi^{\rm s}_{0}(\theta^{\rm r}_{\rm a}|\theta^{\rm s}_{\rm m})=1-\beta$$

(31)

with $\beta\in[0,1]$. The malicious sender does not know the true receiver type, i.e., whether the sender is aware of the vulnerability or not. The given initial beliefs are summarized in Table 1.

In accordance with the introduction of the type structure, the definition of strategies and the solution concept are needed to be slightly modified. The contrasting ingredients of the game with symmetric recognition and the one with asymmetric recognition are listed in Table 2, where those with asymmetric recognition can analogically be defined. The conditional probability $\mathbb{P}^{s}(\cdot|\Theta^{\rm s}=\theta^{\rm s},\Theta^{\rm r}=\theta^{\rm r})$, which is the probability measure induced by $s^{\rm s}(\theta^{\rm s},\cdot)$ and $s^{\rm r}(\theta^{\rm r},\cdot)$, is denoted by $\mathbb{P}^{s}_{\theta^{\rm s},\theta^{\rm r}}$. The sender’s expected average utility at the $k$th step with the horizon length $T$ is given by

$$\begin{array}[]{l}\bar{U}^{\rm s}_{k,T}(s_{k:k+T}|\theta^{\rm s},x_{0:k}):=\dfrac{1}{T+1}\\ \displaystyle{\small{\times\mathbb{E}^{s}\left[\left.\sum_{\tau=k}^{k+T}U^{\rm s}(\Theta^{\rm s},X_{\tau},s^{\rm s}_{\tau}(\Theta^{\rm s},X_{0:\tau}),s^{\rm r}_{\tau}(\Theta^{\rm r},X_{0:\tau}))\right|\theta^{\rm s},x_{0:k}\right].}}\end{array}$$

(32)

The receiver’s expected average utility at the $k$th step with the horizon length $T$ is given by

$$\begin{array}[]{l}\bar{U}^{\rm r}_{k,T}(s_{k:k+T}|\theta^{\rm r},x_{0:k}):=\dfrac{1}{T+1}\\ \displaystyle{\small{\times\mathbb{E}^{s}\left[\left.\sum_{\tau=k}^{k+T}U^{\rm r}(\Theta^{\rm s},X_{\tau},s^{\rm s}_{\tau}(\Theta^{\rm s},X_{0:\tau}),s^{\rm r}_{\tau}(\Theta^{\rm r},X_{0:\tau}))\right|\theta^{\rm r},x_{0:k}\right].}}\end{array}$$

(33)

A strategy $s$ is said to be a PBE when the limit of the utilities $(\bar{U}^{\rm s}_{k},\bar{U}^{\rm r}_{k})$ satisfies

$$\left\{\begin{array}[]{l}s^{\rm s}_{r:\infty}\in{\rm BR}^{\rm s}_{k}(s^{\rm r}_{r:\infty}|\theta^{\rm s},x_{0:k}),\quad\forall\theta^{\rm s}\in{\it\Theta}^{\rm s},\\ s^{\rm r}_{k:\infty}\in{\rm BR}^{\rm r}_{k}(s^{\rm s}_{k:\infty}|\theta^{\rm r},x_{0:k}),\quad\forall\theta^{\rm r}\in{\it\Theta}^{\rm r},\end{array}\right.$$

(34)

for any $k\in{\mathbb{Z}_{+}}$ and $x_{0:k}\in\mathcal{X}^{k+1}$ where

$$\begin{array}[]{l}{\rm BR}^{\rm s}_{k}(s^{\rm r}_{k:\infty}|\theta^{\rm s},x_{0:k}):=\displaystyle{\operatorname*{arg\,max}_{\tilde{s}^{\rm s}_{k:\infty}\in\mathcal{S}^{\rm s}_{k:\infty}}\ \bar{U}^{\rm s}_{k}((\tilde{s}^{\rm s}_{k:\infty},s^{\rm r}_{k:\infty})|\theta^{\rm s},x_{0:k})},\\ {\rm BR}^{\rm r}_{k}(s^{\rm s}_{k:\infty}|\theta^{\rm r},x_{0:k}):=\displaystyle{\operatorname*{arg\,max}_{\tilde{s}^{\rm r}_{k:\infty}\in\mathcal{S}^{\rm r}_{k:\infty}}\ \bar{U}^{\rm r}_{k}((s^{\rm s}_{k:\infty},\tilde{s}^{\rm r}_{k:\infty})|\theta^{\rm r},x_{0:k})}.\end{array}$$

(35)

We define the game formulated above by

$$\mathcal{G}_{2}:=(\mathcal{M},\mathcal{S},U,({\it\Theta}^{\rm s},{\it\Theta}^{\rm r}),(\pi^{\rm s}_{0},\pi^{\rm r}_{0})),$$

(36)

where the defender’s initial belief is not common information in contrast to $\mathcal{G}_{1}$.

In the following discussion, we analyze $\mathcal{G}_{2}$ through $\mathcal{G}_{1}$. To clarify their relationship, we describe the game $\mathcal{G}_{1}$ using the modified formulation. Define another game

$$\hat{\mathcal{G}}_{2}:=(\mathcal{M},\mathcal{S},U,({\it\Theta}^{\rm s},{\it\Theta}^{\rm r}),(\hat{\pi}^{\rm s}_{0},\pi^{\rm r}_{0})),$$

(37)

where

$$\hat{\pi}^{\rm s}_{0}(\theta^{\rm r}_{\rm a}|\theta^{\rm s}_{\rm m})=1.$$

(38)

The initial belief means that the adversary believes that the defender is aware of the vulnerability. The situation of $\hat{\mathcal{G}}_{2}$ is the same as that of $\mathcal{G}_{1}$ if the defender is aware of the vulnerability. Thus, these games lead to the same consequence when the true types are $\theta^{\rm s}_{\rm m}$ and $\theta^{\rm r}_{\rm a}$. The following lemma holds.

We extend the notions of detection-averse utilities and asymptotically benign strategies to $\mathcal{G}_{2}$. Our objective is to investigate the effectiveness of the proposed defensive deception. It is possible to define detection-averse utilities directly using the game $\mathcal{G}_{2}$ as utilities where the resulting equilibrium leads the adversary to avoid being detected. However, this definition immediately means that the defensive deception works well, and any results from the definition cannot show its effectiveness. Instead, we say that utilities in $\mathcal{G}_{2}$ are detection-averse when the adversary avoids being detected if she is certain that the defender is aware of the vulnerability.

Note that Definition 3 is a necessary requirement to make the game interesting, because the adversary is not afraid of being detected at all without this condition.

Next, we define desirable strategies that should be achieved by Bayesian defense mechanisms. We say a strategy in $\mathcal{G}_{2}$ to be asymptotically benign when it becomes benign regardless of the defender’s awareness.

Note that Definition 4 requires the strategy to be asymptotically benign for any $\theta^{\rm r}\in{\it\Theta}^{\rm r}$. In other words, the strategy is needed to be asymptotically benign even if the defender is unaware of the vulnerability.

We expect that there exists a chance of preventing attacks that exploit unnoticed vulnerabilities if the state does not possess information about the defender’s recognition. To formally verify this expectation, we define passively bluffing strategies.

Definition 5 requires the sender’s belief to be invariant over time. If the strategy is passively bluffing, the adversary cannot identify whether the defender is aware of the exploited vulnerability or not even in an asymptotic sense. Note that the introduced passively bluffing strategies can be regarded as a commitment. It is well known that restricting feasible strategies, referred to as commitment, can be beneficial in a game [47, 48]. In what follows, we investigate the effectiveness of the specific commitment.

Passively bluffing strategies can relax the condition for asymptotically benign strategies. The following lemma holds.

The difference between (42) and (44) is the required receiver type. Lemma 5 implies that if a passively bluffing strategy profile is asymptotically benign when the receiver is aware of the vulnerability then the strategy is needed to be asymptotically benign even when the receiver is unaware of the vulnerability.

Remark: Although Definition 5 depends not only on the receiver’s strategy but also on the sender’s strategy for generality, the bluffing should be realized only by the defender in practice. A simple defender’s approach to achieving the bluffing is to choose reactions that do not influence the system’s behavior. Let $\mathcal{R}_{\rm pb}\subset\mathcal{R}$ be the set of reactions such that the system’s dynamics is independent of the reaction, i.e., the transition probability satisfies

$$P(x^{\prime}|x,a,r)=P(x^{\prime}|x,a,r^{\prime})$$

(45)

for any $x^{\prime}\in\mathcal{X},x\in\mathcal{X}$, $a\in\mathcal{A}$, $r\in\mathcal{R}_{\rm pb}$, $r^{\prime}\in\mathcal{R}_{\rm pb}$. If the receiver’s strategy takes only reactions in $\mathcal{R}_{\rm pb},$ every strategy profile is passively bluffing. Indeed, because the transition probability is independent of $r\in\mathcal{R}_{\rm pb}$, the probability distribution of the state is independent of $\theta^{\rm r}$. Thus, from Bayes’ rule, we have

$$\begin{array}[]{cl}\pi^{\rm s}_{k}(\theta^{\rm r}|x_{0:k},\theta^{\rm s})&=\dfrac{p^{s}_{\theta^{\rm s},\theta^{\rm r}}(x_{0:k})\pi^{\rm s}_{0}(\theta^{\rm r}|\theta^{\rm s})}{\sum_{\phi^{\rm r}\in{\it\Theta}^{\rm r}}p^{s}_{\theta^{\rm s},\phi^{\rm r}}(x_{0:k})\pi^{\rm s}_{0}(\phi^{\rm r}|\theta^{\rm s})}\\ &=\dfrac{p^{s}_{\theta^{\rm s}}(x_{0:k})\pi^{\rm s}_{0}(\theta^{\rm r}|\theta^{\rm s})}{\sum_{\phi^{\rm r}\in{\it\Theta}^{\rm r}}p^{s}_{\theta^{\rm s}}(x_{0:k})\pi^{\rm s}_{0}(\phi^{\rm r}|\theta^{\rm s})}\\ &=\dfrac{\pi^{\rm s}_{0}(\theta^{\rm r}|\theta^{\rm s})}{\sum_{\phi^{\rm r}\in{\it\Theta}^{\rm r}}\pi^{\rm s}_{0}(\phi^{\rm r}|\theta^{\rm s})}\\ &=\pi^{\rm s}_{0}(\theta^{\rm r}|\theta^{\rm s})\end{array}$$

(46)

when $p^{s}_{\theta^{\rm s},\theta^{\rm r}}(x_{0:k})\neq 0$. An example of such reactions is just analyzing the network log and raising an alarm inside the operation room without applying control on the system itself. Note that the reaction still affects the players’ decision making through their utility functions, even if (45) holds.