Influence Maximization over Markovian Graphs: A Stochastic Optimization Approach

Markovian Graph Process: The social network graph at discrete time instants $n=0,1,\dots$ is modeled as a directed graph $G_{n}=(V,E_{n})$, consisting of a fixed set of individuals $V$, connected by the set of directed edges $E_{n}$. The graph evolves as a Markov process $\{G_{n}=(V,E_{n})\}_{n\geq 0}$ with a finite state space $\mathcal{G}$, and a parameterized regular transition matrix ${P_{\theta}}$ with a unique stationary distribution ${\pi_{\theta}}$ (where, $\theta$ denotes the parameter vector which lies in a compact subset of an $M$-dimensional Euclidean space). Henceforth, $n$ is used to denote the discrete time scale on which the Markovian graph process evolves.

Frequency of the Messages: An influence maximizer distributes messages to the nodes in this evolving network. We assume that the messages are distributed periodically⁴⁴4The assumption of periodic messages is not required for the problem considered in this paper and the proposed algorithms. It suffices if the messages are distributed with some minimum time gap between them (on the time scale $n$). at time instants $n=kN$ where, the positive integer $N$ denotes the period.

Observations of the Finite Sample Paths: When the influence maximizer distributes the $(k+1)$^th message at time ${n=(k+1)N}$, only the finite sample path of the Markovian graph process $\{G_{n}\}_{kN\leq n<kN+\bar{N}}$ for some fixed $\bar{N}\in\mathbb{N}$ with $\bar{N}<N$, is visible to the influence maximizer.

Information Diffusion model: As explained in the example in Sec. I, nodes pass messages they receive to their neighbors with random delays. This method of information spreading in graphs is formalized by independent cascade (IC) model of information diffusion. Various versions of this IC model have been studied in literature. We use a slightly different version of the IC model utilized in [13, 17] and, it is as follows briefly. When the influence maximizer gives a message to a set of seed nodes⁵⁵5We consider the case where the influence maximizer selects a set of seed nodes instead of one seed node (as in the motivating example in Sec. I) to keep the definitions of this section more general. ${\mathcal{I}_{n}}\subseteq V$ at time $n$, a time variable $t\in\mathbb{R}_{\geq 0}$ is initialized at $t=0$. Here, $t$ is the continuous time scale on which the diffusion of the message takes place and, is different from the discrete time scale $n\in\mathbb{Z}_{\geq 0}$ on which the graph evolves (and the messages are distributed periodically). Further, the time scale $t$ is nested in the time scale $n$: $t$ is set to $t=0$ with each new message distributed by the influence maximizer. The set of seed nodes ${\mathcal{I}_{n}}$ (that receive the message from influence maximizer at $n$ and $t=0$) transmits the message through edges attached to them in graph $G_{n}=(V,E_{n})$. Each edge $(j,i)\in E_{n}$ induces a random delay distributed according to probability density function ${\pi_{tt}}(\tau_{ji})$ (called transmission time distribution). Further, each edge transmits the message only once. Only the neighbor which infects a node first will be considered as the true parent node (considering the infection propagation) of the infected node. This process continues until the message expires at $t=T$ (henceforth referred to as the message expiration time) and, the diffusion process stops at that time. It is assumed that the discrete time scale $n$ on which the graph evolves is slower than $T$ i.e. the graph will remain the same at least for $t\in[0,T]$. Then, the same message spreading process will take place when the next message is distributed.

For any realization of this random message spreading process (taking place in $t\in[0,T]$), the subgraph of $G_{n}$ which is induced by the set of edges through which the message propagated, constitutes a Directed Acyclic Graph (DAG). Further, due to the DAG induced by the propagation of a message, the infection time $t_{i}$ of each node $i\in V$ satisfies the shortest path property: conditional on a graph $G_{n}=(V,E_{n})$, and a set of pairwise transmission times $\{\tau_{ji}\}_{(ji)\in E_{n}}$, the infection time $t_{i}$ of $i\in V$ is given by,

where, $g_{i}(\{\tau_{ji}\}_{(ji)\in E_{n}}|\,{\mathcal{I}})$ denotes the length of the shortest path from ${\mathcal{I}}\subseteq V$ to $i\in V$, with edge lengths $\{\tau_{ji}\}_{(ji)\in E_{n}}$.

We use the following definition (from [13, 21]) of influence of a set of nodes ${\mathcal{I}}\subseteq V$, on a graph $G=(V,E)$, for the diffusion model introduced in Sec. II-A. Further, $\mathbb{E}_{{\pi_{tt}}}\{\cdot\}$ is used to denote expectation over a set of pairwise transmission times (associated with each edge) sampled independently from ${\pi_{tt}}$.

In (2), the influence $\sigma_{G}({\mathcal{I}},T)$ of the set of nodes ${{\mathcal{I}}\subset V}$ is the expected number of nodes infected within time $T$, by the diffusion process (characterized by the distribution of transmission delays ${\pi_{tt}}$) on the graph $G$, that started with the set of seed nodes ${\mathcal{I}}$.

Note that the set of infection times $\{t_{i}\}_{i\in V}$, in (2) are dependent random variables. Therefore, obtaining closed form expressions for marginal cumulative distributions in (3) involves computing $|V|-1$ dimensional integral which is not possible (in closed form) for many general forms of the transmission delay distribution. Further, numerical evaluation of these marginal distributions is also not feasible since it will involve discretizing the domain $[0,\infty)$ (refer [13] for a more detailed description about the computational infeasibility of the calculation of the expected value in (4)).

Why randomized selection of seed nodes? Assume that, at time instant $n=kN$, an influence maximizer needs to find a seed node $i^{*}\in V$ that maximizes $\sigma_{G_{n}}(\{i\},T)$ (in order to distribute the $k$^th message, as explained in Sec. II-A). To achieve this, the influence maximizer needs to know the graph $G_{n}$, calculate the influence $\sigma_{G_{n}}(\{i\},T)$ for each $i\in V$ and then, locate the $i^{*}\in V$ that has the largest influence. However, performing all these steps for each message is not feasible from a practical perspective. Especially, monitoring the graph in real time is practically difficult. Hence, a natural alternative is to use a randomized seed selection method. We consider a case where the seed node $i\in V$ is sampled from a parameterized probability distribution $p_{\theta}\in\Delta(V)$ such that, the expected size of the cascade initiated by the sampled node $i$ is largest when $\theta=\theta^{*}$. This random seed selection approach leads to the formal definition of the main problem which we call the randomized influence maximization.

We define the influence of a parameterized probability distribution ${p_{\theta}},\,\theta\in\mathbb{R}^{M}$, on a Markovian graph process as below (henceforth, we will use $\sigma_{G}(i,T)$ to denote $\sigma_{G}(\{i\},T)$ with a slight notational misuse).

Eq. (5) averages influence $\sigma_{G}(i,T)$ using the sampling distribution ${p_{\theta}}$, to obtain $c(\theta,G)$, which is the influence of the sampling distribution ${p_{\theta}}$ conditional on the graph $G$. Then, (4) averages $c(\theta,G)$ using the unique stationary distribution ${\pi_{\theta}}$ of the graph process in order to obtain $C(\theta)$, which is the influence of the sampling distribution ${p_{\theta}}$ over the Markovian graph process. We will refer to $c(\theta,G_{n})$ as the conditional (on graph $G_{n}$) influence function (at time $n$) and, $C(\theta)$ as the influence function, of the sampling distribution ${p_{\theta}}$.

In this context, the main problem studied in this paper can be defined as follows.

Networks as Markovian Graphs: We assumed that the graph evolution is Markovian. In a similar context to ours, [22] states that “Markovian evolving graphs are a natural and very general class of models for evolving graphs” and, studies the information spreading protocols on them during the stationary phase. Further, [23] considers information broadcasting methods on Markovian graph processes since they are “general enough for allowing us to model basically any kind of network evolution”.

Functional Dependency of the Sampling Process and Graph Evolution: [24] considers a weakly adversarial random broadcasting network: a randomly evolving broadcast network whose state at the next time instant is sampled from a distribution that minimizes the probability of successful communications. Analogous to this, the functional dependency in our model may represent how the network evolves adversely to the influence maximization process (or some other underlying network dynamic which is responsive to the influence maximization). The influence maximizer need not be aware of such dependencies to apply the algorithms that will be presented in the next sections.

The exact computation of the node influence $\sigma_{G}(i,T)$ in closed form or estimating it with a naive sampling approach is computationally infeasible (as explained in Sec. II-B). As a solution, [13] shows that the shortest path property of the IC model (explained in Sec. II-A) can be used to convert (2) into an expression involving a set of independent random variables as follows:

where, $g_{i}(\{\tau_{ji}\}_{(ji)\in E}|{\mathcal{I}})$ is the shortest path as defined previously in (1). Further, note from (8) that influence of the set ${\mathcal{I}}$, $\sigma_{G}({\mathcal{I}},T)$ is the expected $T$-distance neighborhood (expected number of nodes within $T$ distance from the seed nodes) i.e.

where,

Hence, we only need a neighborhood size estimation algorithm and samples from ${\pi_{tt}}$ to estimate the influence $\sigma_{G}({\mathcal{I}},T)$ of the set ${\mathcal{I}}\subset V$.

We propose Algorithm 2 for solving the optimization problem (6), utilizing the conditional influence estimates obtained via Algorithm 1. Algorithm 2 is based on the Simultaneous Perturbation Stochastic Approximation (SPSA) algorithm (see [26, 27] for details). In general, SPSA algorithm utilizes a finite difference estimate $\hat{\nabla}C_{k}(\theta_{k})$, of the gradient of the function $C(\theta)$ at the point $\theta_{k}$ in the ($k$^th iteration of the) recursion,

In the $k$^th iteration of the Algorithm 2, the influence maximizer passes a message to the network using a seed node sampled from the distribution ${p_{\theta}}$ where, $\theta=\theta_{k}+\Delta d_{k}$ (step 3). Sampling with ${p_{\theta}}$ causes the transition matrix to be become ${P_{\theta}}$ (recall Remark 1). Then, in step 4, (16) averages the conditional influence estimates ${\hat{c}(\theta,G_{n})}$ over $\bar{N}$ consecutive time instants (where, $\bar{N}$ is the length of the available sample path as defined in Sec. II-A) to obtain ${\hat{C}_{k}(\theta_{k}+\Delta d_{k})}$, which is an asymptotically convergent estimate (by the law of large numbers for Markov Chains [28, 29]) of ${C}(\theta_{k}+\Delta d_{k})$. Similarly, steps 5 and 6 obtain ${\hat{C}_{k}(\theta_{k}-\Delta d_{k})}$, which is an estimate of ${C}(\theta_{k}-\Delta d_{k})$. Using these influence function estimates, (17) computes the finite difference gradient estimate $\hat{\nabla}C_{k}(\theta_{k})$ in step 7. Finally, step 8 updates the $M$-dimensional parameter vector using the gradient estimate computed in (17). Some remarks about this algorithm are as follows.

Next, we establish the convergence of Algorithm 2 using standard results which gives sufficient conditions for the convergence of recursive stochastic gradient algorithms (for details, see [27, 30, 31]).

In some applications, the influence maximizer can observe only a small part of the full network $G_{n}$, at any time instant $n$. Let $G^{\bar{V}}$ denote the subgraph of $G=(V,E)$, induced by the set of nodes $\bar{V}\subseteq V$. Then, we consider the case where, the observable part is the subgraph of $G_{n}$ which is induced by a fixed subset of nodes $\bar{V}\subseteq V$ i.e. the observable part at time $n$ is $G_{n}^{\bar{V}}$ ($G^{\bar{V}}$ denotes the subgraph of $G$, induced by the set of nodes $\bar{V}$)⁶⁶6For example consider the friendship network $G_{n}=(V,E_{n})$ of all the high school students in a city at time $n$. The smaller observable part could be the friendship network formed by the set of students in a particular high school $V^{\prime}\subset V$, which is a subgraph of the friendship network $G_{n}$. The influence maximizer then needs to perform influence maximization by observing this subgraph.. Then, the observation space $\bar{\mathcal{G}}$ of the Markovian graph process can be defined as,

which consists of the subgraphs induced by $\bar{V}$ in each graph $G\in\mathcal{G}$ (the case ${\bar{V}=V}$ corresponds to the perfectly observed case). For each $G\in\mathcal{G}$ and $\bar{G}\in\bar{\mathcal{G}}$, the observation likelihoods, denoted by $B_{G\bar{G}}$ are defined as,

In our system model, these observation likelihoods can take only binary values:

i.e. $B_{G\bar{G}}=1$ if the subgraph formed by the set of nodes $\bar{V}$ in graph $G\in\mathcal{G}$ is $\bar{G}\in\mathcal{Y}$ and, $0$ otherwise. In this setting, our main assumption is the following.

Then, the aim of the influence maximizer is to recursively solve the problem of influence maximization given in (6), utilizing the information assumed to be known in Assumption 2. However, solving (6) is non-trivial even under the Assumption 2, as emphasized in the following remark.

Assumption 2 made in IV-A makes it possible to implement an HMM filter (see [32] for a detailed treatment of HMM filters and related results). The HMM filter is a finite dimensional Bayesian filter which recursively (with each observation) computes $\pi_{n}^{\theta}$ which is the probability distribution of the state of the graph, conditional on the sequence of observations $\{G^{\bar{V}}_{0},\,\dots,G^{\bar{V}}_{n}\}$. Algorithm 3 gives the HMM filter algorithm and, Theorem 3 establishes the asymptotic convergence of the influence function estimate obtained from from it.

Finally, the Algorithm 2 can be modified by replacing the estimates computed in steps 4 and 6 with the influence function estimates obtained using the HMM filter in Algorithm 3.

We use the Stochastic Block Model (SBM) as a generative models to create the graphs used in this section. These models have been widely studied in statistics [33, 34, 35, 36, 37] and network science [38, 39] as generative models that closely resemble the real world networks.

State space of the graph process: We consider the graph process obtained by Markovian switching between the two graphs in Fig. 2: a graph where two dense equal sized clusters exist (Graph $G^{1}$) and, a graph where most of the nodes (45 out of 50) are in a single dense cluster (Graph $G^{2}$). These graphs are sampled from SBM models with the following parameter values: $G^{1}$ with cluster sizes 25, 25 with, within cluster edge probability $p_{w}^{SBM}=0.3$, between cluster edge probability $p_{b}^{SBM}=0.01$ and, $G^{2}$ with cluster sizes 45, 5 with, within cluster edge probability $p_{w}^{SBM}=0.3$, between cluster edge probability $p_{b}^{SBM}=0.01$. This graph process is motivated by the clustered and non-clustered states of a social network.

Sampling distribution and the graph evolution: We consider the case where the influence maximizer samples from a subset of nodes that consists of the two nodes indexed by $v_{1}$ and $v_{2}$ using the parameterized probability distribution ${{p_{\theta}}=[\cos^{2}(\theta)\quad\sin^{2}(\theta)]}^{\prime}$. These two nodes are located in different clusters in the graphs $G^{1},G^{2}$. Also, the transition probabilities may depend on this sampling distribution (representing for example, the adversarial networks/nodes as explained in Sec. II-D). However, exact functional characterizations of such dependencies are not known in many practical applications. Also, the form of these dependencies may change over time as well (recall Remark 1). This experimental setup considers a graph process with a stationary distribution ${\pi_{\theta}}={p_{\theta}}$ in order to have a closed form influence function as a ground truth (in order to compare the accuracy of the estimates). In an actual implementation of the algorithm, this functional dependency need not be known to the influence maximizer.

Influence Functions: The transmission time distribution was selected to be an exponential distribution with mean $1$ for edges within a cluster and, an exponential distribution with mean $10$ for between cluster edges (in the SBM). Further, the message expiration time was selected to be $T=1.5$. Then, the influences of nodes $v_{1},v_{2}$ on graphs $G_{1}$ and $G_{2}$ were estimated to be as follows by evaluating the integral in Definition 1 with a naive simulation method (repeated use of the shortest path algorithm): $\sigma_{G^{1}}(v_{1},T)=25.2,\,\sigma_{G^{1}}(v_{2},T)=23.2,\,\sigma_{G^{2}}(v_{1},T)=45.1,\,\sigma_{G^{2}}(v_{2},T)=5.8$. These values, along with the expressions for $p_{\theta}$ and ${\pi_{\theta}}$ were used to obtain the following expression for the influence function (defined in Definition 2) to be compared with the outputs of the algorithm estimates:

In this context, the goal of our algorithm is to locate the value of $\theta$ which maximizes this function, without using knowledge of $C(\theta)$ or ${\pi_{\theta}}$.

Algorithm 2 was utilized in an experimental setting with the parameters specified in the Sec. V-A. For this, the length of the observable sample path of the graph process was assumed to be $\bar{N}=30$. Further, in the Algorithm 1, the number of transmission time sets ($s$) and the number of exponential random variable sets ($m$) were both set to be $10$.

With these numerical values for the parameters, Fig. 3 shows the variation of the absolute error (absolute value of the difference between the current and maximum influence) and the parameter value, against the iteration of the algorithm. From this, it can be seen that the algorithm finds optimal parameter in less than 50 iterations. Further, any change in the system model will result in a suboptimal (expected) influence only for 50 iterations since the algorithm is capable of tracking the time evolving optima. Hence, this shows that the stochastic optimization algorithm is capable of estimating the optimal parameter value with a smaller (less than 50) number of iterations.

Here we aim to see how the proposed stochastic approximation algorithm can track the optima when the system model changes on a slower time scale and, the effect of reduced variance Algorithm 1 on the accuracy of tracking. For this, the experimental setup described in V-A was utilized again and, a sudden change in the influence function (by changing the the state space of the graph process and the functional dependency) was introduced at the iteration number 200. In this setting, Fig. 4 depicts the variation of the absolute error in influence with the algorithm iteration for two cases: with the reduced variance Algorithm 1 (red curve) and without the variance reduction approach (black curve). It can be seen from Fig. 4 that the variance reduction method improves the speed of convergence to the optima initially (iterations 1 to 50) and, also in tracking the optima after a sudden change in the influence function (iterations 200 to 300). Further, after convergence (between iterations 50 to 200 and 300 to 500), it can be seen that reduced variance approach is less noisy compared to the method without variance reduction. Hence, this shows that the proposed approach is capable of tracking the optimal sampling distribution in a slowly evolving system such as, varying graph state space, evolving functional dependencies, etc.