Restricted Regression in Networks

Network data are measurements about the relationships between pairs of entities. These network measurements can be visualized as measurements on the edges between nodes of a graph (Becker et al.,, 1995). Examples of network data include relationships between potential borrowers on peer-to-peer lending platforms (Lee and Sohn,, 2022) or annual migration between countries (Aleskerov et al.,, 2017). Data are typically represented as a matrix, $\bm{Y}$, where $y_{ij}$ is the value in row $i$ and column $j$, for $i,j=1,\dots,n$. The value $y_{ij}$ is the measurement about the dyadic relationship between the sending node $i$ and the receiving node $j$. If $y_{ij}=y_{ji}$ for all $i,j$, then the network is called undirected, otherwise it is called directed.

Network regression uses covariates measured on the node pairs to model the dyadic relationships. An example of such a model is

	$\displaystyle y_{ij}$	$\displaystyle=g(z_{ij}),$		(1)
	$\displaystyle z_{ij}$	$\displaystyle=\bm{x}_{ij}^{\top}\bm{\beta}+\gamma_{ij},$		(2)

where $y_{ij}$ is the observed network measure, $g(\cdot)$ is a function mapping latent continuous values, $z_{ij}$, to the observed $y_{ij}$, $\bm{x}_{ij}$ is a $p$-vector of covariates related to node $i$, node $j$ or the relationship from node $i$ to node $j$, $\bm{\beta}$ is a $p$-vector of regression coefficients, and $\gamma_{ij}$ is random error. For a binary observation $y_{ij}\in\{0,1\}$ indicating the presence or absence of an edge from node $i$ to node $j$, a probit version of (1) is given by setting

	$\displaystyle g(z_{ij})$	$\displaystyle=I(z_{ij}>0),$		(3)
	$\displaystyle\gamma_{ij}$	$\displaystyle\overset{\text{i.i.d}}{\sim}N(0,\sigma^{2}),$		(4)

where the function $I(\cdot)$ is the indicator function (Albert and Chib,, 1993). For the rest of this paper, we will use $z_{ij}$ to refer to a continuous latent variable resulting from the regression equation, and $y_{ij}$ to refer to observations related to $z_{ij}$ through a function, $g(\cdot)$, possibly the identity function.

Network regression has found applications in medical meta-analysis (e.g., Li et al.,, 2018; Gwon et al.,, 2020), analysis of international politics (e.g., Campbell et al.,, 2019), and social networks (e.g., Cantner and Graf,, 2006). Other methods for modeling network data include stochastic block models (Holland et al.,, 1983) and exponential-family random graph models (Robins et al.,, 2007), which allow inference on latent aspects of the network structure such as clusters and density. While many different properties of a network may be of interest, our primary interest is inference for the regression coefficients, $\bm{\beta}$.

The network structure of the data creates dependence among the measurements. Observations $\{y_{ij}\}$ with one or more nodes in common, for example $y_{ij}$ and $y_{ij^{\prime}}$, can be dependent due to their common node, $i$. If not all of this dependence is captured by the covariates, latent random effects can be used to account for excess variation (Holland et al.,, 1983; Wang and Wong,, 1987; Hoff et al.,, 2002; Li and Loken,, 2002; Hoff,, 2005). In this approach, the measurements are modeled as conditionally independent given the latent structure. Additive node random effects can be used to account for dependence in the observations,

	$\displaystyle\gamma_{ij}$	$\displaystyle=a_{i}+b_{j}+\varepsilon_{ij},$		(5)
	$\displaystyle a_{i}$	$\displaystyle\sim g_{a}(\bm{\theta}_{a}),\quad b_{j}\sim g_{b}(\bm{\theta}_{b}),$		(6)
	$\displaystyle\varepsilon_{ij}$	$\displaystyle\overset{\text{i.i.d}}{\sim}N(0,\sigma^{2}).$		(7)

The terms $a_{i}$ and $b_{j}$ are called sender and receiver random effects, respectively, and are meant to capture variation due to unobserved node factors, for example, sociability and popularity in social networks. The random effects have distributions, $g_{a}$ and $g_{b}$, parameterized by parameters $\bm{\theta}_{a}$ and $\bm{\theta}_{b}$. These effects first appeared in the social relations model of Warner et al., (1979). Node random effects have also been incorporated in more complex network models such as popularity-adjusted block models (Sengupta and Chen,, 2018) and additive and multiplicative effects network models (Hoff,, 2021). An alternate approach is to model the correlation between measurements on dyads which share a node by the residual covariance matrix under an assumption of exchangeable errors (Marrs et al.,, 2022).

A key difference between the model in (2) with the error structure in (4) compared to the error structure in (5) is the interpretation of the regression parameters. With the error structure in (4), the regression parameters are called the unconditional regression effects because they capture the marginal effect of $\bm{X}$ on $\bm{z}$. With (5), the regression parameters are called the conditional regression effects because their value is interpreted as the effect of $\bm{X}$ conditioned on the random effects $\bm{a}=(a_{1},\dots,a_{n})$ and $\bm{b}=(b_{1},\dots,b_{n})$. For the rest of this paper, we will distinguish between these interpretations by writing $\bm{\beta}$ for conditional regression effects and $\bm{\delta}$ for unconditional regression effects in (2).

In network-structured data, covariates can occupy the same linear space as the random effects $a_{i}$ or $b_{j}$. We define this collinearity as network confounding. Despite potential impacts of bias due to network confounding, including random effects is typically desired to account for correlation between observations due to network dependence and to allow for more accurate uncertainty quantification for the regression effects. A method that mitigates network confounding would allow for accurate estimation of the unconditional regression effects, while using random effects to account for unobserved network-structured variability in the response.

Confounding between covariates and random effects has been of significant interest in the spatial statistics literature, where it is referred to as spatial confounding (Clayton et al.,, 1993; Reich et al.,, 2006; Hodges and Reich,, 2010). A typical spatial model for continuous areal data is the Intrinsic Conditional Autoregressive (ICAR; Besag et al.,, 1991) model:

	$\displaystyle\bm{y}$	$\displaystyle=\bm{X}\bm{\beta}+\bm{\eta}+\bm{\varepsilon},$		(8)
	$\displaystyle p(\bm{\eta})$	$\displaystyle\propto\tau_{s}^{n-G}\exp\left(-0.5\tau_{s}\bm{\eta}^{\top}\bm{Q}\bm{\eta}\right),$		(9)
	$\displaystyle\bm{\varepsilon}$	$\displaystyle\sim N(\bm{0},\sigma^{2}\bm{I}).$		(10)

The random effect $\bm{\eta}$ is intended to capture spatial variation in the response $\bm{z}$ not accounted for by the predictors $\bm{X}$. The random effect is regularized to have spatial variation via the matrix $\bm{Q}$, the Laplacian of the graph of neighboring areas. Spatial confounding occurs when a covariate is smoothly spatially varying, as this creates a scenario when there is collinearity or near collinearity between the fixed covariates $\bm{X}$ and spatial random effects $\bm{\eta}$ such that both the random effect and the covariate are attempting to capture similar structure. Hodges and Reich, (2010) noted that estimates of regression coefficients $\bm{\beta}$ can be dramatically affected by the inclusion or exclusion of random effects $\bm{\eta}$, and introduced restricted spatial regression to resolve the confounding of the random effects and covariates by orthogonalizing $\bm{\eta}$ against $\bm{X}$. The restricted spatial regression equation is expressed as

$$\bm{y}=\bm{X}\bm{\delta}+(\bm{I}-\bm{P}_{X})\bm{\eta}+\bm{\varepsilon},$$

(11)

where $\bm{P}_{X}=\bm{X}(\bm{X}^{\top}\bm{X})^{-1}\bm{X}^{\top}$ is the linear projection matrix onto the column space of $\bm{X}$. Restricted spatial regression has been studied extensively in the spatial statistics literature as a means to alleviate spatial confounding (e.g., Hodges and Reich,, 2010; Hughes and Haran,, 2013; Hanks et al.,, 2015; Khan and Calder,, 2022; Zimmerman and Hoef,, 2022). Hanks et al., (2015) show that in a Bayesian setting, inference on both $\bm{\beta}$ and $\bm{\delta}$ can be achieved simultaneously by calculating $\bm{\delta}=\bm{\beta}+(\bm{X}^{\top}\bm{X})^{-1}\bm{X}^{\top}\bm{\eta}$ for each posterior sample.

However, the dependence in network data is more complex than the dependence in spatial data. In the network setting, observations are made on the relationships between nodes, and shared nodes between two observations create dependence. This is in contrast to the areal spatial setting, where observations are made on discrete, disjoint areas and the neighbor relation between areas creates dependence (Figure 1, A & B). The additive random effects in (5), reflect two different ways in which network observations can be related: sharing a sender node (observations $y_{ij}$ and $y_{ij^{\prime}}$) and sharing a receiver node (observations $y_{ij}$ and $y_{i^{\prime}j}$). These two types of dependence result in a complex dependence structure (Figure 1, C & D). For each of these two types of dependence, the $n^{2}$ observations are divided into $n$ groups of $n$ observations which are all mutually dependent within a group.

Spatial models in which regions can be neighboring in two distinct ways have been studied in Reich et al., (2007) for the case of periodontal health measurements. Measurements were considered to be neighboring other measurements on the same tooth, or measurements on an adjacent tooth, but these kinds of neighbor relations were considered to create different kinds of dependence between measurements. The dependence in the periodontal health measurements was modeled using the random effect prior

$$p(\bm{\eta})\propto c(\tau_{1},\tau_{2})\exp\{-0.5\bm{\eta}^{\top}(\tau_{1}\bm{Q}_{1}+\tau_{2}\bm{Q}_{2})\bm{\eta}\},$$

(12)

where $\bm{Q}_{1}$ and $\bm{Q}_{2}$ are Laplacians for each neighbor relation. However, the distribution in (12) cannot represent the same dependence as the sender and receiver random effects in (5) because the distribution of $\bm{\eta}$ is full rank, while the distribution of the vectorized $(a_{i}+b_{j})$ is not full rank.

In this paper we introduce Restricted Network Regression as a method that mitigates network confounding. We approach network regression in a Bayesian framework, estimating parameters using their posterior distributions given the data. We characterize the posterior mean and variance of regression parameters in Restricted Network Regression with continuous data and show through simulation that Restricted Network Regression mitigates network confounding for continuous and binary data. Specifically, we demonstrate that a Restricted Network Regression model results in smaller bias and posterior credible intervals that are more appropriately calibrated to capture the generative parameter values than corresponding network regression models without random effects or with non-restricted random effects.

The remainder of this paper is organized as follows. Section 2 defines network confounding and requirements for methods to “alleviate” and “mitigate” network confounding. Section 3 introduces Restricted Network Regression, characterizes the collinearity of effects within the Restricted Network Regression model, and provides theorems about the posterior distribution of the regression parameters in the continuous Restricted Network Regression model. Section 4 describes the results of a simulation study involving both continuous and binary network regression. Section 5 is a case study of Eurovision Song Contest voting data showing the changes in inference that occur when using a Restricted Network Regression approach, relative to models without random effects and non-restricted random effect models. Finally, Section 6 closes with a discussion.

Network confounding is the collinearity between fixed effects and network-structured random effects in a network regression model. This collinearity creates difficulty in estimating regression parameters by introducing bias in estimates and increasing their posterior variance. In this section we explore network confounding in more detail and define conditions for a method to alleviate or mitigate network confounding.

Consider the network regression model,

	$\displaystyle y_{ij}$	$\displaystyle=g^{-1}(z_{ij}),\ \ 1\leq i,j\leq n,$		(13)
	$\displaystyle\bm{z}$	$\displaystyle=\bm{X}\bm{\beta}+\bm{A}\bm{a}+\bm{B}\bm{b}+\bm{\varepsilon},$		(14)
	$\displaystyle a_{i}$	$\displaystyle\overset{\text{i.i.d.}}{\sim}g_{a}(\bm{\theta_{a}}),$		(15)
	$\displaystyle b_{j}$	$\displaystyle\overset{\text{i.i.d.}}{\sim}g_{b}(\bm{\theta_{b}}),$		(16)
	$\displaystyle\bm{\varepsilon}$	$\displaystyle\sim N(\bm{0},\sigma^{2}_{e}\bm{I}),$		(17)

where $\bm{z}$ is an $n^{2}$-vector of latent continuous responses, $\bm{X}$ is an $n^{2}\times p$ fixed matrix of covariates, $\bm{\beta}$ is a $p$-vector of regression parameters, and $\bm{a}$ and $\bm{b}$ are $n$-vectors composed of sender and receiver random effects, respectively, for each node in the network. The matrices $\bm{A}$ and $\bm{B}$ are $n^{2}\times n$ matrices of zeros and ones which broadcast the elements of $\bm{a}$ and $\bm{b}$ into the appropriate rows of $\bm{z}$ depending on the sender and receiver of each dyad. To match the vectorized form of $\bm{z}$, we write $\bm{y}$ as an $n^{2}$-vector of the observed network data.

Now consider partitioning $\bm{X}=[\bm{1}\ \bm{X}_{s}\ \bm{X}_{r}\ \bm{X}_{d}]$ into an intercept, an $n^{2}\times p_{s}$ matrix of sender covariates $\bm{X}_{s}$, an $n^{2}\times p_{r}$ matrix of receiver covariates $\bm{X}_{r}$, and an $n^{2}\times p_{d}$ matrix of dyadic covariates $\bm{X}_{d}$ as done in Hoff, (2021), where $p_{s}$, $p_{r}$, and $p_{d}$ are the number of sender, receiver, and dyadic covariates, respectively, and $p=1+p_{s}+p_{r}+p_{d}$. Similarly, partition $\bm{\beta}=(\beta_{0}\ \bm{\beta}_{s}^{\top}\ \bm{\beta}_{r}^{\top}\ \bm{\beta}_{d}^{\top})^{\top}$. The column space of the matrix $\bm{A}$ contains all $n^{2}$-vectors that have repeated values for dyads with a common sender. Similarly, the column space of the matrix $\bm{B}$ contains all $n^{2}$-vectors that have repeated values for dyads with a common receiver. Since the columns of $\bm{X}_{s}$ are sender covariates, every column of $\bm{X}_{s}$ is in the column space of $\bm{A}$. Therefore, we can write $\bm{X}_{s}=\bm{A}\bm{X}_{s}^{\prime}$, where $\bm{X}_{s}^{\prime}$ is an $n\times p_{s}$ matrix created by collapsing equivalent rows of $\bm{X}_{s}$. Since the columns of $\bm{X}_{r}$ are receiver covariates, every column of $\bm{X}_{r}$ is in the column space of $\bm{B}$, allowing us to write $\bm{X}_{r}=\bm{B}\bm{X}_{r}^{\prime}$. The model in (14) can be written

	$\displaystyle\bm{z}$	$\displaystyle=[\bm{1}\ \bm{X}_{d}](\beta_{0}\ \bm{\beta}_{d}^{\top})^{\top}+\bm{X}_{s}\bm{\beta}_{s}+\bm{X}_{r}\bm{\beta}_{r}+\bm{A}\bm{a}+\bm{B}\bm{b}+\bm{\varepsilon},$		(18)
		$\displaystyle=[\bm{1}\ \bm{X}_{d}](\beta_{0}\ \bm{\beta}_{d}^{\top})^{\top}+\bm{A}(\bm{X}_{s}^{\prime}\bm{\beta}_{s}+\bm{a})+\bm{B}(\bm{X}_{r}^{\prime}\bm{\beta}_{r}+\bm{b})+\bm{\varepsilon}.$		(19)

From this, we can see that $\bm{\beta}_{s}$ and $\bm{a}$ are confounded in the sense that they occupy the same linear space in the response, i.e., ${\cal C}(\bm{X}_{s})\subset{\cal C}(\bm{A})$, where ${\cal C}$ is the column space operator. Similarly, $\bm{\beta}_{r}$ and $\bm{b}$ are also confounded. By restricting the random effects to be orthogonal to the fixed effects, Restricted Network Regression (introduced formally in the next section) fixes this collinearity by removing the intersection of the column spaces of $\bm{X}$, $\bm{A}$, and $\bm{B}$.

We now distinguish between two types of methods: those that alleviate network confounding and those that mitigate network confounding. We adapt a definition from the spatial statistics literature to define what it means for a method to alleviate network confounding:

Definition 1 is adapted from the definition of spatial confounding in Khan and Calder, (2022). This definition of alleviating network confounding reflects the intuition that models with network confounding will result in excess uncertainty on regression parameter estimates, reflected in large posterior variances of those parameters and that a method that alleviates network confounding should have lower posterior variances than one exhibiting network confounding. At the same time, a model that alleviates network confounding should model the unconditional effects of the fixed covariates, and so is expected to have the same posterior means as the model without network-structured random effects.

Alleviation of network confounding has useful interpretation in terms of parameter uncertainty, however, it does not relate directly to the accuracy of estimates made using models with network confounding. Also, Definition 1 provides no way to compare two models if neither alleviates network confounding. For these reasons, we introduce an alternative notion of network confounding mitigation. We expect a method that mitigates network confounding relative to another approach to produce better estimates and uncertainty quantification of the unconditional regression effects $\bm{\delta}$. We give a more precise definition of this mitigation:

Definition 2 combines two useful model evaluations, bias and credible interval coverage, into a comparison which can be used to evaluate the relative improvement of one model over another in the presence of network confounding. Even if a model does not meet the requirements of Definition 1, it can be compared to other models using Definition 2 to assess its mitigation of network confounding.

For the network regression model given in (14) - (17), we propose the following Restricted Network Regression model:

	$\displaystyle\bm{z}$	$\displaystyle=\bm{X}\bm{\delta}+(\bm{I}-\bm{P}_{X})(\bm{A}\bm{a}+\bm{B}\bm{b})+\bm{\varepsilon},$		(20)
	$\displaystyle a_{i}$	$\displaystyle\overset{\text{i.i.d.}}{\sim}g_{a}(\bm{\theta_{a}}),$		(21)
	$\displaystyle b_{j}$	$\displaystyle\overset{\text{i.i.d.}}{\sim}g_{b}(\bm{\theta_{b}}),$		(22)
	$\displaystyle\bm{\varepsilon}$	$\displaystyle\sim\mathrm{N}(\bm{0},\sigma^{2}\bm{I}_{n^{2}}),$		(23)

where $\bm{z}$, $\bm{X}$, $\bm{a}$, $\bm{b}$, $\bm{A}$, $\bm{B}$, and $\bm{P}_{X}$ are as described earlier. This model is distinguished from the model in (14)-(17) by the application of the projection matrix $\bm{I}-\bm{P}_{X}$ projecting the random effects orthogonal to the column space of $\bm{X}$, and the interpretation of the regression effects, $\bm{\delta}$, as unconditional on the values of the random effects. With $\bm{\delta}$ related to the unconditional regression parameters, $\bm{\beta}$ by $\bm{\delta}=\bm{\beta}+(\bm{X}^{\top}\bm{X})^{-1}\bm{X}^{\top}(\bm{A}\bm{a}+\bm{B}\bm{b}),$ the regression equation (20) is equivalent to (14).

In this section we investigate properties of Restricted Network Regression through simulations. We confirm the theoretical results from Section 3.1 using continuous network data and show that neither continuous nor binary Restricted Network Regression alleviate network confounding. However, we show that both mitigate network confounding relative to non-restricted network regression and network regression with no random effects. All of the following simulations involve data with varying levels of excess nodal variation, using models with both sender/receiver covariates and sender/receiver random effects.

In addition to evaluating Restricted Network Regression, we chose to evaluate choices of prior distribution on $\sigma^{2}_{a}$ and $\sigma^{2}_{b}$. A common choice is the inverse-gamma distribution, e.g., in the amen package (Hoff et al.,, 2020). However, the half-Cauchy distribution, $\sigma_{a}\sim\mathrm{Cauchy}^{+}(0,1)$, is a less informative distribution recommended by Gelman, (2006) for random effect variances in hierarchical models. We compare five models:

1. (NoRE)

   Network model with no random effects,

2. (NR.ig)

   Network model with additive random effects and inverse-gamma priors,

3. (NR.hc)

   Network model with additive random effects and half-Cauchy priors,

4. (RNR.ig)

   Network model with restricted additive random effects and inverse-gamma priors,

5. (RNR.hc)

   Network model with restricted additive random effects and half-Cauchy priors.

We show that Restricted Network Regression mitigates network confounding by providing estimates of $\bm{\delta}$ with lower bias and properly calibrated credible intervals.

The Eurovision Song Contest is an annual competition in which European countries compete by submitting the best song by an artist from their country. The contest culminates in a final round, where the remaining 26 competitors perform their songs for a TV audience. All participating countries then vote for their top ten songs through judges and/or phone-in voting. Points are awarded according to votes (12 points for first, 10 points for second, then 8 through 1 points for third through tenth) and the total determines the winner.

The contest is extremely popular, drawing 182 million viewers in 2019 (Eurovision,, 2019). This popularity has meant that the contest has been of interest for study, especially the study of voting patterns (e.g., Ginsburgh and Noury,, 2008; Spierdijk and Vellekoop,, 2009). Because competing countries are a subset of voting countries, vote data can naturally be represented as a directed graph with the countries as nodes and an edge from country $i$ to country $j$ representing a top-10 vote by country $i$ for country $j$. Edges may be labeled with ranked votes if desired. Eurovision votes have been analyzed as a network by, for example, Yair, (1995), Fenn et al., (2006), and D’Angelo et al., (2019).

Countries may have other measurable qualities that are related to how well they score in the contest results. For example, countries with larger populations have more musicians from which to select a contestant. A country’s wealth may be associated with the reach of its cultural exports, leading to more votes received from its trading partners. The Eurovision Song Contest is also the subject of bets predicting its winner. Betting markets reflect the collective knowledge of their participants, which in this case includes knowledge of the specific songs entered by each country, and it is reasonable to think they may be predictive of the outcome. For example, Spann and Skiera, (2009) finds betting odds to be predictive of match results in the German premier soccer league. Analyzing the relationship between population, wealth, and betting odds, and the Eurovision contest voting outcome fits a network regression framework naturally.

We restrict our analysis to the year 2015 and the 27 countries with entries in the final round of that year (Australia, as a new contestant, was given an automatic berth to the final round). The response data consist of a vote network represented as a $27\times 27$ matrix (Figure 6). The receiver covariate data consist of three dimension 27 vectors of song or country attributes: the log median odds from 16 popular European betting sites for each song to win the contest, the log 2015 population of each competing country, and the log 2015 GDP per capita of each competing country. We log-transform the receiver covariates before using them as covariates in a regression model because they are either right-skewed (GDP, population) or because we believe there to be a logarithmic relationship between the predictor and the response (betting odds). We also include a dyadic covariate for country contiguity which was found to be explanatory in D’Angelo et al., (2019). Country contiguity is an undirected network represented as a symmetric $27\times 27$ binary matrix where a 1 indicates that two countries share a border and a 0 indicates otherwise. Visualizations of these covariates are available in Appendix C of the Supplementary Material (Taylor et al.,, 2023). Votes were represented as ranks (1-10 with 10 the highest). This data is freely available and easily compiled by hand (Eurovision,, 2015; Eurovisionworld,, 2015; Conte et al.,, 2022; United Nations,, 2015; World Bank,, 2022). Any pairs $i,j$ where country $i$ did not vote for country $j$ in its top 10 were coded as zero. Our focus on a single year is due to the fact that both betting odds and column random effects are song-specific, and therefore year-specific. Therefore additional years in this analysis cannot be treated like replicates in the style of D’Angelo et al., (2019). We examine the effects of the covariates on Eurovision voting by performing a network regression analysis without random effects, with receiver random effects, and with restricted receiver random effects. We investigate the effect of Restricted Network Regression on regression parameter estimates and interpretation compared to either alternative model.

In this paper, we introduced Restricted Network Regression for models with additive network random effects and established its connection to restricted spatial regression. We characterized the network confounding of the network regression model with additive random effects and node-level covariates, which Restricted Network Regression addresses by forcing the column spaces of the fixed and random effects to be mutually orthogonal. We provided conditions for network regression models to alleviate and mitigate network confounding, and proved that Restricted Network Regression does not alleviate network confounding with theoretical results and through simulation. However, we showed through simulation that Restricted Network Regression does mitigate network confounding relative to network regression with no random effects and non-restricted network regression with continuous and binary response data. Restricted Network Regression produces less bias and properly calibrated credible intervals for regression parameters relative to network regression without random effects and non-restricted network regression.

We also explored through simulation the effect of a half-Cauchy prior on the variance components of the network random effects. We found that this prior resulted in comparable bias and credible interval coverage for the unconditional regression effects to the conjugate inverse-gamma prior in probit Restricted Network Regression, with better bias and credible interval coverage in some scenarios.

Finally, we applied Restricted Network Regression to a dataset of Eurovision Song Contest voting. We interpreted the model estimates of Restricted Network Regression alongside those from a model without network-structured random effects and those from a non-restricted network regression model. For the three receiver covariates in the model, the choice of model affected both their posterior mean point estimates and their credible interval estimates. The change in credible interval width is noticeable for all three receiver covariates. Uncertainty, as indicated by the width of posterior credible intervals, increases after adding random effects to the model, but decreases again after restricting them. The widths of the credible intervals for the receiver covariates with restricted random effects is larger than without random effects but smaller than with non-restricted random effects.

Future work in this area includes developing Restricted Network Regression for other forms of network random effects such as multiplicative effects (Hoff,, 2021) or latent space distance effects (Hoff et al.,, 2002). It is less clear which covariates may be confounded with such effects, and whether restricting these random effects can have the same benefits as with additive effects and non-Gaussian data. Theoretical results for binary or other non-Gaussian data that describe the posterior distribution are also needed to make stronger conclusions about Restricted Network Regression on non-Gaussian data. Restricted network regression can also be expanded to include bipartite network data or longitudinal network data (e.g., Marrs et al.,, 2020).

Supplement to “Restricted Regression in Networks”

This supplementary document contains proofs of theorems in this paper, additional theorems not presented in this paper, details of simulation studies and details of data analysis.

K. Keller acknowledges the support of NSF Grant 1856229 for this work.