Synthetic Regressing Control Method

This article is closely related to the studies that investigate the SC estimator when the pre-treatment fit is imperfect. One approach to addressing the issue is to relax the restriction that the weights are nonnegative. Doudchenko and Imbens (2016) argues that negative weights would be beneficial in many settings and proposes adding an intercept into the SC problem. Similarly, Amjad et al. (2018) proposes a denoising algorithm that combines negative weights with a preprocessing step, and Ferman and Pinto (2021) also argues that a demeaned version of the SC method is already efficient. Another approach is to use an outcome model for reducing the imperfect fit. Powell (2018) allows for extrapolation by constructing the SC unit based on the fitted values on unit-specific time periods. Ben-Michael et al. (2021) proposes the augmented synthetic control method, which uses an outcome model to estimate bias resulting from imperfect pre-treatment fit and de-biases the original SC estimator. The SRC method relates these estimators in the sense of addressing the issue of imperfect pre-treatment fit. However, it differs from them in intent. The concept of SRC aims to mitigate interpolation bias through unit regressing while concurrently reducing extrapolation bias by synthesizing all fitted control units by non-negative weights.

Several related articles have addressed the challenge of dealing with datasets that include too many control units, leading to that the solution of the SC estimator is not unique (Abadie et al., 2015). Robbins et al. (2017) and Abadie and L’Hour (2021) adapt the original SC proposal to incorporate a penalty on the weights into the SC optimization problem. Gobillon and Magnac (2016) makes use of dimension reduction strategies to improve the estimator’s performance. Doudchenko and Imbens (2016) suggests selecting the set of best controls by restricting the number of controls allowed to be different from zero using an $l_{0}$-penalty on the weights. While the SRC method does not employ a penalty to tackle the problem of an excessive number of unit, the criterion used in a penalty-style fashion stems from constructing an unbiased estimator of the risk associated with the synthetic estimator. In situations where the number of units is large with respect to the number of time periods, a preprocessing step of screening units can extend the SRC method.

Our article is related to Kellogg et al. (2021), which proposes the matching and synthetic control (MASC) estimator by using a weighted average of the SC and matching estimators to balance interpolation and extrapolation bias. Our SRC method differs from MASC in several ways. First, SRC does not use the matching estimator and instead considers pre-treatment fit of each control unit that aligns the treated unit. Second, while the MASC estimator is a weighted average between the SC estimator and the matching estimator, the SRC estimator is a synthetic estimator that incorporates all fitted controls. Finally, the methods also differ in terms of how the weights are chosen. For the MASC estimator, the only weight is chosen by the cross-validation method. In contrast, the SRC method involves solving multiple weights, and we employ the unbiased risk estimator criterion to determine these weights.

Our article is also connected to Athey et al. (2019) and Viviano and Bradic (2023), which have explored the advantage of model averaging within the realm of synthetic control. While Athey et al. (2019) combines several regularized SC and matrix completion estimators developed in Doudchenko and Imbens (2016) and Athey et al. (2021), and Viviano and Bradic (2023) combines a large number of estimators from the machine learning literature. In contrast, the weighting scheme in our SRC method aims to ensemble all fitted units, mitigating the risk associated with the resulting synthetic estimator. We leverage the pre-treatment fitted estimators from each control unit as the proxied control units for synthesis. The averaging process in our approach serves to alleviate the inherent risk in the synthetic estimator. This distinguishes our method from previous approaches, which involve averaging various types of estimators or combining multiple estimators from the machine learning literature.

Finally, in addition to SC-style weighting strategies, there have been articles that directly use outcome modeling approaches. These include the panel data approach in Hsiao et al. (2012), the generalized synthetic control method in Xu (2017), the matrix completion method in Athey et al. (2021), and the synthetic difference-in-differences method in Arkhangelsky et al. (2021). In this article we focus on studying the synthetic control framework.

The goal of unit regressing is to establish a correspondence between each control unit and the treated unit, aiming to minimize the distance between them. This process entails conducting a univariate regression analysis where the treated unit is regressed on the control unit. By doing so, we can estimate the counterfactual outcome for each control unit based on the pre-intervention regression fit.

We approximate $\mu_{1t}$ in (1) by control unit $j$ in the following working model

where $e_{jt}$ represents the approximation error of $\mu_{1t}$ by $\theta_{j}Y_{jt}$, and $\alpha_{j}$ and $\theta_{j}$ are the intercept and coefficient, respectively, of the regression. Given $\theta_{j}$, $\alpha_{j}$ is estimated by

We construct the fitted control

where $\theta_{j}$ is obtained by minimizing the simple Euclidean distance: $\min_{\theta_{j}\in\mathbb{R}}\|\mathbf{y}_{1}-\bar{y}_{1}\mathbf{1}+\theta_{j}(\mathbf{y}_{j}-\bar{y}_{j}\mathbf{1})\|_{\mathbf{V}}^{2}$. In practice, we use the procedure employed in the SC method to choose $\mathbf{V}$ (Abadie and Gardeazabal, 2003, Abadie et al., 2010, and Abadie et al., 2015). Because $\mathbf{V}$ can be absorbed into $\mathbf{y}_{1}$ and $\mathbf{y}_{j}$, without loss of generality we simply rewrite the minimization as

Using the regression method is to mimic the behavior of the treated unit before the intervention as closely as possible. It follows the least squares estimator

Consequently, the fitted control unit $j$ at time $t$ is given by

We utilize the synthetic method by assigning a weight for the matching estimator for each control unit. Let $\boldsymbol{w}=(w_{2},\cdots,w_{J+1})^{\top}$ be the weight vector in the simplex in $\mathbb{R}^{J}$:

Note we do not require that $\sum_{j=2}^{J+1}w_{j}=1$. Then the synthetic regressing control estimator is

Let us call it “Synthetic Regressing Control” (SRC). In contrast to SC, the weight in SRC is separate from the regression coefficient. The weight represents the degree to which the matched characteristic is considered in the synthesis process, while the regression coefficient reflects the relationship between the treated and control units. Similar to SC, SRC leverages the synthetic method to control extrapolation error by assigning more weight to the matched characteristics that demonstrate higher similarity.

To perform inference on the estimated causal effect, we apply the placebo permutation-based approach test (Abadie et al., 2010). It applies the synthetic controls estimator to each control unit by pretending this control unit is the treated one. If there is an actual treatment effect only in the treatment group post-intervention, then the estimated effect for the actual treatment unit should be among the most extreme.

Algorithm 2 provides the pseudo-code for the placebo permutation test. The obtained probability $p_{t}$ provides the probability of observing a difference between the observable $Y_{1t}(0)$ and the estimated counterfactual $\hat{Y}_{1t}(0)$ given all permutations of the treatment and control units.

We extend the application of the SRC method to cases where $J\geq T_{0}$ or $J\approx T_{0}$. To accomplish this, we propose a practical procedure that involves screening the units using the sure independent ranking and screening (SIRS) method (Zhu et al., 2011) to reduce the number of units. In high-dimensional statistics, Theorems 2 and 3 in Zhu et al. (2011) indicate that SIRS can reduce the dimensionality without losing any active variables with a probability approaching one. We prefer SIRS over the original sure independence screening proposed by Fan and Lv (2008) because it allows us to assume that no linear candidate model is correct. In model (3), any working model is an approximation of the expected counterfactual value $\mu_{1t}$.

For applying SIRS into the control units, we assume that $\mu_{1t}$ depends only on some of the control units, called as active units, in this study. SIRS screens the units based on the magnitude of the following statistics instead of the marginal correlation,

Derivation and interpretation of this statistics can be found in Zhu et al. (2011). We use the statistics $\tilde{\eta}_{j}$ for screening units, then obtain a set that involves any activate units. Once the screened units are reduced, we perform the SRC method on these units to obtain the estimator. We summarize it as Algorithm 3 below.

We have focused on matching pre-treatment values of the outcome variable. In practice, we typically observe a set of auxiliary covariates as well. For example, in the study of Proposition 99, Abadie et al. (2010) considers the following covariates: average retail price of cigarettes, per capita state personal income, per capita beer consumption, and the percentage of the population age 15–24.

It is natural to incorporate auxiliary covariates in applying the SRC method. For unit $j$, denote $\mathbf{x}_{j}$ as a $(p\times 1)$ vector of observed covariates that are not affected by the intervention. Let $\mathbf{X}=(\mathbf{x}_{1},\mathbf{x}_{2},\cdots,\mathbf{x}_{J+1})$. Analogous to the SC method (Abadie et al., 2010), We define the augmented $(T_{d}\times 1)$, where $T_{d}=T_{0}+p$, vector of pre-intervention characteristics for the treated unit $\mathbf{z}_{1}=(\mathbf{y}_{1}^{\top},\mathbf{x}_{1})^{\top}\in\mathbb{R}^{T_{d}}$. Similarly, $\mathbf{Z}_{0}$ is a $(T_{d}\times J_{0})$ matrix that contains the same variables for the control units.

We apply Algorithm 1 on $\mathbf{z}_{1}$ and $\mathbf{Z}_{0}$ to obtain $\hat{w}_{j}^{(\mathbf{z})}$ and $\hat{\theta}_{j}^{(\mathbf{z})}$, and then obtain the SRC estimator

We summarize it as Algorithm 4 below.

Now we investigate the finite sample performance of alternative estimators in the simulation experiments using a factor model. We compare several representative synthetic estimators, including: (a) the original SC method (SC) in Abadie et al. (2010), (b) the de-meaned SC (dSC) in Ferman and Pinto (2021), (c) the augmented SC Method (ASC) in Ben-Michael et al. (2021), (d) the matching and SC method (MASC) in Kellogg et al. (2021), (e) OLS in Hsiao et al. (2012), and (f) the constrained lasso (lasso) in Chernozhukov et al. (2021).

In this experiment, all units are generated according to the factor model as follows

where time specific terms $\alpha_{t}\sim\mathcal{N}(0,1)$, and unobserved factors $\lambda_{t}\sim\mathcal{N}(0,1)$. Similarly as Li (2020), we consider three sets of factor loadings $f_{j}$. (1) F1: $f_{j}=1$ for $j=1,\cdots,7$, and $f_{j}=0$ for $j=8,\cdots,J+1$; (2) F2: $f_{1}=3$, $f_{j}=1$ for $j=2,\cdots,7$, and $f_{j}=0$ for $j=8,\cdots,J+1$; (3) F3: $f_{1}=3$, $f_{j}=1$ for $j=2,\cdots,J+1$; For F1, all nonzero factor loadings are set to be ones so that both treated and control units with nonzero loadings are drawn from a common distribution. In contrast, for F2, treated and control units are drawn from two heterogeneous distributions, since loadings for the treated unit all equal to 3, and the control units’ nonzero loadings all equal to 1. The setting of F3 is similar to F2, except that control units’ loadings all equal to 1. We set $T=50$ with $T_{0}=40$ and $J=20$. The errors $\epsilon_{jt}\sim\mathcal{N}(0,\sigma^{2})$, where we use three values of $\sigma$ values, 1, 0.5, and 0.1, to investigate the impact of $\sigma$.

To evaluate each estimator, we compute the mean squared prediction error (MSPE), which is defined as $\text{MSPE}=(T-T_{0})^{-1}\sum\limits_{t=T_{0}+1}^{T}\|\hat{Y}_{1,t}(0)-Y_{1,t}(0)\|^{2}$, by calculating the average loss across 500 simulations. The results are reported in Table 1. For the homoscedastic setting of F1, the SC, dSC, ASC, and MASC methods perform similarly as lasso, and are better than SRC. For the heteroscedastic settings of F2 and F3, SRC is better than other methods except for the case of F2 and $\sigma=0.1$, where it is slightly worse than ASC. Especially for the dense setting F3, the SC, dSC, ASC, and MASC methods work as badly as lasso, while SRC works pretty well. The observation regarding SRC aligns with the findings in Section 3.1.1, indicating that the extent of improvement is anticipated to increase as the disparity between $f_{1}f_{j}$ and $f_{j}^{2}$, where $j\neq 1$, becomes more pronounced. Meanwhile, OLS works well in this setting, but is still worse than SRC.

Comparing the results across various $\sigma$ values, we find that the above observations hold true. Notably, we observe that the MSPE values of both the SRC and OLS estimators approach zero as $\sigma$ decreases from 1 to 0.1 across all three scenarios of $\boldsymbol{f}$. However, we do not observe this convergence in the SC, dSC, MASC, and lasso methods for the heteroscedastic settings, nor in ASC for the setting of F3.

In order to assess the impact of $(T,J)$, we consider three scenarios: $(T,J)$ takes on values of $(100,20)$, $(50,10)$, and $(50,50)$. The results are reported in Section S1 of the Online Information. We find that these methods consistently demonstrate robust performance across different values of $(T,J)$. It is noteworthy that in the case of $(50,50)$, we have $T_{0}<J$. For this case, we employ the SRC estimator by applying Algorithm 3 with SIRS preprocessing on screening units as described in Section 4.1. These results obtained in the $(50,50)$ case highlight the effectiveness of the SIRS preprocessing step on screening units within the extended SRC method.

We study the effect of terrorism on per capita GDP in Basque, Spain. The Basque dataset is from Abadie and Gardeazabal (2003). It consists of per capita GDP of 17 regions in Spain from 1955 to 1997, and 12 other covariates of each region over the same time interval, representing education, investment, sectional shares, and population density in each region. We incorporate auxiliary covariates which include averages for the 13 characteristics from 1960 to 1969, and scale each covariate so that it has equal variance of outcomes. In this study, the treated unit is the Basque Country, and the treatment is the onset of separatist terrorism, which begins in 1970.

Placebo Analyses. Similar to Abadie and Gardeazabal (2003), we conduct a placebo study to compare alternative estimators in the real data. We perform placebo analyses on each region, excluding Basque, as the placebo region. We calculate the mean squared prediction error (MSPE) for each region by taking the differences between its actual and fitted outcome paths in each of the post-period years (1970-1997), squaring these differences and then averaging them among these years. The results of our analysis are presented in Table 2, which shows that, on average, SRC tends to have the lowest MSPE. In addition, we include the pre-period fit of these estimator in Section S2 of the Online Information to further demonstrate their performance. Interestingly, we observe that SRC does not exhibit the best pre-period fit on average (it is the second best), while ASC demonstrates the best pre-period fit on average. This observation suggests that SRC is less prone to over-fitting compared to ASC.

Synthetic Basque. We estimate the effect of exposure to terrorism on GDP per capita in Basque, Spain. We present the GDP per capita for both Basque and its synthetic control, generated using the SC and SRC methods, in Figure 1. We also show the difference between the actual values and the synthetic values. Due to saving space and for ease of comparison, we just report the comparison between SRC and the original SC method, ignoring other estimators. To gain a better understanding of the SRC method, we inspect the comprehensive weight $\theta_{j}w_{j}$ assigned to each control unit $j$ and compare it with the weights obtained from other methods. The results are reported in Table 3. The SC, dSC, and MASC methods assign non-zero weights to just two control units. The lasso method assigns non-zero weights to six control units. Comparing to them, SRC assigns non-zero weights to five units. It is important to note that OLS does not have zero weight since it is not subject to constraints. These findings suggest that, in contrast to other constrained methods, the greater flexibility of SRC may contribute to enhanced predictive power.