Randomization Inference Tests for Shift-Share Designs¹¹1We would like to thank Peter Hull for excellent comments and suggestions.

Suppose our goal is to test the null that $\beta=b$ against either a unilateral or bilateral alternative. Let $\hat{T}=\bar{T}(g,\boldsymbol{S},\boldsymbol{X},\boldsymbol{Y})$ be a test statistic, where large values of $\hat{T}$ constitute evidence against the null. We assume the following condition on the test statistic.

That is, under the null, the test-statistic depends on $\boldsymbol{Y}$ and $\boldsymbol{X}$ solely through the “null-imposed residuals” $\boldsymbol{e}_{b}=\boldsymbol{Y}-\boldsymbol{X}b$.

Suppose now that the researcher has a guess on the shock assignment mechanism, i.e on the conditional probabilities $\mathbb{P}[g\leq v|\boldsymbol{S},\boldsymbol{\epsilon}]$ for each $v\in\mathbb{R}^{J}$. Let us denote such guess by a conditional distribution function $\mathbb{H}(\cdot|\boldsymbol{s},\boldsymbol{e})$ which specifies, for each $(\boldsymbol{s},\boldsymbol{e})$ in the support of $(\boldsymbol{S},\boldsymbol{\epsilon})$, a cummulative distribution function $\mathbb{H}(\cdot|\boldsymbol{s},\boldsymbol{e})$ on $\mathbb{R}^{J}$. In this case, the researcher is able to compute critical values by analysing the quantiles of

Such quantity is easily estimable by simulation. Indeed, if we are able to draw $L$ independent draws $g^{*}_{l}$, $l=1,\ldots,L$, from $\mathbb{H}[\cdot|\boldsymbol{S},\boldsymbol{e}_{b}]$, then $H_{b}(c|\boldsymbol{S},\boldsymbol{X},\boldsymbol{Y})$ may be estimated as

Clearly, if the shock-assignment process is correctly specified, then the procedure above provides valid inference.

The result presented above is similar to Proposition S.3 from BH, with a couple of minor differences. First, we allow the shock assignment mechanism to depend on the non-observables. This way, we allow for the distribution of $S$ to depend on $\boldsymbol{\epsilon}$ (though in practice we expect that applied researchers would rarely choose a $\mathbb{H}[\cdot|\boldsymbol{S},\boldsymbol{e}_{b}]$ that depends on $\boldsymbol{e}_{b}$). Also, we provide level guarantees with a finite number of simulations. We present details of the proof in Appendix A.1.1.

In their paper, BH consider basing inference on the test statistic

which depends on $(\boldsymbol{X},\boldsymbol{Y})$ solely through $\boldsymbol{e}_{b}$, so Assumption 2 holds.

In contrast, we consider the test-statistic

where $\hat{V}_{N,b}$ is the null-imposed variance estimator from AKM and BHJ,⁷⁷7In a setting without an intercept or controls, BHJ show their variance estimator collapses to AKM’s.

Observe that, under the null, the above map satisfies

so this test statistic also satisfies Assumption 2. This corresponds to a rescaled version of the test statistic in BH.

Inference may be conducted as follows:

In this section, we consider the asymptotic properties of the simulation-based approach. We consider the properties of the tests in a framework where the number of sectors, $J$, is large. The number of units, $N$, is (implicitly) indexed by $J$, and is also allowed to grow.

We adopt a finite population perspective and allow for treatment effects to vary by unit. Formally, for a given $J\in\mathbb{N}$, potential outcomes are given by

and observed outcomes are given by

whereas the instrument is given by $Z_{i,J}=\boldsymbol{s}_{i,J}^{\prime}g_{J}$. We will treat the $\epsilon_{i,J}$, $\beta_{i,J}$ and $\boldsymbol{s}_{i,J}$ as nonrandom throughout – the only source of randomness stems from the assignment of $g_{J}$ and the treatment $X_{i,J}$. In other words, we follow a “design-based” approach. In this setting, the shift-share identification assumption is written as follows.

We rewrite the outcome model as

where

and

We consider the goal of the researcher to be to conduct inference on $\beta_{J}$, an affine combination of individual treatment effects. Specifically, she would like to test the null that $\beta_{J}=b_{J}$, and for that she uses one of the procedures described in the previous section.

Following AKM and BHJ, we put $v_{J}=\sum_{j=1}^{J}|\sum_{i=1}^{N}{s}_{i,j,J}|^{2}$ and assume $v_{J}\to\infty$. In the next proposition, we provide conditions for (conditional) asymptotic normality of $T^{*}_{1}=T_{1}(g^{*}_{J},\boldsymbol{S},\boldsymbol{Y}-b_{J}\boldsymbol{X})$, the test statistic ${T}_{1}$ constructed under simulated shocks $g^{*}_{J}$.

We present details of the proof in Appendix A.1.2. Proposition 2 provides high-level conditions for conditional asymptotic normality of the simulated statistic. In Appendix A.2.1, we show that these conditions are satisfied for three examples of simulation distributions: (i) when we sample with replacement from the (recentered) empirical distribution of shocks; (ii) when we consider shocks iid $N(0,1)$, independently from $\mathbf{X}$ and $g_{J}$, and; (iii) when we consider sign-changes of observed shocks.⁹⁹9In the Appendix, we consider sign changes without recentering shocks ($m=0$). We note, however, that convergence would hold for any choice of recentering parameter $m$, including the case in which it is misspecified, and the case in which $m$ is replaced by an estimator such as the sample mean of shocks; provided we work with the shift-share estimator that uses demeaned shocks (as per footnote 2). The crucial point is that we consider a studentized test statistic, so the simulated test statistic is asymptotically $N(0,1)$. In contrast, if we considered alternative test statistics, such as $\hat{T}_{0}$, then we would not reach this conclusion. Studentizing the test statistic using robust standard errors would also generally not work.

As a byproduct of Proposition 2, whenever inference based on $\hat{T}_{1}$ and normal critical values provides asymptotically conservative inference, the simulation-based approach will also lead to asymptotically conservative inference. We summarize this fact in the corollary below.

When there is no treatment effect heterogeneity, it follows that, under the conditions in AKM and BHJ, $v=1$. These conditions include that shocks are independent, that the number of sectors increase, and that the relevance of sectors are asymptotically negligible (AKM and BHJ consider alternatives that relax the assumption that shocks are independent, and we discuss that in Remark 4). In this case, inference based on the simulation approach is asymptotically size $\alpha$. More generally, when there is treatment effect heterogeneity, AKM provide sufficient conditions for inference based on $\hat{T}_{1}$ and normal critical-values being conservative (i.e. $v\geq 1$). These conditions limit the correlation between treatment effect heterogeneity and exposure weights. In this case, our simulation-based approach will also lead to conservative inference. Notice that, in contrast to our finite sample results, which require homogeneous treatment effects, asymptotically our method may be able to provide conservative inference under treatment effect heterogeneity.

Next, we analyze the test statistic $\hat{T}_{2}$. In this case, since the shift-share estimator is being recomputed across samples and then used in the calculation of the standard error, we need to ensure that $\hat{\beta}_{SS}^{*}$, the simulated shift-share estimator from Algorithm 2, is consistent at a given rate. In addition to the assumptions in Proposition 2, we require a “strong simulated shock” assumption that ensures that the variance of the simulated shift-share regressor does not vanish asymptotically; as well as conditions that ensure the estimation error of the standard error vanishes. We state these requirements in the proposition below:

We present details of the proof in Appendix A.1.3. In Appendix A.2.2, we discuss assumptions (i)-(iii) of the proposition in the context of our three examples of simulation distributions.