Diff in Diff: From 2x2 to GxT DID

Advances, Problems and Solutions

Fernando Rios-Avila

Levy Economics Institute of Bard College

How GxT DID was suppoused to work

While the 2x2 DID design is simple to understand, it does not reflect the type of information that is available in most cases. (G>2 and T>2)
Until few years ago, when this kind of data was available, the standard approach was to use a generalized DID design, which extended the use of Fixed Effects:

$\begin{aligned} 2\times2: Y_{i,t} &= \beta_0 + \beta_1 T + \beta_2 D_i + \beta_3 (D_i \times T) + \epsilon_{i,t} \\ G\times T: Y_{i,t} &= \sum \gamma_t + \sum \delta_i + \theta^{fe} PT_{i,t} + \epsilon_{i,t} \\ \end{aligned}$

where $PT_{i,t}$ assume the value of 1 for the treated group After the treatment is applied, and 0 otherwise, and $\theta^{fe}$ representing the treatment effect.

Little that we know that this approach only works if the treatment effect is homogeneous across all groups and periods.

$\widetilde{PT}_{it}=PT_{it}-\overline{PT}_{i}-\overline{PT}_{t}+\overline{PT}$

$\overline{PT}_{i}$ is the share of periods a unit is observed to be treated. (larger for early treated)
$\overline{PT}_{t}$ is the share of units treated at time $t$ . (increasing)
$\overline{PT}$ Share of treated “unit-periods”

Thus,

Units that are treated early, (high $\overline{PT}_{i}$ )
but are analyzed at later periods ( $\overline{PT}_{t}$ increasing in $t$ )

Will be more likely to have a negative weight $w_{it}$ , thus contaminating the ATT estimate.

GxT DID: Generalized DID

Setup

In the Generalized DID
- One has access to multiple periods of data: $t = 1,2,...,T$ .
- And units can be treated at any point before, during or after the available Data $G = -k,..1,2,..,T+l$ .
  - This is the cohort or group.
- We also assume that once a unit is treated, it remains treated. (no reversals/no cohort-change)
From our perspective, units treated at or before $t=1$ will be considered as Allways treated.
- This units cannot be used for analysis.
For any practical purpose, if we do not observe a unit being treated in the window of time observed, we assume its Never Treated.
- For Notation we say these units belong to $g=\infty$ (or that they could be treated at some point in the far future)

GxT DID: Generalized DID

Potential Outcomes

In contrast with previous setup, with the GDID, we believe observations have multiple potential outcomes, for each period of time.
- $Y_{i,t}(G)$ is the potenital outcome for individual $i$ at time $t$ , if this unit would be treated at time $G$ .
- Thus we state that depending on “when” a unit is treated, the potential outcomes could be different.
- This is what it means allowing for heterogeneity.

GxT DID: Generalized DID

Parallel Trends Assumption

PTA assumption is also slightly modified. Because we can differentiate between never-treated and not-yet-treated, one could impose those PTA assumptions.

Never Treated $E(Y_{i,t}(\infty) - Y_{i,t-s}(\infty)|G=g) = E(Y_{i,t}(\infty) - Y_{i,t-s}(\infty)|G=\infty) \forall s>0$

Not Yet Treated

$E(Y_{i,t}(\infty) - Y_{i,t-s}(\infty)|G=g) = E(Y_{i,t}(\infty) - Y_{i,t-s}(\infty)|G=g') \forall s>0$
Which suggests PTA hold for all pre- and post-treatment periods.
Some methods only rely on Post-treatment PTA.

Avoid using bad controls: `did_imputation` and `did2s`

Borusyak, Jaravel, and Spiess (2023) and Gardner (2022)

Both methods are based on the idea of “imputing” the missing potential outcomes $Y_{it}(0)$ for the treated group, using a method similar to the one used for 2x2 DID. A two-stage approach.

We know that estimating the following model would be incorrect:

$y_{it} = \delta_i + \gamma_t + \theta D_{it} + \epsilon_{it}$

However, what this authors suggest is to identify $\delta_i$ and $\gamma_t$ using pre-treatment data only:

$y_{it} = \delta_i + \gamma_t + \epsilon_{it} \text{ if } t<g$

This helps identifying the fixed effects, without any contamination. (its a model for the potential outcome of no treatment)

Allow for Heterogeneity: `jwdid` and `eventstudyinteract`

Wooldridge (2021) and Sun and Abraham (2021)

The work by Wooldridge (2021) suggested that the GDID-TWFE estimator was not a problem perse.
The problem was that the GDID-TWFE estimator was simply misspecified.
Instead of modeling: $Y_{i,t} = \delta_i + \gamma_t + \theta^{fe} PT_{i,t} + \epsilon_{i,t}$
One should allow for a full set of interactions between the group and time dummies:

$Y_{i,t} = \delta_i + \gamma_t + \sum_{g=2}^T \sum_{t=g}^T \theta_{g,t} \mathbb{1}(G=g,T=t) + \epsilon_{i,t}$

In this framework, each $\theta_{g,t}$ represents the ATT for each group at a particular period.

Allow for Heterogeneity: `jwdid` and `eventstudyinteract`

Wooldridge (2021) and Sun and Abraham (2021)

In the basic setup, this approach is basically the same as the method proposed by Borusyak, Jaravel, and Spiess (2023) and Gardner (2022).
Wooldridge, however, was not the first approach that aim to “allow for heterogeneity” in the treatment effect. Early attempts were done by using a dynamic events structure, using both leads ands lags of the treatment variable. $Y_{i,t} = \delta_i + \gamma_t + \sum_{e=-k}^{-2} \theta_e \mathbb{1}(t-G_i=e) + \sum_{e=0}^L \theta_e \mathbb{1}(t-G_i=e) + \epsilon_{i,t}$
This not only allows for heterogenous effects across time, but also allows you to analyze pre-treatments effects.
However Sun and Abraham (2021) showed that this approach could also be wrong, if dynamic effects are also heterogenous across groups.

Allow for Heterogeneity: `jwdid` and `eventstudyinteract`

Wooldridge (2021) and Sun and Abraham (2021)

As solution, Sun and Abraham (2021) propose to use a full set of interactions between the group dummies and the event-study dummies. This is similar to Wooldridge (2021). $Y_{i,t} = \delta_i + \gamma_t + \sum_{g=2}^T \sum_{e=-k}^{-2} \theta_{g,e} \mathbb{1}(t-G_i=e, G=g) + \sum_{g=2}^T \sum_{e=0}^L \theta_{g,e} \mathbb{1}(t-G_i=e, G=g) + \epsilon_{i,t}$
In fact, if we write the “event” as “time”, it would look very similar to the model proposed by Wooldridge (2021).

$Y_{i,t} = \delta_i + \gamma_t + \sum_{g=2}^T \sum_{t=1}^{g-2} \theta_{g,t} \mathbb{1}(T=t, G=g) + \sum_{g=2}^T \sum_{t=g}^{T} \theta_{g,t} \mathbb{1}(T=t, G=g) + \epsilon_{i,t}$

Thus, both approaches are identical if we allow for Full interactions before and after treatment.

Using Good DID Designs: `csdid` and `csdid2`

Callaway and Sant’Anna (2021)

The proposed estimator starts with the assumption that one is interested in the ATTGT:

$ATT(g,t) = E(y_{i,t}(G) - y_{i,t}(\infty) |G_i =g)$

As in the simpler 2x2 case, however, we cannot observe the $y_{i,t}(\infty)$ after they are treated.
What Callaway and Sant’Anna (2021) does is to impute this piece applyint PTA and no Anticipation.
${E(y_{i,t}(\infty)|G_i =g)} \approx E(y_{i,G-k}|G_i =g) + E(y_{i,t}-y_{i,G-k}| G_i \in Control)$
Thus, the estimator for the ATT(g,t) becomes: $\widehat{ATT(g,t)} = E(y_{i,t}- y_{i,G-k}|G_i =g)- E(y_{i,t}- y_{i,G-k}|G_i \in Control)$

Using Good DID Designs: `csdid` and `csdid2`

Callaway and Sant’Anna (2021)

$\widehat{ATT(g,t)} = E(y_{i,t}- y_{i,G-k}|G_i =g)- E(y_{i,t}- y_{i,G-k}|G_i \in Control)$

With no anticipation and PTA, ANY period before treatment ( $G-k$ ) could be used to construct the ATT(g,t).
Because of this, it only relies on Post-treatment PTA. (Violations of PTA before treatment have no impact on the estimates)
Depending on the analysis of interest, one could choose different “control groups”
- Never treated: Most common
- Not-yet-treated: Include observations that up to time $t$ have not been treated.
- For pre-treatment ATT(g,t)’s, the Not-yet treated cound include all cohorts not treated until $t$ R or those not treat until $t$ nor $g$ Stata.

Using Good DID Designs: `csdid` and `csdid2`

Callaway and Sant’Anna (2021)

This approach is relatively easy to implement. The main difficulty is keeping track of ALL the ATT(g,t)s that are estimated (and their VCV)
However, once all ATT(g,t)’s are obtained, aggregation is straight forward:

$AGG(ATT(g,t)) = \frac{\sum_{g,t \in G\times T} ATT(g,t) * w_{g,t} * sel_{g,t} } {\sum_{g,t \in G\times T} w_{g,t} * sel_{g,t} }$

where $w_{g,t}$ represents the size of cohort $g$ at time $t$ used in the estimation of ATT(g,t).
and $sel_{g,t}$ is an indicator for whether that ATTGT will be used in the aggregation.

Using Good DID Designs: `csdid` and `csdid2`

Callaway and Sant’Anna (2021)

Typical Aggregations:

Simple: $sel_{g,t}=1$ if $t>=g$
Group/cohort: $sel_{g,t}=1$ if $t>=g$ and $g=G$
Calendar: $sel_{g,t}=1$ if $t>=g$ and $t=T$
Event: $sel_{g,t}=1$ if $t-g=e$
Cevent: $sel_{g,t}=1$ if $t-g \in [ll,\dots,uu]$

and may be possible to combine some of these restrictions

Also, because the method is based on 2x2 DID, it allows to easily implement various methodologies, including the Doubly Robust.

1 / 56

Diff in Diff: From 2x2 to GxT DID Advances, Problems and Solutions Fernando Rios-Avila Levy Economics Institute of Bard College

Diff in Diff: From 2x2 to GxT DID
Why differences in diffences?
2x2 DID: Cannonic Design
Potential outcomes and Treatment Effects
The DID Estimator: Assumptions
The DID Estimator
Graphically
How to estimate it?
Extension: Adding Controls
Problems of Adding Controls
Potential Solution: Sant’Anna and Zhao (2020)
Added Assumption
Potential Solution: drdid, teffects
Panel Data
Panel Data: Adding controls
Panel Data: Methods
Panel Data: Methods
Example
Highlights I: Identification
Highlighs II: Overlapping
Highlighs III: Controls
From 2x2 to GxT DID
How GxT DID was suppoused to work
Why it doesn’t work?
The “Bad Controls” Problem
The “Bad Controls” Problem
Negative Weights
\[\widetilde{PT}_{it}=PT_{it}-\overline{PT}_{i}-\overline{PT}_{t}+\overline{PT}\]...
Graphically
Summarizing the Problem
GxT DID: Generalized DID
GxT DID: Generalized DID
GxT DID: Generalized DID
GxT DID: Generalized DID
Solutions
General Overview
Avoid using bad controls: did_imputation and did2s
Avoid using bad controls: did_imputation and did2s
Avoid using bad controls: did_imputation and did2s
Example
Allow for Heterogeneity: jwdid and eventstudyinteract
Allow for Heterogeneity: jwdid and eventstudyinteract
Allow for Heterogeneity: jwdid and eventstudyinteract
Allow for Heterogeneity: jwdid and eventstudyinteract
Implementation
Using Good DID Designs: csdid and csdid2
Using Good DID Designs: csdid and csdid2
Using Good DID Designs: csdid and csdid2
Using Good DID Designs: csdid and csdid2
Using Good DID Designs: csdid and csdid2
Map: How Callaway and Sant’Anna (2021) relates to Wooldridge (2021) and Borusyak, Jaravel, and Spiess (2023)
Example
Conclusions
Conclusions
Thank you!
Suggested readings