Before Identification and Inference: Where the Causal Claim Enters
Several recent posts by Carlos Chavez are on themes close to my recent posts on causality, quasi-experiments etc. I thought discussing their similarities and differences would be interesting. Specifically his posts on inference and identification, and mine on the question: where do causal claims enter research designs at all?
Read together, they help explain why applied work can be technically careful and still rest on weak causal reasoning.
What Chavez Clarifies
Chavez is precise about what inference is in the frequentist tradition: reasoning about uncertainty from finite data.
The target parameter is fixed; estimates vary across hypothetical repetitions.
Standard errors, confidence intervals, and p-values describe that variation. They are not probabilities about hypotheses or effects.
A central point in his inference piece is that none of this makes sense without a real source of variation. Sampling, randomization, or a well-defined assignment mechanism has to exist. Without it, inferential results drift away from the problem they are supposed to address.
This is why choices that look technical matter:
clustering decisions
heteroskedastic-robust errors
weak-instrument diagnostics
RD bandwidths
They encode assumptions about dependence, repetition, and what would change if the study were run again.
In his post on identification, Chavez pushes the logic one step back. Identification is not about precision but whether the assumptions plus the data pin down an object at all. Infinite data cannot rescue a failure of identification; noisy data do not preclude it.
Across IV, DiD, RD, bounds, or macro restrictions, the logic is the same. Assumptions define the object. The data rule out alternatives.
Both posts draw a clear boundary. Even perfect identification and correct inference do not answer questions about scale, transportability, or mechanism. Those sit outside the statistical apparatus.
It is worth being explicit that this limitation is not specific to frequentist inference. Bayesian inference does not resolve it either. It relocates the causal claim into priors rather than asymptotics, but the object still has to be defined before probability enters. When the counterfactual itself is not generated by a repeatable process, no inferential formalism – frequentist or Bayesian – can supply it.
Where My Essay Starts
My essay begins earlier, before identification or inference are even in view.
In most quasi-experimental work:
the population is fixed
history is singular
no unit is observed under both regimes
There is no literal counterfactual in the data.
The causal claim enters when the researcher defines:
the baseline regime – intercepts, trends, fixed effects
the perturbation – policy changes, instruments, timing
the units or periods standing in for the counterfactual
Those choices already assert a path outcomes would have followed absent the perturbation. That assertion is the causal model.
Estimation measures the implied contrast.
Inference assesses its uncertainty or stability.
Neither creates causality.
Many applied errors follow from misreading what the design actually delivered. A local, design-defined contrast is treated as a portable parameter, then scaled as if the baseline regime were known or irrelevant.
The “missing intercept” problem is not technical sloppiness. It is a mistake about where causal content lives.
How the Pieces Fit Together
Seen this way, the posts fit together well. Chavez shows that inference only has meaning relative to a real source of variation, while I argue that the same variation also fixes what the causal claim is.
The variation does double duty. It determines:
what can be inferred
what the causal statement means
Once this is clear, the workflow becomes explicit:
Causal contrast (design choice)
→ Identification
→ Inference
→ Generalization
Design and interpretation do most of the work. Statistics occupies the narrow space between them.
Why This Matters
Many applied debates mislocate causal force. First-stage F-statistics, parallel-trends tests, and clustered standard errors speak to internal consistency and precision. They do not justify the counterfactual being invoked.
Credibility comes from clarity about:
the baseline being assumed
the variation doing the work
how local and conditional the resulting claim is
The hardest work happens before the regression runs. Chavez’s posts make that clear for inference and identification while my makes the same point for causality. Nuanced econometrics is necessary but alone still is not enough.
Fin