Spec rebuild defaults PUF donor fits to unweighted, reversing the verified landmine fix

[MaxGhenis]

Summary

The spec-driven rebuild defaults donor fits to unweighted, and specs/us-2024.yaml declares no weights: on any imputation step — including the PUF steps. This silently reverses the verified fix for the eCPS "landmine" failure mode, two days after the root cause was confirmed.

Where the default lives

• Blueprint (microplex/docs/spec-driven-rebuild.md §1): "Donor fits are unweighted by default. Set weights: donor_weight_column on a step only when deliberately testing a weighted donor fit." — i.e. weighted fits are framed as the experiment, unweighted as canonical.
• Engine (microplex/src/microplex/imputation.py, _resolve_step_weight_column and the surrounding fit path): "Omitted step.weights means the fit is intentionally unweighted, even if the donor has survey weights."
• Spec: grep -n "weights" specs/us-2024.yaml → no matches (both the copy here and microplex/packs/us/specs/us-2024.yaml).

Why this is the landmine bug class

The 2026-06-06 full-population analysis traced the eCPS income-tax blowup to QRF donor fits trained unweighted on the tail-oversampled PUF (policyengine-us-data puf_impute.py fits with no weight_col while _stratified_subsample_index keeps all top-0.5%-AGI donors). The PUF is stratified on the outcome (income), so unweighted conditional fits are biased toward the tail within demographic cells, and rare extreme donors get broadcast as point-mass landmines that detonate under any reweight. Measured then: eCPS miscellaneous_income max $795M with ~3,940 records >$100M (2,301 landmines surviving calibration); the weighted-fit microplex path produced max ~$1.4–2.2M and zero landmines, with calibrated income tax at a sane 9.3% of GDP.

The code that carried that fix — pe_source_impute_engine.py's run_conditioned_block fitting with weight_col="weight" from household_weight — was deleted in PolicyEngine/microplex-us#261 (last present at f3af332) and its behavior was not carried into the spec.

What's needed

A deliberate decision, recorded in the blueprint, between the two coherent designs:

1. Weighted donor fits (the verified fix): declare weights: <PUF design weight> on the PUF steps in us-2024.yaml, and flip the blueprint's default-framing for outcome-stratified donors. The engine already supports this (ImputationStep.weights).
2. Unweighted fits as a deliberate pool-building choice (eCPS-style: tail-rich record pool, calibration owns representativeness): then the compensating control must exist — a hard weight-ratio bound in the calibration solve policy (calibration/solve_policy.py has no such bound today), because this is exactly the configuration that produced the landmines in eCPS.

Either way this should be resolved before any spec-built candidate is scored against frozen eCPS — it changes what the scores mean.

🤖 Generated with Claude Code

[MaxGhenis]

Correction to the framing after reviewing the 06-07 session record: the unweighted default was a deliberate design choice, not an accidental regression — the rationale (relayed from an external review) treats imputation as a support-building stage (tail-rich record pool is a feature) and calibration as the representation stage, with weights: kept available per-step but intentionally empty for now.

That makes this issue a measured-adjudication question rather than a bug report. The two designs and their obligations:

1. Unweighted fits (current default): calibration owns representativeness → needs the compensating control the eCPS path lacked: a hard weight-ratio bound in the calibration solve policy, so tail-pool donors can't become population-scale point masses. calibration/solve_policy.py has no such bound today.
2. Weighted fits: the 06-06 measured result (zero landmines, sane income tax) — declare weights: on PUF steps.

Both should be scored through the sound comparison on the same baseline; the gates adjudicate. Transferring this issue to the engine repo since the spec + imputation default live there under the consolidation plan.

🤖 Generated with Claude Code

[MaxGhenis]

Measured adjudication, full-scale spec-built candidates (sound comparison, matched 41,314 households, ~3,704-target surface, eCPS baseline refit = 1.405 full / 0.317 holdout):

arm	donor-fit weighting	full loss	holdout	notes
A	unweighted (the blueprint default)	130.1	33.6	landmines at scale: candidate LTCG $201T vs $257B, qualified dividends $18.5T vs $154B — every synthetic unit draws from the PUF's high-income-oversampled sample conditionals
B	weights: weight step param	130.1-equivalent (totals unchanged)	—	the param is silently ignored for all numeric targets: regime_gated.fit never threads weights into the numeric chain, and quantile-forest's native sample_weight is a no-op for the predictive distribution. Fixed with regression tests in microimpute#196 @ 676a5ab
C	population-resampled donor (importance bootstrap ≡ weighted fit)	1.260	0.215	beats eCPS on full, holdout, and unweighted MSRE (1.04 vs 1.32)

Resolution: the unweighted default is unsafe as a default — it reproduces the eCPS landmine mechanism at three orders of magnitude when the synthetic records carry real weights. The support-building argument is real but belongs in pool design (deliberate tail strata at design weights — free support, calibration owns representation), not in fit weighting. Concretely: fits should default to the donor's design weight, with tail support added as an explicit verbatim stratum (capital gains and dividends are the two protected families still worse than eCPS in arm C, and they're exactly the tail-concentrated ones).

Blueprint default should flip once microimpute#196 (with 676a5ab) lands; until then weighted fits require the resampling workaround.

🤖 Generated with Claude Code