# Hindcasts

A hindcast runs the live method over events that already happened and scores it against the federal record. It is the fastest honest answer to "show me skill": the live track record grows one storm at a time, the hindcast covers decades on day one, and the live ledger then extends it.

`hurricane-loss.md` and `hurricane-loss.json` are generated by `scripts/hindcast-hurricane-loss.ts` and nothing in them is typed by hand. This file is the hand-written reading of them: what the numbers say, what they do not, and what to do about it. Rewrite it when the script's output changes.

## Hurricane loss, 1996–2024 (first run, 2026-10-28, model 4.2)

**What was run.** The crossing estimate the platform freezes for a live storm (`lib/case-studies/freeze-records.ts`: Holland wind field × 2020 housing within 450 km × Hazus curves), applied to every continental US hurricane strike in NOAA/HRD's list since 1996 at each crossing found in the IBTrACS best track. 53 of 56 strikes were estimated; Alex 2004 and Ophelia 2005 never put their center over US land (HRD flags them so), and Dorian 2019's Hatteras landfall falls between six-hourly fixes.

**What it was scored against.** NCEI's billion-dollar list (total cost, all perils and lines, 2024 dollars; 40 of the 53 storms reached it) and OpenFEMA's IHP + PA dollars under the storm's declarations (41 storms with dollars). Neither is residential wind loss, so the level of the ratio means little; its spread is the signal.

### What the numbers say

- **Against NCEI's total cost, 2010–2024: median 0.27, p10–p90 0.09–1.44, 48% of storms within a factor of 3 of the median** (n = 23). The whole record reads median 0.40 with the same p10 and a wider p90, which is exposure growth doing what the generated doc says it does.
- **The misses sort by physics the model does not have.** The lowest ratios are the surge and rain storms: Irene 2011 (0.01), Sandy 2012 (0.03), Matthew 2016 (0.09), Florence 2018 (0.10), Katrina 2005 (0.06 against $201bn). The model estimates wind on houses; those storms' cost was water. This is the model declining to estimate what it does not model, and it is the right behavior for a figure that is labeled residential wind — but it means the figure is not a storm-total and must never be presented as one.
- **The over-estimates are small intense storms.** Charley 2004 reads 9.3× ($249bn modeled against $26.7bn total): a Category 4 at Punta Gorda with a radius of maximum winds of about 10 km, which the best track does not carry, so the wind field took its size from intensity and latitude (Willoughby et al. 2006) and got roughly three times Charley's. Dennis 2005 (5.9×), Bonnie 1998 (4.5×) and Jeanne 2004 (3.5×) are the same shape. Ian 2022, the same coast at the same wind, reads 1.69×; Milton 2024 reads 1.46× against the total and 1.38× against reported insured loss (the existing validation).

### What to do about it

1. **Storm size is the first model finding.** The live path reads the advisory's 34-kt radius when it has one; the best track in `historical_hurricane_tracks` carries none. IBTrACS publishes the USA_R34/R50/R64 quadrant radii from 2004 on; ingesting them (a migration adding four columns and a backfill) would let the hindcast use measured size for the twenty-year window that matters most, and would show how much of Charley's 9.3× is size. Until then the SW Florida Cat 4 result is a known hot spot, not a calibrated one.
2. **Say "residential wind" every time.** The surge storms make the case: a reader who takes the figure as the storm's cost will be wrong by 10–30× on exactly the storms they care most about.
3. **The FEMA axis is weaker than the NCEI axis.** IHP + PA is the federal response, lags by years, carries no dollars for most 1990s disasters, and in a busy season declarations can belong to the next storm; the matcher uses a 7-day-before / 14-day-after window around the first crossing for that reason. Use it as a second opinion, not a reference.
4. **Re-run after every model change** and keep each run's document; the ratio band moving is the evidence that a change helped or hurt. The script is deterministic given the database and the catalogues.

### What this is not

Not a validation verdict (those are in `docs/stochastic-validation/`, held byte-equal by tests). Not a calibration: no parameter was changed in response to these numbers. Not a claim about the stochastic event set, which samples storms this replay does not.

## Pre-event hurricane loss, 1998–2024 (first run, 2026-10-31, model 4.2)

**What was run.** The same crossing estimate, run not on the storm as it was but on the storm as NHC **forecast** it 72, 48 and 24 hours before the actual first crossing: the official forecast (ATCF a-deck `OFCL` lines, fetched from NHC's archive; `scripts/hindcast-pre-event-hurricane.ts`, `lib/modeling/hindcast/pre-event-hurricane.ts`), interpolated to six-hourly fixes, its own first crossing onto US land found with the live finder, and the estimate run there with the forecast's intensity and size. 50 strikes had a crossing estimate to compare with; every one was matched to its a-deck. `pre-event-hurricane.md` and `.json` are generated; this is the reading.

**What the numbers say.**

- **Pre-event against at-crossing, 24 hours out: median 0.62, p10–p90 0.07–6.08, 51% of storms within a factor of 3** (n = 39). At 48 hours, median 0.47 (n = 36); at 72, 0.73 with only a third within 3× (n = 27). The model is held fixed in every pair, so this band is forecast error in dollars, and it is wide: a day before landfall, half the storms' pre-event figures were more than 3× away from what the same model said at the crossing.
- **The misses are the storms everyone remembers.** Michael 2018 read $0.0bn at 72 h and $1.7bn at 24 h against $45bn at the crossing: the forecast intensity was far below what made landfall. Ian 2022 read $3.5bn at 48 h against $202bn: the track shift from Tampa Bay to Fort Myers. Milton 2024 read $239bn at 48 h against $50bn: the same shift in the other direction. Irma 2017 read $562bn at 72 h against $15bn: the forecast had a Category 5 over Miami.
- **A forecast track that never reaches US land is a real pre-event outcome and reads as $0:** 8 of 39 at 72 h, 4 of 48 at 24 h (Harvey, Matthew, Beryl among them). A customer holding only the forecast would have been shown nothing for those storms at that lead.
- **Against NCEI's total cost, 2010–2024, the pre-event figure reads median 0.05–0.13 by lead**, far below the crossing hindcast's 0.27, for both reasons at once: residential wind only, and the forecast.

**What to do about it.**

1. **Show the band, never a point.** The product this exists for is a pre-event range per county, sealed before the event; the range across leads (the `Range` column) is the honest shape, and the storms where it is widest are the ones where a point would have been most wrong.
2. **Score the record as it accrues.** Every live storm from now on gets its pre-event figures sealed in the daily digest and scored here after the fact. That is the asset: nobody publishes their pre-event number with a provable timestamp, and after a few seasons this table is the evidence.
3. **Read `historical_advisories` once it is loaded** (migration 228) rather than the archive files: same data, one source, and the live path and the hindcast then share it.
4. **Caveats to keep.** Older forecasts carry no 96/120 h point and some carry no intensity at the long taus, so a long-lead forecast whose landfall lay past its last tau reads as a coast miss. The issue time is approximated as synoptic time plus three hours. Size comes from the forecast's 34-kt radii where it carried them.

**What this is not.** Not a forecast verification — NHC publishes its own, in miles and knots; this is the same error carried through one model to dollars. Not a validation verdict and not a calibration. Not the storm's cost.