--- type: run --- # Collection sweep, 19 Sep 2026 (staging), before the review passes Opened by the owner session after Stream K12 deployed (0.51.0, migration `k12a1b2c3d4e`, `backfill stuck_runs` 0, `backfill prune_lines` 20). One process per host so each host's crawl delay applies: the ten Avolta airports in sequence in one process; Extime, Shilla, the four Shopify shops and The Loop each in their own. Logs in `sweep-2026-09-19/.log`. Watch it on `/collectors#live`. Skipped on purpose: the five refused sources `sweep plan` lists (never contacted), and **dubai-duty-free**, whose 403 to our declared identity is not recorded as a run (running list: `issue-dubai-duty-free-is-listed-as-reachable-by-sweep-plan-bec`); rian settles that, no bypass. **While this window is open: no deploy of staging** (a deploy kills a running collection). Close this entry with the per-source outcome when the last process ends. ## Closed 18 Sep 22:35 — the host rebooted mid-sweep; six of eight sources landed Mosiah ran out of memory at 20:54 and was rebooted at 20:57, killing everything still collecting. The cause was not ours: five global OOM kills across 15–18 Sep, all `CONSTRAINT_NONE`, all `uid=1001`, each 10.7–13 GB; **zero** `CONSTRAINT_MEMCG` kills, and no docker cgroup appears in any OOM process table. Our collectors run inside the app container under its 1 GB cap (the database publishes no port and the render sidecar is on an internal network, so nothing outside it can collect). In its last thirteen minutes avolta-ath logged `Temporary failure in name resolution`: systemd-resolved was flushing caches in a loop under the memory pressure. A symptom, not a cause. | source | outcome | |---|---| | shilla-icn | ok, 192 prices | | shopify-bog | ok, 2478 | | shopify-pty | ok, 2351 | | shopify-sal | ok, 793 | | shopify-yul | ok, 1049 | | the-loop-dublin-cork | ok, 730 | | **avolta-ath** (run 156) | **killed**, 200 prices banked, 2h20m in | | **extime-paris** (run 155) | **killed**, 1600 prices banked | | the other nine avolta airports | **never started** — the ten run in sequence in one process and ATH is page-one-only (`robots forbids query strings`) plus per-item rereads, so it held the process for the whole window | Both killed runs kept `status = running`, and at three hours old `backfill stuck_runs` declined them as too young: the approval gate would have refused every decision on `/review` until 19:08 on 19 Sep. Fixed in code rather than by hand — a run that began before the app container did is dead whatever the timer says (`ingest.process_boot`, CHANGELOG Unreleased) — and both rows are now `error: stuck: process ended without finishing`. **Nothing collected after 20:51, so the window is closed and staging may be deployed.** The rest of the sweep — avolta (ten airports) and extime-paris — needs a fresh window. Prices already banked by the killed runs stand: observations are timestamped, so a re-run adds to them and cannot double-count.