---
type: run
---

# Window W2, 19 Sep 2026 (staging, after deploy D3): the live collectors page drives one small run

Written by Stream AW4 (task AW4.5) before the window; **opened by the orchestrator after deploy D3,
never by a build agent** (`AWAY-2026-09-19.md` sections 1 and 4). One fast source, `shopify-yul`
(Montreal-Trudeau, `www.montrealdutyfree.ca`, CAD, the Shopify platform, text fetch), collects with
`--limit 30` while every page action of `/collectors#live` is pressed once: Start in both modes,
Pause, Resume, the pace, Stop, and the refusals a person can provoke. Nothing else is collected.
**Minutes, not hours**: a recheck of 30 held listings at a 3 s pace is about two minutes of
requests; the pause in the middle adds whatever it is held for.

Why Montreal: its robots.txt (read 19 Sep 2026 through the collector's reader) is `User-agent: *`,
`Allow: /`, no Crawl-delay, with only admin, cart, checkout, account and Shopify-internal paths
disallowed; `read_one` reads `/products/<handle>.json`, which is permitted. So the floor is our own
1 s, a pace of 3 s is honoured exactly and 6 s halves the rate, which is what the pace steps
measure. Its last run (19 Sep 02:08, ok) and its held listings with a URL (about 1,060 on the
19 Sep dump) make the recheck count exact: `expected_total` is 30, the page says "of 30".

## Two hands

**The page half is rian's.** The six routes and `GET /api/collectors/live` sit under
`sources.manage`, which no seeded level holds (`ACCOUNTS.md`, Permissions and levels): on staging
only the owner's session can press Start. The orchestrator cannot sign in as him and never asks
for a credential. So the orchestrator opens the window when rian says he is at a browser
(section 2 protocol: one Telegram line, the two-hour deadline), then runs the shell half below
beside him: the gate, the `psql` check after each step, the container log, the marker, the close.

**Without rian at a browser, only the shell lane runs** (the last section): it proves the control
plane inside the container (the run row's pid and heartbeat, the lock, the signal, the marker)
and leaves the page half for the first hour he is at a laptop. The window stays open until both
halves are done or rian says stop; no deploy while it is open.

Shell lines run from the main checkout as `rian`; `psql` is
`docker exec dutyfreeprofessor-db psql -U dfp -d dfp -tA -F' ' -c "<sql>"` (call it `Q` below).
The run's own log is the app container's (`docker logs dutyfreeprofessor-app --since 5m`): the
child a page Start spawns inherits uvicorn's stdio.

## The gate (before the first click)

```bash
cd /srv/apps/dutyfreeprofessor
curl -s https://dutyfreeprofessor.demoing.info/api/health                       # the D3 version
docker exec dutyfreeprofessor-app alembic current 2>/dev/null | grep -c aw4a1b2c3d4e   # 1
docker exec dutyfreeprofessor-app python -m app.cli status | grep -cE ' (running|stopping|pausing|paused|stalled|dead) '   # 0
docker exec dutyfreeprofessor-app sh -c 'for p in /proc/[0-9]*; do tr "\0" " " < $p/cmdline 2>/dev/null; echo; done' | grep -c 'app.cli collect'   # 0
test ! -e uploads/.collect-freeze && echo "marker absent"                       # D3 removed it after health
docker exec dutyfreeprofessor-app python -m app.cli status | grep -E '^ +shopify-yul '   # idle  ok (N prices)
srv-gw notify --from "AW4 W2" "W2 open: shopify-yul recheck 30 driven from /collectors#live; no deploy until closed (minutes)"
```

**Fail lines:** the marker present means D3's deploy did not reach its `rm -f` (read the deploy
log; remove it by hand only once nothing is deploying). `shopify-yul` reading `refused` means a
robots or edge refusal since the dump: the window does not open and nothing lifts it but rian's
recorded decision.

## The sequence (rian at the page, the orchestrator at the shell)

Each step: what rian does, what the page must show, the `Q` line, and what a miss means. The page
polls every 3 s while visible, so "within one poll" is under five seconds.

1. **Pace 3 s.** Expand `shopify-yul` (▸), type `3` in "Pace, seconds between requests", Set pace.
   Page: "pace set to 3 s"; the line reads "host asks nothing · floor 1 s · recommended 1 s · cap
   600 s · set by rian <time>".
   `Q "SELECT delay_seconds, delay_set_by FROM sources WHERE slug='shopify-yul'"` → `3.00 rian`.
   Miss: a `PACE_BELOW_FLOOR` here means `robots_crawl_delay` on the row is above 3 (a host that
   started publishing one); read the `pace` line and choose above the floor.
2. **Start, recheck, limit 30.** Open "Start ▾", choose "Recheck: read back the N held listings"
   (that radio posts `/mode` at once), limit `30`, Start. Page: "started, waiting for the run",
   then within one poll the row is `running` with "collecting since <time>: 0 prices, 1 requests"
   and the bar "0 of 30 checked".
   `Q "SELECT id, status, pid, mode, limit_n, expected_total, started_by, heartbeat_at IS NOT NULL FROM collection_runs WHERE source_id=(SELECT id FROM sources WHERE slug='shopify-yul') ORDER BY id DESC LIMIT 1"`
   → `<id> running <pid> recheck 30 30 rian t`. Note the id as `RUN`.
   Log: `docker logs dutyfreeprofessor-app --since 2m | grep -E 'collector_spawned|collector_locked'`
   shows one `collector_spawned source=shopify-yul mode=recheck limit=30 by=rian pid=<pid>`.
   Miss: `SOURCE_REFUSED` or `COLLECT_FROZEN` as the disabled button's title means the gate above
   was wrong; `NOTHING_TO_RECHECK` means the dump's listings lost their URLs (a data question, not
   this window's). A row that never appears: `docker logs` for the child's traceback.
3. **Percent and ETA after three units.** Wait for "3 of 30 checked" (about ten seconds). Page:
   "3 of 30 checked · about N s to go", the bar a tenth full (`aria-valuenow` 10 on the progress
   bar; a recheck prints its count, the percent drives the bar); no bar width and no ETA before
   the third unit; never a clock time.
   `Q "SELECT existing_checked + existing_missing, requests_made, heartbeat_at > now() - interval '15 seconds' FROM collection_runs WHERE id=RUN"`
   → `3 or more, 3 or more, t` (the heartbeat moves with every request).
   Miss: a percent at 1 or 2 units, or "100%", is a page bug (`percent_done`); a heartbeat older
   than 15 s while the counter moves is the hook not committing (`control.py` `_poll`).
4. **Start while running.** Press Start again (the menu is still there for a second tab; in this
   tab the row shows Pause and Stop, so use the second tab or the console line below).
   Console, on the page's own tab (same origin, the cookie rides):
   `fetch('/api/collectors/shopify-yul/start',{method:'POST',headers:{'Content-Type':'application/json'},body:'{}'}).then(r=>r.json()).then(console.log)`
   → `{detail: {error_code: "SOURCE_RUNNING", summary: "run RUN has been collecting since …; stop it first."}}`.
   `Q "SELECT count(*) FROM collection_runs WHERE source_id=(SELECT id FROM sources WHERE slug='shopify-yul') AND status='running'"` → `1`.
   Miss: a second `running` row means the lock let two children through (`ingest._try_lock`).
5. **Pause.** Press Pause. Page: `pausing` with "pausing: next request boundary in about N s", then
   within `CONTROL_POLL` (2 s) plus one poll, `paused` with "paused 12 s by rian · holds approvals
   while paused" (the seconds climb); the bar keeps its figures, no ETA; the buttons are Resume and Stop.
   `Q "SELECT control, control_set_by, control_set_at >= (SELECT started_at FROM collection_runs WHERE id=RUN) FROM sources WHERE slug='shopify-yul'"` → `pause rian t`.
   Then, twenty seconds apart, twice:
   `Q "SELECT requests_made, heartbeat_at FROM collection_runs WHERE id=RUN"`
   → `requests_made` the same both times, `heartbeat_at` newer the second time (a pause
   heartbeats every 2 s and is never silent).
   Miss: `requests_made` still climbing means the hook did not read the control (the `sources`
   row's `control_set_at` must be after `started_at`; `effective_control`); a heartbeat that does
   not move while paused is the pause loop not committing.
6. **The refusals a paused run lets you provoke** (the run is held, so there is time):
   - **pace 0.5 s.** The input refuses it in the browser (`min` is the floor), which is the
     design; the server's answer is the console line
     `fetch('/api/collectors/shopify-yul/pace',{method:'POST',headers:{'Content-Type':'application/json'},body:'{"delay_seconds":0.5}'}).then(r=>r.json()).then(console.log)`
     → `PACE_BELOW_FLOOR`, "0.5 s is under the floor: our own floor is 1 s."
     `Q "SELECT delay_seconds FROM sources WHERE slug='shopify-yul'"` → still `3.00`.
   - **Stop now, fresh.** The page never offers Stop now before a stop was asked for and 60 s
     passed (`STOP_GRACE`); the server's answer is
     `fetch('/api/collectors/shopify-yul/stop',{method:'POST',headers:{'Content-Type':'application/json'},body:'{"force":true}'}).then(r=>r.json()).then(console.log)`
     → `STOP_NOT_REQUESTED`, "ask it to stop first; Stop now is for a run that did not answer."
     `Q "SELECT status FROM collection_runs WHERE id=RUN"` → `running` (nothing was signalled).
   - **pace 6 s.** Type `6`, Set pace. Page: "pace set to 6 s".
     `Q "SELECT delay_seconds, delay_set_by FROM sources WHERE slug='shopify-yul'"` → `6.00 rian`.
   Miss: any of the three answering 202 or 200 is a route bug; the pace tests
   (`tests/test_collector_control.py` `TestPaceRefusal`, `TestPauseResumeStop`) pin the sentences.
7. **Resume, and the rate halves.** Press Resume. Page: `running` within one poll, the counter
   moving again. Then two readings 30 s apart:
   `Q "SELECT requests_made, existing_checked + existing_missing FROM collection_runs WHERE id=RUN"`
   → at 6 s about 4 to 5 requests in 30 s, where step 3's 3 s pace gave about 7 to 10 (each
   request is the pace plus the fetch itself). The `pace` line on the page reads 6 s.
   `Q "SELECT control FROM sources WHERE slug='shopify-yul'"` → `run`.
   Miss: the rate unchanged means the hook read a stale `delay_seconds` (the identity-mapped
   `Source` instead of the column SELECT); a rate faster than the pace is a request storm and the
   window closes with Stop at once.
8. **Stop.** Press Stop. Page: `stopping` with "stopping by rian N s ago; waits for the request in
   flight", then within one poll the row leaves the live group and reads `idle` with "last:
   stopped <time>, N prices in M s"; Stop now never appeared (the run answered inside the grace).
   `Q "SELECT status, stopped_by, finished_at IS NOT NULL, prices_written, existing_checked, existing_missing FROM collection_runs WHERE id=RUN"`
   → `stopped rian t <N> <N> <0 or more>`: the prices written before the stop are kept (the batch
   commits at every boundary and once more at the stop).
   `Q "SELECT count(*) FROM price_observations WHERE run_id=RUN"` → the same `N` as `prices_written`.
   Miss: `error` with "batch lost at stop" means the final commit failed and the row alone was
   written; `running` after a minute is the hook not polling (`docker logs` for the child).
9. **Two Starts within a second.** The lock, not the route, is the guard. From the shell, both in
   the same second (the page's button is disabled while one click is in flight, so a double click
   from one tab cannot do this; two tabs can, with the same outcome):
   ```bash
   docker exec dutyfreeprofessor-app python -m app.cli collect --source shopify-yul --mode recheck --limit 5 --by w2-orchestrator > /tmp/w2-a.log 2>&1 &
   docker exec dutyfreeprofessor-app python -m app.cli collect --source shopify-yul --mode recheck --limit 5 --by w2-orchestrator > /tmp/w2-b.log 2>&1 &
   wait; grep -h 'collector_locked\|locked (' /tmp/w2-a.log /tmp/w2-b.log
   ```
   → exactly one `collector_locked source=shopify-yul (another process holds the collector lock)`
   and one `shopify-yul: locked (…)` (that process exits 3); the other prints
   `shopify-yul: stopped …` or `ok …` when it ends.
   `Q "SELECT count(*) FROM collection_runs WHERE source_id=(SELECT id FROM sources WHERE slug='shopify-yul') AND started_at > now() - interval '2 minutes'"` → `1`.
   Page: one run came and went in the live group; the 24 h table shows it with `started_by`
   `w2-orchestrator`.
   Miss: two rows, or no `collector_locked` line, means `pg_try_advisory_lock` did not hold (the
   lock connection is not autocommit, or the key differs between the two).
10. **Discover, limit 30.** In "Start ▾" choose "Look for new", limit `30`, Start. Page: `running`,
    "N of about 30" (an estimate: the newest ok unlimited run's `products_seen` capped by the
    limit), the percent never above 99 while running, then `idle` with "last: ok <time>, 30 prices
    in M s"; expanded, the last run's "existing missing" prints **not measured** (a limited run
    measures nothing).
    `Q "SELECT status, mode, limit_n, expected_total, products_seen, existing_missing, new_found FROM collection_runs WHERE source_id=(SELECT id FROM sources WHERE slug='shopify-yul') ORDER BY id DESC LIMIT 1"`
    → `ok discover 30 30 30 0 <0 or more>`.
    Miss: `existing_missing` above 0 on a limited run is D5 broken (only an unlimited ok discover
    measures it); "100%" while `running` is `PERCENT_CAP_RUNNING` not applied.
11. **The freeze marker.** With nothing running:
    ```bash
    printf 'W2 test %s by %s\n' "$(date -u +%FT%TZ)" "$USER" > uploads/.collect-freeze
    ```
    Page, within one poll: the banner "Collection is frozen for a deploy: W2 test <stamp>" and
    every Start button disabled, its title the summary "collection is frozen for a deploy: W2 test
    … (0 min ago)". Shell: `docker exec dutyfreeprofessor-app python -m app.cli collect --source shopify-yul --limit 1; echo "exit $?"`
    → `collect_frozen marker=/srv/uploads/.collect-freeze stamp='W2 test …'` and `exit 3`, and
    `Q "SELECT count(*) FROM collection_runs WHERE started_at > now() - interval '1 minute'"` → `0`.
    Then `rm -f uploads/.collect-freeze`; the banner is gone within one poll and Start is back.
    Miss: no banner means the container reads another path than the host wrote (`uploads_dir`,
    the compose bind mount); a run row means the CLI skipped the marker.
12. **The audit trail, and the source left tidy.** Set the Start mode back to "Look for new" (the
    sweep's default) and leave the pace at 3 s (slower than before; the page names who set it):
    type `3`, Set pace.
    `Q "SELECT to_char(at,'HH24:MI:SS'), action, detail::text FROM audit_log WHERE entity_type='source' AND entity_key='shopify-yul' AND at > now() - interval '2 hours' ORDER BY at"`
    → in order: `source.pace {from 1, to 3, floor 1}`, `source.mode {to recheck}`, `source.start
    {mode recheck, limit 30, pid}`, `source.pause`, `source.pace {to 6}`, `source.resume`,
    `source.stop {run_id}`, `source.mode {to discover}`, `source.start {mode discover, limit 30}`,
    `source.pace {to 3}`; none for the two console refusals, none for the shell starts, none for
    the pause that changed nothing.
    Page: the five most recent of those under the row's events.

## Close

```bash
docker exec dutyfreeprofessor-app python -m app.cli status | grep -cE ' (running|stopping|pausing|paused|stalled|dead) '   # 0
test ! -e uploads/.collect-freeze && echo "marker absent"
srv-gw notify --from "AW4 W2" "W2 closed: <12 or N of 12> steps passed on shopify-yul; staging may deploy again"
```

Append the closing section below (which steps passed, the run ids, the figures step 3, 7 and 8
printed, anything that missed and what it points at) to **the main checkout's copy of this file**,
and fold it into the worktree with the next stream's commit, as W1's was. Until then no stream
commits a change to this file, or the fast-forward at the next deploy point fails.

## The shell lane alone (no browser signed in as the owner)

The page half waits; this proves the container. Same gate, same announce, then:

```bash
docker exec dutyfreeprofessor-app python -m app.cli collect --source shopify-yul --mode recheck --limit 30 --by w2-orchestrator > .logs/runs/w2-shell-2026-09-19.log 2>&1 &
sleep 8; docker exec dutyfreeprofessor-db psql -U dfp -d dfp -tA -F' ' -c "SELECT id, status, pid, mode, limit_n, expected_total, started_by, requests_made, heartbeat_at > now() - interval '15 seconds' FROM collection_runs WHERE source_id=(SELECT id FROM sources WHERE slug='shopify-yul') ORDER BY id DESC LIMIT 1"
# → <id> running <pid> recheck 30 30 w2-orchestrator <2 or more> t ; status prints `running` for the row
docker exec dutyfreeprofessor-app python -m app.cli status | grep -E '^ +shopify-yul +running'
# step 9 exactly as above (the lock), then the signal, which is what guard.sh's shed sends:
PID=$(docker exec dutyfreeprofessor-db psql -U dfp -d dfp -tAc "SELECT pid FROM collection_runs WHERE status='running' AND source_id=(SELECT id FROM sources WHERE slug='shopify-yul')")
docker exec dutyfreeprofessor-app python -c "import os,signal; os.kill($PID, signal.SIGTERM)"
sleep 6; docker exec dutyfreeprofessor-db psql -U dfp -d dfp -tA -F' ' -c "SELECT status, stopped_by, prices_written, existing_checked FROM collection_runs WHERE pid=$PID ORDER BY id DESC LIMIT 1"
# → stopped signal <N> <N>: the batch committed, the row ended by the flag at the next poll
# step 11 (the marker) exactly as above; then close.
```

What this lane cannot show, and the page lane must: `pausing` to `paused` with the heartbeat
moving, the pace change landing mid-run, `SOURCE_RUNNING`, `PACE_BELOW_FLOOR` and
`STOP_NOT_REQUESTED` from the routes, `stopped_by` as a username, the audit rows, the banner.
Every one of those is pinned by `tests/test_collector_control.py` on SQLite; the window is the
proof on Postgres in the container, which is why it waits for rian rather than being skipped.

The closing section below is the orchestrator's to write.

## Closed 2026-09-20 02:46 UTC (19 Sep 19:46 local): the shell lane alone

Run by the orchestrator under **the shell lane alone** (no browser signed in as the owner),
against staging on `0.52.0`, migration head `aw4a1b2c3d4e`. **The page half was not run**: steps
1 to 8, 10 and 12 need the owner's session and are still owed. What this lane proved is the
control plane inside the container — the run row's pid and heartbeat, the advisory lock, the
signal, and the freeze marker.

**One standing deviation, and everything that followed from it.** The orchestrator capped the
collect at `--limit 5` where the lane says 30. A 5-listing recheck at this source's 1 s pace is
about nine seconds, not two minutes, so the main run ended on its own before the signal step.
See D1 to D3 under Deviations.

### Per step

| # | Step | Expected | Observed | |
|---|---|---|---|---|
| G1 | gate: health | the D3 version | `{"status":"ok","version":"0.52.0","database":"ok"}` | pass |
| G2 | gate: migration head | `1` | `1` (`alembic current` → `aw4a1b2c3d4e (head)`) | pass |
| G3 | gate: nothing live | `0` | `0` | pass |
| G4 | gate: no collect process | `0` | `0` | pass |
| G5 | gate: marker absent | `marker absent` | `marker absent` | pass |
| G6 | gate: the source reads idle | `idle  ok (N prices)` | `shopify-yul  idle  ok (1049 prices)` | pass |
| S1 | the run row, 8 s in | `<id> running <pid> recheck 30 30 w2-orchestrator <2 or more> t` | `183 running 66 recheck 5 5 w2-orchestrator 5 t` (5 and 5 are the capped limit) | pass |
| S2 | `app.cli status` names it running | a line matching `^ +shopify-yul +running` | `shopify-yul  running  running (1 prices)` | pass |
| S3 | step 9, the lock | exactly one `collector_locked`, one `shopify-yul: locked (…)` exiting 3, the other `ok …`; one run row in 2 min | one `collector_locked source=shopify-yul (another process holds the collector lock)`; `shopify-yul: locked (shopify-yul: another collector process holds the lock)`, `Exit 3`; the other `shopify-yul: ok prices=5`; count `1` | pass |
| S4 | the signal | `stopped signal <N> <N>` | `stopped signal 0 0` — status and `stopped_by` exact; N is 0, see D3 | pass, with D3 |
| S5 | step 11, the freeze marker | `collect_frozen marker=/srv/uploads/.collect-freeze stamp='W2 test …'`, `exit 3`, count `0` | `collect_frozen marker=/srv/uploads/.collect-freeze stamp='W2 test 2026-09-20T02:45:16Z by rian' (use --ignore-freeze to override)`, `exit 3`, count `0`; marker removed, `marker absent` | pass |
| C1 | close: nothing live | `0` | `0` (and `0` for a plain `' running '` grep) | pass |
| C2 | close: no collect process | `0` | `0` | pass |
| C3 | close: marker absent | `marker absent` | `marker absent` | pass |

**12 of 12 shell-lane checks passed; none failed.** No step was skipped for want of the page
beyond the page half itself (steps 1 to 8, 10 and 12), which is rian's.

### The runs

Three run rows, all `shopify-yul`, all `recheck`, all `--limit 5 --by w2-orchestrator`:

```
id  status   pid mode    limit expected by               stopped_by reqs prices checked missing started  finished
183 ok       66  recheck 5     5        w2-orchestrator  -          5    5      5       0       02:41:32 02:41:39
184 ok       78  recheck 5     5        w2-orchestrator  -          5    5      5       0       02:43:32 02:43:39
185 stopped  90  recheck 5     5        w2-orchestrator  signal     1    0      0       0       02:44:00 02:44:02
```

183 is the lane's main run. 184 is step 9's winner (its twin took the lock refusal and wrote no
row). 185 is the signal target (D2). Ten price observations were written to staging, by 183 and
184; that is what a recheck does and nothing else was touched.

The run row sampled as 183 went (the lane asks only for the 8 s reading; the others are
read-only extra observation, D1):

```
T~2s  183 running 66 recheck 5 5 w2-orchestrator 0 <blank>
T~5s  183 running 66 recheck 5 5 w2-orchestrator 2 t
T~8s  183 running 66 recheck 5 5 w2-orchestrator 5 t
```

`heartbeat_at` is NULL until the first request boundary, so the predicate reads blank rather than
`t` if it is asked before the first request. The lane's own reading, at 8 s, is `t`.

The signal, in full (`/tmp/w2-sig.log`):

```
2026-09-20 02:44:01,056 WARNING collector_signalled source=shopify-yul signal=15
2026-09-20 02:44:02,056 WARNING collector_stopped source=shopify-yul reason=signal by=signal
2026-09-20 02:44:02,096 INFO collector_finished source=shopify-yul status=stopped prices=0
shopify-yul: stopped prices=0 error=-
```

One second between the signal and the stop: the handler set the flag and the hook raised at the
next poll, never mid-flush, exactly as `control.py` says.

### Where the source is left

```
slug         mode      delay_seconds delay_set_by control control_set_by enabled robots_crawl_delay robots_read_at
shopify-yul  discover  1.00          (null)       run     (null)         t       (null)             2026-09-20 02:44:01Z
```

- **Pace 1.00 s, set by nobody** — untouched. The pace steps (1, 6, 7) are page actions; this lane
  never set one, so the pace is as the window found it, not the 3 s step 12 would leave.
- **Mode `discover`** — the sweep's default, untouched. The CLI's `--mode recheck` is a per-run
  argument and never writes the column, so step 12's "set the Start mode back" has nothing to undo.
- **Control `run`, enabled, not refused.** `app.cli status` now reads `shopify-yul  idle  stopped
  (0 prices)` rather than `idle  ok (1049 prices)`: the line names the newest run, which is the
  signalled one. Cosmetic, and the next ok run restores it — a `stopped` last run refuses no
  start (only `blocked` does, `start_refusal`).
- **`robots_read_at` is now stamped and `robots_crawl_delay` is still NULL** — corroborating this
  window's premise: Montreal publishes no Crawl-delay, so the floor is our own 1 s.
- **No audit rows.** `audit_log` for `entity_type='source', entity_key='shopify-yul'` over the last
  two hours is empty, as the lane says it should be: a shell start is not a page action.

### Deviations, all forced by the limit cap

- **D1 — `--limit 5`, not 30.** The orchestrator's standing boundary. The run is then about nine
  seconds, so the row was also read at ~2 s and ~5 s, not only at the lane's `sleep 8`. Read-only
  observation; nothing extra was collected.
- **D2 — one further run, to give the signal step a live target.** With the cap, the main run
  (183) had finished before the signal step, and step 9's pair (184 and its locked twin) with it,
  so `SELECT pid … WHERE status='running'` had nothing to return. Rather than improvise, the
  lane's own main-run command was run once more, unchanged (`--source shopify-yul --mode recheck
  --limit 5 --by w2-orchestrator`), and signalled: that is run 185. Same source, same limit, same
  command the lane names.
- **D3 — the signal landed during startup, so N is 0.** A collector spends about three seconds
  starting (imports, the DB connection, robots.txt) before its first request, and the signal was
  sent three seconds in. So 185 stopped at `requests_made` 1 with nothing yet written, and the
  reading is `stopped signal 0 0`. The shape and `stopped_by` are exactly right and the
  cooperative path is proven end to end; what is **not** proven here is the other half of the
  lane's comment — *"the batch committed"* — because there was no batch. **Page-lane step 8 still
  owes that**: prices written before a Stop being kept is checked there, against
  `prices_written` and `count(*) FROM price_observations WHERE run_id=RUN`.

### Two notes for whoever runs the page half

- **The container log will not show these runs.** `docker logs dutyfreeprofessor-app --since 3m |
  grep collector_signalled` printed nothing. A collector started by `docker exec` owns its own
  stdio (it went to the log files named above); only a child the page spawns inherits uvicorn's,
  which is what the lane's log line at step 2 is for. Not a fault — but do not read an empty
  container log as a missing run when the start came from a shell.
- **The lane's own ordering would make step 9 read differently from the page lane's.** At the
  lane's `--limit 30` the main run is still holding the lock when step 9 fires, so *both* of its
  processes would print `collector_locked` and the "exactly one … the other prints ok" wording
  would not hold. It read literally here only because the capped run had already finished. Worth
  a word in the file before the next window reuses this block at full limit.

### The gate

Open again: 0 sources in a live state, 0 `app.cli collect` processes in the container, no freeze
marker, health `ok` on `0.52.0`. **Staging may be deployed again** as far as this lane is
concerned — but the window itself stays open until the page half is done or rian says stop.

## Page half, part A (the orchestrator through rian's signed-in browser, 19 Sep 19:47 to 19:55 PDT)

Steps 1 to 3 passed on `shopify-yul`: pace 3 s set by rian (row `3.00 rian`, event "to 3, from 1,
floor 1"); Start recheck limit 30 from the page (run 186, pid 149, `collector_spawned … by=rian`);
the row read `running` with "7 of 30 checked · about 1 min to go", pid 149, 8 requests, RSS 144 MiB.

**Then a real finding.** Request nine of the recheck read
`https://www.montrealdutyfree.ca/products/rouge-clair-by-michel-chapoutier-750ml.json`, which
answers HTTP 404 (the product is gone from the shop). `fetch()` raises `SourceBlocked` for every
HTTP error, so run 186 ended `blocked` after 8 prices and the source reads `refused` on the page
with Start disabled. A vanished product is `existing_missing`, not a refusal by the host. A fix is
in progress on the branch (404 and 410 become "not found", counted as missing in a recheck, skipped
in a walk, `error` never `blocked` if it escapes) with an idempotent `backfill false_refusals` that
reclassifies run 186; both ride the next deploy. Steps 4 to 8 and 10 on Montreal wait for that.

A discover run on `shopify-sal` (pace 3 s set by rian, limit 30) finished in one request and 1 s
(a Shopify feed is one request per 250 products), so it cannot be paused: the pace and the audit
rows landed, the run is `ok 30 prices`.

**Substitute for the long-run steps, announced here before the first request:** `avolta-lgw`
(London Gatwick, `london-gatwick.worlddutyfree.com`, robots `Crawl-delay: 30`, page-one-only),
discover, limit 30: about four category pages at 30 s each. Steps 4 to 8 run on it with the pace
steps adapted to its floor (3 s must answer `PACE_BELOW_FLOOR`; the "rate halves" reading is 30 s
against 60 s). Nothing else is collected.

## Page half, part A: results (closed 19 Sep 20:00 PDT; the orchestrator through rian's signed-in browser)

| step | source | expected | observed | |
|---|---|---|---|---|
| 1 pace | shopify-yul | "pace set to 3 s", row `3.00 rian` | as expected; event "to 3, from 1, floor 1" | pass |
| 2 start recheck 30 | shopify-yul | `running` within a poll; run row `recheck 30 30 rian t`; `collector_spawned` | run 186, pid 149, "collecting since …: 0 prices, 1 requests" then "7 of 30 checked · about 1 min to go" | pass |
| 3 percent and ETA | shopify-yul | after three units | 7 of 30, ETA shown, pid 149, 8 requests, RSS 144 MiB | pass |
| finding | shopify-yul | | request 9 read a vanished product's `.json` (HTTP 404); `fetch()` raised `SourceBlocked`; run 186 `blocked` after 8 prices; the source reads `refused`, Start disabled | **bug, fix on the branch** |
| 10 discover 30 | shopify-sal | `ok discover 30 30 30 0` | run 187: one request, 1 s, 30 prices; row `ok discover 30 30 30 0 0`; a Shopify feed cannot be paused | pass |
| 2 start discover 30 | avolta-lgw | `running`, "N of about 30" | run 188, pid 153, "0 of about 30"; robots read 30 s, floor 30 recorded at the run's start | pass |
| 4 SOURCE_RUNNING | avolta-lgw | 409, "run 188 has been collecting since …; stop it first." | exactly that | pass |
| 6 PACE_BELOW_FLOOR | avolta-lgw | 3 s refused | 409, "3 s is under the floor: the host asks for at least 30 s (robots.txt read 20 Sep 02:54)." | pass |
| 6 STOP_NOT_REQUESTED | avolta-lgw | force stop refused | 409, "ask it to stop first; Stop now is for a run that did not answer." | pass |
| 5 pause | avolta-lgw | `paused … by rian · holds approvals while paused`; control `pause rian t`; requests still, heartbeat moving | paused within 7 s; `pause | rian | t`; requests 3 and 3, heartbeat 02:55:13 then 02:55:34 | pass |
| 6 pace above floor | avolta-lgw | "pace set to 60 s", `60.00 rian` | as expected, while paused | pass |
| 7 resume, rate | avolta-lgw | `running` within a poll; fewer requests per minute | running in 6 s with the in-flight batch (8 prices, 4 requests); 4 requests still 50 s later (pace 60), 26 %, ETA 461 s; control `run` | pass |
| 8 stop | avolta-lgw | `stopped rian t <N> <N>`; observations = prices | idle within 4 s, "last: stopped 20 Sep 02:54, 8 prices in 2 min"; row `stopped | rian | t | 8 | 8 | 0 | 0 | 4 | 8`; 8 observations. Closes the shell lane's open point (a non-zero batch committed at stop) | pass |
| 11 freeze marker | all | banner and every Start disabled with the stamp; gone after `rm` | API `COLLECT_FROZEN` at once; the page drew the banner and disabled 26 Starts at its next poll (the tab counts as hidden in the automation window, so 30 s cadence); gone within one poll after removal | pass |
| 12 audit and tidy | all | the rows in order, none for refusals | ten rows in order (pace, mode, start, pace, start, start, pause, pace, resume, stop); none for the three refusals or the shell starts. Left: yul pace 3 s mode discover; sal pace 3 s; lgw pace 30 s (the host's ask) | pass |

Step 9 (two Starts in a second) was proven by the shell lane. **Still owed on Montreal after the fix and the backfill ride the next deploy:** steps 2 to 8 in recheck mode, so the recheck counters ("N of 30 checked", existing checked and missing) are seen on the page with a vanished product counted as missing.

Two smaller notes for the collectors' owner. Until a source has run once under the control plane, its row says "host asks nothing · floor 1 s" even when its robots publishes a crawl delay (Gatwick read floor 1 s before run 188 and 30 s after: `robots_crawl_delay` is written by the loop). The floor the loop honours is right either way; the page's floor is stale before the first run. And the accessibility tree the automation reads did not carry the freeze banner although the DOM did; not a page defect.

Runs: 186 (yul recheck, blocked by the finding), 187 (sal discover, ok), 188 (lgw discover, stopped by rian). Gate at close: 0 live, 0 collect processes, marker absent.

## Page half, part B: Montreal's recheck after the fix (closed 19 Sep 22:22 PDT)

After deploy D4 (the branch's `6ce158b`: HTTP 404 and 410 are "not found", never a refusal;
`backfill false_refusals` moved run 186 from `blocked` to `error`, so the source read `idle`,
Start enabled). From the page: Recheck, limit 30, Start (run 189, started by rian). The row read
`running`, "6 of 30 checked · about 1 min to go" at 12 requests with existing missing already 5:
the same held listings that refused the whole source the evening before now count as missing and
the walk continues. It ended on its own: run 189 `ok`, recheck, 30 requests, 25 checked, 0
changed, **5 missing**, 25 prices, 25 observations; the source reads `idle ok (25 prices)`.
Gate at close: 0 live, 0 collect processes, marker absent. W2 is complete: every step of the plan
has its proof on Postgres in the container.

One observation for the collectors' owner: while run 189 ran, the collapsed row's summary line
read "prices 0 · checked 0 · pace 3 s · 143 MiB" although the state detail beside it said "6
prices, 12 requests" and the bar "6 of 30 checked"; the two figures on that line appear to be the
previous run's, not the live one's.