---
description: "Stream AW1: three new collectors for the away run of 19 Sep (a scout that reads robots and one listing per candidate host inside its own announced window, three sources added as configuration on the Avolta, Shopify or ARI platform with tests on the real tiles, every new airport hidden, and window W1 with its scripts and commands handed to the orchestrator)"
---

You are running **Stream AW1** of the Duty Free Professor away run of 19 to 20 September 2026, as
an autonomous session or as the orchestrator's workflow of subagents. The design is the brief; do
not redesign it. Where the brief is silent, choose the option that touches fewer files and says
less to a retailer, and write the choice in your handoff.

**Start by checking your ground.** `pwd` must be
`/srv/apps/dutyfreeprofessor/.claude/worktrees/collectors-live-page-plan-8951e2` and `git branch
--show-current` must print `claude/collectors-live-page-plan-8951e2`; `git log --oneline -15` must
include `05941b4` or later. `.venv-dev` and `main/web/node_modules` are symlinks there. If any of
these is not so, stop and say so. Do not `cd` to the main checkout; it is dirty on purpose.

Read first, in this order: your brief `.logs/planning/streams/AW1-collectors.md`; the away plan
`.logs/planning/streams/AWAY-2026-09-19.md` (sections 1, 2, 4 and 7 bind you);
`.logs/planning/streams/OVERNIGHT-RULES.md`; `agents.md` (the whole of "Legal and collection
posture"); `brief.md`; then the docs the tasks need: `main/docs/COLLECTORS.md` (the robots policy
and the posture table), `main/docs/VOCABULARY.md`, and the Collecting section of
`main/docs/RUNBOOK.md`. Then the code the brief names, thoroughly: `main/app/services/collectors/`
(`registry.py`, `base.py`, `avolta.py`, `shopify.py`, `ari.py`, `robots.py`, `fetch.py`),
`services/ingest.py` (`ensure_retailer_and_shops`, `run_collector`), `services/collector_view.py`,
`services/urls.py`, `web/src/lib/countries.ts`, and the tests that pin them
(`tests/test_collectors_avolta.py`, `test_robots.py`, `test_parser_version.py`, `test_read_one.py`,
`test_collector_view.py`, `test_beauty_widening.py`). Read the last three entries of
`.logs/handoff.md`, not the whole file. Read `.logs/runs/window-2026-09-19b-sweep.md` and the
three scripts in `.logs/runs/sweep-2026-09-19b/`.

**Register the tasks on `/plan` first**, one per task, then work them in order, AW1.1 to AW1.5:

```
python3 main/scripts/plan-set.py AW1.1 todo "Stream AW1" --add --stream AW1 --stream-title "Three new collectors" --title "The scout: enumerate, read robots, one listing read each, rank, choose three"
python3 main/scripts/plan-set.py AW1.2 todo "Stream AW1" --add --stream AW1 --title "First chosen source: configuration, registry, tests on the real tiles"
python3 main/scripts/plan-set.py AW1.3 todo "Stream AW1" --add --stream AW1 --title "Second chosen source: configuration, registry, tests on the real tiles"
python3 main/scripts/plan-set.py AW1.4 todo "Stream AW1" --add --stream AW1 --title "Third chosen source: configuration, registry, tests on the real tiles"
python3 main/scripts/plan-set.py AW1.5 todo "Stream AW1" --add --stream AW1 --title "Window W1 for the orchestrator, and what to read afterwards"
```

Mark each `doing` when you start it (`python3 main/scripts/plan-set.py AW1.2 doing "Stream AW1"`)
and `done` with a note saying what proves it. Each task ends with `main/check.sh` printing
"checks passed" and one commit prefixed `AW1:` with a subject line and a body that says what it
prevents or what it cost. AW1.1 also commits the scout window file and the brief's filled design
section. A bug you fix becomes a test in `main/tests/` written as the real record that failed,
with what it cost in the docstring. Rehearse anything that reads a database on `dfp-devdb`
(127.0.0.1:5433) from a fresh copy of the newest dump in `backups/`; never on staging.

**Network, exactly this and no more.** AW1.1 may read `robots.txt`, the discovery page and one
listing page per candidate host, each after the host's published crawl delay, only inside
`.logs/runs/window-2026-09-19c-scout.md`, which you write and announce before the first request
and close with the per-host table. A host that refuses at robots (401 or 403) or on content is
REFUSED, recorded, never asked again. AW1.2 to AW1.5 make no request beyond one `robots.txt` read
per host to test code. Window W1 is the orchestrator's to launch after deploy D1; you write its
file, its scripts and its commands, and you never run `app.cli collect` yourself.

**Hard boundaries.** Never deploy; never write to staging's or production's database; never run a
collection; never log in to a retailer, accept terms, or change `identity_mode`; never contact a
host `sweep plan` lists as refused, nor Dubai, nor Singapore; never create a source, retailer,
shop or place row by hand (the collector's first run creates them, `backfill places` follows);
never show a new airport (`Shop.visible` stays false); never edit `services/identity.py`,
`keying.py`, `normalize.py`, `merges.py`, `ingest.py`, `collectors/{base,fetch,robots,targets}.py`,
the parsing functions of `avolta.py` or `shopify.py`, `main/docs/REVIEW-PROCESS.md`, migrations,
`docker-compose.yml`, `.env`, `.app.env`, `main/check.sh`, `.logs/runs/sweep-2026-09-19b/`,
`.logs/runs/uptime.log` or `notes/`; never `git push` or rewrite history. Client-facing copy has no
em dashes and never the words "cheap" or "free"; one word per concept (`VOCABULARY.md`).

Tell rian as you go with `srv-gw notify --from "AW1" "<one line>"`: on start ("AW1 starting: 5
tasks, three new collectors"), when the scout window opens and closes, on each task done with the
count ("AW1.2 done (2/5): <slug> configured and tested"), on blocked, and at hand-back ("AW1 done:
5/5, check green, handoff written; D1 then W1 may start"). Never a secret, never a URL with a
token. If the command fails, note it in the handoff and carry on.

When you hit a decision only rian can make, do not stop: add it to the running list
(`python3 main/scripts/items.py add --kind decide --by "Stream AW1" --blocks AW1.<n> --weight costly
--title ... --detail ... --option ... --assumption ...`) with the options, the assumption you are
proceeding under and what you would redo, then carry on with everything that does not depend on it.
The brief's four questions already carry their assumptions.

If a task needs more than about three hours, split it, record the split with a note, and keep the
part that lands. When the brief is done, prepend the handoff entry to `.logs/handoff.md` (at most
25 lines: the three sources with their slugs, delays and expected W1 duration; the W1 commands by
reference to the window file; the after-run checks; the four questions with their assumptions;
"green and ready") and stop. The orchestrator deploys D1 and launches W1; you do not.

```
/goal Stream AW1 is complete: every task AW1.1 to AW1.5 is done on /plan, or blocked with a note saying why; the scout window .logs/runs/window-2026-09-19c-scout.md is announced and closed with its per-host table and the brief's design section holds the ranking; three sources are registered as configuration with tests on the tiles the scout fetched; the W1 window file, its adapted admit-queue, guard and finish scripts, and the exact launch and after-run commands exist under .logs/runs/; main/check.sh has printed "checks passed" on the final tree; every finished task has its own commit prefixed "AW1:"; nothing was deployed, no database but a restored copy was written, no collection was run, no new airport was shown. Or stop after 120 turns.
```