---
type: plan
version: 3
status: decisions made 2026-09-04 (§9); reviewed adversarially the same day (16 agents, 69 findings kept); corrections applied; §10 lists what rian still decides; kickoffs and briefs to be rewritten next
supersedes: v2 (git ce2b3f7..abc0006)
---
# Soft-launch build plan v3: parallel sessions, with quality, viability and documentation as first-class work

Accepted scope: $8,400 (The Ambitious Version + three new collectors). Delivery **Fri 18 Sep**.
Buffer Sat 19 – Fri 26. **Cannes opens Sun 27 Sep.** Written **Fri 4 Sep** (v2 mislabelled it
Thursday; every wave date below is corrected). Nine working days remain: Mon 7 – Fri 11,
Mon 14 – Fri 18, of which Wave 4 takes the last two and a half as one session.

This is the third pass. v2 was written against a six-lens read-only map of the codebase and
then reviewed by eight adversarial lenses with refutation. What survived is folded in below;
what still needs rian is in §10. Every claim of "verified" here was checked against code or
the database on 2026-09-04.

## 1. Where we actually are (measured, not remembered)

| Stream | v1 said | Now |
|---|---|---|
| Step zero: git | not done | **Done** 3 Sep, secrets excluded, single alembic head `b0c1d2e3f4a5` |
| S. Structure proposal | Mon 7 | **Done** as document + hidden `/structure` page. **Not yet sent to Mark: owner action.** The page is the live copy; the .md is superseded. |
| C. Design | pass due Mon 7 | **Pass one done** (logo, #003F6C palette, stand-in serif, v0.30.0). Mega menu, prominence: not started. |
| A. Collectors | Mon 7 → | **Paris built and fully collected** (7,032 listings, 5h09m, clean). Sixteen not activated. Beauty exists only at Paris (3,889) and Athens (20, uncategorised). |
| A2. Rendered fetch | (new) | **Starting now**, separate session: Playwright sidecar, then Singapore, then one Shilla probe. |
| B. Pages | Mon 7 → | Not started. Body served to any non-JS client is still `<div id="root"></div>`. |
| D. Content | Mon 14 → | Not started. |
| E. Infrastructure | now → | Nothing provisioned. |
| F. Awards | Mon 14 → | Not started (a card-corner "top award" exists). |

Of sixteen accepted quote lines: **one delivered, two partial, thirteen have no code.**

**Facts that shape everything:** Paris was publicly visible on the demo by a model default
until hidden today (Decision 1). All 13,140 products carry `vertical='liquor'`. Every launch
price except Paris was last observed 21–25 Aug. Dubai (39% of launch products, 77% of barcodes)
refuses our declared User-Agent (Akamai 403). **Seoul (Shilla) challenges every product page**
(Cloudflare, verified today) and **Singapore is client-rendered with a token-gated API**
(verified today); both need a real browser. **Four Avolta stores forbid query-string pagination**
(`Disallow: /*?` at LHR, ATH, MAD, BCN; verified today), so category discovery there is page 1
only and the current collector has been fetching pages the hosts forbid. **The robots matcher
cannot hear a refusal aimed at us by name** (it keys on the first UA token, "mozilla"; verified).
**The database had no backup since 24 Aug** until today's dump; the container auto-migrates on
every start.

## 2. What is shared, and how it is controlled

One working tree, one database with one migration chain, one deploy that kills a running
collection. Three tokens, granted by rian, one holder at a time:

| Token | Rule |
|---|---|
| **Deploy** | Sessions build and test locally, never deploy. Rian deploys at two fixed checkpoints (midday, evening). `main/check.sh` must pass first. A deploy recreates the container and **kills any running collection**: check `.logs/runs/` and `pgrep -f app.cli` first. **Before any deploy that carries a migration, rian runs** `docker exec dutyfreeprofessor-db pg_dump -U dfp -Fc dfp > backups/dfp-<date>-<reason>.dump`. |
| **Migration** | `alembic heads` shows one head before writing. **Migrations are schema-only** (create tables, add nullable columns or columns with `server_default`); **every backfill is an idempotent `app.cli backfill <name>` or `rederive` run after deploy**, never Python inside Alembic. Ordered holders below. |
| **Crawl** | Long runs announce a window in `.logs/runs/`; no deploy inside it. Overnight is the natural home. |

**Migration order (one head, serialized):**

| # | Owner | When | Tables / columns |
|---|---|---|---|
| 1 | A (or rian) | Sat 6 evening deploy, **before any recollection and before A2's first real SIN ingest** | `accounts(id, email NULL, display_name, created_at)` seeded with rian; `raw_records(listing_id, run_id, payload JSONB, parser_version)`; `price_observations.source_kind` (`server_default 'collector'`); `sources.identity_mode` + `permission_record` (nullable); `products.vertical` populated by collector; `Location.visible` default False. Plus `RawListing.raw` and `RawListing.vertical` (dataclass, no migration). |
| 2 | Q | Sun 7 | `verification_runs`, `verification_checks(listing_id, observation_id, …)`, `audit_snapshots`, `rejected_observations(stage collector\|ingest, reason, source_sku, url, payload, run_id)`; split `CollectionRun.skipped_no_price` into per-reason counters. |
| 3 | A | Wave 2 | `brands(id, slug UNIQUE, name, canonical_id NULL)` + `products.brand_id`; `size_value`, `size_unit`; per-vertical attribute column; `identity_rules_version`; `products.merged_into_id`; `merge_candidates`; `product_merges(from_id, to_id, by, at, detail JSONB)`; `reverifications`. |
| 4 | R | Wave 2 start | `identities`, `sessions`, `audit_log`, `overrides` (see §4). |
| 5 | D | Wave 2 | `articles`, `subscribers`. |

**Who-columns rule:** every `*_by`, `set_by`, `author` column on a new table is `FK accounts.id
NULL`; `decided_by` stays the enum `{algorithm, human}` with the FK as the pointer. Existing
comment `author` strings are backfilled as `guest:<name>` identities when R lands.

**File ownership** lets sessions run blind to each other; each stream is its own session and
stays in its column. The table is in §7.

## 3. The three first-class concerns

### 3a. Data quality, truthfulness, verification (rian's first priority)

The audit's verdict: the database is honest at the observation level (0 zero prices, 0
`was<=price`, 0 duplicate emissions, 0 bad check digits, live FX every run) and there is **no
repeatable verification mechanism anywhere.** Several derived values are published as if
observed. **Stream Q** owns three deliverables:

1. **`app.cli verify`: live sample verification, stored.** Q defines `read_one(listing)` on the
   `Collector` protocol and builds `verify.py` against a stub on day one; **A implements
   `read_one`** for Avolta, Shopify, ARI and Extime in its first pass (Dubai and the Heinemann
   family return `SourceBlocked` until unblocked); **A2 implements it for Changi via `render()`**,
   capped at 20 per run. Fetched through our own layer: honest UA, robots re-checked, crawl-delay
   honoured; **abort a source at the first `SourceBlocked`** and record one BLOCKED check; **run one
   process per host**, as collection does; run the post-collection check **the next morning**, and
   use the run's own counters (prices written vs previous run, skip ratio, PARSE_FAIL count) as
   the immediate tripwire. Verdicts: `PASS`, `PRICE_MOVED`, `MISMATCH_SIZE`, `MISMATCH_IDENTITY`,
   `MISMATCH_CURRENCY`, `GONE`, `PARSE_FAIL`, `BLOCKED`, plus a no-network `REVIEW` when the
   re-read price is under half or over 2.5× the product's cross-shop median, or a `PRICE_MOVED`
   exceeds 50%. Where a parser already has two channels (Extime JSON-LD offer vs RSC variation
   price) compare them. **Verify writes a `source_kind='verify'` observation on `PRICE_MOVED`**
   (§10 #11 to ratify), so the price table gets fresher and every price still carries its date.
   **Sampling v1 is hardcoded:** census of every listing behind the unscoped featured eight and
   the top-20 savings (cheapest and dearest listing), plus every row moved >20%, then uniform
   random to fill N per source. Render-cost sources are verified only on the targeted set.
2. **`app.cli audit`: no-network internal consistency, snapshotted nightly** into
   `audit_snapshots`: vertical-vs-category mismatches, location currency vs observation currency,
   orphan `-P` tiles without a `::size` sibling, duplicate match_key groups by gtin kind, drinks
   with price and no size, size >5,000 ml outside multipacks, Extime capacity vs
   `price_per_quantity`, same-day flips without `was_price`, cross-shop ratio >2.5, `in_stock=true`
   rate per source, fallback-FX use, image provenance split, uncategorised per source, stuck runs,
   observation age per visible location. Until its table lands, audit writes JSON under
   `.logs/verification/`. Quarantine and veto decisions are **stored** in `rejected_observations`
   (both ingest-layer rejections and collector-layer skips, via a run-scoped hook A adds).
3. **A dated human review**: **Mon 21 Sep**, in rian's calendar, ~5 hours, with audit emitting
   the lists (brand folds, ratios, discounts, sample rows with retailer URLs) so the human only
   judges "right product, right size, right currency". It is the only independent reader of
   correctness and **the only check Dubai can get** ("DXB 20-row browser check" is Dubai's
   publication condition). Monthly thereafter.

**What verify measures, said plainly (goes in QUALITY.md):** freshness and parser drift.
Re-reading through the same parser cannot catch a parser that reads the wrong field; that is
guarded by the ingest distribution checks and the human review. Do not describe verify as
proof of correctness.

**Specific defects fixed in Stream A's first pass** (all measured): set `vertical` from the
collector and backfill; purge or re-resolve the 25 orphan configurable tiles (ATH 21, MEX 3,
YYZ 1) and re-crawl ATH with the variant cap raised; on `gtin_size_veto` do not create a
wrong-size product (two Glenfiddichs are stored as 7,000 ml); Extime cross-checks capacity
against `price_per_quantity` and `net_weight`; Dubai `in_stock=None` unless the API exposes
stock; MEX location currency; persist FX rate and source per run, add HKD/MXN/SGD; split
`image_source` into barcode vs name provenance and require number tokens to agree; stamp
`observed_at` per fetch; mark the 8 stuck runs as error; `Location.visible` default False.

### 3b. Long-term collector viability (rian's second priority)

**How collection works, in one paragraph** (becomes `COLLECTORS.md`): one shared pipeline,
`fetch.py → collector.collect() → RawListing → ingest.run_collector → normalize / taxonomy /
fx → Product / Listing / PriceObservation`. Seven collector modules, six classes, twenty-one
registered sources. **One collector per commerce platform, airports as configuration rows**,
plus bespoke modules for platforms seen once: Avolta/Magento (10), Shopify (4), Heinemann
platform (4, robots-blocked), Dubai (Oracle Commerce API), ARI Dublin (Salesforce), Extime
Paris (Next.js). **No step uses an LLM or any AI.** Adding an airport: one config line on an
existing platform; ~150 lines for a new JSON-API platform; a bespoke parser for server-rendered
HTML; **a `render()` call for client-rendered sites once A2 lands it.**

| Fragility | Evidence | Plan |
|---|---|---|
| Robots matcher keys on the first UA token ("mozilla") and so ignores a group addressed to `DutyFreeProfessorBot` | reproduced 2026-09-04 | **A, first hour, before A2 imports it:** `collectors/robots.py` with `BOT_NAME`, group match on case-insensitive substring of the bot name, crawl-delay from the same rule, stdlib parser retired from avolta.py, the by-name refusal as a test |
| Three robots policies and three collectors with none | extime stops on unreadable; heinemann continues; avolta disables all path checks; ARI/Dubai/Shopify never check at runtime | **One policy (RFC 9309 §2.3.1 base, §10 #13 to ratify):** 404/410 → unrestricted, logged; **401/403 → refusal**; 5xx/timeout → stop the run, retry next run. Applied at the top of every `collect()`, `read_one()`, `render()`. |
| Four Avolta stores forbid `?p=` pagination | LHR, ATH, MAD, BCN carry `Disallow: /*?`; advertised sitemaps 404 or WAF-challenge | Category discovery is **page 1 only** there; the recollection **also re-reads every product URL already held in `listings.url`** (permitted, no query string) so page-2+ products do not vanish. Measure the shrink at LHR first. Pages-2+ observations already collected against robots: **§10 #14.** |
| `MAX_CATEGORIES=16` alphabetical slice | it once dropped whisky/wine at LHR; `beauty/*` sorts before `liquor/*` | **Walk per vertical with its own slug list**, drop the global alphabetical cap; log every category any cap drops, naming the store; raise `MAX_VARIANT_LOOKUPS` for the targeted set. |
| Bot managers refuse declared bots | Dubai 403; Shilla challenge; Heathrow sitemaps WAF | Decision 3 (bot identity page, then the ask). **The page is its own static project, never an alias of the app** (§9 #3). |
| Platforms redesign | any regex breaks | `verify` PARSE_FAIL within one run; real trimmed fixtures per collector; posture per source in the owner area |
| Client-rendered stores | SIN, Shilla, Madrid-class grids | A2's `render()`: own Chromium in an isolated sidecar, honest UA, robots, floor delay, no dialog click-through, stop on challenge |
| Permission is the scarce asset | most operators cooperate or permit (trade interview) | posture per source with robots read date; **`identity_mode` switches only with a `permission_record`**; Cannes is where asking happens (Dubai, Avolta pagination/sitemaps, Shilla) |
| Egress IP reputation on the production move | untested; 15 of 16 answer this server's IP | **E's first deliverable:** one-hour droplet, fetch robots + one product page from all 21 sources through `fetch()`, record per source; then decide where collectors run (§10 #7) |

### 3c. Documentation a stranger can use (rian's third priority)

Three tiers, one owner file per concern, nothing typed that can be generated.

- **Tier 1, always loaded.** `agents.md` ≤140 lines (180 today after the review trim; the Docs
  session finishes it): rules only, ≤4 lines each with past-tense provenance, no counts, no
  paths that can move; today's decision blocks are 2–3 line pointers to §9. `brief.md` ≤40
  lines with a six-line **NOW** block rewritten in place by the session at checkpoint.
- **Tier 2, on demand, `main/docs/`,** each opening "Sources of truth: <dirs>" at directory
  granularity: `ARCHITECTURE.md` (module map **generated** from first docstring lines by
  `scripts/docmap.py`), `DATA-MODEL.md` (tables generated from SQLAlchemy metadata), `COLLECTORS.md`
  (§3b paragraph, per-platform recipes, robots policy, the source table from `app.cli status`,
  the rendered-fetch section), `SEO.md`, `CLIENT-SURFACES.md`, `RUNBOOK.md` (dev loop, deploy
  with dump-first, CLI from `--help`, the three tokens, what a deploy kills), `QUALITY.md`.
- **Tier 3.** `issues.md` with "last swept <date>"; handoff entries ≤25 lines and never the only
  home of a mechanism; `poc-dfp.md` → `status: delivered`, guardrails moved to `COLLECTORS.md`;
  `site-structure-proposal.md` banner'd superseded by the page.

**Vigilance, two tiers so one session's commit cannot fail another's hand-back:** a small
`main/docs-check.sh` called from `check.sh`. **FAIL:** `agents.md` >140 lines; any generated
map out of date; brief NOW-block version ≠ `APP_VERSION`; `CHANGELOG.md`'s first **versioned**
heading ≠ `APP_VERSION`. **WARN (FAIL only in `--strict`, run in Wave 4 and the post-launch
sweep):** a prose doc whose named source directory changed in git after the doc; `issues.md`
"last swept" older than 7 days; a handoff entry over 25 lines (23 of 66 existing already
exceed). Sessions add CHANGELOG lines under **Unreleased**; rian bumps the version once per
deploy checkpoint. A project `.claude/commands/checkpoint.md` extends the server command with
the four duties (NOW block, issues sweep, doc gates, diary); **it loads only when the session
cwd is the project**, so every kickoff states `cd /srv/apps/dutyfreeprofessor`, the old
folder's CLAUDE.md gets a one-line redirect, and rian gets `alias dfp='cd
/srv/apps/dutyfreeprofessor && claude'`.

**Code documentation standard:** every module says what it is for and what it depends on (the
map is built from that line); every non-obvious decision says why and what it cost; trivial glue
gets nothing. Measured: 88% of modules and 45% of public functions have docstrings.

**Hygiene now:** five "professormediagroup" headers; brief retitle and rewrite; delete
`nginx.conf` and `.logs/deploy-failed.log`; `docs/` created (A2 starts `COLLECTORS.md`).

## 4. The backend that does not exist yet, designed as data shape

1. **One record of every human decision: `overrides`.** `(entity_type, entity_key TEXT,
   field, value JSONB, collected_value JSONB, set_by FK accounts, set_at, reason,
   collector_disagrees_since NULL)` with `UNIQUE(entity_type, entity_key, field)`. `entity_key`
   is text so a brand slug, category slug, airport IATA or product id all fit. Readers resolve
   override → collected. Collectors never write it. **A "verify" is an override whose value equals
   the collected value at the time.** Computed surfaces use virtual fields: `featured_pin`,
   `award_pin`, `published`, `image`. **There are no per-field `verified_by/at` columns on
   products**; Principle C lives in this table.
2. **`decided_by` is derived:** an override row exists ⇒ human; otherwise algorithm. The algorithm
   proposes into candidates; a human pins. **An award pin is stored by natural key**
   `(product_id, competition_slug, year, medal)`, never `awards.id` (ids churn on `--rebuild`);
   if the key no longer matches, fall back to the algorithm and write a `reverifications` row.
3. **Disagreement is scored and queued.** When a collector disagrees with an overridden value, a
   scorer estimates meaningfulness (size/barcode change, sustained price move, name change beyond
   punctuation → high; single-run blip, whitespace → low) and writes `reverifications(row, field,
   old, new, score, reason, status)`. Humans clear from the top; their verdicts feed a re-ranking
   pass. Thousands are expected; the design assumes triage.
4. **Identity for growth (Decision 6, amended).** `vertical` from the collector; `size_value +
   size_unit` with `size_ml` kept as a derived column so liquid keys and featured are unchanged;
   one per-vertical attribute column (`expression` / `concentration` / `shade`) that **vetoes a
   join when both sides are known and differ, and never enters the key**; when null on one side
   the join proceeds and a `merge_candidates` row is queued. Edition and pack tokens join identity
   (splits 3 products, all genuinely different). **Ambiguity rule:** if two or more GTIN-holders
   share a key and the incoming row has no gtin, create a new product and queue it.
5. **Brands are a table.** `brands(id, slug UNIQUE, name, canonical_id NULL)`, `products.brand_id`,
   backfilled by `app.cli backfill brands` folding case, accents, punctuation and trailing
   category words; aliases are rows pointing at a canonical. B reads slugs from the table so URLs
   never move. Brand merges flow through the same queue.
6. **Merges are recorded and tombstoned.** `products.merged_into_id`; one merge service repoints
   listings, awards (skip collisions), overrides and pins, moves the gtin to the canonical,
   records detail; `_resolve_product` and every catalogue query filter `merged_into_id IS NULL`;
   one `resolve_product_id()` helper follows the chain and drives the 301 in `main.py`,
   `routers/catalog.py` and the sitemap exclusion.
7. **Everything re-doable.** `raw_records` per listing per run with `parser_version`;
   `identity_rules_version` on Product; `app.cli rederive` re-runs rules over stored raw with no
   network.
8. **Editorial storage.** `articles(id, slug, title, standfirst, body_md, kind article |
   airport_writeup | category_intro, category, author FK, status draft|published, published_at,
   airport_code NULL, brand_id NULL, …)`; public reads `published` only; intake `app.cli
   articles import <file.md>`; editor later.
9. **The owner area (Stream R, §10 #1 decides its timing).** Five surfaces: issues dashboard,
   data summaries, collector posture, comment inbox, drafts. **Deliverable 1 is an ASGI middleware
   that denies any non-GET request and any `/owner*` or `/api/owner*` path without a valid
   session, except exact paths on an initially-empty `PUBLIC_MUTATIONS` allowlist; a test walks
   `app.routes` and fails if a mutating route is neither allowlisted nor covered, run by
   `check.sh`; the kill switch is the first route behind it.** Session cookie Secure/HttpOnly/
   SameSite=Lax, Origin check on mutations, argon2, rate limit and lockout on login.
10. **AI-assisted QA later** proposes into the same candidate queues; humans confirm in batch.
11. **Consent converts a refusal.** `sources.identity_mode` switches to `browser_like` only when
    `permission_record` names who agreed, when and how; a UA change is the ceiling.

## 5. Comments (Decision 9, lowest priority)

Step 1 (owner inbox, no schema change) after Q, A and R work; step 2 (the BW Interaction
Standard shape) in Wave 2 or the buffer. Authors become `guest:<name>` identities when R lands.
**Sixteen of Adam's quote-thread replies are unanswered as of today; a by-hand reply pass on
Mon 7 is the vehicle for every client message in §10.**

## 6. The SEO/AEO plan

Built and real: server-stamped head (title, description, canonical, OG, Product +
AggregateOffer + award JSON-LD, BreadcrumbList, WebSite, sitemap with lastmod, robots, noindex
on internal pages). What dominates: **the body is empty** to any non-rendering client (Stream B
deliverable 1, needs no structure decision); the public host is gated until launch; JSON-LD URLs
are relative and the canonical host comes from the Host header (`PUBLIC_BASE_URL`, absolute
`url`/`offers.url`/breadcrumb `item`, `@id`).

Then, all Stream B: gtin13/12/8/14 by width; availability omitted where in_stock is null;
per-airport `Offer` nested with `priceCurrency`; Organization; **drop SearchAction** (retired
Nov 2024) keep WebSite; real 404s; HTTP HEAD; ETag/Last-Modified on HTML, sitemap, robots;
`no-store` on HTML and JSON so a production CDN cannot cache admin changes; favicon and
manifest; IndexNow from the collector; per-bot robots policy with Content-Signal (after the
Cloudflare zone check); RSS for articles; `llms.txt`; Dataset markup for a public data page.
Hub pages (brand, category, airport, category-at-airport) with ItemList + CollectionPage +
Airport **after Mark's comments and after the brand table and uncategorised backfill land.**
Every merged product 301s by `resolve_product_id()`.

**The structure page needs a half-day sweep before it goes to Mark** (web/ owner): "the sixteen
collected today; nineteen at launch with Paris, Seoul and Singapore"; SiteMap caption to four
airports with a linked live number; SearchAction line aligned with this section; the four
"free / almost nothing" phrases replaced ("a rules change on our side", "low cost to emit", "a
declaration rather than a rebuild"); beauty counts re-measured; a sentence on which page types
ship 18 Sep (§10 #10).

## 7. Streams, ownership, waves

| Stream | Owns | Deliverables now |
|---|---|---|
| **A. Collectors & data** | `app/services/collectors/*` **except** the files A2 owns below, `normalize.py`, `ingest.py`, `taxonomy.py`, `cli.py`, `models/catalog.py`, base.py, tests | **First pass (Fri 4 – Sat 6):** `robots.py` (matcher + policy + crawl-delay), migration #1, `RawListing.raw/vertical` with every collector populating it, `read_one` on Avolta/Shopify/ARI/Extime, the §3a defect list, the `/bot` UA repoint once the bot host exists. **Wave 2:** beauty widening (per-vertical walk, Dublin prefixes, Shopify Spanish/beauty shelves, page-1-only at the four query-blocked stores + re-read of held URLs), the **targeted 40 list by Tue 8** (method below), migration #3, brand fold + uncategorised backfill + the 52 launch-scale duplicate groups as recorded merges, `app.cli images import <dir>`, sixteen recollection (drinks refresh exempt from Decision 4, after robots.py). **Not Shilla, not SIN.** |
| **A2. Rendered fetch** | `fetch.py` (`render` addition), `changi.py`, `shilla.py`, `registry.py` (two lines), `fx.py` (SGD), `dfp-browser` in compose, `models/catalog.py` for the one `visible=False` line, their tests | Per `streams/A2-rendered-fetch.md` incl. the 2026-09-04 additions. Shilla probe first. A requests changes to A2's files via rian until A2 hands back. |
| **Q. Quality & verification** | `verify.py`, `audit.py`, their CLI subcommands, migration #2, `QUALITY.md` | audit first (writes JSON until its table lands), then verify against A's `read_one`; per-host runs; BLOCKED rule; QUALITY.md's plain-language explanation |
| **B. Pages & SEO** | `seo.py`, `main.py` routes, `routers/catalog.py`, new page components, `lib/urls.ts`, the structure page sweep | server-rendered body first; §6 corrections; hub pages after Mark + brand table |
| **C. Design** | `styles/*`, `public/` assets, header/footer/announcement, `HomePage.tsx`, all `*.css` | mega menu, prominence; CSS for Q/R surfaces on request; Instagram/YouTube links |
| **D. Content** | `articles` table + pipeline + import command, email capture (migration #5), sponsor slots | articles as table with `kind`; subscriber fields checklist to Adam by Fri 11 |
| **E. Infrastructure** | production host, DNS, certs, backups, monitoring, deploy path, runbook, host cron | **egress test first**; nightly `pg_dump` cron with 14-day retention + one rehearsed restore; sizing and real monthly cost to rian by Tue 8; `PUBLIC_BASE_URL`; zone checklist (Bot Fight Mode / AI-bot blocking / managed robots.txt off, cache rules bypass `/api/*` and `/owner/*`); cron for `app.cli audit` nightly, `verify --n 20` after any collect, `verify --n 200` weekly, on whichever machine the egress decision picks; sidecar (2 GB / 1 CPU) and nineteen airports in the sizing input |
| **F. Awards** | `awards_import.py`, picker, `featured.py`, `Medal.tsx` | strategy doc for Adam, then picker; `--rebuild` upserts by natural key; pin via `overrides.award_pin` |
| **R. Owner area** | `routers/owner.py`, `web/src/pages/owner/*`, migration #4, the auth middleware + route-inventory test | **§10 #1 decides whether R runs before or after 18 Sep.** Middleware and route test come first regardless. |
| **Docs** | `main/docs/*`, `scripts/docmap.py`, `docs-check.sh`, `agents.md`/`brief.md` trims, `.claude/commands/checkpoint.md` | one session (Sun 6): hygiene, the tier-1 trims to ≤140, `COLLECTORS.md`/`RUNBOOK.md`/`QUALITY.md` shells, the FAIL gates only; WARN gates and the rest in the buffer |

**Targeted-40 selection method (Stream A, by Tue 8):** seed = Paris beauty rows with GTIN
(2,937) plus the 9 ATH×CDG match_key hits; discovery at near-zero cost = Dublin beauty slugs from
its product sitemap (0 fetches), Shopify beauty rows from `/products.json` already fetched,
Changi beauty slugs from `sitemap-products.xml` (0 renders), Avolta **page-1** tiles of the
beauty categories per store (5–8 fetches per store at 30–60 s, one process per host); normalise
to brand + line + size + concentration; count distinct stores; pick 20 fragrance / 20 skincare
with 3+ stores, preferring GTIN-bearing rows. Only those ~40 product pages are then fetched
everywhere. Adam's images list follows from it.

**Wave 1 (Fri 4 – Sun 6): decide, unblock, back up.** A2 (running) + one other seat: **A first
pass Fri–Sat, Docs session Sun.** Rian: dump taken (done), **send Mark the `/structure` URL after
B's half-day sweep, comments by Thu 10**, answer Adam's sixteen open threads Mon 7 (§10). Q
starts Mon 7 as the second seat; E is rian-driven between sessions. Overnight: **targeted
collection of the beauty candidates only after `robots.py` lands; no full crawls (Decision 4);
the drinks refresh of already-collected sources is exempt and runs after migration #1.**

**Wave 2 (Mon 7 – Fri 11): the heavy build.** B: body rendering, §6 corrections, structure
sweep, then hub pages **Fri 11 either way** (brand table gates them, not Mark's silence). A:
beauty widening, targeted list, migration #3, brand fold, duplicates. Q: audit, migration #2,
verify. C: mega menu, prominence. D: articles table and import against placeholders. R and
comments step 2: **only if §10 #1 keeps them in.** **Checkpoint Fri 11** (the quote's date; §10
#2 decides what it means).

**Wave 3 (Mon 14 – Tue 15): features on top.** F: strategy then picker. D: email capture,
sponsor slots, Adam's articles. A: two categories at twenty each; full recollection after the
widening. Q: weekly N=200.

**Wave 4 (Wed 16 – Fri 18): one session.** Integration, full test run, cross-device and
accessibility, performance, `docs-check.sh --strict`, launch checklist including **curl every
`App.tsx` route and its `/api/*` reads unauthenticated on the production host and confirm the
internal set is closed**, walkthrough.

**Buffer (Sat 19 – Fri 26):** whatever §10 #1 moved; Adam's late content; **Mon 21 human
review**; the Cannes ask list (Dubai, Avolta pagination and sitemaps, Shilla).

## 8. Daily rhythm and what to watch

Morning: each session's last handoff line, the overnight audit diff and verify failures, the
day's tokens. Midday and evening: deploy checkpoints (dump first if a migration rides). Every
session before handing back: `check.sh` green, own commit prefixed with its letter, one handoff
line ≤25 lines, its doc updated, CHANGELOG line under Unreleased.

Watch: Adam's content and his sixteen unanswered threads; Singapore's sidecar; Dubai (39% of
launch products, stale); scope arrives by conversation and goes to the board; **no client-facing
figure is quoted without a `verify` run or a dated human check behind it.**

## 9. Decisions made 2026-09-04 (thirteen; amended where the review found a defect)

1. **Paris hidden on the demo** until switched on deliberately. `Location.visible` default →
   False (migration #1 / A2's one line).
2. **Auth is app-owned, on FastAPI + React, one domain** (reaffirms poc-dfp D4; django.md case 3
   with the case-2 check answered). Accounts skeleton lands in migration #1 so who-columns have
   an FK from day one. Post-Cannes checkpoint: cost Django against the real admin.
3. **Dubai path A, amended.** (i) **The bot identity page is its own static project**
   (`srv-gw create-project --name dfp-bot --type custom --domain <bot host>`, one `index.html`,
   then `id-gate --site <bot host> --mode public` on that host) — **never a second hostname on the
   app container while any write route is unauthenticated** (an alias would expose `/quote`,
   `/structure`, `/discuss`, `/settings` and the kill switch). Hostname on a zone rian controls
   (§10 #3). UA repointed once it serves. (ii) The ask to Dubai when rian judges the relationship
   ready; **Dubai launches on dated 21 Aug prices** (§10 #5 ratifies the BLOCKED rule).
4. **The launch set is nineteen; beauty collected at all airports; targeted first, full crawls
   last.** Amended: the drinks refresh of already-collected sources is exempt; the four
   query-blocked Avolta stores are page-1-only plus held-URL re-reads; per-vertical walks replace
   the alphabetical cap; the targeted-40 method is in §7.
5. **Rendered fetch built now for Seoul and Singapore** (A2). Shilla probe first. If Shilla
   refuses, a replacement airport on a genuinely new platform (§10 #9 names candidates now).
6. **Identity for growth, amended:** per-vertical attribute vetoes, never keys; brands become a
   table; merges tombstone via `merged_into_id`; raw records first (migration #1).
7. **Override model, amended:** `overrides` is the single record of human decision
   (`entity_key` text, `UNIQUE`, `collected_value`, `collector_disagrees_since`); no per-field
   verified columns; award pins by natural key; reverification queue scored and re-ranked.
8. **Articles are a table** with `kind` (article | airport_writeup | category_intro) and status.
9. **Comments: owner inbox first, standard later; lowest priority.**
10. **Verification bar, amended.** N=20 after a collection is a **tripwire, not a rate**: any
    `MISMATCH_*` or `PARSE_FAIL` blocks publication of that source until reviewed; `PRICE_MOVED`
    and `GONE` are reported. The per-source pass rate is a **rolling window over all checks**,
    shown in the owner area and reviewed after two weeks of real data, **never a launch condition
    at n=20** (20/20 bounds the true rate only at ≥86%). **A source whose checks are BLOCKED is
    governed by its last-observed date**: prices stay visible with "last checked" frozen and a
    flag on the dashboard. Public wording: per row **"Seen <date>"** (exists today), per source
    **"Spot-checked <date>"** with a one-line sampling note, shown only after a run with zero
    correctness failures; **never a per-row "Verified"**, never a rate, never a failure list.
    Freshness failures keep rows visible with their date; correctness failures hide the affected
    rows or source until a human clears them.
11. **Documentation with gated vigilance** (§3c, two tiers of gate).
12. **Owner area v1 = five surfaces**; `/structure` moves behind it **after Mark has commented or
    Thu 10 passes**.
13. **Capacity: two sessions, three briefly, fixed deploy times.** Amended drop order (§10 #1):
    **unpaid infrastructure drops before any quote line.**

## 10. What rian still decides (from the review; ranked)

1. **Scope vs capacity.** The plan schedules roughly twice what two seats can do in nine days,
   and the overflow is mostly unpaid back-office work. Move to the buffer or after Cannes: Stream
   R's five surfaces and the app-owned login screens; comments step 2; the reverification
   scorer and merge-queue *behaviour* (keep the columns); the WARN-tier doc gates and the full
   seven-doc restructure (keep hygiene, COLLECTORS/RUNBOOK/QUALITY shells, FAIL gates). Keep
   from the unpaid work only the cheap enablers already in migration #1–#2 and the auth
   middleware + route test (they close nine open write routes). Confirm the inverted drop order.
2. **The Fri 11 milestone Adam approved** ("production live, sixteen public, crawlable pages").
   Either (a) Fri 11 uses the quote's own bar: fresh drinks recollection minus Dubai + a one-off
   human 30-page check, sixteen public on the demo host, production live if E lands it; verify's
   tripwire becomes the 18 Sep gate; or (b) re-date with Adam now: Fri 11 delivers crawlable
   product pages and brand pass two; sixteen public and production move to Fri 18. **Tell him
   this week either way.**
3. **The bot identity hostname**, permanent, on a zone rian controls (it is baked into every UA
   string and retailer log forever).
4. **Client surfaces on production** (`/quote`, `/discuss`, `/structure`, `/sources`,
   `/settings` and their `/api` reads): (a) off behind a `feature_client_surfaces` flag in
   production, or (b) behind the login with a `client` capability so Adam and Mark keep
   commenting after the move. `/sources` and `/settings` go behind the owner login either way.
5. **Ratify the BLOCKED rule** (Decision 10 amendment): Dubai public on dated prices, flagged.
6. **Hosting figure and message.** $250/month was the 1 Sep anchor; the quote page says $50 and
   promises real numbers before production; E gives a number by Tue 8. Decide (a) production
   from Fri 11 at $X, or (b) $50 holds through soft launch and steps up when scheduled collection
   starts. Say it on the `quote:core-environment` thread Adam opened.
7. **Where collectors run** after E's egress test: on the production host; on this server
   writing to production Postgres over Tailscale; or a runner with a reserved IP. This also
   fixes which machine hosts the cron.
8. **Zone control**: fold into the Tue 8 domain ask to Adam — the zone moves to rian's Cloudflare
   account, or Adam issues a scoped API token staged into a 0600 file (never in chat); confirm
   the WPE placeholder stops resolving at cutover.
9. **Shilla replacement candidates**: name two airports on genuinely new platforms this week; a
   session runs the 20-minute robots + product-page probe and records verdicts, so the swap is
   one sentence to Adam if needed.
10. **Which page types ship 18 Sep**: "all six where thresholds are met; beauty categories switch
    on as coverage crosses the bar", or "category and crossing pages in the buffer week". One
    sentence on the structure page and in the note to Mark.
11. **Verify writes observations on PRICE_MOVED** (`source_kind='verify'`) — recommended yes.
12. **Host cron for audit/verify** owned by E — recommended yes; state in COLLECTORS.md that this
    is not "automated collection" (collection stays hand-run, as told to Adam).
13. **Unreadable-robots policy**: ratify 404→open, 401/403→refusal, 5xx→stop.
14. **Pages-2+ observations at the four query-blocked Avolta stores**, collected against their
    robots: keep (dated) or purge.
15. **Client messages by Mon 7** (rian, ~30 min, one reply per thread): Mark — the `/structure`
    URL and password after the sweep, comments by Thu 10, "if I have nothing by Fri 11 I build to
    the proposal as written". Adam — Seoul's product pages began answering a security challenge
    this week; we are building the browser reader for Singapore now and will try Shilla with it;
    we do not play cat and mouse, a block is a stop, and a different airport from your list takes
    its place if needed, so please send the 20+ airport list; Dubai began refusing our declared
    reader on 3 Sep after five clean runs, it is 39 percent of launch products, we will not go
    around it, at launch it shows 21 Aug prices with the date, and your Dubai Duty Free
    conversation before Cannes would help (forward them the identity page when it exists);
    images: yes start sourcing, the list by Tue 8, each product shows at the two to five airports
    that stock it, images after Fri 11 land in the buffer, a page with no image shows none;
    airport write-ups and one 200–300 word paragraph per category by Fri 11 as two new asks
    (start with the four visible airports plus Paris); the hosting figure or the explicit hold;
    awards override: yes, via rian until the admin area; articles handover to Kristine/Matt after
    launch; subscriber fields checklist by Fri 11; sponsor sizes noted; corrected weekdays for the
    Tue 8 and Fri 11 cut-offs. **No em dashes; never "cheap", "free" or "the code is yours".**