# dutyfreeprofessor — agents.md

Project constitution. Inherits from `/srv/CLAUDE.md` (server) and `/srv/apps/CLAUDE.md`
(tenant); `CLAUDE.md` is a symlink to this file. Rules only: every rule here is
failure-backed, tool-enforceable, decision-encoding or triggerable. How things work lives in
`main/docs/`. Budget 140 lines, enforced by `main/docs-check.sh`.

## Read first (in order)
1. `brief.md`: what this is, who it is for, and the NOW block.
2. `main/docs/ARCHITECTURE.md`: the request path, the pipeline, the module map.
3. The running list: `python3 main/scripts/items.py list` (decisions for rian, his to-dos, open issues; shown on `/issues`). `.logs/issues.md` and `.logs/decisions-for-rian.md` are generated from it.
4. The last three entries of `.logs/handoff.md`, not the whole file.
5. What rian said to Claude on the running list since then: `docker exec dutyfreeprofessor-app python -m app.cli discussion inbox --for claude --since <date>` (a comment naming `@claude` on `/issues`; it notifies nobody).

Then the stack standards: `/srv/projects/standards/README.md` → `coding.md` → `react.md` →
`frontend.md`. A stream session is also bound by `.logs/planning/streams/OVERNIGHT-RULES.md`.

## Legal and collection posture
- **A price is an observation, never a fact.** Duty-free pricing varies by destination,
  loyalty tier and traveller. Store timestamped observations with context, show the date
  in the UI, and never write code or copy that implies a canonical "the price".
- **Collect facts, not expression.** Prices, sizes, barcodes, stock and brands only; no
  retailer marketing copy, no rehosted photography. Every plaintiff in the comparable
  litigation won on imagery and branding, never on prices (`main/docs/COLLECTORS.md`).
- **Never log in to a retailer, never create an account, never accept terms**, not even
  for testing. Logging in is what creates the contract; the legal posture rests on it.
- **Robots.txt is re-read on every run, matched with modern wildcards, and ANY matching
  Disallow is a no.** The stdlib parser read `Disallow: /*/search/` as permission and a
  retail family was collected for days on it; permission was once withdrawn between runs.
  Honour intent too: a host that permits its global catalogue but disallows per-airport
  paths gives us identity, and its prices become a partnership ask.
- **Crawl-delay is the slower of theirs and ours,** per host. Sibling airports on one
  platform asked for different rates. Following published crawling rules is the opposite
  of evasion; what matters is whether they said no, not whether a request is automated.
- **A block is a refusal, not an obstacle.** Record it and stop; never escalate to
  stealth tooling or paid unblockers. The client's business is relationships with these
  same retailers. The fetch port enforces this by raising `SourceBlocked`.
- **Consent converts a refusal, and consent is data.** A source runs in a browser-like
  identity mode only when its permission record names who agreed, when and how; without
  the record the collector refuses to switch. A user-agent change is the ceiling.
- **Validate fetches on content, never on status code.** A retail edge answered blocked
  requests with HTTP 202 and an empty body; a naive client recorded success and ingested
  nothing.
- **Harvest listing pages, not product pages.** Same data at a small fraction of the
  request volume; this decision dominated running cost more than any vendor choice.

## Identity and matching
- **GTIN is the product identity.** Barcodes make cross-retailer matching a join; the
  normalised brand+name+size key is only a fallback. Prefer sources that publish one, and
  weigh a new shop on barcode coverage before product count: without barcodes its
  products only ever match others from the same operator.
- **A fallback identity lookup must consider every key-holder.** Resolving by match key
  took one arbitrary row; when it held a barcode the guard rejected it and minted a fresh
  duplicate per airport per run for two days. Iterate all candidates; create only when
  none is compatible.
- **Medal matching is precision-first.** The competition network has no barcodes, so
  awards match on brand and name. A medal on the wrong bottle is worse than a medal
  missing: conservative thresholds, and review samples after any change.
- **One tokenizer does not fit every matcher.** The catalogue's stopwords delete
  "original", "reserve" and ages, exactly the words that tell an expression from its
  sibling, and medal matching silently landed on the wrong bottle. Each matcher owns its
  token rules: medals keep expression words, require brand agreement, veto number
  mismatches, and treat a tie between different names as no match.
- **A multi-size tile's price is a family's cheapest, and which member differs per
  store.** One configurable slug defaulted to the 1.5L at one airport and the 75cl at
  another, and the slug's own size lies ("-15l" means 1.5 litres). Collect each priced
  size as its own listing, mint no barcode from a parent SKU, skip what cannot be read.
- **An extreme spread seen at only two shops does not headline until a third
  corroborates.** The dataset's single worst outlier was almost certainly a retailer's
  own entry error.

## Prices
- **Never infer currency from a symbol; read what the page declares.** One store
  rendered `US$` and `MX$` figures on the same line; a `$` matcher plus a per-store
  default published the whole shop at a fraction of its real prices and put a cheap XO
  cognac at the top of "cheapest anywhere". Prefer `priceCurrency` microdata, and treat
  a store an order of magnitude off its peers as a bug, not a bargain.
- **Parse the price the shopper pays, and quarantine the implausible.** The first price
  in a tile was the pre-promotion or small-variant price, and an impossible litre of
  vodka led the savings table. Target the typed final price, record the crossed-out one
  as was-price, and let ingest refuse anything under half the product's cross-shop median.

## Code and data
- **A GET must never write.** A listing endpoint created missing rows on read and
  deadlocked against a running collection holding those rows; the page simply hung.
  Collectors create their own rows when they run. The one named exception is the View As
  fail-closed auto-stop, confined to the session row on its own connection.
- **Collectors run concurrently, so every shared row must tolerate a race.** Two airports
  stocking the same bottle both tried to create it and the barcode constraint killed one,
  poisoning the session for everything after. Create inside a savepoint and re-read on
  IntegrityError; a plain select-then-insert lost whole runs.
- **One collector per platform, not per shop.** Operators on the same commerce software
  share a parameterised collector, so a new shop is configuration. Each domain's own
  robots.txt is still checked before it is added; permission does not transfer.
- **A bug that shipped becomes a test, in `main/tests/`,** written as the real record
  that failed with what it cost in the docstring. Pure logic only (parsing, matching,
  normalising, robots): no network, no database, seconds not minutes; the accounts suites
  run against an in-memory SQLite, still no server and no network.
- **Run `main/check.sh` before every deploy.** Tests, the typed API client regenerated
  from the backend's OpenAPI schema, the frontend typecheck, and the doc gates, in
  seconds; a Docker build finds the same errors minutes later.
- **Import a component's stylesheet in the component.** A CSS file nothing imports fails
  silently: the page rendered unstyled and nothing errored.
- **A catch-all route makes a missing route look like a working one.** An explainer page
  with no route returned 200 with the app inside it for days. Serve a directory of pages
  by rule, 404 an absent page, verify a static page by a string only it contains.
- **A personalised label needs a personalised query behind it.** A related-products rail
  said "in your shops" while naming shops the shopper had never chosen. Scope the query
  or drop the possessive.
- **Build-time toggles belong in Settings, not the storefront,** so the shopper-facing
  filter row stays as short as a real store's.

## Decided 2026-09-04, pointers only
- **Auth is app-owned identity on the current stack, one domain.** The BW kit is vendored
  byte-identical and never imported; `X-Auth-User` is never read; the host is never registered
  as a BW client, which would un-gate the demo (build plan §9 #2). Every route is in exactly
  one class of `services/access.py` or it is refused; the owner is `ACCOUNT_OWNER`, code not a
  row; a token rides only in a link's fragment and a JSON body, never a path (the access log
  prints every request line); `SITE_ACCESS=members` until the go-live flip (`main/docs/ACCOUNTS.md`).
- **Data shape: keep the raw record; empty beats guessed; merges are recorded, never
  deleted; a human value is never overwritten by a machine.** Migrations are schema-only
  with server defaults; every backfill is an idempotent CLI command (§9 #6–#7; `app.cli backfill
  <name>` and `rederive`, `main/docs/RUNBOOK.md`).
- **No second hostname on the app container while any write route is unauthenticated.**
  The bot identity page is its own static project (§9 #3; served at bot.dutyfreeprofessor.com,
  named in the reader's user agent).
- **Verification is a tripwire, not a rate.** Public wording is a date, never a per-row
  "Verified", never a rate or a failure list (§9 #10; `app.cli verify`, `main/docs/QUALITY.md`).
- **Docs are gated, not remembered.** `main/docs-check.sh` fails on a stale generated map,
  an over-budget tier-1 file or a version mismatch; one home per mechanism, the handoff
  links; comments say why and what it cost, module docstrings feed the map (§9 #11).
- **Open sessions in this workspace.** The checkpoint hook walks up from the cwd for
  `.logs/`; a session opened elsewhere wrote no diary for eleven days.

## Operating the thing
Dev loop, deploy, CLI, tokens, backups: `main/docs/RUNBOOK.md`. Layout and request path:
`main/docs/ARCHITECTURE.md`. Secrets live in `.app.env` and are never printed.