# Stream AW3: pictures at three levels

**Status 2026-09-19 (written by the away run's orchestrator from rian's ask of 19 Sep 10:00):** not
started. Second in `AWAY-2026-09-19.md` section 3, after AW1, before AW2 (reads `image_tier`); D2.

## Goal

Every card, page and server-rendered body shows the best picture the catalogue holds for a product
variant, else its product line, else its brand, with the level exposed so a brand-level fallback is
styled as one and never enters structured data as a product's image. Adam's folder becomes a
reviewable manifest, staged into `uploads/` and imported with provenance (who supplied it, under
what licence); an admin-supplied picture outranks any fetched one and no machine replaces it. Then
Wikidata and Commons logos, OFF photos promoted to the line, and `/images` by level.

## What already exists, so you build on it

- `services/images.py`: OFF by barcode (`lookup`), the strict name search (`search_by_name`,
  `numbers_agree`), `enrich_product_variants` writing the four variant columns; `USER_AGENT` names
  the bot page. **Bug to carry:** `search_by_name` reads `payload.get("product_variants")`; OFF
  answers under `products`, so the name fallback has found nothing since the K1 rename (AW3.4).
- `models/catalog.py`: `ProductVariant.image_url` (600), `thumb_url`, `image_source` (80),
  `image_checked`; no image columns on `Brand` or `ProductLine`. `merges.py` `_ENRICH_FIELDS` and
  `tests/test_inheritance.py`: a survivor inherits a picture it lacks.
- `cli.py`: `cmd_images` (the bare `app.cli images` is the OFF enrich), `BACKFILLS`, `cmd_backfill`
  (`--check` flushes then rolls back); sub-CLIs register through `register(sub)` (`cli_hours.py`).
  `services/image_ask.py`, `routers/image_ask.py`, `pages/ImagesPage.tsx`: the client's sourcing
  list under `client.view`; `test_image_ask.py`.
- `catalog_queries.py` reads the columns in the product list, `with_image_only`, the `multi_only`
  ordering, `ProductDetail`, `LineVariantOut`, `has_image`, `BrandSummary`, `BrandDetail`;
  `representative_variant()` is most units, then lowest id.
- `seo.py`: `absolutise()` (JSON-LD `image`/`logo`), `Head.image`, `product_jsonld`, `line_jsonld`,
  `line_body`, `product_body`, `head_for_brand` (`about: Brand`, no logo), the credit span. Shapes
  in `schemas.py` and `hubs.py`. `Thumb.tsx` (falls back to `BottleMark` on 404) is on every card.
- `main.py` `mount_site` (mounts `fonts medals flags media`), `cache_headers`; `access.py`
  `PUBLIC_ALWAYS` (`MOUNT /media`); `test_route_inventory.py` fails on an unclassified mount.
  `uploads/` is bind-mounted rw at `/srv/uploads`, `import/` ro at `/srv/import`. `DATA-MODEL.md`
  "Facts, not expression", "What a merge inherits (M7)"; `COLLECTORS.md` L2; `QUALITY.md` metric 16.
- The folder, read-only, main checkout only: `/srv/apps/dutyfreeprofessor/notes/images for the
  site/` (73 files, 300 MB), surveyed 19 Sep. "Duty Free Professor Imagery" is William Grant & Sons,
  one folder per product line; names carry size, view (`FRONT`, `BACK`, `CLOSE_UP`, `TUBE`,
  `BOTTLE_AND_BOX`, `TRANSPARENT`, `LAY BACK`, `CARTON`, `GROUP`) and sometimes a UPC or EAN; up to
  8192 px and 14 MB, RGBA PNGs, one CMYK JPEG, byte duplicates. "Bottle Images To Upload" is
  Bruichladdich x4, Port Charlotte 10 1L, and a zip of the same Pernod export. `products.csv` (438
  rows): `Core Data - Marketing Brand (Computed)`, `Core Data - UPC` (12 digits, the bottle), `Core
  Data - GTIN` (14 digits, often a case code with a leading 1), `Net Content (Computed)`, `Digital
  Assets - Front Bottle Image` (437), `Digital Assets - Brand Logo` (433). Pillow 12.3 has WebP.

## Design decisions (a change goes here and in the handoff, never silently)

- **Columns, not registry entries.** The registry describes per-variant kinds that decide sameness;
  a picture decides none, is shown by one rule everywhere, and lives on three tables the registry
  does not cover. Typed columns, the same names on all three tables.
- **Provenance vocabulary** (`image_source`, controlled, tested): `admin:<supplier-slug>` (such as
  `admin:william-grant-via-adam`), `public:openfoodfacts:barcode`, `public:openfoodfacts:name`,
  `public:wikimedia-commons`; `imagery.credit(source)` prints it; `backfill image_sources` moves
  the two old strings.
- **Storage.** Originals `uploads/originals/<level>/<id>-<sha8>.<ext>` (kept, never served);
  derivatives `uploads/images/<level>/<id>-<sha8>.webp` (long side 1200) and `-thumb.webp` (400),
  alpha kept, EXIF dropped, CMYK converted; `<sha8>` is the content hash, so `/uploads/images/`
  joins the one-year immutable prefix. Mounted by `mount_site` (directory created if absent),
  `MOUNT /uploads/images` in `PUBLIC_ALWAYS` like `/media`.
- **One cascade, one tier.** `services/imagery.py` (pure): `resolve(variant, line, brand) ->
  Picture(url, thumb_url, level, source, credit)`; `image_tier(variant, line, brand) -> int`: `0
  none, 1 public brand, 2 public line, 3 public variant, 4 admin brand, 5 admin line, 6 admin
  variant`. Every reader uses `resolve`; JSON-LD emits `image` for `variant` or `line` only.
- **The human rule.** `imagery.set_image(row, ...)` refuses to replace an `admin:*` source with a
  `public:*` one (`IMAGE_ADMIN_KEPT`, counted); every writer goes through it. No ledger row here.
- **Two commands, because the folder is not in the container.** `images stage` is run by this
  stream from the worktree, `--uploads /srv/apps/dutyfreeprofessor/uploads` (the mounted home; files
  only), and the manifest is committed with its `url` fields; `images import` (database only) and
  `backfill image_sources` are the orchestrator's, in the container, after the deploy.

## Tasks, in order (commit prefix `AW3:`)

### AW3.1 Survey and manifest (2 to 3 h)

- Open six to ten files with the Read tool, one per naming family, then trust the names. Then
  `services/image_manifest.py` (pure): `classify_file(path) -> Classified(level, view, quality,
  gtin, size_ml, brand_words, line_words, usable, reason)` from the path and name alone; quality
  `transparent bottle > bottle with box or tube > e-commerce bottle > lifestyle`; `BACK`, `CLOSE_UP`,
  `ICONS`, `LAY BACK`, `CARTON` alone, `GROUP`, `PACK`/`BOX` alone, `BWC` unusable with the reason.
  `classify_csv_row(row)`: brand from `Marketing Brand`, line words from the title before the first
  comma minus the brand, size from `Net Content`, barcode from `UPC` and from `GTIN` only when its
  first digit is 0 (a leading 1 is a case), through `normalize.clean_gtin`; `Front Bottle Image`
  is the variant picture, `Brand Logo` the brand picture; nothing else is used.
- `image_manifest.match(db, classified)`, precision first: GTIN equal to `ProductVariant.gtin` wins;
  else brand through `keying.load_maps().brand_of`, line through `product_lines.product_line_key`
  under that brand, size against `quantity_ml` for a variant. Two candidates, or a brand resolving
  to nothing, is `unassigned` with the reason. The line picture is the best-quality front bottle
  shot, preferring the `representative_variant`'s size, else 700 or 750 ml; a hand, a sash or a
  travel wrap makes it variant-only. The brand picture is the Pernod `Brand Logo`, once per brand.
- Output `import/images/adam-2026-09-19.json` (tracked, small): per file or URL `source_path` or
  `url`, `level`, `target` (`brand:<slug>`, `line:<slug>`, `variant:<id>`, the gtin when matched by
  it), `chosen`, `reason`, `image_source`, `licence`. `app.cli images manifest --folder --csv --out`,
  on the host with `DATABASE_URL` at `dfp-devdb` holding a fresh restore of the newest dump.
- Handoff: usable and chosen per level, unassigned with the top three reasons. Tests
  `tests/test_image_manifest.py` on the real names: `..._BACK.png` unusable,
  `GLENFIDDICH_750ML_21YO_BOTTLE_AND_BOX_TRANSPARENT.png` line-usable at box quality,
  `Glenfiddich_18YO_083664990412_Front_750ml-2.jpg` yields the UPC, a case GTIN-14 is dropped, two
  same-quality candidates unassigned. Doc: `RUNBOOK.md`, "Manifest".

### AW3.2 Storage, provenance and the cascade (2 to 3 h)

- Migration (schema only, server defaults, one head after `k12a1b2c3d4e`): on `brands` and
  `product_lines` add `image_url` String(600), `thumb_url` String(600), `image_source` String(80),
  `image_level` String(16), `image_licence` String(200), `image_attribution` String(400),
  `image_set_at` DateTime(tz); on `product_variants` the last four. Rehearse up, down, up.
  `image_level` is what the picture depicts, never inferred from the table a row sits in.
- `services/imagery.py`: the vocabulary, `credit()`, `resolve()`, `image_tier()`, `set_image()`,
  `store(bytes, level, id) -> (original_path, url, thumb_url)` with Pillow (`exif_transpose`,
  `thumbnail`, WebP quality 85), `derivative_paths()`. No network imports.
- `main.py` mount and cache prefix; `access.py` the class; `backfill image_sources` in `cli.py`;
  `audit.py` and `QUALITY.md` metric 16 read the new values; `merges.py` inheritance calls
  `set_image` (`test_inheritance.py`: an admin picture on the survivor is kept).
- Tests `tests/test_imagery.py`: the cascade picks variant over line over brand with the level;
  `image_tier` returns all seven values; `set_image` keeps admin against public, replaces public
  with admin; `credit()` for every value; `store` on a 4 px RGBA PNG and a CMYK JPEG writes both
  derivatives. Docs: `DATA-MODEL.md` "Pictures at three levels"; `VOCABULARY.md` (picture level).

### AW3.3 Stage, import, every reader on the cascade (2 to 3 h)

- `app/cli_images.py` as the `images` group: `fetch` (the former bare command), `manifest`,
  `stage`, `import`, `logos`, `lines`, `coverage`. `images stage --manifest <path> --folder <path>
  [--uploads <dir>] [--check]`: for each chosen entry, `store()` the file or the downloaded Pernod
  URL (one request per second, the bot user agent, refused on a non-image content type); skips an
  entry whose hashed derivative exists; writes `url` and `thumb_url` back into the manifest.
- Before the first download, announce `.logs/runs/window-2026-09-19-images.md` (type run): the
  hosts, the expected counts, one request per second; close it with the counts. Then run `stage`
  from the worktree with `--uploads /srv/apps/dutyfreeprofessor/uploads`; commit the manifest.
- `images import --manifest <path> [--check]`: for each chosen entry with a `url`, verifies the
  derivative under `/srv/uploads`, resolves the target by id, slug or gtin, calls `set_image` with
  source, level, licence, attribution, `image_set_at = now`; an unresolved target is counted and
  skipped; an applied entry is a no-op. Prints counts per level and `IMAGE_ADMIN_KEPT`.
- Every reader on the cascade: `catalog_queries` loads the line and brand rows by id set (one query
  each) and fills `thumb_url`, `image_url`, `image_source`, new `image_level` and `image_credit`
  from `resolve()`; `ProductLineDetail` and `BrandSummary` gain the four; `with_image_only` and the
  `multi_only` ordering still read the variant's own column. `seo.py`: `product_jsonld` and
  `line_jsonld` emit `image` for `variant` or `line` only; `head_for_brand` adds `logo` on the
  `Brand` node when a brand-level picture exists; bodies print `image_credit`. SPA: `Thumb` takes
  `level`, adds `thumb--brand` (`Thumb.css`, imported in `Thumb.tsx`); `BrandPage` shows the picture.
- Tests: `test_seo_body.py` and `test_seo_line.py` gain a brand-level fallback case (the brand
  class, no `image` in JSON-LD); `tests/test_cli_images.py` runs `stage` and `import --check` on a
  two-entry manifest against a tmp dir and the in-memory SQLite. Docs: `RUNBOOK.md`; `SEO.md`.

### AW3.4 Wider legitimate coverage (2 to 3 h)

- Fix the `products` key in `search_by_name`, with a test replaying a real OFF search payload
  (five candidates) that asserts the strict match still refuses the 18 for the 12. Then
  `services/images_public.py` `wikidata_brand_logo(brand_name)`: `wbsearchentities` on the exact
  name; accept only an entity whose label equals the brand after NFKD case folding and whose `P31`
  is in a small allow set (brand, trademark, business, distillery, winery, perfume house, cosmetics
  company); `P154`, else `P18`; Commons `imageinfo` with `extmetadata` (`LicenseShortName`,
  `LicenseUrl`, `Artist`, `Credit`, description URL); a free licence or public domain only; download
  (never hotlink), `store()`, `set_image` with `public:wikimedia-commons`, the licence short name,
  attribution "Wikimedia Commons: <File> by <Artist>, <URL>". One request per second, the bot user
  agent, `--limit`, `--brand`. `app.cli images logos`.
- `app.cli images lines [--limit N] [--check]`: a product line with no picture whose
  `representative_variant` carries a `public:openfoodfacts:*` picture gets it (same url, level
  `line`); a line whose pictures sit only on other sizes stays empty (empty beats guessed).
- Tests `tests/test_images_public.py`: a label off by one word, a `P31` outside the set and a
  non-free licence are each refused; the promotion picks the representative; fetchers take an
  injectable `fetch_json`. The one live call is `images logos --limit 3 --check` on the dev copy.

### AW3.5 The /images page, coverage, docs (1 to 2 h)

- `app.cli images coverage`: per level, counts with a picture split admin and public, and without;
  run before and after the import, both in the handoff. `image_ask.py`: `summary` gains the
  per-level block; product rows carry `image_level` and the credit; brand rows gain `has_image` and
  `image_source`; a `wanted` list per level (top by airports carried, no picture at any level).
  `ImagesPage.tsx`: stat cards per level; the pill reads `Adam's`, `public` or `none` from the
  prefix; the brand table gains "Brand image now". `test_image_ask.py` pins both.
- Docs: `RUNBOOK.md` images section complete (every command; the order written as "after a deploy
  that carries the image migration: `backfill image_sources --check`, then `images stage`, then
  `images import --check`, then `images import`", no deploy label; "after D2" stays in the handoff);
  `COLLECTORS.md` posture bullet (brand-supplied, openly licensed, takedown on complaint, every
  picture carries a source and a level); `CHANGELOG.md` Unreleased.

## Working assumptions (rian's to overturn)

- A clean bottle shot whose only size cue is the label's print is usable for the line; a hand, a
  sash or a travel wrap makes it variant-only. *Overturn: re-run `images manifest`, re-import.*
- An admin picture is a column value with an `admin:*` source, not a ledger decision. *Overturn: a
  later stream writes one `decisions` row per admin picture and guards the columns.*
- Commons logos show at brand level only, takedown on complaint (*overturn: `images logos --remove`,
  written then*); `app.cli images` becomes `images fetch` (*an alias*); derivatives are WebP (*a flag*).

## Questions for rian (none critical; each task proceeds under its assumption)

- Is the Pernod `Brand Logo` column covered by Adam's approval? (assumed yes; overturn: `chosen:
  false`, re-import.) Does a brand-level picture show on product cards, or only on brand and line
  pages? (assumed on cards, styled as a brand mark.) Fetch Wikidata logos before launch at all?
  (assumed the `--limit 3 --check` probe only in this run; the full `images logos` is rian's to run.)

## Owns

`services/imagery.py`, `image_manifest.py`, `images_public.py`, `images.py`, `image_ask.py`,
`app/cli_images.py`, `routers/image_ask.py`, the new migration, `import/images/`,
`.logs/runs/window-2026-09-19-images.md`, `Thumb.tsx`
(+ `Thumb.css`), `ImagesPage.tsx`/`.css`, tests for all of it; image-only lines in `cli.py`,
`models/catalog.py`, `schemas.py`, `hubs.py`, `catalog_queries.py`, `seo.py`, `merges.py`,
`audit.py`, `main.py`, `access.py`, `BrandPage.tsx`; `RUNBOOK.md` (images section), `DATA-MODEL.md`,
`VOCABULARY.md` (one row), `COLLECTORS.md` (one bullet), `SEO.md` (one line), `CHANGELOG.md`.

## Must not touch

Staging's and production's databases; `.env`, `.app.env`, `docker-compose.yml`; the notes folder
(read only, never moved, never committed); `.logs/runs/` except the images window file; `featured.py`, `airport_featured.py`
(AW2); collectors and identity rules; `decisions/`; the vendored kit; `check.sh`.

## Rules that bite here

- Never rehost retailer photography: brand-supplied and openly licensed only, each with a recorded
  source, licence and level (COLLECTORS.md L2; agents.md "Collect facts, not expression").
- Empty beats guessed, a wrong picture is worse than none: precision-first matching, ambiguous
  means unassigned with the reason, a brand logo never enters a product's structured data.
- A human value is never overwritten by a machine: `set_image` enforces it for every writer.
  Migrations schema-only with server defaults; every data move an idempotent `app.cli` command
  with `--check`, rehearsed on `dfp-devdb`, never on staging. A GET never writes; `resolve()` is pure.
- No image file is ever committed; `uploads/` is gitignored; the folder stays where it is; the
  manifest holds paths and URLs, not bytes. One word per concept (product variant, product line,
  brand, picture level; never bare "product"); client copy without em dashes, "cheap" or "free"; a
  component imports its own stylesheet; `main/check.sh` green before every commit; a bug found
  becomes a test with the real record; no deploy, no collection, no `git push`.

## Out of scope, noted

- An upload backend for Adam (the manifest and `store()` are its seam); the export's `Country Of
  Origin` and `Awards` columns; `raw.tsv`, superseded by `images coverage`; `image_checked` as is.