# Collectors

Sources of truth: `main/app/services/collectors/`, `main/app/cli.py`, `main/app/services/ingest.py`.

## How collection works
One shared pipeline: `fetch.py → collector.collect() → RawListing → ingest.run_collector →
normalize / taxonomy / fx → Product / Listing / PriceObservation`. **One collector per
commerce platform, airports as configuration rows**, plus bespoke modules for platforms seen
once. The registry (`collectors/registry.py`) is the list; `python -m app.cli status` is the
live source table. **No step uses an LLM or any AI.** Adding an airport is one configuration
line on an existing platform; a new JSON-API platform is a small module; server-rendered HTML
needs a bespoke parser; a client-rendered site needs the rendered fetch (Stream A2's section
below). Listing pages are harvested, not product pages; product pages are read one at a time
only to verify (`read_one`).

A staging refresh (`RUNBOOK.md`, Staging refresh) replaces every collection run, listing and
observation on staging with production's and keeps only what people wrote on the site; never
run one while a collection is in progress, and never point it at production.

## Robots policy
Applied at the top of every `collect()`, `read_one()` and `render()` by `collectors/robots.py`,
on every run, because permission can be withdrawn between runs and once was:
- The group is matched on the bot's own name (`DutyFreeProfessorBot`, case-insensitive
  substring), then `*`; the crawl-delay comes from the same group and the run uses the slower
  of theirs and ours.
- Modern wildcard matching; **any matching Disallow is a no**, even when a longer Allow would
  technically win.
- robots.txt itself: 404 or 410 → unrestricted, logged; **401 or 403 → a refusal**, recorded,
  run ends; 5xx or timeout → stop this run and retry next run (RFC 9309 §2.3.1; build plan
  §10 #13 ratifies).
- A block on any page (challenge, WAF, empty body behind a 2xx) is a refusal too: recorded,
  never worked around (`fetch()` raises `SourceBlocked`).
- Identity mode is honest by default. The rule (build plan section 4 item 11): a source
  switches to a browser-like identity only with a `permission_record` naming who agreed, when
  and how. Today `sources.identity_mode` and `permission_record` exist as columns and nothing
  reads them: every collector sends the declared identity and no code path can switch, so the
  rule holds by absence; the refusal the rule describes is an open issue for Stream A
  (`issue-identity-mode-is-a-column-nothing-enforces`), to land before any switch is possible.

## Per-platform recipes

Recipes exist below for the two rendered sources; the six text-fetch platforms (Avolta,
Shopify, ARI, Extime, Dubai, Heinemann and its platform shops) are documented by their
module docstrings and the posture table until their recipes are written
(`issue-per-platform-recipes-for-the-six-text-fetch-collectors`, Docs). Operational
companions: the per-source kill switch is `sources.enabled` (`POST /api/sources/{slug}/enabled`,
owner only; `python -m app.cli sources`), the no-network audit and the re-read check are
`python -m app.cli audit` and `verify` (`QUALITY.md`), and the skip reasons a run counts are
the strings passed to `report_skip()` in each collector (`no_price`, `no_offer`,
`not_a_product_page`, `reread_failed`, `variant_page_unreadable`, `not_offered`,
`multi_variant_unpriced`).
Stream A fills this: one subsection per collector module with the discovery method, the
pagination rule, where the price and currency are read from, and the known fragilities.

### iShopChangi, Singapore (`changi.py`, rendered)
Discovery: `/en/sitemap-products.xml` (about thirty thousand product URLs; the trailing
`mp` code is the identity), filtered to `import/changi-targets.json` unless the collector is
built with `full_catalogue=True` (rian's crawl token). Per URL: one render with
`assets.adobedtm.com` declared as the only asset host (the storefront bundle crashes without
its tag manager; beacon hosts stay aborted), waiting for a price element; the page's own
`pdp/inventory.json` response is read, never requested directly. Price: the `DEPARTURE`
channel of the variant's selected offer, accepted only when its string form reads `S$`
(`price_type` "departure"; Arrival is the same figure, non-traveller delivery is about a
fifth higher and is never published). Size from the name, else the bare sale-measure
qualifier; ABV from the `alcohol_percentage` classification; stock from the offer; category
from the breadcrumb. **No barcode anywhere**, so SIN rows match other shops on brand, name
and size only. The seller (a marketplace offer names one) is kept in the raw record.
Fragilities: the tag-manager dependency; three channels; a variant per size under one
product code. Tests: `tests/test_collectors_changi.py` on the real Johnnie Walker Blue Label
record.

### The Shilla Duty Free, Seoul (`shilla.py`, rendered)
Discovery: the home page (the one page that serves to a text client) and each product
page's related-product rail, walked breadth-first under the run's render budget (200) from
the liquor, skin care, makeup and fragrance trees; the sitemap is empty and the category
grids are drawn by scripts robots.txt disallows, so they are unreadable to a compliant bot
and their loader is not read either. Per URL: one render at `domcontentloaded` with every
asset request refused (robots); the document is complete without them. Price: the hidden
`prdPriceDollar` field, equal to the visible "Discount price" (`price_type` "discount"),
the "Price" tier as the crossed-out figure, the online-member tier kept in raw only.
Barcode from "REF.NO" (a UPC); the neighbouring "SKU.NO" is not one. Stock: the sold-out
block is in every page's markup and means sold out only when it is shown. Size from the
"Weight,Volume" specification row (a weight yields no size). Category from the page's
three hidden depth fields. Fragilities: the Cloudflare judgement (a challenge to the
browser one day is a refusal, recorded, never solved); robots.txt now names many AI and
rendering crawlers by name, which is a signal about intent worth raising in the
partnership ask. Tests: `tests/test_collectors_shilla.py` on the real NARS record.

## Rendered fetch
`fetch.render(url)` is the second door for sources that draw their prices with JavaScript
(a text fetch of such a page sees a shell and no price). It keeps the `fetch()` contract:
the same honest identity, robots re-read per run through `robots.py` (or the run's own
`Robots` passed in), the host's crawl delay with a **ten-second floor** where none is
published, validation on content never status (`cf-mitigated: challenge`, a challenge
title, a shell that drew fewer than forty characters → `SourceBlocked`), and a per-run
render budget (`render_budget(cap)`; the count is logged at exit; the default cap outside
a declared budget is small on purpose). A render is roughly fifty times a text fetch: it is
a per-URL tool for the comparisons that matter, never a mode for a whole source.

**Pace per source.** `render_wait` takes the slowest of the host's Crawl-delay, the run's
delay and the source's own floor: a rendering collector declares `render_floor_seconds`
(None keeps the general floor) when a host has shown it wants a slower pace, so the pace is
the source's declaration rather than a flag someone remembers; one run is slowed with
`collect --delay N` (or the source's `delay_seconds`), and every `rendered` log line carries
the pace used as `wait=`, so a change of pace and what followed it read together in the run
log. The wait is measured from the start of the previous render, so a page that takes nine
seconds to draw under a ten-second wait leaves the host about one second idle.

**The sidecar** (`main/browser/`, service `browser` in compose) is Playwright's bundled
Chromium behind a small HTTP server (`GET /health`, `POST /render`). Isolation is
structural: its own compose network with only the app on it (the database is unreachable),
no `env_file`, no published port, the image's non-root user, `cap_drop: ALL`,
`no-new-privileges`, and Chromium's own renderer sandbox ON, made possible by
`browser/seccomp.json` (Docker's default profile plus the user-namespace and `chroot`
calls the sandbox needs; measured 2026-09-05, `chroot` was the one Docker's default blocks).
Every render gets a fresh context (no cookies, no storage) and an **egress allowlist**:
`route('**/*')` aborts any request to an IP literal, loopback, private or link-local
address, or a host outside the source's origin plus its named asset hosts; aborted hosts
are reported back and logged once per render. It also captures the JSON bodies of the API
calls the page makes for itself (`Rendered.api_responses`), so a collector can read what
the page fetched without calling the API itself.

**What it never does:** click, type, scroll, or dismiss a cookie, age, terms or
shopper-mode dialog. If a price appears only after one, that is a STOP and an item on the
running list (`items.py add --kind issue`). And a challenge page is a refusal like any other: recorded, the source disabled,
never solved by other means.

`render()` also takes `wait_for` (a CSS selector the caller expects once the page has drawn)
and `settle_ms`; the sidecar reports the page's console errors and uncaught exceptions
(`Rendered.console`), which is how a shell that drew nothing explains itself. A refusal on
content raises `RenderRefused` (a `SourceBlocked`) carrying the `Rendered` it judged, so a
probe can still read the headers and captured API bodies. The host's own Disallow rules are
sent along as regexes and bind the page's subresource requests too (`aborted_paths`).

Probe a new source with `scripts/render-probe.py <url>` (announce it in `.logs/runs/`
first). Tests: `tests/test_render.py`. Local run of the sidecar for a probe:
`docker run` the image with the same flags compose declares and a port on `127.0.0.1`.

## Posture per source
Stream A fills this: robots read date, identity mode, permission record, and whether prices
are collected, frozen, or a partnership ask. Never a typed count; the numbers come from
`python -m app.cli status`.

| Source | Robots read | Identity | Access | Posture |
|---|---|---|---|---|
| Singapore, iShopChangi (SIN) | 2026-09-05: product paths permitted, no Crawl-delay (floor 10 s) | honest UA | client-rendered; the page's own inventory response read through the sidecar; tag-manager host declared | collector built (`changi.py`), targeted set seeded (`.logs/runs/changi-targets-2026-09-05.md`); first run after the sidecar deploy; no barcodes |
| Seoul, Shilla (ICN) | 2026-09-05: product paths permitted, Crawl-delay 5, assets under `/estore/_ui/`, `/medias/`, `/files/` disallowed | honest UA | text client is challenged by Cloudflare; the browser passes; document is server-rendered (read at `domcontentloaded`, assets never fetched) | confirmed possible (`.logs/runs/shilla-probe-2026-09-05.log`); collector built (`shilla.py`), discovery by home page + rails under a 200-render budget; first run after the sidecar deploy |

## Legal guardrails (researched 2026-08-21)

**Not legal advice, and rian is not a lawyer.** These are conservative engineering
practices drawn from current US case law. **Action item: 30 minutes with an actual
lawyer before LAUNCH — not before the PoC.** A few dozen products, logged-out,
facts-only, at trivial volume is a negligible-risk proof of concept; 168 airports
refreshed daily as a commercial product is a different conversation.

**L1. NEVER LOG IN. Never create an account, never accept terms of service, never
authenticate to a retailer site.** This is the sharpest practical line in scraping law.
In *Meta v. Bright Data* (N.D. Cal. 2024) Bright Data won summary judgment on Meta's
breach-of-contract claim specifically because it scraped **only while logged out** — the
terms didn't reach that conduct. Conversely *hiQ v. LinkedIn* won on the CFAA and still
**lost on breach of contract**. Contract, not copyright, is the realistic exposure, and
logging in is what creates the contract. Happy convergence: this is also why Club Avolta
member prices are unreachable (F4) — the technical limit and the legal safe harbour
point the same way. Treat "log in to get better data" as permanently off the table.

**L2. Take facts, not expression.** Price, SKU, size, ABV, currency, stock status,
brand and airport are **facts** — uncopyrightable under *Feist*, and an actually-charged
retail price sits at the *NYMEX v. ICE* end of the spectrum (held not copyrightable)
rather than the *CDN v. Kapes* end (derived/estimated price guides, held copyrightable).
**But marketing copy and product photography are creative expression and are the real
copyright exposure in retail scraping** — see *FragranceNet.com v. FragranceX.com*, a
direct e-commerce competitor-scraping case where photographic originality survived a
motion to dismiss. Build rules: store structured attributes; **do not ingest or
republish retailer marketing prose**; **do not rehost retailer product photography** —
use brand-supplied or licensed imagery, or ship without photos, and revisit images as a
partnership deliverable. Always **link and attribute to the source retailer** (good
product design anyway, and non-attribution is an element of the "hot news" theory).

**L3. Buying scraped data does not transfer the risk.** In *Reddit v. SerpApi*
(S.D.N.Y., 31 Jul 2026) the court held at the pleading stage that Perplexity — which
merely *purchased* SerpApi's output — could be a **direct** circumventer because it "set
the parameters for circumvention activity." Consequence for us: using Firecrawl or any
paid fetcher is a convenience and a capability, **not** a liability shield. Choose it on
engineering merit (D1), not in the belief that it moves risk off Adam.

**L4. The controlling question is genuinely unsettled, and it splits in our favour.**
Two federal courts reached **opposite results on the same anti-bot technology 11 days
apart**: *Google v. SerpApi* (N.D. Cal., 20 Jul 2026) dismissed DMCA §1201 claims
**without leave to amend** where the protected material wasn't copyrightable at all —
which is exactly the posture of bare price and SKU data; *Reddit v. SerpApi*
(S.D.N.Y., 31 Jul 2026) held defeating a bot wall by proxying, IP rotation and spoofed
user-agents **is** circumvention. **The best argument available to a price scraper is
therefore "the measure does not control access to a copyrighted work," not "our copying
was fair use."** L2 is what keeps that argument available — the moment we ingest
photography and prose, we hand the other side the copyrighted work it needs.

**L6. EUROPE INVERTS THE RANKING — the most counterintuitive finding, and the one most
likely to catch us out.** The controlling case is *Ryanair v PR Aviation* (CJEU
C-30/14, 2015) — **literally a flight price-comparison site scraping an airline**, i.e.
our exact product. Held: where a database is protected by **neither** copyright **nor**
the sui generis database right, the Database Directive's mandatory lawful-user rights
don't apply, so **the owner is free to impose contractual restrictions**. The paradox:
**an unprotected database gets MORE effective protection than a protected one.**
Concluding "prices are just facts, so no database right" does not clear us — it moves
us into the zone where the retailer's terms bite hardest. **In Europe, contract is the
primary exposure, not IP**, which makes L1 (never log in, never accept terms) the
single highest-value control in the whole build.

**L7. Architecture is legally constrained: crawl-and-index, NEVER live query
passthrough.** *Innoweb v Wegener* (CJEU C-202/12) held that a dedicated meta search
engine which **translates a user's query in real time into the target's own search
engine** and re-presents the results is unlawful re-utilisation — "very close to the
manufacture of a parasitic competing product." A periodic crawl feeding **our own
index** is materially different. So: never proxy a user's search into a retailer's
search. (*CV-Online Latvia* C-762/19 further requires an adverse effect on the owner's
investment, which gives a comparator that **drives referral traffic** a real defence —
another reason every price should link back to the retailer.)

**L8. Jurisdiction matters, and it is not uniform.** **Germany is favourable** — the
BGH held (Ryanair v Cheaptickets, 2014) that automated retrieval for display on a
comparison site is not unfair obstruction **even where terms prohibit scraping**,
because price transparency benefits consumers. **France is the highest-risk** —
*parasitisme* requires neither confusion nor lost turnover, and copying a competitor's
**product descriptions** has been held unfair on its own (note `aeliadutyfree.fr` is a
plausible target). **The UK has no commercial TDM exception at all** — s.29A CDPA is
non-commercial research only. And *Getty v Stability* [2025] EWHC 2863 turned partly on
Getty being unable to show the acts happened in the UK: **where our crawling
infrastructure physically sits is a legal fact, not just an ops detail.**

**L9. The closest analogues to this product are the airline cases, and they all turned
on the same thing.** In *American Airlines v Skiplagged* (N.D. Tex. 2024) the
**contract claim failed** and the **copyright claim succeeded** — via AA's **logo and
creative content**, not via fare scraping. *Southwest v Kiwi.com* likewise ran on
contract plus **trademark**. **Every plaintiff who won, won on pictures and branding,
not on prices.** This is the strongest possible confirmation of L2: scrape prices,
never imagery or marketing prose, and never reproduce a retailer's logo.

**L10. The retail baseline is genuinely favourable — and the scary airline cases don't
generalise to us.** There is essentially **no US case law of a general retailer suing a
price-comparison service over price data**. The retail baseline is instead:
- ***Rapaport USA v. Nivoda*** (S.D.N.Y., Rakoff, May 2025) — **the single best
  authority for this product.** A subscription diamond price-list publisher sued a
  retailer for displaying its prices alongside its own. **Held: prices are NOT
  copyrightable, under the merger doctrine** — "the only way to express the price of a
  particular diamond is with the specific number corresponding to that price."
  (Two Second Circuit appeals pending; watch them.)
- ***Meta v. Bright Data*** — terms construed **against the drafter** as an adhesion
  contract and held **not to reach logged-off scraping of public data**.
- ***X Corp v. Bright Data*** (Alsup) — a contract claim whose real object is
  controlling copying/resale of content is "camouflage" for a copyright claim and is
  **preempted**. A second line of defence behind L1.
- ***Ryanair v. Booking*** — a pure CFAA scraping case litigated **to a jury verdict**,
  where the airline recovered **nothing**: only $2,457.72 of loss survived *Van Buren*'s
  "technological harms" limit, below the $5,000 threshold. Appeal voluntarily dismissed
  Aug 2025, so it stands. At summary judgment: **public pages are not accessed "without
  authorization."**
- ***Intel v. Hamidi*** — trespass to chattels requires actual impairment.

**Four things make the airline wins work, and none applies to us:** (1) a *booking
relationship* — Kiwi bought 20,000+ tickets through clickwrap, Booking's vendor created
accounts and transacted through a password-gated page; (2) a *logo hook* — American's
entire $9.4M was its Flight Symbol, and its **contract claim actually lost** (time-barred);
(3) a *consumer-harm narrative* (inflated fares, hidden-city disruption); (4) *Contract
of Carriage* violations. A read-only comparison site that never transacts has none of
these. Note also the Skiplagged jury found **nominative fair use** for using the
airline's mark to identify whose fare it was — which is exactly our attribution case.
**Identify, don't decorate.**

**L11. Two operational controls with real legal weight.**
- **Rate-limit hard.** *QVC v. Resultly* is the only retail case in this set with real
  exposure on the facts, and it was **200–300 requests/minute spiking to 36,000**,
  crashing QVC's site. It was never about the price data. Politeness delays and low
  concurrency aren't manners here, they're the damages defence — civil CFAA needs
  ≥$5,000 loss and trespass needs actual impairment.
- **If anyone asks us to stop, stop that day.** A cease-and-desist is the moment
  unenforceable browsewrap becomes an enforceable contract (*Southwest v. BoardFirst* —
  actual knowledge plus continued use equals assent; it's how Southwest has won
  repeatedly). The *QVC* court pointedly noted there was **no allegation Resultly
  refused to stop when asked**. **Build rule: a per-source kill switch in the admin, so
  a collector can be disabled in seconds without a deploy**, and route any retailer
  contact to rian immediately.

**L5. Everything above is provisional.** Nearly every case expanding liability here is
a district-court motion-to-dismiss ruling, which decides only that allegations state a
claim. The only binding appellate decisions in the set — *Cox v. Sony* (SCOTUS, Mar
2026) and *Amazon v. Perplexity* (9th Cir., Aug 2026) — both **narrow** liability. Note
*Amazon v. Perplexity* does **not** help us: it turned on an agent running in the end
user's own browser under their own credentials. A collector hitting a retailer from our
server is *us* accessing their servers, squarely outside that holding.

## Robots policy (`collectors/robots.py`, Stream A, 2026-09-04)

One module, one identity (`BOT_NAME = "DutyFreeProfessorBot"`), one policy; every
`collect()` and `read_one()` opens with `check_allowed(base_url, paths)` and takes its
crawl delay from the returned `Robots.delay_for(requested)`.

- A group applies to us when its agent value is a case-insensitive substring of
  `BOT_NAME` or the reverse. Never the first User-Agent token (that misreading once let a
  run past a group addressed to us by name).
- Paths match RFC 9309 style (`*` spans anything, `$` anchors). **Any matching Disallow is
  a no**, in our group or `*`, and a longer Allow does not out-lawyer it.
- Crawl-delay: our group's, else `*`'s; the run waits the slower of that and its own delay.
- Unreadable robots.txt: 404/410 (and other plain 4xx) means no rules, proceed and log the
  date; **401/403 is a refusal** (`SourceBlocked`, run status `blocked`); 5xx, 429 or a
  timeout stops the run without calling it a block (`RobotsUnavailable`, run status
  `error`, retried next run).
- Per-URL checks inside a run use `robots.allows(url)` offline; no second read.
- Measured 2026-09-04: LHR asks 30 s, the other nine Avolta stores 60 s; LHR, ATH, MAD and
  BCN forbid `/*?`, so they are page-1-only. Tests: `tests/test_robots.py`.

## read_one (`Collector.read_one(listing)`, Stream A, 2026-09-04)

One published listing read back from its source, fresh, for verification. Takes a
database `Listing` or a `ListingRef(source_sku, url, location_code)`; returns a
`RawListing`, or `None` when the source no longer offers it (404/410, the page now serves
another SKU, the size is no longer priced); raises `SourceBlocked` when the host refuses
and `FetchError` when it should be retried. Opens with the same robots check as `collect()`
(cached ten minutes per host) and waits the host's crawl delay before the request.

| Collector | Reads | Pure function tested offline |
|---|---|---|
| Avolta | the product page: offer microdata for a plain SKU, `variant_prices()` for a `parent::size` row | `AvoltaCollector.listing_from_product_page` |
| Shopify | `/products/<handle>.json`, variant picked by id | `ShopifyCollector.listing_from_product_json` |
| ARI | the product page's JSON-LD (query string dropped) | `ari.listing_from_page` |
| Extime | the family page, the row with the same `source_sku` | `Extime._listings` |
| Dubai, Heinemann family | nothing: raises `SourceBlocked` (no permitted read path) | |

## Audit defects fixed in the first pass (Stream A, 2026-09-04)

Build plan §3a; every fix is a test in `tests/test_audit_defects.py` with the record that
failed. What changed at the collection layer:

- **Ingest refuses a barcode/size contradiction** (`RejectedListing("gtin_size_veto")`) instead
  of re-routing to brand+name+size, which once created two 7,000 ml Glenfiddichs. Rejections
  go through `record_rejection()`: counted on the run and stored in `rejected_observations`
  (migration #2, `d1e2f3a4b5c6`; the insert is guarded, so a schema mismatch cannot end a run).
- **`observed_at` is stamped per listing**, and each observation carries `fx_rate`; each run
  carries `fx_source` (live | fallback) and `fx_fetched_at`.
- **A source's earlier run still "running" is marked error when a new run starts.**
- **A location's currency follows its collector's `LocationSpec`** (`sync_location_currency`).
- **Dubai `in_stock` is None** unless the API item carries a stock field (`active` is not one).
- **Extime cross-checks the declared capacity** against `price_per_quantity` / `net_weight` when
  present and returns an unknown size on disagreement. The live shape of those two fields is
  pinned from a raw page (`tests/fixtures/extime_cheirosa_76.html`, 5 Sep); the older readings
  stay for the shapes seen before the fixture existed.
- **Image provenance** is `Open Food Facts (barcode)` or `(name)`; a name match requires the
  number tokens (ages, editions) to agree.
- Every `backfill <name>` takes `--check` (print what a run would change, write nothing).
- **Orphan parent tiles** (`-P` with no `::size` sibling) are purged by `backfill orphan_tiles`;
  the next collection of that store reads the product page and creates the per-size rows.

## Beauty widening (Stream A, 2026-09-05)

Decision 4: beauty is collected at every airport, targeted pages first and full crawls
last. The targeted set is `import/beauty-targets.json` (twenty fragrance and twenty skincare
folds, written by `scripts/beauty-candidates.py` from rows already held plus feeds already
fetched; the list with its evidence is `.logs/runs/beauty-candidates-<date>.md`). It is
mounted read-only into the container like the other import files and read by
`collectors/targets.py`; **an absent or empty file means no beauty product pages, never all**.

| Platform | Drinks | Beauty | Product-page fetches |
|---|---|---|---|
| Avolta | its own slug list, paged until a page adds nothing | its own slug list, **page 1 only** | a multi-size family only when `targets.matches(name, brand)`; drinks and beauty have separate budgets (`MAX_VARIANT_LOOKUPS`, `MAX_BEAUTY_LOOKUPS`) |
| ARI Dublin | every `/alcohol/` URL in the product sitemap | `/beauty/` (and sibling prefixes) only when the URL path names a targeted line (`ari.wanted_urls`; the slug omits the brand, so line words plus size) | one per wanted URL, as before |
| Shopify | the shop's shelf names, English and Spanish (`shopify.shelf_vertical`: Licores, Vinos, Rones, Champanas, Cordiales, Icewine, Eau de Vie...) | every beauty shelf (Fragancias, Rostro, Cuerpo, Capilar, Women's Perfume, Men's Cologne...): the feed arrives whole either way, so keeping every row costs no request | none (the feed is the read) |

- **Two trees, two caps, at Avolta.** Each tree is capped at `MAX_CATEGORIES_PER_VERTICAL`
  and a cap that drops anything logs `categories_capped store=<code> dropped=[...]`. The
  old single alphabetical cap once dropped whisky and wine at Heathrow because "beauty"
  sorts first.
- **Page-1-only stores re-read what they hold.** Where robots forbids the paging query
  string (LHR, ATH, MAD, BCN carry `Disallow: /*?`), the collector detects it
  (`page_one_only`), walks page 1, then re-reads every product URL the database already
  holds for the store that the walk did not reach (`AvoltaCollector._reread_held`; ingest
  hands the URLs over through `wants_held_listings`), one fetch per URL, bounded by
  `MAX_HELD_REREADS` and logged as `held_reread store=... unseen_urls=N reading=M`. The
  shrink at such a store is `prices_written` of its run against the previous one, plus the
  `held_reread_capped` line if the bound bit.
- **Shop-hidden products are not on the shelf.** Shopify tags such as `auto-oculto` and
  `Apagar` skip the product (`not_offered`, counted as a collector skip).
- **Extime's capacity is cross-checked against the live shape**: each price tier carries
  `price_per_quantity` and `price_per_quantity_type` ("100ml"); a declared capacity that
  disagrees with the implied size beyond the 700-vs-750 tolerance becomes an unknown size.
  The page that failed (a mist declared at 7,624 ml) is `tests/fixtures/extime_cheirosa_76.html`.
  `backfill implausible_sizes` clears sizes already stored that no single item comes in
  (`normalize.size_is_implausible`).