# SEO and AEO

Sources of truth: `main/app/services/seo.py`, `main/app/services/urls.py`, `main/app/main.py`,
`main/app/services/publish.py`, `main/app/routers/pages.py`,
`main/app/models/hubs.py`, `main/app/services/robots_policy.py`, `main/app/services/feeds.py`,
`main/app/services/indexnow.py`, `main/app/cli_pages.py`, `main/web/src/lib/urls.ts`,
`main/web/src/lib/structure.ts`.

The machine-readable layer (title, description, canonical, social tags, JSON-LD) is stamped
server-side into the SPA shell for every page. Product pages also get a server-rendered body
that matches the SPA's default state exactly; anything else is cloaking (`seo.py` docstring).
Client surfaces are noindexed, and their writes are the only ones open past the shared
password: every other write route is closed by the default-deny middleware
(`CLIENT-SURFACES.md`, `app/services/mutations.py`). Page-type decisions and their evidence
are on `/structure`; the plan is build plan §6.

## Which shops the site shows
Every catalogue query, `/api/stats`, the sitemap and the IndexNow list scope their shops through
`catalog_queries.publishable(db)`: visible shops (Decision 1) minus any shop whose source
verify has blocked (Decision 10, `catalog_queries.blocked_shop_ids`, fed by
`verify.blocked_sources`). A blocked shop is hidden everywhere at once until a human clears
the check (`app.cli verify-clear`): its prices leave every card and price table, its airport
page is a 404 like a hidden airport's, its product variants leave the sitemap. Nothing else hides a
row; a refusal or a moved price keeps its date. The `/trip` comparison and the coverage page
apply the same rule (`tests/test_trip_publication_block.py`; the aliased pair query takes
`blocked_shop_ids` by hand). Tests: `tests/test_publication_block.py`; rehearsal 2026-09-05 on a
scratch copy: one MISMATCH_SIZE on a JFK listing removed JFK from stats, browse, a product variant's
price table, the airports index and page, the dataset facts and the sitemap; clearing it
restored every figure exactly.

## URL scheme and redirects
- Every URL shape lives in `app/services/urls.py`, mirrored by `lib/urls.ts`; the two must
  agree or server canonicals fight client links (`tests/test_seo.py::TestSlugs`,
  `tests/test_seo_airport.py::TestAirportUrls`). `seo.py` re-exports the helpers.
- Product: `/products/<slug>-<id>`. The trailing number is the identity; the words are for
  humans.
- Category at airport: `/airports/<airport>/<category-slug>` (`airport_category_path`): the
  airport's own address, then the category's word from the declared table `CATEGORY_SLUGS` in
  `urls.py`, mirrored exactly by `lib/urls.ts` (`tests/test_airport_category.py` pins the two
  equal). A page exists only where the pairing is at or over `coverage.CATEGORY_AT_AIRPORT_MIN_PRODUCTS`
  published product variants, read at request time; under it the address is a real 404 and absent from
  the sitemap. The airport part canonicalises like the airport page (a stale slug 301s, keeping
  the query); the category word must be exact. The shopping-feature flags, the sort and the page
  are views for people; every one names the clean pair address as its canonical.
- The airport page's shopping-feature chips (`?multi_only|awarded_only|exclusives_only=true`, the
  names `/api/products` takes) and the comparison preset (`/savings?from=<IATA>`) are views too:
  the clean airport address stays the canonical of every filtered view.
- A bare id (`/products/60`) or a stale slug 301s to the current canonical path.
- Airport: `/airports/<common-name>-<iata>-<city>` (`airport_path`), the review's shape:
  the airport's common name leads because it is what people search ("heathrow", not "lhr"),
  then the code, then the city unless the name already says it (`/airports/heathrow-lhr-london`,
  `/airports/hong-kong-hkg`, `/airports/paris-cdg` for CDG and Orly together). JFK is the one
  airport known by its code (`/airports/jfk-new-york`); any other exception is Adam's to name.
  The name is declared data: one table keyed by IATA, `AIRPORT_SLUGS` in `urls.py`, mirrored
  exactly by `lib/urls.ts` (`tests/test_seo_airport.py::TestAirportSlugTable` reads the TS
  file and pins the two equal). The IATA code stays the identity; the words never come from
  a request. An airport with no entry keeps the earlier `<iata>-<city>` shape from its
  `shops` row so nothing 404s. `parse_airport_slug` resolves, in order: a declared slug
  exactly; a declared code leading the slug (the bare code, the earlier `lhr-london`); a
  declared code in any later segment (a stale new-shape slug); an undeclared code leading
  it. A three-letter word is never read as a code unless it leads or is declared
  (`san-salvador-sal` is SAL, never San Diego). Every shape that resolves and is not the
  canonical path 301s to it, keeping any query string; an airport we do not show (hidden,
  catalogue-only, no stock) is a 404. The API sends every airport link ready-made
  (`AirportSummary.path`, `PriceOut.shop_path`, `CoverageShop.path`), so the SPA
  rebuilds no address; the TS table serves `airportIataFromSlug` on the airport page. Filtered
  and paged views (`?category=`, `?sort=`, `?page=`) render server-side too but canonicalise
  to the clean page: they are for people. `page` and `sort` are read leniently (garbage means
  page one, featured), never a 422 where a page should be.
- Brand: `/brands/<slug>` (`brand_page_path`), the slug being `brands.slug` from migration #3,
  never a string folded from a name. An alias row (`alias_of_id` set) and a differently cased
  slug 301 to the brand's path, keeping the query; an unknown slug, or a brand under the floor,
  is a 404 and has no sitemap entry. The floor (review, 9 Sep) is `BRAND_PAGE_MIN_PRODUCTS`
  (three) published product variants priced, between them, at `BRAND_PAGE_MIN_AIRPORTS` (two) distinct
  airports by IATA; read that way because the page is a comparison, and stated on the running
  list for Adam to correct in one line. `catalog_queries._brand_counts` is the one definition,
  shared by the index, the page, the sitemap and the product page's brand link. Filters and paging as on airport
  pages. `brand_path(name, slug)` / `brandPath(name, slug)` link to the page when the API handed
  over the slug and fall back to a catalogue search otherwise. `ProductSummary.brand_slug`
  (`catalog_queries.brand_page_slugs`) is that slug, and only for a brand `list_brands` would
  list, so the product page's brand name (body and SPA) reaches the page and can never 404.
- Article: `/articles/<slug>` (`editorial.article_path`, the row's slug verbatim; Stream D
  owns the slugging) with `/articles` the centre they sit in. A draft or unknown slug is a real
  404 exactly like a slug that never existed, so nothing about unpublished work is inferable
  from outside. The centre takes `?page=` and `?tag=`, both read leniently and both
  canonicalising to `/articles`: a page number and a tag are ways of walking one room, not
  addresses of their own.
- Family landing: `/alcohol`, `/beauty` (`family_path`), the word being the shopper's search
  term rather than the column name (`liquor` is `alcohol` in an address), from the declared
  table `FAMILY_SLUGS` in `urls.py`, mirrored exactly by `lib/urls.ts`. One concrete route per
  word, never a parameterised `/{slug}`: the access policy classifies by route key before any
  handler runs, so a one-segment pattern would silently reclassify `/docs`, `/login` and every
  member-only SPA path as a public storefront page.
- Category: `/alcohol/whisky` (`category_page_path`): the family's word, then the category's
  own. Nested under the family rather than at the root, where a category word could collide
  with a brand name. A category addressed under the wrong family 301s to its own address,
  keeping the query. Filtered and sorted views canonicalise to the clean page like every
  other list.
- A category page exists from the moment the category has a published product, a declared
  word and a family with a landing; no floor decides existence (plan W18). `category_path` /
  `categoryPath` therefore point at the page for every shown category in a family with a
  landing, and keep the old `/products?category=` filter only for the rest.
- Both are `noindex, follow` until a person approves the address (`publish.INDEXED_PAGES`),
  and only an approved address reaches the sitemap (`indexed_category_rows`). They are the one
  page kind with no row of their own, so the approval is a declared constant rather than a
  ledger decision; move it into the ledger the day categories become rows.
- How we choose what is featured: `/how-we-choose` (`seo.METHOD_PATH`), a page route ahead of
  the catch-all like `/data`, whose body is `main/docs/FEATURED.md` read per request
  (`process_doc.read_doc("FEATURED")`, the same dict `GET /api/featured/method` returns and the
  SPA is seeded with as `window.__DFP_METHOD__`); an absent or unversioned doc is a real 404 on
  both. `noindex, follow` until a person names the address in `publish.INDEXED_PAGES`. Every
  featured rail's head carries `<a class="how-we-choose" href="/how-we-choose">How we choose</a>`
  (`seo.METHOD_LINK`; the body tests assert it exactly when the rail is a featured one).
- The privacy policy and the terms of use: `/privacy` and `/terms` (`seo.LEGAL_PATHS`), one
  concrete page route each ahead of the catch-all, whose body is `main/docs/legal/<name>.md`
  read per request (`process_doc.legal_page`, the same dict `GET /api/legal/{name}` returns and
  the SPA is seeded with as `window.__DFP_LEGAL__`); an absent or unversioned doc is a real 404
  on both. `noindex, follow` until a person names the address in `publish.INDEXED_PAGES`. The
  footer's bar links both.
- Every merged product variant 301s to its survivor through `catalog_queries.resolve_variant_id()`,
  which the page route and both product API routes already call (the API keeps the query
  string), following `product_variants.merged_into_id` (migration #3).

## What renders server-side, and how to verify it
- Every route: head only (title, description, canonical, OG, JSON-LD, noindex where due).
  Titles and every served string follow the house style, no em dash among them
  (`DEFAULT_TITLE`, `tests/test_house_style.py`, `tests/test_seo_body.py`).
- `/products/<slug>-<id>`: the whole body. `product_body()` mirrors `ProductPage.tsx` with
  no airports chosen: header nav, hero, meta line, medals, verdict, price table. Each helper
  names the component it mirrors in a comment block in `seo.py`; change one side, carry the
  other. The detail object is seeded as `window.__DFP_PRODUCT__` so React's first commit is
  the finished page (no skeleton, no product request). The seed is the no-airport medal
  order; a shopper with saved airports gets one request with `at=` for their order
  (`useProduct`), the seed standing in as placeholder meanwhile, so the served body is
  never swapped for the skeleton.
- `/how-we-choose`: the whole body (`method_body()` mirrors `HowWeChoosePage.tsx`: the hero
  with the lede and the version line, the "On this page" list, one `method-section` per heading,
  the factors table as the doc's own table). Verify by `curl -s https://<host>/how-we-choose |
  grep -E '<h1|method-version'`: the h1 and `Version N, <date>` are the strings only this page
  carries (`tests/test_site_routes.py::TestMethodPage`).
- `/privacy` and `/terms`: the whole body (`legal_body()` mirrors `LegalPage.tsx`: the hero with
  the opening paragraph and `Last updated <date>`, the "On this page" list, one `legal-section`
  per heading). Verify by `curl -s https://<host>/privacy | grep -E '<h1|legal-updated'`
  (`tests/test_site_routes.py::TestLegalPages`).
- Omitted on purpose: the "On your trip" block (the shopper's airports live in their
  browser), the related-products rail (a second request after mount), the footer.
- The header (`header_html`) is `SiteHeader.tsx` closed: announcement bar, brand, the no-JS
  search form (`action=/products`, the one deliberate difference from the SPA's; the input is
  SiteSearch's combobox, closed, with its empty hidden suggestion list), the airports
  button and the profile circle drawn signed out (the SPA swaps in the initials at the same
  size), the nav with its four departments (each a `site-nav__group`: the department's link
  and its panel toggle), the rest of the row after a divider and My savings at its end, and
  the panel's empty hidden shell (`#site-mega-menu`), exactly what the SPA renders before a
  panel opens; its links are never pre-filled server-side, because a visitor does not get
  them until they open it. The active item follows the SPA's: Airports on an airport page,
  Articles on an article, none on a product page. A header change in the SPA is a mirror change here the same
  day (`tests/test_seo_body.py::TestHeaderMirror`).
- `/airports/<common-name>-<iata>-<city>`: the whole body. `airport_body()` mirrors `AirportPage.tsx` with
  no airports chosen: breadcrumb, hero with the lede sentence, the facts panel (summary, the
  standing facts, the four counts), the terminal rows where a guide is written, the write-up,
  then the one shelf of products the URL asks for — the savings (the default), the travel
  exclusives, or the full list with its category chips, sort control and pager
  (`product_card_html()` mirrors `ProductCard.tsx`). `?tab=` names the shelf and a filter, a
  sort or a page implies the full list (`urls.airport_tab`, mirrored in `lib/urls.ts`); every
  shelf carries the clean page as its canonical, so they never compete with it. Every figure comes from `catalog_queries.airport_detail()`, the same object
  `GET /api/airports/{iata}` returns and the SPA is seeded with (`window.__DFP_AIRPORT__`,
  used only when the seeded view matches the URL's filters). Verified by diffing the served
  root against headless Chrome's DOM of the same URL: identical apart from the header's
  no-JS search form attributes and react-router's `active` class.
- Verify without JavaScript: `curl -s https://<host>/products/<slug>-<id> | grep '<h1'` must
  show the name, and `price-table__usd` rows must carry prices. The pure renderer is pinned by
  `tests/test_seo_body.py`; the routes by `tests/test_site_routes.py` (no database).
- Performance is measured, not assumed: Lighthouse mobile, local, gzip on both sides.
  2026-09-04: CLS 0.003 to 0, LCP 3.2s to 3.0s, performance 93 both. 2026-09-10, home page
  after the airport picker drew loading placeholders at the rows' final height: CLS 0.017 to
  0.001, LCP 2.4s, performance 97. Re-measure after any change to the header mirror or the
  hero; a header of a different height shifts the hero on mount. A local server caches the
  shell at startup, so restart it after a rebuild or Lighthouse reports no paint at all.

## Structured data per page type
- Home: `Organization` (`@id /#organization`, name, url, logo; `sameAs` once the client
  confirms handles) and `WebSite` (`@id /#website`, publisher). No `SearchAction`: Google
  retired the sitelinks search box in November 2024.
- Product: `Product` with `@id`, `url`, `description` (the same factual sentence as the meta
  description), `brand`, the barcode as `gtin8/12/13/14` by width, `image` (only when the
  picture shown is the bottle's own or its product line's, `image_level` variant or line; a
  brand mark standing in for a bottle is never a product's `image`, nor its `og:image`, and the
  Brand node on a brand page carries it as `logo` instead; Stream AW3), `category`,
  `award`, and `offers` as an `AggregateOffer` (`offerCount`, `url`) holding one `Offer` per
  shop in the shop's own currency with `seller`, `availableAtOrFrom` and the shop's URL. The
  price of record (review, 9 Sep): the shop's own price in its own currency is the price on
  the page and the only price in the markup; the combined offer states `priceCurrency`,
  `lowPrice` and `highPrice` only when every shop quoted one currency, from those quotes, and
  carries no range otherwise. Our USD conversion never enters the markup: on the page it sits
  second, labelled as our conversion at the rate held on the date checked. Airport and brand
  pages carry no offer markup at all (`tests/test_seo_body.py::TestOfferMarkup`). `availability` appears only when a stock flag was
  observed: any InStock wins, all OutOfStock says so, unknown is omitted, never defaulted.
  `aggregateRating`/`review` are deliberately absent: there are no reviews.
- Product: `BreadcrumbList` (Products, category, name) with absolute `item` URLs.
- Airport: `CollectionPage` (`@id <path>#page`, `url`, `name`, `description`, `isPartOf` the
  WebSite) whose `about` is an `Airport` (`name`, `iataCode`, and a `PostalAddress` holding
  only the locality and country the `shops` row has: no street, no coordinates, because
  we do not hold them) and whose `mainEntity` is an `ItemList` (`numberOfItems` = the
  airport's product total; one `ListItem` with `position`, `url`, `name` per card on the
  served page, positions continuing across pages). Plus `BreadcrumbList` (Airports we price,
  the airport).
- Brand and category hub pages (B5, B6): `ItemList` + `CollectionPage` the same way, with
  `Brand` on brand pages.
- Brand: `CollectionPage` (`@id <path>#page`, `isPartOf` the WebSite) whose `about` is a `Brand`
  (`@id <path>#brand`, `name`, `url`) and whose `mainEntity` is an `ItemList` as on airport
  pages; `BreadcrumbList` (Products, the brand). Body: `brand_body()` mirrors `BrandPage.tsx`:
  breadcrumb, hero with the lede (product variants, airports, comparable count and the airport most
  often cheapest, all counted), four counts, "Where to buy" (one row per airport that stocks
  the brand with its product variant count, how many comparable bottles it is cheapest for, checked
  date), category chips only for a brand with several categories, then the shared list section
  (`_list_section`, also the airport page's). Data: `catalog_queries.brand_detail()`
  (`GET /api/brands/{slug}`, seeded as `window.__DFP_BRAND__`); `list_brands()` feeds the
  sitemap and `GET /api/brands`.
- Article: `Article` (`@id <path>#article`, `url`, `mainEntityOfPage` the WebPage, `headline`,
  `description`, `datePublished`, `dateModified` = the last edit, `articleSection` = the
  category when set, `image` = the hero when set, `publisher` the Organization, `isPartOf` the
  WebSite; no `author` until the rows carry a name) plus `BreadcrumbList` (Articles, the
  piece); `og:type` is `article` there and `website` everywhere else.
- `/data`: `Dataset` (`@id /data#dataset`, `creator`/`publisher` the Organization,
  `isAccessibleForFree`, `temporalCoverage` first to last observation, `dateModified`,
  `spatialCoverage` one `Airport` per airport page, `variableMeasured` from
  `DATASET_VARIABLES`, `measurementTechnique` from `DATASET_METHOD`, `size`). No `license` and
  no `distribution`: neither is decided (running list), and an invented one is a promise.
  Plus `BreadcrumbList`. The body (`dataset_body`) mirrors `DataPage.tsx`; every figure is
  `catalog_queries.dataset_facts()` (`GET /api/dataset`, seeded as `window.__DFP_DATASET__`).

- Organization (`organization_jsonld`): name, url, logo, and `sameAs` for the social profiles
  that exist. The profiles have one home, `SOCIAL_INSTAGRAM_URL` / `SOCIAL_YOUTUBE_URL` in
  `.app.env` (`settings.social_profiles`: https only, placeholders ignored): the shell's
  `window.__DFP_SOCIAL__` (the SPA's header and footer links), the header mirror and `sameAs`
  all read it, so nothing can show a profile the markup does not claim, or the reverse.

Every URL in JSON-LD is written site-relative and made absolute at apply time
(`absolutise`), so heads can be module constants and still carry the configured origin.

## Origin, caching, conditional requests
- `PUBLIC_BASE_URL` (in `.app.env`) is the origin for canonical, sitemap and JSON-LD URLs.
  Outside production an unset value falls back to the Host header; in production it does not,
  and the startup log warns. Set it before the first production deploy.
- HTML and JSON are `Cache-Control: no-store` so a CDN never serves an owner's edit late.
  `/assets` and `/fonts` are immutable for a year; `/medals` and `/flags` for 30 days.
- HTML, `sitemap.xml` and `robots.txt` carry a weak `ETag` and `Last-Modified` (a product's is
  its newest observation; the sitemap's is its newest lastmod) and answer `If-None-Match` /
  `If-Modified-Since` with 304. HEAD is supported on all of them.
- `/articles/<slug>`: the whole body. `article_body()` mirrors `ArticlePage.tsx`: eyebrow,
  title, standfirst, the published/updated line (`updated_after_published`: a later UTC day,
  as `formatDate` counts it), hero figure, the rendered Markdown in `<div class="prose">` (one
  renderer, `services/markdown.py`, so the API and the body cannot differ), the "All articles"
  link. The piece is seeded as `window.__DFP_ARTICLE__` (`api/editorial.ts` reads it) and
  `ArticlePage` ships in the main bundle so React's first render redraws the body rather than
  a Suspense blank. Omitted on purpose: the subscribe form (a JavaScript form whose ids React
  mints at mount) and the footer. A `sample` piece prints one line above its head
  (`seo.SAMPLE_MARK`), so nothing the launch step will take to draft reads as the client's own
  text. Omitted too: the "What this is about" rail (the brand's page and a few of its product
  lines from `GET /api/brands/<slug>`, the category page, the piece's tags), fetched after
  mount like "More from the Professor".
- `/articles`: the whole grid. `articles_body()` mirrors `ArticlesPage.tsx` (`_article_card`
  mirrors `ArticleCard.tsx`): the ruled head, the tag rail, the newest piece large on page one,
  the grid, the pager with the tag kept in every link. The page is seeded as
  `window.__DFP_ARTICLES__` for the view it drew, and the SPA takes the seed only when the page
  number and the tag match. Its head is a `CollectionPage` whose `ItemList` names the pieces on
  the page (headline, url, datePublished, image when there is one) plus a `BreadcrumbList`.
  Tag chips on a card are spans, not links: the card is one anchor already, and an anchor
  inside an anchor is not a link.
- An airport's published write-up rides in `AirportDetail.writeup` (`editorial.airport_writeup`),
  so `airport_body()` and `AirportPage.tsx` both draw it from the same object, with
  `EditorialBlock.tsx`'s classes (`editorial_block_html()`), after the shops line and before
  the savings cards; no second request, no shift. Category intros wait for B6.
- The written guide to an airport's duty free (where the shops are by terminal, the specialty
  boutiques, a service, a map link) rides in `AirportDetail.guide` from
  `services/airport_guides.py`, in code because the data holds no terminals;
  `airport_facts_html()` and `airport_terminals_html()` mirror `AirportGuide.tsx`'s two
  components class for class. The demo's placeholder view, which marks the parts
  of the template nobody has filled (`web/src/lib/airportTemplate.ts`), is a browser setting and
  never reaches the server body: a crawler sees the page without the part, never a note about it
  (`tests/test_seo_airport.py::TestAirportGuide`).
- Unknown product ids and unknown routes are real 404s (the shell with a noindex head, so the
  SPA draws its not-found page). `is_known_route` mirrors `App.tsx`;
  `tests/test_seo.py::TestRouteInventory` reads `App.tsx` and fails on drift.

## The crawl surface (robots, IndexNow, feed, llms.txt)
- `/robots.txt` is `robots_policy.robots_txt()` once `SITE_ACCESS=public`; while the site is
  members-only (`ACCOUNTS.md`) it answers `User-agent: *` / `Disallow: /` with no Sitemap line,
  the machine files 404, and every response carries `X-Robots-Tag: noindex, nofollow`, so nothing
  under a product URL is indexed before launch. Public: a `*` group with the three Content Signals
  (`search=yes, ai-input=yes, ai-train=no`), `Allow: /`, `Disallow: /api/`; then one group per
  named bot whose crawl serves a signal (`NAMED_BOTS`), allowed iff any of its signals is on.
  The stance is the one dict `CONTENT_SIGNALS`; flipping `ai-train` flips GPTBot, ClaudeBot,
  CCBot, Bytespider, Applebot-Extended, cohere-ai with it. Google-Extended and
  meta-externalagent also govern grounding, so they list `ai-input` too and stay open. The
  file is written to the same standard the collectors' `robots.py` reads by (a bot addressed
  by name gets its own group); `tests/test_crawl_surface.py` parses our file with our matcher.
  Zone rule (Stream E's checklist): Cloudflare's managed robots.txt and AI-bot blocking stay
  off, or the edge overrides this file.
- IndexNow (`indexnow.py`, `cli_pages.py`): `python -m app.cli_pages indexnow --since <ISO>`
  (or `--hours N`, `--dry-run`) inside the container after a collection. Changed URLs come
  from the database, not from a list a collector keeps: products with an observation after
  `since`, the airport pages whose shops gained one, and the sitemap. One POST per 10,000 URLs
  to `api.indexnow.org`; the key is `INDEXNOW_KEY` in `.app.env` (any alphanumeric string we
  mint, not a secret) and is served at `/<key>.txt` only when set. Without `PUBLIC_BASE_URL`
  or the key the command lists and exits 2. The subcommand is registered in `app.cli`
  (`register_pages(sub)`), so `python -m app.cli indexnow` is the same command.
- `/feed.xml` (`feeds.rss_xml`): RSS 2.0, one channel newest first: the fifty product variants most
  recently added to the catalogue at a shop the site shows (`catalog_queries.newest_product_variants`,
  `pubDate` = the product variant's `created_at`) and the twenty newest published articles
  (`editorial.feed_items`, `pubDate` = `published_at`, the description never the body), absolute
  links, `lastBuildDate` and `Last-Modified` = the newest item. Every head carries
  `<link rel="alternate" type="application/rss+xml">`.
- `/llms.txt` (`feeds.llms_txt`): markdown from `dataset_facts()`: what the site is, how to read
  a price, the live counts, every airport page with its count and checked date, the
  categories, the machine surfaces and the reader's identity page. `Last-Modified` = the
  newest observation. No number in it is typed (`TestLlmsTxt::test_no_typed_numbers`).
- All of these answer HEAD and conditional GET like the pages (`conditional()` in `main.py`).

## The structure proposal page
`/structure` (`lib/structure.ts`, `components/SiteMap.tsx`) is the page Mark comments on. Its
figures are measured at the sixteen airports collected today and say so; the launch set is
nineteen. The published-pages count and switched-on airport count are read live from
`/api/stats`, never typed. House style there: no em dashes, never "cheap" or "free".
Page rows carry one of three states (`PageStatus`): `live`, `built` (the code has landed and
waits for rian's deploy; drawn like a live node, tagged BUILT, "Built, waiting for the next deploy"
in the table) and `build`. Flip `built` to `live` at the deploy checkpoint, never before,
because Mark reads the page as fact.

## Sitemap
- In: `/`, `/products`, `/savings` (when the my-airports feature is on), `/exclusives`,
  `/awards`, `/airports`, `/data`, every airport page (`catalog_queries.list_airports`),
  every brand page (`list_brands`, brands that meet the floor), each with
  `lastmod` = its newest observation, every published article (`editorial.sitemap_rows`,
  `lastmod` = its last edit), and every product variant with at least one observation at a shop the
  site shows (`publishable`), with `lastmod` = its newest observation.
- Out: noindexed client surfaces (`/discuss` and its tabs, `/settings`, `/plan`, `/issues`,
  `/sources`, `/collectors`; the moved `/todo`, `/structure` and `/quote` answer a 301 before any
  head, `CLIENT-SURFACES.md`), `/trip`, `/feature/*`, hidden or blocked shops' product variants, thin
  brands, filtered or paged hub views. Category pages join when they ship (B6).
- `/articles` is the one hub that waits: it is `noindex, follow` and out of the sitemap until a
  person names it in `publish.INDEXED_PAGES`, while the pieces inside it index on publish. One
  list, `seo.static_sitemap_paths()`, is read by the sitemap and by `publish.status_of_path`,
  so the file and the badge can never disagree about it.

## Page status: what a signed-in admin sees

A signed-in account holding `pages.status` (the owner and the `admin` level, `ACCOUNTS.md`) sees, in
one corner of every page, which of four words describes the address it is standing on. The words are
the app's own, and each is a test rather than a feeling:

| Word | What it means | Which pages |
|---|---|---|
| **indexable** | No robots directive, and the address is in the sitemap. | The static hubs, every published article, and a brand, product line, place, family or category page a person approved. |
| **noindex** | Reachable and linked, `noindex, follow`, out of the sitemap. | Every generated page until a person approves it, and `/articles` until it is named in `INDEXED_PAGES`. |
| **unlisted** | Reachable only by direct link, `noindex, nofollow`, never in the sitemap. | The client, owner and account surfaces in `STATIC_HEADS`, the explainer pages, the not-found shell. |
| **hidden** | The address answers a redirect, so nobody stands on it. | A hidden brand, product line or place 302s to `hidden_forward`; an alias, a retired slug and an old product variant address 301. The status names the target. |

An address that answers a real 404, a draft article or an unknown route, is `missing`: not a page, so
not one of the four. A category at an airport is that too under the coverage bar, whatever its
airport's decision, because the pair has no page there. Two facts ride beside the word:
`in_sitemap`, read from the list the sitemap itself builds rather than worked out a second time
(an address a person named is still out of the file where no page qualifies), and whether the site
is open.
While `SITE_ACCESS` is `members` every response carries `X-Robots-Tag: noindex, nofollow`, so the
word says what the address would be once the site is public and the badge says "members-only:
nothing indexes until the flip" ahead of it.

`publish.status_of_path(db, path)` is the one resolver: pure over rows and the declared tables, it
calls no head builder and writes nothing (a GET must never write). `GET /api/pages/status?path=`
(`app/routers/pages.py`) is the read, refused to anyone without the permission.
`web/src/components/PageStatusBadge.tsx` draws it, mounted once at the root and null unless
`/api/bw/me` reports the capability. No head builder, no `header_html` and no server-rendered body
ever carries the badge (`tests/test_page_status.py` greps for it): a crawler and a visitor are
served the same page whoever is signed in, and the resolver is pinned against the head builders'
`noindex` and `follow` and against membership of `sitemap_entries`, so the word cannot drift from
what the page actually sends.

## Build plan §6 corrections
| Correction | Status |
|---|---|
| Server-rendered product body | done 2026-09-04 (B1) |
| `PUBLIC_BASE_URL`; absolute `url`, `offers.url`, breadcrumb `item`, `@id` | done (B2); value to be set in `.app.env` |
| gtin by width | done (B2) |
| availability omitted when unknown | done (B2) |
| per-shop `Offer` with `priceCurrency` | done (B2) |
| `Organization`; `WebSite` without `SearchAction` | done (B2); `sameAs` pending handles |
| real 404s, HEAD, ETag/Last-Modified, `no-store` | done (B2) |
| favicon and manifest | done (C, 11 Sep): `favicon.ico`, PNG icons, `site.webmanifest` in `web/public`, pinned by `test_tokens_contrast.py` |
| IndexNow, per-bot robots with Content-Signal, RSS, `llms.txt`, Dataset page | done (B7); `INDEXNOW_KEY` to be set in `.app.env` |
| airport pages with ItemList/CollectionPage/Airport | done (B4) |
| brand pages with ItemList/CollectionPage/Brand on the brands table | done (B5); product pages link the brand name to its page through `brand_slug` |
| family landings and category pages with ItemList/CollectionPage/BreadcrumbList | done (B6); `noindex` until an address is approved |
| category-at-airport pages | built, behind `FEATURE_CATEGORY_AT_AIRPORT`; waits on rian's page-types answer |
| merged product variants 301 | done: `resolve_variant_id()` follows `merged_into_id` (A, migration #3) |

The owner's image sourcing list at `/images` is `noindex`, like every owner surface (`app/main.py` `HEADS`).