# Architecture Sources of truth: `main/app/`, `main/app/routers/`, `main/app/services/`, `main/web/src/`, `main/Dockerfile`, `docker-compose.yml`. One Python process serves both the API and the built React SPA. Postgres sits beside it on a private compose network. Collectors are plain Python modules run from the same image via `python -m app.cli`. There is no queue, no cache tier, no worker fleet, and no step that uses an LLM. ## The request path 1. Caddy terminates TLS and proxies the host to the container's published port on the Docker bridge (managed by the gateway; the port is in `.env`). 2. Every request, reads and pages included, passes two middlewares before routing: the session loader (`app/services/sessions.py`, the account cookie to `request.state.session`) and the access policy (`app/services/access.py`), which puts every route, page, mount and SPA path in exactly one class: public always, public when `SITE_ACCESS=public` and otherwise any signed-in account, any member, or one permission. A route in no class is refused before its handler runs; `tests/test_route_inventory.py` fails on one. Every write needs a matching Origin. While the site is members-only an anonymous page request is one 302 to `/login`. The whole account system: `ACCOUNTS.md`; the procedure for a host: `RUNBOOK.md` (Accounts); which surfaces the client sees: `CLIENT-SURFACES.md`. 3. FastAPI (`app/main.py`) routes `/api/*` to the routers; `/assets`, `/fonts`, `/medals` and `/flags` (the airport cards' country flags) are real static mounts so HEAD and range requests work for crawlers. 4. Any other path returns the built SPA shell, **stamped server-side** by `app/services/seo.py`: title, description, canonical, social tags and JSON-LD per page, and for the product, airport, brand, article and data pages the body itself, so a client that runs no JavaScript reads the same page the SPA draws. The same process serves the machine surfaces (`/robots.txt` with per-bot policy, `/sitemap.xml`, `/feed.xml`, `/llms.txt`, the IndexNow key file); each mechanism and how to verify it is in `SEO.md`. Feature flags ride in the shell (`window.__DFP_FLAGS__`, from `app/config.py`). 5. Explainer pages in `public/` are served at `/.html` by rule; an absent page is a 404, never the SPA shell (`agents.md`, the catch-all rule). 6. The SPA (`web/src/App.tsx`) owns the routes; its typed client (`web/src/api/schema.ts`) is generated from the backend's OpenAPI schema at build time and by `check.sh`. 7. The site chrome is the SPA's (`web/src/components/SiteHeader.tsx`, `SiteFooter.tsx`), and the server-rendered pages carry a copy of the header in its closed state (`seo.py header_html()`) so nothing moves when React mounts; a change to the header's shape changes the mirror in the same deploy (`SEO.md`). The top row is the logo, the search (`SiteSearch.tsx`: suggestions as one types, airports and brands matched in the browser from the lists it holds, products from `/api/search/suggest`, a match-and-rank query without the price aggregation that makes `/api/products` too slow per keystroke; a code or a city typed whole opens that airport, `lib/search.ts`), the airports button and the profile circle (`AccountMenu.tsx`: the account links and the build-discussion pages this account may open, from `lib/clientPages.ts`). The nav leads with four departments (Airports, Drinks, Beauty, Articles; `MegaMenu.tsx` DEPARTMENTS, the families being taxonomy's), each a link to its own page and the toggle for its panel, and ends with My savings. The panel reads `/api/stats` (shelves by family), `/api/brands` (each brand's `family`), `/api/airports` (and warms its flags), the featured savings and the latest articles once the page is idle, or sooner on hover, focus or the drawer, so the first hover finds them in hand; a list the API returns empty is simply absent. "All drinks" is `/products?family=liquor`, the products API's `family` filter over the family's shown categories. Social profile links (`SocialLinks.tsx`) render only for URLs the shell injects as `window.__DFP_SOCIAL__`, the same mechanism as the feature flags. 8. The SPA draws its controls by capability: one `GET /api/bw/me` (the vendored pack's provider in `web/src/lib/auth.tsx`, refetched on focus and after writes, never on a timer) carries the kit's booleans and DFP's `capabilities` block, and `/plan`, `/sources`, curator mode, the client pages and the account menu draw from it; the server refuses the same routes whatever the SPA draws (step 2). The sign-in pages are DFP's own (`web/src/pages/LoginPage.tsx`, `ForgotPage`, `SetPasswordPage`, `AccountPage`); the People, Levels and Access panels on `/admin` are the pack's. Every string a shopper or the client reads is checked for house style by `tests/test_house_style.py`. ## The pipeline `fetch.py → .collect() → RawListing → ingest.run_collector → normalize / taxonomy / fx → Product / Listing / PriceObservation`. Every collector yields the same `RawListing` shape; ingest owns identity resolution (GTIN first, the v3 key as fallback: `services/keying.py` resolves the house through the brand alias and the variation through the alias table, then `normalize.match_key` composes house, line, variation, size and form from the pure rules in `services/lines.py`), the plausibility guards, and the race-tolerant row creation. Details and per-platform recipes: `COLLECTORS.md`. The schema and its rationale: `DATA-MODEL.md`. What merges and what waits for a person, and the merge session behind `/collectors`: `services/merges.py`, `services/suggest.py`, `services/merge_session.py` (their docstrings), and `RUNBOOK.md`. Medals join onto products after the fact (`app/services/awards_import.py`, one row per product, competition, year and medal, reconciled in place); which medal a card or page shows is `app/services/award_picker.py`. ## Build and run Three Docker stages (`main/Dockerfile`): Python deps and an OpenAPI dump; the typed client and SPA build against that dump; a runtime image that is Python only and runs `alembic upgrade head` then uvicorn. Compose (`docker-compose.yml`): `app` and `db`, the database with no published port, `import/` and `public/` mounted read-only. ## Module map Generated from each module's first docstring line by `main/scripts/docmap.py`; a module without a docstring shows as such, which is the signal to write one. Do not edit by hand. - `app/cli.py`: Operational commands. - `app/cli_accounts.py`: The account commands: `accounts`, `sessions` and `audit-log` (docs/ACCOUNTS.md). - `app/cli_editorial.py`: The editorial commands: articles in, subscribers out. - `app/cli_pages.py`: CLI subcommands for the crawl surface (Stream B): `indexnow`. - `app/cli_quality.py`: The quality commands: `audit` (no network) and `verify` (re-reads, per host). - `app/config.py`: Application settings. Values come from the environment (see .app.env). - `app/db.py`: Database engine and session handling. - `app/main.py`: Duty Free Professor - application entrypoint. - `app/models/__init__.py`: (no docstring) - `app/models/accounts.py`: Accounts: credentials, levels, members, grants, sessions, tokens, the audit log, and overrides. - `app/models/base.py`: (no docstring) - `app/models/catalog.py`: Catalog schema. - `app/models/client.py`: The client's to-do list and the files they hand us. - `app/models/editorial.py`: Editorial storage: articles as rows, and the people who asked to hear from us. - `app/models/hubs.py`: API response shapes for the hub pages (airports now; brands and categories follow). - `app/models/owner.py`: Rian's running list: the state he sets on items the sessions raise. - `app/models/quality.py`: What verification and the audit record: runs, checks, snapshots, refusals. - `app/models/schemas.py`: API response shapes. The frontend's TypeScript client is generated from these. - `app/routers/__init__.py`: (no docstring) - `app/routers/articles.py`: Public reads of editorial text: the article list, one article, and the two keyed blocks. - `app/routers/auth.py`: The app's own sign-in: `/api/auth/*` (login, logout, forgot, token-state, welcome, reset, - `app/routers/catalog.py`: (no docstring) - `app/routers/collectors.py`: The collector and catalogue review area: /collectors, and its merge session. - `app/routers/discussion.py`: The running list of things to raise with the client, and every comment thread. - `app/routers/health.py`: (no docstring) - `app/routers/items.py`: The running list on /plan: decisions for rian, things he must do, issues to address. - `app/routers/ops.py`: The owner's operational reads under `/api/ops/*` (R1: active impersonations only). - `app/routers/plan.py`: The build plan as data, for the /plan page. - `app/routers/quote.py`: The client-facing quote builder. - `app/routers/sources.py`: Collection health, and the per-source kill switch. - `app/routers/subscribers.py`: The one public write for email capture: POST /api/subscribers. - `app/routers/todos.py`: The client's to-do page: items, completion, and file hand-in. - `app/routers/trip.py`: (no docstring) - `app/services/access.py`: The route policy: every route in exactly one class, the pure `decide()`, the middleware. - `app/services/accounts.py`: The account system's front door to the vendored BW kit: the mapped store, the permissions, - `app/services/airport_guides.py`: The written half of an airport page: where the duty free actually is. - `app/services/audit.py`: Is this data true? The no-network audit: metrics with thresholds, and the human review's lists. - `app/services/audit_log.py`: The audit sink every account mutation calls, and the kit's hooks into it. - `app/services/award_picker.py`: Which medal a product shows when it holds several. - `app/services/awards_import.py`: Attaching competition medals to duty-free products. - `app/services/catalog_queries.py`: Read queries behind the catalog API. - `app/services/collector_view.py`: The collectors and catalogue, as the review page reads them. - `app/services/collectors/ari.py`: ARI's "The Loop" — Dublin and Cork airport duty-free. - `app/services/collectors/avolta.py`: Avolta storefronts — World Duty Free and Shop Duty Free. - `app/services/collectors/base.py`: Shared shape for every collector. - `app/services/collectors/changi.py`: iShopChangi, the online duty-free marketplace of Singapore Changi (Changi Airport Group). - `app/services/collectors/dubai.py`: Dubai Duty Free. - `app/services/collectors/extime.py`: Extime, the Paris airports storefront. - `app/services/collectors/fetch.py`: The fetch port. - `app/services/collectors/heinemann.py`: Gebr. Heinemann -- global online catalogue only. - `app/services/collectors/heinemann_platform.py`: Shops running Gebr. Heinemann's storefront platform. - `app/services/collectors/registry.py`: Every collector the app knows about, in one place. - `app/services/collectors/robots.py`: The one robots.txt policy every collector applies. - `app/services/collectors/shilla.py`: The Shilla Duty Free, Seoul Incheon's online store, read through the browser sidecar. - `app/services/collectors/shopify.py`: Duty-free shops running on Shopify. - `app/services/collectors/targets.py`: The targeted beauty set: which lines a collector reads beauty pages for (Decision 4). - `app/services/coverage.py`: What we cover, what we do not, and how good each source's data is. - `app/services/directory.py`: The local directory: what the vendored kit calls `bw_auth`, plus the token mint and the - `app/services/discussion_import.py`: Load the decisions-and-realities list from a JSON file (idempotent, upsert by title). - `app/services/editorial.py`: Articles: reading a hand-in into a row, and the published reads every page uses. - `app/services/featured.py`: Which savings the home page leads with. - `app/services/feeds.py`: The plain-text and syndication surfaces: the RSS feed and llms.txt. - `app/services/fx.py`: Currency conversion, so prices from different airports are comparable. - `app/services/identity.py`: Who the caller is: the two request-scoped resolvers the whole app runs on. - `app/services/images.py`: Product imagery, from an openly licensed source. - `app/services/indexnow.py`: IndexNow: tell the search engines which pages changed, instead of waiting. - `app/services/ingest.py`: Turning collected listings into catalog rows. - `app/services/items.py`: The running list: merge the items file with rian's recorded state. - `app/services/keying.py`: The keying call: what ingest and rederive ask before a product is keyed (Stream M). - `app/services/lines.py`: The line above the product, the variation vocabulary, and the form: the pure rules. - `app/services/mail.py`: The mail seam: a `Mailer` sends one plain-text message; `none` is the launch provider. - `app/services/mail_resend.py`: The Resend provider behind `MAIL_PROVIDER=resend`: one HTTPS POST per message, stdlib only. - `app/services/markdown.py`: Markdown to HTML for editorial text, one renderer for the API and the server-rendered page. - `app/services/merge_session.py`: The merge session: the queue a person works through, and what confirming or rejecting a - `app/services/merges.py`: Recorded merges: two product rows that are one bottle become one, with a record; and - `app/services/normalize.py`: Turning messy retailer strings into comparable facts. - `app/services/passwords.py`: Passwords: argon2id hashing and verification, the rules, the generated-password alphabet. - `app/services/robots_policy.py`: Our own robots.txt: what we tell other readers, by name. - `app/services/seo.py`: Server-rendered SEO layer. - `app/services/sessions.py`: Sessions: the store, the `SessionRow` mapping the kit reads, the cookie, the loader middleware. - `app/services/subscribers.py`: Email capture: taking a subscription and exporting the list, with the personal-data rules. - `app/services/suggest.py`: Merge suggestions at every level, rules first: what the queue offers a person (Stream M). - `app/services/taxonomy.py`: Consumer-facing product categories. - `app/services/throttle.py`: A small in-process rate limiter for the few public routes that write. - `app/services/trip.py`: Savings across the shopper's airports. - `app/services/uploads.py`: Validation and storage for files the client hands us. - `app/services/urls.py`: The site's URL shapes, in one place. - `app/services/verify.py`: Re-read a sample of published listings and say what moved, what broke, what refused. - `app/services/view_as.py`: View As: DFP's policy over the vendored `bw_view_as` mechanism, and its reporting. - `app/vendor/__init__.py`: The BW application kit, vendored byte-identical (plan: accounts-2026-09 §4.1). - `app/vendor/bw_accounts.py`: bw_accounts.py — the standard BW-Auth app admin framework (P4, external-users plan). - `app/vendor/bw_admin_api.py`: bw_admin_api.py — the standard BW-Auth admin API as a mountable FastAPI router. - `app/vendor/bw_store_sqlalchemy.py`: bw_store_sqlalchemy.py — a ready SQLAlchemy store for the BW-Auth accounts kit. - `app/vendor/bw_view_as.py`: bw_view_as.py — standard "View As" (impersonation) for BW-Auth (Pattern B) apps. - `app/version.py`: Single source of truth for the app version (surfaced by /api/health and the UI).