# Stream K3: certain-only identity, and the word lists as proposal generators
(written 2026-09-16 Wed, from the accepted plan §0, W7, W9, W10, W14 and §8; rian's ruling: escalation 1 is not a separate think, K3 rehearses two or three versions of the certain boundary on the staging copy and picks from the numbers; part of the Thursday-night wave with K4 and K5)

**Runs:** Thu 17 Sep night into Fri 18, in the wave, after K2's hand-back, on the refactor branch
in the worktree. Never alongside a collection; never alongside another stream that touches
identity (none in this wave does).

## Goal
A rule extracts only what is certain and the automatic merge fires only on certain agreement;
everything interpretive is a proposal on the sheet with its reason; a stated attribute that
differs vetoes the automatic merge in every category; a person's Keep separate is honoured by every
machine path; the Attenza shade ships as a shop-published option carried as a field, not read back
from a name; and the boundary between certain and uncertain is chosen from measured numbers on a
copy of staging, then written into `REVIEW-PROCESS.md` as a rules version.

## What already exists, so you build on it
- `product_lines.py` (K1's name for `lines.py`): the residual-name key with its hand-kept lists
  (`_DRINK_WORDS` 72, `_FORMAT_WORDS` 66, skin-type tails, cask words, articles, connectors,
  concentration rules), the marked-tail reader (` / ` tails), `_PACK_EXPRESSION_WORDS`, the
  display-name function; `normalize.py`'s `_BRAND_TRAILERS`, `_STOPWORDS`, `_LEADING_FILLER`;
  `keying.py`'s `Maps`, `listing_key`, `product_key`, `line_for`; `merges.py`'s
  `duplicate_groups`, `_conflicts` (barcode, set, unstated quantity, `attributes` disagreement),
  `merge_duplicates`, the alias appliers; `suggest.py`'s rule pairs; `ingest.py`'s
  `_resolve_product`, `_follow_tombstone`, `_resolve_listing`; `cmd_rederive` re-lining every row.
- The 15 Sep tests pin today's behaviour: `test_lines.py` (63), `test_keying.py`, `test_merges.py`,
  `test_aliases.py`, `test_decided.py`, `test_line_backfills.py`, `test_suggest.py`. Many of them
  pin applied list deletions; those become proposal tests (below).
- K2 provides: the attribute registry (`attributes.py`: kinds per vertical and category with
  `identity` and `display`; the accessor over columns and JSONB; `quantity` typed;
  `not_applicable`), the `decisions` table with `decisions.effective()` the key path reads through
  `Maps.decided`, `Maps.separated` (every Keep separate, survivor-chased), and the `proposals` table
  with `proposals.write(db, pass_name, rows, ...)`; K2's brief fixes the exact signatures K3 calls.
- The plan's §0 fact: 9,865 of 16,761 variants carry a barcode; the rule-based fold produced 612
  merges in the catalogue's life.

## The certain key (plan W14)
`brand | residual name | identity attributes | quantity`, where:
- **brand** is the brand row's slug after `alias_of_id` (a Confirm same on a brand pair still
  reaches every key under it);
- **residual name** is the listed name with only these removed: the brand as matched to its row
  (a head match, or the exact brand words anywhere); the quantity statement the parser read as a
  number; an ABV percentage; a pack figure ("3x1L"); every shop-published option value; the
  closed concentration vocabulary folded to its canonical word. Case, accents, punctuation and
  glyphs fold. Nothing else is deleted;
- **identity attributes** are the variant's attributes whose kind has `identity: always` for its
  category, sorted by kind name, each `kind=value` (K2's registry; the shop-published options are
  identity by default);
- **quantity** as today (`100ml`, `50g`, `200pcs`, `unknown`, and K2's `n/a` for a category that
  has none).

Two or more of these versions are rehearsed (task K3.2); the one chosen becomes identity rules v6.

## Tasks, in order (ids on /plan; commit prefix `K3:`)
1. **K3.1 Options as fields from the collectors.** `RawListing` gains `options: list[(name, value)]`
   as the shop published them; the Shopify collector fills it from `option1`/`option2`/`option3`
   with the product's option names (`Tamaño`, `Color`, `Talla`) and stops gluing the variant title
   into the name; Extime fills it from `variation.name` and the capacity fields; ingest writes each
   as an attribute `option:<shop name>` (certain, identity) until the registry maps the shop's
   name to a kind (`color`, `size_label`); `listed_variant` keeps the shop's own string. The
   ` / ` tail reader stays only as the fallback for fragments stored before this change, and a
   test pins that a fresh Attenza fragment never reaches it. The RingConn ring at Panama (ring size
   6.0, colour Plateado, no category, no quantity) is a fixture: two attributes, `n/a` quantity, its
   own product line, nothing guessed.
2. **K3.2 The boundary, rehearsed.** On a fresh copy of the newest staging dump (`dfp_k3` on
   `dfp-devdb`), rederive under three boundaries and report, read from the database, for each:
   product lines; variants that split apart against today's grouping and the comparisons lost by
   it (a comparison is a variant at two or more comparison units); variants that join and the
   comparisons gained; the groups the automatic merge would fold, listed by name so a person can
   read them for wrong joins. The three: **(a)** the certain key above as written; **(b)** (a) plus
   the drink category words as certain removals (whisky, gin, vodka, blended, single malt: a closed
   list of category nouns, not descriptors); **(c)** (b) plus the format words that never name a
   product (spray, vaporisateur, bottle, ml, cl). Pick the boundary whose lost comparisons are
   fewest against zero wrong joins in the listed groups (a wrong join outweighs any number of
   splits, per the constitution), write the numbers to the handoff and the chosen list into
   `REVIEW-PROCESS.md` "What a rule may do without review" as a rules version bump; stamp
   `IDENTITY_RULES_VERSION = "6"`.
3. **K3.3 The lists become proposal generators.** Every open list (`_DRINK_WORDS`, `_FORMAT_WORDS`
   beyond the chosen certain subset, region words, cask words, `_PACK_EXPRESSION_WORDS`, the
   skin-type tails, `_BRAND_TRAILERS`, `_STOPWORDS`, the unmarked-name shade shapes) is moved into
   `proposal_rules.py`: each still computes what it would have grouped or extracted, and writes
   that as a proposal (`proposals.write(db, pass_name="rule:<list>:6", rows, kind="rule", ...)`, one
   pass per list so a bad list withdraws alone, with the rule's name as the reason and the span it read) through the same store the AI pass uses; nothing applies. The
   `suggest.py` pair rules (name within, words shared, namesake, rebrand) stay as suggestions.
4. **K3.4 The vetoes and the honoured decisions.** `_conflicts` reads every identity attribute
   through the accessor (ABV included, so 40 against 43 vetoes; numbers compare as numbers; a
   stated value against an absent one is not a disagreement), and reads every Keep separate through
   `Maps.separated` (K2 fills it from the ledger, survivor-chased), so the automatic merge never folds a pair a
   person kept separate. The veto binds machines only: `merge_session.confirm` still lets a person
   Confirm same across a stated difference and then set the value as a decision (plan §8's
   answer to rian's W9 question); pin with a test that the person's merge captures the next
   sighting of either key (`_follow_tombstone` under the certain key, where the differing ABV is
   part of the key).
5. **K3.5 New listings after an approved sheet.** Ingest lands a new listing by barcode, else by
   the certain key onto a decided variant (a decided variant's key is its effective key: a person's
   line, attributes and name decisions read through `Maps`); a listing that matches nothing becomes
   its own variant on its own new product line and is written as a proposal "addition to <nearest
   approved line>" (nearest by brand and residual-name overlap), never joined automatically.
   `rederive` never moves a decided variant off its decided line and never re-mints a decided line;
   `prune_lines` is unchanged.
6. **K3.6 The slug hook.** At the point `apply_brand_alias` and `apply_line_alias` record the alias,
   call K6's `publish.slug_at_confirm(row, chosen_name)` (signature fixed in K2's brief; a no-op
   stub until K6 lands, so K3 does not wait).
7. **K3.7 Tests and the rederive proof.** The proposal tests: every former list-deletion test now
   asserts the proposal it writes (the "Triple Cask" rows propose "triple is the expression" with
   `form: single` as the reason; the shade shapes propose a shade); the certain key on the fourteen
   1 Million rows (EDT synonyms fold; the set stays apart; nothing else joins); the Keep separate
   veto; the ABV veto and the person's override of it; the option fields; the RingConn fixture;
   `rederive` twice on the copy printing zero on the second run; `backfill merges --check` listing
   zero wrong groups by a person's reading.

## Owns
`app/services/product_lines.py`, `keying.py`, `normalize.py`, `merges.py` (the automatic paths;
K6 supplies the slug function), `suggest.py`, `proposal_rules.py` (new), `ingest.py`, `collected.py`,
`collectors/shopify.py`, `collectors/extime.py`, `collectors/base.py` (the `options` field),
`cli.py`'s `rederive` and the identity backfills, `models/catalog.py`'s `IDENTITY_RULES_VERSION`,
their tests, `REVIEW-PROCESS.md`'s "What a rule may do without review" section (the chosen
boundary, as a rules version).

## Must not touch
The schema and the accessor (K2; K3 asks through the running list for a column it lacks), the
review sheet and its CLI (K4; K3 writes proposals through K2's store only), the line page (K5),
`publish.py` (K6), the merge session's person-facing routes beyond keeping the person's Confirm
same allowed.

## Rehearsal and acceptance
K3.2's three-way rehearsal is the acceptance for the boundary; the handoff carries its table.
Then, on the same copy under the chosen boundary: `backfill variations`, `backfill lines`,
`rederive` twice, `backfill merges --check` read and listed, `suggest`, `prune_lines`; the
proposal count by rule; `main/check.sh` green.

## Rollback
The rules version: `IDENTITY_RULES_VERSION` back to 5 and `rederive` restores the previous keys;
proposals written are withdrawn per list (`proposals withdraw --pass rule:<list>:6`).

## Hand-back
Handoff (≤25 lines) with the boundary table, the chosen boundary, the rules version, the
after-deploy commands in order, and "K3 green and ready".