# Shilla (ICN) rendered probe, Stream A2, announced 2026-09-05 09:45 local
# Brief: A2-rendered-fetch.md deliverable 3 + "Shilla probe first". Decision 5.
# Scope: one robots.txt read, one text read of the home page to find a product id,
# ONE rendered fetch of ONE product page through the sidecar, honest UA, floor delay 10 s,
# no dialog click-through, no retries past a refusal. Nothing else is fetched from the host.
# Sidecar: dfp-browser-test (the same image and flags compose declares), 127.0.0.1:18080.
robots: status=200 disallows=['/files/', '/medias/', '/estore/_ui/', '/comm/kr/ko/storeMain?org_cd=08', '/comm/kr/ko/storeFloor?org_cd=08', '/comm/kr/ko/storeMain?store_dtl_id=6?actr=2'] crawl_delay=5.0
home: HTTP 200 bytes=674514
product ids on home: 28 distinct; first 8: ['5581103', '5301333', '5098145', '5542729', '5677766', '5505172', '4925576', '5363471']
first link text: 5581103 ''
2026-09-05 09:44:06,776 INFO render_budget label=probe cap=1 used=1
ERROR: browser sidecar answered HTTP 502: TimeoutError: Page.goto: Timeout 45000ms exceeded.
Call log:
  - navigating to "https://www.shilladfs.com/estore/kr/en/p/5581103", waiting until "load"

  - navigating to "https://www.shilladfs.com/estore/kr/en/p/5581103", waiting until "load"

# second and final attempt: the same page, with the challenge script host declared as an asset host (challenges.cloudflare.com), because a browser that cannot load the page's own challenge script cannot 'simply be a browser'. Still our declared UA; no other change.
2026-09-05 09:45:49,659 INFO render_aborted_hosts url=https://www.shilladfs.com/estore/kr/en/p/5581103 hosts=['connect.facebook.net', 'image1.shilladfs.com', 'image2.shilladfs.com', 'media.shilladfs.com', 'www.googletagmanager.com']
2026-09-05 09:45:49,660 INFO render_disallowed_paths url=https://www.shilladfs.com/estore/kr/en/p/5581103 paths=['/estore/_ui/desktop/common/css/jquery.colorbox-1.3.16.css', '/estore/_ui/desktop/common/images/No_Image_320X.jpg', '/estore/_ui/desktop/common/js/acc.common.js', '/estore/_ui/desktop/common/js/jquery-1.11.2.min.js', '/estore/_ui/desktop/common/js/jquery-migrate-1.2.1.min.js', '/estore/_ui/desktop/common/js/jquery-ui-1.11.2.min.js', '/estore/_ui/desktop/common/js/jquery.colorbox.custom-1.3.16.js', '/estore/_ui/desktop/common/js/jquery.dotdotdot.js', '/estore/_ui/desktop/common/js/jquery.jcarousel.min.js', '/estore/_ui/desktop/common/js/jquery.price.format.js', '/estore/_ui/desktop/common/js/jquery.query-2.1.7.js', '/estore/_ui/desktop/common/js/jquery.validate.min.js', '/estore/_ui/desktop/common/shilladfshome/css/jquery-ui.css', '/estore/_ui/desktop/common/shilladfshome/css/jquery.mCustomScrollbar.css', '/estore/_ui/desktop/common/shilladfshome/en/css/common.css', '/estore/_ui/desktop/common/shilladfshome/en/css/sub.css', '/estore/_ui/desktop/common/shilladfshome/en/css/sub_common.css', '/estore/_ui/desktop/common/shilladfshome/en/css/sub_dev.css', '/estore/_ui/desktop/common/shilladfshome/en/img/common/bg_arrow_down.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/bg_arrow_up.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/bg_sky_scrapper_qr.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/footer_certify_ccm02.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/footer_certify_ncsi.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_cart.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_certify_r01.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_certify_r02.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_certify_r03.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_clock.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_close_17px.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_01.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_02.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_03.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_04.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_05.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_07.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_c.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_k.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_s.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_w.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_search5.png', '/estore/_ui/desktop/common/shilladfshome/en/img/ico_mlevel_s_40.png', '/estore/_ui/desktop/common/shilladfshome/en/js/adultCertification/shilla.adultCe', '/estore/_ui/desktop/common/shilladfshome/en/js/boutique.js', '/estore/_ui/desktop/common/shilladfshome/en/js/common.js', '/estore/_ui/desktop/common/shilladfshome/en/js/main.js', '/estore/_ui/desktop/common/shilladfshome/en/js/product_detail.js', '/estore/_ui/desktop/common/shilladfshome/en/js/shilla.cart.js', '/estore/_ui/desktop/common/shilladfshome/en/js/shilla.customer.js', '/estore/_ui/desktop/common/shilladfshome/en/js/shilla.miniCalculator.js', '/estore/_ui/desktop/common/shilladfshome/en/js/shilla.product.js', '/estore/_ui/desktop/common/shilladfshome/en/js/shilla.product.quickview.js', '/estore/_ui/desktop/common/shilladfshome/en/js/sjhm.load.min.js', '/estore/_ui/desktop/common/shilladfshome/en/js/slick.min.js', '/estore/_ui/desktop/common/shilladfshome/images/product/productDetail/common/ico', '/estore/_ui/desktop/common/shilladfshome/js/breadcrumb.js', '/estore/_ui/desktop/common/shilladfshome/js/default.js', '/estore/_ui/desktop/common/shilladfshome/js/heartcode-canvasloader-min-0.9.1.js', '/estore/_ui/desktop/common/shilladfshome/js/html5.js', '/estore/_ui/desktop/common/shilladfshome/js/jquery-ui-Datepicker-1.11.2.js', '/estore/_ui/desktop/common/shilladfshome/js/jquery.cookie.js', '/estore/_ui/desktop/common/shilladfshome/js/jquery.mCustomScrollbar.concat.min.j', '/estore/_ui/desktop/common/shilladfshome/js/netfunnel.js', '/estore/_ui/desktop/common/shilladfshome/js/shilla.customer.common.js', '/estore/_ui/desktop/common/shilladfshome/js/shilla.orderAvailableTime.js', '/estore/_ui/desktop/common/shilladfshome/js/shilla.share.js', '/estore/_ui/desktop/common/shilladfshome/js/skin.js', '/estore/_ui/desktop/common/shilladfshome/js/util.all.js', '/estore/_ui/desktop/common/shilladfshome/kr_v2/resource/images/img_default.gif', '/estore/_ui/desktop/theme-blue/css/changes.css', '/estore/_ui/xtractor/xtractor-cookie.js', '/estore/_ui/xtractor/xtractor-ebm.js']
2026-09-05 09:45:49,660 INFO rendered url=https://www.shilladfs.com/estore/kr/en/p/5581103 status=200 text=67361 api=0 timed_out=True ms=47239 budget=1/1
2026-09-05 09:45:49,660 INFO render_budget label=probe cap=1 used=1
status=200 final_url=https://www.shilladfs.com/estore/kr/en/p/5581103
headers={'content-type': 'text/html;charset=UTF-8', 'server': 'cloudflare'} timed_out=True
title='The Shilla Duty Free Shop' text_length=67361 html_bytes=675610 elapsed_ms=47239
aborted_hosts=['connect.facebook.net', 'image1.shilladfs.com', 'image2.shilladfs.com', 'media.shilladfs.com', 'www.googletagmanager.com'] disallowed_paths=['/estore/_ui/desktop/common/css/jquery.colorbox-1.3.16.css', '/estore/_ui/desktop/common/images/No_Image_320X.jpg', '/estore/_ui/desktop/common/js/acc.common.js', '/estore/_ui/desktop/common/js/jquery-1.11.2.min.js', '/estore/_ui/desktop/common/js/jquery-migrate-1.2.1.min.js', '/estore/_ui/desktop/common/js/jquery-ui-1.11.2.min.js', '/estore/_ui/desktop/common/js/jquery.colorbox.custom-1.3.16.js', '/estore/_ui/desktop/common/js/jquery.dotdotdot.js', '/estore/_ui/desktop/common/js/jquery.jcarousel.min.js', '/estore/_ui/desktop/common/js/jquery.price.format.js', '/estore/_ui/desktop/common/js/jquery.query-2.1.7.js', '/estore/_ui/desktop/common/js/jquery.validate.min.js', '/estore/_ui/desktop/common/shilladfshome/css/jquery-ui.css', '/estore/_ui/desktop/common/shilladfshome/css/jquery.mCustomScrollbar.css', '/estore/_ui/desktop/common/shilladfshome/en/css/common.css', '/estore/_ui/desktop/common/shilladfshome/en/css/sub.css', '/estore/_ui/desktop/common/shilladfshome/en/css/sub_common.css', '/estore/_ui/desktop/common/shilladfshome/en/css/sub_dev.css', '/estore/_ui/desktop/common/shilladfshome/en/img/common/bg_arrow_down.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/bg_arrow_up.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/bg_sky_scrapper_qr.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/footer_certify_ccm02.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/footer_certify_ncsi.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_cart.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_certify_r01.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_certify_r02.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_certify_r03.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_clock.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_close_17px.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_01.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_02.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_03.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_04.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_05.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_07.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_c.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_k.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_s.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_rate_w.png', '/estore/_ui/desktop/common/shilladfshome/en/img/common/ico_search5.png', '/estore/_ui/desktop/common/shilladfshome/en/img/ico_mlevel_s_40.png', '/estore/_ui/desktop/common/shilladfshome/en/js/adultCertification/shilla.adultCe', '/estore/_ui/desktop/common/shilladfshome/en/js/boutique.js', '/estore/_ui/desktop/common/shilladfshome/en/js/common.js', '/estore/_ui/desktop/common/shilladfshome/en/js/main.js', '/estore/_ui/desktop/common/shilladfshome/en/js/product_detail.js', '/estore/_ui/desktop/common/shilladfshome/en/js/shilla.cart.js', '/estore/_ui/desktop/common/shilladfshome/en/js/shilla.customer.js', '/estore/_ui/desktop/common/shilladfshome/en/js/shilla.miniCalculator.js', '/estore/_ui/desktop/common/shilladfshome/en/js/shilla.product.js', '/estore/_ui/desktop/common/shilladfshome/en/js/shilla.product.quickview.js', '/estore/_ui/desktop/common/shilladfshome/en/js/sjhm.load.min.js', '/estore/_ui/desktop/common/shilladfshome/en/js/slick.min.js', '/estore/_ui/desktop/common/shilladfshome/images/product/productDetail/common/ico', '/estore/_ui/desktop/common/shilladfshome/js/breadcrumb.js', '/estore/_ui/desktop/common/shilladfshome/js/default.js', '/estore/_ui/desktop/common/shilladfshome/js/heartcode-canvasloader-min-0.9.1.js', '/estore/_ui/desktop/common/shilladfshome/js/html5.js', '/estore/_ui/desktop/common/shilladfshome/js/jquery-ui-Datepicker-1.11.2.js', '/estore/_ui/desktop/common/shilladfshome/js/jquery.cookie.js', '/estore/_ui/desktop/common/shilladfshome/js/jquery.mCustomScrollbar.concat.min.j', '/estore/_ui/desktop/common/shilladfshome/js/netfunnel.js', '/estore/_ui/desktop/common/shilladfshome/js/shilla.customer.common.js', '/estore/_ui/desktop/common/shilladfshome/js/shilla.orderAvailableTime.js', '/estore/_ui/desktop/common/shilladfshome/js/shilla.share.js', '/estore/_ui/desktop/common/shilladfshome/js/skin.js', '/estore/_ui/desktop/common/shilladfshome/js/util.all.js', '/estore/_ui/desktop/common/shilladfshome/kr_v2/resource/images/img_default.gif', '/estore/_ui/desktop/theme-blue/css/changes.css', '/estore/_ui/xtractor/xtractor-cookie.js', '/estore/_ui/xtractor/xtractor-ebm.js']
api_responses=0
saved /tmp/claude-1000/-srv-apps-dutyfreeprofessor/317745c9-7e4e-4331-bfb0-dcf67a62a3c8/scratchpad/shilla-5581103.html
2026-09-05 16:45:49,633 INFO rendered url=https://www.shilladfs.com/estore/kr/en/p/5581103 status=200 text=67361 api=0 aborted=['connect.facebook.net', 'image1.shilladfs.com', 'image2.shilladfs.com', 'media.shilladfs.com', 'www.googletagmanager.com'] denied_paths=4139 timed_out=True ms=47239

# RESULT (Stream A2, 2026-09-05 09:46 local). Two renders in total, ten seconds apart or more.
# 1. Product page /estore/kr/en/p/5581103, allowlist = origin only: the sidecar aborted the
#    challenge script host and goto() timed out at 45 s with nothing returned (the sidecar then
#    gained partial-DOM-on-timeout).
# 2. Same page, challenges.cloudflare.com declared as an asset host, our declared UA unchanged:
#    HTTP 200, server: cloudflare, NO cf-mitigated header, 675,610 bytes, 67,361 characters of
#    text, body class "pageType-ProductPage". The browser passed simply by being a browser. The
#    text client's 403 challenge on 4 Sep was Cloudflare judging the client, not the identity.
# What the document holds, server-rendered (no JS ran that mattered; the page's own scripts
# live under /estore/_ui/ which robots.txt disallows for *, so the sidecar refused all 4,139
# requests for them, and that is why "load" never fired):
#   productCode 5581103; brand NARS; name "NARS LIGHT REFLECTING SETTING POWDER-PRESSED (NEW)";
#   REF.NO 607845058946 (a UPC-A, the GTIN); SKU.NO 096311001883; category Makeup > Base >
#   Powder/Compact Foundation; Price $43; Discount price $25.37 (41% off); "Price for online
#   member" $25.37; Temporarily Sold Out; Weight,Volume 10G; hidden inputs prdPriceDollar=25.37,
#   exchangeRate=156.04. Three price tiers on the page: list, discount, online member.
# robots.txt (read 09:44): * group, Crawl-delay 5, Disallow /files/ /medias/ /estore/_ui/ and
# three /comm/kr/ko/store* paths. Product paths are permitted. Sitemap is empty.
# Aborted third-party hosts: connect.facebook.net, www.googletagmanager.com (trackers), and
# image1/image2/media.shilladfs.com (imagery; we store URLs only, never fetch).
# Conclusion: Seoul is CONFIRMED POSSIBLE through the rendered fetch, as a server-rendered
# source read at domcontentloaded, 10 s between pages (their 5 s, our floor). Discovery is the
# open question (empty sitemap; home lists ~40 products; category pages are the likely walk)
# and needs its own announced probe. Nothing further was fetched from the host tonight.
# 2026-09-05 10:20 addendum: ONE more render, a category listing page (/estore/kr/en/c/1211, Blended Whisky), to learn the listing tile shape and pagination for discovery, because the sitemap is empty. domcontentloaded, assets still refused per robots, ten seconds after the last render.
status=200 timed_out=False title='The Shilla Duty Free Shop' text_length=58482 html_bytes=594564 elapsed_ms=3648 headers={'content-type': 'text/html; charset=utf-8', 'server': 'cloudflare'}
aborted_hosts=['connect.facebook.net', 'media.shilladfs.com', 'www.googletagmanager.com'] denied_paths=164
product links on the category page: 1
pagination hints: [] ['class="main page-shilladfsProductGridPage pageType-CategoryPage template-pages-category-shilladfsENProductGridPage  language-en"', 'class="power_search category_page"', 'class="pagination facet-pagination"', 'class="pagination facet-pagination"', 'class="pageloading_wrap"']
TILE: None

# ADDENDUM RESULT (10:25): the category page is a shell whose grid is drawn by facet.search()
# in scripts under /estore/_ui/ (robots-disallowed for *); the sidecar refused 164 requests and
# the grid never appeared. We do not read their loader and do not guess its endpoint. Discovery
# for shilla.py = home page (text) + related-product rails (40 links per product page), bounded.
# robots.txt re-read once more (text): named groups now disallow Amazonbot, Applebot-Extended,
# Bytespider, CCBot, ClaudeBot, CloudflareBrowserRenderingCrawler, Google-Extended, GPTBot,
# meta-externalagent; * has Crawl-delay 5 and the asset Disallows. Three renders total on this
# host tonight. Nothing else was fetched.