Skip to main content
Lab Grimoire
TW EN
Coffee
Site

Staging robots.txt locked down via SSR endpoint

Changes (3)
  • Staging (test.labgrimoire.com) /robots.txt now returns `User-agent: * / Disallow: /` for every request, replacing the prod allow-list that the Cloudflare Assets layer was previously serving on staging
  • Production /robots.txt content is byte-for-byte unchanged but now served by an env-aware SSR endpoint instead of a static asset — required because the Astro Cloudflare adapter routes known assets directly to env.ASSETS.fetch before middleware runs, which made the staging override silently ineffective
  • Verification: staging body length dropped from 2084 bytes to 26 bytes, X-Robots-Tag: noindex,nofollow,noarchive header added; prod etag unchanged