User-agent: *
Allow: /
Disallow: /admin/
Disallow: /api/

# The paginated listings are server-rendered per page against D1, ~2,900 pages
# deep, and OFFSET makes each one cost more than the last. Crawling them was a
# large share of the daily D1 read allowance and discovered nothing new: pages
# 2+ are noindex, follow, and the sitemaps below list every article directly.
# Caveat: a ?page= URL already indexed when this went in cannot be dropped while it stands, since Google never re-reads the noindex. See docs/PAGINATED_URLS.md
Disallow: /*?page=

Sitemap: https://kol-il.news/sitemap.xml