An AI crawler policy is a short, deliberate document that articulates which AI crawlers your site allows, which it blocks, and the reasoning. It typically lives as comments at the top of robots.txt plus a longer policy page (often `/ai-policy/` or part of the legal section). **Why a deliberate policy matters in 2026:** **Defensibility.** If a vendor or partner asks why your site allows or blocks specific crawlers, a documented policy provides a clear answer. Vendors increasingly review prospective partners' AI policies as part of procurement. **Avoid accidental blocking.** Many Canadian sites blocked AI crawlers in 2023–2024 reactively, without revisiting the decision. A periodic policy review catches outdated decisions that may no longer serve current business goals. **Legal clarity.** Canadian copyright law (the Copyright Act, plus the Online News Act for news publishers) intersects with AI training and inference in ways that are still being tested in court. A documented policy supports the position that your stance on AI use of your content was deliberate, not implicit. **Selective blocking is now standard.** Most enterprise Canadian sites in 2026 allow live-citation crawlers (OAI-SearchBot, Claude-Web, PerplexityBot, Googlebot) while blocking pure training crawlers (GPTBot, ClaudeBot, Google-Extended, Applebot-Extended). This selective stance preserves citation visibility while declining to contribute to model training without compensation. **Sample policy structure:** ``` # AI crawler policy v2026.04 — see /ai-policy/ for full text # Live-citation crawlers ALLOWED (we want our content cited) User-agent: OAI-SearchBot Allow: / User-agent: Claude-Web Allow: / User-agent: PerplexityBot Allow: / # Training crawlers BLOCKED (no compensation framework yet) User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Google-Extended Disallow: / User-agent: Applebot-Extended Disallow: / ``` **The companion `/ai-policy/` page** typically explains: which crawlers are allowed and why; which are blocked and why; the contact for licensing inquiries; the date of last review; the next scheduled review date. **Review cadence.** AI search is evolving rapidly. Schedule policy reviews every 6 months to reflect new crawlers, new business priorities, and new licensing options that may emerge from publisher–AI vendor agreements.