User-agent: GPTBot Disallow: / User-agent: ClaudeBot Disallow: /gated/ User-agent: * Disallow: /private/ Crawl-delay: 10 Allow: /public/