Applebot

What does Applebot do?

Applebot is Apple's web crawler that discovers and indexes publicly available web content to power search and suggestion features across Apple products, including Spotlight, Siri, and Safari Suggestions. Crawled content also feeds into Apple Intelligence and generative AI features. Applebot can drive referral traffic to your site through clickable search results, suggestions, titles, snippets, and links presented to users across Apple platforms.

Should I allow and optimize for Applebot to drive organic growth?

Applebot directly powers search results and suggestions across Spotlight, Siri, and Safari on hundreds of millions of Apple devices. Allowing it means your content can appear as clickable results with titles, snippets, and links that drive users back to your site. Blocking Applebot removes your content from these surfaces entirely. Even the training component has indirect value: your content helps shape the AI models behind Apple Intelligence features, which may surface your brand or information in AI-powered responses. For most sites, allowing Applebot (and optionally blocking only Applebot-Extended if you want to opt out of training) is the right call.

Data Usage & Training

Content crawled by Applebot may be used to help train Apple's foundation models. Apple applies filters to remove profane or low-quality content and reduce inclusion of personally identifiable information. You can opt out of training use specifically by blocking Applebot-Extended in your robots.txt, which stops training crawls while still allowing Applebot to index your content for search and suggestions.

How Applebot Accesses Content

Here's how Applebot accesses your site and understands your content:

• Fetches HTML and fully renders JavaScript (including CSS and XHR requests)
• Uses a browser-like user-agent string containing 'Applebot/0.1' and a link to apple.com/go/applebot
• Respects robots.txt Disallow and Allow directives, plus Sitemap references
• Honors meta robots tags including noindex, nosnippet, and nofollow
• IP addresses can be verified via reverse DNS (PTR records under applebot.apple.com) and the published CIDR list

Applebot performs periodic, ongoing crawls to maintain Apple's server-side search index. This is continuous scheduled crawling rather than on-demand fetching.

How to Block or Control Applebot

To block all Applebot crawling: User-agent: Applebot Disallow: / To block only AI training use while allowing search indexing: User-agent: Applebot-Extended Disallow: / For page-level control, use meta robots tags (noindex, nosnippet, nofollow). You can verify and block Applebot by IP using the published CIDR ranges at http://search.developer.apple.com/applebot.json. For removal requests, contact applebot@apple.com or use Apple's Intelligence Privacy Inquiries form.

Common Issues & Troubleshooting

Watch out for these common problems when working with Applebot:

• Blocking JavaScript, CSS, or XHR resources prevents Applebot from rendering pages fully, which degrades index quality
• Blocking only Applebot-Extended does not stop search crawling; Applebot will still index your pages for Spotlight and Siri
• Robots.txt rules targeting other crawlers (like Googlebot) can unintentionally affect Applebot if no Applebot-specific rules are present
• Crawl-delay is not supported, so you cannot throttle Applebot through robots.txt
• Sites relying solely on IP blocking may miss new CIDR ranges if they don't periodically refresh from applebot.json