Google Feedfetcher

What does Google Feedfetcher do?

Google Feedfetcher retrieves RSS and Atom feeds on behalf of users who have subscribed to or requested those feeds through Google products. It powers feed display in services like Google News and other feed-reading surfaces. Because these products typically show clickable links back to the original content, Feedfetcher can drive meaningful referral traffic to your site.

Should I allow and optimize for Google Feedfetcher to drive organic growth?

Google Feedfetcher directly powers feed display in Google News and other feed-reading surfaces. These products show clickable links back to your original content, creating a direct referral traffic channel. If you publish RSS or Atom feeds, allowing Feedfetcher means your content can appear in front of users who have actively subscribed to your feed through Google products. Blocking it removes your content from these surfaces entirely.

Data Usage & Training

Google's official Feedfetcher documentation describes the bot's purpose as serving and caching feeds for user-requested products. It does not state whether fetched content is used to train AI models. Training usage is unclear from available documentation.

How Google Feedfetcher Accesses Content

Here's how Google Feedfetcher accesses your site and understands your content:

• Fetches RSS and Atom feed URLs via standard HTTP requests
• Does not render JavaScript
• Ignores robots.txt directives because fetches are user-initiated
• Fetches from multiple machines and IP addresses across Google's infrastructure
• Periodically refreshes stored feeds, generally not more than once per hour per site

Fetches are triggered when a user subscribes to or requests a feed through a Google product. Stored feeds are then refreshed periodically, generally not more than once per hour on average for most sites.

How to Block or Control Google Feedfetcher

Robots.txt does not work for Feedfetcher. It ignores Disallow, Allow, and Crawl-delay directives because its fetches are user-initiated. To block Feedfetcher, you have a few options. Serve a 404 or 410 response to requests with the FeedFetcher-Google user-agent string. Require authentication on your feed URLs. Block the IP ranges published at https://developers.google.com/static/crawling/ipranges/user-triggered-fetchers-google.json. Or remove or change the feed URL entirely. To verify a request is genuinely from Feedfetcher, check the user-agent string for "FeedFetcher-Google", match the source IP against Google's published IP ranges, and perform a reverse DNS lookup to confirm a Google-owned hostname.

Common Issues & Troubleshooting

Watch out for these common problems when working with Google Feedfetcher:

• Robots.txt rules have no effect on Feedfetcher, which surprises many site owners who expect standard blocking to work
• Feedfetcher uses multiple IPs across Google's infrastructure, making simple IP-based blocking unreliable without the published IP range list
• Malformed or invalid RSS/Atom feeds may cause incomplete or failed fetches
• Rate limiting on your server may inadvertently block periodic feed refreshes
• Feed URLs that require cookies or session-based authentication will fail silently