Amazon Product Discovery

What does Amazon Product Discovery do?

Amazon Product Discovery crawls publicly available product details from selling partner, brand, and retailer websites to improve the accuracy and completeness of product information in Amazon's catalog. The data feeds into Amazon's product detail pages and seller-initiated listing workflows. It does not drive referral traffic or link back to your site as a source.

Should I allow and optimize for Amazon Product Discovery to drive organic growth?

Amazon Product Discovery does not generate referral traffic or citations back to your site. Its purpose is to enrich Amazon's own product catalog. If you sell products on Amazon, allowing this crawler may help keep your Amazon listings accurate and complete. If you don't sell on Amazon, there's little direct benefit. For sellers, accurate product data in Amazon's catalog can indirectly support your Amazon storefront, but the growth path is narrow and indirect.

Data Usage & Training

Whether content crawled by Amazon Product Discovery is used for AI model training is unclear. Amazon's general Amazonbot documentation states crawled content "may be used to train Amazon AI models," but the Product Discovery-specific documentation on Vendor Central does not address this directly. If this concerns you, contact amazonbot@amazon.com for clarification.

How Amazon Product Discovery Accesses Content

Here's how Amazon Product Discovery accesses your site and understands your content:

• Fetches HTML via standard HTTP requests using a Firefox-based user-agent string
• Respects robots.txt Disallow directives for the AmazonProductDiscoverybot token
• Does not support Crawl-delay, nofollow, or noindex directives
• JavaScript rendering capability is unknown
• IP addresses are published at https://developer.amazon.com/amazonbot/searchbot-ip-addresses/

Amazon does not publish a specific crawl schedule. Documentation implies continuous or scheduled crawling rather than on-demand fetches. Robots.txt changes may take up to 24 hours to be reflected in Amazon's systems.

How to Block or Control Amazon Product Discovery

To block Amazon Product Discovery via robots.txt: User-agent: AmazonProductDiscoverybot Disallow: / Changes may take up to 24 hours to propagate in Amazon's systems. For IP-based blocking, use the published prefixes at https://developer.amazon.com/amazonbot/searchbot-ip-addresses/ and verify requests with reverse DNS lookups to confirm Amazon-owned hostnames. Do not rely on Crawl-delay, nofollow, or noindex directives, as this crawler ignores all three. For content removal requests or questions, contact amazonbot@amazon.com or use Vendor Central support channels.

Common Issues & Troubleshooting

Watch out for these common problems when working with Amazon Product Discovery:

• Robots.txt changes can take up to 24 hours to take effect in Amazon's systems
• Crawl-delay, nofollow, and noindex directives are all ignored by this crawler
• Amazon publishes many IP prefixes, making IP-based blocking error-prone if you don't use the exact published list
• Simple user-agent string blocking is easy to spoof and can produce false positives
• JavaScript-heavy product pages may not be fully crawled, since rendering capability is unknown