Semrush Split Signal

What does Semrush Split Signal do?

SplitSignalBot is a Semrush crawler that visits pages to collect content, links, and on-page data needed to run SEO A/B experiments through the SplitSignal product. It feeds data into SplitSignal reports, helping site owners measure the impact of on-page SEO changes. It does not drive referral traffic or produce public-facing citations to your site.

Should I allow and optimize for Semrush Split Signal to drive organic growth?

SplitSignalBot does not drive direct referral traffic. It feeds Semrush's SplitSignal A/B testing product, which presents findings within the Semrush interface rather than in public search results. Allowing it is primarily useful if you or your team actively use SplitSignal for SEO experimentation. If you don't use Semrush's SplitSignal product, there is limited direct benefit to allowing this bot. That said, Semrush is a widely used SEO platform, and ensuring its crawlers can access your site keeps your data accurate across Semrush's product suite, which SEO professionals and potential partners may rely on when evaluating your site.

Data Usage & Training

It is unclear whether content crawled by SplitSignalBot is used to train AI models. Semrush documents that crawled pages feed products like Site Audit and SplitSignal for indexing and analytics, but does not explicitly address AI training. If this matters to you, contact [email protected] for clarification.

How Semrush Split Signal Accesses Content

Here's how Semrush Split Signal accesses your site and understands your content:

• Fetches HTML and renders JavaScript fully
• Collects content, links, and on-page data from visited pages
• Respects robots.txt Allow and Disallow directives
• Supports non-standard Crawl-delay with an operator-imposed cap on large values
• Recognizes Sitemap directives in robots.txt
• Treats 4xx responses on robots.txt as "no robots.txt" and stops crawling on 5xx responses

SplitSignalBot crawls on an adaptive, periodic basis. Frequency adjusts based on server load, robots.txt rules, and internal crawl-delay caps. Changes to robots.txt may take up to one hour or roughly 100 requests to take effect.

How to Block or Control Semrush Split Signal

To block SplitSignalBot via robots.txt, add the following to your site root: User-agent: SplitSignalBot Disallow: / Make sure your robots.txt returns a 200 status code. A 4xx response is treated as no robots.txt (crawling proceeds), while a 5xx response stops crawling entirely. IP-based blocking is possible but discouraged because Semrush does not publish consecutive IP ranges. For verification or opt-out support, contact [email protected].

Common Issues & Troubleshooting

Watch out for these common problems when working with Semrush Split Signal:

• Missing or misconfigured robots.txt at the site root causes the bot to crawl as if no restrictions exist
• Robots.txt returning 4xx is treated as nonexistent, so the bot will crawl freely
• Changes to robots.txt can take up to one hour or approximately 100 requests to be recognized
• Setting extremely large Crawl-delay values gets overridden by Semrush's internal cap
• IP-based blocking is unreliable because Semrush does not publish consecutive IP ranges
• Meta robots tag handling is not explicitly documented by Semrush