YouBot
AI search indexerThe crawler behind You.com's AI search engine, indexing pages to ground its answers with cited sources.
| Operator | You.com |
| Type | AI search indexer |
| Respects robots.txt | Yes (documented) |
| Documentation | Official operator documentation |
YouBot user-agent string
This is how YouBot identifies itself in your server logs:
Mozilla/5.0 (compatible; YouBot (+http://www.you.com))How to block YouBot in robots.txt
Add this to the robots.txt file at the root of your site:
Block YouBot
User-agent: YouBot
Disallow: /Explicitly allow YouBot
User-agent: YouBot
Allow: /What blocking changes: Your site is excluded from You.com's index and its cited AI answers.
Want to decide for all AI bots at once? Use the AI robots.txt generator.
Other ai search indexers
- OAI-SearchBot (OpenAI)
- Claude-SearchBot (Anthropic)
- Googlebot (Google)
- PerplexityBot (Perplexity)
- Bingbot (Microsoft)
- Applebot (Apple)
Is You.com sending you visitors?
Glyphex classifies AI referral traffic from ChatGPT, Perplexity, Claude, Gemini and more. See which assistants cite your pages and what those visitors do.
See your AI traffic