Meta-ExternalFetcher
AI assistant fetcherMeta's fetcher for user-initiated link loads inside Meta AI products. Because fetches happen on behalf of a specific user request, Meta documents that it may bypass robots.txt rules.
| Operator | Meta |
| Type | AI assistant fetcher |
| Respects robots.txt | Partially |
| Documentation | Official operator documentation |
Meta-ExternalFetcher user-agent string
This is how Meta-ExternalFetcher identifies itself in your server logs:
meta-externalfetcher/1.1How to block Meta-ExternalFetcher in robots.txt
Add this to the robots.txt file at the root of your site:
Block Meta-ExternalFetcher
User-agent: meta-externalfetcher
Disallow: /Explicitly allow Meta-ExternalFetcher
User-agent: meta-externalfetcher
Allow: /What blocking changes: Robots.txt is documented as possibly bypassed for these user-initiated fetches; a network-level block is the reliable option if you want Meta AI unable to read your pages on request.
Want to decide for all AI bots at once? Use the AI robots.txt generator.
Other ai assistant fetchers
- ChatGPT-User (OpenAI)
- Claude-User (Anthropic)
- Perplexity-User (Perplexity)
- MistralAI-User (Mistral AI)
- DuckAssistBot (DuckDuckGo)
Is Meta sending you visitors?
Glyphex classifies AI referral traffic from ChatGPT, Perplexity, Claude, Gemini and more. See which assistants cite your pages and what those visitors do.
See your AI traffic