What is GEO Optimization (Generative Engine Optimization)?

GEO Optimization (Generative Engine Optimization) is the targeted adaptation of web content so that AI-based search systems like ChatGPT, Perplexity AI and Google Gemini recognize the page as a trustworthy source and cite it in their answers.

How does the snaff.ai GEO Checker work?

The snaff.ai GEO Checker analyzes a URL for over 30 GEO signals: AI bot access in robots.txt, llms.txt, ai.txt, Schema.org JSON-LD (Organization, FAQPage, Article), E-E-A-T signals, semantic HTML and citability. The result is a weighted score from 0-100.

What is llms.txt and why does it matter?

llms.txt is a new file in the root directory of a website that provides language models (LLMs) with a structured summary of the website in Markdown. AI systems can use it to immediately understand the context of a brand without parsing the entire HTML.

Which AI bots should I allow in robots.txt?

The most important AI bots are: GPTBot and OAI-SearchBot (OpenAI/ChatGPT), ClaudeBot and anthropic-ai (Anthropic/Claude), Google-Extended (Google Gemini Training), PerplexityBot (Perplexity AI), Bingbot (Bing/Copilot) and Applebot-Extended (Apple Intelligence). All should be allowed with Allow: / in robots.txt.

What is the difference between SEO and GEO?

SEO (Search Engine Optimization) optimizes websites for classic search engines like Google that show blue links in result lists. GEO (Generative Engine Optimization) optimizes for AI systems that generate direct answers while citing sources. GEO focuses more on structured data, citability, E-E-A-T and AI bot access.

The snaff crawler

snaff-GEO-Checker

When a website blocks automated access, snaff identifies itself with its own, unique crawler. Allow it – and snaff analyzes protected sites in full.

Mozilla/5.0 (compatible; snaff-GEO-Checker/1.0; +https://snaff.ai/bot)

What the crawler does

Polite, transparent, on request only

snaff is not a search or training crawler. It only fetches what a GEO analysis needs.

public

Public pages only

Homepage, robots.txt, sitemap.xml, llms.txt – no forms, no logins, no protected areas.

touch_app

Only when someone checks

It only runs when a check is started or monitoring is set up – no mass crawling.

badge

Clearly identifiable

The user agent always contains “snaff-GEO-Checker” and links to this page – ideal for a targeted allow rule.

Allow it in Cloudflare

Three steps to a full analysis

One WAF rule lets the snaff crawler past your bot protection – every other rule stays active.

1Create a rule

Security → WAF → Custom rules → Create rule.

2Set the condition

Field User Agent, operator contains, value snaff-GEO-Checker – or paste the expression below.

3Action “Skip”

Tick “All remaining custom rules”, “Rate limiting rules”, “Managed rules” and – if available – “Super Bot Fight Mode”. Move the rule to the top.

(http.user_agent contains "snaff-GEO-Checker")

Important: the free “Bot Fight Mode” (Security → Bots) cannot be skipped by rules. If it is on, it blocks snaff and most AI crawlers too – it has to be turned off. A user-agent rule can be imitated by others; to avoid that, enable it only while you run the check.

Visibility in AI answers

What snaff sees, AI crawlers see

If your bot protection blocks snaff, it usually blocks ChatGPT, Claude and Perplexity too. Then your site can hardly be cited there.

smart_toy

Allow AI crawlers

Cloudflare: Security → Bots – turn off “Block AI Scrapers and Crawlers” or allow the crawlers you want under AI Crawl Control. Akamai, Imperva, DataDome, Sucuri: set them to “allow” in the bot manager.

GPTBotOAI-SearchBotChatGPT-UserClaudeBotClaude-SearchBotPerplexityBotGoogle-ExtendedApplebot-Extended

description

Keep robots.txt reachable

robots.txt must be reachable without a browser check – otherwise crawlers can’t tell what they may do. snaff checks this automatically for every blocked site.

Allowed it? Check again now.

Questions or problems with the crawler: hello@snaff.ai

Check a website arrow_forward