Original research
How AI search engines and their crawlers actually behave, measured
English editions of SEO GEO Lab's original research: measurements from our own server logs and ChatGPT conversation data on how AI search engines and their crawlers actually behave. SEO GEO Lab is a Japanese publication on SEO and GEO (generative engine optimization).
10 ways of writing that AI search never cites, checked against 9 ChatGPT logs and 187 candidate URLs
Nine questions in ChatGPT: 187 candidate URLs, 40 citations. The 147 dropped share ten habits: an opening that is not the topic, answers ending in "it depends", tables with no sentence beside them.
ChatGPTAI searchCitationsOfficial sites
Comparison sites are dying in AI search: they make the candidate list and get dropped
Five credit card questions in ChatGPT: 236 candidate URLs, 23 citations. All 9 cited domains were issuer official sites or public agencies, and comparison sites took zero.
ChatGPTAI searchCitationsComparison sites
Republishing the English Version on a Subdomain Cut Googlebot to About a Quarter
I published the same English version in a subdirectory and on a subdomain, and compared Googlebot over the first 6 days. Per page the subdomain got about a quarter, and a tenth of pages were visited.
SubdomainSubdirectoryMultilingual sitesGooglebot
Google's AI Cited Only the English Version: Measured on a Topic With No Competition
I published English versions of articles from a Japanese site. About 96% of Search Console's generative AI impressions landed on the English URLs, though normal search showed Japanese 3x more.
AI OverviewsAI ModeMultilingual sitesSearch Console
Inside ChatGPT Sol's Conversation Log: It Searches in Multiple Stages
I read a ChatGPT Sol conversation JSON for a credit card recommendation question, message by message from 0 to 15. It searched in 3 stages; here is how that differs from the older gpt-5-6 model.
ChatGPTAI searchGPT-5.6Fan-out queries
1 Added Condition Swapped ChatGPT's Brands; 0 Overlap at 3 Conditions
Adding conditions to the same ChatGPT question swapped the brands it picks before searching; no conditions vs. three shared zero. Some queries shifted overnight, others held.
ChatGPTAI searchBrandsOriginal research
ChatGPT Snippets Are the First ~200 Characters of Your Body Text
We pulled 383 snippets from ChatGPT conversation logs. Every one was cut at 200–206 characters of extracted body text, and none contained meta description wording.
ChatGPTAI searchSnippetsmeta description
GoogleOther Outcrawled Googlebot 15x: 29 Days of IP-Verified Logs
In 29 days of logs from a site I run, GoogleOther made 109,551 requests and Googlebot 10,856. All 43 GoogleOther IPs were genuine; 27.5% of Googlebot-labeled requests were not.
GoogleOtherGooglebotCrawlersLog analysis
llms.txt Was Never Fetched by AI Crawlers in 29 Days: 19,347 Requests
I analyzed 29 days of access logs from a Japanese site I run: of 19,347 requests from 8 AI crawlers, zero fetched llms.txt, while robots.txt in the same root directory was fetched.
llms.txtAI crawlersLog analysisGEO
Crawlers don't follow sitemap order: 190K log requests analyzed
We analyzed 190,179 bot requests in our own access logs. Crawl order shows near-zero correlation with sitemap order, and ClaudeBot and GPTBot never fetched a single child sitemap.
CrawlersServer logsSitemapsClaudeBot
ChatGPT fan-out queries in real logs: 4 of 36 candidate URLs cited
I traced one ChatGPT conversation JSON from 3 fan-out queries to 36 candidate URLs to the 4 URLs actually cited. Comparison sites made the candidate list but got zero citations.
ChatGPTAI searchCitationsFan-out queries