AI Bot Log Analyzer: how GPTBot, ClaudeBot and others crawl your site
See how AI crawlers crawl your site from your server access log.
- Free to use
- Runs in your browser
- No sign-up
Or paste log lines
Want an expert to check this result?
Send it to us in one click. We read it and reply within one business day with what we would fix first.
Embed this tool on your websiteFree. Paste one code; it runs in your visitors' browsers.
Copy this code into a page of your site. The tool sets no cookies and sends nothing your visitors type to us. Change height="760" if it needs more room.
About the AI Bot Log Analyzer
If AI crawlers cannot fetch your important pages, those pages cannot be cited. Open your server access log (Apache or Nginx, plain or .gz) and we find every visit from AI crawlers such as GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot and others, next to Googlebot and Bingbot. You see visits per crawler, the pages they read, errors and blocked requests, and what to fix. The file is read in your browser and never uploaded.
How it works
-
Step 1
Download your access log: in cPanel under Metrics > Raw Access, from your host, or from /var/log/nginx or /var/log/apache2.
-
Step 2
Drop the file here (plain or .gz) or paste some lines. It is processed in your browser.
-
Step 3
You get visits per crawler, top pages, errors and blocks, daily activity and recommendations.
Your log stays on your computer
The analysis runs in your browser with JavaScript. Nothing is uploaded to our server, so you can use it with real logs.
Crawler names can be faked
Anyone can send a user agent such as "GPTBot". To confirm a visit is genuine, compare the IP address with the ranges the company publishes, or use a reverse DNS lookup. Treat the numbers as a strong hint.
What to look for
403 or 429 errors mean a firewall or rate limit is blocking a crawler. 404s waste crawls. If AI search crawlers never visit, check robots.txt with the AI Crawler Checker and make sure your pages are linked and in the sitemap.
Frequently asked questions
Which log formats work?
The common and combined formats used by Apache, Nginx, LiteSpeed and most cPanel hosts. Lines in other formats are skipped.
How large can the file be?
Large logs work because the file is read in pieces. Very large files take longer; a week or a month of logs is usually enough.
Why do I see no AI crawlers?
Your log may only cover a short period, your firewall may block them, or they have not found the site yet. Check robots.txt and the firewall, then look again in a few weeks.
Related reading
Rather have us do it for you?
Tell us about your business and what you want to fix. We reply within one business day.
Talk to us Our service: Technical SEO servicesMore free tools
All tools- Website SEO Audit Crawl up to 50 pages and get a health score, every issue and a 30-day plan.
- Redirect Checker Trace 301/302 redirects, chains and loops to the final URL.
- Robots.txt Tester Test a URL against robots.txt and see which rule blocks or allows it.
- Email Deliverability Checker Check SPF, DKIM, DMARC and MX against the Gmail, Yahoo and Outlook.com sender rules.
Want us to fix what the tool found?
Tell us about your business and what you want to grow. We reply within one business day with next steps, not a sales script.
- A reply within one business day
- A quick look at your site and search visibility before we talk
- Your details are only used to answer your enquiry