Which AI bots actually visited?
Your server logs are the only record of what really happened. Paste them and see which AI crawlers came, what they fetched, and what your server told them.
The only record that cannot lie.
Export your access logs
Most hosting panels offer raw access logs under a Logs or Statistics section. Cloudflare and other CDNs can export them too. Combined format includes the user agent, which is what we need.
Paste a sample
A few thousand lines is plenty for a directional view. Parsing happens entirely in your browser, so your logs never leave your machine.
Read what really happened
Which AI crawlers came, how often, which pages they preferred, and what status codes your server returned to them.
Tools estimate. Logs prove.
Every SEO tool you use models crawler behaviour. Your server logs record it. For questions about what actually happened, there is no substitute, and this is the one dataset your competitors cannot see.
This matters more in AI search than it ever did in traditional SEO, because there is no Search Console for AI engines. Marketers consistently report being unable to monitor whether AI bots are crawling their content at all. Your logs are the only first-party answer available.
What you learn is concrete. Whether GPTBot has ever visited. Whether ClaudeBot is hammering one section and ignoring another. Whether crawlers are being served 404s on pages you thought existed. Whether the crawl budget you assumed was going to your money pages is actually being spent on parameter URLs.
Enterprise platforms sell this as a premium feature. The analysis itself is not complicated, it just needs the data. If crawl efficiency is a serious constraint on a large site, full log file analysis goes considerably deeper than a pasted sample can.
Log analysis questions.
Where do I get my server logs?
Most hosting control panels provide raw access logs, often under Logs, Statistics or Metrics. On cPanel it is usually Raw Access. If you use Cloudflare or another CDN, they may hold the more complete record since they sit in front of your origin. Your developer or host can export them if you cannot find them.
Is it safe to paste logs here?
Parsing happens entirely in your browser using JavaScript, and nothing is transmitted to us or anywhere else. That said, access logs contain visitor IP addresses, so treat them carefully as a matter of habit, and remove anything sensitive before sharing logs with any third party.
What log format do I need?
Combined format, which is the default for both Apache and Nginx. It includes the user agent string at the end of each line, which is how crawlers are identified. Common format omits the user agent, so bot detection is impossible from it.
Can user agents be faked?
Yes. Anyone can claim to be GPTBot. For casual analysis this rarely matters, but for high-stakes work you verify identity by reverse DNS lookup on the IP, or by checking against the published IP ranges the major operators provide.
How many lines should I paste?
A few thousand gives a useful directional picture. Bear in mind that crawler visits are sporadic, so a sample covering only a few hours may show nothing even on a healthy site. A few days of data is far more reliable than a few hours.
Want the full picture?
A pasted sample shows a snapshot. We analyse complete log sets to find wasted crawl budget, indexation gaps and error patterns, then hand you a prioritised fix list.