The AI search toolkit · 2.0Technical · Tool 10
Free AI Bot Log Analyzer
Turn a server-log excerpt into an actionable crawler report. Group requests by claimed bot, inspect response statuses and identify requested paths—all from a local analysis of the log data you provide.
Analyse local log sample
Processed in your browserExample data is loaded. These are fictional demonstration inputs, not a DigiMSM or client performance result. Use Clear to start your own analysis; editing example values keeps the run labelled as an example.
The interactive tool requires JavaScript. If the form does not appear, check that your WordPress setup allows this fragment’s inline script. The methods and FAQs below remain readable.
Inputs changed. Run the tool again to refresh the result and enable exports.
Your evidence comes first.
Analyse combined or JSONL access logs for claimed crawler requests and errors. Enter your information or load the labelled example to see the workflow.
No score, scan or platform result is invented while you wait.
A practical workflow
From input to a useful next step.
Export a bounded log sample
Use Apache or Nginx combined access logs, or one JSON object per line. Start with a representative period and keep the input within 2 MB of text and 10,000 lines.
Inspect the response patterns
Review recognized user-agent tokens, 2xx success responses, redirects, client errors and server errors. The tool reports unsupported lines instead of silently treating them as valid.
Verify before drawing conclusions
Check the relevant platform’s bot-verification guidance, then investigate repeated errors or unexpected paths. Export summaries and retain the raw evidence securely in your own workflow.
Know what you are measuring
The method behind the result.
Analyse combined or JSONL access logs for claimed crawler requests and errors. The scope and limitations below help you interpret the output and decide what to verify next.
User-agent claims stay qualified
A request can carry a crawler name without coming from that platform. The report labels recognized tokens as claimed crawler activity. It does not perform IP-range or reverse-DNS verification.
Documented input formats
JSONL accepts user_agent, http_user_agent, userAgent or ua; status or status_code; and request, request_uri, path or uri. Common timestamp aliases are read, while unsupported line formats are counted separately.
Useful summaries with less raw data
Exports omit client IPs, referrers and query strings. Path names can still contain sensitive information, so review exports before sharing. The fragment keeps the pasted sample in page memory only.
Clear answers
Before you act on the result.
Understand the inputs, what the output can establish and how to use it in a broader audit.
Which log formats are supported?
Apache/Nginx combined logs and JSONL with documented user-agent, request and status fields are supported. Custom formats may need conversion. Malformed or unsupported lines are counted explicitly.
Does a GPTBot user-agent prove OpenAI visited?
No. User-agent strings can be copied or spoofed. Verify the request origin using the platform’s published verification method before treating it as authenticated crawler activity.
Are Google-Extended requests counted?
Google-Extended is a robots control token, not an HTTP user-agent. It is deliberately excluded from the recognized request-token list.
What do 403 and 429 responses indicate?
They record forbidden and rate-limited responses respectively. Investigate the surrounding server and security configuration before assuming the cause. This tool groups observed statuses without changing your server.
Do successful bot requests prove my pages were cited?
No. A successful response is evidence about a request, not about later use in an answer. Track actual answer citations separately.
Is my raw log uploaded?
This fragment processes the selected text in your browser and makes no log-analysis network request. It does not store the log between visits. Your wider WordPress site and its plugins have their own data behaviour.
Continue your investigation
The next check is one click away.
AI Crawler Checker
Test robots.txt rules by path for OpenAI, Perplexity, Claude and search bots.
Open this tool ↗XML Sitemap Auditor
Audit sitemap XML structure, duplicate locations and selected URL metadata.
Open this tool ↗AI Visibility Checker
Review brand mentions and supplied citation links in a real AI answer.
Open this tool ↗DigiMSM · Server log analysis and technical SEO
Connect crawler activity to a technical fix.
DigiMSM can review a properly scoped log sample, verify crawler identity and investigate the response patterns affecting important pages.
Built for DigiMSM · Methods reviewed 14 September 2026 · Free tool; service work is scoped separately.