Overview
The Agent Analytics settings page is where you tell Sona how to see AI crawler traffic on your website. It tracks which AI agents crawl your site and forwards that traffic to Sona at the edge, so visits from Claude, GPTBot, Perplexity and other AI agents show up in your reporting.
The page has two parts: Connect your crawler source, which is required, and Enhance with analytics, which is optional and correlates crawler activity with search performance and human traffic. Once traffic is flowing, you read the results in the
When to use these settings
Use the Agent Analytics settings when you need to:
Start tracking AI crawler activity on your website for the first time.
Switch the platform that forwards traffic to Sona, for example moving from Cloudflare to your own servers.
Change which brand's AI content a Cloudflare Worker serves to recognized crawlers.
Correlate crawler activity with search performance or human traffic.
Prerequisites:
Access to Workspace Settings in Sona.
For the Cloudflare Workers path: a Cloudflare account (the free tier works) and a domain added to Cloudflare with DNS proxied through it.
For the Fluent Bit path: a Linux server running Apache or Nginx, with systemctl available.
Getting to Agent Analytics settings
Open Workspace Settings.
Select Agent Analytics at the bottom of the settings list.
The breadcrumb at the top confirms you are on Workspace Settings › Agent Analytics.
Note: View Setup Docs in the top right opens the full Agent Analytics setup documentation in a new tab.
Step 1: connect your crawler source
The Connect your crawler source card carries a status pill that reads Not connected until a source is set up. While it is not connected, Sona shows a warning: without a crawler source, visits from Claude, GPTBot, Perplexity and other AI agents will not appear in your Agent Analytics dashboard.
Open the selector to pick the platform that hosts your site. There are two options:
Cloudflare Workers: observe AI crawler traffic at the edge, in front of your origin.
Fluent Bit: ship access logs to Sona from your own web servers.
Pick the one that matches how your site is served:
Option | Best for |
Cloudflare Workers | Sites already behind Cloudflare. Captures every request at the edge, before it reaches your origin server, and can also serve AI-agent-friendly content to recognized crawlers. |
Fluent Bit | Self-hosted Apache or Nginx servers. Tails your existing access logs and streams them to Sona in real time. |
Step 2: pick the brand (Cloudflare Workers)
Under Brand, choose the brand whose AI content crawl run the worker should serve. Its crawl run ID is filled into the script below automatically, so you do not have to look it up.
This is what lets the worker return clean, AI-agent-friendly content to recognized crawlers instead of your standard page. If you are not using that feature, the worker still forwards traffic to Sona for reporting.
Step 3: deploy the worker (Cloudflare Workers)
The Deploy the worker section shows a ready-to-use script named sona-traffic-observer.js, with your organization ID and crawl run ID already filled in. You do not need to edit it.
Click Copy code to copy the script.
In your Cloudflare dashboard, go to Workers & Pages and create a Worker (the Hello World template is fine), then open Edit code.
Replace the default script with the copied Sona script and deploy it.
In the Worker's Settings tab, under Domains & Routes, add a Route pointed at your domain, for example example.com/*, so every request passes through the worker.
Set Failure mode to Fail open (proceed) so your site stays reachable even if the worker errors.
The worker inspects each request's user agent, matches it against Sona's list of known AI crawlers, and forwards the request details to Sona in the background. The original request passes through to your origin untouched, so there is no impact on response time.
Tip: the full walkthrough with Cloudflare screenshots is in the Cloudflare Worker Setup guide.
Using Fluent Bit instead
If you run your own Apache or Nginx servers rather than sitting behind Cloudflare, select Fluent Bit as your crawler source. Fluent Bit is a lightweight log forwarder that tails your access logs on the server and streams them to Sona in real time. Because your access logs already record every request's user agent, Sona can identify AI crawlers from the forwarded logs.
At a high level the setup is:
Install Fluent Bit on the server using the official install script.
Edit /etc/fluent-bit/fluent-bit.conf with the Sona configuration for Apache, Nginx, or both, replacing the organization UUID, hostname, and server timezone placeholders.
Run systemctl enable fluent-bit and systemctl start fluent-bit so it starts now and on every reboot.
Tip: the exact configuration blocks and commands are in the Fluent Bit Setup guide.
Step 4: enhance with analytics (optional)
The Enhance with analytics card correlates AI crawler activity with search performance, human traffic, and referrals. You can connect any, all, or none of these.
Sona Tracking Script: tracks human visitors and AI-referred traffic on your website. Click Setup Guide for the installation steps.
Google Search Console: correlates search performance with AI crawler activity. Click Connect Account, then answer Which property should we track? by choosing a property from the dropdown and clicking Confirm.
These connections are additive. Skipping them does not stop Agent Analytics from reporting crawler activity, it just means you see crawler data on its own rather than alongside search and human traffic.
Saving your changes
Click Save at the bottom of the page to store your crawler source and brand selection. The Not connected pill on the crawler source card is your check that traffic is reaching Sona, so revisit the page after deploying the worker or starting Fluent Bit.
Which AI crawlers Sona identifies
Both crawler sources match requests against the same list of known AI agents. In the Cloudflare Worker this list is visible in the script as the AI_BOTS pattern, so you can always confirm exactly which agents are matched.
Company or source | Crawlers identified |
OpenAI | GPTBot, ChatGPT-User, OAI-SearchBot |
Anthropic | ClaudeBot, Claude-SearchBot, anthropic-ai |
Perplexity | PerplexityBot |
Google-Extended | |
Apple | Applebot-Extended |
Amazon | Amazonbot |
Meta | Meta-ExternalAgent, Meta-ExternalFetcher |
ByteDance | Bytespider |
Cohere | cohere-ai |
Common Crawl | CCBot |
Other | YouBot, Timpibot, Kangaroo Bot, magpie-crawler, ICC-Crawler, iaskspider/2.0 |
Note: this list reflects the crawlers Sona recognizes today and is expanded as new AI agents appear.
Key concepts and best practices
One source is enough: pick the crawler source that matches how your site is served. Cloudflare Workers for edge traffic, Fluent Bit for your own servers. You do not need both.
Route the worker widely: a route that only covers your apex domain misses subdomain traffic. Add routes for each subdomain you care about, or use a wildcard pattern.
Fail open: always set the Cloudflare route's failure mode to fail open so a worker error never takes your site down.
Copy, do not retype: use Copy code rather than transcribing the script. Your organization ID and crawl run ID are already embedded in it.
Connect Search Console early: the search correlation is far more useful with history behind it, so connect it at setup rather than later.
FAQs
Why do I need to connect a crawler source?
Why do I need to connect a crawler source?
Sona cannot see AI crawler traffic on your site unless something forwards it. The crawler source is that forwarder. Without one connected, visits from Claude, GPTBot, Perplexity and other AI agents will not appear in your Agent Analytics dashboard.
Should I choose Cloudflare Workers or Fluent Bit?
Should I choose Cloudflare Workers or Fluent Bit?
Choose Cloudflare Workers if your site already sits behind Cloudflare. It captures every request at the edge, before your origin server, and can also serve AI-agent-friendly content to recognized crawlers. Choose Fluent Bit if you run your own Apache or Nginx servers and would rather stream your existing access logs.
What does the Brand selector do?
What does the Brand selector do?
It picks which brand's AI content crawl run the worker should serve. The selected brand's crawl run ID is filled into the worker script automatically, so recognized crawlers can be served that brand's AI-agent-friendly content.
Do I need to edit the worker script?
Do I need to edit the worker script?
No. Your organization ID and the selected brand's crawl run ID are already filled in. Click Copy code, paste it into your Cloudflare Worker, and deploy.
Will the Cloudflare Worker slow down my site?
Will the Cloudflare Worker slow down my site?
No. The worker passes the original request through to your origin untouched and sends the traffic details to Sona in the background, so visitors are not waiting on it.
Do I have to connect the tracking script and Google Search Console?
Do I have to connect the tracking script and Google Search Console?
No, both are optional. They correlate crawler activity with human traffic and search performance. Skipping them means you still see crawler data, just without those extra dimensions alongside it.