Which AI Agents Visit Your Site?

Which AI agents visit your site is a question a hit counter answers badly. A model training on your pages, an index for an answer engine, and a fetch that happened because a person asked an assistant something are different events. A request that only claims to be one of those, with a user agent anybody can type, is a fourth. Adding them up into "AI traffic" throws away the only part worth having.

SwiftlyBotKit is the package we built for that split, for Vapor apps. We run it on the four Swiftly sites: this one, marketing, consulting and workspace. This article describes what it records. It does not quote a number from the dashboard. A count without the window, the filter and the purpose is how those numbers get waved around, and we are not going to start.

The catalog, and what a visit was for

Every request is classified against a catalog of 167 known AI agents, generated from the community ai.robots.txt list. A match is stored with a purpose. The README's names are training, aiSearch, userTriggered, agent and scraper. Training is a crawl that feeds a model. AI search is an index for an answer engine. User-triggered is the fetch a person's question caused. The last two cover agents and scrapers that do not fit the first three.

Claimed identity is not trusted on its own. OpenAI, Anthropic and Perplexity publish IP ranges. The package checks a claimed agent against those ranges and stores the visit as verified, unverified or spoofed. A forged ChatGPT-User header and a fetch that really came from OpenAI's range are not the same row. The check is only as good as the ranges those operators publish. Agents outside those three stay unverified, which is an honest gap, not a hidden pass.

People who arrive from an assistant are recorded too, as referrals rather than as agents: ChatGPT, Claude, Perplexity, Gemini, Copilot, Grok, Le Chat and others, on the web and from their Android apps. A referral is a browser hit with a referring host. It is not an agent fetch, and the dashboard keeps them apart.

What it refuses to record

Classification is one in-memory lookup. The database write happens after the response, in a detached task, so the package does not hold the page. Ordinary human traffic and static assets are not stored as agent visits. robots.txt and sitemap.xml are recorded on purpose, because those are exactly what a crawler comes for first.

There is an optional second feature, and it is easy to mix up with the first. Anonymous page views count how often people load each HTML page, per site, path and quarter-hour. No cookie, no IP address, no user agent and no referrer is stored on that counter. The agent table is different: it keeps a keyed hash of the client IP, not the address itself. If you turn page views on, you are not turning on a second copy of the agent log. The dashboard's page tab shows the human counts beside the agent reads for the same path. Time on page, when enabled, is another counter, in duration bands, from a beacon that sends seconds and a path.

The figures we recorded for the public repository on 10 October 2026 showed no stars. The latest GitHub release is 0.11.0, published on 27 September 2026. The README's install example in this repository still says from version 0.10.0. Depend on 0.11.0 if you want the release this article is describing. The licence is MIT. The dashboard needs PostgreSQL. It is server-rendered HTML, behind the password you configure, with a site switcher when one app serves several domains, and it can export CSV.

SwiftlyBotKit on GitHub

Why the user-triggered row is the one to watch

A training crawl tells you a model might have seen the page. A user-triggered fetch tells you a person asked a question and an assistant came to read. That is the visit a citation can come from. It is still not a citation. The package cannot see whether the assistant quoted you. It can see that the assistant fetched you, and whether the IP matched the operator.

Several of those crawlers download the HTML and do not run JavaScript. The measurement we rely on is the one Vercel and MERJ published in December 2024: GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot and others fetched pages and did not execute scripts. Only Googlebot and Applebot rendered. So the text that exists for them is the text in the response body. That is why these sites are server-rendered, and why a dashboard that itself needs a client framework would be a strange thing to ship next to that finding. SwiftlyBotKit's dashboard is HTML.

The same fact cuts the other way. If your product page only fills in after a script runs, the user-triggered fetch still "visited" you and still did not see the offer. The agent row will look healthy. The page will not be quotable. Fix the HTML before you celebrate the chart.

What this article does not tell you

• How many agents fetched swiftly-developed.com last week. The dashboard has that. This page does not, on purpose.
• Which assistant cites us. A fetch is not a citation.
• A complete list of every agent in the catalog. It is generated, and it moves. The README's 167 is the size, not an invitation to retype the list.
• Legal advice about the EU AI Act. Article 50, which has applied since August 2026, is a different article. It is about what your app owes the people using its AI features, not about who crawled your marketing site.

AI Act Article 50: what it means for the AI features in your app

If you run Vapor and you want the breakdown instead of a badge that says "AI bots: 12,000", the package is the one we use ourselves. Wire it, look at purpose and verification, and leave the grand total in the drawer.