ParadigmaBot
The crawler behind Paradigma's site checks. It visits a site only after its owner connects and verifies it.
User agent
Requests from the crawler send this user agent:
ParadigmaBot/1.0 (+https://paradigma.click/bot)
The robots.txt token is ParadigmaBot.
When it visits
ParadigmaBot crawls a website only after someone with access to the site proves ownership in Paradigma with a DNS record, a file or a meta tag. It does not crawl the open web.
- Site checks run weekly on the Free plan and daily on paid plans.
- Each check reads robots.txt, up to 10 same-site sitemaps and up to 100 pages, a few at a time.
- For connected repositories, it can also read the preview deployment of a pull request to compare it with production.
robots.txt
ParadigmaBot reads robots.txt before each check and skips any URL it disallows for ParadigmaBot, including redirect targets. To block it:
User-agent: ParadigmaBot
Disallow: /Groups written for its former name, OrganicAgentBot, still apply when robots.txt has no ParadigmaBot group.
To show owners what search and AI crawlers see, a check can also:
- request a page or sitemap with the Googlebot or OAI-SearchBot user agent;
- request up to three pages per crawl with the Bingbot, ChatGPT-User, PerplexityBot, Claude-SearchBot and Claude-User user agents.
Only where robots.txt allows that crawler, and only for status codes, indexing directives and bot challenge detection.
Limits
- HTTPS only, on the verified domain and its public addresses.
- At most 5 redirects and 2 MB per response, 10-second timeout.
- Page HTML is discarded after analysis. See Data handling.
Contact
Seeing unexpected traffic from ParadigmaBot? Email support@paradigma.click with the domain and time, and we will look into it.