ParadigmaBot

The crawler behind Paradigma's site checks. It visits a site only after its owner connects and verifies it.

Requests from the crawler send this user agent:

ParadigmaBot/1.0 (+https://paradigma.click/bot)

The robots.txt token is ParadigmaBot.

ParadigmaBot crawls a website only after someone with access to the site proves ownership in Paradigma with a DNS record, a file or a meta tag. It does not crawl the open web.

  • Site checks run weekly on the Free plan and daily on paid plans.
  • Each check reads robots.txt, up to 10 same-site sitemaps and up to 100 pages, a few at a time.
  • For connected repositories, it can also read the preview deployment of a pull request to compare it with production.

ParadigmaBot reads robots.txt before each check and skips any URL it disallows for ParadigmaBot, including redirect targets. To block it:

User-agent: ParadigmaBot
Disallow: /

Groups written for its former name, OrganicAgentBot, still apply when robots.txt has no ParadigmaBot group.

To show owners what search and AI crawlers see, a check can also:

  • request a page or sitemap with the Googlebot or OAI-SearchBot user agent;
  • request up to three pages per crawl with the Bingbot, ChatGPT-User, PerplexityBot, Claude-SearchBot and Claude-User user agents.

Only where robots.txt allows that crawler, and only for status codes, indexing directives and bot challenge detection.

  • HTTPS only, on the verified domain and its public addresses.
  • At most 5 redirects and 2 MB per response, 10-second timeout.
  • Page HTML is discarded after analysis. See Data handling.

Seeing unexpected traffic from ParadigmaBot? Email support@paradigma.click with the domain and time, and we will look into it.