protogrid

Search / com.atalaya-ai/ai-crawler-check

Atalaya AI crawler check

R0· no sign-in

Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why.

v1.1.0 · active · website · descriptor JSON · versions

An agent can connect right now, no human step.Remote endpoint, no authentication, reachable on the last probe.
Quality89new
Trust80of 100
Uptime, 30 days83%
Latency p50263 ms

Connect

8 clients · secrets stay placeholders
{
  "mcpServers": {
    "ai-crawler-check": {
      "type": "http",
      "url": "https://atalaya-ai.com/mcp"
    }
  }
}
claude mcp add --transport http ai-crawler-check https://atalaya-ai.com/mcp
[mcp_servers.ai-crawler-check]
url = "https://atalaya-ai.com/mcp"
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "ai-crawler-check": {
      "type": "remote",
      "url": "https://atalaya-ai.com/mcp",
      "enabled": true
    }
  }
}
{
  "mcpServers": {
    "ai-crawler-check": {
      "type": "http",
      "url": "https://atalaya-ai.com/mcp"
    }
  },
  "deeplink": "cursor://anysphere.cursor-deeplink/mcp/install?name=ai-crawler-check&config=eyJ0eXBlIjoiaHR0cCIsInVybCI6Imh0dHBzOi8vYXRhbGF5YS1haS5jb20vbWNwIn0%3D"
}
{
  "servers": {
    "ai-crawler-check": {
      "type": "http",
      "url": "https://atalaya-ai.com/mcp"
    }
  }
}
{
  "mcpServers": {
    "ai-crawler-check": {
      "httpUrl": "https://atalaya-ai.com/mcp"
    }
  }
}
extensions:
  "ai-crawler-check":
    enabled: true
    name: "ai-crawler-check"
    type: streamable_http
    uri: "https://atalaya-ai.com/mcp"
    timeout: 300

Quality 89 / 100new

protocol75 · weight 25
  • warn
    protocol.modernnewest supported version is 2025-11-25; 2026-07-28 not supported, older versions are deprecated until 2027-07-28
  • n/a
    protocol.statelessonly applies to 2026-07-28 servers
  • pass
    protocol.transportstreamable HTTP
  • n/a
    protocol.list_ttlonly applies to 2026-07-28 servers
authorizationn/a · weight 15
  • n/a
    auth.prm, auth.as_metadata, auth.cimdno remote uses OAuth
  • n/a
    auth.secret_in_urlno templated URL
tool hygiene100 · weight 30
  • pass
    tools.descriptionsevery tool has a description
  • pass
    tools.description_lengthdescriptions are concise
  • pass
    tools.schemasevery tool has an object input schema
  • pass
    tools.annotations2 of 2 tools declare readOnlyHint or destructiveHint
  • pass
    tools.directory_hintsevery tool declares readOnlyHint, destructiveHint, idempotentHint and openWorldHint
  • pass
    tools.token_costabout 350 tokens to load every tool
  • pass
    tools.api_dumptools are not a one-to-one API dump
stabilityn/a · weight 15
  • n/a
    stability.changes, stability.rug_pulltool history recorded for 3 of the 7 days needed
dependenciesn/a · weight 15
  • n/a
    deps.known_vulns, deps.mcp_sdk_version, deps.resolvableno npm or PyPI package

2 tools, about 350 tokens to load them all · tool set d774452416b6 · checked 2026-11-07 16:55 UTC

Tool changes

No tool definition changes recorded. A server's first observation is its baseline; later probes record what changes.

Checks run on what our credential-free, read-only probes observe; tools are never called and no code audit is performed. How it is computed.

Trust 80 / 100

no-repository
provenance55
liveness93
freshness100
hygiene80

+dns-namespace +website +reachable +uptime-83% +updated-4d-ago -no-repository

Derived from observable signals (official registry feed and our own credential-free probes); no code audit performed. How it is computed.

Endpoints

remoteauthreachableuptime 30dp50protocollast ok
https://atalaya-ai.com/mcp streamable-httpnoneyes83%263 ms2025-11-252026-11-07

Tools (2)

check_ai_crawler_accessread-onlyChecks whether AI crawlers can reach and read a public website or web app: which AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, PerplexityBot, Google-Extended and more) robots.txt allows and the exact rule for each, how much text the homepage HTML carries for crawlers that do not run JavaScript (flags client-rendered app shells, common in apps built with Lovable, Bolt, Base44 or Replit), title/meta description/H1, llms.txt, sitemap, and the platform or host setting to change (Cloudflare, Vercel, Netlify). Use it when someone asks why their site or app does not show up in ChatGPT, Claude, Perplexity or other AI answers.
list_ai_crawlersread-onlyLists the AI crawler user-agent tokens Atalaya checks, by vendor and purpose (training, search index, user-triggered fetch), so you can write a robots.txt that keeps AI search while opting out of training.

Schemas: list_tools.

Similar MCP servers

Official server.json
{
  "name": "com.atalaya-ai/ai-crawler-check",
  "icons": [
    {
      "src": "https://atalaya-ai.com/logo-512.png",
      "sizes": [
        "512x512"
      ],
      "mimeType": "image/png"
    },
    {
      "src": "https://atalaya-ai.com/favicon.svg",
      "sizes": [
        "any"
      ],
      "mimeType": "image/svg+xml"
    }
  ],
  "title": "Atalaya AI crawler check",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
  "remotes": [
    {
      "url": "https://atalaya-ai.com/mcp",
      "type": "streamable-http"
    }
  ],
  "version": "1.1.0",
  "websiteUrl": "https://atalaya-ai.com/mcp-server/",
  "description": "Which AI crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) a website allows, and why."
}