Connect

8 clients · secrets stay placeholders

JSON (mcpServers)Claude CodeCodexOpenCodeCursorVS CodeGemini CLIGoose

{
  "mcpServers": {
    "robotsgate": {
      "type": "http",
      "url": "https://robotsgate.mike-tusa.workers.dev/mcp",
      "headers": {
        "Authorization": "Bearer ${ROBOTSGATE_TOKEN}"
      }
    }
  }
}

claude mcp add --transport http robotsgate https://robotsgate.mike-tusa.workers.dev/mcp --header "Authorization: Bearer ${ROBOTSGATE_TOKEN}"

[mcp_servers.robotsgate]
url = "https://robotsgate.mike-tusa.workers.dev/mcp"
bearer_token_env_var = "ROBOTSGATE_TOKEN"

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "robotsgate": {
      "type": "remote",
      "url": "https://robotsgate.mike-tusa.workers.dev/mcp",
      "enabled": true,
      "headers": {
        "Authorization": "Bearer {env:ROBOTSGATE_TOKEN}"
      }
    }
  }
}

{
  "mcpServers": {
    "robotsgate": {
      "type": "http",
      "url": "https://robotsgate.mike-tusa.workers.dev/mcp",
      "headers": {
        "Authorization": "Bearer ${ROBOTSGATE_TOKEN}"
      }
    }
  },
  "deeplink": "cursor://anysphere.cursor-deeplink/mcp/install?name=robotsgate&config=eyJ0eXBlIjoiaHR0cCIsInVybCI6Imh0dHBzOi8vcm9ib3RzZ2F0ZS5taWtlLXR1c2Eud29ya2Vycy5kZXYvbWNwIiwiaGVhZGVycyI6eyJBdXRob3JpemF0aW9uIjoiQmVhcmVyICR7Uk9CT1RTR0FURV9UT0tFTn0ifX0%3D"
}

{
  "servers": {
    "robotsgate": {
      "type": "http",
      "url": "https://robotsgate.mike-tusa.workers.dev/mcp",
      "headers": {
        "Authorization": "Bearer ${ROBOTSGATE_TOKEN}"
      }
    }
  }
}

{
  "mcpServers": {
    "robotsgate": {
      "httpUrl": "https://robotsgate.mike-tusa.workers.dev/mcp",
      "headers": {
        "Authorization": "Bearer ${ROBOTSGATE_TOKEN}"
      }
    }
  }
}

extensions:
  "robotsgate":
    enabled: true
    name: "robotsgate"
    type: streamable_http
    uri: "https://robotsgate.mike-tusa.workers.dev/mcp"
    headers:
      "Authorization": "Bearer ${ROBOTSGATE_TOKEN}"
    timeout: 300

Placeholders to fill from your own secret store: ${ROBOTSGATE_TOKEN}. The registry never accepts secret values.

Quality 100 / 100new

protocol100 · weight 25

pass
protocol.modernsupports 2026-07-28 (server/discover)
pass
protocol.statelessanswers without a session
pass
protocol.transportstreamable HTTP
pass
protocol.list_ttltool list is cacheable (ttlMs 3600000)

authorizationn/a · weight 15

n/a
auth.prm, auth.as_metadata, auth.cimdno remote uses OAuth
n/a
auth.secret_in_urlno templated URL

tool hygiene100 · weight 30

pass
tools.descriptionsevery tool has a description
pass
tools.description_lengthdescriptions are concise
pass
tools.schemasevery tool has an object input schema
pass
tools.annotations4 of 4 tools declare readOnlyHint or destructiveHint
pass
tools.directory_hintsevery tool declares readOnlyHint, destructiveHint, idempotentHint and openWorldHint
pass
tools.token_costabout 1,204 tokens to load every tool
pass
tools.api_dumptools are not a one-to-one API dump

stabilityn/a · weight 15

n/a
stability.changes, stability.rug_pulltool history recorded for 2 of the 7 days needed

dependenciesn/a · weight 15

n/a
deps.known_vulns, deps.mcp_sdk_version, deps.resolvableno npm or PyPI package

4 tools, about 1,204 tokens to load them all · tool set 1704eb6c05d2 · checked 2026-10-19 10:20 UTC

Tool changes

No tool definition changes recorded. A server's first observation is its baseline; later probes record what changes.

Checks run on what our credential-free, read-only probes observe; tools are never called and no code audit is performed. How it is computed.

Trust 81 / 100

no-repository

provenance55

liveness96

freshness100

hygiene80

+dns-namespace +website +reachable +uptime-91% +updated-3d-ago -no-repository

Derived from observable signals (official registry feed and our own credential-free probes); no code audit performed. How it is computed.

Endpoints

remote	auth	reachable	uptime 30d	p50	protocol	last ok
`https://robotsgate.mike-tusa.workers.dev/mcp` streamable-http	none	yes	91%	197 ms	2026-07-28, 2025-11-25, 2025-06-18, 2025-03-26, 2024-11-05	2026-10-19

Tools (4)

`check_site`read-only	Fetch a site's /robots.txt (http/https on the default port only; private, loopback and internal addresses are refused; up to 512,000 bytes; results cached for 5 minutes) and report, for one path, which registry AI crawlers are allowed or blocked, plus validation errors and warnings. Read-only: one GET of /robots.txt with a RobotsGate user-agent. Same engine as GET /api/check.
`generate_robots`read-only	Build a robots.txt from per-category choices (allow, block or omit for training, dataset, search, user_fetch or other), optional per-token overrides, a default policy for other bots, paths, sitemaps and custom rules. Returns the file text, a per-crawler summary and notes. Read-only: it only returns text and changes nothing. Same engine as POST /api/generate.
`list_crawlers`read-only	Return RobotsGate's AI crawler registry: each crawler's robots.txt token, operator, category (training, dataset, search, user_fetch or other), whether the operator says it respects robots.txt, and a link to the operator's documentation, plus unverified entries and notes. Read-only, no network access. Same data as GET /api/crawlers.
`validate_robots`read-only	Validate robots.txt text against RFC 9309 and report errors, warnings and, for one URL path, which registry AI crawlers are allowed or blocked and by which rule. Read-only, no network access. Same engine as POST /api/validate. Up to 50,000 characters over MCP (the whole JSON-RPC request must also fit in 64 KB); for larger files use POST /api/validate or check_site.

Schemas: list_tools.

Similar MCP servers

Official server.json

{
  "name": "dev.workers.mike-tusa.robotsgate/robotsgate",
  "title": "RobotsGate",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
  "remotes": [
    {
      "url": "https://robotsgate.mike-tusa.workers.dev/mcp",
      "type": "streamable-http",
      "headers": [
        {
          "name": "Authorization",
          "isSecret": true,
          "description": "Optional. 'Bearer <RobotsGate Pro license key>' for higher limits. Works without a key."
        }
      ]
    }
  ],
  "version": "0.3.1",
  "websiteUrl": "https://robotsgate.mike-tusa.workers.dev/",
  "description": "Generate, validate and check robots.txt rules for AI crawlers. Free; optional Pro key."
}