Connect

8 clients · secrets stay placeholders

JSON (mcpServers)Claude CodeCodexOpenCodeCursorVS CodeGemini CLIGoose

{
  "mcpServers": {
    "octocrawl": {
      "type": "http",
      "url": "https://mcp.octocrawl.dev/mcp"
    }
  }
}

claude mcp add --transport http octocrawl https://mcp.octocrawl.dev/mcp

[mcp_servers.octocrawl]
url = "https://mcp.octocrawl.dev/mcp"

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "octocrawl": {
      "type": "remote",
      "url": "https://mcp.octocrawl.dev/mcp",
      "enabled": true
    }
  }
}

{
  "mcpServers": {
    "octocrawl": {
      "type": "http",
      "url": "https://mcp.octocrawl.dev/mcp"
    }
  },
  "deeplink": "cursor://anysphere.cursor-deeplink/mcp/install?name=octocrawl&config=eyJ0eXBlIjoiaHR0cCIsInVybCI6Imh0dHBzOi8vbWNwLm9jdG9jcmF3bC5kZXYvbWNwIn0%3D"
}

{
  "servers": {
    "octocrawl": {
      "type": "http",
      "url": "https://mcp.octocrawl.dev/mcp"
    }
  }
}

{
  "mcpServers": {
    "octocrawl": {
      "httpUrl": "https://mcp.octocrawl.dev/mcp"
    }
  }
}

extensions:
  "octocrawl":
    enabled: true
    name: "octocrawl"
    type: streamable_http
    uri: "https://mcp.octocrawl.dev/mcp"
    timeout: 300

Quality 83 / 100good

protocol88 · weight 25

pass
protocol.modernsupports 2026-07-28 (server/discover)
pass
protocol.statelessanswers without a session
pass
protocol.transportstreamable HTTP
warn
protocol.list_ttltools/list declares ttlMs 0, so clients re-fetch it on every use

authorization100 · weight 15

pass
auth.prmProtected Resource Metadata (RFC 9728) published
pass
auth.as_metadataauthorization server metadata reachable
pass
auth.cimdClient ID Metadata Documents supported
n/a
auth.secret_in_urlno templated URL

tool hygiene100 · weight 30

pass
tools.descriptionsevery tool has a description
pass
tools.description_lengthdescriptions are concise
pass
tools.schemasevery tool has an object input schema
pass
tools.annotations2 of 2 tools declare readOnlyHint or destructiveHint
pass
tools.directory_hintsevery tool declares readOnlyHint, destructiveHint, idempotentHint and openWorldHint
pass
tools.token_costabout 2,466 tokens to load every tool
pass
tools.api_dumptools are not a one-to-one API dump

stability25 · weight 15

warn
stability.changes13 tool changes in 30 days
fail
stability.rug_pull1 tools kept their name but changed most of their description in 30 days; review before trusting them

dependenciesn/a · weight 15

n/a
deps.known_vulns, deps.mcp_sdk_version, deps.resolvabledeps.dev does not know @octocrawl/[email protected]

2 tools, about 2,466 tokens to load them all · tool set a5d1aa20ae35 · tools last changed 2026-10-19 · checked 2026-10-20 13:17 UTC

Tool changes

2026-10-19scrapechangedinput schema
2026-10-18mapchangedinput schema
2026-10-17scrapechangedoutput schema
2026-10-17mapchangeddescription, input schema, output schema
before
List a site's URLs without fetching each page: the start URL, the links on its page (read on the http lane alone; no browser) and the entries of the sitemaps the site declares (robots.txt Sitemap: lines, else /sitemap.xml), inside one deadline. Every URL is in the crawl's scope (the start host and its www twin, the start URL's path subtree, assets left out, similar URLs folded) and allowed by its host's robots.txt unless ignoreRobotsTxt is set; what was left out is counted. A title is never fetched: the start page's own, an anchor's text or a sitemap's news title. At the deadline the answer is what was found, status partial (failed when nothing), stoppedBy timeout. Compact by default ({ id, status, stoppedBy, links: [{ url, title?, description?, robots? }], warning?, agentHints?, counts }; robots only on a link robots.txt keeps out, under ignoreRobotsTxt); debug=true returns the full map with each link's evidence (via, sitemapFile, lastmod, robots), the sources read and the refusals. One page body is read at most: a site without a sitemap maps only its start page's links; crawl reads further pages.
after · 54% of words in common
List a site's URLs without reading each page: the start page's links (read over HTTP alone) and the entries of the sitemaps the site declares, within one deadline. URLs stay in scope (the start host and its www twin, the start path's subtree, assets left out, similar URLs folded) and robots.txt applies unless ignoreRobotsTxt is set; what was left out is counted. No page title is fetched. At the deadline the answer is what was found (partial, stoppedBy timeout). Compact by default; debug adds each link's evidence and the sources read. Not for: reading pages, as one page body is read at most.
2026-10-17scrapechangeddescription, input schemameaning changed
before
Fetch one URL through the Octocrawl coverage ladder. Compact by default; set debug=true for the full audit. The result's warnings name what its content cannot vouch for: robots_overridden (robots.txt disallows the URL; a local server fetched it because you named it), or client_rendered_suspected when the HTTP page looks like a shell its scripts fill in and the browser rung found nothing better. Its agentHints, when present, say what to change next time (a login wall, a robots.txt rule, a gate, a wait).
after · 40% of words in common
Read one URL as Markdown with its Evidence Record (final URL, fetch time, HTTP status, robots.txt decision, hashes), over HTTP first and in the browser when the page needs it. Not for: many URLs or a whole site section. Compact by default; debug adds the full audit. warnings name what the content cannot vouch for (client_rendered_suspected, robots_overridden), and agentHints the next step.
2026-10-17mapchangedtitle, input schema, annotations
2026-10-17scrapechangedtitle, input schema, annotations
2026-10-16scrapechangeddescription
before
Fetch one URL through the Octocrawl coverage ladder. Compact by default; set debug=true for the full audit. The result's warnings name what its content cannot vouch for: robots_overridden (robots.txt disallows the URL; a local server fetched it because you named it), or client_rendered_suspected when the HTTP page looks like a shell its scripts fill in and the browser rung found nothing better. Its agentHints, when present, say what to change next time (a login wall, a robots.txt rule, a gate, a wait). metadata.scrapeId names the call's record for get_scrape.
after · 91% of words in common
Fetch one URL through the Octocrawl coverage ladder. Compact by default; set debug=true for the full audit. The result's warnings name what its content cannot vouch for: robots_overridden (robots.txt disallows the URL; a local server fetched it because you named it), or client_rendered_suspected when the HTTP page looks like a shell its scripts fill in and the browser rung found nothing better. Its agentHints, when present, say what to change next time (a login wall, a robots.txt rule, a gate, a wait).
2026-10-15scrapechangedinput schema
2026-10-12scrapechangedinput schema

Showing the latest 10. Full history: list_changes · trend: quality JSON.

Checks run on what our credential-free, read-only probes observe; tools are never called and no code audit is performed. How it is computed.

Trust 97 / 100

provenance90

liveness100

freshness100

hygiene100

+repository +namespace-matches-repo +website +reachable +uptime-100% +updated-3d-ago

Derived from observable signals (official registry feed and our own credential-free probes); no code audit performed. How it is computed.

Dependencies

@octocrawl/[email protected] npmthis version is not known to the dependency resolver

Dependency graphs from deps.dev; advisories from OSV.dev, including the GitHub Advisory Database and the PyPI Advisory Database; all CC BY 4.0. The graph is what a clean install of this version resolves today; nothing was installed or run. How it is checked.

Endpoints

remote	auth	reachable	uptime 30d	p50	protocol	last ok
`https://mcp.octocrawl.dev/mcp` streamable-http	none	yes	100%	650 ms	2026-07-28	2026-10-20
`https://mcp.octocrawl.dev/mcp-oauth` streamable-http	oauth2 AS: https://octocrawl.dev	no (1 failures)	94%	682 ms	–	2026-10-20

package	registry	version	runtime	secrets
`@octocrawl/mcp`	npm	0.7.0	–	`W2L_API_TOKEN`

Tools (2)

mapread-only List a site's URLs without reading each page: the start page's links (read over HTTP alone) and the entries of the sitemaps the site declares, within one deadline. URLs stay in scope (the start host and its www twin, the start path's subtree, assets left out, similar URLs folded) and robots.txt applies unless ignoreRobotsTxt is set; what was left out is counted. No page title is fetched. At the deadline the answer is what was found (partial, stoppedBy timeout). Compact by default; debug adds each link's evidence and the sources read. Not for: reading pages, as one page body is read at most.

scraperead-only Read one URL as Markdown with its Evidence Record (final URL, fetch time, HTTP status, robots.txt decision, hashes), over HTTP first and in the browser when the page needs it. Not for: many URLs or a whole site section. Compact by default; debug adds the full audit. warnings name what the content cannot vouch for (client_rendered_suspected, robots_overridden), and agentHints the next step.

Schemas: list_tools.

Similar MCP servers

Official server.json

{
  "name": "io.github.77777R7/octocrawl",
  "title": "Octocrawl",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
  "remotes": [
    {
      "url": "https://mcp.octocrawl.dev/mcp",
      "type": "streamable-http"
    },
    {
      "url": "https://mcp.octocrawl.dev/mcp-oauth",
      "type": "streamable-http"
    }
  ],
  "version": "0.7.0",
  "packages": [
    {
      "version": "0.7.0",
      "transport": {
        "type": "stdio"
      },
      "identifier": "@octocrawl/mcp",
      "registryType": "npm",
      "registryBaseUrl": "https://registry.npmjs.org",
      "environmentVariables": [
        {
          "name": "W2L_API_URL",
          "description": "The Octocrawl API the server calls: npx octocrawl serve on this computer (default http://127.0.0.1:8787), or https://api.octocrawl.dev"
        },
        {
          "name": "W2L_API_TOKEN",
          "isSecret": true,
          "description": "A bearer token for a hosted or self-hosted API; none for a local one"
        },
        {
          "name": "W2L_MCP_TOOLSETS",
          "description": "The tool groups to list, comma-separated: core, crawl, batch, products, logins, monitor, delivery, or all (default all); the same as --toolsets"
        },
        {
          "name": "W2L_MCP_READ_ONLY",
          "description": "1 lists read-only tools alone, with a scrape that does not act as the person; the same as --read-only"
        }
      ]
    }
  ],
  "repository": {
    "url": "https://github.com/77777R7/Octocrawl",
    "source": "github"
  },
  "websiteUrl": "https://octocrawl.dev/docs/connect-mcp/?utm_source=mcp-registry&utm_medium=listing",
  "description": "Web pages as clean Markdown and JSON for AI agents, in fewer tokens, with an Evidence Record."
}