protogrid

Search / io.github.pipeworx-io/crawlbase

Crawlbase

R0· no sign-in

Crawlbase MCP — wraps the Crawlbase Crawling API (crawlbase.com, formerly

v0.1.2 · active · repository · website · descriptor JSON · versions

An agent can connect right now, no human step.Remote endpoint, no authentication, reachable on the last probe.
Quality–not scored
Trust97of 100
Uptime, 30 days100%
Latency p5028 ms

Connect

8 clients · secrets stay placeholders
{
  "mcpServers": {
    "crawlbase": {
      "type": "http",
      "url": "https://gateway.pipeworx.io/crawlbase/mcp"
    }
  }
}
claude mcp add --transport http crawlbase https://gateway.pipeworx.io/crawlbase/mcp
[mcp_servers.crawlbase]
url = "https://gateway.pipeworx.io/crawlbase/mcp"
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "crawlbase": {
      "type": "remote",
      "url": "https://gateway.pipeworx.io/crawlbase/mcp",
      "enabled": true
    }
  }
}
{
  "mcpServers": {
    "crawlbase": {
      "type": "http",
      "url": "https://gateway.pipeworx.io/crawlbase/mcp"
    }
  },
  "deeplink": "cursor://anysphere.cursor-deeplink/mcp/install?name=crawlbase&config=eyJ0eXBlIjoiaHR0cCIsInVybCI6Imh0dHBzOi8vZ2F0ZXdheS5waXBld29yeC5pby9jcmF3bGJhc2UvbWNwIn0%3D"
}
{
  "servers": {
    "crawlbase": {
      "type": "http",
      "url": "https://gateway.pipeworx.io/crawlbase/mcp"
    }
  }
}
{
  "mcpServers": {
    "crawlbase": {
      "httpUrl": "https://gateway.pipeworx.io/crawlbase/mcp"
    }
  }
}
extensions:
  "crawlbase":
    enabled: true
    name: "crawlbase"
    type: streamable_http
    uri: "https://gateway.pipeworx.io/crawlbase/mcp"
    timeout: 300

Quality – / 100not scored

Not scored: fewer than three checks apply, usually because no remote answered a credential-free probe or the server is a local package.

protocol75 · weight 25
  • warn
    protocol.modernnewest supported version is 2025-03-26; 2026-07-28 not supported, older versions are deprecated until 2027-07-28
  • n/a
    protocol.statelessonly applies to 2026-07-28 servers
  • pass
    protocol.transportstreamable HTTP
  • n/a
    protocol.list_ttlonly applies to 2026-07-28 servers
authorizationn/a · weight 15
  • n/a
    auth.prm, auth.as_metadata, auth.cimdno remote uses OAuth
  • n/a
    auth.secret_in_urlno templated URL
tool hygienen/a · weight 30
  • n/a
    tools.descriptions, tools.description_length, tools.schemas, tools.annotations, tools.directory_hints, tools.token_cost, tools.api_dumptool list unknown (behind auth, not probed, or empty)
stabilityn/a · weight 15
  • n/a
    stability.changes, stability.rug_pulltool list unknown
dependenciesn/a · weight 15
  • n/a
    deps.known_vulns, deps.mcp_sdk_version, deps.resolvableno npm or PyPI package

Tool list unknown · tools last changed 2026-10-20 · checked 2026-10-20 12:13 UTC

Tool changes

  • crawlbase_scraperemoved

    Scrape any website through Crawlbase rotating residential proxies with server-side anti-bot bypass (Cloudflare, DataDome, hCaptcha solved for you). Returns the page as HTML (default), clean markdown (format:"md" — great for LLM context), or a JSON envelope with metadata (format:"json"). JavaScript-heavy pages (SPAs) render when you pass your Crawlbase JavaScript token as _apiKey; the Normal token does fast static fetches. Example: crawlbase_scrape({ url: "https://example.com", format: "md", _apiKey: "your-crawlbase-token" })

  • forgetremoved

    Delete a previously stored memory by key. Use when context is stale, the task is done, or you want to clear sensitive data the agent saved earlier. Pair with remember and recall.

  • rememberremoved

    Save data the agent will need to reuse later — across this conversation or across sessions. Use when you discover something worth carrying forward (a resolved ticker, a target address, a user preference, a research subject) so you don't have to look it up again. Stored as a key-value pair scoped by your identifier. Authenticated users get persistent memory; anonymous sessions retain memory for 24 hours. Pair with recall to retrieve later, forget to delete.

  • pipeworx_feedbackremoved

    Tell the Pipeworx team something is broken, missing, or needs to exist. Use when a tool returns wrong/stale data (bug), when a tool you wish existed isn't in the catalog (feature/data_gap), or when something worked surprisingly well (praise). ONLY for tools served by this Pipeworx connection — if the tool came from a different MCP server in your client (another vendor's Gmail, Splunk, Slack, etc. connector), we cannot fix it and reporting it here only delays you; file it with that server instead. Not sure? Pipeworx tool names are the ones this connection lists. Describe the issue in terms of Pipeworx tools/packs — don't paste the end-user's prompt. Filing without an account returns a `claim_token`; pass it back later as pipeworx_feedback({claim_token:"pwfb_…"}) to read whether it was fixed and what changed. The team reads digests daily and signal directly affects roadmap. Rate-limited to 5 per identifier per day. Free; doesn't count against your tool-call quota.

  • scan_competitor_ai_presenceremoved

    Compare AI visibility across multiple entities side-by-side. Probes each entity (your brand + N competitors) with ai_visibility_check, ranks by score, surfaces which is most/least recognized. Useful for competitive AI-marketing audits: "does Claude know about us as well as our competitors?". Returns ranked list with score, confidence, signal density per entity.

  • polymarket_edge_trackerremoved

    Edge persistence and decay telemetry built from daily polymarket_edges snapshots. Answers "how long has this edge existed and is it shrinking?" — a fresh wide edge and a 3-week-old wide edge are different trades (the latter is wide for a reason nobody is willing to take). Args: days (lookback, default 14, max 30), window (snapshot family, default "1wk"). RESPONSE: tracked[] = every opportunity in the LATEST snapshot with its full edge_pp_net time-series across prior snapshots, first_seen, trend (new | widening | stable | decaying) and decay_pp_per_day (both computed on |edge_pp_net| — the value itself is signed by trade direction, negative = SELL YES); expired[] = opportunities that appeared in earlier snapshots but are GONE from the latest (closed, resolved, or arbed away) with their lifespan_days — the median lifespan is your competition clock; snapshot_dates[] = which days actually have data (snapshots are written when polymarket_edges runs on a cache-miss, so gaps mean nobody scanned that day). LIMITS: history depth is bounded by the 60-day snapshot TTL and starts from when snapshotting was enabled; decay numbers come from daily closes of edge_pp_net (net of default slippage AND Polymarket's own taker fee — see polymarket_edges), not intraday.

  • polymarket_fill_riskremoved

    Realizable-vs-theoretical edge check against live CLOB order-book depth. REQUIRES one of `market` (single-market mode) or `event` (basket/partition mode). SINGLE-MARKET: pass a market slug/URL + side (buy_yes|sell_yes|buy_no|sell_no, default buy_yes) + size_usd (default 1000 — max spend on buys, target proceeds on sells); walks the ladder and returns top_of_book, vwap_fill_price, slippage_pp, shares_filled, max_fillable_usd, and a verdict (clean|degraded|cannot_fill). BASKET: pass an event slug/URL + side (sell_yes = capture overround by selling every leg, buy_yes = capture underround; default auto from partition sum) + size_usd interpreted as settlement notional S (shares per leg; each share pays $1); returns theoretical_sum vs realizable_sum (top-of-book vs VWAP across all legs), capture_ratio, profit_usd at executed size, per-leg fill detail, thin_legs[], max_clean_notional_usd, and forced_directional_risk naming the legs most likely to strand you unhedged. USE THIS before acting on any polymarket_arbitrage SELL/BUY-EVERY-LEG signal or any polymarket_edges trade above ~$500 — theoretical overround on thin books is not capturable, and partial basket fills convert an arb into an unhedged directional position (the dominant loss mode in real arb-bot P&L). FEES ARE NOT MODELLED HERE: vwap_fill_price/profit_usd are GROSS of Polymarket's own taker fee (rate 0.04-0.07 by category — see polymarket_edges/fees.ts), on top of which this tool prices depth-crossing cost; a thin-margin fill that looks clean here can still be net-negative after the fee.

  • discover_toolsremoved

    Find tools by describing the data or task. Use when you need to browse, search, look up, or discover what tools exist for: SEC filings, financials, revenue, profit, FDA drugs, adverse events, FRED economic data, Census demographics, BLS jobs/unemployment/inflation, ATTOM real estate, ClinicalTrials, USPTO patents, weather, news, crypto, stocks. Returns the top-N most relevant tools with names, descriptions, and full input schemas (with curated examples) — each result is ready to call directly, no second schema lookup needed. Call this FIRST when you have many tools available and want to see the option set (not just one answer).

  • deep_researchremoved

    ACCOUNT REQUIRED (free — sign in via GitHub at https://pipeworx.io/signup; depth:"thorough" needs a paid plan). If you are not signed in, use ask_pipeworx instead — it works on every tier. Grounded multi-source research across Pipeworx's 1847 STRUCTURED data sources (SEC filings, FRED/BLS economics, FDA, USPTO patents, markets, science, government records, etc.) in ONE call — this is NOT open-web search. Decomposes your question into focused facets, routes each to the right one of 7,013 tools IN PARALLEL, and returns a findings packet: verbatim evidence + confidence + source + fetched_at + a stable pipeworx:// citation per finding, with explicit gaps[] for facets the data couldn't answer (never invented). Best for broad/multi-part questions over structured data ("compare X and Y's regulatory + financial exposure", "research the filings + market picture for ACME"). For a single lookup use ask_pipeworx (one LLM call, not many). For BREAKING or colloquial CURRENT-NEWS / "what's the world saying about X" topics, prefer ask_pipeworx — it routes to live news APIs and the *-news-feeds packs; deep_research returns mostly empty gaps[] when the topic isn't in the structured catalog. Second-hop iteration: depth:"standard" re-angles unanswered gaps (gap recovery); depth:"thorough" additionally chases the best leads from the first pass — so multi-step questions resolve in one call. Every finding carries a `hop` field and a citation_uri — a resolvable pipeworx:// record URI, present only when the source emits one that resources/read can actually serve, so a citation you get back is always fetchable. "standard" and "thorough" also return contradictions[] flagging findings that disagree. Large records are semantically excerpted to the passages relevant to each facet (not head-truncated), so answers deep in a long filing/series aren't missed. Expect 15-60s (thorough with its follow-up + contradiction pass: up to ~90s).

  • company_factsremoved

    TYPED, DETERMINISTIC financial facts for a US public company for an EXPLICITLY NAMED reporting period — "Apple revenue for fiscal 2023", "Walmart net income FY2026 Q3", "Microsoft cash at the end of fiscal 2024". PREFER OVER entity_profile / get_company_financials whenever the period matters: those answer "the most recent figures" and will happily hand back FY2025 when you asked about FY2019, and neither separates a discrete quarter from a year-to-date figure. This one refuses instead — it NEVER substitutes the latest period for the period requested, NEVER returns 0 for missing data, NEVER lets a 9-month YTD number answer a quarterly question, and NEVER converts a currency. Every answer carries the exact us-gaap concept it came from, what that concept MEASURES (NetIncomeLoss excludes non-controlling interests, ProfitLoss includes them — not synonyms), the accession number and a link to the filing on sec.gov, the restatement trail of any superseded figures, and a contract + derivation version to pin against. Fiscal periods are the FILER'S OWN, anchored on their fiscal-year end, so Walmart's year ending 2026-01-31 is FY2026 and Apple's ending 2025-09-27 is FY2025. Attributes: revenue, net_income, cash, gross_profit, operating_income, eps_diluted, capex, operating_cash_flow, long_term_debt, shares_outstanding (the dei COVER-PAGE share count, so its period end is the cover date, after the quarter closed — that is the most current count the filing carries). Units are not all USD: shares_outstanding is in `shares` with currency null, eps_diluted in `USD/shares`. A discrete quarter the filer never filed — every Q4, and Q2/Q3 of cash-flow lines, which are filed year-to-date — is DERIVED as YTD(q) minus YTD(q-1) and comes back with `reported: false`, the subtraction named in `derivation` (fy_minus_9m / ytd_9m_minus_6m / ytd_6m_minus_3m) and both legs with their filings in `derived_from`; test `reported` if you will only accept a figure lifted from a filing. Per-share and balance-sheet figures are NEVER derived. Every non-answer is a named status — `unavailable` (the filer did not report it for that period; the periods that DO exist are listed, without values), `unsupported` (outside what v1 covers — a non-us-gaap filer, an unknown attribute, a non-USD unit), `ambiguous` (the company name matched two filers equally well; both are named), `conflicting` (two filings the same day disagree; both are returned and neither is picked), `partial` (a value with no accession behind it). Source: SEC EDGAR, XBRL financial facts, one publisher read once — see `corroboration`. Same response is served at POST https://gateway.pipeworx.io/v1/facts for non-MCP callers.

Showing the latest 10. Full history: list_changes · trend: quality JSON.

Checks run on what our credential-free, read-only probes observe; tools are never called and no code audit is performed. How it is computed.

Trust 97 / 100

provenance90
liveness100
freshness100
hygiene100

+repository +namespace-matches-repo +website +reachable +uptime-100% +updated-25d-ago

Derived from observable signals (official registry feed and our own credential-free probes); no code audit performed. How it is computed.

Endpoints

remoteauthreachableuptime 30dp50protocollast ok
https://gateway.pipeworx.io/crawlbase/mcp streamable-httpnoneyes100%28 ms2025-03-262026-10-20

Tools (0)

No tools observed. Tools appear once a credential-free probe can list them.

Schemas: list_tools.

Similar MCP servers

More from io.github.pipeworx-io

And 1,855 more from this publisher.

Official server.json
{
  "name": "io.github.pipeworx-io/crawlbase",
  "title": "Crawlbase",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
  "remotes": [
    {
      "url": "https://gateway.pipeworx.io/crawlbase/mcp",
      "type": "streamable-http"
    }
  ],
  "version": "0.1.2",
  "repository": {
    "url": "https://github.com/pipeworx-io/mcp-crawlbase",
    "source": "github"
  },
  "websiteUrl": "https://pipeworx.io/packs/crawlbase",
  "description": "Crawlbase MCP — wraps the Crawlbase Crawling API (crawlbase.com, formerly"
}