Search / tech.ratlabs/rat-radar
protogridR0· no sign-inAsk before you scrape: is a site open, may I fetch this URL, and how do I extract a page. No key.
v1.4.0 · active · website · descriptor JSON · versions
{
"mcpServers": {
"rat-radar": {
"type": "http",
"url": "https://rat-radar.github-a18.workers.dev/mcp"
}
}
}claude mcp add --transport http rat-radar https://rat-radar.github-a18.workers.dev/mcp[mcp_servers.rat-radar]
url = "https://rat-radar.github-a18.workers.dev/mcp"
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"rat-radar": {
"type": "remote",
"url": "https://rat-radar.github-a18.workers.dev/mcp",
"enabled": true
}
}
}{
"mcpServers": {
"rat-radar": {
"type": "http",
"url": "https://rat-radar.github-a18.workers.dev/mcp"
}
},
"deeplink": "cursor://anysphere.cursor-deeplink/mcp/install?name=rat-radar&config=eyJ0eXBlIjoiaHR0cCIsInVybCI6Imh0dHBzOi8vcmF0LXJhZGFyLmdpdGh1Yi1hMTgud29ya2Vycy5kZXYvbWNwIn0%3D"
}{
"servers": {
"rat-radar": {
"type": "http",
"url": "https://rat-radar.github-a18.workers.dev/mcp"
}
}
}{
"mcpServers": {
"rat-radar": {
"httpUrl": "https://rat-radar.github-a18.workers.dev/mcp"
}
}
}extensions:
"rat-radar":
enabled: true
name: "rat-radar"
type: streamable_http
uri: "https://rat-radar.github-a18.workers.dev/mcp"
timeout: 300
protocol.modernprotocol.statelessprotocol.transportprotocol.list_ttlauth.prm, auth.as_metadata, auth.cimdauth.secret_in_urltools.descriptionstools.description_lengthtools.schemastools.annotationstools.directory_hintstools.token_costtools.api_dumptools.hidden_textstability.changesstability.rug_pullstability.new_linksdeps.known_vulns, deps.mcp_sdk_version, deps.resolvablecheck_sitechangeddescriptionCan I scrape this site, and how? Returns a verdict (open, guarded, hostile, blocked), a hostility score, a recommended approach with steps and risks, bot protections, robots.txt and AI-crawler policy, machine-readable surfaces (sitemap, llms.txt, JSON-LD), and eu_ai_training: the signals the EDPB draft Guidelines 03/2026 on web scraping for generative AI count against collecting personal data (robots.txt, ai.txt, CAPTCHAs, login walls), as none, some or strong. Signals, not legal advice. Uses a weekly-probed dataset of several hundred sites; for any other domain it probes live from a datacenter IP.
Can I scrape this site, and how? Returns a verdict (open, guarded, hostile, blocked), a hostility score, a recommended approach with steps and risks, bot protections, robots.txt and AI-crawler policy, machine-readable surfaces (sitemap, llms.txt, JSON-LD), and eu_ai_training: the signals the EDPB draft Guidelines 03/2026 on web scraping for generative AI count against collecting personal data (robots.txt blocks on AI training crawlers, a Content-Signal of ai-train=no, ai.txt, CAPTCHAs, login walls), as none, some or strong. Signals, not legal advice. Uses a weekly-probed dataset of several hundred sites; for any other domain it probes live from a datacenter IP.
Full history: list_changes · trend: quality JSON.
Checks run on what our credential-free, read-only probes observe; tools are never called and no code audit is performed. How it is computed.
+dns-namespace +website +reachable +uptime-100% +updated-16d-ago -no-repository
Derived from observable signals (official registry feed and our own credential-free probes); no code audit performed. How it is computed.
| remote | auth | reachable | uptime 30d | p50 | protocol | last ok |
|---|---|---|---|---|---|---|
https://rat-radar.github-a18.workers.dev/mcp streamable-http | none | yes | 100% | 189 ms | 2025-11-25 | 2026-11-14 |
can_fetch | Does robots.txt allow fetching this exact URL? Fetches the live robots.txt and applies the longest-match rule for a user agent (default: * plus GPTBot, ClaudeBot, Google-Extended, CCBot, PerplexityBot). Returns allowed or not, the rule that decided it, and any crawl delay. A courtesy check, not a legal opinion. |
check_site | Can I scrape this site, and how? Returns a verdict (open, guarded, hostile, blocked), a hostility score, a recommended approach with steps and risks, bot protections, robots.txt and AI-crawler policy, machine-readable surfaces (sitemap, llms.txt, JSON-LD), and eu_ai_training: the signals the EDPB draft Guidelines 03/2026 on web scraping for generative AI count against collecting personal data (robots.txt blocks on AI training crawlers, a Content-Signal of ai-train=no, ai.txt, CAPTCHAs, login walls), as none, some or strong. Signals, not legal advice. Uses a weekly-probed dataset of several hundred sites; for any other domain it probes live from a datacenter IP. |
get_playbook | What should I build with this site, and how? Give a domain and, optionally, a goal in plain words (for example "price alerts", "sales leads", "new jobs", "change log", "knowledge base"). Returns the best-matching workflow for that site: what it does, its cadence, a trigger, collect, shape, store, act pipeline, the fields to keep, and step-by-step build recipes for n8n, Make, Zapier and Apify (or just one with platform). Workflows marked found_on_this_site come from what the weekly probe saw on the site itself. Also returns the other workflows, the golden data, and the guardrails. |
list_sites | List the sites in the dataset, optionally by category or verdict. Use it to find easy targets or to see what is covered. |
search_sites | Find sites in the dataset by a word: a name, host, category (jobs, shop, news, gov), country or verdict. Use it to discover what is covered before calling check_site. |
summary | Headline numbers across the dataset: verdict mix, how many sites block AI crawlers (and which), protection vendors, by category and country. |
xray_diagram | Draw a page's anatomy as an Excalidraw diagram: its regions, the ranked ways to get its data, proof rows, the best route and what the page is made of. Returns a download link for the .excalidraw file (give it to the user; it opens at excalidraw.com or in Obsidian) and a file_name. With include_scene, also the scene itself (a few hundred KB). |
xray_page | How do I extract the data from THIS page? Fetches one public page and returns a ranked extraction plan: embedded JSON, JSON-LD, repeated records with CSS selectors and field names, tables, feeds, hidden API calls. Each source has a reliability, proof rows pulled from the real page, and the plan notes pagination and blockers. Set include_code for a starter Python script. Works on the raw HTML only, so JavaScript-built pages report a blocker instead of data. Use check_site first to learn whether the site allows scraping at all. |
Schemas: list_tools.
{
"name": "tech.ratlabs/rat-radar",
"icons": [
{
"src": "https://ratlabs.tech/radar/assets/icons/icon-192.png",
"sizes": [
"192x192"
],
"mimeType": "image/png"
}
],
"title": "Rat Radar",
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
"remotes": [
{
"url": "https://rat-radar.github-a18.workers.dev/mcp",
"type": "streamable-http"
}
],
"version": "1.4.0",
"websiteUrl": "https://ratlabs.tech/radar/",
"description": "Ask before you scrape: is a site open, may I fetch this URL, and how do I extract a page. No key."
}