protogrid

Search / io.github.mauricekleine/nonobench

Nonobench

R0· no sign-in

An open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20.

v1.1.0 · active · repository · website · descriptor JSON · versions

An agent can connect right now, no human step.Remote endpoint, no authentication, reachable on the last probe.
Quality89good
Trust97of 100
Uptime, 30 days96%
Latency p50451 ms

Connect

8 clients · secrets stay placeholders
{
  "mcpServers": {
    "nonobench": {
      "type": "http",
      "url": "https://www.nonobench.com/mcp"
    }
  }
}
claude mcp add --transport http nonobench https://www.nonobench.com/mcp
[mcp_servers.nonobench]
url = "https://www.nonobench.com/mcp"
{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "nonobench": {
      "type": "remote",
      "url": "https://www.nonobench.com/mcp",
      "enabled": true
    }
  }
}
{
  "mcpServers": {
    "nonobench": {
      "type": "http",
      "url": "https://www.nonobench.com/mcp"
    }
  },
  "deeplink": "cursor://anysphere.cursor-deeplink/mcp/install?name=nonobench&config=eyJ0eXBlIjoiaHR0cCIsInVybCI6Imh0dHBzOi8vd3d3Lm5vbm9iZW5jaC5jb20vbWNwIn0%3D"
}
{
  "servers": {
    "nonobench": {
      "type": "http",
      "url": "https://www.nonobench.com/mcp"
    }
  }
}
{
  "mcpServers": {
    "nonobench": {
      "httpUrl": "https://www.nonobench.com/mcp"
    }
  }
}
extensions:
  "nonobench":
    enabled: true
    name: "nonobench"
    type: streamable_http
    uri: "https://www.nonobench.com/mcp"
    timeout: 300

Quality 89 / 100good

protocol100 · weight 25
  • pass
    protocol.modernsupports 2026-07-28 (server/discover)
  • pass
    protocol.statelessanswers without a session
  • pass
    protocol.transportstreamable HTTP
  • pass
    protocol.list_ttltool list is cacheable (ttlMs 3600000)
authorizationn/a · weight 15
  • n/a
    auth.prm, auth.as_metadata, auth.cimdno remote uses OAuth
  • n/a
    auth.secret_in_urlno templated URL
tool hygiene86 · weight 30
  • pass
    tools.descriptionsevery tool has a description
  • pass
    tools.description_lengthdescriptions are concise
  • pass
    tools.schemasevery tool has an object input schema
  • pass
    tools.annotations11 of 11 tools declare readOnlyHint or destructiveHint
  • fail
    tools.directory_hintsno tool declares all four of readOnlyHint, destructiveHint, idempotentHint and openWorldHint; missing: list_families (destructiveHint, idempotentHint); compare_models (destructiveHint, idempotentHint); get_model_results (destructiveHint, idempotentHint); …
  • pass
    tools.token_costabout 1,464 tokens to load every tool
  • pass
    tools.api_dumptools are not a one-to-one API dump
stability75 · weight 15
  • warn
    stability.changes11 tool changes in 30 days
  • pass
    stability.rug_pullno tool changed its meaning under the same name
dependenciesn/a · weight 15
  • n/a
    deps.known_vulns, deps.mcp_sdk_version, deps.resolvableno npm or PyPI package

11 tools, about 1,464 tokens to load them all · tool set ccce2e6a6d8e · tools last changed 2026-09-30 · checked 2026-10-20 15:26 UTC

Tool changes

  • list_runschangedinput schema, output schema, annotations
  • get_model_puzzleschangedinput schema, output schema, annotations
  • get_puzzle_resultschangedinput schema, output schema, annotations
  • check_solutionchangedoutput schema, annotations
  • get_puzzlechangedoutput schema, annotations
  • list_puzzleschangedoutput schema, annotations
  • get_model_resultschangedoutput schema, annotations
  • compare_modelschangedinput schema, output schema, annotations
  • list_familieschangedoutput schema, annotations
  • list_providerschangedoutput schema, annotations

Showing the latest 10. Full history: list_changes · trend: quality JSON.

Checks run on what our credential-free, read-only probes observe; tools are never called and no code audit is performed. How it is computed.

Trust 97 / 100

provenance90
liveness99
freshness100
hygiene100

+repository +namespace-matches-repo +website +reachable +uptime-96% +updated-21d-ago

Derived from observable signals (official registry feed and our own credential-free probes); no code audit performed. How it is computed.

Endpoints

remoteauthreachableuptime 30dp50protocollast ok
https://www.nonobench.com/mcp streamable-httpnoneyes96%451 ms2026-07-282026-10-20

Tools (11)

check_solutionread-onlyCheck a grid against a puzzle's clues, using the same rule as the benchmark grader. Reports which rows and columns do not match.
compare_modelsread-onlySide-by-side core overall and per-size accuracy, cost, latency and token results for model or family names.
get_leaderboardread-onlyModels ranked by accuracy; defaults to all effort levels for compatibility.
get_model_puzzlesread-onlyWhich puzzles one model solved, missed, timed out on, or has not run.
get_model_resultsread-onlyAccuracy, cost, latency and token use for one model, broken down by grid size.
get_puzzleread-onlyOne puzzle, including the clue text models were prompted with. The reference solution is only included on request; some puzzles have several valid solutions.
get_puzzle_resultsread-onlyPer-model outcomes for one puzzle. Answers are omitted unless requested.
list_familiesread-onlyModel families, available efforts and best variants.
list_providersread-onlyProvider ids, names, families and variant counts.
list_puzzlesread-onlyThe benchmark puzzles with their ids and row/column clues.
list_runsread-onlyIndividual benchmark runs (one model on one puzzle), optionally with the raw prompt and model output.

Schemas: list_tools.

Similar MCP servers

Official server.json
{
  "name": "io.github.mauricekleine/nonobench",
  "icons": [
    {
      "src": "https://www.nonobench.com/icon.svg",
      "mimeType": "image/svg+xml"
    }
  ],
  "title": "Nonobench",
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
  "remotes": [
    {
      "url": "https://www.nonobench.com/mcp",
      "type": "streamable-http"
    }
  ],
  "version": "1.1.0",
  "repository": {
    "id": "1124680684",
    "url": "https://github.com/mauricekleine/nonobench",
    "source": "github"
  },
  "websiteUrl": "https://www.nonobench.com",
  "description": "An open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20."
}