# Nonobench

> An open-source benchmark of how well LLMs solve nonogram puzzles, from 5x5 to 20x20.

- name: io.github.mauricekleine/nonobench
- version: 1.1.0
- connection class: R0 (autonomous)
- trust: 97/100
- quality: 89/100 (good)
- owner: not claimed
- descriptor: https://api.protogrid.dev/v1/servers/io.github.mauricekleine%2Fnonobench
- tools: https://api.protogrid.dev/v1/servers/io.github.mauricekleine%2Fnonobench/tools

**An agent can connect right now, no human step.** Remote endpoint, no authentication, reachable on the last probe.

## Connection (mcpServers)

```json
{
  "mcpServers": {
    "nonobench": {
      "type": "http",
      "url": "https://www.nonobench.com/mcp"
    }
  }
}
```


## Trust

- hygiene: 100
- liveness: 99
- freshness: 100
- provenance: 90
- drivers: +repository +namespace-matches-repo +website +reachable +uptime-96% +updated-21d-ago
- Derived from observable signals (official registry feed and our own credential-free probes); no code audit performed.

## Quality 89/100 (good)

### protocol: 100 (weight 25)

- pass `protocol.modern`: supports 2026-07-28 (server/discover)
- pass `protocol.stateless`: answers without a session
- pass `protocol.transport`: streamable HTTP
- pass `protocol.list_ttl`: tool list is cacheable (ttlMs 3600000)

### authorization: n/a (weight 15)

- n/a `auth.prm`, `auth.as_metadata`, `auth.cimd`: no remote uses OAuth
- n/a `auth.secret_in_url`: no templated URL

### tool hygiene: 86 (weight 30)

- pass `tools.descriptions`: every tool has a description
- pass `tools.description_length`: descriptions are concise
- pass `tools.schemas`: every tool has an object input schema
- pass `tools.annotations`: 11 of 11 tools declare readOnlyHint or destructiveHint
- fail `tools.directory_hints`: no tool declares all four of readOnlyHint, destructiveHint, idempotentHint and openWorldHint; missing: list_families (destructiveHint, idempotentHint); compare_models (destructiveHint, idempotentHint); get_model_results (destructiveHint, idempotentHint); …
- pass `tools.token_cost`: about 1,464 tokens to load every tool
- pass `tools.api_dump`: tools are not a one-to-one API dump

### stability: 75 (weight 15)

- warn `stability.changes`: 11 tool changes in 30 days
- pass `stability.rug_pull`: no tool changed its meaning under the same name

### dependencies: n/a (weight 15)

- n/a `deps.known_vulns`, `deps.mcp_sdk_version`, `deps.resolvable`: no npm or PyPI package

- tools: 11, about 1,464 tokens to load them all
- tool set hash: ccce2e6a6d8e5af0dd36d1cfab830ab8
- tools last changed: 2026-09-30T03:09:58.981Z
- badge: [![protogrid quality](https://protogrid.dev/badge/io.github.mauricekleine/nonobench.svg)](https://protogrid.dev/servers/io.github.mauricekleine/nonobench)
- Checks run on what our credential-free, read-only probes observe; tools are never called and no code audit is performed.

### Tool changes

- 2026-09-30 `list_runs` changed (inputSchema, outputSchema, annotations)
- 2026-09-30 `get_model_puzzles` changed (inputSchema, outputSchema, annotations)
- 2026-09-30 `get_puzzle_results` changed (inputSchema, outputSchema, annotations)
- 2026-09-30 `check_solution` changed (outputSchema, annotations)
- 2026-09-30 `get_puzzle` changed (outputSchema, annotations)
- 2026-09-30 `list_puzzles` changed (outputSchema, annotations)
- 2026-09-30 `get_model_results` changed (outputSchema, annotations)
- 2026-09-30 `compare_models` changed (inputSchema, outputSchema, annotations)
- 2026-09-30 `list_families` changed (outputSchema, annotations)
- 2026-09-30 `list_providers` changed (outputSchema, annotations)

Full history: https://api.protogrid.dev/v1/servers/io.github.mauricekleine%2Fnonobench/changes

## Remotes

- https://www.nonobench.com/mcp (streamable-http, auth none, reachable true, uptime30d 0.962963)

## Packages

_none_

## Tools (11)

- `check_solution`: Check a grid against a puzzle's clues, using the same rule as the benchmark grader. Reports which rows and columns do not match.
- `compare_models`: Side-by-side core overall and per-size accuracy, cost, latency and token results for model or family names.
- `get_leaderboard`: Models ranked by accuracy; defaults to all effort levels for compatibility.
- `get_model_puzzles`: Which puzzles one model solved, missed, timed out on, or has not run.
- `get_model_results`: Accuracy, cost, latency and token use for one model, broken down by grid size.
- `get_puzzle`: One puzzle, including the clue text models were prompted with. The reference solution is only included on request; some puzzles have several valid solutions.
- `get_puzzle_results`: Per-model outcomes for one puzzle. Answers are omitted unless requested.
- `list_families`: Model families, available efforts and best variants.
- `list_providers`: Provider ids, names, families and variant counts.
- `list_puzzles`: The benchmark puzzles with their ids and row/column clues.
- `list_runs`: Individual benchmark runs (one model on one puzzle), optionally with the raw prompt and model output.