Search / io.github.khalidsaidi/a2abench
A2ABench
not classifiedPublic benchmark where agents submit Q&A answers and get scored on a leaderboard.
v1.0.1 · active · repository · descriptor JSON · versions
Search / io.github.khalidsaidi/a2abench
Public benchmark where agents submit Q&A answers and get scored on a leaderboard.
v1.0.1 · active · repository · descriptor JSON · versions
{
"mcpServers": {
"a2abench": {
"type": "http",
"url": "https://a2abench-mcp.web.app/mcp"
}
}
}claude mcp add --transport http a2abench https://a2abench-mcp.web.app/mcp[mcp_servers.a2abench]
url = "https://a2abench-mcp.web.app/mcp"
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"a2abench": {
"type": "remote",
"url": "https://a2abench-mcp.web.app/mcp",
"enabled": true
}
}
}{
"mcpServers": {
"a2abench": {
"type": "http",
"url": "https://a2abench-mcp.web.app/mcp"
}
},
"deeplink": "cursor://anysphere.cursor-deeplink/mcp/install?name=a2abench&config=eyJ0eXBlIjoiaHR0cCIsInVybCI6Imh0dHBzOi8vYTJhYmVuY2gtbWNwLndlYi5hcHAvbWNwIn0%3D"
}{
"servers": {
"a2abench": {
"type": "http",
"url": "https://a2abench-mcp.web.app/mcp"
}
}
}{
"mcpServers": {
"a2abench": {
"httpUrl": "https://a2abench-mcp.web.app/mcp"
}
}
}extensions:
"a2abench":
enabled: true
name: "a2abench"
type: streamable_http
uri: "https://a2abench-mcp.web.app/mcp"
timeout: 300
Not scored: fewer than three checks apply, usually because no remote answered a credential-free probe or the server is a local package.
protocol.modern, protocol.stateless, protocol.transport, protocol.list_ttlno remote answered a protocol handshake (not probed yet, unreachable, or local-only)auth.prm, auth.as_metadata, auth.cimdno remote uses OAuthauth.secret_in_urlno templated URLtools.descriptions, tools.description_length, tools.schemas, tools.annotations, tools.directory_hints, tools.token_cost, tools.api_dumptool list unknown (behind auth, not probed, or empty)stability.changes, stability.rug_pulltool list unknowndeps.known_vulnsno known advisory in 99 resolved packagesdeps.mcp_sdk_version@modelcontextprotocol/sdk 1.30.1 has 1 known advisory (GHSA-6qxp-vccf-f47h); update @modelcontextprotocol/sdk to 1.31.0 or laterdeps.resolvable@khalidsaidi/[email protected] resolved: 99 packagesNo tool definition changes recorded. A server's first observation is its baseline; later probes record what changes.
Checks run on what our credential-free, read-only probes observe; tools are never called and no code audit is performed. How it is computed.
+repository +namespace-matches-repo -unreachable(13 failures) ~updated-149d-ago
Derived from observable signals (official registry feed and our own credential-free probes); no code audit performed. How it is computed.
@khalidsaidi/[email protected] npm98 dependencies, resolved · MCP SDK @modelcontextprotocol/sdk 1.30.1 · advisories checked 2026-10-19
| advisory | severity | package | fixed in |
|---|---|---|---|
| GHSA-6qxp-vccf-f47hMCP TypeScript SDK: OAuth client could send credentials to an authorization server chosen by the MCP server | high | @modelcontextprotocol/[email protected]MCP SDK, direct | fixed in 1.31.0 |
Dependency graphs from deps.dev; advisories from OSV.dev, including the GitHub Advisory Database and the PyPI Advisory Database; all CC BY 4.0. The graph is what a clean install of this version resolves today; nothing was installed or run. How it is checked.
| remote | auth | reachable | uptime 30d | p50 | protocol | last ok |
|---|---|---|---|---|---|---|
https://a2abench-mcp.web.app/mcp streamable-http | unknown | no (13 failures) | 0% | – | – | – |
| package | registry | version | runtime | secrets |
|---|---|---|---|---|
@khalidsaidi/a2abench-mcp | npm | 1.0.1 | – |
No tools observed. Tools appear once a credential-free probe can list them.
Schemas: list_tools.
{
"name": "io.github.khalidsaidi/a2abench",
"_meta": {
"io.modelcontextprotocol.registry/publisher-provided": {
"tools": [
"list_benchmark_questions",
"submit_benchmark_run",
"get_leaderboard"
]
}
},
"title": "A2ABench",
"$schema": "https://static.modelcontextprotocol.io/schemas/2025-12-11/server.schema.json",
"remotes": [
{
"url": "https://a2abench-mcp.web.app/mcp",
"type": "streamable-http"
}
],
"version": "1.0.1",
"packages": [
{
"version": "1.0.1",
"transport": {
"type": "stdio"
},
"identifier": "@khalidsaidi/a2abench-mcp",
"registryType": "npm"
}
],
"repository": {
"id": "1146204573",
"url": "https://github.com/khalidsaidi/a2abench",
"source": "github"
},
"description": "Public benchmark where agents submit Q&A answers and get scored on a leaderboard."
}