# panto-mcp: plan An MCP client for panto, shipped as an out-of-tree Lua extension (rock `panto-mcp`, entry name `mcp`), plus the general-purpose panto core additions it needs. Written 2026-11-16 from a planning session; readable without that conversation. Research reports referenced below were written to `/tmp` (volatile); the facts this plan depends on are restated inline. - `/tmp/mcp-research.md`: spec revisions, transports, lifecycle, auth, configs. - `/tmp/mcp-discovery-research.md`: opencode/pi discovery, provider-native deferred tools, generic-dispatch prior art. - `/tmp/mcp-optional-features.md`: prompts, resources, elicitation, tasks. - `/tmp/whiteboard-mcp-optional-features.html`: whiteboard of the above. ## Goals - Configure MCP servers in panto's layered `config.toml`: remote by URL, local by command. - Speak the latest released revision (**2026-07-28**) and fall back to **2025-11-25** (and the 2025-06-18 / 2025-03-26 versions it negotiates down to). Most servers in the wild still speak the handshake revisions. - Never leave orphaned server processes under any normal or abnormal panto exit (one accepted gap, below). - Make large MCP catalogs cheap: deferred discovery through `mcp.search` / `mcp.call`, and "code mode" through an `mcp` namespace in `std.lua`. - Tools, resources and prompts. Remote OAuth. ## Non-goals (v1) | Feature | Why not | Revisit when | |---|---|---| | Elicitation | Needs core "user input for extensions" (TODOs.md). Not advertised, so servers must not send it; GitHub hides `delete_repository`. | After that core project; noted on its TODOs.md entry. | | Tasks (2026 extension) | No popular server needs it for ordinary calls. Travis plans background-able tool calls first. | After background-able tool calls. | | Tasks (2025 experimental wire) | Superseded, wire-incompatible. | Never, unless a concrete server requires it. | | Sampling, roots, logging | Deprecated in 2026-07-28. Not advertised. | — | | Resource subscriptions / `subscriptions/listen` | Long-lived stream, ack/demux/reconnect; on-demand reads suffice. | A real server workflow needs push updates. | | 2024-11-05 HTTP+SSE transport | Deprecated; SHOULD NOT adopt. | A user hits a server that only speaks it. | | Native provider deferred loading (`defer_loading`) | Only Anthropic Messages and OpenAI Responses have it; both still need the catalog up front. `mcp.search`/`mcp.call` is provider-neutral and cache-stable. | — | | Importing `.mcp.json` / Claude Desktop configs | Not needed at first. | Later convenience. | | Persistent broker surviving `/new`/`/resume`/`/reload` | Restarting is far simpler; remote servers dominate; switches are infrequent. | Startup cost becomes a real complaint. | | Windows | panto doesn't target it. | — | ## Key facts that shape the design - **stdio is newline-delimited JSON-RPC** on the child's stdin/stdout; stderr is free-form logging. No port handoff exists. A local server that speaks HTTP is configured by URL. - **2026-07-28 is stateless.** No `initialize`, no sessions, no ping, no SSE resumability. Every request carries `_meta` protocol version, client capabilities and client info; servers MUST implement `server/discover`. Results carry `resultType`. HTTP is one POST per request, answered with JSON or a request-scoped SSE stream, with required `MCP-Protocol-Version`, `Mcp-Method` and (for `tools/call`, `resources/read`, `prompts/get`) `Mcp-Name` headers. - **2025-11-25 is handshake-based:** `initialize` → `notifications/initialized`, optional `Mcp-Session-Id`, GET notification stream, DELETE to end a session. - **MCP has no tool search.** All proposals were closed or are dormant; the official client guidance puts progressive discovery in the host. - **Spec shutdown order for stdio:** close the server's stdin, wait, SIGTERM, wait, SIGKILL (current opencode: 2s / 2s, whole process group). - **panto constraints found in code:** - Tools and commands are harvested only during `activate()` (`src/lua_runtime.zig` `activateEntry`); a `ToolSource`'s `tools` slice is fixed at registration. - `panto.ext.http_request` is GET-only, fully buffered, `keep_alive = false`, 120s max timeout, 16 global concurrent requests, auth over HTTPS only (`src/lua_http.zig`). Its `Service` is created per Lua runtime and destroyed on every session switch (`src/live_session.zig`). - The TUI's SIGINT/SIGTERM/SIGHUP handlers restore the terminal and re-raise without running `session_end` (`src/tui_terminal.zig`), so extension cleanup cannot be relied on for abnormal exits. - `session_end` gets at most 16 non-blocking loop passes before teardown (`src/main.zig`), so it cannot wait out grace periods. - Core auth has API keys and device-code OAuth only (libpantograph `src/auth.zig`); no authorization-code + PKCE flow. - `agent:conversation()` on the host agent returns a borrowed, mutable view with `add_user_blocks` / `add_assistant_message`; `agent:submit` accepts blocks. Persistence and rendering of view edits are unverified (spike S2). ## Architecture ```mermaid flowchart LR subgraph panto["panto process (Lua runtime, rebuilt per session)"] EXT["mcp extension
config · catalog · search · validate"] NS["std.lua `mcp` namespace"] T["tools: mcp.search / mcp.call / mcp.read_resource
+ eager mcp.<server>.<tool>"] C["commands: /mcp:<server>:<prompt>"] H["core http_request
(process-lifetime pool)"] A["core auth (oauth_pkce)"] MC[("metadata cache
(disk, per server)")] end SUP["panto lua supervisor
(own process group)"] SRV["stdio server
(own process group)"] REM["remote server"] EXT --- NS & T & C EXT <-->|"load at activation · save on connect"| MC EXT -- "stdin/stdout NDJSON (pipes)" --> SRV EXT -- "fd 3 lifeline" --> SUP SUP -- "owns / signals group" --> SRV EXT -- "POST · JSON or SSE" --> H --> REM H -. "Bearer" .- A ``` ### Lifetime One Lua runtime per session. Servers start **lazily**, driven by a disk metadata cache, so a session switch normally spawns nothing. On `session_end` the extension closes everything without waiting (the supervisor finishes local teardown; HTTP requests are cancelled). Running servers simply restart on their next use after `/new`, `/resume`, `/reload`. **Metadata cache.** Every successful connect saves the server's discovery result: detected era, server info/instructions, capabilities, and the full tool, prompt, resource and resource-template lists. - One JSON file per server, keyed by a **config fingerprint**: a hash (FNV in Lua) of the connection identity (`url`, or `command`/`args`/`env`/`cwd`, plus `auth`). A config change means a cold cache. Writes are atomic (temp file + rename). - Location: `${XDG_DATA_HOME:-$HOME/.local/share}/panto-mcp/`, resolved in Lua with the same XDG rule panto uses (`$XDG_DATA_HOME` if set and non-empty, else `$HOME/.local/share`). The extension owns this directory; no core cache-dir API. - Replaces the earlier plan to keep the detected era in `panto.ext.state`. **Activation:** - **Warm cache** (entry with a matching fingerprint): register `/mcp::` commands and any eager `mcp..` tools straight from the cache. **No process starts.** Eager tools are therefore present from the first turn, without post-activation registration. - **Cold cache** (first run, or config changed): connect that server in the background (non-blocking) to populate the cache, then register its prompts and eager tools post-activation (core C1/C2). **What starts a server:** `mcp.call`, an eager tool call, `mcp.read_resource`, running a prompt command, or `/mcp refresh [server]`. `mcp.search` searches the cache and starts nothing (a cold server still connecting is waited for). Every connect re-lists everything, rewrites the cache, and reconciles registrations: new prompts/eager tools are registered, removed ones unregistered (C1/C2; an eager change busts the prompt cache once, which is acceptable for a rare server-side change). **Staleness:** the cache is refreshed on every connect (no `ttlMs`-based expiry in v1). `mcp.call` validates against the **live** schema, since the connect it triggers re-lists first. A tool missing after the refresh errors with the closest matches; a removed prompt fails on `prompts/get` and its command is unregistered. A call to a server that is still connecting waits for it (bounded by the server's startup timeout). A failed connection is retried lazily on the next use, with the error visible in `mcp.search` / `/mcp` status. ### Configuration Extension-owned tables, read from `panto.ext.config` (merged layers; rigs and project configs may define servers, which is no worse than shipping Lua). ```toml [mcp] startup_timeout_ms = 30000 # per server: spawn/connect + discovery call_timeout_ms = 300000 # per request; progress does not extend past this idle_timeout_ms = 60000 # HTTP streams: max silence between bytes [mcp.servers.github] url = "https://api.githubcopilot.com/mcp/" expose = "search" # "search" (default) | "eager" auth = "mcp.github" # optional: named panto [auth.*] session headers = { "X-Tenant" = "acme" } # non-secret; ${env:VAR} expanded enabled = true enabled_tools = [] # empty = all disabled_tools = ["delete_*"] # glob over MCP tool names; applied after enabled_tools [mcp.servers.fs] command = "npx" args = ["-y", "@modelcontextprotocol/server-filesystem", "${project_root}"] env = { "TOKEN" = "${env:FS_TOKEN}" } cwd = "." # relative to project root ``` - Exactly one of `url` / `command` per server; invalid servers are reported and skipped, never fatal to panto. - `panto.ext.config` leaves `${env:VAR}` literal; the extension expands it in `args`, `env`, `headers`, `url`. `${project_root}` comes from `panto.ext.dirs`. Missing env → empty string, matching core semantics. - Server names use `^[A-Za-z0-9_-]+$`, but also cannot contain `__` or end in `_`: panto decodes `__` as a provider-name separator, so those names make server/tool boundaries ambiguous. Invalid server entries are warned about and skipped, never fatal to panto. - Secrets for remote servers should go through `auth = ""` so they never pass through Lua; `headers` with `${env:}` remain possible. - For automatic OAuth, omit `auth`; optional `[mcp.servers..oauth]` accepts `client_id`, required matching `issuer` with that ID, optional `client_secret` (`${env:VAR}` expanded) and fallback `scopes`. Otherwise use public DCR. - Child env: inherit panto's environment plus `env`. (Codex-style `env_vars` allowlisting is skipped; add if needed.) ### Tool surface and token cost Every registered tool costs context tokens, so nothing is unconditional. | Exposed to model | When | |---|---| | `mcp.search`, `mcp.call`, `mcp.read_resource` | At activation, iff ≥1 enabled server has `expose = "search"`. Static declarations: cache-stable for the session. | | `mcp..` per eager tool | At activation from a warm cache; otherwise after that eager server's first connect (post-activation registration, C1), visible from the next provider request. | | `mcp.read_resource` (eager-only configs) | Registered when the first eager server (cached or live) advertises `resources`. | | std.lua `mcp` namespace | Iff ≥1 enabled server. No tool cost; a short hint. | | System hint | Iff ≥1 enabled server: one line per server (name, transport, mode) built from **config only**, so it never changes mid-session; points at search tools when exposed, otherwise at eager tools and docs. | Eager tools can miss the first turn only on a cold cache (the user submits before the first-ever connect finishes). Search mode has no such race (`mcp.call` waits). **`mcp.search {query?, server?, limit?}`**: connects the relevant servers if needed and searches the cached catalog (tools, resources, resource templates) client-side. Ranking: simple term scoring over name, title, description and parameter names/descriptions (BM25-ish; no dependency). Each hit returns `server`, `name`, title/description and the **full input schema**, so search doubles as describe. Server `instructions` (from discover/initialize) are returned when `server` is given, truncated. **`mcp.call {server, tool, input}`**: exact `server` + `tool` only; never fuzzy execution. Unknown names return an error listing the closest matches. `input` is validated against the tool's live input schema before sending; failure returns the validation errors plus the schema. Declared non-strict (an `any`-shaped `input` cannot be expressed in OpenAI/Anthropic strict mode). **`mcp.read_resource {server, uri}`**: `resources/read` on the originating server. Never fetches arbitrary URLs itself; a `resource_link` in a tool result names its server. Eager tool names are `mcp..`, with the MCP tool name sanitized to `[A-Za-z0-9_-]` and `__` collapsed (panto encodes `.` as `__` on the wire). Every tool gets a suffix derived only from its own MCP name, so adding another tool cannot rename it. Truncation keeps the encoded wire name within **64 bytes**. The limit is the strictest declared provider cap checked: Anthropic Messages tool names allow 128 characters; OpenAI Chat Completions function names cap at 64; OpenAI Responses documents function names as strings without a stated limit. Sources: [Anthropic Messages](https://platform.claude.com/docs/en/api/http/messages/create), [OpenAI Chat Completions](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create), [OpenAI Responses](https://developers.openai.com/api/reference/resources/responses/methods/create). Eager tools are validated the same way as `mcp.call` (panto has no schema validation of its own). ### std.lua `mcp` namespace ("code mode") Registered with `require("_std").lua_namespace("mcp", factory)`; available to child agents too. Functions run in the std.lua handler coroutine and yield on luv: ```lua mcp.servers() -- name, transport, mode, era, state, error mcp.search(query, {server=, limit=}) mcp.call(server, tool, input) -- validated; returns {text, attachments, structured, is_error} mcp.start(server, tool, input) -- handle; handle:await(); for parallel calls mcp.read_resource(server, uri) mcp.prompts(server), mcp.prompt(server, name, args) ``` Reference docs ship as a require-able module (`require("panto_mcp.docs")`), like `subagents.docs` and `_std.docs`: discovery, examples, result shapes, errors. Only the one-line hint is injected. Per-server proxy sugar (`mcp.github.create_issue{}`) is skipped; add it if models reach for it. ### Input validation A pure-Lua JSON Schema subset validator, ported from panto-subagents' `workflow.lua` (copied, not a dependency). Covers `type`, `properties`, `required`, `additionalProperties`, `enum`/`const`, `items`/`prefixItems`, numeric/string/array bounds, `anyOf`/`oneOf`/`allOf`/`not`, local `$ref`/ `$defs`. Unknown keywords (including `pattern`/`format` if unsupported) are **ignored, not rejected**: validation exists to catch cheap mistakes, and a false rejection would block a valid call. Errors carry JSON-pointer paths. ### Result mapping (tool calls, resources, prompts) | MCP content | panto result | |---|---| | `text` | text | | `image` | attachment (base64 decoded in Lua; size-capped) | | `audio` | text placeholder with MIME type and size | | `resource` (embedded text) | text, prefixed with its URI | | `resource` (embedded blob) | attachment if image, else placeholder | | `resource_link` | text line: name, URI, description, and "read with `mcp.read_resource{server=…, uri=…}`" (or `mcp.read_resource` in std.lua) | | `structuredContent` | JSON text, only if no text block already carries it | | `isError: true` | tool error (model-visible) with the content | | JSON-RPC `error` | tool error: `: ()` | | `resultType: input_required` | tool error: server requested user input, which panto does not support yet | Output is capped (configurable later if needed); truncation is stated in the result. ### Prompts - Each server prompt becomes a slash command **`/mcp::`**, registered at activation from the metadata cache, or after the first connect on a cold cache (core C2), and unregistered if a later connect shows it gone. Description = prompt title/description plus its argument names, so the fuzzy command selector finds it. - Arguments: `name=value` tokens with shell-style quoting; a prompt with exactly one argument accepts the trimmed raw remainder when no matching `name=` token appears. Missing required arguments → error listing them. - On invoke, start `prompts/get` in an async coroutine from the synchronous command callback. Refuse if a turn or non-prompt command is active, and re-check when the result arrives. Prompt responses append real user/assistant roles to the borrowed host conversation; a trailing user message is submitted to start a turn. Assistant media is represented by text placeholders. - Prompt names the core rejects (slash/whitespace) are skipped with a warning visible in `/mcp`. Async failures mount a visible error above the input until the next `command_start`. One-argument positional values preserve the trimmed raw remainder when no matching `name=` argument is present. - **Assistant-tail rule:** if the prompt does not end with a user message, append all messages and skip `agent:submit`. Emit `agent_submission` so the idle TUI sync renders and persists the tail without starting a turn. - All prompt message blocks are mapped before conversation mutation. Only `image/*` content becomes user media; assistant media and non-image blobs become text placeholders. This avoids partial history from rejected media. - **Conversation sync verified in ws-1:** idle `onPaint` syncs the conversation tail and calls `flushPersist`; idle submission draining syncs/persists before queued `agent:submit` work starts. Assistant-tail prompts emit `agent_submission` to request that same idle sync. No C5 core change is needed. ### Resources Built in v1 because tool results can carry `resource_link`s that are only usable with `resources/read`. - Catalog: `resources/list` and `resources/templates/list` (paginated, cached per `ttlMs`), searchable through `mcp.search`. - Reading: `mcp.read_resource` tool and namespace function; templates are filled by the model/user into a concrete URI, then read. - Only servers advertising `resources` are queried. Some legacy clients treat `-32601` from list methods as fatal; we never call unadvertised methods. ## Protocol client Layers (each a module; transports are interchangeable under the client): 1. **Transport** - *stdio:* NDJSON framing over luv pipes; partial-line buffering; stderr captured into a bounded per-server ring for diagnostics (never treated as an error signal). - *HTTP (modern):* POST per request with the required headers; response is JSON or a request-scoped SSE stream parsed in Lua. `x-mcp-header` tool annotations are validated and mirrored as `Mcp-Param-*` headers. Cancellation = cancel the request (close its stream). - *HTTP (legacy 2025-11-25):* adds `Mcp-Session-Id` capture/echo, re-init on 404, best-effort DELETE on close. The optional GET notification stream opens for eager servers advertising `tools.listChanged`, closes with the session, retries after an established stream ends, and is replaced after legacy re-init. 2. **JSON-RPC:** id allocation, pending-request table, per-request timeout (`call_timeout_ms`), cancellation (`notifications/cancelled` on stdio), late responses to cancelled ids ignored. Progress notifications are accepted and ignored (no progress token is sent in v1). 3. **Era negotiation:** - Modern first: `server/discover` with 2026-07-28 `_meta`. Success → modern; `UnsupportedProtocolVersion (-32022)` → pick from `supported`. - Any other error, a recognizable legacy rejection, an empty/unrecognized HTTP 400, or a probe timeout → legacy `initialize` (offer `2025-11-25`, accept the server's counter-offer among versions we implement), then `notifications/initialized`. - The detected era is stored in the server's metadata cache entry to skip the probe next time; if the cached era fails, re-probe. 4. **Capabilities advertised:** none (`{}`): no elicitation, sampling, roots, tasks. Client info `{name = "panto-mcp", version}`. 5. **Server→client traffic (legacy):** answer `ping`; reply `-32601` to any other request. Handle `notifications/tools|resources|prompts/list_changed` by re-listing (eager: unregister/re-register; search: invalidate catalog; prompts: re-register commands). ## Local server lifecycle (no orphans) ```mermaid sequenceDiagram participant P as panto (extension) participant W as supervisor (panto lua) participant S as server P->>W: spawn detached: fds 0,1,2 = server pipes, fd 3 = lifeline W->>S: spawn detached (new process group), inheriting fds 0,1,2 W-->>W: close own copies of 0/1; report server pid on stderr marker P->>S: JSON-RPC over the pipes (no relaying through W) alt normal session_end / quit P->>S: close stdin write end P->>W: close lifeline (fd 3) else panto dies (SIGKILL, crash, SIGHUP, OOM) Note over P,W: kernel closes panto's fds: stdin EOF + lifeline EOF end W->>W: wait grace (2s) for S to exit W->>S: SIGTERM process group, wait 2s W->>S: SIGKILL process group W-->>W: exit once group is gone ``` - One supervisor per local server: `uv.spawn(uv.exepath(), {"lua", supervisor_path, ...})`, starts in ~16ms. `supervisor_path` resolved via `package.searchpath` so it works from a rock or a checkout. - Both supervisor and server run in their own process groups (`detached`), so terminal Ctrl+C / Ctrl+Z / SIGHUP do not hit them directly; panto's death is observed only through the lifeline. The supervisor also traps INT/TERM/HUP and runs the same teardown. - The server's group is signalled, catching `npx`/`docker`-style wrappers that leave grandchildren. - panto never waits during `session_end`: it closes two fds and returns; process handles are unref'd so they don't hold the loop. - **Accepted gap:** if panto and the supervisor are both SIGKILLed at once, a server that ignores stdin EOF survives. No pidfile sweep (rare enough). - Precedent: `../panto-claudesub/panto-claudesub/_session.lua` (detached group, bounded TERM → KILL escalation, confirmed exit). ## Remote auth Order of preference per server: `auth = ""` (static API key or OAuth session in `[auth.*]`), then automatic MCP OAuth on a 401. **Extension (MCP policy):** 1. On 401, parse `WWW-Authenticate` for `resource_metadata`; else try the RFC 9728 well-known PRM URIs (endpoint path, then origin root). 2. Fetch authorization-server metadata (RFC 8414, then OIDC discovery variants); require an exact `issuer` match and PKCE (`S256`) support. 3. Client registration: configured `client_id` requires an exact `oauth.issuer` binding (optional env-expanded `client_secret`) → public Dynamic Client Registration. Confidential DCR is refused; no client secret is stored in `panto.ext.state`. Public registrations are stored per issuer and retained on every failure. Any configured issuer pins selection from authorization_servers, even without a client ID. CIMD is deferred (`TODOs.md`). 4. Define a runtime `oauth_pkce` session `mcp..` (≤128 bytes), using 128 bits of SHA-256 over canonical resource, exact issuer and client ID. Retire the old name with core logout on changes. Use discovered endpoints, challenge → PRM → configured scopes, and `resource` = canonical server URI. Warm activation restores discovery without network work; no attestation flag. 5. Login is user-initiated: `/mcp-login [callback-url | --reregister]`. Reuse a definition unless missing or core login fails; keep registrations on all failures. Only a changed discovered issuer or explicit --reregister discards one. The escape hatch rediscovers/registers once; errors mention it. Consent with an existing usable token does not lock out tools, and give-up clears login_required. 6. Authenticated 401: one forced refresh, then one retry; parallel calls await the shared refresh. Scope 403 redefines the scope union and asks for re-consent, without disabling working calls or an artificial attempt counter. Other 403s report permission errors; explicit non-PKCE auth errors point to core auth. Legacy GET reconnect backs off to 60 seconds and stops with a /mcp warning on 401. 7. Root PRM fallback validates `resource` against the origin. Loopback discovery/ endpoints require a loopback resource; reject leading-zero IPv4 octets. Cache status reads/poll at ~1 second and back off failed discovery from 5 to 60 seconds. **Core (credential mechanism, C4):** generic `oauth_pkce` auth type (authorization code + PKCE S256, `state` check, optional `iss` validation, `resource` parameter, loopback callback on an ephemeral port, **pasted callback URL fallback** for SSH/headless, token storage under `auth/`, refresh, header injection) plus `panto.ext` APIs to define a named session at runtime and start its login. Requests then use `http_request{auth = name}`; tokens never enter Lua. ## Core additions (pantograph / libpanto) New `panto.ext` surfaces are co-designed with Travis before building; each passes the AGENTS.md tests (second consumer, no "mcp" in core names, mechanism not policy). | # | Addition | Second consumer | |---|---|---| | C1 | `register_tool` after activation + `unregister_tool(name)`. Changes take effect at the next turn (per-turn snapshot); in-flight calls unaffected; same collision rules; child agents sharing the Lua tool source see it. Needs copy-on-write declarations in the Lua `ToolSource` (provider serialization reads them off the owner thread). | Any extension whose tools depend on async discovery: plugin loaders, per-skill tools, a tool set that changes with project state. | | C2 | `register_command` after activation + `unregister_command(name)`; the fuzzy selector and `render_header` command list pick them up. | subagents' `/workflow:` when profiles change on disk; skills exposed as commands. | | C3 | `http_request`: `method` (GET/POST/PUT/PATCH/DELETE), `body`; streaming via `on_head(head)` / `on_data(chunk)` hooks (presence of `on_data` disables buffering; final callback still fires with status/headers or error); `idle_timeout_ms`; no total cap for streams (`timeout_ms = 0`); keep-alive with a process-lifetime connection pool (callbacks still die with the runtime); auth allowed over plain HTTP to loopback; streams counted separately from the 16 buffered slots; bounded chunk queue for backpressure. | Lua protocol providers streaming a custom wire API; webhook/REST clients; the messaging bridge's long-poll. | | C4 | `oauth_pkce` auth type + runtime session definition and login APIs (above). Seeded `config.toml` documents the new auth fields. | Messaging bridge (Slack/Telegram OAuth); any extension calling an OAuth-protected API. | | C5 | *Only if spike S2 fails:* host-agent method to append arbitrary-role messages that persists and renders. | Fork/import tools, transcript surgery extensions, prompt-library extensions. | Draft Lua shape for C3 (to be co-designed): ```lua local handle, err = panto.ext.http_request({ url = url, method = "POST", body = json, auth = "mcp.github", headers = { ["content-type"] = "application/json", accept = "application/json, text/event-stream" }, timeout_ms = 0, idle_timeout_ms = 60000, on_head = function(head) end, -- {status, headers} on_data = function(chunk) end, -- raw bytes; SSE parsing is the caller's }, function(response, err) end) -- buffered: {status, body, headers}; streamed: no body handle:cancel() ``` ## Build phases Spikes first, then extension work proceeds in parallel with core work (stdio + search mode needs no core change). - [x] **S1 supervisor spike:** luv `spawn` with a 4th stdio slot from Lua; detached supervisor + detached server; EOF propagation when panto is `kill -9`ed; group kill of an `sh -c 'trap "" TERM; sleep …'` tree. - [x] **S2 conversation spike:** source-verified: asynchronous idle edits sync through `onPaint`; `agent:submit` synchronizes before the TUI drains its queued work. Assistant-tail prompts explicitly emit `agent_submission` to wake that same render/persist path. C5 is unnecessary. - [x] **S3 tool-source mutability:** read libpanto's registry/`ToolSource` snapshot path to size C1. - [x] **E1 core client:** rock skeleton (`panto-mcp` → `init.lua`, `panto_mcp.*` modules), config parsing/expansion, JSON-RPC, stdio transport + supervisor, era negotiation, catalog + metadata cache (lazy start), `mcp.search` / `mcp.call`, validator, result mapping, `session_end` teardown, `/mcp` status command. - [x] **E2 code mode:** std.lua `mcp` namespace, `panto_mcp.docs`, system hint. - [x] **C3 HTTP** → **E3 HTTP transport:** modern POST/SSE, legacy sessions, static headers and `auth = ""`. - [x] **E4 resources:** list/templates/read, `mcp.read_resource`, resource blocks in results, catalog search over resources. - [x] **C1/C2 dynamic registration** → **E5 eager mode** (including legacy HTTP GET notifications) and **E6 prompts** (`/mcp::`); no C5 fix. - [x] **C4 OAuth** → **E7 MCP OAuth:** PRM/AS discovery, configured/DCR registration, `/mcp-login` (loopback + pasted callback), refresh-on-401 and non-blocking scope step-up; issuer-bound configured clients, identity-scoped sessions, public-only DCR, rediscovery/backoff/status caching, warm restoration and real-core TCP-fake checks. Real-provider pass remains below. - [ ] **E8 release:** README, rockspec, mise tasks (copy panto-web's `version`/`release`), luarocks upload. Files done; tag/upload left to Travis. - [x] Note on pantograph `TODOs.md` "User input for extensions" that it enables MCP elicitation in panto-mcp. ## Testing Per pantograph AGENTS.md: never test timeouts by sleeping; inject the clock / timer factory so timeouts fire deterministically. - `_selfcheck.lua` run with `panto lua` (like panto-web/panto-claudesub): fake stdio servers written in Lua for both eras (modern discover; legacy initialize; legacy server that errors on pre-initialize requests; server that ignores EOF; server that spawns a grandchild), and a luv TCP fake for HTTP (JSON response, SSE response, legacy session 404 → re-init, 401 → PRM). - Eager tools/prompts: warm activation is idle; a cold `mcp.search` connect reconciles eager declarations; `list_changed` reconciles add/remove/schema changes. Tests cover multi-server prompt preservation, `/compact` command busy state, visible async failures, image-only media and all-message preflight, raw one-argument remainders, invalid command/server names, stable eager names, exact cache pruning, and legacy GET reconnect/re-init. - Validator, SSE parser, NDJSON framer, result mapping, name sanitizing, arg parsing: table-driven unit checks in the same selfcheck. - OAuth: table-driven challenge/URL/issuer/S256/scope checks; injected HTTP/auth/timers cover discovery → public DCR → user login → reconnect, refresh and scope upgrades. Regressions cover foreign-issuer credentials, identity-scoped sessions, non-locking 403s, concurrent refresh calls, confidential-DCR refusal, rediscovery and backoff. `PANTO_MCP_HTTP_E2E=1` adds a new-core browser/pasted-callback flow, forced refresh, scope re-consent and warm reuse against a local TCP fake (no real credentials). - Orphan check: spawn supervisor+server from a stand-in parent process, `kill -9` the parent, wait on the server group's exit event (bounded), assert nothing survives. - Core additions get Zig tests in pantograph/libpanto alongside their code. - One manual end-to-end pass against real servers: a filesystem stdio server via `npx`, a remote legacy server, a remote 2026 server, and one OAuth server (GitHub or Linear). ## Resolved design checks - **Provider tool-name limits:** use 64 encoded wire bytes, the strictest declared bound in the primary provider docs linked above.