# panto-mcp: plan
An MCP client for panto, shipped as an out-of-tree Lua extension (rock
`panto-mcp`, entry name `mcp`), plus the general-purpose panto core additions
it needs. Written 2026-11-16 from a planning session; readable without that
conversation.
Research reports referenced below were written to `/tmp` (volatile); the facts
this plan depends on are restated inline.
- `/tmp/mcp-research.md`: spec revisions, transports, lifecycle, auth, configs.
- `/tmp/mcp-discovery-research.md`: opencode/pi discovery, provider-native
deferred tools, generic-dispatch prior art.
- `/tmp/mcp-optional-features.md`: prompts, resources, elicitation, tasks.
- `/tmp/whiteboard-mcp-optional-features.html`: whiteboard of the above.
## Goals
- Configure MCP servers in panto's layered `config.toml`: remote by URL, local
by command.
- Speak the latest released revision (**2026-07-28**) and fall back to
**2025-11-25** (and the 2025-06-18 / 2025-03-26 versions it negotiates down
to). Most servers in the wild still speak the handshake revisions.
- Never leave orphaned server processes under any normal or abnormal panto
exit (one accepted gap, below).
- Make large MCP catalogs cheap: deferred discovery through `mcp.search` /
`mcp.call`, and "code mode" through an `mcp` namespace in `std.lua`.
- Tools, resources and prompts. Remote OAuth.
## Non-goals (v1)
| Feature | Why not | Revisit when |
|---|---|---|
| Elicitation | Needs core "user input for extensions" (TODOs.md). Not advertised, so servers must not send it; GitHub hides `delete_repository`. | After that core project; noted on its TODOs.md entry. |
| Tasks (2026 extension) | No popular server needs it for ordinary calls. Travis plans background-able tool calls first. | After background-able tool calls. |
| Tasks (2025 experimental wire) | Superseded, wire-incompatible. | Never, unless a concrete server requires it. |
| Sampling, roots, logging | Deprecated in 2026-07-28. Not advertised. | — |
| Resource subscriptions / `subscriptions/listen` | Long-lived stream, ack/demux/reconnect; on-demand reads suffice. | A real server workflow needs push updates. |
| 2024-11-05 HTTP+SSE transport | Deprecated; SHOULD NOT adopt. | A user hits a server that only speaks it. |
| Native provider deferred loading (`defer_loading`) | Only Anthropic Messages and OpenAI Responses have it; both still need the catalog up front. `mcp.search`/`mcp.call` is provider-neutral and cache-stable. | — |
| Importing `.mcp.json` / Claude Desktop configs | Not needed at first. | Later convenience. |
| Persistent broker surviving `/new`/`/resume`/`/reload` | Restarting is far simpler; remote servers dominate; switches are infrequent. | Startup cost becomes a real complaint. |
| Windows | panto doesn't target it. | — |
## Key facts that shape the design
- **stdio is newline-delimited JSON-RPC** on the child's stdin/stdout; stderr is
free-form logging. No port handoff exists. A local server that speaks HTTP is
configured by URL.
- **2026-07-28 is stateless.** No `initialize`, no sessions, no ping, no SSE
resumability. Every request carries `_meta` protocol version, client
capabilities and client info; servers MUST implement `server/discover`.
Results carry `resultType`. HTTP is one POST per request, answered with JSON
or a request-scoped SSE stream, with required `MCP-Protocol-Version`,
`Mcp-Method` and (for `tools/call`, `resources/read`, `prompts/get`)
`Mcp-Name` headers.
- **2025-11-25 is handshake-based:** `initialize` → `notifications/initialized`,
optional `Mcp-Session-Id`, GET notification stream, DELETE to end a session.
- **MCP has no tool search.** All proposals were closed or are dormant; the
official client guidance puts progressive discovery in the host.
- **Spec shutdown order for stdio:** close the server's stdin, wait, SIGTERM,
wait, SIGKILL (current opencode: 2s / 2s, whole process group).
- **panto constraints found in code:**
- Tools and commands are harvested only during `activate()`
(`src/lua_runtime.zig` `activateEntry`); a `ToolSource`'s `tools` slice is
fixed at registration.
- `panto.ext.http_request` is GET-only, fully buffered, `keep_alive = false`,
120s max timeout, 16 global concurrent requests, auth over HTTPS only
(`src/lua_http.zig`). Its `Service` is created per Lua runtime and
destroyed on every session switch (`src/live_session.zig`).
- The TUI's SIGINT/SIGTERM/SIGHUP handlers restore the terminal and re-raise
without running `session_end` (`src/tui_terminal.zig`), so extension
cleanup cannot be relied on for abnormal exits.
- `session_end` gets at most 16 non-blocking loop passes before teardown
(`src/main.zig`), so it cannot wait out grace periods.
- Core auth has API keys and device-code OAuth only (libpantograph
`src/auth.zig`); no authorization-code + PKCE flow.
- `agent:conversation()` on the host agent returns a borrowed, mutable view
with `add_user_blocks` / `add_assistant_message`; `agent:submit` accepts
blocks. Persistence and rendering of view edits are unverified (spike S2).
## Architecture
```mermaid
flowchart LR
subgraph panto["panto process (Lua runtime, rebuilt per session)"]
EXT["mcp extension
config · catalog · search · validate"]
NS["std.lua `mcp` namespace"]
T["tools: mcp.search / mcp.call / mcp.read_resource
+ eager mcp.<server>.<tool>"]
C["commands: /mcp:<server>:<prompt>"]
H["core http_request
(process-lifetime pool)"]
A["core auth (oauth_pkce)"]
MC[("metadata cache
(disk, per server)")]
end
SUP["panto lua supervisor
(own process group)"]
SRV["stdio server
(own process group)"]
REM["remote server"]
EXT --- NS & T & C
EXT <-->|"load at activation · save on connect"| MC
EXT -- "stdin/stdout NDJSON (pipes)" --> SRV
EXT -- "fd 3 lifeline" --> SUP
SUP -- "owns / signals group" --> SRV
EXT -- "POST · JSON or SSE" --> H --> REM
H -. "Bearer" .- A
```
### Lifetime
One Lua runtime per session. Servers start **lazily**, driven by a disk
metadata cache, so a session switch normally spawns nothing. On `session_end`
the extension closes everything without waiting (the supervisor finishes local
teardown; HTTP requests are cancelled). Running servers simply restart on
their next use after `/new`, `/resume`, `/reload`.
**Metadata cache.** Every successful connect saves the server's discovery
result: detected era, server info/instructions, capabilities, and the full
tool, prompt, resource and resource-template lists.
- One JSON file per server, keyed by a **config fingerprint**: a hash (FNV in
Lua) of the connection identity (`url`, or `command`/`args`/`env`/`cwd`,
plus `auth`). A config change means a cold cache. Writes are atomic
(temp file + rename).
- Location: `${XDG_DATA_HOME:-$HOME/.local/share}/panto-mcp/`, resolved in
Lua with the same XDG rule panto uses (`$XDG_DATA_HOME` if set and
non-empty, else `$HOME/.local/share`). The extension owns this directory;
no core cache-dir API.
- Replaces the earlier plan to keep the detected era in `panto.ext.state`.
**Activation:**
- **Warm cache** (entry with a matching fingerprint): register
`/mcp::` commands and any eager `mcp..` tools
straight from the cache. **No process starts.** Eager tools are therefore
present from the first turn, without post-activation registration.
- **Cold cache** (first run, or config changed): connect that server in the
background (non-blocking) to populate the cache, then register its prompts
and eager tools post-activation (core C1/C2).
**What starts a server:** `mcp.call`, an eager tool call,
`mcp.read_resource`, running a prompt command, or `/mcp refresh [server]`.
`mcp.search` searches the cache and starts nothing (a cold server still
connecting is waited for). Every connect re-lists everything, rewrites the
cache, and reconciles registrations: new prompts/eager tools are registered,
removed ones unregistered (C1/C2; an eager change busts the prompt cache once,
which is acceptable for a rare server-side change).
**Staleness:** the cache is refreshed on every connect (no `ttlMs`-based
expiry in v1). `mcp.call` validates against the **live** schema, since the
connect it triggers re-lists first. A tool missing after the refresh errors
with the closest matches; a removed prompt fails on `prompts/get` and its
command is unregistered.
A call to a server that is still connecting waits for it (bounded by the
server's startup timeout). A failed connection is retried lazily on the next
use, with the error visible in `mcp.search` / `/mcp` status.
### Configuration
Extension-owned tables, read from `panto.ext.config` (merged layers; rigs and
project configs may define servers, which is no worse than shipping Lua).
```toml
[mcp]
startup_timeout_ms = 30000 # per server: spawn/connect + discovery
call_timeout_ms = 300000 # per request; progress does not extend past this
idle_timeout_ms = 60000 # HTTP streams: max silence between bytes
[mcp.servers.github]
url = "https://api.githubcopilot.com/mcp/"
expose = "search" # "search" (default) | "eager"
auth = "mcp.github" # optional: named panto [auth.*] session
headers = { "X-Tenant" = "acme" } # non-secret; ${env:VAR} expanded
enabled = true
enabled_tools = [] # empty = all
disabled_tools = ["delete_*"] # glob over MCP tool names; applied after enabled_tools
[mcp.servers.fs]
command = "npx"
args = ["-y", "@modelcontextprotocol/server-filesystem", "${project_root}"]
env = { "TOKEN" = "${env:FS_TOKEN}" }
cwd = "." # relative to project root
```
- Exactly one of `url` / `command` per server; invalid servers are reported and
skipped, never fatal to panto.
- `panto.ext.config` leaves `${env:VAR}` literal; the extension expands it in
`args`, `env`, `headers`, `url`. `${project_root}` comes from
`panto.ext.dirs`. Missing env → empty string, matching core semantics.
- Server names use `^[A-Za-z0-9_-]+$`, but also cannot contain `__` or end in
`_`: panto decodes `__` as a provider-name separator, so those names make
server/tool boundaries ambiguous. Invalid server entries are warned about
and skipped, never fatal to panto.
- Secrets for remote servers should go through `auth = ""` so they
never pass through Lua; `headers` with `${env:}` remain possible.
- For automatic OAuth, omit `auth`; optional `[mcp.servers..oauth]` accepts
`client_id`, required matching `issuer` with that ID, optional `client_secret`
(`${env:VAR}` expanded) and fallback `scopes`. Otherwise use public DCR.
- Child env: inherit panto's environment plus `env`. (Codex-style `env_vars`
allowlisting is skipped; add if needed.)
### Tool surface and token cost
Every registered tool costs context tokens, so nothing is unconditional.
| Exposed to model | When |
|---|---|
| `mcp.search`, `mcp.call`, `mcp.read_resource` | At activation, iff ≥1 enabled server has `expose = "search"`. Static declarations: cache-stable for the session. |
| `mcp..` per eager tool | At activation from a warm cache; otherwise after that eager server's first connect (post-activation registration, C1), visible from the next provider request. |
| `mcp.read_resource` (eager-only configs) | Registered when the first eager server (cached or live) advertises `resources`. |
| std.lua `mcp` namespace | Iff ≥1 enabled server. No tool cost; a short hint. |
| System hint | Iff ≥1 enabled server: one line per server (name, transport, mode) built from **config only**, so it never changes mid-session; points at search tools when exposed, otherwise at eager tools and docs. |
Eager tools can miss the first turn only on a cold cache (the user submits
before the first-ever connect finishes). Search mode has no such race
(`mcp.call` waits).
**`mcp.search {query?, server?, limit?}`**: connects the relevant servers if
needed and searches the cached catalog (tools, resources, resource templates)
client-side. Ranking: simple term scoring over name, title, description and
parameter names/descriptions (BM25-ish; no dependency). Each hit returns
`server`, `name`, title/description and the **full input schema**, so search
doubles as describe. Server `instructions` (from discover/initialize) are
returned when `server` is given, truncated.
**`mcp.call {server, tool, input}`**: exact `server` + `tool` only; never fuzzy
execution. Unknown names return an error listing the closest matches.
`input` is validated against the tool's live input schema before sending;
failure returns the validation errors plus the schema. Declared non-strict
(an `any`-shaped `input` cannot be expressed in OpenAI/Anthropic strict mode).
**`mcp.read_resource {server, uri}`**: `resources/read` on the originating server.
Never fetches arbitrary URLs itself; a `resource_link` in a tool result names
its server.
Eager tool names are `mcp..`, with the MCP tool name sanitized
to `[A-Za-z0-9_-]` and `__` collapsed (panto encodes `.` as `__` on the wire).
Every tool gets a suffix derived only from its own MCP name, so adding another
tool cannot rename it. Truncation keeps the encoded wire name within **64
bytes**. The limit is the strictest declared provider cap checked: Anthropic
Messages tool names allow 128 characters; OpenAI Chat Completions function
names cap at 64; OpenAI Responses documents function names as strings without
a stated limit. Sources: [Anthropic Messages](https://platform.claude.com/docs/en/api/http/messages/create),
[OpenAI Chat Completions](https://developers.openai.com/api/reference/resources/chat/subresources/completions/methods/create),
[OpenAI Responses](https://developers.openai.com/api/reference/resources/responses/methods/create).
Eager tools are validated the same way as `mcp.call` (panto has no schema
validation of its own).
### std.lua `mcp` namespace ("code mode")
Registered with `require("_std").lua_namespace("mcp", factory)`; available to
child agents too. Functions run in the std.lua handler coroutine and yield on
luv:
```lua
mcp.servers() -- name, transport, mode, era, state, error
mcp.search(query, {server=, limit=})
mcp.call(server, tool, input) -- validated; returns {text, attachments, structured, is_error}
mcp.start(server, tool, input) -- handle; handle:await(); for parallel calls
mcp.read_resource(server, uri)
mcp.prompts(server), mcp.prompt(server, name, args)
```
Reference docs ship as a require-able module (`require("panto_mcp.docs")`),
like `subagents.docs` and `_std.docs`: discovery, examples, result shapes,
errors. Only the one-line hint is injected. Per-server proxy sugar
(`mcp.github.create_issue{}`) is skipped; add it if models reach for it.
### Input validation
A pure-Lua JSON Schema subset validator, ported from panto-subagents'
`workflow.lua` (copied, not a dependency). Covers `type`, `properties`,
`required`, `additionalProperties`, `enum`/`const`, `items`/`prefixItems`,
numeric/string/array bounds, `anyOf`/`oneOf`/`allOf`/`not`, local `$ref`/
`$defs`. Unknown keywords (including `pattern`/`format` if unsupported) are
**ignored, not rejected**: validation exists to catch cheap mistakes, and a
false rejection would block a valid call. Errors carry JSON-pointer paths.
### Result mapping (tool calls, resources, prompts)
| MCP content | panto result |
|---|---|
| `text` | text |
| `image` | attachment (base64 decoded in Lua; size-capped) |
| `audio` | text placeholder with MIME type and size |
| `resource` (embedded text) | text, prefixed with its URI |
| `resource` (embedded blob) | attachment if image, else placeholder |
| `resource_link` | text line: name, URI, description, and "read with `mcp.read_resource{server=…, uri=…}`" (or `mcp.read_resource` in std.lua) |
| `structuredContent` | JSON text, only if no text block already carries it |
| `isError: true` | tool error (model-visible) with the content |
| JSON-RPC `error` | tool error: `: ()` |
| `resultType: input_required` | tool error: server requested user input, which panto does not support yet |
Output is capped (configurable later if needed); truncation is stated in the
result.
### Prompts
- Each server prompt becomes a slash command **`/mcp::`**,
registered at activation from the metadata cache, or after the first connect
on a cold cache (core C2), and unregistered if a later connect shows it gone. Description = prompt title/description plus its argument names, so the
fuzzy command selector finds it.
- Arguments: `name=value` tokens with shell-style quoting; a prompt with exactly
one argument accepts the trimmed raw remainder when no matching `name=` token
appears. Missing required arguments → error listing them.
- On invoke, start `prompts/get` in an async coroutine from the synchronous
command callback. Refuse if a turn or non-prompt command is active, and
re-check when the result arrives. Prompt responses append real user/assistant
roles to the borrowed host conversation; a trailing user message is submitted
to start a turn. Assistant media is represented by text placeholders.
- Prompt names the core rejects (slash/whitespace) are skipped with a warning
visible in `/mcp`. Async failures mount a visible error above the input until
the next `command_start`. One-argument positional values preserve the trimmed
raw remainder when no matching `name=` argument is present.
- **Assistant-tail rule:** if the prompt does not end with a user message,
append all messages and skip `agent:submit`. Emit `agent_submission` so the
idle TUI sync renders and persists the tail without starting a turn.
- All prompt message blocks are mapped before conversation mutation. Only
`image/*` content becomes user media; assistant media and non-image blobs
become text placeholders. This avoids partial history from rejected media.
- **Conversation sync verified in ws-1:** idle `onPaint` syncs the conversation
tail and calls `flushPersist`; idle submission draining syncs/persists before
queued `agent:submit` work starts. Assistant-tail prompts emit
`agent_submission` to request that same idle sync. No C5 core change is needed.
### Resources
Built in v1 because tool results can carry `resource_link`s that are only
usable with `resources/read`.
- Catalog: `resources/list` and `resources/templates/list` (paginated, cached
per `ttlMs`), searchable through `mcp.search`.
- Reading: `mcp.read_resource` tool and namespace function; templates are filled by the
model/user into a concrete URI, then read.
- Only servers advertising `resources` are queried. Some legacy clients treat
`-32601` from list methods as fatal; we never call unadvertised methods.
## Protocol client
Layers (each a module; transports are interchangeable under the client):
1. **Transport**
- *stdio:* NDJSON framing over luv pipes; partial-line buffering; stderr
captured into a bounded per-server ring for diagnostics (never treated as
an error signal).
- *HTTP (modern):* POST per request with the required headers; response is
JSON or a request-scoped SSE stream parsed in Lua. `x-mcp-header` tool
annotations are validated and mirrored as `Mcp-Param-*` headers.
Cancellation = cancel the request (close its stream).
- *HTTP (legacy 2025-11-25):* adds `Mcp-Session-Id` capture/echo, re-init on
404, best-effort DELETE on close. The optional GET notification stream opens
for eager servers advertising `tools.listChanged`, closes with the session,
retries after an established stream ends, and is replaced after legacy re-init.
2. **JSON-RPC:** id allocation, pending-request table, per-request timeout
(`call_timeout_ms`), cancellation (`notifications/cancelled` on stdio),
late responses to cancelled ids ignored. Progress notifications are accepted
and ignored (no progress token is sent in v1).
3. **Era negotiation:**
- Modern first: `server/discover` with 2026-07-28 `_meta`. Success → modern;
`UnsupportedProtocolVersion (-32022)` → pick from `supported`.
- Any other error, a recognizable legacy rejection, an empty/unrecognized
HTTP 400, or a probe timeout → legacy `initialize` (offer `2025-11-25`,
accept the server's counter-offer among versions we implement), then
`notifications/initialized`.
- The detected era is stored in the server's metadata cache entry to skip
the probe next time; if the cached era fails, re-probe.
4. **Capabilities advertised:** none (`{}`): no elicitation, sampling, roots,
tasks. Client info `{name = "panto-mcp", version}`.
5. **Server→client traffic (legacy):** answer `ping`; reply `-32601` to any
other request. Handle `notifications/tools|resources|prompts/list_changed`
by re-listing (eager: unregister/re-register; search: invalidate catalog;
prompts: re-register commands).
## Local server lifecycle (no orphans)
```mermaid
sequenceDiagram
participant P as panto (extension)
participant W as supervisor (panto lua)
participant S as server
P->>W: spawn detached: fds 0,1,2 = server pipes, fd 3 = lifeline
W->>S: spawn detached (new process group), inheriting fds 0,1,2
W-->>W: close own copies of 0/1; report server pid on stderr marker
P->>S: JSON-RPC over the pipes (no relaying through W)
alt normal session_end / quit
P->>S: close stdin write end
P->>W: close lifeline (fd 3)
else panto dies (SIGKILL, crash, SIGHUP, OOM)
Note over P,W: kernel closes panto's fds: stdin EOF + lifeline EOF
end
W->>W: wait grace (2s) for S to exit
W->>S: SIGTERM process group, wait 2s
W->>S: SIGKILL process group
W-->>W: exit once group is gone
```
- One supervisor per local server: `uv.spawn(uv.exepath(), {"lua",
supervisor_path, ...})`, starts in ~16ms. `supervisor_path` resolved via
`package.searchpath` so it works from a rock or a checkout.
- Both supervisor and server run in their own process groups (`detached`), so
terminal Ctrl+C / Ctrl+Z / SIGHUP do not hit them directly; panto's death is
observed only through the lifeline. The supervisor also traps
INT/TERM/HUP and runs the same teardown.
- The server's group is signalled, catching `npx`/`docker`-style wrappers that
leave grandchildren.
- panto never waits during `session_end`: it closes two fds and returns; process
handles are unref'd so they don't hold the loop.
- **Accepted gap:** if panto and the supervisor are both SIGKILLed at once, a
server that ignores stdin EOF survives. No pidfile sweep (rare enough).
- Precedent: `../panto-claudesub/panto-claudesub/_session.lua` (detached group,
bounded TERM → KILL escalation, confirmed exit).
## Remote auth
Order of preference per server: `auth = ""` (static API key or OAuth
session in `[auth.*]`), then automatic MCP OAuth on a 401.
**Extension (MCP policy):**
1. On 401, parse `WWW-Authenticate` for `resource_metadata`; else try the
RFC 9728 well-known PRM URIs (endpoint path, then origin root).
2. Fetch authorization-server metadata (RFC 8414, then OIDC discovery
variants); require an exact `issuer` match and PKCE (`S256`) support.
3. Client registration: configured `client_id` requires an exact `oauth.issuer`
binding (optional env-expanded `client_secret`) → public Dynamic Client Registration.
Confidential DCR is refused; no client secret is stored in `panto.ext.state`.
Public registrations are stored per issuer and retained on every failure. Any
configured issuer pins selection from authorization_servers, even without a client ID.
CIMD is deferred (`TODOs.md`).
4. Define a runtime `oauth_pkce` session `mcp..` (≤128 bytes),
using 128 bits of SHA-256 over canonical resource, exact issuer and client ID.
Retire the old name with core logout on changes. Use discovered endpoints,
challenge → PRM → configured scopes, and `resource` = canonical server URI.
Warm activation restores discovery without network work; no attestation flag.
5. Login is user-initiated: `/mcp-login [callback-url | --reregister]`.
Reuse a definition unless missing or core login fails; keep registrations on all
failures. Only a changed discovered issuer or explicit --reregister discards one.
The escape hatch rediscovers/registers once; errors mention it. Consent with an
existing usable token does not lock out tools, and give-up clears login_required.
6. Authenticated 401: one forced refresh, then one retry; parallel calls await the
shared refresh. Scope 403 redefines the scope union and asks for re-consent,
without disabling working calls or an artificial attempt counter. Other 403s
report permission errors; explicit non-PKCE auth errors point to core auth.
Legacy GET reconnect backs off to 60 seconds and stops with a /mcp warning on 401.
7. Root PRM fallback validates `resource` against the origin. Loopback discovery/
endpoints require a loopback resource; reject leading-zero IPv4 octets. Cache
status reads/poll at ~1 second and back off failed discovery from 5 to 60 seconds.
**Core (credential mechanism, C4):** generic `oauth_pkce` auth type
(authorization code + PKCE S256, `state` check, optional `iss` validation,
`resource` parameter, loopback callback on an ephemeral port, **pasted
callback URL fallback** for SSH/headless, token storage under `auth/`,
refresh, header injection) plus `panto.ext` APIs to define a named session at
runtime and start its login. Requests then use `http_request{auth = name}`;
tokens never enter Lua.
## Core additions (pantograph / libpanto)
New `panto.ext` surfaces are co-designed with Travis before building; each
passes the AGENTS.md tests (second consumer, no "mcp" in core names, mechanism
not policy).
| # | Addition | Second consumer |
|---|---|---|
| C1 | `register_tool` after activation + `unregister_tool(name)`. Changes take effect at the next turn (per-turn snapshot); in-flight calls unaffected; same collision rules; child agents sharing the Lua tool source see it. Needs copy-on-write declarations in the Lua `ToolSource` (provider serialization reads them off the owner thread). | Any extension whose tools depend on async discovery: plugin loaders, per-skill tools, a tool set that changes with project state. |
| C2 | `register_command` after activation + `unregister_command(name)`; the fuzzy selector and `render_header` command list pick them up. | subagents' `/workflow:` when profiles change on disk; skills exposed as commands. |
| C3 | `http_request`: `method` (GET/POST/PUT/PATCH/DELETE), `body`; streaming via `on_head(head)` / `on_data(chunk)` hooks (presence of `on_data` disables buffering; final callback still fires with status/headers or error); `idle_timeout_ms`; no total cap for streams (`timeout_ms = 0`); keep-alive with a process-lifetime connection pool (callbacks still die with the runtime); auth allowed over plain HTTP to loopback; streams counted separately from the 16 buffered slots; bounded chunk queue for backpressure. | Lua protocol providers streaming a custom wire API; webhook/REST clients; the messaging bridge's long-poll. |
| C4 | `oauth_pkce` auth type + runtime session definition and login APIs (above). Seeded `config.toml` documents the new auth fields. | Messaging bridge (Slack/Telegram OAuth); any extension calling an OAuth-protected API. |
| C5 | *Only if spike S2 fails:* host-agent method to append arbitrary-role messages that persists and renders. | Fork/import tools, transcript surgery extensions, prompt-library extensions. |
Draft Lua shape for C3 (to be co-designed):
```lua
local handle, err = panto.ext.http_request({
url = url, method = "POST", body = json, auth = "mcp.github",
headers = { ["content-type"] = "application/json",
accept = "application/json, text/event-stream" },
timeout_ms = 0, idle_timeout_ms = 60000,
on_head = function(head) end, -- {status, headers}
on_data = function(chunk) end, -- raw bytes; SSE parsing is the caller's
}, function(response, err) end) -- buffered: {status, body, headers}; streamed: no body
handle:cancel()
```
## Build phases
Spikes first, then extension work proceeds in parallel with core work
(stdio + search mode needs no core change).
- [x] **S1 supervisor spike:** luv `spawn` with a 4th stdio slot from Lua;
detached supervisor + detached server; EOF propagation when panto is
`kill -9`ed; group kill of an `sh -c 'trap "" TERM; sleep …'` tree.
- [x] **S2 conversation spike:** source-verified: asynchronous idle edits sync
through `onPaint`; `agent:submit` synchronizes before the TUI drains its
queued work. Assistant-tail prompts explicitly emit `agent_submission`
to wake that same render/persist path. C5 is unnecessary.
- [x] **S3 tool-source mutability:** read libpanto's registry/`ToolSource`
snapshot path to size C1.
- [x] **E1 core client:** rock skeleton (`panto-mcp` → `init.lua`,
`panto_mcp.*` modules), config parsing/expansion, JSON-RPC, stdio
transport + supervisor, era negotiation, catalog + metadata cache
(lazy start), `mcp.search` /
`mcp.call`, validator, result mapping, `session_end` teardown, `/mcp`
status command.
- [x] **E2 code mode:** std.lua `mcp` namespace, `panto_mcp.docs`, system hint.
- [x] **C3 HTTP** → **E3 HTTP transport:** modern POST/SSE, legacy sessions,
static headers and `auth = ""`.
- [x] **E4 resources:** list/templates/read, `mcp.read_resource`, resource blocks in
results, catalog search over resources.
- [x] **C1/C2 dynamic registration** → **E5 eager mode** (including legacy HTTP
GET notifications) and **E6 prompts** (`/mcp::`); no C5 fix.
- [x] **C4 OAuth** → **E7 MCP OAuth:** PRM/AS discovery, configured/DCR registration,
`/mcp-login` (loopback + pasted callback), refresh-on-401 and non-blocking scope step-up;
issuer-bound configured clients, identity-scoped sessions, public-only DCR,
rediscovery/backoff/status caching, warm restoration and real-core TCP-fake checks.
Real-provider pass remains below.
- [ ] **E8 release:** README, rockspec, mise tasks (copy panto-web's
`version`/`release`), luarocks upload. Files done; tag/upload left to Travis.
- [x] Note on pantograph `TODOs.md` "User input for extensions" that it enables
MCP elicitation in panto-mcp.
## Testing
Per pantograph AGENTS.md: never test timeouts by sleeping; inject the clock /
timer factory so timeouts fire deterministically.
- `_selfcheck.lua` run with `panto lua` (like panto-web/panto-claudesub):
fake stdio servers written in Lua for both eras (modern discover; legacy
initialize; legacy server that errors on pre-initialize requests; server that
ignores EOF; server that spawns a grandchild), and a luv TCP fake for HTTP
(JSON response, SSE response, legacy session 404 → re-init, 401 → PRM).
- Eager tools/prompts: warm activation is idle; a cold `mcp.search` connect
reconciles eager declarations; `list_changed` reconciles add/remove/schema
changes. Tests cover multi-server prompt preservation, `/compact` command
busy state, visible async failures, image-only media and all-message preflight,
raw one-argument remainders, invalid command/server names, stable eager names,
exact cache pruning, and legacy GET reconnect/re-init.
- Validator, SSE parser, NDJSON framer, result mapping, name sanitizing, arg
parsing: table-driven unit checks in the same selfcheck.
- OAuth: table-driven challenge/URL/issuer/S256/scope checks; injected HTTP/auth/timers
cover discovery → public DCR → user login → reconnect, refresh and scope upgrades.
Regressions cover foreign-issuer credentials, identity-scoped sessions, non-locking
403s, concurrent refresh calls, confidential-DCR refusal, rediscovery and backoff.
`PANTO_MCP_HTTP_E2E=1` adds a new-core browser/pasted-callback flow, forced refresh,
scope re-consent and warm reuse against a local TCP fake (no real credentials).
- Orphan check: spawn supervisor+server from a stand-in parent process,
`kill -9` the parent, wait on the server group's exit event (bounded), assert
nothing survives.
- Core additions get Zig tests in pantograph/libpanto alongside their code.
- One manual end-to-end pass against real servers: a filesystem stdio server
via `npx`, a remote legacy server, a remote 2026 server, and one OAuth server
(GitHub or Linear).
## Resolved design checks
- **Provider tool-name limits:** use 64 encoded wire bytes, the strictest declared
bound in the primary provider docs linked above.