# panto-subagents refactor: extension-API-first

## Goal

The v0 implementation works end to end, but pantograph absorbed the feature:
`subagent_host.zig`, `subagent_runtime.zig`, and `tui_subagents.zig` are a
subagent backend in core, and the rock is mostly validation and formatting
glue. Per the "Extension features: two responses" rubric in pantograph's
AGENTS.md, this refactor takes response 2 retroactively: the missing *general*
capabilities are added to libpanto-lua and `panto.ext`, subagent policy moves
into this rock, and the feature-shaped core surface is deleted.

Definition of done, in one line: `rg -i subagent pantograph/src` returns
nothing, and every behavior in DESIGN.md still holds through the real binary.

The product surface (DESIGN.md) is unchanged. This is a re-division of the
implementation, not a redesign. DESIGN.md's "Generic pantograph child-job
seam" section is superseded by this document and should be rewritten to match
when this lands.

## What exists today (all currently uncommitted working-tree state)

Generic and staying (invisible runtime correctness, landed with v0):

- The multiplexed Lua executor: tool batches keyed by id instead of a single
  `current_batch`, cross-thread posted jobs onto the Lua owner thread
  (`lua_executor.zig`, `lua_runtime.zig`).
- Lane identity: `panto.ext.agent` resolves through `coroutine.running()`;
  per-open protocol stream state and stream-level `cancel_turn`
  (`lua_provider.zig`).
- libpantograph: message metadata survives the JSONL load path;
  `UserMessage.metadata` exists on `Agent.run`.
- `panto.ext.models` (bounded catalog queries) and `panto.ext.session_info()`.

Feature-shaped and going: `subagent_host.zig` (vtable), `subagent_runtime.zig`
(spawn transaction, worker threads, `Gate` concurrency bound, metadata
stamping, manifest extraction, resume defaults, one-shot workers, progress
queue), `tui_subagents.zig` (hardcoded progress panel), and the
`spawn_agent`/`await_jobs` thunks in `lua_bridge.zig`.

## New libpanto-lua APIs

These are library bindings, useful to any embedder of libpanto-lua; nothing
about them is subagent-specific.

### Async agent jobs

```lua
local job = agent:run_async {
  prompt = "...",                 -- or blocks, mirroring agent:run
  metadata = { ... },             -- optional user-message metadata (JSON-encoded)
  dispatch_tools = false,         -- optional: end the turn at the first
                                  -- assistant response instead of dispatching
                                  -- its tool calls; they are reported settled
  wake_fd = fd,                   -- optional: a byte is written on every event
                                  -- arrival and on settle
}

job:next_event()   -- -> event table | nil  (non-blocking; owned copy)
job:result()       -- -> settled result | nil while running
job:request_cancel()
job:close()        -- join + release; safe after settle or cancel
```

- Implemented with a binding-internal pump thread per job: it drives the
  blocking `stream.next()`, copies each event payload (payloads die at the
  next pull), buffers under a mutex, and writes one byte to `wake_fd`.
- The fd contract is deliberately loop-agnostic: a luv consumer arms
  `uv.new_poll(fd)`; a Go or C embedder selects on it. No loop dependency
  enters libpanto-lua.
- Tool dispatch inside the turn is unchanged libpantograph behavior; when the
  agent's tools include panto's Lua tool source, its `invoke_batch` posts to
  the Lua owner thread through the existing executor — that path already
  exists and does not change.
- `dispatch_tools = false` is the generic knob that makes structured one-shot
  workers a pure consumer feature: the settled result carries the assistant's
  tool calls (name + input JSON) unresolved, and no second model round runs.

### Conversation message metadata

```lua
conv:message_metadata(i)            -- -> table | nil (decoded JSON)
conv:set_message_metadata(i, tbl)   -- JSON-encodes; nil clears
```

Plus optional `metadata` on `conv:add_system_message`/`add_user_message`.
The disk format already round-trips metadata; this is access only.

### Agent tool control

```lua
agent:tools()          -- -> array of decl tables { name, description, schema },
                       --    a mutable copy
agent:set_tools(decls) -- replace this agent's tool list with these decls;
                       --    handlers resolve by name through the owning source
```

Copying between agents works because dispatch is name-keyed on the shared
runtime. `agent:set_config` additionally accepts `tool_choice` (force a named
tool). This retires the Zig-side `toolSourceExcluding`.

## New/changed pantograph surfaces

- `panto.ext.resolve_model { model = "provider:alias", reasoning = label }`
  → an **opaque config userdata** accepted by `panto.agent { config = ... }`
  and `agent:set_config`. Resolution and validation reuse the existing
  registry/reasoning pipeline and fail before inference; credentials are
  embedded host-side and never readable from Lua. This is the one place the
  credential boundary requires a host API rather than a binding API.
- `panto.ext.models` and `panto.ext.session_info()` remain as-is (already
  generic).
- `spawn_agent`, `await_jobs`, the job-handle userdata, and the
  `SubagentHost` vtable are deleted once the rock is ported. Awaiting becomes
  ordinary luv: the rock arms `uv.new_poll` on each job's `wake_fd` and its
  tool-handler coroutine yields/resumes through the standard async-tool
  contract.
- Progress rendering: `tui_subagents.zig` is deleted. The rock pulls its own
  job events and surfaces activity through the existing extension event and
  component machinery (`panto.ext.on`/`emit` + component swap). If that
  machinery proves insufficient for a live multi-card panel, the fallback is
  one small *generic* host API (a "live status card" keyed by an id — second
  consumer: any long-running extension activity), to be co-designed before
  building, per AGENTS.md.

## Policy migration map (the closure check)

Every deleted Zig behavior must have a Lua expression using only the APIs
above. This table is the completeness test for the API set:

| Behavior (today in Zig)                  | New home in the rock |
|------------------------------------------|----------------------|
| Child store layout `<sd>/subagents/<id>` | string assembly (already rock-side) |
| Store create/resolve, ownership boundary | `panto.file_system_jsonl_store(dir)` + `store:resolve/load` |
| Manifest on profile system message       | `conv:set_message_metadata` at seeding |
| Per-turn model/reasoning stamp           | `run_async { metadata = ... }` |
| Resume defaults from last user message   | `store:load` + `conv:message_metadata` scan |
| Primary system context for new children  | copy system messages from `panto.ext.agent:conversation()` |
| Concurrency bound (4) + queue + cancel-while-queued | rock-side gate: start ≤4 jobs, queue the rest, drop queued on cancel (~15 lines of Lua) |
| `subagents.*` exclusion (no recursion)   | `agent:set_tools(filtered)` from the primary's `agent:tools()` |
| One-shot structured workers              | new agent + single synthetic tool + `tool_choice` + `dispatch_tools = false` + `panto.null_store()`; validate with the existing jsonschema path |
| `resumable` determination                | fresh `store:resolve(id)` after failure |
| Result shaping (id/agent/status/output)  | already rock-side; loses the host `JobResult` intermediary |
| Progress cards                           | rock consumes `job:next_event()`; renders via event/component machinery |
| Escape cancels all children              | rock tracks its live jobs; a host cancellation event (or the turn-teardown hook that fires it) calls `job:request_cancel()` on each — verify the existing bus exposes interrupt; if not, add a generic turn-lifecycle event |

Two rows need small verifications during implementation (marked above):
turn-lifecycle/interrupt visibility to extensions, and whether the component
machinery can host the live panel. Both resolutions must stay feature-neutral.

## Deletions

pantograph: `subagent_host.zig`, `subagent_runtime.zig`, `tui_subagents.zig`,
the spawn/await thunks and job userdata in `lua_bridge.zig`, the subagent
wiring in `main.zig`/`tui_app.zig`, and `subagent_integration_tests.zig` in
its current form (see validation). The executor, lane machinery, and
`lua_provider` lane calls remain as internal runtime correctness — reviewed
afterward for members only the deleted code used.

## Validation

- The v0 end-to-end proof must survive: a scripted extension-protocol fixture
  driving the real binary with the real rock — parallel `subagents.run` +
  unknown-agent error in one batch, child resume with history, workflow
  diamond, sandbox fan-out, catalog round-trip, child JSONL
  isolation/manifest/metadata asserted from bytes on disk. (Reconstruct the
  harness from this list; treat it as the primary gate.)
- Rock suite (`mise run check`) extended to cover the migrated policy:
  gating/bound behavior, resume-default extraction, one-shot capture via
  unresolved tool calls, tool filtering, manifest seeding.
- pantograph integration tests rewritten against the *generic* surfaces
  (async jobs driving concurrent agents, batch isolation, lane agent
  resolution, cancellation) with no subagent vocabulary.
- `rg -i subagent /Users/travis/Code/pantograph/src` → empty.
- All suites green: `mise exec -- zig build test`, `mise run check`.

## Delivery order

1. libpanto-lua: `run_async` jobs + fd contract + `dispatch_tools` knob, with
   binding-level tests (fixture provider, no panto involvement).
2. libpanto-lua: message-metadata access; agent tool get/set + `tool_choice`.
3. pantograph: `resolve_model` opaque config; verify interrupt/lifecycle
   event visibility (add the generic event if missing).
4. Rock: port spawn/await/bound/policy onto the new APIs behind the same tool
   surfaces; port progress to event/component rendering.
5. Delete the pantograph subagent files and thunks; rewrite integration tests
   generically; run the full validation list.
6. Update DESIGN.md's seam sections to describe the final division.

## Interop with the panto-on-libuv project (separate; order-independent)

The `wake_fd` contract works identically in both worlds: today the rock arms
it with `uv.new_poll` on luv's loop; after the libuv unification it is the
same call on the shared process loop. If the libuv project lands first, the
executor internals this refactor relies on (posted jobs onto the Lua owner)
will already use `uv_async` instead of the pipe waker — no change to anything
specified here. Neither project should block on the other.