diff options
| author | t <t@tjp.lol> | 2026-08-18 14:18:14 -0600 |
|---|---|---|
| committer | t <t@tjp.lol> | 2026-08-18 14:20:28 -0600 |
| commit | cc69a2e431779528578211a5cb3385bb23e076bf (patch) | |
| tree | 8833171e25bb3dfb71fc4a2e0b8fe583782da764 /REFACTOR.md | |
| parent | 372ef8ff40991644ec2654c61328f31779f4ad21 (diff) | |
Persist and replay subagent progress across sessions
Record durable child-turn metadata, restore bounded progress cards during
startup replay, and keep settled cards attached to transcript entries. Add
workflow-local Lua profiles and expose discovered agents and workflows in the
session header.
Diffstat (limited to 'REFACTOR.md')
| -rw-r--r-- | REFACTOR.md | 196 |
1 files changed, 0 insertions, 196 deletions
diff --git a/REFACTOR.md b/REFACTOR.md deleted file mode 100644 index 3eec504..0000000 --- a/REFACTOR.md +++ /dev/null @@ -1,196 +0,0 @@ -# panto-subagents refactor: extension-API-first - -## Goal - -The v0 implementation works end to end, but pantograph absorbed the feature: -`subagent_host.zig`, `subagent_runtime.zig`, and `tui_subagents.zig` are a -subagent backend in core, and the rock is mostly validation and formatting -glue. Per the "Extension features: two responses" rubric in pantograph's -AGENTS.md, this refactor takes response 2 retroactively: the missing *general* -capabilities are added to libpanto-lua and `panto.ext`, subagent policy moves -into this rock, and the feature-shaped core surface is deleted. - -Definition of done, in one line: `rg -i subagent pantograph/src` returns -nothing, and every behavior in DESIGN.md still holds through the real binary. - -The product surface (DESIGN.md) is unchanged. This is a re-division of the -implementation, not a redesign. DESIGN.md's "Generic pantograph child-job -seam" section is superseded by this document and should be rewritten to match -when this lands. - -## What exists today (all currently uncommitted working-tree state) - -Generic and staying (invisible runtime correctness, landed with v0): - -- The multiplexed Lua executor: tool batches keyed by id instead of a single - `current_batch`, cross-thread posted jobs onto the Lua owner thread - (`lua_executor.zig`, `lua_runtime.zig`). -- Lane identity: `panto.ext.agent` resolves through `coroutine.running()`; - per-open protocol stream state and stream-level `cancel_turn` - (`lua_provider.zig`). -- libpantograph: message metadata survives the JSONL load path; - `UserMessage.metadata` exists on `Agent.run`. -- `panto.ext.models` (bounded catalog queries) and `panto.ext.session_info()`. - -Feature-shaped and going: `subagent_host.zig` (vtable), `subagent_runtime.zig` -(spawn transaction, worker threads, `Gate` concurrency bound, metadata -stamping, manifest extraction, resume defaults, one-shot workers, progress -queue), `tui_subagents.zig` (hardcoded progress panel), and the -`spawn_agent`/`await_jobs` thunks in `lua_bridge.zig`. - -## New libpanto-lua APIs - -These are library bindings, useful to any embedder of libpanto-lua; nothing -about them is subagent-specific. - -### Async agent jobs - -```lua -local job = agent:run_async { - prompt = "...", -- or blocks, mirroring agent:run - metadata = { ... }, -- optional user-message metadata (JSON-encoded) - dispatch_tools = false, -- optional: end the turn at the first - -- assistant response instead of dispatching - -- its tool calls; they are reported settled - wake_fd = fd, -- optional: a byte is written on every event - -- arrival and on settle -} - -job:next_event() -- -> event table | nil (non-blocking; owned copy) -job:result() -- -> settled result | nil while running -job:request_cancel() -job:close() -- join + release; safe after settle or cancel -``` - -- Implemented with a binding-internal pump thread per job: it drives the - blocking `stream.next()`, copies each event payload (payloads die at the - next pull), buffers under a mutex, and writes one byte to `wake_fd`. -- The fd contract is deliberately loop-agnostic: a luv consumer arms - `uv.new_poll(fd)`; a Go or C embedder selects on it. No loop dependency - enters libpanto-lua. -- Tool dispatch inside the turn is unchanged libpantograph behavior; when the - agent's tools include panto's Lua tool source, its `invoke_batch` posts to - the Lua owner thread through the existing executor — that path already - exists and does not change. -- `dispatch_tools = false` is the generic knob that makes structured one-shot - workers a pure consumer feature: the settled result carries the assistant's - tool calls (name + input JSON) unresolved, and no second model round runs. - -### Conversation message metadata - -```lua -conv:message_metadata(i) -- -> table | nil (decoded JSON) -conv:set_message_metadata(i, tbl) -- JSON-encodes; nil clears -``` - -Plus optional `metadata` on `conv:add_system_message`/`add_user_message`. -The disk format already round-trips metadata; this is access only. - -### Agent tool control - -```lua -agent:tools() -- -> array of decl tables { name, description, schema }, - -- a mutable copy -agent:set_tools(decls) -- replace this agent's tool list with these decls; - -- handlers resolve by name through the owning source -``` - -Copying between agents works because dispatch is name-keyed on the shared -runtime. `agent:set_config` additionally accepts `tool_choice` (force a named -tool). This retires the Zig-side `toolSourceExcluding`. - -## New/changed pantograph surfaces - -- `panto.ext.resolve_model { model = "provider:alias", reasoning = label }` - → an **opaque config userdata** accepted by `panto.agent { config = ... }` - and `agent:set_config`. Resolution and validation reuse the existing - registry/reasoning pipeline and fail before inference; credentials are - embedded host-side and never readable from Lua. This is the one place the - credential boundary requires a host API rather than a binding API. -- `panto.ext.models` and `panto.ext.session_info()` remain as-is (already - generic). -- `spawn_agent`, `await_jobs`, the job-handle userdata, and the - `SubagentHost` vtable are deleted once the rock is ported. Awaiting becomes - ordinary luv: the rock arms `uv.new_poll` on each job's `wake_fd` and its - tool-handler coroutine yields/resumes through the standard async-tool - contract. -- Progress rendering: `tui_subagents.zig` is deleted. The rock pulls its own - job events and surfaces activity through the existing extension event and - component machinery (`panto.ext.on`/`emit` + component swap). If that - machinery proves insufficient for a live multi-card panel, the fallback is - one small *generic* host API (a "live status card" keyed by an id — second - consumer: any long-running extension activity), to be co-designed before - building, per AGENTS.md. - -## Policy migration map (the closure check) - -Every deleted Zig behavior must have a Lua expression using only the APIs -above. This table is the completeness test for the API set: - -| Behavior (today in Zig) | New home in the rock | -|------------------------------------------|----------------------| -| Child store layout `<sd>/subagents/<id>` | string assembly (already rock-side) | -| Store create/resolve, ownership boundary | `panto.file_system_jsonl_store(dir)` + `store:resolve/load` | -| Manifest on profile system message | `conv:set_message_metadata` at seeding | -| Per-turn model/reasoning stamp | `run_async { metadata = ... }` | -| Resume defaults from last user message | `store:load` + `conv:message_metadata` scan | -| Primary system context for new children | copy system messages from `panto.ext.agent:conversation()` | -| Concurrency bound (4) + queue + cancel-while-queued | rock-side gate: start ≤4 jobs, queue the rest, drop queued on cancel (~15 lines of Lua) | -| `subagents.*` exclusion (no recursion) | `agent:set_tools(filtered)` from the primary's `agent:tools()` | -| One-shot structured workers | new agent + single synthetic tool + `tool_choice` + `dispatch_tools = false` + `panto.null_store()`; validate with the existing jsonschema path | -| `resumable` determination | fresh `store:resolve(id)` after failure | -| Result shaping (id/agent/status/output) | already rock-side; loses the host `JobResult` intermediary | -| Progress cards | rock consumes `job:next_event()`; renders via event/component machinery | -| Escape cancels all children | rock tracks its live jobs; a host cancellation event (or the turn-teardown hook that fires it) calls `job:request_cancel()` on each — verify the existing bus exposes interrupt; if not, add a generic turn-lifecycle event | - -Two rows need small verifications during implementation (marked above): -turn-lifecycle/interrupt visibility to extensions, and whether the component -machinery can host the live panel. Both resolutions must stay feature-neutral. - -## Deletions - -pantograph: `subagent_host.zig`, `subagent_runtime.zig`, `tui_subagents.zig`, -the spawn/await thunks and job userdata in `lua_bridge.zig`, the subagent -wiring in `main.zig`/`tui_app.zig`, and `subagent_integration_tests.zig` in -its current form (see validation). The executor, lane machinery, and -`lua_provider` lane calls remain as internal runtime correctness — reviewed -afterward for members only the deleted code used. - -## Validation - -- The v0 end-to-end proof must survive: a scripted extension-protocol fixture - driving the real binary with the real rock — parallel `subagents.run` + - unknown-agent error in one batch, child resume with history, workflow - diamond, sandbox fan-out, catalog round-trip, child JSONL - isolation/manifest/metadata asserted from bytes on disk. (Reconstruct the - harness from this list; treat it as the primary gate.) -- Rock suite (`mise run check`) extended to cover the migrated policy: - gating/bound behavior, resume-default extraction, one-shot capture via - unresolved tool calls, tool filtering, manifest seeding. -- pantograph integration tests rewritten against the *generic* surfaces - (async jobs driving concurrent agents, batch isolation, lane agent - resolution, cancellation) with no subagent vocabulary. -- `rg -i subagent /Users/travis/Code/pantograph/src` → empty. -- All suites green: `mise exec -- zig build test`, `mise run check`. - -## Delivery order - -1. libpanto-lua: `run_async` jobs + fd contract + `dispatch_tools` knob, with - binding-level tests (fixture provider, no panto involvement). -2. libpanto-lua: message-metadata access; agent tool get/set + `tool_choice`. -3. pantograph: `resolve_model` opaque config; verify interrupt/lifecycle - event visibility (add the generic event if missing). -4. Rock: port spawn/await/bound/policy onto the new APIs behind the same tool - surfaces; port progress to event/component rendering. -5. Delete the pantograph subagent files and thunks; rewrite integration tests - generically; run the full validation list. -6. Update DESIGN.md's seam sections to describe the final division. - -## Interop with the panto-on-libuv project (separate; order-independent) - -The `wake_fd` contract works identically in both worlds: today the rock arms -it with `uv.new_poll` on luv's loop; after the libuv unification it is the -same call on the shared process loop. If the libuv project lands first, the -executor internals this refactor relies on (posted jobs onto the Lua owner) -will already use `uv_async` instead of the pipe waker — no change to anything -specified here. Neither project should block on the other. |
