summaryrefslogtreecommitdiff
path: root/REFACTOR.md
diff options
context:
space:
mode:
authort <t@tjp.lol>2026-08-18 14:18:14 -0600
committert <t@tjp.lol>2026-08-18 14:20:28 -0600
commitcc69a2e431779528578211a5cb3385bb23e076bf (patch)
tree8833171e25bb3dfb71fc4a2e0b8fe583782da764 /REFACTOR.md
parent372ef8ff40991644ec2654c61328f31779f4ad21 (diff)
Persist and replay subagent progress across sessions
Record durable child-turn metadata, restore bounded progress cards during startup replay, and keep settled cards attached to transcript entries. Add workflow-local Lua profiles and expose discovered agents and workflows in the session header.
Diffstat (limited to 'REFACTOR.md')
-rw-r--r--REFACTOR.md196
1 files changed, 0 insertions, 196 deletions
diff --git a/REFACTOR.md b/REFACTOR.md
deleted file mode 100644
index 3eec504..0000000
--- a/REFACTOR.md
+++ /dev/null
@@ -1,196 +0,0 @@
-# panto-subagents refactor: extension-API-first
-
-## Goal
-
-The v0 implementation works end to end, but pantograph absorbed the feature:
-`subagent_host.zig`, `subagent_runtime.zig`, and `tui_subagents.zig` are a
-subagent backend in core, and the rock is mostly validation and formatting
-glue. Per the "Extension features: two responses" rubric in pantograph's
-AGENTS.md, this refactor takes response 2 retroactively: the missing *general*
-capabilities are added to libpanto-lua and `panto.ext`, subagent policy moves
-into this rock, and the feature-shaped core surface is deleted.
-
-Definition of done, in one line: `rg -i subagent pantograph/src` returns
-nothing, and every behavior in DESIGN.md still holds through the real binary.
-
-The product surface (DESIGN.md) is unchanged. This is a re-division of the
-implementation, not a redesign. DESIGN.md's "Generic pantograph child-job
-seam" section is superseded by this document and should be rewritten to match
-when this lands.
-
-## What exists today (all currently uncommitted working-tree state)
-
-Generic and staying (invisible runtime correctness, landed with v0):
-
-- The multiplexed Lua executor: tool batches keyed by id instead of a single
- `current_batch`, cross-thread posted jobs onto the Lua owner thread
- (`lua_executor.zig`, `lua_runtime.zig`).
-- Lane identity: `panto.ext.agent` resolves through `coroutine.running()`;
- per-open protocol stream state and stream-level `cancel_turn`
- (`lua_provider.zig`).
-- libpantograph: message metadata survives the JSONL load path;
- `UserMessage.metadata` exists on `Agent.run`.
-- `panto.ext.models` (bounded catalog queries) and `panto.ext.session_info()`.
-
-Feature-shaped and going: `subagent_host.zig` (vtable), `subagent_runtime.zig`
-(spawn transaction, worker threads, `Gate` concurrency bound, metadata
-stamping, manifest extraction, resume defaults, one-shot workers, progress
-queue), `tui_subagents.zig` (hardcoded progress panel), and the
-`spawn_agent`/`await_jobs` thunks in `lua_bridge.zig`.
-
-## New libpanto-lua APIs
-
-These are library bindings, useful to any embedder of libpanto-lua; nothing
-about them is subagent-specific.
-
-### Async agent jobs
-
-```lua
-local job = agent:run_async {
- prompt = "...", -- or blocks, mirroring agent:run
- metadata = { ... }, -- optional user-message metadata (JSON-encoded)
- dispatch_tools = false, -- optional: end the turn at the first
- -- assistant response instead of dispatching
- -- its tool calls; they are reported settled
- wake_fd = fd, -- optional: a byte is written on every event
- -- arrival and on settle
-}
-
-job:next_event() -- -> event table | nil (non-blocking; owned copy)
-job:result() -- -> settled result | nil while running
-job:request_cancel()
-job:close() -- join + release; safe after settle or cancel
-```
-
-- Implemented with a binding-internal pump thread per job: it drives the
- blocking `stream.next()`, copies each event payload (payloads die at the
- next pull), buffers under a mutex, and writes one byte to `wake_fd`.
-- The fd contract is deliberately loop-agnostic: a luv consumer arms
- `uv.new_poll(fd)`; a Go or C embedder selects on it. No loop dependency
- enters libpanto-lua.
-- Tool dispatch inside the turn is unchanged libpantograph behavior; when the
- agent's tools include panto's Lua tool source, its `invoke_batch` posts to
- the Lua owner thread through the existing executor — that path already
- exists and does not change.
-- `dispatch_tools = false` is the generic knob that makes structured one-shot
- workers a pure consumer feature: the settled result carries the assistant's
- tool calls (name + input JSON) unresolved, and no second model round runs.
-
-### Conversation message metadata
-
-```lua
-conv:message_metadata(i) -- -> table | nil (decoded JSON)
-conv:set_message_metadata(i, tbl) -- JSON-encodes; nil clears
-```
-
-Plus optional `metadata` on `conv:add_system_message`/`add_user_message`.
-The disk format already round-trips metadata; this is access only.
-
-### Agent tool control
-
-```lua
-agent:tools() -- -> array of decl tables { name, description, schema },
- -- a mutable copy
-agent:set_tools(decls) -- replace this agent's tool list with these decls;
- -- handlers resolve by name through the owning source
-```
-
-Copying between agents works because dispatch is name-keyed on the shared
-runtime. `agent:set_config` additionally accepts `tool_choice` (force a named
-tool). This retires the Zig-side `toolSourceExcluding`.
-
-## New/changed pantograph surfaces
-
-- `panto.ext.resolve_model { model = "provider:alias", reasoning = label }`
- → an **opaque config userdata** accepted by `panto.agent { config = ... }`
- and `agent:set_config`. Resolution and validation reuse the existing
- registry/reasoning pipeline and fail before inference; credentials are
- embedded host-side and never readable from Lua. This is the one place the
- credential boundary requires a host API rather than a binding API.
-- `panto.ext.models` and `panto.ext.session_info()` remain as-is (already
- generic).
-- `spawn_agent`, `await_jobs`, the job-handle userdata, and the
- `SubagentHost` vtable are deleted once the rock is ported. Awaiting becomes
- ordinary luv: the rock arms `uv.new_poll` on each job's `wake_fd` and its
- tool-handler coroutine yields/resumes through the standard async-tool
- contract.
-- Progress rendering: `tui_subagents.zig` is deleted. The rock pulls its own
- job events and surfaces activity through the existing extension event and
- component machinery (`panto.ext.on`/`emit` + component swap). If that
- machinery proves insufficient for a live multi-card panel, the fallback is
- one small *generic* host API (a "live status card" keyed by an id — second
- consumer: any long-running extension activity), to be co-designed before
- building, per AGENTS.md.
-
-## Policy migration map (the closure check)
-
-Every deleted Zig behavior must have a Lua expression using only the APIs
-above. This table is the completeness test for the API set:
-
-| Behavior (today in Zig) | New home in the rock |
-|------------------------------------------|----------------------|
-| Child store layout `<sd>/subagents/<id>` | string assembly (already rock-side) |
-| Store create/resolve, ownership boundary | `panto.file_system_jsonl_store(dir)` + `store:resolve/load` |
-| Manifest on profile system message | `conv:set_message_metadata` at seeding |
-| Per-turn model/reasoning stamp | `run_async { metadata = ... }` |
-| Resume defaults from last user message | `store:load` + `conv:message_metadata` scan |
-| Primary system context for new children | copy system messages from `panto.ext.agent:conversation()` |
-| Concurrency bound (4) + queue + cancel-while-queued | rock-side gate: start ≤4 jobs, queue the rest, drop queued on cancel (~15 lines of Lua) |
-| `subagents.*` exclusion (no recursion) | `agent:set_tools(filtered)` from the primary's `agent:tools()` |
-| One-shot structured workers | new agent + single synthetic tool + `tool_choice` + `dispatch_tools = false` + `panto.null_store()`; validate with the existing jsonschema path |
-| `resumable` determination | fresh `store:resolve(id)` after failure |
-| Result shaping (id/agent/status/output) | already rock-side; loses the host `JobResult` intermediary |
-| Progress cards | rock consumes `job:next_event()`; renders via event/component machinery |
-| Escape cancels all children | rock tracks its live jobs; a host cancellation event (or the turn-teardown hook that fires it) calls `job:request_cancel()` on each — verify the existing bus exposes interrupt; if not, add a generic turn-lifecycle event |
-
-Two rows need small verifications during implementation (marked above):
-turn-lifecycle/interrupt visibility to extensions, and whether the component
-machinery can host the live panel. Both resolutions must stay feature-neutral.
-
-## Deletions
-
-pantograph: `subagent_host.zig`, `subagent_runtime.zig`, `tui_subagents.zig`,
-the spawn/await thunks and job userdata in `lua_bridge.zig`, the subagent
-wiring in `main.zig`/`tui_app.zig`, and `subagent_integration_tests.zig` in
-its current form (see validation). The executor, lane machinery, and
-`lua_provider` lane calls remain as internal runtime correctness — reviewed
-afterward for members only the deleted code used.
-
-## Validation
-
-- The v0 end-to-end proof must survive: a scripted extension-protocol fixture
- driving the real binary with the real rock — parallel `subagents.run` +
- unknown-agent error in one batch, child resume with history, workflow
- diamond, sandbox fan-out, catalog round-trip, child JSONL
- isolation/manifest/metadata asserted from bytes on disk. (Reconstruct the
- harness from this list; treat it as the primary gate.)
-- Rock suite (`mise run check`) extended to cover the migrated policy:
- gating/bound behavior, resume-default extraction, one-shot capture via
- unresolved tool calls, tool filtering, manifest seeding.
-- pantograph integration tests rewritten against the *generic* surfaces
- (async jobs driving concurrent agents, batch isolation, lane agent
- resolution, cancellation) with no subagent vocabulary.
-- `rg -i subagent /Users/travis/Code/pantograph/src` → empty.
-- All suites green: `mise exec -- zig build test`, `mise run check`.
-
-## Delivery order
-
-1. libpanto-lua: `run_async` jobs + fd contract + `dispatch_tools` knob, with
- binding-level tests (fixture provider, no panto involvement).
-2. libpanto-lua: message-metadata access; agent tool get/set + `tool_choice`.
-3. pantograph: `resolve_model` opaque config; verify interrupt/lifecycle
- event visibility (add the generic event if missing).
-4. Rock: port spawn/await/bound/policy onto the new APIs behind the same tool
- surfaces; port progress to event/component rendering.
-5. Delete the pantograph subagent files and thunks; rewrite integration tests
- generically; run the full validation list.
-6. Update DESIGN.md's seam sections to describe the final division.
-
-## Interop with the panto-on-libuv project (separate; order-independent)
-
-The `wake_fd` contract works identically in both worlds: today the rock arms
-it with `uv.new_poll` on luv's loop; after the libuv unification it is the
-same call on the shared process loop. If the libuv project lands first, the
-executor internals this refactor relies on (posted jobs onto the Lua owner)
-will already use `uv_async` instead of the pipe waker — no change to anything
-specified here. Neither project should block on the other.