1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
|
# panto-subagents refactor: extension-API-first
## Goal
The v0 implementation works end to end, but pantograph absorbed the feature:
`subagent_host.zig`, `subagent_runtime.zig`, and `tui_subagents.zig` are a
subagent backend in core, and the rock is mostly validation and formatting
glue. Per the "Extension features: two responses" rubric in pantograph's
AGENTS.md, this refactor takes response 2 retroactively: the missing *general*
capabilities are added to libpanto-lua and `panto.ext`, subagent policy moves
into this rock, and the feature-shaped core surface is deleted.
Definition of done, in one line: `rg -i subagent pantograph/src` returns
nothing, and every behavior in DESIGN.md still holds through the real binary.
The product surface (DESIGN.md) is unchanged. This is a re-division of the
implementation, not a redesign. DESIGN.md's "Generic pantograph child-job
seam" section is superseded by this document and should be rewritten to match
when this lands.
## What exists today (all currently uncommitted working-tree state)
Generic and staying (invisible runtime correctness, landed with v0):
- The multiplexed Lua executor: tool batches keyed by id instead of a single
`current_batch`, cross-thread posted jobs onto the Lua owner thread
(`lua_executor.zig`, `lua_runtime.zig`).
- Lane identity: `panto.ext.agent` resolves through `coroutine.running()`;
per-open protocol stream state and stream-level `cancel_turn`
(`lua_provider.zig`).
- libpantograph: message metadata survives the JSONL load path;
`UserMessage.metadata` exists on `Agent.run`.
- `panto.ext.models` (bounded catalog queries) and `panto.ext.session_info()`.
Feature-shaped and going: `subagent_host.zig` (vtable), `subagent_runtime.zig`
(spawn transaction, worker threads, `Gate` concurrency bound, metadata
stamping, manifest extraction, resume defaults, one-shot workers, progress
queue), `tui_subagents.zig` (hardcoded progress panel), and the
`spawn_agent`/`await_jobs` thunks in `lua_bridge.zig`.
## New libpanto-lua APIs
These are library bindings, useful to any embedder of libpanto-lua; nothing
about them is subagent-specific.
### Async agent jobs
```lua
local job = agent:run_async {
prompt = "...", -- or blocks, mirroring agent:run
metadata = { ... }, -- optional user-message metadata (JSON-encoded)
dispatch_tools = false, -- optional: end the turn at the first
-- assistant response instead of dispatching
-- its tool calls; they are reported settled
wake_fd = fd, -- optional: a byte is written on every event
-- arrival and on settle
}
job:next_event() -- -> event table | nil (non-blocking; owned copy)
job:result() -- -> settled result | nil while running
job:request_cancel()
job:close() -- join + release; safe after settle or cancel
```
- Implemented with a binding-internal pump thread per job: it drives the
blocking `stream.next()`, copies each event payload (payloads die at the
next pull), buffers under a mutex, and writes one byte to `wake_fd`.
- The fd contract is deliberately loop-agnostic: a luv consumer arms
`uv.new_poll(fd)`; a Go or C embedder selects on it. No loop dependency
enters libpanto-lua.
- Tool dispatch inside the turn is unchanged libpantograph behavior; when the
agent's tools include panto's Lua tool source, its `invoke_batch` posts to
the Lua owner thread through the existing executor — that path already
exists and does not change.
- `dispatch_tools = false` is the generic knob that makes structured one-shot
workers a pure consumer feature: the settled result carries the assistant's
tool calls (name + input JSON) unresolved, and no second model round runs.
### Conversation message metadata
```lua
conv:message_metadata(i) -- -> table | nil (decoded JSON)
conv:set_message_metadata(i, tbl) -- JSON-encodes; nil clears
```
Plus optional `metadata` on `conv:add_system_message`/`add_user_message`.
The disk format already round-trips metadata; this is access only.
### Agent tool control
```lua
agent:tools() -- -> array of decl tables { name, description, schema },
-- a mutable copy
agent:set_tools(decls) -- replace this agent's tool list with these decls;
-- handlers resolve by name through the owning source
```
Copying between agents works because dispatch is name-keyed on the shared
runtime. `agent:set_config` additionally accepts `tool_choice` (force a named
tool). This retires the Zig-side `toolSourceExcluding`.
## New/changed pantograph surfaces
- `panto.ext.resolve_model { model = "provider:alias", reasoning = label }`
→ an **opaque config userdata** accepted by `panto.agent { config = ... }`
and `agent:set_config`. Resolution and validation reuse the existing
registry/reasoning pipeline and fail before inference; credentials are
embedded host-side and never readable from Lua. This is the one place the
credential boundary requires a host API rather than a binding API.
- `panto.ext.models` and `panto.ext.session_info()` remain as-is (already
generic).
- `spawn_agent`, `await_jobs`, the job-handle userdata, and the
`SubagentHost` vtable are deleted once the rock is ported. Awaiting becomes
ordinary luv: the rock arms `uv.new_poll` on each job's `wake_fd` and its
tool-handler coroutine yields/resumes through the standard async-tool
contract.
- Progress rendering: `tui_subagents.zig` is deleted. The rock pulls its own
job events and surfaces activity through the existing extension event and
component machinery (`panto.ext.on`/`emit` + component swap). If that
machinery proves insufficient for a live multi-card panel, the fallback is
one small *generic* host API (a "live status card" keyed by an id — second
consumer: any long-running extension activity), to be co-designed before
building, per AGENTS.md.
## Policy migration map (the closure check)
Every deleted Zig behavior must have a Lua expression using only the APIs
above. This table is the completeness test for the API set:
| Behavior (today in Zig) | New home in the rock |
|------------------------------------------|----------------------|
| Child store layout `<sd>/subagents/<id>` | string assembly (already rock-side) |
| Store create/resolve, ownership boundary | `panto.file_system_jsonl_store(dir)` + `store:resolve/load` |
| Manifest on profile system message | `conv:set_message_metadata` at seeding |
| Per-turn model/reasoning stamp | `run_async { metadata = ... }` |
| Resume defaults from last user message | `store:load` + `conv:message_metadata` scan |
| Primary system context for new children | copy system messages from `panto.ext.agent:conversation()` |
| Concurrency bound (4) + queue + cancel-while-queued | rock-side gate: start ≤4 jobs, queue the rest, drop queued on cancel (~15 lines of Lua) |
| `subagents.*` exclusion (no recursion) | `agent:set_tools(filtered)` from the primary's `agent:tools()` |
| One-shot structured workers | new agent + single synthetic tool + `tool_choice` + `dispatch_tools = false` + `panto.null_store()`; validate with the existing jsonschema path |
| `resumable` determination | fresh `store:resolve(id)` after failure |
| Result shaping (id/agent/status/output) | already rock-side; loses the host `JobResult` intermediary |
| Progress cards | rock consumes `job:next_event()`; renders via event/component machinery |
| Escape cancels all children | rock tracks its live jobs; a host cancellation event (or the turn-teardown hook that fires it) calls `job:request_cancel()` on each — verify the existing bus exposes interrupt; if not, add a generic turn-lifecycle event |
Two rows need small verifications during implementation (marked above):
turn-lifecycle/interrupt visibility to extensions, and whether the component
machinery can host the live panel. Both resolutions must stay feature-neutral.
## Deletions
pantograph: `subagent_host.zig`, `subagent_runtime.zig`, `tui_subagents.zig`,
the spawn/await thunks and job userdata in `lua_bridge.zig`, the subagent
wiring in `main.zig`/`tui_app.zig`, and `subagent_integration_tests.zig` in
its current form (see validation). The executor, lane machinery, and
`lua_provider` lane calls remain as internal runtime correctness — reviewed
afterward for members only the deleted code used.
## Validation
- The v0 end-to-end proof must survive: a scripted extension-protocol fixture
driving the real binary with the real rock — parallel `subagents.run` +
unknown-agent error in one batch, child resume with history, workflow
diamond, sandbox fan-out, catalog round-trip, child JSONL
isolation/manifest/metadata asserted from bytes on disk. (Reconstruct the
harness from this list; treat it as the primary gate.)
- Rock suite (`mise run check`) extended to cover the migrated policy:
gating/bound behavior, resume-default extraction, one-shot capture via
unresolved tool calls, tool filtering, manifest seeding.
- pantograph integration tests rewritten against the *generic* surfaces
(async jobs driving concurrent agents, batch isolation, lane agent
resolution, cancellation) with no subagent vocabulary.
- `rg -i subagent /Users/travis/Code/pantograph/src` → empty.
- All suites green: `mise exec -- zig build test`, `mise run check`.
## Delivery order
1. libpanto-lua: `run_async` jobs + fd contract + `dispatch_tools` knob, with
binding-level tests (fixture provider, no panto involvement).
2. libpanto-lua: message-metadata access; agent tool get/set + `tool_choice`.
3. pantograph: `resolve_model` opaque config; verify interrupt/lifecycle
event visibility (add the generic event if missing).
4. Rock: port spawn/await/bound/policy onto the new APIs behind the same tool
surfaces; port progress to event/component rendering.
5. Delete the pantograph subagent files and thunks; rewrite integration tests
generically; run the full validation list.
6. Update DESIGN.md's seam sections to describe the final division.
## Interop with the panto-on-libuv project (separate; order-independent)
The `wake_fd` contract works identically in both worlds: today the rock arms
it with `uv.new_poll` on luv's loop; after the libuv unification it is the
same call on the shared process loop. If the libuv project lands first, the
executor internals this refactor relies on (posted jobs onto the Lua owner)
will already use `uv_async` instead of the pipe waker — no change to anything
specified here. Neither project should block on the other.
|
