From 003908344336cc13e74618291aa9f3af137f030f Mon Sep 17 00:00:00 2001 From: t Date: Fri, 5 Jun 2026 10:38:54 -0600 Subject: docs * note on the claude subscription provider plan that this is an extension, not panto core, not to be published * archive that old "overview" doc * new doc: pluggable session stores * new doc: libpanto C wrappers, FFI wrappers for python and go --- docs/anthropic-subscription-provider.md | 4 +- docs/archive/overview.md | 103 ++++++++++ docs/libpanto-bindings.md | 323 ++++++++++++++++++++++++++++++++ docs/overview.md | 103 ---------- docs/pluggable-session-store.md | 236 +++++++++++++++++++++++ 5 files changed, 665 insertions(+), 104 deletions(-) create mode 100644 docs/archive/overview.md create mode 100644 docs/libpanto-bindings.md delete mode 100644 docs/overview.md create mode 100644 docs/pluggable-session-store.md diff --git a/docs/anthropic-subscription-provider.md b/docs/anthropic-subscription-provider.md index c6d44eb..941e33d 100644 --- a/docs/anthropic-subscription-provider.md +++ b/docs/anthropic-subscription-provider.md @@ -1,12 +1,14 @@ # Anthropic subscription provider sketch +**This is implemented as a Lua extension, not built into pantograph's core.** Everything described here — the provider, PTY management, MCP tool bridge, and hook harness — lives in a Lua extension that pantograph loads. Pantograph's core has no knowledge of Claude Code, subscription auth, or any of the mechanics below. The extension uses only the public extension/provider API surface that any third-party extension could use. + Goal: expose a pantograph provider backed by an interactive Claude Code subscription session, without using Anthropic API keys, `claude -p`, or Agent SDK metering. ## Core idea Run `claude` as an interactive TUI process under a PTY, but make it behave like a minimal local model provider: -- pantograph owns the outer UX and provider API. +- pantograph (via the Lua extension) owns the outer UX and provider API. - Claude Code provides subscription-backed model turns. - Claude built-in tools are disabled. - pantograph tools are exposed to Claude Code through MCP. diff --git a/docs/archive/overview.md b/docs/archive/overview.md new file mode 100644 index 0000000..916e904 --- /dev/null +++ b/docs/archive/overview.md @@ -0,0 +1,103 @@ +# pantograph — Overview + +`pantograph` is a minimal coding agent built for performance, efficiency, correctness, and a small core that can be extended deliberately. + +## Ethos + +**Batteries optional.** A full-featured coding agent experience ships by default — but everything can be deactivated. The standard distribution includes a curated set of coding-oriented tools and settings, all of which can be turned off. Strip out every coding tool and `pantograph` becomes a general-purpose LLM chat client. Additional capabilities may ship in the base but remain deactivated by default, waiting to be opted into. Nothing is mandatory; everything is intentional. + +**Small core, deliberate extension.** The core runtime does as little as possible. Features that would be built-ins in other agents are extensions in `pantograph` — including the fundamental tools like `read`, `write`, `edit`, and `bash`. The extension system is the primary mechanism for adding capability. + +**Conservative provider support.** Provider integrations are careful and complete rather than broad and broken. `pantograph` supports Anthropic-shaped and OpenAI-shaped APIs with arbitrary base URLs. A provider integration that partially works is worse than no integration at all. + +**Own your data model.** `pantograph` defines its own internal conversation representation and maps to/from provider wire formats. No provider's API shape is treated as canonical. This ensures that adding a new provider never requires contorting the core model. + +**Lean on the terminal.** The TUI does not try to be a full application framework. Scrollback, selection, and search are handled by the surrounding terminal (ghostty, tmux, etc.). The TUI's job is to present output clearly and offer targeted enhancements — like expanding or collapsing tool-call blocks — by clearing and re-rendering its own output region. This keeps the TUI simple while still providing a much nicer experience than a raw CLI. + +## Architecture + +### Data model + +The conversation model is provider-agnostic. It uses a flat message list where each message has a role and a list of typed content blocks: + +``` +Conversation = ordered list of Messages +Message = { role: system | user | assistant, content: []ContentBlock } +ContentBlock = Text | Thinking | ToolUse | ToolResult +``` + +- `Text` and `Thinking` use a shared `TextualBlock` type that grows incrementally via an internal `ArrayList(u8)` — amortized O(1) appends during streaming, no O(n²) re-copying. +- `ToolUse` and `ToolResult` arrive complete (not streamed incrementally) and store their data as owned byte slices. +- System messages may contain multiple `Text` blocks, which are concatenated when a provider expects a single system prompt string (e.g., Anthropic). + +### Library structure + +`pantograph` is a library first. The core agent functionality lives in `libpanto`, a Zig module. The CLI is a thin consumer of the library. A C ABI build of `libpanto` will be produced when the extension system needs it (for Lua interop), implemented as thin `export fn` wrappers around the Zig API. + +### Provider abstraction + +Providers implement a streaming interface: given a conversation, stream a response message back via a Receiver (callback-based). The Receiver delivers incremental content deltas for real-time display and a complete assembled message when the stream ends. Adding a new provider means implementing this interface and writing the serialization for the provider's wire format. + +### Extension system + +Extensions will initially be written in Lua, requiring a C ABI surface on `libpanto`. Future support for shared-object extensions (Zig, Rust, C, C++) will use the same C ABI. Core tools like `read`, `write`, `edit`, and `bash` are extensions — individually disableable, included in the standard distribution but not hardcoded into the runtime. + +### Server/proxy mode + +In a future phase, `pantograph` will be able to run as a server exposing OpenAI-compatible and Anthropic-compatible APIs, acting as a lightweight provider router/proxy to its configured backends. This is not yet planned in detail. + +## Phase Roadmap + +| Phase | Deliverable | Doc | +|-------|-------------|-----| +| 1 | libpanto — minimal chat library, OpenAI provider, streaming, minimal CLI | [phase-1.md](phase-1.md) | +| 2 | Anthropic provider — second provider, validates the abstraction | phase-2.md | +| 3 | Extension API — Lua runtime, extension loading, tool registration | phase-3.md | +| 4 | Conversation serialization — JSONL event log, session save/resume, crash recovery | [phase-4.md](phase-4.md) | +| 5 | Core tools — read/write/edit/bash as distributable extensions | phase-5.md | +| 6 | Rounded coding agent — slash commands, TOML config, extended TUI | phase-6.md | + +### Phase 1: libpanto + +A Zig library that holds a streaming conversation with an LLM via an OpenAI-compatible API. No tools, no extensions — just chat. Ships a minimal CLI (`panto` binary) for live testing: readline, send, print streamed response, repeat. The conversation model is established with all four ContentBlock variants defined (ToolUse and ToolResult exist in the type but are never produced in this phase). + +### Phase 2: Anthropic Provider + +A second provider implementation targeting Anthropic's API shape. Validates that the Provider abstraction and internal conversation model are genuinely provider-agnostic — not just OpenAI in disguise. This is a focused phase: if the abstraction is right, it's mostly serialization work. If it's wrong, we find out here rather than later. + +### Phase 3: Extension API + +Introduces a Lua extension runtime and the extension loading mechanism. Extensions can register tools, access configuration, and participate in the agent loop. This phase also produces the C ABI build of `libpanto` needed for Lua interop. Tools exist but none ship yet — the extension system is the deliverable, not the tools. + +### Phase 4: Conversation Serialization + +Save and resume conversations. Sessions are stored as append-only JSONL event logs — recording every message and model change — and fully rebuilt from the log on resume. Disk persistence so a coding agent can survive restarts and be reviewed later. See [phase-4.md](phase-4.md). + +### Phase 5: Core Tools as Extensions + +The fundamental coding tools — `read`, `write`, `edit`, `bash` — are implemented as extensions (initially Lua, eventually native). They live under the `std` namespace: `std.read`, `std.write`, `std.edit`, `std.bash`. The `std` package is a curated set of coding-oriented extensions — some enabled by default, some available but deactivated — embodying the "batteries optional" ethos. They ship with the standard distribution but are individually disableable. This is where `pantograph` becomes a functional coding agent rather than just a chat client. + +### Phase 6: Rounded Coding Agent + +Polish and capstone features that make `pantograph` a well-rounded coding agent experience: + +- **Slash commands** — an extensible framework for `/`-prefixed commands (e.g., `/help`, `/model`, `/clear`) +- **TOML configuration** — a config file for persistent settings (default model, enabled/disabled extensions, provider configs, system prompt templates) +- **Extended TUI** — smarter output rendering while remaining lightweight: expand/collapse tool call blocks (via clear-and-reprint), structured display of thinking content, prompt decoration +- **Compaction with custom compaction prompts** — LLM-based context pruning for long conversations. Older messages are summarized to free context window space. The `compaction` entry type in the event log records the summary and a reference to the first kept message, so the log remains append-only and the full history is never destroyed — only omitted from the LLM context. Extension hooks allow custom compaction prompts, so users can guide how their history is summarized (e.g., preserving details about a specific task, emphasizing code changes over conversation). + +## Future (Unplanned) + +These are recorded from the initial ideas but do not yet have phase documents or detailed plans: + +- **Server/proxy mode** — run `pantograph` as a server exposing OpenAI-compatible and Anthropic-compatible APIs, routing to configured backends +- **Shared-object extensions** — extend the extension system beyond Lua to support native shared libraries via the C ABI (Zig, Rust, C, C++) +- **System prompt construction framework** — opinionated system for assembling system prompts from composable parts (templates, project context, extension contributions) +- **Google API provider** — native integration with Google's Gemini API (rather than their OpenAI-compatibility layer), unlocking richer capabilities specific to that API shape. Low priority compared to Anthropic and OpenAI support. +- **C ABI distribution of libpanto** — `export fn` wrappers exposing libpanto functionality through a C calling convention, enabling external programs to embed or build on pantograph from C, Rust, or other native languages. Not a separate library — the C ABI is a second interface on the same `libpanto` artifact, compiled from `export fn` shims that translate between Zig types and C types. Needed eventually for shared-object extensions (Zig, Rust, C, C++) beyond Lua. + +## Punted + +Deliberate decisions to defer functionality that came up during phase planning but doesn't fit cleanly into the existing phase roadmap. Each one is captured here with enough context to pick up later. + +- **Tool-call cancellation / timeout via process isolation.** First raised in phase 3. There is no clean POSIX mechanism for cancelling a thread mid-execution with a guarantee of no further side effects — `pthread_kill` with SIGKILL terminates the entire process, `pthread_cancel` is widely considered unusable, and signal-based interruption can't safely unwind arbitrary code. Lua's `lua_sethook` provides cooperative cancellation between VM steps but doesn't interrupt handlers blocked in C calls (filesystem, subprocess, network). The mechanism that actually works is **process isolation**: run tool invocations in a helper subprocess, SIGKILL the subprocess on timeout. The intended approach is `fork+exec` into a small `panto-tool-worker` binary that statically links libpanto's extension machinery. libpanto would define the wire protocol (tool name + input bytes over a pipe, result bytes back) and own the fork/exec/timeout/kill orchestration; the embedder supplies the helper binary path. This makes sandboxing a library-level capability available to all embedders, not just the panto CLI. Open design questions when this is picked up: extension loading strategy in the worker (rescan-per-fork vs long-lived worker pool), file descriptor inheritance policy, working directory and environment handling, and how registry contents are communicated to the worker. Until this lands, tool handlers run to completion in-process — if a tool hangs, the user kills `panto`. diff --git a/docs/libpanto-bindings.md b/docs/libpanto-bindings.md new file mode 100644 index 0000000..9b9ef2c --- /dev/null +++ b/docs/libpanto-bindings.md @@ -0,0 +1,323 @@ +# Plan: `libpanto` language bindings (Go + Python) + +## Goal + +Expose `libpanto` to other languages, targeting **Go** and **Python** first. +The bindings are organized as a family of sibling packages, named uniformly: + +| package | language | role | +| ------------- | -------- | ------------------------------------------------------------- | +| `libpanto` | Zig | the core library (exists today) | +| `libpanto-c` | Zig | a C-ABI shared library + header wrapping the public Zig API | +| `libpanto-go` | Go | idiomatic Go bindings over `libpanto-c` via cgo | +| `libpanto-py` | Zig | a CPython extension implemented in pure Zig (`@cImport`) | + +Two consumers, two paths to the core: + +- **Go → `libpanto-c` → `libpanto`.** cgo is effectively the only option, and + cgo can only call C. So a C ABI is mandatory. +- **Python → `libpanto`** directly. `libpanto-py` is a native CPython + extension written *in Zig*: it `@cImport`s `Python.h`, builds `PyObject` + glue against the translated C types, and `build.zig` emits the loadable + `.so`. It calls the Zig API directly and **does not** depend on `libpanto-c`. + +### Why this split (decisions settled) + +- **`libpanto-c` is built regardless**, because Go requires it. It doubles as + the stable-ABI validation platform for the C surface. +- **No cffi.** A pure-Python cffi binding was considered and rejected. Its only + real appeal (no native build step, trivial wheels) holds solidly *only* in + ABI mode, which is also where streaming callbacks and silent struct/ABI + drift bite hardest; API mode reintroduces the C-compile/per-platform-wheel + cost without the control of a real native module. Either way it would be a + throwaway second validator of the same C surface that `libpanto-go` already + exercises. Net: redundant. We'd reimplement it natively on the soon side. +- **`libpanto-py` is pure Zig, not C-over-`libpanto-c`.** The decisive win is + that a Zig→`.so` compilation involves **no C translation units in the repo at + all** — one toolchain, header translation done at compile time by `@cImport`. + Going straight to the Zig API (rather than through `libpanto-c`) also skips a + redundant marshalling layer. + +## The core architectural decision: ship a *pull* streaming API + +This is the spine of the whole project, and it requires an **internal refactor +of `libpanto` first** (Phase 0). Everything else wraps it. + +### Why pull, not push + +`libpanto` today streams via a **push** `Receiver` vtable +(`provider.zig`): the provider loop calls `onMessageStart`, `onContentDelta`, +`onBlockComplete`, `onMessageComplete`, etc. Push is the wrong primitive to +*export*, for two independent reasons that land on the same conclusion: + +1. **It doesn't compose into the idiomatic surfaces we want.** + - Go: the idiomatic streaming shape is a single goroutine driving a + range-over-func iterator (`for ev := range stream.Iter`) and/or a channel. + cgo callbacks *into* Go are slow and awkward (`//export`, pointer rules, + no closures). A goroutine draining a pull API is the clean build. + - Python: a (sync or async) generator **is** a pull construct — + `__next__` *is* "give me the next event." A push callback can't `yield`; + to turn pushed values into a `for ev in stream`, you must run the producer + and consumer in different stack frames concurrently (a thread + queue). + So push → generator *forces* a thread+queue regardless. Pull → generator + is a 1:1 mapping with zero adaptation. + +2. **Pull is the more primitive primitive.** Push composes trivially on top of + pull (`while (try s.next()) |ev| receiver.onEvent(ev);` — five lines). + Pull does **not** compose cheaply on top of push: to expose `next()` over a + `Receiver`-driven core you must suspend *inside* a callback and resume the + provider loop later, which means a thread+queue or hand-rolled coroutine + state — i.e. you rebuild pull anyway, the hard way. + +**Conclusion:** ship only the pull API. `Receiver` leaves the public surface +entirely. Anyone who wants push writes the five-line wrapper themselves. + +### The Zig interface + +```zig +// run() drops the receiver parameter entirely and returns a Stream. +// Whether it also drops `conv` depends on sequencing against +// docs/pluggable-session-store.md (see note below) — either form is fine. +pub fn run(self: *Agent) !Stream // or: run(self: *Agent, conv: *Conversation) !Stream + +// Stream is a resumable handle, closely linked to SSEParser + the provider +// read loop. next() yields events until the turn is exhausted. +pub fn next(self: *Stream) !?Event +``` + +`Stream.next()` returns `!?Event`, which gives **three orthogonal channels**: + +| signal | meaning | +| ------------------ | ---------------------------------------------------- | +| `Event` (a value) | streaming progress, including the terminal event | +| `null` | the stream is exhausted (already past the terminal) | +| `error.X` | a genuine failure (network, parse, provider error) | + +### The terminal-event invariant (documented contract) + +> Every `next()` call on a stream **after** it has yielded a `MessageComplete` +> event returns `null`, and `null` is **never** returned before +> `MessageComplete`. + +Consequences, and why this exact shape: + +- **`MessageComplete` is a normal `Event` variant**, not an error and not + signalled by `null`. It is the in-band success terminal. +- **Intelligent consumers never observe `null`.** They stop after *consuming* + `MessageComplete`. `null` exists only as the defensive answer to "you called + `next()` one time too many," distinct from a real `error`. +- Python: emit `MessageComplete`, then raise `StopIteration` on the *next* + call (which sees `null`) — or, more precisely, the generator stops right + after yielding `MessageComplete`, so well-behaved code raises `StopIteration` + without ever materializing a `None`. +- Go: the `Iter` range-over-func stops *after* yielding `MessageComplete`, not + by waiting to observe a `null` sentinel. +- Mid-stream **provider errors surface as a Zig `error` from `next()`** (the + `!` in `!?Event`), not as an `Event.Error` variant. This keeps the `Event` + union success-only and maps cleanly to a raised Python exception and a Go + `error`. (`error.StreamExhausted` is *not* used — exhaustion is `null`, not + an error; we reserve errors for failures.) + +### `Event` spans the whole turn, not one HTTP response + +Verified from the code: `Agent.runStep` wraps **multiple** `streamStep` calls +in a tool-using turn (one provider stream per assistant message), with +concurrent tool dispatch *between* them (`agent.zig`). The `Stream` therefore +spans the entire `run()` loop — provider streaming **and** tool dispatch — not +a single SSE response. The `Event` union must express both layers. + +The exact variant list is to be finalized against what the provider loop and +agent loop emit today (see `ReceiverVTable` in `provider.zig` and the dispatch +path in `agent.zig`), but it covers at least: + +- message lifecycle: `MessageStart`, `MessageComplete` +- block lifecycle: `BlockStart`, `BlockComplete` +- content: `ContentDelta` +- tool identity: `ToolDetails` (id + name) +- tool dispatch boundaries: tool-call start / tool-result (so Go/Python + consumers can observe the agent running tools between provider turns) +- provider retry notices: `ProviderRetry` (informational; today `onProviderRetry`) + +`Event` is an `enum union`; it is the single type every binding marshals into +its native form. Define it once, in `libpanto`. + +## The hard part of Phase 0: making `Stream.next()` resumable + +Verified from `provider_openai_chat.zig`: the current provider loop is a +single-stack-frame loop — + +``` +readVec(body_reader) -> parser.feed(chunk) -> handleEvent(...) -> receiver.*(...) +``` + +— with all state (the HTTP body reader + `transfer_buf`, the `SSEParser`, the +per-response `StreamState`) living **on the stack**. A pull `next()` must +*return* between events, so that state has to move into the `Stream` handle. +Three ways to get there: + +1. **State machine (recommended target).** The `Stream` owns the socket/body + reader, the `SSEParser`, and the decode state; `next()` reads/parses just + enough to produce one `Event` and returns. No threads. Cleanest and most + portable across all four packages. Most upfront refactoring effort, since + the provider loop is inverted into a resumable step function. + +2. **Thread + queue inside the handle.** Keep the existing + `streamStep(receiver)` loop verbatim; run it on a thread whose receiver + pushes events into a bounded queue; `next()` pops. Minimal change to proven + code, but costs a thread per active stream plus shutdown/backpressure care. + A reasonable interim if (1) proves invasive. + +3. **Zig async/suspend.** Language-level coroutines. Availability depends on the + toolchain's async status at the time of implementation; treat as + not-reliably-available without checking first. + +The internal mechanism (1 vs 2) is swappable later **without changing the +exported contract** — the public surface is pull either way. Recommendation: +target (1); fall back to (2) only if the provider-loop inversion is too costly +for v1. + +> Expectation: **Phase 0 is a net-red diff.** Deleting `Receiver` from the +> public API, the `CompactionCapture` no-op receiver, and the CLI's +> `CLIReceiver` vtable plumbing should remove more than the `Stream` machinery +> adds. + +--- + +## Phase 0 — internal refactor of `libpanto` to a pull API + +The real work. Everything after this is wrapping. + +1. **Define the `Event` union** (success-only `enum union`) from the current + `ReceiverVTable` callbacks plus the agent's tool-dispatch boundaries. This + is the type every binding marshals. +2. **Define the `Stream` type**: a resumable handle owning the provider read + loop's state (body reader, `SSEParser`, decode state) and the agent's + tool-dispatch position. `fn next(self: *Stream) !?Event`. This is where the + state-machine-vs-thread decision lands. +3. **`Agent.run() -> Stream`.** With `conversation` owned by the `Agent` (per + `docs/pluggable-session-store.md`) and `receiver` removed, `run()` takes + only `self`. The agent loop (provider stream → tool dispatch → repeat) + becomes the thing `Stream.next()` drives incrementally. +4. **Delete `Receiver` from the public API** (`root.zig`, `provider.zig`). + Internal seams that genuinely want push (if any) get the trivial pull→push + wrapper, in caller code, not in the library surface. +5. **Rebuild the CLI on the pull loop.** `src/main.zig`'s `CLIReceiver` + collapses into a `while (try stream.next()) |ev| render(ev)` loop. Prove the + refactor end-to-end against the existing test suite; CLI behavior unchanged. + +**Exit criteria:** `Receiver` gone from `root.zig`; `Stream.next()` is the only +streaming primitive; CLI output and the full test suite are unchanged. + +> Sequencing with `docs/pluggable-session-store.md` is **flexible**, not a +> dependency. That plan moves `Conversation` onto the `Agent`; this one removes +> `receiver`. The two are orthogonal: +> - If the session-store work lands first, `run()` takes only `self`. +> - If this work lands first, `run(self, conv)` takes a single `conv` argument +> (just `runStep` minus `receiver`), and the session-store refactor drops +> `conv` later. +> +> Either way `receiver` is gone after Phase 0. Don't block on ordering. + +## Phase 1 — `libpanto-c` (C ABI) + +6. **Opaque handles** for the major types: `PantoAgent`, `PantoStream` (and + whatever construction requires — config, conversation/session store). + Consumers never see the layout. +7. **`panto_next_event(stream, *PantoEvent) -> status`** is the C projection of + `!?Event`. The `!?Event` triple maps onto a status enum plus an out-param: + + | Zig `next()` | C status | out-param | + | ----------------------- | ------------ | ---------------- | + | `Event` value | `EVENT` (0) | filled | + | `null` | `DONE` (1) | untouched | + | `error.X` | `ERROR` (2) | error detail | + +8. **`@export` wrappers** for construction, `run()`, `next_event`, teardown, + and a free function for any owned event payloads. +9. **The C header is committed and hand-maintained, not generated by + `build.zig`.** (Open question 1 below: we are *not* having `build.zig` emit + `panto.h`.) A stable, hand-written `include/panto.h` is the ABI contract — + it should change deliberately, be reviewable in diffs, and not be a build + artifact. `build.zig` emits the shared library and stages the committed + header; it does not author it. +10. **Define the event-marshalling C structs once.** `PantoEvent` (a tagged + union mirroring the Zig `Event`) is read by every downstream binding. + +## Phase 2 — `libpanto-go` (validates `libpanto-c`) + +11. **cgo bindings**: `Agent`, `Stream`, and the raw + `Stream.Next() (Event, bool, error)` (or `(Event, error)` with a separate + done signal) mapping `EVENT`/`DONE`/`ERROR` onto Go's idioms. +12. **`Stream.Iter() iter.Seq[Event]`** — the modern range-over-func form, so + consumers write `for ev := range stream.Iter { ... }`. Single goroutine, + auto-terminates after yielding `MessageComplete`, surfaces a failure via a + trailing `stream.Err()` after the range (the idiomatic Go pattern since + `bufio.Scanner`). This is the primary Go surface. +13. **A channel wrapper too.** Go users expect channels; a goroutine drains + `Next()` into a channel. Cheap over a pull core. Ship both the raw + iterator and the channel form. + +`libpanto-go` is the validation harness for the `libpanto-c` ABI — if Go can +drive a full streaming turn idiomatically, the C surface is sound. + +## Phase 3 — `libpanto-py` (pure Zig CPython extension) + +14. **`build.zig` does `@cImport(@cInclude("Python.h"))`** and emits + `_panto.so` (a loadable extension module). No C in this package. Decide + **stable ABI (`abi3` / limited API)** here to collapse the + CPython-version × platform wheel matrix to one artifact per platform. +15. **`module.zig`** implements `PyModuleDef`, `PyMethodDef`, and an `Agent` + type and `Stream` type as `PyTypeObject`s, calling the **Zig** `libpanto` + API directly. The `Stream` type's `tp_iternext` calls Zig `Stream.next()`: + - `Event` → build and return the `PyObject` event. + - `null` → set `StopIteration` (return `NULL` with no error set). + - `error.X` → set the mapped Python exception. + - Wrap the blocking `next()` in `Py_BEGIN_ALLOW_THREADS` / + `Py_END_ALLOW_THREADS` so streaming doesn't serialize the whole + interpreter; re-acquire the GIL to build the event `PyObject`. +16. **`panto/__init__.py`** is a thin Pythonic surface over the native + `_panto`: a real exception hierarchy, context managers, idiomatic event + objects. **The async generator is pure Python** — wrap the sync iterator + with `asyncio.to_thread` (blocking `next()` runs in a worker thread). No + file descriptors, no ABI change. (See "Out of scope for v1" below.) + +## The one contract that unifies all four packages + +| layer | progress / terminal | exhausted | failure | +| -------- | ------------------------- | ------------------ | ---------------------- | +| Zig | `Event` / `MessageComplete` | `null` | `error.X` | +| C | `PantoEvent` / status `EVENT` | status `DONE` | status `ERROR` | +| Go | `Event` (Iter yields) | Iter stops | `error` / `stream.Err()` | +| Python | event object (yielded) | `StopIteration` | raised exception | + +Design every binding to this single table. Pull-shaped, success-only events, +terminal-by-`MessageComplete`, exhaustion-by-`null`, failure-by-error. + +## Out of scope for v1 (deliberately deferred) + +- **A pollable fd / `panto_step_poll(timeout)`.** This is the only thing that + makes async *natively* non-blocking on both Go (`select`) and Python + (`await` on readiness). It is real work in `libpanto-c` and unnecessary for + v1. **Design the C ABI so a pollable fd can be added later without breaking + the existing pull surface**, but do not build it now. +- **A native async Python API.** Without an fd, async Python is *just* the sync + pull API wrapped in `asyncio.to_thread`. That is pure-Python glue in + `panto/__init__.py`; no native or ABI work, so it isn't a binding deliverable. +- **Languages beyond Go and Python.** `libpanto-c` is the reuse point for any + future cgo-style or cffi-style consumer (Ruby, Node N-API, …); none are in + scope now. + +## Open questions / decisions to finalize before coding + +1. **`panto.h` generation — resolved: no.** The header is committed and + hand-maintained as the ABI contract; `build.zig` stages it but does not emit + it. (Captured in Phase 1, step 9.) +2. **Resumable-handle strategy (state machine vs. thread+queue)** — target the + state machine; decide finally once the provider-loop inversion is scoped. +3. **Exact `Event` variants** — finalize against `ReceiverVTable` + (`provider.zig`) and the tool-dispatch path (`agent.zig`). +4. **What moves onto `Agent` vs. stays per-`run()`** — driven by + `docs/pluggable-session-store.md`'s final shape; `conversation` is moving, + confirm nothing else needs to. +5. **CPython stable-ABI (`abi3`) commitment** — decide in Phase 3 to fix the + wheel matrix early. diff --git a/docs/overview.md b/docs/overview.md deleted file mode 100644 index 916e904..0000000 --- a/docs/overview.md +++ /dev/null @@ -1,103 +0,0 @@ -# pantograph — Overview - -`pantograph` is a minimal coding agent built for performance, efficiency, correctness, and a small core that can be extended deliberately. - -## Ethos - -**Batteries optional.** A full-featured coding agent experience ships by default — but everything can be deactivated. The standard distribution includes a curated set of coding-oriented tools and settings, all of which can be turned off. Strip out every coding tool and `pantograph` becomes a general-purpose LLM chat client. Additional capabilities may ship in the base but remain deactivated by default, waiting to be opted into. Nothing is mandatory; everything is intentional. - -**Small core, deliberate extension.** The core runtime does as little as possible. Features that would be built-ins in other agents are extensions in `pantograph` — including the fundamental tools like `read`, `write`, `edit`, and `bash`. The extension system is the primary mechanism for adding capability. - -**Conservative provider support.** Provider integrations are careful and complete rather than broad and broken. `pantograph` supports Anthropic-shaped and OpenAI-shaped APIs with arbitrary base URLs. A provider integration that partially works is worse than no integration at all. - -**Own your data model.** `pantograph` defines its own internal conversation representation and maps to/from provider wire formats. No provider's API shape is treated as canonical. This ensures that adding a new provider never requires contorting the core model. - -**Lean on the terminal.** The TUI does not try to be a full application framework. Scrollback, selection, and search are handled by the surrounding terminal (ghostty, tmux, etc.). The TUI's job is to present output clearly and offer targeted enhancements — like expanding or collapsing tool-call blocks — by clearing and re-rendering its own output region. This keeps the TUI simple while still providing a much nicer experience than a raw CLI. - -## Architecture - -### Data model - -The conversation model is provider-agnostic. It uses a flat message list where each message has a role and a list of typed content blocks: - -``` -Conversation = ordered list of Messages -Message = { role: system | user | assistant, content: []ContentBlock } -ContentBlock = Text | Thinking | ToolUse | ToolResult -``` - -- `Text` and `Thinking` use a shared `TextualBlock` type that grows incrementally via an internal `ArrayList(u8)` — amortized O(1) appends during streaming, no O(n²) re-copying. -- `ToolUse` and `ToolResult` arrive complete (not streamed incrementally) and store their data as owned byte slices. -- System messages may contain multiple `Text` blocks, which are concatenated when a provider expects a single system prompt string (e.g., Anthropic). - -### Library structure - -`pantograph` is a library first. The core agent functionality lives in `libpanto`, a Zig module. The CLI is a thin consumer of the library. A C ABI build of `libpanto` will be produced when the extension system needs it (for Lua interop), implemented as thin `export fn` wrappers around the Zig API. - -### Provider abstraction - -Providers implement a streaming interface: given a conversation, stream a response message back via a Receiver (callback-based). The Receiver delivers incremental content deltas for real-time display and a complete assembled message when the stream ends. Adding a new provider means implementing this interface and writing the serialization for the provider's wire format. - -### Extension system - -Extensions will initially be written in Lua, requiring a C ABI surface on `libpanto`. Future support for shared-object extensions (Zig, Rust, C, C++) will use the same C ABI. Core tools like `read`, `write`, `edit`, and `bash` are extensions — individually disableable, included in the standard distribution but not hardcoded into the runtime. - -### Server/proxy mode - -In a future phase, `pantograph` will be able to run as a server exposing OpenAI-compatible and Anthropic-compatible APIs, acting as a lightweight provider router/proxy to its configured backends. This is not yet planned in detail. - -## Phase Roadmap - -| Phase | Deliverable | Doc | -|-------|-------------|-----| -| 1 | libpanto — minimal chat library, OpenAI provider, streaming, minimal CLI | [phase-1.md](phase-1.md) | -| 2 | Anthropic provider — second provider, validates the abstraction | phase-2.md | -| 3 | Extension API — Lua runtime, extension loading, tool registration | phase-3.md | -| 4 | Conversation serialization — JSONL event log, session save/resume, crash recovery | [phase-4.md](phase-4.md) | -| 5 | Core tools — read/write/edit/bash as distributable extensions | phase-5.md | -| 6 | Rounded coding agent — slash commands, TOML config, extended TUI | phase-6.md | - -### Phase 1: libpanto - -A Zig library that holds a streaming conversation with an LLM via an OpenAI-compatible API. No tools, no extensions — just chat. Ships a minimal CLI (`panto` binary) for live testing: readline, send, print streamed response, repeat. The conversation model is established with all four ContentBlock variants defined (ToolUse and ToolResult exist in the type but are never produced in this phase). - -### Phase 2: Anthropic Provider - -A second provider implementation targeting Anthropic's API shape. Validates that the Provider abstraction and internal conversation model are genuinely provider-agnostic — not just OpenAI in disguise. This is a focused phase: if the abstraction is right, it's mostly serialization work. If it's wrong, we find out here rather than later. - -### Phase 3: Extension API - -Introduces a Lua extension runtime and the extension loading mechanism. Extensions can register tools, access configuration, and participate in the agent loop. This phase also produces the C ABI build of `libpanto` needed for Lua interop. Tools exist but none ship yet — the extension system is the deliverable, not the tools. - -### Phase 4: Conversation Serialization - -Save and resume conversations. Sessions are stored as append-only JSONL event logs — recording every message and model change — and fully rebuilt from the log on resume. Disk persistence so a coding agent can survive restarts and be reviewed later. See [phase-4.md](phase-4.md). - -### Phase 5: Core Tools as Extensions - -The fundamental coding tools — `read`, `write`, `edit`, `bash` — are implemented as extensions (initially Lua, eventually native). They live under the `std` namespace: `std.read`, `std.write`, `std.edit`, `std.bash`. The `std` package is a curated set of coding-oriented extensions — some enabled by default, some available but deactivated — embodying the "batteries optional" ethos. They ship with the standard distribution but are individually disableable. This is where `pantograph` becomes a functional coding agent rather than just a chat client. - -### Phase 6: Rounded Coding Agent - -Polish and capstone features that make `pantograph` a well-rounded coding agent experience: - -- **Slash commands** — an extensible framework for `/`-prefixed commands (e.g., `/help`, `/model`, `/clear`) -- **TOML configuration** — a config file for persistent settings (default model, enabled/disabled extensions, provider configs, system prompt templates) -- **Extended TUI** — smarter output rendering while remaining lightweight: expand/collapse tool call blocks (via clear-and-reprint), structured display of thinking content, prompt decoration -- **Compaction with custom compaction prompts** — LLM-based context pruning for long conversations. Older messages are summarized to free context window space. The `compaction` entry type in the event log records the summary and a reference to the first kept message, so the log remains append-only and the full history is never destroyed — only omitted from the LLM context. Extension hooks allow custom compaction prompts, so users can guide how their history is summarized (e.g., preserving details about a specific task, emphasizing code changes over conversation). - -## Future (Unplanned) - -These are recorded from the initial ideas but do not yet have phase documents or detailed plans: - -- **Server/proxy mode** — run `pantograph` as a server exposing OpenAI-compatible and Anthropic-compatible APIs, routing to configured backends -- **Shared-object extensions** — extend the extension system beyond Lua to support native shared libraries via the C ABI (Zig, Rust, C, C++) -- **System prompt construction framework** — opinionated system for assembling system prompts from composable parts (templates, project context, extension contributions) -- **Google API provider** — native integration with Google's Gemini API (rather than their OpenAI-compatibility layer), unlocking richer capabilities specific to that API shape. Low priority compared to Anthropic and OpenAI support. -- **C ABI distribution of libpanto** — `export fn` wrappers exposing libpanto functionality through a C calling convention, enabling external programs to embed or build on pantograph from C, Rust, or other native languages. Not a separate library — the C ABI is a second interface on the same `libpanto` artifact, compiled from `export fn` shims that translate between Zig types and C types. Needed eventually for shared-object extensions (Zig, Rust, C, C++) beyond Lua. - -## Punted - -Deliberate decisions to defer functionality that came up during phase planning but doesn't fit cleanly into the existing phase roadmap. Each one is captured here with enough context to pick up later. - -- **Tool-call cancellation / timeout via process isolation.** First raised in phase 3. There is no clean POSIX mechanism for cancelling a thread mid-execution with a guarantee of no further side effects — `pthread_kill` with SIGKILL terminates the entire process, `pthread_cancel` is widely considered unusable, and signal-based interruption can't safely unwind arbitrary code. Lua's `lua_sethook` provides cooperative cancellation between VM steps but doesn't interrupt handlers blocked in C calls (filesystem, subprocess, network). The mechanism that actually works is **process isolation**: run tool invocations in a helper subprocess, SIGKILL the subprocess on timeout. The intended approach is `fork+exec` into a small `panto-tool-worker` binary that statically links libpanto's extension machinery. libpanto would define the wire protocol (tool name + input bytes over a pipe, result bytes back) and own the fork/exec/timeout/kill orchestration; the embedder supplies the helper binary path. This makes sandboxing a library-level capability available to all embedders, not just the panto CLI. Open design questions when this is picked up: extension loading strategy in the worker (rescan-per-fork vs long-lived worker pool), file descriptor inheritance policy, working directory and environment handling, and how registry contents are communicated to the worker. Until this lands, tool handlers run to completion in-process — if a tool hangs, the user kills `panto`. diff --git a/docs/pluggable-session-store.md b/docs/pluggable-session-store.md new file mode 100644 index 0000000..9b45864 --- /dev/null +++ b/docs/pluggable-session-store.md @@ -0,0 +1,236 @@ +# Plan: Agent-owned, pluggable session persistence + +## Problem + +Session logging is implemented as a single concrete type, `SessionManager` +(`libpanto/src/session_manager.zig`), that hard-codes filesystem JSONL I/O, +the `.jsonl` filename scheme, and the directory layout. There is no +interface seam: a different `libpanto` consumer (e.g. a web service backed by +Postgres) cannot swap in its own persistence. + +Worse, persistence is currently driven *entirely outside* `libpanto`. The +`panto` CLI translates in-memory `Conversation` messages into on-disk +`DiskMessage`s (`src/session_persist.zig`), supplies metadata +(provider/model/`stop_reason`/`Usage`), and sequences the append calls around +`agent.runStep`. The `Agent` never touches the session log. + +## Goal + +`libpanto` consumers get persistence **for free**. The `Agent` owns the live +`Conversation` and a `SessionStore`, and persists everything it generates as +turns progress. `libpanto` ships two backends: `FSJSONLStore` (today's +behavior, constructed with a directory) and `NullStore` (no-op, for embedders +who opt out). The `panto` CLI resolves the directory, constructs an +`FSJSONLStore`, and hands it to the `Agent`. + +## Design decisions (settled) + +1. **`Agent` centralizes.** The `Agent` owns its `Conversation` (as + `agent.conversation`) and its `SessionStore`. Turn-driving methods stop + taking a `*Conversation` parameter and operate on the owned conversation. + Fewer, more powerful types on the API surface. `Conversation` and + `SessionStore` are sub-components of the `Agent`'s responsibility. + +2. **Persistence binds to conversation mutation, not to `runStep`.** Every + message that enters the conversation persists when added: + - `agent.submitUserMessage(text)` — adds + persists a user entry, stamped + with `config.provider`/`model`. The user prompt is durably logged + *immediately on submission*, before any provider call. + - `agent.addSystemMessage(text, mode)` — adds + persists a system entry. + - `agent.runStep(receiver)` — persists assistant + tool-result messages it + generates internally. + - `agent.compact(...)` — persists its own result. + + The embedder never calls a persist function. + +3. **The store has a read side, on the interface.** A `SessionStore` must be + able to reconstruct a `Conversation` (a single linear chain). Required on + the interface — a store you can't read is not a valid store. Later `/tree` + functionality (out of scope here) will use the read side to enumerate many + `Conversation`s (one per leaf / per unique prefix); the current + `ArrayList(Message)` shape supports that naturally. + +4. **`Conversation` is always a single linear chain.** Session branching + (future) lives in the store's tree of entries, not in `Conversation`. The + read side flattens a chosen path into one `Conversation`. + +5. **Resume returns `{ Conversation, ?dangling_user }`.** The read side + returns the reconstructed conversation plus an optional dangling trailing + user-prompt string (a user entry with no following assistant — e.g. a + crash/quit right after submission). The reconstructed `Conversation` + **excludes** that dangling user turn, so a resumed agent never auto-sends + it. Today's `panto` throws the optional string away. A future TUI will + prefill it for editing — which becomes the first real **session branch** + (an edited resubmission is a tree sibling sharing the original's + `parent_id`). Do not build branching now; just shape the return type so the + dangling prompt is discoverable. + +6. **`Agent.init` takes an optional `Conversation` to adopt.** `null` → fresh + empty conversation. Non-null → adopt (take ownership) as the live + conversation. Resume is an embedder choice expressed through types: open the + store, ask it for the conversation, hand it to `Agent.init`. No hidden + "replay mode" inside the agent. + +7. **Naming:** `FSJSONLStore`, `SessionStore`, `NullStore` (CamelCase acronyms + fully capitalized per `AGENTS.md`). + +## Verified facts (from code, not assumed) + +- **`Agent` does not touch persistence today.** It drives the provider/tool + loop and mutates a `Conversation`. So there is nothing to "unhook" — only new + ownership to add. +- **`Message.usage` is canonical.** Both providers write usage onto the + conversation message via `Conversation.addAssistantMessageWithUsage` + (`provider_anthropic_messages.zig:426`, `provider_openai_chat.zig:541`) with + the same `?Usage` they pass to `onMessageComplete`. The agent can capture + per-message usage directly from `conv.messages[i].usage` — **no receiver + dependency.** The CLIReceiver's `per_message_usage` list was a redundant + second copy and can be dropped from the persistence path. +- **Construction ordering.** System-prompt seeding needs + `luarocks_rt.layout.agent_dir` (only available after luarocks bootstrap) and + needs the store. The `Agent` holds a `*const Config` whose `registry` pointer + can be populated *after* the agent is constructed. So the new order works: + build store → build empty registry → build agent (adopts conversation + + store) → luarocks bootstrap → seed/reconcile system prompt **through the + agent** → load extensions into the registry. No deep dependency conflict; + it's a reshuffle. + +## Persistence call sites today (all in `panto`) + +- `src/main.zig`: `openSession` (init/open/resume), `appendUserPromptToSession`, + `persistTurn`/`persistCompaction` after `runStep`, plus + `getEntries`/`getSessionId`/`rebuildConversation`. +- `src/system_prompt.zig`: `seedFresh`/`reconcileResume` append system entries + via `mgr.appendMessage`. +- `src/compaction.zig` (`/compact`): `persistCompaction`. +- `src/command.zig`: `Context` holds `*SessionManager`; Lua commands reach it. + +All of these collapse into agent methods or move into `libpanto`. + +--- + +## Phase 1 — Define the interface (`libpanto/src/session_store.zig`, new) + +`pub const SessionStore = struct { ptr: *anyopaque, vtable: *const VTable }`, +matching the existing `Tool`/`ToolSource`/`Provider`/`Receiver` seams. + +VTable methods (each takes `ctx`): + +- `appendMessages(ctx, alloc, []DiskMessage, providers, models) !void` — the + batch-atomic primitive (mirrors today's `appendMessagesAtomic`; single append + is len-1). +- `loadConversation(ctx, alloc) !LoadedSession` where + `LoadedSession = struct { conversation: Conversation, dangling_user: ?[]const u8 }`. + Required read side (decision 3 + 5). +- `sessionId(ctx) []const u8`. +- `activeModel(ctx) ?struct { provider: []const u8, model: []const u8 }`. + +Notes: +- Re-export the disk types (`DiskMessage`, `DiskContentBlock`, `Usage`, …) here + so the interface is self-contained. The wire types in `session.zig` remain + the *default format* but the interface traffics in `DiskMessage` as the + neutral in-memory representation (a Postgres backend maps `DiskMessage` to + columns; it need not emit JSONL). +- **Drop `getSessionFile()` from the neutral interface** (filesystem-specific). + Keep it only on the concrete `FSJSONLStore` for CLI display/resume. +- Catalog operations (`listSessions`, `findMostRecentSession`, + `resolveSessionId`) are **not** on the interface — they are backend-specific + resume/listing helpers (a web backend lists via SQL). They stay as free + functions on the `FSJSONLStore` module. + +## Phase 2 — Backends + +- **`FSJSONLStore`** (rename within `session_manager.zig`, or new + `libpanto/src/fs_jsonl_store.zig` wrapping it): keep concrete constructors + `init(alloc, io, dir, cwd)` and `open(alloc, io, file_path)`; add + `store(self) SessionStore` returning the vtable wrapper (like + `rt.toolSource()`). Implement the vtable by delegating to existing methods + (`appendMessagesAtomic`, `activeModel`, …). Implement `loadConversation` from + the existing `rebuildConversation`, adding dangling-user detection (trailing + user entry with no following assistant → exclude from the rebuilt + conversation, return its text as `dangling_user`). +- **`NullStore`** (`libpanto/src/null_store.zig`, new): all appends no-op, + `loadConversation` returns an empty conversation + `null`, `activeModel` + null, `sessionId` a fixed/empty id. + +## Phase 3 — Agent owns the conversation, store, and persistence + +- Add fields: `conversation: Conversation` and `session_store: SessionStore` + (default `NullStore` singleton so existing `Agent.init` callers/tests keep + compiling with no persistence). +- `Agent.init(alloc, io, config, store, maybe_conversation)`: + - `maybe_conversation == null` → fresh empty `Conversation`. + - non-null → adopt it (take ownership). +- New / changed methods (drop the `*conv` parameter; operate on + `self.conversation`): + - `submitUserMessage(text)` — `conversation.addUserMessage` + persist a user + entry stamped from `self.config.provider`/`model`. + - `addSystemMessage(text, mode)` — add + persist a system entry. + - `runStep(receiver)` — snapshot `conversation.messages.len` at entry; on exit + (including error paths that committed messages), persist `[start..]` as + assistant/tool-result entries. **Move** `persistTurn`, + `hasToolUseWithoutFollowingResults`, and the disk-mapping out of + `src/session_persist.zig` into `libpanto` (agent or a new + `libpanto/src/turn_persist.zig`). + - `compact(prompt, extra)` — persist its own result (absorb + `persistCompaction`). +- **Usage capture:** read from `self.conversation.messages[i].usage` (verified + canonical). No receiver involvement. +- **`stop_reason`:** keep `"stop"` initially (matches today). Real wire value is + a follow-up (needs Receiver/provider plumbing). +- **Partial turns on error:** the agent now owns persisting messages it + committed before a `runStep` error (today the CLI persists after a failed + `runStep`). Preserve that behavior. + +## Phase 4 — `panto` CLI shrinks + +- **Delete `src/session_persist.zig` entirely** (logic moved into `libpanto`). +- `openSession`: construct an `FSJSONLStore` (concrete), keep the handle for + resume/listing, call `store.loadConversation` on resume to get + `{ conversation, dangling_user }`. Pass `store.store()` and the conversation + into `Agent.init`. Ignore `dangling_user` for now (do-nothing per decision 5). +- **Reorder construction** (decision: verified safe): store → empty registry → + agent (adopts conversation + store) → luarocks bootstrap → seed/reconcile + system prompt **through the agent** → load extensions into the registry. +- `src/system_prompt.zig`: `seedFresh`/`reconcileResume` call + `agent.addSystemMessage` instead of `mgr.appendMessage`. (System-prompt + sourcing stays CLI policy; only the persist mechanism changes.) +- REPL loop: `agent.submitUserMessage(line)` then `agent.runStep(&recv)`. + Remove `entries_before_step`, the `persistTurn`/`persistCompaction` calls, and + the `per_message_usage` → persistence plumbing. (CLIReceiver keeps usage only + if it still *displays* it.) +- `/compact` command: just `agent.compact(...)`. +- `src/command.zig` `Context`: drop the separate `*conv`; commands reach + `ctx.agent.conversation`. Keep a concrete `*FSJSONLStore` (or the interface) + for Lua commands that use `getEntries`/`getSessionId` — confirm exact usage. + +## Phase 5 — Tests & docs + +- Add an in-memory capturing `SessionStore` test double in `libpanto`; move the + `persistTurn` round-trip coverage there (verifies the agent persists the right + `DiskMessage`s without touching disk). +- Keep `FSJSONLStore` file-format tests where they are (concrete backend). +- Add a `NullStore` test (agent runs a turn; nothing persisted; no error). +- Add a `loadConversation` dangling-user test (trailing user entry excluded + from the conversation, returned as `dangling_user`). +- Update `root.zig` exports: `session_store`, the renamed `FSJSONLStore` + module, `null_store`. +- Update `docs/overview.md` persistence description and any libpanto docs. + +## Open / deferred (explicitly out of scope now) + +- **`/tree` and full session branching.** The read side is shaped to support it + (returns conversation-shaped data; dangling prompt discoverable) but + enumeration of all leaves/prefixes and leaf selection is future work. +- **Real `stop_reason`** plumbing through the Receiver. +- **TUI resume prefill** of the dangling user prompt (and the sibling-branch + write it implies). No-op for now. + +## Suggested landing order + +1. Phase 1 + 2 (interface + `FSJSONLStore`/`NullStore`), with the CLI still + calling persistence *through the interface* — de-risks the metadata/usage + move while keeping behavior identical. +2. Phase 3 — move persistence into the agent; make `Agent` own the conversation. +3. Phase 4 — collapse the CLI call sites and reorder construction. +4. Phase 5 — tests + docs throughout (not deferred to the end in practice). -- cgit v1.3