docs

* note on the claude subscription provider plan that this is an
  extension, not panto core, not to be published
* archive that old "overview" doc
* new doc: pluggable session stores
* new doc: libpanto C wrappers, FFI wrappers for python and go

4 files changed, 562 insertions, 1 deletions

diff --git a/docs/anthropic-subscription-provider.md b/docs/anthropic-subscription-provider.md
index c6d44eb..941e33d 100644
--- a/docs/anthropic-subscription-provider.md
+++ b/docs/anthropic-subscription-provider.md
@@ -1,12 +1,14 @@
 # Anthropic subscription provider sketch
 
+**This is implemented as a Lua extension, not built into pantograph's core.** Everything described here — the provider, PTY management, MCP tool bridge, and hook harness — lives in a Lua extension that pantograph loads. Pantograph's core has no knowledge of Claude Code, subscription auth, or any of the mechanics below. The extension uses only the public extension/provider API surface that any third-party extension could use.
+
 Goal: expose a pantograph provider backed by an interactive Claude Code subscription session, without using Anthropic API keys, `claude -p`, or Agent SDK metering.
 
 ## Core idea
 
 Run `claude` as an interactive TUI process under a PTY, but make it behave like a minimal local model provider:
 
-- pantograph owns the outer UX and provider API.
+- pantograph (via the Lua extension) owns the outer UX and provider API.
 - Claude Code provides subscription-backed model turns.
 - Claude built-in tools are disabled.
 - pantograph tools are exposed to Claude Code through MCP.
diff --git a/docs/overview.md b/docs/archive/overview.md
index 916e904..916e904 100644
--- a/docs/overview.md
+++ b/docs/archive/overview.md
diff --git a/docs/libpanto-bindings.md b/docs/libpanto-bindings.md
new file mode 100644
index 0000000..9b9ef2c
--- /dev/null
+++ b/docs/libpanto-bindings.md
@@ -0,0 +1,323 @@
+# Plan: `libpanto` language bindings (Go + Python)
+
+## Goal
+
+Expose `libpanto` to other languages, targeting **Go** and **Python** first.
+The bindings are organized as a family of sibling packages, named uniformly:
+
+| package       | language | role                                                          |
+| ------------- | -------- | ------------------------------------------------------------- |
+| `libpanto`    | Zig      | the core library (exists today)                               |
+| `libpanto-c`  | Zig      | a C-ABI shared library + header wrapping the public Zig API   |
+| `libpanto-go` | Go       | idiomatic Go bindings over `libpanto-c` via cgo               |
+| `libpanto-py` | Zig      | a CPython extension implemented in pure Zig (`@cImport`)      |
+
+Two consumers, two paths to the core:
+
+- **Go → `libpanto-c` → `libpanto`.** cgo is effectively the only option, and
+  cgo can only call C. So a C ABI is mandatory.
+- **Python → `libpanto`** directly. `libpanto-py` is a native CPython
+  extension written *in Zig*: it `@cImport`s `Python.h`, builds `PyObject`
+  glue against the translated C types, and `build.zig` emits the loadable
+  `.so`. It calls the Zig API directly and **does not** depend on `libpanto-c`.
+
+### Why this split (decisions settled)
+
+- **`libpanto-c` is built regardless**, because Go requires it. It doubles as
+  the stable-ABI validation platform for the C surface.
+- **No cffi.** A pure-Python cffi binding was considered and rejected. Its only
+  real appeal (no native build step, trivial wheels) holds solidly *only* in
+  ABI mode, which is also where streaming callbacks and silent struct/ABI
+  drift bite hardest; API mode reintroduces the C-compile/per-platform-wheel
+  cost without the control of a real native module. Either way it would be a
+  throwaway second validator of the same C surface that `libpanto-go` already
+  exercises. Net: redundant. We'd reimplement it natively on the soon side.
+- **`libpanto-py` is pure Zig, not C-over-`libpanto-c`.** The decisive win is
+  that a Zig→`.so` compilation involves **no C translation units in the repo at
+  all** — one toolchain, header translation done at compile time by `@cImport`.
+  Going straight to the Zig API (rather than through `libpanto-c`) also skips a
+  redundant marshalling layer.
+
+## The core architectural decision: ship a *pull* streaming API
+
+This is the spine of the whole project, and it requires an **internal refactor
+of `libpanto` first** (Phase 0). Everything else wraps it.
+
+### Why pull, not push
+
+`libpanto` today streams via a **push** `Receiver` vtable
+(`provider.zig`): the provider loop calls `onMessageStart`, `onContentDelta`,
+`onBlockComplete`, `onMessageComplete`, etc. Push is the wrong primitive to
+*export*, for two independent reasons that land on the same conclusion:
+
+1. **It doesn't compose into the idiomatic surfaces we want.**
+   - Go: the idiomatic streaming shape is a single goroutine driving a
+     range-over-func iterator (`for ev := range stream.Iter`) and/or a channel.
+     cgo callbacks *into* Go are slow and awkward (`//export`, pointer rules,
+     no closures). A goroutine draining a pull API is the clean build.
+   - Python: a (sync or async) generator **is** a pull construct —
+     `__next__` *is* "give me the next event." A push callback can't `yield`;
+     to turn pushed values into a `for ev in stream`, you must run the producer
+     and consumer in different stack frames concurrently (a thread + queue).
+     So push → generator *forces* a thread+queue regardless. Pull → generator
+     is a 1:1 mapping with zero adaptation.
+
+2. **Pull is the more primitive primitive.** Push composes trivially on top of
+   pull (`while (try s.next()) |ev| receiver.onEvent(ev);` — five lines).
+   Pull does **not** compose cheaply on top of push: to expose `next()` over a
+   `Receiver`-driven core you must suspend *inside* a callback and resume the
+   provider loop later, which means a thread+queue or hand-rolled coroutine
+   state — i.e. you rebuild pull anyway, the hard way.
+
+**Conclusion:** ship only the pull API. `Receiver` leaves the public surface
+entirely. Anyone who wants push writes the five-line wrapper themselves.
+
+### The Zig interface
+
+```zig
+// run() drops the receiver parameter entirely and returns a Stream.
+// Whether it also drops `conv` depends on sequencing against
+// docs/pluggable-session-store.md (see note below) — either form is fine.
+pub fn run(self: *Agent) !Stream  // or: run(self: *Agent, conv: *Conversation) !Stream
+
+// Stream is a resumable handle, closely linked to SSEParser + the provider
+// read loop. next() yields events until the turn is exhausted.
+pub fn next(self: *Stream) !?Event
+```
+
+`Stream.next()` returns `!?Event`, which gives **three orthogonal channels**:
+
+| signal             | meaning                                              |
+| ------------------ | ---------------------------------------------------- |
+| `Event` (a value)  | streaming progress, including the terminal event     |
+| `null`             | the stream is exhausted (already past the terminal)  |
+| `error.X`          | a genuine failure (network, parse, provider error)   |
+
+### The terminal-event invariant (documented contract)
+
+> Every `next()` call on a stream **after** it has yielded a `MessageComplete`
+> event returns `null`, and `null` is **never** returned before
+> `MessageComplete`.
+
+Consequences, and why this exact shape:
+
+- **`MessageComplete` is a normal `Event` variant**, not an error and not
+  signalled by `null`. It is the in-band success terminal.
+- **Intelligent consumers never observe `null`.** They stop after *consuming*
+  `MessageComplete`. `null` exists only as the defensive answer to "you called
+  `next()` one time too many," distinct from a real `error`.
+- Python: emit `MessageComplete`, then raise `StopIteration` on the *next*
+  call (which sees `null`) — or, more precisely, the generator stops right
+  after yielding `MessageComplete`, so well-behaved code raises `StopIteration`
+  without ever materializing a `None`.
+- Go: the `Iter` range-over-func stops *after* yielding `MessageComplete`, not
+  by waiting to observe a `null` sentinel.
+- Mid-stream **provider errors surface as a Zig `error` from `next()`** (the
+  `!` in `!?Event`), not as an `Event.Error` variant. This keeps the `Event`
+  union success-only and maps cleanly to a raised Python exception and a Go
+  `error`. (`error.StreamExhausted` is *not* used — exhaustion is `null`, not
+  an error; we reserve errors for failures.)
+
+### `Event` spans the whole turn, not one HTTP response
+
+Verified from the code: `Agent.runStep` wraps **multiple** `streamStep` calls
+in a tool-using turn (one provider stream per assistant message), with
+concurrent tool dispatch *between* them (`agent.zig`). The `Stream` therefore
+spans the entire `run()` loop — provider streaming **and** tool dispatch — not
+a single SSE response. The `Event` union must express both layers.
+
+The exact variant list is to be finalized against what the provider loop and
+agent loop emit today (see `ReceiverVTable` in `provider.zig` and the dispatch
+path in `agent.zig`), but it covers at least:
+
+- message lifecycle: `MessageStart`, `MessageComplete`
+- block lifecycle: `BlockStart`, `BlockComplete`
+- content: `ContentDelta`
+- tool identity: `ToolDetails` (id + name)
+- tool dispatch boundaries: tool-call start / tool-result (so Go/Python
+  consumers can observe the agent running tools between provider turns)
+- provider retry notices: `ProviderRetry` (informational; today `onProviderRetry`)
+
+`Event` is an `enum union`; it is the single type every binding marshals into
+its native form. Define it once, in `libpanto`.
+
+## The hard part of Phase 0: making `Stream.next()` resumable
+
+Verified from `provider_openai_chat.zig`: the current provider loop is a
+single-stack-frame loop —
+
+```
+readVec(body_reader) -> parser.feed(chunk) -> handleEvent(...) -> receiver.*(...)
+```
+
+— with all state (the HTTP body reader + `transfer_buf`, the `SSEParser`, the
+per-response `StreamState`) living **on the stack**. A pull `next()` must
+*return* between events, so that state has to move into the `Stream` handle.
+Three ways to get there:
+
+1. **State machine (recommended target).** The `Stream` owns the socket/body
+   reader, the `SSEParser`, and the decode state; `next()` reads/parses just
+   enough to produce one `Event` and returns. No threads. Cleanest and most
+   portable across all four packages. Most upfront refactoring effort, since
+   the provider loop is inverted into a resumable step function.
+
+2. **Thread + queue inside the handle.** Keep the existing
+   `streamStep(receiver)` loop verbatim; run it on a thread whose receiver
+   pushes events into a bounded queue; `next()` pops. Minimal change to proven
+   code, but costs a thread per active stream plus shutdown/backpressure care.
+   A reasonable interim if (1) proves invasive.
+
+3. **Zig async/suspend.** Language-level coroutines. Availability depends on the
+   toolchain's async status at the time of implementation; treat as
+   not-reliably-available without checking first.
+
+The internal mechanism (1 vs 2) is swappable later **without changing the
+exported contract** — the public surface is pull either way. Recommendation:
+target (1); fall back to (2) only if the provider-loop inversion is too costly
+for v1.
+
+> Expectation: **Phase 0 is a net-red diff.** Deleting `Receiver` from the
+> public API, the `CompactionCapture` no-op receiver, and the CLI's
+> `CLIReceiver` vtable plumbing should remove more than the `Stream` machinery
+> adds.
+
+---
+
+## Phase 0 — internal refactor of `libpanto` to a pull API
+
+The real work. Everything after this is wrapping.
+
+1. **Define the `Event` union** (success-only `enum union`) from the current
+   `ReceiverVTable` callbacks plus the agent's tool-dispatch boundaries. This
+   is the type every binding marshals.
+2. **Define the `Stream` type**: a resumable handle owning the provider read
+   loop's state (body reader, `SSEParser`, decode state) and the agent's
+   tool-dispatch position. `fn next(self: *Stream) !?Event`. This is where the
+   state-machine-vs-thread decision lands.
+3. **`Agent.run() -> Stream`.** With `conversation` owned by the `Agent` (per
+   `docs/pluggable-session-store.md`) and `receiver` removed, `run()` takes
+   only `self`. The agent loop (provider stream → tool dispatch → repeat)
+   becomes the thing `Stream.next()` drives incrementally.
+4. **Delete `Receiver` from the public API** (`root.zig`, `provider.zig`).
+   Internal seams that genuinely want push (if any) get the trivial pull→push
+   wrapper, in caller code, not in the library surface.
+5. **Rebuild the CLI on the pull loop.** `src/main.zig`'s `CLIReceiver`
+   collapses into a `while (try stream.next()) |ev| render(ev)` loop. Prove the
+   refactor end-to-end against the existing test suite; CLI behavior unchanged.
+
+**Exit criteria:** `Receiver` gone from `root.zig`; `Stream.next()` is the only
+streaming primitive; CLI output and the full test suite are unchanged.
+
+> Sequencing with `docs/pluggable-session-store.md` is **flexible**, not a
+> dependency. That plan moves `Conversation` onto the `Agent`; this one removes
+> `receiver`. The two are orthogonal:
+> - If the session-store work lands first, `run()` takes only `self`.
+> - If this work lands first, `run(self, conv)` takes a single `conv` argument
+>   (just `runStep` minus `receiver`), and the session-store refactor drops
+>   `conv` later.
+>
+> Either way `receiver` is gone after Phase 0. Don't block on ordering.
+
+## Phase 1 — `libpanto-c` (C ABI)
+
+6. **Opaque handles** for the major types: `PantoAgent`, `PantoStream` (and
+   whatever construction requires — config, conversation/session store).
+   Consumers never see the layout.
+7. **`panto_next_event(stream, *PantoEvent) -> status`** is the C projection of
+   `!?Event`. The `!?Event` triple maps onto a status enum plus an out-param:
+
+   | Zig `next()`            | C status     | out-param        |
+   | ----------------------- | ------------ | ---------------- |
+   | `Event` value           | `EVENT` (0)  | filled           |
+   | `null`                  | `DONE` (1)   | untouched        |
+   | `error.X`               | `ERROR` (2)  | error detail     |
+
+8. **`@export` wrappers** for construction, `run()`, `next_event`, teardown,
+   and a free function for any owned event payloads.
+9. **The C header is committed and hand-maintained, not generated by
+   `build.zig`.** (Open question 1 below: we are *not* having `build.zig` emit
+   `panto.h`.) A stable, hand-written `include/panto.h` is the ABI contract —
+   it should change deliberately, be reviewable in diffs, and not be a build
+   artifact. `build.zig` emits the shared library and stages the committed
+   header; it does not author it.
+10. **Define the event-marshalling C structs once.** `PantoEvent` (a tagged
+    union mirroring the Zig `Event`) is read by every downstream binding.
+
+## Phase 2 — `libpanto-go` (validates `libpanto-c`)
+
+11. **cgo bindings**: `Agent`, `Stream`, and the raw
+    `Stream.Next() (Event, bool, error)` (or `(Event, error)` with a separate
+    done signal) mapping `EVENT`/`DONE`/`ERROR` onto Go's idioms.
+12. **`Stream.Iter() iter.Seq[Event]`** — the modern range-over-func form, so
+    consumers write `for ev := range stream.Iter { ... }`. Single goroutine,
+    auto-terminates after yielding `MessageComplete`, surfaces a failure via a
+    trailing `stream.Err()` after the range (the idiomatic Go pattern since
+    `bufio.Scanner`). This is the primary Go surface.
+13. **A channel wrapper too.** Go users expect channels; a goroutine drains
+    `Next()` into a channel. Cheap over a pull core. Ship both the raw
+    iterator and the channel form.
+
+`libpanto-go` is the validation harness for the `libpanto-c` ABI — if Go can
+drive a full streaming turn idiomatically, the C surface is sound.
+
+## Phase 3 — `libpanto-py` (pure Zig CPython extension)
+
+14. **`build.zig` does `@cImport(@cInclude("Python.h"))`** and emits
+    `_panto.so` (a loadable extension module). No C in this package. Decide
+    **stable ABI (`abi3` / limited API)** here to collapse the
+    CPython-version × platform wheel matrix to one artifact per platform.
+15. **`module.zig`** implements `PyModuleDef`, `PyMethodDef`, and an `Agent`
+    type and `Stream` type as `PyTypeObject`s, calling the **Zig** `libpanto`
+    API directly. The `Stream` type's `tp_iternext` calls Zig `Stream.next()`:
+    - `Event` → build and return the `PyObject` event.
+    - `null` → set `StopIteration` (return `NULL` with no error set).
+    - `error.X` → set the mapped Python exception.
+    - Wrap the blocking `next()` in `Py_BEGIN_ALLOW_THREADS` /
+      `Py_END_ALLOW_THREADS` so streaming doesn't serialize the whole
+      interpreter; re-acquire the GIL to build the event `PyObject`.
+16. **`panto/__init__.py`** is a thin Pythonic surface over the native
+    `_panto`: a real exception hierarchy, context managers, idiomatic event
+    objects. **The async generator is pure Python** — wrap the sync iterator
+    with `asyncio.to_thread` (blocking `next()` runs in a worker thread). No
+    file descriptors, no ABI change. (See "Out of scope for v1" below.)
+
+## The one contract that unifies all four packages
+
+| layer    | progress / terminal       | exhausted          | failure                |
+| -------- | ------------------------- | ------------------ | ---------------------- |
+| Zig      | `Event` / `MessageComplete` | `null`           | `error.X`              |
+| C        | `PantoEvent` / status `EVENT` | status `DONE`  | status `ERROR`         |
+| Go       | `Event` (Iter yields)     | Iter stops         | `error` / `stream.Err()` |
+| Python   | event object (yielded)    | `StopIteration`    | raised exception       |
+
+Design every binding to this single table. Pull-shaped, success-only events,
+terminal-by-`MessageComplete`, exhaustion-by-`null`, failure-by-error.
+
+## Out of scope for v1 (deliberately deferred)
+
+- **A pollable fd / `panto_step_poll(timeout)`.** This is the only thing that
+  makes async *natively* non-blocking on both Go (`select`) and Python
+  (`await` on readiness). It is real work in `libpanto-c` and unnecessary for
+  v1. **Design the C ABI so a pollable fd can be added later without breaking
+  the existing pull surface**, but do not build it now.
+- **A native async Python API.** Without an fd, async Python is *just* the sync
+  pull API wrapped in `asyncio.to_thread`. That is pure-Python glue in
+  `panto/__init__.py`; no native or ABI work, so it isn't a binding deliverable.
+- **Languages beyond Go and Python.** `libpanto-c` is the reuse point for any
+  future cgo-style or cffi-style consumer (Ruby, Node N-API, …); none are in
+  scope now.
+
+## Open questions / decisions to finalize before coding
+
+1. **`panto.h` generation — resolved: no.** The header is committed and
+   hand-maintained as the ABI contract; `build.zig` stages it but does not emit
+   it. (Captured in Phase 1, step 9.)
+2. **Resumable-handle strategy (state machine vs. thread+queue)** — target the
+   state machine; decide finally once the provider-loop inversion is scoped.
+3. **Exact `Event` variants** — finalize against `ReceiverVTable`
+   (`provider.zig`) and the tool-dispatch path (`agent.zig`).
+4. **What moves onto `Agent` vs. stays per-`run()`** — driven by
+   `docs/pluggable-session-store.md`'s final shape; `conversation` is moving,
+   confirm nothing else needs to.
+5. **CPython stable-ABI (`abi3`) commitment** — decide in Phase 3 to fix the
+   wheel matrix early.
diff --git a/docs/pluggable-session-store.md b/docs/pluggable-session-store.md
new file mode 100644
index 0000000..9b45864
--- /dev/null
+++ b/docs/pluggable-session-store.md
@@ -0,0 +1,236 @@
+# Plan: Agent-owned, pluggable session persistence
+
+## Problem
+
+Session logging is implemented as a single concrete type, `SessionManager`
+(`libpanto/src/session_manager.zig`), that hard-codes filesystem JSONL I/O,
+the `<uuidv7>.jsonl` filename scheme, and the directory layout. There is no
+interface seam: a different `libpanto` consumer (e.g. a web service backed by
+Postgres) cannot swap in its own persistence.
+
+Worse, persistence is currently driven *entirely outside* `libpanto`. The
+`panto` CLI translates in-memory `Conversation` messages into on-disk
+`DiskMessage`s (`src/session_persist.zig`), supplies metadata
+(provider/model/`stop_reason`/`Usage`), and sequences the append calls around
+`agent.runStep`. The `Agent` never touches the session log.
+
+## Goal
+
+`libpanto` consumers get persistence **for free**. The `Agent` owns the live
+`Conversation` and a `SessionStore`, and persists everything it generates as
+turns progress. `libpanto` ships two backends: `FSJSONLStore` (today's
+behavior, constructed with a directory) and `NullStore` (no-op, for embedders
+who opt out). The `panto` CLI resolves the directory, constructs an
+`FSJSONLStore`, and hands it to the `Agent`.
+
+## Design decisions (settled)
+
+1. **`Agent` centralizes.** The `Agent` owns its `Conversation` (as
+   `agent.conversation`) and its `SessionStore`. Turn-driving methods stop
+   taking a `*Conversation` parameter and operate on the owned conversation.
+   Fewer, more powerful types on the API surface. `Conversation` and
+   `SessionStore` are sub-components of the `Agent`'s responsibility.
+
+2. **Persistence binds to conversation mutation, not to `runStep`.** Every
+   message that enters the conversation persists when added:
+   - `agent.submitUserMessage(text)` — adds + persists a user entry, stamped
+     with `config.provider`/`model`. The user prompt is durably logged
+     *immediately on submission*, before any provider call.
+   - `agent.addSystemMessage(text, mode)` — adds + persists a system entry.
+   - `agent.runStep(receiver)` — persists assistant + tool-result messages it
+     generates internally.
+   - `agent.compact(...)` — persists its own result.
+
+   The embedder never calls a persist function.
+
+3. **The store has a read side, on the interface.** A `SessionStore` must be
+   able to reconstruct a `Conversation` (a single linear chain). Required on
+   the interface — a store you can't read is not a valid store. Later `/tree`
+   functionality (out of scope here) will use the read side to enumerate many
+   `Conversation`s (one per leaf / per unique prefix); the current
+   `ArrayList(Message)` shape supports that naturally.
+
+4. **`Conversation` is always a single linear chain.** Session branching
+   (future) lives in the store's tree of entries, not in `Conversation`. The
+   read side flattens a chosen path into one `Conversation`.
+
+5. **Resume returns `{ Conversation, ?dangling_user }`.** The read side
+   returns the reconstructed conversation plus an optional dangling trailing
+   user-prompt string (a user entry with no following assistant — e.g. a
+   crash/quit right after submission). The reconstructed `Conversation`
+   **excludes** that dangling user turn, so a resumed agent never auto-sends
+   it. Today's `panto` throws the optional string away. A future TUI will
+   prefill it for editing — which becomes the first real **session branch**
+   (an edited resubmission is a tree sibling sharing the original's
+   `parent_id`). Do not build branching now; just shape the return type so the
+   dangling prompt is discoverable.
+
+6. **`Agent.init` takes an optional `Conversation` to adopt.** `null` → fresh
+   empty conversation. Non-null → adopt (take ownership) as the live
+   conversation. Resume is an embedder choice expressed through types: open the
+   store, ask it for the conversation, hand it to `Agent.init`. No hidden
+   "replay mode" inside the agent.
+
+7. **Naming:** `FSJSONLStore`, `SessionStore`, `NullStore` (CamelCase acronyms
+   fully capitalized per `AGENTS.md`).
+
+## Verified facts (from code, not assumed)
+
+- **`Agent` does not touch persistence today.** It drives the provider/tool
+  loop and mutates a `Conversation`. So there is nothing to "unhook" — only new
+  ownership to add.
+- **`Message.usage` is canonical.** Both providers write usage onto the
+  conversation message via `Conversation.addAssistantMessageWithUsage`
+  (`provider_anthropic_messages.zig:426`, `provider_openai_chat.zig:541`) with
+  the same `?Usage` they pass to `onMessageComplete`. The agent can capture
+  per-message usage directly from `conv.messages[i].usage` — **no receiver
+  dependency.** The CLIReceiver's `per_message_usage` list was a redundant
+  second copy and can be dropped from the persistence path.
+- **Construction ordering.** System-prompt seeding needs
+  `luarocks_rt.layout.agent_dir` (only available after luarocks bootstrap) and
+  needs the store. The `Agent` holds a `*const Config` whose `registry` pointer
+  can be populated *after* the agent is constructed. So the new order works:
+  build store → build empty registry → build agent (adopts conversation +
+  store) → luarocks bootstrap → seed/reconcile system prompt **through the
+  agent** → load extensions into the registry. No deep dependency conflict;
+  it's a reshuffle.
+
+## Persistence call sites today (all in `panto`)
+
+- `src/main.zig`: `openSession` (init/open/resume), `appendUserPromptToSession`,
+  `persistTurn`/`persistCompaction` after `runStep`, plus
+  `getEntries`/`getSessionId`/`rebuildConversation`.
+- `src/system_prompt.zig`: `seedFresh`/`reconcileResume` append system entries
+  via `mgr.appendMessage`.
+- `src/compaction.zig` (`/compact`): `persistCompaction`.
+- `src/command.zig`: `Context` holds `*SessionManager`; Lua commands reach it.
+
+All of these collapse into agent methods or move into `libpanto`.
+
+---
+
+## Phase 1 — Define the interface (`libpanto/src/session_store.zig`, new)
+
+`pub const SessionStore = struct { ptr: *anyopaque, vtable: *const VTable }`,
+matching the existing `Tool`/`ToolSource`/`Provider`/`Receiver` seams.
+
+VTable methods (each takes `ctx`):
+
+- `appendMessages(ctx, alloc, []DiskMessage, providers, models) !void` — the
+  batch-atomic primitive (mirrors today's `appendMessagesAtomic`; single append
+  is len-1).
+- `loadConversation(ctx, alloc) !LoadedSession` where
+  `LoadedSession = struct { conversation: Conversation, dangling_user: ?[]const u8 }`.
+  Required read side (decision 3 + 5).
+- `sessionId(ctx) []const u8`.
+- `activeModel(ctx) ?struct { provider: []const u8, model: []const u8 }`.
+
+Notes:
+- Re-export the disk types (`DiskMessage`, `DiskContentBlock`, `Usage`, …) here
+  so the interface is self-contained. The wire types in `session.zig` remain
+  the *default format* but the interface traffics in `DiskMessage` as the
+  neutral in-memory representation (a Postgres backend maps `DiskMessage` to
+  columns; it need not emit JSONL).
+- **Drop `getSessionFile()` from the neutral interface** (filesystem-specific).
+  Keep it only on the concrete `FSJSONLStore` for CLI display/resume.
+- Catalog operations (`listSessions`, `findMostRecentSession`,
+  `resolveSessionId`) are **not** on the interface — they are backend-specific
+  resume/listing helpers (a web backend lists via SQL). They stay as free
+  functions on the `FSJSONLStore` module.
+
+## Phase 2 — Backends
+
+- **`FSJSONLStore`** (rename within `session_manager.zig`, or new
+  `libpanto/src/fs_jsonl_store.zig` wrapping it): keep concrete constructors
+  `init(alloc, io, dir, cwd)` and `open(alloc, io, file_path)`; add
+  `store(self) SessionStore` returning the vtable wrapper (like
+  `rt.toolSource()`). Implement the vtable by delegating to existing methods
+  (`appendMessagesAtomic`, `activeModel`, …). Implement `loadConversation` from
+  the existing `rebuildConversation`, adding dangling-user detection (trailing
+  user entry with no following assistant → exclude from the rebuilt
+  conversation, return its text as `dangling_user`).
+- **`NullStore`** (`libpanto/src/null_store.zig`, new): all appends no-op,
+  `loadConversation` returns an empty conversation + `null`, `activeModel`
+  null, `sessionId` a fixed/empty id.
+
+## Phase 3 — Agent owns the conversation, store, and persistence
+
+- Add fields: `conversation: Conversation` and `session_store: SessionStore`
+  (default `NullStore` singleton so existing `Agent.init` callers/tests keep
+  compiling with no persistence).
+- `Agent.init(alloc, io, config, store, maybe_conversation)`:
+  - `maybe_conversation == null` → fresh empty `Conversation`.
+  - non-null → adopt it (take ownership).
+- New / changed methods (drop the `*conv` parameter; operate on
+  `self.conversation`):
+  - `submitUserMessage(text)` — `conversation.addUserMessage` + persist a user
+    entry stamped from `self.config.provider`/`model`.
+  - `addSystemMessage(text, mode)` — add + persist a system entry.
+  - `runStep(receiver)` — snapshot `conversation.messages.len` at entry; on exit
+    (including error paths that committed messages), persist `[start..]` as
+    assistant/tool-result entries. **Move** `persistTurn`,
+    `hasToolUseWithoutFollowingResults`, and the disk-mapping out of
+    `src/session_persist.zig` into `libpanto` (agent or a new
+    `libpanto/src/turn_persist.zig`).
+  - `compact(prompt, extra)` — persist its own result (absorb
+    `persistCompaction`).
+- **Usage capture:** read from `self.conversation.messages[i].usage` (verified
+  canonical). No receiver involvement.
+- **`stop_reason`:** keep `"stop"` initially (matches today). Real wire value is
+  a follow-up (needs Receiver/provider plumbing).
+- **Partial turns on error:** the agent now owns persisting messages it
+  committed before a `runStep` error (today the CLI persists after a failed
+  `runStep`). Preserve that behavior.
+
+## Phase 4 — `panto` CLI shrinks
+
+- **Delete `src/session_persist.zig` entirely** (logic moved into `libpanto`).
+- `openSession`: construct an `FSJSONLStore` (concrete), keep the handle for
+  resume/listing, call `store.loadConversation` on resume to get
+  `{ conversation, dangling_user }`. Pass `store.store()` and the conversation
+  into `Agent.init`. Ignore `dangling_user` for now (do-nothing per decision 5).
+- **Reorder construction** (decision: verified safe): store → empty registry →
+  agent (adopts conversation + store) → luarocks bootstrap → seed/reconcile
+  system prompt **through the agent** → load extensions into the registry.
+- `src/system_prompt.zig`: `seedFresh`/`reconcileResume` call
+  `agent.addSystemMessage` instead of `mgr.appendMessage`. (System-prompt
+  sourcing stays CLI policy; only the persist mechanism changes.)
+- REPL loop: `agent.submitUserMessage(line)` then `agent.runStep(&recv)`.
+  Remove `entries_before_step`, the `persistTurn`/`persistCompaction` calls, and
+  the `per_message_usage` → persistence plumbing. (CLIReceiver keeps usage only
+  if it still *displays* it.)
+- `/compact` command: just `agent.compact(...)`.
+- `src/command.zig` `Context`: drop the separate `*conv`; commands reach
+  `ctx.agent.conversation`. Keep a concrete `*FSJSONLStore` (or the interface)
+  for Lua commands that use `getEntries`/`getSessionId` — confirm exact usage.
+
+## Phase 5 — Tests & docs
+
+- Add an in-memory capturing `SessionStore` test double in `libpanto`; move the
+  `persistTurn` round-trip coverage there (verifies the agent persists the right
+  `DiskMessage`s without touching disk).
+- Keep `FSJSONLStore` file-format tests where they are (concrete backend).
+- Add a `NullStore` test (agent runs a turn; nothing persisted; no error).
+- Add a `loadConversation` dangling-user test (trailing user entry excluded
+  from the conversation, returned as `dangling_user`).
+- Update `root.zig` exports: `session_store`, the renamed `FSJSONLStore`
+  module, `null_store`.
+- Update `docs/overview.md` persistence description and any libpanto docs.
+
+## Open / deferred (explicitly out of scope now)
+
+- **`/tree` and full session branching.** The read side is shaped to support it
+  (returns conversation-shaped data; dangling prompt discoverable) but
+  enumeration of all leaves/prefixes and leaf selection is future work.
+- **Real `stop_reason`** plumbing through the Receiver.
+- **TUI resume prefill** of the dangling user prompt (and the sibling-branch
+  write it implies). No-op for now.
+
+## Suggested landing order
+
+1. Phase 1 + 2 (interface + `FSJSONLStore`/`NullStore`), with the CLI still
+   calling persistence *through the interface* — de-risks the metadata/usage
+   move while keeping behavior identical.
+2. Phase 3 — move persistence into the agent; make `Agent` own the conversation.
+3. Phase 4 — collapse the CLI call sites and reorder construction.
+4. Phase 5 — tests + docs throughout (not deferred to the end in practice).