archive the session-store doc

1 files changed, 0 insertions, 236 deletions

diff --git a/docs/pluggable-session-store.md b/docs/pluggable-session-store.md
deleted file mode 100644
index 9b45864..0000000
--- a/docs/pluggable-session-store.md
+++ /dev/null
@@ -1,236 +0,0 @@
-# Plan: Agent-owned, pluggable session persistence
-
-## Problem
-
-Session logging is implemented as a single concrete type, `SessionManager`
-(`libpanto/src/session_manager.zig`), that hard-codes filesystem JSONL I/O,
-the `<uuidv7>.jsonl` filename scheme, and the directory layout. There is no
-interface seam: a different `libpanto` consumer (e.g. a web service backed by
-Postgres) cannot swap in its own persistence.
-
-Worse, persistence is currently driven *entirely outside* `libpanto`. The
-`panto` CLI translates in-memory `Conversation` messages into on-disk
-`DiskMessage`s (`src/session_persist.zig`), supplies metadata
-(provider/model/`stop_reason`/`Usage`), and sequences the append calls around
-`agent.runStep`. The `Agent` never touches the session log.
-
-## Goal
-
-`libpanto` consumers get persistence **for free**. The `Agent` owns the live
-`Conversation` and a `SessionStore`, and persists everything it generates as
-turns progress. `libpanto` ships two backends: `FSJSONLStore` (today's
-behavior, constructed with a directory) and `NullStore` (no-op, for embedders
-who opt out). The `panto` CLI resolves the directory, constructs an
-`FSJSONLStore`, and hands it to the `Agent`.
-
-## Design decisions (settled)
-
-1. **`Agent` centralizes.** The `Agent` owns its `Conversation` (as
-   `agent.conversation`) and its `SessionStore`. Turn-driving methods stop
-   taking a `*Conversation` parameter and operate on the owned conversation.
-   Fewer, more powerful types on the API surface. `Conversation` and
-   `SessionStore` are sub-components of the `Agent`'s responsibility.
-
-2. **Persistence binds to conversation mutation, not to `runStep`.** Every
-   message that enters the conversation persists when added:
-   - `agent.submitUserMessage(text)` — adds + persists a user entry, stamped
-     with `config.provider`/`model`. The user prompt is durably logged
-     *immediately on submission*, before any provider call.
-   - `agent.addSystemMessage(text, mode)` — adds + persists a system entry.
-   - `agent.runStep(receiver)` — persists assistant + tool-result messages it
-     generates internally.
-   - `agent.compact(...)` — persists its own result.
-
-   The embedder never calls a persist function.
-
-3. **The store has a read side, on the interface.** A `SessionStore` must be
-   able to reconstruct a `Conversation` (a single linear chain). Required on
-   the interface — a store you can't read is not a valid store. Later `/tree`
-   functionality (out of scope here) will use the read side to enumerate many
-   `Conversation`s (one per leaf / per unique prefix); the current
-   `ArrayList(Message)` shape supports that naturally.
-
-4. **`Conversation` is always a single linear chain.** Session branching
-   (future) lives in the store's tree of entries, not in `Conversation`. The
-   read side flattens a chosen path into one `Conversation`.
-
-5. **Resume returns `{ Conversation, ?dangling_user }`.** The read side
-   returns the reconstructed conversation plus an optional dangling trailing
-   user-prompt string (a user entry with no following assistant — e.g. a
-   crash/quit right after submission). The reconstructed `Conversation`
-   **excludes** that dangling user turn, so a resumed agent never auto-sends
-   it. Today's `panto` throws the optional string away. A future TUI will
-   prefill it for editing — which becomes the first real **session branch**
-   (an edited resubmission is a tree sibling sharing the original's
-   `parent_id`). Do not build branching now; just shape the return type so the
-   dangling prompt is discoverable.
-
-6. **`Agent.init` takes an optional `Conversation` to adopt.** `null` → fresh
-   empty conversation. Non-null → adopt (take ownership) as the live
-   conversation. Resume is an embedder choice expressed through types: open the
-   store, ask it for the conversation, hand it to `Agent.init`. No hidden
-   "replay mode" inside the agent.
-
-7. **Naming:** `FSJSONLStore`, `SessionStore`, `NullStore` (CamelCase acronyms
-   fully capitalized per `AGENTS.md`).
-
-## Verified facts (from code, not assumed)
-
-- **`Agent` does not touch persistence today.** It drives the provider/tool
-  loop and mutates a `Conversation`. So there is nothing to "unhook" — only new
-  ownership to add.
-- **`Message.usage` is canonical.** Both providers write usage onto the
-  conversation message via `Conversation.addAssistantMessageWithUsage`
-  (`provider_anthropic_messages.zig:426`, `provider_openai_chat.zig:541`) with
-  the same `?Usage` they pass to `onMessageComplete`. The agent can capture
-  per-message usage directly from `conv.messages[i].usage` — **no receiver
-  dependency.** The CLIReceiver's `per_message_usage` list was a redundant
-  second copy and can be dropped from the persistence path.
-- **Construction ordering.** System-prompt seeding needs
-  `luarocks_rt.layout.agent_dir` (only available after luarocks bootstrap) and
-  needs the store. The `Agent` holds a `*const Config` whose `registry` pointer
-  can be populated *after* the agent is constructed. So the new order works:
-  build store → build empty registry → build agent (adopts conversation +
-  store) → luarocks bootstrap → seed/reconcile system prompt **through the
-  agent** → load extensions into the registry. No deep dependency conflict;
-  it's a reshuffle.
-
-## Persistence call sites today (all in `panto`)
-
-- `src/main.zig`: `openSession` (init/open/resume), `appendUserPromptToSession`,
-  `persistTurn`/`persistCompaction` after `runStep`, plus
-  `getEntries`/`getSessionId`/`rebuildConversation`.
-- `src/system_prompt.zig`: `seedFresh`/`reconcileResume` append system entries
-  via `mgr.appendMessage`.
-- `src/compaction.zig` (`/compact`): `persistCompaction`.
-- `src/command.zig`: `Context` holds `*SessionManager`; Lua commands reach it.
-
-All of these collapse into agent methods or move into `libpanto`.
-
----
-
-## Phase 1 — Define the interface (`libpanto/src/session_store.zig`, new)
-
-`pub const SessionStore = struct { ptr: *anyopaque, vtable: *const VTable }`,
-matching the existing `Tool`/`ToolSource`/`Provider`/`Receiver` seams.
-
-VTable methods (each takes `ctx`):
-
-- `appendMessages(ctx, alloc, []DiskMessage, providers, models) !void` — the
-  batch-atomic primitive (mirrors today's `appendMessagesAtomic`; single append
-  is len-1).
-- `loadConversation(ctx, alloc) !LoadedSession` where
-  `LoadedSession = struct { conversation: Conversation, dangling_user: ?[]const u8 }`.
-  Required read side (decision 3 + 5).
-- `sessionId(ctx) []const u8`.
-- `activeModel(ctx) ?struct { provider: []const u8, model: []const u8 }`.
-
-Notes:
-- Re-export the disk types (`DiskMessage`, `DiskContentBlock`, `Usage`, …) here
-  so the interface is self-contained. The wire types in `session.zig` remain
-  the *default format* but the interface traffics in `DiskMessage` as the
-  neutral in-memory representation (a Postgres backend maps `DiskMessage` to
-  columns; it need not emit JSONL).
-- **Drop `getSessionFile()` from the neutral interface** (filesystem-specific).
-  Keep it only on the concrete `FSJSONLStore` for CLI display/resume.
-- Catalog operations (`listSessions`, `findMostRecentSession`,
-  `resolveSessionId`) are **not** on the interface — they are backend-specific
-  resume/listing helpers (a web backend lists via SQL). They stay as free
-  functions on the `FSJSONLStore` module.
-
-## Phase 2 — Backends
-
-- **`FSJSONLStore`** (rename within `session_manager.zig`, or new
-  `libpanto/src/fs_jsonl_store.zig` wrapping it): keep concrete constructors
-  `init(alloc, io, dir, cwd)` and `open(alloc, io, file_path)`; add
-  `store(self) SessionStore` returning the vtable wrapper (like
-  `rt.toolSource()`). Implement the vtable by delegating to existing methods
-  (`appendMessagesAtomic`, `activeModel`, …). Implement `loadConversation` from
-  the existing `rebuildConversation`, adding dangling-user detection (trailing
-  user entry with no following assistant → exclude from the rebuilt
-  conversation, return its text as `dangling_user`).
-- **`NullStore`** (`libpanto/src/null_store.zig`, new): all appends no-op,
-  `loadConversation` returns an empty conversation + `null`, `activeModel`
-  null, `sessionId` a fixed/empty id.
-
-## Phase 3 — Agent owns the conversation, store, and persistence
-
-- Add fields: `conversation: Conversation` and `session_store: SessionStore`
-  (default `NullStore` singleton so existing `Agent.init` callers/tests keep
-  compiling with no persistence).
-- `Agent.init(alloc, io, config, store, maybe_conversation)`:
-  - `maybe_conversation == null` → fresh empty `Conversation`.
-  - non-null → adopt it (take ownership).
-- New / changed methods (drop the `*conv` parameter; operate on
-  `self.conversation`):
-  - `submitUserMessage(text)` — `conversation.addUserMessage` + persist a user
-    entry stamped from `self.config.provider`/`model`.
-  - `addSystemMessage(text, mode)` — add + persist a system entry.
-  - `runStep(receiver)` — snapshot `conversation.messages.len` at entry; on exit
-    (including error paths that committed messages), persist `[start..]` as
-    assistant/tool-result entries. **Move** `persistTurn`,
-    `hasToolUseWithoutFollowingResults`, and the disk-mapping out of
-    `src/session_persist.zig` into `libpanto` (agent or a new
-    `libpanto/src/turn_persist.zig`).
-  - `compact(prompt, extra)` — persist its own result (absorb
-    `persistCompaction`).
-- **Usage capture:** read from `self.conversation.messages[i].usage` (verified
-  canonical). No receiver involvement.
-- **`stop_reason`:** keep `"stop"` initially (matches today). Real wire value is
-  a follow-up (needs Receiver/provider plumbing).
-- **Partial turns on error:** the agent now owns persisting messages it
-  committed before a `runStep` error (today the CLI persists after a failed
-  `runStep`). Preserve that behavior.
-
-## Phase 4 — `panto` CLI shrinks
-
-- **Delete `src/session_persist.zig` entirely** (logic moved into `libpanto`).
-- `openSession`: construct an `FSJSONLStore` (concrete), keep the handle for
-  resume/listing, call `store.loadConversation` on resume to get
-  `{ conversation, dangling_user }`. Pass `store.store()` and the conversation
-  into `Agent.init`. Ignore `dangling_user` for now (do-nothing per decision 5).
-- **Reorder construction** (decision: verified safe): store → empty registry →
-  agent (adopts conversation + store) → luarocks bootstrap → seed/reconcile
-  system prompt **through the agent** → load extensions into the registry.
-- `src/system_prompt.zig`: `seedFresh`/`reconcileResume` call
-  `agent.addSystemMessage` instead of `mgr.appendMessage`. (System-prompt
-  sourcing stays CLI policy; only the persist mechanism changes.)
-- REPL loop: `agent.submitUserMessage(line)` then `agent.runStep(&recv)`.
-  Remove `entries_before_step`, the `persistTurn`/`persistCompaction` calls, and
-  the `per_message_usage` → persistence plumbing. (CLIReceiver keeps usage only
-  if it still *displays* it.)
-- `/compact` command: just `agent.compact(...)`.
-- `src/command.zig` `Context`: drop the separate `*conv`; commands reach
-  `ctx.agent.conversation`. Keep a concrete `*FSJSONLStore` (or the interface)
-  for Lua commands that use `getEntries`/`getSessionId` — confirm exact usage.
-
-## Phase 5 — Tests & docs
-
-- Add an in-memory capturing `SessionStore` test double in `libpanto`; move the
-  `persistTurn` round-trip coverage there (verifies the agent persists the right
-  `DiskMessage`s without touching disk).
-- Keep `FSJSONLStore` file-format tests where they are (concrete backend).
-- Add a `NullStore` test (agent runs a turn; nothing persisted; no error).
-- Add a `loadConversation` dangling-user test (trailing user entry excluded
-  from the conversation, returned as `dangling_user`).
-- Update `root.zig` exports: `session_store`, the renamed `FSJSONLStore`
-  module, `null_store`.
-- Update `docs/overview.md` persistence description and any libpanto docs.
-
-## Open / deferred (explicitly out of scope now)
-
-- **`/tree` and full session branching.** The read side is shaped to support it
-  (returns conversation-shaped data; dangling prompt discoverable) but
-  enumeration of all leaves/prefixes and leaf selection is future work.
-- **Real `stop_reason`** plumbing through the Receiver.
-- **TUI resume prefill** of the dangling user prompt (and the sibling-branch
-  write it implies). No-op for now.
-
-## Suggested landing order
-
-1. Phase 1 + 2 (interface + `FSJSONLStore`/`NullStore`), with the CLI still
-   calling persistence *through the interface* — de-risks the metadata/usage
-   move while keeping behavior identical.
-2. Phase 3 — move persistence into the agent; make `Agent` own the conversation.
-3. Phase 4 — collapse the CLI call sites and reorder construction.
-4. Phase 5 — tests + docs throughout (not deferred to the end in practice).