From 9c64a7d4462a11674e2dea481b037b5f5d9c62fc Mon Sep 17 00:00:00 2001 From: t Date: Tue, 2 Jun 2026 09:02:49 -0600 Subject: system prompt building and logging --- docs/archive/system-prompt.md | 299 ++++++++++++++++++++++++++++++++++++++++++ docs/system-prompt.md | 289 ---------------------------------------- 2 files changed, 299 insertions(+), 289 deletions(-) create mode 100644 docs/archive/system-prompt.md delete mode 100644 docs/system-prompt.md (limited to 'docs') diff --git a/docs/archive/system-prompt.md b/docs/archive/system-prompt.md new file mode 100644 index 0000000..e44f953 --- /dev/null +++ b/docs/archive/system-prompt.md @@ -0,0 +1,299 @@ +# System Prompt + +Status: implemented (libpanto data model + serializers + session store + +CLI sourcing & resume reconciliation). See `src/system_prompt.zig` and the +seeding/reconciliation wiring in `src/main.zig`. + +This document specifies first-class support for the system prompt across +libpanto (the conversation + session-store data model and provider +serializers) and the `panto` CLI (file-based sourcing across config +layers, plus reconciliation on resume). + +Today there is no system-prompt API at all: the CLI hard-codes +`"You are a helpful assistant."` and seeds it as a single `.system`-role +message (`src/main.zig`). This design replaces that with a model where the +system prompt is conversation data that can **change over the life of a +conversation** and is **faithfully reconstructable at any earlier point in +time** (a prerequisite for a future pi-style `/tree` command). + +## Principles + +- **The system prompt is conversation state, not config state.** It lives + in the conversation and the session log, never in the per-turn `Config` + snapshot. The agent does not own or inject it. +- **It changes over time, and the log records that change.** Every + mutation is its own positioned log entry. Truncating the message list at + position _N_ and re-deriving the prompt yields exactly the prompt as it + was at _N_. This is what makes `/tree` faithful. +- **Append-only history, replace-capable semantics.** Mutations only ever + *append* log entries. A mutation may carry `mode = replace`, which means + "from here on, discard all prior system text," but it does so by adding a + new entry — it never rewrites history. +- **Convention over configuration in the CLI.** No new TOML keys. The + prompt is sourced from `SYSTEM.md` / `APPEND_SYSTEM.md` files discovered + across the existing config layers. + +## Part 1 — libpanto / session-store data model + +### 1.1 A `.System` content block with a mode + +System prompts remain `.system`-**role** messages. What changes is the +*content block*: instead of a plain `.Text` block, a system message +carries a new `System` content-block variant that records its mode. + +```zig +pub const SystemBlock = struct { + text: TextualBlock = .empty, + mode: SystemMode = .append, + + pub fn deinit(self: *SystemBlock, alloc: Allocator) void { + self.text.deinit(alloc); + } +}; + +pub const SystemMode = enum { append, replace }; + +pub const ContentBlock = union(enum) { + Text: TextualBlock, + Thinking: ThinkingBlock, + ToolUse: ToolUseBlock, + ToolResult: ToolResultBlock, + System: SystemBlock, // new + + // deinit gains a `.System => |*b| b.deinit(alloc)` arm. +}; +``` + +Rationale for a distinct block (rather than a `mode` field hung off +`Message`): the mode is only meaningful for system content. Putting it on +`Message` would leave a meaningless field on every user/assistant/tool +message. Keeping the `.system` *role* (option (a) from design discussion) +keeps the blast radius minimal: serializers and the session manager +already filter on `role == .system`; only the block payload grows. + +### 1.2 Conversation methods + +```zig +/// Append a system message in `append` mode. Adds to the effective +/// system prompt. (Back-compatible: same external behavior as today.) +pub fn addSystemMessage(self: *Conversation, text: []const u8) !void; + +/// Append a system message in `replace` mode. When the effective prompt +/// is rebuilt, this discards all prior system text and starts fresh. +pub fn replaceSystemMessage(self: *Conversation, text: []const u8) !void; +``` + +Both append a `.system`-role message whose single content block is a +`.System` block; they differ only in the recorded `mode`. Both are +available to extensions, so an extension can grow *or* wholesale replace +the system prompt at any point mid-conversation. + +### 1.3 Deriving the effective system prompt + +A single shared rule governs both provider serializers and session +rebuild. Walk the conversation messages in order; for each `.system` +message's `.System` block: + +- `append`: add the block's text to the running list of effective system + blocks. +- `replace`: **clear the running list**, then add this block's text. + +The result is an ordered list of surviving system-text blocks. "Replace +means replace" — it wipes everything collected so far, regardless of how +those earlier blocks were joined. + +Because each mutation is a positioned entry, this same walk over a +*prefix* of the messages reconstructs the prompt as of that point — the +`/tree` property. + +### 1.4 Provider serialization + +**Anthropic (`anthropic_messages_json.zig`).** The wire format requires a +single top-level `system` string. `collectSystemPrompt` already walks all +`.system` messages; update it to: + +1. Apply the append/replace derivation above. +2. For each surviving block, **strip trailing newlines**. +3. **Join with `\n\n---\n\n`** (double-newline / horizontal rule / + double-newline), instead of the current single `\n`. + +Emit the joined string as the top-level `system` field (omit the field +entirely if empty, as today). + +**OpenAI (`openai_chat_json.zig`).** Today `.system` messages are emitted +positionally as ordinary messages. Change to: + +1. Apply the append/replace derivation. +2. Emit the surviving system blocks as **separate leading `system`-role + messages**, in order, before any non-system message. +3. Emit all non-system messages in their original order afterward. + +Do **not** concatenate OpenAI system messages into one. Keeping them as +separate, individually-positioned messages preserves block-level +addressability (again, for `/tree`-style truncation). The double-rule +join is an Anthropic-only concession to its single-string wire format. + +### 1.5 Session log format + +`system`-role log objects gain an optional `mode` field: + +```json +{"...","message":{"role":"system","mode":"replace","content":[{"type":"text","text":"..."}]}} +``` + +- `mode` is `"append"` or `"replace"`. +- **Absent `mode` defaults to `"append"`** — existing logs read back + identically, no migration needed. + +On write, the session manager records the block's mode. On +`rebuildConversation`, it reads `mode` (defaulting to `append`) and +reconstructs the corresponding `.System` block. + +## Part 2 — `panto` CLI: sourcing & reconciliation + +### 2.1 File discovery across layers + +The system prompt is sourced from files discovered across the same three +layers the CLI already uses for config / extensions / tools, in +precedence order **base → user → project** (project highest): + +| Layer | Directory | +|---------|------------------------------------------------------| +| base | `${XDG_DATA_HOME:-$HOME/.local/share}/panto/` | +| user | `${XDG_CONFIG_HOME:-$HOME/.config}/panto/` | +| project | `./.panto/` | + +In each layer we look for two files: + +- **`SYSTEM.md`** — the base/seed system prompt. +- **`APPEND_SYSTEM.md`** — an additional appended system block. + +There is **no TOML key** for the system prompt. Convention only. + +### 2.2 Resolution rules + +- **`SYSTEM.md`:** the **highest layer present wins** (whole-file + override, matching how scalar config values already override across + layers). The winning file's content becomes the **seed** system block. +- **`APPEND_SYSTEM.md`:** **every** layer's file is respected; each + becomes its own appended system block. +- **Built-in default:** if no `SYSTEM.md` exists at any layer, fall back + to a built-in default seed. (The current default, + `"You are a helpful assistant."`, needs a rewrite — tracked separately.) + +### 2.3 Ordering of the resolved blocks + +The resolved sequence of system blocks for a fresh session is: + +1. **Seed** (`SYSTEM.md` winner, or built-in default) — emitted first, so + it reads earliest / highest-salience. +2. **Appends**, collected base → user → project, then **emitted in + reversed order: project → user → base.** + +Reversing the appends places the project-layer append earliest among the +appends (right after the seed). The working hypothesis is that LLMs weight +earlier prompt text more heavily, so the most-specific (project) layer +should lead. This is a *defensible default heuristic*, not a proven +optimum — primacy vs. recency weighting in long context is an open +empirical question. What we guarantee is **determinism and consistency**; +the ordering can be retuned later as a localized change. + +> Example: base has `SYSTEM.md` + `APPEND_SYSTEM.md`; user has +> `APPEND_SYSTEM.md`; project has `SYSTEM.md` + `APPEND_SYSTEM.md`. +> Resolved blocks, in emission order: +> 1. seed = project `SYSTEM.md` (highest layer wins) +> 2. project `APPEND_SYSTEM.md` +> 3. user `APPEND_SYSTEM.md` +> 4. base `APPEND_SYSTEM.md` + +### 2.4 Fresh session + +Seed the conversation with the resolved blocks (§2.3) and append matching +entries to the session log: + +- Seed → `addSystemMessage` (an `append`-mode block; nothing precedes it + so it is effectively the foundation). +- Each append → `addSystemMessage` in the resolved order. + +### 2.5 Resume reconciliation + +`SYSTEM.md` / `APPEND_SYSTEM.md` are moving targets: they can change +between the session that created a log and a later resume. On resume we +re-consult configuration and, if it has changed, append new log entries so +the conversation continues under the updated prompt — **without rewriting +history** (preserving `/tree` faithfulness) and **without clobbering +extension-authored prompt edits**. + +**Positional comparison (no provenance tag).** Resolve the current config +blocks per §2.3: this yields an ordered list — `[seed, append₁, …, +appendₙ]` of length `K`. Compare these, **by position and exact text**, +against the **current effective config window**: the last `replace`-mode +`.System` block in the rebuilt conversation, plus every `.System` block +after it (i.e. the system blocks from the most recent re-seed onward). + +> **Why not the *first* `K` blocks?** Reconciliation appends a +> `replace + N×append` sequence to the log. A second resume must compare +> against *that* sequence, not the session's original seed — otherwise a +> no-op resume after a prior reconciliation would mismatch the stale +> original blocks and needlessly re-replace on every load. Anchoring to +> the latest `replace` makes a no-op resume a true no-op. (A fresh +> session has no `replace` block; the window is then the leading system +> blocks from the start — the original seed sequence, the intended +> behavior for a first resume.) + +- **All `K` match (text-equal, in order):** config is unchanged relative + to the current effective prompt. Do nothing. +- **Any difference (any of the `K` positions differs, or the window has a + different number of blocks):** append a fresh reconciliation sequence to + the log: + 1. one `replace`-mode entry carrying the current seed text, then + 2. one `append`-mode entry per current append block, in resolved order. + + Because the leading entry is `replace`, the derivation (§1.3) discards + the stale config blocks *and* any earlier system text, then re-applies + the current config prompt. Extension-authored system edits that occurred + *after* the original config blocks are **also** discarded by the + `replace` — this is the accepted semantics: a config change re-seeds the + prompt wholesale. (Extensions that need to survive a config re-seed can + re-apply their edit on the next turn.) + +Comparing the current effective window positionally is deliberately simple +and needs no extra log surface (no provenance/source field). A provenance +flag would buy the freedom to, e.g., change append ordering without +triggering reconciliation, but that flexibility is explicitly **not** +wanted right now — consistent ordering is the contract. + +> Note: this means changing `APPEND_SYSTEM.md` ordering or content always +> triggers a full `replace + N×append` on the next resume. That is +> intended. + +## Out of scope (tracked separately) + +- Rewriting the built-in default system prompt. +- CLI flags (`--system-prompt`, etc.). Convention-first; flags can be + layered on later if a one-off override is wanted. +- The `/tree` command itself — this design only guarantees the data model + can support it. + +## Implementation order + +1. ✅ **libpanto data model:** `SystemMode`, `SystemBlock`, `ContentBlock` + arm + `deinit`; `addSystemMessage` (mode-aware) + `replaceSystemMessage`; + shared `effectiveSystemBlocks` derivation helper. +2. ✅ **Serializers:** OpenAI leading-system hoist (separate messages); + Anthropic strip-trailing-newlines + `\n\n---\n\n` join. Both via the + shared derivation. Tests added. +3. ✅ **Session store:** optional `mode` field on system entries (read + + write, default `append`); rebuild reconstructs `.System` blocks. Tests. +4. ✅ **CLI sourcing** (`src/system_prompt.zig`): discover `SYSTEM.md` / + `APPEND_SYSTEM.md` across the three layers (base = `$PANTO_HOME/agent`, + user = `${XDG_CONFIG_HOME:-$HOME/.config}/panto`, project = `./.panto`); + resolve + order per §2.2–§2.3; seed fresh sessions. +5. ✅ **CLI resume reconciliation:** positional comparison against the + current effective config window (anchored to the latest `replace`); + append `replace + N×append` on any difference. Tests. + +> Note: the base layer directory is `$PANTO_HOME/agent` (i.e. +> `$XDG_DATA_HOME/panto/agent`), matching the convention used for extension +> and tool discovery, rather than the bare `$XDG_DATA_HOME/panto/` sketched +> in §2.1. diff --git a/docs/system-prompt.md b/docs/system-prompt.md deleted file mode 100644 index df296f6..0000000 --- a/docs/system-prompt.md +++ /dev/null @@ -1,289 +0,0 @@ -# System Prompt - -Status: design / not yet implemented. - -This document specifies first-class support for the system prompt across -libpanto (the conversation + session-store data model and provider -serializers) and the `panto` CLI (file-based sourcing across config -layers, plus reconciliation on resume). - -Today there is no system-prompt API at all: the CLI hard-codes -`"You are a helpful assistant."` and seeds it as a single `.system`-role -message (`src/main.zig`). This design replaces that with a model where the -system prompt is conversation data that can **change over the life of a -conversation** and is **faithfully reconstructable at any earlier point in -time** (a prerequisite for a future pi-style `/tree` command). - -## Principles - -- **The system prompt is conversation state, not config state.** It lives - in the conversation and the session log, never in the per-turn `Config` - snapshot. The agent does not own or inject it. -- **It changes over time, and the log records that change.** Every - mutation is its own positioned log entry. Truncating the message list at - position _N_ and re-deriving the prompt yields exactly the prompt as it - was at _N_. This is what makes `/tree` faithful. -- **Append-only history, replace-capable semantics.** Mutations only ever - *append* log entries. A mutation may carry `mode = replace`, which means - "from here on, discard all prior system text," but it does so by adding a - new entry — it never rewrites history. -- **Convention over configuration in the CLI.** No new TOML keys. The - prompt is sourced from `SYSTEM.md` / `APPEND_SYSTEM.md` files discovered - across the existing config layers. - -## Part 1 — libpanto / session-store data model - -### 1.1 A `.System` content block with a mode - -System prompts remain `.system`-**role** messages. What changes is the -*content block*: instead of a plain `.Text` block, a system message -carries a new `System` content-block variant that records its mode. - -```zig -pub const SystemBlock = struct { - text: TextualBlock = .empty, - mode: SystemMode = .append, - - pub fn deinit(self: *SystemBlock, alloc: Allocator) void { - self.text.deinit(alloc); - } -}; - -pub const SystemMode = enum { append, replace }; - -pub const ContentBlock = union(enum) { - Text: TextualBlock, - Thinking: ThinkingBlock, - ToolUse: ToolUseBlock, - ToolResult: ToolResultBlock, - System: SystemBlock, // new - - // deinit gains a `.System => |*b| b.deinit(alloc)` arm. -}; -``` - -Rationale for a distinct block (rather than a `mode` field hung off -`Message`): the mode is only meaningful for system content. Putting it on -`Message` would leave a meaningless field on every user/assistant/tool -message. Keeping the `.system` *role* (option (a) from design discussion) -keeps the blast radius minimal: serializers and the session manager -already filter on `role == .system`; only the block payload grows. - -### 1.2 Conversation methods - -```zig -/// Append a system message in `append` mode. Adds to the effective -/// system prompt. (Back-compatible: same external behavior as today.) -pub fn addSystemMessage(self: *Conversation, text: []const u8) !void; - -/// Append a system message in `replace` mode. When the effective prompt -/// is rebuilt, this discards all prior system text and starts fresh. -pub fn replaceSystemMessage(self: *Conversation, text: []const u8) !void; -``` - -Both append a `.system`-role message whose single content block is a -`.System` block; they differ only in the recorded `mode`. Both are -available to extensions, so an extension can grow *or* wholesale replace -the system prompt at any point mid-conversation. - -### 1.3 Deriving the effective system prompt - -A single shared rule governs both provider serializers and session -rebuild. Walk the conversation messages in order; for each `.system` -message's `.System` block: - -- `append`: add the block's text to the running list of effective system - blocks. -- `replace`: **clear the running list**, then add this block's text. - -The result is an ordered list of surviving system-text blocks. "Replace -means replace" — it wipes everything collected so far, regardless of how -those earlier blocks were joined. - -Because each mutation is a positioned entry, this same walk over a -*prefix* of the messages reconstructs the prompt as of that point — the -`/tree` property. - -### 1.4 Provider serialization - -**Anthropic (`anthropic_messages_json.zig`).** The wire format requires a -single top-level `system` string. `collectSystemPrompt` already walks all -`.system` messages; update it to: - -1. Apply the append/replace derivation above. -2. For each surviving block, **strip trailing newlines**. -3. **Join with `\n\n---\n\n`** (double-newline / horizontal rule / - double-newline), instead of the current single `\n`. - -Emit the joined string as the top-level `system` field (omit the field -entirely if empty, as today). - -**OpenAI (`openai_chat_json.zig`).** Today `.system` messages are emitted -positionally as ordinary messages. Change to: - -1. Apply the append/replace derivation. -2. Emit the surviving system blocks as **separate leading `system`-role - messages**, in order, before any non-system message. -3. Emit all non-system messages in their original order afterward. - -Do **not** concatenate OpenAI system messages into one. Keeping them as -separate, individually-positioned messages preserves block-level -addressability (again, for `/tree`-style truncation). The double-rule -join is an Anthropic-only concession to its single-string wire format. - -### 1.5 Session log format - -`system`-role log objects gain an optional `mode` field: - -```json -{"...","message":{"role":"system","mode":"replace","content":[{"type":"text","text":"..."}]}} -``` - -- `mode` is `"append"` or `"replace"`. -- **Absent `mode` defaults to `"append"`** — existing logs read back - identically, no migration needed. - -On write, the session manager records the block's mode. On -`rebuildConversation`, it reads `mode` (defaulting to `append`) and -reconstructs the corresponding `.System` block. - -## Part 2 — `panto` CLI: sourcing & reconciliation - -### 2.1 File discovery across layers - -The system prompt is sourced from files discovered across the same three -layers the CLI already uses for config / extensions / tools, in -precedence order **base → user → project** (project highest): - -| Layer | Directory | -|---------|------------------------------------------------------| -| base | `${XDG_DATA_HOME:-$HOME/.local/share}/panto/` | -| user | `${XDG_CONFIG_HOME:-$HOME/.config}/panto/` | -| project | `./.panto/` | - -In each layer we look for two files: - -- **`SYSTEM.md`** — the base/seed system prompt. -- **`APPEND_SYSTEM.md`** — an additional appended system block. - -There is **no TOML key** for the system prompt. Convention only. - -### 2.2 Resolution rules - -- **`SYSTEM.md`:** the **highest layer present wins** (whole-file - override, matching how scalar config values already override across - layers). The winning file's content becomes the **seed** system block. -- **`APPEND_SYSTEM.md`:** **every** layer's file is respected; each - becomes its own appended system block. -- **Built-in default:** if no `SYSTEM.md` exists at any layer, fall back - to a built-in default seed. (The current default, - `"You are a helpful assistant."`, needs a rewrite — tracked separately.) - -### 2.3 Ordering of the resolved blocks - -The resolved sequence of system blocks for a fresh session is: - -1. **Seed** (`SYSTEM.md` winner, or built-in default) — emitted first, so - it reads earliest / highest-salience. -2. **Appends**, collected base → user → project, then **emitted in - reversed order: project → user → base.** - -Reversing the appends places the project-layer append earliest among the -appends (right after the seed). The working hypothesis is that LLMs weight -earlier prompt text more heavily, so the most-specific (project) layer -should lead. This is a *defensible default heuristic*, not a proven -optimum — primacy vs. recency weighting in long context is an open -empirical question. What we guarantee is **determinism and consistency**; -the ordering can be retuned later as a localized change. - -> Example: base has `SYSTEM.md` + `APPEND_SYSTEM.md`; user has -> `APPEND_SYSTEM.md`; project has `SYSTEM.md` + `APPEND_SYSTEM.md`. -> Resolved blocks, in emission order: -> 1. seed = project `SYSTEM.md` (highest layer wins) -> 2. project `APPEND_SYSTEM.md` -> 3. user `APPEND_SYSTEM.md` -> 4. base `APPEND_SYSTEM.md` - -### 2.4 Fresh session - -Seed the conversation with the resolved blocks (§2.3) and append matching -entries to the session log: - -- Seed → `addSystemMessage` (an `append`-mode block; nothing precedes it - so it is effectively the foundation). -- Each append → `addSystemMessage` in the resolved order. - -### 2.5 Resume reconciliation - -`SYSTEM.md` / `APPEND_SYSTEM.md` are moving targets: they can change -between the session that created a log and a later resume. On resume we -re-consult configuration and, if it has changed, append new log entries so -the conversation continues under the updated prompt — **without rewriting -history** (preserving `/tree` faithfulness) and **without clobbering -extension-authored prompt edits**. - -**Positional comparison (no provenance tag).** Resolve the current config -blocks per §2.3: this yields an ordered list — `[seed, append₁, …, -appendₙ]` of length `K`. Compare these, **by position and exact text**, -against the **current effective config window**: the last `replace`-mode -`.System` block in the rebuilt conversation, plus every `.System` block -after it (i.e. the system blocks from the most recent re-seed onward). - -> **Why not the *first* `K` blocks?** Reconciliation appends a -> `replace + N×append` sequence to the log. A second resume must compare -> against *that* sequence, not the session's original seed — otherwise a -> no-op resume after a prior reconciliation would mismatch the stale -> original blocks and needlessly re-replace on every load. Anchoring to -> the latest `replace` makes a no-op resume a true no-op. (A fresh -> session has no `replace` block; the window is then the leading system -> blocks from the start — the original seed sequence, the intended -> behavior for a first resume.) - -- **All `K` match (text-equal, in order):** config is unchanged relative - to the current effective prompt. Do nothing. -- **Any difference (any of the `K` positions differs, or the window has a - different number of blocks):** append a fresh reconciliation sequence to - the log: - 1. one `replace`-mode entry carrying the current seed text, then - 2. one `append`-mode entry per current append block, in resolved order. - - Because the leading entry is `replace`, the derivation (§1.3) discards - the stale config blocks *and* any earlier system text, then re-applies - the current config prompt. Extension-authored system edits that occurred - *after* the original config blocks are **also** discarded by the - `replace` — this is the accepted semantics: a config change re-seeds the - prompt wholesale. (Extensions that need to survive a config re-seed can - re-apply their edit on the next turn.) - -Comparing the current effective window positionally is deliberately simple -and needs no extra log surface (no provenance/source field). A provenance -flag would buy the freedom to, e.g., change append ordering without -triggering reconciliation, but that flexibility is explicitly **not** -wanted right now — consistent ordering is the contract. - -> Note: this means changing `APPEND_SYSTEM.md` ordering or content always -> triggers a full `replace + N×append` on the next resume. That is -> intended. - -## Out of scope (tracked separately) - -- Rewriting the built-in default system prompt. -- CLI flags (`--system-prompt`, etc.). Convention-first; flags can be - layered on later if a one-off override is wanted. -- The `/tree` command itself — this design only guarantees the data model - can support it. - -## Implementation order - -1. **libpanto data model:** `SystemMode`, `SystemBlock`, `ContentBlock` - arm + `deinit`; `addSystemMessage` (mode-aware) + `replaceSystemMessage`; - shared append/replace derivation helper. -2. **Serializers:** OpenAI leading-system hoist (separate messages); - Anthropic strip-trailing-newlines + `\n\n---\n\n` join. Both via the - shared derivation. Add/extend tests. -3. **Session store:** optional `mode` field on system entries (read + - write, default `append`); rebuild reconstructs `.System` blocks. Tests. -4. **CLI sourcing:** discover `SYSTEM.md` / `APPEND_SYSTEM.md` across the - three layers; resolve + order per §2.2–§2.3; seed fresh sessions. -5. **CLI resume reconciliation:** positional first-`K` comparison; append - `replace + N×append` on any difference. Tests. -- cgit v1.3