path: root/docs/system-prompt.md

# System Prompt

Status: design / not yet implemented.

This document specifies first-class support for the system prompt across
libpanto (the conversation + session-store data model and provider
serializers) and the `panto` CLI (file-based sourcing across config
layers, plus reconciliation on resume).

Today there is no system-prompt API at all: the CLI hard-codes
`"You are a helpful assistant."` and seeds it as a single `.system`-role
message (`src/main.zig`). This design replaces that with a model where the
system prompt is conversation data that can **change over the life of a
conversation** and is **faithfully reconstructable at any earlier point in
time** (a prerequisite for a future pi-style `/tree` command).

## Principles

- **The system prompt is conversation state, not config state.** It lives
  in the conversation and the session log, never in the per-turn `Config`
  snapshot. The agent does not own or inject it.
- **It changes over time, and the log records that change.** Every
  mutation is its own positioned log entry. Truncating the message list at
  position _N_ and re-deriving the prompt yields exactly the prompt as it
  was at _N_. This is what makes `/tree` faithful.
- **Append-only history, replace-capable semantics.** Mutations only ever
  *append* log entries. A mutation may carry `mode = replace`, which means
  "from here on, discard all prior system text," but it does so by adding a
  new entry — it never rewrites history.
- **Convention over configuration in the CLI.** No new TOML keys. The
  prompt is sourced from `SYSTEM.md` / `APPEND_SYSTEM.md` files discovered
  across the existing config layers.

## Part 1 — libpanto / session-store data model

### 1.1 A `.System` content block with a mode

System prompts remain `.system`-**role** messages. What changes is the
*content block*: instead of a plain `.Text` block, a system message
carries a new `System` content-block variant that records its mode.

```zig
pub const SystemBlock = struct {
    text: TextualBlock = .empty,
    mode: SystemMode = .append,

    pub fn deinit(self: *SystemBlock, alloc: Allocator) void {
        self.text.deinit(alloc);
    }
};

pub const SystemMode = enum { append, replace };

pub const ContentBlock = union(enum) {
    Text: TextualBlock,
    Thinking: ThinkingBlock,
    ToolUse: ToolUseBlock,
    ToolResult: ToolResultBlock,
    System: SystemBlock,   // new

    // deinit gains a `.System => |*b| b.deinit(alloc)` arm.
};
```

Rationale for a distinct block (rather than a `mode` field hung off
`Message`): the mode is only meaningful for system content. Putting it on
`Message` would leave a meaningless field on every user/assistant/tool
message. Keeping the `.system` *role* (option (a) from design discussion)
keeps the blast radius minimal: serializers and the session manager
already filter on `role == .system`; only the block payload grows.

### 1.2 Conversation methods

```zig
/// Append a system message in `append` mode. Adds to the effective
/// system prompt. (Back-compatible: same external behavior as today.)
pub fn addSystemMessage(self: *Conversation, text: []const u8) !void;

/// Append a system message in `replace` mode. When the effective prompt
/// is rebuilt, this discards all prior system text and starts fresh.
pub fn replaceSystemMessage(self: *Conversation, text: []const u8) !void;
```

Both append a `.system`-role message whose single content block is a
`.System` block; they differ only in the recorded `mode`. Both are
available to extensions, so an extension can grow *or* wholesale replace
the system prompt at any point mid-conversation.

### 1.3 Deriving the effective system prompt

A single shared rule governs both provider serializers and session
rebuild. Walk the conversation messages in order; for each `.system`
message's `.System` block:

- `append`: add the block's text to the running list of effective system
  blocks.
- `replace`: **clear the running list**, then add this block's text.

The result is an ordered list of surviving system-text blocks. "Replace
means replace" — it wipes everything collected so far, regardless of how
those earlier blocks were joined.

Because each mutation is a positioned entry, this same walk over a
*prefix* of the messages reconstructs the prompt as of that point — the
`/tree` property.

### 1.4 Provider serialization

**Anthropic (`anthropic_messages_json.zig`).** The wire format requires a
single top-level `system` string. `collectSystemPrompt` already walks all
`.system` messages; update it to:

1. Apply the append/replace derivation above.
2. For each surviving block, **strip trailing newlines**.
3. **Join with `\n\n---\n\n`** (double-newline / horizontal rule /
   double-newline), instead of the current single `\n`.

Emit the joined string as the top-level `system` field (omit the field
entirely if empty, as today).

**OpenAI (`openai_chat_json.zig`).** Today `.system` messages are emitted
positionally as ordinary messages. Change to:

1. Apply the append/replace derivation.
2. Emit the surviving system blocks as **separate leading `system`-role
   messages**, in order, before any non-system message.
3. Emit all non-system messages in their original order afterward.

Do **not** concatenate OpenAI system messages into one. Keeping them as
separate, individually-positioned messages preserves block-level
addressability (again, for `/tree`-style truncation). The double-rule
join is an Anthropic-only concession to its single-string wire format.

### 1.5 Session log format

`system`-role log objects gain an optional `mode` field:

```json
{"...","message":{"role":"system","mode":"replace","content":[{"type":"text","text":"..."}]}}
```

- `mode` is `"append"` or `"replace"`.
- **Absent `mode` defaults to `"append"`** — existing logs read back
  identically, no migration needed.

On write, the session manager records the block's mode. On
`rebuildConversation`, it reads `mode` (defaulting to `append`) and
reconstructs the corresponding `.System` block.

## Part 2 — `panto` CLI: sourcing & reconciliation

### 2.1 File discovery across layers

The system prompt is sourced from files discovered across the same three
layers the CLI already uses for config / extensions / tools, in
precedence order **base → user → project** (project highest):

| Layer   | Directory                                            |
|---------|------------------------------------------------------|
| base    | `${XDG_DATA_HOME:-$HOME/.local/share}/panto/`        |
| user    | `${XDG_CONFIG_HOME:-$HOME/.config}/panto/`           |
| project | `./.panto/`                                           |

In each layer we look for two files:

- **`SYSTEM.md`** — the base/seed system prompt.
- **`APPEND_SYSTEM.md`** — an additional appended system block.

There is **no TOML key** for the system prompt. Convention only.

### 2.2 Resolution rules

- **`SYSTEM.md`:** the **highest layer present wins** (whole-file
  override, matching how scalar config values already override across
  layers). The winning file's content becomes the **seed** system block.
- **`APPEND_SYSTEM.md`:** **every** layer's file is respected; each
  becomes its own appended system block.
- **Built-in default:** if no `SYSTEM.md` exists at any layer, fall back
  to a built-in default seed. (The current default,
  `"You are a helpful assistant."`, needs a rewrite — tracked separately.)

### 2.3 Ordering of the resolved blocks

The resolved sequence of system blocks for a fresh session is:

1. **Seed** (`SYSTEM.md` winner, or built-in default) — emitted first, so
   it reads earliest / highest-salience.
2. **Appends**, collected base → user → project, then **emitted in
   reversed order: project → user → base.**

Reversing the appends places the project-layer append earliest among the
appends (right after the seed). The working hypothesis is that LLMs weight
earlier prompt text more heavily, so the most-specific (project) layer
should lead. This is a *defensible default heuristic*, not a proven
optimum — primacy vs. recency weighting in long context is an open
empirical question. What we guarantee is **determinism and consistency**;
the ordering can be retuned later as a localized change.

> Example: base has `SYSTEM.md` + `APPEND_SYSTEM.md`; user has
> `APPEND_SYSTEM.md`; project has `SYSTEM.md` + `APPEND_SYSTEM.md`.
> Resolved blocks, in emission order:
> 1. seed = project `SYSTEM.md` (highest layer wins)
> 2. project `APPEND_SYSTEM.md`
> 3. user `APPEND_SYSTEM.md`
> 4. base `APPEND_SYSTEM.md`

### 2.4 Fresh session

Seed the conversation with the resolved blocks (§2.3) and append matching
entries to the session log:

- Seed → `addSystemMessage` (an `append`-mode block; nothing precedes it
  so it is effectively the foundation).
- Each append → `addSystemMessage` in the resolved order.

### 2.5 Resume reconciliation

`SYSTEM.md` / `APPEND_SYSTEM.md` are moving targets: they can change
between the session that created a log and a later resume. On resume we
re-consult configuration and, if it has changed, append new log entries so
the conversation continues under the updated prompt — **without rewriting
history** (preserving `/tree` faithfulness) and **without clobbering
extension-authored prompt edits**.

**Positional comparison (no provenance tag).** Resolve the current config
blocks per §2.3: this yields an ordered list — `[seed, append₁, …,
appendₙ]` of length `K`. Compare these, **by position and exact text**,
against the **current effective config window**: the last `replace`-mode
`.System` block in the rebuilt conversation, plus every `.System` block
after it (i.e. the system blocks from the most recent re-seed onward).

> **Why not the *first* `K` blocks?** Reconciliation appends a
> `replace + N×append` sequence to the log. A second resume must compare
> against *that* sequence, not the session's original seed — otherwise a
> no-op resume after a prior reconciliation would mismatch the stale
> original blocks and needlessly re-replace on every load. Anchoring to
> the latest `replace` makes a no-op resume a true no-op. (A fresh
> session has no `replace` block; the window is then the leading system
> blocks from the start — the original seed sequence, the intended
> behavior for a first resume.)

- **All `K` match (text-equal, in order):** config is unchanged relative
  to the current effective prompt. Do nothing.
- **Any difference (any of the `K` positions differs, or the window has a
  different number of blocks):** append a fresh reconciliation sequence to
  the log:
  1. one `replace`-mode entry carrying the current seed text, then
  2. one `append`-mode entry per current append block, in resolved order.

  Because the leading entry is `replace`, the derivation (§1.3) discards
  the stale config blocks *and* any earlier system text, then re-applies
  the current config prompt. Extension-authored system edits that occurred
  *after* the original config blocks are **also** discarded by the
  `replace` — this is the accepted semantics: a config change re-seeds the
  prompt wholesale. (Extensions that need to survive a config re-seed can
  re-apply their edit on the next turn.)

Comparing the current effective window positionally is deliberately simple
and needs no extra log surface (no provenance/source field). A provenance
flag would buy the freedom to, e.g., change append ordering without
triggering reconciliation, but that flexibility is explicitly **not**
wanted right now — consistent ordering is the contract.

> Note: this means changing `APPEND_SYSTEM.md` ordering or content always
> triggers a full `replace + N×append` on the next resume. That is
> intended.

## Out of scope (tracked separately)

- Rewriting the built-in default system prompt.
- CLI flags (`--system-prompt`, etc.). Convention-first; flags can be
  layered on later if a one-off override is wanted.
- The `/tree` command itself — this design only guarantees the data model
  can support it.

## Implementation order

1. **libpanto data model:** `SystemMode`, `SystemBlock`, `ContentBlock`
   arm + `deinit`; `addSystemMessage` (mode-aware) + `replaceSystemMessage`;
   shared append/replace derivation helper.
2. **Serializers:** OpenAI leading-system hoist (separate messages);
   Anthropic strip-trailing-newlines + `\n\n---\n\n` join. Both via the
   shared derivation. Add/extend tests.
3. **Session store:** optional `mode` field on system entries (read +
   write, default `append`); rebuild reconstructs `.System` blocks. Tests.
4. **CLI sourcing:** discover `SYSTEM.md` / `APPEND_SYSTEM.md` across the
   three layers; resolve + order per §2.2–§2.3; seed fresh sessions.
5. **CLI resume reconciliation:** positional first-`K` comparison; append
   `replace + N×append` on any difference. Tests.