compaction v1 doc

1 files changed, 276 insertions, 0 deletions

diff --git a/docs/compaction-v1.md b/docs/compaction-v1.md
new file mode 100644
index 0000000..481dcac
--- /dev/null
+++ b/docs/compaction-v1.md
@@ -0,0 +1,276 @@
+# Compaction v1
+
+## Goals
+
+Compaction reduces context size by summarizing older conversation history into a synthetic seed message.
+
+v1 goals:
+- keep the runtime model simple and robust
+- preserve coherent recent context verbatim
+- avoid depending on user-configured context window sizes for correctness
+- support a separate compaction prompt and optional compaction model override
+- recover automatically from context overflow by compacting and retrying once
+
+Non-goals for v1:
+- proactive compaction based on configured context window thresholds
+- splitting oversized turns
+- side-channel metadata such as tracked files outside the summary text
+- pre-truncating tool results before compaction
+- exact tokenizer-based sizing for every provider
+
+## High-level model
+
+Compaction uses a separate system prompt from normal agent execution.
+
+The old portion of the conversation is converted into a transcript artifact and sent to the model as the user prompt for a compaction request. The model returns a summary. That summary is stored as a first-class compaction content block in a new effective context, followed by a recent suffix of verbatim conversation turns.
+
+So after compaction, the effective context is:
+1. normal agent system prompt
+2. one compaction content block carrying the summary/seed
+3. recent verbatim turns kept from the original conversation
+
+This is prefix compaction, not a total reset.
+
+Compaction replay is reset-like:
+- when replaying history, a compaction block clears earlier effective context
+- only the latest compaction block and the messages after it contribute to active context
+- therefore any verbatim turns that should survive compaction must be duplicated after the compaction block when the compaction entry is written
+
+## Triggering
+
+### Automatic compaction
+
+Auto-compaction triggers only after a prompt attempt fails because the provider rejected the request as too large.
+
+Flow:
+1. send normal request
+2. provider rejects it for context/input length
+3. compact the conversation
+4. retry the failed request once against the compacted context
+
+This avoids relying on configured context window sizes for correctness.
+
+### Manual compaction
+
+`/compact`
+
+Manual compaction runs immediately against the current conversation.
+
+`/compact $ARGUMENTS`
+
+Arguments are treated as additional instructions for that compaction run. They are appended to the compaction system prompt, not treated as conversation content.
+
+Example:
+
+```text
+/compact be sure and carefully keep our full understanding of bugs #3 and #4
+```
+
+## Prompting model
+
+### Compaction prompt file
+
+Compaction uses a dedicated prompt file:
+
+- base/default prompt
+- user-level override
+- project-level override
+- last one wins
+
+The file name is:
+
+- `COMPACTION.md`
+
+This file acts as the system prompt for compaction.
+
+### Manual compaction instructions
+
+For `/compact $ARGUMENTS`, panto appends a section to the compaction system prompt such as:
+
+```md
+## Additional instructions for this compaction run
+
+...
+```
+
+### User prompt for compaction
+
+The compaction request user prompt is just the transcript artifact, optionally wrapped in a tiny fixed envelope.
+
+The transcript is treated as material to summarize, not as live chat history to continue.
+
+On repeated compactions, the request may also include the previous compaction summary in a separate tagged section such as:
+
+```text
+<previous-summary>
+...
+</previous-summary>
+```
+
+This is not an accumulating list of all historical summaries. Each new compaction run receives at most one previous summary: the latest one currently active in context.
+
+## Transcript serialization
+
+The summarized portion of the conversation is serialized into plain text.
+
+The exact formatting is flexible, but it should clearly mark message roles and important structure, e.g.:
+
+```text
+[User]: ...
+[Assistant]: ...
+[Assistant tool calls]: ...
+[Tool result]: ...
+```
+
+Rationale:
+- keeps the compaction task clearly distinct from ongoing conversation
+- makes the summarized material feel like an artifact under analysis
+- avoids needing to replay old messages as normal chat turns during the compaction request
+
+v1 does not pre-truncate tool results specifically for compaction.
+
+## Retention policy
+
+Compaction preserves a recent suffix of conversation turns verbatim.
+
+### Unit of retention
+
+The retention unit is a whole turn.
+
+A turn is the full request/response cycle needed to preserve protocol correctness. In practice, compaction logic should treat whole API message units conservatively and never split a turn.
+
+### Turn-boundary rule
+
+Compaction walks backward through whole recent turns and keeps the longest suffix that fits the keep-verbatim budget.
+
+Everything before that suffix is summarized.
+
+If adding one more whole turn would exceed the budget, that turn is not kept verbatim. It is included in the summarized prefix instead.
+
+v1 never splits a turn.
+
+### Why keep a suffix
+
+Keeping a recent verbatim suffix provides a gradient:
+- older history is compressed aggressively
+- recent history remains concrete and detailed
+- the post-compaction context still feels like a continuation of the current conversation
+
+## Compaction content block
+
+The compaction result is stored as one first-class compaction content block in the effective context.
+
+That compaction block is the seed that stands in for the compacted prefix.
+
+This should be modeled as a distinct content block type rather than ordinary user prose. Like a replacing system prompt part, a compaction block changes replay semantics: it resets prior effective conversation state, then later content is appended after it.
+
+## Chained compactions
+
+Repeated compactions produce a chain of replacement summaries:
+
+- compaction 1 produces `S1`
+- compaction 2 summarizes new old material plus `S1` as `<previous-summary>`, producing `S2`
+- compaction 3 summarizes newer old material plus `S2` as `<previous-summary>`, producing `S3`
+
+So the active chain is always represented by one latest summary, not an ever-growing stack of historical summaries.
+
+This gives panto a clean invariant:
+- one latest compaction block is sufficient to represent all prior compacted history
+- kept verbatim turns are replayed after that block
+- older compaction blocks remain in persisted history but do not contribute to current effective context once a newer compaction block supersedes them
+
+## Compaction model selection
+
+By default, compaction uses the active chat model.
+
+v1 also supports an optional config override:
+
+- `compaction_model`
+
+If set, panto first tries the configured compaction model.
+
+If compaction fails, panto falls back to the active chat model.
+
+This fallback is primarily for cases like:
+- compaction model rejects the request for context length
+- compaction model is unavailable
+- compaction model call otherwise fails
+
+The active chat model is the reliability fallback because it already accepted nearly all of the current conversation shape in normal use.
+
+v1 does not try to predict whether the compaction model will fit ahead of time.
+
+## Sizing and budgeting
+
+### Keep-verbatim budget
+
+Compaction needs a budget for how much recent conversation to keep verbatim.
+
+This budget is used only to choose the kept suffix versus summarized prefix.
+
+### Preferred size source
+
+When usage data is available, panto should use provider-reported usage to infer message or turn sizes.
+
+Useful facts:
+- assistant responses often provide exact `output_tokens`
+- assistant responses may also provide exact `input_tokens`
+- adjacent assistant usage records can be used to infer the size of the material added between them
+
+This is good enough to drive whole-turn retention decisions.
+
+### Fallback size source
+
+Some providers do not provide usable token accounting.
+
+When exact or inferred token usage is unavailable, panto falls back to a cheap monotonic heuristic such as word count.
+
+This fallback is not exact, but it is sufficient for choosing a recent suffix conservatively.
+
+## Correctness rules
+
+1. Never split a turn.
+2. Never leave protocol-dependent fragments dangling.
+3. A compaction block resets earlier effective context during replay.
+4. Kept verbatim turns that survive compaction must be replayed after the compaction block.
+5. Repeated compactions pass only the latest active summary as `<previous-summary>`, not a list of all prior summaries.
+6. After automatic compaction on overflow, retry the failed request once.
+7. If compaction with `compaction_model` fails, retry compaction once with the active model.
+8. If sizing data is missing or poor, prefer conservative retention logic.
+
+## Error handling
+
+### Overflow recovery
+
+On provider context-overflow error:
+1. stop normal request handling
+2. run compaction
+3. rebuild effective context from seed message plus kept suffix
+4. retry the user request once
+
+If the retry also fails for context size, surface the error.
+
+### Compaction model failure
+
+If a configured compaction model fails, retry compaction once with the active model.
+
+### Missing usage data
+
+If token usage data is unavailable, use fallback sizing heuristics.
+
+## Configuration surface
+
+Initial v1 configuration:
+- `compaction_model` optional override
+- keep-verbatim budget
+
+Configured context window sizes may still be useful for UI display, but they are not used for correctness-critical auto-compaction behavior in v1.
+
+## Open questions
+
+1. Exact keep-verbatim budget name and units.
+2. Exact transcript serialization format.
+3. Exact wire/storage shape of the compaction content block.
+4. Exact `<previous-summary>` prompt wording and placement.
+5. Exact overflow-error detection mapping across providers.
+6. Whether manual `/compact` should default to the active model even when `compaction_model` is configured, or use the same selection/fallback logic as automatic compaction.