path: root/docs/compaction-v1.md

# Compaction v1

## Goals

Compaction reduces context size by summarizing older conversation history into a synthetic seed message.

v1 goals:
- keep the runtime model simple and robust
- preserve coherent recent context verbatim
- avoid depending on user-configured context window sizes for correctness
- support a separate compaction prompt and optional compaction model override
- recover automatically from context overflow by compacting and retrying once

Non-goals for v1:
- proactive compaction based on configured context window thresholds
- splitting oversized turns
- side-channel metadata such as tracked files outside the summary text
- pre-truncating tool results before compaction
- exact tokenizer-based sizing for every provider

## High-level model

Compaction uses a separate system prompt from normal agent execution.

The old portion of the conversation is converted into a transcript artifact and sent to the model as the user prompt for a compaction request. The model returns a summary. That summary is stored as a first-class compaction content block in a new effective context, followed by a recent suffix of verbatim conversation turns.

So after compaction, the effective context is:
1. normal agent system prompt
2. one compaction content block carrying the summary/seed
3. recent verbatim turns kept from the original conversation

This is prefix compaction, not a total reset.

Compaction replay is reset-like:
- when replaying history, a compaction block clears earlier effective context
- only the latest compaction block and the messages after it contribute to active context
- therefore any verbatim turns that should survive compaction must be duplicated after the compaction block when the compaction entry is written

## Triggering

### Automatic compaction

Auto-compaction triggers only after a prompt attempt fails because the provider rejected the request as too large.

Flow:
1. send normal request
2. provider rejects it for context/input length
3. compact the conversation
4. retry the failed request once against the compacted context

This avoids relying on configured context window sizes for correctness.

### Manual compaction

`/compact`

Manual compaction runs immediately against the current conversation.

`/compact $ARGUMENTS`

Arguments are treated as additional instructions for that compaction run. They are appended to the compaction system prompt, not treated as conversation content.

Example:

```text
/compact be sure and carefully keep our full understanding of bugs #3 and #4
```

## Prompting model

### Compaction prompt file

Compaction uses a dedicated prompt file:

- base/default prompt
- user-level override
- project-level override
- last one wins

The file name is:

- `COMPACTION.md`

This file acts as the system prompt for compaction.

### Manual compaction instructions

For `/compact $ARGUMENTS`, panto appends a section to the compaction system prompt such as:

```md
## Additional instructions for this compaction run

...
```

### User prompt for compaction

The compaction request user prompt is just the transcript artifact, optionally wrapped in a tiny fixed envelope.

The transcript is treated as material to summarize, not as live chat history to continue.

On repeated compactions, the request may also include the previous compaction summary in a separate tagged section such as:

```text
<previous-summary>
...
</previous-summary>
```

This is not an accumulating list of all historical summaries. Each new compaction run receives at most one previous summary: the latest one currently active in context.

## Transcript serialization

The summarized portion of the conversation is serialized into plain text.

The exact formatting is flexible, but it should clearly mark message roles and important structure, e.g.:

```text
[User]: ...
[Assistant]: ...
[Assistant tool calls]: ...
[Tool result]: ...
```

Rationale:
- keeps the compaction task clearly distinct from ongoing conversation
- makes the summarized material feel like an artifact under analysis
- avoids needing to replay old messages as normal chat turns during the compaction request

v1 does not pre-truncate tool results specifically for compaction.

## Retention policy

Compaction preserves a recent suffix of conversation turns verbatim.

### Unit of retention

The retention unit is a whole turn.

A turn is the full request/response cycle needed to preserve protocol correctness. In practice, compaction logic should treat whole API message units conservatively and never split a turn.

### Turn-boundary rule

Compaction walks backward through whole recent turns and keeps the longest suffix that fits the keep-verbatim budget.

Everything before that suffix is summarized.

If adding one more whole turn would exceed the budget, that turn is not kept verbatim. It is included in the summarized prefix instead.

v1 never splits a turn.

### Why keep a suffix

Keeping a recent verbatim suffix provides a gradient:
- older history is compressed aggressively
- recent history remains concrete and detailed
- the post-compaction context still feels like a continuation of the current conversation

## Compaction content block

The compaction result is stored as one first-class compaction content block in the effective context.

That compaction block is the seed that stands in for the compacted prefix.

This should be modeled as a distinct content block type rather than ordinary user prose. Like a replacing system prompt part, a compaction block changes replay semantics: it resets prior effective conversation state, then later content is appended after it.

## Chained compactions

Repeated compactions produce a chain of replacement summaries:

- compaction 1 produces `S1`
- compaction 2 summarizes new old material plus `S1` as `<previous-summary>`, producing `S2`
- compaction 3 summarizes newer old material plus `S2` as `<previous-summary>`, producing `S3`

So the active chain is always represented by one latest summary, not an ever-growing stack of historical summaries.

This gives panto a clean invariant:
- one latest compaction block is sufficient to represent all prior compacted history
- kept verbatim turns are replayed after that block
- older compaction blocks remain in persisted history but do not contribute to current effective context once a newer compaction block supersedes them

## Compaction model selection

By default, compaction uses the active chat model.

v1 also supports an optional config override:

- `compaction_model`

If set, panto first tries the configured compaction model.

If compaction fails, panto falls back to the active chat model.

This fallback is primarily for cases like:
- compaction model rejects the request for context length
- compaction model is unavailable
- compaction model call otherwise fails

The active chat model is the reliability fallback because it already accepted nearly all of the current conversation shape in normal use.

v1 does not try to predict whether the compaction model will fit ahead of time.

## Sizing and budgeting

### Keep-verbatim budget

Compaction needs a budget for how much recent conversation to keep verbatim.

This budget is used only to choose the kept suffix versus summarized prefix.

### Preferred size source

When usage data is available, panto should use provider-reported usage to infer message or turn sizes.

Useful facts:
- assistant responses often provide exact `output_tokens`
- assistant responses may also provide exact `input_tokens`
- adjacent assistant usage records can be used to infer the size of the material added between them

This is good enough to drive whole-turn retention decisions.

### Fallback size source

Some providers do not provide usable token accounting.

When exact or inferred token usage is unavailable, panto falls back to a cheap monotonic heuristic such as word count.

This fallback is not exact, but it is sufficient for choosing a recent suffix conservatively.

## Correctness rules

1. Never split a turn.
2. Never leave protocol-dependent fragments dangling.
3. A compaction block resets earlier effective context during replay.
4. Kept verbatim turns that survive compaction must be replayed after the compaction block.
5. Repeated compactions pass only the latest active summary as `<previous-summary>`, not a list of all prior summaries.
6. After automatic compaction on overflow, retry the failed request once.
7. If compaction with `compaction_model` fails, retry compaction once with the active model.
8. If sizing data is missing or poor, prefer conservative retention logic.

## Error handling

### Overflow recovery

On provider context-overflow error:
1. stop normal request handling
2. run compaction
3. rebuild effective context from seed message plus kept suffix
4. retry the user request once

If the retry also fails for context size, surface the error.

### Compaction model failure

If a configured compaction model fails, retry compaction once with the active model.

### Missing usage data

If token usage data is unavailable, use fallback sizing heuristics.

## Configuration surface

Initial v1 configuration:
- `compaction_model` optional override
- keep-verbatim budget

Configured context window sizes may still be useful for UI display, but they are not used for correctness-critical auto-compaction behavior in v1.

## Open questions

1. Exact keep-verbatim budget name and units.
2. Exact transcript serialization format.
3. Exact wire/storage shape of the compaction content block.
4. Exact `<previous-summary>` prompt wording and placement.
5. Exact overflow-error detection mapping across providers.
6. Whether manual `/compact` should default to the active model even when `compaction_model` is configured, or use the same selection/fallback logic as automatic compaction.