failure retries scheme

1 files changed, 341 insertions, 0 deletions

diff --git a/docs/archive/retries.md b/docs/archive/retries.md
new file mode 100644
index 0000000..26aced1
--- /dev/null
+++ b/docs/archive/retries.md
@@ -0,0 +1,341 @@
+# Message-level error retries
+
+## Summary
+
+The agent loop should make an explicit decision for every failure that occurs while producing the next conversation message. Depending on the failure, the agent should either:
+
+1. retry the same provider request with backoff,
+2. compact/rewrite the conversation and retry the same provider request,
+3. append conversation-visible tool error results and continue the agent loop, or
+4. hard-fail and return the error to the embedder.
+
+The important distinction is between provider/API failures and tool-call failures:
+
+- Provider/API failures usually mean the assistant message was not produced. The correct recovery is to retry the same request without mutating the conversation.
+- Tool-call failures usually mean the model made a recoverable choice. The correct recovery is to append `ToolResult` blocks describing the failures, then let the model continue from there.
+
+This plan treats retries as **message-level**: a failed provider attempt should not create a partial duplicate message, and a tool failure should be represented as the next conversation message in the normal assistant/tool-result exchange.
+
+## Existing behavior
+
+We already have several special cases:
+
+- Context overflow errors are handled by automatic compaction followed by one retry of the same request.
+- Max-tokens overflow during tool argument generation can surface as a partially generated tool input; this is currently handled as invalid/incomplete JSON-ish tool input and sent back as a tool result.
+- Some tool-result post-processing failures are already degraded into text, e.g. unrecognized binary attachments are dropped with a note.
+
+However, error handling is not yet holistic:
+
+- Provider HTTP failures are often collapsed into broad errors such as `HttpError`, losing retryability information.
+- Transient provider/API failures do not have a general backoff retry loop.
+- Tool execution errors usually abort the agent loop instead of becoming tool error results.
+- Retry attempts are not surfaced to the UI, which makes coding-agent behavior opaque during provider outages or rate limits.
+
+## Error actions
+
+### 1. Retry the same provider request
+
+Use this when the provider request appears transiently unavailable or unreliable, and the conversation has not gained a completed assistant message.
+
+Examples:
+
+- HTTP 429 rate limit.
+- HTTP 408, 409, 425 where retry is appropriate.
+- HTTP 500, 502, 503, 504.
+- Connection reset, DNS/connect failure, TLS failure, timeout.
+- Stream interruption or malformed provider stream before a complete assistant message.
+- Possibly an empty assistant response, with a small retry cap.
+
+Behavior:
+
+- Do not append or modify conversation messages.
+- Retry the same request against the same conversation snapshot.
+- Use exponential backoff with jitter.
+- Honor `Retry-After` when the provider sends it.
+- Stop after a configurable attempt cap and then hard-fail.
+- Notify the receiver/UI before each delayed retry.
+
+### 2. Compact/rewrite context, then retry the same provider request
+
+Use this when the provider rejects the request because the input context is too large.
+
+Examples:
+
+- OpenAI-compatible `context_length_exceeded` / `maximum context length` responses.
+- Anthropic `prompt is too long` responses.
+- Other provider-specific context-window diagnostics.
+
+Behavior:
+
+- Run automatic compaction if configured.
+- Rewrite the conversation into the compacted form.
+- Retry the same request once against the compacted conversation.
+- If compaction cannot shed enough context, or if the retry also overflows, hard-fail.
+
+This is already mostly implemented; it should be integrated into the same provider-error decision path as ordinary retries.
+
+### 3. Append tool error results and continue
+
+Use this when the model emitted a tool call and the failure is meaningful to the model.
+
+Examples:
+
+- Unknown tool name.
+- Invalid/incomplete tool input.
+- Future schema validation failures.
+- Native tool handler returned an error.
+- Lua/runtime exception.
+- `ToolSource` per-call error.
+- `ToolSource` whole-batch failure, mapped to each affected call.
+- Tool output shape/encoding problems that can be explained to the model.
+- Media processing failures where the rest of the agent is healthy.
+
+Behavior:
+
+- Append a user message containing one `ToolResult` for every `ToolUse` block, preserving original call order.
+- Successful calls keep their normal results.
+- Failed calls get model-readable error text.
+- The agent loop continues, allowing the model to correct arguments, try a different tool, or explain the failure to the user.
+- Prefer adding an internal `is_error` flag to tool results, serialized where supported.
+
+Example error text:
+
+```text
+Tool execution failed for `std.read`: file not found: src/foo.zig
+You may fix the arguments, try a different tool, or explain the failure to the user.
+```
+
+Provider serialization:
+
+- Anthropic supports an `is_error`-style tool result marker; serialize it when present.
+- OpenAI Chat Completions has no direct equivalent; include the error as textual tool content.
+
+### 4. Hard-fail
+
+Use this when retrying would be unsafe or misleading, or when the failure belongs to the host application rather than the model/provider interaction.
+
+Examples:
+
+- User cancellation / abort.
+- Out of memory.
+- Receiver/UI/session persistence failure.
+- Invalid local configuration: missing API key, malformed URL, unknown model alias.
+- Provider authentication/authorization failure: HTTP 401/403.
+- Non-retryable provider request errors: most HTTP 400s except context overflow.
+- Internal invariant violation / conversation corruption.
+- Concurrency unavailable, unless we add a sequential fallback.
+
+Behavior:
+
+- Return the error to the embedder.
+- Do not synthesize a model-visible tool result for host/system failures.
+
+## Provider retry notifications
+
+Retry visibility should be front-loaded. Coding-agent UIs need to show that the agent is waiting on the provider rather than silently hanging.
+
+Add a receiver callback for provider retry scheduling, something like:
+
+```zig
+onProviderRetry(info: ProviderRetryInfo) void
+```
+
+Possible fields:
+
+```zig
+const ProviderRetryInfo = struct {
+    attempt: usize,          // next attempt number, 1-based or 0-based; define clearly
+    max_attempts: usize,
+    delay_ms: u64,
+    err: anyerror,
+    status_code: ?u16 = null,
+    retry_after_ms: ?u64 = null,
+    message: ?[]const u8 = null,
+};
+```
+
+Semantics:
+
+- Fire after a provider attempt fails and before sleeping for the next attempt.
+- Fire for ordinary provider retry/backoff.
+- Fire for context-overflow compaction retry too, either through this callback or through a sibling callback that says compaction is being attempted.
+- Do not fire for tool-call failures that are being sent back as tool results; those are normal conversation continuations.
+
+Initial CLI behavior can be simple, e.g. print a dim status line:
+
+```text
+provider unavailable: retrying in 2.4s (attempt 2/4)
+```
+
+## Provider error classification
+
+The providers should stop collapsing all HTTP/API failures into a single broad error when the agent needs to distinguish retryable from non-retryable failures.
+
+Minimum useful public errors:
+
+```zig
+error.ProviderRateLimited
+error.ProviderUnavailable
+error.ProviderServerError
+error.ProviderTransport
+error.ProviderStreamMalformed
+error.ProviderAuthFailed
+error.ProviderBadRequest
+error.ProviderModelNotFound
+error.ContextOverflow
+```
+
+Suggested HTTP mapping:
+
+- `400` + context marker: `ContextOverflow`
+- other `400`: `ProviderBadRequest`
+- `401`, `403`: `ProviderAuthFailed`
+- `404`: `ProviderModelNotFound` or `ProviderBadRequest`, depending on provider/API shape
+- `408`: retryable transport/request timeout
+- `409`, `425`: retryable if provider semantics indicate temporary conflict/not-ready
+- `429`: `ProviderRateLimited`
+- `500..599`: `ProviderServerError` / `ProviderUnavailable`
+
+Longer term, `anyerror` alone is too lossy for status code, provider message, and `Retry-After`. A richer provider error/reporting channel would be useful. For the first implementation, distinct error names plus logging may be enough.
+
+## Agent retry policy
+
+Add a retry policy to config, with conservative defaults.
+
+Possible shape:
+
+```zig
+const RetryConfig = struct {
+    max_attempts: usize = 4,
+    initial_delay_ms: u64 = 500,
+    max_delay_ms: u64 = 10_000,
+    multiplier: f64 = 2.0,
+    jitter: bool = true,
+};
+```
+
+Rules:
+
+- Attempt count includes the first attempt; `max_attempts = 4` means one initial try plus up to three retries.
+- Retry only errors classified as retryable provider failures.
+- Never retry cancellation, local receiver errors, auth failures, or bad local configuration.
+- If the provider supplies `Retry-After`, prefer that delay, capped by policy.
+- Keep context-overflow compaction retry as a separate one-shot path, not an unbounded retry loop.
+
+## Tool error result policy
+
+Change tool dispatch so normal tool failures do not abort the turn.
+
+Current behavior to replace:
+
+- Unknown tool returns `error.UnknownTool`.
+- `Tool.invoke` error records `FlatCall.err`; after dispatch, the first error aborts the turn.
+- `ToolSource` per-call errors and whole-batch errors abort the turn.
+
+Desired behavior:
+
+- Unknown tool becomes an error `ToolResult`.
+- Invalid/incomplete input remains an error `ToolResult`.
+- Native tool errors become error `ToolResult`s unless classified as hard host failures.
+- Source per-call errors become per-call error `ToolResult`s.
+- Source whole-batch errors become error `ToolResult`s for every member call unless the batch failed due to hard host failure.
+- Successful calls in the same assistant message should still return successful results.
+
+Hard host failures should still abort, e.g. cancellation and out-of-memory.
+
+This requires a helper like:
+
+```zig
+fn classifyToolError(err: anyerror) ToolErrorAction {
+    return switch (err) {
+        error.Canceled, error.OutOfMemory => .hard_fail,
+        else => .tool_result,
+    };
+}
+```
+
+The exact list will need auditing across native tools, Lua tools, source runtimes, and media processing.
+
+## Data model changes
+
+Consider extending `ToolResultBlock` with an error marker:
+
+```zig
+pub const ToolResultBlock = struct {
+    tool_use_id: []const u8,
+    parts: std.ArrayList(ResultPartStored),
+    is_error: bool = false,
+};
+```
+
+Update:
+
+- conversation cloning/deinit paths,
+- session persistence format,
+- Anthropic serializer/parser,
+- OpenAI serializer/parser,
+- tests.
+
+Backward compatibility:
+
+- Missing `is_error` in old sessions should default to `false`.
+- OpenAI Chat serialization can ignore the flag except for the textual content already present.
+
+## Implementation plan
+
+### Phase 1: Classify provider errors
+
+- Expand provider HTTP status handling to return specific provider errors.
+- Preserve context-overflow detection.
+- Add tests for status-to-error mapping where practical.
+
+### Phase 2: Add provider retry loop
+
+- Add retry config defaults.
+- Implement `streamWithRetries` around `stream_fn` in `Agent.runStep`.
+- Retry only classified transient provider errors.
+- Add exponential backoff with jitter and `Retry-After` support if available.
+- Ensure failed attempts do not mutate the conversation.
+
+### Phase 3: Add retry notifications
+
+- Extend `ReceiverVTable` with provider retry notification.
+- Update all receivers, test receivers, and CLI receivers.
+- Print visible retry status in the CLI.
+
+### Phase 4: Convert tool failures to tool results
+
+- Add helper to build textual error `ResultPart`s.
+- Change unknown-tool handling to append an error result instead of aborting.
+- Change `FlatCall.err` assembly to synthesize error results for model-visible failures.
+- Keep hard host failures as returned errors.
+- Update tool API docs to say returned errors normally become model-visible tool failures.
+
+### Phase 5: Add tool-result error marker
+
+- Add `is_error` to `ToolResultBlock`.
+- Serialize to Anthropic where supported.
+- Preserve compatibility for OpenAI and old session files.
+- Update tests.
+
+### Phase 6: Holistic tests
+
+Add tests for:
+
+- provider 429 retries and then succeeds without duplicate messages,
+- provider 500 retries with backoff notification,
+- provider auth failure does not retry,
+- context overflow still compacts once and retries,
+- unknown tool produces an error tool result and the agent continues,
+- tool handler error produces an error tool result and the model gets another turn,
+- source per-call and batch errors produce error results,
+- cancellation/OOM-like errors still hard-fail,
+- retry notification callbacks are delivered before delay.
+
+## Open questions
+
+- Should malformed provider streams always be retryable, or only before any complete content block has been committed?
+- Do we need a richer provider error object sooner rather than later, mainly for `Retry-After` and UI diagnostics?
+- Should tool implementations have a first-class `model_error` return path distinct from host errors, instead of overloading `anyerror`?
+- Should `ConcurrencyUnavailable` fall back to sequential tool execution instead of hard-failing?
+- How should retry notifications be represented in persisted sessions, if at all? Initial answer: probably not persisted; they are UI/runtime events, not conversation history.