docs

* note on the claude subscription provider plan that this is an
  extension, not panto core, not to be published
* archive that old "overview" doc
* new doc: pluggable session stores
* new doc: libpanto C wrappers, FFI wrappers for python and go

1 files changed, 0 insertions, 103 deletions

diff --git a/docs/overview.md b/docs/overview.md
deleted file mode 100644
index 916e904..0000000
--- a/docs/overview.md
+++ /dev/null
@@ -1,103 +0,0 @@
-# pantograph — Overview
-
-`pantograph` is a minimal coding agent built for performance, efficiency, correctness, and a small core that can be extended deliberately.
-
-## Ethos
-
-**Batteries optional.** A full-featured coding agent experience ships by default — but everything can be deactivated. The standard distribution includes a curated set of coding-oriented tools and settings, all of which can be turned off. Strip out every coding tool and `pantograph` becomes a general-purpose LLM chat client. Additional capabilities may ship in the base but remain deactivated by default, waiting to be opted into. Nothing is mandatory; everything is intentional.
-
-**Small core, deliberate extension.** The core runtime does as little as possible. Features that would be built-ins in other agents are extensions in `pantograph` — including the fundamental tools like `read`, `write`, `edit`, and `bash`. The extension system is the primary mechanism for adding capability.
-
-**Conservative provider support.** Provider integrations are careful and complete rather than broad and broken. `pantograph` supports Anthropic-shaped and OpenAI-shaped APIs with arbitrary base URLs. A provider integration that partially works is worse than no integration at all.
-
-**Own your data model.** `pantograph` defines its own internal conversation representation and maps to/from provider wire formats. No provider's API shape is treated as canonical. This ensures that adding a new provider never requires contorting the core model.
-
-**Lean on the terminal.** The TUI does not try to be a full application framework. Scrollback, selection, and search are handled by the surrounding terminal (ghostty, tmux, etc.). The TUI's job is to present output clearly and offer targeted enhancements — like expanding or collapsing tool-call blocks — by clearing and re-rendering its own output region. This keeps the TUI simple while still providing a much nicer experience than a raw CLI.
-
-## Architecture
-
-### Data model
-
-The conversation model is provider-agnostic. It uses a flat message list where each message has a role and a list of typed content blocks:
-
-```
-Conversation = ordered list of Messages
-Message      = { role: system | user | assistant, content: []ContentBlock }
-ContentBlock = Text | Thinking | ToolUse | ToolResult
-```
-
-- `Text` and `Thinking` use a shared `TextualBlock` type that grows incrementally via an internal `ArrayList(u8)` — amortized O(1) appends during streaming, no O(n²) re-copying.
-- `ToolUse` and `ToolResult` arrive complete (not streamed incrementally) and store their data as owned byte slices.
-- System messages may contain multiple `Text` blocks, which are concatenated when a provider expects a single system prompt string (e.g., Anthropic).
-
-### Library structure
-
-`pantograph` is a library first. The core agent functionality lives in `libpanto`, a Zig module. The CLI is a thin consumer of the library. A C ABI build of `libpanto` will be produced when the extension system needs it (for Lua interop), implemented as thin `export fn` wrappers around the Zig API.
-
-### Provider abstraction
-
-Providers implement a streaming interface: given a conversation, stream a response message back via a Receiver (callback-based). The Receiver delivers incremental content deltas for real-time display and a complete assembled message when the stream ends. Adding a new provider means implementing this interface and writing the serialization for the provider's wire format.
-
-### Extension system
-
-Extensions will initially be written in Lua, requiring a C ABI surface on `libpanto`. Future support for shared-object extensions (Zig, Rust, C, C++) will use the same C ABI. Core tools like `read`, `write`, `edit`, and `bash` are extensions — individually disableable, included in the standard distribution but not hardcoded into the runtime.
-
-### Server/proxy mode
-
-In a future phase, `pantograph` will be able to run as a server exposing OpenAI-compatible and Anthropic-compatible APIs, acting as a lightweight provider router/proxy to its configured backends. This is not yet planned in detail.
-
-## Phase Roadmap
-
-| Phase | Deliverable | Doc |
-|-------|-------------|-----|
-| 1 | libpanto — minimal chat library, OpenAI provider, streaming, minimal CLI | [phase-1.md](phase-1.md) |
-| 2 | Anthropic provider — second provider, validates the abstraction | phase-2.md |
-| 3 | Extension API — Lua runtime, extension loading, tool registration | phase-3.md |
-| 4 | Conversation serialization — JSONL event log, session save/resume, crash recovery | [phase-4.md](phase-4.md) |
-| 5 | Core tools — read/write/edit/bash as distributable extensions | phase-5.md |
-| 6 | Rounded coding agent — slash commands, TOML config, extended TUI | phase-6.md |
-
-### Phase 1: libpanto
-
-A Zig library that holds a streaming conversation with an LLM via an OpenAI-compatible API. No tools, no extensions — just chat. Ships a minimal CLI (`panto` binary) for live testing: readline, send, print streamed response, repeat. The conversation model is established with all four ContentBlock variants defined (ToolUse and ToolResult exist in the type but are never produced in this phase).
-
-### Phase 2: Anthropic Provider
-
-A second provider implementation targeting Anthropic's API shape. Validates that the Provider abstraction and internal conversation model are genuinely provider-agnostic — not just OpenAI in disguise. This is a focused phase: if the abstraction is right, it's mostly serialization work. If it's wrong, we find out here rather than later.
-
-### Phase 3: Extension API
-
-Introduces a Lua extension runtime and the extension loading mechanism. Extensions can register tools, access configuration, and participate in the agent loop. This phase also produces the C ABI build of `libpanto` needed for Lua interop. Tools exist but none ship yet — the extension system is the deliverable, not the tools.
-
-### Phase 4: Conversation Serialization
-
-Save and resume conversations. Sessions are stored as append-only JSONL event logs — recording every message and model change — and fully rebuilt from the log on resume. Disk persistence so a coding agent can survive restarts and be reviewed later. See [phase-4.md](phase-4.md).
-
-### Phase 5: Core Tools as Extensions
-
-The fundamental coding tools — `read`, `write`, `edit`, `bash` — are implemented as extensions (initially Lua, eventually native). They live under the `std` namespace: `std.read`, `std.write`, `std.edit`, `std.bash`. The `std` package is a curated set of coding-oriented extensions — some enabled by default, some available but deactivated — embodying the "batteries optional" ethos. They ship with the standard distribution but are individually disableable. This is where `pantograph` becomes a functional coding agent rather than just a chat client.
-
-### Phase 6: Rounded Coding Agent
-
-Polish and capstone features that make `pantograph` a well-rounded coding agent experience:
-
-- **Slash commands** — an extensible framework for `/`-prefixed commands (e.g., `/help`, `/model`, `/clear`)
-- **TOML configuration** — a config file for persistent settings (default model, enabled/disabled extensions, provider configs, system prompt templates)
-- **Extended TUI** — smarter output rendering while remaining lightweight: expand/collapse tool call blocks (via clear-and-reprint), structured display of thinking content, prompt decoration
-- **Compaction with custom compaction prompts** — LLM-based context pruning for long conversations. Older messages are summarized to free context window space. The `compaction` entry type in the event log records the summary and a reference to the first kept message, so the log remains append-only and the full history is never destroyed — only omitted from the LLM context. Extension hooks allow custom compaction prompts, so users can guide how their history is summarized (e.g., preserving details about a specific task, emphasizing code changes over conversation).
-
-## Future (Unplanned)
-
-These are recorded from the initial ideas but do not yet have phase documents or detailed plans:
-
-- **Server/proxy mode** — run `pantograph` as a server exposing OpenAI-compatible and Anthropic-compatible APIs, routing to configured backends
-- **Shared-object extensions** — extend the extension system beyond Lua to support native shared libraries via the C ABI (Zig, Rust, C, C++)
-- **System prompt construction framework** — opinionated system for assembling system prompts from composable parts (templates, project context, extension contributions)
-- **Google API provider** — native integration with Google's Gemini API (rather than their OpenAI-compatibility layer), unlocking richer capabilities specific to that API shape. Low priority compared to Anthropic and OpenAI support.
-- **C ABI distribution of libpanto** — `export fn` wrappers exposing libpanto functionality through a C calling convention, enabling external programs to embed or build on pantograph from C, Rust, or other native languages. Not a separate library — the C ABI is a second interface on the same `libpanto` artifact, compiled from `export fn` shims that translate between Zig types and C types. Needed eventually for shared-object extensions (Zig, Rust, C, C++) beyond Lua.
-
-## Punted
-
-Deliberate decisions to defer functionality that came up during phase planning but doesn't fit cleanly into the existing phase roadmap. Each one is captured here with enough context to pick up later.
-
-- **Tool-call cancellation / timeout via process isolation.** First raised in phase 3. There is no clean POSIX mechanism for cancelling a thread mid-execution with a guarantee of no further side effects — `pthread_kill` with SIGKILL terminates the entire process, `pthread_cancel` is widely considered unusable, and signal-based interruption can't safely unwind arbitrary code. Lua's `lua_sethook` provides cooperative cancellation between VM steps but doesn't interrupt handlers blocked in C calls (filesystem, subprocess, network). The mechanism that actually works is **process isolation**: run tool invocations in a helper subprocess, SIGKILL the subprocess on timeout. The intended approach is `fork+exec` into a small `panto-tool-worker` binary that statically links libpanto's extension machinery. libpanto would define the wire protocol (tool name + input bytes over a pipe, result bytes back) and own the fork/exec/timeout/kill orchestration; the embedder supplies the helper binary path. This makes sandboxing a library-level capability available to all embedders, not just the panto CLI. Open design questions when this is picked up: extension loading strategy in the worker (rescan-per-fork vs long-lived worker pool), file descriptor inheritance policy, working directory and environment handling, and how registry contents are communicated to the worker. Until this lands, tool handlers run to completion in-process — if a tool hangs, the user kills `panto`.