initial documentation

1 files changed, 97 insertions, 0 deletions

diff --git a/docs/overview.md b/docs/overview.md
new file mode 100644
index 0000000..962516d
--- /dev/null
+++ b/docs/overview.md
@@ -0,0 +1,97 @@
+# awl — Overview
+
+`awl` is a minimal coding agent built for performance, efficiency, correctness, and a small core that can be extended deliberately.
+
+## Ethos
+
+**Batteries optional.** A full-featured coding agent experience ships by default — but everything can be deactivated. The standard distribution includes a curated set of coding-oriented tools and settings, all of which can be turned off. Strip out every coding tool and `awl` becomes a general-purpose LLM chat client. Additional capabilities may ship in the base but remain deactivated by default, waiting to be opted into. Nothing is mandatory; everything is intentional.
+
+**Small core, deliberate extension.** The core runtime does as little as possible. Features that would be built-ins in other agents are extensions in `awl` — including the fundamental tools like `read`, `write`, `edit`, and `bash`. The extension system is the primary mechanism for adding capability.
+
+**Conservative provider support.** Provider integrations are careful and complete rather than broad and broken. `awl` supports Anthropic-shaped and OpenAI-shaped APIs with arbitrary base URLs. A provider integration that partially works is worse than no integration at all.
+
+**Own your data model.** `awl` defines its own internal conversation representation and maps to/from provider wire formats. No provider's API shape is treated as canonical. This ensures that adding a new provider never requires contorting the core model.
+
+**Lean on the terminal.** The TUI does not try to be a full application framework. Scrollback, selection, and search are handled by the surrounding terminal (ghostty, tmux, etc.). The TUI's job is to present output clearly and offer targeted enhancements — like expanding or collapsing tool-call blocks — by clearing and re-rendering its own output region. This keeps the TUI simple while still providing a much nicer experience than a raw CLI.
+
+## Architecture
+
+### Data model
+
+The conversation model is provider-agnostic. It uses a flat message list where each message has a role and a list of typed content blocks:
+
+```
+Conversation = ordered list of Messages
+Message      = { role: system | user | assistant, content: []ContentBlock }
+ContentBlock = Text | Thinking | ToolUse | ToolResult
+```
+
+- `Text` and `Thinking` use a shared `TextualBlock` type that grows incrementally via an internal `ArrayList(u8)` — amortized O(1) appends during streaming, no O(n²) re-copying.
+- `ToolUse` and `ToolResult` arrive complete (not streamed incrementally) and store their data as owned byte slices.
+- System messages may contain multiple `Text` blocks, which are concatenated when a provider expects a single system prompt string (e.g., Anthropic).
+
+### Library structure
+
+`awl` is a library first. The core agent functionality lives in `libawl`, a Zig module. The CLI is a thin consumer of the library. A C ABI build of `libawl` will be produced when the extension system needs it (for Lua interop), implemented as thin `export fn` wrappers around the Zig API.
+
+### Provider abstraction
+
+Providers implement a streaming interface: given a conversation, stream a response message back via a Receiver (callback-based). The Receiver delivers incremental content deltas for real-time display and a complete assembled message when the stream ends. Adding a new provider means implementing this interface and writing the serialization for the provider's wire format.
+
+### Extension system
+
+Extensions will initially be written in Lua, requiring a C ABI surface on `libawl`. Future support for shared-object extensions (Zig, Rust, C, C++) will use the same C ABI. Core tools like `read`, `write`, `edit`, and `bash` are extensions — individually disableable, included in the standard distribution but not hardcoded into the runtime.
+
+### Server/proxy mode
+
+In a future phase, `awl` will be able to run as a server exposing OpenAI-compatible and Anthropic-compatible APIs, acting as a lightweight provider router/proxy to its configured backends. This is not yet planned in detail.
+
+## Phase Roadmap
+
+| Phase | Deliverable | Doc |
+|-------|-------------|-----|
+| 1 | libawl — minimal chat library, OpenAI provider, streaming, minimal CLI | [phase-1.md](phase-1.md) |
+| 2 | Anthropic provider — second provider, validates the abstraction | phase-2.md |
+| 3 | Extension API — Lua runtime, extension loading, tool registration | phase-3.md |
+| 4 | Conversation serialization — JSONL event log, session save/resume, crash recovery | [phase-4.md](phase-4.md) |
+| 5 | Core tools — read/write/edit/bash as distributable extensions | phase-5.md |
+| 6 | Rounded coding agent — slash commands, TOML config, extended TUI | phase-6.md |
+
+### Phase 1: libawl
+
+A Zig library that holds a streaming conversation with an LLM via an OpenAI-compatible API. No tools, no extensions — just chat. Ships a minimal CLI (`awl` binary) for live testing: readline, send, print streamed response, repeat. The conversation model is established with all four ContentBlock variants defined (ToolUse and ToolResult exist in the type but are never produced in this phase).
+
+### Phase 2: Anthropic Provider
+
+A second provider implementation targeting Anthropic's API shape. Validates that the Provider abstraction and internal conversation model are genuinely provider-agnostic — not just OpenAI in disguise. This is a focused phase: if the abstraction is right, it's mostly serialization work. If it's wrong, we find out here rather than later.
+
+### Phase 3: Extension API
+
+Introduces a Lua extension runtime and the extension loading mechanism. Extensions can register tools, access configuration, and participate in the agent loop. This phase also produces the C ABI build of `libawl` needed for Lua interop. Tools exist but none ship yet — the extension system is the deliverable, not the tools.
+
+### Phase 4: Conversation Serialization
+
+Save and resume conversations. Sessions are stored as append-only JSONL event logs — recording every message and model change — and fully rebuilt from the log on resume. Disk persistence so a coding agent can survive restarts and be reviewed later. See [phase-4.md](phase-4.md).
+
+### Phase 5: Core Tools as Extensions
+
+The fundamental coding tools — `read`, `write`, `edit`, `bash` — are implemented as extensions (initially Lua, eventually native). They live under the `std` namespace: `std.read`, `std.write`, `std.edit`, `std.bash`. The `std` package is a curated set of coding-oriented extensions — some enabled by default, some available but deactivated — embodying the "batteries optional" ethos. They ship with the standard distribution but are individually disableable. This is where `awl` becomes a functional coding agent rather than just a chat client.
+
+### Phase 6: Rounded Coding Agent
+
+Polish and capstone features that make `awl` a well-rounded coding agent experience:
+
+- **Slash commands** — an extensible framework for `/`-prefixed commands (e.g., `/help`, `/model`, `/clear`)
+- **TOML configuration** — a config file for persistent settings (default model, enabled/disabled extensions, provider configs, system prompt templates)
+- **Extended TUI** — smarter output rendering while remaining lightweight: expand/collapse tool call blocks (via clear-and-reprint), structured display of thinking content, prompt decoration
+- **Compaction with custom compaction prompts** — LLM-based context pruning for long conversations. Older messages are summarized to free context window space. The `compaction` entry type in the event log records the summary and a reference to the first kept message, so the log remains append-only and the full history is never destroyed — only omitted from the LLM context. Extension hooks allow custom compaction prompts, so users can guide how their history is summarized (e.g., preserving details about a specific task, emphasizing code changes over conversation).
+
+## Future (Unplanned)
+
+These are recorded from the initial ideas but do not yet have phase documents or detailed plans:
+
+- **Server/proxy mode** — run `awl` as a server exposing OpenAI-compatible and Anthropic-compatible APIs, routing to configured backends
+- **Shared-object extensions** — extend the extension system beyond Lua to support native shared libraries via the C ABI (Zig, Rust, C, C++)
+- **System prompt construction framework** — opinionated system for assembling system prompts from composable parts (templates, project context, extension contributions)
+- **Google API provider** — native integration with Google's Gemini API (rather than their OpenAI-compatibility layer), unlocking richer capabilities specific to that API shape. Low priority compared to Anthropic and OpenAI support.
+- **C ABI distribution of libawl** — `export fn` wrappers exposing libawl functionality through a C calling convention, enabling external programs to embed or build on awl from C, Rust, or other native languages. Not a separate library — the C ABI is a second interface on the same `libawl` artifact, compiled from `export fn` shims that translate between Zig types and C types. Needed eventually for shared-object extensions (Zig, Rust, C, C++) beyond Lua.