path: root/docs/phase-1.md

# Phase 1: libawl — Minimal Chat Library

## Goal

A Zig library that can hold a streaming conversation with an LLM via an OpenAI-compatible API. No tools, no extensions — just chat. Includes a minimal CLI for live testing.

## Deliverable

A `libawl` Zig module importable by other Zig code, plus a `awl` binary that wires it into a basic read/print loop. At the end of this phase, you can:

- Start `awl`, type a message, and receive a streamed response from an OpenAI-compatible LLM.
- Send follow-up messages that include full conversation history.
- See thinking tokens and text tokens stream in as they arrive.
- Have the complete conversation available in memory for the duration of the session.

## What is usable at the end

| Capability | How to exercise it |
|---|---|
| Open a conversation | `awl.conversation.Conversation.init(allocator)` |
| Add a user message | `conversation.addUserMessage("hello")` |
| Run an agent step (streaming) | `agent.runStep(conversation, &receiver)` |
| See streamed output | CLI prints thinking/text chunks as they arrive |
| Conversation persists across turns | Follow-up messages include prior history |

## What is explicitly out of scope

- Tools and tool-use (phase 3+)
- Extensions and extension API (phase 3+)
- C ABI (phase 3+, when needed for Lua extensions)
- Anthropic provider (phase 2)
- Disk persistence / session save (later phase)
- Server/proxy mode (future, undefined phase)
- System prompt construction framework (later phase — the model supports system messages, but no opinionated assembly system yet)

---

## Data Model

### TextualBlock (shared streaming buffer)

```
TextualBlock = struct {
    buf: std.ArrayList(u8),
    allocator: std.mem.Allocator,

    pub fn init(allocator: std.mem.Allocator) TextualBlock
    pub fn content(self: *const TextualBlock) []const u8   // self.buf.items
    pub fn append(self: *TextualBlock, delta: []const u8) !void
    pub fn deinit(self: *TextualBlock) void               // self.buf.deinit()
}
```

`Text` and `Thinking` blocks both use `TextualBlock` as their payload. They share the same append-based streaming behavior — deltas arrive incrementally and are appended via an internal `ArrayList(u8)`, giving amortized O(1) appends and avoiding the O(n²) re-copying that would result from storing `[]const u8` slices.

The `TextualBlock` stores its own allocator reference so that `deinit()` needs no external context. Each `TextualBlock` has an `init()`/`deinit()` pair. This also means a `ContentBlock` can clean itself up without the caller providing an allocator.

### ContentBlock (tagged union)

```
ContentBlock = union(enum) {
    Text: TextualBlock,
    Thinking: TextualBlock,
    ToolUse: ToolUseBlock,       // phase 3+
    ToolResult: ToolResultBlock, // phase 3+

    pub fn deinit(self: *ContentBlock) void {
        switch (self.*) {
            .Text, .Thinking => |*b| b.deinit(),
            .ToolUse => |b| { /* free id, name, input */ },
            .ToolResult => |b| { /* free tool_use_id, content */ },
        }
    }
}

ToolUseBlock = struct {
    id: []const u8,       // owned copy
    name: []const u8,     // owned copy
    input: []const u8,    // raw JSON bytes, owned copy
}

ToolResultBlock = struct {
    tool_use_id: []const u8,  // owned copy
    content: []const u8,     // owned copy
}
```

`ToolUse` and `ToolResult` are defined in the model now but not populated or processed until the extensions phase. This avoids a model refactor later — the types exist, we just never encounter them in phase 1.

The `input` field of `ToolUse` is stored as raw JSON bytes (`[]const u8`) rather than a parsed structure. We are not in the business of understanding tool input schemas; we pass them through.

`ToolUse` blocks also stream incrementally — both providers send tool input as JSON fragments across multiple deltas. Therefore `ToolUse.input` also uses `TextualBlock` for assembly. `id` and `name` arrive at block-start time and are stored as owned `[]const u8` copies.

`ToolResult` blocks are constructed by `awl` itself (not streamed from a provider), so `content` could be a simple `[]const u8`. However, for consistency and to allow progressive construction of results, it also uses `TextualBlock`.

Updated types:
```
ToolUseBlock = struct {
    id: []const u8,           // owned copy, from onBlockStart metadata
    name: []const u8,         // owned copy, from onBlockStart metadata
    input: TextualBlock,     // accumulated from onContentDelta
}

ToolResultBlock = struct {
    tool_use_id: []const u8,  // owned copy
    content: TextualBlock,    // accumulated content
}
```

**Memory discipline**: When a `ContentBlock` is moved into a `Message`'s content list (stored in `std.ArrayList(ContentBlock)`), the TextualBlock's internal ArrayList buffer pointer remains valid — it points to the same heap allocation. The caller must ensure each block's `deinit()` is called exactly once, and must not copy a ContentBlock without clearing the source (standard Zig move semantics).

### Message

```
Message = {
  role: enum { system, user, assistant },
  content: []ContentBlock,
}
```

A system message may contain multiple `Text` blocks. When serializing to Anthropic's API (phase 2), these are concatenated into the single system prompt string.

An assistant message is assembled incrementally during streaming. A user message containing tool results (phase 3+) naturally groups multiple `ToolResult` blocks.

### Conversation

```
Conversation = {
  messages: std.ArrayList(Message),
  allocator: std.mem.Allocator,
}
```

Ordered list of messages. Methods:

- `init(allocator)` → Conversation
- `addSystemMessage(text)` → appends `Message{ .system, [TextBlock(text)] }`
- `addUserMessage(text)` → appends `Message{ .user, [TextBlock(text)] }`
- `addAssistantMessage(blocks)` → appends `Message{ .assistant, blocks }` (called by agent loop after streaming completes)
- `deinit()` → frees all owned memory

All `[]const u8` fields in ContentBlocks and Messages are owned by the Conversation and freed on `deinit()`. Content is stored as copies, not slices into external buffers. `TextualBlock` fields back their content with a heap-allocated `ArrayList(u8)` that grows incrementally during streaming and is freed on `deinit()`.

---

## Module Structure

```
src/
  root.zig              // public API re-exports
  conversation.zig      // Message, ContentBlock, Conversation
  provider.zig          // Provider interface, StreamEvent, StreamResult
  provider_openai.zig   // OpenAI-compatible implementation
  sse.zig               // SSE line parser
  agent.zig             // Agent loop: runStep, Receiver interface
  config.zig            // Config struct (api_key, base_url, model)
  json.zig              // Serialization helpers (model → wire JSON, deltas → ContentBlocks)
```

### `conversation.zig`

Defines `Message`, `ContentBlock`, `Conversation`. All serialization to/from provider wire formats lives in `json.zig` — conversation.zig is pure data structure.

Tests: create conversations, add messages, verify content, free without leaks.

### `provider.zig`

Defines the `Provider` interface:

```
Provider = struct {
  ptr: *anyopaque,
  vtable: *const VTable,

  VTable = struct {
    streamStep: *const fn(*anyopaque, conversation: *Conversation, receiver: *Receiver) anyerror!void,
    deinit: *const fn(*anyopaque) void,
  };

  pub fn streamStep(self, conversation, receiver) !void
  pub fn deinit(self) void
};
```

And the `Receiver` interface for streaming callbacks:

```
Receiver = struct {
  ptr: *anyopaque,
  vtable: *const ReceiverVTable,

  ReceiverVTable = struct {
    onMessageStart:    *const fn(*anyopaque, role: MessageRole) void,
    onBlockStart:      *const fn(*anyopaque, block_type: ContentBlockType, index: usize, meta: ?BlockMeta) void,
    onContentDelta:    *const fn(*anyopaque, block_index: usize, delta: []const u8) void,
    onBlockComplete:   *const fn(*anyopaque, block_index: usize, block: ContentBlock) void,
    onMessageComplete:*const fn(*anyopaque, message: Message) void,
  };

  pub fn onMessageStart(self, role) void
  pub fn onBlockStart(self, block_type, index, meta) void
  pub fn onContentDelta(self, block_index, delta) void
  pub fn onBlockComplete(self, block_index, block) void
  pub fn onMessageComplete(self, message) void
};

BlockMeta = struct {
  // Only populated for ToolUse blocks. Null for Text/Thinking.
  tool_id: ?[]const u8,
  tool_name: ?[]const u8,
};
```

**Callback contract:**

- Callbacks are always invoked in this order for every block, regardless of which provider is active.
- `onMessageStart` is called when the stream begins delivering a new message.
- `onBlockStart` is called when a new content block begins. `meta` carries block-type-specific metadata (tool id/name for ToolUse, null for Text/Thinking).
- `onContentDelta` is called zero or more times per block with raw byte fragments. For Text/Thinking these are word fragments; for ToolUse these are JSON fragments. The receiver does not need to interpret them. `delta` is a `[]const u8` — libawl does not parse tool input content, it passes bytes through.
- `onBlockComplete` is called when a block is finished. The `block` parameter contains the fully assembled ContentBlock. The receiver that only needs complete content can ignore deltas and use this.
- `onMessageComplete` is called when the stream ends. The `message` parameter contains the fully assembled Message with all blocks.
- Providers guarantee that `onBlockComplete`'s `block` and `onMessageComplete`'s `message` are always fully assembled and valid.

This uniform callback sequence means the TUI and agent loop don't need to know which provider is active. Anthropic (phase 2) maps its structured events directly to these callbacks; OpenAI synthesizes block boundaries from delta field transitions (see below).

### `provider_openai.zig`

Implements `Provider` for OpenAI-compatible APIs.

- Converts `Conversation` → OpenAI wire JSON (see [OpenAI serialization](#openai-serialization) below)
- Makes HTTP POST to `{base_url}/chat/completions` with `stream: true`
- Reads SSE events, parses each `data: {...}` line as complete JSON
- Synthesizes block boundaries from delta field transitions (OpenAI has no explicit block boundary events)
- Calls the full Receiver callback sequence (onMessageStart → onBlockStart → onContentDelta → onBlockComplete → onMessageComplete)
- Accumulates deltas into ContentBlocks via TextualBlocks

Construction:

```
OpenAIProvider.init(allocator, config) !OpenAIProvider
```

### OpenAI block boundary synthesis

OpenAI's streaming deltas have no explicit block boundaries. The provider tracks a state machine to infer when blocks start and end:

```
StreamingState = struct {
    active_block_type: enum { none, thinking, text, tool_use },
    active_block_index: usize,
    // assembly buffers per block
}

On each SSE event:
  1. If delta.role == "assistant" → emit onMessageStart(.assistant)
  2. If delta.reasoning_content present:
       - If active_block_type != .thinking:
           - If active_block_type != .none → emit onBlockComplete for prior block
           - Emit onBlockStart(.Thinking, index, null)
           - active_block_type = .thinking
       - Emit onContentDelta(index, delta.reasoning_content)
  3. If delta.content present:
       - If active_block_type != .text:
           - If active_block_type != .none → emit onBlockComplete for prior block
           - Emit onBlockStart(.Text, index, null)
           - active_block_type = .text
       - Emit onContentDelta(index, delta.content)
  4. If delta.tool_calls present:
       - If active_block_type != .tool_use:
           - If active_block_type != .none → emit onBlockComplete for prior block
           - Emit onBlockStart(.ToolUse, index, .{ .tool_id = ..., .tool_name = ... })
           - active_block_type = .tool_use
       - Emit onContentDelta(index, delta.tool_calls[].function.arguments)
  5. If finish_reason != null:
       - If active_block_type != .none → emit onBlockComplete for current block
       - Emit onMessageComplete(assembled_message)
```

A transition in `active_block_type` means the previous block is done and a new one has started. The state machine also handles the case where the same block type appears again after an intervening type (e.g., thinking → text → thinking), which would open a new Thinking block at a new index.

### `sse.zig`

Incremental SSE line parser. The HTTP client delivers arbitrary-sized read buffers; this module reassembles them into complete `data: ...\n\n` events.

```
SSEParser = struct {
  buf: std.ArrayList(u8),

  pub fn init(allocator) SSEParser
  pub fn feed(self, chunk: []const u8) ![]const []const u8   // returns slice of complete event strings
  pub fn deinit(self) void
};
```

`feed()` may return zero events (partial line buffered) or multiple events (chunk contained several). The caller does not need to worry about line boundaries.

Tests: feed partial chunks, verify events emitted at correct boundaries; multi-event in single chunk; empty lines; `data: [DONE]`.

### `json.zig`

Two responsibilities:

1. **Serialize Conversation → OpenAI request body** — Convert our `Message`/`ContentBlock` model into the JSON shape OpenAI expects. See below.
2. **Parse SSE chunk deltas → ContentBlock updates** — Each SSE event's JSON contains a `choices[0].delta` object. Extract text/thinking content from it.

### `agent.zig`

The agent loop. In phase 1, it's simple:

```
Agent = struct {
  provider: Provider,
  allocator: std.mem.Allocator,

  pub fn init(allocator, provider) Agent
  pub fn runStep(self, conversation: *Conversation, receiver: *Receiver) !void
  pub fn deinit(self) void
};
```

`runStep` does:
1. Call `provider.streamStep(conversation, receiver)` — this streams the response and calls the full Receiver callback sequence on the receiver
2. The `onMessageComplete` callback appends the finished Message to the conversation (the agent itself can wire this, or the caller handles it — TBD during implementation)

In later phases, `runStep` gains the tool-call loop: check for ToolUse blocks, execute them, feed results back, call provider again. But the shape stays the same — one `runStep` invocation carries the conversation through a full agent turn.

### `config.zig`

```
Config = struct {
  api_key: []const u8,
  base_url: []const u8,     // e.g. "https://api.openai.com/v1"
  model: []const u8,        // e.g. "gpt-4o"
};
```

Populated from environment variables (`AWL_API_KEY`, `AWL_BASE_URL`, `AWL_MODEL`) with defaults for base_url and model.

### `root.zig`

Public API. Re-exports the types and functions that external Zig code needs:

```
pub const conversation = @import("conversation.zig");
pub const provider = @import("provider.zig");
pub const agent = @import("agent.zig");
pub const config = @import("config.zig");
```

Does not re-export provider_openai, sse, or json — those are internal.

---

## OpenAI Serialization

### Request

Our `Conversation` → OpenAI `chat/completions` request body:

```
{
  "model": config.model,
  "stream": true,
  "messages": [
    // For each Message in conversation:
    //
    // role=.system → { "role": "system", "content": "<concatenated text blocks>" }
    // role=.user   → { "role": "user",   "content": "<concatenated text blocks>" }
    //                (ToolResult blocks pulled out into separate role:tool messages in phase 3+)
    // role=.assistant → { "role": "assistant", "content": [
    //                      ...text blocks as { "type": "text", "text": "..." },
    //                      ...thinking blocks as { "type": "thinking", "thinking": "..." },
    //                      ...tool_use blocks become function_call/function entries in phase 3+
    //                    ] }
  ]
}
```

For phase 1, all content blocks we encounter are `Text` or `Thinking`, so serialization is straightforward.

### Response (streaming)

Each SSE event is a complete JSON object:

```
data: {"id":"chatcmpl-123","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}
data: {"id":"chatcmpl-123","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}
data: {"id":"chatcmpl-123","choices":[{"index":0,"delta":{"content":" world"},"finish_reason":null}]}
data: {"id":"chatcmpl-123","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```

We parse each event's `choices[0].delta` and drive the block boundary state machine:
- `delta.role == "assistant"` → emit onMessageStart, marks the start of a new assistant message
- `delta.reasoning_content` → transition to Thinking block if needed, append via onContentDelta
- `delta.content` → transition to Text block if needed, append via onContentDelta
- `delta.tool_calls` → transition to ToolUse block if needed, append arguments via onContentDelta (phase 3+)

The `finish_reason: "stop"` signals stream end → emit onBlockComplete for any active block, then onMessageComplete.

---

## Minimal CLI

```
awl/
  src/
    main.zig    // CLI entry point
```

Behavior:
1. Read `AWL_API_KEY`, `AWL_BASE_URL`, `AWL_MODEL` from environment
2. Create a Conversation, add a system message (default: "You are a helpful assistant.")
3. Print a prompt (`> `), read a line from stdin
4. Add user message, call `agent.runStep()`, print streamed deltas to stdout
5. Repeat step 3 until EOF (Ctrl+D)

There is no line editing, no scrolling, no syntax highlighting. Just `readline` → `print`. The sole purpose is exercising libawl against a real API.

---

## Testing Strategy

### Unit tests (automated, per module)

| Module | What to test |
|---|---|
| `conversation.zig` | Create conversation, add messages of each role, verify content block storage, free without leaks |
| `sse.zig` | Feed partial chunks, verify event boundaries; multi-event chunks; `data: [DONE]`; empty lines between events |
| `json.zig` | Serialize conversation → OpenAI JSON; parse delta JSON objects → content updates |
| `config.zig` | Parse from env vars; defaults for missing optional fields |

### Integration test (manual)

- Run `awl` binary with a real API key
- Hold a multi-turn conversation
- Verify responses stream to stdout
- Verify follow-up messages include prior context (ask the model "what did I just say?")

---

## Open Questions (to resolve during implementation)

1. **Thinking token support in OpenAI API**: OpenAI's `reasoning_content` field in streaming deltas is not universally present across models/endpoints. We need to handle its absence gracefully (just skip it, don't crash).
2. **Error handling in streams**: Mid-stream HTTP errors, rate limiting, truncated responses. How do we represent these to the caller? An `onError` callback on the Receiver seems likely.
3. **HTTP connection lifecycle**: Does `std.http.Client` support long-lived streaming connections cleanly? We may need to manage connection pooling or timeouts.
4. **Memory strategy for long conversations**: We're storing full message content in memory. For phase 1 this is fine, but we should define the interface so a later phase can introduce message summarization or offloading without changing the agent loop.