diff options
Diffstat (limited to 'docs/libpanto-bindings.md')
| -rw-r--r-- | docs/libpanto-bindings.md | 323 |
1 files changed, 323 insertions, 0 deletions
diff --git a/docs/libpanto-bindings.md b/docs/libpanto-bindings.md new file mode 100644 index 0000000..9b9ef2c --- /dev/null +++ b/docs/libpanto-bindings.md @@ -0,0 +1,323 @@ +# Plan: `libpanto` language bindings (Go + Python) + +## Goal + +Expose `libpanto` to other languages, targeting **Go** and **Python** first. +The bindings are organized as a family of sibling packages, named uniformly: + +| package | language | role | +| ------------- | -------- | ------------------------------------------------------------- | +| `libpanto` | Zig | the core library (exists today) | +| `libpanto-c` | Zig | a C-ABI shared library + header wrapping the public Zig API | +| `libpanto-go` | Go | idiomatic Go bindings over `libpanto-c` via cgo | +| `libpanto-py` | Zig | a CPython extension implemented in pure Zig (`@cImport`) | + +Two consumers, two paths to the core: + +- **Go → `libpanto-c` → `libpanto`.** cgo is effectively the only option, and + cgo can only call C. So a C ABI is mandatory. +- **Python → `libpanto`** directly. `libpanto-py` is a native CPython + extension written *in Zig*: it `@cImport`s `Python.h`, builds `PyObject` + glue against the translated C types, and `build.zig` emits the loadable + `.so`. It calls the Zig API directly and **does not** depend on `libpanto-c`. + +### Why this split (decisions settled) + +- **`libpanto-c` is built regardless**, because Go requires it. It doubles as + the stable-ABI validation platform for the C surface. +- **No cffi.** A pure-Python cffi binding was considered and rejected. Its only + real appeal (no native build step, trivial wheels) holds solidly *only* in + ABI mode, which is also where streaming callbacks and silent struct/ABI + drift bite hardest; API mode reintroduces the C-compile/per-platform-wheel + cost without the control of a real native module. Either way it would be a + throwaway second validator of the same C surface that `libpanto-go` already + exercises. Net: redundant. We'd reimplement it natively on the soon side. +- **`libpanto-py` is pure Zig, not C-over-`libpanto-c`.** The decisive win is + that a Zig→`.so` compilation involves **no C translation units in the repo at + all** — one toolchain, header translation done at compile time by `@cImport`. + Going straight to the Zig API (rather than through `libpanto-c`) also skips a + redundant marshalling layer. + +## The core architectural decision: ship a *pull* streaming API + +This is the spine of the whole project, and it requires an **internal refactor +of `libpanto` first** (Phase 0). Everything else wraps it. + +### Why pull, not push + +`libpanto` today streams via a **push** `Receiver` vtable +(`provider.zig`): the provider loop calls `onMessageStart`, `onContentDelta`, +`onBlockComplete`, `onMessageComplete`, etc. Push is the wrong primitive to +*export*, for two independent reasons that land on the same conclusion: + +1. **It doesn't compose into the idiomatic surfaces we want.** + - Go: the idiomatic streaming shape is a single goroutine driving a + range-over-func iterator (`for ev := range stream.Iter`) and/or a channel. + cgo callbacks *into* Go are slow and awkward (`//export`, pointer rules, + no closures). A goroutine draining a pull API is the clean build. + - Python: a (sync or async) generator **is** a pull construct — + `__next__` *is* "give me the next event." A push callback can't `yield`; + to turn pushed values into a `for ev in stream`, you must run the producer + and consumer in different stack frames concurrently (a thread + queue). + So push → generator *forces* a thread+queue regardless. Pull → generator + is a 1:1 mapping with zero adaptation. + +2. **Pull is the more primitive primitive.** Push composes trivially on top of + pull (`while (try s.next()) |ev| receiver.onEvent(ev);` — five lines). + Pull does **not** compose cheaply on top of push: to expose `next()` over a + `Receiver`-driven core you must suspend *inside* a callback and resume the + provider loop later, which means a thread+queue or hand-rolled coroutine + state — i.e. you rebuild pull anyway, the hard way. + +**Conclusion:** ship only the pull API. `Receiver` leaves the public surface +entirely. Anyone who wants push writes the five-line wrapper themselves. + +### The Zig interface + +```zig +// run() drops the receiver parameter entirely and returns a Stream. +// Whether it also drops `conv` depends on sequencing against +// docs/pluggable-session-store.md (see note below) — either form is fine. +pub fn run(self: *Agent) !Stream // or: run(self: *Agent, conv: *Conversation) !Stream + +// Stream is a resumable handle, closely linked to SSEParser + the provider +// read loop. next() yields events until the turn is exhausted. +pub fn next(self: *Stream) !?Event +``` + +`Stream.next()` returns `!?Event`, which gives **three orthogonal channels**: + +| signal | meaning | +| ------------------ | ---------------------------------------------------- | +| `Event` (a value) | streaming progress, including the terminal event | +| `null` | the stream is exhausted (already past the terminal) | +| `error.X` | a genuine failure (network, parse, provider error) | + +### The terminal-event invariant (documented contract) + +> Every `next()` call on a stream **after** it has yielded a `MessageComplete` +> event returns `null`, and `null` is **never** returned before +> `MessageComplete`. + +Consequences, and why this exact shape: + +- **`MessageComplete` is a normal `Event` variant**, not an error and not + signalled by `null`. It is the in-band success terminal. +- **Intelligent consumers never observe `null`.** They stop after *consuming* + `MessageComplete`. `null` exists only as the defensive answer to "you called + `next()` one time too many," distinct from a real `error`. +- Python: emit `MessageComplete`, then raise `StopIteration` on the *next* + call (which sees `null`) — or, more precisely, the generator stops right + after yielding `MessageComplete`, so well-behaved code raises `StopIteration` + without ever materializing a `None`. +- Go: the `Iter` range-over-func stops *after* yielding `MessageComplete`, not + by waiting to observe a `null` sentinel. +- Mid-stream **provider errors surface as a Zig `error` from `next()`** (the + `!` in `!?Event`), not as an `Event.Error` variant. This keeps the `Event` + union success-only and maps cleanly to a raised Python exception and a Go + `error`. (`error.StreamExhausted` is *not* used — exhaustion is `null`, not + an error; we reserve errors for failures.) + +### `Event` spans the whole turn, not one HTTP response + +Verified from the code: `Agent.runStep` wraps **multiple** `streamStep` calls +in a tool-using turn (one provider stream per assistant message), with +concurrent tool dispatch *between* them (`agent.zig`). The `Stream` therefore +spans the entire `run()` loop — provider streaming **and** tool dispatch — not +a single SSE response. The `Event` union must express both layers. + +The exact variant list is to be finalized against what the provider loop and +agent loop emit today (see `ReceiverVTable` in `provider.zig` and the dispatch +path in `agent.zig`), but it covers at least: + +- message lifecycle: `MessageStart`, `MessageComplete` +- block lifecycle: `BlockStart`, `BlockComplete` +- content: `ContentDelta` +- tool identity: `ToolDetails` (id + name) +- tool dispatch boundaries: tool-call start / tool-result (so Go/Python + consumers can observe the agent running tools between provider turns) +- provider retry notices: `ProviderRetry` (informational; today `onProviderRetry`) + +`Event` is an `enum union`; it is the single type every binding marshals into +its native form. Define it once, in `libpanto`. + +## The hard part of Phase 0: making `Stream.next()` resumable + +Verified from `provider_openai_chat.zig`: the current provider loop is a +single-stack-frame loop — + +``` +readVec(body_reader) -> parser.feed(chunk) -> handleEvent(...) -> receiver.*(...) +``` + +— with all state (the HTTP body reader + `transfer_buf`, the `SSEParser`, the +per-response `StreamState`) living **on the stack**. A pull `next()` must +*return* between events, so that state has to move into the `Stream` handle. +Three ways to get there: + +1. **State machine (recommended target).** The `Stream` owns the socket/body + reader, the `SSEParser`, and the decode state; `next()` reads/parses just + enough to produce one `Event` and returns. No threads. Cleanest and most + portable across all four packages. Most upfront refactoring effort, since + the provider loop is inverted into a resumable step function. + +2. **Thread + queue inside the handle.** Keep the existing + `streamStep(receiver)` loop verbatim; run it on a thread whose receiver + pushes events into a bounded queue; `next()` pops. Minimal change to proven + code, but costs a thread per active stream plus shutdown/backpressure care. + A reasonable interim if (1) proves invasive. + +3. **Zig async/suspend.** Language-level coroutines. Availability depends on the + toolchain's async status at the time of implementation; treat as + not-reliably-available without checking first. + +The internal mechanism (1 vs 2) is swappable later **without changing the +exported contract** — the public surface is pull either way. Recommendation: +target (1); fall back to (2) only if the provider-loop inversion is too costly +for v1. + +> Expectation: **Phase 0 is a net-red diff.** Deleting `Receiver` from the +> public API, the `CompactionCapture` no-op receiver, and the CLI's +> `CLIReceiver` vtable plumbing should remove more than the `Stream` machinery +> adds. + +--- + +## Phase 0 — internal refactor of `libpanto` to a pull API + +The real work. Everything after this is wrapping. + +1. **Define the `Event` union** (success-only `enum union`) from the current + `ReceiverVTable` callbacks plus the agent's tool-dispatch boundaries. This + is the type every binding marshals. +2. **Define the `Stream` type**: a resumable handle owning the provider read + loop's state (body reader, `SSEParser`, decode state) and the agent's + tool-dispatch position. `fn next(self: *Stream) !?Event`. This is where the + state-machine-vs-thread decision lands. +3. **`Agent.run() -> Stream`.** With `conversation` owned by the `Agent` (per + `docs/pluggable-session-store.md`) and `receiver` removed, `run()` takes + only `self`. The agent loop (provider stream → tool dispatch → repeat) + becomes the thing `Stream.next()` drives incrementally. +4. **Delete `Receiver` from the public API** (`root.zig`, `provider.zig`). + Internal seams that genuinely want push (if any) get the trivial pull→push + wrapper, in caller code, not in the library surface. +5. **Rebuild the CLI on the pull loop.** `src/main.zig`'s `CLIReceiver` + collapses into a `while (try stream.next()) |ev| render(ev)` loop. Prove the + refactor end-to-end against the existing test suite; CLI behavior unchanged. + +**Exit criteria:** `Receiver` gone from `root.zig`; `Stream.next()` is the only +streaming primitive; CLI output and the full test suite are unchanged. + +> Sequencing with `docs/pluggable-session-store.md` is **flexible**, not a +> dependency. That plan moves `Conversation` onto the `Agent`; this one removes +> `receiver`. The two are orthogonal: +> - If the session-store work lands first, `run()` takes only `self`. +> - If this work lands first, `run(self, conv)` takes a single `conv` argument +> (just `runStep` minus `receiver`), and the session-store refactor drops +> `conv` later. +> +> Either way `receiver` is gone after Phase 0. Don't block on ordering. + +## Phase 1 — `libpanto-c` (C ABI) + +6. **Opaque handles** for the major types: `PantoAgent`, `PantoStream` (and + whatever construction requires — config, conversation/session store). + Consumers never see the layout. +7. **`panto_next_event(stream, *PantoEvent) -> status`** is the C projection of + `!?Event`. The `!?Event` triple maps onto a status enum plus an out-param: + + | Zig `next()` | C status | out-param | + | ----------------------- | ------------ | ---------------- | + | `Event` value | `EVENT` (0) | filled | + | `null` | `DONE` (1) | untouched | + | `error.X` | `ERROR` (2) | error detail | + +8. **`@export` wrappers** for construction, `run()`, `next_event`, teardown, + and a free function for any owned event payloads. +9. **The C header is committed and hand-maintained, not generated by + `build.zig`.** (Open question 1 below: we are *not* having `build.zig` emit + `panto.h`.) A stable, hand-written `include/panto.h` is the ABI contract — + it should change deliberately, be reviewable in diffs, and not be a build + artifact. `build.zig` emits the shared library and stages the committed + header; it does not author it. +10. **Define the event-marshalling C structs once.** `PantoEvent` (a tagged + union mirroring the Zig `Event`) is read by every downstream binding. + +## Phase 2 — `libpanto-go` (validates `libpanto-c`) + +11. **cgo bindings**: `Agent`, `Stream`, and the raw + `Stream.Next() (Event, bool, error)` (or `(Event, error)` with a separate + done signal) mapping `EVENT`/`DONE`/`ERROR` onto Go's idioms. +12. **`Stream.Iter() iter.Seq[Event]`** — the modern range-over-func form, so + consumers write `for ev := range stream.Iter { ... }`. Single goroutine, + auto-terminates after yielding `MessageComplete`, surfaces a failure via a + trailing `stream.Err()` after the range (the idiomatic Go pattern since + `bufio.Scanner`). This is the primary Go surface. +13. **A channel wrapper too.** Go users expect channels; a goroutine drains + `Next()` into a channel. Cheap over a pull core. Ship both the raw + iterator and the channel form. + +`libpanto-go` is the validation harness for the `libpanto-c` ABI — if Go can +drive a full streaming turn idiomatically, the C surface is sound. + +## Phase 3 — `libpanto-py` (pure Zig CPython extension) + +14. **`build.zig` does `@cImport(@cInclude("Python.h"))`** and emits + `_panto.so` (a loadable extension module). No C in this package. Decide + **stable ABI (`abi3` / limited API)** here to collapse the + CPython-version × platform wheel matrix to one artifact per platform. +15. **`module.zig`** implements `PyModuleDef`, `PyMethodDef`, and an `Agent` + type and `Stream` type as `PyTypeObject`s, calling the **Zig** `libpanto` + API directly. The `Stream` type's `tp_iternext` calls Zig `Stream.next()`: + - `Event` → build and return the `PyObject` event. + - `null` → set `StopIteration` (return `NULL` with no error set). + - `error.X` → set the mapped Python exception. + - Wrap the blocking `next()` in `Py_BEGIN_ALLOW_THREADS` / + `Py_END_ALLOW_THREADS` so streaming doesn't serialize the whole + interpreter; re-acquire the GIL to build the event `PyObject`. +16. **`panto/__init__.py`** is a thin Pythonic surface over the native + `_panto`: a real exception hierarchy, context managers, idiomatic event + objects. **The async generator is pure Python** — wrap the sync iterator + with `asyncio.to_thread` (blocking `next()` runs in a worker thread). No + file descriptors, no ABI change. (See "Out of scope for v1" below.) + +## The one contract that unifies all four packages + +| layer | progress / terminal | exhausted | failure | +| -------- | ------------------------- | ------------------ | ---------------------- | +| Zig | `Event` / `MessageComplete` | `null` | `error.X` | +| C | `PantoEvent` / status `EVENT` | status `DONE` | status `ERROR` | +| Go | `Event` (Iter yields) | Iter stops | `error` / `stream.Err()` | +| Python | event object (yielded) | `StopIteration` | raised exception | + +Design every binding to this single table. Pull-shaped, success-only events, +terminal-by-`MessageComplete`, exhaustion-by-`null`, failure-by-error. + +## Out of scope for v1 (deliberately deferred) + +- **A pollable fd / `panto_step_poll(timeout)`.** This is the only thing that + makes async *natively* non-blocking on both Go (`select`) and Python + (`await` on readiness). It is real work in `libpanto-c` and unnecessary for + v1. **Design the C ABI so a pollable fd can be added later without breaking + the existing pull surface**, but do not build it now. +- **A native async Python API.** Without an fd, async Python is *just* the sync + pull API wrapped in `asyncio.to_thread`. That is pure-Python glue in + `panto/__init__.py`; no native or ABI work, so it isn't a binding deliverable. +- **Languages beyond Go and Python.** `libpanto-c` is the reuse point for any + future cgo-style or cffi-style consumer (Ruby, Node N-API, …); none are in + scope now. + +## Open questions / decisions to finalize before coding + +1. **`panto.h` generation — resolved: no.** The header is committed and + hand-maintained as the ABI contract; `build.zig` stages it but does not emit + it. (Captured in Phase 1, step 9.) +2. **Resumable-handle strategy (state machine vs. thread+queue)** — target the + state machine; decide finally once the provider-loop inversion is scoped. +3. **Exact `Event` variants** — finalize against `ReceiverVTable` + (`provider.zig`) and the tool-dispatch path (`agent.zig`). +4. **What moves onto `Agent` vs. stays per-`run()`** — driven by + `docs/pluggable-session-store.md`'s final shape; `conversation` is moving, + confirm nothing else needs to. +5. **CPython stable-ABI (`abi3`) commitment** — decide in Phase 3 to fix the + wheel matrix early. |