1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
|
//! Batch-dispatched tool extension API: `ToolSource`.
//!
//! Where `Tool` is a single, thread-safe handler (one tool, one vtable,
//! reentrant), `ToolSource` is a single owner of many tools whose runtime
//! prefers to receive calls in *batches* on a single thread.
//!
//! Motivation: Lua. A Lua extension runtime maintains one long-lived
//! `lua_State` so that module-globals, lazy connection pools, rate
//! limiters, etc. survive across calls. A single `lua_State` is not safe
//! for concurrent host entry, so the runtime can't satisfy `Tool`'s
//! thread-safety contract directly. The runtime *can* dispatch many calls
//! cooperatively (coroutines + an event loop), but it needs to be told
//! all of them at once.
//!
//! The contract libpanto provides:
//!
//! - For a given turn, every `ToolUse` block whose tool name belongs to
//! a particular source is delivered in a single `invoke_batch` call,
//! on one thread.
//! - Distinct sources still execute concurrently (one OS thread per
//! source per turn), so a Lua source and a native source can run in
//! parallel.
//! - Single `Tool` registrations are unchanged. They each get their own
//! thread when they appear alongside other tool calls in a turn.
//!
//! The "thread-safe" promise that `Tool.invoke` carries relaxes to
//! "coroutine-safe within the source's runtime" for source-backed tools —
//! enforcement is the source's problem.
const std = @import("std");
const Allocator = std.mem.Allocator;
const tool = @import("tool.zig");
/// Tool metadata: re-exported from `tool.zig`, which owns the more atomic
/// type. `ToolSource`s declare their tools this way because they share a
/// single dispatch path.
pub const ToolDecl = tool.ToolDecl;
pub const ResultPart = tool.ResultPart;
pub const ResultParts = tool.ResultParts;
/// One pending invocation passed to `invoke_batch`. Slices borrowed from
/// the caller for the duration of the call.
pub const Call = struct {
/// Which of the source's declared tools this call targets.
tool_name: []const u8,
/// Raw JSON bytes the provider sent. Borrowed.
input: []const u8,
};
/// Result for a single call. Mirrors the success/error split of
/// `Tool.invoke`'s return shape. Owned by the caller-supplied allocator.
pub const CallResult = union(enum) {
/// Owned parts (the `ResultParts` slice + each part's bytes), freed by
/// libpanto after assembling the ToolResult block (see
/// `tool.ResultParts.deinit`).
ok: ResultParts,
err: anyerror,
};
/// A grouped tool runtime.
pub const ToolSource = struct {
/// Diagnostic name; surfaced in error messages and logs. Example
/// values: `"panto-lua"`, `"panto-python"`. Borrowed; lifetime owned
/// by the source.
name: []const u8,
/// Tool metadata for every tool this source owns. Borrowed.
tools: []const ToolDecl,
ctx: *anyopaque,
vtable: *const VTable,
pub const VTable = struct {
/// libpanto guarantees: for a given turn, every ToolUse call
/// whose tool name belongs to this source is delivered in one
/// `invoke_batch`, on one thread. Different sources still
/// execute in parallel.
///
/// `calls` and `results` are parallel arrays of length N.
/// `results` is pre-allocated by libpanto; the source fills each
/// slot. The source decides internal scheduling — sequential,
/// coroutine fan-out, worker pool, etc.
///
/// Two failure modes, both normally model-visible:
/// - Per-call: record `.{ .err = e }` in a `results[i]` slot.
/// That call gets an error `ToolResult`; siblings are
/// unaffected.
/// - Whole-batch: return an error from this function. libpanto
/// frees any `ok` slots already filled and maps the error onto
/// *every* member call as an error `ToolResult`.
/// In both cases the agent loop continues so the model can react.
/// Only hard host failures (`error.Canceled`, `error.OutOfMemory`)
/// abort the whole turn and propagate to the embedder.
invoke_batch: *const fn (
ctx: *anyopaque,
calls: []const Call,
results: []CallResult,
allocator: Allocator,
) anyerror!void,
/// Called when the source is removed from the registry or the
/// registry is torn down. Frees any resources owned by `ctx`,
/// including `ctx` itself if heap-allocated.
deinit: *const fn (ctx: *anyopaque, allocator: Allocator) void,
};
};
|
