1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
|
# panto-warming
Session warming for [Pantograph](https://github.com/travisp/pantograph),
modeled on [opencode's warming
feature](https://opencode.ai/v2/docs/warming/). While you pause between
prompts, the extension periodically sends a **minimal transient prompt** on
top of the session's current context so the provider-side prompt cache stays
warm — then throws the prompt and the response away. Nothing is added to the
context window or the session transcript.
Disabled by default. Warming requests are real provider requests: they
consume tokens, cost money, and count against rate limits.
## Install
```toml
[extensions]
rocks = ["panto-warming"]
```
For a local checkout, use `paths = ["/path/to/panto-warming"]` instead. A
`paths` entry loads the code but installs nothing, so put the dependencies
in panto's own rocks tree first (`mise run deps`, or the `luarocks` line it
contains).
## Host compatibility
This source requires Pantograph's callback form
`agent:run_async(request, callback)`; hosts exposing only the older no-callback
job API are unsupported.
## Configuration
Enable with defaults in any layered `config.toml` (the built-in base layer,
`${XDG_CONFIG_HOME:-$HOME/.config}/panto/`, `.panto/`, and `.panto/local/`):
```toml
warming = true
```
Or configure it as a table — its presence enables warming:
```toml
[warming]
prompt = "Do not perform any work. Reply with exactly: OK"
interval = "5 minutes"
duration = "1 hour"
```
| Field | Default | Purpose |
| ---------- | ------------------------------------------------------------------- | -------------------------------------------------------------------------- |
| `prompt` | keep-alive instruction (do no work, no tools, reply `OK`) | appended as a transient user message for each warming request |
| `interval` | `"4 minutes"` | idle time before the next warming request |
| `duration` | `"30 minutes"` | warming window measured from the latest non-warming model request |
| `enabled` | `true` when the table is present | opt out in one layer (`enabled = false`, or the scalar `warming = false`) |
Omitted fields keep their defaults. `interval` and `duration` take duration
strings (`"30 seconds"`, `"4 minutes"`, `"1 hour"`; singular and a few
abbreviations work too) and must be greater than zero — anything else fails
activation with a message naming the file, because a silently misread setting
in a feature that spends money is worse than a loud error.
Settings merge across the config layers, lowest precedence first; later
layers win per key. A higher layer can therefore shorten the interval, extend
the window, or turn warming off entirely.
### Timing
Warming starts when the UI is ready or after any non-warming model request, and any
new real activity resets both timers. A warming request never extends the
window; after each attempt (success or failure) the next one waits out a full
interval:
```
00:00 real activity window = [00:00, 00:30]
00:04 warming request
00:08 warming request
...
00:30 warming stops
```
If `interval >= duration`, the window always expires before a warming
request is due and none is ever sent. Failures are silent (extensions have
no logging channel): the session is unchanged and the next interval retries.
Nothing warms while a real turn is in flight.
## How a warming request works
```mermaid
sequenceDiagram
participant Timer
participant Warm as warm agent (throwaway)
participant Store as session store
participant Provider
Timer->>Store: replay transcript (same as /resume)
Timer->>Warm: panto.agent { conversation, model, tools, null_store }
Timer->>Warm: run_async { transient prompt, dispatch_tools = false }, callback
Warm->>Provider: [session prefix] + [transient user message]
Provider-->>Warm: one response
Warm--)Timer: callback caches termination; response discarded
```
The warm request shares the session's prompt prefix and differs only by the
transient user message, so a provider that caches prefixes (Anthropic with
its `cache_control` breakpoints, OpenAI's prompt caching) sees the shared
prefix again and keeps it hot. Then everything is dropped:
- **Transcript** — reloaded from the session's own JSONL store with the same
machinery `/resume` uses. System blocks, tool inputs, and thinking
signatures replay verbatim, so the messages half of the prefix is
byte-identical to the session's own requests.
- **Model** — `panto.ext.resolve_model` for the session's current
model/reasoning label, inheriting the live credentialed transport.
- **Tools** — the session agent's declarations, copied onto the warm agent.
- **Turn** — one callback-form `run_async` on its own pump thread (the UI
never blocks), with `dispatch_tools = false`: a misbehaving model cannot run
real tools from a keep-alive prompt, and tool calls are handed back
unresolved and discarded with the rest. Progress deliveries are discarded;
only the terminal status/error is cached for the next timer tick.
The warm turn's user message and response live only in the throwaway
agent's conversation and a `null_store` — the live session, its transcript,
and its persistence are never touched.
### Fidelity caveat: tool schemas
Tool declarations cross a decode/re-encode seam when copied to the warm
agent (`agent:tools()` decodes the session's schema JSON, `set_tools`
re-serializes it), and Lua table iteration can reorder JSON object keys
along the way. Both Anthropic and OpenAI key their caches on the **exact**
rendered prefix — tools included, at position 0 — so when the re-encoded
schema bytes differ from the session's own, that warming request cannot
write a cache entry the session's next request will hit, and buys nothing.
This is unfixable from an extension: no binding surface passes raw schema
bytes to a second agent. A small `schema_json` passthrough in
`libpanto-lua` (`agent:tools()` exposing the raw bytes, `set_tools`
accepting them) would make the copy exact. Until that lands, the self-check
measures how often re-encoding drifts on the local codec:
```
$ panto lua _selfcheck.lua
probe: schema re-encode drift 3/4 (tool prefix fidelity degraded; see README)
```
The drift is a property of the shared JSON codec's table iteration, not of
this rock; the messages half of the prefix is unaffected.
## Costs
Warming requests are billed like any other. Shorter intervals increase
request volume; longer durations allow more of them. Enable warming only
when the provider-side benefit is worth the added usage — with the defaults,
one uninterrupted 30-minute window sends at most seven warming requests.
## Development
```sh
mise trust # once
mise run deps # install toml2lua into panto's rocks tree (for `paths` use)
mise run check # self-check plus real-binding HTTP integration
```
`warming/policy.lua` and `warming/config.lua` are pure (durations, layer
merging, the timing state machine) and fully covered by the self-check;
`warming/warm.lua` is the binding glue. `mise run check` also runs
`binding-check.lua` against the real callback binding and local HTTP fixtures.
Release plumbing: `mise run version <x.y.z>`, commit, `mise run release`.
|
