diff options
Diffstat (limited to 'docs/step19-cli-panto-module-plan.md')
| -rw-r--r-- | docs/step19-cli-panto-module-plan.md | 144 |
1 files changed, 0 insertions, 144 deletions
diff --git a/docs/step19-cli-panto-module-plan.md b/docs/step19-cli-panto-module-plan.md deleted file mode 100644 index f162403..0000000 --- a/docs/step19-cli-panto-module-plan.md +++ /dev/null @@ -1,144 +0,0 @@ -# Step 19 plan: CLI consumes `libpanto-lua` as a Lua module (Option A) - -Goal: inside the panto CLI's embedded Lua VM, `require('panto')` returns the -**native** `libpanto-lua` agent/stream table **plus** `panto.ext`. The CLI -obtains that table only through Lua's normal module machinery (`require`) — it -never calls `luaopen_panto` directly and knows nothing of the module's -internals. The single invariant relied upon: `luaopen_panto` returns a fresh -table (already true). - -## Mechanism - -1. **Embed `panto.so` in the CLI binary and stage it at bootstrap** into the - versioned rocks tree's C-module dir (`<tree>/lib/lua/5.4/panto.so`), which - is already on `package.cpath` (see `configurePackagePaths`). The staged - `.so` leaves `lua_*` undefined and resolves them against the host binary at - `dlopen` time — identical to how `luv.so` already works (`exe.rdynamic = - true`). - -2. **After luarocks bootstrap configures cpath**, the CLI: - ```lua - local panto = require('panto') -- finds staged panto.so - panto.ext = <ext table built in Zig> -- augment host's fresh copy - package.preload['panto'] = function() return panto end -- any later require gets ext too - ``` - `require` caches the augmented table in `package.loaded['panto']`, and the - preload entry (searcher slot 1) makes any *future* `require('panto')` — e.g. - from an extension — return the same augmented table without re-hitting cpath. - -## File-by-file changes - -### `build.zig` (CLI) -- Add `libpanto-lua` as a path dependency (`build.zig.zon`). -- Get its compiled shared object via `dep.artifact("panto").getEmittedBin()`. -- New codegen step `gen_panto_so_embed.zig`: emit a Zig module that - `@embedFile`s the `.so` bytes (binary embed is fine). Mirror - `generateLuaHeadersEmbed`. Materialize the `.so` next to the generated file - via `addCopyFile` so `@embedFile` can reach it. -- Add the embed module import to `exe_mod` and `test_mod` as - `embedded_panto_so`. - -### new `src/embedded_panto_so` import -- A single `pub const bytes: []const u8 = @embedFile("panto.so");` style module - (generated). Name: `embedded_panto_so`. - -### `src/luarocks_runtime.zig` -- New step `stagePantoModule`: write `embedded_panto_so.bytes` to - `<lib_lua_dir>/panto.so` via `writeIfDifferent`. Call it from `bootstrap` - after the tree dirs exist (alongside `stageLuaHeaders`). -- `lib_lua_dir` must exist (create if missing) before writing. - -### `src/lua_bridge.zig` -- **Stop building the native `panto` table and stop installing the preload - loader.** `install()` becomes "create the registrations tables + build the - `ext` table, stash `ext` in a registry slot" — it no longer fabricates a - `panto` table or touches `package.preload`. -- Keep all `ext` thunks (`register_tool`/`register_command`/`on`/`emit`) and - the registrations-table machinery unchanged. -- `pushPantoTable` is repurposed (or replaced by `pushExtTable`): internal Zig - code that today adds members to `panto` (`installEmit`, `_record_result`, - wrapper closure) must target the **augmented native** table after it is - obtained. Decision below. - -### `src/lua_runtime.zig` -- `create()` no longer relies on a Zig-built `panto` table. Split startup: - - `create()` opens the state, `openlibs`, builds the `ext` table + regs - tables (via the slimmed `lua_bridge.install`), creates the EventBridge. - **Defer** `installEmit` until the native table exists. - - New `installPantoModule(self)`: run the `require('panto')` + attach-`ext` + - set-`preload` sequence in Zig (a `luaL_dostring`-style snippet plus stack - surgery), then stash the augmented table in the registry slot so - `pushPantoTable` keeps working, then `installEmit`. -- `installScheduler` already runs after bootstrap; `installPantoModule` runs - between bootstrap and `installScheduler` (it needs cpath configured; the - scheduler's `_record_result`/wrapper need the table present). - -### `src/main.zig` -- After `luarocks_runtime.bootstrap(...)` returns (cpath configured, `panto.so` - staged) and before `rt.installScheduler()`, call `try rt.installPantoModule()`. - -## Sequencing (the hazard) - -Today, ordering is: `create` (builds table + preload + installEmit) → -bootstrap → installScheduler (adds `_record_result`, wrapper, caches uv.run). - -New ordering: -1. `create` — state, libs, `ext` + regs tables, EventBridge. **No table, no - preload, no emit yet.** -2. `bootstrap` — stages `panto.so`, configures cpath/path, installs luv. -3. `installPantoModule` — `require('panto')`, attach `ext`, set preload, stash - in registry slot, `installEmit`. -4. `installScheduler` — `_record_result` + wrapper closure (both reach the now- - augmented native table via the registry slot) + cache uv.run. -5. extension loading — `require('panto')` returns the augmented table. - -Everything that previously assumed "the `panto` table exists from `create`" -moves to step 3. The only consumers are `installEmit` and the scheduler, both -already post-bootstrap-or-movable. - -## Open decision: registry slot semantics - -`lua_bridge.panto_table_key` currently holds the Zig-built table. Two choices: - -- **(chosen)** Keep the slot; in `installPantoModule`, after building the - augmented table, `lua_rawsetp` it into `panto_table_key`. `pushPantoTable` - is unchanged and now yields the native+ext table. `installEmit` / - `_record_result` / wrapper all keep using `pushPantoTable`. Minimal churn. -- Alternative: split into `pushExtTable` (for `ext` members) vs `pushPantoTable` - (whole module). More precise but more edits. Not needed — `ext` lives on the - module table, so one slot pointing at the module suffices. - -## Tests to update / add - -- `lua_bridge.zig` tests assume `install` builds a `panto` table reachable via - `require('panto')`/`pushPantoTable`. These must change: either keep a tiny - test-only table builder, or rewrite the tests to drive `ext` directly. Since - `ext` is what those tests actually exercise (register_tool/command/on), point - them at the `ext` table via a test helper that fabricates a minimal module - table `{ ext = <ext> }` and stashes it — preserving coverage without the - native `.so`. -- `lua_runtime.zig` tests call `LuaRuntime.create` and `loadExtension` without - bootstrap, so `require('panto')` would fail there (no staged `.so`, no - preload). Add a test-only `installTestPantoTable(self)` that fabricates the - `{ ext = <ext> }` module table and registers the preload loader, mirroring - what the old `create` did — so the runtime tests keep passing without a real - native module. `writeTempScript` already prepends `require('panto')`. -- New integration check (best-effort, may be manual): in a built CLI, - `require('panto')` yields a table with both `agent` (native) and `ext`. - -## Risks - -- **Binary size**: embedding `panto.so` (~5.6 MB debug) bloats the CLI binary. - ReleaseFast/strip will shrink it; acceptable (luv and the agent tree are also - embedded). Confirm size is tolerable; consider stripping the `.so` in the - module's ReleaseSafe/Fast build. -- **Two Lua builds must be ABI-identical**: the CLI's statically-linked Lua - (5.4.7) and the headers `panto.so` was compiled against (5.4.7, same pin). - Both come from the same `lua_src` tarball hash — consistent by construction. -- **`dep.artifact("panto")`**: `libpanto-lua/build.zig` currently emits the - `.so` through `addLibrary` + a manual install-file rename. `artifact("panto")` - returns the `addLibrary` Compile step (named `panto`), whose `getEmittedBin()` - is `libpanto.so`/`libpanto.dylib`. Embedding by bytes is name-agnostic; we - stage it as `panto.so`. Verify the dependency exposes the artifact (it must be - `b.installArtifact`'d or at least addressable; may need a small tweak to - `libpanto-lua/build.zig` to expose it cleanly to dependents). |