# Extension loading: unified policy, entry/activate, rocks & paths

Status: **implemented** (2026-07-01). Governs the panto CLI's extension/tool
loading, the `[extensions]` config section, and loading luarocks packages and
extra filesystem paths as extension sources.

## Implementation status

Landed and covered by `zig build test` (666 tests green):

- `src/glob.zig` — `Specificity` + `specificity()`, with tests.
- `src/config_file.zig` — single `[extensions]` section; `Policy` is now a
  layered, specificity-sorted rule list resolved by last-match-wins;
  `[tools]` removed; `extensions.paths`/`extensions.rocks` parsed per-layer
  into `Config.ext_paths`/`ext_rocks`.
- `src/lua_runtime.zig` — `Entry` + `evalEntries`/`evalEntriesFromModule`/
  `activateEntry`/`dropEntry`; `loadExtension`/`loadTool` removed. Sugar tool
  tables still work.
- `src/extension_loader.zig` — two-pass (eval → shadow → filter → activate);
  `Source {layer, origin}` precedence (`rocks < paths < dir`, `project > user
  > base`); scans dirs + `paths`, requires `rocks`.
- `src/luarocks_runtime.zig` — `installRock` / `installRockIfMissing`
  (in-process luarocks driver, exact-pin fast-path skip).
- `src/main.zig` — installs `extensions.rocks` (best-effort) then discovers.
- `examples/extensions/{greet,echo}.lua` migrated; `agent/tools/*.lua`
  unchanged (sugar). `wc.lua` unchanged.

Known follow-ups / not yet done:

- **Rocks end-to-end is unverified against a real registry rock** — no such
  rock exists yet and the sandbox has no network. The `require`→entries path
  is unit-tested via `package.preload`; the install path reuses the
  battery-install machinery. First real integration test is pending the
  published `agent.*` rock.
- **Integrity (#9)** — first-party GPG signing + `luarocks --verify` + embedded
  pubkey is designed but NOT wired. Until then `rocks` pins *which version*,
  not *which bytes*. The bespoke sha256 lockfile stays deferred (YAGNI).
- Rock module name is assumed equal to the rock name (spec's first token);
  rocks whose Lua module differs would need a mapping.

Context: a separately-published luarocks package (the `agent.*` suite — skills,
rules, subagents) will register extensions into panto. That suite lives in its
own repo/rock; this work is only the *loading mechanism* in panto core that
lets a rock (or an extra path) contribute extensions, plus the config surface
to select among them.

## Decisions

### 1. One `[extensions]` config section

The old split of `[tools]` (gates registered tool names post-load) and
`[extensions]` (gates script logical names pre-load) collapses into a single
`[extensions]` section governing every lua-authored capability, whether it
comes from the `extensions/` dir, the `tools/` dir, a rock, or an extra path.
No backwards compatibility — `[tools]` is removed.

```toml
[extensions]
allow = ["agent.rules"]
deny  = ["agent.*"]
rocks = ["panto-agent 1.4.2-1"]
paths = ["./dev/my-extension"]
```

### 2. Registration redesign: entry + deferred `activate()`

Every extension source (disk file or rock module) is **always eval'd** and
returns an *entry*, or a list of entries. An entry is `{ name, activate }`:

```lua
return {
    name = "agent.skills",         -- identity; matched against allow/deny
    activate = function()          -- run ONLY if name is permitted
        panto.ext.register_tool { ... }
        panto.ext.register_command { ... }
    end,
}
```

- `name` is the dotted identity the policy matches on.
- `activate()` is deferred: the runtime calls it only when `name` is permitted.
  All `register_tool` / `register_command` / `on` side effects move inside it.
- A source returning a list registers multiple independently-gateable entries
  (this is how one rock contributes many namespaces).
- **Sugar:** a returned table with a `handler` (and no `activate`) is a bare
  tool — normalized internally to an entry whose activation registers itself.
  The built-in `agent/tools/*.lua` keep working unchanged through this path.

**Hard rule:** eval/`require` must be side-effect-free — it only *returns*
entries, never registers. This is what makes Pass 1 (below) safe to run over
*every* candidate, including denied ones.

This collapses the old two-stage gate (pre-load logical name + post-load tool
name) into a single post-eval gate on the declared `name`. Identity is the
declared `name`, uniform across disk/rock/path — there is no filename-derived
identity anymore.

### 3. Two-pass load

- **Pass 1 — availability.** Walk sources in order, eval/`require` everything
  (side-effect-free), collect `name → entry`. Later trumps earlier for the same
  name (shadowing). Shadow key is the declared `name` (the old `(kind, name)`
  key is gone — `kind` is dropped).
- **Pass 2 — activation.** Resolve the policy (below) and call `entry.activate()`
  for every permitted name.

Same-name duplicate **within one source** = error; **across sources** = shadow.

### 4. Source precedence

Cross-layer: `project > user > base` (unchanged).
Within a layer: `rocks < paths < dir` — the canonical `extensions/`/`tools/`
dir is most authoritative locally, an explicit `paths` entry next, a rock least
(most upstream). This lets a local dev copy shadow a packaged rock.

### 5. Policy resolution: specificity + last-match-wins

Neither clobber nor naive union across layers works (both silently drop another
layer's rules; and "carve one out of a denied group" needs allow to beat a
broader deny). The model:

- Every layer contributes `allow`/`deny` glob patterns. **Rules are never
  clobbered or merged away** — all layers' rules are kept.
- Build one ordered rule list: layers low→high; within a layer, sort
  least-specific→most-specific, with `deny` after `allow` at equal specificity.
- Resolve a name by scanning the list and taking the **last** matching rule.
  That yields: the highest layer that touches the name wins, and within it the
  most-specific rule wins; a higher layer's broad rule beats a lower layer's
  specific rule (consistent with panto's higher-layer-authoritative model).
- Default is **allow** (empty policy permits everything). A pure whitelist is
  expressed explicitly as `deny = ["**"]` + specific `allow` carve-outs.

**Specificity** (only compared when two patterns match the same name), in
order: (1) more literal segments win (`a.b.* > a.*`, `a.b.** > a.**`);
(2) more total segments win (`a.*.* > a.*`); (3) fewer `**` win (`a.* > a.**`).

Example — base pins one thing out of an otherwise-denied group, user adds an
unrelated carve-out; both survive:

```toml
# base
[extensions]
deny  = ["agent.*"]
allow = ["agent.rules"]   # more specific than agent.* → wins → agent.rules on
# user
[extensions]
allow = ["foo.bar"]       # unions in, touches nothing else
```

### 6. `extensions.rocks` — the trust boundary

`rocks` is a plain list of **luarocks dependency strings** passed straight
through to embedded luarocks. No custom grammar.

```toml
rocks = ["panto-agent 1.4.2-1"]
```

- luarocks operators: `==` `~=` `<` `>` `<=` `>=` `~>`; no operator = implicit
  `==`; `~>` is pessimistic ("compatible with").
- **`rocks` is the code-execution trust boundary** — listing a rock is the
  "I will run this" decision. `allow`/`deny` is a *feature toggle*, applied
  after eval, and is **not** a security boundary (a hostile rock runs on
  `require` regardless of any deny).
- Install via embedded luarocks into `<data-home>/rocks/...`. Startup only
  installs a rock that is *missing* (exact pin: that version absent; range/bare:
  no version of the name present) — never a per-startup network hit otherwise.
  `panto update` force-(re)installs every `extensions.rocks` entry (the one
  place upgrades/re-resolves happen).
- **Pin gotcha (verified in luarocks v3.13.0 `core/vers.lua`):** `1.4.2`
  without the `-R` revision floats across `1.4.2-1`/`1.4.2-2` — the revision is
  only compared when the constraint carries one. A true pin needs
  `version-revision`. We do *not* enforce this; pin discipline lives in docs.

### 7. `extensions.paths` — extra search dirs

A plain list of filesystem dirs appended to the scan at the listing layer's
precedence. No registry — reuses the dir scan. Relative paths resolve against
cwd (project) for now.

### 8. Integrity (deferred piece)

luarocks provides **no** content-hash pinning: no sha256, no `--checksum`, only
an optional author-set `source.md5` for source archives. `--pin`/`luarocks.lock`
pins *versions, not bytes*. The only native integrity is GPG detached `.asc`
signatures via `luarocks install --verify` (plain gpg against the local
keyring; weak/optional in general).

Plan: for the first-party `agent.*` rocks we publish, sign releases + embed our
public key in panto + install with `--verify` → a real boundary for first-party
code at low cost. Defer (YAGNI) a bespoke sha256 TOFU lockfile for unsigned
third-party rocks. Until that lands, docs must say `rocks` pins *which version*,
not *which bytes*.

## Implementation notes (anchors)

- `src/glob.zig` — add specificity comparator + rule-list resolution.
- `src/config_file.zig` — single `[extensions]` policy; capture per-layer
  `allow`/`deny` in `loadFromPaths` (before the clobbering `mergeTable`),
  build the ordered rule list. Drop `[tools]`. `Policy` becomes a rule list.
- `src/lua_runtime.zig` / `src/lua_bridge.zig` — new unified load path: eval
  file, read returned entry/entries (or sugar tool table), hold `activate`
  refs, activate permitted ones. Keep `register_tool`/`register_command`/`on`.
- `src/extension_loader.zig` — two-pass by declared name; `Source` gains
  ordering for `rocks`/`paths`/dir; scan `paths`; resolve `rocks`.
- `src/luarocks_runtime.zig` / `src/manifest.zig` — install user `rocks`.
- `agent/tools/*.lua` — unchanged (sugar). `examples/extensions/*.lua` —
  migrate to entry/activate.