diff options
Diffstat (limited to 'docs/oauth-provider-auth.md')
| -rw-r--r-- | docs/oauth-provider-auth.md | 495 |
1 files changed, 495 insertions, 0 deletions
diff --git a/docs/oauth-provider-auth.md b/docs/oauth-provider-auth.md new file mode 100644 index 0000000..9ae63a7 --- /dev/null +++ b/docs/oauth-provider-auth.md @@ -0,0 +1,495 @@ +# OAuth provider auth + +Goal: make provider authentication a core, reusable capability instead of +embedding auth details directly in each provider entry. A provider should name +the auth session it uses; core resolves that auth session into request-ready +credentials before a provider stream opens. + +This covers today's static API-key providers, GitHub Copilot subscription +login, and OpenAI Codex/ChatGPT subscription login with one configuration +shape: + +```toml +[providers.openai] +style = "openai_chat" +base_url = "https://api.openai.com/v1" +auth = "openai_api" + +[auth.openai_api] +type = "api_key" +key_env_var = "OPENAI_API_KEY" +``` + +Providers no longer carry `api_key` or `api_key_env_var` directly. All +providers use `auth = "<name>"`; all credentials live under `[auth.<name>]`. +OAuth is just one family of auth configuration types in that namespace. + +--- + +## Why config-only core auth + +The earlier Copilot design put most auth behavior in Lua: HTTP requests, token +storage, refresh-before-turn, config mutation, and retry-after-401. With named +`[auth.<name>]` blocks, that split no longer buys enough. Auth can be entirely +described in config and executed by core. + +Core should own: + +- loading and validating named auth blocks +- API-key resolution from literal config or environment +- OAuth login flows +- token persistence under `$PANTO_HOME` +- refresh-before-turn with an expiry margin +- refresh-after-401/403 with one retry +- final request-header injection +- dynamic `base_url` updates when an auth exchange returns one + +Lua is not required for API-key, GitHub Copilot, or OpenAI Codex auth +correctness. Extensions may still package optional provider presets or UI +polish, but the flagship auth stories should work from config alone. + +--- + +## OAuth flow choices + +`oauth_pkce` means OAuth 2.0 Authorization Code flow with PKCE. + +PKCE is "Proof Key for Code Exchange." The client creates a one-time secret +called a `code_verifier`, sends only a hashed `code_challenge` in the browser +authorization URL, then later proves it owns the original verifier when it +exchanges the returned authorization code for tokens. This protects native +apps and CLIs that cannot safely keep a client secret. + +For pantograph, `oauth_pkce` is the browser-login flow: + +1. Start a short-lived localhost callback server. +2. Generate PKCE verifier/challenge and state. +3. Open or print an authorization URL. +4. Receive the callback with `code` and `state`. +5. Exchange the code + verifier at the token endpoint. +6. Persist returned tokens and refresh them later. + +This is different from `oauth_device`, which prints a URL and user code, then +polls until the browser-side authorization completes. Device flow is better for +headless or remote terminals because it needs no local callback server. + +For the first implementation, prefer `oauth_device` if it can cover both +flagship user stories: + +1. It avoids callback URLs, so it works naturally through SSH and remote + servers where `localhost` is ambiguous. +2. It avoids implementing a local Zig web server in the first pass. +3. GitHub officially positions device flow for headless apps and CLIs. +4. OpenAI Codex already supports device-code login for headless environments. + +Keep `oauth_pkce` in the design as a known later extension, not as a first-pass +requirement, unless Codex subscription auth proves device flow is unavailable +for the accounts we need to support. + +--- + +## Configuration model + +### API key auth + +```toml +[auth.openai_api] +type = "api_key" +key_env_var = "OPENAI_API_KEY" +``` + +or: + +```toml +[auth.local_proxy_key] +type = "api_key" +key = "sk-..." +``` + +Rules: + +- `key` wins over `key_env_var` if both are present. +- if `key_env_var` is set but absent from the environment, the auth session is + unresolved +- a provider with unresolved auth may remain visible/selectable, but the first + request should fail with a clear auth error unless the auth type can launch an + interactive login +- literal `key` is supported for completeness, but env vars remain preferred + +### OAuth device auth + +GitHub Copilot: + +```toml +[auth.github_copilot] +type = "oauth_device" +dialect = "token" +client_id = "Iv1.b507a08c87ecfe98" +device_code_url = "https://github.com/login/device/code" +token_url = "https://github.com/login/oauth/access_token" +scope = "read:user" +token_request_format = "form" # or "json"; default TBD +``` + +OpenAI Codex: + +```toml +[auth.openai_codex] +type = "oauth_device" +dialect = "codex" +client_id = "app_EMoamEEZ73f0CkXaXp7hrann" +issuer = "https://auth.openai.com" +device_code_url = "https://auth.openai.com/api/accounts/deviceauth/usercode" +device_poll_url = "https://auth.openai.com/api/accounts/deviceauth/token" +verification_url = "https://auth.openai.com/codex/device" +token_url = "https://auth.openai.com/oauth/token" +``` + +Core owns requesting the device code, presenting `verification_uri` and +`user_code`, polling on `interval`, and storing the durable OAuth token. +`dialect` selects the device-flow completion shape: + +- `token` — standard OAuth device flow. The poll endpoint returns the OAuth + token response directly, including `access_token` and optional + `refresh_token` / `id_token`. +- `codex` — OpenAI Codex device flow. The poll endpoint returns an + authorization code plus PKCE verifier data; core exchanges those at + `token_url` to obtain `id_token`, `access_token`, and `refresh_token`. + +### OAuth PKCE auth (deferred) + +```toml +[auth.openai_codex] +type = "oauth_pkce" +client_id = "app_EMoamEEZ73f0CkXaXp7hrann" +issuer = "https://auth.openai.com" +authorize_url = "https://auth.openai.com/oauth/authorize" +token_url = "https://auth.openai.com/oauth/token" +redirect_port = 1455 +scopes = [ + "openid", + "profile", + "email", + "offline_access", + "api.connectors.read", + "api.connectors.invoke", +] + +[auth.openai_codex.authorize_params] +id_token_add_organizations = "true" +codex_cli_simplified_flow = "true" +``` + +Core can later own PKCE generation, callback handling, code exchange, token +persistence, JWT expiry parsing, and refresh-token grants. This is explicitly +not required for the first pass if `oauth_device` covers Copilot and Codex. + +### Optional token exchange + +Some OAuth flows do not return the token that should be sent to the model API. +GitHub Copilot is the important example: GitHub device auth returns `ghu_...`, +then Copilot requires a second exchange to get a short-lived chat token and API +endpoint. + +```toml +[auth.github_copilot.exchange] +method = "GET" +url = "https://api.github.com/copilot_internal/v2/token" +bearer = "oauth_access_token" +token_json_path = "token" +expires_at_json_path = "expires_at" +base_url_json_path = "endpoints.api" + +[auth.github_copilot.exchange.headers] +User-Agent = "GitHubCopilotChat/0.26.7" +Editor-Version = "vscode/1.99.0" +Editor-Plugin-Version = "copilot-chat/0.26.7" +Copilot-Integration-Id = "vscode-chat" +``` + +The exchange result becomes the request credential used by the provider. It may +also override the provider `base_url`. + +### Provider reference + +```toml +[providers.copilot] +style = "openai_chat" +base_url = "https://api.individual.githubcopilot.com" # fallback/placeholder +auth = "github_copilot" + +[providers.copilot.extra_headers] +User-Agent = "GitHubCopilotChat/0.26.7" +Editor-Version = "vscode/1.99.0" +Editor-Plugin-Version = "copilot-chat/0.26.7" +Copilot-Integration-Id = "vscode-chat" +X-Initiator = "user" +``` + +```toml +[providers.codex] +style = "openai_chat" # pending endpoint verification +base_url = "https://chatgpt.com/backend-api" # placeholder +auth = "openai_codex" +``` + +`extra_headers` remains a provider capability because it applies to the model +request, not necessarily to the auth endpoints. Auth exchanges can have their +own headers. + +--- + +## GitHub Copilot auth + +Verified against opencode and copilot-api reference implementations. + +Two tokens, two phases: + +1. **One-time browser login (OAuth device flow).** Yields a long-lived OAuth + token (`ghu_...`). This is the durable credential; persist it. The device + flow needs no redirect URI and no local HTTP server. +2. **Per-~30-min token exchange.** GET the Copilot token endpoint with the + `ghu_` token as bearer; receive a short-lived API token, its expiry, and the + API base URL. Re-run with the same `ghu_` token to refresh. + +Constants: + +```text +CLIENT_ID = "Iv1.b507a08c87ecfe98" +DEVICE_CODE_URL = "https://github.com/login/device/code" +ACCESS_TOKEN_URL = "https://github.com/login/oauth/access_token" +TOKEN_URL = "https://api.github.com/copilot_internal/v2/token" +SCOPE = "read:user" +GRANT_TYPE = "urn:ietf:params:oauth:grant-type:device_code" +``` + +Device-code request: + +```json +{ "client_id": CLIENT_ID, "scope": "read:user" } +``` + +Poll request: + +```json +{ "client_id": CLIENT_ID, "device_code": "...", "grant_type": GRANT_TYPE } +``` + +Copilot exchange response: + +```json +{ + "token": "...", + "expires_at": 1700000000, + "refresh_in": 1500, + "endpoints": { + "api": "https://api.individual.githubcopilot.com" + } +} +``` + +`token` is opaque and becomes the provider bearer credential. +`endpoints.api` becomes the runtime `base_url`. + +Required Copilot headers ride on the token exchange and chat requests: + +```text +User-Agent: GitHubCopilotChat/<ver> +Editor-Version: vscode/<ver> +Editor-Plugin-Version: copilot-chat/<ver> +Copilot-Integration-Id: vscode-chat +``` + +Chat requests commonly also send `X-Initiator: user` or `agent`. + +--- + +## OpenAI Codex auth + +Official Codex documentation describes two OpenAI sign-in methods: + +- ChatGPT sign-in for subscription access +- API-key sign-in for usage-based access + +The open-source Codex client currently implements ChatGPT sign-in with OAuth +PKCE and also supports a device-code variant for headless environments. For +pantograph, prefer the device-code variant first. + +Device-code login uses the ChatGPT auth issuer plus Codex-specific device auth +endpoints: + +```text +issuer/client auth base: https://auth.openai.com +device user-code URL: https://auth.openai.com/api/accounts/deviceauth/usercode +device poll URL: https://auth.openai.com/api/accounts/deviceauth/token +browser verification: https://auth.openai.com/codex/device +token endpoint: https://auth.openai.com/oauth/token +client_id: app_EMoamEEZ73f0CkXaXp7hrann +``` + +The device poll response returns an authorization code plus PKCE verifier data; +Codex then exchanges that code at the token endpoint. That means OpenAI's +device flow is not identical to GitHub's, but it still has the CLI-friendly +property we care about: no local callback URL and no local web server. + +Browser/PKCE login uses: + +```text +issuer/client auth base: https://auth.openai.com +token endpoint: https://auth.openai.com/oauth/token +client_id: app_EMoamEEZ73f0CkXaXp7hrann +default callback port: 1455 +fallback callback port: 1457 +``` + +The token exchange returns: + +```json +{ + "id_token": "...", + "access_token": "...", + "refresh_token": "..." +} +``` + +Core should persist these as one auth session. Refresh uses +`grant_type = "refresh_token"` against the same token endpoint. The access +token is a JWT in normal ChatGPT auth sessions, so core can refresh +proactively when its `exp` claim is within the safety margin. + +Codex-backed requests use: + +```text +Authorization: Bearer <access_token> +ChatGPT-Account-ID: <account_id> # when present +X-OpenAI-Fedramp: true # when the ID token says FedRAMP +``` + +Open question: the exact provider wire endpoint pantograph should call for +ChatGPT subscription-backed Codex use. The auth machinery is clear; the model +request shape still needs verification before we commit to treating this as a +plain `openai_chat` provider. + +--- + +## Core implementation plan + +### C1. Parse named auth blocks + +Add `Config.auths: []AuthConfig` and make `Provider.auth` required for +networked providers. Move `api_key` / `api_key_env_var` out of providers and +into `[auth.<name>]`. + +Compatibility path: accept provider-level `api_key` and `api_key_env_var` for +one release by synthesizing hidden auth entries, then warn. + +### C2. Resolve provider auth before each request + +The active provider config should be built from: + +- provider transport fields (`style`, `base_url`, `extra_headers`) +- selected model alias +- resolved auth session (`api_key`, OAuth access token, exchanged token, extra + auth-derived headers, dynamic base URL) + +This replaces the old "drop providers whose env var is absent" behavior. A +provider can survive config resolution even if its auth is not currently +resolved. + +### C3. Core HTTP client + +Core auth needs HTTPS for OAuth and token exchange. Implement a small +request/response HTTP helper backed by Zig `std.http.Client`. Because this is +core Zig auth, it does not need the Lua coroutine surface from the old design. +If Lua still needs HTTP later, expose a separate `panto.http` wrapper. + +### C4. Token storage + +Store under `$PANTO_HOME/auth/<name>.json` initially: + +```json +{ + "type": "oauth_device", + "access_token": "...", + "refresh_token": "...", + "id_token": "...", + "expires_at": 1700000000, + "exchange": { + "token": "...", + "expires_at": 1700000000, + "base_url": "https://api.individual.githubcopilot.com" + } +} +``` + +Only fields relevant to the auth type are present. Treat these files like +passwords. A future credential-store backend can keep the same logical API. + +### C5. Refresh lifecycle + +Before opening a provider stream: + +1. Load the named auth session. +2. If absent and interactive login is possible, run login. +3. If token is within the refresh margin, refresh. +4. If an exchange is configured and stale, run the exchange. +5. Build request auth headers and dynamic provider config. + +On 401/403: + +1. Force one refresh/exchange. +2. Retry the same turn once. +3. Surface the original provider error if it still fails. + +### C6. Extra headers + +Add generic `extra_headers` to `OpenAIChatConfig` and +`AnthropicMessagesConfig`, then thread config TOML through libpanto. + +Auth-derived headers and provider `extra_headers` should merge deterministically. +Provider request-specific defaults should not leak into OAuth endpoints unless +configured under `[auth.<name>.exchange.headers]`. + +### C7. No Lua dependency + +Built-in auth should not depend on Lua. Optional Lua surfaces can come later if +they are useful: + +- auth status query +- command to trigger login/logout +- UI overrides for device-code prompts +- provider packages that install config/model defaults + +None of these are required for Copilot or Codex auth correctness. + +--- + +## Tests + +- API-key auth: literal key, env var present, env var absent, provider survives + with clear unresolved-auth error. +- Config migration: provider-level legacy key fields synthesize auth entries. +- OAuth device: device-code response parsing, polling pending/success/error, + persisted token shape. +- OAuth device: GitHub-style direct device-token polling. +- OAuth device: OpenAI Codex-style device polling followed by authorization-code + exchange. +- OAuth PKCE later: authorize URL construction, state validation, code + exchange, token persistence. +- JWT expiry parsing: refresh inside safety margin, no-op when fresh. +- Token exchange: Copilot `token`, `expires_at`, and `endpoints.api` mapping. +- Request headers: bearer auth, `ChatGPT-Account-ID`, FedRAMP header, + provider `extra_headers`, exchange-only headers. +- 401 retry: refresh/exchange once and retry the same turn. + +## Open questions + +- Exact TOML names: `key_env_var` vs. current `api_key_env_var`; this doc uses + `key_env_var` because the `[auth]` section already establishes the domain. +- Whether `auth = "<name>"` should be mandatory for no-auth local providers or + whether `auth = null` / omitted means unauthenticated. +- How much of OpenAI Codex device auth to model generically versus as an + OpenAI-specific device-flow dialect. +- The correct Codex subscription-backed model request endpoint and wire API. +- Whether to add OS keychain storage in the first pass or after file-backed + auth works end to end. |