feat(auth): sign in with a ChatGPT subscription for inference

Add an OAuth-based path to run Strix on a user's ChatGPT Plus/Pro
subscription instead of a metered API key, modeled on OpenAI's Codex CLI.

Auth:
- strix/auth: Codex OAuth login (authorization-code + PKCE), a 0600 token
  store, refresh-on-expiry, and an AsyncOpenAI client that routes inference
  through the ChatGPT backend (chatgpt.com/backend-api/codex) with a
  per-request auth hook so long scans survive token expiry.
- `strix auth login|logout|status` CLI (browser loopback on :1455 with a
  manual-paste fallback); STRIX_AUTH_MODE=subscription persisted to config.

Inference wiring:
- Subscription branch in configure_sdk_model_defaults installs the Codex
  client and the Responses API.
- _CodexResponsesModel always streams (the backend rejects non-streamed
  requests) and aggregates back for the non-streaming get_response path.
- store=false + encrypted reasoning for the stateless backend; models
  coerced to plan-available names (default gpt-5.4 — 5.5+ apply stricter
  content moderation that interferes with security testing).

UX / reporting:
- Track tokens but report $0.00 in the TUI, completion panel, and web
  viewer run details; record auth_mode in run.json and PostHog/Scarf.
- Graceful, actionable errors for unavailable models and expired sign-in.
- Restyled OAuth callback page (Strix branding + link to strix.ai).

Tests: PKCE/URL/redirect parsing, token refresh + account-id, streaming
aggregation, cost zeroing, CLI routing/provider aliasing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Jonathan Singer
2026-07-22 16:03:24 -04:00
co-authored by Claude Fable 5
parent 89a707ff51
commit d35af02e47
26 changed files with 1839 additions and 88 deletions
+17
View File
@@ -267,6 +267,23 @@ export STRIX_REASONING_EFFORT="high" # control thinking effort (default: high,
> [!NOTE]
> Strix automatically saves your configuration to `~/.strix/cli-config.json`, so you don't have to re-enter it on every run.
#### Sign in with a ChatGPT subscription
Instead of a metered API key, you can run Strix on your ChatGPT Plus/Pro subscription:
```bash
strix auth login chatgpt # opens your browser to sign in with ChatGPT
strix --target ./app-directory
strix auth status # show the active sign-in
strix auth logout # revert to API-key billing
```
This uses OpenAI's Codex OAuth flow: inference is billed to your ChatGPT plan rather than per token. Strix defaults to `gpt-5.4` here — newer models apply stricter content moderation that interferes with security-testing prompts, so `gpt-5.4` is recommended for scans. You can override the model with `strix auth login chatgpt --model <name>`. Note that the models a ChatGPT plan exposes are a narrower set than the OpenAI API. If the browser can't open, the command falls back to pasting the redirect URL by hand. Tokens are stored in `~/.strix/subscription-auth.json` (`0600`) and refreshed automatically.
> [!NOTE]
> Using a ChatGPT subscription outside OpenAI's own products is not officially supported by OpenAI and may be subject to its terms of use. For unattended/CI runs, prefer an API key.
**Recommended models for best results:**
- [OpenAI GPT-5.4](https://openai.com/api/) - `openai/gpt-5.4`
+4
View File
@@ -213,6 +213,7 @@ ignore = [
# Test doubles use fixture tokens/passwords and match a callee signature whose
# args they intentionally ignore.
"tests/test_viewer_auth.py" = ["S105", "S106", "ARG001"]
"tests/test_codex_auth.py" = ["S105", "S106", "SLF001"]
"tests/test_report_pdf.py" = ["S105", "S106"]
# Stdlib HTTP handler overrides (do_GET/do_POST) and lazy imports that avoid a
# circular dependency with strix.telemetry / strix.viewer.report_pdf.
@@ -254,6 +255,9 @@ ignore = [
"strix/report/state.py" = ["TC003", "PLR0912", "PLR0915", "E501", "PERF401"]
"strix/report/usage.py" = ["PLC0415"]
"strix/config/models.py" = ["PLC0415"]
# Heavy inference deps (httpx, openai) imported lazily so auth-status checks
# don't pull them in.
"strix/auth/codex.py" = ["PLC0415"]
# Interface utility branches per scope-mode / target-type combination;
# splitting would obscure the decision tree without simplifying it.
"strix/interface/utils.py" = ["PLR0912", "BLE001", "PLC0415"]
+12
View File
@@ -0,0 +1,12 @@
"""Subscription-based model authentication.
Strix normally authenticates to a model provider with an API key. This package
adds a second path: signing in with a model *subscription* (currently ChatGPT
Plus/Pro, via the OpenAI Codex OAuth flow) and using that for inference instead
of metered API billing.
- :mod:`strix.auth.store` — the on-disk credential store
(``~/.strix/subscription-auth.json``, ``0600``), one record per provider.
- :mod:`strix.auth.codex` — the ChatGPT (Codex) OAuth flow, token refresh, and
the OpenAI client that routes inference through the ChatGPT backend.
"""
+412
View File
@@ -0,0 +1,412 @@
"""ChatGPT (Codex) subscription OAuth: login, token refresh, and the OpenAI
client that routes inference through the ChatGPT backend.
This lets a user run Strix on their ChatGPT Plus/Pro subscription instead of a
metered OpenAI API key. The mechanism mirrors OpenAI's own Codex CLI: an OAuth
2.0 authorization-code flow with PKCE against ``auth.openai.com``, whose access
token is then sent as a ``Bearer`` token to ``chatgpt.com/backend-api/codex``
(the subscription endpoint) rather than to the metered ``api.openai.com``.
Terms of service: using a ChatGPT subscription outside OpenAI's own products is
not officially supported by OpenAI and may be against its terms. Strix identifies
as the Codex client (``originator: codex_cli_rs``) because the backend requires
it. This path exists as an option; the user chooses it knowingly.
The OAuth constants (client id, endpoints, headers, model names) are copied from
OpenAI's Codex CLI and must be kept in step with it if OpenAI changes them.
"""
from __future__ import annotations
import base64
import contextlib
import hashlib
import json
import secrets
import threading
import time
import urllib.error
import urllib.parse
import urllib.request
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from openai import AsyncOpenAI
from strix.auth import store
PROVIDER = "codex"
# --- OAuth client (from openai/codex) --------------------------------------
CLIENT_ID = "app_EMoamEEZ73f0CkXaXp7hrann"
AUTHORIZE_URL = "https://auth.openai.com/oauth/authorize"
TOKEN_URL = "https://auth.openai.com/oauth/token" # noqa: S105 - URL, not a secret
CALLBACK_HOST = "localhost"
CALLBACK_PORT = 1455
CALLBACK_PATH = "/auth/callback"
REDIRECT_URI = f"http://{CALLBACK_HOST}:{CALLBACK_PORT}{CALLBACK_PATH}"
SCOPE = "openid profile email offline_access"
# --- inference (ChatGPT subscription backend) ------------------------------
CODEX_BASE_URL = "https://chatgpt.com/backend-api/codex"
ORIGINATOR = "codex_cli_rs"
_ACCOUNT_CLAIM = "https://api.openai.com/auth"
# Models the ChatGPT Codex backend serves to subscription accounts (bare names,
# no provider prefix). Note this set is NARROWER than the OpenAI API and is
# account-specific: the ``*-codex`` variants are API-only and are rejected for a
# ChatGPT account ("model is not supported when using Codex with a ChatGPT
# account"). These are the general GPT-5.x models a Plus/Pro plan exposes.
#
# Default is gpt-5.4: gpt-5.5 and newer apply stricter content moderation that
# interferes with security-testing prompts, so we prefer 5.4 for scans.
DEFAULT_CODEX_MODEL = "gpt-5.4"
CODEX_MODELS: tuple[str, ...] = (
"gpt-5.4",
"gpt-5.5",
)
_TOKEN_TIMEOUT = 30
# Refresh a little before the token actually expires so a request never goes out
# with a token that lapses in flight.
_EXPIRY_SKEW_S = 300
_refresh_lock = threading.Lock()
class CodexAuthError(Exception):
"""A Codex auth step failed. ``code`` is a stable, machine-readable reason."""
def __init__(self, code: str, message: str | None = None) -> None:
self.code = code
super().__init__(message or code)
# --- PKCE + authorization ---------------------------------------------------
def _b64url(raw: bytes) -> str:
"""Unpadded base64url (RFC 7636). Padding here breaks the challenge check."""
return base64.urlsafe_b64encode(raw).rstrip(b"=").decode("ascii")
def generate_pkce() -> tuple[str, str]:
"""Return ``(verifier, challenge)`` for a fresh PKCE exchange."""
verifier = _b64url(secrets.token_bytes(64))
challenge = _b64url(hashlib.sha256(verifier.encode("ascii")).digest())
return verifier, challenge
def create_state() -> str:
"""Return a random ``state`` value for CSRF protection on the callback."""
return secrets.token_hex(16)
def build_authorize_url(challenge: str, state: str) -> str:
"""Build the browser URL the user visits to approve the sign-in."""
params = {
"response_type": "code",
"client_id": CLIENT_ID,
"redirect_uri": REDIRECT_URI,
"scope": SCOPE,
"code_challenge": challenge,
"code_challenge_method": "S256",
"state": state,
"id_token_add_organizations": "true",
"codex_cli_simplified_flow": "true",
"originator": ORIGINATOR,
}
return f"{AUTHORIZE_URL}?{urllib.parse.urlencode(params)}"
def parse_redirect_input(value: str) -> tuple[str | None, str | None]:
"""Extract ``(code, state)`` from whatever the user pastes back.
Accepts a full redirect URL, a ``code#state`` fragment, a bare query string,
or just the code — so the manual-paste fallback is forgiving.
"""
value = (value or "").strip()
if not value:
return None, None
with contextlib.suppress(ValueError):
parsed = urllib.parse.urlparse(value)
if parsed.scheme and parsed.query:
query = urllib.parse.parse_qs(parsed.query)
return _first(query, "code"), _first(query, "state")
if "#" in value:
code, _, state = value.partition("#")
return code or None, state or None
if "code=" in value:
query = urllib.parse.parse_qs(value)
return _first(query, "code"), _first(query, "state")
return value, None
def _first(query: dict[str, list[str]], key: str) -> str | None:
values = query.get(key)
return values[0] if values else None
# --- token exchange / refresh ----------------------------------------------
def _post_form(payload: dict[str, str]) -> dict[str, Any]:
body = urllib.parse.urlencode(payload).encode("ascii")
request = urllib.request.Request( # noqa: S310 - fixed https OAuth endpoint
TOKEN_URL,
data=body,
headers={
"Content-Type": "application/x-www-form-urlencoded",
"Accept": "application/json",
},
method="POST",
)
try:
with urllib.request.urlopen(request, timeout=_TOKEN_TIMEOUT) as response: # noqa: S310
data = json.loads(response.read() or b"{}")
except urllib.error.HTTPError as exc:
detail = exc.read().decode("utf-8", "replace")[:300]
raise CodexAuthError("token_http_error", f"HTTP {exc.code}: {detail}") from exc
except (urllib.error.URLError, TimeoutError, OSError) as exc:
raise CodexAuthError("unavailable", str(exc)) from exc
if not isinstance(data, dict):
raise CodexAuthError("bad_response", "token endpoint returned non-object")
return data
def _record_from_token_response(
data: dict[str, Any], refresh_fallback: str | None = None
) -> dict[str, Any]:
access = data.get("access_token")
# A refresh response may omit refresh_token when it isn't rotated; keep the old one.
refresh = data.get("refresh_token") or refresh_fallback
expires_in = data.get("expires_in")
if not isinstance(access, str) or not access:
raise CodexAuthError("bad_response", "token response missing access_token")
if not isinstance(refresh, str) or not refresh:
raise CodexAuthError("bad_response", "token response missing refresh_token")
account_id = _account_id_from_jwt(access) or _account_id_from_jwt(
data.get("id_token") if isinstance(data.get("id_token"), str) else ""
)
if not account_id:
raise CodexAuthError("no_account_id", "could not read chatgpt_account_id from token")
ttl = expires_in if isinstance(expires_in, (int, float)) else 3600
return {
"type": "oauth",
"provider": PROVIDER,
"access": access,
"refresh": refresh,
"account_id": account_id,
"expires_at": time.time() + ttl,
}
def exchange_code(code: str, verifier: str) -> dict[str, Any]:
"""Exchange an authorization code + PKCE verifier for a token record."""
data = _post_form(
{
"grant_type": "authorization_code",
"client_id": CLIENT_ID,
"code": code,
"code_verifier": verifier,
"redirect_uri": REDIRECT_URI,
}
)
return _record_from_token_response(data)
def refresh_tokens(refresh_token: str) -> dict[str, Any]:
"""Exchange a refresh token for a new token record."""
data = _post_form(
{
"grant_type": "refresh_token",
"client_id": CLIENT_ID,
"refresh_token": refresh_token,
}
)
return _record_from_token_response(data, refresh_fallback=refresh_token)
def _account_id_from_jwt(token: str | None) -> str | None:
"""Read the ChatGPT account id out of a JWT payload without verifying it.
The token's authenticity is enforced by the server on use; here we only need
to read the account id claim so we can send the ``chatgpt-account-id`` header.
"""
if not token or token.count(".") != 2:
return None
payload_b64 = token.split(".")[1]
padding = "=" * (-len(payload_b64) % 4)
try:
payload = json.loads(base64.urlsafe_b64decode(payload_b64 + padding))
except (ValueError, json.JSONDecodeError):
return None
if not isinstance(payload, dict):
return None
auth = payload.get(_ACCOUNT_CLAIM)
if isinstance(auth, dict):
account_id = auth.get("chatgpt_account_id")
if isinstance(account_id, str) and account_id:
return account_id
organizations = payload.get("organizations")
if isinstance(organizations, list) and organizations and isinstance(organizations[0], dict):
org_id = organizations[0].get("id")
if isinstance(org_id, str) and org_id:
return org_id
return None
# --- stored credential access ----------------------------------------------
def read_record() -> dict[str, Any] | None:
"""Return the stored Codex token record if it is complete, else None."""
record = store.read_provider(PROVIDER)
if not record or record.get("type") != "oauth":
return None
if not (record.get("access") and record.get("refresh") and record.get("account_id")):
return None
return record
def is_authenticated() -> bool:
"""True when a usable Codex token record exists on disk."""
return read_record() is not None
def save_record(record: dict[str, Any]) -> None:
store.write_provider(PROVIDER, record)
def logout() -> None:
store.forget(PROVIDER)
def _near_expiry(record: dict[str, Any]) -> bool:
expires_at = record.get("expires_at")
if not isinstance(expires_at, (int, float)):
return True
return expires_at - _EXPIRY_SKEW_S <= time.time()
def get_valid_token() -> tuple[str, str]:
"""Return ``(access_token, account_id)``, refreshing if near expiry.
Refreshes under a lock and re-reads the store after acquiring it, so that
when many agents fire at once only one refresh happens — OpenAI invalidates a
refresh token as soon as it is used, so a concurrent stampede would fail.
"""
record = read_record()
if record is None:
raise CodexAuthError("not_authenticated", "not signed in; run: strix auth login")
if not _near_expiry(record):
return record["access"], record["account_id"]
with _refresh_lock:
record = read_record()
if record is None:
raise CodexAuthError("not_authenticated", "not signed in; run: strix auth login")
if not _near_expiry(record):
return record["access"], record["account_id"]
refreshed = refresh_tokens(record["refresh"])
save_record(refreshed)
return refreshed["access"], refreshed["account_id"]
# --- inference client -------------------------------------------------------
def build_openai_client() -> AsyncOpenAI:
"""Build an ``AsyncOpenAI`` that routes inference through the ChatGPT backend.
A per-request hook refreshes the token if needed and stamps the auth headers,
so a long scan that outlives a single access token keeps working. The
``api_key`` passed to the SDK is a placeholder — the hook always overwrites
the ``Authorization`` header with the current token.
"""
import asyncio
import httpx
from openai import AsyncOpenAI
# Validate up front so sign-in problems surface at configure time, not
# mid-scan, and to fail fast if the stored refresh token is dead.
_access, account_id = get_valid_token()
async def _auth_hook(request: httpx.Request) -> None:
access, acct = await asyncio.to_thread(get_valid_token)
request.headers["Authorization"] = f"Bearer {access}"
request.headers["chatgpt-account-id"] = acct
http_client = httpx.AsyncClient(
timeout=httpx.Timeout(600.0, connect=30.0),
event_hooks={"request": [_auth_hook]},
)
return AsyncOpenAI(
api_key="strix-codex-oauth",
base_url=CODEX_BASE_URL,
http_client=http_client,
default_headers={
"chatgpt-account-id": account_id,
"OpenAI-Beta": "responses=experimental",
"originator": ORIGINATOR,
},
)
def is_backend_compatible(model_name: str | None) -> bool:
"""True if ``model_name`` could be served by the ChatGPT subscription backend.
A bare name or an ``openai/`` prefix is plausible (the backend validates the
exact name against the account). Any other provider prefix — ``anthropic/``,
``deepseek/``, ``vertex_ai/``, … — never can be, so it's treated as
incompatible.
"""
name = (model_name or "").strip()
if not name:
return False
if "/" not in name:
return True
return name.split("/", 1)[0].lower() == "openai"
def normalize_model(model_name: str | None) -> str:
"""Return the bare model name to send to the ChatGPT backend.
``openai/gpt-5.4`` → ``gpt-5.4``. A configured-but-unlisted OpenAI/bare name
is passed through as-is (availability is account-specific and the backend is
the authority). An empty value, or a model from another provider that a
ChatGPT subscription can't serve, falls back to the default so a stray
``STRIX_LLM`` never sends an impossible name to the backend. See
``is_backend_compatible``.
"""
name = (model_name or "").strip()
if not is_backend_compatible(name):
return DEFAULT_CODEX_MODEL
if "/" in name:
name = name.rsplit("/", 1)[-1]
return name
__all__ = [
"CODEX_MODELS",
"DEFAULT_CODEX_MODEL",
"PROVIDER",
"CodexAuthError",
"build_authorize_url",
"build_openai_client",
"create_state",
"exchange_code",
"generate_pkce",
"get_valid_token",
"is_authenticated",
"is_backend_compatible",
"logout",
"normalize_model",
"parse_redirect_input",
"read_record",
"refresh_tokens",
"save_record",
]
+71
View File
@@ -0,0 +1,71 @@
"""Local credential store for subscription-based model auth.
Mirrors the viewer-auth store (see ``strix/viewer/auth.py``): a single JSON file
under ``~/.strix`` written ``0600``, holding one record per provider. It is kept
deliberately separate from ``cli-config.json`` — that file only ever holds env
vars, so OAuth tokens never leak into the env-var config that gets echoed back
to the user or copied into CI secrets.
"""
from __future__ import annotations
import contextlib
import json
from pathlib import Path
from typing import Any
AUTH_PATH = Path.home() / ".strix" / "subscription-auth.json"
def read_all() -> dict[str, Any]:
"""Return the full provider→record mapping, or ``{}`` if absent/unreadable."""
try:
data = json.loads(AUTH_PATH.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return {}
return data if isinstance(data, dict) else {}
def read_provider(provider: str) -> dict[str, Any] | None:
"""Return the stored record for ``provider``, or None."""
record = read_all().get(provider)
return record if isinstance(record, dict) else None
def write_provider(provider: str, record: dict[str, Any]) -> None:
"""Persist ``record`` for ``provider``, preserving other providers' records."""
data = read_all()
data[provider] = record
_atomic_write(data)
def forget(provider: str) -> None:
"""Delete the stored record for ``provider``. No-op if it is absent.
Removes the file entirely once the last record is gone, so ``logout`` leaves
nothing behind.
"""
data = read_all()
if provider not in data:
return
del data[provider]
if data:
_atomic_write(data)
return
with contextlib.suppress(OSError):
AUTH_PATH.unlink()
def _atomic_write(data: dict[str, Any]) -> None:
AUTH_PATH.parent.mkdir(parents=True, exist_ok=True)
tmp = AUTH_PATH.with_suffix(".json.tmp")
tmp.write_text(json.dumps(data, indent=2), encoding="utf-8")
with contextlib.suppress(OSError):
tmp.chmod(0o600)
tmp.replace(AUTH_PATH)
with contextlib.suppress(OSError):
AUTH_PATH.chmod(0o600)
__all__ = ["AUTH_PATH", "forget", "read_all", "read_provider", "write_provider"]
+72 -3
View File
@@ -3,10 +3,16 @@
from __future__ import annotations
import os
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, Any
from agents import set_default_openai_api, set_default_openai_key, set_tracing_disabled
from agents import (
set_default_openai_api,
set_default_openai_client,
set_default_openai_key,
set_tracing_disabled,
)
from agents.models.multi_provider import MultiProvider
from agents.models.openai_responses import OpenAIResponsesModel
from agents.retry import (
ModelRetryBackoffSettings,
ModelRetrySettings,
@@ -16,7 +22,7 @@ from agents.retry import (
if TYPE_CHECKING:
from agents.models.interface import ModelProvider
from agents.models.interface import Model, ModelProvider
from strix.config.settings import Settings
@@ -36,6 +42,33 @@ def _retry_statusless_provider_errors(context: RetryPolicyContext) -> bool:
return normalized.status_code is None
class _CodexResponsesModel(OpenAIResponsesModel):
"""Responses model for the ChatGPT Codex backend, which rejects non-streamed
requests with ``{"detail": "Stream must be set to true"}``.
It always calls the API in streaming mode. The natively-streamed run path
(``Runner.run_streamed`` → ``stream_response``) is forwarded unchanged; the
non-streaming ``get_response`` path (warm-up, dedupe) transparently issues a
streamed request and aggregates the events back into a single ``Response``,
so those callers don't need to know the backend only speaks SSE.
"""
async def _fetch_response(self, *args: Any, stream: bool = False, **kwargs: Any) -> Any:
# Always call the backend streamed (it rejects non-streamed requests).
events = await super()._fetch_response(*args, stream=True, **kwargs) # type: ignore[call-overload]
if stream:
return events
# Non-streaming caller: aggregate the SSE events into a single Response.
final_response = None
async for event in events: # iterate to exhaustion so the stream cleans up
if getattr(event, "type", None) == "response.completed":
final_response = event.response
if final_response is None:
msg = "ChatGPT backend stream ended without a completed response"
raise RuntimeError(msg)
return final_response
class StrixProvider(MultiProvider):
"""Route any non-OpenAI prefix through LiteLLM with the prefix preserved,
so users type ``deepseek/deepseek-chat`` rather than
@@ -59,6 +92,18 @@ class StrixProvider(MultiProvider):
return self._get_fallback_provider("litellm"), f"ollama_chat/{stripped_model_name}"
return self._get_fallback_provider("litellm"), original_model_name
def get_model(self, model_name: str | None) -> Model:
model = super().get_model(model_name)
# In subscription mode the OpenAI route talks to the ChatGPT Codex
# backend, which requires streamed requests. Promote the responses model
# to the streaming-aware subclass (no added state, so the in-place class
# swap is safe) rather than duplicating the SDK's provider wiring.
from strix.config.loader import load_settings
if load_settings().llm.auth_mode == "subscription" and type(model) is OpenAIResponsesModel:
model.__class__ = _CodexResponsesModel
return model
DEFAULT_MODEL_RETRY = ModelRetrySettings(
max_retries=5,
@@ -117,6 +162,9 @@ def configure_sdk_model_defaults(settings: Settings) -> None:
"""Apply Strix config to SDK-native defaults."""
llm = settings.llm
set_tracing_disabled(True)
if llm.auth_mode == "subscription":
_configure_subscription_defaults()
return
_configure_litellm_compatibility()
_configure_openrouter_attribution(llm.model)
if llm.api_key:
@@ -131,6 +179,23 @@ def configure_sdk_model_defaults(settings: Settings) -> None:
set_default_openai_api("responses")
def _configure_subscription_defaults() -> None:
"""Route inference through a model subscription instead of an API key.
Builds the subscription-backed OpenAI client (currently ChatGPT/Codex) and
installs it as the SDK default, so ``StrixProvider`` uses it for the OpenAI
route. The client carries the OAuth bearer token and refreshes it per
request; the Codex backend speaks the Responses API. Statelessness
(``store=false`` and encrypted reasoning) is applied per call in
``make_model_settings``.
"""
from strix.auth import codex
client = codex.build_openai_client()
set_default_openai_client(client, use_for_tracing=False)
set_default_openai_api("responses")
def _mirror_api_key_to_provider_env(model_name: str | None, api_key: str) -> None:
if not model_name:
return
@@ -211,6 +276,10 @@ def _configure_litellm_default(name: str, value: str) -> None:
def uses_chat_completions_tool_schema(model_name: str, settings: Settings) -> bool:
"""Return whether the resolved SDK route can only receive JSON function tools."""
# Subscription mode (ChatGPT/Codex) speaks the Responses API, so it takes the
# native reasoning-model tool schema regardless of the model name's shape.
if settings.llm.auth_mode == "subscription":
return False
model = model_name.strip().lower()
if "/" in model and not model.startswith("openai/"):
return True
+5
View File
@@ -9,6 +9,7 @@ from pydantic_settings import BaseSettings, SettingsConfigDict
ReasoningEffort = Literal["none", "minimal", "low", "medium", "high", "xhigh"]
AuthMode = Literal["api_key", "subscription"]
_BASE_CONFIG = SettingsConfigDict(
case_sensitive=False,
@@ -21,6 +22,10 @@ class LlmSettings(BaseSettings):
model_config = _BASE_CONFIG
model: str | None = Field(default=None, alias="STRIX_LLM")
# "api_key" (default) reads a provider API key from ``api_key`` below.
# "subscription" ignores the API key and authenticates with a model
# subscription instead (see ``strix/auth``); set by ``strix auth login``.
auth_mode: AuthMode = Field(default="api_key", alias="STRIX_AUTH_MODE")
api_key: str | None = Field(
default=None,
validation_alias=AliasChoices("LLM_API_KEY", "OPENAI_API_KEY"),
+30 -4
View File
@@ -128,6 +128,7 @@ def make_model_settings(
model_name: str,
force_required_tool_choice: bool = False,
request_timeout: float | None = None,
codex_subscription: bool = False,
) -> ModelSettings:
model_settings = ModelSettings(
parallel_tool_calls=False,
@@ -135,10 +136,16 @@ def make_model_settings(
include_usage=True,
extra_args=request_timeout_extra_args(request_timeout),
)
if (
reasoning_effort is not None
and reasoning_effort != "none"
and model_supports_reasoning(model_name)
if codex_subscription:
# The ChatGPT Codex backend is stateless: it requires store=false and
# carries reasoning context forward via encrypted reasoning content
# replayed in the input rather than server-side state.
model_settings = model_settings.resolve(
ModelSettings(store=False, response_include=["reasoning.encrypted_content"]),
)
reasoning_effort = _resolve_reasoning_effort(reasoning_effort, codex_subscription)
if reasoning_effort is not None and (
codex_subscription or model_supports_reasoning(model_name)
):
model_settings = model_settings.resolve(
ModelSettings(reasoning=Reasoning(effort=reasoning_effort)),
@@ -148,6 +155,25 @@ def make_model_settings(
return model_settings
def _resolve_reasoning_effort(
reasoning_effort: ReasoningEffort | None, codex_subscription: bool
) -> ReasoningEffort | None:
"""Normalize the configured effort, clamping to what the Codex backend accepts.
The ChatGPT Codex backend rejects ``minimal`` and only some models accept
``xhigh``; both are clamped rather than passed through, so a valid Strix
config doesn't fail at the provider.
"""
if reasoning_effort is None or reasoning_effort == "none":
return None
if codex_subscription:
if reasoning_effort == "minimal":
return "low"
if reasoning_effort == "xhigh":
return "high"
return reasoning_effort
def child_initial_input(
*,
name: str,
+8
View File
@@ -148,7 +148,14 @@ async def run_strix_scan(
settings = load_settings()
configure_sdk_model_defaults(settings)
codex_subscription = settings.llm.auth_mode == "subscription"
resolved_model = (model or settings.llm.model or "").strip()
if codex_subscription:
# Map whatever is configured to a name the ChatGPT Codex backend accepts,
# so a stray STRIX_LLM value can't send an unsupported model.
from strix.auth import codex
resolved_model = codex.normalize_model(resolved_model)
if not resolved_model:
raise RuntimeError(
"No LLM model configured. Set STRIX_LLM env or pass model= to run_strix_scan().",
@@ -216,6 +223,7 @@ async def run_strix_scan(
model_name=resolved_model,
force_required_tool_choice=settings.llm.force_required_tool_choice,
request_timeout=settings.llm.timeout,
codex_subscription=codex_subscription,
)
run_config = RunConfig(
model=resolved_model,
+466
View File
@@ -0,0 +1,466 @@
"""`strix auth` — manage model-subscription sign-in.
Subcommands:
- ``strix auth login chatgpt [--model NAME] [--manual]`` — OAuth sign-in with a
ChatGPT Plus/Pro subscription. Opens a browser and catches the redirect on a
local server; ``--manual`` (or a failure to open the browser / bind the port)
falls back to pasting the redirect URL by hand.
- ``strix auth status`` — show whether a subscription sign-in is active.
- ``strix auth logout`` — forget the stored sign-in.
Signing in persists ``STRIX_AUTH_MODE=subscription`` and a Codex model to
``~/.strix/cli-config.json`` so subsequent ``strix`` runs use the subscription.
Tokens live separately in ``~/.strix/subscription-auth.json`` (see
``strix/auth/store.py``); they are never written to the env-var config.
"""
from __future__ import annotations
import argparse
import base64
import logging
import os
import threading
import webbrowser
from http.server import BaseHTTPRequestHandler, HTTPServer
from pathlib import Path
from typing import Any
from urllib.parse import parse_qs, urlparse
from rich.console import Console
from rich.panel import Panel
from rich.text import Text
from strix.auth import codex
from strix.config import load_settings, persist_current
logger = logging.getLogger(__name__)
_CALLBACK_TIMEOUT_S = 300
# CLI-facing name for the login provider. Internally this is the Codex OAuth
# flow (``codex.PROVIDER``), but users know it as ChatGPT, so that's what the
# command and messaging say. ``codex`` is accepted as an alias.
LOGIN_PROVIDER = "chatgpt"
_ACCEPTED_PROVIDERS = frozenset({LOGIN_PROVIDER, codex.PROVIDER})
_USAGE = (
"Usage:\n"
" strix auth login chatgpt [--model NAME] [--manual]\n"
" strix auth status\n"
" strix auth logout"
)
def run_auth(argv: list[str]) -> int:
"""Entry point for ``strix auth …``. Returns a process exit code."""
console = Console()
subcommand = argv[0] if argv else "login"
rest = argv[1:]
if subcommand in ("-h", "--help", "help"):
console.print(_USAGE)
return 0
if subcommand == "login":
return _login(console, rest)
if subcommand == "status":
return _status(console)
if subcommand == "logout":
return _logout(console)
# Bare `strix auth` (no subcommand) defaults to login; anything else is an error.
if not argv:
return _login(console, [])
console.print(f"[red]Unknown auth command:[/] {subcommand}\n")
console.print(_USAGE)
return 2
def _login(console: Console, argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog="strix auth login", add_help=True)
parser.add_argument(
"provider",
nargs="?",
default=LOGIN_PROVIDER,
help="Model provider to sign in with (default: chatgpt).",
)
parser.add_argument(
"--model",
default=codex.DEFAULT_CODEX_MODEL,
help=f"ChatGPT model to use (default: {codex.DEFAULT_CODEX_MODEL}).",
)
parser.add_argument(
"--manual",
action="store_true",
help="Skip the local callback server and paste the redirect URL by hand.",
)
try:
args = parser.parse_args(argv)
except SystemExit as exc: # argparse already printed the message
return int(exc.code or 2)
if args.provider.lower() not in _ACCEPTED_PROVIDERS:
console.print(
f"[red]Unsupported provider:[/] {args.provider}. "
f"Only '{LOGIN_PROVIDER}' (ChatGPT subscription) is supported."
)
return 2
# Capture any shell-exported STRIX_LLM before we persist our own, so we can
# warn if it would override the model chosen here (env wins over the config
# file we write). This must be read before _persist_subscription_config sets it.
preexisting_llm = os.environ.get("STRIX_LLM")
verifier, challenge = codex.generate_pkce()
state = codex.create_state()
authorize_url = codex.build_authorize_url(challenge, state)
console.print()
console.print("[bold]Signing in with ChatGPT[/] [dim](provider: chatgpt)[/]")
console.print(
"[dim]This uses your ChatGPT Plus/Pro plan for inference instead of a metered API key.[/]"
)
console.print()
try:
record = _run_oauth_flow(console, authorize_url, verifier, state, manual=args.manual)
except codex.CodexAuthError as exc:
return _fail(console, exc)
except KeyboardInterrupt:
console.print("\n[yellow]Sign-in cancelled.[/]")
return 130
codex.save_record(record)
_persist_subscription_config(args.model)
_warn_if_env_overrides_model(console, preexisting_llm, args.model)
stored_model = load_settings().llm.model or f"openai/{codex.normalize_model(args.model)}"
_print_success(console, stored_model)
return 0
def _warn_if_env_overrides_model(console: Console, preexisting: str | None, chosen: str) -> None:
"""Warn when a shell-exported STRIX_LLM will override the model just saved.
The config file we write loses to an environment variable at load time, so a
lingering ``export STRIX_LLM=…`` would silently win. Only warn when it
resolves to a different model than the one chosen here (same value, or an
incompatible one that coerces to the same default, is not a real conflict).
"""
if not preexisting or not preexisting.strip():
return
if codex.normalize_model(preexisting) == codex.normalize_model(chosen):
return
console.print(
f"[yellow]Note:[/] STRIX_LLM is set in your shell to "
f"[bold]{preexisting.strip()}[/], which overrides the model just saved. "
f"Run [bold cyan]unset STRIX_LLM[/] so the subscription model is used."
)
def _run_oauth_flow(
console: Console,
authorize_url: str,
verifier: str,
state: str,
*,
manual: bool,
) -> dict[str, Any]:
"""Drive the browser (or manual) OAuth flow and return a token record."""
server = None if manual else _try_start_callback_server()
console.print("Open this URL in your browser to authorize:")
console.print(f"[cyan]{authorize_url}[/]")
console.print()
if not manual:
try:
webbrowser.open(authorize_url)
except Exception: # noqa: BLE001 - opening a browser is best-effort
logger.debug("could not open browser", exc_info=True)
if server is not None:
console.print("[dim]Waiting for you to finish signing in…[/]")
result = server.wait(_CALLBACK_TIMEOUT_S)
server.shutdown()
if result is not None:
code, returned_state, error = result
if error:
raise codex.CodexAuthError("oauth_error", error)
return _finish(code, returned_state, verifier, state)
console.print("[yellow]Timed out waiting for the browser. Falling back to manual paste.[/]")
# Manual fallback: the user completes sign-in and pastes the redirect URL
# (the browser lands on a localhost page that won't load if no server is up;
# the address bar still holds the code+state).
console.print()
try:
pasted = console.input("Paste the full redirect URL (or code#state): ").strip()
except EOFError as exc:
raise codex.CodexAuthError("no_input", "no redirect URL provided") from exc
code, returned_state = codex.parse_redirect_input(pasted)
return _finish(code, returned_state, verifier, state)
def _finish(
code: str | None, returned_state: str | None, verifier: str, expected_state: str
) -> dict[str, Any]:
if not code:
raise codex.CodexAuthError("no_code", "no authorization code found in the redirect")
if returned_state is not None and returned_state != expected_state:
raise codex.CodexAuthError("state_mismatch", "state did not match; possible CSRF")
return codex.exchange_code(code, verifier)
class _CallbackServer:
"""A one-shot local HTTP server that catches the OAuth redirect."""
def __init__(self, httpd: HTTPServer, event: threading.Event, holder: dict[str, Any]) -> None:
self._httpd = httpd
self._event = event
self._holder = holder
self._thread = threading.Thread(target=httpd.serve_forever, daemon=True)
self._thread.start()
def wait(self, timeout: float) -> tuple[str | None, str | None, str | None] | None:
if not self._event.wait(timeout):
return None
return (
self._holder.get("code"),
self._holder.get("state"),
self._holder.get("error"),
)
def shutdown(self) -> None:
self._httpd.shutdown()
self._httpd.server_close()
def _try_start_callback_server() -> _CallbackServer | None:
event = threading.Event()
holder: dict[str, Any] = {}
class Handler(BaseHTTPRequestHandler):
def log_message(self, *args: Any) -> None: # silence default stderr logging
pass
def do_GET(self) -> None:
parsed = urlparse(self.path)
if parsed.path != codex.CALLBACK_PATH:
self.send_response(404)
self.end_headers()
return
query = parse_qs(parsed.query)
holder["code"] = _first(query, "code")
holder["state"] = _first(query, "state")
holder["error"] = _first(query, "error_description") or _first(query, "error")
body = _render_callback_html().encode("utf-8")
self.send_response(200)
self.send_header("Content-Type", "text/html; charset=utf-8")
self.send_header("Content-Length", str(len(body)))
self.end_headers()
self.wfile.write(body)
event.set()
try:
httpd = HTTPServer(("127.0.0.1", codex.CALLBACK_PORT), Handler)
except OSError:
logger.debug("could not bind callback port %d", codex.CALLBACK_PORT, exc_info=True)
return None
return _CallbackServer(httpd, event, holder)
def _first(query: dict[str, list[str]], key: str) -> str | None:
values = query.get(key)
return values[0] if values else None
def _persist_subscription_config(model: str) -> None:
"""Persist subscription mode + model to cli-config.json for later runs."""
normalized = codex.normalize_model(model)
os.environ["STRIX_AUTH_MODE"] = "subscription"
os.environ["STRIX_LLM"] = f"openai/{normalized}"
persist_current()
def _status(console: Console) -> int:
record = codex.read_record()
if record is None:
console.print("[yellow]Not signed in.[/] Run [cyan]strix auth login[/] to sign in.")
return 1
settings = load_settings()
console.print("[green]Signed in[/] with a ChatGPT subscription (Codex).")
console.print(f" Account: [bold]{record.get('account_id')}[/]")
console.print(f" Model: [bold]{settings.llm.model or codex.DEFAULT_CODEX_MODEL}[/]")
if settings.llm.auth_mode != "subscription":
console.print(
" [yellow]Note:[/] STRIX_AUTH_MODE is not 'subscription'; "
"runs will use API-key billing until you re-run [cyan]strix auth login[/]."
)
return 0
def _logout(console: Console) -> int:
codex.logout()
console.print("[green]Signed out.[/] Stored subscription credentials removed.")
console.print(
"[dim]Runs still set to subscription mode will ask you to sign in again. "
"Set STRIX_AUTH_MODE=api_key (and LLM_API_KEY) to use metered billing.[/]"
)
return 0
def _fail(console: Console, exc: codex.CodexAuthError) -> int:
error_text = Text()
error_text.append("SIGN-IN FAILED", style="bold red")
error_text.append("\n\n", style="white")
error_text.append(f"{exc}", style="white")
console.print()
console.print(
Panel(
error_text,
title="[bold white]STRIX",
title_align="left",
border_style="red",
padding=(1, 2),
)
)
return 1
def _print_success(console: Console, model: str) -> None:
text = Text()
text.append("Signed in with your ChatGPT subscription", style="bold #22c55e")
text.append("\n\n", style="white")
text.append("Model", style="dim")
text.append(" ")
text.append(model, style="bold white")
text.append("\n")
text.append("Billing", style="dim")
text.append(" ")
text.append("your ChatGPT plan (no per-token API charges)", style="white")
text.append("\n\n", style="white")
text.append("Run a scan as usual, e.g. ", style="white")
text.append("strix --target https://example.com", style="bold cyan")
text.append("\nChange model with ", style="dim")
text.append("strix auth login chatgpt --model <name>", style="cyan")
console.print()
console.print(
Panel(
text,
title="[bold white]STRIX",
title_align="left",
border_style="#22c55e",
padding=(1, 2),
)
)
console.print()
_LOGO_PATH = Path(__file__).resolve().parent.parent / "viewer" / "static" / "logo.png"
def _logo_img_tag() -> str:
"""Return an ``<img>`` for the Strix logo as an inline data URI, or "".
The callback page is served offline by the local OAuth server, so the logo
is embedded rather than linked. Missing/unreadable file degrades to just the
"Strix" wordmark.
"""
try:
data = _LOGO_PATH.read_bytes()
except OSError:
return ""
encoded = base64.b64encode(data).decode("ascii")
return f'<img class="logo" src="data:image/png;base64,{encoded}" alt="" />'
def _render_callback_html() -> str:
return _CALLBACK_HTML.replace("<!--LOGO-->", _logo_img_tag())
_CALLBACK_HTML = """<!doctype html>
<html lang="en"><head><meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Strix — signed in</title>
<style>
:root { color-scheme: dark; }
* { box-sizing: border-box; }
body {
margin: 0; min-height: 100vh; padding: 24px;
font-family: 'Geist', 'Geist Sans', ui-sans-serif, system-ui, -apple-system,
"Segoe UI", Roboto, Helvetica, Arial, sans-serif;
-webkit-font-smoothing: antialiased; -moz-osx-font-smoothing: grayscale;
background: #000; color: #ededed;
display: flex; flex-direction: column; align-items: center; justify-content: center;
}
.topbar {
position: absolute; top: 20px; left: 22px;
display: flex; align-items: center; gap: 6px; text-decoration: none;
}
.topbar .logo { width: 40px; height: 40px; display: block; }
.topbar span {
font-size: 1.1rem; font-weight: 600; letter-spacing: -.01em; color: #fff;
transition: color .15s ease;
}
.topbar:hover span { color: #c9c9c9; }
.brand {
font-size: 2.1rem; font-weight: 700; letter-spacing: -.02em; color: #fff;
text-align: center; margin: 0 0 10px;
}
h1 {
font-size: 1.35rem; font-weight: 600; letter-spacing: -.01em; color: #f5f5f5;
text-align: center; margin: 0 0 28px;
}
.card {
width: 100%; max-width: 430px; text-align: center;
background: #171717; border: 1px solid rgba(255, 255, 255, .06);
border-radius: 24px; padding: 40px 40px 34px;
}
.badge {
margin: 0 auto 22px; width: 52px; height: 52px; border-radius: 50%;
display: flex; align-items: center; justify-content: center; font-size: 23px; color: #fff;
background: rgba(255, 255, 255, .05); border: 1px solid rgba(255, 255, 255, .14);
}
.msg { margin: 0 auto; max-width: 34ch; color: #b5b5b5; line-height: 1.6; font-size: .98rem; }
.rule { height: 1px; background: rgba(255, 255, 255, .07); margin: 26px 0 0; }
.tagline { margin: 22px 0 0; color: #7c7c7c; font-size: .9rem; line-height: 1.55; }
.tagline b { color: #ededed; font-weight: 500; }
.links {
margin-top: 18px; display: flex; gap: 8px; justify-content: center;
align-items: center; flex-wrap: wrap; font-size: .84rem;
}
.links a { color: #a3a3a3; text-decoration: none; transition: color .15s ease; }
.links a:hover { color: #fff; }
.links .dot { color: #3a3a3a; }
.close { margin: 24px 0 0; color: #5a5a5a; font-size: .78rem; text-align: center; }
</style></head>
<body>
<a class="topbar" href="https://strix.ai" target="_blank" rel="noopener"
aria-label="Strix — strix.ai">
<!--LOGO-->
<span>Strix</span>
</a>
<div class="brand">Strix</div>
<h1>You're signed in</h1>
<main class="card">
<div class="badge">✓</div>
<p class="msg">Strix is connected to your ChatGPT subscription. Head back to your
terminal — your security test runs there.</p>
<div class="rule"></div>
<p class="tagline">Autonomous AI hackers that <b>find and fix</b> your app's
vulnerabilities.</p>
<nav class="links">
<a href="https://strix.ai" target="_blank" rel="noopener">strix.ai</a>
<span class="dot">·</span>
<a href="https://docs.strix.ai" target="_blank" rel="noopener">docs</a>
<span class="dot">·</span>
<a href="https://discord.gg/strix-ai" target="_blank" rel="noopener">community</a>
</nav>
</main>
<p class="close">You can close this tab.</p>
</body></html>"""
__all__ = ["run_auth"]
+165 -13
View File
@@ -85,6 +85,10 @@ def validate_environment() -> None:
settings = load_settings()
if settings.llm.auth_mode == "subscription":
_validate_subscription_environment(console)
return
if not settings.llm.model:
missing_required_vars.append("STRIX_LLM")
@@ -203,6 +207,86 @@ def validate_environment() -> None:
)
def _validate_subscription_environment(console: Console) -> None:
"""Gate a subscription-mode run on being signed in.
In subscription mode there is no API key to check; instead we require a
stored sign-in. The model name is optional here — the runner falls back to a
default model the subscription backend accepts.
"""
from strix.auth import codex
if codex.is_authenticated():
_warn_if_incompatible_subscription_model(console)
logger.info("Environment OK (subscription mode, signed in)")
return
logger.error("Subscription mode but not signed in")
error_text = Text()
error_text.append("NOT SIGNED IN", style="bold red")
error_text.append("\n\n", style="white")
error_text.append(
"Strix is set to use a model subscription, but no sign-in was found.\n",
style="white",
)
error_text.append("Sign in with:\n\n", style="white")
error_text.append("strix auth login", style="bold cyan")
error_text.append("\n\nOr switch back to API-key billing with:\n", style="white")
error_text.append("export STRIX_AUTH_MODE=api_key", style="bold cyan")
error_text.append(" # and set LLM_API_KEY\n", style="dim white")
console.print("\n")
console.print(
Panel(
error_text,
title="[bold white]STRIX",
title_align="left",
border_style="red",
padding=(1, 2),
)
)
console.print()
sys.exit(1)
def _warn_if_incompatible_subscription_model(console: Console) -> None:
"""Warn (once, at startup) when the configured model can't run on the plan.
In subscription mode a model from another provider (e.g. ``anthropic/…``) is
coerced to the default Codex model so the run still works; surface that so the
override isn't silent. A bare/``openai/`` name is left alone — the backend
validates it.
"""
from strix.auth import codex
configured = (load_settings().llm.model or "").strip()
if not configured or codex.is_backend_compatible(configured):
return
effective = codex.normalize_model(configured)
warn_text = Text()
warn_text.append("MODEL NOT AVAILABLE ON SUBSCRIPTION", style="bold yellow")
warn_text.append("\n\n", style="white")
warn_text.append("STRIX_LLM is set to ", style="white")
warn_text.append(f"'{configured}'", style="bold cyan")
warn_text.append(", which a ChatGPT subscription can't serve.\n", style="white")
warn_text.append("Using ", style="white")
warn_text.append(f"'{effective}'", style="bold cyan")
warn_text.append(" instead.\n\n", style="white")
warn_text.append("Pick a supported model with: ", style="white")
warn_text.append("strix auth login chatgpt --model gpt-5.4", style="bold cyan")
console.print(
Panel(
warn_text,
title="[bold white]STRIX",
title_align="left",
border_style="yellow",
padding=(1, 2),
)
)
def check_docker_installed() -> None:
if shutil.which("docker") is None:
logger.error("Docker CLI not found in PATH")
@@ -267,6 +351,35 @@ def _provider_import_hint(exc: BaseException, model: str) -> str | None:
return None
def _subscription_error_hint(exc: BaseException) -> str | None:
"""Return an actionable hint for a known ChatGPT-subscription error, or None.
The ChatGPT Codex backend rejects a model the account can't use, and a few
other conditions, with clear 400s. In subscription mode we translate those
into a fix rather than surfacing a raw provider traceback.
"""
if load_settings().llm.auth_mode != "subscription":
return None
joined = " ".join(_exception_messages(exc)).lower()
if "not supported when using codex with a chatgpt account" in joined:
return (
"The selected model isn't available on your ChatGPT subscription.\n"
"Sign in again with a supported model, for example:\n"
" strix auth login chatgpt --model gpt-5.4"
)
if "stream must be set to true" in joined:
return (
"The ChatGPT backend requires streamed requests, but this call wasn't "
"streamed. This is an internal Strix issue on this path — please report it."
)
if "401" in joined or "unauthorized" in joined or "invalid_grant" in joined:
return (
"Your ChatGPT sign-in has expired or was revoked. Sign in again:\n"
" strix auth login chatgpt"
)
return None
async def warm_up_llm(show_model_warning: bool = True) -> None:
console = Console()
logger.info("Warming up LLM connection")
@@ -276,10 +389,22 @@ async def warm_up_llm(show_model_warning: bool = True) -> None:
settings = load_settings()
configure_sdk_model_defaults(settings)
llm = settings.llm
subscription = llm.auth_mode == "subscription"
warm_model_settings = ModelSettings()
if subscription:
from strix.auth import codex
raw_model = codex.normalize_model(llm.model)
warm_model_settings = ModelSettings(
store=False, response_include=["reasoning.encrypted_content"]
)
else:
raw_model = (llm.model or "").strip()
raw_model = (llm.model or "").strip()
if (
raw_model
not subscription
and raw_model
and "/" not in raw_model
and not is_known_openai_bare_model(raw_model)
and not llm.api_base
@@ -309,7 +434,12 @@ async def warm_up_llm(show_model_warning: bool = True) -> None:
)
sys.exit(1)
if show_model_warning and raw_model and not is_recommended_or_frontier_model(raw_model):
if (
show_model_warning
and not subscription
and raw_model
and not is_recommended_or_frontier_model(raw_model)
):
warn_text = Text()
warn_text.append("MODEL QUALITY WARNING", style="bold yellow")
warn_text.append("\n\n", style="white")
@@ -340,7 +470,7 @@ async def warm_up_llm(show_model_warning: bool = True) -> None:
model.get_response(
system_instructions="You are a helpful assistant.",
input="Reply with just 'OK'.",
model_settings=ModelSettings(),
model_settings=warm_model_settings,
tools=[],
output_schema=None,
handoffs=[],
@@ -356,20 +486,33 @@ async def warm_up_llm(show_model_warning: bool = True) -> None:
except Exception as e:
logger.exception("LLM warm-up failed")
error_text = Text()
error_text.append("LLM CONNECTION FAILED", style="bold red")
error_text.append("\n\n", style="white")
error_text.append("Could not establish connection to the language model.\n", style="white")
error_text.append("Please check your configuration and try again.\n", style="white")
hint = _provider_import_hint(e, raw_model)
if hint is not None:
error_text.append(f"\n{hint}\n", style="bold yellow")
error_text.append(f"\nError: {e}", style="dim white")
sub_hint = _subscription_error_hint(e)
if sub_hint is not None:
# The model/backend answered with a clear, actionable rejection —
# show that instead of a generic "connection failed".
border_style = "yellow"
error_text.append("MODEL NOT AVAILABLE ON SUBSCRIPTION", style="bold yellow")
error_text.append("\n\n", style="white")
error_text.append(f"{sub_hint}\n", style="white")
error_text.append(f"\nDetails: {e}", style="dim white")
else:
border_style = "red"
error_text.append("LLM CONNECTION FAILED", style="bold red")
error_text.append("\n\n", style="white")
error_text.append(
"Could not establish connection to the language model.\n", style="white"
)
error_text.append("Please check your configuration and try again.\n", style="white")
hint = _provider_import_hint(e, raw_model)
if hint is not None:
error_text.append(f"\n{hint}\n", style="bold yellow")
error_text.append(f"\nError: {e}", style="dim white")
panel = Panel(
error_text,
title="[bold white]STRIX",
title_align="left",
border_style="red",
border_style=border_style,
padding=(1, 2),
)
@@ -664,6 +807,7 @@ def _persist_run_record(args: argparse.Namespace) -> None:
"status": "running",
"start_time": datetime.now(UTC).isoformat(),
"end_time": None,
"auth_mode": load_settings().llm.auth_mode,
"targets_info": args.targets_info,
"scan_mode": args.scan_mode,
"instruction": args.instruction,
@@ -880,6 +1024,13 @@ def main() -> None:
run_view(sys.argv[2:])
return
# `strix auth …` manages model-subscription sign-in and exits; it needs no
# target, Docker, or scan setup.
if len(sys.argv) > 1 and sys.argv[1] == "auth":
from strix.interface.auth_cli import run_auth
sys.exit(run_auth(sys.argv[2:]))
args = parse_arguments()
if args.config:
@@ -941,6 +1092,7 @@ def main() -> None:
_telemetry_start_kwargs = {
"model": load_settings().llm.model,
"auth_mode": load_settings().llm.auth_mode,
"scan_mode": args.scan_mode,
"is_whitebox": is_whitebox_scan(args.targets_info),
"interactive": not args.non_interactive,
+36 -6
View File
@@ -253,6 +253,18 @@ def _llm_usage(report_state: Any) -> dict[str, Any]:
return usage if isinstance(usage, dict) else {}
def _is_subscription(report_state: Any) -> bool:
"""Whether this run uses a model subscription (no metered cost).
Prefers the run record so it's correct for hydrated/resumed runs; falls back
to current settings.
"""
record = getattr(report_state, "run_record", None)
if isinstance(record, dict) and record.get("auth_mode"):
return record.get("auth_mode") == "subscription"
return load_settings().llm.auth_mode == "subscription"
def _int_stat(usage: dict[str, Any], key: str) -> int:
try:
return max(0, int(usage.get(key) or 0))
@@ -283,11 +295,16 @@ def _build_llm_usage_stats(
*,
live: bool = False,
) -> None:
subscription = _is_subscription(report_state)
usage = _llm_usage(report_state)
if not usage or _int_stat(usage, "requests") <= 0:
stats_text.append("\n")
stats_text.append("Cost ", style="dim")
stats_text.append("$0.0000 ", style="#fbbf24")
if subscription:
stats_text.append("$0.00 ", style="#22c55e")
stats_text.append("(subscription) ", style="dim")
else:
stats_text.append("$0.0000 ", style="#fbbf24")
stats_text.append("· ", style="dim white")
stats_text.append("Tokens ", style="dim")
stats_text.append("0", style="white")
@@ -312,7 +329,12 @@ def _build_llm_usage_stats(
stats_text.append("Output Tokens ", style="dim")
stats_text.append(format_token_count(output_tokens), style="white")
if live or cost > 0:
if subscription:
stats_text.append(" · ", style="dim white")
stats_text.append("Cost ", style="dim")
stats_text.append("$0.00", style="#22c55e")
stats_text.append(" (subscription)", style="dim")
elif live or cost > 0:
stats_text.append(" · ", style="dim white")
stats_text.append("Cost ", style="dim")
stats_text.append(f"${cost:.4f}", style="#fbbf24")
@@ -337,6 +359,9 @@ def build_live_stats_text(report_state: Any) -> Text:
model = load_settings().llm.model or "unknown"
stats_text.append("Model ", style="dim")
stats_text.append(str(model), style="white")
if _is_subscription(report_state):
stats_text.append(" · ", style="dim white")
stats_text.append("ChatGPT subscription", style="#22c55e")
stats_text.append("\n")
vuln_count = len(report_state.vulnerability_reports)
@@ -379,6 +404,10 @@ def build_tui_stats_text(report_state: Any) -> Text:
model = load_settings().llm.model or "unknown"
stats_text.append(str(model), style="white")
subscription = _is_subscription(report_state)
if subscription:
stats_text.append("\n")
stats_text.append("ChatGPT subscription", style="#22c55e")
usage = _llm_usage(report_state)
if usage and _int_stat(usage, "total_tokens") > 0:
@@ -388,7 +417,10 @@ def build_tui_stats_text(report_state: Any) -> Text:
style="white",
)
cost = _float_stat(usage, "cost")
if cost > 0:
if subscription:
stats_text.append(" · ", style="white")
stats_text.append("$0.00", style="white")
elif cost > 0:
stats_text.append(" · ", style="white")
stats_text.append(f"${cost:.2f}", style="white")
@@ -1147,9 +1179,7 @@ def read_target_list_file(path_str: str) -> list[str]:
if (target := line.strip()) and not target.startswith("#")
]
except UnicodeDecodeError as e:
raise ValueError(
f"Target list file '{path_str}' must be valid UTF-8 text: {e!s}"
) from e
raise ValueError(f"Target list file '{path_str}' must be valid UTF-8 text: {e!s}") from e
except OSError as e:
raise ValueError(f"Failed to read target list file '{path_str}': {e!s}") from e
+6
View File
@@ -10,6 +10,7 @@ from uuid import uuid4
from agents.usage import Usage
from strix.config.loader import load_settings
from strix.core.paths import run_dir_for
from strix.report.sarif import write_sarif
from strix.report.usage import LLMUsageLedger
@@ -117,12 +118,17 @@ class ReportState:
self.scan_results: dict[str, Any] | None = None
self.scan_config: dict[str, Any] | None = None
self._llm_usage = LLMUsageLedger()
# In subscription mode inference is covered by the user's plan, so there
# is no metered cost — track tokens but report $0.
auth_mode = load_settings().llm.auth_mode
self._llm_usage.zero_cost = auth_mode == "subscription"
self.run_record: dict[str, Any] = {
"run_id": self.run_id,
"run_name": self.run_name,
"start_time": self.start_time,
"end_time": None,
"status": "running",
"auth_mode": auth_mode,
"targets_info": [],
"llm_usage": self._build_llm_usage_record(),
}
+6 -1
View File
@@ -19,6 +19,9 @@ class LLMUsageLedger:
self._agent_usage: dict[str, Usage] = {}
self._agent_metadata: dict[str, dict[str, str]] = {}
self._total_cost = 0.0
# When True, tokens are still tracked but cost stays $0 — the run is on a
# model subscription, so there is no metered per-token charge to report.
self.zero_cost = False
def record(
self,
@@ -41,7 +44,7 @@ class LLMUsageLedger:
if model:
metadata["model"] = model
if not _is_litellm_routed(model):
if not self.zero_cost and not _is_litellm_routed(model):
estimated = _estimate_litellm_cost(usage, model)
if estimated:
self._total_cost += estimated
@@ -49,6 +52,8 @@ class LLMUsageLedger:
return True
def record_observed_cost(self, cost: float) -> None:
if self.zero_cost:
return
if isinstance(cost, int | float) and cost > 0:
self._total_cost += float(cost)
+3
View File
@@ -58,12 +58,14 @@ def start(
is_whitebox: bool,
interactive: bool,
has_instructions: bool,
auth_mode: str | None = None,
) -> None:
_send(
"scan_started",
{
**base_props(),
"model": model or "unknown",
"auth_mode": auth_mode or "api_key",
"scan_mode": scan_mode or "unknown",
"scan_type": "whitebox" if is_whitebox else "blackbox",
"interactive": interactive,
@@ -133,6 +135,7 @@ def end(report_state: "ReportState", exit_reason: str = "completed") -> None:
"scan_ended",
{
**base_props(),
"auth_mode": report_state.run_record.get("auth_mode") or "api_key",
"exit_reason": report_state.scan_ended_exit_reason,
"duration_seconds": round(duration),
"vulnerabilities_total": len(report_state.vulnerability_reports),
+3
View File
@@ -59,6 +59,7 @@ def start(
is_whitebox: bool,
interactive: bool,
has_instructions: bool,
auth_mode: str | None = None,
) -> None:
_send(
"scan_started",
@@ -66,6 +67,7 @@ def start(
**base_props(),
"session": SESSION_ID,
"model": model or "unknown",
"auth_mode": auth_mode or "api_key",
"scan_mode": scan_mode or "unknown",
"scan_type": "whitebox" if is_whitebox else "blackbox",
"interactive": interactive,
@@ -140,6 +142,7 @@ def end(report_state: ReportState, exit_reason: str = "completed") -> None:
{
**base_props(),
"session": SESSION_ID,
"auth_mode": report_state.run_record.get("auth_mode") or "api_key",
"exit_reason": report_state.scan_ended_exit_reason,
"duration_seconds": round(duration),
"vulnerabilities_total": len(report_state.vulnerability_reports),
@@ -94,6 +94,7 @@ export function RunDetails({
const reasoning = num(rec(arr(usage.output_tokens_details)[0]).reasoning_tokens);
const totalTokens = num(usage.total_tokens);
const cost = num(usage.cost);
const subscription = str(raw.auth_mode) === "subscription";
const sub = (n: number, word: string) => (
<span className="text-[#666]"> ({formatNumber(n)} {word})</span>
@@ -169,6 +170,15 @@ export function RunDetails({
{hasUsage ? (
<dl className="space-y-2.5 tabular-nums">
<Field label="Model">{models.length ? models.join(", ") : "n/a"}</Field>
{subscription && (
<Field label="Provider">
<span className="inline-flex items-center gap-1.5">
<span className="rounded-full border border-[#22c55e]/40 bg-[#22c55e]/10 px-2 py-0.5 text-[11px] text-[#22c55e]">
ChatGPT subscription
</span>
</span>
</Field>
)}
<Field label="Run time">{fmtDuration(durationSeconds)}</Field>
{requests != null && <Field label="Requests">{formatNumber(requests)}</Field>}
{inputTokens != null && (
@@ -184,7 +194,14 @@ export function RunDetails({
</Field>
)}
{totalTokens != null && <Field label="Total tokens">{formatNumber(totalTokens)}</Field>}
{cost != null && <Field label="Cost">${cost.toFixed(2)}</Field>}
{subscription ? (
<Field label="Cost">
<span className="text-[#22c55e]">$0.00</span>
<span className="text-[#666]"> (subscription)</span>
</Field>
) : (
cost != null && <Field label="Cost">${cost.toFixed(2)}</Field>
)}
{agents.length > 0 && <Field label="Agents">{formatNumber(agents.length)}</Field>}
</dl>
) : (
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+2 -2
View File
@@ -6,8 +6,8 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="color-scheme" content="dark" />
<title>Strix Results</title>
<script type="module" crossorigin src="./assets/index-BU_tk5L-.js"></script>
<link rel="stylesheet" crossorigin href="./assets/index-C0NveaV7.css">
<script type="module" crossorigin src="./assets/index-Bv6n0dcC.js"></script>
<link rel="stylesheet" crossorigin href="./assets/index-Cu_Cf9io.css">
</head>
<body>
<div id="root"></div>
+69
View File
@@ -0,0 +1,69 @@
"""Tests for the `strix auth` CLI: subcommand routing and provider naming."""
from __future__ import annotations
from typing import TYPE_CHECKING, Any
import pytest
from strix.auth import codex, store
from strix.interface import auth_cli
if TYPE_CHECKING:
from pathlib import Path
@pytest.fixture(autouse=True)
def _tmp_store(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(store, "AUTH_PATH", tmp_path / "home" / ".strix" / "subscription-auth.json")
def test_login_provider_is_chatgpt() -> None:
assert auth_cli.LOGIN_PROVIDER == "chatgpt"
assert codex.PROVIDER in auth_cli._ACCEPTED_PROVIDERS
assert "chatgpt" in auth_cli._ACCEPTED_PROVIDERS
def test_unknown_subcommand_returns_usage_error() -> None:
assert auth_cli.run_auth(["bogus"]) == 2
def test_help_returns_zero() -> None:
assert auth_cli.run_auth(["--help"]) == 0
def test_status_not_signed_in() -> None:
assert auth_cli.run_auth(["status"]) == 1
def test_login_rejects_unsupported_provider(monkeypatch: pytest.MonkeyPatch) -> None:
def _should_not_run(*_args: Any, **_kwargs: Any) -> dict[str, Any]:
msg = "OAuth flow must not start for an unsupported provider"
raise AssertionError(msg)
monkeypatch.setattr(auth_cli, "_run_oauth_flow", _should_not_run)
assert auth_cli.run_auth(["login", "gemini"]) == 2
@pytest.mark.parametrize("provider", ["chatgpt", "codex", "ChatGPT"])
def test_login_accepts_provider_aliases(provider: str, monkeypatch: pytest.MonkeyPatch) -> None:
reached = {"flow": False}
def _fake_flow(*_args: Any, **_kwargs: Any) -> dict[str, Any]:
reached["flow"] = True
return {
"type": "oauth",
"provider": "codex",
"access": "a",
"refresh": "r",
"account_id": "acct",
"expires_at": 0,
}
monkeypatch.setattr(auth_cli, "_run_oauth_flow", _fake_flow)
monkeypatch.setattr(codex, "save_record", lambda _record: None)
monkeypatch.setattr(auth_cli, "_persist_subscription_config", lambda _model: None)
assert auth_cli.run_auth(["login", provider]) == 0
assert reached["flow"] is True
+189
View File
@@ -0,0 +1,189 @@
"""Tests for ChatGPT (Codex) subscription auth: PKCE, token handling, store."""
from __future__ import annotations
import base64
import hashlib
import json
import time
from typing import TYPE_CHECKING, Any
import pytest
from strix.auth import codex, store
if TYPE_CHECKING:
from pathlib import Path
def _fake_jwt(account_id: str) -> str:
def seg(obj: dict[str, Any]) -> str:
return base64.urlsafe_b64encode(json.dumps(obj).encode()).rstrip(b"=").decode()
header = seg({"alg": "none"})
payload = seg({"https://api.openai.com/auth": {"chatgpt_account_id": account_id}})
return f"{header}.{payload}.sig"
@pytest.fixture(autouse=True)
def _tmp_store(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
path = tmp_path / "home" / ".strix" / "subscription-auth.json"
monkeypatch.setattr(store, "AUTH_PATH", path)
return path
def test_pkce_challenge_matches_verifier_and_is_unpadded() -> None:
verifier, challenge = codex.generate_pkce()
expected = (
base64.urlsafe_b64encode(hashlib.sha256(verifier.encode()).digest()).rstrip(b"=").decode()
)
assert challenge == expected
assert "=" not in verifier
assert "=" not in challenge
def test_authorize_url_carries_pkce_and_client() -> None:
url = codex.build_authorize_url("chal", "st8")
assert codex.AUTHORIZE_URL in url
assert "code_challenge=chal" in url
assert "code_challenge_method=S256" in url
assert f"client_id={codex.CLIENT_ID}" in url
assert "state=st8" in url
@pytest.mark.parametrize(
("value", "expected"),
[
("http://localhost:1455/auth/callback?code=AAA&state=BBB", ("AAA", "BBB")),
("AAA#BBB", ("AAA", "BBB")),
("code=AAA&state=BBB", ("AAA", "BBB")),
("AAA", ("AAA", None)),
("", (None, None)),
],
)
def test_parse_redirect_input(value: str, expected: tuple[str | None, str | None]) -> None:
assert codex.parse_redirect_input(value) == expected
@pytest.mark.parametrize(
("model", "expected"),
[
("openai/gpt-5.5", "gpt-5.5"),
("gpt-5.4", "gpt-5.4"),
# A configured-but-unlisted OpenAI/bare name is passed through; the backend validates.
("openai/gpt-5.6", "gpt-5.6"),
# Another provider can't be served by a ChatGPT subscription → coerced to default.
("anthropic/claude-opus-4-8", codex.DEFAULT_CODEX_MODEL),
("deepseek/deepseek-v4-pro", codex.DEFAULT_CODEX_MODEL),
("vertex_ai/gemini-3-pro", codex.DEFAULT_CODEX_MODEL),
(None, codex.DEFAULT_CODEX_MODEL),
("", codex.DEFAULT_CODEX_MODEL),
],
)
def test_normalize_model(model: str | None, expected: str) -> None:
assert codex.normalize_model(model) == expected
@pytest.mark.parametrize(
("model", "compatible"),
[
("openai/gpt-5.5", True),
("gpt-5.4", True),
("gpt-5.1-codex", True), # bare name: backend is the authority
("anthropic/claude-opus-4-8", False),
("deepseek/deepseek-v4-pro", False),
("", False),
(None, False),
],
)
def test_is_backend_compatible(model: str | None, compatible: bool) -> None:
assert codex.is_backend_compatible(model) is compatible
def test_account_id_from_jwt() -> None:
assert codex._account_id_from_jwt(_fake_jwt("acct-42")) == "acct-42"
assert codex._account_id_from_jwt("not-a-jwt") is None
assert codex._account_id_from_jwt("") is None
def test_store_roundtrip_and_logout() -> None:
assert codex.read_record() is None
assert codex.is_authenticated() is False
codex.save_record(
{
"type": "oauth",
"provider": "codex",
"access": _fake_jwt("acct-42"),
"refresh": "r1",
"account_id": "acct-42",
"expires_at": time.time() + 3600,
}
)
record = codex.read_record()
assert record is not None
assert record["account_id"] == "acct-42"
assert codex.is_authenticated() is True
codex.logout()
assert codex.read_record() is None
codex.logout() # no-op when already gone
def test_read_record_rejects_incomplete_records() -> None:
store.write_provider("codex", {"type": "oauth", "access": "a"}) # missing refresh/account
assert codex.read_record() is None
assert codex.is_authenticated() is False
def test_get_valid_token_returns_stored_when_fresh(monkeypatch: pytest.MonkeyPatch) -> None:
def _boom(_payload: dict[str, str]) -> dict[str, Any]:
msg = "should not refresh a fresh token"
raise AssertionError(msg)
monkeypatch.setattr(codex, "_post_form", _boom)
codex.save_record(
{
"type": "oauth",
"provider": "codex",
"access": "access-fresh",
"refresh": "r1",
"account_id": "acct-42",
"expires_at": time.time() + 3600,
}
)
assert codex.get_valid_token() == ("access-fresh", "acct-42")
def test_get_valid_token_refreshes_and_persists_rotation(monkeypatch: pytest.MonkeyPatch) -> None:
calls = {"n": 0}
def _fake_post(payload: dict[str, str]) -> dict[str, Any]:
calls["n"] += 1
assert payload["grant_type"] == "refresh_token"
assert payload["refresh_token"] == "r1"
return {"access_token": _fake_jwt("acct-42"), "refresh_token": "r2", "expires_in": 3600}
monkeypatch.setattr(codex, "_post_form", _fake_post)
codex.save_record(
{
"type": "oauth",
"provider": "codex",
"access": "stale",
"refresh": "r1",
"account_id": "acct-42",
"expires_at": time.time() - 10, # already expired
}
)
_access, account_id = codex.get_valid_token()
assert calls["n"] == 1
assert account_id == "acct-42"
# Rotated refresh token was written back to the store.
assert codex.read_record()["refresh"] == "r2"
def test_get_valid_token_raises_when_not_signed_in() -> None:
with pytest.raises(codex.CodexAuthError) as exc:
codex.get_valid_token()
assert exc.value.code == "not_authenticated"
+138
View File
@@ -0,0 +1,138 @@
"""Regression test for the ChatGPT Codex backend's streaming requirement.
The backend rejects non-streamed requests with ``{"detail": "Stream must be set
to true"}``. ``_CodexResponsesModel`` must therefore issue a streamed request
even from the non-streaming ``get_response`` path and aggregate the events into
a single response. A local server that mimics that behaviour proves the wrapper
works where the stock responses model would fail.
"""
from __future__ import annotations
import json
import threading
from http.server import BaseHTTPRequestHandler, HTTPServer
from typing import TYPE_CHECKING, Any
import pytest
from agents.model_settings import ModelSettings
from agents.models.interface import ModelTracing
from agents.models.openai_responses import OpenAIResponsesModel
from openai import AsyncOpenAI, BadRequestError
from strix.config.models import _CodexResponsesModel
if TYPE_CHECKING:
from collections.abc import Iterator
def _response_payload() -> dict[str, Any]:
return {
"id": "resp_1",
"object": "response",
"created_at": 0,
"status": "completed",
"model": "gpt-5.5",
"output": [
{
"type": "message",
"id": "m1",
"status": "completed",
"role": "assistant",
"content": [{"type": "output_text", "text": "OK", "annotations": []}],
}
],
"usage": {
"input_tokens": 1,
"output_tokens": 1,
"total_tokens": 2,
"input_tokens_details": {"cached_tokens": 0},
"output_tokens_details": {"reasoning_tokens": 0},
},
"parallel_tool_calls": False,
"tool_choice": "auto",
"tools": [],
"metadata": {},
"temperature": 1.0,
"top_p": 1.0,
"error": None,
"incomplete_details": None,
"instructions": None,
"max_output_tokens": None,
}
class _Handler(BaseHTTPRequestHandler):
def log_message(self, *args: Any) -> None:
pass
def do_POST(self) -> None:
length = int(self.headers.get("Content-Length", 0))
body = json.loads(self.rfile.read(length) or b"{}")
if not body.get("stream"):
payload = json.dumps({"detail": "Stream must be set to true"}).encode()
self.send_response(400)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(payload)))
self.end_headers()
self.wfile.write(payload)
return
event = {
"type": "response.completed",
"sequence_number": 0,
"response": _response_payload(),
}
frame = f"event: response.completed\ndata: {json.dumps(event)}\n\n".encode()
self.send_response(200)
self.send_header("Content-Type", "text/event-stream")
self.end_headers()
self.wfile.write(frame)
@pytest.fixture
def backend_url() -> Iterator[str]:
server = HTTPServer(("127.0.0.1", 0), _Handler)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
try:
yield f"http://127.0.0.1:{server.server_address[1]}/backend-api/codex"
finally:
server.shutdown()
server.server_close()
def _client(base_url: str) -> AsyncOpenAI:
return AsyncOpenAI(api_key="tok", base_url=base_url)
def _call_kwargs() -> dict[str, Any]:
return {
"system_instructions": "s",
"input": "hi",
"model_settings": ModelSettings(
store=False, response_include=["reasoning.encrypted_content"]
),
"tools": [],
"output_schema": None,
"handoffs": [],
"tracing": ModelTracing.DISABLED,
"previous_response_id": None,
"conversation_id": None,
"prompt": None,
}
@pytest.mark.asyncio
async def test_stock_model_fails_on_non_streamed_backend(backend_url: str) -> None:
model = OpenAIResponsesModel(model="gpt-5.5", openai_client=_client(backend_url))
with pytest.raises(BadRequestError, match="Stream must be set to true"):
await model.get_response(**_call_kwargs())
@pytest.mark.asyncio
async def test_codex_model_streams_and_aggregates(backend_url: str) -> None:
model = _CodexResponsesModel(model="gpt-5.5", openai_client=_client(backend_url))
response = await model.get_response(**_call_kwargs())
assert response.output[0].content[0].text == "OK"
assert response.usage.total_tokens == 2
+1
View File
@@ -35,6 +35,7 @@ async def test_persistent_rate_limit_stops_gracefully(
settings = types.SimpleNamespace(
llm=types.SimpleNamespace(
model="openai/gpt-4o",
auth_mode="api_key",
reasoning_effort="high",
force_required_tool_choice=False,
timeout=300,
+3 -1
View File
@@ -43,10 +43,12 @@ def _patch_engine_scaffold(
settings = types.SimpleNamespace(
llm=types.SimpleNamespace(
model="openai/gpt-4o",
auth_mode="api_key",
reasoning_effort="high",
force_required_tool_choice=False,
timeout=300,
)
),
runtime=types.SimpleNamespace(max_context_images=3),
)
monkeypatch.setattr(runner, "load_settings", lambda: settings)
monkeypatch.setattr(runner, "configure_sdk_model_defaults", lambda _settings: None)
+46
View File
@@ -0,0 +1,46 @@
"""Subscription runs track tokens but report zero cost."""
from __future__ import annotations
from agents.usage import Usage
from strix.report.usage import LLMUsageLedger
def _usage() -> Usage:
usage = Usage()
usage.requests = 1
usage.input_tokens = 1000
usage.output_tokens = 200
usage.total_tokens = 1200
return usage
def test_zero_cost_ledger_keeps_tokens_but_reports_no_cost() -> None:
ledger = LLMUsageLedger()
ledger.zero_cost = True
ledger.record(agent_id="a", usage=_usage(), agent_name="strix", model="gpt-5.5")
record = ledger.to_record()
assert record["cost"] == 0.0
assert record["total_tokens"] == 1200
assert record["input_tokens"] == 1000
assert record["output_tokens"] == 200
assert ledger.total_cost == 0.0
def test_zero_cost_ledger_ignores_observed_cost() -> None:
ledger = LLMUsageLedger()
ledger.zero_cost = True
ledger.record_observed_cost(4.20)
assert ledger.total_cost == 0.0
def test_normal_ledger_still_estimates_cost() -> None:
# Sanity check the flag is opt-in: without it, an OpenAI-native model still
# accrues an estimated cost (proves zeroing is what suppresses it).
ledger = LLMUsageLedger()
ledger.record(agent_id="a", usage=_usage(), agent_name="strix", model="gpt-5.5")
assert ledger.to_record()["total_tokens"] == 1200
# Cost estimation depends on litellm's cost map; it should be >= 0 and not error.
assert ledger.total_cost >= 0.0