* fix(llm): cap the tool calls one assistant response may queue
* fix(llm): cap the subscription backend's responses too
---------
Co-authored-by: Ahmed Allam <ahmed39652003@gmail.com>
Some OpenAI-compatible gateways don't support Server-Sent Events (or
deliver them unreliably), but the SDK run loop Strix uses only issues
streamed requests, so such a gateway fails every turn. Add an opt-in
LLM_DISABLE_STREAMING setting that wraps the resolved model in
_NonStreamingModel: each turn makes one non-streaming get_response and
replays the completed result as a single terminal stream event, so tool
calls, usage, and the rest of the agent loop are unchanged. Subscription
(ChatGPT) models are always streamed and are not wrapped.
An httpx.Timeout in ModelSettings.extra_args crashes
ModelSettings.to_json_dict() (PydanticSerializationError) on the Chat
Completions and LiteLLM model paths, which serialize settings for their
tracing generation span — failing every model turn on those paths. Pass
the timeout as a plain float, which httpx-based clients apply as the
read (inactivity) timeout.
The SDK's http_status retry policy only retries errors carrying a known
HTTP status code, but quota/billing (and other provider-side) failures
often surface inside a streamed response as a bare error with no status
code, so they were failing on the first attempt. Add a statusless retry
policy to DEFAULT_MODEL_RETRY (retry count and backoff unchanged) so they
are retried before a genuine exhaustion fails the run; user aborts are
never retried.
The SDK's http_status retry policy only retries errors carrying a known
HTTP status code, but quota/billing (and other provider-side) failures
often surface inside a streamed response as a bare error with no status
code, so they were failing on the first attempt. Add a statusless retry
policy to DEFAULT_MODEL_RETRY so they are retried (before any content is
streamed; user aborts are never retried), restoring the pre-SDK engine's
resilience. If the provider is genuinely exhausted, the error still
propagates and fails the scan after retries.
Register a litellm.success_callback that captures kwargs['response_cost']
into a new observed-cost bucket on LLMUsageLedger. record() skips the
tokens-times-registry estimate for LiteLLM-routed models so we do not
double-count with the callback; OpenAI direct routes keep estimating
since LiteLLM is not invoked for them. Per-agent attribution for
LiteLLM-routed calls is apportioned by token share at to_record() time.
OpenAI's Responses API rejects reasoning.effort on non-reasoning
models like gpt-4o with `unsupported_parameter`, so any scan with
the default STRIX_REASONING_EFFORT=high against gpt-4o crashed at
the first model call. drop_params=True absorbs the rejected param
on LiteLLM-routed models but the SDK's native OpenAI path has no
equivalent.
Lift model_supports_reasoning to a public helper that strips
litellm/, any-llm/, openai/ prefixes and falls back to last-segment
lookup so prefixed forms like anthropic/claude-opus-4-7 resolve
through the bare model_cost entry. make_model_settings regains
model_name and skips Reasoning() when the registry doesn't confirm
support. uses_chat_completions_tool_schema reuses the same helper
(was duplicating the lookup under a misleading name).
OpenAI's Responses API rejects tools[i].type="custom" on non-reasoning
models like gpt-4o (400 with code=unknown_parameter, param=tools).
Strix's SDK-native Filesystem capability registers CustomTool entries
by default, so a bare STRIX_LLM=gpt-4o run failed at the first tool
invocation even though warm-up (a tool-less call) succeeded.
uses_chat_completions_tool_schema now consults
litellm.model_cost[<name>].supports_reasoning for OpenAI routes and
flips to the chat-completions function-tool schema for models that
don't carry the reasoning flag. Same registry-lookup pattern as
is_known_openai_bare_model. Non-OpenAI prefixes and configs with
LLM_API_BASE are unchanged (still function tools).
litellm.suppress_debug_info silences two unsolicited print() calls in
LiteLLM core: the "Provider List: https://docs.litellm.ai/docs/providers"
banner emitted by get_llm_provider_logic and the "Give Feedback /
Get Help" + "If you need to debug this error, use litellm._turn_on_debug()"
pair emitted by exception_mapping_utils on every LiteLLM exception.
Both are unconditional print() calls, not logger output, so log-level
config can't catch them. LiteLLM's own router and proxy_server set the
same flag for the same reason.
Warn on bare unknown model names before warm-up. is_known_openai_bare_model
consults litellm.model_cost and matches only entries whose
litellm_provider == "openai". When the configured STRIX_LLM has no
provider prefix, isn't a known OpenAI model, and no LLM_API_BASE is
set, show a clear panel pointing the user at the <provider>/<model>
form and exit before issuing the doomed request — no more chasing an
"Incorrect API key" 401 from OpenAI when the user actually meant
deepseek/, anthropic/, etc. Custom-base configs are still allowed
through unconfirmed.
Disable LiteLLM's message-logging and streaming-logging knobs to cut
noise and skip one of the two end-of-stream submit paths. The other
path at streaming_handler.py:2206 schedules work on a global
ThreadPoolExecutor that loses to atexit shutdown when the interpreter
is winding down; the SDK's stream consumer surfaces that as a fatal
"cannot schedule new futures after shutdown" RuntimeError even though
the actual stream content was already delivered. Catch and swallow
that specific RuntimeError in _run_cycle so the scan isn't killed by
an upstream end-of-stream logging race.
Drop every hand-rolled provider table and per-model gating that had
accumulated in the model-handling layer:
* normalize_model_name no longer auto-prefixes bare claude-* / gemini-*
names. Users supply the full <provider>/<model> form. The function
became literally model_name.strip(), so callers now inline that and
the function is removed.
* tool_choice="required" is gone everywhere. Thinking-mode endpoints
(Anthropic, DeepSeek /beta) reject it; modern reasoning models don't
need it; non-interactive runs already have
_append_noninteractive_tool_required_message as the convergence
backstop. model_supports_reasoning, model_known_to_registry, and
_model_cost_entry were only used to gate this and follow it out.
* Reasoning(effort=...) is now attached whenever
STRIX_REASONING_EFFORT is non-none. litellm.drop_params=True absorbs
it for non-reasoning models.
* Warm-up's bare-name OpenAI 401 hint is removed (false-positive prone,
relied on substring matching).
* reset_tool_choice on SandboxAgent is no-op now (no tool_choice gets
set) and is removed.
* report/dedupe.py was still routing through stock MultiProvider, so
non-OpenAI configs failed the dedupe LLM pass; switch it to
StrixProvider.
Verified end-to-end against modern provider strings (openai/gpt-5.4,
anthropic/claude-opus-4-7, deepseek/deepseek-reasoner,
gemini/gemini-2.5-pro, groq/, xai/, mistral/, together_ai/, perplexity/,
openrouter/, litellm/ legacy form, and whitespace-padded input): 18/18
cases route correctly, env vars mirror via litellm.validate_environment,
and ModelSettings carries no tool_choice. mypy strict passes.
Users had to type STRIX_LLM=litellm/deepseek/deepseek-chat — the
litellm/ wrapper was Strix-internal plumbing surfacing in user config.
Add StrixProvider, a MultiProvider subclass that routes any non-OpenAI
prefix (deepseek/, anthropic/, groq/, xai/, mistral/, openrouter/, …)
through LitellmProvider with the prefix preserved. normalize_model_name
no longer adds litellm/ to anything; bare claude-* / gemini-* shorthands
expand to anthropic/<model> / gemini/<model> instead of the wrapped form.
Wire StrixProvider into warm_up_llm and RunConfig.model_provider.
litellm/<provider>/<model> and any-llm/<provider>/<model> still resolve
unchanged for users on older config.
Refresh stale model names in the env-validation messages and the
warm-up hint (gpt-5.4, claude-opus-4-7, deepseek-reasoner).
Verified 24-case end-to-end matrix: OpenAI direct vs. LitellmProvider
routing, env-var mirroring via validate_environment, supports_reasoning
detection, and tool_choice gating all behave correctly across modern
providers including the user's unknown DeepSeek SKU.
When the user opts into reasoning_effort but the configured model
isn't in litellm.model_cost at all (private SKUs, fresh releases the
registry hasn't picked up — e.g. deepseek/deepseek-v4-pro), we can't
confirm thinking support and were sending tool_choice="required",
which thinking-mode endpoints reject ("Thinking mode does not support
this tool_choice").
Add model_known_to_registry() and split the decision: when the user
wants reasoning AND the model is either confirmed-reasoning OR
unknown-to-registry, drop tool_choice. The Reasoning(effort=...) param
still only attaches for confirmed-reasoning models, so we don't send
reasoning hints to known non-reasoning models.
Known non-reasoning models (gpt-4o, registry-confirmed) keep
tool_choice="required" unchanged.
Naively uppercasing the routing prefix breaks for providers whose
LiteLLM env var name doesn't match the prefix verbatim:
together_ai/... needs TOGETHERAI_API_KEY (no underscore)
perplexity/... needs PERPLEXITYAI_API_KEY
Ask LiteLLM directly via litellm.validate_environment(model=...) which
env vars it consults for the chosen provider, then setdefault each one
to LLM_API_KEY. This is the SDK-blessed lookup and stays correct for
every provider LiteLLM supports without a hand-maintained name map.
Lowercase the routed model name before lookup so mixed-case user input
(e.g. Together_AI/...) still resolves.
normalize_model_name expands `claude-*` and `gemini-*` shorthands into
`litellm/anthropic/...` and `litellm/gemini/...` at routing time, but
the mirror helper was looking at the raw pre-normalization name — bare
shorthands had no `/` and hit the early return, so ANTHROPIC_API_KEY /
GEMINI_API_KEY were never populated for those users.
Run the same normalization inside the mirror helper so the provider
prefix is consistent with what LiteLLM actually sees downstream.
LiteLLM's per-provider branches (deepseek, anthropic, groq, etc.)
don't consult ``litellm.api_key`` (the module global Strix sets).
They only check the per-call ``api_key`` kwarg and the
``<PROVIDER>_API_KEY`` env var. The SDK's LitellmModel passes
``api_key=None`` by default, so requests went out with an empty
bearer and DeepSeek (and friends) returned 401.
Mirror the user's LLM_API_KEY into the provider-specific env var
(``DEEPSEEK_API_KEY`` for ``deepseek/...``, ``ANTHROPIC_API_KEY``
for ``anthropic/...``, etc.) using LiteLLM's documented convention.
``os.environ.setdefault`` is used so an explicit user env is never
clobbered. The OpenAI branch was already working via
``set_default_openai_key`` + the existing ``litellm.api_key`` global
fallback.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Five rounds of sweep across the tree. Net ~544 lines removed.
Removed:
- Section-divider banners and one-line section labels (# Display
utilities, # ----- list_requests -----, # CVSS breakdown, etc.).
- Module-level prose docstrings on internal modules. Kept one-line
summaries; trimmed multi-paragraph narration about SDK/Strix
responsibility splits, cache strategies, three-source precedence.
- Internal-helper docstrings that just restate the function name —
caido_api helpers (caido_url, get_client, view_request, etc.),
settings-class one-liners (LLMSettings, RuntimeSettings, ...),
UI helper docstrings.
- Args/Returns blocks on non-LLM-facing internal helpers
(build_strix_agent, render_system_prompt, create_or_reuse,
bootstrap_caido) — kept only the genuinely non-obvious params.
- Internal-history phrasing — "Mirrors main-branch shape",
"pre-SDK harness", "previous lookup matched no attribute".
- Narrative comments inside function bodies that explained what the
next line does, design rationale obvious from the surrounding code,
or "we used to..." asides.
- Trailing periods on every error-string literal across the tool tree.
- Duplicated roundtripTime quirk comment (kept the LLM-facing copy in
tools/proxy/tools.py).
Kept (every one names an upstream bug, vendored-code provenance, or
non-obvious data quirk):
- core/runner.py: SDK replay-with-empty-initial-input + on_agent_end
lifecycle gap.
- runtime/docker_client.py: VERBATIM COPY block of the upstream
_create_container body, pinned to SDK v0.14.6.
- runtime/session_manager.py: NO_PROXY for agent-browser CDP loopback.
- tools/proxy/caido_api.py: generated-pydantic Request.raw quirk,
replay double-history pitfall.
- tools/proxy/tools.py: Caido roundtripTime=0 quirk for proxy
captures.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
devin-ai-integration[bot]