The cache breakpoints are gated on _is_claude_model (name contains
"claude"), but LiteLLM's AnthropicCacheControlHook only *consumes*
cache_control_injection_points for models it recognises as cache-capable
via its statically bundled model map. On a Bedrock route whose model
isn't in that map, the marker passes straight through and Bedrock's
Converse API rejects it outright:
ValidationException: cache_control_injection_points: Extra inputs are
not permitted
— which fails the whole scan at the first LLM call. This bites any
Bedrock Claude model LiteLLM hasn't mapped yet (a just-released model),
and is made worse when LiteLLM can't refresh its remote model map (e.g.
behind a TLS-intercepting corporate proxy) and falls back to a stale
local copy. Observed live on bedrock/global.anthropic.claude-sonnet-5.
Fix: withhold the marker only for a Bedrock route LiteLLM can't confirm
supports prompt caching. Scope is deliberately narrow — Anthropic-native,
Vertex, and OpenRouter Claude tolerate/ignore the marker (or LiteLLM maps
them under keys we don't resolve), so gating those on confirmed support
would DISABLE caching for capable models — the opposite of this PR's
intent. Only Bedrock hard-rejects, so only Bedrock is guarded.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A Strix scan is a long multi-turn agentic loop that re-sends a large, STABLE
prefix every turn — the system prompt plus the tool schemas — while only the
conversation tail changes. Without a caching breakpoint that whole prefix is
re-tokenised and billed at full input rate on every turn; on Bedrock Claude
it's the single biggest lever on scan cost. Measured on a real scan: cache-read
went 0% -> 57% once these injection points are set (roughly halving input cost,
and the ratio climbs on longer scans where the stable prefix dominates more
turns).
LiteLLM already implements this end to end: when `cache_control_injection_points`
is present in the call kwargs its `AnthropicCacheControlHook` fires and emits the
provider-appropriate breakpoint (Anthropic `cache_control`; Bedrock Converse
`cachePoint`), honouring Anthropic's 4-breakpoint cap. `LitellmModel` forwards
`ModelSettings.extra_args` straight into `litellm.acompletion()`, so passing the
points there is all that's needed. We mark the two big stable segments (system
prompt + tool_config = 2 of 4 breakpoints, headroom left).
Deliberately kept at the LiteLLM-config layer rather than a general ModelSettings
caching flag — that's the direction the Agents SDK maintainer prescribed when
declining a native `cache_system_prompt` field
(openai/openai-agents-python#3008 / #3009): caching is a LiteLLM/provider
behaviour, and a ModelSettings flag would let strict OpenAI-compatible paths emit
non-standard cache_control parts. Gating on Claude keeps it a strict no-op for
every other provider (no injection points -> the hook never fires); only
Claude-family routes (Anthropic native, Bedrock, Vertex, OpenRouter -> Claude)
honour the marker.
Tests: parametrised, non-vacuous — Claude routes (bedrock/native/openrouter) get
the two injection points; non-Claude (gpt-5/gemini/o3) get extra_args=None.
feat(inputs): implement logic for required tool choice based on model
test(inputs): add tests for force_required_tool_choice behavior
test(runner): update tests to include force_required_tool_choice in settings
