mirror of
https://github.com/usestrix/strix.git
synced 2026-08-16 17:27:26 +02:00
seanturner83andClaude Opus 4.8
cc2b3351b8
The cache breakpoints are gated on _is_claude_model (name contains "claude"), but LiteLLM's AnthropicCacheControlHook only *consumes* cache_control_injection_points for models it recognises as cache-capable via its statically bundled model map. On a Bedrock route whose model isn't in that map, the marker passes straight through and Bedrock's Converse API rejects it outright: ValidationException: cache_control_injection_points: Extra inputs are not permitted — which fails the whole scan at the first LLM call. This bites any Bedrock Claude model LiteLLM hasn't mapped yet (a just-released model), and is made worse when LiteLLM can't refresh its remote model map (e.g. behind a TLS-intercepting corporate proxy) and falls back to a stale local copy. Observed live on bedrock/global.anthropic.claude-sonnet-5. Fix: withhold the marker only for a Bedrock route LiteLLM can't confirm supports prompt caching. Scope is deliberately narrow — Anthropic-native, Vertex, and OpenRouter Claude tolerate/ignore the marker (or LiteLLM maps them under keys we don't resolve), so gating those on confirmed support would DISABLE caching for capable models — the opposite of this PR's intent. Only Bedrock hard-rejects, so only Bedrock is guarded. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>