get_available_skills() now yields {name, description} records, so the
coverage gap check reads names out of them rather than iterating the
records themselves.
A real scan surfaced both. An agent carrying path_traversal_lfi_rfi
records "Path Traversal", not the skill's filename, so requiring every
token of the leaf published "treat it as unexamined" for a class that
had been tested and had a finding filed against it; 18 multi-token
skills were exposed. Each vulnerability skill now declares how it can
appear in a ledger row, with a test that a new skill must do the same.
The root agent delegates rather than tests, so it recorded nothing on
every clean run and every report carried a false Not Covered line. It is
exempt while it has children, and only when the snapshot identifies one
unambiguous root -- a root that worked alone is still held to the rule.
Promote the coverage ledger from runtime state to a deliverable:
coverage.json beside vulnerabilities.json, a Coverage section rendered
into the report from the ledger rather than transcribed by an agent, and
SARIF pass / notApplicable / open results so a consumer can tell 'tested
and clean' from 'never tested'.
Ground it in what the runtime observed rather than only what agents
claimed: a risk class an agent carried a skill for and never accounted
for is published as a gap (and surfaced back to the root agent from
finish_scan while it can still act), and a run cut short is stamped
incomplete on both the artifact and the SARIF invocation.
Also: make the ledger's duplicate check and insertion one critical
section and persist under the lock; key a checkout and the URL it was
cloned from onto one threat-model identity; render the calibration
metadata (counterevidence, confidence, severity change conditions, fix
verification) that was being stored and then dropped.
Treat tool_output_max_tokens as a ceiling so an explicit model-supplied
cap can't exceed it, and derive the truncation notice's dropped-line
count from the lines actually kept after the byte-trim pass. Also cast
the pygments fallback lexer so it satisfies the resolve_lexer return
type under the pre-commit mypy hook.
* fix(report): prevent code-fence breakout in vulnerability markdown
render_vulnerability_md wrapped LLM-authored poc_script_code and code
snippet values in a fixed three-backtick fence, so a triple-backtick inside
the value closed the fence early and the rest rendered as live markdown
(headings, tracking-beacon images) in the shareable report deliverable.
Open each such block with a fence one backtick longer than the longest
backtick run in the payload (CommonMark: a block closes only on a fence at
least as long as the opener), so the content always renders verbatim. The
adjacent ```diff block is already safe (its lines are '- '/'+ ' prefixed and
so can never be a bare-backtick closing fence) and is left unchanged.
Fixes#815
* fix(report): indent multiline snippets
---------
Co-authored-by: thejesh23 <thejesh23@users.noreply.github.com>
Co-authored-by: Alex Schapiro <bearsyankees@gmail.com>
Builds on the SARIF 2.1.0 emitter (#626): give each SARIF rule one or more
`stride:<leg>` tags (Spoofing / Tampering / Repudiation / Information
disclosure / Denial of service / Elevation of privilege) derived from the
finding's CWE, so consumers — the GitHub code-scanning Security tab, ASPM
dashboards, coverage reports — can group and filter findings by
threat-model leg. SARIF results inherit their rule's tags via ruleId, so
tagging the rule is sufficient.
- _CWE_TO_STRIDE maps common CWEs to legs (dominant leg first where a CWE
spans several); unmapped / no-CWE findings fall back to a default
(tampering + information-disclosure) so every finding carries >=1 leg
and downstream reports have no coverage gaps.
- Includes mappings for CWEs surfaced by real scans: 798 (hardcoded
creds), 862 (missing authz), 259 (hardcoded password), 1391 (weak
credential).
Tests: tests/report/test_sarif_stride.py (14 cases — mapping, normalization
of CWE-306/306/"cwe: 306" forms, default fallback, rule-tag emission).
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(report): SARIF 2.1.0 emitter for CI / code-scanning integration
Strix emits CSV + markdown + JSON but no SARIF, so findings can't feed
GitHub code-scanning, an ASPM, or any SARIF-consuming CI gate. Add a
stdlib-only emitter (strix/report/sarif.py) and always write findings.sarif
from ReportState._save_artifacts, beside the existing artifacts.
Design invariants (learned from running this in production):
- Stable partialFingerprints.primaryLocationLineHash per finding, so a
re-scan that re-words a title doesn't churn code-scanning alert IDs.
- Class/category hashing so the same vuln class maps to a stable ruleId
across scans rather than drifting.
- Findings with no code location anchor to SECURITY.md with a synthetic
location marker instead of being silently dropped.
- Always emit (even with zero findings) so a clean re-scan overwrites a
stale findings.sarif and code-scanning auto-resolves fixed alerts.
- tool.driver.version reports the strix package version.
- Fully isolated in its own try/except: a SARIF build error must never
break the CSV/MD/run-record path.
Verified end-to-end on v1.0.4 against a SQLi/cmd-inj/weak-hash fixture:
3 findings -> valid SARIF 2.1.0, 3 results, real code locations, distinct
per-finding fingerprints.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(report): complete SARIF code scanning metadata
---------
Co-authored-by: bearsyankees <bearsyankees@gmail.com>
Line 72 was over-indented, causing an IndentationError on import of strix/report/writer.py and breaking main. Also bump the mirrors-mypy pre-commit hook to v1.17.1 to avoid the mypy 1.16.0 internal crash (python/mypy#19412) on openai/_client.py.
* fix: resolve pre-commit check failures
- Change RuntimeError to TypeError for type validation in report/writer.py
- Update pyupgrade to v3.21.2 for Python 3.14 compatibility
* feat(cli): add --max-budget-usd flag
Raises BudgetExceededError in ReportUsageHooks after each LLM call when
accumulated cost reaches the limit, with clean "stopped" status and
child-agent cancellation in non-interactive mode.
* test: add budget enforcement unit tests
7 tests covering no-budget, under-budget, at-limit, over-limit, error
message content, None report state, and exception hierarchy.
Also adds pytest/pytest-asyncio to dev deps and a mypy override for tests.
* fix(budget): validate positive budget and check the live cost ledger
Two hardening fixes for --max-budget-usd enforcement:
- Reject non-positive budgets. ReportUsageHooks now raises ValueError for
max_budget_usd <= 0, and the CLI validates the flag via a custom argparse
type so '--max-budget-usd 0' fails fast with a friendly message instead of
silently killing the scan on the first model response.
- Read the live cost. The budget check now reads ReportState.get_total_llm_cost()
(the live ledger) instead of the persisted run-record snapshot, so it stays
accurate even when a usage save fails after a model call.
* fix(budget): stop the entire scan deterministically when the limit is hit
Previously a BudgetExceededError was handled per-agent: it was swallowed in
interactive mode (the loop kept waiting), a child's error escaped its detached
task as an unretrieved-exception warning, the parent was never released from
wait_for_message, and the stop was logged at ERROR with a traceback as if the
agent had failed.
Replace that with a single scan-wide signal on the coordinator:
- AgentCoordinator.trigger_budget_stop() sets a flag and wakes every parked
agent; wait_for_message returns as soon as the flag is set.
- The run loops check coordinator.budget_stopped and raise to exit cleanly,
marking themselves 'stopped'. The root's exception reaches run_strix_scan's
handler, which cancels descendants and tears the scan down once; child
exceptions are swallowed in their detached task.
- The budget stop is logged at INFO, not as a failure.
This is deterministic regardless of tree depth or which agent first sees the
limit, fixing the interactive/TUI hang where a deep agent's stop never reached
a parked root. Also re-raises BudgetExceededError explicitly in the stream
handler so it can't be mistaken for the LiteLLM 'after shutdown' race.
* fix(budget): treat a budget stop as a clean stop in the TUI
Add an explicit BudgetExceededError handler in the TUI scan thread so that, if
the error ever reaches it, the budget stop is logged as a graceful stop rather
than surfaced as a red scan error by the broad 'except Exception'. The runner
normally absorbs the error and returns cleanly, so this is defensive depth for
a money-spending feature.
* docs(cli): document --max-budget-usd behavior and limitations
Clarify that the budget is cumulative across all agents, checked after each
model response, that the scan stops cleanly (not as a failure), that the value
must be > 0, and that spend can slightly overshoot due to in-flight calls and
best-effort cost estimation.
* Apply suggestions from code review
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Register a litellm.success_callback that captures kwargs['response_cost']
into a new observed-cost bucket on LLMUsageLedger. record() skips the
tokens-times-registry estimate for LiteLLM-routed models so we do not
double-count with the callback; OpenAI direct routes keep estimating
since LiteLLM is not invoked for them. Per-agent attribution for
LiteLLM-routed calls is apportioned by token share at to_record() time.
Drop every hand-rolled provider table and per-model gating that had
accumulated in the model-handling layer:
* normalize_model_name no longer auto-prefixes bare claude-* / gemini-*
names. Users supply the full <provider>/<model> form. The function
became literally model_name.strip(), so callers now inline that and
the function is removed.
* tool_choice="required" is gone everywhere. Thinking-mode endpoints
(Anthropic, DeepSeek /beta) reject it; modern reasoning models don't
need it; non-interactive runs already have
_append_noninteractive_tool_required_message as the convergence
backstop. model_supports_reasoning, model_known_to_registry, and
_model_cost_entry were only used to gate this and follow it out.
* Reasoning(effort=...) is now attached whenever
STRIX_REASONING_EFFORT is non-none. litellm.drop_params=True absorbs
it for non-reasoning models.
* Warm-up's bare-name OpenAI 401 hint is removed (false-positive prone,
relied on substring matching).
* reset_tool_choice on SandboxAgent is no-op now (no tool_choice gets
set) and is removed.
* report/dedupe.py was still routing through stock MultiProvider, so
non-OpenAI configs failed the dedupe LLM pass; switch it to
StrixProvider.
Verified end-to-end against modern provider strings (openai/gpt-5.4,
anthropic/claude-opus-4-7, deepseek/deepseek-reasoner,
gemini/gemini-2.5-pro, groq/, xai/, mistral/, together_ai/, perplexity/,
openrouter/, litellm/ legacy form, and whitespace-padded input): 18/18
cases route correctly, env vars mirror via litellm.validate_environment,
and ModelSettings carries no tool_choice. mypy strict passes.
Both backends share session/version/first-run helpers in
strix/telemetry/_common.py and fire from the same four call sites in
strix/interface/main.py and strix/report/state.py. STRIX_TELEMETRY is
the single toggle for both.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Devin AI