From 256df30d287c6d89be77302fe3012a9f376546ec Mon Sep 17 00:00:00 2001 From: Ahmed Allam Date: Mon, 17 Aug 2026 09:11:59 +0000 Subject: [PATCH] reporting: require the source-to-sink trace in reachability evidence, not just CVSS reasoning --- .../skills/custom/dependency_cve_scanning.md | 43 +++++++++++-------- strix/tools/reporting/tool.py | 26 +++++++---- tests/test_reporting_fields.py | 1 + 3 files changed, 43 insertions(+), 27 deletions(-) diff --git a/strix/skills/custom/dependency_cve_scanning.md b/strix/skills/custom/dependency_cve_scanning.md index 1b9a4536..5f4d09a2 100644 --- a/strix/skills/custom/dependency_cve_scanning.md +++ b/strix/skills/custom/dependency_cve_scanning.md @@ -161,7 +161,23 @@ fi verdict/evidence onto its siblings; run the symbol search against each CVE's own affected-symbol list. The import check (step 1) is the only part shared across a package's CVEs. -3. If the analysis was not performed or is inconclusive (obfuscated code, +3. **Source-to-sink trace — do this whenever step 2 found a symbol hit.** A + symbol hit alone says the code calls the vulnerable API; it does not say + who can reach it. Start at the sink (the exact line that calls the + vulnerable function) and walk backwards hop by hop to the source: the + entry point that carries untrusted input (HTTP route, CLI argument, queue + or webhook payload, uploaded file, config value). Read each intermediate + function; when a hop is a thin wrapper, go one step deeper — never stop at + the first caller. Record what each hop enforces: authentication, a role + check, validation, a feature flag, a size or type limit, a default that is + off in production. + Write the chain into `reachability_evidence` as + `entry point -> intermediate call -> package call` with a + repository-relative `file:line` for every hop, and say who controls the + input. If no source reaches the sink, say that too — the level stays + `vulnerable_symbol_used` (the call is real), and the trace is what tells + the reader it is only reachable from, say, an operator CLI. +4. If the analysis was not performed or is inconclusive (obfuscated code, dynamic loading, unparsable sources) ⇒ `unknown` and say why in `assumptions`. @@ -258,24 +274,15 @@ Set only what your usage analysis supports: - `CR`/`IR`/`AR` `H`/`M`/`L` — the security requirement of the data or service the package handles (credentials or payment data raise `CR`). -Ground every metric in a **source-to-sink trace**, not in a general impression -of the package. Before you set any metric: +Ground every metric in the **source-to-sink trace** from the usage analysis +(step 3 above), not in a general impression of the package. Derive the metrics +from that chain: `MAV`, `MPR`, and `MUI` come from what the source requires; +`MAC` comes from the preconditions the hops enforce; `MC`, `MI`, and `MA` come +from the data and privileges available at the sink; `CR`, `IR`, and `AR` come +from what that data is worth. -1. Find the sink: the exact line where this codebase calls the vulnerable - function or class of the package. -2. Walk backwards hop by hop to the source: the entry point that carries - untrusted input (HTTP route, CLI argument, queue or webhook payload, - uploaded file, config value). Read each intermediate function. When a hop - is a thin wrapper, go one step deeper — never stop at the first caller. -3. Record what each hop enforces: authentication, a role check, validation, a - feature flag, a size or type limit, a default that is off in production. -4. Derive the metrics from that chain. `MAV`, `MPR`, and `MUI` come from what - the source requires. `MAC` comes from the preconditions on the hops. `MC`, - `MI`, and `MA` come from the data and privileges available at the sink. - `CR`, `IR`, and `AR` come from what that data is worth. - -If the chain breaks — no source reaches the sink, or you cannot follow a hop — -say so and omit the contextual fields instead of guessing. +No trace, no contextual metrics: if you did not reach a symbol hit, or you +could not follow a hop, omit the contextual fields instead of guessing. `contextual_cvss_reasoning` is required with the metrics. Write two to four sentences that another engineer can check without opening the repository. Name diff --git a/strix/tools/reporting/tool.py b/strix/tools/reporting/tool.py index d0685119..1fb7b83f 100644 --- a/strix/tools/reporting/tool.py +++ b/strix/tools/reporting/tool.py @@ -1188,6 +1188,17 @@ async def create_dependency_report( (required for any level other than ``unknown``): repo-relative ``file:line`` of the import or symbol usage, the matched advisory symbols, or the govulncheck call-path excerpt. + Whenever you found the vulnerable symbol in use, also give the + **source-to-sink trace** here: start at the vulnerable package + call site and walk backwards hop by hop to the entry point + that carries untrusted input (HTTP route, CLI argument, queue + message, webhook, config file), going one step deeper whenever + a hop is a wrapper. Write it as ``entry point -> intermediate + call -> package call`` with a ``file:line`` per hop, name what + each hop enforces (auth, role check, validation, a flag that + is off in production), and say who controls the input. State + it plainly when no entry point reaches the sink — that is the + most useful result a reader can get. contextual_cvss_metrics: Optional CVSS v3.1 **environmental** metrics that reframe the published score for this codebase, as a mapping of metric to value: ``MAV`` (N/A/L/P), ``MAC`` @@ -1196,15 +1207,12 @@ async def create_dependency_report( Set only the metrics your evidence supports (for example ``{"MAC": "H", "MC": "L"}`` when the vulnerable path needs a precondition this deployment enforces and the data at risk is - limited). Base the values on a **source-to-sink trace**: start - at the entry point that carries untrusted input (HTTP route, - CLI argument, queue message, webhook, config file), follow - each hop of the data through this codebase, and end at the - vulnerable package call site. Go one step deeper whenever a - hop is a wrapper — never stop at the first caller. Adjust - ``MAV`` / ``MPR`` / ``MUI`` from what that entry point - actually requires, and ``MC`` / ``MI`` / ``MA`` from the data - and privileges reachable at the sink. You never supply base + limited). Derive every value from the **source-to-sink + trace** you recorded in ``reachability_evidence``: adjust + ``MAV`` / ``MPR`` / ``MUI`` from what the entry point + actually requires, ``MAC`` from the preconditions the hops + enforce, and ``MC`` / ``MI`` / ``MA`` from the data and + privileges reachable at the sink. You never supply base metrics or a score: the base vector comes from the advisory and the adjusted score is computed from the resulting vector. Omit the field when the trace does not change the published diff --git a/tests/test_reporting_fields.py b/tests/test_reporting_fields.py index 688c88ec..2fa78547 100644 --- a/tests/test_reporting_fields.py +++ b/tests/test_reporting_fields.py @@ -887,6 +887,7 @@ def test_dep_tool_exposes_contextual_cvss_params() -> None: ): assert field in dep_props assert "source-to-sink" in dep_props["contextual_cvss_metrics"]["description"].lower() + assert "source-to-sink" in dep_props["reachability_evidence"]["description"].lower() assert "file:line" in dep_props["contextual_cvss_reasoning"]["description"].lower()