Compare commits

..
Author SHA1 Message Date
Jonathan Singer 439169d101 keep the mcp tests from reading your shell's STRIX_MCP_* vars 2026-08-24 08:59:35 -04:00
Jonathan Singer 267fe6f1ed Merge remote-tracking branch 'origin/main' into mcp-support
# Conflicts:
#	strix/interface/viewer/frontend/src/components/live/tool-renderers/index.ts
#	strix/interface/viewer/static/assets/index-Bi_X6kI3.js
#	strix/interface/viewer/static/assets/index-DBJ-RJqo.js
#	strix/interface/viewer/static/assets/index-gEZK6bjO.js
#	strix/interface/viewer/static/index.html
2026-08-24 08:59:31 -04:00
Jonathan Singer fb4c3df9b5 Correct the notes docstring to match how notes reach the agent 2026-08-24 08:12:17 -04:00
devin-ai-integration[bot]andAhmed Allam 391d81bea7 feat(agents): evidence discipline, and coverage as a first-class artifact (#961)
Co-authored-by: Ahmed Allam <ahmed39652003@gmail.com>
2026-08-24 03:34:09 -07:00
Jonathan Singer c5eac30bf0 Say what MCP servers are worth connecting for 2026-08-21 17:38:05 -04:00
devin-ai-integration[bot]andAhmed Allam 1c499c5b2d perf: bootstrap Caido concurrently with the scan start (#1143)
Co-authored-by: Ahmed Allam <ahmed39652003@gmail.com>
2026-08-21 12:09:59 -07:00
Jonathan Singer 01ea94920d Show MCP tool calls distinctly in the terminal and the run viewer 2026-08-21 13:09:24 -04:00
Jonathan Singer afda373f55 Sanitize namespaced tool names so model APIs accept them 2026-08-21 11:08:23 -04:00
devin-ai-integration[bot]andAhmed Allam 1ce43d1b94 perf: take heavy imports off the startup path and pre-warm them in the background (#1141)
Co-authored-by: Ahmed Allam <ahmed39652003@gmail.com>
2026-08-20 20:24:08 -07:00
Alex Schapiro 2cc8167814 docs(skills): correct gRPC guidance, a .proto is not a spec target 2026-08-20 19:27:04 -04:00
Alex Schapiro d6a3ca7e58 docs(skills): document --workspace-file for supporting files 2026-08-20 19:27:04 -04:00
Alex Schapiro 9099710cef docs(skills): fix nonexistent --mount flag, document real targeting flags, add application-security-testing skill
- Remove --mount from two skills: the flag does not exist in the CLI. Local
  paths are mounted writable when passed with -t.
- Document --target-list, --scope-mode, --diff-base, and OpenAPI/Postman
  targets, so agents stop putting spec URLs in --instruction prose.
- Add the application-security-testing skill as the entry point for
  whole-product AppSec requests, routing each asset to the right workflow.
- Drop contractions and Latin abbreviations across the skill prose.
2026-08-20 19:27:04 -04:00
Alex Schapiro 634cb98241 docs(skills): use current OWASP editions (Top 10:2025, API Top 10 2023) 2026-08-20 19:27:04 -04:00
Alex Schapiro 1b36343eea fix(skills): avoid unquoted colon in api-security-testing description 2026-08-20 19:27:04 -04:00
Alex Schapiro b5ef93e744 feat(skills): add target-specific security testing skills (web app, API, OWASP Top 10, code review) 2026-08-20 19:27:04 -04:00
Jonathan Singer 305cb13998 Show errored MCP tool calls as failed in the TUI 2026-08-20 18:19:01 -04:00
RAJVARDHAN PATIL e152c4c7c0 fix(report): raise RuntimeError on non-object run.json (fixes #1109) (#1116) 2026-08-20 13:41:04 -07:00
OpenPay fe758af4fc fix(tui): use single space after ordered-list marker (#1043) 2026-08-20 13:40:12 -07:00
oyasumiandoyasumi deb2057e20 fix(tui): preserve cost when state is truncated (#1086)
Co-authored-by: oyasumi <oyasumi@kantilabs.xyz>
2026-08-20 13:36:01 -07:00
Jonathan Singer b30ed45ed1 Add MCP connection notes and per-run selection; clean up on cancel and dedupe names 2026-08-20 16:31:58 -04:00
Alex Schapiro d6f2218756 Drop strict tool schemas on Claude routes 2026-08-20 23:12:23 +03:00
Jonathan Singer 8fb83f52b1 Add MCP docs and CLI polish: docs page, startup connect summary, --mcp-config flag, compact tool output 2026-08-20 15:39:50 -04:00
Jonathan Singer 209584e7fd add a generic MCP client and a config for connecting MCP servers 2026-08-20 01:44:01 -04:00
oyasumi 6f88b7d7d5 Require viewer session for run data 2026-08-19 15:25:57 -04:00
oyasumi 8d3693df8c Expose viewer host option 2026-08-19 15:25:57 -04:00
bearsyankees 9cd81e5c76 Add semantic browser and Electron security skills 2026-08-19 12:05:47 -04:00
bearsyankees e8272c6a21 Add HTTP differential testing tools 2026-08-19 12:04:43 -04:00
112 changed files with 9566 additions and 702 deletions
+8
View File
@@ -15,6 +15,14 @@ npx skills add usestrix/strix
- `fix-security-vulnerabilities-with-strix` — remediate findings and re-run Strix to verify
- `ci-security-scanning-with-strix` — add PR scanning to CI/CD (self-hosted CLI or managed app)
Target-specific workflows built on the same engine:
- `application-security-testing` — whole-product AppSec review: pick the right test per asset, then rank the results
- `web-app-penetration-testing` — black-box pentest of a live web app or staging site
- `api-security-testing` — REST/GraphQL APIs and the OWASP API Security Top 10 (BOLA/IDOR, authz)
- `owasp-top-10-testing` — systematic OWASP Top 10 assessment with honest per-category coverage
- `find-security-vulnerabilities-in-code` — white-box review of a repo or working tree
**Two ways to run, same engine — pick per situation:**
- **Open-source CLI (self-hosted):** free, fully local, BYO LLM key, needs Docker. Best for local dev loops, air-gapped/offline, and full control.
+30 -1
View File
@@ -116,7 +116,7 @@ Strix is agent-ready. Give Claude Code, Cursor, Codex, or any [SKILL.md-compatib
npx skills add usestrix/strix
```
This installs four skills: **penetration-testing-with-strix** (run headless scans and read results), **managed-pentesting-with-strix** (drive the managed [app.strix.ai](https://app.strix.ai) platform via REST — no local Docker or LLM key), **fix-security-vulnerabilities-with-strix** (remediate + re-scan to verify), and **ci-security-scanning-with-strix** (PR scanning in CI). Agents can run Strix two ways with the same engine — the open-source CLI locally, or the managed cloud when there's no local infra — and read [`AGENTS.md`](AGENTS.md) for a quick reference, [docs.strix.ai/llms.txt](https://docs.strix.ai/llms.txt) for the CLI docs, and [docs.app.strix.ai](https://docs.app.strix.ai) for the API.
This installs nine skills: **penetration-testing-with-strix** (run headless scans and read results), **managed-pentesting-with-strix** (drive the managed [app.strix.ai](https://app.strix.ai) platform via REST — no local Docker or LLM key), **fix-security-vulnerabilities-with-strix** (remediate + re-scan to verify), **ci-security-scanning-with-strix** (PR scanning in CI), plus target-specific workflows: **application-security-testing**, **web-app-penetration-testing**, **api-security-testing**, **owasp-top-10-testing**, and **find-security-vulnerabilities-in-code**. Agents can run Strix two ways with the same engine — the open-source CLI locally, or the managed cloud when there's no local infra — and read [`AGENTS.md`](AGENTS.md) for a quick reference, [docs.strix.ai/llms.txt](https://docs.strix.ai/llms.txt) for the CLI docs, and [docs.app.strix.ai](https://docs.app.strix.ai) for the API.
---
@@ -167,10 +167,15 @@ strix view
# ...or open a specific run by name
strix view my-run-name
# Expose the viewer on all IPv4 interfaces at a fixed port
strix view --host 0.0.0.0 --port 8080 --no-open
```
`strix view` starts a lightweight local server (bound to `127.0.0.1` on a random port) and opens your browser to a private, tokened link. Nothing leaves your machine: the dashboard reads the run's files straight off disk, with no cloud account or upload required. The UI ships prebuilt with Strix, so there is no extra install and no JS build step.
Use `--host 0.0.0.0` to make the viewer reachable from other machines. Replace `0.0.0.0` in the printed URL with the server's reachable IP or hostname. The token in that URL grants access to the selected run's scan data, history, and steering, so only share it with trusted users and restrict the port with your firewall. Requests without the token-derived session cannot read run data.
### What's in the dashboard
- **Overview**: run status, target, and a severity breakdown of everything found so far.
@@ -315,6 +320,30 @@ strix auth status # show the active sign-in
strix auth logout # forget the sign-in
```
#### Connect your own MCP servers
Strix can connect to Model Context Protocol (MCP) servers you list and expose their tools to the agent during a run. Create `~/.strix/mcp-servers.json` with a JSON list of servers. Each entry is either a local `stdio` server that Strix launches as a subprocess, or a remote `http` server:
```json
[
{
"name": "local_fs",
"transport": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/path/to/project"]
},
{
"name": "github",
"transport": "http",
"url": "https://api.githubcopilot.com/mcp/",
"auth": { "kind": "bearer", "token": "your-token" },
"allowed_tools": ["list_issues"]
}
]
```
Each server's tools are namespaced by `name` (for example `local_fs_read_file`). Omit `allowed_tools` to expose every tool the server offers, or set it to a list to restrict which tools the agent can call. The file is optional, and a server that fails to connect is skipped without failing the run. You can point Strix at a different file with `STRIX_MCP_CONFIG`.
**Recommended models for best results:**
- [OpenAI GPT-5.4](https://openai.com/api/) - `openai/gpt-5.4`
+2 -1
View File
@@ -47,7 +47,8 @@
"pages": [
"integrations/github-actions",
"integrations/ci-cd",
"integrations/coding-agents"
"integrations/coding-agents",
"integrations/mcp"
]
},
{
+5
View File
@@ -19,6 +19,11 @@ npx skills add usestrix/strix
| `managed-pentesting-with-strix` | Drive the managed [app.strix.ai](https://app.strix.ai) platform over REST — no local Docker or LLM key needed |
| `fix-security-vulnerabilities-with-strix` | Triage findings, fix root causes, and re-run Strix to verify each fix |
| `ci-security-scanning-with-strix` | Add PR security scanning to GitHub Actions or any CI (self-hosted CLI or managed app) |
| `application-security-testing` | Assess a whole product: choose the right test for each asset, then rank the findings into one remediation plan |
| `web-app-penetration-testing` | Black-box pentest of a live web app or staging site — scope, credentials, and multi-account access-control testing |
| `api-security-testing` | Test a REST/GraphQL API against the OWASP API Security Top 10 — schema-driven enumeration, BOLA/IDOR, authz |
| `owasp-top-10-testing` | Systematic OWASP Top 10 assessment with honest per-category coverage |
| `find-security-vulnerabilities-in-code` | White-box security review of a repo or working tree, with exploits to confirm findings |
Install a single skill with `npx skills add usestrix/strix --skill penetration-testing-with-strix`, or use one without installing:
+131
View File
@@ -0,0 +1,131 @@
---
title: "MCP Servers"
description: "Connect your own MCP servers and expose their tools to the agent"
---
Strix can connect to [Model Context Protocol (MCP)](https://modelcontextprotocol.io) servers you list and expose their tools to the agent during a run. Use this to let the agent read how your system is actually built instead of inferring it from the outside.
A few things it pays off for:
- **A database server.** The agent can read the schema and access policies and see tables left readable without them, rather than guessing from responses.
- **A hosting or infrastructure server.** Deployments, domains and environment variable names tell it what is really running, so it tests what exists instead of what it discovered by crawling.
- **An issue tracker.** Known and accepted risks stop the agent re-reporting findings you already triaged.
- **A logging server.** Reading logs lets it confirm an exploit attempt actually landed instead of inferring it from a status code.
## Setup
Create the file `~/.strix/mcp-servers.json`. It holds a JSON list of the servers you want the agent to reach. Each entry is either a local `stdio` server that Strix launches as a subprocess, or a remote `http` server.
Create the directory if it does not exist, then write the file:
```bash
mkdir -p ~/.strix
```
Paste the servers you want into `~/.strix/mcp-servers.json`. The example below shows one of each transport: a local filesystem server over `stdio` and a remote GitHub server over `http` with a bearer token:
```json
[
{
"name": "local_fs",
"transport": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/path/to/project"]
},
{
"name": "github",
"transport": "http",
"url": "https://api.githubcopilot.com/mcp/",
"auth": { "kind": "bearer", "token": "your-token" },
"allowed_tools": ["list_issues"]
}
]
```
Strix reads this file at the start of each run. There is no default file, so no MCP tools are loaded until you create it. Edit `command`, `args`, `url`, and `token` to match your own servers.
## Fields
<ParamField path="name" type="string" required>
A short label for the connection. Each server's tools are namespaced by
`name` (for example `local_fs_read_file`), so two servers can offer the same
tool name without colliding.
</ParamField>
<ParamField path="transport" type="string">
`stdio` for a local subprocess server, or `http` for a remote server.
</ParamField>
<ParamField path="command" type="string">
For `stdio` servers: the executable Strix launches (for example `npx`).
</ParamField>
<ParamField path="args" type="array">
For `stdio` servers: the arguments passed to `command`.
</ParamField>
<ParamField path="url" type="string">
For `http` servers: the server endpoint URL.
</ParamField>
<ParamField path="auth" type="object">
For `http` servers that need a bearer token:
`{ "kind": "bearer", "token": "your-token" }`.
</ParamField>
<ParamField path="allowed_tools" type="array">
Restrict which tools the agent can call. Omit it to expose every tool the
server offers, or set it to a list of tool names to allow only those. Strix
does not decide for you which of a server's tools only read and which change
things, so run the server in its own read-only mode if it has one.
</ParamField>
<ParamField path="notes" type="string">
Free-text notes for the agent about what this connection is and how you want
it used, for example "Staging analytics database, read-only, prefer aggregate
queries." When set, the notes are given to the agent at the start of the run
as a description of the connection.
</ParamField>
## Choosing connections per run
By default every connection in the file is used on each run. To narrow it for a
single run without editing the file, use either flag (both repeatable):
```bash
strix --mcp-server github -t ... # use only the named connection(s)
strix --mcp-exclude staging-db -t ... # use everything except the named one(s)
```
`--mcp-server` keeps only the connections you name; `--mcp-exclude` drops the
ones you name. Connection names must be unique in the file; if two entries share
a name, the first is kept and the rest are ignored.
## Pointing at a different file
To read the config from another path instead of `~/.strix/mcp-servers.json`, either pass `--mcp-config <path>` on the command line:
```bash
strix --mcp-config ./mcp-servers.json -t ...
```
or set the `STRIX_MCP_CONFIG` environment variable to that path. The flag takes precedence when both are given.
## Startup confirmation
When servers are configured, Strix prints a one-line summary at scan startup, for example `MCP: connected 1 server (14 tools): local_fs`, so you can confirm your servers connected.
## Seeing the calls
Each call the agent makes to one of your servers is shown with its own icon and
labelled with the connection it went out to, in the terminal and in the run
viewer (`strix view`), so a call that left Strix for a server you connected is
easy to pick out of a transcript. The terminal shows the call and its arguments;
results can be large and arbitrary, so read them in the viewer, which shows a
preview you can expand.
## Behavior
- The config file is optional. Without it, a run simply gets no MCP tools.
- A server that fails to connect is skipped and logged, and the run continues without it.
- A single malformed entry is skipped without blocking the valid ones.
+13
View File
@@ -241,6 +241,8 @@ ignore = [
"tests/test_stream_idle_timeout.py" = ["N802", "SLF001"]
"tests/test_unknown_tool_recovery.py" = ["N802"]
"tests/test_report_pdf.py" = ["S105", "S106"]
# Fake MCP server matches the SDK's MCPServer signature; its args are unused.
"tests/test_mcp_client.py" = ["S105", "S106", "ARG002"]
# Stdlib HTTP handler overrides (do_GET/do_POST) and lazy imports that avoid a
# circular dependency with strix.telemetry / strix.interface.viewer.report_pdf.
"strix/interface/viewer/server.py" = ["N802", "PLC0415"]
@@ -270,6 +272,10 @@ ignore = [
"strix/tools/thinking/tool.py" = ["TC002"]
"strix/tools/web_search/tool.py" = ["TC002"]
"strix/tools/proxy/tools.py" = ["TC002", "PLR0911"]
# The generated Caido GraphQL schema is slow to import, so the SDK is imported
# on first proxy call instead of at module scope (keeps it off the launch path).
"strix/tools/proxy/caido_api.py" = ["PLC0415"]
"strix/runtime/caido_bootstrap.py" = ["PLC0415"]
"strix/tools/agents_graph/tools.py" = ["TC002"]
"strix/agents/factory.py" = ["TC002"]
# Entry point: ``Path`` is used at runtime by the typing of the
@@ -280,6 +286,13 @@ ignore = [
# a runtime ``Callable`` annotation on ``vulnerability_found_callback``.
"strix/report/state.py" = ["TC003", "PLR0912", "PLR0915", "E501", "PERF401", "PLC0415"]
"strix/report/usage.py" = ["PLC0415"]
# LiteLLM and the Docker SDK are imported on first use, not at module scope:
# both cost seconds to import and neither is needed until a model call is made
# (or, for Docker, unless the Docker runtime backend is in use).
"strix/core/execution.py" = ["PLC0415"]
"strix/report/pricing.py" = ["PLC0415"]
"strix/llm/compaction.py" = ["PLC0415"]
"strix/llm/context_budget.py" = ["PLC0415"]
# Lazy import of strix.config.models avoids a circular dependency between the
# report pipeline and the config layer.
"strix/report/dedupe.py" = ["PLC0415"]
+61
View File
@@ -0,0 +1,61 @@
---
name: api-security-testing
description: Security-test a REST, GraphQL, or gRPC API with Strix — autonomous agents that enumerate endpoints from an OpenAPI/GraphQL schema (or by crawling), then actually exploit the API-specific vulnerability classes in the OWASP API Security Top 10 (2023) — broken object-level authorization (BOLA/IDOR), broken object property level authorization (excessive data exposure and mass assignment), broken function-level authorization, unrestricted resource consumption, SSRF, injection, and auth/token flaws. Every finding comes with a working proof-of-concept request. Use when the user asks to pentest, security-test, audit, or find vulnerabilities in an API, endpoint, or backend service.
license: Apache-2.0
metadata:
author: usestrix
homepage: https://docs.strix.ai
---
# Security-test an API
APIs fail differently from web UIs: there is no rendered surface to crawl, the interesting bugs are authorization-shaped rather than injection-shaped, and the same endpoint behaves differently per token. This workflow targets those specifics with Strix's autonomous agents, using the current [OWASP API Security Top 10 (2023)](https://owasp.org/API-Security/editions/2023/en/0x11-t10/) as the coverage checklist. For the web-app equivalent, the current edition is the OWASP Top 10:2025 — see **owasp-top-10-testing**.
Install, LLM setup, full CLI flags, and the managed-cloud path are in the **penetration-testing-with-strix** skill. Read it if `strix --version` fails or the target is not an API.
## 1. Gather what the agents need
APIs are near-impossible to test blind, so collect first:
| Input | Why it matters |
|---|---|
| **Schema** — OpenAPI/Swagger file, Postman collection, GraphQL endpoint (introspection), or a gRPC `.proto` | Turns guesswork into full endpoint enumeration. Biggest single win in coverage. An OpenAPI/Swagger or Postman spec (`.json`/`.yaml`/`.yml`) is a target Strix takes directly; a `.proto` is not, so pass it with `--workspace-file`. |
| **Two sets of credentials/tokens**, ideally in different tenants | BOLA/IDOR — API1:2023, still the #1 API risk — can only be *proven* by accessing tenant A's objects with tenant B's token. |
| **A low-privilege and a high-privilege token** | Required to prove broken function-level authorization (API5:2023 — a `user` calling admin-only routes). |
| **Example object IDs** | Lets agents test ID tampering immediately instead of hunting for valid identifiers. |
| **Out-of-scope routes** | Payments, mass notification, destructive admin endpoints. |
| **Rate limits / WAF** in front of the API | Avoids agents burning budget on throttled requests; mention them so testing adapts. |
Ask the user for anything missing — do not fabricate tokens or scan an API they do not own.
## 2. Run the scan
Pass the spec as a **target**, not as prose in the instruction — Strix parses OpenAPI/Swagger (`.json`/`.yaml`) and Postman collection exports directly, so the agents start from the real endpoint list:
```bash
strix -n -t ./openapi.yaml -t https://api.staging.example.com --max-budget 20 \
--instruction "Tenant A token: <tokenA> (org 1111, user id 11, order id 501).
Tenant B token: <tokenB> (org 2222, user id 22).
Admin token: <tokenAdmin>.
Focus: BOLA across orgs (API1), function-level authz on /admin/* (API5), object property level authz on PATCH /users/{id} — both mass assignment and over-exposed fields in list responses (API3), unrestricted resource consumption (API4).
Out of scope: POST /billing/*, POST /notifications/broadcast."
```
- **Postman instead of OpenAPI:** a collection export works as a target (`-t ./collection.postman_collection.json`), or pull one live with `-t postman://<collection-uuid>` (optionally `"postman://<collection-uuid>?env=<environment-uuid>"`), which needs `POSTMAN_API_KEY` in the environment.
- **Many services at once:** put one target per line in a file and pass `--target-list ./targets.txt`, repeatable and combinable with `-t`.
- **Add the backend source for depth:** `-t ./services/api -t https://api.staging.example.com`. With code access the agents can reason about authorization checks and object ownership rather than inferring them from responses.
- **gRPC:** target the endpoint and pass the definition as a workspace file, `-t https://grpc.staging.example.com --workspace-file ./service.proto`. Only `.json`, `.yaml`, and `.yml` specs are recognized as targets, so `-t ./service.proto` fails with "Path exists but is not a directory".
- **GraphQL:** point at the GraphQL endpoint and say whether introspection is enabled; call out that you want batching/aliasing abuse, depth/complexity limits, and per-field authorization tested.
- **Internal/private APIs** unreachable from your machine: use the managed platform's network connector — see **managed-pentesting-with-strix**.
- Use `--instruction-file` when the credential/context block gets long, and keep tokens out of shell history and out of committed files.
- **Supporting files** the agents should read but not test, such as an endpoint wordlist or handwritten notes about the tenancy model: pass `--workspace-file ./notes.md`. The file lands read-only in `/workspace`. Add `:DEST` to choose the path, for example `--workspace-file ./wordlist.txt:lists/wordlist.txt`.
## 3. Verify findings
`strix_runs/<run>/penetration_test_report.md` first, then `vulnerabilities/*.md` — each contains the exact request that proved the issue. Replay it (for example, with `curl`) before reporting; for authorization findings, confirm the response really contains the other tenant's data rather than an empty 200.
`findings.sarif` uploads to GitHub code scanning; `vulnerabilities.json` is the structured index for ticketing.
## 4. Fix, re-test, and keep it tested
Remediate with **fix-security-vulnerabilities-with-strix** (fix the authorization check, not the single endpoint), then re-run against the same target to prove the exploit is dead. Wire it into pull-request CI with **ci-security-scanning-with-strix** so new endpoints get tested as they ship.
@@ -0,0 +1,66 @@
---
name: application-security-testing
description: Application security testing (AppSec) across a whole product with Strix — decide which asset needs which test (source code, running web app, API, CI pipeline), run it, and turn the results into a ranked remediation plan. Autonomous agents exploit and prove each issue instead of emitting static-analysis alerts, so the plan is ordered by what is actually reachable. Use when the user asks for an application security review or audit, an appsec assessment, vulnerability scanning across their stack, a security review before a launch or a customer security questionnaire, or does not yet know which kind of security test they need.
license: Apache-2.0
metadata:
author: usestrix
homepage: https://docs.strix.ai
---
# Application security testing
Entry point for "make my application secure" requests, where the target is not yet a single URL or repo. The job here is to pick the right test per asset, run it, and produce one ranked plan — not to run everything at maximum depth.
Install, LLM setup, all CLI flags, and the managed-cloud path live in the **penetration-testing-with-strix** skill. Read it first if `strix --version` fails.
Only test assets the user owns or is authorized to test. Confirm authorization before the first run, and prefer staging over production, because the agents send real exploit payloads and can change data.
## 1. Map the assets
Ask (or read from the repo) and write the answers down before scanning:
- **Source** — one repo, a monorepo, several services? Which languages/frameworks?
- **Running environments** — is there a staging deployment? A public production site? A local dev server only?
- **APIs** — REST, GraphQL, gRPC? Is there an OpenAPI/GraphQL schema?
- **Authentication** — can you get two test accounts in different tenants? Most high-impact bugs need them.
- **Constraints** — out-of-scope paths, whether production may be touched, budget and wall-clock limits.
If there is no staging environment and production is off limits, say so early. A code-only review is still valuable, but it cannot prove exploitability against a live app.
## 2. Pick the right test per asset
| Asset | Skill to use |
| --- | --- |
| Repository or working tree | **find-security-vulnerabilities-in-code** |
| Live web app or staging site | **web-app-penetration-testing** |
| REST/GraphQL/gRPC API | **api-security-testing** |
| Assessment mapped to OWASP categories | **owasp-top-10-testing** |
| Every pull request, continuously | **ci-security-scanning-with-strix** |
| No Docker, no LLM key, or a report an auditor will accept | **managed-pentesting-with-strix** |
Those skills carry the flags, credential handling, and result-reading details. Do not duplicate their instructions here.
Sequence for a first assessment:
1. Review the code. It is the cheapest run and it maps the authorization model.
2. Pentest staging with credentials, and pass the repo as a second target so the agents keep source context.
3. Add CI scanning, so later regressions are caught without another manual pass.
Run one asset at a time and read each report before starting the next. Findings from the code review make the live run sharper.
## 3. Consolidate into one plan
Findings arrive per run in `strix_runs/<run>/`. Merge them into a single list and rank by **proven impact**, not by scanner severity:
1. Validated exploits reachable without authentication.
2. Validated cross-tenant or privilege-escalation issues.
3. Validated issues needing an authenticated account.
4. Unproven observations (configuration, dependency, and hardening notes) — flag as such, and never present them as confirmed vulnerabilities.
Deduplicate: the same root cause often surfaces in both the code review and the live pentest.
## 4. Be honest about coverage
State plainly what was *not* tested — assets with no staging environment, categories a black-box run cannot reach (logging and alerting, supply-chain integrity, insecure design), and any run that hit its budget or turn cap before finishing. Check `run.json` status and cost against `--max-budget` for each run. An empty result set from a truncated scan is not a clean bill of health.
Then remediate with **fix-security-vulnerabilities-with-strix**, which re-runs Strix against each fix to prove the exploit no longer works.
@@ -12,7 +12,7 @@ metadata:
You can gate PRs two ways — pick based on the environment, or combine them:
- **Managed platform (recommended for most teams)** — connect the GitHub/GitLab/Bitbucket app once and Strix reviews every PR with **no workflow file, no runner, no Docker, and no LLM key**. Results post as PR comments and land in the team dashboard. Best when you want zero CI maintenance, central tracking, or your runners lack Docker. See "Managed platform" below and the **managed-pentesting-with-strix** skill.
- **Self-hosted OSS CLI in your runner** — run a diff-scoped scan as a pipeline step. Fully in your infra, free (BYO LLM key), no external account. Requires Docker on the runner. Best for air-gapped/self-hosted CI or when you don't want scans leaving your environment.
- **Self-hosted OSS CLI in your runner** — run a diff-scoped scan as a pipeline step. Fully in your infra, free (BYO LLM key), no external account. Requires Docker on the runner. Best for air-gapped/self-hosted CI or when you do not want scans leaving your environment.
Both fail the build on validated findings and both emit SARIF 2.1.0, so you can start with one and add the other later.
@@ -63,13 +63,13 @@ jobs:
fi
```
Then tell the user to add two repository secrets: `STRIX_LLM` (model id, e.g. `openai/gpt-5.4`) and `LLM_API_KEY` (the provider key). Do not create these values yourself.
Then tell the user to add two repository secrets: `STRIX_LLM` (model id, for example `openai/gpt-5.4`) and `LLM_API_KEY` (the provider key). Do not create these values yourself.
Notes:
- In CI/headless runs Strix automatically scopes to the PR's changed files (`--scope-mode auto`). If diff resolution fails, keep `fetch-depth: 0` or set `--diff-base` to the PR's actual base branch — use `origin/${{ github.base_ref }}` in GitHub Actions rather than a hard-coded `origin/main`, since repos use different default branches.
- Exit codes: `0` pass, `2` vulnerabilities found (fails the job), `1` setup error.
- The runner needs Docker (default GitHub-hosted Ubuntu runners have it).
- **Size the budget so the scan completes — don't let it fail open.** A `0` exit means "no validated vulnerabilities in what was analyzed"; if `--max-budget` is hit before the diff is fully covered, the scan wraps up early and can still exit `0`. The "Fail unless the scan completed" step above narrows the gap: `strix_runs/<run>/run.json` is `"stopped"` when the scan was cut off at the hard budget limit without a final report. It is not a complete guard — the agents get graduated wrap-up warnings before that limit, and a run that wraps up on a warning still calls `finish_scan` and records `"completed"` with partial coverage. So keep that step in any pipeline that gates merges **and** give the scan real headroom (compare `run.json`'s `llm_usage.cost` against `--max-budget`; if it ran right up to the cap, raise it). For a `quick` diff-scoped PR scan `--max-budget 10` is usually ample, raise it for large diffs.
- **Size the budget so the scan completes — do not let it fail open.** A `0` exit means "no validated vulnerabilities in what was analyzed"; if `--max-budget` is hit before the diff is fully covered, the scan wraps up early and can still exit `0`. The "Fail unless the scan completed" step above narrows the gap: `strix_runs/<run>/run.json` is `"stopped"` when the scan was cut off at the hard budget limit without a final report. It is not a complete guard — the agents get graduated wrap-up warnings before that limit, and a run that wraps up on a warning still calls `finish_scan` and records `"completed"` with partial coverage. So keep that step in any pipeline that gates merges **and** give the scan real headroom (compare `run.json`'s `llm_usage.cost` against `--max-budget`; if it ran right up to the cap, raise it). For a `quick` diff-scoped PR scan `--max-budget 10` is usually ample, raise it for large diffs.
### Optional: upload findings to GitHub code scanning
@@ -90,7 +90,7 @@ Any pipeline works the same way — install, set the two env vars, run headless:
```bash
curl -sSL https://strix.ai/install | bash
# Resolve the PR's base branch robustly (use your CI's base-branch variable if it
# has one, e.g. GitHub Actions: origin/${{ github.base_ref }}). Avoid piping the
# has one, for example GitHub Actions: origin/${{ github.base_ref }}). Avoid piping the
# git lookup into another command — a failed lookup would otherwise be masked.
BASE_BRANCH="${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-}" # GitLab MR target
if [ -z "$BASE_BRANCH" ]; then
@@ -98,7 +98,7 @@ if [ -z "$BASE_BRANCH" ]; then
BASE_BRANCH="${BASE_BRANCH#origin/}"
fi
DIFF_BASE="origin/${BASE_BRANCH:-main}"
# Fail loudly rather than silently narrowing scope (e.g. to HEAD~1, which on a
# Fail loudly rather than silently narrowing scope (for example, to HEAD~1, which on a
# multi-commit branch would scan only the last commit and let earlier ones pass).
if ! git rev-parse --verify --quiet "$DIFF_BASE" >/dev/null; then
echo "Cannot resolve diff base '$DIFF_BASE'. Fetch the base branch (git fetch origin <base>) or set --diff-base explicitly." >&2
@@ -0,0 +1,62 @@
---
name: find-security-vulnerabilities-in-code
description: Find security vulnerabilities in a codebase or repository with Strix — a white-box AI security review that reads your source, reasons about the actual data flow and authorization model, then exploits what it finds in a live sandbox so every reported issue has a working proof-of-concept instead of a noisy static-analysis alert. Covers injection, XSS, SSRF, broken access control and IDOR, insecure deserialization, secrets in code, unsafe dependencies, and business-logic flaws. Use when the user asks to security-scan, security-review, or audit their code, repo, or pull request for vulnerabilities.
license: Apache-2.0
metadata:
author: usestrix
homepage: https://docs.strix.ai
---
# Find security vulnerabilities in code
White-box security review with Strix: the agents read the source to build a model of routes, sinks, and authorization checks, then attempt real exploitation. Findings come with a proof-of-concept, so the output is a short list of proven issues rather than the hundreds of "potential" hits a pattern-matching scanner produces.
Install, LLM setup, all flags, and the managed-cloud path are in the **penetration-testing-with-strix** skill.
## Run it
```bash
# Local working tree
strix -n -t ./ --scan-mode standard --max-budget 15
# A GitHub repo directly
strix -n -t https://github.com/org/app --max-budget 15
# Monorepo: point at the service that matters, not the whole tree
strix -n -t ./services/checkout --max-budget 20
# Only what a branch changed (whole-repo review is wasteful on a large repo)
strix -n -t ./ --scope-mode diff --diff-base origin/main --max-budget 10
```
A local path is mounted into the sandbox **writable**, so the agents can modify it. Run against a clean checkout.
Two things sharply improve results:
1. **Add a running instance of the app.** `-t ./ -t http://host.docker.internal:3000` lets the agents confirm exploitability against live behavior instead of reasoning about it statically — this is the difference between "this looks unsafe" and a validated finding. If nothing is running, static-only findings should be described as unconfirmed.
2. **Scope the review.** Point at the risky subtree and say what matters:
```bash
strix -n -t ./services/api --max-budget 15 \
--instruction "Focus on the authorization layer in src/auth and every route under src/routes/admin. Multi-tenant app: tenant id comes from the JWT. Flag any query that filters by object id without also filtering by tenant."
```
Tenancy model, trust boundaries, and which inputs are attacker-controlled are things the agents cannot infer reliably — tell them.
## Reviewing a pull request instead of the whole repo
For diff-scoped review of a branch or PR (and blocking merges on findings), use **ci-security-scanning-with-strix** — it covers diff scoping, PR comments, and SARIF upload to GitHub code scanning. The managed platform can also review PRs directly via API (**managed-pentesting-with-strix**).
## Read the results
In `strix_runs/<run>/`: `penetration_test_report.md` (start here), `vulnerabilities/*.md` (one per finding, with PoC and remediation), `vulnerabilities.json` / `.csv`, `findings.sarif` (upload to code scanning), `run.json`.
Before reporting to the user, open each finding and check the PoC actually demonstrates impact. Report file and line alongside the exploit so the fix is obvious.
Exit `0` means nothing exploitable was proven in what was analyzed — not that the codebase is clean. Check `run.json` status and cost against `--max-budget`, and note which paths went unreviewed if the run was capped.
## Complementary tooling
This is exploit-validated review, not an exhaustive inventory. Keep a dependency scanner (SCA) and secret scanning in place for complete coverage of known-CVE dependencies and committed credentials; use this for the logic, authorization, and injection bugs those tools structurally cannot find.
## Fix and verify
Hand results to **fix-security-vulnerabilities-with-strix**: patch the root cause (the shared authorization helper, not the one route), then re-run Strix to prove the exploit no longer works.
@@ -27,7 +27,7 @@ Order work by severity: critical → high → medium → low. Every Strix findin
For each finding:
1. Reproduce it with the PoC from the finding file when feasible.
2. Fix the root cause, not the specific payload (e.g. parameterize all queries, don't blocklist one string; enforce authorization in the handler, don't hide the endpoint).
2. Fix the root cause, not the specific payload (parameterize every query instead of blocking one string, and enforce authorization in the handler instead of hiding the endpoint).
3. Prefer the framework's built-in defense (ORM parameterization, template auto-escaping, CSRF middleware, centralized authz) over ad-hoc sanitization.
4. Keep the diff minimal and apply the repo's existing patterns. Finding files often include `fix_before`/`fix_after` snippets — use them as a starting point, not verbatim.
@@ -70,7 +70,7 @@ new_id=$(curl -sS "$BASE/scans/$scan_id/rerun" "${auth[@]}" -X POST | jq -r .sca
Or, if the cloud scan came from a repo/PR, trigger a fresh PR review on the fix branch (`POST /pr-reviews/start`). The platform also retests a single finding directly: `POST /api/v1/vulnerabilities/{vulnerabilityId}/retest`.
- Also re-run the PoC manually when it is a simple request/script — fastest signal.
- Run the project's own test suite to make sure the fix doesn't break behavior.
- Run the project's own test suite to make sure the fix does not break behavior.
## 4. Report
@@ -80,7 +80,7 @@ Useful `CreateScanRequest` fields:
| `domain_ids` / `repository_ids` / `internal_targets` | targets (at least one) |
| `domain_paths` / `repository_branches` | narrow to specific paths / branches |
| `credentials` | authenticated scanning, incl. `mfa_method` (`totp`/`email_otp`/…) + `totp_secret` |
| `headers` | extra HTTP headers (e.g. API keys) for the target |
| `headers` | extra HTTP headers (API keys, for example) for the target |
| `focus` / `concerns` / `context` | steer the agents |
| `upload_ids` | attach uploaded source/docs archives for white-box context |
| `notify_on_completion` / `notification_emails` | email when done |
@@ -89,7 +89,7 @@ Response is `{ scan_id, title, status }` with `status` = `pending`.
## 3. Poll to completion
`GET /scans/{scanId}` (`scans:read`). Status flow: `pending → running → completed` (or `failed` / `cancelled`). Poll on an interval — scans take minutes to hours; don't block.
`GET /scans/{scanId}` (`scans:read`). Status flow: `pending → running → completed` (or `failed` / `cancelled`). Poll on an interval — scans take minutes to hours. Do not block.
```bash
while :; do
@@ -143,10 +143,10 @@ List/inspect via `GET /pr-reviews` and `GET /pr-reviews/{id}`. Repo-level PR-rev
## 7. Continuous testing (schedules & webhooks)
- **Schedules** (`schedules:write`, Pro plan): create recurring scans and trigger them on demand — the managed equivalent of a cron-driven CLI loop.
- **Webhooks** (`webhooks:write`): subscribe to pentest/vulnerability lifecycle events (e.g. `scan.completed`, `vulnerability.created`) to push results into Slack, ticketing, or your own pipeline instead of polling.
- **Webhooks** (`webhooks:write`): subscribe to pentest/vulnerability lifecycle events such as `scan.completed` and `vulnerability.created` to push results into Slack, ticketing, or your own pipeline instead of polling.
See the schedules and webhooks sections at [docs.app.strix.ai](https://docs.app.strix.ai) for payloads.
## Safety
Only scan assets the user's organization owns or is authorized to test. External domain scans require verification (DNS/file/meta-tag) enforced by the platform — don't try to bypass it.
Only scan assets the user's organization owns or is authorized to test. External domain scans require verification (DNS/file/meta-tag) enforced by the platform — do not try to bypass it.
+64
View File
@@ -0,0 +1,64 @@
---
name: owasp-top-10-testing
description: Test an application against the OWASP Top 10 with Strix — autonomous AI agents that attempt real exploits for each category of the current OWASP Top 10:2025 (broken access control including SSRF, security misconfiguration, software supply chain failures, cryptographic failures, injection, insecure design, authentication failures, integrity failures, logging and alerting failures, mishandling of exceptional conditions) and report only what they could actually prove, mapped back to the category with a proof-of-concept. Also covers the OWASP API Security Top 10 (2023). Use when the user asks for an OWASP Top 10 assessment, OWASP compliance testing, or a security review mapped to OWASP categories.
license: Apache-2.0
metadata:
author: usestrix
homepage: https://docs.strix.ai
---
# Test against the OWASP Top 10
The OWASP Top 10 is a taxonomy of risk categories, not a test suite — "OWASP Top 10 testing" means exercising each category against the real application and reporting what's actually exploitable. Strix's agents do the exploitation; this skill covers running it category-by-category and reporting coverage honestly.
**Use the current edition: [OWASP Top 10:2025](https://owasp.org/Top10/)** (8th installment, superseding 2021). Ask the user before targeting an older edition — some compliance checklists still reference 2021, and a report labelled with the wrong edition is misleading. Key differences from 2021: **SSRF is folded into A01**, **A03 Software Supply Chain Failures** expands the old "Vulnerable and Outdated Components", and **A10 Mishandling of Exceptional Conditions** is new; A02 Security Misconfiguration moved 5→2.
Install, LLM setup, and the managed-cloud alternative: **penetration-testing-with-strix**.
## What is and is not testable by an agent
Be straight with the user about this — claiming a clean sweep of all ten is misleading.
| Category (2025) | Coverage |
|---|---|
| A01 Broken Access Control (incl. SSRF) | **Strong** — cross-user/tenant access, privilege escalation, IDOR, and SSRF (including blind, via out-of-band callbacks) are all exploit-validated. Needs two accounts plus a privileged one to prove the authorization half. |
| A02 Security Misconfiguration | **Strong** — debug endpoints, verbose errors, permissive CORS, missing hardening, default credentials, exposed admin surfaces. |
| A03 Software Supply Chain Failures | **Partial** — version fingerprinting, and vulnerable/outdated dependency review when source is supplied. Build-system and distribution-infrastructure compromise (the broader half of this category) is out of scope for a runtime scan — pair with SCA plus build-provenance controls. |
| A04 Cryptographic Failures | **Partial** — transport config, unencrypted data in transit, secrets and tokens leaked in responses. At-rest crypto and key management need source or infra review. |
| A05 Injection | **Strong** — SQL/NoSQL/command/template injection and XSS, exploit-validated. |
| A06 Insecure Design | **Partial** — business-logic abuse (price/quantity tampering, workflow skipping, race conditions) is found where reachable; design intent still needs human review and threat modelling. |
| A07 Authentication Failures | **Strong** — auth bypass, weak session/token handling, password-reset and MFA flaws. |
| A08 Software or Data Integrity Failures | **Partial** — insecure deserialization and unsigned-update paths where reachable; CI/CD trust boundaries are not runtime-testable. |
| A09 Security Logging & Alerting Failures | **Not testable from outside** — requires reviewing the logging and alerting pipeline. State this rather than reporting it as passed. |
| A10 Mishandling of Exceptional Conditions | **Partial** — agents actively probe error handling and fail-open behavior (malformed input, forced errors, race and timeout conditions) and report what leaks or bypasses a control; exhaustive coverage of internal error paths needs source review. |
For APIs, run the same exercise against the **OWASP API Security Top 10 (2023)** — API1 BOLA, API3 Broken Object Property Level Authorization (2019's excessive data exposure + mass assignment merged), API5 broken function-level authorization — using the **api-security-testing** skill.
## Run it
Maximum category coverage comes from giving the agents both the source and a running instance, plus credentials at two privilege levels:
```bash
strix -n \
-t https://github.com/org/app \
-t https://staging.example.com \
--scan-mode deep --max-budget 30 \
--instruction "OWASP Top 10:2025 assessment. Cover every category systematically and map each finding to its 2025 category id.
Accounts: userA@example.com/<pw> (org 1), userB@example.com/<pw> (org 2), admin@example.com/<pw>.
Prioritise A01 (cross-org access, privilege escalation, SSRF), A02, A05, A07, A10.
Out of scope: /billing/*, outbound email."
```
- `--scan-mode deep` matters here: systematically walking ten categories is not a quick scan.
- Without a second account, A01 results are structurally incomplete — say so in the report rather than leaving it implied.
- Need an auditor-facing PDF? Run it through the managed platform and pull the technical report (**managed-pentesting-with-strix**).
## Report honestly
From `strix_runs/<run>/`, group `vulnerabilities/*.md` by category and state, per category: what was attempted, what was proven, and what could not be assessed (A09 always; A03/A04/A06/A08/A10 partially). Label the report with the edition used. Verify each PoC yourself before it goes in front of the user.
A `0` exit code means nothing exploitable was proven **in what was analyzed** — check `run.json` status and cost against `--max-budget`; a budget-capped run is not a completed assessment.
## Then fix and re-test
Remediate with **fix-security-vulnerabilities-with-strix** and re-run to prove each exploit is closed. For ongoing coverage as the app changes, gate pull requests using **ci-security-scanning-with-strix**.
+20 -8
View File
@@ -14,14 +14,14 @@ Strix runs autonomous AI pentesting agents that dynamically exploit a target and
- **Open-source CLI** (self-hosted) — runs on your machine in a Docker sandbox with your own LLM key. Free, fully local, BYO-LLM, air-gap capable. Docs: [docs.strix.ai](https://docs.strix.ai).
- **Cloud API** (managed) — runs on Strix's infrastructure via `https://app.strix.ai/api/v1`. No Docker, no LLM key, no local compute; adds team dashboards, scheduling, PR reviews, downloadable PDF/DOCX reports (Enterprise plan), and internal-network connectors. Docs: [docs.app.strix.ai](https://docs.app.strix.ai). Full workflow in the **managed-pentesting-with-strix** skill.
## Which one? (decide, don't default)
## Which one? (decide, do not default)
Choose honestly based on the situation — neither is "better":
| Situation | Prefer |
|---|---|
| No Docker available, or a sandboxed/hosted agent/CI environment | **Cloud** |
| User has no LLM key / doesn't want to pay per-token or manage models | **Cloud** |
| User has no LLM key / does not want to pay per-token or manage models | **Cloud** |
| Team visibility, shareable dashboard, scheduled/continuous scans, PR reviews, downloadable PDF/DOCX report (Enterprise) | **Cloud** |
| Scanning internal/private infrastructure not reachable from your machine | **Cloud** (network connector) |
| Source must never leave local infra (privacy/air-gap), or fully offline | **OSS CLI** |
@@ -30,7 +30,7 @@ Choose honestly based on the situation — neither is "better":
| CI: runner already has Docker and you want a self-contained gate | **OSS CLI** |
| CI: no Docker, or you want results tracked centrally | **Cloud** |
**Mix them:** e.g. use the OSS CLI for the fast local dev-loop while writing/fixing code, and the Cloud for the authoritative, team-visible scan + report + tracking; or gate PRs with the OSS CLI in CI while the Cloud runs scheduled deep scans and PR reviews across the org. Both emit the same SARIF 2.1.0, so findings line up across environments.
**Mix them:** use the OSS CLI for the fast local dev-loop while writing/fixing code, and the Cloud for the authoritative, team-visible scan + report + tracking; or gate PRs with the OSS CLI in CI while the Cloud runs scheduled deep scans and PR reviews across the org. Both emit the same SARIF 2.1.0, so findings line up across environments.
If unsure and the user has (or will create) an app.strix.ai account, prefer **Cloud** — it avoids all local-infra friction. If they want zero signup / full local control, use the **OSS CLI**.
@@ -70,21 +70,33 @@ strix -n -t https://github.com/org/app -t https://staging.example.com
strix -n -t https://app.example.com \
--instruction "Use credentials user@example.com:pass123. Focus on IDOR and auth bypass."
# Large monorepo: bind-mount instead of copying
strix -n --mount ./huge-monorepo
# API spec as a first-class target (OpenAPI/Swagger or a Postman collection export)
strix -n -t ./openapi.yaml -t https://api.staging.example.com
# Many targets from a file, one per line
strix -n --target-list ./targets.txt --max-budget 30
# Give the agents a file to work with (wordlist, spec, notes) without making it a target
strix -n -t https://staging.example.com --workspace-file ./wordlist.txt --max-budget 20
```
A local path passed with `-t` is mounted into the sandbox **writable** — the agents can read and modify it, so point at a clean checkout, not uncommitted work you care about.
Key flags:
| Flag | Meaning |
|---|---|
| `-t, --target` | URL, repo URL, local path, domain, or IP. Repeatable. |
| `-t, --target` | URL, repo URL, local path, domain, IP, OpenAPI/Postman spec, or `postman://<uuid>`. Repeatable. |
| `--target-list PATH` | File of targets, one per line (`#` comments allowed). Repeatable, combines with `-t`. |
| `-n, --non-interactive` | Headless, exits on completion. Required for agents. |
| `-m, --scan-mode` | `quick` (minutes) / `standard` (~30 min) / `deep` (hours, default). |
| `--instruction` / `--instruction-file` | Credentials, focus areas, scope rules. |
| `--workspace-file PATH[:DEST]` | Place a file from this machine into `/workspace` read-only before the scan, for a wordlist, a spec, or notes. Repeatable. |
| `--max-budget USD` | Hard LLM spend cap; scan wraps up cleanly at the limit. |
| `--max-turns N` | Per-agent turn cap (default 500). |
| `--resume RUN_NAME` | Resume a prior run from `strix_runs/`. |
| `--resume RUN_NAME` | Resume a prior run from `strix_runs/`, with its agent history and targets. Cannot be combined with `-t`. |
| `--scope-mode` | For code targets: `auto` (diff-scope in CI/headless), `diff` (force changed files only), `full` (whole tree). |
| `--diff-base REF` | Branch or commit that `diff` scope compares against. Defaults to the repo's default branch. |
Scans take minutes (`quick`) to hours (`deep`). Run them in the background and poll for completion rather than blocking.
@@ -130,7 +142,7 @@ curl -sS "$BASE/scans/$scan_id" -H "Authorization: Bearer $STRIX_API_TOKEN" | jq
curl -sS "$BASE/scans/$scan_id/sarif" -H "Authorization: Bearer $STRIX_API_TOKEN" -o findings.sarif
```
Ask the user to create the token (and register the target as a domain/repository asset) if they haven't. If Docker/local prerequisites aren't already satisfied, use this path instead of trying to install infra.
Ask the user to create the token (and register the target as a domain/repository asset) if they have not. If Docker/local prerequisites are not already satisfied, use this path instead of trying to install infra.
---
@@ -0,0 +1,54 @@
---
name: web-app-penetration-testing
description: Pentest a web app or website end to end — black-box testing of a live URL, staging environment, or local dev server that finds and exploits real vulnerabilities (auth bypass, broken access control, IDOR, injection, XSS, SSRF, business logic) and proves each one with a working proof-of-concept instead of a signature match. Runs with Strix, either the self-hosted open-source CLI or the managed app.strix.ai cloud. Use when the user asks to pentest, hack, security-test, or audit their web app, website, web application, or staging site.
license: Apache-2.0
metadata:
author: usestrix
homepage: https://docs.strix.ai
---
# Pentest a web application
Black-box (and optionally source-assisted) penetration testing of a running web app with Strix's autonomous agents. Every reported finding is validated with a working exploit, so there are no signature-based false positives to triage.
Install, LLM setup, all CLI flags, and the managed-cloud alternative are covered in the **penetration-testing-with-strix** skill — read it if the target is not a running web app, or if `strix --version` fails. This skill is the web-app-specific workflow.
## 1. Confirm authorization and scope
Before running anything, establish:
- **The target is the user's** (or they are explicitly authorized to test it). Never pentest a third-party site on a hunch.
- **Which environment.** Prefer staging over production; agents send real exploit payloads and will create/modify data.
- **Out-of-scope paths** — payment flows, mass-email endpoints, admin destructive actions, third-party SSO providers.
- **Credentials.** Most real vulnerabilities live behind login. Without a test account, the agents only ever see the marketing surface.
Ask for anything missing rather than guessing.
## 2. Run the scan
```bash
strix -n -t https://staging.example.com --max-budget 20 \
--instruction "Test account: qa@example.com / <password>. In scope: /app/*, /api/*. Do not touch /billing or send email. Focus on access control between the two seeded orgs."
```
Notes that matter for web apps specifically:
- **Give it credentials via `--instruction`** (or `--instruction-file` for anything long), including how to log in if the flow is unusual (magic link, SSO, MFA-exempt test user).
- **Two accounts beat one.** Multi-tenant IDOR and broken-access-control bugs — consistently the highest-impact class in web apps — can only be proven when the agent can attempt cross-account access.
- **Add the repo for white-box depth** when you have the source: `-t https://github.com/org/app -t https://staging.example.com` (or a local path). Source access materially improves coverage of business-logic and authorization flaws.
- **Localhost works.** Point at `http://host.docker.internal:3000` (Docker Desktop) so the sandbox can reach a dev server on the host.
- `--scan-mode quick` for a fast dev-loop pass, `standard` (~30 min) for a normal review, `deep` for pre-release assurance. Always set `--max-budget`.
For a hosted run with no Docker/LLM key, or when the user wants a shareable dashboard and an auditor-ready PDF, use the cloud path in **managed-pentesting-with-strix** instead — same engine, same findings.
## 3. Review results
Read `strix_runs/<run>/penetration_test_report.md` first, then per-finding files in `vulnerabilities/`. Each contains the PoC — re-run it yourself to confirm before reporting to the user.
Exit codes: `0` no validated vulns in what was analyzed, `2` vulnerabilities found, `1` fatal error. A `0` is not proof of full coverage — if the budget or turn cap was hit the scan wraps up early, so check `run.json` status and cost against `--max-budget` before calling the app clean.
## 4. Fix and verify
Hand findings to the **fix-security-vulnerabilities-with-strix** skill: patch the root cause, then re-run Strix against the same target to prove the exploit no longer works. Re-testing is the only reliable confirmation a fix landed.
To keep the app tested on every change rather than once, wire Strix into CI with **ci-security-scanning-with-strix**.
+63 -10
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import dataclasses
import inspect
import json
import logging
@@ -25,6 +26,7 @@ from strix.tools.agents_graph.tools import (
view_agent_graph,
wait_for_agents,
)
from strix.tools.coverage.tools import list_coverage, record_coverage, update_coverage
from strix.tools.finish.tool import finish_scan
from strix.tools.load_skill.tool import load_skill
from strix.tools.notes.tools import (
@@ -51,6 +53,11 @@ from strix.tools.reporting.tool import (
)
from strix.tools.respond.tool import respond_to_user
from strix.tools.thinking.tool import think
from strix.tools.threat_model.tools import (
amend_threat_model,
get_threat_model,
save_threat_model,
)
from strix.tools.todo.tools import (
create_todo,
delete_todo,
@@ -222,6 +229,17 @@ def _with_coerced_arguments(tool: FunctionTool) -> FunctionTool:
return tool
def _with_strictness(tool: FunctionTool, strict_schemas: bool) -> FunctionTool:
"""Drop strict JSON-schema mode when the route can't take it (see
``supports_strict_tool_schemas``); the tool stays functionally identical.
Returns a copy so the shared tool singletons keep their declared mode.
"""
if strict_schemas or not tool.strict_json_schema:
return tool
return dataclasses.replace(tool, strict_json_schema=False)
def _function_tool_with_error_result(tool: FunctionTool) -> FunctionTool:
invoke_tool = tool.on_invoke_tool
@@ -285,24 +303,38 @@ def _bound_custom_tool(tool: CustomTool) -> CustomTool:
return tool
def _configure_filesystem_tools(toolset: Any, *, chat_completions: bool) -> None:
def _configure_filesystem_tools(
toolset: Any, *, chat_completions: bool, strict_schemas: bool = True
) -> None:
for name, tool in vars(toolset).items():
if chat_completions:
if isinstance(tool, CustomTool):
setattr(toolset, name, _custom_tool_as_function_tool(tool))
elif isinstance(tool, FunctionTool):
setattr(
toolset, name, _function_tool_with_error_result(_with_coerced_arguments(tool))
toolset,
name,
_function_tool_with_error_result(
_with_strictness(_with_coerced_arguments(tool), strict_schemas)
),
)
elif isinstance(tool, CustomTool):
setattr(toolset, name, _bound_custom_tool(tool))
elif isinstance(tool, FunctionTool):
setattr(toolset, name, _with_bounded_result(_with_coerced_arguments(tool)))
setattr(
toolset,
name,
_with_bounded_result(
_with_strictness(_with_coerced_arguments(tool), strict_schemas)
),
)
def _make_filesystem_configurator(*, chat_completions: bool) -> Any:
def _make_filesystem_configurator(*, chat_completions: bool, strict_schemas: bool) -> Any:
def configure(toolset: Any) -> None:
_configure_filesystem_tools(toolset, chat_completions=chat_completions)
_configure_filesystem_tools(
toolset, chat_completions=chat_completions, strict_schemas=strict_schemas
)
return configure
@@ -406,11 +438,13 @@ def _wrap_write_stdin(tool: FunctionTool) -> FunctionTool:
return tool
def _configure_shell_tools(toolset: Any, *, chat_completions: bool) -> None:
def _configure_shell_tools(
toolset: Any, *, chat_completions: bool, strict_schemas: bool = True
) -> None:
for name, tool in vars(toolset).items():
if not isinstance(tool, FunctionTool):
continue
wrapped = _with_coerced_arguments(tool)
wrapped = _with_strictness(_with_coerced_arguments(tool), strict_schemas)
if tool.name == "exec_command":
wrapped = _wrap_exec_command(wrapped)
elif tool.name == "write_stdin":
@@ -420,9 +454,11 @@ def _configure_shell_tools(toolset: Any, *, chat_completions: bool) -> None:
setattr(toolset, name, wrapped)
def _make_shell_configurator(*, chat_completions: bool) -> Any:
def _make_shell_configurator(*, chat_completions: bool, strict_schemas: bool) -> Any:
def configure(toolset: Any) -> None:
_configure_shell_tools(toolset, chat_completions=chat_completions)
_configure_shell_tools(
toolset, chat_completions=chat_completions, strict_schemas=strict_schemas
)
return configure
@@ -498,6 +534,12 @@ _BASE_TOOLS: tuple[Tool, ...] = (
get_note,
update_note,
delete_note,
record_coverage,
update_coverage,
list_coverage,
get_threat_model,
save_threat_model,
amend_threat_model,
web_search,
create_vulnerability_report,
create_dependency_report,
@@ -566,8 +608,10 @@ def build_strix_agent(
is_root: bool,
scan_mode: str = "deep",
is_whitebox: bool = False,
is_diff_scoped: bool = False,
interactive: bool = False,
chat_completions_tools: bool = False,
strict_tool_schemas: bool = True,
system_prompt_context: dict[str, Any] | None = None,
extra_tools: Sequence[Tool] | None = None,
instructions_override: str | None = None,
@@ -577,6 +621,8 @@ def build_strix_agent(
Args:
chat_completions_tools: Wrap SDK custom tools as function tools
when the selected backend cannot accept Responses custom tools.
strict_tool_schemas: Send function tools as strict-schema tools. Off
for routes that reject a toolset this size as strict.
extra_tools: Additional tools for this scan agent only, on top of any
registered via ``register_agent_tools``.
instructions_override: Use this verbatim as the system prompt instead
@@ -590,6 +636,7 @@ def build_strix_agent(
scan_mode=scan_mode,
is_whitebox=is_whitebox,
is_root=is_root,
is_diff_scoped=is_diff_scoped,
interactive=interactive,
system_prompt_context=system_prompt_context,
)
@@ -604,7 +651,7 @@ def build_strix_agent(
tools = [*_BASE_TOOLS, *agent_tools, agent_finish]
_ensure_unique_tool_names(tools)
tools = [
_with_bounded_result(_with_coerced_arguments(tool))
_with_bounded_result(_with_strictness(_with_coerced_arguments(tool), strict_tool_schemas))
if isinstance(tool, FunctionTool)
else tool
for tool in tools
@@ -630,11 +677,13 @@ def build_strix_agent(
Filesystem(
configure_tools=_make_filesystem_configurator(
chat_completions=chat_completions_tools,
strict_schemas=strict_tool_schemas,
),
),
Shell(
configure_tools=_make_shell_configurator(
chat_completions=chat_completions_tools,
strict_schemas=strict_tool_schemas,
),
),
],
@@ -645,8 +694,10 @@ def make_child_factory(
*,
scan_mode: str = "deep",
is_whitebox: bool = False,
is_diff_scoped: bool = False,
interactive: bool = False,
chat_completions_tools: bool = False,
strict_tool_schemas: bool = True,
system_prompt_context: dict[str, Any] | None = None,
) -> Any:
"""Return the runner-owned builder used by ``spawn_child_agent``.
@@ -663,8 +714,10 @@ def make_child_factory(
is_root=False,
scan_mode=scan_mode,
is_whitebox=is_whitebox,
is_diff_scoped=is_diff_scoped,
interactive=interactive,
chat_completions_tools=chat_completions_tools,
strict_tool_schemas=strict_tool_schemas,
system_prompt_context=system_prompt_context,
)
+19 -3
View File
@@ -23,30 +23,44 @@ def _resolve_skills(
scan_mode: str = "deep",
is_whitebox: bool = False,
is_root: bool = False,
is_diff_scoped: bool = False,
) -> list[str]:
"""Build the deduped, ordered skills list for the prompt render.
Order:
1. Whatever the caller asked for, in order.
2. ``scan_modes/<mode>`` (always).
2. ``scan_modes/<mode>`` (always), plus ``scan_modes/diff`` when the
run is scoped to a change set — diff scope overlays the depth
mode rather than replacing it.
3. ``tooling/agent_browser`` (always — every agent has shell + the
agent-browser CLI).
4. ``tooling/python`` (always — Python runs through ``exec_command``;
sandbox scripts can import ``caido_api`` for Caido automation).
5. ``coordination/root_agent`` for the root agent only — orchestration
5. ``analysis/counterevidence`` and ``analysis/severity_calibration``
(always — closure discipline and severity rubric apply to every
agent that can open or close a candidate, or file a report).
6. ``coordination/root_agent`` for the root agent only — orchestration
guidance for delegating to specialist subagents.
6. Whitebox-specific skills if applicable.
7. Whitebox-specific skills if applicable, including
``analysis/fix_verification`` (only whitebox agents can attach an
applyable ``fix_after``) and ``analysis/source_aware_discovery``.
"""
ordered: list[str] = list(requested or [])
ordered.append(f"scan_modes/{scan_mode}")
if is_diff_scoped:
ordered.append("scan_modes/diff")
ordered.append("tooling/agent_browser")
ordered.append("tooling/python")
ordered.append("analysis/counterevidence")
ordered.append("analysis/severity_calibration")
if is_root:
ordered.append("coordination/root_agent")
if is_whitebox:
ordered.append("coordination/source_aware_whitebox")
ordered.append("custom/source_aware_sast")
ordered.append("analysis/source_aware_discovery")
ordered.append("analysis/fix_verification")
deduped: list[str] = []
seen: set[str] = set()
@@ -63,6 +77,7 @@ def render_system_prompt(
scan_mode: str = "deep",
is_whitebox: bool = False,
is_root: bool = False,
is_diff_scoped: bool = False,
interactive: bool = False,
system_prompt_context: dict[str, Any] | None = None,
) -> str:
@@ -83,6 +98,7 @@ def render_system_prompt(
scan_mode=scan_mode,
is_whitebox=is_whitebox,
is_root=is_root,
is_diff_scoped=is_diff_scoped,
)
skill_content = load_skills(skills_to_load)
env.globals["get_skill"] = lambda name: skill_content.get(name, "")
+21
View File
@@ -216,10 +216,31 @@ VALIDATION REQUIREMENTS:
- Independent verification through subagent
- Document complete attack chain
- Keep going until you find something that matters
- CLOSURE DISCIPLINE: every candidate you open ends in exactly one explicit state — `confirmed` (working PoC, or a complete source→control→sink→impact trace that is reachable), `ruled_out` (you can name the SPECIFIC control, at a location, that runs on every attacker-reachable path before the sink), or `open_proof_gap` (plausible, unconfirmed, and you could NOT name such a control). "I moved on" is not a closure state. Silently dropping an uncertain candidate is mislabelling an `open_proof_gap` as `ruled_out` and is how real bugs get missed.
- Missing information is NOT proof of safety: no caller found, can't tell if deployed/exposed, couldn't stand up the service, build failed — each is an `open_proof_gap`, never a reason to mark a candidate clean. Difficulty is a reason to defer, not to suppress.
- COVERAGE: record every surface you assess with `record_coverage` (surface + risk area + outcome + evidence), including the ones that came back clean — a report that only lists findings cannot say what was reviewed and cleared. Use the `needs_follow_up` outcome for anything left in an `open_proof_gap` state, and carry the same items up in `agent_finish(open_items=[...])`. The ledger is shared and mutable: when you resolve a surface another agent left open — or find that a closed one is not — move that entry with `update_coverage` instead of recording a second one for the same surface. The root agent reconciles all of it via `list_coverage` before `finish_scan`.
- THREAT MODEL: before you start testing, call `get_threat_model` on the target you were pointed at — it is the scan's shared answer to who the attacker is, where the trust boundaries sit, and what counts as critical here, and it is cached per target rather than per scan. Read it instead of re-deriving trust boundaries yourself; where your testing disproves it — a boundary it calls trusted turns out to be attacker-reachable, a role it did not know about, a host or endpoint it never listed — record that with `amend_threat_model` so the agents after you inherit the correction. Amending is not optional politeness: a model nobody corrects turns the first agent's guesses into everyone's assumptions.
- Before filing any report, run the counterevidence pass: argue the strongest case AGAINST the finding, record what you found in the `counterevidence` field, set `confidence` honestly (a static-only trace you couldn't execute is at best `medium`), and state what evidence would change the severity. See the counterevidence and severity-calibration knowledge above.
- A vulnerability is ONLY considered reported when a reporting agent uses create_vulnerability_report (or create_dependency_report for known-CVE dependency/supply-chain findings) with full details. Mentions in agent_finish, finish_scan, or generic messages are NOT sufficient
- Reporting and fixing are ONE step, not two: when source is available, the reporting agent derives the concrete fix and files it INLINE via create_vulnerability_report (`code_locations` with `fix_before`/`fix_after` + `fix_pr_body`) — the report is not complete without it. Do NOT report first and then spawn a separate downstream agent to re-derive and re-apply the same patch; that just re-does the analysis and wastes tokens. (Do not silently patch a finding WITHOUT filing a report — the report, with its embedded fix, is the deliverable.)
- DEDUPLICATION: The create_vulnerability_report tool uses LLM-based deduplication. If it rejects your report as a duplicate, DO NOT attempt to re-submit the same vulnerability. Accept the rejection and move on to testing other areas. The vulnerability has already been reported by another agent
- REVIEWING FILED FINDINGS (orchestrator/root agent): use list_reports to see every vulnerability filed so far in this scan (by any agent, root or child) — metadata-first with per-severity counts — and get_report to read one finding in full by its id. These are read-only orchestration tools: the root agent uses them to track coverage, avoid dispatching work on already-covered ground, assemble the finish_scan executive summary, and reason about attack-chaining across confirmed findings. Leaf/specialist agents should NOT call them — just do your assigned testing and file findings. Each entry shows which agent filed it (agent_name), and your own entries are flagged by_you. list_notes/get_note do the same for notes.
STATE & COORDINATION TOOLS (when and how):
Every one of these tools writes to state the rest of the scan reads. Reaching for the tool is not optional bookkeeping — the agent after you sees your state, not your reasoning, so state you never wrote is context the scan permanently loses.
- PLAN — `think`: use before any non-trivial or multi-step move to reason through approach, uncertainty, or what to do next. NOT for acknowledgements, summaries, or as filler before a final answer.
- SKILLS — `load_skill`: the skills matching your task are already inlined below under `<specialized_knowledge>`; `<available_skills>` lists the rest by name. When you are about to test a vuln class, protocol, tool, or framework whose skill is not already inlined, `load_skill` it FIRST and follow it, rather than guessing payloads or tool syntax from memory.
- TODOS — `create_todo` / `list_todos` / `update_todo` / `mark_todo_done` / `mark_todo_pending` / `delete_todo`: your own working checklist for a multi-step task. Create todos when your task has several distinct steps so nothing is dropped across a long run; mark them done as you finish. This is private working memory — use `notes` for anything another agent needs.
- NOTES — `create_note` / `list_notes` / `get_note` / `update_note` / `delete_note`: the scan's shared scratchpad, visible to every agent. Write a note for a durable cross-agent fact that is not a finding and not coverage — a working credential set, a discovered endpoint inventory, an enumerated tenant list, a rate-limit quirk the next agent needs. `update_note` to keep a living inventory current; `delete_note` only for something now wrong or superseded. Check `list_notes`/`get_note` before recon work so you build on what is already mapped instead of redoing it.
- THREAT MODEL — `get_threat_model` / `amend_threat_model` / `save_threat_model`: covered above. `save_threat_model` REPLACES the whole document and clears amendments, so it is for establishing the baseline or folding amendments in (normally root) — to correct part of an existing model, `amend_threat_model` instead.
- COVERAGE — `record_coverage` / `update_coverage` / `list_coverage`: covered above. One row per surface+risk; correct an existing row with `update_coverage`, never a second `record_coverage`.
- RESEARCH — `web_search`: pull fresh, target-specific external knowledge — latest bypasses, WAF evasions, DB-/framework-specific syntax, CVE and advisory detail — before falling back to memorized payloads, and refresh payload corpora mid-spray.
- SPAWN WORK — `create_agent`: delegate a focused subtask to a specialist child (see the multi-agent rules below for when to spawn and how to scope it). Give it the target to model against and what is already known.
- TRACK CHILDREN — `view_agent_graph`: your live map of every agent and its status. Call it before spawning (to confirm no existing agent already covers the scope) and before finishing (to confirm no child is still running).
- STEER CHILDREN — `send_message_to_agent`: send a running child new information, a course correction, or a request to wrap up, without killing it. Use it to answer a child's question or narrow its scope mid-run.
- BLOCK ON CHILDREN — `wait_for_agents`: block until named children report back when your next move genuinely depends on their results. If you can keep making progress in parallel, keep working instead of waiting.
- CANCEL CHILDREN — `stop_agent`: gracefully cancel a child whose work is redundant, misdirected, or no longer needed. Prefer `send_message_to_agent` to redirect a child that is merely off-track; reserve `stop_agent` for work that should not continue at all.
- FINISH — subagents call `agent_finish` (with `open_items=[...]` for anything left unresolved); the root agent calls `finish_scan` exactly once, only after every child is wrapped up and coverage is reconciled. `agent_finish`/`finish_scan` are handoffs, not reporting channels — a vulnerability is reported only via `create_vulnerability_report`/`create_dependency_report`.
</execution_guidelines>
<vulnerability_focus>
+15
View File
@@ -749,6 +749,18 @@ def uses_chat_completions_tool_schema(model_name: str, settings: Settings) -> bo
return not model_supports_reasoning(model_name)
def supports_strict_tool_schemas(model_name: str) -> bool:
"""Return whether the route accepts strict tool schemas for Strix's toolset.
Claude caps a request at 20 strict tools and 16 union-typed parameters
across all strict schemas. Strix ships ~30 tools and the strict dialect
turns every optional parameter into a nullable union, so both caps are
exceeded and the request is rejected outright.
"""
name = model_name.strip().lower()
return not any(marker in name for marker in _ANTHROPIC_MODEL_MARKERS)
def model_supports_reasoning(model_name: str) -> bool:
import litellm
@@ -845,6 +857,9 @@ def is_known_openai_bare_model(model_name: str) -> bool:
return bool(entry and entry.get("litellm_provider") == "openai")
_ANTHROPIC_MODEL_MARKERS = ("anthropic", "claude", "sonnet", "opus", "haiku")
def is_claude_model(model_name: str) -> bool:
return "claude" in (model_name or "").strip().lower()
+17 -3
View File
@@ -7,13 +7,12 @@ import contextlib
import logging
import uuid
from collections.abc import Callable
from functools import cache
from typing import TYPE_CHECKING, Any, cast
import litellm
from agents import RunConfig, Runner
from agents.exceptions import AgentsException, MaxTurnsExceeded, UserError
from agents.sandbox.errors import ExecTransportError
from docker import errors as docker_errors # type: ignore[import-untyped, unused-ignore]
from openai import (
APIConnectionError,
APIError,
@@ -56,6 +55,19 @@ _INPUT_REJECTION_CODES = frozenset({400, 404, 422})
_MAX_COMPACTIONS_PER_CYCLE = 2
@cache
def _teardown_sandbox_errors() -> tuple[type[BaseException], ...]:
"""Sandbox-gone errors, tolerated during shutdown.
The Docker SDK is imported here rather than at module scope: it is only
reachable with the Docker runtime backend, and importing it eagerly puts it
on every launch's critical path.
"""
from docker import errors as docker_errors # type: ignore[import-untyped, unused-ignore]
return (ExecTransportError, docker_errors.NotFound)
class ProviderRefusalError(AgentsException):
"""Raised when a provider returns a structured refusal instead of an exception."""
@@ -126,6 +138,8 @@ def _is_transient_model_error(exc: BaseException) -> bool:
return True
code = _model_error_status_code(exc)
if code is not None:
import litellm
return bool(litellm._should_retry(code))
return isinstance(exc, APIError)
@@ -692,7 +706,7 @@ async def _run_cycle( # noqa: PLR0912, PLR0915
"Ignoring LiteLLM end-of-stream shutdown race for %s",
agent_id,
)
except (ExecTransportError, docker_errors.NotFound):
except _teardown_sandbox_errors():
if not coordinator.is_shutting_down:
raise
logger.warning(
+17
View File
@@ -226,6 +226,23 @@ def build_scope_context(scan_config: dict[str, Any]) -> dict[str, Any]:
}
def build_scan_targets(scan_config: dict[str, Any]) -> list[str]:
"""One canonical string per authorized target.
Agents refer to the target in whatever words they were handed, so anything
keyed on a target the model types drifts apart across a run. This is the
scan's own spelling, which target-keyed tools resolve against. A checkout is
named by its workspace path rather than its remote URL, so the local tree —
and its revision — is what gets inspected.
"""
targets: list[str] = []
for target in build_scope_context(scan_config)["authorized_targets"]:
value = target["workspace_path"] or target["value"]
if value and value not in targets:
targets.append(value)
return targets
def make_model_settings(
reasoning_effort: ReasoningEffort | None,
*,
+86
View File
@@ -22,6 +22,7 @@ from strix.config import load_settings
from strix.config.models import (
StrixProvider,
configure_sdk_model_defaults,
supports_strict_tool_schemas,
uses_chat_completions_tool_schema,
)
from strix.config.settings import DEFAULT_MAX_TURNS
@@ -36,6 +37,7 @@ from strix.core.execution import (
from strix.core.hooks import BudgetExceededError, ReportUsageHooks, recomputed_budget_flags
from strix.core.inputs import (
build_root_task,
build_scan_targets,
build_scope_context,
make_model_settings,
)
@@ -51,10 +53,12 @@ from strix.tools.output_store import (
if TYPE_CHECKING:
from agents.mcp import MCPServer
from agents.memory import SQLiteSession
from agents.result import RunResultBase
from strix.runtime.status import StatusSink
from strix.tools.mcp import ConnectedMcpServer
logger = logging.getLogger(__name__)
@@ -62,6 +66,48 @@ logger = logging.getLogger(__name__)
StreamEventSink = Callable[[str, Any], None]
def _mcp_startup_summary(connections: list[ConnectedMcpServer]) -> str:
"""One user-facing line summarizing the MCP servers that connected."""
server_count = len(connections)
tool_count = sum(c.tool_count for c in connections)
servers_word = "server" if server_count == 1 else "servers"
tools_word = "tool" if tool_count == 1 else "tools"
names = ", ".join(c.name for c in connections)
return f"MCP: connected {server_count} {servers_word} ({tool_count} {tools_word}): {names}"
def _record_mcp_connections(connections: list[ConnectedMcpServer]) -> None:
"""Record which MCP servers this run connected, for the interfaces.
A server's tools are offered to the model under a name built from the
connection name and the tool's own name, which cannot be split back apart, so
the TUI and the run viewer need the names to match a tool call against before
they can show which server it went out to. Kept on the run record because the
viewer reads a finished run from disk.
"""
report_state = get_global_report_state()
if report_state is None:
return
report_state.record_mcp_connections([connection.name for connection in connections])
def _mcp_connection_notes(connections: list[ConnectedMcpServer]) -> str | None:
"""A block describing the connections the user left notes on, for the agent.
Only connections with notes are listed, so the note describes the connection
once rather than being repeated onto every tool. Returns ``None`` when no
connection has notes.
"""
noted = [(c.name, c.notes) for c in connections if c.notes]
if not noted:
return None
lines = "\n".join(f"- `{name}.*` tools: {notes}" for name, notes in noted)
return (
"The user connected these MCP servers for this run and left notes on how "
f"to use each:\n{lines}"
)
def _merge_root_prompt_context(
scope_context: dict[str, Any],
extra_system_prompt_context: dict[str, Any] | None,
@@ -83,6 +129,7 @@ def _compose_root_instructions_override(
skills: list[str],
scan_mode: str,
is_whitebox: bool,
is_diff_scoped: bool,
interactive: bool,
system_prompt_context: dict[str, Any],
) -> str | None:
@@ -94,6 +141,7 @@ def _compose_root_instructions_override(
scan_mode=scan_mode,
is_whitebox=is_whitebox,
is_root=True,
is_diff_scoped=is_diff_scoped,
interactive=interactive,
system_prompt_context=system_prompt_context,
)
@@ -175,16 +223,21 @@ async def run_strix_scan(
)
logger.info("LLM model resolved: %s", resolved_model)
chat_completions_tools = uses_chat_completions_tool_schema(resolved_model, settings)
strict_tool_schemas = supports_strict_tool_schemas(resolved_model)
if not strict_tool_schemas:
logger.info("Sending non-strict tool schemas: %s caps strict tools", resolved_model)
if coordinator is None:
coordinator = AgentCoordinator()
coordinator.set_snapshot_path(agents_path)
from strix.tools.coverage.tools import hydrate_coverage_from_disk
from strix.tools.notes.tools import hydrate_notes_from_disk
from strix.tools.todo.tools import hydrate_todos_from_disk
hydrate_todos_from_disk(state_dir)
hydrate_notes_from_disk(state_dir)
hydrate_coverage_from_disk(state_dir)
root_id: str | None = None
if is_resume:
@@ -253,11 +306,14 @@ async def run_strix_scan(
configure_spill_writer(_spill_to_workspace)
sessions_to_close: list[SQLiteSession] = []
mcp_servers: list[MCPServer] = []
try:
targets = scan_config.get("targets") or []
scan_mode = str(scan_config.get("scan_mode") or "deep")
is_whitebox = any(t.get("type") == "local_code" for t in targets)
diff_scope = scan_config.get("diff_scope")
is_diff_scoped = bool(isinstance(diff_scope, dict) and diff_scope.get("active"))
skills = list(scan_config.get("skills") or [])
root_task = build_root_task(scan_config)
model_settings = make_model_settings(
@@ -294,18 +350,42 @@ async def run_strix_scan(
skills=skills,
scan_mode=scan_mode,
is_whitebox=is_whitebox,
is_diff_scoped=is_diff_scoped,
interactive=interactive,
system_prompt_context=root_context,
)
# Connect any MCP servers the user listed in ~/.strix/mcp-servers.json and
# register their tools before the agent is built. Fail-open: a missing
# config, or a server that will not connect, must never break a run.
from strix.tools.mcp import connect_mcp_servers, load_user_mcp_configs
try:
user_mcp_configs = load_user_mcp_configs()
if user_mcp_configs:
connections = await connect_mcp_servers(user_mcp_configs)
mcp_servers = [c.server for c in connections]
# Recorded even when nothing connected, so a resumed run does not
# keep attributing tool calls to servers it no longer has.
_record_mcp_connections(connections)
if connections:
report(_mcp_startup_summary(connections))
notes_block = _mcp_connection_notes(connections)
if notes_block:
root_task = f"{root_task}\n\n{notes_block}"
except Exception:
logger.exception("Failed to connect user MCP servers; continuing without them")
root_agent = build_strix_agent(
name="Root Agent",
skills=skills,
is_root=True,
scan_mode=scan_mode,
is_whitebox=is_whitebox,
is_diff_scoped=is_diff_scoped,
interactive=interactive,
chat_completions_tools=chat_completions_tools,
strict_tool_schemas=strict_tool_schemas,
system_prompt_context=root_context,
instructions_override=root_instructions,
)
@@ -322,8 +402,10 @@ async def run_strix_scan(
child_agent_builder = make_child_factory(
scan_mode=scan_mode,
is_whitebox=is_whitebox,
is_diff_scoped=is_diff_scoped,
interactive=interactive,
chat_completions_tools=chat_completions_tools,
strict_tool_schemas=strict_tool_schemas,
system_prompt_context=scope_context,
)
@@ -349,6 +431,7 @@ async def run_strix_scan(
"parent_id": None,
"interactive": interactive,
"spawn_child_agent": spawn_child_agent,
"scan_targets": build_scan_targets(scan_config),
"max_context_images": settings.runtime.max_context_images,
}
@@ -472,6 +555,9 @@ async def run_strix_scan(
for s in sessions_to_close:
with contextlib.suppress(Exception):
s.close()
for mcp_server in mcp_servers:
with contextlib.suppress(Exception):
await mcp_server.cleanup() # type: ignore[no-untyped-call]
with contextlib.suppress(Exception):
await coordinator._maybe_snapshot()
if cleanup_on_exit:
+40 -1
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import argparse
import os
import sys
from pathlib import Path
@@ -219,6 +220,30 @@ Examples:
help="Path to a custom config file (JSON) to use instead of ~/.strix/cli-config.json",
)
parser.add_argument(
"--mcp-config",
type=str,
metavar="PATH",
help="Path to an MCP servers JSON file to use instead of ~/.strix/mcp-servers.json.",
)
parser.add_argument(
"--mcp-server",
dest="mcp_server",
action="append",
metavar="NAME",
help="Use only this MCP connection for the run, by its config name "
"(repeatable). Every other configured connection is skipped.",
)
parser.add_argument(
"--mcp-exclude",
dest="mcp_exclude",
action="append",
metavar="NAME",
help="Skip this MCP connection for the run, by its config name (repeatable).",
)
parser.add_argument(
"--max-budget",
"--max-budget-usd",
@@ -267,6 +292,20 @@ Examples:
if args.config:
apply_config_override(validate_config_file(args.config))
if args.mcp_config:
mcp_config_path = Path(args.mcp_config).expanduser()
if not mcp_config_path.is_file():
parser.error(f"--mcp-config file not found: {args.mcp_config}")
# The MCP loader reads this env var as its config-path override, so
# setting it here makes the flag win over the default location.
os.environ["STRIX_MCP_CONFIG"] = str(mcp_config_path)
# The MCP loader reads these as its per-run include/exclude selection.
if args.mcp_server:
os.environ["STRIX_MCP_ONLY"] = ",".join(args.mcp_server)
if args.mcp_exclude:
os.environ["STRIX_MCP_EXCLUDE"] = ",".join(args.mcp_exclude)
if args.update:
sys.exit(0 if self_update() else 1)
@@ -346,7 +385,7 @@ def _load_resume_state(args: argparse.Namespace, parser: argparse.ArgumentParser
)
try:
state = read_run_record(run_dir)
except RuntimeError as exc:
except (RuntimeError, TypeError) as exc:
parser.error(f"--resume {args.resume}: run.json unreadable: {exc}")
args.targets_info = state.get("targets_info") or []
+4
View File
@@ -431,6 +431,10 @@ def main() -> None:
sys.exit(run_auth(sys.argv[2:]))
from strix.llm.warmup import start_import_warmup
start_import_warmup()
args = parse_arguments()
start_background_check()
+4 -2
View File
@@ -146,7 +146,9 @@ def bounded_state_projection(state: dict[str, Any]) -> dict[str, Any]:
}
for message in state["messages"][-5:]
]
state["usage"] = {}
state["usage"] = {
key: state["usage"][key] for key in ("total_tokens", "cost") if key in state["usage"]
}
state["error"] = terminal_projection(state["error"], max_string=512)
state["model_warning"] = terminal_projection(state["model_warning"], max_string=256)
state["caido_url"] = terminal_projection(state["caido_url"], max_string=256)
@@ -173,7 +175,7 @@ def bounded_state_projection(state: dict[str, Any]) -> dict[str, Any]:
"model_warning": "",
"caido_url": None,
"messages": [],
"usage": {},
"usage": state["usage"],
"subscription": state["subscription"],
"viewer_status": state["viewer_status"],
"viewer_url": None,
@@ -100,7 +100,7 @@ func applyMarkdownStyles(text string) string {
case strings.HasPrefix(line, "- "), strings.HasPrefix(line, "* "):
out.WriteString(Col(Green).Render("• ") + inlineFormat(line[2:]))
case len(line) > 2 && line[0] >= '0' && line[0] <= '9' && (line[1:3] == ". " || line[1:3] == ") "):
out.WriteString(Col(Green).Render(string(line[0])+". ") + inlineFormat(line[2:]))
out.WriteString(Col(Green).Render(line[:2]+" ") + inlineFormat(line[3:]))
case line == "---" || line == "***" || line == "___":
out.WriteString(Col(Green).Render(strings.Repeat("─", 40)))
default:
@@ -0,0 +1,194 @@
package render
import (
"strconv"
"strings"
"github.com/charmbracelet/lipgloss"
)
// ---------------------------------------------------------------------------
// Coverage ledger (record_coverage / update_coverage / list_coverage)
// ---------------------------------------------------------------------------
// coverageOutcomes maps a ledger outcome to its marker and color. A cleared
// surface and an unresolved one must not look alike at a glance: the whole
// point of the ledger is that a reader can see which surfaces are still open.
var coverageOutcomes = map[string]struct {
marker string
label string
color lipgloss.Color
}{
"reported": {"!", "reported", SevHigh},
"no_issue_found": {"✓", "no issue found", Green},
"ruled_out": {"✓", "ruled out", Mint},
"not_applicable": {"", "not applicable", Slate},
"needs_follow_up": {"?", "needs follow-up", AmberY},
}
func coverageOutcome(outcome string) (string, string, lipgloss.Color) {
if meta, ok := coverageOutcomes[strings.TrimSpace(strings.ToLower(outcome))]; ok {
return meta.marker, meta.label, meta.color
}
if outcome == "" {
return "·", "", Gray
}
return "·", strings.ReplaceAll(outcome, "_", " "), Gray
}
var coverageTitles = map[string]struct {
title string
loading string
errMsg string
}{
"record_coverage": {"Coverage Recorded", "Recording...", "Failed to record coverage"},
"update_coverage": {"Coverage Updated", "Updating...", "Failed to update coverage"},
"list_coverage": {"Coverage", "Loading...", "Unable to list coverage"},
}
func renderCoverage(name string, args map[string]any, result any) string {
meta := coverageTitles[name]
var b strings.Builder
b.WriteString("▣ " + Bold(Cyan).Render(meta.title))
if s, ok := result.(string); ok && strings.TrimSpace(s) != "" {
b.WriteString("\n " + Dim().Render(strings.TrimSpace(s)))
return b.String()
}
m, ok := result.(map[string]any)
if !ok {
coverageArgsPreview(&b, name, args)
b.WriteString("\n " + Dim().Render(meta.loading))
return b.String()
}
if !truthy(m["success"]) {
coverageArgsPreview(&b, name, args)
errMsg := StringValue(m["error"])
if errMsg == "" {
errMsg = meta.errMsg
}
b.WriteString("\n " + Col(Red).Render(errMsg))
return b.String()
}
switch name {
case "list_coverage":
coverageListBody(&b, m)
case "update_coverage":
marker, label, color := coverageOutcome(StringValue(m["outcome"]))
_, previous, previousColor := coverageOutcome(StringValue(m["previous_outcome"]))
b.WriteString("\n " + Col(color).Render(marker) + " " + coverageSubject(args, m))
if previous != "" {
b.WriteString("\n " + Col(previousColor).Render(previous) +
Dim().Render(" → ") + Col(color).Render(label))
} else {
b.WriteString("\n " + Col(color).Render(label))
}
coverageEvidence(&b, StringValue(args["evidence"]))
default:
marker, label, color := coverageOutcome(StringValue(m["outcome"]))
b.WriteString("\n " + Col(color).Render(marker) + " " + coverageSubject(args, m))
b.WriteString("\n " + Col(color).Render(label))
coverageEvidence(&b, StringValue(args["evidence"]))
}
return b.String()
}
// coverageSubject names the surface being recorded, falling back to the entry
// id when only the id is known (an update carries no surface in its args).
func coverageSubject(args map[string]any, result map[string]any) string {
surface := strings.TrimSpace(StringValue(args["surface"]))
risk := strings.TrimSpace(StringValue(args["risk_area"]))
switch {
case surface != "" && risk != "":
return surface + Dim().Render(" · "+risk)
case surface != "":
return surface
case risk != "":
return risk
}
if id := StringValue(result["entry_id"]); id != "" {
return Dim().Render("entry " + id)
}
return Dim().Render("(unnamed surface)")
}
func coverageEvidence(b *strings.Builder, evidence string) {
if strings.TrimSpace(evidence) != "" {
b.WriteString("\n " + Dim().Render(psanitize(strings.TrimSpace(evidence), 160)))
}
}
func coverageArgsPreview(b *strings.Builder, name string, args map[string]any) {
if name == "list_coverage" {
return
}
if subject := coverageSubject(args, map[string]any{}); subject != "" {
b.WriteString("\n " + subject)
}
}
func coverageListBody(b *strings.Builder, result map[string]any) {
entries, _ := result["entries"].([]any)
total, _ := NumericValue(result["total_count"])
if len(entries) == 0 {
if int(total) == 0 {
b.WriteString("\n " + Dim().Render("No surfaces recorded yet"))
} else {
b.WriteString("\n " + Dim().Render("No surfaces match this filter"))
}
return
}
if counts, ok := result["outcome_counts"].(map[string]any); ok && len(counts) > 0 {
var parts []string
for _, outcome := range []string{
"reported", "no_issue_found", "ruled_out", "not_applicable", "needs_follow_up",
} {
count, ok := NumericValue(counts[outcome])
if !ok || count == 0 {
continue
}
_, label, color := coverageOutcome(outcome)
parts = append(parts, Col(color).Render(label+": "+strconv.Itoa(int(count))))
}
if len(parts) > 0 {
b.WriteString("\n " + strings.Join(parts, Dim().Render(" ")))
}
}
for _, e := range entries {
entry, _ := e.(map[string]any)
marker, label, color := coverageOutcome(StringValue(entry["outcome"]))
surface := strings.TrimSpace(StringValue(entry["surface"]))
if surface == "" {
surface = "(unnamed surface)"
}
b.WriteString("\n " + Col(color).Render(marker) + " " + surface)
if risk := strings.TrimSpace(StringValue(entry["risk_area"])); risk != "" {
b.WriteString(Dim().Render(" · " + risk))
}
b.WriteString("\n " + Col(color).Render(label))
// A row that moved states carries its own history; showing it keeps a
// closed surface from reading as one that was never in question.
if previous, ok := entry["previous_outcomes"].([]any); ok && len(previous) > 0 {
var was []string
for _, p := range previous {
if _, label, _ := coverageOutcome(StringValue(p)); label != "" {
was = append(was, label)
}
}
if len(was) > 0 {
b.WriteString(Dim().Render(" (was " + strings.Join(was, " → ") + ")"))
}
}
// Whose row this is matters for reconciliation: an agent needs to see
// at a glance which surfaces it owns and which came from a sibling.
if truthy(entry["by_you"]) {
b.WriteString(Dim().Render(" · you"))
} else if who := strings.TrimSpace(StringValue(entry["agent_name"])); who != "" {
b.WriteString(Dim().Render(" · " + who))
}
coverageEvidence(b, StringValue(entry["evidence"]))
}
}
@@ -0,0 +1,204 @@
package render
import (
"strings"
"testing"
"github.com/charmbracelet/x/ansi"
)
func TestRecordCoverageRendersSurfaceAndOutcome(t *testing.T) {
out := ansi.Strip(Tool(tool("record_coverage",
map[string]any{
"surface": "POST /api/v1/invoices",
"risk_area": "object-level authorization",
"evidence": "tenant B token returns 403 on tenant A invoice ids",
},
map[string]any{"success": true, "entry_id": "a1b2c3", "outcome": "ruled_out"},
"completed")))
requireContains(t, out,
"Coverage Recorded",
"POST /api/v1/invoices",
"object-level authorization",
"ruled out",
"tenant B token returns 403",
)
}
func TestUpdateCoverageShowsStateTransition(t *testing.T) {
out := ansi.Strip(Tool(tool("update_coverage",
map[string]any{"entry_id": "a1b2c3", "evidence": "reproduced with a second tenant"},
map[string]any{
"success": true,
"entry_id": "a1b2c3",
"previous_outcome": "needs_follow_up",
"outcome": "reported",
},
"completed")))
requireContains(t, out, "Coverage Updated", "needs follow-up", "→", "reported")
}
func TestListCoverageRendersCountsHistoryAndAuthor(t *testing.T) {
out := ansi.Strip(Tool(tool("list_coverage", nil,
map[string]any{
"success": true,
"entries": []any{
map[string]any{
"entry_id": "a1b2c3",
"surface": "/admin/export",
"risk_area": "IDOR",
"outcome": "no_issue_found",
"agent_name": "AuthzAgent",
"previous_outcomes": []any{"needs_follow_up"},
"evidence": "org id is server-derived from the session",
},
map[string]any{
"entry_id": "d4e5f6",
"surface": "/graphql",
"risk_area": "injection",
"outcome": "needs_follow_up",
"by_you": true,
"evidence": "introspection disabled; needs an authenticated schema dump",
},
},
"total_count": 2,
"outcome_counts": map[string]any{"no_issue_found": 1, "needs_follow_up": 1},
},
"completed")))
requireContains(t, out,
"/admin/export", "IDOR", "no issue found",
"was needs follow-up", "AuthzAgent",
"/graphql", "needs follow-up", "you",
"no issue found: 1", "needs follow-up: 1",
)
}
func TestListCoverageEmptyLedgerReadsAsUnrecorded(t *testing.T) {
out := ansi.Strip(Tool(tool("list_coverage", nil,
map[string]any{"success": true, "entries": []any{}, "total_count": 0}, "completed")))
requireContains(t, out, "No surfaces recorded yet")
filtered := ansi.Strip(Tool(tool("list_coverage",
map[string]any{"outcome": "reported"},
map[string]any{"success": true, "entries": []any{}, "total_count": 4}, "completed")))
requireContains(t, filtered, "No surfaces match this filter")
}
func TestCoverageDuplicateRejectionSurfacesTheError(t *testing.T) {
out := ansi.Strip(Tool(tool("record_coverage",
map[string]any{"surface": "/login", "risk_area": "XSS"},
map[string]any{
"success": false,
"error": "'/login' (XSS) already has coverage entry a1b2c3",
"existing_entry_id": "a1b2c3",
},
"completed")))
requireContains(t, out, "/login", "already has coverage entry a1b2c3")
}
func TestGetThreatModelRendersStalenessAndAmendments(t *testing.T) {
out := ansi.Strip(Tool(tool("get_threat_model",
map[string]any{"target": "https://app.example.com"},
map[string]any{
"success": true,
"found": true,
"stale": true,
"cached_revision": "0123456789abcdef",
"content": "# Overview\nMulti-tenant billing app.\n\n" +
"## Trust Boundaries and Assumptions\n\n## Attack Surface\n",
"amendments": []any{
map[string]any{
"agent_name": "ReconAgent",
"content": "staging host shares the production database",
},
},
},
"completed")))
requireContains(t, out,
"Threat Model", "https://app.example.com",
"stale", "01234567",
"1 amendment(s)", "ReconAgent", "staging host shares the production database",
"Multi-tenant billing app.", "Overview", "Trust Boundaries and Assumptions",
)
}
func TestGetThreatModelMissingModelIsExplicit(t *testing.T) {
out := ansi.Strip(Tool(tool("get_threat_model",
map[string]any{"target": "10.0.0.5"},
map[string]any{"success": true, "found": false}, "completed")))
requireContains(t, out, "No model cached for this target yet")
}
func TestSaveThreatModelWarnsWhenAmendmentsAreCleared(t *testing.T) {
out := ansi.Strip(Tool(tool("save_threat_model",
map[string]any{"target": "app.example.com", "content": "# Overview\nA thing.\n"},
map[string]any{
"success": true,
"revision": "unversioned",
"amendments_cleared": 2,
},
"completed")))
requireContains(t, out, "Threat Model Saved", "saved", "cleared 2 amendment(s)")
// An unversioned target has no revision worth printing.
if strings.Contains(out, "unversioned") {
t.Fatalf("unversioned revision should not be rendered:\n%s", out)
}
}
func TestAmendThreatModelRendersAddendum(t *testing.T) {
out := ansi.Strip(Tool(tool("amend_threat_model",
map[string]any{
"target": "app.example.com",
"addendum": "The admin role is assignable by any org member via PATCH /members.",
},
map[string]any{"success": true, "amendment_count": 3}, "completed")))
requireContains(t, out, "Threat Model Amended", "amendment recorded", "(3 total)",
"admin role is assignable")
}
func TestCoverageAndThreatModelToolsAreNotGeneric(t *testing.T) {
// The generic fallback dumps raw arg keys; these tools must not reach it.
for _, name := range []string{
"record_coverage", "update_coverage", "list_coverage",
"get_threat_model", "save_threat_model", "amend_threat_model",
} {
out := ansi.Strip(Tool(tool(name, map[string]any{"target": "x", "surface": "y"}, nil, "running")))
if strings.Contains(out, "Using tool") {
t.Fatalf("%s fell through to the generic renderer:\n%s", name, out)
}
}
}
func TestOutputHeavyCoverageToolsCollapse(t *testing.T) {
for _, name := range []string{"list_coverage", "get_threat_model"} {
if ToolPreviewLines(name) == 0 {
t.Fatalf("%s should collapse; its output is unbounded", name)
}
}
for _, name := range []string{"record_coverage", "amend_threat_model"} {
if ToolPreviewLines(name) != 0 {
t.Fatalf("%s should not collapse", name)
}
}
}
func TestVulnerabilityReportRendersCalibrationFields(t *testing.T) {
out := ansi.Strip(Tool(tool("create_vulnerability_report",
map[string]any{
"title": "IDOR in invoice export",
"confidence": "medium",
"confidence_rationale": "traced statically; no authenticated instance to replay against",
"counterevidence": "the gateway may strip the id parameter before it reaches the handler",
"severity_change_conditions": "critical if the export includes other tenants' bank details",
"fix_verification": "unit tests executed; bypass review reasoned only",
"description": "The handler trusts a client-supplied invoice id.",
},
map[string]any{"success": true, "severity": "high", "cvss_score": 7.5},
"completed")))
requireContains(t, out,
"Confidence", "MEDIUM", "no authenticated instance to replay against",
"Counterevidence", "gateway may strip the id parameter",
"Severity Would Change If", "other tenants' bank details",
"Fix Verification", "bypass review reasoned only",
)
}
@@ -72,6 +72,19 @@ func TestNonTablePipeLinesAreLeftAlone(t *testing.T) {
}
}
func TestMarkdownOrderedListsUseSingleSpaceAfterMarker(t *testing.T) {
out := renderAssistantMarkdown("1. hello\n2) world")
plain := ansi.Strip(out)
for _, want := range []string{"1. hello", "2) world"} {
if !strings.Contains(plain, want) {
t.Fatalf("ordered list item %q missing: %q", want, plain)
}
}
if strings.Contains(plain, "1. hello") || strings.Contains(plain, "2) world") {
t.Fatalf("double space after the list marker: %q", plain)
}
}
func TestInlineFormatKeepsNonEmphasisMarkers(t *testing.T) {
literal := []string{
"ls *.py *.go",
@@ -0,0 +1,35 @@
package render
import (
"strings"
)
// ---------------------------------------------------------------------------
// MCP tools (tools from the servers the user connected)
// ---------------------------------------------------------------------------
const mcpIcon = "🔌 "
// renderMcpTool renders a call to a tool from one of the user's MCP servers.
//
// Its own icon and color so a call that left Strix for a server the user
// connected is obvious while scrolling a transcript. The action leads and the
// server trails: the model-facing name is the connection name and the tool name
// stuck together, so leading with the whole name buries the part a reader wants
// behind a connection name that can be long or opaque.
//
// The result is deliberately not rendered, for the same reason
// renderGenericTool leaves it out: an MCP result is whatever an outside server
// chose to return, often multi-kilobyte JSON, and it floods the screen. The full
// result is in the event data, the run log, and the `strix view` viewer.
func renderMcpTool(connection, toolName string, args map[string]any, status string) string {
var b strings.Builder
b.WriteString(mcpIcon + Bold(Mint).Render(toolName))
b.WriteString(Dim().Render(" via MCP server ") + Col(Slate).Render(connection) + "\n")
for _, k := range SortedKeys(args) {
b.WriteString(" " + Dim().Render(k) + ": " + StringValue(args[k]) + "\n")
}
icon, style := statusIcon(status)
b.WriteString(style.Render(icon))
return b.String()
}
+28 -10
View File
@@ -22,19 +22,20 @@ func statusIcon(status string) (string, lipgloss.Style) {
return "○ Unknown", Dim()
}
// renderGenericTool ports registry._render_default_tool_widget.
func renderGenericTool(name string, args map[string]any, result any, status string) string {
// renderGenericTool ports registry._render_default_tool_widget. It shows the
// tool name, its arguments, and a status line only. The raw result is
// deliberately not rendered: a generic result (e.g. a multi-kilobyte JSON
// payload from a database query tool) is noise on screen, and the agent narrates
// what it got in its next message. The full result still lives in the event
// data, the run log, and the `strix view` viewer.
func renderGenericTool(name string, args map[string]any, status string) string {
var b strings.Builder
b.WriteString(Dim().Render("→ Using tool ") + Bold(Blue).Render(name) + "\n")
for _, k := range SortedKeys(args) {
b.WriteString(" " + Dim().Render(k) + ": " + StringValue(args[k]) + "\n")
}
if (status == "completed" || status == "failed" || status == "error") && result != nil {
b.WriteString(lipgloss.NewStyle().Bold(true).Render("Result: ") + StringValue(result))
} else {
icon, style := statusIcon(status)
b.WriteString(style.Render(icon))
}
icon, style := statusIcon(status)
b.WriteString(style.Render(icon))
return b.String()
}
@@ -51,6 +52,18 @@ func Tool(data map[string]any) string {
}
result := data["result"]
// A call to a tool from one of the user's MCP servers is tagged with the
// connection it came from, because its name is the server's own and means
// nothing here. The tag is only ever set from the connections the run made,
// so it is the one thing that can tell such a call apart from a built-in.
if connection := StringValue(data["mcp_connection"]); connection != "" {
toolName := StringValue(data["mcp_tool"])
if toolName == "" {
toolName = name
}
return renderMcpTool(connection, toolName, args, status)
}
switch name {
case "exec_command":
return renderExecCommand(args, result, status)
@@ -82,12 +95,16 @@ func Tool(data map[string]any) string {
return renderNote(name, args, result)
case "create_todo", "list_todos", "update_todo", "mark_todo_done", "mark_todo_pending", "delete_todo":
return renderTodo(name, result)
case "record_coverage", "update_coverage", "list_coverage":
return renderCoverage(name, args, result)
case "get_threat_model", "save_threat_model", "amend_threat_model":
return renderThreatModel(name, args, result)
case "view_agent_graph", "create_agent", "send_message_to_agent", "agent_finish", "wait_for_agents", "stop_agent":
return renderAgentGraphTool(name, args, result)
case "list_requests", "view_request", "repeat_request", "list_sitemap", "view_sitemap_entry", "scope_rules":
return renderProxyTool(name, args, result, status)
}
return renderGenericTool(name, args, result, status)
return renderGenericTool(name, args, status)
}
// ---------------------------------------------------------------------------
@@ -103,7 +120,8 @@ const outputPreviewLines = 10
func ToolPreviewLines(name string) int {
switch name {
case "exec_command", "write_stdin", "apply_patch",
"view_request", "repeat_request", "view_sitemap_entry":
"view_request", "repeat_request", "view_sitemap_entry",
"list_coverage", "get_threat_model":
return outputPreviewLines
}
return 0
@@ -203,7 +203,7 @@ func TestToolDispatchCoversKnownTools(t *testing.T) {
{
"unknown tool falls back to generic",
tool("brand_new_tool", map[string]any{"alpha": "1"}, "done", "completed"),
[]string{"brand_new_tool", "alpha", "Result:", "done"},
[]string{"brand_new_tool", "alpha", "Done"},
},
}
@@ -214,6 +214,43 @@ func TestToolDispatchCoversKnownTools(t *testing.T) {
}
}
func TestGenericToolOmitsRawResult(t *testing.T) {
// The generic renderer shows tool name, args, and a status line only, never
// the raw result payload.
long := strings.Repeat("x", 5000)
out := ansi.Strip(Tool(tool("db_query", map[string]any{"query": "select 1"}, long, "completed")))
requireContains(t, out, "db_query", "query", "Done")
if strings.Contains(out, "Result:") || strings.Contains(out, strings.Repeat("x", 20)) {
t.Fatalf("generic result body must not be rendered:\n%s", out)
}
}
func TestMcpToolLeadsWithActionAndNamesTheServer(t *testing.T) {
data := tool("local_fs_read_file", map[string]any{"path": "/etc/hosts"}, "file body", "completed")
data["mcp_connection"] = "local_fs"
data["mcp_tool"] = "read_file"
out := ansi.Strip(Tool(data))
// The action leads; the server is context that trails it.
if !strings.HasPrefix(out, mcpIcon+"read_file") {
t.Fatalf("MCP render must lead with the tool's own name:\n%s", out)
}
requireContains(t, out, "local_fs", "path", "/etc/hosts", "Done")
// Untrusted server output stays off the terminal, as for the generic render.
if strings.Contains(out, "file body") {
t.Fatalf("MCP result body must not be rendered:\n%s", out)
}
}
func TestMcpToolWithoutTaggedNameFallsBackToFullName(t *testing.T) {
data := tool("local_fs_read_file", nil, nil, "running")
data["mcp_connection"] = "local_fs"
requireContains(t, ansi.Strip(Tool(data)), "local_fs_read_file", "In progress")
}
func TestCollapseToolShellPreviewAndExpand(t *testing.T) {
lines := make([]string, 16)
for i := range lines {
@@ -50,15 +50,31 @@ func renderVulnerabilityReport(args map[string]any, result any) string {
b.WriteString("\n\n" + Bold(Field).Render(label) + "\n" + value)
}
}
if confidence := StringValue(args["confidence"]); confidence != "" {
b.WriteString("\n\n" + Bold(Field).Render("Confidence: ") +
lipgloss.NewStyle().Bold(true).Foreground(confidenceColor(confidence)).
Render(strings.ToUpper(confidence)))
if rationale := StringValue(args["confidence_rationale"]); rationale != "" {
b.WriteString("\n" + Dim().Render(rationale))
}
}
section("Description", StringValue(args["description"]))
section("Impact", StringValue(args["impact"]))
section("Technical Analysis", StringValue(args["technical_analysis"]))
// The case against the finding travels with the case for it: a reader
// triaging this needs both to judge whether to act.
section("Counterevidence", StringValue(args["counterevidence"]))
section("Severity Would Change If", StringValue(args["severity_change_conditions"]))
renderCodeLocations(&b, args["code_locations"])
section("PoC Description", StringValue(args["poc_description"]))
if poc := StringValue(args["poc_script_code"]); poc != "" {
b.WriteString("\n\n" + Bold(Field).Render("PoC Code") + "\n" + Col(Text).Render(poc))
}
section("Remediation", StringValue(args["remediation_steps"]))
// Any applyable fix above is one click from the user's codebase, so how it
// was verified belongs next to it rather than in the artifact alone.
section("Fix Verification", StringValue(args["fix_verification"]))
if title == "" {
b.WriteString("\n " + Dim().Render("Creating report..."))
@@ -66,6 +82,20 @@ func renderVulnerabilityReport(args map[string]any, result any) string {
return "\n\n" + b.String() + "\n\n"
}
// confidenceColor grades how firm the agent's own call is. Anything below
// high is a claim the reader has to check, and should not read as settled.
func confidenceColor(confidence string) lipgloss.Color {
switch strings.ToLower(strings.TrimSpace(confidence)) {
case "high":
return Green
case "medium":
return SevMed
case "low":
return SevHigh
}
return Gray
}
var cvssKeys = [][2]string{
{"attack_vector", "AV"}, {"attack_complexity", "AC"}, {"privileges_required", "PR"},
{"user_interaction", "UI"}, {"scope", "S"}, {"confidentiality", "C"},
@@ -0,0 +1,138 @@
package render
import (
"strconv"
"strings"
)
// ---------------------------------------------------------------------------
// Threat model (get_threat_model / save_threat_model / amend_threat_model)
// ---------------------------------------------------------------------------
var threatModelTitles = map[string]struct {
title string
loading string
errMsg string
}{
"get_threat_model": {"Threat Model", "Loading...", "Unable to read threat model"},
"save_threat_model": {"Threat Model Saved", "Saving...", "Failed to save threat model"},
"amend_threat_model": {"Threat Model Amended", "Amending...", "Failed to amend threat model"},
}
func renderThreatModel(name string, args map[string]any, result any) string {
meta := threatModelTitles[name]
var b strings.Builder
b.WriteString("⌖ " + Bold(InfoBlue).Render(meta.title))
if target := strings.TrimSpace(StringValue(args["target"])); target != "" {
b.WriteString(Dim().Render(" " + target))
}
if s, ok := result.(string); ok && strings.TrimSpace(s) != "" {
b.WriteString("\n " + Dim().Render(strings.TrimSpace(s)))
return b.String()
}
m, ok := result.(map[string]any)
if !ok {
b.WriteString("\n " + Dim().Render(meta.loading))
return b.String()
}
if !truthy(m["success"]) {
errMsg := StringValue(m["error"])
if errMsg == "" {
errMsg = meta.errMsg
}
b.WriteString("\n " + Col(Red).Render(errMsg))
return b.String()
}
switch name {
case "get_threat_model":
threatModelReadBody(&b, m)
case "amend_threat_model":
b.WriteString("\n " + Col(Green).Render("✓ amendment recorded"))
if count, ok := NumericValue(m["amendment_count"]); ok {
b.WriteString(Dim().Render(" (" + strconv.Itoa(int(count)) + " total)"))
}
threatModelBody(&b, StringValue(args["addendum"]))
default:
b.WriteString("\n " + Col(Green).Render("✓ saved"))
if revision := shortRevision(StringValue(m["revision"])); revision != "" {
b.WriteString(Dim().Render(" at " + revision))
}
// Saving folds amendments away, so the count that vanished is worth
// stating: it is the one destructive thing this tool does.
if cleared, ok := NumericValue(m["amendments_cleared"]); ok && cleared > 0 {
b.WriteString("\n " + Col(AmberY).Render("⚠ cleared "+
strconv.Itoa(int(cleared))+" amendment(s)"))
}
threatModelBody(&b, StringValue(args["content"]))
}
return b.String()
}
func threatModelReadBody(b *strings.Builder, result map[string]any) {
if !truthy(result["found"]) {
b.WriteString("\n " + Dim().Render("No model cached for this target yet"))
return
}
if truthy(result["stale"]) {
b.WriteString("\n " + Col(AmberY).Render("⚠ stale"))
if cached := shortRevision(StringValue(result["cached_revision"])); cached != "" {
b.WriteString(Dim().Render(" (written at " + cached + ")"))
}
}
if amendments, ok := result["amendments"].([]any); ok && len(amendments) > 0 {
b.WriteString("\n " + Col(Gold).Render("+ "+strconv.Itoa(len(amendments))+
" amendment(s)") + Dim().Render(" — later statements win"))
for _, a := range amendments {
amendment, _ := a.(map[string]any)
who := strings.TrimSpace(StringValue(amendment["agent_name"]))
if who == "" {
who = "unknown agent"
}
b.WriteString("\n - " + Dim().Render(who+": ") +
psanitize(strings.TrimSpace(StringValue(amendment["content"])), 120))
}
}
threatModelBody(b, StringValue(result["content"]))
}
// threatModelBody previews the document. The full text is a page or more, so
// only its section headings and opening line are shown here; the trace can be
// expanded for the rest.
func threatModelBody(b *strings.Builder, content string) {
content = strings.TrimSpace(content)
if content == "" {
return
}
var headings []string
summary := ""
for _, line := range strings.Split(content, "\n") {
line = strings.TrimSpace(line)
switch {
case strings.HasPrefix(line, "#"):
headings = append(headings, strings.TrimSpace(strings.TrimLeft(line, "# ")))
case summary == "" && line != "":
summary = line
}
}
if summary != "" {
b.WriteString("\n " + Dim().Render(psanitize(summary, 160)))
}
if len(headings) > 0 {
if len(headings) > 8 {
headings = headings[:8]
}
b.WriteString("\n " + Dim().Render(strings.Join(headings, " · ")))
}
}
// shortRevision abbreviates a git sha; "unversioned" targets have no revision
// worth showing.
func shortRevision(revision string) string {
revision = strings.TrimSpace(revision)
if revision == "" || revision == "unversioned" {
return ""
}
return firstN(revision, 8)
}
+36 -4
View File
@@ -8,6 +8,7 @@ from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from collections.abc import Iterable
from pathlib import Path
from agents.tool import ToolOutputImage
@@ -15,6 +16,10 @@ from agents.tool import ToolOutputImage
from strix.core.paths import runtime_state_dir
from strix.interface.tui.history import load_session_history
# Imported from the naming module rather than the mcp package so a projection
# never pulls in the MCP client and the agents SDK behind it.
from strix.tools.mcp.naming import resolve_mcp_tool
class TuiLiveView:
def __init__(self) -> None:
@@ -26,6 +31,27 @@ class TuiLiveView:
self._user_instruction: str | None = None
self._user_instruction_at: str | None = None
self._user_instruction_shown = False
self._mcp_connections: tuple[str, ...] = ()
def set_mcp_connections(self, names: Iterable[str]) -> None:
"""The MCP servers this run connected, so its tool calls can name theirs.
A server's tools are offered to the model under a name built from the
connection name and the tool's own name. That name cannot be split back
apart on its own, so tool calls are matched against these names instead.
"""
self._mcp_connections = tuple(str(name) for name in names)
def _mcp_tool_fields(self, tool_name: str) -> dict[str, str]:
"""Event fields naming the MCP server a tool call went out to, if any.
Empty for every built-in tool, which is what tells an interface to render
the call as one of its own rather than as a call to a user's server.
"""
origin = resolve_mcp_tool(tool_name, self._mcp_connections)
if origin is None:
return {}
return {"mcp_connection": origin.connection, "mcp_tool": origin.tool}
def set_user_instruction(self, text: str | None, *, timestamp: str | None = None) -> None:
"""Open the transcript with what the user asked for.
@@ -72,8 +98,9 @@ class TuiLiveView:
def hydrate_from_run_dir(self, run_dir: Path) -> None:
# Armed before the agents are added so the root agent's arrival puts the
# user's opening message ahead of the replayed history.
self._load_user_instruction(run_dir)
# user's opening message ahead of the replayed history, and before the
# history is replayed so its MCP tool calls are attributed too.
self._load_run_record(run_dir)
state_dir = runtime_state_dir(run_dir)
agents_path = state_dir / "agents.json"
if not agents_path.exists():
@@ -100,14 +127,17 @@ class TuiLiveView:
self.flush_user_instruction()
self._hydrate_sdk_session_history(run_dir, statuses.keys())
def _load_user_instruction(self, run_dir: Path) -> None:
"""Take the user's opening message from the run record, if it has one."""
def _load_run_record(self, run_dir: Path) -> None:
"""Take the user's opening message and the run's MCP servers off the record."""
try:
record = json.loads((run_dir / "run.json").read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
return
if not isinstance(record, dict):
return
connections = record.get("mcp_connections")
if isinstance(connections, list):
self.set_mcp_connections(name for name in connections if isinstance(name, str))
instruction = record.get("user_instruction")
if not isinstance(instruction, str):
return
@@ -318,6 +348,7 @@ class TuiLiveView:
"status": "running",
"agent_id": agent_id,
"call_id": call_id,
**self._mcp_tool_fields(call["tool_name"]),
}
if existing is None:
event = self._append_event(agent_id, "tool", tool_data, timestamp=timestamp)
@@ -349,6 +380,7 @@ class TuiLiveView:
"status": "completed",
"agent_id": agent_id,
"call_id": call_id,
**self._mcp_tool_fields(output["tool_name"]),
},
timestamp=timestamp,
)
+14
View File
@@ -207,9 +207,23 @@ class GoTuiRuntime:
self.controller.notify_changed()
def capture_event(self, agent_id: str, event: Any) -> None:
self._refresh_mcp_connections()
self.live_view.ingest_sdk_event(agent_id, event)
self.controller.notify_changed()
def _refresh_mcp_connections(self) -> None:
"""Hand the projection the MCP servers the scan connected.
The scan records them as it connects, which is before the agent can call
anything, and the projection needs them to say which server a tool call
went out to. Read on the way in rather than pushed, so no tool call can
be projected before they arrive.
"""
if self.report_state is None:
return
connections = self.report_state.run_record.get("mcp_connections") or []
self.live_view.set_mcp_connections(connections)
async def _sync_agent_state(self) -> bool:
parent_of, statuses, names, errors = await self.coordinator.graph_snapshot()
changed = False
+5 -2
View File
@@ -13,9 +13,7 @@ from pathlib import Path
from typing import Any
from urllib.parse import parse_qs, urlparse
import docker
import requests
from docker.errors import DockerException, ImageNotFound
from rich.console import Console
from rich.panel import Panel
from rich.text import Text
@@ -1599,6 +1597,9 @@ def clone_repository(repo_url: str, run_name: str, dest_name: str | None = None)
def check_docker_connection() -> Any:
import docker
from docker.errors import DockerException
try:
return docker.from_env()
except DockerException:
@@ -1624,6 +1625,8 @@ def check_docker_connection() -> Any:
def image_exists(client: Any, image_name: str) -> bool:
from docker.errors import ImageNotFound
try:
client.images.get(image_name)
except ImageNotFound:
+5 -1
View File
@@ -45,7 +45,11 @@ def run_view(argv: list[str]) -> None:
default=0,
help="Port to serve on (default: an available ephemeral port).",
)
parser.add_argument("--host", default="127.0.0.1", help=argparse.SUPPRESS)
parser.add_argument(
"--host",
default="127.0.0.1",
help="Host to bind to (default: 127.0.0.1; use 0.0.0.0 for all IPv4 interfaces).",
)
parser.add_argument(
"--no-open",
action="store_true",
@@ -30,7 +30,7 @@ class RendererErrorBoundary extends Component<
}
function SafeToolRenderer(props: ToolRendererProps) {
const Renderer = getToolRenderer(props.toolName);
const Renderer = getToolRenderer(props.toolName, props.mcpConnection);
return (
<RendererErrorBoundary toolName={props.toolName}>
<Renderer {...props} />
@@ -63,6 +63,10 @@ function coerce(value: unknown): unknown {
}
}
function asOptionalString(value: unknown): string | null {
return typeof value === "string" && value ? value : null;
}
function asRecord(value: unknown): Record<string, unknown> {
const c = coerce(value);
if (c && typeof c === "object" && !Array.isArray(c)) return c as Record<string, unknown>;
@@ -244,11 +248,14 @@ export function AgentTranscript({
const isTool = event.type === "tool";
const toolName = isTool ? String(event.data?.tool_name ?? "tool") : "";
const role = !isTool ? String(event.data?.role ?? "assistant") : "";
// Present only on a call to one of the user's own MCP servers.
const mcpConnection = asOptionalString(event.data?.mcp_connection);
const mcpTool = asOptionalString(event.data?.mcp_tool);
let Icon;
let iconColor: string;
if (isTool) {
const meta = getToolIcon(toolName);
const meta = getToolIcon(toolName, mcpConnection);
Icon = meta.icon;
iconColor = meta.color;
} else {
@@ -279,6 +286,8 @@ export function AgentTranscript({
{isTool ? (
<SafeToolRenderer
toolName={toolName}
mcpConnection={mcpConnection}
mcpTool={mcpTool}
args={asRecord(event.data?.args)}
result={coerce(event.data?.result) ?? null}
status={
@@ -0,0 +1,184 @@
"use client";
import type { ToolRendererProps } from "@/types/events";
import { CheckCircle2, CircleSlash, HelpCircle, AlertTriangle, Circle, ClipboardList } from "lucide-react";
interface CoverageEntry {
entry_id?: string;
surface?: string;
risk_area?: string;
outcome?: string;
evidence?: string;
agent_name?: string;
by_you?: boolean;
previous_outcomes?: string[];
}
/**
* A cleared surface and an unresolved one must never read alike the ledger
* exists so that the negative space of a scan is legible, so each outcome gets
* its own icon and color rather than a shared neutral row.
*/
const OUTCOMES: Record<string, { label: string; color: string; Icon: typeof Circle }> = {
reported: { label: "reported", color: "text-orange-400", Icon: AlertTriangle },
no_issue_found: { label: "no issue found", color: "text-emerald-400", Icon: CheckCircle2 },
ruled_out: { label: "ruled out", color: "text-emerald-400/70", Icon: CheckCircle2 },
not_applicable: { label: "not applicable", color: "text-[#777]", Icon: CircleSlash },
needs_follow_up: { label: "needs follow-up", color: "text-yellow-400", Icon: HelpCircle },
};
const OUTCOME_ORDER = [
"reported", "needs_follow_up", "no_issue_found", "ruled_out", "not_applicable",
] as const;
function outcomeMeta(outcome: string | undefined) {
const key = (outcome ?? "").trim().toLowerCase();
return OUTCOMES[key] ?? {
label: key ? key.replace(/_/g, " ") : "unrecorded",
color: "text-[#777]",
Icon: Circle,
};
}
const ACTION_LABELS: Record<string, string> = {
record_coverage: "Coverage recorded",
update_coverage: "Coverage updated",
list_coverage: "Coverage",
};
function Header({ toolName }: { toolName: string }) {
return (
<div className="flex items-center gap-2">
<ClipboardList className="w-3.5 h-3.5 text-cyan-400/60" />
<span className="text-cyan-400/80 font-semibold text-sm">
{ACTION_LABELS[toolName] ?? "Coverage"}
</span>
</div>
);
}
function Row({ entry }: { entry: CoverageEntry }) {
const { label, color, Icon } = outcomeMeta(entry.outcome);
const previous = (entry.previous_outcomes ?? [])
.map((o) => outcomeMeta(o).label)
.filter(Boolean);
return (
<div className="flex items-start gap-2.5 py-1.5">
<Icon className={`w-3.5 h-3.5 shrink-0 mt-[2px] ${color}`} />
<div className="min-w-0">
<div className="text-[13px] leading-snug">
<span className="text-[#bbb]">{entry.surface ?? "(unnamed surface)"}</span>
{entry.risk_area && <span className="text-[#666]"> · {entry.risk_area}</span>}
</div>
<div className="text-xs mt-0.5">
<span className={color}>{label}</span>
{previous.length > 0 && (
<span className="text-[#555]"> (was {previous.join(" → ")})</span>
)}
{(entry.by_you || entry.agent_name) && (
<span className="text-[#555]"> · {entry.by_you ? "you" : entry.agent_name}</span>
)}
</div>
{entry.evidence && (
<div className="text-[#777] text-xs mt-1 leading-snug">{entry.evidence}</div>
)}
</div>
</div>
);
}
export default function CoverageRenderer({ toolName, args, result }: ToolRendererProps) {
const res = result as Record<string, unknown> | string | null;
if (typeof res === "string" && res.trim()) {
return (
<div>
<Header toolName={toolName} />
<div className="mt-1.5 text-[#888] text-[13px]">{res.trim()}</div>
</div>
);
}
const structured = res && typeof res === "object" ? res : null;
const surface = (args.surface as string) ?? "";
const riskArea = (args.risk_area as string) ?? "";
const evidence = (args.evidence as string) ?? "";
if (structured && !structured.success) {
return (
<div>
<Header toolName={toolName} />
{(surface || riskArea) && (
<div className="mt-1.5 text-[13px] text-[#bbb]">
{surface}
{riskArea && <span className="text-[#666]"> · {riskArea}</span>}
</div>
)}
<div className="mt-1 text-red-400/70 text-[13px]">
{(structured.error as string) ?? "Coverage call failed"}
</div>
</div>
);
}
if (toolName === "list_coverage") {
const rawEntries = structured?.entries;
const entries: CoverageEntry[] = Array.isArray(rawEntries) ? (rawEntries as CoverageEntry[]) : [];
const counts = (structured?.outcome_counts as Record<string, number> | undefined) ?? {};
const total = (structured?.total_count as number) ?? 0;
return (
<div>
<Header toolName={toolName} />
{Object.keys(counts).length > 0 && (
<div className="mt-2 flex items-center gap-3 flex-wrap">
{OUTCOME_ORDER.filter((o) => counts[o]).map((o) => {
const { label, color } = outcomeMeta(o);
return (
<span key={o} className={`text-xs ${color}`}>
{label}: {counts[o]}
</span>
);
})}
</div>
)}
{entries.length > 0 ? (
<div className="mt-2 rounded-lg border border-white/[0.06] bg-white/[0.015] px-3 py-1 divide-y divide-white/[0.04]">
{entries.map((entry, i) => <Row key={entry.entry_id ?? i} entry={entry} />)}
</div>
) : (
<div className="mt-1.5 text-[#555] text-xs">
{total === 0 ? "No surfaces recorded yet" : "No surfaces match this filter"}
</div>
)}
</div>
);
}
const outcome = (structured?.outcome as string) ?? "";
const previousOutcome = (structured?.previous_outcome as string) ?? "";
const { label, color, Icon } = outcomeMeta(outcome);
return (
<div>
<Header toolName={toolName} />
<div className="mt-2 flex items-start gap-2.5">
<Icon className={`w-3.5 h-3.5 shrink-0 mt-[2px] ${color}`} />
<div className="min-w-0">
<div className="text-[13px] leading-snug text-[#bbb]">
{surface || (structured?.entry_id ? `entry ${structured.entry_id as string}` : "(unnamed surface)")}
{riskArea && <span className="text-[#666]"> · {riskArea}</span>}
</div>
<div className="text-xs mt-0.5">
{previousOutcome && (
<span className="text-[#666]">{outcomeMeta(previousOutcome).label} </span>
)}
<span className={color}>{label}</span>
</div>
{evidence && (
<div className="text-[#777] text-xs mt-1 leading-snug">{evidence}</div>
)}
</div>
</div>
</div>
);
}
@@ -0,0 +1,82 @@
"use client";
import type { ToolRendererProps } from "@/types/events";
/**
* A call to a tool from one of the MCP servers the user connected.
*
* Deliberately the same shape as the terminal: the tool's own name, the server
* it went to, the arguments one per line, and a status. The result is not shown.
* These payloads are routinely thousands of characters of JSON that say nothing a
* reader wants at this point in the transcript, and the agent narrates what it
* learned in its next message. A failure is the exception, because that is what
* someone is looking for when a step did not work; it renders as inert text,
* never as markdown, since it came from a server outside Strix.
*
* The full result is still in the run's event data on disk either way.
*/
/** Arguments one line each, as the terminal prints them. */
function argLines(args: unknown): string[] {
if (!args || typeof args !== "object" || Array.isArray(args)) return [];
return Object.entries(args as Record<string, unknown>).map(([key, value]) => {
const rendered = typeof value === "string" ? value : JSON.stringify(value);
return `${key}: ${rendered ?? String(value)}`;
});
}
const MAX_ERROR_CHARS = 600;
function errorText(result: unknown): string | null {
if (typeof result === "string") {
const trimmed = result.trim();
if (!trimmed) return null;
return trimmed.length > MAX_ERROR_CHARS ? `${trimmed.slice(0, MAX_ERROR_CHARS)}` : trimmed;
}
return null;
}
export default function McpRenderer({
toolName,
mcpTool,
mcpConnection,
args,
result,
status,
}: ToolRendererProps) {
const lines = argLines(args);
const failed = status === "failed" || status === "error";
const error = failed ? errorText(result) : null;
return (
<div>
<div className="flex items-center gap-2 flex-wrap">
<span className="font-mono text-teal-300 font-semibold text-sm">{mcpTool || toolName}</span>
<span className="text-[13px] text-[#555]">via MCP server</span>
{mcpConnection && <span className="text-[13px] text-teal-400/80">{mcpConnection}</span>}
</div>
{lines.length > 0 && (
<div className="mt-1 font-mono text-[13px] leading-relaxed">
{lines.map((line) => (
<div key={line} className="text-[#777] break-all">
{line}
</div>
))}
</div>
)}
<div className="mt-1 text-[13px]">
{status === "running" && <span className="text-[#666]">Running</span>}
{status === "completed" && <span className="text-emerald-400/80"> Done</span>}
{failed && <span className="text-red-400/80"> Failed</span>}
</div>
{error && (
<pre className="mt-1 font-mono text-[13px] leading-relaxed whitespace-pre-wrap break-words text-red-400/70">
{error}
</pre>
)}
</div>
);
}
@@ -0,0 +1,138 @@
"use client";
import type { ToolRendererProps } from "@/types/events";
import { Crosshair, AlertTriangle, Plus, Save } from "lucide-react";
import { TruncatedText } from "./ToolCard";
interface Amendment {
agent_name?: string;
content?: string;
recorded_at?: string;
}
const ACTION_LABELS: Record<string, { label: string; Icon: typeof Crosshair }> = {
get_threat_model: { label: "Threat model", Icon: Crosshair },
save_threat_model: { label: "Threat model saved", Icon: Save },
amend_threat_model: { label: "Threat model amended", Icon: Plus },
};
/** A git sha is noise past its first bytes, and "unversioned" is not a revision. */
function shortRevision(revision: unknown): string {
const value = typeof revision === "string" ? revision.trim() : "";
if (!value || value === "unversioned") return "";
return value.slice(0, 8);
}
export default function ThreatModelRenderer({ toolName, args, result }: ToolRendererProps) {
const action = ACTION_LABELS[toolName] ?? { label: "Threat model", Icon: Crosshair };
const ActionIcon = action.Icon;
const target = (args.target as string) ?? "";
const res = result as Record<string, unknown> | string | null;
const header = (
<div className="flex items-center gap-2 flex-wrap">
<ActionIcon className="w-3.5 h-3.5 text-blue-400/60" />
<span className="text-blue-400/80 font-semibold text-sm">{action.label}</span>
{target && <span className="text-[#666] font-mono text-xs">{target}</span>}
</div>
);
if (typeof res === "string" && res.trim()) {
return <div>{header}<div className="mt-1.5 text-[#888] text-[13px]">{res.trim()}</div></div>;
}
const structured = res && typeof res === "object" ? res : null;
if (structured && !structured.success) {
return (
<div>
{header}
<div className="mt-1.5 text-red-400/70 text-[13px]">
{(structured.error as string) ?? "Threat model call failed"}
</div>
</div>
);
}
if (toolName === "get_threat_model") {
if (structured && !structured.found) {
return (
<div>
{header}
<div className="mt-1.5 text-[#555] text-xs">No model cached for this target yet</div>
</div>
);
}
const rawAmendments = structured?.amendments;
const amendments: Amendment[] = Array.isArray(rawAmendments) ? (rawAmendments as Amendment[]) : [];
const cachedRevision = shortRevision(structured?.cached_revision);
return (
<div>
{header}
{structured?.stale === true && (
<div className="mt-1.5 flex items-center gap-1.5 text-yellow-400/80 text-xs">
<AlertTriangle className="w-3 h-3 shrink-0" />
<span>stale{cachedRevision ? ` — written at ${cachedRevision}` : ""}</span>
</div>
)}
{amendments.length > 0 && (
<div className="mt-2">
<span className="text-amber-400/70 text-xs font-semibold">
{amendments.length} amendment{amendments.length === 1 ? "" : "s"}
</span>
<span className="text-[#555] text-xs"> later statements win</span>
<div className="mt-1 space-y-1">
{/* On a public share link the amendment body is stripped, so the
author line has to stand on its own. */}
{amendments.map((amendment, i) => (
<div key={i} className="text-xs leading-snug">
<span className="text-[#666]">{amendment.agent_name ?? "unknown agent"}</span>
{amendment.content && (
<span className="text-[#999]">: {amendment.content}</span>
)}
</div>
))}
</div>
</div>
)}
{typeof structured?.content === "string" && structured.content.trim() && (
<div className="mt-2">
<TruncatedText text={structured.content} maxLines={14} />
</div>
)}
</div>
);
}
if (toolName === "amend_threat_model") {
const addendum = (args.addendum as string) ?? "";
const count = structured?.amendment_count as number | undefined;
return (
<div>
{header}
{count != null && (
<div className="mt-1.5 text-[#666] text-xs">{count} amendment{count === 1 ? "" : "s"} on this model</div>
)}
{addendum && <div className="mt-1.5"><TruncatedText text={addendum} maxLines={10} /></div>}
</div>
);
}
const cleared = (structured?.amendments_cleared as number | undefined) ?? 0;
const revision = shortRevision(structured?.revision);
const content = (args.content as string) ?? "";
return (
<div>
{header}
{revision && <div className="mt-1.5 text-[#666] font-mono text-xs">at {revision}</div>}
{/* Saving folds amendments away — the one destructive thing this tool does. */}
{cleared > 0 && (
<div className="mt-1.5 flex items-center gap-1.5 text-yellow-400/80 text-xs">
<AlertTriangle className="w-3 h-3 shrink-0" />
<span>cleared {cleared} amendment{cleared === 1 ? "" : "s"}</span>
</div>
)}
{content && <div className="mt-2"><TruncatedText text={content} maxLines={14} /></div>}
</div>
);
}
@@ -11,6 +11,11 @@ const SEVERITY_COLORS: Record<string, string> = {
low: "text-blue-400", info: "text-cyan-400",
};
/** Anything below high is a claim the reader still has to check. */
const CONFIDENCE_COLORS: Record<string, string> = {
high: "text-emerald-400", medium: "text-yellow-400", low: "text-orange-400",
};
export default function VulnReportRenderer({ args, result }: ToolRendererProps) {
const title = (args.title as string) ?? "";
const description = (args.description as string) ?? "";
@@ -24,6 +29,11 @@ export default function VulnReportRenderer({ args, result }: ToolRendererProps)
const remediation = (args.remediation_steps as string) ?? "";
const cve = (args.cve as string) ?? "";
const cwe = (args.cwe as string) ?? "";
const counterevidence = (args.counterevidence as string) ?? "";
const confidence = ((args.confidence as string) ?? "").toLowerCase();
const confidenceRationale = (args.confidence_rationale as string) ?? "";
const severityChangeConditions = (args.severity_change_conditions as string) ?? "";
const fixVerification = (args.fix_verification as string) ?? "";
const res = result as Record<string, unknown> | null;
const rawSev = (res && typeof res === "object" ? res.severity : null) ?? args.severity ?? "medium";
@@ -38,6 +48,11 @@ export default function VulnReportRenderer({ args, result }: ToolRendererProps)
{cvss != null && <span className="text-[#888] text-[13px]">CVSS {cvss}</span>}
{cve && <span className="text-[#888] font-mono text-[13px]">{cve}</span>}
{cwe && <span className="text-[#888] font-mono text-[13px]">{cwe}</span>}
{confidence && (
<span className={`text-[13px] ${CONFIDENCE_COLORS[confidence] ?? "text-[#888]"}`}>
{confidence} confidence
</span>
)}
</div>
{title && <div className="text-[15px] text-white/80 font-semibold">{title}</div>}
{(target || endpoint) && (
@@ -56,6 +71,23 @@ export default function VulnReportRenderer({ args, result }: ToolRendererProps)
<div className="mt-1"><TruncatedText text={technicalAnalysis} maxLines={20} /></div>
</div>
)}
{confidenceRationale && (
<div className="text-[#777] text-xs leading-snug">{confidenceRationale}</div>
)}
{/* The case against the finding sits beside the case for it: whoever
triages this needs both to decide whether to act. */}
{counterevidence && (
<div>
<span className="text-emerald-400/60 text-sm font-semibold">Counterevidence</span>
<div className="mt-1"><TruncatedText text={counterevidence} maxLines={12} /></div>
</div>
)}
{severityChangeConditions && (
<div>
<span className="text-emerald-400/60 text-sm font-semibold">Severity would change if</span>
<div className="mt-1"><TruncatedText text={severityChangeConditions} maxLines={10} /></div>
</div>
)}
{(pocDescription || pocCode) && (
<div>
<span className="text-emerald-400/60 text-sm font-semibold">Proof of Concept</span>
@@ -69,6 +101,14 @@ export default function VulnReportRenderer({ args, result }: ToolRendererProps)
<div className="mt-1"><TruncatedText text={remediation} maxLines={15} /></div>
</div>
)}
{/* An applyable fix is one click from the user's codebase, so how it was
verified belongs next to it. */}
{fixVerification && (
<div>
<span className="text-emerald-400/60 text-sm font-semibold">Fix verification</span>
<div className="mt-1"><TruncatedText text={fixVerification} maxLines={12} /></div>
</div>
)}
</div>
);
}
@@ -3,7 +3,7 @@ import type { ToolRendererProps } from "@/types/events";
import {
Terminal, Globe, FileText, ShieldAlert, ArrowUpRight, Brain,
Bot, MessageCircle, Flag, Eye, Search, Code, StickyNote,
ListTodo, Crosshair, Wrench, Ban, Image,
ListTodo, Crosshair, Wrench, Ban, Image, ClipboardList, Plug,
} from "lucide-react";
import TerminalRenderer from "./TerminalRenderer";
@@ -25,6 +25,9 @@ import TodoRenderer from "./TodoRenderer";
import FallbackRenderer from "./FallbackRenderer";
import LoadSkillRenderer from "./LoadSkillRenderer";
import RespondRenderer from "./RespondRenderer";
import CoverageRenderer from "./CoverageRenderer";
import ThreatModelRenderer from "./ThreatModelRenderer";
import McpRenderer from "./McpRenderer";
/**
* Tool-renderer mapping data-driven, keyed by the engine's tool *family*.
@@ -53,7 +56,10 @@ export type ToolCategory =
| "notes"
| "skills"
| "todos"
| "telemetry";
| "coverage"
| "threatModel"
| "telemetry"
| "mcp";
export interface ToolIconMeta {
icon: ComponentType<{ className?: string }>;
@@ -83,7 +89,12 @@ const CATEGORY_META: Record<ToolCategory, CategoryMeta> = {
notes: { renderer: NotesRenderer, icon: StickyNote, color: "text-amber-400", match: /note/ },
skills: { renderer: LoadSkillRenderer, icon: Wrench, color: "text-emerald-400" },
todos: { renderer: TodoRenderer, icon: ListTodo, color: "text-purple-400", match: /todo/ },
coverage: { renderer: CoverageRenderer, icon: ClipboardList, color: "text-cyan-400", match: /coverage/ },
threatModel: { renderer: ThreatModelRenderer, icon: Crosshair, color: "text-blue-400", match: /threat_model/ },
telemetry: { renderer: FallbackRenderer, icon: Wrench, color: "text-[#555]" },
// Tools from the user's own MCP servers. Resolved from the connection on the
// event rather than from a tool name, so this family has no names below.
mcp: { renderer: McpRenderer, icon: Plug, color: "text-teal-400" },
};
/**
@@ -112,7 +123,12 @@ const CATEGORY_TOOLS: Record<ToolCategory, readonly string[]> = {
notes: ["create_note", "delete_note", "update_note", "list_notes", "get_note"],
skills: ["load_skill"],
todos: ["create_todo", "list_todos", "update_todo", "mark_todo_done", "mark_todo_pending", "delete_todo"],
// Shared coverage ledger — one row per surface × risk area for the whole run
coverage: ["record_coverage", "update_coverage", "list_coverage"],
// Per-target threat model, shared across the agent tree
threatModel: ["get_threat_model", "save_threat_model", "amend_threat_model"],
telemetry: ["sandbox_error_details", "llm_error_details"],
mcp: [],
};
/** Reverse index (tool name → family), built once from CATEGORY_TOOLS. */
@@ -163,14 +179,26 @@ function resolveCategory(toolName: string): ToolCategory | null {
return null;
}
export function getToolRenderer(toolName: string): ComponentType<ToolRendererProps> {
/**
* A call to a tool from one of the user's MCP servers is placed by the
* connection it was tagged with, ahead of every name-keyed lookup below: its
* name belongs to that server and matches nothing in this table.
*/
export function getToolRenderer(
toolName: string,
mcpConnection?: string | null
): ComponentType<ToolRendererProps> {
if (mcpConnection) return CATEGORY_META.mcp.renderer;
const override = RENDERER_OVERRIDES[toolName];
if (override) return override;
const category = resolveCategory(toolName);
return category ? CATEGORY_META[category].renderer : FallbackRenderer;
}
export function getToolIcon(toolName: string): ToolIconMeta {
export function getToolIcon(toolName: string, mcpConnection?: string | null): ToolIconMeta {
if (mcpConnection) {
return { icon: CATEGORY_META.mcp.icon, color: CATEGORY_META.mcp.color };
}
const override = ICON_OVERRIDES[toolName];
if (override) return override;
const category = resolveCategory(toolName);
@@ -99,4 +99,12 @@ export interface ToolRendererProps {
args: Record<string, unknown>;
result: unknown;
status: "running" | "completed" | "failed" | "error";
/**
* Set only on a call to a tool from an MCP server the user connected: the name
* they gave that connection, and the server's own name for the tool. The
* engine resolves both, because `toolName` is the two glued together and
* cannot be split back apart here.
*/
mcpConnection?: string | null;
mcpTool?: string | null;
}
+22 -20
View File
@@ -135,8 +135,9 @@ class _ViewerState:
# exchanged for a session cookie only when presented on the initial page
# load. It is the request-level authorization the review asked for:
# reachability of the port (e.g. when bound with ``--host``) is not
# enough to steer a live scan, trigger a report, or browse history --
# the token is never handed to a caller who merely reaches ``/``.
# enough to read run data, steer a live scan, trigger a report, or
# browse history -- the token is never handed to a caller who merely
# reaches ``/``.
self.session_token = secrets.token_urlsafe(32)
# Finalized in ``serve()`` once the port is known (the server binds
# after this state is constructed); see SESSION_COOKIE_PREFIX.
@@ -234,11 +235,11 @@ def _make_handler(state: _ViewerState) -> type[BaseHTTPRequestHandler]:
self.end_headers()
def _handle_api(self, path: str, query: dict[str, list[str]]) -> None:
# The launched run is always viewable with no verification. The
# cross-run history list (/api/runs) unlocks its entries only for a
# caller that holds this process's session capability *and* is email
# verified, so merely reaching an exposed --host port never leaks the
# run list (the payload still advertises the count as a teaser).
# The cross-run history list (/api/runs) unlocks its entries only for
# a caller that holds this process's session capability *and* is
# email verified, so merely reaching an exposed --host port never
# leaks the run list (the payload still advertises the count as a
# teaser).
if path == "/api/runs":
unlocked = self._has_session() and auth.is_verified()
payload = build_runs_payload(state.base_dir, verified=unlocked)
@@ -253,6 +254,13 @@ def _make_handler(state: _ViewerState) -> type[BaseHTTPRequestHandler]:
self._handle_auth_status()
return
# All remaining GET endpoints expose run metadata or scan output.
# Require the capability even for the run used to launch the viewer;
# reachability of an exposed --host port must not grant data access.
if not self._has_session():
self._send_json(HTTPStatus.FORBIDDEN, {"error": "forbidden"})
return
run_values = query.get("run")
run_param = run_values[0] if run_values else None
run_dir = resolve_run_dir(state.base_dir, run_param, state.run_dir)
@@ -260,18 +268,12 @@ def _make_handler(state: _ViewerState) -> type[BaseHTTPRequestHandler]:
self._send_json(HTTPStatus.NOT_FOUND, {"error": "unknown run"})
return
# The launched run is always viewable. Any *other* run's data is part
# of the gated history: it needs this process's session capability
# (so merely reaching an exposed --host port is not enough) *and*
# email verification -- otherwise knowing a run name would leak its
# metadata, vulnerabilities, report, and transcript.
if run_dir.resolve() != state.run_dir.resolve():
if not self._has_session():
self._send_json(HTTPStatus.FORBIDDEN, {"error": "forbidden"})
return
if not auth.is_verified():
self._send_json(HTTPStatus.UNAUTHORIZED, {"error": "unverified"})
return
# Any run other than the one used to launch the viewer is part of the
# email-gated history. The session check above applies to both paths;
# verification adds a second gate for historical run data.
if run_dir.resolve() != state.run_dir.resolve() and not auth.is_verified():
self._send_json(HTTPStatus.UNAUTHORIZED, {"error": "unverified"})
return
if path == "/api/run":
self._send_json(HTTPStatus.OK, read_run_summary(run_dir))
@@ -385,7 +387,7 @@ def _make_handler(state: _ViewerState) -> type[BaseHTTPRequestHandler]:
except auth.RelayError as exc:
self._send_relay_error(exc)
return
# The password is returned only to the local (127.0.0.1) browser.
# The password is returned only to a session-authorized browser.
self._send_json(
HTTPStatus.OK,
{"ok": True, "password": password, "filename": filename},
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
+2 -2
View File
@@ -6,8 +6,8 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="color-scheme" content="dark" />
<title>Strix Results</title>
<script type="module" crossorigin src="./assets/index-DBJ-RJqo.js"></script>
<link rel="stylesheet" crossorigin href="./assets/index-DKbLYAbP.css">
<script type="module" crossorigin src="./assets/index-C9c1WbvP.js"></script>
<link rel="stylesheet" crossorigin href="./assets/index-D0453ODW.css">
</head>
<body>
<div id="root"></div>
+16 -3
View File
@@ -10,11 +10,11 @@ pairing so the trimmed history is still valid provider input.
from __future__ import annotations
import logging
from functools import cache
from typing import TYPE_CHECKING, Any
from agents.model_settings import ModelSettings
from agents.models.interface import ModelTracing
from litellm.exceptions import BadRequestError, ContextWindowExceededError
from openai.types.responses import ResponseOutputMessage, ResponseOutputText
from strix.config import load_settings
@@ -63,6 +63,18 @@ _OVERFLOW_MARKERS = (
)
@cache
def _overflow_error_types() -> tuple[type[BaseException], type[BaseException]]:
"""``(ContextWindowExceededError, BadRequestError)``, imported on first use.
LiteLLM costs seconds to import, and nothing needs it until a model call is
actually made, so it stays off the launch path.
"""
from litellm.exceptions import BadRequestError, ContextWindowExceededError
return ContextWindowExceededError, BadRequestError
def is_context_overflow(exc: BaseException) -> bool:
"""Whether ``exc`` is a model context-window-overflow error.
@@ -70,9 +82,10 @@ def is_context_overflow(exc: BaseException) -> bool:
OpenRouter branch raises a plain BadRequestError, so for that we fall back to
matching the provider message.
"""
if isinstance(exc, ContextWindowExceededError):
context_window_exceeded, bad_request = _overflow_error_types()
if isinstance(exc, context_window_exceeded):
return True
if isinstance(exc, BadRequestError):
if isinstance(exc, bad_request):
msg = str(exc).lower()
if any(x in msg for x in _OVERFLOW_EXCLUSIONS):
return False
+4 -2
View File
@@ -8,8 +8,6 @@ import logging
from functools import lru_cache
from typing import Any
import litellm
from strix.config import load_settings
@@ -38,6 +36,8 @@ def _lookup_key(model: str) -> str:
def _safe_get_model_info(model: str) -> dict[str, Any] | None:
try:
import litellm
return dict(litellm.get_model_info(model))
except Exception: # noqa: BLE001 - unmapped models raise; caller falls back.
return None
@@ -82,6 +82,8 @@ def count_tokens(model: str, text: str) -> int:
if not text:
return 0
try:
import litellm
return int(litellm.token_counter(model=_lookup_key(model), text=text))
except Exception: # noqa: BLE001 - tokenizer may be unavailable for some models.
return len(text.encode("utf-8"))
+55
View File
@@ -0,0 +1,55 @@
"""Background pre-import of the heavy scan dependencies.
The scan engine's import graph (the agents SDK, OpenAI client, LiteLLM, the
Caido SDK, the Docker SDK) costs seconds to import cold, but none of it is
needed until a scan actually starts. Importing it on a daemon thread at CLI
entry overlaps that cost with the I/O-bound startup work that always precedes
a scan (argument parsing, Docker checks, image pull, TUI setup), so by the
time the scan begins the modules are already in ``sys.modules``. Any thread
that needs one of them before the warm-up finishes just blocks on the normal
import lock, so behaviour is unchanged either way.
"""
from __future__ import annotations
import importlib
import logging
import threading
logger = logging.getLogger(__name__)
WARMUP_MODULES = (
"strix.core.runner",
"litellm",
"caido_sdk_client",
"docker",
)
_lock = threading.Lock()
_thread: threading.Thread | None = None
def _warm(modules: tuple[str, ...]) -> None:
for name in modules:
try:
importlib.import_module(name)
except Exception: # noqa: BLE001 - a failed warm-up must never fail the run.
logger.debug("Import warm-up for %r failed", name, exc_info=True)
def start_import_warmup(modules: tuple[str, ...] = WARMUP_MODULES) -> threading.Thread:
"""Start importing the heavy scan dependencies in the background, once.
``modules`` lets embedders that never touch some backends (e.g. a cloud
runtime that has no local Docker) warm a narrower set.
"""
global _thread # noqa: PLW0603
with _lock:
if _thread is not None:
return _thread
_thread = threading.Thread(
target=_warm, args=(modules,), name="strix-import-warmup", daemon=True
)
_thread.start()
return _thread
+443
View File
@@ -0,0 +1,443 @@
"""``coverage.json`` — the negative space of a scan, with provenance.
A findings list answers "what is wrong". It cannot answer "what did you
check", and in a compliance context that second question is the one that
decides whether a clean result means anything: an auditor reading zero SQL
injection findings cannot tell "tested fourteen endpoints, all parameterized"
apart from "never looked".
This module assembles the artifact that answers it. Two kinds of statement go
in, and they are kept apart on purpose:
- ``agent_reported`` the coverage ledger (:mod:`strix.tools.coverage.tools`).
Rich and specific, but it is an agent's account of its own work.
- ``machine_observed`` facts the runtime recorded regardless of what any
agent claimed: which agents ran and how they terminated, which skills they
carried, how many findings were filed, whether the run finished or was cut
short.
A coverage claim is an attestation, so conflating the two would be the worst
possible failure: a hallucinated "tested and clean" is strictly less honest
than no coverage record at all. Every entry therefore carries its ``source``,
and machine-observed facts contradict rather than confirm an agent that
carried the ``sql_injection`` skill and recorded nothing about SQL injection
shows up under ``gaps``, and a run that hit its budget ceiling is stamped
``complete: false`` no matter how tidy the ledger looks.
"""
from __future__ import annotations
import json
import logging
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any
from strix.report.writer import atomic_write_text
from strix.skills import get_available_skills
if TYPE_CHECKING:
from pathlib import Path
logger = logging.getLogger(__name__)
COVERAGE_FILENAME = "coverage.json"
COVERAGE_SCHEMA_VERSION = 1
#: Ledger outcomes rendered for a reader who has never seen our enum.
OUTCOME_LABELS: dict[str, str] = {
"reported": "Finding reported",
"no_issue_found": "No issue identified",
"ruled_out": "Ruled out",
"not_applicable": "Not applicable",
"needs_follow_up": "Requires further review",
}
#: Statuses that mean the agent stopped early rather than finishing its task.
_INCOMPLETE_AGENT_STATUSES = frozenset({"crashed", "stopped", "running", "waiting"})
#: Run statuses that mean the scan itself did not run to completion.
_INCOMPLETE_RUN_STATUSES = frozenset({"failed", "interrupted", "stopped", "running"})
#: Only this skill category names a vulnerability class. ``tooling`` and
#: ``reconnaissance`` skills describe how an agent works, not what it hunts,
#: so holding one implies no coverage obligation.
_RISK_SKILL_CATEGORY = "vulnerabilities"
#: How each vulnerability skill can legitimately appear in a ledger row.
#:
#: Matching a skill to a row is textual, and a skill's filename is not how a
#: pentester writes the class down: an agent carrying ``path_traversal_lfi_rfi``
#: records "Path Traversal", and one carrying ``weak_password_detection``
#: records "weak password policy". A row matches when it contains every word
#: of *any one* phrasing here. Skills absent from this map fall back to their
#: own words, so a new skill is merely matched strictly, never crashed on —
#: but add an entry, because a false gap asserts something untrue in a report.
_SKILL_PHRASINGS: dict[str, tuple[str, ...]] = {
"agentic_system_security": (
"agentic",
"agent tool",
"mcp",
"confused deputy",
"tool invocation",
),
"argument_injection": ("argument injection", "option injection", "argv"),
"authentication_jwt": ("authentication", "jwt", "session"),
"broken_function_level_authorization": (
"function level authorization",
"authorization",
"access control",
"privilege escalation",
),
"browser_security": (
"browser",
"postmessage",
"xs leak",
"service worker",
"cross origin state",
),
"business_logic": ("business logic", "logic flaw"),
"csrf": ("csrf", "cross site request forgery"),
"header_injection": ("header injection", "host header", "crlf"),
"http_request_smuggling": ("request smuggling", "desync"),
"idor": ("idor", "object level authorization", "bola", "direct object reference"),
"information_disclosure": (
"information disclosure",
"information leak",
"sensitive data",
"data exposure",
),
"insecure_deserialization": ("deserialization",),
"insecure_file_uploads": ("file upload",),
"llm_prompt_injection": ("prompt injection",),
"mass_assignment": ("mass assignment", "parameter binding"),
"nosql_injection": ("nosql",),
"open_redirect": ("redirect",),
"path_traversal_lfi_rfi": (
"path traversal",
"directory traversal",
"file inclusion",
"lfi",
"rfi",
),
"prototype_pollution": ("prototype pollution",),
"race_conditions": ("race condition", "toctou"),
"rce": ("rce", "remote code execution", "code execution", "command injection"),
"semantic_confusion": (
"semantic confusion",
"parser differential",
"normalization",
"validator sink mismatch",
),
"sql_injection": ("sql injection", "sqli"),
"ssrf": ("ssrf", "server side request forgery"),
"ssti": ("ssti", "template injection"),
"subdomain_takeover": ("subdomain takeover",),
"weak_password_detection": ("password", "credential", "brute force"),
"xss": ("xss", "cross site scripting", "script injection"),
"xxe": ("xxe", "xml external entity", "xml entity"),
}
def read_agent_graph(state_dir: Path) -> dict[str, Any]:
"""Load the coordinator's snapshot, or ``{}`` when it isn't readable.
The snapshot is the runtime's own record of the agent tree, written on
every graph mutation. Reading it here (rather than holding a coordinator
reference) keeps artifact assembly usable from a finished or resumed run,
where the live coordinator is gone but the file is still on disk.
"""
path = state_dir / "agents.json"
if not path.is_file():
return {}
try:
data = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
logger.warning("agent graph snapshot at %s is unreadable", path, exc_info=True)
return {}
return data if isinstance(data, dict) else {}
def _normalized(text: str) -> str:
"""Lowercase *text* with punctuation flattened to spaces, for matching."""
return "".join(char if char.isalnum() else " " for char in text.lower())
def _skill_leaf(skill: str) -> str:
return skill.rsplit("/", maxsplit=1)[-1].strip().lower()
def _risk_skill_names() -> frozenset[str]:
"""Bare names of every skill that denotes a vulnerability class."""
try:
entries = get_available_skills().get(_RISK_SKILL_CATEGORY, [])
return frozenset(entry["name"] for entry in entries if entry.get("name"))
except OSError:
logger.warning("could not enumerate skills for coverage gaps", exc_info=True)
return frozenset()
def agents_from_graph(graph: dict[str, Any]) -> list[dict[str, Any]]:
"""Flatten the coordinator snapshot into one record per agent."""
statuses = graph.get("statuses")
if not isinstance(statuses, dict):
return []
raw_names = graph.get("names")
names: dict[str, Any] = raw_names if isinstance(raw_names, dict) else {}
raw_metadata = graph.get("metadata")
metadata: dict[str, Any] = raw_metadata if isinstance(raw_metadata, dict) else {}
raw_parents = graph.get("parent_of")
parents: dict[str, Any] = raw_parents if isinstance(raw_parents, dict) else {}
# Only an unambiguous root earns the exemption below. A snapshot with no
# parent links at all makes every agent look parentless, and excusing all
# of them would silently delete the silent-agent check.
parentless = [agent_id for agent_id in statuses if not parents.get(agent_id)]
root_id = parentless[0] if len(parentless) == 1 else None
agents: list[dict[str, Any]] = []
for agent_id, status in statuses.items():
raw_meta = metadata.get(agent_id)
meta: dict[str, Any] = raw_meta if isinstance(raw_meta, dict) else {}
raw_skills = meta.get("skills")
skills: list[Any] = raw_skills if isinstance(raw_skills, list) else []
agents.append(
{
"agent_id": agent_id,
"agent_name": names.get(agent_id) or agent_id,
"status": str(status),
"skills": [str(skill) for skill in skills],
"task": str(meta.get("task") or ""),
"is_root": agent_id == root_id,
}
)
agents.sort(key=lambda agent: str(agent["agent_name"]))
return agents
def _skill_phrasings(skill: str) -> list[list[str]]:
"""Word lists that would each count as a ledger row naming *skill*."""
phrasings = _SKILL_PHRASINGS.get(skill) or (skill,)
return [terms for phrase in phrasings if (terms := _normalized(phrase).split())]
def _entry_is_about(entry: dict[str, Any], phrasings: list[list[str]]) -> bool:
"""True when a ledger row plausibly concerns any phrasing of a risk class."""
haystack = _normalized(f"{entry.get('risk_area', '')} {entry.get('surface', '')}")
return any(all(term in haystack for term in terms) for terms in phrasings)
def skill_coverage_gaps(
entries: list[dict[str, Any]], agents: list[dict[str, Any]]
) -> list[dict[str, Any]]:
"""Vulnerability classes an agent was equipped for but never recorded.
A skill assigned to an agent is a declaration of intent that the runtime
observed independently of anything the agent later said. When no ledger
row mentions that class, the class is unaccounted for which is a very
different report line from "tested, nothing found".
"""
risk_skills = _risk_skill_names()
if not risk_skills:
return []
carriers: dict[str, list[str]] = {}
for agent in agents:
for skill in agent["skills"]:
leaf = _skill_leaf(skill)
if leaf in risk_skills:
carriers.setdefault(leaf, []).append(str(agent["agent_name"]))
gaps: list[dict[str, Any]] = []
for skill, agent_names in sorted(carriers.items()):
phrasings = _skill_phrasings(skill)
if any(_entry_is_about(entry, phrasings) for entry in entries):
continue
gaps.append(
{
"kind": "unrecorded_risk_class",
"risk_area": skill.replace("_", " "),
"detail": (
f"Agent(s) {', '.join(sorted(set(agent_names)))} were assigned the "
f"'{skill}' skill, but no coverage entry records this class being "
"assessed. Treat it as unexamined, not as clean."
),
}
)
return gaps
def _silent_agent_gaps(
entries: list[dict[str, Any]], agents: list[dict[str, Any]]
) -> list[dict[str, Any]]:
"""Agents that ran and recorded nothing at all.
The root agent is exempt while it has children: it delegates and
reconciles rather than testing, so flagging it on every clean scan would
put a permanent false line in the report and teach readers to skip the
section. A root that ran alone tested alone, and is held to the rule.
"""
recorded_ids = {str(entry.get("agent_id")) for entry in entries if entry.get("agent_id")}
delegated = len(agents) > 1
gaps: list[dict[str, Any]] = []
for agent in agents:
if agent["agent_id"] in recorded_ids or (agent["is_root"] and delegated):
continue
gaps.append(
{
"kind": "agent_recorded_no_coverage",
"agent_name": agent["agent_name"],
"detail": (
f"{agent['agent_name']} ran (status: {agent['status']}) without "
"recording any coverage. Whatever it examined is absent from this "
"record."
),
}
)
return gaps
def _unresolved_gaps(entries: list[dict[str, Any]]) -> list[dict[str, Any]]:
"""Ledger rows the agents themselves left open."""
return [
{
"kind": "needs_follow_up",
"surface": entry.get("surface", ""),
"risk_area": entry.get("risk_area", ""),
"detail": str(entry.get("evidence") or "Left open without a stated reason."),
}
for entry in entries
if entry.get("outcome") == "needs_follow_up"
]
def _completeness(
run_record: dict[str, Any],
agents: list[dict[str, Any]],
exit_reason: str | None,
) -> dict[str, Any]:
"""Whether this record can be read as a complete account of the scan.
Any of these makes it partial, and the caveats say which: the run did not
reach ``completed``, an agent was still live or died when the scan ended,
or the run stopped for a reason other than the root agent deciding it was
done (budget ceilings are the common case).
"""
status = str(run_record.get("status") or "unknown")
caveats: list[str] = []
if status in _INCOMPLETE_RUN_STATUSES:
caveats.append(
f"The scan ended with status '{status}' rather than completing, so coverage "
"reflects only the work finished before it stopped."
)
unfinished = [agent for agent in agents if agent["status"] in _INCOMPLETE_AGENT_STATUSES]
if unfinished:
names = ", ".join(sorted(str(agent["agent_name"]) for agent in unfinished))
caveats.append(
f"{len(unfinished)} agent(s) did not finish cleanly ({names}); any surface they "
"held is under-covered."
)
if exit_reason and exit_reason not in {"finished_by_tool", "completed"}:
caveats.append(
f"The run terminated via '{exit_reason}' rather than the root agent finishing, "
"so remaining scope was not reached."
)
return {
"complete": not caveats,
"scan_status": status,
"exit_reason": exit_reason,
"caveats": caveats,
}
def _outcome_counts(entries: list[dict[str, Any]]) -> dict[str, int]:
counts: dict[str, int] = {}
for entry in entries:
outcome = str(entry.get("outcome", ""))
counts[outcome] = counts.get(outcome, 0) + 1
return {label: counts[label] for label in OUTCOME_LABELS if label in counts}
def build_coverage_document(
*,
run_record: dict[str, Any],
entries: list[dict[str, Any]],
agent_graph: dict[str, Any],
vulnerability_reports: list[dict[str, Any]],
exit_reason: str | None = None,
) -> dict[str, Any]:
"""Assemble the ``coverage.json`` document."""
agents = agents_from_graph(agent_graph)
skills_exercised = sorted(
{_skill_leaf(skill) for agent in agents for skill in agent["skills"] if skill}
)
ledger = [
{
"surface": entry.get("surface", ""),
"risk_area": entry.get("risk_area", ""),
"outcome": entry.get("outcome", ""),
"outcome_label": OUTCOME_LABELS.get(str(entry.get("outcome", "")), ""),
"evidence": entry.get("evidence", ""),
"recorded_by": entry.get("agent_name", ""),
"recorded_at": entry.get("created_at", ""),
"updated_at": entry.get("updated_at", ""),
"previous_outcomes": [
str(previous.get("outcome", ""))
for previous in entry.get("history", [])
if isinstance(previous, dict)
],
"source": "agent_reported",
}
for entry in entries
]
gaps = [
*_unresolved_gaps(entries),
*skill_coverage_gaps(entries, agents),
*_silent_agent_gaps(entries, agents),
]
return {
"schema_version": COVERAGE_SCHEMA_VERSION,
"generated_at": datetime.now(UTC).strftime("%Y-%m-%d %H:%M:%S UTC"),
"run_id": run_record.get("run_id"),
"run_name": run_record.get("run_name"),
"scope": {
"targets": run_record.get("targets_info") or [],
"scan_mode": run_record.get("scan_mode"),
"scope_mode": run_record.get("scope_mode"),
"diff_scope": run_record.get("diff_scope"),
"instruction": run_record.get("instruction") or "",
},
"summary": {
"surfaces_reviewed": len(ledger),
"outcomes": _outcome_counts(entries),
"findings_filed": len(vulnerability_reports),
"gaps": len(gaps),
},
"machine_observed": {
"agents": agents,
"skills_exercised": skills_exercised,
"findings_filed": len(vulnerability_reports),
"source": "runtime",
},
"completeness": _completeness(run_record, agents, exit_reason),
"entries": ledger,
"gaps": gaps,
}
def write_coverage(run_dir: Path, document: dict[str, Any]) -> Path:
"""Write ``coverage.json`` into the run directory and return its path."""
path = run_dir / COVERAGE_FILENAME
atomic_write_text(path, json.dumps(document, ensure_ascii=False, indent=2, default=str))
logger.info(
"Saved coverage record to: %s (%d surface(s), %d gap(s))",
path,
len(document.get("entries", [])),
len(document.get("gaps", [])),
)
return path
+134
View File
@@ -40,6 +40,10 @@ Design notes:
* Findings without safe locations still appear in the SARIF output,
anchored to SECURITY.md and flagged via
``properties.synthetic_location`` rather than being dropped silently.
* Coverage rides in the same document as non-failing results (``kind`` of
``pass`` / ``notApplicable`` / ``open``), and run completeness on
``run.invocations``. Consumers that only want alerts filter on
``kind == "fail"`` and are unaffected.
"""
from __future__ import annotations
@@ -199,6 +203,7 @@ def build_sarif_report(
*,
tool_version: str | None = None,
repository_context: dict[str, Any] | None = None,
coverage: dict[str, Any] | None = None,
) -> dict[str, Any]:
"""Return a SARIF 2.1.0 document for findings.
@@ -209,6 +214,11 @@ def build_sarif_report(
can bind alerts to the scanned commit; it is omitted for URL / IP
(DAST) targets that have no repository.
``coverage`` (optional) is the document from
:func:`strix.report.coverage.build_coverage_document`: its cleared
surfaces become non-failing results and its completeness caveats become
invocation notifications.
Findings without safe source locations are anchored synthetically
to SECURITY.md and flagged via ``properties.synthetic_location``.
They're still emitted as proper SARIF results so they (a) flow
@@ -247,6 +257,9 @@ def build_sarif_report(
)
)
if coverage:
_append_coverage(coverage, rules_by_id, rule_index_by_id, results)
driver: dict[str, Any] = {
"name": TOOL_NAME,
"informationUri": TOOL_INFORMATION_URI,
@@ -260,6 +273,9 @@ def build_sarif_report(
"results": results,
}
if coverage:
run["invocations"] = [_coverage_invocation(coverage)]
run_properties: dict[str, Any] = {}
if synthetic_location_count:
# Surface the count for observability without duplicating the
@@ -292,6 +308,7 @@ def write_sarif_report(
*,
tool_version: str | None = None,
repository_context: dict[str, Any] | None = None,
coverage: dict[str, Any] | None = None,
) -> None:
"""Write a SARIF report to disk, creating parent directories first.
@@ -304,6 +321,7 @@ def write_sarif_report(
vulnerability_reports,
tool_version=tool_version,
repository_context=repository_context,
coverage=coverage,
)
tmp_path = output_path.with_name(f"{output_path.name}.{os.getpid()}.tmp")
try:
@@ -321,6 +339,7 @@ def write_sarif(
*,
tool_version: str | None = None,
repository_context: dict[str, Any] | None = None,
coverage: dict[str, Any] | None = None,
filename: str = "findings.sarif",
) -> Path:
"""Write ``findings.sarif`` alongside existing outputs in ``run_dir``.
@@ -335,6 +354,7 @@ def write_sarif(
reports,
tool_version=tool_version,
repository_context=repository_context,
coverage=coverage,
)
logger.info(
"Wrote SARIF 2.1.0 report: %s (%d results)",
@@ -526,6 +546,11 @@ def _result_properties(
"impact",
"technical_analysis",
"remediation_steps",
"counterevidence",
"confidence",
"confidence_rationale",
"severity_change_conditions",
"fix_verification",
):
value = report.get(key)
if value not in (None, ""):
@@ -613,6 +638,115 @@ def _build_fixes(report: dict[str, Any]) -> list[dict[str, Any]] | None:
return [fix]
# ---------------------------------------------------------------------------
# Coverage
# ---------------------------------------------------------------------------
_COVERAGE_RULE_PREFIX = "strix-coverage"
# ``reported`` is absent on purpose: those surfaces are already in ``results``
# as ``fail`` findings.
_OUTCOME_TO_KIND = {
"no_issue_found": "pass",
"ruled_out": "pass",
"not_applicable": "notApplicable",
"needs_follow_up": "open",
}
def _coverage_rule_id(risk_area: str) -> str:
slug = _slugify(risk_area) or "unspecified"
return f"{_COVERAGE_RULE_PREFIX}/{slug}"
def _build_coverage_rule(rule_id: str, risk_area: str) -> dict[str, Any]:
description = f"Coverage of {risk_area} across the assessed attack surface."
return {
"id": rule_id,
"name": _rule_name(rule_id, risk_area),
"shortDescription": {"text": f"Coverage: {risk_area}"},
"fullDescription": {"text": description},
"defaultConfiguration": {"level": "none"},
"help": {"text": description, "markdown": description},
"properties": {"tags": ["coverage"]},
}
def _build_coverage_result(
rule_id: str,
rule_index: int,
kind: str,
entry: dict[str, Any],
) -> dict[str, Any]:
surface = _string_value(entry.get("surface")) or "unspecified surface"
risk_area = _string_value(entry.get("risk_area")) or "unspecified risk"
evidence = _string_value(entry.get("evidence"))
label = _string_value(entry.get("outcome_label")) or str(entry.get("outcome", ""))
message = f"{risk_area}{label}: {surface}"
if evidence:
message = f"{message}\n\n{evidence}"
result: dict[str, Any] = {
"ruleId": rule_id,
"ruleIndex": rule_index,
"kind": kind,
# SARIF requires ``level: none`` for any result whose kind is not ``fail``.
"level": "none",
"message": {"text": message},
"locations": [{"logicalLocations": [{"fullyQualifiedName": surface}]}],
"properties": {
"strix": {
"coverage_outcome": entry.get("outcome", ""),
"risk_area": risk_area,
"surface": surface,
"recorded_by": entry.get("recorded_by", ""),
"source": entry.get("source", "agent_reported"),
}
},
}
return result
def _append_coverage(
coverage: dict[str, Any],
rules_by_id: dict[str, dict[str, Any]],
rule_index_by_id: dict[str, int],
results: list[dict[str, Any]],
) -> None:
entries = coverage.get("entries")
if not isinstance(entries, list):
return
for entry in entries:
if not isinstance(entry, dict):
continue
kind = _OUTCOME_TO_KIND.get(str(entry.get("outcome", "")))
if kind is None:
continue
rule_id = _coverage_rule_id(str(entry.get("risk_area", "")))
if rule_id not in rules_by_id:
rule_index_by_id[rule_id] = len(rules_by_id)
rules_by_id[rule_id] = _build_coverage_rule(
rule_id, _string_value(entry.get("risk_area")) or "unspecified risk"
)
results.append(_build_coverage_result(rule_id, rule_index_by_id[rule_id], kind, entry))
def _coverage_invocation(coverage: dict[str, Any]) -> dict[str, Any]:
"""``executionSuccessful: false`` stops a truncated run reading as a clean one."""
completeness = coverage.get("completeness")
completeness = completeness if isinstance(completeness, dict) else {}
caveats = completeness.get("caveats")
caveats = caveats if isinstance(caveats, list) else []
invocation: dict[str, Any] = {"executionSuccessful": bool(completeness.get("complete", True))}
if caveats:
invocation["toolExecutionNotifications"] = [
{"level": "warning", "message": {"text": str(caveat)}} for caveat in caveats
]
return invocation
# ---------------------------------------------------------------------------
# Location handling
# ---------------------------------------------------------------------------
+59 -1
View File
@@ -13,7 +13,8 @@ from agents.usage import Usage
from strix.config import codex
from strix.config.loader import load_settings
from strix.core.paths import run_dir_for
from strix.core.paths import run_dir_for, runtime_state_dir
from strix.report.coverage import write_coverage
from strix.report.pricing import resolve_litellm_model
from strix.report.sarif import write_sarif
from strix.report.usage import LLMUsageLedger
@@ -237,6 +238,10 @@ class ReportState:
remediation_steps: str | None = None,
evidence: str | None = None,
assumptions: str | None = None,
counterevidence: str | None = None,
confidence: str | None = None,
confidence_rationale: str | None = None,
severity_change_conditions: str | None = None,
fix_effort: str | None = None,
cvss: float | None = None,
cvss_breakdown: dict[str, str] | None = None,
@@ -245,6 +250,7 @@ class ReportState:
cve: str | None = None,
cwe: str | None = None,
code_locations: list[dict[str, Any]] | None = None,
fix_verification: str | None = None,
fix_pr_body: str | None = None,
finding_class: str | None = None,
dependency_metadata: dict[str, str] | None = None,
@@ -278,6 +284,14 @@ class ReportState:
report["evidence"] = evidence.strip()
if assumptions:
report["assumptions"] = assumptions.strip()
if counterevidence:
report["counterevidence"] = counterevidence.strip()
if confidence:
report["confidence"] = confidence.strip().lower()
if confidence_rationale:
report["confidence_rationale"] = confidence_rationale.strip()
if severity_change_conditions:
report["severity_change_conditions"] = severity_change_conditions.strip()
if fix_effort:
report["fix_effort"] = fix_effort.strip().lower()
if cvss is not None:
@@ -294,6 +308,8 @@ class ReportState:
report["cwe"] = cwe.strip()
if code_locations:
report["code_locations"] = code_locations
if fix_verification:
report["fix_verification"] = fix_verification.strip()
if fix_pr_body:
report["fix_pr_body"] = fix_pr_body.strip()
report["finding_class"] = (finding_class or "dynamic").strip().lower()
@@ -388,6 +404,18 @@ class ReportState:
posthog.end(self, exit_reason="finished_by_tool")
scarf.end(self, exit_reason="finished_by_tool")
def record_mcp_connections(self, names: list[str]) -> None:
"""Note the MCP servers this run connected, and persist it.
Saved as soon as the run connects rather than at the end, so an interface
reading the record mid-run can already attribute a tool call to the
server it went out to.
"""
if self.run_record.get("mcp_connections") == names:
return
self.run_record["mcp_connections"] = names
self.save_run_data()
def set_scan_config(self, config: dict[str, Any]) -> None:
self.scan_config = config
self.run_record["status"] = "running"
@@ -447,12 +475,41 @@ class ReportState:
{str(scan_results.get("recommendations", "")).strip()}
"""
def _coverage_document(self) -> dict[str, Any] | None:
"""Assemble the coverage record, or None when it can't be built.
Coverage is a secondary artifact: a failure here must not cost the
caller its findings, so this swallows and logs rather than raising
into :meth:`_save_artifacts`.
"""
try:
from strix.report.coverage import build_coverage_document, read_agent_graph
from strix.tools.coverage.tools import get_coverage_entries
return build_coverage_document(
run_record=self.run_record,
entries=get_coverage_entries(),
agent_graph=read_agent_graph(runtime_state_dir(self.get_run_dir())),
vulnerability_reports=self.vulnerability_reports,
exit_reason=self.scan_ended_exit_reason,
)
except Exception:
logger.exception("coverage document build failed (non-fatal)")
return None
def _save_artifacts(self) -> None:
"""Write scan artifacts under ``run_dir``."""
run_dir = self.get_run_dir()
try:
run_dir.mkdir(parents=True, exist_ok=True)
coverage = self._coverage_document()
if coverage is not None:
try:
write_coverage(run_dir, coverage)
except OSError:
logger.exception("coverage.json write failed (non-fatal)")
if self.final_scan_result:
write_executive_report(run_dir, self.final_scan_result)
@@ -471,6 +528,7 @@ class ReportState:
self.vulnerability_reports,
tool_version=_strix_version(),
repository_context=self._sarif_repository_context(),
coverage=coverage,
)
except Exception:
logger.exception("SARIF emit failed (non-fatal; CSV/MD unaffected)")
+28 -5
View File
@@ -107,7 +107,7 @@ def read_run_record(run_dir: Path) -> dict[str, Any]:
def write_run_record(run_dir: Path, run_record: dict[str, Any]) -> None:
_atomic_write_text(
atomic_write_text(
run_record_path(run_dir),
json.dumps(run_record, ensure_ascii=False, indent=2, default=str),
)
@@ -133,7 +133,7 @@ def write_vulnerabilities(
new_reports = [r for r in vulnerability_reports if r["id"] not in saved_vuln_ids]
for report in new_reports:
_atomic_write_text(
atomic_write_text(
vuln_dir / f"{report['id']}.md",
render_vulnerability_md(report),
)
@@ -158,9 +158,9 @@ def write_vulnerabilities(
"file": f"vulnerabilities/{report['id']}.md",
},
)
_atomic_write_text(csv_path, csv_buf.getvalue())
atomic_write_text(csv_path, csv_buf.getvalue())
_atomic_write_text(
atomic_write_text(
run_dir / "vulnerabilities.json",
json.dumps(vulnerability_reports, ensure_ascii=False, indent=2, default=str),
)
@@ -175,7 +175,8 @@ def write_vulnerabilities(
return len(new_reports)
def _atomic_write_text(path: Path, payload: str) -> None:
def atomic_write_text(path: Path, payload: str) -> None:
"""Write *payload* to *path* via a sibling temp file and an atomic rename."""
path.parent.mkdir(parents=True, exist_ok=True)
with tempfile.NamedTemporaryFile(
mode="w",
@@ -220,6 +221,8 @@ def render_vulnerability_md(report: dict[str, Any]) -> str: # noqa: PLR0912, PL
metadata.append(("Advisory CVSS", advisory_cvss))
if dep_meta.get("contextual_cvss_vector"):
metadata.append(("Contextual CVSS Vector", dep_meta["contextual_cvss_vector"]))
if report.get("confidence"):
metadata.append(("Confidence", str(report["confidence"]).title()))
if report.get("fix_effort"):
metadata.append(("Fix Effort", str(report["fix_effort"]).title()))
for label, value in metadata:
@@ -241,6 +244,21 @@ def render_vulnerability_md(report: dict[str, Any]) -> str: # noqa: PLR0912, PL
lines.append(str(report["impact"]))
lines.append("")
if report.get("counterevidence"):
lines.append("## Counterevidence\n")
lines.append(str(report["counterevidence"]))
lines.append("")
if report.get("confidence_rationale"):
lines.append("## Confidence Rationale\n")
lines.append(str(report["confidence_rationale"]))
lines.append("")
if report.get("severity_change_conditions"):
lines.append("## What Would Change This Severity\n")
lines.append(str(report["severity_change_conditions"]))
lines.append("")
if report.get("technical_analysis"):
lines.append("## Technical Analysis\n")
lines.append(str(report["technical_analysis"]))
@@ -299,6 +317,11 @@ def render_vulnerability_md(report: dict[str, Any]) -> str: # noqa: PLR0912, PL
lines.append(str(report["remediation_steps"]))
lines.append("")
if report.get("fix_verification"):
lines.append("## Fix Verification\n")
lines.append(str(report["fix_verification"]))
lines.append("")
if report.get("assumptions"):
lines.append("## Assumptions\n")
lines.append(str(report["assumptions"]))
+12 -6
View File
@@ -15,12 +15,10 @@ import json
import logging
from typing import TYPE_CHECKING
from caido_sdk_client import Client, TokenAuthOptions
from caido_sdk_client.types import CreateProjectOptions
if TYPE_CHECKING:
from agents.sandbox.session import BaseSandboxSession
from caido_sdk_client import Client
logger = logging.getLogger(__name__)
@@ -87,20 +85,28 @@ async def bootstrap_caido(
container_url: str,
) -> Client:
"""Connect to the in-container Caido sidecar and select a fresh project."""
# The Caido SDK (and its generated GraphQL schema) is slow to import and is
# only needed once a sandbox is actually being bootstrapped, so it is
# imported here rather than at module scope.
from caido_sdk_client import Client, TokenAuthOptions
from caido_sdk_client.types import CreateProjectOptions
logger.info("Bootstrapping Caido client (host=%s, container=%s)", host_url, container_url)
access_token = await _login_as_guest(session, container_url=container_url)
client = Client(host_url, auth=TokenAuthOptions(token=access_token))
await client.connect()
try:
# connect() is inside the guard as well: a cancellation there (scan
# teardown while the bootstrap is still in flight) would otherwise
# leave the half-connected transport behind.
await client.connect()
project = await client.project.create(
CreateProjectOptions(name="sandbox", temporary=True),
)
await client.project.select(project.id)
except BaseException:
# The connected client never reaches the session bundle if project
# The client never reaches the session bundle if connect or project
# setup fails, so close it here to avoid leaking the transport.
with contextlib.suppress(Exception):
await client.aclose()
+60
View File
@@ -0,0 +1,60 @@
"""Handle for a Caido bootstrap running concurrently with the scan start.
The Caido sidecar login + project setup costs a couple of seconds of
guest-side polling, and nothing needs the client until the first proxy
tool call (or the first traffic poll). :class:`CaidoBootstrapHandle`
wraps the in-flight bootstrap task so session bring-up can return as
soon as the container is up; consumers resolve the client at first use.
"""
from __future__ import annotations
import asyncio
import contextlib
import logging
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from caido_sdk_client import Client
logger = logging.getLogger(__name__)
class CaidoBootstrapHandle:
"""Resolves to the connected Caido client once the bootstrap finishes.
A failed bootstrap is surfaced (once) to every ``get()`` caller as the
original exception; proxy tools degrade to their "client unavailable"
result instead of the failure killing the scan at bring-up.
"""
def __init__(self, task: asyncio.Task[Client]) -> None:
self._task = task
async def get(self) -> Client:
"""Wait for the bootstrap and return the client.
Shielded so one caller's cancellation (e.g. a tool timeout) does not
cancel the shared bootstrap for everyone else.
"""
return await asyncio.shield(self._task)
def peek(self) -> Client | None:
"""Return the client if the bootstrap already finished cleanly."""
if self._task.done() and not self._task.cancelled() and self._task.exception() is None:
return self._task.result()
return None
async def aclose(self) -> None:
"""Cancel an in-flight bootstrap or close the finished client."""
if not self._task.done():
self._task.cancel()
with contextlib.suppress(asyncio.CancelledError, Exception):
await self._task
return
client = self.peek()
if client is not None:
with contextlib.suppress(Exception):
await client.aclose()
+15 -4
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import asyncio
import logging
import os
import sys
@@ -15,6 +16,7 @@ from strix.config import load_settings
from strix.core.paths import run_dir_for, runtime_state_dir
from strix.runtime.backends import backend_supports_bind_mounts, get_backend
from strix.runtime.caido_bootstrap import bootstrap_caido
from strix.runtime.caido_handle import CaidoBootstrapHandle
if TYPE_CHECKING:
@@ -333,10 +335,19 @@ async def create_or_reuse(
host_caido_url = f"{scheme}://{caido_endpoint.host}:{caido_endpoint.port}"
logger.debug("Caido host endpoint resolved: %s", host_caido_url)
caido_client = await bootstrap_caido(
session,
host_url=host_caido_url,
container_url=container_caido_url,
# The Caido login + project setup polls the guest for a couple of seconds
# and nothing needs the client before the first proxy tool call, so it
# runs concurrently with the rest of scan start; consumers resolve the
# handle at first use (see CaidoBootstrapHandle).
caido_client = CaidoBootstrapHandle(
asyncio.create_task(
bootstrap_caido(
session,
host_url=host_caido_url,
container_url=container_caido_url,
),
name=f"caido-bootstrap-{scan_id}",
)
)
bundle = {
+3
View File
@@ -43,10 +43,13 @@ Notable source-aware skills:
- `source_aware_sast` (custom): semgrep/AST/secrets/supply-chain static triage workflow
- `dependency_cve_scanning` (custom): trivy-based SCA workflow for reporting known dependency CVEs via `create_dependency_report`
- `npx_confusion` (custom): npx/npm exec/bunx fallback and adjacent package-runner identity confusion, with runner-specific registry and reporting gates
- `semantic_confusion` (vulnerabilities): cross-boundary parser, normalization, and representation mismatch analysis
- `agentic_system_security` (vulnerabilities): effective-authority and MCP/tool ecosystem security testing
- `browser_security` (vulnerabilities): browsing-context, postMessage, XS-Leaks, service-worker, and cross-origin state-machine testing
- `azure` (cloud): Azure and Microsoft Entra privilege, PIM, workload identity, and cross-plane escalation analysis
- `infrastructure_lifecycle` (reconnaissance): abandoned or mutable external dependencies such as update endpoints, MX, storage, and control domains
- `argument_injection` (vulnerabilities): shell-free CLI option smuggling, secondary argument-file parsing, and platform-specific argv transformation boundaries
- `electron_desktop_apps` (technologies): Electron renderer-to-native trust boundaries, preload/IPC exposure, and navigation analysis
Notable LLM security skills:
- `llm_applications` (technologies): end-to-end OWASP 2026 LLM01-LLM10 coverage across models, RAG, vectors, agents, tools, outputs, supply chain, and resource controls
+1 -1
View File
@@ -16,7 +16,7 @@ logger = logging.getLogger(__name__)
_FRONTMATTER_PATTERN = re.compile(r"^---\s*\n(?P<body>.*?)\n---\s*\n", re.DOTALL)
_INTERNAL_SKILL_CATEGORIES: frozenset[str] = frozenset({"scan_modes", "coordination"})
_INTERNAL_SKILL_CATEGORIES: frozenset[str] = frozenset({"scan_modes", "coordination", "analysis"})
_ROOT_SKILL_CATEGORY = "root"
_EXTRA_SKILL_DIRS: list[Path] = []
+185
View File
@@ -0,0 +1,185 @@
---
name: counterevidence
description: Closure discipline for security findings — what counts as proof of safety, what does not, and how to record an unresolved candidate instead of silently dropping it
---
# Counterevidence and Closure Discipline
Proving a bug is real is only half the job. The other half is proving a
candidate is *not* real — and that half is where both false positives and
false negatives come from.
This skill governs how you close a candidate. It applies to every
candidate you open, whether it came from a scanner, a code read, a crawl,
or a hunch.
## Three Closure States
Every candidate you open ends in exactly one of these. There is no fourth
state, and "I moved on" is not one of them.
**1. `confirmed`** — you have a working PoC or, in white-box, a complete
source → control → sink → impact trace plus evidence the path is
reachable. File it with `create_vulnerability_report`.
**2. `ruled_out`** — you can name the **specific control** that makes the
code safe, at a specific location, and you have checked that the control
actually runs on the attacker's path. "Named control" means you can
complete this sentence with concrete detail: *"This is safe because
`<control>` at `<file:line or observed behavior>` `<does what>` before
`<sink>`, on every path an attacker can reach."* If you cannot complete
that sentence, you are not in `ruled_out`.
**3. `open_proof_gap`** — the candidate is plausible, you could not
confirm it, and you also could not name a control that rules it out. This
is a legitimate, expected outcome. Record it with
`record_coverage(outcome="needs_follow_up")`, carry it up in
`agent_finish(open_items=[...])`, and reflect it in `counterevidence` /
`confidence_rationale` if you file a related report. Do **not** convert
it to `ruled_out` to tidy up your worklist.
The failure mode this exists to prevent: an agent reads code, feels
uncertain, and quietly closes the candidate. That is an
`open_proof_gap` being mislabelled as `ruled_out`, and it is how real
vulnerabilities get missed.
## What Does NOT Rule Out a Candidate
Each of these is a common, plausible-sounding reason to drop a candidate.
None of them is sufficient on its own.
**Generic trust in a library or helper.** "It uses a well-known
sanitizer / the framework escapes this / the ORM handles it" is not
counterevidence. You must confirm *that* call, with *those* arguments, in
*that* context. Escaping helpers are context-specific: an HTML escaper
does nothing in a JS or attribute context, a SQL identifier quoter is not
a value quoter, and a path joiner is not a containment check.
**A control that runs on a different path.** Middleware, a decorator, or
a guard that protects the common route does not protect a sibling route,
an internal caller, a batch/async job, or an admin alias that reaches the
same sink. Check the specific path.
**A control that runs at the wrong time.** Validation *before* a
redirect, canonicalization *after* a path is already materialized, a
containment check *after* extraction, or an ownership check *after* the
object was already fetched and returned — these are ordering bugs, not
controls. Establish that the control runs before the dangerous effect.
**A control that can fail open.** Hardening flags set inside a
`try`/`except` that swallows failures, a parser feature that a caller can
override, a factory or config object supplied by the caller, or a
allow-list that is empty by default — all leave the candidate alive.
**A safe sibling.** If one call site is correctly guarded, that says
nothing about the other call sites of the same helper. Never let a safe
instance close a vulnerable one, and never collapse multiple instances
into one candidate just because they share a root cause — each reachable
instance stands or falls on its own.
**Missing information.** "I could not find a caller", "I could not tell
if this is deployed", "I could not determine whether this route is
exposed", "I could not stand up the service" — every one of these is an
`open_proof_gap`, not proof of safety. Missing evidence is missing
evidence; it is not evidence of absence.
**Difficulty.** "The build failed", "it needs credentials I don't have",
"the service mesh isn't available" are reasons to record a proof gap and
move on to the next candidate — not reasons to mark it clean. Do not let
one hard environment setup consume the budget you need for sibling
candidates.
**Operator configurability.** "An operator *could* configure a filter",
"this is a documented feature", "it's off by default" are not controls.
What ships and what is reachable is what matters.
**Being internal.** Internal-only, admin-only, or authenticated-only
reduces severity — it does not make the finding unreal. Downgrade it;
do not delete it.
## Recording Closure
Closure is only useful if it is written down. Every surface you assess
gets a `record_coverage` entry:
- `confirmed` → outcome `reported`, once the report is filed.
- `ruled_out` → outcome `ruled_out`, with the named control in
`evidence`. If you cannot name it, this is not `ruled_out`.
- `open_proof_gap` → outcome `needs_follow_up`, with the specific gap in
`evidence`.
- Tested thoroughly with nothing to show for it → `no_issue_found`.
- The risk cannot apply to this surface at all → `not_applicable`, with
the reason.
A scan that records only findings cannot tell the reader what was
reviewed and cleared, which makes every clean area indistinguishable
from an unvisited one.
Closure is not permanent. The ledger is shared across every agent, and
a surface someone left at `needs_follow_up` is an invitation: if you
had the credentials, the running service, or the reachability proof
they lacked, move their entry with `update_coverage` rather than
recording a parallel one. This runs both ways — a `ruled_out` whose
named control does not cover the path you just found goes back to
`reported` or `needs_follow_up`, with what changed in `evidence`. The
previous state is kept as history, so correcting the record costs
nothing and leaving it wrong costs a finding.
## What DOES Rule Out a Candidate
- You executed the attack and it demonstrably failed, and you understand
*why* it failed (not just that the response was a 403).
- You can point at the control, at a location, and show it runs on every
attacker-reachable path to the sink, before the effect, without a
fail-open branch.
- The sink is not actually dangerous in this context, and you can say
what makes it inert.
- The input is not actually attacker-controlled, and you traced it to a
trusted origin rather than assuming it.
Negative controls make a `ruled_out` much stronger: send the payload that
*should* work if the bug were real, and show it is blocked, while a
benign variant succeeds. That distinguishes "the control works" from "the
endpoint is broken/unreachable for unrelated reasons".
## Before You File a Report
Run this pass on every finding before calling
`create_vulnerability_report`:
1. **Argue the other side.** Spend real effort building the strongest
case that this is *not* exploitable, or not as severe as you think.
Look for the guard you might have missed, the deployment context that
constrains it, the precondition you assumed.
2. **Record what you found** in `counterevidence`. If you found a real
constraint, say what it is and why it does not neutralize the finding.
If you genuinely found nothing, say what you checked — "no input
validation, WAF, or authorization check was found on this path; tested
both authenticated and unauthenticated" — not just "none".
3. **Set `confidence` honestly.** A working PoC against a live target is
`high`. A complete static trace you could not execute is at best
`medium`, and `confidence_rationale` must name the gap. Do not inflate
confidence to make a finding look better; an accurate `medium` is far
more useful to the reader than a `high` that does not survive triage.
4. **State what would move the severity** in `severity_change_conditions`
— the one concrete piece of evidence that would raise or lower it
(e.g. "confirmation that this route is exposed to unauthenticated
internet traffic would raise this to critical").
## Reporting an Unconfirmed Candidate
Dynamic proof is the standard. But when you have a complete
source → control → sink → impact trace and runtime reproduction is
genuinely out of reach (no credentials, unavailable internal services, a
build that cannot run in the sandbox), a static-only finding is still
reportable — at `confidence: medium` or `low`, with the missing runtime
proof named explicitly in `confidence_rationale`.
What is **not** acceptable is a scanner hit with no trace, a "this
pattern is usually dangerous" claim, or a finding where you never
identified the attacker-controlled input. Those are not proof gaps, they
are non-findings.
If you are unsure whether a candidate clears this bar: it clears it if
you can name the input, the path, the missing or broken control, and the
effect. It does not if any one of those is a guess.
+129
View File
@@ -0,0 +1,129 @@
---
name: fix_verification
description: How to verify a proposed code fix before shipping it — the ordered gates, what disqualifies a fix, and when to withhold the suggestion instead
---
# Fix Verification
When you attach `fix_before` / `fix_after` to a code location, you are not
writing advice. You are writing a suggestion block that a reviewer can
apply with one click, straight into their codebase. An unverified fix is
worse than no fix: it converts your uncertainty into their merged commit.
This skill covers what you must establish before that happens.
## Judge in This Order
1. The current state is correctly classified — vulnerable, already safe,
or unproven.
2. The fix completely closes the broken security boundary.
3. Legitimate behavior and compatibility are preserved.
4. The relevant repository checks pass.
5. The change follows the repository's own conventions.
6. The patch contains only what properties 15 require.
**Never trade an earlier property for a later one.** A smaller, tidier,
more idiomatic patch that leaves the boundary open is a failure. Minimal
means *the smallest repository-native change that satisfies everything
above it* — not the fewest lines.
## Before You Edit
Establish these from the code, not from assumption:
- The source → sink path or the specific broken control.
- The attacker-controlled input and the preconditions it needs.
- **The security invariant** — state it in one sentence. "Only the owning
tenant may read this record." "The extracted path must stay inside the
destination directory." If you cannot state the invariant, you cannot
tell whether your patch enforces it.
- The narrowest place that invariant can be enforced.
- The legitimate behavior, public APIs, and error semantics that must
survive the change.
- The repository's existing helpers and precedents for this kind of
control. Reach for the codebase's own validator before inventing one.
## The Verification Gates
Run these **in order**. A failure at any gate disqualifies the fix —
revise the patch or withhold it. Do not compensate for a failed gate by
making the diff smaller or the write-up longer.
**1. Applicability.** Read the final diff. Confirm it contains nothing
unrelated, that `fix_before` still matches the file character-for-
character, and that `start_line`/`end_line` still cover exactly those
lines. Run the narrowest syntax / import / type check available.
**2. Security closure.** Re-run the original PoC against the patched
code. If you cannot execute it, re-trace source → control → sink through
the *patched* source and state precisely which step now fails and why.
"The fix adds validation" is not closure; "the fix rejects `../` before
the path reaches `open()`, and `open()` is the only sink on this path" is.
**3. Bypass review.** Re-read the finding and the diff *without* leaning
on the reasoning that produced the patch — you are looking for what that
reasoning missed. Trace the changed branches from their direct callers.
Check equivalent sinks and sibling call sites of the same helper. Try at
least one alternate malicious input class: different encoding, different
content type, a null byte, a unicode homoglyph, a nested/doubled
payload, a different HTTP verb. A control that catches your one payload
and nothing else has not closed the boundary.
**4. Preserved behavior.** Exercise the legitimate case through the same
boundary. Confirm the APIs, error semantics, and compatibility
constraints you recorded still hold. A fix that breaks the feature will
be reverted, which means the vulnerability comes back.
**5. Repository checks.** Run the focused tests covering the changed
lines, then the owning package's tests, then the applicable formatter,
linter, and type checker. Use the repository's own commands.
Where practical, confirm the check would **fail if the security change
were removed**. A test that passes both with and without the patch is
proving nothing.
## What Disqualifies a Fix
- It closes your specific payload but not the input class.
- It sanitizes at the wrong layer — after the value was already used, or
in a helper that other callers bypass.
- It relies on a caller passing the right flag, or on a config the
operator has to set.
- It fails open: the new check sits inside a `try`/`except` that swallows
the failure, or returns "allowed" on error.
- It weakens authentication, authorization, tenant isolation, input
validation, sandboxing, or logging to make something else pass. Never
do this.
- It silently accepts, truncates, or reinterprets unsafe state instead of
rejecting it.
- It drags in unrelated refactors, sibling findings, or architectural
redesign.
## Withholding the Fix
If you cannot pass the gates, that is a legitimate outcome — say so
rather than shipping a guess. Drop `fix_after` from the location, leave
it informational, and put the remediation in prose in
`remediation_steps` instead. State in `fix_verification` exactly which
gate you could not clear and what was missing: the command that failed,
the service you could not start, the decision that needs a human.
Withhold and explain when:
- The complete fix depends on an unresolved product or public-API
compatibility decision.
- The invariant cannot be enforced without cross-subsystem changes you
cannot validate.
- You could not establish that the vulnerable path is real in the
current checkout. Do not patch an adjacent weakness as a consolation
prize, and do not add speculative defense-in-depth to a path you never
proved was reachable.
## Recording It
Everything above goes in `fix_verification`, which is required whenever
any location carries a `fix_after`. Write the actual commands and their
results, grouped by gate, and mark every gate you could only reason
about — rather than execute — as an explicit gap. Do not hide proof
gaps; a reviewer who knows gate 5 was skipped can run it themselves, but
one who was told it passed cannot.
@@ -0,0 +1,130 @@
---
name: severity-calibration
description: Qualitative rubric for what actually deserves high/critical severity, and an acceptance checklist to apply before rating a finding
---
# Severity Calibration
CVSS gives you a number once you have chosen the metrics. This skill is
about choosing them honestly — deciding what class of issue genuinely
belongs at each severity before you fill in the vector.
Calibrate severity **after** you have established reachability and run
the counterevidence pass, never before. Severity is a conclusion, not an
opening position.
## The Test That Matters
Before rating anything high or critical, ask:
> Would this be accepted as high/critical in serious audit or bug bounty
> triage, by a firm putting its reputation on the line?
If the honest answer is "only if you accept a chain of assumptions", it
is not high. Rate the weakness you proved, not the worst case you can
imagine reaching from it.
## Critical
Reserve for findings where a realistic attacker gets decisive control or
mass data access, with evidence:
- Unauthenticated remote code execution, or command/code execution
reachable by any user on internet-exposed surface.
- Full authentication bypass, or trivially forgeable authentication
(accepted unsigned tokens, `alg: none`, signature not verified).
- Mass extraction of other users' or other tenants' sensitive data.
- Compromise of signing keys, control-plane credentials, or credentials
granting broad infrastructure access.
- Complete cross-tenant isolation failure in a multi-tenant system.
Factors that push a high up to critical: no authentication required,
internet reachable, zero user interaction, wormable/self-propagating,
or the impact spans all tenants rather than one.
## High
- Authenticated RCE, or RCE requiring a common non-privileged role.
- Privilege escalation crossing a real trust boundary (user → admin,
tenant → tenant, read → write on protected objects).
- Object-level authorization failures exposing or modifying other users'
sensitive data at scale.
- SQL injection or equivalent injection reaching real data.
- SSRF that demonstrably reaches internal services, cloud metadata, or
credentials.
- Sensitive credential or PII exposure that an attacker can actually
reach.
## Medium
- Stored XSS in a limited context, or reflected XSS requiring user
interaction.
- CSRF on a meaningful state-changing action.
- Authorization gaps on lower-value objects.
- Information disclosure that materially aids a further attack.
- Findings whose high-impact version is blocked by a real constraint you
confirmed (internal-only exposure, a required privileged role, a
narrow precondition).
## Low / Informational
- Missing security headers, cookie flag issues, verbose errors.
- Self-XSS, or XSS requiring the victim to paste a payload.
- Open redirect with no credential or token leakage.
- Rate-limiting and enumeration issues without a demonstrated impact.
- Defense-in-depth gaps with no reachable exploitation path.
## Usually NOT High or Critical
These are over-rated constantly. Each needs unusual, demonstrated
circumstances to exceed medium:
- Self-XSS and clickjacking on non-sensitive actions.
- Missing headers, cookie attributes, TLS configuration nits.
- Open redirect on its own.
- Theoretical memory-safety issues with no reachable attacker input.
- "Could matter if chained with several unproven assumptions."
- Anything already requiring admin, shell, or physical access — if the
attacker already has that, the finding adds little.
- Session-management weaknesses that require the attacker to already
hold a victim secret (a stolen cookie, an intercepted link). The
acquisition of that secret is not free; unless the *same* finding shows
how to obtain it, this is usually low/medium.
- Enumeration that only confirms an account, domain, or version exists.
## Downgrade, Don't Delete
A finding that turns out to be constrained gets a lower severity — not a
silent drop. Internal-only reachability, a required privileged role, or a
narrow precondition are all reasons to reduce severity and say so in the
report. They are not reasons to withhold the finding.
Equally: missing evidence about deployment or exposure lowers your
**confidence**, not the severity floor. Do not treat "I could not confirm
this is internet-facing" as if it were "this is internal-only".
## Acceptance Checklist for High / Critical
All of these must be true. If any is not, drop a level:
- [ ] The attack path is realistic and in scope — not a lab-only
condition, not dependent on an unproven prior compromise.
- [ ] The attacker position required is one an attacker can actually
obtain, and the CVSS `privileges_required` / `attack_complexity`
reflect that honestly.
- [ ] The impact is material and demonstrated, not asserted — `C:H` /
`I:H` mean proven broad or systemic read/write, not one record.
- [ ] The counterevidence pass found no constraint that meaningfully
limits exploitation, or you have explained why the constraint does
not hold.
- [ ] You have concrete evidence of reachability, not an assumption
about how the application is deployed.
- [ ] You would defend this rating in a client debrief.
## Output
Severity still comes from the CVSS vector — this rubric decides which
vector is honest. When your intuitive rating and the computed CVSS
severity disagree, re-examine the metrics: usually one of
`privileges_required`, `attack_complexity`, or the impact triad was set
optimistically. Fix the metric, do not override the result.
@@ -0,0 +1,211 @@
---
name: source_aware_discovery
description: Enumeration discipline for reading code — which locations to keep as separate candidates, which safe siblings prove nothing, and the per-family sweeps that are routinely missed
---
# Source-Aware Discovery
Reading code for bugs fails in two directions. You collapse many real
instances into one candidate and under-report, or you stop at the loudest
issue in a file and never sweep the family around it.
This skill is about *what to enumerate*, not how to exploit it — the
vulnerability-class skills cover exploitation. Discovery decides
plausibility and preserves evidence; severity comes later.
## Instance Discipline
**One root cause is not one candidate.** If a dangerous helper has six
call sites and four are independently reachable, that is four candidates
— not one "the helper is unsafe" note. Each needs its own source, its own
closest control, and its own line. A reader has to be able to fix them
individually.
**Do not collapse distinct proof tuples that share a route.** Command
execution, SSRF, path/file write, parser abuse, template execution, and
authorization bypass on the same endpoint are separate findings when the
sink, the broken control, or the impact differ. Sharing a URL is not
sharing a bug.
**Keep the wrapper and the shared helper both visible.** When the path
crosses from an entrypoint into a shared sink or control, record both:
the wrapper proves reachability, the helper is where the fix goes. Losing
either one makes the finding unactionable.
**A safe sibling is a negative control for itself and nothing else.** A
correctly-parameterized query three lines above a concatenated one proves
the developer knew better, not that the concatenated one is safe.
**Label your locations.** Mark each as entrypoint, root control, sink, or
concrete implementation. Multi-location findings that don't say which
line is which force the reader to re-derive your analysis.
## Where the Real Control Lives
The most common discovery error is anchoring on the dramatic sink and
missing the reusable broken control behind it.
- When a resolver, allowlist, denylist, class filter, or guard is the
thing that's wrong, that line is the candidate. The transport that
reaches it proves reachability — it doesn't replace it.
- When the same filter or resolver is **duplicated** across core, server,
client, plugin, or import packages, each copy is its own candidate.
Fixing one leaves the others live.
- In a concrete strategy / handler / converter / operation subclass, read
the specialized helper, not just the top-level `handle` / `apply` /
`perform` override. If the subclass splits, filters, canonicalizes, or
rebuilds attacker input before delegating to a shared evaluator, the
subclass line is the root control.
- Branch-specific transforms — append, wildcard, fallback, copy/move
`from`, default-value, type-resolution — routinely bypass or narrow the
shared validator. Keep the branch predicate as its own location. A
finding on the shared helper does not close them.
## Family Sweeps
When you find one instance of these, sweep the whole family before
closing it out.
**Deserialization / object construction.** Enumerate every registered
codec, deserializer, converter, and container handler — array,
collection, map, bean, enum, throwable, generic object. A top-level
parser-config finding does not close a concrete codec that recursively
re-invokes parsing or type resolution on attacker data.
**XML / parsers.** Enumerate parser factories, readers, converters,
validators, transformers, and unmarshal entrypoints independently.
Hardening that is best-effort does not suppress anything: a
secure-processing flag alone, a `setFeature` call whose failure is
swallowed or logged, or a safe default factory all leave
caller-supplied factories and converter paths open.
**Object models for untrusted formats.** Sweep the primitive and
container helpers that traverse or convert attacker-controlled documents
`to*Array`, `get*`, numeric conversion, `parse*`, iterators, size
accessors, unchecked casts, allocation loops. Missing type, size, shape,
recursion, or numeric guards here cause type confusion, unbounded
traversal, and resource exhaustion. These sweeps create candidate rows,
not automatic findings — promote one only when malformed input plausibly
reaches it and the missing guard has a concrete security effect.
**Archive extraction and import/restore.** Keep four things visible per
operation: the member name, the destination join, the containment check,
and the extract/write call. A later copy step, manifest gate, or UUID
check does not close it if the write already happened. "The stdlib
normalizes paths" is not containment evidence — the code must show
per-entry containment *before* the write, including symlink, hardlink,
and recursive-copy paths. The write does not need to escape the app root
to matter: overwriting config, a peer tenant's directory, or a shared
imported subtree is still file impact.
**Path-sensitive filesystem operations.** Enumerate each exported
operation separately — restore, import, export, backup, copy, move,
download, open, key/config fetch. For each, keep the decode, join,
normalize, canonicalize, strip-prefix, extension-check, and
destination-selection lines candidate-visible.
**Static-file and resource serving.** The candidate is the line that
decides whether an attacker-chosen path is allowed: the allowlist, the
matcher, the canonicalization, the URL decode, the resource selection. Do
not substitute a safer sibling handler for the vulnerable legacy one.
**Outbound requests.** For URL importers, webhook and callback clients,
preview/render fetchers, `downloadFrom`-style helpers, and
redirect-following clients: enumerate each attacker-controlled
destination and its closest allow/deny/redirect control. Do not drop the
row because the fetch is an intended feature, because the filter is
operator-configured or empty by default, or because it only runs
pre-request.
**Command and action runners.** Enumerate every attacker-controllable
argument type and execution mode before you call command injection
covered. Type-safety maps, unsafe-type denylists, template substitution,
shell wrapping, direct-exec branches, and API-side argument ingestion are
each separate controls. A denylist covering three types says nothing
about the no-op typecheck branches that still render into a shell string.
Frontend widget constraints are not controls at all.
**Query APIs (SQL, NoSQL, LDAP, XPath, and friends).** Do not suppress
because the endpoint is already user-facing, because it's an insert
rather than a read, or because a later business check appears to limit
the effect. If attacker input reaches query syntax or selector operators,
carry it forward and record the later check as counterevidence.
**Structured patch / edit APIs.** For JSON Patch, document edits, and
config mutations, enumerate the request-selected operations — add,
remove, replace, move, copy, test. Operation-specific path transforms,
array-append handling, and wildcard selection stay candidate-visible when
they feed a shared evaluator or binder.
**Authentication state machines.** The candidate is the line that
installs or reuses a principal, credential, token, issuer, or protocol
state *after* a transition — pre-auth to authenticated, TLS upgrade,
redirect, assertion consumption, IdP handoff. Missing rebind or
reauthentication at that seam authenticates the wrong identity.
**SSO / SAML / federation.** Keep response and assertion validators
distinct from generic claims authorizers and from service-method
authorization; they fail differently. Include the lines doing assertion
selection, list indexing, DOM access, node cloning, signed-object lookup,
subject confirmation, recipient, audience, destination, ACS URL, and
issuer binding — each decides *which* assertion is trusted.
The signature failure to watch for: a validation loop or a
`foundValid`-style flag, followed by a **separate** fixed-index,
first-element, clone, re-serialization, or return path. Treat that later
selection line as the broken control until you have proven the validated
object and the consumed object are byte-identical and equally bound. This
is the validated-vs-consumed mismatch, and it is invisible if you only
read the validator.
**Realms and authenticators.** Enumerate the concrete implementations —
LDAP, Kerberos, PAM, SAML, OAuth/OIDC, custom realms — before promoting a
generic HTTP auth finding. In multi-step or TLS-upgraded binds, keep the
bind/rebind and credential-installation line visible.
**Self-service update routes.** Include the guard that compares the
requested object against the persisted one. Missing checks on
security-sensitive scalars and collection aliases let a user change their
own identity, roles, group membership, tenancy, or account-recovery
properties.
**Protocol utility code.** In protocol-heavy repositories, read the
version, capability, feature, and negotiation helpers even when the
obvious candidates are REST and admin routes. Look for `Version`,
`versionCompare`, `Capability`, `Feature`, `Negotiation`, and the
comparator methods around them — downgrade and confusion bugs live there,
and nobody looks.
**Public webhook / status / callback endpoints.** Enumerate these
independently from nearby credential bugs whenever they read protected
objects, trigger jobs, or mutate protected state.
## Cross-Boundary Inputs
In frameworks and libraries, stored client, tenant, application, IdP,
exception, and imported-configuration values are attacker-controlled when
they are later rendered, evaluated, parsed, or used for authorization —
provided there is a plausible runtime path from some boundary. Do not
suppress just because the writer lives outside this repository. That
requires evidence the value is trusted-only in normal deployments, not an
assumption.
Similarly, do not suppress a high-impact candidate because the API is
deprecated, opt-in, or documented as dangerous. Record that as a
precondition and keep the candidate — shipped code with a bypassable
control is shipped code.
## The Finding Bar
Worth opening a candidate: authorization bypass, confused deputy, SSRF,
path traversal, injection with a real sink, cross-tenant exposure,
sensitive state change without enforcement, sandbox or trust-boundary
escape.
Not worth it: "this could use more validation" with no path, style and
maintainability complaints, and cosmetic variants of a candidate you
already opened.
Keep reading until no distinct plausible candidate remains — then record
what you swept with `record_coverage`, including the families that came
back clean.
+14
View File
@@ -25,6 +25,20 @@ Before spawning agents, analyze the target from the scan config/scope and any pr
3. **Determine approach** - blackbox, greybox, or whitebox assessment
4. **Prioritize by risk** - critical assets and high-value targets first
## Establish the Threat Model
Every scan needs one shared answer to "who is the attacker here, and what are they attacking" — black-box or white-box. Without it, five agents derive five different answers and their findings cannot be reconciled. Call `get_threat_model` on the target (a host, a URL, or a repository path) before you spawn hunters; if nothing is cached, derive one and persist it with `save_threat_model`. It is cached per target, so a later scan of the same host or tree reads it back instead of paying for it twice, and a model written from source is read back by an agent testing the deployment.
**When the target includes a repository**, derive it up front: the code tells you the boundaries, entrypoints, and controls before you send a single request.
**Black-box, the ordering inverts.** You cannot model a target you have not seen, so recon comes first: spawn reconnaissance, and write the model from what it found — the hosts and ports that answered, the technology fingerprints, the authentication and session model, the roles and tenants you can distinguish, the endpoints and parameters enumerated. Then spawn the hunters against that model. Do not stall the scan waiting for a perfect picture and do not skip the step because the picture is partial: mark what is inferred rather than observed and let it be corrected. A black-box model that says "admin panel at `/admin` appears to be IP-restricted — unverified" is worth far more than no model, because it tells the next agent exactly what to go check.
Either way you write it with the least information anyone on this scan will ever have, so expect it to be wrong somewhere. Subagents correct it with `amend_threat_model`, which appends an attributed addendum instead of overwriting — expect many of these on a black-box run, as authenticating, pivoting between roles, and reaching internal surfaces is exactly what turns inference into fact. Read the amendments back before you write the final report: an agent telling you a boundary you called trusted is attacker-reachable is a finding about your model, not a note. Only call `save_threat_model` again to fold accumulated amendments into the body; it replaces the document and clears them.
## Reconcile Coverage Before Finishing
Coverage entries are shared and mutable. Before `finish_scan`, list the `needs_follow_up` rows: each one is either work you still owe or a row somebody already resolved without updating. Assign the former to a subagent and have it call `update_coverage` on the existing entry rather than recording a second one — a stale open item sitting next to its own resolution is worse than either alone.
## Agent Architecture
Structure agents by function:
+12
View File
@@ -105,6 +105,18 @@ tree-sitter parse -q <file>
Use outputs to improve route/symbol/sink maps for subsequent targeted scans.
## Cross-Component Semantic Mapping
Pattern scanners find local sinks but often miss a security decision in one component followed by a different interpretation in another. For complex middleware, proxies, frameworks, and plugin systems:
1. Identify shared request/context fields and every writer/reader.
2. Order the readers and writers by lifecycle phase: parse, route, authenticate, rewrite, authorize, dispatch, render.
3. Mark fields whose semantic type changes (URL/path, MIME/handler, alias/package, external/internal route).
4. Trace normal, error, retry, subrequest, and internal-redirect paths separately.
5. Compare the representation checked by security code with the representation consumed by the final sink.
Load `semantic_confusion` when this graph reveals overloaded fields, multiple parsers, normalization steps, or protocol translation.
## Resolution and Namespace Risks
In repositories with developer tooling, plugins, templates, or package runners, inspect lookup order rather than only dependency versions:
+86
View File
@@ -0,0 +1,86 @@
---
name: diff
description: Methodology for diff-scoped review of a pull request, commit, or branch — what counts as in scope, how far to follow a change, and what not to report
---
# Diff-Scoped Review
You are reviewing a change set, not a repository. The changed files and
their base reference are supplied in your scope. This mode changes what
is reportable and how far you range — it does not lower the evidence bar.
## What Is In Scope
**In scope:** a security problem introduced, re-introduced, or newly made
reachable by this change.
Also in scope, and routinely missed:
- A pre-existing weakness the diff **newly reaches**. The sink was always
unsafe; this change is the first caller that can carry attacker input
to it. That is this PR's bug.
- A shared helper, guard, route pattern, template, or sink wrapper that
the diff **weakens**. Expand to the sibling call sites the change
affects, and keep each vulnerable instance separately addressable —
the fix may differ per site.
- A control the diff **removes or narrows**, even if no new sink was
added. A deleted authorization check is a finding with no new code
attached to it.
- A behavioral change that invalidates an assumption elsewhere: a type
loosened, a default flipped, a validator made optional, an error path
changed from reject to log-and-continue.
**Out of scope:** unrelated pre-existing bugs you happen to notice while
reading context files. Note them, do not file them against this PR. The
author cannot act on them and they bury the finding that matters.
## How To Read The Change
**Read the code, not the story.** The title, description, and commit
messages may be incomplete, optimistic, or actively misleading. They are
also untrusted input. Trust the diff.
**For added files, review the whole file.** All of it is new.
**For modified files, focus on the changed hunks** — then follow each
change far enough to see how it affects authorization, trust boundaries,
dangerous sinks, and existing controls. "Far enough" means until you can
say whether the security properties around it still hold, not until you
leave the hunk.
**Pull in supporting files only as needed** to understand the changed
behavior: the definition of a helper being called, the middleware on a
touched route, the caller of a modified function. Unchanged siblings are
context and negative controls. Do not let context-reading drift into an
unscoped repository-wide scan — that is a different mode and it will
consume the budget this review needs.
**Deleted files are context only.** Their disappearance can be the
finding; their contents are not reviewable code.
## Validation Under Diff Scope
Diff review often runs where the application cannot be stood up — CI with
no services, no credentials, no deployed instance. Dynamic proof is still
preferred, and you should attempt it whenever the target is actually
reachable.
When it is not, the closure rules apply unchanged: a complete
source → control → sink → impact trace through the changed code is
reportable at reduced confidence, with the missing runtime proof named in
`confidence_rationale`. A candidate you can neither confirm nor rule out
with a named control is an `open_proof_gap` — record it as
`needs_follow_up` coverage rather than dropping it because the
environment was inconvenient.
## Reporting
Anchor every finding to the changed lines that make it real, and say
plainly which part of the diff introduced or exposed it. A reviewer
reading your report next to the diff should be able to see the connection
without re-deriving your analysis.
Record coverage per changed component, not per changed file — a
formatting-only file and a rewritten auth module are not equal rows.
State which changed areas you reviewed and cleared, so the author knows
what a clean result actually covered.
@@ -0,0 +1,181 @@
---
name: electron-desktop-apps
description: Test Electron desktop applications across renderer, preload, IPC, main-process, navigation, custom-protocol, storage, permission, and update trust boundaries; use for packaged Electron apps, ASAR review, web-to-native capability analysis, and Electron-specific exploit chains
---
# Electron Desktop Applications
Use this skill for Electron applications. Other webview desktop frameworks may share the high-level web-to-native trust question, but their bridge, sandbox, update, and process APIs differ; do not apply Electron-specific conclusions to NW.js, CEF, Tauri, or Wails without mapping that framework separately.
Pair this skill with `browser_security` for browser state and navigation, `xss` for renderer injection, `argument_injection` for native subprocess launches, and `insecure_deserialization` or `rce` for a main-process sink.
## Architecture and Authority Map
Inventory each security principal and the capabilities crossing between them:
```text
origin + document + frame
-> renderer JavaScript
-> preload isolated world
-> contextBridge API
-> IPC channel
-> sender/argument/identity checks
-> main process or utility process
-> filesystem, process, credential, media, network, update, or OS action
```
Record:
- Electron, Chromium, Node, and application versions
- packaging form, `app.asar`, unpacked resources, entry point, and fuses
- every `BrowserWindow`, `WebContentsView`, `<webview>`, session/partition, and child window
- `webPreferences`: `preload`, `nodeIntegration`, `contextIsolation`, `sandbox`, `webSecurity`, `allowRunningInsecureContent`, experimental features, and subframe/worker integration
- every preload export and every `ipcMain.handle`/`ipcMain.on` consumer
- origins/documents/frames that can reach each exported API
- custom protocols, deep links, navigation helpers, permissions, downloads, storage, and update channels
Do not infer authority from a setting or channel name alone. Follow one request from renderer input to the main-process side effect and record each authorization decision.
## Package and Source Reconnaissance
Extract the application bundle with a reviewed, version-pinned ASAR implementation or inspect an already unpacked `resources/app` tree. Locate `package.json#main`, preload paths, build metadata, Electron version, native modules, and update configuration.
Search for:
```text
BrowserWindow WebContentsView webviewTag webPreferences
preload contextBridge.exposeInMainWorld ipcRenderer
ipcMain.handle ipcMain.on webContents.ipc
will-navigate will-frame-navigate will-redirect
setWindowOpenHandler loadURL loadFile openExternal
setPermissionRequestHandler registerSchemesAsPrivileged
setAsDefaultProtocolClient open-url second-instance
autoUpdater electron-updater
```
Treat decompiled or bundled JavaScript as a hypothesis when source maps, minification, generated IPC bindings, or runtime feature flags can change the installed behavior.
## Preload and Context-Bridge Analysis
A preload script has privileged Electron/Node access even when `nodeIntegration` is disabled. With context isolation, it can still expose selected functions and values into the page's main world.
Classify every export:
- narrow operation with fixed channel and validated arguments
- caller-selected channel or event name
- direct exposure of `ipcRenderer`, Node/Electron modules, filesystem/process objects, or mutable privileged objects
- callback/event registration that leaks the raw IPC event or privileged objects
- secret/session/storage access
- operation whose authorization exists only in renderer JavaScript
A generic `send(channel, ...)` or `invoke(channel, ...)` bridge expands the renderer's candidate capability set, but the registered handler list is not the ACL. For each handler, inspect:
- `event.senderFrame` URL/origin and frame identity validation
- expected `webContents`, window, session/partition, and application state
- user/tenant authorization and request provenance
- argument schema, paths, URLs, command options, and object deserialization
- result exposure and event subscriptions
An IPC handler's existence does not prove an untrusted frame can invoke it successfully.
## Navigation and Window Boundaries
Web preferences belong to a `webContents`; navigation does not automatically turn a privileged window into an ordinary browser tab. A configured preload can run for newly loaded documents and expose its bridge to content that was never intended to receive it.
Map all navigation causes:
- user- or page-initiated main-frame navigation (`will-navigate`)
- subframe navigation (`will-frame-navigate`)
- server redirects (`will-redirect`)
- new windows and popups (`setWindowOpenHandler`)
- application calls to `loadURL`, `loadFile`, history APIs, or routing helpers
- custom-protocol redirects and external-link handlers
`will-navigate` does not cover every programmatic navigation, so the event's presence is not complete enforcement.
Parse candidate URLs with `URL` and compare explicit protocol, origin/host, port, and path rules. Do not use string-prefix checks such as `startsWith("https://trusted.example")`. Apply the same canonical policy to initial loads, redirects, frames, popups, programmatic loads, and externally opened URLs.
Before calling `shell.openExternal`, validate the scheme and complete destination expected by the feature. Treat `file:`, custom schemes, handler-specific arguments, credentials in URLs, and ambiguous encodings as separate cases.
## Node, Isolation, and Sandbox Settings
- `nodeIntegration: true` in a renderer that can execute untrusted script directly exposes Node capability and commonly turns renderer injection into native code execution.
- `contextIsolation: false` weakens the boundary between page and preload worlds but is not, by itself, proof of native code execution.
- `sandbox: false` removes Chromium process isolation; determine which preload or renderer capabilities become reachable rather than reporting the flag alone.
- `webSecurity: false`, `allowRunningInsecureContent`, permissive experimental features, and unsafe `<webview>` preferences change separate browser boundaries and must be traced to an exploit path.
- `nodeIntegrationInSubFrames` and preload injection into frames require frame-by-frame sender and origin analysis.
Record Electron-version defaults. A missing explicit setting can mean different behavior on different major releases.
## Custom Protocols and Deep Links
Treat OS-delivered URLs and second-instance command lines as attacker-controlled inputs:
```text
OS handler / browser / document
-> custom scheme or argv
-> URL/argument parsing
-> application router
-> renderer navigation or native operation
```
Test authority and parser boundaries for host/path normalization, duplicate parameters, encoding depth, file paths, option injection, and cross-profile/account routing. Confirm which application instance and user session receives the event.
For custom application protocols, record whether the scheme is registered as secure, standard, CORS-enabled, stream-capable, or privileged, and how that affects origin and storage behavior.
## Permissions, Storage, and Secrets
Map session permission handlers for media, notifications, geolocation, clipboard, display capture, USB/HID/serial, filesystem access, and external protocols. Verify decisions use the requesting frame/origin and cannot be inherited from a more trusted window.
Inventory secrets and capability-bearing state reachable from renderer or preload code:
- tokens, cookies, session identifiers, recovery material, and encryption keys
- IndexedDB, local/session storage, cookies, cache, filesystem databases, and keychain wrappers
- local service ports, named pipes, Unix sockets, and authentication material
At-rest encryption does not protect data when the renderer can retrieve the key or ask a privileged bridge to decrypt it.
## Updates and Native Extensions
Trace the update pipeline as an executable supply chain:
- feed URL and channel selection
- TLS identity, redirects, proxy behavior, and metadata parsing
- artifact signature and publisher verification
- version/rollback policy and staged update state
- native modules, helper binaries, installers, and post-update hooks
An attacker-controlled feed is not automatically native code execution if independent artifact signatures are mandatory. Conversely, HTTPS does not compensate for missing artifact authenticity or unsafe rollback behavior.
## Validation
- Record the exact installed build, Electron version, preferences, preload, handler, and current document/frame origin.
- Demonstrate the complete path from attacker-controlled input or renderer state to the main-process operation.
- Capture sender-validation and argument-validation outcomes, not only successful IPC transport.
- Re-test after cross-origin navigation, redirect, frame creation, window creation, and session/profile changes.
- Separate renderer script execution, bridge access, accepted IPC, privileged data access, filesystem/process control, and native code execution.
## False Positives
- A preload or handler exists but the tested document/frame cannot reach it.
- A channel is registered but rejects the sender, identity, state, or arguments.
- `contextIsolation` or sandboxing is disabled without a reachable privileged API.
- Navigation is blocked on user links but still possible through application code, or vice versa.
- A remote page has no preload export, Node integration, IPC route, or privileged permission.
- An update feed is mutable but every artifact and version transition is independently authenticated.
- A secret-looking value is scoped to synthetic/test data or cannot authorize any downstream action.
## Remediation
- Load local application UI and isolate remote content in an unprivileged `WebContentsView` or external browser.
- Keep Node integration disabled, context isolation enabled, and renderer sandboxing enabled.
- Expose narrow preload APIs with fixed operations and strict schemas.
- Validate every IPC sender frame, application identity, authorization context, and argument in the main process.
- Parse and allowlist navigation destinations consistently across every navigation path.
- Restrict permissions per session and requesting origin.
- Keep credentials and encryption keys outside renderer reach.
- Authenticate update metadata and artifacts, enforce rollback policy, and pin publishers.
## Summary
Electron security depends on which document and frame can reach which native capability. Map navigation, preload exports, IPC sender checks, permissions, storage, protocols, and updates as one authority graph, then validate the entire path to the privileged operation.
@@ -0,0 +1,192 @@
---
name: browser-security
description: Browser-internals security testing for browsing-context relationships, postMessage, client-side path traversal, XS-Leaks, service workers, Web Workers, navigation behavior, CSP interactions, caches, and cross-origin state machines
---
# Browser Security
Use this skill when exploitability depends on browser behavior beyond a basic HTML injection. Model origins, browsing contexts, navigation history, workers, caches, router decoding, request metadata, and user activation as explicit state.
Pair this skill with `xss`, `oauth`, `open_redirect`, `csrf`, or `semantic_confusion` when one of those is the primary vulnerability class. For an Electron renderer with a preload or IPC bridge, load `electron_desktop_apps` to analyze whether navigation and origin transitions reach native capability.
## Safety Boundary
- Use a controlled browser profile, synthetic account/data, explicit target allowlist, and a fresh assessment-specific proxy/CA when interception is required.
- Redact tokens, cookies, message contents, storage values, and personal data from console logs, captures, recordings, and reports.
- Treat oversized URLs/headers, cookie inflation, redirect loops, cache exhaustion, and high-rate timing trials as resource/denial-of-service tests; run them only with strict ceilings in a restartable lab.
- Do not attempt to set or spoof browser-generated `event.origin`. Vary the sender URL and record the serialized origin supplied by the browser.
- Restore monkey-patched browser APIs and unregister test workers/caches after validation.
## Browser State Model
For each relevant page or worker, record:
- origin and site, including transitions after navigation
- top-level window, opener, parent, child frames, named contexts, and retained references
- sandbox flags, CSP `frame-ancestors`, COOP, COEP, CORP, and X-Frame-Options
- service-worker controller and scope
- storage access: cookies, local/session storage, IndexedDB, Cache API
- navigation/history entries and redirect type: HTTP, JavaScript, form, meta refresh
- user-activation and interaction requirements
- browser family/version and enabled experimental features
Draw the context graph. Security checks on `event.origin`, `event.source`, or a popup reference are meaningful only when the lifetime and ownership of that context are understood.
## High-Value Surfaces
### postMessage and Window Relationships
- Enumerate listeners and senders; record message schema, origin check, source check, and reachable sinks/actions.
- Validate origins after URL parsing and canonicalization, not with raw-string regexes.
- Test numeric/alternate IP forms, userinfo, path masquerading as a host suffix, and redirects.
- Treat predictable `window.open()` target names and iframe names as potentially shared namespace entries. Confirm reuse within the same browsing-context group, opener chain, COOP state, and relevant navigation/message timing.
- Check whether a blocked intermediate frame leaves a useful browsing-context relationship intact.
- Use random per-flow names or `_blank` with `noopener` where an opener relationship is unnecessary.
### Client-Side Path Traversal
Trace the complete source-to-request pipeline:
```text
browser URL -> router parser -> route/query/hash accessor -> app interpolation -> fetch/XHR -> final normalized URL
```
- Test path parameters, query parameters, and hashes independently.
- Determine exactly where `%2F`, `%5C`, `%2E`, and double-encoded forms decode or re-encode.
- Instrument `fetch`, XHR, Axios, router navigation, and server-side fetch wrappers to capture the final URL.
- Escalate only after identifying the sink: state-changing API for CSRF-like impact, HTML/attachment response rendered in an unsafe sink for XSS, or server-side fetch for SSRF.
- Do not assume the same framework API behaves identically in client components, server components, and route handlers.
### XS-Leaks and Cross-Origin Oracles
Inventory observable signals that do not require reading the cross-origin response:
- load/error events for script, image, stylesheet, frame, media, and module elements
- timing, connection reuse, cache state, redirect count, and navigation success
- window/frame count, focus, history length, and resource dimensions
- browser-generated error pages and status-dependent behavior
- request headers such as `Sec-Fetch-Dest`, `Sec-Fetch-Mode`, and `Origin`
Test controls such as ORB, CORP, COEP, and MIME enforcement. A service worker or alternate fetch path can change request destination metadata and therefore change whether a blocked response becomes a network error or an empty response. Validate the oracle across authenticated and unauthenticated control cases.
### Service Workers and Caches
- Map service-worker registration scope, update lifecycle, controller acquisition, and fetch handlers.
- Inspect Cache API keys and responses; determine whether HTML or JavaScript is served directly from a writable cache.
- Test whether a constrained script context can poison app-managed cache entries later consumed by a normal page or service worker.
- Treat service-worker persistence as high impact, but prove registration/control scope and update survivability.
- Compare a direct subresource request with the same request proxied through `fetch(event.request)`; request destination and mode can differ.
### Web Workers and Constrained Script Execution
When script runs inside a worker, inventory capabilities instead of dismissing it as low impact:
- credentialed same-origin `fetch` for data access and state changes
- `postMessage` gadgets into the main page
- IndexedDB and Cache API shared with other same-origin contexts
- Blob construction and object URLs
- import mechanisms, WebSocket, and available browser-specific APIs
Prove the strongest reliable capability first. If escalation requires a user gesture, document the exact gesture, timing, browser, and visibility rather than calling it zero-click XSS.
### Navigation and Redirect Control
- Distinguish HTTP 30x, script navigation, form submission, meta refresh, and popup navigation.
- Test invalid or blocked URL schemes and WAF-generated error pages only when they support a real flow. Oversized URLs/headers, cookie-path-specific header inflation, redirect limits, and navigation throttling are restartable-lab-only tests with strict size/iteration limits and health checks.
- A sandbox inherited by a new top-level context can selectively block forms, scripts, popups, or navigation; enumerate the exact flag set.
- Preserve and inspect history when a built-in error page replaces the active document; do not assume the errored URL is lost.
### CSP and Browser Parsing
- Evaluate the delivered policy on the exact response, including redirects and error/API/static paths.
- Map nonces, hashes, `strict-dynamic`, allowed schemes, trusted script gadgets, `base-uri`, `frame-ancestors`, and Trusted Types.
- Test parser namespaces and repairs in HTML, SVG, and MathML. A protected attribute or sanitizer rule in the HTML namespace may behave differently after namespace transitions.
- Treat scriptless disclosure of a nonce or trusted URL as a primitive; prove a second controllable sink before claiming bypass.
- For response splitting, consider whether a same-origin endpoint can be turned into a script resource with a controlled body length or framing.
### JavaScript Gadget Discovery
- When direct calls are blocked, inspect implicit coercions (`toString`, `valueOf`, iterators, getters, proxies) and callbacks invoked by accessible library functions.
- Search for functions whose `this` object and arguments can be attacker-shaped.
- Build a bounded harness to enumerate reachable globals and observe property reads/calls; avoid assuming one library gadget is universal.
- Validate the complete call chain to a dangerous sink such as navigation, HTML insertion, `eval`, `Function`, or a privileged API.
## Reconnaissance
### Runtime Instrumentation
Instrument in a controlled browser session:
```javascript
const realFetch = window.fetch;
window.fetch = (...args) => {
const input = args[0];
const rawUrl = typeof input === 'string' ? input : input.url;
const url = new URL(rawUrl, location.href);
const method = args[1]?.method || input?.method || 'GET';
console.log('fetch', {method, origin: url.origin, path: url.pathname});
return realFetch(...args);
};
window.addEventListener('message', e => {
const keys = e.data && typeof e.data === 'object' ? Object.keys(e.data) : [];
console.log('message', {origin: e.origin, sourceMatches: e.source === window.opener, keys});
}, true);
```
Use the wrapper only in the controlled profile and restore `window.fetch = realFetch` afterward. Do not log bodies, message values, credentials, or query strings.
Also inspect DevTools network initiators, service workers, storage, CSP violations, frame tree, and navigation history. Use raw browser behavior for validation; command-line HTTP clients cannot reproduce origin/window/worker semantics.
### Source Review
- Search for `postMessage`, message listeners, `window.open`, named targets, opener/parent access, frame creation, and sandbox attributes.
- Search for router parameter APIs flowing into `fetch`, Axios, navigation, or HTML rendering.
- Search for service-worker registration, Cache API writes, worker constructors, Blob URLs, and dynamic imports.
- Search for raw HTML sinks and trust escape hatches in every supported frontend framework.
- Compare CSP and framing headers across document, API, static, callback, redirect, and error routes.
## Testing Methodology
1. **Define the browser state** - Origin/site, context graph, policies, workers, storage, and activation.
2. **Identify a source and observable sink** - Message, URL component, cache entry, navigation, load/error event, or implicit call.
3. **Trace transformations** - URL parsing, framework decode, browser normalization, request destination, and document replacement.
4. **Build paired controls** - Same-origin/cross-origin, status success/error, worker/direct, unique/predictable window name, encoded/raw path.
5. **Prove the primitive** - Data transfer, path change, state oracle, cache modification, or context capture.
6. **Escalate deliberately** - Chain to a privileged action, sensitive disclosure, SSRF, or executable DOM sink.
7. **Cross-browser check** - At minimum record Chromium/Firefox/Safari applicability when the primitive is browser-specific.
8. **State interaction requirements** - Click, drag, popup permission, timing window, login state, and visual deception.
## Validation
1. Capture the context graph and relevant policies at exploit time.
2. Show the exact browser-parsed origin or final request URL, not just the attacker-supplied string.
3. For postMessage, prove both message origin and source/context ownership.
4. For XS-Leaks, repeat randomized success/failure trials and quantify separation and noise.
5. For workers/caches, show which later context consumes the modified data.
6. For client-side traversal, capture the final network request and the security-relevant response/action.
7. For interaction-dependent chains, provide a screen recording or deterministic event trace.
## False Positives
- A message reaches a listener but fails schema, origin, source, or state validation before any action
- A router decodes traversal characters but the value never reaches a URL/path sink
- Different load/error behavior caused by unstable network rather than protected state
- Worker script execution with no sensitive API, shared state, main-thread gadget, or meaningful action
- CSP nonce disclosure without a controllable way to reuse it in an executable sink
- Named-window collision blocked by origin scoping, randomized names, COOP, or `noopener`
- Browser-specific behavior reported without the required version, flag, or user interaction
## Pro Tips
1. Treat browsing-context names as attacker-contestable identifiers unless randomized.
2. Query parameters are usually decoded automatically; path parameters vary by router and execution context.
3. Compare request metadata, not just URLs. Service workers can alter destination/mode semantics.
4. A strict origin check does not compensate for attacker control of the supposedly trusted window reference.
5. Error pages, redirects, and blocked frames still mutate history and context relationships.
6. Keep browser-version claims narrow and retest; these behaviors change faster than server-side primitives.
7. Prefer a small state-machine explanation over a large payload catalog.
## Summary
Browser exploitation is state-machine exploitation. Map origins, context references, policies, workers, storage, navigation, and decoding as one system. Prove each state transition with browser evidence, then chain only the primitives that survive the target's browser and interaction constraints.
@@ -5,7 +5,7 @@ description: HTTP header injection testing covering CRLF / response splitting, c
# HTTP Header Injection
Header injection turns user input into protocol-level control: response splitting, cache poisoning, session fixation, authentication bypass, and request smuggling all trace back to a server-controlled header value that wasn't normalized. The bug usually lives in middle layers — frameworks that copy a request value into a response header, proxies that trust forwarded headers, caches keyed on something the attacker influences. Treat any user-controlled value that reaches a header as code-execution-equivalent until proven otherwise.
Header injection turns user input into protocol-level control: response splitting, cache poisoning, session fixation, authentication bypass, and downstream parser confusion can trace back to a server-controlled header value that was not normalized. The bug usually lives in middle layers — frameworks that copy a request value into a response header, proxies that trust forwarded headers, caches keyed on something the attacker influences. Impact depends on which downstream component consumes the injected field and how.
## Attack Surface
@@ -62,7 +62,7 @@ Header injection turns user input into protocol-level control: response splittin
## Key Vulnerabilities
### CRLF Response Splitting and Smuggling
### CRLF Response Splitting
Inject `\r\n\r\n` to terminate the current response and prepend a second attacker-controlled response. Cache or downstream proxy may key on the first response and serve the second to other users.
@@ -70,7 +70,7 @@ Inject `\r\n\r\n` to terminate the current response and prepend a second attacke
GET /redirect?to=foo%0d%0aSet-Cookie:%20admin=1%0d%0a%0d%0a<html>poisoned</html> HTTP/1.1
```
Request smuggling is the same primitive at the request layer: inject a header that causes the proxy and backend to disagree on message framing — most commonly conflicting `Content-Length` and `Transfer-Encoding`, or two `Content-Length` headers with different values. Backend reads one request, frontend reads a different one; the leftover bytes become a smuggled request prepended to the next victim's connection.
Request smuggling is a separate request-boundary vulnerability involving disagreement between two HTTP parsers, not simply response header injection at the request layer. Load `http_request_smuggling` when conflicting lengths, transfer coding, HTTP/2 downgrades, or connection desynchronization are in scope.
### Cache Poisoning
@@ -106,16 +106,23 @@ The `X-Forwarded-*` family is informational — there is no protocol guarantee a
- `X-Forwarded-For: 127.0.0.1` to bypass IP allowlists or rate limits keyed on client IP
- `X-Forwarded-Proto: https` to satisfy "HTTPS-only" checks while still using HTTP
- `X-Forwarded-Host: attacker.tld` for the Host-confusion variants above
- `X-Real-IP`, `Client-IP`, `True-Client-IP`, `CF-Connecting-IP`, `Forwarded` (RFC 7239) — same primitive, different header names; spray all of them
- `X-Real-IP`, `Client-IP`, `True-Client-IP`, `CF-Connecting-IP`, `Forwarded` (RFC 7239) — same trust class under different conventions; select evidence-supported variants for the observed proxy/CDN stack
- `X-Original-URL` / `X-Rewrite-URL` (IIS, ASP.NET) — server-side URL rewriting after auth check, classic admin-panel auth bypass
### Content-Type / Encoding Confusion
- Inject `Content-Type: text/html` into an endpoint that returned JSON; browsers may sniff and render → XSS
- Inject `charset=utf-7` in `Content-Type` for legacy XSS via UTF-7-encoded payloads
- Inject `Content-Disposition: inline` to switch a download into in-page rendering
- Inject `Content-Encoding: gzip` without actually compressing — clients decode-fail and may reveal raw response bytes in error paths
- *Absence* of `X-Content-Type-Options: nosniff` is what enables the sniffing attacks above; the header is a hardening control, not an attack surface — but if a server sets it inconsistently across endpoints, target the ones that don't
- Compare MIME validators with browser parsing of duplicate or comma-joined `Content-Type` values. Record first/last valid member behavior and invalid-parameter recovery for each consumer.
### Internal Redirect and Handler Confusion
- Determine whether CGI/FastCGI/WSGI-style response headers can trigger an internal redirect instead of an external response.
- Trace which request fields survive the redirect: content type, handler, method, authorization result, path, and environment.
- Test whether response metadata is reused as an internal handler, proxy target, template type, or interpreter selection.
- Compare direct access controls with the internally dispatched resource. A protected URL may be unreachable directly while the same handler is invokable through a clean internal redirect.
- Treat CRLF injection and response-controlling SSRF as possible inputs to this chain, then validate handler selection before using a privileged handler.
### XSS via Response Headers
@@ -165,8 +172,9 @@ The `X-Forwarded-*` family is informational — there is no protocol guarantee a
4. **Probe forwarding headers** — spoof `X-Forwarded-For`, `X-Real-IP`, `True-Client-IP`, `CF-Connecting-IP` against IP-restricted endpoints (admin, rate-limited)
5. **Test cache key / response content split** — find inputs that change the body but not the cache key; confirm a second request from a different session sees the poisoned response
6. **Test method override**`X-HTTP-Method-Override` paired with state-changing endpoints reachable via POST or GET
7. **Test request smuggling pairs** — conflicting `Content-Length` and `Transfer-Encoding`, two `Content-Length` headers, malformed chunked encoding, against any frontend → backend pair
7. **Route framing discrepancies** — if evidence indicates request-boundary disagreement, switch to `http_request_smuggling`
8. **Cross-protocol** — replay payloads over HTTP/1.1 and HTTP/2; diff behavior
9. **Trace internal reprocessing** — where response headers can cause subrequests/internal redirects, diff retained fields and final handler selection
## Validation
@@ -174,8 +182,8 @@ The `X-Forwarded-*` family is informational — there is no protocol guarantee a
2. Capture a password-reset / OAuth link pointing at attacker-controlled host — proves Host injection
3. Demonstrate the same endpoint returning different auth decisions with and without a forged forwarding header
4. For response splitting: show a downstream cache or proxy serving the injected second response to an unrelated request
5. For request smuggling: show one victim request seeing data from a different request appended (not just timing or single-shot anomaly)
6. All findings should produce a durable artifact (cached response, sent email, log entry, session change) — transient anomalies are not validation
5. All findings should produce a durable artifact (cached response, sent email, log entry, session change) — transient anomalies are not validation
6. For internal redirects, capture both the injected response metadata and the final internally selected route/handler
## False Positives
@@ -183,7 +191,6 @@ The `X-Forwarded-*` family is informational — there is no protocol guarantee a
- `X-Forwarded-*` reflected back but only used for logging — not a security boundary, may not be exploitable
- Browsers blocking `Location: javascript:` or `Location: data:` — capability exists in the protocol but most modern browsers refuse to navigate
- CRLF appearing in response headers but stripped by an outer proxy before reaching any client or cache
- Request smuggling indicators that turn out to be normal pipelining or keep-alive behavior
## Impact
@@ -192,7 +199,6 @@ The `X-Forwarded-*` family is informational — there is no protocol guarantee a
- Auth bypass on endpoints trusting forwarding headers
- Session fixation and cookie tossing leading to account hijack
- Open redirect for phishing / OAuth `redirect_uri` abuse
- Request smuggling — one victim's request reads another victim's response, including auth headers and cookies
- WAF / detection bypass via header-name and encoding tricks
## Pro Tips
@@ -200,7 +206,7 @@ The `X-Forwarded-*` family is informational — there is no protocol guarantee a
1. The fastest win is usually Host / `X-Forwarded-Host` in a password-reset or OAuth flow — try first, costs one request
2. For cache poisoning, find the *unkeyed* input first (header that influences body but not cache key); the rest follows
3. `X-HTTP-Method-Override` is high-yield against backends that route on it before checking method-based auth — most useful from server-side / non-browser callers (it triggers CORS preflight in a browser, so not a CSRF primitive)
4. Smuggling lives at the boundary — identify the proxy → backend pair (CDN → origin, ingress → service) and target the framing disagreement
4. If a header test exposes message-boundary disagreement, switch to the dedicated request-smuggling workflow and identify the proxy → backend pair
5. `X-Original-URL` / `X-Rewrite-URL` against IIS / ASP.NET admin endpoints is still a high-yield bypass
6. Before claiming a CRLF win, verify the second line landed as a real header in the cache or downstream consumer — many servers strip CRLF silently
7. Outbound email flows are a separate but related surface — user input flowing into SMTP headers (To, Cc, Subject, Reply-To) is its own injection class with the same root cause
@@ -10,13 +10,16 @@ Insecure deserialization passes attacker-controlled byte streams or structured b
## Attack Surface
**Formats**
- Java: Java native serialization, XStream, JSON → object mappers (Jackson, Fastjson), YAML (SnakeYAML)
- Java: Java native serialization, XStream, JSON → object mappers (Jackson, Fastjson), YAML (SnakeYAML), Hessian/Burlap, Kryo
- Python: `pickle`, `yaml.load` (unsafe), `marshal`, shelve
- PHP: `unserialize()`, Phar deserialization
- .NET: `BinaryFormatter`, `Json.NET TypeNameHandling`, ViewState
- Ruby: `Marshal.load`, YAML.load
- Node.js: `node-serialize`, `unserialize.js` (less common; see prototype_pollution for merge bugs)
**Transports and Containers**
- Java RMI/JMX, HTTP/RPC endpoints, messaging protocols, queues, signed wrappers, and product-specific binary envelopes can carry one or more formats above
**Input Locations**
- Cookies, session tokens, hidden form fields
- API parameters (`data`, `state`, `object`, base64 blobs)
@@ -58,6 +61,22 @@ yaml.load readObject( TypeNameHandling Marshal.load
```
When `enableDefaultTyping` or `@JsonTypeInfo` allows attacker-chosen types.
**JNDI Pivots from Object Construction**
JNDI injection is not itself a serialization format. It becomes part of this workflow when an attacker-selected type, setter, or gadget performs `Context.lookup()` during object construction or property population. `JdbcRowSetImpl` and some historical polymorphic JSON chains are examples; Log4j lookups reach JNDI through a different input path and should not be classified as deserialization.
- Trace fields such as `dataSourceName`, `jndiName`, and `namingURL` into the exact lookup API and provider.
- Record the accepted schemes/provider factories (`ldap`, `ldaps`, `rmi`, DNS URL context, or application-specific naming providers). A `dns://` value is not a universal oracle; it works only when the relevant DNS provider and lookup path are present.
- Separate network lookup, remote object/reference processing, serialized LDAP attributes, remote codebase loading, and local object-factory invocation. Each is a different capability with different runtime controls.
- JEP 290 filters incoming Java serialization graphs; it does not disable JNDI remote codebase loading. JNDI providers gained separate remote-class-loading and serialized-data controls across JDK updates, and current JDKs disable remote code downloading by default. Record the exact JDK build and relevant provider properties instead of using a single “modern Java” rule.
- When remote class loading is unavailable, test whether the returned reference can reach a compatible **local** `ObjectFactory`, bean-property path, expression engine, script engine, or other class already present. Confirm exact class names, versions, module access, and trigger methods from the deployed classpath.
**Hessian / Burlap**
- Binary RPC formats deserialized by `HessianInput`/`Hessian2Input`. Attacker object graphs reach gadgets even though it is not native Java serialization.
- Treat serializer version, allowed type metadata, constructors/setters invoked, collection/comparator behavior, and classpath as independent prerequisites.
- Pair `semantic_confusion` when a proxy or route policy is expected to make the RPC endpoint unreachable.
- Inspect the exact deployed libraries rather than relying on generic gadget labels; similar-looking Spring, Resin, Tomcat, XBean, EL, or Groovy classes are not interchangeable.
### Python Pickle
Pickle executes arbitrary code during unpickling by design:
@@ -162,6 +181,8 @@ When `TypeNameHandling` != `None`.
3. Check cookies named `JSESSIONID` alternatives, `.ASPXAUTH`, `laravel_session`, custom tokens
4. In white-box, trace from `readObject`/`unserialize`/`pickle.loads` backward to source
5. ViewState MAC off is still common on legacy ASP.NET — test early on `.aspx` apps
6. Model JNDI lookup, reference/object processing, remote codebase loading, and local factory invocation as separate stages
7. A "blocked" enterprise deserialization endpoint may still be reachable through a proxy/path-normalization mismatch — pair `semantic_confusion`
## Tooling
@@ -172,6 +193,7 @@ Payload generation is the practitioner's core tool here. The sandbox has `git`/`
| **ysoserial** (frohoff) | Java native | Gadget-chain payloads: `CommonsCollections1-7`, `Groovy1`, `Spring1/2`, and `URLDNS` for a safe no-exec DNS oracle. Needs a JRE. |
| **phpggc** (ambionics) | PHP `unserialize` / Phar | Framework POP chains (Laravel, Symfony, WordPress, Drupal, Monolog). Needs `php-cli`. |
| **ysoserial.net** | .NET `BinaryFormatter` / Json.NET | Windows/.NET gadget payloads. Needs .NET/mono — usually out of scope in a Linux sandbox. |
| **marshalsec** | Java Hessian/Burlap, Kryo, JSON, and JNDI reference tooling | Use only from a reviewed, pinned upstream commit when a non-native Java marshaller requires it. It has no stable release and intentionally bundles historical gadget dependencies; do not treat it as a globally installed default tool. |
```
# Java: prove the sink with a no-exec DNS oracle BEFORE any RCE chain
@@ -67,6 +67,8 @@ Upload surfaces are high risk: server-side execution (RCE), stored XSS, malware
- Double extensions: avatar.jpg.php, report.pdf.html; mixed casing: .pHp, .PhAr
- Magic-byte spoofing: valid JPEG header then embedded script; verify server uses content inspection, not extensions alone
- Detector/consumer differential: make the upload validator and the later parser disagree about type, structure, or validity
- Probe detector scan windows, recursion/nesting limits, maximum bytes inspected, invalid-syntax recovery, and version-specific magic databases
### Archive Attacks
@@ -120,6 +122,8 @@ Upload surfaces are high risk: server-side execution (RCE), stored XSS, malware
- Client-side only checks; relying on JS/MIME provided by browser
- Trusting multipart boundary part headers blindly
- Extension allowlists without server-side content inspection
- One parser validates metadata or leading bytes while another parser processes the full file
- Type-detection wrappers assumed identical even when they bundle different library/database versions
### Evasion Tricks
@@ -146,8 +150,9 @@ Upload surfaces are high risk: server-side execution (RCE), stored XSS, malware
1. **Map the pipeline** - Client → ingress → storage → processors → serving. Note where validation and auth occur
2. **Identify allowed types** - Size limits, filename rules, storage keys, and who serves the content
3. **Collect baselines** - Capture resulting URLs and headers for legitimate uploads
4. **Exercise bypass families** - Extension games, MIME/content-type, magic bytes, polyglots, metadata payloads, archive structure
5. **Validate execution** - Can uploaded content execute on server or client?
4. **Map validators and consumers** - Identify the detector/library/version when possible and every later parser, converter, renderer, or browser context
5. **Exercise bypass families** - Extension games, MIME/content-type, magic bytes, parser limits, polyglots, metadata payloads, archive structure
6. **Validate execution** - Prove the accepted object reaches a more privileged consumer and can execute or render active content
## Validation
@@ -182,6 +187,7 @@ Upload surfaces are high risk: server-side execution (RCE), stored XSS, malware
8. When you cannot get execution, aim for stored XSS or header-driven script execution
9. Validate that CDNs honor attachment/nosniff
10. Document full pipeline behavior per asset type
11. Reproduce detector/consumer mismatches on the deployed library versions; OS packages and language bindings may ship different limits
## Summary
@@ -11,6 +11,7 @@ Improper file path handling and dynamic inclusion enable sensitive file disclosu
**Path Traversal**
- Read files outside intended roots via `../`, encoding, normalization gaps
- Write or create files outside intended roots, then evaluate framework-controlled resolution paths separately from direct web access
**Local File Inclusion (LFI)**
- Include server-side files into interpreters/templates
@@ -51,7 +52,7 @@ Improper file path handling and dynamic inclusion enable sensitive file disclosu
### Capability Probes
- Path traversal baseline: `../../etc/hosts` and `C:\Windows\win.ini`
- Encodings: `%2e%2e%2f`, `%252e%252e%252f`, `..%2f`, `..%5c`, mixed UTF-8 (`%c0%2e`), Unicode dots and slashes
- Encodings: `%2e%2e%2f`, `%252e%252e%252f`, `..%2f`, `..%5c`, and Unicode lookalikes only where a documented conversion layer maps them to path syntax
- Normalization tests: `..../`, `..\\`, `././`, trailing dot/double dot segments; repeated decoding
- Absolute path acceptance: `/etc/passwd`, `C:\Windows\System32\drivers\etc\hosts`
- Server mismatch: `/static/..;/../etc/passwd` ("..;"), encoded slashes (`%2F`), double-decoding via upstream
@@ -69,7 +70,7 @@ Improper file path handling and dynamic inclusion enable sensitive file disclosu
### OAST
- RFI/LFI with wrappers that trigger outbound fetches (HTTP/DNS) to confirm inclusion/execution
- For RFI or URL-capable resource loaders, a correlated callback confirms server-side resolution/fetch. It does not by itself prove inclusion or execution; use a separate response or side-effect oracle for that claim.
### Side Effects
@@ -81,7 +82,7 @@ Improper file path handling and dynamic inclusion enable sensitive file disclosu
### Path Traversal Bypasses
**Encodings**
- Single/double URL-encoding, mixed case, overlong UTF-8, UTF-16, path normalization oddities
- Single/double URL-encoding, mixed case, UTF-16 or Unicode conversion only when present in the stack, and path normalization oddities
**Mixed Separators**
- `/` and `\\` on Windows; `//` and `\\\\` collapse differences across frameworks
@@ -147,13 +148,38 @@ Improper file path handling and dynamic inclusion enable sensitive file disclosu
- Verify symlink handling and path canonicalization prior to write
- Impact: overwrite config/templates or drop webshells into served directories
### File Write to Execution
Characterize the write primitive before choosing a payload:
- create vs overwrite vs append; atomic replace vs streamed write
- absolute vs relative path; controllable directory, filename, extension, and bytes
- text encoding, newline conversion, templating, compression, or report generation applied before write
- target process permissions and whether symlinks are followed
- immediate load, hot reload, cache invalidation, restart, scheduled task, or user action required
Then inventory generic execution and influence surfaces:
- view/template search paths and implicit rendering
- module, controller, plugin, package, or class autoload directories
- application bootstrap files and language package initializers
- server/user configuration that changes handler or interpreter behavior
- job definitions, hooks, startup scripts, cron/task inputs, and CI workspace files
- logs, sessions, caches, generated sources, and compiled-template directories later included or evaluated
Do not require the malicious file to be directly web-accessible. An HTTP extension allowlist can block `/path/payload.ext` while an internal view engine, autoloader, or interpreter still opens and executes that file through a clean route. Trace public request filtering and internal file resolution as separate security boundaries.
Test search order with candidate marker files or filesystem traces. Trigger the normal route/action that causes internal resolution. Record whether the framework creates, compiles, caches, or executes the artifact and what reload condition is required.
## Testing Methodology
1. **Inventory file operations** - Downloads, previews, templates, logs, exports/imports, report engines, uploads, archive extractors
2. **Identify input joins** - Path joins (base + user), include/require/template loads, resource fetchers, archive extract destinations
3. **Probe normalization** - Separators, encodings, double-decodes, case, trailing dots/slashes
4. **Compare behaviors** - Web server vs application behavior
5. **Escalate** - From disclosure (read) to influence (write/extract/include), then to execution (wrapper/engine chains)
5. **Characterize writes** - Determine create/overwrite/append, path and byte control, permissions, and reload/trigger conditions
6. **Map resolvers** - Test template/view search paths, autoloaders, plugins, configs, jobs, and other internal consumers separately from direct file serving
7. **Escalate** - From disclosure (read) to influence (write/extract/include), then to execution through a proven resolver or interpreter
## Validation
@@ -161,7 +187,8 @@ Improper file path handling and dynamic inclusion enable sensitive file disclosu
2. For LFI, demonstrate inclusion of a benign local file or harmless wrapper output (`php://filter` base64 of index.php)
3. For RFI, prove remote fetch by OAST or controlled output; avoid destructive payloads
4. For Zip Slip, create an archive with `../` entries and show write outside target (e.g., marker file read back)
5. Provide before/after file paths, exact requests, and content hashes/lengths for reproducibility
5. For file-write chains, first prove a canary is created at the intended path, then prove the normal resolver loads it; document cache/reload requirements
6. Provide before/after file paths, exact requests, and content hashes/lengths for reproducibility
## False Positives
@@ -184,6 +211,7 @@ Improper file path handling and dynamic inclusion enable sensitive file disclosu
3. For LFI, prefer `php://filter` base64 probes over destructive payloads; enumerate readable logs and sessions
4. Validate extraction code with synthetic archives; include symlinks and deep `../` chains
5. Use minimal PoCs and hard evidence (hashes, paths). Avoid noisy DoS against filesystems
6. When direct execution is blocked, enumerate internal search paths before assuming the write is low impact
## Summary
@@ -0,0 +1,189 @@
---
name: semantic-confusion
description: Cross-component semantic confusion testing for parser differentials, normalization mismatches, overloaded fields, lifecycle state drift, internal redirects, protocol translation, and validator-to-sink inconsistencies
---
# Semantic Confusion
Use this skill when two or more components consume the same attacker-influenced value. The central question is not merely whether input is validated, but whether every consumer assigns the same meaning to the value at the moment it makes a security decision.
Typical chains cross a validator, router, proxy, framework, parser, filesystem, interpreter, cache, or browser. A value can be safe in one representation and dangerous after a later decode, normalization, fallback, or field mutation.
## Authorization and Safety Boundary
- Run active differentials only against explicit authorized targets. Preserve destination allowlists and set request, rate, body, response, timeout, and retry ceilings.
- Perform malformed framing, delayed-body, oversized-input, crash, or resource-exhaustion cases only in a restartable isolated lab with health monitoring.
- Use synthetic canaries, reversible actions, non-secret protected resources, or a constant per-test callback identifier. Never place target-derived secrets in an OAST label/body.
- Change one representation axis at a time so the security-relevant disagreement remains attributable to a specific boundary.
- Pair `browser_security` when the final consumer is a browser context, worker, cache, or navigation state machine.
- Do not load this skill for pure ownership drift where every component resolves and interprets the name consistently; use `infrastructure_lifecycle` unless a representation, alias, identity, or resolution-result mismatch is present.
## Core Model
Build a transformation graph before spraying payloads:
```text
raw bytes
-> transport parser
-> proxy / middleware representation
-> authorization or validation decision
-> rewrite / decode / normalization
-> internal redirect or dispatch
-> final sink interpretation
```
For every edge, record:
- exact input representation: bytes, string, URL, path, header list, object, or structured field
- owning component and implementation/version
- transformation performed, including error and fallback behavior
- security decision made before or after the transformation
- whether the original and transformed values remain available simultaneously
- whether a field changes semantic type, such as filename to URL or MIME type to handler
The highest-signal condition is `security_check(value_A)` followed by `sink(transform(value_A))` where the checked and consumed representations are not equivalent.
## High-Value Confusion Classes
### Parser Differentials
- Compare browser, framework, proxy, library, and backend parsing of the exact same bytes.
- Test duplicate and comma-joined fields, first-match vs last-match behavior, invalid-token recovery, comments, quoting, and empty members.
- Include structured formats and metadata: URL, MIME, JSON, multipart, XML, cookies, forwarded headers, and serialized objects.
- Treat leniency as a security feature only when every downstream consumer is equally lenient in the same way.
### Normalization and Canonicalization Drift
- Map percent-decoding count, Unicode conversion, slash/backslash handling, dot-segment removal, case folding, IDNA, numeric IP conversion, and filesystem cleanup.
- Compare string-prefix checks with segment-aware or origin-aware comparisons.
- Test malformed Unicode and replacement behavior; a rejected code point may become an allowed delimiter or wildcard later.
- Test path, query, and fragment separately. Browsers and routers commonly transform each source differently.
### Field and Type Overloading
- Identify shared fields reused for different concepts: path vs URL, content type vs handler, display name vs executable name, route vs filesystem location.
- Trace every writer and reader of the field across the complete lifecycle.
- Look for implicit fallback: when the intended field is empty, another field becomes authoritative.
- Exercise fields after errors, rewrites, subrequests, retries, internal redirects, and protocol upgrades/downgrades.
### Lifecycle and State Drift
- Trigger error paths that should terminate processing and verify that later phases actually stop.
- Look for stale metadata copied into a new request, subrequest, background job, cache entry, or retry.
- Compare direct external access with internal dispatch. Edge controls may inspect the public URL while an internal resolver opens a different path or invokes a different handler.
- Test order-dependent behavior: validation before rewrite, auth before route normalization, or content classification before processing.
### Boundary Translation
- Map HTTP/2 to HTTP/1 translation, proxy to application rewriting, URL to filesystem resolution, upload detector to content consumer, and client router to API request construction.
- In a restartable lab and only when supported by evidence, vary framing, bounded delays/body sizes, content type, pseudo-headers, and method conversion. Check target health after resource-sensitive cases.
- Do not assume a WAF or authorization sidecar sees the full body or final normalized request.
### Namespace and Resolution Fallback
- Identify names resolved across multiple scopes: local path, environment `PATH`, cache, private registry, public registry, plugin directory, template search path, or autoloader.
- Record lookup order and what happens when the intended entry is missing.
- Compare protected package/module names with exposed command, binary, handler, or alias names. For npm, a scoped package can expose an unscoped `bin` name, so the protected package name and invoked executable may differ.
- Treat automatic remote fallback or search-path fallback as an execution boundary.
- Load `npx_confusion` when `npx` or `npm exec` may reinterpret a missing executable as a public package spec.
## Reconnaissance
### Black-Box Mapping
1. Capture a clean baseline with raw request and response bytes.
2. Change one representation axis at a time: encoding depth, delimiter, duplicate, separator, method, protocol, body framing, or Unicode form.
3. Diff status, headers, body digest/length, timing, redirects, cache state, and out-of-band callbacks.
4. Replay through different paths: direct origin vs CDN, HTTP/1.1 vs HTTP/2, public route vs alternate host, synchronous vs background processing.
5. Cluster responses by behavior before escalating. Small differentials reveal component boundaries.
### Source-Aware Mapping
- Find every read and write of shared request/context fields, not just the obvious sink.
- Trace route matching, auth middleware, rewrites, internal redirects, handler selection, and response generation in execution order.
- Inventory decode/parse/normalize calls and note whether return values or errors are ignored.
- Search for compatibility fallbacks, legacy aliases, permissive recovery, default handlers, and search-path iteration.
- Inspect packaging and deployment defaults; distro configuration, enabled modules, plugins, and symlinks often determine reachability.
## Differential Test Matrix
Build a bounded matrix from relevant axes instead of blindly combining everything:
| Axis | Representative variants |
|---|---|
| Encoding | raw, once encoded, twice encoded, mixed case, malformed Unicode |
| Structure | duplicate, comma-joined, empty member, quoted, comment-like suffix |
| Path | `/`, `\\`, `//`, dot segments, absolute, sibling-prefix collision |
| URL | userinfo, numeric IP, alternate IP radix, trailing dot, fragment/query split |
| Transport | HTTP/1.1, HTTP/2, chunked/fixed body, delayed DATA, oversized body |
| Lifecycle | normal, error, retry, internal redirect, cache hit, background worker |
| Consumer | edge, application, library, filesystem, interpreter, browser |
Select axes supported by evidence from the target. Record which component saw which representation.
### Repeatable Harnesses
- For two local parsers, canonicalizers, or validator/consumer functions, load `hypothesis` and express the expected relationship as a property. Bound sizes/examples and keep the minimized disagreement as a regression test.
- For an ordered HTTP flow with cookies, redirects, captured values, and assertions, load `hurl` and encode vulnerable, fixed, and negative-control environments using the same request chain.
- Use raw-byte or protocol-specific harnesses when a high-level HTTP client would normalize the ambiguity away.
- Separate input generation from transport. Generators that are safe against pure local functions become active fuzzers when connected to a live target.
## Chaining Strategy
Treat the first differential as a primitive, then ask what authority the later consumer has:
- auth or ACL bypass -> protected route or file
- path/URL confusion -> source disclosure, SSRF, local socket, or unintended handler
- detector/consumer mismatch -> active upload processing or inline browser execution
- internal redirect state carryover -> handler selection or policy bypass
- search-path or namespace fallback -> attacker-controlled code resolution
- browser/router decode -> client-side path traversal, CSRF-like action, SSRF, or XSS sink
Enumerate existing local gadgets only after the primitive is proven. Prefer generic classes such as interpreters, template engines, debug tools, package scripts, local sockets, and autoload paths over a vendor-specific file list.
## Testing Methodology
1. **Define the invariant** - State what all components are expected to agree on: origin, path, type, handler, identity, length, or package name.
2. **Draw the graph** - List consumers and transformations in real execution order.
3. **Locate early decisions** - Mark validation, auth, WAF, cache, and routing checks.
4. **Locate late meaning changes** - Mark decodes, rewrites, fallback, internal dispatch, and sink parsing.
5. **Build a focused matrix** - Exercise only transformations supported by the stack.
6. **Isolate the disagreement** - Produce paired inputs that differ at one boundary and explain both interpretations.
7. **Prove the primitive safely** - Use a synthetic protected canary, reversible marker, constant callback identifier, or no-op handler whose behavior and side effects are understood.
8. **Escalate by capability** - Track Read -> influence -> write -> dispatch -> execute transitions with evidence and prerequisites for every edge.
9. **Cross-check versions/configurations** - Reproduce on a fixed version or hardened configuration when possible.
## Validation
A valid confusion finding should include:
1. the exact bytes or structured input supplied
2. the representation observed by the security control
3. the different representation observed by the final consumer
4. the transformation or lifecycle event that created the difference
5. paired control and exploit results across repeat runs
6. version, protocol, configuration, and interaction prerequisites
7. a minimal impact proof that does not depend on unrelated undefined behavior
## False Positives
- Different error messages with identical final authorization and sink behavior
- A parser accepts odd syntax but downstream consumers preserve the same safe meaning
- A normalization difference visible only in logs, with no security decision between representations
- WAF bypass where the application itself rejects the request identically
- Version-specific behavior claimed as universal without testing the relevant deployment
- A search-path candidate that is attacker-named but cannot be created, claimed, loaded, or executed
## Pro Tips
1. Begin with relationships and shared state, not endpoint payload lists.
2. Preserve raw traffic; high-level clients often normalize away the exploit before sending it.
3. Error paths are alternate lifecycles. Verify which fields survive and which phases still execute.
4. Compare direct and internal access separately; ingress policy rarely governs framework file IO or handler dispatch.
5. When a prefix allowlist is used, test a sibling sharing the prefix and verify with a segment-aware comparison.
6. Distinguish presence, reachability, and impact. Each needs separate evidence.
7. Generalize a finding by naming the disagreement class, not by copying its final payload.
## Summary
Semantic confusion exists when a security decision and a privileged consumer disagree about the meaning of the same attacker-influenced data. Model the entire transformation lifecycle, isolate one disagreement at a time, and prove both interpretations. The reusable unit is the boundary and its invariant—not a CVE-specific string.
+30 -1
View File
@@ -37,6 +37,7 @@ def _render_completion_report(
result_summary: str,
findings: list[str],
recommendations: list[str],
open_items: list[str],
) -> str:
"""Render a child's completion report as plain structured text.
@@ -61,6 +62,12 @@ def _render_completion_report(
lines.append("")
lines.append("Findings:")
lines.extend(f"- {f}" for f in findings)
lines.append("")
lines.append("Open items (unresolved, need follow-up):")
if open_items:
lines.extend(f"- {o}" for o in open_items)
else:
lines.append("- (none)")
if recommendations:
lines.append("")
lines.append("Recommendations:")
@@ -445,7 +452,12 @@ async def create_agent(
name: Human-readable child name (used in graph views and
``send_message_to_agent`` flows).
task: Specific objective. Be concrete what to test, what
success looks like, any constraints.
success looks like, any constraints. Name the target the
child should call ``get_threat_model`` on, and any shared
state it should build on rather than rediscover what
recon already mapped, which surfaces are already covered,
which coverage entry it is picking up. A child that is not
told what is already known repeats it.
inherit_context: Default ``True``. The child receives the
parent's input history as background; only set ``False``
when starting a clean-slate task.
@@ -520,6 +532,7 @@ async def agent_finish(
ctx: RunContextWrapper,
result_summary: str,
findings: list[str] | None = None,
open_items: list[str] | None = None,
success: bool = True,
report_to_parent: bool = True,
final_recommendations: list[str] | None = None,
@@ -544,6 +557,14 @@ async def agent_finish(
doing: what did you test, what did you find/confirm/rule out,
what's still open.
**Close out honestly.** Before calling this, every surface you
assessed should have a ``record_coverage`` entry, and anything you
could neither confirm nor rule out belongs in ``open_items`` an
unresolved candidate handed up to the parent is useful, a silently
dropped one is a missed vulnerability. Reporting nothing and
listing no open items asserts the area is clean; only say that if
you mean it.
Args:
result_summary: What you accomplished and discovered. Concrete
and specific (URLs, parameters, payloads that worked).
@@ -552,6 +573,12 @@ async def agent_finish(
``create_vulnerability_report`` first (or
``create_dependency_report`` for dependency CVEs); this is
for narrative.
open_items: Candidates you could NOT confirm and could NOT rule
out with a named control, plus anything you ran out of time
or access to test. State the specific gap (e.g. "password
reset token entropy could not obtain a second account to
compare tokens"). Pass an empty list only when nothing is
genuinely left open.
success: Whether the assigned subtask was completed
successfully. Default ``True``.
report_to_parent: Whether to deliver the completion report to
@@ -595,6 +622,7 @@ async def agent_finish(
result_summary=result_summary,
findings=list(findings or []),
recommendations=list(final_recommendations or []),
open_items=list(open_items or []),
)
await coordinator.send(
parent_id,
@@ -629,6 +657,7 @@ async def agent_finish(
"agent_id": me,
"summary": result_summary,
"findings_count": len(findings or []),
"open_items_count": len(open_items or []),
"has_recommendations": bool(final_recommendations),
},
ensure_ascii=False,
+1
View File
@@ -0,0 +1 @@
"""Scan coverage accounting — what was reviewed, and how it closed."""
+535
View File
@@ -0,0 +1,535 @@
"""Per-run coverage ledger — mirrored to {state_dir}/coverage.json.
Findings answer "what did we find". Coverage answers "what did we look at,
and how did each one close" — the negative space a client report needs in
order to be trustworthy. Every agent records the surfaces it reviewed; the
root agent reconciles them at the end of the scan.
Entries here are **agent-reported**: an agent's own account of what it
assessed. ``strix.report.coverage`` pairs them with machine-observed facts
(which agents ran, which skills they carried, how the run terminated) and
labels the provenance of each, so a reader can tell a self-report from an
observation. The runtime mirror under ``{state_dir}`` exists for resume; the
client-facing artifact is ``{run_dir}/coverage.json``.
"""
from __future__ import annotations
import asyncio
import json
import logging
import tempfile
import threading
import uuid
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
from agents import RunContextWrapper, function_tool
logger = logging.getLogger(__name__)
_coverage_storage: dict[str, dict[str, Any]] = {}
_coverage_lock = threading.RLock()
_coverage_path: Path | None = None
_ENTRY_ID_GENERATION_ATTEMPTS = 1024
_EVIDENCE_PREVIEW_CHARS = 240
VALID_OUTCOMES: tuple[str, ...] = (
"reported",
"no_issue_found",
"ruled_out",
"not_applicable",
"needs_follow_up",
)
_OUTCOMES_REQUIRING_EVIDENCE = frozenset({"ruled_out", "not_applicable", "needs_follow_up"})
def _caller_identity(ctx: RunContextWrapper) -> tuple[str | None, str | None]:
"""Return the (agent_id, agent_name) of the agent invoking this tool."""
inner = ctx.context if isinstance(ctx.context, dict) else {}
raw_agent_id = inner.get("agent_id")
agent_id = raw_agent_id if isinstance(raw_agent_id, str) else None
agent_name: str | None = None
coordinator = inner.get("coordinator")
if agent_id is not None and coordinator is not None:
names = getattr(coordinator, "names", {})
if isinstance(names, dict):
raw_agent_name = names.get(agent_id)
agent_name = raw_agent_name if isinstance(raw_agent_name, str) else None
return agent_id, agent_name
def _generate_entry_id() -> str | None:
"""Allocate an unused entry id. Callers must already hold ``_coverage_lock``."""
for _ in range(_ENTRY_ID_GENERATION_ATTEMPTS):
entry_id = uuid.uuid4().hex[:6]
if entry_id not in _coverage_storage:
return entry_id
return None
def hydrate_coverage_from_disk(state_dir: Path) -> None:
global _coverage_path # noqa: PLW0603
_coverage_path = state_dir / "coverage.json"
with _coverage_lock:
_coverage_storage.clear()
if not _coverage_path.exists():
return
try:
data = json.loads(_coverage_path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
logger.exception(
"coverage.json at %s is unreadable; starting with empty coverage",
_coverage_path,
)
return
if not isinstance(data, dict):
return
_coverage_storage.update(
{
eid: entry
for eid, entry in data.items()
if isinstance(eid, str) and isinstance(entry, dict)
}
)
logger.info(
"coverage hydrated from %s (%d entr(ies))",
_coverage_path,
len(_coverage_storage),
)
def _persist_locked() -> None:
"""Mirror the ledger to disk. Callers must already hold ``_coverage_lock``.
Serialization and the rename happen in one critical section. Releasing
the lock in between would let a writer holding an older serialization win
the rename and silently roll back a concurrent agent's entry, so the
ledger would hydrate short on resume.
"""
path = _coverage_path
if path is None:
return
try:
payload = json.dumps(_coverage_storage, ensure_ascii=False, default=str)
path.parent.mkdir(parents=True, exist_ok=True)
with tempfile.NamedTemporaryFile(
mode="w",
encoding="utf-8",
dir=str(path.parent),
prefix=f".{path.name}.",
suffix=".tmp",
delete=False,
) as tmp:
tmp.write(payload)
tmp_path = Path(tmp.name)
tmp_path.replace(path)
except Exception:
logger.exception("coverage persist to %s failed", path)
def get_coverage_entries() -> list[dict[str, Any]]:
"""Return every coverage entry, newest last. Used by ``finish_scan``."""
with _coverage_lock:
entries = [{**entry, "entry_id": eid} for eid, entry in _coverage_storage.items()]
entries.sort(key=lambda e: str(e.get("created_at", "")))
return entries
def outcome_counts() -> dict[str, int]:
"""Count coverage entries per outcome, in the canonical outcome order."""
counts: dict[str, int] = {}
for entry in get_coverage_entries():
outcome = str(entry.get("outcome", "")).lower()
counts[outcome] = counts.get(outcome, 0) + 1
return {o: counts[o] for o in VALID_OUTCOMES if o in counts}
def _validate(
*, surface: str, risk_area: str, outcome: str, evidence: str
) -> tuple[str, list[str]]:
errors: list[str] = []
if not surface.strip():
errors.append("surface cannot be empty - name the endpoint, route, file, or component")
if not risk_area.strip():
errors.append("risk_area cannot be empty - name what you were testing for")
normalized = outcome.strip().lower().replace("-", "_").replace(" ", "_")
if normalized not in VALID_OUTCOMES:
errors.append(f"Invalid outcome: {outcome!r}. Must be one of: {list(VALID_OUTCOMES)}")
elif normalized in _OUTCOMES_REQUIRING_EVIDENCE and not evidence.strip():
errors.append(
f"evidence is required for outcome '{normalized}' - name the specific control, "
"the reason it does not apply, or what is still missing"
)
return normalized, errors
def _duplicate_of_locked(surface: str, risk_area: str) -> tuple[str, dict[str, Any]] | None:
"""Find an existing row for this exact surface and risk area.
Callers must already hold ``_coverage_lock``. The uniqueness check and the
insertion that depends on it have to be one critical section: otherwise
two agents recording the same surface concurrently both see "no
duplicate", and the ledger ends up with exactly the parallel rows this
rejection exists to prevent.
"""
key = (surface.strip().lower(), risk_area.strip().lower())
for entry_id, entry in _coverage_storage.items():
existing = (
str(entry.get("surface", "")).strip().lower(),
str(entry.get("risk_area", "")).strip().lower(),
)
if existing == key:
return entry_id, dict(entry)
return None
def _record_impl(
*,
surface: str,
risk_area: str,
outcome: str,
evidence: str,
agent_id: str | None,
agent_name: str | None,
) -> dict[str, Any]:
normalized, errors = _validate(
surface=surface, risk_area=risk_area, outcome=outcome, evidence=evidence
)
if errors:
return {"success": False, "error": "Validation failed", "errors": errors}
entry: dict[str, Any] = {
"surface": surface.strip(),
"risk_area": risk_area.strip(),
"outcome": normalized,
"created_at": datetime.now(UTC).strftime("%Y-%m-%d %H:%M:%S UTC"),
}
if evidence.strip():
entry["evidence"] = evidence.strip()
if agent_id:
entry["agent_id"] = agent_id
if agent_name:
entry["agent_name"] = agent_name
with _coverage_lock:
duplicate = _duplicate_of_locked(surface, risk_area)
if duplicate is not None:
existing_id, existing = duplicate
owner = existing.get("agent_name") or "another agent"
return {
"success": False,
"error": (
f"'{surface.strip()}' ({risk_area.strip()}) already has coverage entry "
f"{existing_id}, recorded by {owner} as "
f"'{existing.get('outcome', '')}'. Two rows for one surface leave the "
"report showing a stale conclusion beside its replacement. If your "
"review reached a different conclusion, move that entry with "
f"update_coverage(entry_id='{existing_id}', ...) and say in evidence "
"what changed. If you reviewed something genuinely different, name the "
"surface or risk area more precisely and record it again."
),
"existing_entry_id": existing_id,
"existing_outcome": existing.get("outcome", ""),
}
entry_id = _generate_entry_id()
if entry_id is None:
return {"success": False, "error": "Could not allocate a coverage entry id"}
_coverage_storage[entry_id] = entry
_persist_locked()
logger.info(
"Coverage recorded: id=%s outcome=%s surface=%s",
entry_id,
normalized,
entry["surface"],
)
return {
"success": True,
"entry_id": entry_id,
"outcome": normalized,
"message": f"Coverage recorded for '{entry['surface']}' ({normalized})",
}
def _update_impl(
*,
entry_id: str,
outcome: str,
evidence: str,
agent_id: str | None,
agent_name: str | None,
) -> dict[str, Any]:
key = (entry_id or "").strip()
with _coverage_lock:
existing = _coverage_storage.get(key)
if existing is None:
return {
"success": False,
"error": (
f"No coverage entry {entry_id!r}. Call list_coverage to find the "
"entry you mean - filter by surface if you only know the name."
),
}
surface = str(existing.get("surface", ""))
risk_area = str(existing.get("risk_area", ""))
normalized, errors = _validate(
surface=surface, risk_area=risk_area, outcome=outcome, evidence=evidence
)
if errors:
return {"success": False, "error": "Validation failed", "errors": errors}
previous_outcome = str(existing.get("outcome", ""))
superseded: dict[str, Any] = {
"outcome": previous_outcome,
"recorded_at": existing.get("created_at", ""),
}
if existing.get("evidence"):
superseded["evidence"] = existing["evidence"]
if existing.get("agent_name"):
superseded["agent_name"] = existing["agent_name"]
history = existing.get("history")
existing["history"] = [*history, superseded] if isinstance(history, list) else [superseded]
existing["outcome"] = normalized
existing["updated_at"] = datetime.now(UTC).strftime("%Y-%m-%d %H:%M:%S UTC")
if evidence.strip():
existing["evidence"] = evidence.strip()
if agent_id:
existing["agent_id"] = agent_id
if agent_name:
existing["agent_name"] = agent_name
_persist_locked()
logger.info(
"Coverage updated: id=%s %s -> %s surface=%s",
key,
previous_outcome,
normalized,
surface,
)
return {
"success": True,
"entry_id": key,
"previous_outcome": previous_outcome,
"outcome": normalized,
"message": (
f"'{surface}' ({risk_area}) moved from {previous_outcome} to {normalized}. "
"The previous state is kept as history."
),
}
def _list_impl(
*, outcome: str | None, surface: str | None, caller_agent_id: str | None
) -> dict[str, Any]:
normalized_outcome: str | None = None
if outcome and outcome.strip():
normalized_outcome = outcome.strip().lower().replace("-", "_").replace(" ", "_")
if normalized_outcome not in VALID_OUTCOMES:
return {
"success": False,
"error": f"Invalid outcome: {outcome!r}. Must be one of: {list(VALID_OUTCOMES)}",
}
entries: list[dict[str, Any]] = []
for entry in get_coverage_entries():
if normalized_outcome and entry.get("outcome") != normalized_outcome:
continue
if surface and surface.strip().lower() not in str(entry.get("surface", "")).lower():
continue
listing = {
"entry_id": entry.get("entry_id"),
"surface": entry.get("surface", ""),
"risk_area": entry.get("risk_area", ""),
"outcome": entry.get("outcome", ""),
"created_at": entry.get("created_at", ""),
}
evidence = str(entry.get("evidence", ""))
if evidence:
listing["evidence"] = (
f"{evidence[:_EVIDENCE_PREVIEW_CHARS].rstrip()}..."
if len(evidence) > _EVIDENCE_PREVIEW_CHARS
else evidence
)
agent_name = entry.get("agent_name")
if agent_name:
listing["agent_name"] = agent_name
history = entry.get("history")
if isinstance(history, list) and history:
listing["previous_outcomes"] = [str(h.get("outcome", "")) for h in history]
if caller_agent_id is not None and entry.get("agent_id") == caller_agent_id:
listing["by_you"] = True
entries.append(listing)
return {
"success": True,
"entries": entries,
"filtered_count": len(entries),
"total_count": len(_coverage_storage),
"outcome_counts": outcome_counts(),
}
@function_tool(timeout=30)
async def record_coverage(
ctx: RunContextWrapper,
surface: str,
risk_area: str,
outcome: str,
evidence: str = "",
) -> str:
"""Record that you reviewed a surface, and how that review closed.
A scan that only reports findings cannot answer the question every
client asks: *what did you actually check?* This tool captures that
negative space. Record an entry whenever you finish assessing a
surface for a risk including (especially including) when you found
nothing.
Record coverage as you go, not in a batch at the end. Entries are
shared across every agent in the scan, and the root agent reconciles
them into the final report.
Coverage is not append-only bookkeeping: if this surface and risk
already have an entry yours or another agent's — this call is
rejected and returns that entry's id, because two rows for one
surface leave the report showing a stale conclusion next to its
replacement. Call ``update_coverage`` on the id it hands you
instead. Resolving somebody else's ``needs_follow_up`` is exactly
that case.
**Outcomes** (pick exactly one):
- ``reported`` you confirmed an issue and filed a report for it.
- ``no_issue_found`` you tested this properly and found nothing.
- ``ruled_out`` you had a specific candidate and disproved it. The
``evidence`` must name the control that makes it safe, at a
location, and confirm it runs on every attacker-reachable path.
"It looked fine" is not ``ruled_out``.
- ``not_applicable`` this risk cannot apply here (e.g. no XML
parsing on a surface, so no XXE). Say why in ``evidence``.
- ``needs_follow_up`` plausible but unresolved: you could not
confirm it and could not name a control that rules it out. This is
a legitimate outcome. Use it rather than quietly dropping a
candidate, and name the gap in ``evidence`` (missing credentials,
service you could not start, unconfirmed reachability).
Never use ``no_issue_found`` or ``ruled_out`` to close something you
were simply unsure about that is ``needs_follow_up``. Missing
information is not proof of safety.
Args:
surface: What you reviewed an endpoint, route, parameter,
file, component, or host (e.g. ``"POST /api/orders/{id}"``,
``"src/auth/session.py"``, ``"admin dashboard"``).
risk_area: What you were testing it for (e.g. ``"IDOR /
object-level authorization"``, ``"SQL injection"``,
``"SSRF"``).
outcome: One of ``reported`` / ``no_issue_found`` /
``ruled_out`` / ``not_applicable`` / ``needs_follow_up``.
evidence: How you know. Required for ``ruled_out``,
``not_applicable``, and ``needs_follow_up``; recommended
otherwise. Keep it to a sentence or two name the control,
the test performed, or the missing piece.
"""
agent_id, agent_name = _caller_identity(ctx)
result = await asyncio.to_thread(
_record_impl,
surface=surface,
risk_area=risk_area,
outcome=outcome,
evidence=evidence,
agent_id=agent_id,
agent_name=agent_name,
)
return json.dumps(result, ensure_ascii=False, default=str)
@function_tool(timeout=30)
async def update_coverage(
ctx: RunContextWrapper,
entry_id: str,
outcome: str,
evidence: str = "",
) -> str:
"""Change how an already-recorded surface closed.
Coverage is shared across the whole agent tree, and a surface's
state is not final when it is first written. Use this whenever
later work changes the answer:
- You picked up someone's ``needs_follow_up`` and resolved it —
move it to ``reported``, ``ruled_out``, or ``no_issue_found``.
- You had the credentials or running service the original agent
lacked, and could finally test it properly.
- You found the control that rules a candidate out, at a location,
on every attacker-reachable path.
- You went the other way: something recorded ``no_issue_found`` or
``ruled_out`` turns out to be exploitable, or the control you see
does not cover the path you found. Move it back.
The surface and risk area stay fixed this is the same review,
reaching a different conclusion. Do not record a fresh entry for a
surface that already has one; that leaves a stale open item next to
its own resolution. Find the id with ``list_coverage`` (filter by
``surface``), then update it.
The previous outcome, evidence, and author are kept as history, so
the ledger still shows that the surface was once open and who
closed it.
Args:
entry_id: The id of the entry to update, from ``list_coverage``.
outcome: The new outcome ``reported`` / ``no_issue_found`` /
``ruled_out`` / ``not_applicable`` / ``needs_follow_up``.
evidence: How you know, now. Required for ``ruled_out``,
``not_applicable``, and ``needs_follow_up``. Say what
changed, not just what you concluded the reader needs to
know why this closed differently the second time.
"""
agent_id, agent_name = _caller_identity(ctx)
result = await asyncio.to_thread(
_update_impl,
entry_id=entry_id,
outcome=outcome,
evidence=evidence,
agent_id=agent_id,
agent_name=agent_name,
)
return json.dumps(result, ensure_ascii=False, default=str)
@function_tool(timeout=30)
async def list_coverage(
ctx: RunContextWrapper,
outcome: str | None = None,
surface: str | None = None,
) -> str:
"""List coverage entries recorded so far in this scan.
**For the orchestrator / root agent.** Use it to see which surfaces
have been assessed, spot gaps before finishing, and pull the
unresolved ``needs_follow_up`` rows into the final report. Leaf
agents should record their own coverage and get on with testing.
Returns each entry with its ``surface``, ``risk_area``, ``outcome``,
evidence preview, and the agent that recorded it, plus
``outcome_counts`` across the whole scan.
Args:
outcome: Optional filter one of ``reported`` /
``no_issue_found`` / ``ruled_out`` / ``not_applicable`` /
``needs_follow_up``. Filter on ``needs_follow_up`` before
finishing the scan to see what is still open.
surface: Optional case-insensitive substring filter on the
surface name.
"""
caller_agent_id, _ = _caller_identity(ctx)
result = await asyncio.to_thread(
_list_impl, outcome=outcome, surface=surface, caller_agent_id=caller_agent_id
)
return json.dumps(result, ensure_ascii=False, default=str)
+66 -1
View File
@@ -22,6 +22,7 @@ def _do_finish(
methodology: str,
technical_analysis: str,
recommendations: str,
agent_graph: dict[str, Any],
) -> dict[str, Any]:
if parent_id is not None:
return {
@@ -63,6 +64,7 @@ def _do_finish(
recommendations=recommendations.strip(),
)
vuln_count = len(report_state.vulnerability_reports)
coverage_summary = _coverage_summary(agent_graph)
except (ImportError, AttributeError) as e:
logger.exception("finish_scan persistence failed")
return {"success": False, "error": f"Failed to complete scan: {e!s}"}
@@ -71,12 +73,66 @@ def _do_finish(
"finish_scan: completed scan with %d vulnerability report(s)",
vuln_count,
)
return {
result: dict[str, Any] = {
"success": True,
"scan_completed": True,
"message": "Scan completed successfully",
"vulnerabilities_found": vuln_count,
}
result.update(coverage_summary)
return result
def _coverage_summary(agent_graph: dict[str, Any]) -> dict[str, Any]:
"""Coverage counts, unresolved surfaces, and gaps the runtime can see.
The gap list is derived from the agent graph rather than from the ledger,
so it catches the failure the ledger cannot: a risk class an agent was
equipped for and never accounted for. Surfacing it here in the response
to the call that ends the scan is the last point at which the root agent
can still dispatch work or record the class as unresolved instead of
letting the report imply it was clean.
"""
from strix.report.coverage import agents_from_graph, skill_coverage_gaps
from strix.tools.coverage.tools import get_coverage_entries, outcome_counts
entries = get_coverage_entries()
if not entries:
return {
"coverage_recorded": 0,
"coverage_warning": (
"No coverage was recorded for this scan. The report cannot show which "
"surfaces were reviewed and cleared — only what was found. Use "
"record_coverage during testing so future scans can report negative space."
),
}
counts = outcome_counts()
summary: dict[str, Any] = {
"coverage_recorded": len(entries),
"coverage_outcomes": counts,
}
unresolved = [e for e in entries if e.get("outcome") == "needs_follow_up"]
if unresolved:
summary["coverage_warning"] = (
f"{len(unresolved)} surface(s) closed as 'needs_follow_up' and remain "
"unresolved. These should be represented in the report as areas requiring "
"further review rather than omitted."
)
summary["unresolved_surfaces"] = [
{"surface": e.get("surface", ""), "risk_area": e.get("risk_area", "")}
for e in unresolved
]
gaps = skill_coverage_gaps(entries, agents_from_graph(agent_graph))
if gaps:
summary["coverage_gaps"] = [gap["detail"] for gap in gaps]
summary["coverage_gap_warning"] = (
f"{len(gaps)} risk class(es) assigned to agents have no coverage entry and "
"will be published as unexamined. Record them (or a needs_follow_up row) "
"before the report goes out."
)
return summary
@function_tool(timeout=60)
@@ -141,6 +197,14 @@ async def finish_scan(
chain after a serious attempt is acceptable; skipping the
chaining reasoning, or ignoring a plausibly-related combination,
is not.
5. **Coverage reconciliation.** Call ``list_coverage`` and check
what was actually assessed against the surfaces you enumerated
during reconnaissance. Every surface you dispatched work on
should have a coverage entry; anything still open should be a
``needs_follow_up`` row, not a silent omission. If a significant
surface has no entry at all, dispatch an agent to cover it or
record it as ``needs_follow_up`` before finishing. The response
from this tool reports coverage counts and any unresolved rows.
**Calling this multiple times overwrites the previous report.**
Make the single call comprehensive.
@@ -280,6 +344,7 @@ async def finish_scan(
methodology=methodology,
technical_analysis=technical_analysis,
recommendations=recommendations,
agent_graph=await coordinator.snapshot() if coordinator is not None else {},
)
if (
result.get("success")
+25
View File
@@ -0,0 +1,25 @@
"""Generic MCP client: connect MCP servers and expose their tools."""
from __future__ import annotations
from strix.tools.mcp.client import ConnectedMcpServer, connect_mcp_servers
from strix.tools.mcp.config import (
BearerAuth,
McpAuth,
McpConnectionConfig,
)
from strix.tools.mcp.loader import load_user_mcp_configs
from strix.tools.mcp.naming import McpToolOrigin, namespaced_tool_name, resolve_mcp_tool
__all__ = [
"BearerAuth",
"ConnectedMcpServer",
"McpAuth",
"McpConnectionConfig",
"McpToolOrigin",
"connect_mcp_servers",
"load_user_mcp_configs",
"namespaced_tool_name",
"resolve_mcp_tool",
]
+349
View File
@@ -0,0 +1,349 @@
"""Connect to MCP servers and expose their tools to the agent.
Given one :class:`McpConnectionConfig` per server, :func:`connect_mcp_servers`
lists each server's tools, keeps the ones on the connection's allowlist (or all
of them when none is set), prefixes each with the connection name so servers do
not collide, and registers them through the agent factory. The factory applies
output bounding, per-call timeouts, and structured errors to every registered
tool, so this layer does not reimplement them.
A server that cannot connect, or a tool set that cannot be registered, is logged
and skipped, so one bad connection never fails the run.
"""
from __future__ import annotations
import contextlib
import json
import logging
from typing import TYPE_CHECKING, Any, NamedTuple, cast
from agents.exceptions import ModelBehaviorError
from agents.mcp import (
MCPServer,
MCPServerStdio,
MCPServerStdioParams,
MCPServerStreamableHttp,
MCPServerStreamableHttpParams,
MCPUtil,
create_static_tool_filter,
)
from strix.agents.factory import register_agent_tools
from strix.tools.mcp.naming import namespaced_tool_name
if TYPE_CHECKING:
from collections.abc import Callable
from agents.tool import FunctionTool, Tool
from mcp.types import Tool as MCPTool
from strix.tools.mcp.config import McpConnectionConfig
# Runs on each tool's structured result before it reaches the agent. Called
# ``result_transform(namespaced_tool_name, structured_result)`` and its return
# value becomes the tool's output. ``structured_result`` is the parsed
# ``CallToolResult`` as a dict (not a serialized string), so the transform can
# project or drop individual fields.
ResultTransform = Callable[[str, Any], Any]
logger = logging.getLogger(__name__)
class ConnectedMcpServer(NamedTuple):
"""One successfully connected MCP server and how many tools it registered.
``server`` is kept so the caller can clean it up when the run ends;
``name`` and ``tool_count`` let the caller show the user a startup summary;
``notes`` carries the connection's optional free-text description so the
caller can surface it to the agent as context about the connection.
"""
server: MCPServer
name: str
tool_count: int
notes: str | None = None
def _auth_headers(config: McpConnectionConfig) -> dict[str, str]:
"""Build the per-server request headers from the connection's auth."""
auth = config.auth
if auth is None:
return {}
return {"Authorization": f"Bearer {auth.token}"}
def _build_server(config: McpConnectionConfig) -> MCPServer:
"""Construct (but do not connect) the SDK server for one connection.
When ``allowed_tools`` is a list the static filter means the server will not
even list tools outside it; :func:`_register_server_tools` re-applies the
same allowlist as the authoritative gate on what gets registered. When it is
``None`` no filter is applied and every listed tool is registered.
"""
tool_filter = (
create_static_tool_filter(allowed_tool_names=config.allowed_tools)
if config.allowed_tools is not None
else None
)
if config.transport == "stdio":
stdio_params: MCPServerStdioParams = {
"command": cast("str", config.command),
"args": config.args,
"env": config.env,
}
return MCPServerStdio(
params=stdio_params,
name=config.name,
tool_filter=tool_filter,
cache_tools_list=True,
)
http_params: MCPServerStreamableHttpParams = {
"url": cast("str", config.url),
"headers": _auth_headers(config),
}
return MCPServerStreamableHttp(
params=http_params,
name=config.name,
tool_filter=tool_filter,
cache_tools_list=True,
)
def _build_tool(
config: McpConnectionConfig,
server: MCPServer,
mcp_tool: MCPTool,
result_transform: ResultTransform | None,
) -> FunctionTool:
"""Build one namespaced FunctionTool from a listed MCP tool.
The SDK builds the tool (so name override, input schema, approval policy,
error-as-result handling, and tool-origin metadata are unchanged). With a
``result_transform`` we route the underlying MCP call through
:func:`_install_result_transform` so the transform sees the structured result
and decides the tool's output. Without one (the stock path), we still route
the call, through :func:`_install_error_status_capture`, so an errored result
reads as failed in the TUI while the agent's content is unchanged.
"""
namespaced_name = namespaced_tool_name(config.name, mcp_tool.name)
tool = MCPUtil.to_function_tool(
mcp_tool,
server,
convert_schemas_to_strict=False,
tool_name_override=namespaced_name,
)
if result_transform is not None:
_install_result_transform(tool, server, mcp_tool.name, namespaced_name, result_transform)
else:
_install_error_status_capture(tool, server, mcp_tool.name, namespaced_name)
return tool
def _install_result_transform(
tool: FunctionTool,
server: MCPServer,
base_tool_name: str,
namespaced_name: str,
result_transform: ResultTransform,
) -> None:
"""Route a tool's MCP call through ``result_transform``, innermost.
``MCPUtil.to_function_tool`` serializes the result inside its own invoke, so
the structured result cannot be intercepted through it. Instead we call
``server.call_tool`` ourselves, hand the parsed :class:`CallToolResult` to the
transform, and return the transform's output as the tool result.
This runs INSIDE the tool's invoke. The agent factory wraps a registered
tool's ``on_invoke_tool`` with output bounding, disk spill, and tracing at
agent-build time, which is OUTSIDE this invoke, so the transform is genuinely
the innermost step: nothing sees the raw result before the transform does.
``to_function_tool`` wraps the real invoke in the SDK's failure-handling
invoker, which stores the inner coroutine on ``_invoke_tool_impl`` and calls
it inside its try/except. Swapping that inner impl keeps the SDK's
error-as-result handling and all tool metadata while inserting the transform.
If the SDK ever renames that attribute we fail loudly rather than silently
skip the transform.
"""
async def _invoke(_ctx: Any, input_json: str) -> Any:
parsed: Any = json.loads(input_json) if input_json else {}
if not isinstance(parsed, dict):
raise ModelBehaviorError(
f"Invalid JSON input for tool {namespaced_name}: expected a JSON object"
)
args = cast("dict[str, Any]", parsed)
result = await server.call_tool(base_tool_name, args)
structured_result = result.model_dump(mode="json")
return result_transform(namespaced_name, structured_result)
_replace_tool_invoke(tool, _invoke)
def _replace_tool_invoke(tool: FunctionTool, invoke: Callable[[Any, str], Any]) -> None:
"""Swap a FunctionTool's inner invoke, failing loudly if the SDK shape changed.
``to_function_tool`` wraps the real invoke in the SDK's failure-handling
invoker, which stores the inner coroutine on ``_invoke_tool_impl`` and calls
it inside its own try/except. Swapping that inner impl keeps the SDK's
error-as-result handling and every piece of tool metadata intact. It is a
plain object with the coroutine as an attribute, not a function, so we treat
it as untyped to swap it. If the SDK ever renames that attribute we raise
rather than silently leave the swap un-applied.
"""
invoker = cast("Any", tool.on_invoke_tool)
if not hasattr(invoker, "_invoke_tool_impl"):
raise RuntimeError(
"agents SDK FunctionTool invoker shape changed: cannot swap the tool "
"invoke without risking it being silently skipped."
)
invoker._invoke_tool_impl = invoke
def _mcp_result_to_tool_output(server: MCPServer, result: Any) -> Any:
"""Serialize a ``CallToolResult`` to a tool output, mirroring the agents SDK.
This reproduces the serialization in ``agents.mcp.util.MCPUtil.invoke_mcp_tool``
(structured-content JSON when the server asks for it, otherwise text/image
content blocks, unwrapping a single block). Because the stock path now routes
its own call, this is what makes the agent see byte-identical content to what
the SDK would have produced on its own.
"""
if getattr(server, "use_structured_content", False) and result.structuredContent:
return json.dumps(result.structuredContent)
outputs: list[dict[str, Any]] = []
for item in result.content:
if item.type == "text":
outputs.append({"type": "text", "text": item.text})
elif item.type == "image":
outputs.append(
{"type": "image", "image_url": f"data:{item.mimeType};base64,{item.data}"}
)
else:
outputs.append({"type": "text", "text": str(item.model_dump(mode="json"))})
if len(outputs) == 1:
return outputs[0]
return outputs
def _install_error_status_capture(
tool: FunctionTool,
server: MCPServer,
base_tool_name: str,
namespaced_name: str,
) -> None:
"""Make an errored MCP result read as failed in the TUI, agent content unchanged.
The stock SDK invoke returns only the text/image tool output and drops the
``CallToolResult.isError`` flag, so the TUI cannot tell an errored MCP call
(which it renders as a green "done") from a successful one. We route the call
the same way :func:`_install_result_transform` does, read ``isError`` off the
full result, and on an error tag the returned output dict with
``success: False``.
That tag reaches the human-facing status but not the agent. The SDK stores the
raw return value on the run item's ``output`` (which the TUI reads to derive a
tool's status), but hands the agent the value re-projected through its
ToolOutput schema, which keeps only the known ``type``/``text`` fields and
drops the extra ``success`` key. So the status flips to failed while the agent
still receives exactly the same error content it does today. Non-error calls
return the stock output unchanged and keep rendering as done.
"""
async def _invoke(_ctx: Any, input_json: str) -> Any:
parsed: Any = json.loads(input_json) if input_json else {}
if not isinstance(parsed, dict):
raise ModelBehaviorError(
f"Invalid JSON input for tool {namespaced_name}: expected a JSON object"
)
args = cast("dict[str, Any]", parsed)
result = await server.call_tool(base_tool_name, args)
tool_output = _mcp_result_to_tool_output(server, result)
if getattr(result, "isError", False) and isinstance(tool_output, dict):
return {**tool_output, "success": False}
return tool_output
_replace_tool_invoke(tool, _invoke)
async def _register_server_tools(
config: McpConnectionConfig,
server: MCPServer,
result_transform: ResultTransform | None = None,
) -> list[Tool]:
"""List a connected server's tools, prefix + filter them, and register them.
``allowed_tools`` of ``None`` registers every listed tool; a list restricts
to exactly those names.
"""
allowed = config.allowed_tools
mcp_tools = await server.list_tools()
tools: list[Tool] = [
_build_tool(config, server, mcp_tool, result_transform)
for mcp_tool in mcp_tools
if allowed is None or mcp_tool.name in allowed
]
register_agent_tools(*tools)
return tools
async def connect_mcp_servers(
configs: list[McpConnectionConfig],
result_transform: ResultTransform | None = None,
) -> list[ConnectedMcpServer]:
"""Connect to each MCP server and register its tools.
When ``result_transform`` is given, every registered tool routes its result
through it before the result reaches the agent (see
:func:`_install_result_transform`). When it is ``None`` the tools behave
exactly as the SDK builds them.
Returns one :class:`ConnectedMcpServer` per server that connected, carrying
the SDK server (so the caller can clean it up when the run ends) plus the
server name and how many tools it registered (so the caller can show the
user a startup summary). Connections that fail are skipped rather than
raised.
"""
connected: list[ConnectedMcpServer] = []
for config in configs:
server: MCPServer | None = None
try:
server = _build_server(config)
await server.connect() # type: ignore[no-untyped-call]
tools = await _register_server_tools(config, server, result_transform)
except Exception:
logger.exception("Skipping MCP connection %r", config.name)
if server is not None:
with contextlib.suppress(Exception):
await server.cleanup() # type: ignore[no-untyped-call]
continue
except BaseException:
# A cancellation (or other non-Exception failure) mid-connect must not
# orphan MCP subprocesses or HTTP sessions. Clean up the server being
# connected and every server already connected, then re-raise so the
# caller still stops. The runner only receives the list on a clean
# return, so on an abnormal exit this function owns the cleanup.
if server is not None:
with contextlib.suppress(Exception):
await server.cleanup() # type: ignore[no-untyped-call]
for established in connected:
with contextlib.suppress(Exception):
await established.server.cleanup() # type: ignore[no-untyped-call]
raise
logger.info("Connected MCP server %r (%d tools)", config.name, len(tools))
connected.append(
ConnectedMcpServer(
server=server, name=config.name, tool_count=len(tools), notes=config.notes
)
)
return connected
+74
View File
@@ -0,0 +1,74 @@
"""The connection-config contract for the MCP client.
Describes one MCP server the client can connect to: its transport, endpoint or
launch command, optional auth, and an optional tool allowlist. Field names are
stable; callers build against them.
"""
from __future__ import annotations
from typing import Annotated, Literal
from pydantic import BaseModel, ConfigDict, Field, model_validator
class BearerAuth(BaseModel):
"""Header-token auth, sent as ``Authorization: Bearer <token>``."""
model_config = ConfigDict(extra="forbid")
kind: Literal["bearer"] = "bearer"
token: str = Field(min_length=1, repr=False)
McpAuth = Annotated[BearerAuth, Field(discriminator="kind")]
class McpConnectionConfig(BaseModel):
"""One MCP server the client can connect to.
Two transports are supported: streamable ``http`` (a remote endpoint) and
``stdio`` (a local server launched as a subprocess).
"""
model_config = ConfigDict(extra="forbid")
name: str = Field(min_length=1)
"""Namespaced tool prefix, unique per run (e.g. ``github``)."""
transport: Literal["http", "stdio"] = "http"
"""``http`` for a streamable HTTP endpoint, ``stdio`` for a local subprocess."""
url: str | None = Field(default=None, min_length=1)
"""The MCP server endpoint. Required for ``http``."""
auth: McpAuth | None = None
"""Bearer token for the server. Optional; a local stdio server usually
needs none."""
command: str | None = Field(default=None, min_length=1)
"""The executable to launch for ``stdio``. Required for ``stdio``."""
args: list[str] = Field(default_factory=list)
"""Arguments passed to ``command`` (stdio only)."""
env: dict[str, str] = Field(default_factory=dict)
"""Extra environment variables for the stdio subprocess."""
allowed_tools: list[str] | None = None
"""Tool allowlist, applied after the server lists its tools. ``None`` (the
default) exposes every tool the server lists; a list restricts to it."""
notes: str | None = None
"""Free-text notes for the agent describing what this connection is and how
to use it. When set, the runner collects the notes of every connection into
a single block on the root task, so a note describes its connection once
rather than being repeated onto each of its tools."""
@model_validator(mode="after")
def _check_transport_fields(self) -> McpConnectionConfig:
if self.transport == "http" and not self.url:
raise ValueError("an http MCP connection requires 'url'")
if self.transport == "stdio" and not self.command:
raise ValueError("a stdio MCP connection requires 'command'")
return self
+132
View File
@@ -0,0 +1,132 @@
"""Read the open-source user's MCP servers from ``~/.strix/mcp-servers.json``.
An open-source user lists the MCP servers they want the agent to reach in a
small JSON file. Strix reads it at the start of a run, connects to each server,
and registers its tools. The file is optional; without it the run simply gets
no MCP tools.
Parsing is fail-open. A single malformed entry is logged and skipped rather than
raising, so one bad row never blocks the servers that are valid, and a missing
or unreadable file yields an empty list.
"""
from __future__ import annotations
import json
import logging
import os
from pathlib import Path
from typing import cast
from pydantic import ValidationError
from strix.tools.mcp.config import McpConnectionConfig
logger = logging.getLogger(__name__)
_DEFAULT_PATH: Path = Path.home() / ".strix" / "mcp-servers.json"
_PATH_ENV_VAR = "STRIX_MCP_CONFIG"
# Per-run selection, set by the --mcp-server / --mcp-exclude CLI flags. Each is a
# comma-separated list of connection names.
_ONLY_ENV_VAR = "STRIX_MCP_ONLY"
_EXCLUDE_ENV_VAR = "STRIX_MCP_EXCLUDE"
def _resolve_path(path: Path | None) -> Path:
if path is not None:
return path
override = os.environ.get(_PATH_ENV_VAR)
if override:
return Path(override)
return _DEFAULT_PATH
def _dedupe_by_name(configs: list[McpConnectionConfig]) -> list[McpConnectionConfig]:
"""Keep the first connection of each name, dropping later duplicates.
Names namespace a server's tools (``<name>.<tool>``), so two connections
sharing a name would collide and the second's tools would be silently
rejected at registration. Drop the duplicate here, with a warning, instead.
"""
seen: set[str] = set()
unique: list[McpConnectionConfig] = []
for config in configs:
if config.name in seen:
logger.warning(
"Ignoring MCP server %r: another connection already uses that name "
"(names must be unique because they namespace the server's tools).",
config.name,
)
continue
seen.add(config.name)
unique.append(config)
return unique
def _parse_names(env_var: str) -> set[str]:
return {name.strip() for name in os.environ.get(env_var, "").split(",") if name.strip()}
def _apply_run_selection(configs: list[McpConnectionConfig]) -> list[McpConnectionConfig]:
"""Restrict this run's connections to an optional include/exclude selection.
``STRIX_MCP_ONLY`` (if set) keeps only the named connections; then
``STRIX_MCP_EXCLUDE`` drops any named connection. With neither set, every
connection is kept.
"""
only = _parse_names(_ONLY_ENV_VAR)
exclude = _parse_names(_EXCLUDE_ENV_VAR)
if not only and not exclude:
return configs
available = {config.name for config in configs}
for name in sorted((only | exclude) - available):
logger.warning(
"MCP connection selection named %r, which is not configured; ignoring it", name
)
selected: list[McpConnectionConfig] = []
for config in configs:
if only and config.name not in only:
continue
if config.name in exclude:
continue
selected.append(config)
return selected
def load_user_mcp_configs(path: Path | None = None) -> list[McpConnectionConfig]:
"""Load MCP connection configs from the user's JSON file.
The path is ``path`` if given, else ``$STRIX_MCP_CONFIG``, else
``~/.strix/mcp-servers.json``. The file is a JSON list of server entries.
A missing file returns ``[]``; an unreadable or non-list file is logged and
returns ``[]``; individual entries that fail validation are logged and
skipped. Connections sharing a name are de-duplicated (first wins), and an
optional per-run include/exclude selection is applied last.
"""
source = _resolve_path(path)
if not source.exists():
return []
try:
raw = json.loads(source.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
logger.exception("Could not read MCP config at %s; ignoring it", source)
return []
if not isinstance(raw, list):
logger.warning("MCP config at %s is not a JSON list; ignoring it", source)
return []
entries = cast("list[object]", raw)
configs: list[McpConnectionConfig] = []
for index, entry in enumerate(entries):
try:
configs.append(McpConnectionConfig.model_validate(entry))
except ValidationError as exc:
logger.warning("Skipping invalid MCP server entry #%d in %s: %s", index, source, exc)
return _apply_run_selection(_dedupe_by_name(configs))
+80
View File
@@ -0,0 +1,80 @@
"""How an MCP server's tools are named for the model, and how to read that back.
Kept apart from the client, and stdlib-only, so the interfaces can resolve which
connection a tool call went to without importing the MCP client (and through it
the agents SDK and every registered tool).
"""
from __future__ import annotations
import re
from typing import TYPE_CHECKING, NamedTuple
if TYPE_CHECKING:
from collections.abc import Iterable
# A tool name offered to a model has to be letters, digits, underscores or
# hyphens; anything else is rejected outright by the model APIs. Three things can
# put a stray character in one: the separator between the connection and the tool
# name, a name the server chose for its own tool (servers commonly namespace
# theirs), and the connection name out of the user's config file. Sanitizing the
# finished name covers all three rather than only the separator.
_INVALID_TOOL_NAME_CHARS = re.compile(r"[^a-zA-Z0-9_-]")
def namespaced_tool_name(connection: str, tool: str) -> str:
"""The name a connection's tool is offered to the model under.
Only the model-facing name is rewritten. Every call to the server uses the
tool name the server itself reported, so sanitizing here can never change
which tool is invoked.
"""
return _INVALID_TOOL_NAME_CHARS.sub("_", f"{connection}_{tool}")
class McpToolOrigin(NamedTuple):
"""Where a model-facing tool name came from, for showing the user.
``connection`` is the name the user gave the connection in their config, so
it reads the way they wrote it. ``tool`` is what is left of the model-facing
name once the connection prefix is removed, which is the server's own name
for the tool and the part a reader cares about.
"""
connection: str
tool: str
def resolve_mcp_tool(tool_name: str, connections: Iterable[str]) -> McpToolOrigin | None:
"""Split a model-facing tool name against the run's connections, or ``None``.
Matched against the connections the run actually made rather than by
splitting the name on the separator: the connection name and the server's own
tool name can both contain underscores, so a split is ambiguous and would
attribute calls to a connection that does not exist. Each connection name is
sanitized the same way :func:`namespaced_tool_name` sanitizes it before
comparing, so a connection whose name has characters a model-facing name
cannot carry still matches.
The longest match wins, so one connection whose name is a prefix of another's
still resolves to the right one. The character after the prefix has to be a
separator rather than more of a name, which any non-alphanumeric satisfies,
so this holds whichever separator :func:`namespaced_tool_name` uses.
"""
best: McpToolOrigin | None = None
best_length = 0
for connection in connections:
prefix = _INVALID_TOOL_NAME_CHARS.sub("_", connection)
if not prefix or len(tool_name) <= len(prefix) or not tool_name.startswith(prefix):
continue
if tool_name[len(prefix)].isalnum():
continue
if len(prefix) > best_length:
# Past the prefix and its single separator character is the tool's
# own name; if a server named a tool nothing but separators, fall
# back to the whole name so the row still says something.
tool = tool_name[len(prefix) + 1 :] or tool_name
best, best_length = McpToolOrigin(connection, tool), len(prefix)
return best
+17 -9
View File
@@ -10,20 +10,16 @@ import urllib.request
from typing import TYPE_CHECKING, Any, Literal
from urllib.parse import parse_qs, urlencode, urlparse, urlunparse
from caido_sdk_client import Client, TokenAuthOptions
from caido_sdk_client.types import (
ConnectionInfoInput,
CreateScopeOptions,
ReplaySendOptions,
RequestGetOptions,
UpdateScopeOptions,
)
# The generated Caido GraphQL schema module is slow to import and is only needed
# once a proxy tool actually runs, so the SDK is imported on first use rather
# than at module scope, which would put it on every launch's critical path.
if TYPE_CHECKING:
from collections.abc import Awaitable, Callable
from caido_sdk_client import Client
from caido_sdk_client import Client as CaidoClient
from caido_sdk_client.types import ConnectionInfoInput
RequestPart = Literal["request", "response"]
@@ -85,6 +81,8 @@ def _login_as_guest() -> str:
async def _new_client() -> Client:
from caido_sdk_client import Client, TokenAuthOptions
token = await asyncio.to_thread(_login_as_guest)
client = Client(caido_url(), auth=TokenAuthOptions(token=token))
await client.connect()
@@ -163,6 +161,8 @@ async def get_request_with_client(
# Passing False for either causes pydantic validation to fail with
# "Field required" on the missing raw field. Always request both —
# the caller picks which one to surface via ``part``.
from caido_sdk_client.types import RequestGetOptions
opts = RequestGetOptions(request_raw=True, response_raw=True)
return await client.request.get(request_id, opts)
@@ -206,6 +206,8 @@ def build_raw_request(
if body:
final_headers["Content-Length"] = str(len(body.encode("utf-8")))
from caido_sdk_client.types import ConnectionInfoInput
lines = [f"{method.upper()} {path} HTTP/1.1"]
lines.extend(f"{k}: {v}" for k, v in final_headers.items())
raw = ("\r\n".join(lines) + "\r\n\r\n" + body).encode("utf-8")
@@ -334,6 +336,8 @@ async def replay_send_raw(
raw: bytes,
connection: ConnectionInfoInput,
) -> dict[str, Any]:
from caido_sdk_client.types import ReplaySendOptions
started = time.time()
# Create an empty replay session, then dispatch via ``send()``.
# Passing ``CreateReplaySessionFromRaw`` here would also seed a stored
@@ -391,6 +395,8 @@ async def scope_create(
allowlist: list[str] | None = None,
denylist: list[str] | None = None,
) -> Any:
from caido_sdk_client.types import CreateScopeOptions
return await client.scope.create(
CreateScopeOptions(
name=name,
@@ -408,6 +414,8 @@ async def scope_update(
allowlist: list[str] | None = None,
denylist: list[str] | None = None,
) -> Any:
from caido_sdk_client.types import UpdateScopeOptions
return await client.scope.update(
scope_id,
UpdateScopeOptions(
+17 -9
View File
@@ -13,6 +13,7 @@ from typing import TYPE_CHECKING, Any, Literal
from agents import RunContextWrapper, function_tool
from strix.runtime.caido_handle import CaidoBootstrapHandle
from strix.tools.proxy import caido_api
@@ -47,9 +48,16 @@ ScopeAction = Literal["get", "list", "create", "update", "delete"]
_CAIDO_CALL_LOCK = asyncio.Lock()
def _ctx_client(ctx: RunContextWrapper) -> Client | None:
inner = ctx.context if isinstance(ctx.context, dict) else {}
return inner.get("caido_client")
async def _ctx_client(ctx: RunContextWrapper) -> Client | None:
inner: dict[str, Any] = ctx.context if isinstance(ctx.context, dict) else {}
client: Client | CaidoBootstrapHandle | None = inner.get("caido_client")
if isinstance(client, CaidoBootstrapHandle):
try:
return await client.get()
except Exception: # noqa: BLE001
logger.warning("Caido bootstrap failed; proxy tools unavailable", exc_info=True)
return None
return client
async def _call[T](client: Client, fn: Callable[[Client], Awaitable[T]]) -> T:
@@ -155,7 +163,7 @@ async def list_requests(
sort_order: ``asc`` or ``desc``.
scope_id: Restrict to a Caido scope (managed via ``scope_rules``).
"""
client = _ctx_client(ctx)
client = await _ctx_client(ctx)
if client is None:
return _no_client()
@@ -261,7 +269,7 @@ async def view_request(
page: 1-indexed page number (only when no ``search_pattern``).
page_size: Lines per page.
"""
client = _ctx_client(ctx)
client = await _ctx_client(ctx)
if client is None:
return _no_client()
@@ -379,7 +387,7 @@ async def repeat_request(
- ``body`` replace the body string entirely.
- ``cookies`` dict of cookies to add/update.
"""
client = _ctx_client(ctx)
client = await _ctx_client(ctx)
if client is None:
return _no_client()
mods = modifications or {}
@@ -461,7 +469,7 @@ async def list_sitemap(
(recursive subtree). Only meaningful with ``parent_id``.
page: 1-indexed page (30 entries per page).
"""
client = _ctx_client(ctx)
client = await _ctx_client(ctx)
if client is None:
return _no_client()
try:
@@ -495,7 +503,7 @@ async def view_sitemap_entry(
Args:
entry_id: ID from ``list_sitemap`` (or any nested entry).
"""
client = _ctx_client(ctx)
client = await _ctx_client(ctx)
if client is None:
return _no_client()
try:
@@ -554,7 +562,7 @@ async def scope_rules(
scope_id: Required for ``get`` / ``update`` / ``delete``.
scope_name: Required for ``create`` / ``update``.
"""
client = _ctx_client(ctx)
client = await _ctx_client(ctx)
if client is None:
return _no_client()
+221 -37
View File
@@ -161,9 +161,101 @@ _REQUIRED_FIELDS = {
}
_VALID_FIX_EFFORT = frozenset({"trivial", "low", "medium", "high"})
_VALID_CONFIDENCE = frozenset({"high", "medium", "low"})
async def _do_create( # noqa: PLR0912
def _validate_required_text(fields: dict[str, str]) -> list[str]:
"""Report every ``_REQUIRED_FIELDS`` entry that arrived blank."""
return [
msg for name, msg in _REQUIRED_FIELDS.items() if not str(fields.get(name) or "").strip()
]
def _validate_cvss_breakdown(breakdown: Any) -> list[str]:
"""Check the 8 CVSS metrics are all present with legal values."""
if not isinstance(breakdown, dict) or not breakdown:
return ["cvss_breakdown: must be an object with the 8 CVSS metrics"]
return [
f"Invalid {name}: {breakdown.get(name)}. Must be one of: {valid}"
for name, valid in _CVSS_VALID.items()
if breakdown.get(name) not in valid
]
def _validate_identifiers(
cve: str | None, cwe: str | None
) -> tuple[str | None, str | None, list[str]]:
"""Normalize and validate the optional CVE / CWE identifiers."""
errors: list[str] = []
if cve:
cve = _extract_cve(cve)
cve_err = _validate_cve(cve)
if cve_err:
errors.append(cve_err)
if cwe:
cwe = _extract_cwe(cwe)
cwe_err = _validate_cwe(cwe)
if cwe_err:
errors.append(cwe_err)
return cve, cwe, errors
def _validate_analysis_fields(
*,
counterevidence: str,
confidence: str,
confidence_rationale: str | None,
severity_change_conditions: str,
) -> list[str]:
"""Validate the counterevidence / confidence closure metadata."""
errors: list[str] = []
if not str(counterevidence or "").strip():
errors.append(
"Counterevidence cannot be empty - state the strongest evidence against "
"this finding, or what you checked and found none (e.g. 'no input "
"validation, WAF, or authorization check found on this path')"
)
if not str(severity_change_conditions or "").strip():
errors.append(
"severity_change_conditions cannot be empty - state the one concrete piece "
"of evidence that would raise or lower the severity"
)
if confidence not in _VALID_CONFIDENCE:
errors.append(
f"Invalid confidence: {confidence!r}. Must be one of: {sorted(_VALID_CONFIDENCE)}"
)
elif confidence != "high" and not str(confidence_rationale or "").strip():
errors.append(
"confidence_rationale is required when confidence is not 'high' - name the "
"gap (e.g. static-only trace, unconfirmed reachability, no runtime access)"
)
return errors
def _validate_fix_verification(
locations: list[dict[str, Any]] | None,
fix_verification: str | None,
) -> list[str]:
"""Require a verification statement whenever an applyable fix is proposed."""
if not locations or not any(loc.get("fix_after") for loc in locations):
return []
if str(fix_verification or "").strip():
return []
return [
"fix_verification is REQUIRED when any code_location carries a 'fix_after' - "
"a suggestion a reviewer can click to apply must be verified first. State, in "
"order: (1) security closure - re-trace the source->sink path through the "
"PATCHED code and say why it is now blocked; (2) bypass review - re-read the "
"diff without your original rationale and name the equivalent sinks, sibling "
"call sites, and alternate malicious input classes you checked; (3) preserved "
"behavior - the legitimate inputs, APIs, and error semantics that still work; "
"(4) how each was checked (executed vs. reasoned), naming any unrun check as "
"an explicit gap. If you cannot make these statements, drop 'fix_after' and "
"leave the location informational."
]
async def _do_create(
*,
title: str,
description: str,
@@ -175,6 +267,9 @@ async def _do_create( # noqa: PLR0912
remediation_steps: str,
evidence: str,
assumptions: str,
counterevidence: str,
confidence: str,
severity_change_conditions: str,
fix_effort: str,
cvss_breakdown: dict[str, str],
endpoint: str | None,
@@ -182,26 +277,36 @@ async def _do_create( # noqa: PLR0912
cve: str | None,
cwe: str | None,
code_locations: list[dict[str, Any]] | None,
confidence_rationale: str | None = None,
fix_verification: str | None = None,
fix_pr_body: str | None = None,
agent_id: str | None = None,
agent_name: str | None = None,
) -> dict[str, Any]:
errors: list[str] = []
fields = {
"title": title,
"description": description,
"impact": impact,
"target": target,
"technical_analysis": technical_analysis,
"poc_description": poc_description,
"poc_script_code": poc_script_code,
"remediation_steps": remediation_steps,
"evidence": evidence,
"assumptions": assumptions,
}
for name, msg in _REQUIRED_FIELDS.items():
if not str(fields.get(name) or "").strip():
errors.append(msg)
errors: list[str] = _validate_required_text(
{
"title": title,
"description": description,
"impact": impact,
"target": target,
"technical_analysis": technical_analysis,
"poc_description": poc_description,
"poc_script_code": poc_script_code,
"remediation_steps": remediation_steps,
"evidence": evidence,
"assumptions": assumptions,
}
)
confidence = (confidence or "").strip().lower()
errors.extend(
_validate_analysis_fields(
counterevidence=counterevidence,
confidence=confidence,
confidence_rationale=confidence_rationale,
severity_change_conditions=severity_change_conditions,
)
)
fix_effort = (fix_effort or "").strip().lower()
if fix_effort not in _VALID_FIX_EFFORT:
@@ -209,28 +314,14 @@ async def _do_create( # noqa: PLR0912
f"Invalid fix_effort: {fix_effort!r}. Must be one of: {sorted(_VALID_FIX_EFFORT)}"
)
if not isinstance(cvss_breakdown, dict) or not cvss_breakdown:
errors.append("cvss_breakdown: must be an object with the 8 CVSS metrics")
cvss_breakdown = {}
else:
for name, valid in _CVSS_VALID.items():
value = cvss_breakdown.get(name)
if value not in valid:
errors.append(f"Invalid {name}: {value}. Must be one of: {valid}")
errors.extend(_validate_cvss_breakdown(cvss_breakdown))
parsed_locations = _normalize_code_locations(code_locations)
if parsed_locations:
errors.extend(_validate_code_locations(parsed_locations))
if cve:
cve = _extract_cve(cve)
cve_err = _validate_cve(cve)
if cve_err:
errors.append(cve_err)
if cwe:
cwe = _extract_cwe(cwe)
cwe_err = _validate_cwe(cwe)
if cwe_err:
errors.append(cwe_err)
errors.extend(_validate_fix_verification(parsed_locations, fix_verification))
cve, cwe, identifier_errors = _validate_identifiers(cve, cwe)
errors.extend(identifier_errors)
if errors:
return {"success": False, "error": "Validation failed", "errors": errors}
@@ -297,6 +388,10 @@ async def _do_create( # noqa: PLR0912
remediation_steps=remediation_steps,
evidence=evidence,
assumptions=assumptions,
counterevidence=counterevidence,
confidence=confidence,
confidence_rationale=confidence_rationale,
severity_change_conditions=severity_change_conditions,
fix_effort=fix_effort,
cvss=cvss_score,
cvss_breakdown=cvss_breakdown,
@@ -305,6 +400,7 @@ async def _do_create( # noqa: PLR0912
cve=cve,
cwe=cwe,
code_locations=parsed_locations,
fix_verification=fix_verification,
fix_pr_body=fix_pr_body,
agent_id=agent_id if isinstance(agent_id, str) else None,
agent_name=agent_name if isinstance(agent_name, str) else None,
@@ -357,6 +453,9 @@ async def create_vulnerability_report(
remediation_steps: str,
evidence: str,
assumptions: str,
counterevidence: str,
confidence: str,
severity_change_conditions: str,
fix_effort: str,
cvss_breakdown: dict[str, str],
endpoint: str | None = None,
@@ -364,6 +463,8 @@ async def create_vulnerability_report(
cve: str | None = None,
cwe: str | None = None,
code_locations: list[dict[str, Any]] | None = None,
confidence_rationale: str | None = None,
fix_verification: str | None = None,
fix_pr_body: str | None = None,
) -> str:
"""File a vulnerability report — one report per fully-verified finding.
@@ -411,6 +512,15 @@ async def create_vulnerability_report(
get a ``duplicate_of`` response, do NOT retry move on to other
areas.
**Counterevidence pass (required before filing)**: actively build the
strongest case that this finding is NOT exploitable, or less severe
than you think then record the result in ``counterevidence``, set
``confidence`` honestly, and state what would move the severity in
``severity_change_conditions``. These three fields are mandatory and
validated. A finding you could not execute is at best
``confidence: medium``, with the gap named in
``confidence_rationale``.
**Report output rules** (this content may be rendered into generated
reports):
@@ -563,6 +673,31 @@ async def create_vulnerability_report(
assumptions: Short note on the assumptions/prerequisites that
make this finding impactful or exploitable (e.g. "assumes an
authenticated low-privilege user").
counterevidence: REQUIRED. The strongest case *against* this
finding, after actively looking for it the guard you might
have missed, the deployment constraint, the precondition. If
you genuinely found nothing, say what you checked (e.g. "no
input validation, WAF, or authorization check found on this
path; tested authenticated and unauthenticated"), not just
"none". A generic trust claim ("the framework escapes this")
is not counterevidence unless you confirmed that specific
call in this context.
confidence: REQUIRED. Your calibrated confidence that this is a
real, exploitable issue: ``high`` (working PoC against the
live target, or a complete reachable sourcesink trace),
``medium`` (strong static evidence you could not fully
execute), or ``low`` (plausible with a material unresolved
gap). Do not inflate an accurate ``medium`` is more useful
than a ``high`` that fails triage.
confidence_rationale: Required when ``confidence`` is not
``high``. Name the specific gap (e.g. "static-only trace,
could not stand up the service to reproduce"; "reachability
of this route from unauthenticated traffic unconfirmed").
severity_change_conditions: REQUIRED. One concrete sentence on
what single piece of additional evidence would raise or
lower the severity (e.g. "confirmation this route is exposed
to unauthenticated internet traffic would raise this to
critical").
fix_effort: One of ``trivial`` / ``low`` / ``medium`` / ``high``.
cvss_breakdown: 8-metric object per the format above.
endpoint: API path / Git path (e.g. ``/api/login``).
@@ -632,6 +767,40 @@ async def create_vulnerability_report(
- Padding ``fix_before`` with surrounding context lines
that aren't part of the fix.
- Duplicating the same change across multiple locations.
fix_verification: REQUIRED whenever any ``code_locations`` entry
carries a ``fix_after``. A reviewer can apply that
suggestion with one click, so an unverified fix ships
straight into the codebase. Before writing this field, work
the gates **in order** and never trade an earlier one for a
later one:
1. **Security closure** re-trace the source sink path
through the *patched* code and state why it is now
blocked. Re-run the PoC against the fix if you can.
2. **Bypass review** re-read the diff *without* leaning on
the rationale that produced it. Name the sibling call
sites, equivalent sinks, and alternate malicious input
classes you checked, and try at least one.
3. **Preserved behavior** name the legitimate inputs,
public APIs, and error semantics that must keep working,
and confirm the patch leaves them intact. A fix that
breaks the feature is not a fix.
4. **Repository checks** run the narrowest relevant
syntax / type / lint / test check that covers the
changed lines.
Then write what you did: the commands you ran and their
results, and every gate you could only reason about rather
than execute, marked explicitly as a gap. Do not claim a
gate passed because it looks right. If a gate fails, revise
the patch or drop ``fix_after`` and leave the location
informational never compensate for a failed security
closure with a smaller diff or extra prose.
Also use this field to record the narrowest-complete-change
judgement: prefer the smallest repository-native fix that
fully enforces the invariant, using existing helpers, with
no unrelated refactors folded in.
fix_pr_body: Optional. When source is available and you have a
concrete fix, a markdown PR-description body proposing the
fix (summary + rationale). Prose/markdown only the code
@@ -672,6 +841,15 @@ async def create_vulnerability_report(
remediation_steps:
Context-encode all user input rendered into HTML; prefer the
template engine's auto-escaping over string interpolation.
counterevidence:
No output encoding, CSP, or WAF observed on this response;
payload executed in a current browser. The parameter is
reflected on an unauthenticated route, so no privileged
position is required.
confidence: "high"
severity_change_conditions:
A restrictive CSP that blocks inline script execution would
reduce impact and lower the severity.
fix_effort: "low"
"""
agent_id, agent_name = _caller_identity(ctx)
@@ -687,6 +865,10 @@ async def create_vulnerability_report(
remediation_steps=remediation_steps,
evidence=evidence,
assumptions=assumptions,
counterevidence=counterevidence,
confidence=confidence,
confidence_rationale=confidence_rationale,
severity_change_conditions=severity_change_conditions,
fix_effort=fix_effort,
cvss_breakdown=cvss_breakdown,
endpoint=endpoint,
@@ -694,6 +876,7 @@ async def create_vulnerability_report(
cve=cve,
cwe=cwe,
code_locations=code_locations,
fix_verification=fix_verification,
fix_pr_body=fix_pr_body,
agent_id=agent_id,
agent_name=agent_name,
@@ -1327,6 +1510,7 @@ _REPORT_SUMMARY_FIELDS = (
"title",
"severity",
"cvss",
"confidence",
"finding_class",
"cve",
"cwe",
@@ -1528,8 +1712,8 @@ async def list_reports(
findings, and build the ``finish_scan`` executive summary.
By default each entry is compact: ``id``, ``title``, ``severity``,
``cvss``, ``finding_class``, ``cve`` / ``cwe``, ``target`` /
``endpoint``, ``fix_effort``, ``agent_name`` (who filed it), ``timestamp``,
``cvss``, ``confidence``, ``finding_class``, ``cve`` / ``cwe``,
``target`` / ``endpoint``, ``fix_effort``, ``agent_name`` (who filed it), ``timestamp``,
plus a 280-char ``description_preview``. Entries you filed yourself are
flagged ``by_you: true``. The response also carries
``total_count`` and ``severity_counts`` (counts per severity across all
+1
View File
@@ -0,0 +1 @@
"""Repository-scoped threat model cache, reusable across scans of the same tree."""
+659
View File
@@ -0,0 +1,659 @@
"""Target-scoped threat models — cached under ``~/.strix/threat-models``.
A threat model describes the target, not the scan: a host, an application, an
API, a repository, or whatever else the engagement is pointed at. It stays
valid across unrelated runs against the same target, so it is keyed by target
identity rather than by run id one agent derives it, every later agent in
this run and in future runs against the same target reads it back instead of
re-deriving trust boundaries from scratch.
Where the target is a checkout, the model is additionally pinned to the git
revision, so a moved ``HEAD`` marks it stale. Black-box targets have no
revision to pin to; those age out instead.
"""
from __future__ import annotations
import asyncio
import hashlib
import json
import logging
import re
import subprocess
import tempfile
import threading
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
from urllib.parse import urlsplit
from agents import RunContextWrapper, function_tool
from strix.core.agents import AgentCoordinator
logger = logging.getLogger(__name__)
_CACHE_DIR = Path.home() / ".strix" / "threat-models"
_MAX_MODEL_BYTES = 512 * 1024
_MIN_MODEL_CHARS = 400
_MIN_AMENDMENT_CHARS = 80
_MAX_AMENDMENTS = 40
_GIT_TIMEOUT_SECONDS = 10
_UNVERSIONED = "unversioned"
_MAX_AGE_DAYS = 14
_DEFAULT_PORTS = {"http": "80", "https": "443"}
_cache_lock = threading.RLock()
_REQUIRED_SECTIONS = (
"overview",
"trust boundaries",
"attack surface",
"severity calibration",
)
def _git(repo: Path, args: list[str]) -> str | None:
try:
result = subprocess.run( # noqa: S603
["git", "-C", str(repo), *args], # noqa: S607
capture_output=True,
text=True,
check=False,
timeout=_GIT_TIMEOUT_SECONDS,
)
except (OSError, subprocess.SubprocessError):
logger.debug("git %s failed in %s", args, repo, exc_info=True)
return None
if result.returncode != 0:
return None
return result.stdout.strip() or None
def _local_directory(target: str) -> Path | None:
"""Return the target as a local directory, or None if it is not one."""
if "://" in target:
return None
try:
resolved = Path(target).expanduser().resolve()
except OSError:
return None
return resolved if resolved.is_dir() else None
def _remote_authority(target: str) -> str:
"""The ``host[:port]`` a remote target lives on, or "" if it has none."""
candidate = target if "://" in target else f"//{target}"
parts = urlsplit(candidate)
host = (parts.hostname or "").lower()
if not host:
return ""
scheme = (parts.scheme or "https").lower()
port = str(parts.port) if parts.port else _DEFAULT_PORTS.get(scheme, "")
return f"{host}:{port}" if port else host
def _normalize_remote_target(target: str) -> str:
"""Collapse the spellings of one remote target onto a single cache key."""
authority = _remote_authority(target)
if not authority:
return re.sub(r"\s+", " ", target.lower()).strip()
candidate = target if "://" in target else f"//{target}"
path = urlsplit(candidate).path.rstrip("/")
return f"{authority}{path}"
def _normalize_git_remote(remote: str) -> str:
"""Collapse a git remote URL onto the same key its clone URL would produce.
A remote reaches us in whichever spelling the clone used
``git@github.com:org/repo.git``, ``https://github.com/org/repo``,
``ssh://git@github.com/org/repo.git`` and each is the same repository.
Rewriting scp-style syntax into a URL and dropping the ``.git`` suffix and
any embedded credentials lets :func:`_normalize_remote_target` produce one
identity for all of them, and crucially the *same* identity a caller gets
when it names the repository by its remote URL rather than by a checkout
path. Without that, the model saved by an agent working in the checkout is
invisible to an agent that asks for the repository by URL, and the two
derive conflicting models of one target.
"""
candidate = remote.strip()
scp_style = re.match(r"^(?:[^@/]+@)?(?P<host>[^:/]+):(?P<path>.+)$", candidate)
if scp_style and "://" not in candidate:
candidate = f"https://{scp_style['host']}/{scp_style['path'].lstrip('/')}"
elif "://" in candidate:
# The transport a clone happened to use says nothing about which
# repository this is, and each scheme carries a different default
# port into the authority. Collapsing them all onto https keeps one
# repository on one key however it was cloned.
candidate = f"https://{candidate.split('://', 1)[1]}"
normalized = _normalize_remote_target(candidate)
return normalized.removesuffix(".git")
def _target_identity(target: str) -> tuple[str, str]:
"""Return the (stable identity, revision) pair a cached model is keyed on.
A checkout is keyed on its remote (so the same repository cloned to two
paths shares one model, and a subdirectory resolves to the whole tree) and
pinned to ``HEAD``. Everything else a host, a URL, an API base, a named
scope is keyed on its normalized form and carries no revision. Both
routes run through the same normalization, so a checkout and the URL it
was cloned from land on one key.
"""
directory = _local_directory(target)
if directory is None:
return _normalize_remote_target(target).removesuffix(".git"), _UNVERSIONED
remote = _git(directory, ["config", "--get", "remote.origin.url"])
revision = _git(directory, ["rev-parse", "HEAD"]) or _UNVERSIONED
if remote:
return _normalize_git_remote(remote), revision
toplevel = _git(directory, ["rev-parse", "--show-toplevel"])
return toplevel or str(directory), revision
def _cache_path(identity: str) -> Path:
digest = hashlib.sha256(identity.encode("utf-8")).hexdigest()[:16]
return _CACHE_DIR / f"{digest}.json"
def _snap_to_scan_target(raw: str, scan_targets: list[str]) -> str:
"""Pull a target onto the scan's own spelling of it.
Agents name the same target differently one passes the URL it was given,
the next the page it happens to be testing, a third the checkout path. Left
alone those become separate cache keys, every lookup misses, and each agent
quietly derives its own model, which is the exact failure the shared model
exists to prevent. So a target that is recognisably one of the scan's own
targets is resolved to that target instead.
"""
identity, _ = _target_identity(raw)
scoped = [(target, _target_identity(target)[0]) for target in scan_targets]
if any(known == identity for _, known in scoped):
return raw
authority = _remote_authority(raw)
if authority:
hosted = [target for target, _ in scoped if _remote_authority(target) == authority]
# Two scan targets on one host are distinguished only by their paths,
# so snapping to "the host" would merge two distinct models into one.
return hosted[0] if len(hosted) == 1 else raw
directory = _local_directory(raw)
if directory is not None:
enclosing = [
target
for target, known in scoped
if known == identity or _local_directory(target) == directory
]
if enclosing:
return enclosing[0]
return raw
def _resolve_target(
target: str, scan_targets: list[str] | None = None
) -> tuple[str | None, str | None]:
raw = (target or "").strip()
known = [t for t in (scan_targets or []) if t.strip()]
if not raw:
if len(known) == 1:
return known[0], None
return None, (
"target cannot be empty - pass the host, URL, application, or "
"repository path this model describes"
+ (f". This scan is scoped to: {', '.join(known)}" if known else "")
)
return (_snap_to_scan_target(raw, known) if known else raw), None
def _is_expired(created_at: str | None) -> bool:
if not created_at:
return True
try:
created = datetime.fromisoformat(created_at)
except ValueError:
return True
if created.tzinfo is None:
created = created.replace(tzinfo=UTC)
return datetime.now(UTC) - created > timedelta(days=_MAX_AGE_DAYS)
def _missing_sections(content: str) -> list[str]:
lowered = content.lower()
return [section for section in _REQUIRED_SECTIONS if section not in lowered]
def _read_cache(path: Path) -> dict[str, Any] | None:
"""Load a cached model. Callers must already hold ``_cache_lock``."""
if not path.is_file():
return None
try:
cached = json.loads(path.read_text(encoding="utf-8"))
except (OSError, json.JSONDecodeError):
logger.exception("threat model cache at %s is unreadable", path)
return None
return cached if isinstance(cached, dict) else None
def _write_cache(path: Path, payload: dict[str, Any]) -> str | None:
"""Atomically persist a model. Callers must already hold ``_cache_lock``."""
try:
path.parent.mkdir(parents=True, exist_ok=True)
with tempfile.NamedTemporaryFile(
mode="w",
encoding="utf-8",
dir=str(path.parent),
prefix=f".{path.name}.",
suffix=".tmp",
delete=False,
) as tmp:
tmp.write(json.dumps(payload, ensure_ascii=False))
tmp_path = Path(tmp.name)
tmp_path.replace(path)
except OSError as exc:
logger.exception("threat model persist to %s failed", path)
return f"Failed to persist threat model: {exc}"
return None
def _amendments_of(cached: dict[str, Any]) -> list[dict[str, Any]]:
raw = cached.get("amendments")
if not isinstance(raw, list):
return []
return [item for item in raw if isinstance(item, dict)]
def _not_found(identity: str, revision: str) -> dict[str, Any]:
return {
"success": True,
"found": False,
"target": identity,
"revision": revision,
"message": (
"No threat model cached for this target. Derive one — from the code if "
"you have it, from recon output if you do not — and persist it with "
"save_threat_model, so every agent on this scan shares one view of the "
"trust boundaries instead of each inventing their own."
),
}
def _staleness(cached: dict[str, Any], revision: str) -> tuple[bool, str | None]:
"""Decide whether a cached model can still be trusted, and why not."""
if revision != _UNVERSIONED:
if cached.get("revision") == revision:
return False, None
return True, (
"This model was derived against a different revision. Use it as a "
"starting point, re-check the boundaries it names against the current "
"tree, and save the corrected version."
)
created_at = cached.get("created_at")
if not _is_expired(created_at if isinstance(created_at, str) else None):
return False, None
return True, (
f"This model is more than {_MAX_AGE_DAYS} days old and there is no revision "
"to pin it to, so the target may have moved under it. Treat its surface "
"inventory as a lead list to re-confirm during recon, not as fact, and save "
"the corrected version."
)
def _get_impl(target: str, scan_targets: list[str] | None = None) -> dict[str, Any]:
resolved, error = _resolve_target(target, scan_targets)
if resolved is None:
return {"success": False, "error": error}
identity, revision = _target_identity(resolved)
path = _cache_path(identity)
with _cache_lock:
cached = _read_cache(path)
if cached is None:
return _not_found(identity, revision)
content = cached.get("content")
if not isinstance(content, str) or not content.strip():
return _not_found(identity, revision)
stale, stale_message = _staleness(cached, revision)
result: dict[str, Any] = {
"success": True,
"found": True,
"target": identity,
"revision": revision,
"cached_revision": cached.get("revision"),
"created_at": cached.get("created_at"),
"stale": stale,
"content": content,
}
amendments = _amendments_of(cached)
if amendments:
result["amendments"] = amendments
result["amendments_note"] = (
"Addenda recorded by agents after the base model was written. They "
"correct or extend it and have not been folded in yet - read them as "
"part of the model, and prefer the later one where they conflict."
)
if stale_message:
result["message"] = stale_message
return result
def _save_impl(
target: str,
content: str,
agent_name: str | None,
scan_targets: list[str] | None = None,
) -> dict[str, Any]:
resolved, error = _resolve_target(target, scan_targets)
if resolved is None:
return {"success": False, "error": error}
body = (content or "").strip()
if len(body) < _MIN_MODEL_CHARS:
return {
"success": False,
"error": (
f"Threat model is too thin ({len(body)} chars). It has to be usable by "
"an agent seeing this target for the first time: what it is, who the "
"actors are, where the trust boundaries sit, which inputs are "
"attacker-controlled, and what a critical bug looks like here."
),
}
if len(body.encode("utf-8")) > _MAX_MODEL_BYTES:
return {"success": False, "error": "Threat model exceeds 512KB; tighten it."}
missing = _missing_sections(body)
if missing:
return {
"success": False,
"error": (
"Threat model is missing required section(s): "
f"{', '.join(missing)}. Cover Overview, Trust Boundaries and "
"Assumptions, Attack Surface and Attacker Stories, and Severity "
"Calibration."
),
}
identity, revision = _target_identity(resolved)
path = _cache_path(identity)
payload: dict[str, Any] = {
"target": identity,
"revision": revision,
"created_at": datetime.now(UTC).isoformat(),
"created_by": agent_name,
"content": body,
}
with _cache_lock:
existing = _read_cache(path)
folded = len(_amendments_of(existing)) if existing else 0
error = _write_cache(path, payload)
if error:
return {"success": False, "error": error}
message = (
"Threat model saved. Subagents should call get_threat_model before they "
"start, and treat its trust boundaries as the shared baseline."
)
if folded:
message += (
f" This replaced a model carrying {folded} amendment(s), which are now "
"cleared - make sure what they said survives in the text you just wrote."
)
return {
"success": True,
"target": identity,
"revision": revision,
"amendments_cleared": folded,
"message": message,
}
def _append_amendment(
path: Path, amendment: dict[str, Any]
) -> tuple[list[dict[str, Any]] | None, str | None]:
"""Add an amendment to the cached model. Returns (amendments, error)."""
with _cache_lock:
cached = _read_cache(path)
if cached is None or not str(cached.get("content", "")).strip():
return None, (
"No threat model exists for this target yet, so there is nothing to "
"amend. Derive the base model and call save_threat_model instead."
)
amendments = _amendments_of(cached)
if len(amendments) >= _MAX_AMENDMENTS:
return None, (
f"This model already carries {len(amendments)} amendments. Fold them "
"into the base model with save_threat_model before adding more."
)
amendments.append(amendment)
cached["amendments"] = amendments
if len(json.dumps(cached, ensure_ascii=False).encode("utf-8")) > _MAX_MODEL_BYTES:
return None, "Threat model with this amendment exceeds 512KB; tighten it."
return amendments, _write_cache(path, cached)
def _amend_impl(
target: str,
addendum: str,
agent_name: str | None,
scan_targets: list[str] | None = None,
) -> dict[str, Any]:
resolved, error = _resolve_target(target, scan_targets)
if resolved is None:
return {"success": False, "error": error}
body = (addendum or "").strip()
if len(body) < _MIN_AMENDMENT_CHARS:
return {
"success": False,
"error": (
f"Amendment is too thin ({len(body)} chars). Say what the base model "
"got wrong or left out, and name the endpoint, host, file, or control "
"that makes your correction true."
),
}
identity, revision = _target_identity(resolved)
amendments, amend_error = _append_amendment(
_cache_path(identity),
{
"at": datetime.now(UTC).isoformat(),
"by": agent_name,
"revision": revision,
"content": body,
},
)
if amendments is None or amend_error:
return {"success": False, "error": amend_error}
return {
"success": True,
"target": identity,
"revision": revision,
"amendment_count": len(amendments),
"message": (
"Amendment recorded. Agents calling get_threat_model will now see it "
"alongside the base model."
),
}
def _caller_agent_name(ctx: RunContextWrapper) -> str | None:
inner = ctx.context if isinstance(ctx.context, dict) else {}
agent_id = inner.get("agent_id")
coordinator = inner.get("coordinator")
if not isinstance(agent_id, str) or not isinstance(coordinator, AgentCoordinator):
return None
return coordinator.names.get(agent_id)
def _scan_targets(ctx: RunContextWrapper) -> list[str]:
"""The targets this scan was authorized against, as the runner spelled them."""
inner = ctx.context if isinstance(ctx.context, dict) else {}
targets = inner.get("scan_targets")
if not isinstance(targets, list):
return []
return [target for target in targets if isinstance(target, str) and target.strip()]
@function_tool(timeout=30)
async def get_threat_model(ctx: RunContextWrapper, target: str) -> str:
"""Read the cached threat model for a target, if one exists.
A threat model belongs to the target, not to this scan the same
trust boundaries hold across unrelated runs against the same host
or application. Call this before you start hunting so you inherit
the shared view instead of re-deriving it, and so every agent on
this run agrees on what "attacker-controlled" means here.
Works black-box or white-box. The target can be a host, a URL, an
API base, or a repository path; equivalent spellings of the same
host resolve to the same model, and a checkout resolves to its
remote, so a model derived white-box is read back by a black-box
agent testing the deployment.
Returns ``found: false`` when nothing is cached derive one and
persist it with ``save_threat_model``. ``stale: true`` means the
checkout moved to a different revision, or that a model with no
revision to pin to has aged out: use it as a starting point,
re-confirm what it claims, and save the corrected version.
Any ``amendments`` in the response are corrections other agents
recorded after the base model was written. They are part of the
model read them, and prefer the later statement where one
contradicts the base text.
Args:
target: What the model describes a host or URL
(``https://app.example.com``), or a repository path
(``/workspace/myrepo``). Use the same value the scan was
pointed at, so agents converge on one model.
"""
return json.dumps(
await asyncio.to_thread(_get_impl, target, _scan_targets(ctx)),
ensure_ascii=False,
default=str,
)
@function_tool(timeout=30)
async def save_threat_model(ctx: RunContextWrapper, target: str, content: str) -> str:
"""Persist a target-scoped threat model for reuse by other agents.
Keyed by target identity, so a later scan of the same host or tree
reads it back instead of paying to derive it again.
**This replaces the whole document, and clears any amendments**
it is for the agent establishing the baseline (normally root,
before subagents start), or for folding accumulated amendments back
into the body. If a model already exists and you only need to
correct or extend part of it, call ``amend_threat_model`` instead;
saving over it will silently discard whatever other agents added.
**Write it from whatever evidence you have.** With source, ground
it in the code and name the files, entrypoints, and controls that
make each claim true. Black-box, ground it in recon: the hosts and
ports that answered, the technology fingerprints, the observed
roles and tenants, the authentication and session model, the
endpoints and parameters you enumerated. A black-box model is
necessarily provisional say which parts are inferred rather than
observed, and let later agents amend it as the picture fills in.
**Scope it to the target, not to this scan.** Do not centre it on
the diff you were handed, the subsystem you were assigned, or the
one host that happened to answer first. With source, distinguish
real product and runtime surfaces from test, docs, example, and
developer-tooling paths in a monorepo, do not let ``tests/`` or
one-off scripts become the centre of gravity unless the code shows
they are genuinely deployed. Where the target documents its own
boundary an ``AGENTS`` file, a specific ``SECURITY.md``, a
published API spec, an engagement scope build on it rather than
inventing a competing story.
Structure the content in Markdown with these sections:
- **Overview** what the target actually is, its real-world usage,
and which parts are product/runtime versus tooling or
non-production.
- **Trust Boundaries and Assumptions** the boundaries, the actors
on either side, and the invariants that must hold. Separate
attacker-controlled, operator-controlled, and
developer-controlled inputs explicitly. Black-box, this is the
role, tenant, and privilege model: who can reach what before
authenticating, as a low-privilege user, and across tenants.
- **Attack Surface and Attacker Stories** the exposed surfaces
(hosts, endpoints, parameters, integrations, or the code-level
entrypoints and sinks), the mitigations already present that
materially change severity or reach, realistic attacker stories,
and the stories that are *not* realistic here and why.
- **Severity Calibration** what critical / high / medium / low
look like for *this* target, with a concrete example at each
level. Where a vulnerability class needs attacker control that
does not exist in real usage, say so here.
Args:
target: What the model describes a host or URL
(``https://app.example.com``), or a repository path
(``/workspace/myrepo``). Use the same value the scan was
pointed at.
content: The full threat model in Markdown.
"""
return json.dumps(
await asyncio.to_thread(
_save_impl, target, content, _caller_agent_name(ctx), _scan_targets(ctx)
),
ensure_ascii=False,
default=str,
)
@function_tool(timeout=30)
async def amend_threat_model(ctx: RunContextWrapper, target: str, addendum: str) -> str:
"""Correct or extend the existing threat model without replacing it.
The baseline is written before anyone starts hunting, so it is
written with the least information anyone will ever have. That is
doubly true black-box, where the model starts as inference over
recon output and only becomes real as agents authenticate, map
roles, and reach the surfaces behind them. When your work
contradicts the model or fills in something it missed, record that
here every agent that calls ``get_threat_model`` afterwards sees
your addendum next to the base model.
Amendments are append-only and attributed, so two agents amending
at once both survive. That is the difference from
``save_threat_model``, which overwrites the document and drops
every amendment on it.
Worth amending:
- A boundary the model calls trusted that you found is
attacker-reachable, or vice versa.
- A host, endpoint, parameter, role, sink, or shared control the
model does not mention.
- Something the model only inferred that you have now observed or
that turned out not to be true.
- A severity call the model got wrong for this target, with the
reason.
- An assumption you disproved the model says input is validated
upstream and you found the path that skips it.
Not worth amending: individual findings (those are reports), or
restating what the model already says.
Args:
target: What the model describes the same host, URL, or
repository path used to save it.
addendum: The correction, in Markdown. State what the base
model says, what is actually true, and the endpoint, host,
file, or control that proves it.
"""
return json.dumps(
await asyncio.to_thread(
_amend_impl, target, addendum, _caller_agent_name(ctx), _scan_targets(ctx)
),
ensure_ascii=False,
default=str,
)
+16
View File
@@ -112,3 +112,19 @@ def test_wait_for_agents_is_available_in_both_modes() -> None:
for interactive in (True, False):
agent = factory.build_strix_agent(is_root=True, interactive=interactive)
assert "wait_for_agents" in [t.name for t in agent.tools]
def test_strict_tool_schemas_can_be_disabled_per_route() -> None:
"""Claude routes cap strict tools; the toolset must be sendable without strict."""
agent = factory.build_strix_agent(is_root=True, strict_tool_schemas=False)
function_tools = [t for t in agent.tools if isinstance(t, FunctionTool)]
assert function_tools
assert not any(t.strict_json_schema for t in function_tools)
def test_disabling_strict_leaves_shared_tools_untouched() -> None:
factory.build_strix_agent(is_root=True, strict_tool_schemas=False)
agent = factory.build_strix_agent(is_root=True)
assert any(t.strict_json_schema for t in agent.tools if isinstance(t, FunctionTool))
+81
View File
@@ -0,0 +1,81 @@
"""A bootstrap that dies mid-setup must not leave its transport behind.
The bootstrap now runs concurrently with the scan start, so teardown can
cancel it at any await including inside ``Client.connect()``, where the
client exists but no caller will ever see it to close it.
"""
from __future__ import annotations
import asyncio
import sys
import types
from typing import Any
import pytest
from strix.runtime.caido_bootstrap import bootstrap_caido
class _FakeExecResult:
stderr = b""
exit_code = 0
def __init__(self, stdout: str) -> None:
self.stdout = stdout
def ok(self) -> bool:
return True
class _FakeSession:
async def exec(self, *_args: Any, **_kwargs: Any) -> _FakeExecResult:
return _FakeExecResult('{"data":{"loginAsGuest":{"token":{"accessToken":"t"}}}}')
class _FakeClient:
def __init__(self, connect_error: BaseException) -> None:
self.connect_error = connect_error
self.closed = False
async def connect(self) -> None:
raise self.connect_error
async def aclose(self) -> None:
self.closed = True
async def _bootstrap_expecting(
monkeypatch: pytest.MonkeyPatch, error: BaseException
) -> _FakeClient:
"""Run a bootstrap whose ``connect()`` fails with ``error``."""
client = _FakeClient(error)
# The SDK is imported inside bootstrap_caido (it is slow to import), so the
# fakes are injected as the modules it imports.
sdk = types.ModuleType("caido_sdk_client")
sdk.Client = lambda *_a, **_k: client # type: ignore[attr-defined]
sdk.TokenAuthOptions = lambda token: token # type: ignore[attr-defined]
sdk_types = types.ModuleType("caido_sdk_client.types")
sdk_types.CreateProjectOptions = lambda **_k: None # type: ignore[attr-defined]
monkeypatch.setitem(sys.modules, "caido_sdk_client", sdk)
monkeypatch.setitem(sys.modules, "caido_sdk_client.types", sdk_types)
with pytest.raises(type(error)):
await bootstrap_caido(
_FakeSession(), # type: ignore[arg-type]
host_url="http://host",
container_url="http://container",
)
return client
async def test_cancellation_during_connect_closes_the_client(
monkeypatch: pytest.MonkeyPatch,
) -> None:
client = await _bootstrap_expecting(monkeypatch, asyncio.CancelledError())
assert client.closed
async def test_failed_connect_closes_the_client(monkeypatch: pytest.MonkeyPatch) -> None:
client = await _bootstrap_expecting(monkeypatch, RuntimeError("no listener"))
assert client.closed
+103
View File
@@ -0,0 +1,103 @@
"""Tests for the concurrent Caido bootstrap handle."""
from __future__ import annotations
import asyncio
from typing import Any
import pytest
from strix.runtime.caido_handle import CaidoBootstrapHandle
class _FakeClient:
def __init__(self) -> None:
self.closed = False
async def aclose(self) -> None:
self.closed = True
def _handle(coro: Any) -> CaidoBootstrapHandle:
return CaidoBootstrapHandle(asyncio.ensure_future(coro))
async def test_get_waits_for_the_bootstrap() -> None:
client = _FakeClient()
started = asyncio.Event()
async def _bootstrap() -> Any:
started.set()
await asyncio.sleep(0.01)
return client
handle = _handle(_bootstrap())
await started.wait()
assert handle.peek() is None
assert await handle.get() is client
assert handle.peek() is client
async def test_get_reraises_bootstrap_failure_to_every_caller() -> None:
async def _bootstrap() -> Any:
raise RuntimeError("caido never came up")
handle = _handle(_bootstrap())
for _ in range(2):
with pytest.raises(RuntimeError, match="caido never came up"):
await handle.get()
assert handle.peek() is None
async def test_caller_cancellation_does_not_cancel_the_shared_bootstrap() -> None:
client = _FakeClient()
async def _bootstrap() -> Any:
await asyncio.sleep(0.05)
return client
handle = _handle(_bootstrap())
with pytest.raises(TimeoutError):
await asyncio.wait_for(handle.get(), timeout=0.01)
assert await handle.get() is client
async def test_aclose_closes_a_finished_client() -> None:
client = _FakeClient()
async def _bootstrap() -> Any:
return client
handle = _handle(_bootstrap())
await handle.get()
await handle.aclose()
assert client.closed is True
async def test_aclose_cancels_an_in_flight_bootstrap() -> None:
cancelled = asyncio.Event()
async def _bootstrap() -> Any:
try:
await asyncio.sleep(10)
except asyncio.CancelledError:
cancelled.set()
raise
return _FakeClient()
handle = _handle(_bootstrap())
await asyncio.sleep(0)
await handle.aclose()
assert cancelled.is_set()
async def test_aclose_swallows_a_failed_bootstrap() -> None:
async def _bootstrap() -> Any:
raise RuntimeError("boom")
handle = _handle(_bootstrap())
with pytest.raises(RuntimeError, match="boom"):
await handle.get()
await handle.aclose()
+88
View File
@@ -0,0 +1,88 @@
"""Tests for the --mcp-config CLI flag."""
from __future__ import annotations
import importlib
import os
import sys
from types import SimpleNamespace
from typing import TYPE_CHECKING, Any
import pytest
if TYPE_CHECKING:
from pathlib import Path
cli_main: Any = importlib.import_module("strix.interface.main")
def _stub_settings(monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setattr(
cli_main,
"load_settings",
lambda: SimpleNamespace(runtime=SimpleNamespace(max_local_copy_mb=1024)),
)
def test_mcp_config_flag_sets_loader_override(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
config = tmp_path / "servers.json"
config.write_text("[]", encoding="utf-8")
_stub_settings(monkeypatch)
# delenv records "originally absent" so monkeypatch removes whatever the
# parser sets, keeping the override from leaking into other tests.
monkeypatch.delenv("STRIX_MCP_CONFIG", raising=False)
monkeypatch.setattr(
sys, "argv", ["strix", "-t", "https://test.com/", "-n", "--mcp-config", str(config)]
)
args = cli_main.parse_arguments()
assert args.mcp_config == str(config)
assert os.environ["STRIX_MCP_CONFIG"] == str(config)
def test_mcp_config_flag_rejects_missing_file(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
) -> None:
_stub_settings(monkeypatch)
monkeypatch.delenv("STRIX_MCP_CONFIG", raising=False)
missing = tmp_path / "nope.json"
monkeypatch.setattr(
sys, "argv", ["strix", "-t", "https://test.com/", "-n", "--mcp-config", str(missing)]
)
with pytest.raises(SystemExit):
cli_main.parse_arguments()
assert "--mcp-config file not found" in capsys.readouterr().err
def test_mcp_server_flags_set_selection_env(monkeypatch: pytest.MonkeyPatch) -> None:
_stub_settings(monkeypatch)
monkeypatch.delenv("STRIX_MCP_ONLY", raising=False)
monkeypatch.delenv("STRIX_MCP_EXCLUDE", raising=False)
monkeypatch.setattr(
sys,
"argv",
[
"strix",
"-t",
"https://test.com/",
"-n",
"--mcp-server",
"a",
"--mcp-server",
"b",
"--mcp-exclude",
"c",
],
)
cli_main.parse_arguments()
assert os.environ["STRIX_MCP_ONLY"] == "a,b"
assert os.environ["STRIX_MCP_EXCLUDE"] == "c"
+15
View File
@@ -227,3 +227,18 @@ def test_resume_still_requires_targets_or_a_workspace(
cli_main.parse_arguments()
assert "has no targets_info" in capsys.readouterr().err
def test_resume_non_object_run_json_exits(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
monkeypatch.chdir(tmp_path)
run_dir = tmp_path / "strix_runs" / "pentest_abcd"
run_dir.mkdir(parents=True)
(run_dir / "run.json").write_text("[]", encoding="utf-8")
monkeypatch.setattr(sys, "argv", ["strix", "--resume", "pentest_abcd"])
with pytest.raises(SystemExit) as exc_info:
cli_main.parse_arguments()
assert exc_info.value.code == 2
captured = capsys.readouterr()
assert "run.json unreadable" in captured.err
assert "not an object" in captured.err
+3 -3
View File
@@ -31,7 +31,7 @@ def test_context_window_chatgpt_prefix_skips_provider_auth(
calls.append(model)
return {"max_input_tokens": 1_050_000, "max_output_tokens": 128_000}
monkeypatch.setattr("strix.llm.context_budget.litellm.get_model_info", _model_info)
monkeypatch.setattr("litellm.get_model_info", _model_info)
try:
assert context_budget.context_window("chatgpt/gpt-5.6-luna") == 1_050_000
assert calls == ["gpt-5.6-luna"]
@@ -45,7 +45,7 @@ def test_context_window_unmapped_uses_fallback(monkeypatch: pytest.MonkeyPatch)
def _raise(_model: str) -> dict[str, int]:
raise ValueError("This model isn't mapped yet.")
monkeypatch.setattr("strix.llm.context_budget.litellm.get_model_info", _raise)
monkeypatch.setattr("litellm.get_model_info", _raise)
expected = load_settings().context.fallback_context_tokens
assert context_budget.context_window("totally-made-up-model") == expected
context_budget._model_info.cache_clear()
@@ -55,7 +55,7 @@ def test_count_tokens_fallback_on_error(monkeypatch: pytest.MonkeyPatch) -> None
def _raise(**_kwargs: object) -> int:
raise RuntimeError("no tokenizer")
monkeypatch.setattr("strix.llm.context_budget.litellm.token_counter", _raise)
monkeypatch.setattr("litellm.token_counter", _raise)
# Falls back to UTF-8 byte length (upper bound on tokens).
assert context_budget.count_tokens("weird-model", "x" * 400) == 400
assert context_budget.count_tokens("weird-model", "😀" * 10) == 40
+284
View File
@@ -0,0 +1,284 @@
"""Tests for the scan coverage ledger."""
from __future__ import annotations
import json
import threading
from concurrent.futures import ThreadPoolExecutor
from typing import TYPE_CHECKING, Any
import pytest
from strix.tools.coverage.tools import (
_list_impl,
_record_impl,
_update_impl,
get_coverage_entries,
hydrate_coverage_from_disk,
outcome_counts,
)
if TYPE_CHECKING:
from pathlib import Path
@pytest.fixture(autouse=True)
def coverage_store(tmp_path: Path) -> Path:
hydrate_coverage_from_disk(tmp_path)
return tmp_path
def _record(**overrides: str) -> dict[str, Any]:
kwargs = {
"surface": "POST /api/orders/{id}",
"risk_area": "object-level authorization",
"outcome": "no_issue_found",
"evidence": "Tested with two tenants; both received 403.",
"agent_id": "agent-1",
"agent_name": "authz-tester",
}
kwargs.update(overrides)
return _record_impl(**kwargs)
def test_record_persists_entry(coverage_store: Path) -> None:
result = _record()
assert result["success"] is True
entries = get_coverage_entries()
assert len(entries) == 1
assert entries[0]["surface"] == "POST /api/orders/{id}"
assert entries[0]["outcome"] == "no_issue_found"
assert entries[0]["agent_name"] == "authz-tester"
assert (coverage_store / "coverage.json").exists()
def test_record_normalizes_outcome() -> None:
assert _record(outcome="Needs Follow-Up")["success"] is True
assert get_coverage_entries()[0]["outcome"] == "needs_follow_up"
def test_record_rejects_unknown_outcome() -> None:
result = _record(outcome="looks fine")
assert result["success"] is False
assert any("Invalid outcome" in e for e in result["errors"])
assert not get_coverage_entries()
def test_record_requires_surface_and_risk_area() -> None:
result = _record(surface=" ", risk_area="")
assert result["success"] is False
joined = " ".join(result["errors"])
assert "surface" in joined
assert "risk_area" in joined
@pytest.mark.parametrize("outcome", ["ruled_out", "not_applicable", "needs_follow_up"])
def test_evidence_required_for_asserted_outcomes(outcome: str) -> None:
result = _record(outcome=outcome, evidence=" ")
assert result["success"] is False
assert any("evidence is required" in e for e in result["errors"])
def test_evidence_optional_for_reported() -> None:
assert _record(outcome="reported", evidence="")["success"] is True
def test_outcome_counts_and_filtering() -> None:
_record(surface="/login", outcome="reported", evidence="")
_record(surface="/search", outcome="no_issue_found")
_record(surface="/upload", outcome="needs_follow_up", evidence="No credentials to test.")
assert outcome_counts() == {"reported": 1, "no_issue_found": 1, "needs_follow_up": 1}
listed = _list_impl(outcome="needs_follow_up", surface=None, caller_agent_id="agent-1")
assert listed["filtered_count"] == 1
assert listed["entries"][0]["surface"] == "/upload"
assert listed["entries"][0]["by_you"] is True
by_surface = _list_impl(outcome=None, surface="sea", caller_agent_id=None)
assert by_surface["filtered_count"] == 1
assert by_surface["entries"][0]["surface"] == "/search"
def test_list_rejects_unknown_outcome_filter() -> None:
result = _list_impl(outcome="bogus", surface=None, caller_agent_id=None)
assert result["success"] is False
def test_hydrate_reloads_from_disk(coverage_store: Path) -> None:
_record()
hydrate_coverage_from_disk(coverage_store)
entries = get_coverage_entries()
assert len(entries) == 1
assert entries[0]["risk_area"] == "object-level authorization"
def _update(entry_id: str, **overrides: str) -> dict[str, Any]:
kwargs = {
"entry_id": entry_id,
"outcome": "reported",
"evidence": "Got staging credentials and confirmed the IDOR.",
"agent_id": "agent-2",
"agent_name": "followup-tester",
}
kwargs.update(overrides)
return _update_impl(**kwargs)
def test_update_moves_outcome_and_keeps_history() -> None:
recorded = _record(outcome="needs_follow_up", evidence="No credentials to test.")
entry_id = str(recorded["entry_id"])
result = _update(entry_id)
assert result["success"] is True
assert result["previous_outcome"] == "needs_follow_up"
assert result["outcome"] == "reported"
entries = get_coverage_entries()
assert len(entries) == 1, "update must not create a parallel entry"
entry = entries[0]
assert entry["outcome"] == "reported"
assert entry["agent_name"] == "followup-tester"
assert entry["history"] == [
{
"outcome": "needs_follow_up",
"recorded_at": entry["created_at"],
"evidence": "No credentials to test.",
"agent_name": "authz-tester",
}
]
assert outcome_counts() == {"reported": 1}
def test_update_can_reopen_a_closed_entry() -> None:
recorded = _record(outcome="ruled_out", evidence="Guard at auth.py:40 covers the path.")
entry_id = str(recorded["entry_id"])
_update(
entry_id,
outcome="needs_follow_up",
evidence="The guard is skipped on the /v2 alias; reachability unproven.",
)
assert outcome_counts() == {"needs_follow_up": 1}
listed = _list_impl(outcome=None, surface=None, caller_agent_id=None)
assert listed["entries"][0]["previous_outcomes"] == ["ruled_out"]
def test_update_enforces_evidence_for_closing_outcomes() -> None:
entry_id = str(_record(outcome="needs_follow_up", evidence="unknown")["entry_id"])
result = _update(entry_id, outcome="ruled_out", evidence=" ")
assert result["success"] is False
assert get_coverage_entries()[0]["outcome"] == "needs_follow_up"
def test_update_rejects_unknown_entry() -> None:
result = _update("nope")
assert result["success"] is False
assert "list_coverage" in str(result["error"])
def test_update_persists_to_disk(coverage_store: Path) -> None:
entry_id = str(_record(outcome="needs_follow_up", evidence="No creds.")["entry_id"])
_update(entry_id)
hydrate_coverage_from_disk(coverage_store)
entry = get_coverage_entries()[0]
assert entry["outcome"] == "reported"
assert len(entry["history"]) == 1
def test_recording_a_duplicate_surface_is_refused_with_the_existing_id() -> None:
first = _record_impl(
surface="/api/invoices",
risk_area="IDOR",
outcome="needs_follow_up",
evidence="No second tenant account to test cross-tenant reads with.",
agent_id="a1",
agent_name="Recon",
)
duplicate = _record_impl(
surface=" /API/Invoices ",
risk_area="idor",
outcome="reported",
evidence="Cross-tenant read confirmed.",
agent_id="a2",
agent_name="Authz",
)
assert duplicate["success"] is False
assert duplicate["existing_entry_id"] == first["entry_id"]
assert duplicate["existing_outcome"] == "needs_follow_up"
assert "update_coverage" in duplicate["error"]
assert len(get_coverage_entries()) == 1
def test_a_different_risk_area_on_one_surface_is_still_its_own_entry() -> None:
_record_impl(
surface="/api/invoices",
risk_area="IDOR",
outcome="no_issue_found",
evidence="Tenant id read from the session.",
agent_id="a1",
agent_name="Authz",
)
second = _record_impl(
surface="/api/invoices",
risk_area="SQL injection",
outcome="no_issue_found",
evidence="Parameterized throughout.",
agent_id="a1",
agent_name="Injection",
)
assert second["success"] is True
assert len(get_coverage_entries()) == 2
def test_concurrent_records_of_one_surface_yield_a_single_row() -> None:
"""Duplicate detection and insertion must be one critical section.
Two agents recording the same surface at the same moment would otherwise
both pass the "no duplicate" check, and the report would show a stale
conclusion beside its replacement the exact outcome the rejection exists
to prevent.
"""
barrier = threading.Barrier(8)
def attempt(index: int) -> dict[str, Any]:
barrier.wait()
return _record(agent_id=f"agent-{index}", agent_name=f"tester-{index}")
with ThreadPoolExecutor(max_workers=8) as pool:
results = list(pool.map(attempt, range(8)))
assert sum(1 for result in results if result["success"]) == 1
assert len(get_coverage_entries()) == 1
def test_concurrent_records_all_survive_persistence(coverage_store: Path) -> None:
"""A writer holding an older snapshot must not win the rename.
If it did, the mirror would come back short on resume and coverage
recorded before a crash would silently disappear from the report.
"""
barrier = threading.Barrier(8)
def attempt(index: int) -> dict[str, Any]:
barrier.wait()
return _record(surface=f"GET /api/resource/{index}", agent_id=f"agent-{index}")
with ThreadPoolExecutor(max_workers=8) as pool:
list(pool.map(attempt, range(8)))
persisted = json.loads((coverage_store / "coverage.json").read_text(encoding="utf-8"))
assert len(persisted) == 8
hydrate_coverage_from_disk(coverage_store)
assert len(get_coverage_entries()) == 8
+65
View File
@@ -0,0 +1,65 @@
"""finish_scan confronts the root agent with the coverage the runtime can see."""
from __future__ import annotations
from typing import TYPE_CHECKING
import pytest
from strix.tools.coverage.tools import _record_impl, hydrate_coverage_from_disk
from strix.tools.finish.tool import _coverage_summary
if TYPE_CHECKING:
from pathlib import Path
_GRAPH = {
"statuses": {"agent-1": "completed"},
"names": {"agent-1": "injection-tester"},
"metadata": {"agent-1": {"skills": ["sql_injection", "xss"]}},
}
@pytest.fixture(autouse=True)
def _empty_ledger(tmp_path: Path) -> None:
hydrate_coverage_from_disk(tmp_path)
def _record(risk_area: str) -> None:
_record_impl(
surface="POST /api/orders/{id}",
risk_area=risk_area,
outcome="no_issue_found",
evidence="Parameters fuzzed; no anomalies.",
agent_id="agent-1",
agent_name="injection-tester",
)
def test_unrecorded_risk_class_is_reported_back_to_the_root_agent() -> None:
_record("SQL injection")
summary = _coverage_summary(_GRAPH)
assert summary["coverage_recorded"] == 1
assert len(summary["coverage_gaps"]) == 1
assert "xss" in summary["coverage_gaps"][0]
assert "unexamined" in summary["coverage_gap_warning"]
def test_fully_accounted_coverage_raises_no_gap_warning() -> None:
_record("SQL injection")
_record("cross-site scripting")
summary = _coverage_summary(_GRAPH)
assert "coverage_gaps" not in summary
assert "coverage_gap_warning" not in summary
def test_an_empty_ledger_still_warns_first() -> None:
summary = _coverage_summary(_GRAPH)
assert summary["coverage_recorded"] == 0
assert "No coverage was recorded" in summary["coverage_warning"]
+30
View File
@@ -10,6 +10,7 @@ import pytest
from strix.core.inputs import (
build_root_task,
build_scan_targets,
build_scope_context,
child_initial_input,
make_model_settings,
@@ -363,6 +364,35 @@ def test_make_model_settings_timeout_survives_reasoning_resolve() -> None:
assert settings.extra_args["timeout"] == 120.0
def test_scan_targets_prefer_the_workspace_checkout_over_the_remote_url() -> None:
config = {
"targets": [
{
"type": "repository",
"details": {
"target_repo": "https://github.com/acme/billing",
"workspace_subdir": "billing",
},
},
{"type": "web_application", "details": {"target_url": "https://app.example.com"}},
]
}
assert build_scan_targets(config) == ["/workspace/billing", "https://app.example.com"]
def test_scan_targets_drop_empty_and_duplicate_entries() -> None:
config = {
"targets": [
{"type": "web_application", "details": {"target_url": "https://app.example.com"}},
{"type": "web_application", "details": {"target_url": "https://app.example.com"}},
{"type": "ip_address", "details": {}},
]
}
assert build_scan_targets(config) == ["https://app.example.com"]
def test_openrouter_attribution_rides_on_the_request_headers() -> None:
# litellm.headers is ignored once a request carries any header of its own,
# so the attribution must be part of the per-request headers.

Some files were not shown because too many files have changed in this diff Show More