mirror of
https://github.com/usestrix/strix.git
synced 2026-08-25 12:22:37 +02:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8e31009c99 | ||
|
|
91afce6847 | ||
|
|
991c017ddf | ||
|
|
4b6b2b7920 | ||
|
|
93e2c3d7e9 | ||
|
|
614d2bfb22 |
@@ -320,42 +320,6 @@ strix auth status # show the active sign-in
|
|||||||
strix auth logout # forget the sign-in
|
strix auth logout # forget the sign-in
|
||||||
```
|
```
|
||||||
|
|
||||||
#### Sign in with an OpenCode subscription
|
|
||||||
|
|
||||||
You can also run Strix on [OpenCode Zen](https://opencode.ai/docs/zen/) credits or an [OpenCode Go](https://opencode.ai/docs/go/) subscription:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
strix auth login opencode # paste your API key from opencode.ai/auth
|
|
||||||
|
|
||||||
export STRIX_LLM="opencode/claude-sonnet-5" # opencode/<model> runs on Zen credits
|
|
||||||
export STRIX_LLM="opencode-go/kimi-k3" # opencode-go/<model> runs on the Go subscription
|
|
||||||
strix --target ./app-directory
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Connect your own MCP servers
|
|
||||||
|
|
||||||
Strix can connect to Model Context Protocol (MCP) servers you list and expose their tools to the agent during a run. Create `~/.strix/mcp-servers.json` with a JSON list of servers. Each entry is either a local `stdio` server that Strix launches as a subprocess, or a remote `http` server:
|
|
||||||
|
|
||||||
```json
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"name": "local_fs",
|
|
||||||
"transport": "stdio",
|
|
||||||
"command": "npx",
|
|
||||||
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/path/to/project"]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "github",
|
|
||||||
"transport": "http",
|
|
||||||
"url": "https://api.githubcopilot.com/mcp/",
|
|
||||||
"auth": { "kind": "bearer", "token": "your-token" },
|
|
||||||
"allowed_tools": ["list_issues"]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
Each server's tools are namespaced by `name` (for example `local_fs_read_file`). Omit `allowed_tools` to expose every tool the server offers, or set it to a list to restrict which tools the agent can call. The file is optional, and a server that fails to connect is skipped without failing the run. You can point Strix at a different file with `STRIX_MCP_CONFIG`.
|
|
||||||
|
|
||||||
**Recommended models for best results:**
|
**Recommended models for best results:**
|
||||||
|
|
||||||
- [OpenAI GPT-5.4](https://openai.com/api/) - `openai/gpt-5.4`
|
- [OpenAI GPT-5.4](https://openai.com/api/) - `openai/gpt-5.4`
|
||||||
|
|||||||
+1
-2
@@ -47,8 +47,7 @@
|
|||||||
"pages": [
|
"pages": [
|
||||||
"integrations/github-actions",
|
"integrations/github-actions",
|
||||||
"integrations/ci-cd",
|
"integrations/ci-cd",
|
||||||
"integrations/coding-agents",
|
"integrations/coding-agents"
|
||||||
"integrations/mcp"
|
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -1,131 +0,0 @@
|
|||||||
---
|
|
||||||
title: "MCP Servers"
|
|
||||||
description: "Connect your own MCP servers and expose their tools to the agent"
|
|
||||||
---
|
|
||||||
|
|
||||||
Strix can connect to [Model Context Protocol (MCP)](https://modelcontextprotocol.io) servers you list and expose their tools to the agent during a run. Use this to let the agent read how your system is actually built instead of inferring it from the outside.
|
|
||||||
|
|
||||||
A few things it pays off for:
|
|
||||||
|
|
||||||
- **A database server.** The agent can read the schema and access policies and see tables left readable without them, rather than guessing from responses.
|
|
||||||
- **A hosting or infrastructure server.** Deployments, domains and environment variable names tell it what is really running, so it tests what exists instead of what it discovered by crawling.
|
|
||||||
- **An issue tracker.** Known and accepted risks stop the agent re-reporting findings you already triaged.
|
|
||||||
- **A logging server.** Reading logs lets it confirm an exploit attempt actually landed instead of inferring it from a status code.
|
|
||||||
|
|
||||||
## Setup
|
|
||||||
|
|
||||||
Create the file `~/.strix/mcp-servers.json`. It holds a JSON list of the servers you want the agent to reach. Each entry is either a local `stdio` server that Strix launches as a subprocess, or a remote `http` server.
|
|
||||||
|
|
||||||
Create the directory if it does not exist, then write the file:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p ~/.strix
|
|
||||||
```
|
|
||||||
|
|
||||||
Paste the servers you want into `~/.strix/mcp-servers.json`. The example below shows one of each transport: a local filesystem server over `stdio` and a remote GitHub server over `http` with a bearer token:
|
|
||||||
|
|
||||||
```json
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"name": "local_fs",
|
|
||||||
"transport": "stdio",
|
|
||||||
"command": "npx",
|
|
||||||
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/path/to/project"]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "github",
|
|
||||||
"transport": "http",
|
|
||||||
"url": "https://api.githubcopilot.com/mcp/",
|
|
||||||
"auth": { "kind": "bearer", "token": "your-token" },
|
|
||||||
"allowed_tools": ["list_issues"]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
Strix reads this file at the start of each run. There is no default file, so no MCP tools are loaded until you create it. Edit `command`, `args`, `url`, and `token` to match your own servers.
|
|
||||||
|
|
||||||
## Fields
|
|
||||||
|
|
||||||
<ParamField path="name" type="string" required>
|
|
||||||
A short label for the connection. Each server's tools are namespaced by
|
|
||||||
`name` (for example `local_fs_read_file`), so two servers can offer the same
|
|
||||||
tool name without colliding.
|
|
||||||
</ParamField>
|
|
||||||
|
|
||||||
<ParamField path="transport" type="string">
|
|
||||||
`stdio` for a local subprocess server, or `http` for a remote server.
|
|
||||||
</ParamField>
|
|
||||||
|
|
||||||
<ParamField path="command" type="string">
|
|
||||||
For `stdio` servers: the executable Strix launches (for example `npx`).
|
|
||||||
</ParamField>
|
|
||||||
|
|
||||||
<ParamField path="args" type="array">
|
|
||||||
For `stdio` servers: the arguments passed to `command`.
|
|
||||||
</ParamField>
|
|
||||||
|
|
||||||
<ParamField path="url" type="string">
|
|
||||||
For `http` servers: the server endpoint URL.
|
|
||||||
</ParamField>
|
|
||||||
|
|
||||||
<ParamField path="auth" type="object">
|
|
||||||
For `http` servers that need a bearer token:
|
|
||||||
`{ "kind": "bearer", "token": "your-token" }`.
|
|
||||||
</ParamField>
|
|
||||||
|
|
||||||
<ParamField path="allowed_tools" type="array">
|
|
||||||
Restrict which tools the agent can call. Omit it to expose every tool the
|
|
||||||
server offers, or set it to a list of tool names to allow only those. Strix
|
|
||||||
does not decide for you which of a server's tools only read and which change
|
|
||||||
things, so run the server in its own read-only mode if it has one.
|
|
||||||
</ParamField>
|
|
||||||
|
|
||||||
<ParamField path="notes" type="string">
|
|
||||||
Free-text notes for the agent about what this connection is and how you want
|
|
||||||
it used, for example "Staging analytics database, read-only, prefer aggregate
|
|
||||||
queries." When set, the notes are given to the agent at the start of the run
|
|
||||||
as a description of the connection.
|
|
||||||
</ParamField>
|
|
||||||
|
|
||||||
## Choosing connections per run
|
|
||||||
|
|
||||||
By default every connection in the file is used on each run. To narrow it for a
|
|
||||||
single run without editing the file, use either flag (both repeatable):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
strix --mcp-server github -t ... # use only the named connection(s)
|
|
||||||
strix --mcp-exclude staging-db -t ... # use everything except the named one(s)
|
|
||||||
```
|
|
||||||
|
|
||||||
`--mcp-server` keeps only the connections you name; `--mcp-exclude` drops the
|
|
||||||
ones you name. Connection names must be unique in the file; if two entries share
|
|
||||||
a name, the first is kept and the rest are ignored.
|
|
||||||
|
|
||||||
## Pointing at a different file
|
|
||||||
|
|
||||||
To read the config from another path instead of `~/.strix/mcp-servers.json`, either pass `--mcp-config <path>` on the command line:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
strix --mcp-config ./mcp-servers.json -t ...
|
|
||||||
```
|
|
||||||
|
|
||||||
or set the `STRIX_MCP_CONFIG` environment variable to that path. The flag takes precedence when both are given.
|
|
||||||
|
|
||||||
## Startup confirmation
|
|
||||||
|
|
||||||
When servers are configured, Strix prints a one-line summary at scan startup, for example `MCP: connected 1 server (14 tools): local_fs`, so you can confirm your servers connected.
|
|
||||||
|
|
||||||
## Seeing the calls
|
|
||||||
|
|
||||||
Each call the agent makes to one of your servers is shown with its own icon and
|
|
||||||
labelled with the connection it went out to, in the terminal and in the run
|
|
||||||
viewer (`strix view`), so a call that left Strix for a server you connected is
|
|
||||||
easy to pick out of a transcript. The terminal shows the call and its arguments;
|
|
||||||
results can be large and arbitrary, so read them in the viewer, which shows a
|
|
||||||
preview you can expand.
|
|
||||||
|
|
||||||
## Behavior
|
|
||||||
|
|
||||||
- The config file is optional. Without it, a run simply gets no MCP tools.
|
|
||||||
- A server that fails to connect is skipped and logged, and the run continues without it.
|
|
||||||
- A single malformed entry is skipped without blocking the valid ones.
|
|
||||||
@@ -241,8 +241,6 @@ ignore = [
|
|||||||
"tests/test_stream_idle_timeout.py" = ["N802", "SLF001"]
|
"tests/test_stream_idle_timeout.py" = ["N802", "SLF001"]
|
||||||
"tests/test_unknown_tool_recovery.py" = ["N802"]
|
"tests/test_unknown_tool_recovery.py" = ["N802"]
|
||||||
"tests/test_report_pdf.py" = ["S105", "S106"]
|
"tests/test_report_pdf.py" = ["S105", "S106"]
|
||||||
# Fake MCP server matches the SDK's MCPServer signature; its args are unused.
|
|
||||||
"tests/test_mcp_client.py" = ["S105", "S106", "ARG002"]
|
|
||||||
# Stdlib HTTP handler overrides (do_GET/do_POST) and lazy imports that avoid a
|
# Stdlib HTTP handler overrides (do_GET/do_POST) and lazy imports that avoid a
|
||||||
# circular dependency with strix.telemetry / strix.interface.viewer.report_pdf.
|
# circular dependency with strix.telemetry / strix.interface.viewer.report_pdf.
|
||||||
"strix/interface/viewer/server.py" = ["N802", "PLC0415"]
|
"strix/interface/viewer/server.py" = ["N802", "PLC0415"]
|
||||||
@@ -272,10 +270,6 @@ ignore = [
|
|||||||
"strix/tools/thinking/tool.py" = ["TC002"]
|
"strix/tools/thinking/tool.py" = ["TC002"]
|
||||||
"strix/tools/web_search/tool.py" = ["TC002"]
|
"strix/tools/web_search/tool.py" = ["TC002"]
|
||||||
"strix/tools/proxy/tools.py" = ["TC002", "PLR0911"]
|
"strix/tools/proxy/tools.py" = ["TC002", "PLR0911"]
|
||||||
# The generated Caido GraphQL schema is slow to import, so the SDK is imported
|
|
||||||
# on first proxy call instead of at module scope (keeps it off the launch path).
|
|
||||||
"strix/tools/proxy/caido_api.py" = ["PLC0415"]
|
|
||||||
"strix/runtime/caido_bootstrap.py" = ["PLC0415"]
|
|
||||||
"strix/tools/agents_graph/tools.py" = ["TC002"]
|
"strix/tools/agents_graph/tools.py" = ["TC002"]
|
||||||
"strix/agents/factory.py" = ["TC002"]
|
"strix/agents/factory.py" = ["TC002"]
|
||||||
# Entry point: ``Path`` is used at runtime by the typing of the
|
# Entry point: ``Path`` is used at runtime by the typing of the
|
||||||
@@ -286,13 +280,6 @@ ignore = [
|
|||||||
# a runtime ``Callable`` annotation on ``vulnerability_found_callback``.
|
# a runtime ``Callable`` annotation on ``vulnerability_found_callback``.
|
||||||
"strix/report/state.py" = ["TC003", "PLR0912", "PLR0915", "E501", "PERF401", "PLC0415"]
|
"strix/report/state.py" = ["TC003", "PLR0912", "PLR0915", "E501", "PERF401", "PLC0415"]
|
||||||
"strix/report/usage.py" = ["PLC0415"]
|
"strix/report/usage.py" = ["PLC0415"]
|
||||||
# LiteLLM and the Docker SDK are imported on first use, not at module scope:
|
|
||||||
# both cost seconds to import and neither is needed until a model call is made
|
|
||||||
# (or, for Docker, unless the Docker runtime backend is in use).
|
|
||||||
"strix/core/execution.py" = ["PLC0415"]
|
|
||||||
"strix/report/pricing.py" = ["PLC0415"]
|
|
||||||
"strix/llm/compaction.py" = ["PLC0415"]
|
|
||||||
"strix/llm/context_budget.py" = ["PLC0415"]
|
|
||||||
# Lazy import of strix.config.models avoids a circular dependency between the
|
# Lazy import of strix.config.models avoids a circular dependency between the
|
||||||
# report pipeline and the config layer.
|
# report pipeline and the config layer.
|
||||||
"strix/report/dedupe.py" = ["PLC0415"]
|
"strix/report/dedupe.py" = ["PLC0415"]
|
||||||
|
|||||||
@@ -26,7 +26,6 @@ from strix.tools.agents_graph.tools import (
|
|||||||
view_agent_graph,
|
view_agent_graph,
|
||||||
wait_for_agents,
|
wait_for_agents,
|
||||||
)
|
)
|
||||||
from strix.tools.coverage.tools import list_coverage, record_coverage, update_coverage
|
|
||||||
from strix.tools.finish.tool import finish_scan
|
from strix.tools.finish.tool import finish_scan
|
||||||
from strix.tools.load_skill.tool import load_skill
|
from strix.tools.load_skill.tool import load_skill
|
||||||
from strix.tools.notes.tools import (
|
from strix.tools.notes.tools import (
|
||||||
@@ -53,11 +52,6 @@ from strix.tools.reporting.tool import (
|
|||||||
)
|
)
|
||||||
from strix.tools.respond.tool import respond_to_user
|
from strix.tools.respond.tool import respond_to_user
|
||||||
from strix.tools.thinking.tool import think
|
from strix.tools.thinking.tool import think
|
||||||
from strix.tools.threat_model.tools import (
|
|
||||||
amend_threat_model,
|
|
||||||
get_threat_model,
|
|
||||||
save_threat_model,
|
|
||||||
)
|
|
||||||
from strix.tools.todo.tools import (
|
from strix.tools.todo.tools import (
|
||||||
create_todo,
|
create_todo,
|
||||||
delete_todo,
|
delete_todo,
|
||||||
@@ -534,12 +528,6 @@ _BASE_TOOLS: tuple[Tool, ...] = (
|
|||||||
get_note,
|
get_note,
|
||||||
update_note,
|
update_note,
|
||||||
delete_note,
|
delete_note,
|
||||||
record_coverage,
|
|
||||||
update_coverage,
|
|
||||||
list_coverage,
|
|
||||||
get_threat_model,
|
|
||||||
save_threat_model,
|
|
||||||
amend_threat_model,
|
|
||||||
web_search,
|
web_search,
|
||||||
create_vulnerability_report,
|
create_vulnerability_report,
|
||||||
create_dependency_report,
|
create_dependency_report,
|
||||||
@@ -608,7 +596,6 @@ def build_strix_agent(
|
|||||||
is_root: bool,
|
is_root: bool,
|
||||||
scan_mode: str = "deep",
|
scan_mode: str = "deep",
|
||||||
is_whitebox: bool = False,
|
is_whitebox: bool = False,
|
||||||
is_diff_scoped: bool = False,
|
|
||||||
interactive: bool = False,
|
interactive: bool = False,
|
||||||
chat_completions_tools: bool = False,
|
chat_completions_tools: bool = False,
|
||||||
strict_tool_schemas: bool = True,
|
strict_tool_schemas: bool = True,
|
||||||
@@ -636,7 +623,6 @@ def build_strix_agent(
|
|||||||
scan_mode=scan_mode,
|
scan_mode=scan_mode,
|
||||||
is_whitebox=is_whitebox,
|
is_whitebox=is_whitebox,
|
||||||
is_root=is_root,
|
is_root=is_root,
|
||||||
is_diff_scoped=is_diff_scoped,
|
|
||||||
interactive=interactive,
|
interactive=interactive,
|
||||||
system_prompt_context=system_prompt_context,
|
system_prompt_context=system_prompt_context,
|
||||||
)
|
)
|
||||||
@@ -694,7 +680,6 @@ def make_child_factory(
|
|||||||
*,
|
*,
|
||||||
scan_mode: str = "deep",
|
scan_mode: str = "deep",
|
||||||
is_whitebox: bool = False,
|
is_whitebox: bool = False,
|
||||||
is_diff_scoped: bool = False,
|
|
||||||
interactive: bool = False,
|
interactive: bool = False,
|
||||||
chat_completions_tools: bool = False,
|
chat_completions_tools: bool = False,
|
||||||
strict_tool_schemas: bool = True,
|
strict_tool_schemas: bool = True,
|
||||||
@@ -714,7 +699,6 @@ def make_child_factory(
|
|||||||
is_root=False,
|
is_root=False,
|
||||||
scan_mode=scan_mode,
|
scan_mode=scan_mode,
|
||||||
is_whitebox=is_whitebox,
|
is_whitebox=is_whitebox,
|
||||||
is_diff_scoped=is_diff_scoped,
|
|
||||||
interactive=interactive,
|
interactive=interactive,
|
||||||
chat_completions_tools=chat_completions_tools,
|
chat_completions_tools=chat_completions_tools,
|
||||||
strict_tool_schemas=strict_tool_schemas,
|
strict_tool_schemas=strict_tool_schemas,
|
||||||
|
|||||||
+3
-19
@@ -23,44 +23,30 @@ def _resolve_skills(
|
|||||||
scan_mode: str = "deep",
|
scan_mode: str = "deep",
|
||||||
is_whitebox: bool = False,
|
is_whitebox: bool = False,
|
||||||
is_root: bool = False,
|
is_root: bool = False,
|
||||||
is_diff_scoped: bool = False,
|
|
||||||
) -> list[str]:
|
) -> list[str]:
|
||||||
"""Build the deduped, ordered skills list for the prompt render.
|
"""Build the deduped, ordered skills list for the prompt render.
|
||||||
|
|
||||||
Order:
|
Order:
|
||||||
|
|
||||||
1. Whatever the caller asked for, in order.
|
1. Whatever the caller asked for, in order.
|
||||||
2. ``scan_modes/<mode>`` (always), plus ``scan_modes/diff`` when the
|
2. ``scan_modes/<mode>`` (always).
|
||||||
run is scoped to a change set — diff scope overlays the depth
|
|
||||||
mode rather than replacing it.
|
|
||||||
3. ``tooling/agent_browser`` (always — every agent has shell + the
|
3. ``tooling/agent_browser`` (always — every agent has shell + the
|
||||||
agent-browser CLI).
|
agent-browser CLI).
|
||||||
4. ``tooling/python`` (always — Python runs through ``exec_command``;
|
4. ``tooling/python`` (always — Python runs through ``exec_command``;
|
||||||
sandbox scripts can import ``caido_api`` for Caido automation).
|
sandbox scripts can import ``caido_api`` for Caido automation).
|
||||||
5. ``analysis/counterevidence`` and ``analysis/severity_calibration``
|
5. ``coordination/root_agent`` for the root agent only — orchestration
|
||||||
(always — closure discipline and severity rubric apply to every
|
|
||||||
agent that can open or close a candidate, or file a report).
|
|
||||||
6. ``coordination/root_agent`` for the root agent only — orchestration
|
|
||||||
guidance for delegating to specialist subagents.
|
guidance for delegating to specialist subagents.
|
||||||
7. Whitebox-specific skills if applicable, including
|
6. Whitebox-specific skills if applicable.
|
||||||
``analysis/fix_verification`` (only whitebox agents can attach an
|
|
||||||
applyable ``fix_after``) and ``analysis/source_aware_discovery``.
|
|
||||||
"""
|
"""
|
||||||
ordered: list[str] = list(requested or [])
|
ordered: list[str] = list(requested or [])
|
||||||
ordered.append(f"scan_modes/{scan_mode}")
|
ordered.append(f"scan_modes/{scan_mode}")
|
||||||
if is_diff_scoped:
|
|
||||||
ordered.append("scan_modes/diff")
|
|
||||||
ordered.append("tooling/agent_browser")
|
ordered.append("tooling/agent_browser")
|
||||||
ordered.append("tooling/python")
|
ordered.append("tooling/python")
|
||||||
ordered.append("analysis/counterevidence")
|
|
||||||
ordered.append("analysis/severity_calibration")
|
|
||||||
if is_root:
|
if is_root:
|
||||||
ordered.append("coordination/root_agent")
|
ordered.append("coordination/root_agent")
|
||||||
if is_whitebox:
|
if is_whitebox:
|
||||||
ordered.append("coordination/source_aware_whitebox")
|
ordered.append("coordination/source_aware_whitebox")
|
||||||
ordered.append("custom/source_aware_sast")
|
ordered.append("custom/source_aware_sast")
|
||||||
ordered.append("analysis/source_aware_discovery")
|
|
||||||
ordered.append("analysis/fix_verification")
|
|
||||||
|
|
||||||
deduped: list[str] = []
|
deduped: list[str] = []
|
||||||
seen: set[str] = set()
|
seen: set[str] = set()
|
||||||
@@ -77,7 +63,6 @@ def render_system_prompt(
|
|||||||
scan_mode: str = "deep",
|
scan_mode: str = "deep",
|
||||||
is_whitebox: bool = False,
|
is_whitebox: bool = False,
|
||||||
is_root: bool = False,
|
is_root: bool = False,
|
||||||
is_diff_scoped: bool = False,
|
|
||||||
interactive: bool = False,
|
interactive: bool = False,
|
||||||
system_prompt_context: dict[str, Any] | None = None,
|
system_prompt_context: dict[str, Any] | None = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
@@ -98,7 +83,6 @@ def render_system_prompt(
|
|||||||
scan_mode=scan_mode,
|
scan_mode=scan_mode,
|
||||||
is_whitebox=is_whitebox,
|
is_whitebox=is_whitebox,
|
||||||
is_root=is_root,
|
is_root=is_root,
|
||||||
is_diff_scoped=is_diff_scoped,
|
|
||||||
)
|
)
|
||||||
skill_content = load_skills(skills_to_load)
|
skill_content = load_skills(skills_to_load)
|
||||||
env.globals["get_skill"] = lambda name: skill_content.get(name, "")
|
env.globals["get_skill"] = lambda name: skill_content.get(name, "")
|
||||||
|
|||||||
@@ -216,31 +216,10 @@ VALIDATION REQUIREMENTS:
|
|||||||
- Independent verification through subagent
|
- Independent verification through subagent
|
||||||
- Document complete attack chain
|
- Document complete attack chain
|
||||||
- Keep going until you find something that matters
|
- Keep going until you find something that matters
|
||||||
- CLOSURE DISCIPLINE: every candidate you open ends in exactly one explicit state — `confirmed` (working PoC, or a complete source→control→sink→impact trace that is reachable), `ruled_out` (you can name the SPECIFIC control, at a location, that runs on every attacker-reachable path before the sink), or `open_proof_gap` (plausible, unconfirmed, and you could NOT name such a control). "I moved on" is not a closure state. Silently dropping an uncertain candidate is mislabelling an `open_proof_gap` as `ruled_out` and is how real bugs get missed.
|
|
||||||
- Missing information is NOT proof of safety: no caller found, can't tell if deployed/exposed, couldn't stand up the service, build failed — each is an `open_proof_gap`, never a reason to mark a candidate clean. Difficulty is a reason to defer, not to suppress.
|
|
||||||
- COVERAGE: record every surface you assess with `record_coverage` (surface + risk area + outcome + evidence), including the ones that came back clean — a report that only lists findings cannot say what was reviewed and cleared. Use the `needs_follow_up` outcome for anything left in an `open_proof_gap` state, and carry the same items up in `agent_finish(open_items=[...])`. The ledger is shared and mutable: when you resolve a surface another agent left open — or find that a closed one is not — move that entry with `update_coverage` instead of recording a second one for the same surface. The root agent reconciles all of it via `list_coverage` before `finish_scan`.
|
|
||||||
- THREAT MODEL: before you start testing, call `get_threat_model` on the target you were pointed at — it is the scan's shared answer to who the attacker is, where the trust boundaries sit, and what counts as critical here, and it is cached per target rather than per scan. Read it instead of re-deriving trust boundaries yourself; where your testing disproves it — a boundary it calls trusted turns out to be attacker-reachable, a role it did not know about, a host or endpoint it never listed — record that with `amend_threat_model` so the agents after you inherit the correction. Amending is not optional politeness: a model nobody corrects turns the first agent's guesses into everyone's assumptions.
|
|
||||||
- Before filing any report, run the counterevidence pass: argue the strongest case AGAINST the finding, record what you found in the `counterevidence` field, set `confidence` honestly (a static-only trace you couldn't execute is at best `medium`), and state what evidence would change the severity. See the counterevidence and severity-calibration knowledge above.
|
|
||||||
- A vulnerability is ONLY considered reported when a reporting agent uses create_vulnerability_report (or create_dependency_report for known-CVE dependency/supply-chain findings) with full details. Mentions in agent_finish, finish_scan, or generic messages are NOT sufficient
|
- A vulnerability is ONLY considered reported when a reporting agent uses create_vulnerability_report (or create_dependency_report for known-CVE dependency/supply-chain findings) with full details. Mentions in agent_finish, finish_scan, or generic messages are NOT sufficient
|
||||||
- Reporting and fixing are ONE step, not two: when source is available, the reporting agent derives the concrete fix and files it INLINE via create_vulnerability_report (`code_locations` with `fix_before`/`fix_after` + `fix_pr_body`) — the report is not complete without it. Do NOT report first and then spawn a separate downstream agent to re-derive and re-apply the same patch; that just re-does the analysis and wastes tokens. (Do not silently patch a finding WITHOUT filing a report — the report, with its embedded fix, is the deliverable.)
|
- Reporting and fixing are ONE step, not two: when source is available, the reporting agent derives the concrete fix and files it INLINE via create_vulnerability_report (`code_locations` with `fix_before`/`fix_after` + `fix_pr_body`) — the report is not complete without it. Do NOT report first and then spawn a separate downstream agent to re-derive and re-apply the same patch; that just re-does the analysis and wastes tokens. (Do not silently patch a finding WITHOUT filing a report — the report, with its embedded fix, is the deliverable.)
|
||||||
- DEDUPLICATION: The create_vulnerability_report tool uses LLM-based deduplication. If it rejects your report as a duplicate, DO NOT attempt to re-submit the same vulnerability. Accept the rejection and move on to testing other areas. The vulnerability has already been reported by another agent
|
- DEDUPLICATION: The create_vulnerability_report tool uses LLM-based deduplication. If it rejects your report as a duplicate, DO NOT attempt to re-submit the same vulnerability. Accept the rejection and move on to testing other areas. The vulnerability has already been reported by another agent
|
||||||
- REVIEWING FILED FINDINGS (orchestrator/root agent): use list_reports to see every vulnerability filed so far in this scan (by any agent, root or child) — metadata-first with per-severity counts — and get_report to read one finding in full by its id. These are read-only orchestration tools: the root agent uses them to track coverage, avoid dispatching work on already-covered ground, assemble the finish_scan executive summary, and reason about attack-chaining across confirmed findings. Leaf/specialist agents should NOT call them — just do your assigned testing and file findings. Each entry shows which agent filed it (agent_name), and your own entries are flagged by_you. list_notes/get_note do the same for notes.
|
- REVIEWING FILED FINDINGS (orchestrator/root agent): use list_reports to see every vulnerability filed so far in this scan (by any agent, root or child) — metadata-first with per-severity counts — and get_report to read one finding in full by its id. These are read-only orchestration tools: the root agent uses them to track coverage, avoid dispatching work on already-covered ground, assemble the finish_scan executive summary, and reason about attack-chaining across confirmed findings. Leaf/specialist agents should NOT call them — just do your assigned testing and file findings. Each entry shows which agent filed it (agent_name), and your own entries are flagged by_you. list_notes/get_note do the same for notes.
|
||||||
|
|
||||||
STATE & COORDINATION TOOLS (when and how):
|
|
||||||
Every one of these tools writes to state the rest of the scan reads. Reaching for the tool is not optional bookkeeping — the agent after you sees your state, not your reasoning, so state you never wrote is context the scan permanently loses.
|
|
||||||
- PLAN — `think`: use before any non-trivial or multi-step move to reason through approach, uncertainty, or what to do next. NOT for acknowledgements, summaries, or as filler before a final answer.
|
|
||||||
- SKILLS — `load_skill`: the skills matching your task are already inlined below under `<specialized_knowledge>`; `<available_skills>` lists the rest by name. When you are about to test a vuln class, protocol, tool, or framework whose skill is not already inlined, `load_skill` it FIRST and follow it, rather than guessing payloads or tool syntax from memory.
|
|
||||||
- TODOS — `create_todo` / `list_todos` / `update_todo` / `mark_todo_done` / `mark_todo_pending` / `delete_todo`: your own working checklist for a multi-step task. Create todos when your task has several distinct steps so nothing is dropped across a long run; mark them done as you finish. This is private working memory — use `notes` for anything another agent needs.
|
|
||||||
- NOTES — `create_note` / `list_notes` / `get_note` / `update_note` / `delete_note`: the scan's shared scratchpad, visible to every agent. Write a note for a durable cross-agent fact that is not a finding and not coverage — a working credential set, a discovered endpoint inventory, an enumerated tenant list, a rate-limit quirk the next agent needs. `update_note` to keep a living inventory current; `delete_note` only for something now wrong or superseded. Check `list_notes`/`get_note` before recon work so you build on what is already mapped instead of redoing it.
|
|
||||||
- THREAT MODEL — `get_threat_model` / `amend_threat_model` / `save_threat_model`: covered above. `save_threat_model` REPLACES the whole document and clears amendments, so it is for establishing the baseline or folding amendments in (normally root) — to correct part of an existing model, `amend_threat_model` instead.
|
|
||||||
- COVERAGE — `record_coverage` / `update_coverage` / `list_coverage`: covered above. One row per surface+risk; correct an existing row with `update_coverage`, never a second `record_coverage`.
|
|
||||||
- RESEARCH — `web_search`: pull fresh, target-specific external knowledge — latest bypasses, WAF evasions, DB-/framework-specific syntax, CVE and advisory detail — before falling back to memorized payloads, and refresh payload corpora mid-spray.
|
|
||||||
- SPAWN WORK — `create_agent`: delegate a focused subtask to a specialist child (see the multi-agent rules below for when to spawn and how to scope it). Give it the target to model against and what is already known.
|
|
||||||
- TRACK CHILDREN — `view_agent_graph`: your live map of every agent and its status. Call it before spawning (to confirm no existing agent already covers the scope) and before finishing (to confirm no child is still running).
|
|
||||||
- STEER CHILDREN — `send_message_to_agent`: send a running child new information, a course correction, or a request to wrap up, without killing it. Use it to answer a child's question or narrow its scope mid-run.
|
|
||||||
- BLOCK ON CHILDREN — `wait_for_agents`: block until named children report back when your next move genuinely depends on their results. If you can keep making progress in parallel, keep working instead of waiting.
|
|
||||||
- CANCEL CHILDREN — `stop_agent`: gracefully cancel a child whose work is redundant, misdirected, or no longer needed. Prefer `send_message_to_agent` to redirect a child that is merely off-track; reserve `stop_agent` for work that should not continue at all.
|
|
||||||
- FINISH — subagents call `agent_finish` (with `open_items=[...]` for anything left unresolved); the root agent calls `finish_scan` exactly once, only after every child is wrapped up and coverage is reconciled. `agent_finish`/`finish_scan` are handoffs, not reporting channels — a vulnerability is reported only via `create_vulnerability_report`/`create_dependency_report`.
|
|
||||||
</execution_guidelines>
|
</execution_guidelines>
|
||||||
|
|
||||||
<vulnerability_focus>
|
<vulnerability_focus>
|
||||||
|
|||||||
+13
-27
@@ -72,32 +72,8 @@ def _write_store(data: dict[str, Any]) -> None:
|
|||||||
write_secret_text(AUTH_PATH, json.dumps(data, indent=2))
|
write_secret_text(AUTH_PATH, json.dumps(data, indent=2))
|
||||||
|
|
||||||
|
|
||||||
def read_provider_record(provider: str) -> dict[str, Any] | None:
|
|
||||||
"""Raw record for *provider* from the shared subscription-auth store."""
|
|
||||||
record = _read_store().get(provider)
|
|
||||||
return record if isinstance(record, dict) else None
|
|
||||||
|
|
||||||
|
|
||||||
def save_provider_record(provider: str, record: dict[str, Any]) -> None:
|
|
||||||
data = _read_store()
|
|
||||||
data[provider] = record
|
|
||||||
_write_store(data)
|
|
||||||
|
|
||||||
|
|
||||||
def remove_provider_record(provider: str) -> None:
|
|
||||||
data = _read_store()
|
|
||||||
if provider not in data:
|
|
||||||
return
|
|
||||||
del data[provider]
|
|
||||||
if data:
|
|
||||||
_write_store(data)
|
|
||||||
return
|
|
||||||
with contextlib.suppress(OSError):
|
|
||||||
AUTH_PATH.unlink()
|
|
||||||
|
|
||||||
|
|
||||||
def read_record() -> dict[str, Any] | None:
|
def read_record() -> dict[str, Any] | None:
|
||||||
record = read_provider_record(PROVIDER)
|
record = _read_store().get(PROVIDER)
|
||||||
if not isinstance(record, dict) or record.get("type") != "oauth":
|
if not isinstance(record, dict) or record.get("type") != "oauth":
|
||||||
return None
|
return None
|
||||||
if not (record.get("access") and record.get("refresh") and record.get("account_id")):
|
if not (record.get("access") and record.get("refresh") and record.get("account_id")):
|
||||||
@@ -110,11 +86,21 @@ def is_authenticated() -> bool:
|
|||||||
|
|
||||||
|
|
||||||
def save_record(record: dict[str, Any]) -> None:
|
def save_record(record: dict[str, Any]) -> None:
|
||||||
save_provider_record(PROVIDER, record)
|
data = _read_store()
|
||||||
|
data[PROVIDER] = record
|
||||||
|
_write_store(data)
|
||||||
|
|
||||||
|
|
||||||
def logout() -> None:
|
def logout() -> None:
|
||||||
remove_provider_record(PROVIDER)
|
data = _read_store()
|
||||||
|
if PROVIDER not in data:
|
||||||
|
return
|
||||||
|
del data[PROVIDER]
|
||||||
|
if data:
|
||||||
|
_write_store(data)
|
||||||
|
return
|
||||||
|
with contextlib.suppress(OSError):
|
||||||
|
AUTH_PATH.unlink()
|
||||||
|
|
||||||
|
|
||||||
@contextlib.contextmanager
|
@contextlib.contextmanager
|
||||||
|
|||||||
+9
-66
@@ -20,7 +20,6 @@ from agents.model_settings import ModelSettings
|
|||||||
from agents.models.fake_id import FAKE_RESPONSES_ID
|
from agents.models.fake_id import FAKE_RESPONSES_ID
|
||||||
from agents.models.interface import Model
|
from agents.models.interface import Model
|
||||||
from agents.models.multi_provider import MultiProvider
|
from agents.models.multi_provider import MultiProvider
|
||||||
from agents.models.openai_chatcompletions import OpenAIChatCompletionsModel
|
|
||||||
from agents.models.openai_responses import OpenAIResponsesModel
|
from agents.models.openai_responses import OpenAIResponsesModel
|
||||||
from agents.retry import (
|
from agents.retry import (
|
||||||
ModelRetryBackoffSettings,
|
ModelRetryBackoffSettings,
|
||||||
@@ -37,7 +36,7 @@ from openai.types.responses import (
|
|||||||
from openai.types.responses.response_usage import ResponseUsage
|
from openai.types.responses.response_usage import ResponseUsage
|
||||||
from openai.types.shared import Reasoning
|
from openai.types.shared import Reasoning
|
||||||
|
|
||||||
from strix.config import codex, opencode
|
from strix.config import codex
|
||||||
from strix.config.loader import load_settings
|
from strix.config.loader import load_settings
|
||||||
from strix.config.tool_call_ids import TurnCallIdRewriter, dedupe_input
|
from strix.config.tool_call_ids import TurnCallIdRewriter, dedupe_input
|
||||||
from strix.config.tool_call_limits import TurnToolCallLimiter
|
from strix.config.tool_call_limits import TurnToolCallLimiter
|
||||||
@@ -80,12 +79,7 @@ def _retry_statusless_provider_errors(context: RetryPolicyContext) -> bool:
|
|||||||
|
|
||||||
|
|
||||||
class _CodexResponsesModel(OpenAIResponsesModel):
|
class _CodexResponsesModel(OpenAIResponsesModel):
|
||||||
"""Responses model for stateless subscription gateways (always streamed).
|
"""Responses model for the ChatGPT subscription backend (always streamed, stateless)."""
|
||||||
|
|
||||||
Used for the ChatGPT subscription backend and for Responses-served models on
|
|
||||||
the OpenCode gateway: neither stores responses server-side, so reasoning is
|
|
||||||
carried inline via ``reasoning.encrypted_content``.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(
|
def __init__(
|
||||||
self,
|
self,
|
||||||
@@ -477,7 +471,6 @@ class StrixProvider(MultiProvider):
|
|||||||
def get_model(self, model_name: str | None) -> Model:
|
def get_model(self, model_name: str | None) -> Model:
|
||||||
llm = load_settings().llm
|
llm = load_settings().llm
|
||||||
slug = codex.subscription_model(model_name)
|
slug = codex.subscription_model(model_name)
|
||||||
oc = opencode.subscription_model(model_name)
|
|
||||||
idle_timeout = float(llm.stream_idle_timeout)
|
idle_timeout = float(llm.stream_idle_timeout)
|
||||||
if slug:
|
if slug:
|
||||||
# The ChatGPT subscription backend is always streamed; it has no
|
# The ChatGPT subscription backend is always streamed; it has no
|
||||||
@@ -488,35 +481,6 @@ class StrixProvider(MultiProvider):
|
|||||||
codex.get_subscription_client(),
|
codex.get_subscription_client(),
|
||||||
reasoning_effort=llm.reasoning_effort,
|
reasoning_effort=llm.reasoning_effort,
|
||||||
)
|
)
|
||||||
elif oc and oc.protocol == opencode.PROTOCOL_RESPONSES:
|
|
||||||
model = _CodexResponsesModel(
|
|
||||||
oc.slug,
|
|
||||||
opencode.get_subscription_client(oc.base_url),
|
|
||||||
reasoning_effort=llm.reasoning_effort,
|
|
||||||
)
|
|
||||||
elif oc and oc.protocol == opencode.PROTOCOL_MESSAGES:
|
|
||||||
# Claude models are served on Anthropic's ``/messages``, which the
|
|
||||||
# OpenAI SDK cannot speak: it has no Messages method and sends the
|
|
||||||
# key as a bearer token rather than ``x-api-key``. LiteLLM's
|
|
||||||
# Anthropic route handles both, so the gateway becomes an Anthropic
|
|
||||||
# base URL with the subscription key.
|
|
||||||
from agents.extensions.models.litellm_model import LitellmModel
|
|
||||||
|
|
||||||
model = LitellmModel(
|
|
||||||
model=f"anthropic/{oc.slug}",
|
|
||||||
base_url=oc.messages_url,
|
|
||||||
api_key=opencode.get_api_key(),
|
|
||||||
)
|
|
||||||
if llm.disable_streaming:
|
|
||||||
model = _NonStreamingModel(model)
|
|
||||||
idle_timeout = 0.0
|
|
||||||
elif oc:
|
|
||||||
model = OpenAIChatCompletionsModel(
|
|
||||||
oc.slug, opencode.get_subscription_client(oc.base_url)
|
|
||||||
)
|
|
||||||
if llm.disable_streaming:
|
|
||||||
model = _NonStreamingModel(model)
|
|
||||||
idle_timeout = 0.0
|
|
||||||
else:
|
else:
|
||||||
model = super().get_model(model_name)
|
model = super().get_model(model_name)
|
||||||
if llm.disable_streaming:
|
if llm.disable_streaming:
|
||||||
@@ -576,24 +540,15 @@ RECOMMENDED_MODEL_NAMES = (
|
|||||||
_RECOMMENDED_MODEL_NAME_SET = frozenset(name.lower() for name in RECOMMENDED_MODEL_NAMES)
|
_RECOMMENDED_MODEL_NAME_SET = frozenset(name.lower() for name in RECOMMENDED_MODEL_NAMES)
|
||||||
|
|
||||||
FRONTIER_MODEL_FAMILIES = (
|
FRONTIER_MODEL_FAMILIES = (
|
||||||
(("azure", "azure_ai", "bedrock_mantle", "chatgpt", "openai", "opencode"), ("gpt-5",)),
|
(("azure", "azure_ai", "bedrock_mantle", "chatgpt", "openai"), ("gpt-5",)),
|
||||||
(
|
(
|
||||||
(
|
("anthropic", "azure_ai", "bedrock", "claude", "databricks", "snowflake", "vertex_ai"),
|
||||||
"anthropic",
|
|
||||||
"azure_ai",
|
|
||||||
"bedrock",
|
|
||||||
"claude",
|
|
||||||
"databricks",
|
|
||||||
"opencode",
|
|
||||||
"snowflake",
|
|
||||||
"vertex_ai",
|
|
||||||
),
|
|
||||||
("claude-fable-5", "claude-opus-5", "claude-opus-4", "claude-sonnet-5", "claude-sonnet-4"),
|
("claude-fable-5", "claude-opus-5", "claude-opus-4", "claude-sonnet-5", "claude-sonnet-4"),
|
||||||
),
|
),
|
||||||
(("google", "gemini", "opencode", "vertex_ai"), ("gemini-3",)),
|
(("google", "gemini", "vertex_ai"), ("gemini-3",)),
|
||||||
(("deepseek", "opencode"), ("deepseek-v4", "deepseek-r1", "deepseek-reasoner")),
|
(("deepseek",), ("deepseek-v4", "deepseek-r1", "deepseek-reasoner")),
|
||||||
(("alibaba", "dashscope", "opencode", "qwen"), ("qwen3.8", "qwen3.7", "qwen3-max")),
|
(("alibaba", "dashscope", "qwen"), ("qwen3.8", "qwen3.7", "qwen3-max")),
|
||||||
(("kimi", "moonshot", "moonshotai", "opencode"), ("kimi-k3", "kimi-k2.7", "kimi-k2.6")),
|
(("moonshot", "moonshotai", "kimi"), ("kimi-k3", "kimi-k2.7", "kimi-k2.6")),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -601,14 +556,7 @@ def configure_sdk_model_defaults(settings: Settings) -> None:
|
|||||||
"""Apply Strix config to SDK-native defaults."""
|
"""Apply Strix config to SDK-native defaults."""
|
||||||
llm = settings.llm
|
llm = settings.llm
|
||||||
set_tracing_disabled(True)
|
set_tracing_disabled(True)
|
||||||
oc = opencode.subscription_model(llm.model)
|
if codex.subscription_model(llm.model):
|
||||||
if codex.subscription_model(llm.model) or oc:
|
|
||||||
# A subscription run carries its own client and credentials, so none of
|
|
||||||
# the api_key/api_base defaults below apply. The Anthropic route is the
|
|
||||||
# exception: it goes through LiteLLM, which still needs the
|
|
||||||
# compatibility flags and the cost callback.
|
|
||||||
if oc is not None and oc.protocol == opencode.PROTOCOL_MESSAGES:
|
|
||||||
_configure_litellm_compatibility()
|
|
||||||
return
|
return
|
||||||
_configure_litellm_compatibility()
|
_configure_litellm_compatibility()
|
||||||
_configure_openrouter_attribution(llm.model)
|
_configure_openrouter_attribution(llm.model)
|
||||||
@@ -793,11 +741,6 @@ def uses_chat_completions_tool_schema(model_name: str, settings: Settings) -> bo
|
|||||||
"""Return whether the resolved SDK route can only receive JSON function tools."""
|
"""Return whether the resolved SDK route can only receive JSON function tools."""
|
||||||
if codex.subscription_model(model_name):
|
if codex.subscription_model(model_name):
|
||||||
return False
|
return False
|
||||||
oc = opencode.subscription_model(model_name)
|
|
||||||
if oc:
|
|
||||||
# Chat Completions takes JSON function tools; so does the LiteLLM
|
|
||||||
# Anthropic route, which translates them to Anthropic tool blocks.
|
|
||||||
return oc.protocol != opencode.PROTOCOL_RESPONSES
|
|
||||||
model = model_name.strip().lower()
|
model = model_name.strip().lower()
|
||||||
if "/" in model and not model.startswith("openai/"):
|
if "/" in model and not model.startswith("openai/"):
|
||||||
return True
|
return True
|
||||||
|
|||||||
@@ -1,225 +0,0 @@
|
|||||||
"""OpenCode subscription auth: API-key sign-in and the clients that route
|
|
||||||
inference through the OpenCode gateway.
|
|
||||||
|
|
||||||
Covers both OpenCode offerings, Zen (pay-as-you-go credits) and Go (the
|
|
||||||
monthly subscription), which share one account and API key but live behind
|
|
||||||
different gateway base URLs. Unlike the ChatGPT subscription there is no
|
|
||||||
OAuth: the user copies a plain API key from https://opencode.ai/auth, and
|
|
||||||
using the gateway from other agents is officially supported.
|
|
||||||
|
|
||||||
The gateway speaks three protocols and serves each model family on exactly
|
|
||||||
one of them (see https://opencode.ai/docs/zen/), answering a request sent to
|
|
||||||
the wrong one with an unhandled 500 rather than a 404. ``_protocol()`` holds
|
|
||||||
the mapping; ``SubscriptionModel.protocol`` carries the result. Claude runs on
|
|
||||||
Anthropic's ``/messages``, which the OpenAI SDK cannot speak, so that route
|
|
||||||
goes through LiteLLM instead of the clients built here.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import httpx
|
|
||||||
import requests
|
|
||||||
from openai import AsyncOpenAI
|
|
||||||
|
|
||||||
from strix.config import codex
|
|
||||||
|
|
||||||
|
|
||||||
PROVIDER = "opencode"
|
|
||||||
|
|
||||||
ZEN_BASE_URL = "https://opencode.ai/zen/v1"
|
|
||||||
GO_BASE_URL = "https://opencode.ai/zen/go/v1"
|
|
||||||
|
|
||||||
# ``opencode/<model>`` runs on Zen credits; ``opencode-go/<model>`` on the Go
|
|
||||||
# subscription (matching OpenCode's own ``opencode-go/`` model ids).
|
|
||||||
ZEN_PREFIX = "opencode/"
|
|
||||||
GO_PREFIX = "opencode-go/"
|
|
||||||
|
|
||||||
AUTH_CONSOLE_URL = "https://opencode.ai/auth"
|
|
||||||
|
|
||||||
_KEY_CHECK_TIMEOUT = 30
|
|
||||||
|
|
||||||
|
|
||||||
class OpencodeAuthError(Exception):
|
|
||||||
def __init__(self, code: str, message: str | None = None) -> None:
|
|
||||||
self.code = code
|
|
||||||
super().__init__(message or code)
|
|
||||||
|
|
||||||
|
|
||||||
PROTOCOL_CHAT = "chat"
|
|
||||||
PROTOCOL_RESPONSES = "responses"
|
|
||||||
PROTOCOL_MESSAGES = "messages"
|
|
||||||
|
|
||||||
PLAN_ZEN = "zen"
|
|
||||||
PLAN_GO = "go"
|
|
||||||
|
|
||||||
_PLAN_LABELS = {PLAN_ZEN: "OpenCode Zen", PLAN_GO: "OpenCode Go"}
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class SubscriptionModel:
|
|
||||||
slug: str
|
|
||||||
base_url: str
|
|
||||||
protocol: str
|
|
||||||
plan: str
|
|
||||||
|
|
||||||
@property
|
|
||||||
def uses_responses(self) -> bool:
|
|
||||||
return self.protocol == PROTOCOL_RESPONSES
|
|
||||||
|
|
||||||
@property
|
|
||||||
def messages_url(self) -> str:
|
|
||||||
"""Anthropic-protocol endpoint for this gateway, e.g. ``.../zen/v1/messages``."""
|
|
||||||
return f"{self.base_url}/messages"
|
|
||||||
|
|
||||||
@property
|
|
||||||
def label(self) -> str:
|
|
||||||
return _PLAN_LABELS[self.plan]
|
|
||||||
|
|
||||||
@property
|
|
||||||
def metered(self) -> bool:
|
|
||||||
"""Whether a run spends money per request.
|
|
||||||
|
|
||||||
Zen bills prepaid credits per request, so its runs cost real money and
|
|
||||||
must not be reported as free. Go is a flat monthly fee, where a run's
|
|
||||||
marginal cost genuinely is zero.
|
|
||||||
"""
|
|
||||||
return self.plan == PLAN_ZEN
|
|
||||||
|
|
||||||
|
|
||||||
def _protocol(slug: str, base_url: str) -> str:
|
|
||||||
"""Which wire protocol the gateway serves *slug* on.
|
|
||||||
|
|
||||||
The gateway routes by model family and answers a request sent to the wrong
|
|
||||||
protocol with an unhandled 500 rather than a 404, so the mapping has to be
|
|
||||||
right. Probed against both gateways per family:
|
|
||||||
|
|
||||||
* Claude on Anthropic's ``/messages``
|
|
||||||
* GPT, Grok (Zen) and Muse on OpenAI's ``/responses``
|
|
||||||
* DeepSeek, MiniMax, Kimi, GLM and Qwen on Chat Completions
|
|
||||||
|
|
||||||
Grok is absent from the Go catalog, so its Zen-only Responses route costs
|
|
||||||
nothing there. Kimi and Qwen also answer on ``/messages``, but Chat
|
|
||||||
Completions works for them on both plans and stays the single mapping.
|
|
||||||
"""
|
|
||||||
lowered = slug.lower()
|
|
||||||
if lowered.startswith("claude-"):
|
|
||||||
return PROTOCOL_MESSAGES
|
|
||||||
if lowered.startswith(("gpt-", "muse-")):
|
|
||||||
return PROTOCOL_RESPONSES
|
|
||||||
if lowered.startswith("grok") and base_url == ZEN_BASE_URL:
|
|
||||||
return PROTOCOL_RESPONSES
|
|
||||||
return PROTOCOL_CHAT
|
|
||||||
|
|
||||||
|
|
||||||
def subscription_model(model_name: str | None) -> SubscriptionModel | None:
|
|
||||||
"""The gateway model behind an ``opencode/`` or ``opencode-go/`` STRIX_LLM."""
|
|
||||||
name = (model_name or "").strip()
|
|
||||||
lowered = name.lower()
|
|
||||||
for prefix, base_url, plan in (
|
|
||||||
(GO_PREFIX, GO_BASE_URL, PLAN_GO),
|
|
||||||
(ZEN_PREFIX, ZEN_BASE_URL, PLAN_ZEN),
|
|
||||||
):
|
|
||||||
if lowered.startswith(prefix):
|
|
||||||
slug = name[len(prefix) :]
|
|
||||||
if not slug:
|
|
||||||
return None
|
|
||||||
return SubscriptionModel(slug, base_url, _protocol(slug, base_url), plan)
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def read_record() -> dict[str, Any] | None:
|
|
||||||
record = codex.read_provider_record(PROVIDER)
|
|
||||||
if not isinstance(record, dict) or record.get("type") != "api_key":
|
|
||||||
return None
|
|
||||||
key = record.get("key")
|
|
||||||
if not isinstance(key, str) or not key:
|
|
||||||
return None
|
|
||||||
return record
|
|
||||||
|
|
||||||
|
|
||||||
def is_authenticated() -> bool:
|
|
||||||
return read_record() is not None
|
|
||||||
|
|
||||||
|
|
||||||
def save_api_key(key: str) -> None:
|
|
||||||
codex.save_provider_record(PROVIDER, {"type": "api_key", "provider": PROVIDER, "key": key})
|
|
||||||
|
|
||||||
|
|
||||||
def logout() -> None:
|
|
||||||
codex.remove_provider_record(PROVIDER)
|
|
||||||
|
|
||||||
|
|
||||||
def get_api_key() -> str:
|
|
||||||
record = read_record()
|
|
||||||
if record is None:
|
|
||||||
raise OpencodeAuthError(
|
|
||||||
"not_authenticated", "not signed in; run: strix auth login opencode"
|
|
||||||
)
|
|
||||||
return str(record["key"])
|
|
||||||
|
|
||||||
|
|
||||||
def validate_api_key(key: str) -> None:
|
|
||||||
"""Check the key against the gateway's models endpoint; raise if rejected."""
|
|
||||||
try:
|
|
||||||
response = requests.get(
|
|
||||||
f"{ZEN_BASE_URL}/models",
|
|
||||||
headers={"Authorization": f"Bearer {key}"},
|
|
||||||
timeout=_KEY_CHECK_TIMEOUT,
|
|
||||||
)
|
|
||||||
except requests.RequestException as exc:
|
|
||||||
raise OpencodeAuthError("unavailable", str(exc)) from exc
|
|
||||||
if response.status_code in (401, 403):
|
|
||||||
raise OpencodeAuthError(
|
|
||||||
"invalid_key", f"OpenCode rejected the API key (HTTP {response.status_code})"
|
|
||||||
)
|
|
||||||
if response.status_code >= 400:
|
|
||||||
raise OpencodeAuthError("http_error", f"HTTP {response.status_code}: {response.text[:300]}")
|
|
||||||
|
|
||||||
|
|
||||||
def build_openai_client(base_url: str) -> AsyncOpenAI:
|
|
||||||
return AsyncOpenAI(
|
|
||||||
api_key=get_api_key(),
|
|
||||||
base_url=base_url,
|
|
||||||
http_client=httpx.AsyncClient(timeout=httpx.Timeout(600.0, connect=30.0)),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
_subscription_clients: dict[str, AsyncOpenAI] = {}
|
|
||||||
|
|
||||||
|
|
||||||
def get_subscription_client(base_url: str) -> AsyncOpenAI:
|
|
||||||
client = _subscription_clients.get(base_url)
|
|
||||||
if client is None:
|
|
||||||
client = build_openai_client(base_url)
|
|
||||||
_subscription_clients[base_url] = client
|
|
||||||
return client
|
|
||||||
|
|
||||||
|
|
||||||
def auth_mode(model_name: str | None) -> str:
|
|
||||||
"""Return "subscription" when STRIX_LLM runs on any subscription
|
|
||||||
(OpenCode or ChatGPT), else "api_key"."""
|
|
||||||
if subscription_model(model_name) or codex.subscription_model(model_name):
|
|
||||||
return "subscription"
|
|
||||||
return "api_key"
|
|
||||||
|
|
||||||
|
|
||||||
def subscription_plan(model_name: str | None) -> str | None:
|
|
||||||
"""Which OpenCode plan STRIX_LLM runs on: "zen", "go", or None.
|
|
||||||
|
|
||||||
Recorded alongside ``subscription_provider`` rather than folded into it, so
|
|
||||||
consumers that compare the provider against "opencode" keep working.
|
|
||||||
"""
|
|
||||||
oc = subscription_model(model_name)
|
|
||||||
return oc.plan if oc else None
|
|
||||||
|
|
||||||
|
|
||||||
def subscription_provider(model_name: str | None) -> str | None:
|
|
||||||
"""The subscription behind STRIX_LLM: "opencode", "chatgpt", or None."""
|
|
||||||
if subscription_model(model_name):
|
|
||||||
return PROVIDER
|
|
||||||
if codex.subscription_model(model_name):
|
|
||||||
return "chatgpt"
|
|
||||||
return None
|
|
||||||
+3
-17
@@ -7,12 +7,13 @@ import contextlib
|
|||||||
import logging
|
import logging
|
||||||
import uuid
|
import uuid
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from functools import cache
|
|
||||||
from typing import TYPE_CHECKING, Any, cast
|
from typing import TYPE_CHECKING, Any, cast
|
||||||
|
|
||||||
|
import litellm
|
||||||
from agents import RunConfig, Runner
|
from agents import RunConfig, Runner
|
||||||
from agents.exceptions import AgentsException, MaxTurnsExceeded, UserError
|
from agents.exceptions import AgentsException, MaxTurnsExceeded, UserError
|
||||||
from agents.sandbox.errors import ExecTransportError
|
from agents.sandbox.errors import ExecTransportError
|
||||||
|
from docker import errors as docker_errors # type: ignore[import-untyped, unused-ignore]
|
||||||
from openai import (
|
from openai import (
|
||||||
APIConnectionError,
|
APIConnectionError,
|
||||||
APIError,
|
APIError,
|
||||||
@@ -55,19 +56,6 @@ _INPUT_REJECTION_CODES = frozenset({400, 404, 422})
|
|||||||
_MAX_COMPACTIONS_PER_CYCLE = 2
|
_MAX_COMPACTIONS_PER_CYCLE = 2
|
||||||
|
|
||||||
|
|
||||||
@cache
|
|
||||||
def _teardown_sandbox_errors() -> tuple[type[BaseException], ...]:
|
|
||||||
"""Sandbox-gone errors, tolerated during shutdown.
|
|
||||||
|
|
||||||
The Docker SDK is imported here rather than at module scope: it is only
|
|
||||||
reachable with the Docker runtime backend, and importing it eagerly puts it
|
|
||||||
on every launch's critical path.
|
|
||||||
"""
|
|
||||||
from docker import errors as docker_errors # type: ignore[import-untyped, unused-ignore]
|
|
||||||
|
|
||||||
return (ExecTransportError, docker_errors.NotFound)
|
|
||||||
|
|
||||||
|
|
||||||
class ProviderRefusalError(AgentsException):
|
class ProviderRefusalError(AgentsException):
|
||||||
"""Raised when a provider returns a structured refusal instead of an exception."""
|
"""Raised when a provider returns a structured refusal instead of an exception."""
|
||||||
|
|
||||||
@@ -138,8 +126,6 @@ def _is_transient_model_error(exc: BaseException) -> bool:
|
|||||||
return True
|
return True
|
||||||
code = _model_error_status_code(exc)
|
code = _model_error_status_code(exc)
|
||||||
if code is not None:
|
if code is not None:
|
||||||
import litellm
|
|
||||||
|
|
||||||
return bool(litellm._should_retry(code))
|
return bool(litellm._should_retry(code))
|
||||||
return isinstance(exc, APIError)
|
return isinstance(exc, APIError)
|
||||||
|
|
||||||
@@ -706,7 +692,7 @@ async def _run_cycle( # noqa: PLR0912, PLR0915
|
|||||||
"Ignoring LiteLLM end-of-stream shutdown race for %s",
|
"Ignoring LiteLLM end-of-stream shutdown race for %s",
|
||||||
agent_id,
|
agent_id,
|
||||||
)
|
)
|
||||||
except _teardown_sandbox_errors():
|
except (ExecTransportError, docker_errors.NotFound):
|
||||||
if not coordinator.is_shutting_down:
|
if not coordinator.is_shutting_down:
|
||||||
raise
|
raise
|
||||||
logger.warning(
|
logger.warning(
|
||||||
|
|||||||
@@ -8,7 +8,6 @@ from typing import TYPE_CHECKING, Any
|
|||||||
from agents.model_settings import ModelSettings
|
from agents.model_settings import ModelSettings
|
||||||
from openai.types.shared import Reasoning
|
from openai.types.shared import Reasoning
|
||||||
|
|
||||||
from strix.config import opencode
|
|
||||||
from strix.config.models import (
|
from strix.config.models import (
|
||||||
DEFAULT_MODEL_RETRY,
|
DEFAULT_MODEL_RETRY,
|
||||||
OPENROUTER_ATTRIBUTION_HEADERS,
|
OPENROUTER_ATTRIBUTION_HEADERS,
|
||||||
@@ -227,23 +226,6 @@ def build_scope_context(scan_config: dict[str, Any]) -> dict[str, Any]:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def build_scan_targets(scan_config: dict[str, Any]) -> list[str]:
|
|
||||||
"""One canonical string per authorized target.
|
|
||||||
|
|
||||||
Agents refer to the target in whatever words they were handed, so anything
|
|
||||||
keyed on a target the model types drifts apart across a run. This is the
|
|
||||||
scan's own spelling, which target-keyed tools resolve against. A checkout is
|
|
||||||
named by its workspace path rather than its remote URL, so the local tree —
|
|
||||||
and its revision — is what gets inspected.
|
|
||||||
"""
|
|
||||||
targets: list[str] = []
|
|
||||||
for target in build_scope_context(scan_config)["authorized_targets"]:
|
|
||||||
value = target["workspace_path"] or target["value"]
|
|
||||||
if value and value not in targets:
|
|
||||||
targets.append(value)
|
|
||||||
return targets
|
|
||||||
|
|
||||||
|
|
||||||
def make_model_settings(
|
def make_model_settings(
|
||||||
reasoning_effort: ReasoningEffort | None,
|
reasoning_effort: ReasoningEffort | None,
|
||||||
*,
|
*,
|
||||||
@@ -321,13 +303,6 @@ def _prompt_cache_extra_args(model_name: str) -> dict[str, Any] | None:
|
|||||||
"""
|
"""
|
||||||
if not is_claude_model(model_name):
|
if not is_claude_model(model_name):
|
||||||
return None
|
return None
|
||||||
# OpenCode's Chat Completions and Responses routes use the raw OpenAI SDK,
|
|
||||||
# which rejects this LiteLLM-only argument. Its Anthropic route does go
|
|
||||||
# through LiteLLM, so the injection points apply there as they would for a
|
|
||||||
# direct Anthropic key.
|
|
||||||
oc = opencode.subscription_model(model_name)
|
|
||||||
if oc is not None and oc.protocol != opencode.PROTOCOL_MESSAGES:
|
|
||||||
return None
|
|
||||||
if is_bedrock_route(model_name) and not bedrock_route_supports_prompt_caching(model_name):
|
if is_bedrock_route(model_name) and not bedrock_route_supports_prompt_caching(model_name):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
|||||||
@@ -37,7 +37,6 @@ from strix.core.execution import (
|
|||||||
from strix.core.hooks import BudgetExceededError, ReportUsageHooks, recomputed_budget_flags
|
from strix.core.hooks import BudgetExceededError, ReportUsageHooks, recomputed_budget_flags
|
||||||
from strix.core.inputs import (
|
from strix.core.inputs import (
|
||||||
build_root_task,
|
build_root_task,
|
||||||
build_scan_targets,
|
|
||||||
build_scope_context,
|
build_scope_context,
|
||||||
make_model_settings,
|
make_model_settings,
|
||||||
)
|
)
|
||||||
@@ -53,12 +52,10 @@ from strix.tools.output_store import (
|
|||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from agents.mcp import MCPServer
|
|
||||||
from agents.memory import SQLiteSession
|
from agents.memory import SQLiteSession
|
||||||
from agents.result import RunResultBase
|
from agents.result import RunResultBase
|
||||||
|
|
||||||
from strix.runtime.status import StatusSink
|
from strix.runtime.status import StatusSink
|
||||||
from strix.tools.mcp import ConnectedMcpServer
|
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -66,48 +63,6 @@ logger = logging.getLogger(__name__)
|
|||||||
StreamEventSink = Callable[[str, Any], None]
|
StreamEventSink = Callable[[str, Any], None]
|
||||||
|
|
||||||
|
|
||||||
def _mcp_startup_summary(connections: list[ConnectedMcpServer]) -> str:
|
|
||||||
"""One user-facing line summarizing the MCP servers that connected."""
|
|
||||||
server_count = len(connections)
|
|
||||||
tool_count = sum(c.tool_count for c in connections)
|
|
||||||
servers_word = "server" if server_count == 1 else "servers"
|
|
||||||
tools_word = "tool" if tool_count == 1 else "tools"
|
|
||||||
names = ", ".join(c.name for c in connections)
|
|
||||||
return f"MCP: connected {server_count} {servers_word} ({tool_count} {tools_word}): {names}"
|
|
||||||
|
|
||||||
|
|
||||||
def _record_mcp_connections(connections: list[ConnectedMcpServer]) -> None:
|
|
||||||
"""Record which MCP servers this run connected, for the interfaces.
|
|
||||||
|
|
||||||
A server's tools are offered to the model under a name built from the
|
|
||||||
connection name and the tool's own name, which cannot be split back apart, so
|
|
||||||
the TUI and the run viewer need the names to match a tool call against before
|
|
||||||
they can show which server it went out to. Kept on the run record because the
|
|
||||||
viewer reads a finished run from disk.
|
|
||||||
"""
|
|
||||||
report_state = get_global_report_state()
|
|
||||||
if report_state is None:
|
|
||||||
return
|
|
||||||
report_state.record_mcp_connections([connection.name for connection in connections])
|
|
||||||
|
|
||||||
|
|
||||||
def _mcp_connection_notes(connections: list[ConnectedMcpServer]) -> str | None:
|
|
||||||
"""A block describing the connections the user left notes on, for the agent.
|
|
||||||
|
|
||||||
Only connections with notes are listed, so the note describes the connection
|
|
||||||
once rather than being repeated onto every tool. Returns ``None`` when no
|
|
||||||
connection has notes.
|
|
||||||
"""
|
|
||||||
noted = [(c.name, c.notes) for c in connections if c.notes]
|
|
||||||
if not noted:
|
|
||||||
return None
|
|
||||||
lines = "\n".join(f"- `{name}.*` tools: {notes}" for name, notes in noted)
|
|
||||||
return (
|
|
||||||
"The user connected these MCP servers for this run and left notes on how "
|
|
||||||
f"to use each:\n{lines}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _merge_root_prompt_context(
|
def _merge_root_prompt_context(
|
||||||
scope_context: dict[str, Any],
|
scope_context: dict[str, Any],
|
||||||
extra_system_prompt_context: dict[str, Any] | None,
|
extra_system_prompt_context: dict[str, Any] | None,
|
||||||
@@ -129,7 +84,6 @@ def _compose_root_instructions_override(
|
|||||||
skills: list[str],
|
skills: list[str],
|
||||||
scan_mode: str,
|
scan_mode: str,
|
||||||
is_whitebox: bool,
|
is_whitebox: bool,
|
||||||
is_diff_scoped: bool,
|
|
||||||
interactive: bool,
|
interactive: bool,
|
||||||
system_prompt_context: dict[str, Any],
|
system_prompt_context: dict[str, Any],
|
||||||
) -> str | None:
|
) -> str | None:
|
||||||
@@ -141,7 +95,6 @@ def _compose_root_instructions_override(
|
|||||||
scan_mode=scan_mode,
|
scan_mode=scan_mode,
|
||||||
is_whitebox=is_whitebox,
|
is_whitebox=is_whitebox,
|
||||||
is_root=True,
|
is_root=True,
|
||||||
is_diff_scoped=is_diff_scoped,
|
|
||||||
interactive=interactive,
|
interactive=interactive,
|
||||||
system_prompt_context=system_prompt_context,
|
system_prompt_context=system_prompt_context,
|
||||||
)
|
)
|
||||||
@@ -231,13 +184,11 @@ async def run_strix_scan(
|
|||||||
coordinator = AgentCoordinator()
|
coordinator = AgentCoordinator()
|
||||||
coordinator.set_snapshot_path(agents_path)
|
coordinator.set_snapshot_path(agents_path)
|
||||||
|
|
||||||
from strix.tools.coverage.tools import hydrate_coverage_from_disk
|
|
||||||
from strix.tools.notes.tools import hydrate_notes_from_disk
|
from strix.tools.notes.tools import hydrate_notes_from_disk
|
||||||
from strix.tools.todo.tools import hydrate_todos_from_disk
|
from strix.tools.todo.tools import hydrate_todos_from_disk
|
||||||
|
|
||||||
hydrate_todos_from_disk(state_dir)
|
hydrate_todos_from_disk(state_dir)
|
||||||
hydrate_notes_from_disk(state_dir)
|
hydrate_notes_from_disk(state_dir)
|
||||||
hydrate_coverage_from_disk(state_dir)
|
|
||||||
|
|
||||||
root_id: str | None = None
|
root_id: str | None = None
|
||||||
if is_resume:
|
if is_resume:
|
||||||
@@ -306,14 +257,11 @@ async def run_strix_scan(
|
|||||||
configure_spill_writer(_spill_to_workspace)
|
configure_spill_writer(_spill_to_workspace)
|
||||||
|
|
||||||
sessions_to_close: list[SQLiteSession] = []
|
sessions_to_close: list[SQLiteSession] = []
|
||||||
mcp_servers: list[MCPServer] = []
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
targets = scan_config.get("targets") or []
|
targets = scan_config.get("targets") or []
|
||||||
scan_mode = str(scan_config.get("scan_mode") or "deep")
|
scan_mode = str(scan_config.get("scan_mode") or "deep")
|
||||||
is_whitebox = any(t.get("type") == "local_code" for t in targets)
|
is_whitebox = any(t.get("type") == "local_code" for t in targets)
|
||||||
diff_scope = scan_config.get("diff_scope")
|
|
||||||
is_diff_scoped = bool(isinstance(diff_scope, dict) and diff_scope.get("active"))
|
|
||||||
skills = list(scan_config.get("skills") or [])
|
skills = list(scan_config.get("skills") or [])
|
||||||
root_task = build_root_task(scan_config)
|
root_task = build_root_task(scan_config)
|
||||||
model_settings = make_model_settings(
|
model_settings = make_model_settings(
|
||||||
@@ -350,39 +298,16 @@ async def run_strix_scan(
|
|||||||
skills=skills,
|
skills=skills,
|
||||||
scan_mode=scan_mode,
|
scan_mode=scan_mode,
|
||||||
is_whitebox=is_whitebox,
|
is_whitebox=is_whitebox,
|
||||||
is_diff_scoped=is_diff_scoped,
|
|
||||||
interactive=interactive,
|
interactive=interactive,
|
||||||
system_prompt_context=root_context,
|
system_prompt_context=root_context,
|
||||||
)
|
)
|
||||||
|
|
||||||
# Connect any MCP servers the user listed in ~/.strix/mcp-servers.json and
|
|
||||||
# register their tools before the agent is built. Fail-open: a missing
|
|
||||||
# config, or a server that will not connect, must never break a run.
|
|
||||||
from strix.tools.mcp import connect_mcp_servers, load_user_mcp_configs
|
|
||||||
|
|
||||||
try:
|
|
||||||
user_mcp_configs = load_user_mcp_configs()
|
|
||||||
if user_mcp_configs:
|
|
||||||
connections = await connect_mcp_servers(user_mcp_configs)
|
|
||||||
mcp_servers = [c.server for c in connections]
|
|
||||||
# Recorded even when nothing connected, so a resumed run does not
|
|
||||||
# keep attributing tool calls to servers it no longer has.
|
|
||||||
_record_mcp_connections(connections)
|
|
||||||
if connections:
|
|
||||||
report(_mcp_startup_summary(connections))
|
|
||||||
notes_block = _mcp_connection_notes(connections)
|
|
||||||
if notes_block:
|
|
||||||
root_task = f"{root_task}\n\n{notes_block}"
|
|
||||||
except Exception:
|
|
||||||
logger.exception("Failed to connect user MCP servers; continuing without them")
|
|
||||||
|
|
||||||
root_agent = build_strix_agent(
|
root_agent = build_strix_agent(
|
||||||
name="Root Agent",
|
name="Root Agent",
|
||||||
skills=skills,
|
skills=skills,
|
||||||
is_root=True,
|
is_root=True,
|
||||||
scan_mode=scan_mode,
|
scan_mode=scan_mode,
|
||||||
is_whitebox=is_whitebox,
|
is_whitebox=is_whitebox,
|
||||||
is_diff_scoped=is_diff_scoped,
|
|
||||||
interactive=interactive,
|
interactive=interactive,
|
||||||
chat_completions_tools=chat_completions_tools,
|
chat_completions_tools=chat_completions_tools,
|
||||||
strict_tool_schemas=strict_tool_schemas,
|
strict_tool_schemas=strict_tool_schemas,
|
||||||
@@ -402,7 +327,6 @@ async def run_strix_scan(
|
|||||||
child_agent_builder = make_child_factory(
|
child_agent_builder = make_child_factory(
|
||||||
scan_mode=scan_mode,
|
scan_mode=scan_mode,
|
||||||
is_whitebox=is_whitebox,
|
is_whitebox=is_whitebox,
|
||||||
is_diff_scoped=is_diff_scoped,
|
|
||||||
interactive=interactive,
|
interactive=interactive,
|
||||||
chat_completions_tools=chat_completions_tools,
|
chat_completions_tools=chat_completions_tools,
|
||||||
strict_tool_schemas=strict_tool_schemas,
|
strict_tool_schemas=strict_tool_schemas,
|
||||||
@@ -431,7 +355,6 @@ async def run_strix_scan(
|
|||||||
"parent_id": None,
|
"parent_id": None,
|
||||||
"interactive": interactive,
|
"interactive": interactive,
|
||||||
"spawn_child_agent": spawn_child_agent,
|
"spawn_child_agent": spawn_child_agent,
|
||||||
"scan_targets": build_scan_targets(scan_config),
|
|
||||||
"max_context_images": settings.runtime.max_context_images,
|
"max_context_images": settings.runtime.max_context_images,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -555,9 +478,6 @@ async def run_strix_scan(
|
|||||||
for s in sessions_to_close:
|
for s in sessions_to_close:
|
||||||
with contextlib.suppress(Exception):
|
with contextlib.suppress(Exception):
|
||||||
s.close()
|
s.close()
|
||||||
for mcp_server in mcp_servers:
|
|
||||||
with contextlib.suppress(Exception):
|
|
||||||
await mcp_server.cleanup() # type: ignore[no-untyped-call]
|
|
||||||
with contextlib.suppress(Exception):
|
with contextlib.suppress(Exception):
|
||||||
await coordinator._maybe_snapshot()
|
await coordinator._maybe_snapshot()
|
||||||
if cleanup_on_exit:
|
if cleanup_on_exit:
|
||||||
|
|||||||
+16
-102
@@ -1,9 +1,8 @@
|
|||||||
"""`strix auth` — subscription sign-in (login / status / logout).
|
"""`strix auth` — ChatGPT subscription sign-in (login / status / logout).
|
||||||
|
|
||||||
Signing in only stores credentials (``~/.strix/subscription-auth.json``); model
|
Signing in only stores OAuth tokens (``~/.strix/subscription-auth.json``); model
|
||||||
selection stays with ``STRIX_LLM``. A ``chatgpt/<model>`` STRIX_LLM runs on the
|
selection stays with ``STRIX_LLM``. A ``chatgpt/<model>`` STRIX_LLM runs on the
|
||||||
ChatGPT subscription; ``opencode/<model>`` (Zen credits) or
|
subscription.
|
||||||
``opencode-go/<model>`` (Go subscription) run on OpenCode.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -22,7 +21,7 @@ from rich.console import Console
|
|||||||
from rich.panel import Panel
|
from rich.panel import Panel
|
||||||
from rich.text import Text
|
from rich.text import Text
|
||||||
|
|
||||||
from strix.config import codex, load_settings, opencode
|
from strix.config import codex, load_settings
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
@@ -33,20 +32,13 @@ logger = logging.getLogger(__name__)
|
|||||||
|
|
||||||
_CALLBACK_TIMEOUT_S = 300
|
_CALLBACK_TIMEOUT_S = 300
|
||||||
|
|
||||||
# CLI-facing name for the default login provider. Internally this is the Codex
|
# CLI-facing name for the login provider. Internally this is the Codex OAuth
|
||||||
# OAuth flow (``codex.PROVIDER``), but users know it as ChatGPT, so that's what
|
# flow (``codex.PROVIDER``), but users know it as ChatGPT, so that's what the
|
||||||
# the command and messaging say. ``codex`` is accepted as an alias.
|
# command and messaging say. ``codex`` is accepted as an alias.
|
||||||
LOGIN_PROVIDER = "chatgpt"
|
LOGIN_PROVIDER = "chatgpt"
|
||||||
_ACCEPTED_PROVIDERS = frozenset({LOGIN_PROVIDER, codex.PROVIDER})
|
_ACCEPTED_PROVIDERS = frozenset({LOGIN_PROVIDER, codex.PROVIDER})
|
||||||
_OPENCODE_PROVIDERS = frozenset({opencode.PROVIDER, "opencode-go", "zen"})
|
|
||||||
|
|
||||||
_USAGE = (
|
_USAGE = "Usage:\n strix auth login chatgpt [--manual]\n strix auth status\n strix auth logout"
|
||||||
"Usage:\n"
|
|
||||||
" strix auth login chatgpt [--manual]\n"
|
|
||||||
" strix auth login opencode\n"
|
|
||||||
" strix auth status\n"
|
|
||||||
" strix auth logout [chatgpt|opencode]"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def run_auth(argv: list[str]) -> int:
|
def run_auth(argv: list[str]) -> int:
|
||||||
@@ -63,7 +55,7 @@ def run_auth(argv: list[str]) -> int:
|
|||||||
handlers: dict[str, Callable[[], int]] = {
|
handlers: dict[str, Callable[[], int]] = {
|
||||||
"login": lambda: _login(console, rest),
|
"login": lambda: _login(console, rest),
|
||||||
"status": lambda: _status(console),
|
"status": lambda: _status(console),
|
||||||
"logout": lambda: _logout(console, rest),
|
"logout": lambda: _logout(console),
|
||||||
}
|
}
|
||||||
handler = handlers.get(subcommand)
|
handler = handlers.get(subcommand)
|
||||||
if handler is not None:
|
if handler is not None:
|
||||||
@@ -92,14 +84,10 @@ def _login(console: Console, argv: list[str]) -> int:
|
|||||||
except SystemExit as exc: # argparse already printed the message
|
except SystemExit as exc: # argparse already printed the message
|
||||||
return int(exc.code or 2)
|
return int(exc.code or 2)
|
||||||
|
|
||||||
if args.provider.lower() in _OPENCODE_PROVIDERS:
|
|
||||||
return _login_opencode(console)
|
|
||||||
|
|
||||||
if args.provider.lower() not in _ACCEPTED_PROVIDERS:
|
if args.provider.lower() not in _ACCEPTED_PROVIDERS:
|
||||||
console.print(
|
console.print(
|
||||||
f"[red]Unsupported provider:[/] {args.provider}. "
|
f"[red]Unsupported provider:[/] {args.provider}. "
|
||||||
f"Supported: '{LOGIN_PROVIDER}' (ChatGPT subscription) and "
|
f"Only '{LOGIN_PROVIDER}' (ChatGPT subscription) is supported."
|
||||||
f"'{opencode.PROVIDER}' (OpenCode Zen/Go)."
|
|
||||||
)
|
)
|
||||||
return 2
|
return 2
|
||||||
|
|
||||||
@@ -127,63 +115,6 @@ def _login(console: Console, argv: list[str]) -> int:
|
|||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def _login_opencode(console: Console) -> int:
|
|
||||||
console.print()
|
|
||||||
console.print("[bold]Signing in with OpenCode[/] [dim](provider: opencode)[/]")
|
|
||||||
console.print(
|
|
||||||
"[dim]This uses your OpenCode Zen credits or Go subscription for inference.\n"
|
|
||||||
f"Get your API key at {opencode.AUTH_CONSOLE_URL}[/]"
|
|
||||||
)
|
|
||||||
console.print()
|
|
||||||
try:
|
|
||||||
key = console.input("Paste your OpenCode API key: ", password=True).strip()
|
|
||||||
except (EOFError, KeyboardInterrupt):
|
|
||||||
console.print("\n[yellow]Sign-in cancelled.[/]")
|
|
||||||
return 130
|
|
||||||
if not key:
|
|
||||||
console.print("[red]No API key provided.[/]")
|
|
||||||
return 2
|
|
||||||
try:
|
|
||||||
opencode.validate_api_key(key)
|
|
||||||
except opencode.OpencodeAuthError as exc:
|
|
||||||
console.print(f"[red]SIGN-IN FAILED:[/] {exc}")
|
|
||||||
return 1
|
|
||||||
opencode.save_api_key(key)
|
|
||||||
_print_opencode_success(console)
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
def _print_opencode_success(console: Console) -> None:
|
|
||||||
text = Text()
|
|
||||||
text.append("Signed in with your OpenCode account", style="bold #22c55e")
|
|
||||||
text.append("\n\n", style="white")
|
|
||||||
text.append("Set ", style="white")
|
|
||||||
text.append("STRIX_LLM", style="bold white")
|
|
||||||
text.append(" to an ", style="white")
|
|
||||||
text.append("opencode/", style="bold cyan")
|
|
||||||
text.append(" model (e.g. ", style="white")
|
|
||||||
text.append("opencode/claude-sonnet-5", style="bold cyan")
|
|
||||||
text.append(") to run on Zen credits, or ", style="white")
|
|
||||||
text.append("opencode-go/", style="bold cyan")
|
|
||||||
text.append(" (e.g. ", style="white")
|
|
||||||
text.append("opencode-go/kimi-k3", style="bold cyan")
|
|
||||||
text.append(") to run on the Go subscription.", style="white")
|
|
||||||
text.append("\n\n", style="white")
|
|
||||||
text.append("Run a scan as usual, e.g. ", style="white")
|
|
||||||
text.append("strix --target https://example.com", style="bold cyan")
|
|
||||||
console.print()
|
|
||||||
console.print(
|
|
||||||
Panel(
|
|
||||||
text,
|
|
||||||
title="[bold white]STRIX",
|
|
||||||
title_align="left",
|
|
||||||
border_style="#22c55e",
|
|
||||||
padding=(1, 2),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
console.print()
|
|
||||||
|
|
||||||
|
|
||||||
def _run_oauth_flow(
|
def _run_oauth_flow(
|
||||||
console: Console,
|
console: Console,
|
||||||
authorize_url: str,
|
authorize_url: str,
|
||||||
@@ -313,41 +244,24 @@ def _first(query: dict[str, list[str]], key: str) -> str | None:
|
|||||||
|
|
||||||
def _status(console: Console) -> int:
|
def _status(console: Console) -> int:
|
||||||
record = codex.read_record()
|
record = codex.read_record()
|
||||||
opencode_signed_in = opencode.is_authenticated()
|
if record is None:
|
||||||
if record is None and not opencode_signed_in:
|
console.print("[yellow]Not signed in.[/] Run [cyan]strix auth login chatgpt[/] to sign in.")
|
||||||
console.print(
|
|
||||||
"[yellow]Not signed in.[/] Run [cyan]strix auth login chatgpt[/] or "
|
|
||||||
"[cyan]strix auth login opencode[/] to sign in."
|
|
||||||
)
|
|
||||||
return 1
|
return 1
|
||||||
settings = load_settings()
|
settings = load_settings()
|
||||||
if record is not None:
|
|
||||||
console.print("[green]Signed in[/] with a ChatGPT subscription.")
|
console.print("[green]Signed in[/] with a ChatGPT subscription.")
|
||||||
console.print(f" Account: [bold]{record.get('account_id')}[/]")
|
console.print(f" Account: [bold]{record.get('account_id')}[/]")
|
||||||
if opencode_signed_in:
|
if codex.subscription_model(settings.llm.model):
|
||||||
console.print("[green]Signed in[/] with an OpenCode account.")
|
|
||||||
if codex.subscription_model(settings.llm.model) or opencode.subscription_model(
|
|
||||||
settings.llm.model
|
|
||||||
):
|
|
||||||
console.print(f" Runs use the subscription (STRIX_LLM=[bold]{settings.llm.model}[/]).")
|
console.print(f" Runs use the subscription (STRIX_LLM=[bold]{settings.llm.model}[/]).")
|
||||||
else:
|
else:
|
||||||
console.print(
|
console.print(
|
||||||
" [yellow]Note:[/] set [cyan]STRIX_LLM[/] to e.g. [cyan]chatgpt/gpt-5.4[/] or "
|
" [yellow]Note:[/] set [cyan]STRIX_LLM[/] to e.g. [cyan]chatgpt/gpt-5.4[/] "
|
||||||
"[cyan]opencode/claude-sonnet-5[/] to run on a subscription."
|
"to run on the subscription."
|
||||||
)
|
)
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|
||||||
def _logout(console: Console, argv: list[str] | None = None) -> int:
|
def _logout(console: Console) -> int:
|
||||||
target = (argv[0].lower() if argv else "") or "all"
|
|
||||||
if target in _ACCEPTED_PROVIDERS or target == "all":
|
|
||||||
codex.logout()
|
codex.logout()
|
||||||
if target in _OPENCODE_PROVIDERS or target == "all":
|
|
||||||
opencode.logout()
|
|
||||||
if target != "all" and target not in _ACCEPTED_PROVIDERS | _OPENCODE_PROVIDERS:
|
|
||||||
console.print(f"[red]Unknown provider:[/] {target}\n")
|
|
||||||
console.print(_USAGE)
|
|
||||||
return 2
|
|
||||||
console.print("[green]Signed out.[/] Stored subscription credentials removed.")
|
console.print("[green]Signed out.[/] Stored subscription credentials removed.")
|
||||||
return 0
|
return 0
|
||||||
|
|
||||||
|
|||||||
@@ -3,7 +3,6 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
import os
|
|
||||||
import sys
|
import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
@@ -220,30 +219,6 @@ Examples:
|
|||||||
help="Path to a custom config file (JSON) to use instead of ~/.strix/cli-config.json",
|
help="Path to a custom config file (JSON) to use instead of ~/.strix/cli-config.json",
|
||||||
)
|
)
|
||||||
|
|
||||||
parser.add_argument(
|
|
||||||
"--mcp-config",
|
|
||||||
type=str,
|
|
||||||
metavar="PATH",
|
|
||||||
help="Path to an MCP servers JSON file to use instead of ~/.strix/mcp-servers.json.",
|
|
||||||
)
|
|
||||||
|
|
||||||
parser.add_argument(
|
|
||||||
"--mcp-server",
|
|
||||||
dest="mcp_server",
|
|
||||||
action="append",
|
|
||||||
metavar="NAME",
|
|
||||||
help="Use only this MCP connection for the run, by its config name "
|
|
||||||
"(repeatable). Every other configured connection is skipped.",
|
|
||||||
)
|
|
||||||
|
|
||||||
parser.add_argument(
|
|
||||||
"--mcp-exclude",
|
|
||||||
dest="mcp_exclude",
|
|
||||||
action="append",
|
|
||||||
metavar="NAME",
|
|
||||||
help="Skip this MCP connection for the run, by its config name (repeatable).",
|
|
||||||
)
|
|
||||||
|
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--max-budget",
|
"--max-budget",
|
||||||
"--max-budget-usd",
|
"--max-budget-usd",
|
||||||
@@ -292,20 +267,6 @@ Examples:
|
|||||||
if args.config:
|
if args.config:
|
||||||
apply_config_override(validate_config_file(args.config))
|
apply_config_override(validate_config_file(args.config))
|
||||||
|
|
||||||
if args.mcp_config:
|
|
||||||
mcp_config_path = Path(args.mcp_config).expanduser()
|
|
||||||
if not mcp_config_path.is_file():
|
|
||||||
parser.error(f"--mcp-config file not found: {args.mcp_config}")
|
|
||||||
# The MCP loader reads this env var as its config-path override, so
|
|
||||||
# setting it here makes the flag win over the default location.
|
|
||||||
os.environ["STRIX_MCP_CONFIG"] = str(mcp_config_path)
|
|
||||||
|
|
||||||
# The MCP loader reads these as its per-run include/exclude selection.
|
|
||||||
if args.mcp_server:
|
|
||||||
os.environ["STRIX_MCP_ONLY"] = ",".join(args.mcp_server)
|
|
||||||
if args.mcp_exclude:
|
|
||||||
os.environ["STRIX_MCP_EXCLUDE"] = ",".join(args.mcp_exclude)
|
|
||||||
|
|
||||||
if args.update:
|
if args.update:
|
||||||
sys.exit(0 if self_update() else 1)
|
sys.exit(0 if self_update() else 1)
|
||||||
|
|
||||||
@@ -385,7 +346,7 @@ def _load_resume_state(args: argparse.Namespace, parser: argparse.ArgumentParser
|
|||||||
)
|
)
|
||||||
try:
|
try:
|
||||||
state = read_run_record(run_dir)
|
state = read_run_record(run_dir)
|
||||||
except (RuntimeError, TypeError) as exc:
|
except RuntimeError as exc:
|
||||||
parser.error(f"--resume {args.resume}: run.json unreadable: {exc}")
|
parser.error(f"--resume {args.resume}: run.json unreadable: {exc}")
|
||||||
|
|
||||||
args.targets_info = state.get("targets_info") or []
|
args.targets_info = state.get("targets_info") or []
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ from rich.console import Console
|
|||||||
from rich.panel import Panel
|
from rich.panel import Panel
|
||||||
from rich.text import Text
|
from rich.text import Text
|
||||||
|
|
||||||
from strix.config import codex, load_settings, opencode
|
from strix.config import codex, load_settings
|
||||||
from strix.interface.utils import (
|
from strix.interface.utils import (
|
||||||
check_docker_connection,
|
check_docker_connection,
|
||||||
image_exists,
|
image_exists,
|
||||||
@@ -37,17 +37,6 @@ def validate_environment() -> None:
|
|||||||
logger.info("Environment OK (ChatGPT subscription)")
|
logger.info("Environment OK (ChatGPT subscription)")
|
||||||
return
|
return
|
||||||
|
|
||||||
oc = opencode.subscription_model(settings.llm.model)
|
|
||||||
if oc:
|
|
||||||
if not opencode.is_authenticated():
|
|
||||||
console.print(
|
|
||||||
f"[red]STRIX_LLM={settings.llm.model} runs on {oc.label}, "
|
|
||||||
"but you're not signed in.[/] Run [cyan]strix auth login opencode[/] first."
|
|
||||||
)
|
|
||||||
sys.exit(1)
|
|
||||||
logger.info("Environment OK (%s)", oc.label)
|
|
||||||
return
|
|
||||||
|
|
||||||
if not settings.llm.model:
|
if not settings.llm.model:
|
||||||
missing_required_vars.append("STRIX_LLM")
|
missing_required_vars.append("STRIX_LLM")
|
||||||
|
|
||||||
|
|||||||
+3
-13
@@ -14,7 +14,7 @@ from rich.console import Console
|
|||||||
from rich.panel import Panel
|
from rich.panel import Panel
|
||||||
from rich.text import Text
|
from rich.text import Text
|
||||||
|
|
||||||
from strix.config import codex, load_settings, opencode, persist_current
|
from strix.config import codex, load_settings, persist_current
|
||||||
from strix.core.paths import run_dir_for
|
from strix.core.paths import run_dir_for
|
||||||
from strix.interface.cli_args import parse_arguments
|
from strix.interface.cli_args import parse_arguments
|
||||||
from strix.interface.environment import (
|
from strix.interface.environment import (
|
||||||
@@ -104,14 +104,8 @@ def _provider_import_hint(exc: BaseException, model: str) -> str | None:
|
|||||||
|
|
||||||
|
|
||||||
def _subscription_error_hint(exc: BaseException) -> str | None:
|
def _subscription_error_hint(exc: BaseException) -> str | None:
|
||||||
"""Return an actionable hint for a known subscription error, or None."""
|
"""Return an actionable hint for a known ChatGPT-subscription error, or None."""
|
||||||
model = load_settings().llm.model
|
if not codex.subscription_model(load_settings().llm.model):
|
||||||
if opencode.subscription_model(model):
|
|
||||||
joined = " ".join(_exception_messages(exc)).lower()
|
|
||||||
if "error code: 401" in joined or "http 401" in joined or "unauthorized" in joined:
|
|
||||||
return "Your OpenCode API key was rejected. Sign in again:\n strix auth login opencode"
|
|
||||||
return None
|
|
||||||
if not codex.subscription_model(model):
|
|
||||||
return None
|
return None
|
||||||
joined = " ".join(_exception_messages(exc)).lower()
|
joined = " ".join(_exception_messages(exc)).lower()
|
||||||
if "not supported when using codex with a chatgpt account" in joined:
|
if "not supported when using codex with a chatgpt account" in joined:
|
||||||
@@ -437,10 +431,6 @@ def main() -> None:
|
|||||||
|
|
||||||
sys.exit(run_auth(sys.argv[2:]))
|
sys.exit(run_auth(sys.argv[2:]))
|
||||||
|
|
||||||
from strix.llm.warmup import start_import_warmup
|
|
||||||
|
|
||||||
start_import_warmup()
|
|
||||||
|
|
||||||
args = parse_arguments()
|
args = parse_arguments()
|
||||||
|
|
||||||
start_background_check()
|
start_background_check()
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import logging
|
|||||||
from datetime import UTC, datetime
|
from datetime import UTC, datetime
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
from strix.config import Settings, load_settings, opencode
|
from strix.config import Settings, codex, load_settings
|
||||||
from strix.core.paths import run_dir_for
|
from strix.core.paths import run_dir_for
|
||||||
from strix.interface.utils import (
|
from strix.interface.utils import (
|
||||||
assign_workspace_subdirs,
|
assign_workspace_subdirs,
|
||||||
@@ -226,7 +226,7 @@ def telemetry_start(args: argparse.Namespace) -> None:
|
|||||||
model = load_settings().llm.model
|
model = load_settings().llm.model
|
||||||
kwargs = {
|
kwargs = {
|
||||||
"model": model,
|
"model": model,
|
||||||
"auth_mode": opencode.auth_mode(model),
|
"auth_mode": codex.auth_mode(model),
|
||||||
"scan_mode": args.scan_mode,
|
"scan_mode": args.scan_mode,
|
||||||
"is_whitebox": is_whitebox_scan(args.targets_info),
|
"is_whitebox": is_whitebox_scan(args.targets_info),
|
||||||
"interactive": not args.non_interactive,
|
"interactive": not args.non_interactive,
|
||||||
@@ -247,8 +247,7 @@ def _persist_run_record(args: argparse.Namespace) -> None:
|
|||||||
"status": "running",
|
"status": "running",
|
||||||
"start_time": datetime.now(UTC).isoformat(),
|
"start_time": datetime.now(UTC).isoformat(),
|
||||||
"end_time": None,
|
"end_time": None,
|
||||||
"auth_mode": opencode.auth_mode(load_settings().llm.model),
|
"auth_mode": codex.auth_mode(load_settings().llm.model),
|
||||||
"subscription_provider": opencode.subscription_provider(load_settings().llm.model),
|
|
||||||
"targets_info": args.targets_info,
|
"targets_info": args.targets_info,
|
||||||
"scan_mode": args.scan_mode,
|
"scan_mode": args.scan_mode,
|
||||||
"instruction": args.instruction,
|
"instruction": args.instruction,
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ from strix.interface.tui.backend.projection import (
|
|||||||
sanitize_terminal_text,
|
sanitize_terminal_text,
|
||||||
terminal_projection,
|
terminal_projection,
|
||||||
)
|
)
|
||||||
from strix.interface.utils import is_subscription_run, subscription_label
|
from strix.interface.utils import is_subscription_run
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
@@ -164,10 +164,6 @@ class TuiController:
|
|||||||
subscription = False
|
subscription = False
|
||||||
with contextlib.suppress(Exception):
|
with contextlib.suppress(Exception):
|
||||||
subscription = is_subscription_run(self.report_state)
|
subscription = is_subscription_run(self.report_state)
|
||||||
label = ""
|
|
||||||
if subscription:
|
|
||||||
with contextlib.suppress(Exception):
|
|
||||||
label = subscription_label()
|
|
||||||
model_warning = ""
|
model_warning = ""
|
||||||
if model and not is_recommended_or_frontier_model(model):
|
if model and not is_recommended_or_frontier_model(model):
|
||||||
model_warning = (
|
model_warning = (
|
||||||
@@ -204,7 +200,6 @@ class TuiController:
|
|||||||
],
|
],
|
||||||
"usage": terminal_projection(usage, max_string=256, max_items=20),
|
"usage": terminal_projection(usage, max_string=256, max_items=20),
|
||||||
"subscription": subscription,
|
"subscription": subscription,
|
||||||
"subscription_label": label,
|
|
||||||
"viewer_status": self.viewer_status,
|
"viewer_status": self.viewer_status,
|
||||||
"viewer_url": terminal_projection(self.viewer_url, max_string=1024),
|
"viewer_url": terminal_projection(self.viewer_url, max_string=1024),
|
||||||
"error": terminal_projection(self.error, max_string=2 * 1024),
|
"error": terminal_projection(self.error, max_string=2 * 1024),
|
||||||
|
|||||||
@@ -146,9 +146,7 @@ def bounded_state_projection(state: dict[str, Any]) -> dict[str, Any]:
|
|||||||
}
|
}
|
||||||
for message in state["messages"][-5:]
|
for message in state["messages"][-5:]
|
||||||
]
|
]
|
||||||
state["usage"] = {
|
state["usage"] = {}
|
||||||
key: state["usage"][key] for key in ("total_tokens", "cost") if key in state["usage"]
|
|
||||||
}
|
|
||||||
state["error"] = terminal_projection(state["error"], max_string=512)
|
state["error"] = terminal_projection(state["error"], max_string=512)
|
||||||
state["model_warning"] = terminal_projection(state["model_warning"], max_string=256)
|
state["model_warning"] = terminal_projection(state["model_warning"], max_string=256)
|
||||||
state["caido_url"] = terminal_projection(state["caido_url"], max_string=256)
|
state["caido_url"] = terminal_projection(state["caido_url"], max_string=256)
|
||||||
@@ -175,7 +173,7 @@ def bounded_state_projection(state: dict[str, Any]) -> dict[str, Any]:
|
|||||||
"model_warning": "",
|
"model_warning": "",
|
||||||
"caido_url": None,
|
"caido_url": None,
|
||||||
"messages": [],
|
"messages": [],
|
||||||
"usage": state["usage"],
|
"usage": {},
|
||||||
"subscription": state["subscription"],
|
"subscription": state["subscription"],
|
||||||
"viewer_status": state["viewer_status"],
|
"viewer_status": state["viewer_status"],
|
||||||
"viewer_url": None,
|
"viewer_url": None,
|
||||||
|
|||||||
@@ -596,11 +596,7 @@ func (m Model) statsView() string {
|
|||||||
if b.Len() > 0 {
|
if b.Len() > 0 {
|
||||||
b.WriteString("\n")
|
b.WriteString("\n")
|
||||||
}
|
}
|
||||||
label := m.snapshot.SubscriptionLabel
|
b.WriteString(lipgloss.NewStyle().Foreground(green).Render("ChatGPT subscription"))
|
||||||
if label == "" {
|
|
||||||
label = "ChatGPT subscription"
|
|
||||||
}
|
|
||||||
b.WriteString(lipgloss.NewStyle().Foreground(green).Render(label))
|
|
||||||
}
|
}
|
||||||
total := numberValue(m.snapshot.Usage["total_tokens"])
|
total := numberValue(m.snapshot.Usage["total_tokens"])
|
||||||
if total > 0 {
|
if total > 0 {
|
||||||
|
|||||||
@@ -68,7 +68,6 @@ type Snapshot struct {
|
|||||||
Vulnerabilities []map[string]any `json:"-"`
|
Vulnerabilities []map[string]any `json:"-"`
|
||||||
Usage map[string]any `json:"usage"`
|
Usage map[string]any `json:"usage"`
|
||||||
Subscription bool `json:"subscription"`
|
Subscription bool `json:"subscription"`
|
||||||
SubscriptionLabel string `json:"subscription_label"`
|
|
||||||
ViewerStatus string `json:"viewer_status"`
|
ViewerStatus string `json:"viewer_status"`
|
||||||
ViewerURL *string `json:"viewer_url"`
|
ViewerURL *string `json:"viewer_url"`
|
||||||
Error *string `json:"error"`
|
Error *string `json:"error"`
|
||||||
|
|||||||
@@ -100,7 +100,7 @@ func applyMarkdownStyles(text string) string {
|
|||||||
case strings.HasPrefix(line, "- "), strings.HasPrefix(line, "* "):
|
case strings.HasPrefix(line, "- "), strings.HasPrefix(line, "* "):
|
||||||
out.WriteString(Col(Green).Render("• ") + inlineFormat(line[2:]))
|
out.WriteString(Col(Green).Render("• ") + inlineFormat(line[2:]))
|
||||||
case len(line) > 2 && line[0] >= '0' && line[0] <= '9' && (line[1:3] == ". " || line[1:3] == ") "):
|
case len(line) > 2 && line[0] >= '0' && line[0] <= '9' && (line[1:3] == ". " || line[1:3] == ") "):
|
||||||
out.WriteString(Col(Green).Render(line[:2]+" ") + inlineFormat(line[3:]))
|
out.WriteString(Col(Green).Render(string(line[0])+". ") + inlineFormat(line[2:]))
|
||||||
case line == "---" || line == "***" || line == "___":
|
case line == "---" || line == "***" || line == "___":
|
||||||
out.WriteString(Col(Green).Render(strings.Repeat("─", 40)))
|
out.WriteString(Col(Green).Render(strings.Repeat("─", 40)))
|
||||||
default:
|
default:
|
||||||
|
|||||||
@@ -1,194 +0,0 @@
|
|||||||
package render
|
|
||||||
|
|
||||||
import (
|
|
||||||
"strconv"
|
|
||||||
"strings"
|
|
||||||
|
|
||||||
"github.com/charmbracelet/lipgloss"
|
|
||||||
)
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// Coverage ledger (record_coverage / update_coverage / list_coverage)
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
// coverageOutcomes maps a ledger outcome to its marker and color. A cleared
|
|
||||||
// surface and an unresolved one must not look alike at a glance: the whole
|
|
||||||
// point of the ledger is that a reader can see which surfaces are still open.
|
|
||||||
var coverageOutcomes = map[string]struct {
|
|
||||||
marker string
|
|
||||||
label string
|
|
||||||
color lipgloss.Color
|
|
||||||
}{
|
|
||||||
"reported": {"!", "reported", SevHigh},
|
|
||||||
"no_issue_found": {"✓", "no issue found", Green},
|
|
||||||
"ruled_out": {"✓", "ruled out", Mint},
|
|
||||||
"not_applicable": {"–", "not applicable", Slate},
|
|
||||||
"needs_follow_up": {"?", "needs follow-up", AmberY},
|
|
||||||
}
|
|
||||||
|
|
||||||
func coverageOutcome(outcome string) (string, string, lipgloss.Color) {
|
|
||||||
if meta, ok := coverageOutcomes[strings.TrimSpace(strings.ToLower(outcome))]; ok {
|
|
||||||
return meta.marker, meta.label, meta.color
|
|
||||||
}
|
|
||||||
if outcome == "" {
|
|
||||||
return "·", "", Gray
|
|
||||||
}
|
|
||||||
return "·", strings.ReplaceAll(outcome, "_", " "), Gray
|
|
||||||
}
|
|
||||||
|
|
||||||
var coverageTitles = map[string]struct {
|
|
||||||
title string
|
|
||||||
loading string
|
|
||||||
errMsg string
|
|
||||||
}{
|
|
||||||
"record_coverage": {"Coverage Recorded", "Recording...", "Failed to record coverage"},
|
|
||||||
"update_coverage": {"Coverage Updated", "Updating...", "Failed to update coverage"},
|
|
||||||
"list_coverage": {"Coverage", "Loading...", "Unable to list coverage"},
|
|
||||||
}
|
|
||||||
|
|
||||||
func renderCoverage(name string, args map[string]any, result any) string {
|
|
||||||
meta := coverageTitles[name]
|
|
||||||
var b strings.Builder
|
|
||||||
b.WriteString("▣ " + Bold(Cyan).Render(meta.title))
|
|
||||||
|
|
||||||
if s, ok := result.(string); ok && strings.TrimSpace(s) != "" {
|
|
||||||
b.WriteString("\n " + Dim().Render(strings.TrimSpace(s)))
|
|
||||||
return b.String()
|
|
||||||
}
|
|
||||||
m, ok := result.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
coverageArgsPreview(&b, name, args)
|
|
||||||
b.WriteString("\n " + Dim().Render(meta.loading))
|
|
||||||
return b.String()
|
|
||||||
}
|
|
||||||
if !truthy(m["success"]) {
|
|
||||||
coverageArgsPreview(&b, name, args)
|
|
||||||
errMsg := StringValue(m["error"])
|
|
||||||
if errMsg == "" {
|
|
||||||
errMsg = meta.errMsg
|
|
||||||
}
|
|
||||||
b.WriteString("\n " + Col(Red).Render(errMsg))
|
|
||||||
return b.String()
|
|
||||||
}
|
|
||||||
|
|
||||||
switch name {
|
|
||||||
case "list_coverage":
|
|
||||||
coverageListBody(&b, m)
|
|
||||||
case "update_coverage":
|
|
||||||
marker, label, color := coverageOutcome(StringValue(m["outcome"]))
|
|
||||||
_, previous, previousColor := coverageOutcome(StringValue(m["previous_outcome"]))
|
|
||||||
b.WriteString("\n " + Col(color).Render(marker) + " " + coverageSubject(args, m))
|
|
||||||
if previous != "" {
|
|
||||||
b.WriteString("\n " + Col(previousColor).Render(previous) +
|
|
||||||
Dim().Render(" → ") + Col(color).Render(label))
|
|
||||||
} else {
|
|
||||||
b.WriteString("\n " + Col(color).Render(label))
|
|
||||||
}
|
|
||||||
coverageEvidence(&b, StringValue(args["evidence"]))
|
|
||||||
default:
|
|
||||||
marker, label, color := coverageOutcome(StringValue(m["outcome"]))
|
|
||||||
b.WriteString("\n " + Col(color).Render(marker) + " " + coverageSubject(args, m))
|
|
||||||
b.WriteString("\n " + Col(color).Render(label))
|
|
||||||
coverageEvidence(&b, StringValue(args["evidence"]))
|
|
||||||
}
|
|
||||||
return b.String()
|
|
||||||
}
|
|
||||||
|
|
||||||
// coverageSubject names the surface being recorded, falling back to the entry
|
|
||||||
// id when only the id is known (an update carries no surface in its args).
|
|
||||||
func coverageSubject(args map[string]any, result map[string]any) string {
|
|
||||||
surface := strings.TrimSpace(StringValue(args["surface"]))
|
|
||||||
risk := strings.TrimSpace(StringValue(args["risk_area"]))
|
|
||||||
switch {
|
|
||||||
case surface != "" && risk != "":
|
|
||||||
return surface + Dim().Render(" · "+risk)
|
|
||||||
case surface != "":
|
|
||||||
return surface
|
|
||||||
case risk != "":
|
|
||||||
return risk
|
|
||||||
}
|
|
||||||
if id := StringValue(result["entry_id"]); id != "" {
|
|
||||||
return Dim().Render("entry " + id)
|
|
||||||
}
|
|
||||||
return Dim().Render("(unnamed surface)")
|
|
||||||
}
|
|
||||||
|
|
||||||
func coverageEvidence(b *strings.Builder, evidence string) {
|
|
||||||
if strings.TrimSpace(evidence) != "" {
|
|
||||||
b.WriteString("\n " + Dim().Render(psanitize(strings.TrimSpace(evidence), 160)))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func coverageArgsPreview(b *strings.Builder, name string, args map[string]any) {
|
|
||||||
if name == "list_coverage" {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if subject := coverageSubject(args, map[string]any{}); subject != "" {
|
|
||||||
b.WriteString("\n " + subject)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func coverageListBody(b *strings.Builder, result map[string]any) {
|
|
||||||
entries, _ := result["entries"].([]any)
|
|
||||||
total, _ := NumericValue(result["total_count"])
|
|
||||||
if len(entries) == 0 {
|
|
||||||
if int(total) == 0 {
|
|
||||||
b.WriteString("\n " + Dim().Render("No surfaces recorded yet"))
|
|
||||||
} else {
|
|
||||||
b.WriteString("\n " + Dim().Render("No surfaces match this filter"))
|
|
||||||
}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if counts, ok := result["outcome_counts"].(map[string]any); ok && len(counts) > 0 {
|
|
||||||
var parts []string
|
|
||||||
for _, outcome := range []string{
|
|
||||||
"reported", "no_issue_found", "ruled_out", "not_applicable", "needs_follow_up",
|
|
||||||
} {
|
|
||||||
count, ok := NumericValue(counts[outcome])
|
|
||||||
if !ok || count == 0 {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
_, label, color := coverageOutcome(outcome)
|
|
||||||
parts = append(parts, Col(color).Render(label+": "+strconv.Itoa(int(count))))
|
|
||||||
}
|
|
||||||
if len(parts) > 0 {
|
|
||||||
b.WriteString("\n " + strings.Join(parts, Dim().Render(" ")))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, e := range entries {
|
|
||||||
entry, _ := e.(map[string]any)
|
|
||||||
marker, label, color := coverageOutcome(StringValue(entry["outcome"]))
|
|
||||||
surface := strings.TrimSpace(StringValue(entry["surface"]))
|
|
||||||
if surface == "" {
|
|
||||||
surface = "(unnamed surface)"
|
|
||||||
}
|
|
||||||
b.WriteString("\n " + Col(color).Render(marker) + " " + surface)
|
|
||||||
if risk := strings.TrimSpace(StringValue(entry["risk_area"])); risk != "" {
|
|
||||||
b.WriteString(Dim().Render(" · " + risk))
|
|
||||||
}
|
|
||||||
b.WriteString("\n " + Col(color).Render(label))
|
|
||||||
// A row that moved states carries its own history; showing it keeps a
|
|
||||||
// closed surface from reading as one that was never in question.
|
|
||||||
if previous, ok := entry["previous_outcomes"].([]any); ok && len(previous) > 0 {
|
|
||||||
var was []string
|
|
||||||
for _, p := range previous {
|
|
||||||
if _, label, _ := coverageOutcome(StringValue(p)); label != "" {
|
|
||||||
was = append(was, label)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if len(was) > 0 {
|
|
||||||
b.WriteString(Dim().Render(" (was " + strings.Join(was, " → ") + ")"))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
// Whose row this is matters for reconciliation: an agent needs to see
|
|
||||||
// at a glance which surfaces it owns and which came from a sibling.
|
|
||||||
if truthy(entry["by_you"]) {
|
|
||||||
b.WriteString(Dim().Render(" · you"))
|
|
||||||
} else if who := strings.TrimSpace(StringValue(entry["agent_name"])); who != "" {
|
|
||||||
b.WriteString(Dim().Render(" · " + who))
|
|
||||||
}
|
|
||||||
coverageEvidence(b, StringValue(entry["evidence"]))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,204 +0,0 @@
|
|||||||
package render
|
|
||||||
|
|
||||||
import (
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"github.com/charmbracelet/x/ansi"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestRecordCoverageRendersSurfaceAndOutcome(t *testing.T) {
|
|
||||||
out := ansi.Strip(Tool(tool("record_coverage",
|
|
||||||
map[string]any{
|
|
||||||
"surface": "POST /api/v1/invoices",
|
|
||||||
"risk_area": "object-level authorization",
|
|
||||||
"evidence": "tenant B token returns 403 on tenant A invoice ids",
|
|
||||||
},
|
|
||||||
map[string]any{"success": true, "entry_id": "a1b2c3", "outcome": "ruled_out"},
|
|
||||||
"completed")))
|
|
||||||
requireContains(t, out,
|
|
||||||
"Coverage Recorded",
|
|
||||||
"POST /api/v1/invoices",
|
|
||||||
"object-level authorization",
|
|
||||||
"ruled out",
|
|
||||||
"tenant B token returns 403",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestUpdateCoverageShowsStateTransition(t *testing.T) {
|
|
||||||
out := ansi.Strip(Tool(tool("update_coverage",
|
|
||||||
map[string]any{"entry_id": "a1b2c3", "evidence": "reproduced with a second tenant"},
|
|
||||||
map[string]any{
|
|
||||||
"success": true,
|
|
||||||
"entry_id": "a1b2c3",
|
|
||||||
"previous_outcome": "needs_follow_up",
|
|
||||||
"outcome": "reported",
|
|
||||||
},
|
|
||||||
"completed")))
|
|
||||||
requireContains(t, out, "Coverage Updated", "needs follow-up", "→", "reported")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestListCoverageRendersCountsHistoryAndAuthor(t *testing.T) {
|
|
||||||
out := ansi.Strip(Tool(tool("list_coverage", nil,
|
|
||||||
map[string]any{
|
|
||||||
"success": true,
|
|
||||||
"entries": []any{
|
|
||||||
map[string]any{
|
|
||||||
"entry_id": "a1b2c3",
|
|
||||||
"surface": "/admin/export",
|
|
||||||
"risk_area": "IDOR",
|
|
||||||
"outcome": "no_issue_found",
|
|
||||||
"agent_name": "AuthzAgent",
|
|
||||||
"previous_outcomes": []any{"needs_follow_up"},
|
|
||||||
"evidence": "org id is server-derived from the session",
|
|
||||||
},
|
|
||||||
map[string]any{
|
|
||||||
"entry_id": "d4e5f6",
|
|
||||||
"surface": "/graphql",
|
|
||||||
"risk_area": "injection",
|
|
||||||
"outcome": "needs_follow_up",
|
|
||||||
"by_you": true,
|
|
||||||
"evidence": "introspection disabled; needs an authenticated schema dump",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"total_count": 2,
|
|
||||||
"outcome_counts": map[string]any{"no_issue_found": 1, "needs_follow_up": 1},
|
|
||||||
},
|
|
||||||
"completed")))
|
|
||||||
requireContains(t, out,
|
|
||||||
"/admin/export", "IDOR", "no issue found",
|
|
||||||
"was needs follow-up", "AuthzAgent",
|
|
||||||
"/graphql", "needs follow-up", "you",
|
|
||||||
"no issue found: 1", "needs follow-up: 1",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestListCoverageEmptyLedgerReadsAsUnrecorded(t *testing.T) {
|
|
||||||
out := ansi.Strip(Tool(tool("list_coverage", nil,
|
|
||||||
map[string]any{"success": true, "entries": []any{}, "total_count": 0}, "completed")))
|
|
||||||
requireContains(t, out, "No surfaces recorded yet")
|
|
||||||
|
|
||||||
filtered := ansi.Strip(Tool(tool("list_coverage",
|
|
||||||
map[string]any{"outcome": "reported"},
|
|
||||||
map[string]any{"success": true, "entries": []any{}, "total_count": 4}, "completed")))
|
|
||||||
requireContains(t, filtered, "No surfaces match this filter")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCoverageDuplicateRejectionSurfacesTheError(t *testing.T) {
|
|
||||||
out := ansi.Strip(Tool(tool("record_coverage",
|
|
||||||
map[string]any{"surface": "/login", "risk_area": "XSS"},
|
|
||||||
map[string]any{
|
|
||||||
"success": false,
|
|
||||||
"error": "'/login' (XSS) already has coverage entry a1b2c3",
|
|
||||||
"existing_entry_id": "a1b2c3",
|
|
||||||
},
|
|
||||||
"completed")))
|
|
||||||
requireContains(t, out, "/login", "already has coverage entry a1b2c3")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGetThreatModelRendersStalenessAndAmendments(t *testing.T) {
|
|
||||||
out := ansi.Strip(Tool(tool("get_threat_model",
|
|
||||||
map[string]any{"target": "https://app.example.com"},
|
|
||||||
map[string]any{
|
|
||||||
"success": true,
|
|
||||||
"found": true,
|
|
||||||
"stale": true,
|
|
||||||
"cached_revision": "0123456789abcdef",
|
|
||||||
"content": "# Overview\nMulti-tenant billing app.\n\n" +
|
|
||||||
"## Trust Boundaries and Assumptions\n\n## Attack Surface\n",
|
|
||||||
"amendments": []any{
|
|
||||||
map[string]any{
|
|
||||||
"agent_name": "ReconAgent",
|
|
||||||
"content": "staging host shares the production database",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"completed")))
|
|
||||||
requireContains(t, out,
|
|
||||||
"Threat Model", "https://app.example.com",
|
|
||||||
"stale", "01234567",
|
|
||||||
"1 amendment(s)", "ReconAgent", "staging host shares the production database",
|
|
||||||
"Multi-tenant billing app.", "Overview", "Trust Boundaries and Assumptions",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGetThreatModelMissingModelIsExplicit(t *testing.T) {
|
|
||||||
out := ansi.Strip(Tool(tool("get_threat_model",
|
|
||||||
map[string]any{"target": "10.0.0.5"},
|
|
||||||
map[string]any{"success": true, "found": false}, "completed")))
|
|
||||||
requireContains(t, out, "No model cached for this target yet")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestSaveThreatModelWarnsWhenAmendmentsAreCleared(t *testing.T) {
|
|
||||||
out := ansi.Strip(Tool(tool("save_threat_model",
|
|
||||||
map[string]any{"target": "app.example.com", "content": "# Overview\nA thing.\n"},
|
|
||||||
map[string]any{
|
|
||||||
"success": true,
|
|
||||||
"revision": "unversioned",
|
|
||||||
"amendments_cleared": 2,
|
|
||||||
},
|
|
||||||
"completed")))
|
|
||||||
requireContains(t, out, "Threat Model Saved", "saved", "cleared 2 amendment(s)")
|
|
||||||
// An unversioned target has no revision worth printing.
|
|
||||||
if strings.Contains(out, "unversioned") {
|
|
||||||
t.Fatalf("unversioned revision should not be rendered:\n%s", out)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAmendThreatModelRendersAddendum(t *testing.T) {
|
|
||||||
out := ansi.Strip(Tool(tool("amend_threat_model",
|
|
||||||
map[string]any{
|
|
||||||
"target": "app.example.com",
|
|
||||||
"addendum": "The admin role is assignable by any org member via PATCH /members.",
|
|
||||||
},
|
|
||||||
map[string]any{"success": true, "amendment_count": 3}, "completed")))
|
|
||||||
requireContains(t, out, "Threat Model Amended", "amendment recorded", "(3 total)",
|
|
||||||
"admin role is assignable")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCoverageAndThreatModelToolsAreNotGeneric(t *testing.T) {
|
|
||||||
// The generic fallback dumps raw arg keys; these tools must not reach it.
|
|
||||||
for _, name := range []string{
|
|
||||||
"record_coverage", "update_coverage", "list_coverage",
|
|
||||||
"get_threat_model", "save_threat_model", "amend_threat_model",
|
|
||||||
} {
|
|
||||||
out := ansi.Strip(Tool(tool(name, map[string]any{"target": "x", "surface": "y"}, nil, "running")))
|
|
||||||
if strings.Contains(out, "Using tool") {
|
|
||||||
t.Fatalf("%s fell through to the generic renderer:\n%s", name, out)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestOutputHeavyCoverageToolsCollapse(t *testing.T) {
|
|
||||||
for _, name := range []string{"list_coverage", "get_threat_model"} {
|
|
||||||
if ToolPreviewLines(name) == 0 {
|
|
||||||
t.Fatalf("%s should collapse; its output is unbounded", name)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
for _, name := range []string{"record_coverage", "amend_threat_model"} {
|
|
||||||
if ToolPreviewLines(name) != 0 {
|
|
||||||
t.Fatalf("%s should not collapse", name)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestVulnerabilityReportRendersCalibrationFields(t *testing.T) {
|
|
||||||
out := ansi.Strip(Tool(tool("create_vulnerability_report",
|
|
||||||
map[string]any{
|
|
||||||
"title": "IDOR in invoice export",
|
|
||||||
"confidence": "medium",
|
|
||||||
"confidence_rationale": "traced statically; no authenticated instance to replay against",
|
|
||||||
"counterevidence": "the gateway may strip the id parameter before it reaches the handler",
|
|
||||||
"severity_change_conditions": "critical if the export includes other tenants' bank details",
|
|
||||||
"fix_verification": "unit tests executed; bypass review reasoned only",
|
|
||||||
"description": "The handler trusts a client-supplied invoice id.",
|
|
||||||
},
|
|
||||||
map[string]any{"success": true, "severity": "high", "cvss_score": 7.5},
|
|
||||||
"completed")))
|
|
||||||
requireContains(t, out,
|
|
||||||
"Confidence", "MEDIUM", "no authenticated instance to replay against",
|
|
||||||
"Counterevidence", "gateway may strip the id parameter",
|
|
||||||
"Severity Would Change If", "other tenants' bank details",
|
|
||||||
"Fix Verification", "bypass review reasoned only",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
@@ -72,19 +72,6 @@ func TestNonTablePipeLinesAreLeftAlone(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestMarkdownOrderedListsUseSingleSpaceAfterMarker(t *testing.T) {
|
|
||||||
out := renderAssistantMarkdown("1. hello\n2) world")
|
|
||||||
plain := ansi.Strip(out)
|
|
||||||
for _, want := range []string{"1. hello", "2) world"} {
|
|
||||||
if !strings.Contains(plain, want) {
|
|
||||||
t.Fatalf("ordered list item %q missing: %q", want, plain)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if strings.Contains(plain, "1. hello") || strings.Contains(plain, "2) world") {
|
|
||||||
t.Fatalf("double space after the list marker: %q", plain)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestInlineFormatKeepsNonEmphasisMarkers(t *testing.T) {
|
func TestInlineFormatKeepsNonEmphasisMarkers(t *testing.T) {
|
||||||
literal := []string{
|
literal := []string{
|
||||||
"ls *.py *.go",
|
"ls *.py *.go",
|
||||||
|
|||||||
@@ -1,35 +0,0 @@
|
|||||||
package render
|
|
||||||
|
|
||||||
import (
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// MCP tools (tools from the servers the user connected)
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
const mcpIcon = "🔌 "
|
|
||||||
|
|
||||||
// renderMcpTool renders a call to a tool from one of the user's MCP servers.
|
|
||||||
//
|
|
||||||
// Its own icon and color so a call that left Strix for a server the user
|
|
||||||
// connected is obvious while scrolling a transcript. The action leads and the
|
|
||||||
// server trails: the model-facing name is the connection name and the tool name
|
|
||||||
// stuck together, so leading with the whole name buries the part a reader wants
|
|
||||||
// behind a connection name that can be long or opaque.
|
|
||||||
//
|
|
||||||
// The result is deliberately not rendered, for the same reason
|
|
||||||
// renderGenericTool leaves it out: an MCP result is whatever an outside server
|
|
||||||
// chose to return, often multi-kilobyte JSON, and it floods the screen. The full
|
|
||||||
// result is in the event data, the run log, and the `strix view` viewer.
|
|
||||||
func renderMcpTool(connection, toolName string, args map[string]any, status string) string {
|
|
||||||
var b strings.Builder
|
|
||||||
b.WriteString(mcpIcon + Bold(Mint).Render(toolName))
|
|
||||||
b.WriteString(Dim().Render(" via MCP server ") + Col(Slate).Render(connection) + "\n")
|
|
||||||
for _, k := range SortedKeys(args) {
|
|
||||||
b.WriteString(" " + Dim().Render(k) + ": " + StringValue(args[k]) + "\n")
|
|
||||||
}
|
|
||||||
icon, style := statusIcon(status)
|
|
||||||
b.WriteString(style.Render(icon))
|
|
||||||
return b.String()
|
|
||||||
}
|
|
||||||
@@ -22,20 +22,19 @@ func statusIcon(status string) (string, lipgloss.Style) {
|
|||||||
return "○ Unknown", Dim()
|
return "○ Unknown", Dim()
|
||||||
}
|
}
|
||||||
|
|
||||||
// renderGenericTool ports registry._render_default_tool_widget. It shows the
|
// renderGenericTool ports registry._render_default_tool_widget.
|
||||||
// tool name, its arguments, and a status line only. The raw result is
|
func renderGenericTool(name string, args map[string]any, result any, status string) string {
|
||||||
// deliberately not rendered: a generic result (e.g. a multi-kilobyte JSON
|
|
||||||
// payload from a database query tool) is noise on screen, and the agent narrates
|
|
||||||
// what it got in its next message. The full result still lives in the event
|
|
||||||
// data, the run log, and the `strix view` viewer.
|
|
||||||
func renderGenericTool(name string, args map[string]any, status string) string {
|
|
||||||
var b strings.Builder
|
var b strings.Builder
|
||||||
b.WriteString(Dim().Render("→ Using tool ") + Bold(Blue).Render(name) + "\n")
|
b.WriteString(Dim().Render("→ Using tool ") + Bold(Blue).Render(name) + "\n")
|
||||||
for _, k := range SortedKeys(args) {
|
for _, k := range SortedKeys(args) {
|
||||||
b.WriteString(" " + Dim().Render(k) + ": " + StringValue(args[k]) + "\n")
|
b.WriteString(" " + Dim().Render(k) + ": " + StringValue(args[k]) + "\n")
|
||||||
}
|
}
|
||||||
|
if (status == "completed" || status == "failed" || status == "error") && result != nil {
|
||||||
|
b.WriteString(lipgloss.NewStyle().Bold(true).Render("Result: ") + StringValue(result))
|
||||||
|
} else {
|
||||||
icon, style := statusIcon(status)
|
icon, style := statusIcon(status)
|
||||||
b.WriteString(style.Render(icon))
|
b.WriteString(style.Render(icon))
|
||||||
|
}
|
||||||
return b.String()
|
return b.String()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -52,18 +51,6 @@ func Tool(data map[string]any) string {
|
|||||||
}
|
}
|
||||||
result := data["result"]
|
result := data["result"]
|
||||||
|
|
||||||
// A call to a tool from one of the user's MCP servers is tagged with the
|
|
||||||
// connection it came from, because its name is the server's own and means
|
|
||||||
// nothing here. The tag is only ever set from the connections the run made,
|
|
||||||
// so it is the one thing that can tell such a call apart from a built-in.
|
|
||||||
if connection := StringValue(data["mcp_connection"]); connection != "" {
|
|
||||||
toolName := StringValue(data["mcp_tool"])
|
|
||||||
if toolName == "" {
|
|
||||||
toolName = name
|
|
||||||
}
|
|
||||||
return renderMcpTool(connection, toolName, args, status)
|
|
||||||
}
|
|
||||||
|
|
||||||
switch name {
|
switch name {
|
||||||
case "exec_command":
|
case "exec_command":
|
||||||
return renderExecCommand(args, result, status)
|
return renderExecCommand(args, result, status)
|
||||||
@@ -95,16 +82,12 @@ func Tool(data map[string]any) string {
|
|||||||
return renderNote(name, args, result)
|
return renderNote(name, args, result)
|
||||||
case "create_todo", "list_todos", "update_todo", "mark_todo_done", "mark_todo_pending", "delete_todo":
|
case "create_todo", "list_todos", "update_todo", "mark_todo_done", "mark_todo_pending", "delete_todo":
|
||||||
return renderTodo(name, result)
|
return renderTodo(name, result)
|
||||||
case "record_coverage", "update_coverage", "list_coverage":
|
|
||||||
return renderCoverage(name, args, result)
|
|
||||||
case "get_threat_model", "save_threat_model", "amend_threat_model":
|
|
||||||
return renderThreatModel(name, args, result)
|
|
||||||
case "view_agent_graph", "create_agent", "send_message_to_agent", "agent_finish", "wait_for_agents", "stop_agent":
|
case "view_agent_graph", "create_agent", "send_message_to_agent", "agent_finish", "wait_for_agents", "stop_agent":
|
||||||
return renderAgentGraphTool(name, args, result)
|
return renderAgentGraphTool(name, args, result)
|
||||||
case "list_requests", "view_request", "repeat_request", "list_sitemap", "view_sitemap_entry", "scope_rules":
|
case "list_requests", "view_request", "repeat_request", "list_sitemap", "view_sitemap_entry", "scope_rules":
|
||||||
return renderProxyTool(name, args, result, status)
|
return renderProxyTool(name, args, result, status)
|
||||||
}
|
}
|
||||||
return renderGenericTool(name, args, status)
|
return renderGenericTool(name, args, result, status)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -120,8 +103,7 @@ const outputPreviewLines = 10
|
|||||||
func ToolPreviewLines(name string) int {
|
func ToolPreviewLines(name string) int {
|
||||||
switch name {
|
switch name {
|
||||||
case "exec_command", "write_stdin", "apply_patch",
|
case "exec_command", "write_stdin", "apply_patch",
|
||||||
"view_request", "repeat_request", "view_sitemap_entry",
|
"view_request", "repeat_request", "view_sitemap_entry":
|
||||||
"list_coverage", "get_threat_model":
|
|
||||||
return outputPreviewLines
|
return outputPreviewLines
|
||||||
}
|
}
|
||||||
return 0
|
return 0
|
||||||
|
|||||||
@@ -203,7 +203,7 @@ func TestToolDispatchCoversKnownTools(t *testing.T) {
|
|||||||
{
|
{
|
||||||
"unknown tool falls back to generic",
|
"unknown tool falls back to generic",
|
||||||
tool("brand_new_tool", map[string]any{"alpha": "1"}, "done", "completed"),
|
tool("brand_new_tool", map[string]any{"alpha": "1"}, "done", "completed"),
|
||||||
[]string{"brand_new_tool", "alpha", "Done"},
|
[]string{"brand_new_tool", "alpha", "Result:", "done"},
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -214,43 +214,6 @@ func TestToolDispatchCoversKnownTools(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestGenericToolOmitsRawResult(t *testing.T) {
|
|
||||||
// The generic renderer shows tool name, args, and a status line only, never
|
|
||||||
// the raw result payload.
|
|
||||||
long := strings.Repeat("x", 5000)
|
|
||||||
out := ansi.Strip(Tool(tool("db_query", map[string]any{"query": "select 1"}, long, "completed")))
|
|
||||||
|
|
||||||
requireContains(t, out, "db_query", "query", "Done")
|
|
||||||
if strings.Contains(out, "Result:") || strings.Contains(out, strings.Repeat("x", 20)) {
|
|
||||||
t.Fatalf("generic result body must not be rendered:\n%s", out)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestMcpToolLeadsWithActionAndNamesTheServer(t *testing.T) {
|
|
||||||
data := tool("local_fs_read_file", map[string]any{"path": "/etc/hosts"}, "file body", "completed")
|
|
||||||
data["mcp_connection"] = "local_fs"
|
|
||||||
data["mcp_tool"] = "read_file"
|
|
||||||
|
|
||||||
out := ansi.Strip(Tool(data))
|
|
||||||
|
|
||||||
// The action leads; the server is context that trails it.
|
|
||||||
if !strings.HasPrefix(out, mcpIcon+"read_file") {
|
|
||||||
t.Fatalf("MCP render must lead with the tool's own name:\n%s", out)
|
|
||||||
}
|
|
||||||
requireContains(t, out, "local_fs", "path", "/etc/hosts", "Done")
|
|
||||||
// Untrusted server output stays off the terminal, as for the generic render.
|
|
||||||
if strings.Contains(out, "file body") {
|
|
||||||
t.Fatalf("MCP result body must not be rendered:\n%s", out)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestMcpToolWithoutTaggedNameFallsBackToFullName(t *testing.T) {
|
|
||||||
data := tool("local_fs_read_file", nil, nil, "running")
|
|
||||||
data["mcp_connection"] = "local_fs"
|
|
||||||
|
|
||||||
requireContains(t, ansi.Strip(Tool(data)), "local_fs_read_file", "In progress")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCollapseToolShellPreviewAndExpand(t *testing.T) {
|
func TestCollapseToolShellPreviewAndExpand(t *testing.T) {
|
||||||
lines := make([]string, 16)
|
lines := make([]string, 16)
|
||||||
for i := range lines {
|
for i := range lines {
|
||||||
|
|||||||
@@ -50,31 +50,15 @@ func renderVulnerabilityReport(args map[string]any, result any) string {
|
|||||||
b.WriteString("\n\n" + Bold(Field).Render(label) + "\n" + value)
|
b.WriteString("\n\n" + Bold(Field).Render(label) + "\n" + value)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if confidence := StringValue(args["confidence"]); confidence != "" {
|
|
||||||
b.WriteString("\n\n" + Bold(Field).Render("Confidence: ") +
|
|
||||||
lipgloss.NewStyle().Bold(true).Foreground(confidenceColor(confidence)).
|
|
||||||
Render(strings.ToUpper(confidence)))
|
|
||||||
if rationale := StringValue(args["confidence_rationale"]); rationale != "" {
|
|
||||||
b.WriteString("\n" + Dim().Render(rationale))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
section("Description", StringValue(args["description"]))
|
section("Description", StringValue(args["description"]))
|
||||||
section("Impact", StringValue(args["impact"]))
|
section("Impact", StringValue(args["impact"]))
|
||||||
section("Technical Analysis", StringValue(args["technical_analysis"]))
|
section("Technical Analysis", StringValue(args["technical_analysis"]))
|
||||||
// The case against the finding travels with the case for it: a reader
|
|
||||||
// triaging this needs both to judge whether to act.
|
|
||||||
section("Counterevidence", StringValue(args["counterevidence"]))
|
|
||||||
section("Severity Would Change If", StringValue(args["severity_change_conditions"]))
|
|
||||||
renderCodeLocations(&b, args["code_locations"])
|
renderCodeLocations(&b, args["code_locations"])
|
||||||
section("PoC Description", StringValue(args["poc_description"]))
|
section("PoC Description", StringValue(args["poc_description"]))
|
||||||
if poc := StringValue(args["poc_script_code"]); poc != "" {
|
if poc := StringValue(args["poc_script_code"]); poc != "" {
|
||||||
b.WriteString("\n\n" + Bold(Field).Render("PoC Code") + "\n" + Col(Text).Render(poc))
|
b.WriteString("\n\n" + Bold(Field).Render("PoC Code") + "\n" + Col(Text).Render(poc))
|
||||||
}
|
}
|
||||||
section("Remediation", StringValue(args["remediation_steps"]))
|
section("Remediation", StringValue(args["remediation_steps"]))
|
||||||
// Any applyable fix above is one click from the user's codebase, so how it
|
|
||||||
// was verified belongs next to it rather than in the artifact alone.
|
|
||||||
section("Fix Verification", StringValue(args["fix_verification"]))
|
|
||||||
|
|
||||||
if title == "" {
|
if title == "" {
|
||||||
b.WriteString("\n " + Dim().Render("Creating report..."))
|
b.WriteString("\n " + Dim().Render("Creating report..."))
|
||||||
@@ -82,20 +66,6 @@ func renderVulnerabilityReport(args map[string]any, result any) string {
|
|||||||
return "\n\n" + b.String() + "\n\n"
|
return "\n\n" + b.String() + "\n\n"
|
||||||
}
|
}
|
||||||
|
|
||||||
// confidenceColor grades how firm the agent's own call is. Anything below
|
|
||||||
// high is a claim the reader has to check, and should not read as settled.
|
|
||||||
func confidenceColor(confidence string) lipgloss.Color {
|
|
||||||
switch strings.ToLower(strings.TrimSpace(confidence)) {
|
|
||||||
case "high":
|
|
||||||
return Green
|
|
||||||
case "medium":
|
|
||||||
return SevMed
|
|
||||||
case "low":
|
|
||||||
return SevHigh
|
|
||||||
}
|
|
||||||
return Gray
|
|
||||||
}
|
|
||||||
|
|
||||||
var cvssKeys = [][2]string{
|
var cvssKeys = [][2]string{
|
||||||
{"attack_vector", "AV"}, {"attack_complexity", "AC"}, {"privileges_required", "PR"},
|
{"attack_vector", "AV"}, {"attack_complexity", "AC"}, {"privileges_required", "PR"},
|
||||||
{"user_interaction", "UI"}, {"scope", "S"}, {"confidentiality", "C"},
|
{"user_interaction", "UI"}, {"scope", "S"}, {"confidentiality", "C"},
|
||||||
|
|||||||
@@ -1,138 +0,0 @@
|
|||||||
package render
|
|
||||||
|
|
||||||
import (
|
|
||||||
"strconv"
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// Threat model (get_threat_model / save_threat_model / amend_threat_model)
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
var threatModelTitles = map[string]struct {
|
|
||||||
title string
|
|
||||||
loading string
|
|
||||||
errMsg string
|
|
||||||
}{
|
|
||||||
"get_threat_model": {"Threat Model", "Loading...", "Unable to read threat model"},
|
|
||||||
"save_threat_model": {"Threat Model Saved", "Saving...", "Failed to save threat model"},
|
|
||||||
"amend_threat_model": {"Threat Model Amended", "Amending...", "Failed to amend threat model"},
|
|
||||||
}
|
|
||||||
|
|
||||||
func renderThreatModel(name string, args map[string]any, result any) string {
|
|
||||||
meta := threatModelTitles[name]
|
|
||||||
var b strings.Builder
|
|
||||||
b.WriteString("⌖ " + Bold(InfoBlue).Render(meta.title))
|
|
||||||
if target := strings.TrimSpace(StringValue(args["target"])); target != "" {
|
|
||||||
b.WriteString(Dim().Render(" " + target))
|
|
||||||
}
|
|
||||||
|
|
||||||
if s, ok := result.(string); ok && strings.TrimSpace(s) != "" {
|
|
||||||
b.WriteString("\n " + Dim().Render(strings.TrimSpace(s)))
|
|
||||||
return b.String()
|
|
||||||
}
|
|
||||||
m, ok := result.(map[string]any)
|
|
||||||
if !ok {
|
|
||||||
b.WriteString("\n " + Dim().Render(meta.loading))
|
|
||||||
return b.String()
|
|
||||||
}
|
|
||||||
if !truthy(m["success"]) {
|
|
||||||
errMsg := StringValue(m["error"])
|
|
||||||
if errMsg == "" {
|
|
||||||
errMsg = meta.errMsg
|
|
||||||
}
|
|
||||||
b.WriteString("\n " + Col(Red).Render(errMsg))
|
|
||||||
return b.String()
|
|
||||||
}
|
|
||||||
|
|
||||||
switch name {
|
|
||||||
case "get_threat_model":
|
|
||||||
threatModelReadBody(&b, m)
|
|
||||||
case "amend_threat_model":
|
|
||||||
b.WriteString("\n " + Col(Green).Render("✓ amendment recorded"))
|
|
||||||
if count, ok := NumericValue(m["amendment_count"]); ok {
|
|
||||||
b.WriteString(Dim().Render(" (" + strconv.Itoa(int(count)) + " total)"))
|
|
||||||
}
|
|
||||||
threatModelBody(&b, StringValue(args["addendum"]))
|
|
||||||
default:
|
|
||||||
b.WriteString("\n " + Col(Green).Render("✓ saved"))
|
|
||||||
if revision := shortRevision(StringValue(m["revision"])); revision != "" {
|
|
||||||
b.WriteString(Dim().Render(" at " + revision))
|
|
||||||
}
|
|
||||||
// Saving folds amendments away, so the count that vanished is worth
|
|
||||||
// stating: it is the one destructive thing this tool does.
|
|
||||||
if cleared, ok := NumericValue(m["amendments_cleared"]); ok && cleared > 0 {
|
|
||||||
b.WriteString("\n " + Col(AmberY).Render("⚠ cleared "+
|
|
||||||
strconv.Itoa(int(cleared))+" amendment(s)"))
|
|
||||||
}
|
|
||||||
threatModelBody(&b, StringValue(args["content"]))
|
|
||||||
}
|
|
||||||
return b.String()
|
|
||||||
}
|
|
||||||
|
|
||||||
func threatModelReadBody(b *strings.Builder, result map[string]any) {
|
|
||||||
if !truthy(result["found"]) {
|
|
||||||
b.WriteString("\n " + Dim().Render("No model cached for this target yet"))
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if truthy(result["stale"]) {
|
|
||||||
b.WriteString("\n " + Col(AmberY).Render("⚠ stale"))
|
|
||||||
if cached := shortRevision(StringValue(result["cached_revision"])); cached != "" {
|
|
||||||
b.WriteString(Dim().Render(" (written at " + cached + ")"))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if amendments, ok := result["amendments"].([]any); ok && len(amendments) > 0 {
|
|
||||||
b.WriteString("\n " + Col(Gold).Render("+ "+strconv.Itoa(len(amendments))+
|
|
||||||
" amendment(s)") + Dim().Render(" — later statements win"))
|
|
||||||
for _, a := range amendments {
|
|
||||||
amendment, _ := a.(map[string]any)
|
|
||||||
who := strings.TrimSpace(StringValue(amendment["agent_name"]))
|
|
||||||
if who == "" {
|
|
||||||
who = "unknown agent"
|
|
||||||
}
|
|
||||||
b.WriteString("\n - " + Dim().Render(who+": ") +
|
|
||||||
psanitize(strings.TrimSpace(StringValue(amendment["content"])), 120))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
threatModelBody(b, StringValue(result["content"]))
|
|
||||||
}
|
|
||||||
|
|
||||||
// threatModelBody previews the document. The full text is a page or more, so
|
|
||||||
// only its section headings and opening line are shown here; the trace can be
|
|
||||||
// expanded for the rest.
|
|
||||||
func threatModelBody(b *strings.Builder, content string) {
|
|
||||||
content = strings.TrimSpace(content)
|
|
||||||
if content == "" {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
var headings []string
|
|
||||||
summary := ""
|
|
||||||
for _, line := range strings.Split(content, "\n") {
|
|
||||||
line = strings.TrimSpace(line)
|
|
||||||
switch {
|
|
||||||
case strings.HasPrefix(line, "#"):
|
|
||||||
headings = append(headings, strings.TrimSpace(strings.TrimLeft(line, "# ")))
|
|
||||||
case summary == "" && line != "":
|
|
||||||
summary = line
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if summary != "" {
|
|
||||||
b.WriteString("\n " + Dim().Render(psanitize(summary, 160)))
|
|
||||||
}
|
|
||||||
if len(headings) > 0 {
|
|
||||||
if len(headings) > 8 {
|
|
||||||
headings = headings[:8]
|
|
||||||
}
|
|
||||||
b.WriteString("\n " + Dim().Render(strings.Join(headings, " · ")))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// shortRevision abbreviates a git sha; "unversioned" targets have no revision
|
|
||||||
// worth showing.
|
|
||||||
func shortRevision(revision string) string {
|
|
||||||
revision = strings.TrimSpace(revision)
|
|
||||||
if revision == "" || revision == "unversioned" {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
return firstN(revision, 8)
|
|
||||||
}
|
|
||||||
@@ -8,7 +8,6 @@ from typing import TYPE_CHECKING, Any
|
|||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from collections.abc import Iterable
|
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from agents.tool import ToolOutputImage
|
from agents.tool import ToolOutputImage
|
||||||
@@ -16,10 +15,6 @@ from agents.tool import ToolOutputImage
|
|||||||
from strix.core.paths import runtime_state_dir
|
from strix.core.paths import runtime_state_dir
|
||||||
from strix.interface.tui.history import load_session_history
|
from strix.interface.tui.history import load_session_history
|
||||||
|
|
||||||
# Imported from the naming module rather than the mcp package so a projection
|
|
||||||
# never pulls in the MCP client and the agents SDK behind it.
|
|
||||||
from strix.tools.mcp.naming import resolve_mcp_tool
|
|
||||||
|
|
||||||
|
|
||||||
class TuiLiveView:
|
class TuiLiveView:
|
||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
@@ -31,27 +26,6 @@ class TuiLiveView:
|
|||||||
self._user_instruction: str | None = None
|
self._user_instruction: str | None = None
|
||||||
self._user_instruction_at: str | None = None
|
self._user_instruction_at: str | None = None
|
||||||
self._user_instruction_shown = False
|
self._user_instruction_shown = False
|
||||||
self._mcp_connections: tuple[str, ...] = ()
|
|
||||||
|
|
||||||
def set_mcp_connections(self, names: Iterable[str]) -> None:
|
|
||||||
"""The MCP servers this run connected, so its tool calls can name theirs.
|
|
||||||
|
|
||||||
A server's tools are offered to the model under a name built from the
|
|
||||||
connection name and the tool's own name. That name cannot be split back
|
|
||||||
apart on its own, so tool calls are matched against these names instead.
|
|
||||||
"""
|
|
||||||
self._mcp_connections = tuple(str(name) for name in names)
|
|
||||||
|
|
||||||
def _mcp_tool_fields(self, tool_name: str) -> dict[str, str]:
|
|
||||||
"""Event fields naming the MCP server a tool call went out to, if any.
|
|
||||||
|
|
||||||
Empty for every built-in tool, which is what tells an interface to render
|
|
||||||
the call as one of its own rather than as a call to a user's server.
|
|
||||||
"""
|
|
||||||
origin = resolve_mcp_tool(tool_name, self._mcp_connections)
|
|
||||||
if origin is None:
|
|
||||||
return {}
|
|
||||||
return {"mcp_connection": origin.connection, "mcp_tool": origin.tool}
|
|
||||||
|
|
||||||
def set_user_instruction(self, text: str | None, *, timestamp: str | None = None) -> None:
|
def set_user_instruction(self, text: str | None, *, timestamp: str | None = None) -> None:
|
||||||
"""Open the transcript with what the user asked for.
|
"""Open the transcript with what the user asked for.
|
||||||
@@ -98,9 +72,8 @@ class TuiLiveView:
|
|||||||
|
|
||||||
def hydrate_from_run_dir(self, run_dir: Path) -> None:
|
def hydrate_from_run_dir(self, run_dir: Path) -> None:
|
||||||
# Armed before the agents are added so the root agent's arrival puts the
|
# Armed before the agents are added so the root agent's arrival puts the
|
||||||
# user's opening message ahead of the replayed history, and before the
|
# user's opening message ahead of the replayed history.
|
||||||
# history is replayed so its MCP tool calls are attributed too.
|
self._load_user_instruction(run_dir)
|
||||||
self._load_run_record(run_dir)
|
|
||||||
state_dir = runtime_state_dir(run_dir)
|
state_dir = runtime_state_dir(run_dir)
|
||||||
agents_path = state_dir / "agents.json"
|
agents_path = state_dir / "agents.json"
|
||||||
if not agents_path.exists():
|
if not agents_path.exists():
|
||||||
@@ -127,17 +100,14 @@ class TuiLiveView:
|
|||||||
self.flush_user_instruction()
|
self.flush_user_instruction()
|
||||||
self._hydrate_sdk_session_history(run_dir, statuses.keys())
|
self._hydrate_sdk_session_history(run_dir, statuses.keys())
|
||||||
|
|
||||||
def _load_run_record(self, run_dir: Path) -> None:
|
def _load_user_instruction(self, run_dir: Path) -> None:
|
||||||
"""Take the user's opening message and the run's MCP servers off the record."""
|
"""Take the user's opening message from the run record, if it has one."""
|
||||||
try:
|
try:
|
||||||
record = json.loads((run_dir / "run.json").read_text(encoding="utf-8"))
|
record = json.loads((run_dir / "run.json").read_text(encoding="utf-8"))
|
||||||
except (OSError, json.JSONDecodeError):
|
except (OSError, json.JSONDecodeError):
|
||||||
return
|
return
|
||||||
if not isinstance(record, dict):
|
if not isinstance(record, dict):
|
||||||
return
|
return
|
||||||
connections = record.get("mcp_connections")
|
|
||||||
if isinstance(connections, list):
|
|
||||||
self.set_mcp_connections(name for name in connections if isinstance(name, str))
|
|
||||||
instruction = record.get("user_instruction")
|
instruction = record.get("user_instruction")
|
||||||
if not isinstance(instruction, str):
|
if not isinstance(instruction, str):
|
||||||
return
|
return
|
||||||
@@ -348,7 +318,6 @@ class TuiLiveView:
|
|||||||
"status": "running",
|
"status": "running",
|
||||||
"agent_id": agent_id,
|
"agent_id": agent_id,
|
||||||
"call_id": call_id,
|
"call_id": call_id,
|
||||||
**self._mcp_tool_fields(call["tool_name"]),
|
|
||||||
}
|
}
|
||||||
if existing is None:
|
if existing is None:
|
||||||
event = self._append_event(agent_id, "tool", tool_data, timestamp=timestamp)
|
event = self._append_event(agent_id, "tool", tool_data, timestamp=timestamp)
|
||||||
@@ -380,7 +349,6 @@ class TuiLiveView:
|
|||||||
"status": "completed",
|
"status": "completed",
|
||||||
"agent_id": agent_id,
|
"agent_id": agent_id,
|
||||||
"call_id": call_id,
|
"call_id": call_id,
|
||||||
**self._mcp_tool_fields(output["tool_name"]),
|
|
||||||
},
|
},
|
||||||
timestamp=timestamp,
|
timestamp=timestamp,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -207,23 +207,9 @@ class GoTuiRuntime:
|
|||||||
self.controller.notify_changed()
|
self.controller.notify_changed()
|
||||||
|
|
||||||
def capture_event(self, agent_id: str, event: Any) -> None:
|
def capture_event(self, agent_id: str, event: Any) -> None:
|
||||||
self._refresh_mcp_connections()
|
|
||||||
self.live_view.ingest_sdk_event(agent_id, event)
|
self.live_view.ingest_sdk_event(agent_id, event)
|
||||||
self.controller.notify_changed()
|
self.controller.notify_changed()
|
||||||
|
|
||||||
def _refresh_mcp_connections(self) -> None:
|
|
||||||
"""Hand the projection the MCP servers the scan connected.
|
|
||||||
|
|
||||||
The scan records them as it connects, which is before the agent can call
|
|
||||||
anything, and the projection needs them to say which server a tool call
|
|
||||||
went out to. Read on the way in rather than pushed, so no tool call can
|
|
||||||
be projected before they arrive.
|
|
||||||
"""
|
|
||||||
if self.report_state is None:
|
|
||||||
return
|
|
||||||
connections = self.report_state.run_record.get("mcp_connections") or []
|
|
||||||
self.live_view.set_mcp_connections(connections)
|
|
||||||
|
|
||||||
async def _sync_agent_state(self) -> bool:
|
async def _sync_agent_state(self) -> bool:
|
||||||
parent_of, statuses, names, errors = await self.coordinator.graph_snapshot()
|
parent_of, statuses, names, errors = await self.coordinator.graph_snapshot()
|
||||||
changed = False
|
changed = False
|
||||||
|
|||||||
@@ -13,7 +13,9 @@ from pathlib import Path
|
|||||||
from typing import Any
|
from typing import Any
|
||||||
from urllib.parse import parse_qs, urlparse
|
from urllib.parse import parse_qs, urlparse
|
||||||
|
|
||||||
|
import docker
|
||||||
import requests
|
import requests
|
||||||
|
from docker.errors import DockerException, ImageNotFound
|
||||||
from rich.console import Console
|
from rich.console import Console
|
||||||
from rich.panel import Panel
|
from rich.panel import Panel
|
||||||
from rich.text import Text
|
from rich.text import Text
|
||||||
@@ -281,27 +283,9 @@ def is_subscription_run(report_state: Any) -> bool:
|
|||||||
record = getattr(report_state, "run_record", None)
|
record = getattr(report_state, "run_record", None)
|
||||||
if isinstance(record, dict) and record.get("auth_mode"):
|
if isinstance(record, dict) and record.get("auth_mode"):
|
||||||
return record.get("auth_mode") == "subscription"
|
return record.get("auth_mode") == "subscription"
|
||||||
from strix.config import opencode
|
from strix.config import codex
|
||||||
|
|
||||||
return opencode.auth_mode(load_settings().llm.model) == "subscription"
|
return codex.auth_mode(load_settings().llm.model) == "subscription"
|
||||||
|
|
||||||
|
|
||||||
def subscription_label() -> str:
|
|
||||||
"""Display name of the subscription behind the configured model."""
|
|
||||||
from strix.config import opencode
|
|
||||||
|
|
||||||
oc = opencode.subscription_model(load_settings().llm.model)
|
|
||||||
if oc:
|
|
||||||
return oc.label
|
|
||||||
return "ChatGPT subscription"
|
|
||||||
|
|
||||||
|
|
||||||
def subscription_is_metered() -> bool:
|
|
||||||
"""Whether the run spends per-request credits rather than a flat plan."""
|
|
||||||
from strix.config import opencode
|
|
||||||
|
|
||||||
oc = opencode.subscription_model(load_settings().llm.model)
|
|
||||||
return oc is not None and oc.metered
|
|
||||||
|
|
||||||
|
|
||||||
def _int_stat(usage: dict[str, Any], key: str) -> int:
|
def _int_stat(usage: dict[str, Any], key: str) -> int:
|
||||||
@@ -344,9 +328,7 @@ def _build_llm_usage_stats(
|
|||||||
if not usage or _int_stat(usage, "requests") <= 0:
|
if not usage or _int_stat(usage, "requests") <= 0:
|
||||||
stats_text.append("\n")
|
stats_text.append("\n")
|
||||||
stats_text.append("Cost ", style="dim")
|
stats_text.append("Cost ", style="dim")
|
||||||
if subscription and subscription_is_metered():
|
if subscription:
|
||||||
stats_text.append("credits ", style="#22c55e")
|
|
||||||
elif subscription:
|
|
||||||
stats_text.append("$0.00 ", style="#22c55e")
|
stats_text.append("$0.00 ", style="#22c55e")
|
||||||
stats_text.append("(subscription) ", style="dim")
|
stats_text.append("(subscription) ", style="dim")
|
||||||
else:
|
else:
|
||||||
@@ -375,19 +357,7 @@ def _build_llm_usage_stats(
|
|||||||
stats_text.append("Output Tokens ", style="dim")
|
stats_text.append("Output Tokens ", style="dim")
|
||||||
stats_text.append(format_token_count(output_tokens), style="white")
|
stats_text.append(format_token_count(output_tokens), style="white")
|
||||||
|
|
||||||
if subscription and subscription_is_metered():
|
if subscription:
|
||||||
# Zen spends prepaid credits per request, so a run is not free. Its
|
|
||||||
# Anthropic route runs through LiteLLM and yields a real charge; the
|
|
||||||
# OpenAI-SDK routes report none, and an unpriced run says so rather
|
|
||||||
# than claiming $0.00.
|
|
||||||
stats_text.append(" · ", style="dim white")
|
|
||||||
stats_text.append("Cost ", style="dim")
|
|
||||||
if cost > 0:
|
|
||||||
stats_text.append(f"${cost:.4f}", style="#22c55e")
|
|
||||||
stats_text.append(" (credits)", style="dim")
|
|
||||||
else:
|
|
||||||
stats_text.append("credits", style="#22c55e")
|
|
||||||
elif subscription:
|
|
||||||
stats_text.append(" · ", style="dim white")
|
stats_text.append(" · ", style="dim white")
|
||||||
stats_text.append("Cost ", style="dim")
|
stats_text.append("Cost ", style="dim")
|
||||||
stats_text.append("$0.00", style="#22c55e")
|
stats_text.append("$0.00", style="#22c55e")
|
||||||
@@ -419,7 +389,7 @@ def build_live_stats_text(report_state: Any) -> Text:
|
|||||||
stats_text.append(str(model), style="white")
|
stats_text.append(str(model), style="white")
|
||||||
if is_subscription_run(report_state):
|
if is_subscription_run(report_state):
|
||||||
stats_text.append(" · ", style="dim white")
|
stats_text.append(" · ", style="dim white")
|
||||||
stats_text.append(subscription_label(), style="#22c55e")
|
stats_text.append("ChatGPT subscription", style="#22c55e")
|
||||||
stats_text.append("\n")
|
stats_text.append("\n")
|
||||||
|
|
||||||
vuln_count = len(report_state.vulnerability_reports)
|
vuln_count = len(report_state.vulnerability_reports)
|
||||||
@@ -465,7 +435,7 @@ def build_tui_stats_text(report_state: Any) -> Text:
|
|||||||
subscription = is_subscription_run(report_state)
|
subscription = is_subscription_run(report_state)
|
||||||
if subscription:
|
if subscription:
|
||||||
stats_text.append("\n")
|
stats_text.append("\n")
|
||||||
stats_text.append(subscription_label(), style="#22c55e")
|
stats_text.append("ChatGPT subscription", style="#22c55e")
|
||||||
|
|
||||||
usage = _llm_usage(report_state)
|
usage = _llm_usage(report_state)
|
||||||
if usage and _int_stat(usage, "total_tokens") > 0:
|
if usage and _int_stat(usage, "total_tokens") > 0:
|
||||||
@@ -1629,9 +1599,6 @@ def clone_repository(repo_url: str, run_name: str, dest_name: str | None = None)
|
|||||||
|
|
||||||
|
|
||||||
def check_docker_connection() -> Any:
|
def check_docker_connection() -> Any:
|
||||||
import docker
|
|
||||||
from docker.errors import DockerException
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
return docker.from_env()
|
return docker.from_env()
|
||||||
except DockerException:
|
except DockerException:
|
||||||
@@ -1657,8 +1624,6 @@ def check_docker_connection() -> Any:
|
|||||||
|
|
||||||
|
|
||||||
def image_exists(client: Any, image_name: str) -> bool:
|
def image_exists(client: Any, image_name: str) -> bool:
|
||||||
from docker.errors import ImageNotFound
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
client.images.get(image_name)
|
client.images.get(image_name)
|
||||||
except ImageNotFound:
|
except ImageNotFound:
|
||||||
|
|||||||
@@ -101,23 +101,6 @@ export function RunDetails({
|
|||||||
const totalTokens = num(usage.total_tokens);
|
const totalTokens = num(usage.total_tokens);
|
||||||
const cost = num(usage.cost);
|
const cost = num(usage.cost);
|
||||||
const subscription = str(raw.auth_mode) === "subscription";
|
const subscription = str(raw.auth_mode) === "subscription";
|
||||||
const subscriptionProvider =
|
|
||||||
str(raw.subscription_provider) ??
|
|
||||||
(models.some((m) => m.toLowerCase().startsWith("opencode")) ? "opencode" : "chatgpt");
|
|
||||||
// Runs recorded before subscription_plan existed still carry the model string,
|
|
||||||
// whose prefix names the plan.
|
|
||||||
const subscriptionPlan =
|
|
||||||
str(raw.subscription_plan) ??
|
|
||||||
(models.some((m) => m.toLowerCase().startsWith("opencode-go/")) ? "go" : "zen");
|
|
||||||
const subscriptionLabel =
|
|
||||||
subscriptionProvider === "opencode"
|
|
||||||
? subscriptionPlan === "go"
|
|
||||||
? "OpenCode Go"
|
|
||||||
: "OpenCode Zen"
|
|
||||||
: "ChatGPT subscription";
|
|
||||||
// Zen bills prepaid credits per request, so its runs are not free and there is
|
|
||||||
// no price table to estimate them from. Go is a flat monthly plan.
|
|
||||||
const metered = subscriptionProvider === "opencode" && subscriptionPlan === "zen";
|
|
||||||
|
|
||||||
const sub = (n: number, word: string) => (
|
const sub = (n: number, word: string) => (
|
||||||
<span className="text-[#666]"> ({formatNumber(n)} {word})</span>
|
<span className="text-[#666]"> ({formatNumber(n)} {word})</span>
|
||||||
@@ -197,7 +180,7 @@ export function RunDetails({
|
|||||||
<Field label="Provider">
|
<Field label="Provider">
|
||||||
<span className="inline-flex items-center gap-1.5">
|
<span className="inline-flex items-center gap-1.5">
|
||||||
<span className="rounded-full border border-[#22c55e]/40 bg-[#22c55e]/10 px-2 py-0.5 text-[11px] text-[#22c55e]">
|
<span className="rounded-full border border-[#22c55e]/40 bg-[#22c55e]/10 px-2 py-0.5 text-[11px] text-[#22c55e]">
|
||||||
{subscriptionLabel}
|
ChatGPT subscription
|
||||||
</span>
|
</span>
|
||||||
</span>
|
</span>
|
||||||
</Field>
|
</Field>
|
||||||
@@ -217,21 +200,7 @@ export function RunDetails({
|
|||||||
</Field>
|
</Field>
|
||||||
)}
|
)}
|
||||||
{totalTokens != null && <Field label="Total tokens">{formatNumber(totalTokens)}</Field>}
|
{totalTokens != null && <Field label="Total tokens">{formatNumber(totalTokens)}</Field>}
|
||||||
{subscription && metered ? (
|
{subscription ? (
|
||||||
<Field label="Cost">
|
|
||||||
{cost != null && cost > 0 ? (
|
|
||||||
<>
|
|
||||||
<span className="text-[#22c55e]">${cost.toFixed(2)}</span>
|
|
||||||
<span className="text-[#666]"> (Zen credits)</span>
|
|
||||||
</>
|
|
||||||
) : (
|
|
||||||
<>
|
|
||||||
<span className="text-[#22c55e]">credits</span>
|
|
||||||
<span className="text-[#666]"> (not priced locally)</span>
|
|
||||||
</>
|
|
||||||
)}
|
|
||||||
</Field>
|
|
||||||
) : subscription ? (
|
|
||||||
<Field label="Cost">
|
<Field label="Cost">
|
||||||
<span className="text-[#22c55e]">$0.00</span>
|
<span className="text-[#22c55e]">$0.00</span>
|
||||||
<span className="text-[#666]"> (subscription)</span>
|
<span className="text-[#666]"> (subscription)</span>
|
||||||
|
|||||||
@@ -30,7 +30,7 @@ class RendererErrorBoundary extends Component<
|
|||||||
}
|
}
|
||||||
|
|
||||||
function SafeToolRenderer(props: ToolRendererProps) {
|
function SafeToolRenderer(props: ToolRendererProps) {
|
||||||
const Renderer = getToolRenderer(props.toolName, props.mcpConnection);
|
const Renderer = getToolRenderer(props.toolName);
|
||||||
return (
|
return (
|
||||||
<RendererErrorBoundary toolName={props.toolName}>
|
<RendererErrorBoundary toolName={props.toolName}>
|
||||||
<Renderer {...props} />
|
<Renderer {...props} />
|
||||||
@@ -63,10 +63,6 @@ function coerce(value: unknown): unknown {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
function asOptionalString(value: unknown): string | null {
|
|
||||||
return typeof value === "string" && value ? value : null;
|
|
||||||
}
|
|
||||||
|
|
||||||
function asRecord(value: unknown): Record<string, unknown> {
|
function asRecord(value: unknown): Record<string, unknown> {
|
||||||
const c = coerce(value);
|
const c = coerce(value);
|
||||||
if (c && typeof c === "object" && !Array.isArray(c)) return c as Record<string, unknown>;
|
if (c && typeof c === "object" && !Array.isArray(c)) return c as Record<string, unknown>;
|
||||||
@@ -248,14 +244,11 @@ export function AgentTranscript({
|
|||||||
const isTool = event.type === "tool";
|
const isTool = event.type === "tool";
|
||||||
const toolName = isTool ? String(event.data?.tool_name ?? "tool") : "";
|
const toolName = isTool ? String(event.data?.tool_name ?? "tool") : "";
|
||||||
const role = !isTool ? String(event.data?.role ?? "assistant") : "";
|
const role = !isTool ? String(event.data?.role ?? "assistant") : "";
|
||||||
// Present only on a call to one of the user's own MCP servers.
|
|
||||||
const mcpConnection = asOptionalString(event.data?.mcp_connection);
|
|
||||||
const mcpTool = asOptionalString(event.data?.mcp_tool);
|
|
||||||
|
|
||||||
let Icon;
|
let Icon;
|
||||||
let iconColor: string;
|
let iconColor: string;
|
||||||
if (isTool) {
|
if (isTool) {
|
||||||
const meta = getToolIcon(toolName, mcpConnection);
|
const meta = getToolIcon(toolName);
|
||||||
Icon = meta.icon;
|
Icon = meta.icon;
|
||||||
iconColor = meta.color;
|
iconColor = meta.color;
|
||||||
} else {
|
} else {
|
||||||
@@ -286,8 +279,6 @@ export function AgentTranscript({
|
|||||||
{isTool ? (
|
{isTool ? (
|
||||||
<SafeToolRenderer
|
<SafeToolRenderer
|
||||||
toolName={toolName}
|
toolName={toolName}
|
||||||
mcpConnection={mcpConnection}
|
|
||||||
mcpTool={mcpTool}
|
|
||||||
args={asRecord(event.data?.args)}
|
args={asRecord(event.data?.args)}
|
||||||
result={coerce(event.data?.result) ?? null}
|
result={coerce(event.data?.result) ?? null}
|
||||||
status={
|
status={
|
||||||
|
|||||||
-184
@@ -1,184 +0,0 @@
|
|||||||
"use client";
|
|
||||||
|
|
||||||
import type { ToolRendererProps } from "@/types/events";
|
|
||||||
import { CheckCircle2, CircleSlash, HelpCircle, AlertTriangle, Circle, ClipboardList } from "lucide-react";
|
|
||||||
|
|
||||||
interface CoverageEntry {
|
|
||||||
entry_id?: string;
|
|
||||||
surface?: string;
|
|
||||||
risk_area?: string;
|
|
||||||
outcome?: string;
|
|
||||||
evidence?: string;
|
|
||||||
agent_name?: string;
|
|
||||||
by_you?: boolean;
|
|
||||||
previous_outcomes?: string[];
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* A cleared surface and an unresolved one must never read alike — the ledger
|
|
||||||
* exists so that the negative space of a scan is legible, so each outcome gets
|
|
||||||
* its own icon and color rather than a shared neutral row.
|
|
||||||
*/
|
|
||||||
const OUTCOMES: Record<string, { label: string; color: string; Icon: typeof Circle }> = {
|
|
||||||
reported: { label: "reported", color: "text-orange-400", Icon: AlertTriangle },
|
|
||||||
no_issue_found: { label: "no issue found", color: "text-emerald-400", Icon: CheckCircle2 },
|
|
||||||
ruled_out: { label: "ruled out", color: "text-emerald-400/70", Icon: CheckCircle2 },
|
|
||||||
not_applicable: { label: "not applicable", color: "text-[#777]", Icon: CircleSlash },
|
|
||||||
needs_follow_up: { label: "needs follow-up", color: "text-yellow-400", Icon: HelpCircle },
|
|
||||||
};
|
|
||||||
|
|
||||||
const OUTCOME_ORDER = [
|
|
||||||
"reported", "needs_follow_up", "no_issue_found", "ruled_out", "not_applicable",
|
|
||||||
] as const;
|
|
||||||
|
|
||||||
function outcomeMeta(outcome: string | undefined) {
|
|
||||||
const key = (outcome ?? "").trim().toLowerCase();
|
|
||||||
return OUTCOMES[key] ?? {
|
|
||||||
label: key ? key.replace(/_/g, " ") : "unrecorded",
|
|
||||||
color: "text-[#777]",
|
|
||||||
Icon: Circle,
|
|
||||||
};
|
|
||||||
}
|
|
||||||
|
|
||||||
const ACTION_LABELS: Record<string, string> = {
|
|
||||||
record_coverage: "Coverage recorded",
|
|
||||||
update_coverage: "Coverage updated",
|
|
||||||
list_coverage: "Coverage",
|
|
||||||
};
|
|
||||||
|
|
||||||
function Header({ toolName }: { toolName: string }) {
|
|
||||||
return (
|
|
||||||
<div className="flex items-center gap-2">
|
|
||||||
<ClipboardList className="w-3.5 h-3.5 text-cyan-400/60" />
|
|
||||||
<span className="text-cyan-400/80 font-semibold text-sm">
|
|
||||||
{ACTION_LABELS[toolName] ?? "Coverage"}
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
function Row({ entry }: { entry: CoverageEntry }) {
|
|
||||||
const { label, color, Icon } = outcomeMeta(entry.outcome);
|
|
||||||
const previous = (entry.previous_outcomes ?? [])
|
|
||||||
.map((o) => outcomeMeta(o).label)
|
|
||||||
.filter(Boolean);
|
|
||||||
return (
|
|
||||||
<div className="flex items-start gap-2.5 py-1.5">
|
|
||||||
<Icon className={`w-3.5 h-3.5 shrink-0 mt-[2px] ${color}`} />
|
|
||||||
<div className="min-w-0">
|
|
||||||
<div className="text-[13px] leading-snug">
|
|
||||||
<span className="text-[#bbb]">{entry.surface ?? "(unnamed surface)"}</span>
|
|
||||||
{entry.risk_area && <span className="text-[#666]"> · {entry.risk_area}</span>}
|
|
||||||
</div>
|
|
||||||
<div className="text-xs mt-0.5">
|
|
||||||
<span className={color}>{label}</span>
|
|
||||||
{previous.length > 0 && (
|
|
||||||
<span className="text-[#555]"> (was {previous.join(" → ")})</span>
|
|
||||||
)}
|
|
||||||
{(entry.by_you || entry.agent_name) && (
|
|
||||||
<span className="text-[#555]"> · {entry.by_you ? "you" : entry.agent_name}</span>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
{entry.evidence && (
|
|
||||||
<div className="text-[#777] text-xs mt-1 leading-snug">{entry.evidence}</div>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
export default function CoverageRenderer({ toolName, args, result }: ToolRendererProps) {
|
|
||||||
const res = result as Record<string, unknown> | string | null;
|
|
||||||
|
|
||||||
if (typeof res === "string" && res.trim()) {
|
|
||||||
return (
|
|
||||||
<div>
|
|
||||||
<Header toolName={toolName} />
|
|
||||||
<div className="mt-1.5 text-[#888] text-[13px]">{res.trim()}</div>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
const structured = res && typeof res === "object" ? res : null;
|
|
||||||
const surface = (args.surface as string) ?? "";
|
|
||||||
const riskArea = (args.risk_area as string) ?? "";
|
|
||||||
const evidence = (args.evidence as string) ?? "";
|
|
||||||
|
|
||||||
if (structured && !structured.success) {
|
|
||||||
return (
|
|
||||||
<div>
|
|
||||||
<Header toolName={toolName} />
|
|
||||||
{(surface || riskArea) && (
|
|
||||||
<div className="mt-1.5 text-[13px] text-[#bbb]">
|
|
||||||
{surface}
|
|
||||||
{riskArea && <span className="text-[#666]"> · {riskArea}</span>}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
<div className="mt-1 text-red-400/70 text-[13px]">
|
|
||||||
{(structured.error as string) ?? "Coverage call failed"}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (toolName === "list_coverage") {
|
|
||||||
const rawEntries = structured?.entries;
|
|
||||||
const entries: CoverageEntry[] = Array.isArray(rawEntries) ? (rawEntries as CoverageEntry[]) : [];
|
|
||||||
const counts = (structured?.outcome_counts as Record<string, number> | undefined) ?? {};
|
|
||||||
const total = (structured?.total_count as number) ?? 0;
|
|
||||||
return (
|
|
||||||
<div>
|
|
||||||
<Header toolName={toolName} />
|
|
||||||
{Object.keys(counts).length > 0 && (
|
|
||||||
<div className="mt-2 flex items-center gap-3 flex-wrap">
|
|
||||||
{OUTCOME_ORDER.filter((o) => counts[o]).map((o) => {
|
|
||||||
const { label, color } = outcomeMeta(o);
|
|
||||||
return (
|
|
||||||
<span key={o} className={`text-xs ${color}`}>
|
|
||||||
{label}: {counts[o]}
|
|
||||||
</span>
|
|
||||||
);
|
|
||||||
})}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
{entries.length > 0 ? (
|
|
||||||
<div className="mt-2 rounded-lg border border-white/[0.06] bg-white/[0.015] px-3 py-1 divide-y divide-white/[0.04]">
|
|
||||||
{entries.map((entry, i) => <Row key={entry.entry_id ?? i} entry={entry} />)}
|
|
||||||
</div>
|
|
||||||
) : (
|
|
||||||
<div className="mt-1.5 text-[#555] text-xs">
|
|
||||||
{total === 0 ? "No surfaces recorded yet" : "No surfaces match this filter"}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
const outcome = (structured?.outcome as string) ?? "";
|
|
||||||
const previousOutcome = (structured?.previous_outcome as string) ?? "";
|
|
||||||
const { label, color, Icon } = outcomeMeta(outcome);
|
|
||||||
|
|
||||||
return (
|
|
||||||
<div>
|
|
||||||
<Header toolName={toolName} />
|
|
||||||
<div className="mt-2 flex items-start gap-2.5">
|
|
||||||
<Icon className={`w-3.5 h-3.5 shrink-0 mt-[2px] ${color}`} />
|
|
||||||
<div className="min-w-0">
|
|
||||||
<div className="text-[13px] leading-snug text-[#bbb]">
|
|
||||||
{surface || (structured?.entry_id ? `entry ${structured.entry_id as string}` : "(unnamed surface)")}
|
|
||||||
{riskArea && <span className="text-[#666]"> · {riskArea}</span>}
|
|
||||||
</div>
|
|
||||||
<div className="text-xs mt-0.5">
|
|
||||||
{previousOutcome && (
|
|
||||||
<span className="text-[#666]">{outcomeMeta(previousOutcome).label} → </span>
|
|
||||||
)}
|
|
||||||
<span className={color}>{label}</span>
|
|
||||||
</div>
|
|
||||||
{evidence && (
|
|
||||||
<div className="text-[#777] text-xs mt-1 leading-snug">{evidence}</div>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
"use client";
|
|
||||||
|
|
||||||
import type { ToolRendererProps } from "@/types/events";
|
|
||||||
|
|
||||||
/**
|
|
||||||
* A call to a tool from one of the MCP servers the user connected.
|
|
||||||
*
|
|
||||||
* Deliberately the same shape as the terminal: the tool's own name, the server
|
|
||||||
* it went to, the arguments one per line, and a status. The result is not shown.
|
|
||||||
* These payloads are routinely thousands of characters of JSON that say nothing a
|
|
||||||
* reader wants at this point in the transcript, and the agent narrates what it
|
|
||||||
* learned in its next message. A failure is the exception, because that is what
|
|
||||||
* someone is looking for when a step did not work; it renders as inert text,
|
|
||||||
* never as markdown, since it came from a server outside Strix.
|
|
||||||
*
|
|
||||||
* The full result is still in the run's event data on disk either way.
|
|
||||||
*/
|
|
||||||
|
|
||||||
/** Arguments one line each, as the terminal prints them. */
|
|
||||||
function argLines(args: unknown): string[] {
|
|
||||||
if (!args || typeof args !== "object" || Array.isArray(args)) return [];
|
|
||||||
return Object.entries(args as Record<string, unknown>).map(([key, value]) => {
|
|
||||||
const rendered = typeof value === "string" ? value : JSON.stringify(value);
|
|
||||||
return `${key}: ${rendered ?? String(value)}`;
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
const MAX_ERROR_CHARS = 600;
|
|
||||||
|
|
||||||
function errorText(result: unknown): string | null {
|
|
||||||
if (typeof result === "string") {
|
|
||||||
const trimmed = result.trim();
|
|
||||||
if (!trimmed) return null;
|
|
||||||
return trimmed.length > MAX_ERROR_CHARS ? `${trimmed.slice(0, MAX_ERROR_CHARS)}…` : trimmed;
|
|
||||||
}
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
|
|
||||||
export default function McpRenderer({
|
|
||||||
toolName,
|
|
||||||
mcpTool,
|
|
||||||
mcpConnection,
|
|
||||||
args,
|
|
||||||
result,
|
|
||||||
status,
|
|
||||||
}: ToolRendererProps) {
|
|
||||||
const lines = argLines(args);
|
|
||||||
const failed = status === "failed" || status === "error";
|
|
||||||
const error = failed ? errorText(result) : null;
|
|
||||||
|
|
||||||
return (
|
|
||||||
<div>
|
|
||||||
<div className="flex items-center gap-2 flex-wrap">
|
|
||||||
<span className="font-mono text-teal-300 font-semibold text-sm">{mcpTool || toolName}</span>
|
|
||||||
<span className="text-[13px] text-[#555]">via MCP server</span>
|
|
||||||
{mcpConnection && <span className="text-[13px] text-teal-400/80">{mcpConnection}</span>}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
{lines.length > 0 && (
|
|
||||||
<div className="mt-1 font-mono text-[13px] leading-relaxed">
|
|
||||||
{lines.map((line) => (
|
|
||||||
<div key={line} className="text-[#777] break-all">
|
|
||||||
{line}
|
|
||||||
</div>
|
|
||||||
))}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
|
|
||||||
<div className="mt-1 text-[13px]">
|
|
||||||
{status === "running" && <span className="text-[#666]">Running</span>}
|
|
||||||
{status === "completed" && <span className="text-emerald-400/80">✓ Done</span>}
|
|
||||||
{failed && <span className="text-red-400/80">✗ Failed</span>}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
{error && (
|
|
||||||
<pre className="mt-1 font-mono text-[13px] leading-relaxed whitespace-pre-wrap break-words text-red-400/70">
|
|
||||||
{error}
|
|
||||||
</pre>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
-138
@@ -1,138 +0,0 @@
|
|||||||
"use client";
|
|
||||||
|
|
||||||
import type { ToolRendererProps } from "@/types/events";
|
|
||||||
import { Crosshair, AlertTriangle, Plus, Save } from "lucide-react";
|
|
||||||
import { TruncatedText } from "./ToolCard";
|
|
||||||
|
|
||||||
interface Amendment {
|
|
||||||
agent_name?: string;
|
|
||||||
content?: string;
|
|
||||||
recorded_at?: string;
|
|
||||||
}
|
|
||||||
|
|
||||||
const ACTION_LABELS: Record<string, { label: string; Icon: typeof Crosshair }> = {
|
|
||||||
get_threat_model: { label: "Threat model", Icon: Crosshair },
|
|
||||||
save_threat_model: { label: "Threat model saved", Icon: Save },
|
|
||||||
amend_threat_model: { label: "Threat model amended", Icon: Plus },
|
|
||||||
};
|
|
||||||
|
|
||||||
/** A git sha is noise past its first bytes, and "unversioned" is not a revision. */
|
|
||||||
function shortRevision(revision: unknown): string {
|
|
||||||
const value = typeof revision === "string" ? revision.trim() : "";
|
|
||||||
if (!value || value === "unversioned") return "";
|
|
||||||
return value.slice(0, 8);
|
|
||||||
}
|
|
||||||
|
|
||||||
export default function ThreatModelRenderer({ toolName, args, result }: ToolRendererProps) {
|
|
||||||
const action = ACTION_LABELS[toolName] ?? { label: "Threat model", Icon: Crosshair };
|
|
||||||
const ActionIcon = action.Icon;
|
|
||||||
const target = (args.target as string) ?? "";
|
|
||||||
const res = result as Record<string, unknown> | string | null;
|
|
||||||
|
|
||||||
const header = (
|
|
||||||
<div className="flex items-center gap-2 flex-wrap">
|
|
||||||
<ActionIcon className="w-3.5 h-3.5 text-blue-400/60" />
|
|
||||||
<span className="text-blue-400/80 font-semibold text-sm">{action.label}</span>
|
|
||||||
{target && <span className="text-[#666] font-mono text-xs">{target}</span>}
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
|
|
||||||
if (typeof res === "string" && res.trim()) {
|
|
||||||
return <div>{header}<div className="mt-1.5 text-[#888] text-[13px]">{res.trim()}</div></div>;
|
|
||||||
}
|
|
||||||
|
|
||||||
const structured = res && typeof res === "object" ? res : null;
|
|
||||||
|
|
||||||
if (structured && !structured.success) {
|
|
||||||
return (
|
|
||||||
<div>
|
|
||||||
{header}
|
|
||||||
<div className="mt-1.5 text-red-400/70 text-[13px]">
|
|
||||||
{(structured.error as string) ?? "Threat model call failed"}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (toolName === "get_threat_model") {
|
|
||||||
if (structured && !structured.found) {
|
|
||||||
return (
|
|
||||||
<div>
|
|
||||||
{header}
|
|
||||||
<div className="mt-1.5 text-[#555] text-xs">No model cached for this target yet</div>
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
const rawAmendments = structured?.amendments;
|
|
||||||
const amendments: Amendment[] = Array.isArray(rawAmendments) ? (rawAmendments as Amendment[]) : [];
|
|
||||||
const cachedRevision = shortRevision(structured?.cached_revision);
|
|
||||||
return (
|
|
||||||
<div>
|
|
||||||
{header}
|
|
||||||
{structured?.stale === true && (
|
|
||||||
<div className="mt-1.5 flex items-center gap-1.5 text-yellow-400/80 text-xs">
|
|
||||||
<AlertTriangle className="w-3 h-3 shrink-0" />
|
|
||||||
<span>stale{cachedRevision ? ` — written at ${cachedRevision}` : ""}</span>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
{amendments.length > 0 && (
|
|
||||||
<div className="mt-2">
|
|
||||||
<span className="text-amber-400/70 text-xs font-semibold">
|
|
||||||
{amendments.length} amendment{amendments.length === 1 ? "" : "s"}
|
|
||||||
</span>
|
|
||||||
<span className="text-[#555] text-xs"> — later statements win</span>
|
|
||||||
<div className="mt-1 space-y-1">
|
|
||||||
{/* On a public share link the amendment body is stripped, so the
|
|
||||||
author line has to stand on its own. */}
|
|
||||||
{amendments.map((amendment, i) => (
|
|
||||||
<div key={i} className="text-xs leading-snug">
|
|
||||||
<span className="text-[#666]">{amendment.agent_name ?? "unknown agent"}</span>
|
|
||||||
{amendment.content && (
|
|
||||||
<span className="text-[#999]">: {amendment.content}</span>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
))}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
{typeof structured?.content === "string" && structured.content.trim() && (
|
|
||||||
<div className="mt-2">
|
|
||||||
<TruncatedText text={structured.content} maxLines={14} />
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
if (toolName === "amend_threat_model") {
|
|
||||||
const addendum = (args.addendum as string) ?? "";
|
|
||||||
const count = structured?.amendment_count as number | undefined;
|
|
||||||
return (
|
|
||||||
<div>
|
|
||||||
{header}
|
|
||||||
{count != null && (
|
|
||||||
<div className="mt-1.5 text-[#666] text-xs">{count} amendment{count === 1 ? "" : "s"} on this model</div>
|
|
||||||
)}
|
|
||||||
{addendum && <div className="mt-1.5"><TruncatedText text={addendum} maxLines={10} /></div>}
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
|
|
||||||
const cleared = (structured?.amendments_cleared as number | undefined) ?? 0;
|
|
||||||
const revision = shortRevision(structured?.revision);
|
|
||||||
const content = (args.content as string) ?? "";
|
|
||||||
return (
|
|
||||||
<div>
|
|
||||||
{header}
|
|
||||||
{revision && <div className="mt-1.5 text-[#666] font-mono text-xs">at {revision}</div>}
|
|
||||||
{/* Saving folds amendments away — the one destructive thing this tool does. */}
|
|
||||||
{cleared > 0 && (
|
|
||||||
<div className="mt-1.5 flex items-center gap-1.5 text-yellow-400/80 text-xs">
|
|
||||||
<AlertTriangle className="w-3 h-3 shrink-0" />
|
|
||||||
<span>cleared {cleared} amendment{cleared === 1 ? "" : "s"}</span>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
{content && <div className="mt-2"><TruncatedText text={content} maxLines={14} /></div>}
|
|
||||||
</div>
|
|
||||||
);
|
|
||||||
}
|
|
||||||
-40
@@ -11,11 +11,6 @@ const SEVERITY_COLORS: Record<string, string> = {
|
|||||||
low: "text-blue-400", info: "text-cyan-400",
|
low: "text-blue-400", info: "text-cyan-400",
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Anything below high is a claim the reader still has to check. */
|
|
||||||
const CONFIDENCE_COLORS: Record<string, string> = {
|
|
||||||
high: "text-emerald-400", medium: "text-yellow-400", low: "text-orange-400",
|
|
||||||
};
|
|
||||||
|
|
||||||
export default function VulnReportRenderer({ args, result }: ToolRendererProps) {
|
export default function VulnReportRenderer({ args, result }: ToolRendererProps) {
|
||||||
const title = (args.title as string) ?? "";
|
const title = (args.title as string) ?? "";
|
||||||
const description = (args.description as string) ?? "";
|
const description = (args.description as string) ?? "";
|
||||||
@@ -29,11 +24,6 @@ export default function VulnReportRenderer({ args, result }: ToolRendererProps)
|
|||||||
const remediation = (args.remediation_steps as string) ?? "";
|
const remediation = (args.remediation_steps as string) ?? "";
|
||||||
const cve = (args.cve as string) ?? "";
|
const cve = (args.cve as string) ?? "";
|
||||||
const cwe = (args.cwe as string) ?? "";
|
const cwe = (args.cwe as string) ?? "";
|
||||||
const counterevidence = (args.counterevidence as string) ?? "";
|
|
||||||
const confidence = ((args.confidence as string) ?? "").toLowerCase();
|
|
||||||
const confidenceRationale = (args.confidence_rationale as string) ?? "";
|
|
||||||
const severityChangeConditions = (args.severity_change_conditions as string) ?? "";
|
|
||||||
const fixVerification = (args.fix_verification as string) ?? "";
|
|
||||||
|
|
||||||
const res = result as Record<string, unknown> | null;
|
const res = result as Record<string, unknown> | null;
|
||||||
const rawSev = (res && typeof res === "object" ? res.severity : null) ?? args.severity ?? "medium";
|
const rawSev = (res && typeof res === "object" ? res.severity : null) ?? args.severity ?? "medium";
|
||||||
@@ -48,11 +38,6 @@ export default function VulnReportRenderer({ args, result }: ToolRendererProps)
|
|||||||
{cvss != null && <span className="text-[#888] text-[13px]">CVSS {cvss}</span>}
|
{cvss != null && <span className="text-[#888] text-[13px]">CVSS {cvss}</span>}
|
||||||
{cve && <span className="text-[#888] font-mono text-[13px]">{cve}</span>}
|
{cve && <span className="text-[#888] font-mono text-[13px]">{cve}</span>}
|
||||||
{cwe && <span className="text-[#888] font-mono text-[13px]">{cwe}</span>}
|
{cwe && <span className="text-[#888] font-mono text-[13px]">{cwe}</span>}
|
||||||
{confidence && (
|
|
||||||
<span className={`text-[13px] ${CONFIDENCE_COLORS[confidence] ?? "text-[#888]"}`}>
|
|
||||||
{confidence} confidence
|
|
||||||
</span>
|
|
||||||
)}
|
|
||||||
</div>
|
</div>
|
||||||
{title && <div className="text-[15px] text-white/80 font-semibold">{title}</div>}
|
{title && <div className="text-[15px] text-white/80 font-semibold">{title}</div>}
|
||||||
{(target || endpoint) && (
|
{(target || endpoint) && (
|
||||||
@@ -71,23 +56,6 @@ export default function VulnReportRenderer({ args, result }: ToolRendererProps)
|
|||||||
<div className="mt-1"><TruncatedText text={technicalAnalysis} maxLines={20} /></div>
|
<div className="mt-1"><TruncatedText text={technicalAnalysis} maxLines={20} /></div>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
{confidenceRationale && (
|
|
||||||
<div className="text-[#777] text-xs leading-snug">{confidenceRationale}</div>
|
|
||||||
)}
|
|
||||||
{/* The case against the finding sits beside the case for it: whoever
|
|
||||||
triages this needs both to decide whether to act. */}
|
|
||||||
{counterevidence && (
|
|
||||||
<div>
|
|
||||||
<span className="text-emerald-400/60 text-sm font-semibold">Counterevidence</span>
|
|
||||||
<div className="mt-1"><TruncatedText text={counterevidence} maxLines={12} /></div>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
{severityChangeConditions && (
|
|
||||||
<div>
|
|
||||||
<span className="text-emerald-400/60 text-sm font-semibold">Severity would change if</span>
|
|
||||||
<div className="mt-1"><TruncatedText text={severityChangeConditions} maxLines={10} /></div>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
{(pocDescription || pocCode) && (
|
{(pocDescription || pocCode) && (
|
||||||
<div>
|
<div>
|
||||||
<span className="text-emerald-400/60 text-sm font-semibold">Proof of Concept</span>
|
<span className="text-emerald-400/60 text-sm font-semibold">Proof of Concept</span>
|
||||||
@@ -101,14 +69,6 @@ export default function VulnReportRenderer({ args, result }: ToolRendererProps)
|
|||||||
<div className="mt-1"><TruncatedText text={remediation} maxLines={15} /></div>
|
<div className="mt-1"><TruncatedText text={remediation} maxLines={15} /></div>
|
||||||
</div>
|
</div>
|
||||||
)}
|
)}
|
||||||
{/* An applyable fix is one click from the user's codebase, so how it was
|
|
||||||
verified belongs next to it. */}
|
|
||||||
{fixVerification && (
|
|
||||||
<div>
|
|
||||||
<span className="text-emerald-400/60 text-sm font-semibold">Fix verification</span>
|
|
||||||
<div className="mt-1"><TruncatedText text={fixVerification} maxLines={12} /></div>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
</div>
|
</div>
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ import type { ToolRendererProps } from "@/types/events";
|
|||||||
import {
|
import {
|
||||||
Terminal, Globe, FileText, ShieldAlert, ArrowUpRight, Brain,
|
Terminal, Globe, FileText, ShieldAlert, ArrowUpRight, Brain,
|
||||||
Bot, MessageCircle, Flag, Eye, Search, Code, StickyNote,
|
Bot, MessageCircle, Flag, Eye, Search, Code, StickyNote,
|
||||||
ListTodo, Crosshair, Wrench, Ban, Image, ClipboardList, Plug,
|
ListTodo, Crosshair, Wrench, Ban, Image,
|
||||||
} from "lucide-react";
|
} from "lucide-react";
|
||||||
|
|
||||||
import TerminalRenderer from "./TerminalRenderer";
|
import TerminalRenderer from "./TerminalRenderer";
|
||||||
@@ -25,9 +25,6 @@ import TodoRenderer from "./TodoRenderer";
|
|||||||
import FallbackRenderer from "./FallbackRenderer";
|
import FallbackRenderer from "./FallbackRenderer";
|
||||||
import LoadSkillRenderer from "./LoadSkillRenderer";
|
import LoadSkillRenderer from "./LoadSkillRenderer";
|
||||||
import RespondRenderer from "./RespondRenderer";
|
import RespondRenderer from "./RespondRenderer";
|
||||||
import CoverageRenderer from "./CoverageRenderer";
|
|
||||||
import ThreatModelRenderer from "./ThreatModelRenderer";
|
|
||||||
import McpRenderer from "./McpRenderer";
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Tool-renderer mapping — data-driven, keyed by the engine's tool *family*.
|
* Tool-renderer mapping — data-driven, keyed by the engine's tool *family*.
|
||||||
@@ -56,10 +53,7 @@ export type ToolCategory =
|
|||||||
| "notes"
|
| "notes"
|
||||||
| "skills"
|
| "skills"
|
||||||
| "todos"
|
| "todos"
|
||||||
| "coverage"
|
| "telemetry";
|
||||||
| "threatModel"
|
|
||||||
| "telemetry"
|
|
||||||
| "mcp";
|
|
||||||
|
|
||||||
export interface ToolIconMeta {
|
export interface ToolIconMeta {
|
||||||
icon: ComponentType<{ className?: string }>;
|
icon: ComponentType<{ className?: string }>;
|
||||||
@@ -89,12 +83,7 @@ const CATEGORY_META: Record<ToolCategory, CategoryMeta> = {
|
|||||||
notes: { renderer: NotesRenderer, icon: StickyNote, color: "text-amber-400", match: /note/ },
|
notes: { renderer: NotesRenderer, icon: StickyNote, color: "text-amber-400", match: /note/ },
|
||||||
skills: { renderer: LoadSkillRenderer, icon: Wrench, color: "text-emerald-400" },
|
skills: { renderer: LoadSkillRenderer, icon: Wrench, color: "text-emerald-400" },
|
||||||
todos: { renderer: TodoRenderer, icon: ListTodo, color: "text-purple-400", match: /todo/ },
|
todos: { renderer: TodoRenderer, icon: ListTodo, color: "text-purple-400", match: /todo/ },
|
||||||
coverage: { renderer: CoverageRenderer, icon: ClipboardList, color: "text-cyan-400", match: /coverage/ },
|
|
||||||
threatModel: { renderer: ThreatModelRenderer, icon: Crosshair, color: "text-blue-400", match: /threat_model/ },
|
|
||||||
telemetry: { renderer: FallbackRenderer, icon: Wrench, color: "text-[#555]" },
|
telemetry: { renderer: FallbackRenderer, icon: Wrench, color: "text-[#555]" },
|
||||||
// Tools from the user's own MCP servers. Resolved from the connection on the
|
|
||||||
// event rather than from a tool name, so this family has no names below.
|
|
||||||
mcp: { renderer: McpRenderer, icon: Plug, color: "text-teal-400" },
|
|
||||||
};
|
};
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -123,12 +112,7 @@ const CATEGORY_TOOLS: Record<ToolCategory, readonly string[]> = {
|
|||||||
notes: ["create_note", "delete_note", "update_note", "list_notes", "get_note"],
|
notes: ["create_note", "delete_note", "update_note", "list_notes", "get_note"],
|
||||||
skills: ["load_skill"],
|
skills: ["load_skill"],
|
||||||
todos: ["create_todo", "list_todos", "update_todo", "mark_todo_done", "mark_todo_pending", "delete_todo"],
|
todos: ["create_todo", "list_todos", "update_todo", "mark_todo_done", "mark_todo_pending", "delete_todo"],
|
||||||
// Shared coverage ledger — one row per surface × risk area for the whole run
|
|
||||||
coverage: ["record_coverage", "update_coverage", "list_coverage"],
|
|
||||||
// Per-target threat model, shared across the agent tree
|
|
||||||
threatModel: ["get_threat_model", "save_threat_model", "amend_threat_model"],
|
|
||||||
telemetry: ["sandbox_error_details", "llm_error_details"],
|
telemetry: ["sandbox_error_details", "llm_error_details"],
|
||||||
mcp: [],
|
|
||||||
};
|
};
|
||||||
|
|
||||||
/** Reverse index (tool name → family), built once from CATEGORY_TOOLS. */
|
/** Reverse index (tool name → family), built once from CATEGORY_TOOLS. */
|
||||||
@@ -179,26 +163,14 @@ function resolveCategory(toolName: string): ToolCategory | null {
|
|||||||
return null;
|
return null;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
export function getToolRenderer(toolName: string): ComponentType<ToolRendererProps> {
|
||||||
* A call to a tool from one of the user's MCP servers is placed by the
|
|
||||||
* connection it was tagged with, ahead of every name-keyed lookup below: its
|
|
||||||
* name belongs to that server and matches nothing in this table.
|
|
||||||
*/
|
|
||||||
export function getToolRenderer(
|
|
||||||
toolName: string,
|
|
||||||
mcpConnection?: string | null
|
|
||||||
): ComponentType<ToolRendererProps> {
|
|
||||||
if (mcpConnection) return CATEGORY_META.mcp.renderer;
|
|
||||||
const override = RENDERER_OVERRIDES[toolName];
|
const override = RENDERER_OVERRIDES[toolName];
|
||||||
if (override) return override;
|
if (override) return override;
|
||||||
const category = resolveCategory(toolName);
|
const category = resolveCategory(toolName);
|
||||||
return category ? CATEGORY_META[category].renderer : FallbackRenderer;
|
return category ? CATEGORY_META[category].renderer : FallbackRenderer;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function getToolIcon(toolName: string, mcpConnection?: string | null): ToolIconMeta {
|
export function getToolIcon(toolName: string): ToolIconMeta {
|
||||||
if (mcpConnection) {
|
|
||||||
return { icon: CATEGORY_META.mcp.icon, color: CATEGORY_META.mcp.color };
|
|
||||||
}
|
|
||||||
const override = ICON_OVERRIDES[toolName];
|
const override = ICON_OVERRIDES[toolName];
|
||||||
if (override) return override;
|
if (override) return override;
|
||||||
const category = resolveCategory(toolName);
|
const category = resolveCategory(toolName);
|
||||||
|
|||||||
@@ -99,12 +99,4 @@ export interface ToolRendererProps {
|
|||||||
args: Record<string, unknown>;
|
args: Record<string, unknown>;
|
||||||
result: unknown;
|
result: unknown;
|
||||||
status: "running" | "completed" | "failed" | "error";
|
status: "running" | "completed" | "failed" | "error";
|
||||||
/**
|
|
||||||
* Set only on a call to a tool from an MCP server the user connected: the name
|
|
||||||
* they gave that connection, and the server's own name for the tool. The
|
|
||||||
* engine resolves both, because `toolName` is the two glued together and
|
|
||||||
* cannot be split back apart here.
|
|
||||||
*/
|
|
||||||
mcpConnection?: string | null;
|
|
||||||
mcpTool?: string | null;
|
|
||||||
}
|
}
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -6,8 +6,8 @@
|
|||||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||||
<meta name="color-scheme" content="dark" />
|
<meta name="color-scheme" content="dark" />
|
||||||
<title>Strix Results</title>
|
<title>Strix Results</title>
|
||||||
<script type="module" crossorigin src="./assets/index-CCwC0agA.js"></script>
|
<script type="module" crossorigin src="./assets/index-DBJ-RJqo.js"></script>
|
||||||
<link rel="stylesheet" crossorigin href="./assets/index-Bko_uVGo.css">
|
<link rel="stylesheet" crossorigin href="./assets/index-DKbLYAbP.css">
|
||||||
</head>
|
</head>
|
||||||
<body>
|
<body>
|
||||||
<div id="root"></div>
|
<div id="root"></div>
|
||||||
|
|||||||
+3
-16
@@ -10,11 +10,11 @@ pairing so the trimmed history is still valid provider input.
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
from functools import cache
|
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
from agents.model_settings import ModelSettings
|
from agents.model_settings import ModelSettings
|
||||||
from agents.models.interface import ModelTracing
|
from agents.models.interface import ModelTracing
|
||||||
|
from litellm.exceptions import BadRequestError, ContextWindowExceededError
|
||||||
from openai.types.responses import ResponseOutputMessage, ResponseOutputText
|
from openai.types.responses import ResponseOutputMessage, ResponseOutputText
|
||||||
|
|
||||||
from strix.config import load_settings
|
from strix.config import load_settings
|
||||||
@@ -63,18 +63,6 @@ _OVERFLOW_MARKERS = (
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@cache
|
|
||||||
def _overflow_error_types() -> tuple[type[BaseException], type[BaseException]]:
|
|
||||||
"""``(ContextWindowExceededError, BadRequestError)``, imported on first use.
|
|
||||||
|
|
||||||
LiteLLM costs seconds to import, and nothing needs it until a model call is
|
|
||||||
actually made, so it stays off the launch path.
|
|
||||||
"""
|
|
||||||
from litellm.exceptions import BadRequestError, ContextWindowExceededError
|
|
||||||
|
|
||||||
return ContextWindowExceededError, BadRequestError
|
|
||||||
|
|
||||||
|
|
||||||
def is_context_overflow(exc: BaseException) -> bool:
|
def is_context_overflow(exc: BaseException) -> bool:
|
||||||
"""Whether ``exc`` is a model context-window-overflow error.
|
"""Whether ``exc`` is a model context-window-overflow error.
|
||||||
|
|
||||||
@@ -82,10 +70,9 @@ def is_context_overflow(exc: BaseException) -> bool:
|
|||||||
OpenRouter branch raises a plain BadRequestError, so for that we fall back to
|
OpenRouter branch raises a plain BadRequestError, so for that we fall back to
|
||||||
matching the provider message.
|
matching the provider message.
|
||||||
"""
|
"""
|
||||||
context_window_exceeded, bad_request = _overflow_error_types()
|
if isinstance(exc, ContextWindowExceededError):
|
||||||
if isinstance(exc, context_window_exceeded):
|
|
||||||
return True
|
return True
|
||||||
if isinstance(exc, bad_request):
|
if isinstance(exc, BadRequestError):
|
||||||
msg = str(exc).lower()
|
msg = str(exc).lower()
|
||||||
if any(x in msg for x in _OVERFLOW_EXCLUSIONS):
|
if any(x in msg for x in _OVERFLOW_EXCLUSIONS):
|
||||||
return False
|
return False
|
||||||
|
|||||||
@@ -8,6 +8,8 @@ import logging
|
|||||||
from functools import lru_cache
|
from functools import lru_cache
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
import litellm
|
||||||
|
|
||||||
from strix.config import load_settings
|
from strix.config import load_settings
|
||||||
|
|
||||||
|
|
||||||
@@ -18,8 +20,6 @@ logger = logging.getLogger(__name__)
|
|||||||
_STRIPPABLE_PREFIXES = (
|
_STRIPPABLE_PREFIXES = (
|
||||||
"openai/",
|
"openai/",
|
||||||
"chatgpt/",
|
"chatgpt/",
|
||||||
"opencode-go/",
|
|
||||||
"opencode/",
|
|
||||||
"litellm/",
|
"litellm/",
|
||||||
"any-llm/",
|
"any-llm/",
|
||||||
"ollama/",
|
"ollama/",
|
||||||
@@ -38,8 +38,6 @@ def _lookup_key(model: str) -> str:
|
|||||||
|
|
||||||
def _safe_get_model_info(model: str) -> dict[str, Any] | None:
|
def _safe_get_model_info(model: str) -> dict[str, Any] | None:
|
||||||
try:
|
try:
|
||||||
import litellm
|
|
||||||
|
|
||||||
return dict(litellm.get_model_info(model))
|
return dict(litellm.get_model_info(model))
|
||||||
except Exception: # noqa: BLE001 - unmapped models raise; caller falls back.
|
except Exception: # noqa: BLE001 - unmapped models raise; caller falls back.
|
||||||
return None
|
return None
|
||||||
@@ -50,11 +48,7 @@ def _model_info(model: str) -> dict[str, int]:
|
|||||||
lookup_key = _lookup_key(model)
|
lookup_key = _lookup_key(model)
|
||||||
# Provider-qualified ChatGPT lookups may start a synchronous device-login
|
# Provider-qualified ChatGPT lookups may start a synchronous device-login
|
||||||
# poll. LiteLLM keys the metadata by the underlying model slug.
|
# poll. LiteLLM keys the metadata by the underlying model slug.
|
||||||
candidates = (
|
candidates = (lookup_key,) if model.startswith("chatgpt/") else (model, lookup_key)
|
||||||
(lookup_key,)
|
|
||||||
if model.startswith(("chatgpt/", "opencode/", "opencode-go/"))
|
|
||||||
else (model, lookup_key)
|
|
||||||
)
|
|
||||||
for candidate in candidates:
|
for candidate in candidates:
|
||||||
info = _safe_get_model_info(candidate)
|
info = _safe_get_model_info(candidate)
|
||||||
if info is not None:
|
if info is not None:
|
||||||
@@ -88,8 +82,6 @@ def count_tokens(model: str, text: str) -> int:
|
|||||||
if not text:
|
if not text:
|
||||||
return 0
|
return 0
|
||||||
try:
|
try:
|
||||||
import litellm
|
|
||||||
|
|
||||||
return int(litellm.token_counter(model=_lookup_key(model), text=text))
|
return int(litellm.token_counter(model=_lookup_key(model), text=text))
|
||||||
except Exception: # noqa: BLE001 - tokenizer may be unavailable for some models.
|
except Exception: # noqa: BLE001 - tokenizer may be unavailable for some models.
|
||||||
return len(text.encode("utf-8"))
|
return len(text.encode("utf-8"))
|
||||||
|
|||||||
@@ -1,55 +0,0 @@
|
|||||||
"""Background pre-import of the heavy scan dependencies.
|
|
||||||
|
|
||||||
The scan engine's import graph (the agents SDK, OpenAI client, LiteLLM, the
|
|
||||||
Caido SDK, the Docker SDK) costs seconds to import cold, but none of it is
|
|
||||||
needed until a scan actually starts. Importing it on a daemon thread at CLI
|
|
||||||
entry overlaps that cost with the I/O-bound startup work that always precedes
|
|
||||||
a scan (argument parsing, Docker checks, image pull, TUI setup), so by the
|
|
||||||
time the scan begins the modules are already in ``sys.modules``. Any thread
|
|
||||||
that needs one of them before the warm-up finishes just blocks on the normal
|
|
||||||
import lock, so behaviour is unchanged either way.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import importlib
|
|
||||||
import logging
|
|
||||||
import threading
|
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
WARMUP_MODULES = (
|
|
||||||
"strix.core.runner",
|
|
||||||
"litellm",
|
|
||||||
"caido_sdk_client",
|
|
||||||
"docker",
|
|
||||||
)
|
|
||||||
|
|
||||||
_lock = threading.Lock()
|
|
||||||
_thread: threading.Thread | None = None
|
|
||||||
|
|
||||||
|
|
||||||
def _warm(modules: tuple[str, ...]) -> None:
|
|
||||||
for name in modules:
|
|
||||||
try:
|
|
||||||
importlib.import_module(name)
|
|
||||||
except Exception: # noqa: BLE001 - a failed warm-up must never fail the run.
|
|
||||||
logger.debug("Import warm-up for %r failed", name, exc_info=True)
|
|
||||||
|
|
||||||
|
|
||||||
def start_import_warmup(modules: tuple[str, ...] = WARMUP_MODULES) -> threading.Thread:
|
|
||||||
"""Start importing the heavy scan dependencies in the background, once.
|
|
||||||
|
|
||||||
``modules`` lets embedders that never touch some backends (e.g. a cloud
|
|
||||||
runtime that has no local Docker) warm a narrower set.
|
|
||||||
"""
|
|
||||||
global _thread # noqa: PLW0603
|
|
||||||
with _lock:
|
|
||||||
if _thread is not None:
|
|
||||||
return _thread
|
|
||||||
_thread = threading.Thread(
|
|
||||||
target=_warm, args=(modules,), name="strix-import-warmup", daemon=True
|
|
||||||
)
|
|
||||||
_thread.start()
|
|
||||||
return _thread
|
|
||||||
@@ -1,443 +0,0 @@
|
|||||||
"""``coverage.json`` — the negative space of a scan, with provenance.
|
|
||||||
|
|
||||||
A findings list answers "what is wrong". It cannot answer "what did you
|
|
||||||
check", and in a compliance context that second question is the one that
|
|
||||||
decides whether a clean result means anything: an auditor reading zero SQL
|
|
||||||
injection findings cannot tell "tested fourteen endpoints, all parameterized"
|
|
||||||
apart from "never looked".
|
|
||||||
|
|
||||||
This module assembles the artifact that answers it. Two kinds of statement go
|
|
||||||
in, and they are kept apart on purpose:
|
|
||||||
|
|
||||||
- ``agent_reported`` — the coverage ledger (:mod:`strix.tools.coverage.tools`).
|
|
||||||
Rich and specific, but it is an agent's account of its own work.
|
|
||||||
- ``machine_observed`` — facts the runtime recorded regardless of what any
|
|
||||||
agent claimed: which agents ran and how they terminated, which skills they
|
|
||||||
carried, how many findings were filed, whether the run finished or was cut
|
|
||||||
short.
|
|
||||||
|
|
||||||
A coverage claim is an attestation, so conflating the two would be the worst
|
|
||||||
possible failure: a hallucinated "tested and clean" is strictly less honest
|
|
||||||
than no coverage record at all. Every entry therefore carries its ``source``,
|
|
||||||
and machine-observed facts contradict rather than confirm — an agent that
|
|
||||||
carried the ``sql_injection`` skill and recorded nothing about SQL injection
|
|
||||||
shows up under ``gaps``, and a run that hit its budget ceiling is stamped
|
|
||||||
``complete: false`` no matter how tidy the ledger looks.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
from datetime import UTC, datetime
|
|
||||||
from typing import TYPE_CHECKING, Any
|
|
||||||
|
|
||||||
from strix.report.writer import atomic_write_text
|
|
||||||
from strix.skills import get_available_skills
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
COVERAGE_FILENAME = "coverage.json"
|
|
||||||
COVERAGE_SCHEMA_VERSION = 1
|
|
||||||
|
|
||||||
#: Ledger outcomes rendered for a reader who has never seen our enum.
|
|
||||||
OUTCOME_LABELS: dict[str, str] = {
|
|
||||||
"reported": "Finding reported",
|
|
||||||
"no_issue_found": "No issue identified",
|
|
||||||
"ruled_out": "Ruled out",
|
|
||||||
"not_applicable": "Not applicable",
|
|
||||||
"needs_follow_up": "Requires further review",
|
|
||||||
}
|
|
||||||
|
|
||||||
#: Statuses that mean the agent stopped early rather than finishing its task.
|
|
||||||
_INCOMPLETE_AGENT_STATUSES = frozenset({"crashed", "stopped", "running", "waiting"})
|
|
||||||
|
|
||||||
#: Run statuses that mean the scan itself did not run to completion.
|
|
||||||
_INCOMPLETE_RUN_STATUSES = frozenset({"failed", "interrupted", "stopped", "running"})
|
|
||||||
|
|
||||||
#: Only this skill category names a vulnerability class. ``tooling`` and
|
|
||||||
#: ``reconnaissance`` skills describe how an agent works, not what it hunts,
|
|
||||||
#: so holding one implies no coverage obligation.
|
|
||||||
_RISK_SKILL_CATEGORY = "vulnerabilities"
|
|
||||||
|
|
||||||
#: How each vulnerability skill can legitimately appear in a ledger row.
|
|
||||||
#:
|
|
||||||
#: Matching a skill to a row is textual, and a skill's filename is not how a
|
|
||||||
#: pentester writes the class down: an agent carrying ``path_traversal_lfi_rfi``
|
|
||||||
#: records "Path Traversal", and one carrying ``weak_password_detection``
|
|
||||||
#: records "weak password policy". A row matches when it contains every word
|
|
||||||
#: of *any one* phrasing here. Skills absent from this map fall back to their
|
|
||||||
#: own words, so a new skill is merely matched strictly, never crashed on —
|
|
||||||
#: but add an entry, because a false gap asserts something untrue in a report.
|
|
||||||
_SKILL_PHRASINGS: dict[str, tuple[str, ...]] = {
|
|
||||||
"agentic_system_security": (
|
|
||||||
"agentic",
|
|
||||||
"agent tool",
|
|
||||||
"mcp",
|
|
||||||
"confused deputy",
|
|
||||||
"tool invocation",
|
|
||||||
),
|
|
||||||
"argument_injection": ("argument injection", "option injection", "argv"),
|
|
||||||
"authentication_jwt": ("authentication", "jwt", "session"),
|
|
||||||
"broken_function_level_authorization": (
|
|
||||||
"function level authorization",
|
|
||||||
"authorization",
|
|
||||||
"access control",
|
|
||||||
"privilege escalation",
|
|
||||||
),
|
|
||||||
"browser_security": (
|
|
||||||
"browser",
|
|
||||||
"postmessage",
|
|
||||||
"xs leak",
|
|
||||||
"service worker",
|
|
||||||
"cross origin state",
|
|
||||||
),
|
|
||||||
"business_logic": ("business logic", "logic flaw"),
|
|
||||||
"csrf": ("csrf", "cross site request forgery"),
|
|
||||||
"header_injection": ("header injection", "host header", "crlf"),
|
|
||||||
"http_request_smuggling": ("request smuggling", "desync"),
|
|
||||||
"idor": ("idor", "object level authorization", "bola", "direct object reference"),
|
|
||||||
"information_disclosure": (
|
|
||||||
"information disclosure",
|
|
||||||
"information leak",
|
|
||||||
"sensitive data",
|
|
||||||
"data exposure",
|
|
||||||
),
|
|
||||||
"insecure_deserialization": ("deserialization",),
|
|
||||||
"insecure_file_uploads": ("file upload",),
|
|
||||||
"llm_prompt_injection": ("prompt injection",),
|
|
||||||
"mass_assignment": ("mass assignment", "parameter binding"),
|
|
||||||
"nosql_injection": ("nosql",),
|
|
||||||
"open_redirect": ("redirect",),
|
|
||||||
"path_traversal_lfi_rfi": (
|
|
||||||
"path traversal",
|
|
||||||
"directory traversal",
|
|
||||||
"file inclusion",
|
|
||||||
"lfi",
|
|
||||||
"rfi",
|
|
||||||
),
|
|
||||||
"prototype_pollution": ("prototype pollution",),
|
|
||||||
"race_conditions": ("race condition", "toctou"),
|
|
||||||
"rce": ("rce", "remote code execution", "code execution", "command injection"),
|
|
||||||
"semantic_confusion": (
|
|
||||||
"semantic confusion",
|
|
||||||
"parser differential",
|
|
||||||
"normalization",
|
|
||||||
"validator sink mismatch",
|
|
||||||
),
|
|
||||||
"sql_injection": ("sql injection", "sqli"),
|
|
||||||
"ssrf": ("ssrf", "server side request forgery"),
|
|
||||||
"ssti": ("ssti", "template injection"),
|
|
||||||
"subdomain_takeover": ("subdomain takeover",),
|
|
||||||
"weak_password_detection": ("password", "credential", "brute force"),
|
|
||||||
"xss": ("xss", "cross site scripting", "script injection"),
|
|
||||||
"xxe": ("xxe", "xml external entity", "xml entity"),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def read_agent_graph(state_dir: Path) -> dict[str, Any]:
|
|
||||||
"""Load the coordinator's snapshot, or ``{}`` when it isn't readable.
|
|
||||||
|
|
||||||
The snapshot is the runtime's own record of the agent tree, written on
|
|
||||||
every graph mutation. Reading it here (rather than holding a coordinator
|
|
||||||
reference) keeps artifact assembly usable from a finished or resumed run,
|
|
||||||
where the live coordinator is gone but the file is still on disk.
|
|
||||||
"""
|
|
||||||
path = state_dir / "agents.json"
|
|
||||||
if not path.is_file():
|
|
||||||
return {}
|
|
||||||
try:
|
|
||||||
data = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
except (OSError, json.JSONDecodeError):
|
|
||||||
logger.warning("agent graph snapshot at %s is unreadable", path, exc_info=True)
|
|
||||||
return {}
|
|
||||||
return data if isinstance(data, dict) else {}
|
|
||||||
|
|
||||||
|
|
||||||
def _normalized(text: str) -> str:
|
|
||||||
"""Lowercase *text* with punctuation flattened to spaces, for matching."""
|
|
||||||
return "".join(char if char.isalnum() else " " for char in text.lower())
|
|
||||||
|
|
||||||
|
|
||||||
def _skill_leaf(skill: str) -> str:
|
|
||||||
return skill.rsplit("/", maxsplit=1)[-1].strip().lower()
|
|
||||||
|
|
||||||
|
|
||||||
def _risk_skill_names() -> frozenset[str]:
|
|
||||||
"""Bare names of every skill that denotes a vulnerability class."""
|
|
||||||
try:
|
|
||||||
entries = get_available_skills().get(_RISK_SKILL_CATEGORY, [])
|
|
||||||
return frozenset(entry["name"] for entry in entries if entry.get("name"))
|
|
||||||
except OSError:
|
|
||||||
logger.warning("could not enumerate skills for coverage gaps", exc_info=True)
|
|
||||||
return frozenset()
|
|
||||||
|
|
||||||
|
|
||||||
def agents_from_graph(graph: dict[str, Any]) -> list[dict[str, Any]]:
|
|
||||||
"""Flatten the coordinator snapshot into one record per agent."""
|
|
||||||
statuses = graph.get("statuses")
|
|
||||||
if not isinstance(statuses, dict):
|
|
||||||
return []
|
|
||||||
raw_names = graph.get("names")
|
|
||||||
names: dict[str, Any] = raw_names if isinstance(raw_names, dict) else {}
|
|
||||||
raw_metadata = graph.get("metadata")
|
|
||||||
metadata: dict[str, Any] = raw_metadata if isinstance(raw_metadata, dict) else {}
|
|
||||||
raw_parents = graph.get("parent_of")
|
|
||||||
parents: dict[str, Any] = raw_parents if isinstance(raw_parents, dict) else {}
|
|
||||||
# Only an unambiguous root earns the exemption below. A snapshot with no
|
|
||||||
# parent links at all makes every agent look parentless, and excusing all
|
|
||||||
# of them would silently delete the silent-agent check.
|
|
||||||
parentless = [agent_id for agent_id in statuses if not parents.get(agent_id)]
|
|
||||||
root_id = parentless[0] if len(parentless) == 1 else None
|
|
||||||
|
|
||||||
agents: list[dict[str, Any]] = []
|
|
||||||
for agent_id, status in statuses.items():
|
|
||||||
raw_meta = metadata.get(agent_id)
|
|
||||||
meta: dict[str, Any] = raw_meta if isinstance(raw_meta, dict) else {}
|
|
||||||
raw_skills = meta.get("skills")
|
|
||||||
skills: list[Any] = raw_skills if isinstance(raw_skills, list) else []
|
|
||||||
agents.append(
|
|
||||||
{
|
|
||||||
"agent_id": agent_id,
|
|
||||||
"agent_name": names.get(agent_id) or agent_id,
|
|
||||||
"status": str(status),
|
|
||||||
"skills": [str(skill) for skill in skills],
|
|
||||||
"task": str(meta.get("task") or ""),
|
|
||||||
"is_root": agent_id == root_id,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
agents.sort(key=lambda agent: str(agent["agent_name"]))
|
|
||||||
return agents
|
|
||||||
|
|
||||||
|
|
||||||
def _skill_phrasings(skill: str) -> list[list[str]]:
|
|
||||||
"""Word lists that would each count as a ledger row naming *skill*."""
|
|
||||||
phrasings = _SKILL_PHRASINGS.get(skill) or (skill,)
|
|
||||||
return [terms for phrase in phrasings if (terms := _normalized(phrase).split())]
|
|
||||||
|
|
||||||
|
|
||||||
def _entry_is_about(entry: dict[str, Any], phrasings: list[list[str]]) -> bool:
|
|
||||||
"""True when a ledger row plausibly concerns any phrasing of a risk class."""
|
|
||||||
haystack = _normalized(f"{entry.get('risk_area', '')} {entry.get('surface', '')}")
|
|
||||||
return any(all(term in haystack for term in terms) for terms in phrasings)
|
|
||||||
|
|
||||||
|
|
||||||
def skill_coverage_gaps(
|
|
||||||
entries: list[dict[str, Any]], agents: list[dict[str, Any]]
|
|
||||||
) -> list[dict[str, Any]]:
|
|
||||||
"""Vulnerability classes an agent was equipped for but never recorded.
|
|
||||||
|
|
||||||
A skill assigned to an agent is a declaration of intent that the runtime
|
|
||||||
observed independently of anything the agent later said. When no ledger
|
|
||||||
row mentions that class, the class is unaccounted for — which is a very
|
|
||||||
different report line from "tested, nothing found".
|
|
||||||
"""
|
|
||||||
risk_skills = _risk_skill_names()
|
|
||||||
if not risk_skills:
|
|
||||||
return []
|
|
||||||
|
|
||||||
carriers: dict[str, list[str]] = {}
|
|
||||||
for agent in agents:
|
|
||||||
for skill in agent["skills"]:
|
|
||||||
leaf = _skill_leaf(skill)
|
|
||||||
if leaf in risk_skills:
|
|
||||||
carriers.setdefault(leaf, []).append(str(agent["agent_name"]))
|
|
||||||
|
|
||||||
gaps: list[dict[str, Any]] = []
|
|
||||||
for skill, agent_names in sorted(carriers.items()):
|
|
||||||
phrasings = _skill_phrasings(skill)
|
|
||||||
if any(_entry_is_about(entry, phrasings) for entry in entries):
|
|
||||||
continue
|
|
||||||
gaps.append(
|
|
||||||
{
|
|
||||||
"kind": "unrecorded_risk_class",
|
|
||||||
"risk_area": skill.replace("_", " "),
|
|
||||||
"detail": (
|
|
||||||
f"Agent(s) {', '.join(sorted(set(agent_names)))} were assigned the "
|
|
||||||
f"'{skill}' skill, but no coverage entry records this class being "
|
|
||||||
"assessed. Treat it as unexamined, not as clean."
|
|
||||||
),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return gaps
|
|
||||||
|
|
||||||
|
|
||||||
def _silent_agent_gaps(
|
|
||||||
entries: list[dict[str, Any]], agents: list[dict[str, Any]]
|
|
||||||
) -> list[dict[str, Any]]:
|
|
||||||
"""Agents that ran and recorded nothing at all.
|
|
||||||
|
|
||||||
The root agent is exempt while it has children: it delegates and
|
|
||||||
reconciles rather than testing, so flagging it on every clean scan would
|
|
||||||
put a permanent false line in the report and teach readers to skip the
|
|
||||||
section. A root that ran alone tested alone, and is held to the rule.
|
|
||||||
"""
|
|
||||||
recorded_ids = {str(entry.get("agent_id")) for entry in entries if entry.get("agent_id")}
|
|
||||||
delegated = len(agents) > 1
|
|
||||||
gaps: list[dict[str, Any]] = []
|
|
||||||
for agent in agents:
|
|
||||||
if agent["agent_id"] in recorded_ids or (agent["is_root"] and delegated):
|
|
||||||
continue
|
|
||||||
gaps.append(
|
|
||||||
{
|
|
||||||
"kind": "agent_recorded_no_coverage",
|
|
||||||
"agent_name": agent["agent_name"],
|
|
||||||
"detail": (
|
|
||||||
f"{agent['agent_name']} ran (status: {agent['status']}) without "
|
|
||||||
"recording any coverage. Whatever it examined is absent from this "
|
|
||||||
"record."
|
|
||||||
),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return gaps
|
|
||||||
|
|
||||||
|
|
||||||
def _unresolved_gaps(entries: list[dict[str, Any]]) -> list[dict[str, Any]]:
|
|
||||||
"""Ledger rows the agents themselves left open."""
|
|
||||||
return [
|
|
||||||
{
|
|
||||||
"kind": "needs_follow_up",
|
|
||||||
"surface": entry.get("surface", ""),
|
|
||||||
"risk_area": entry.get("risk_area", ""),
|
|
||||||
"detail": str(entry.get("evidence") or "Left open without a stated reason."),
|
|
||||||
}
|
|
||||||
for entry in entries
|
|
||||||
if entry.get("outcome") == "needs_follow_up"
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def _completeness(
|
|
||||||
run_record: dict[str, Any],
|
|
||||||
agents: list[dict[str, Any]],
|
|
||||||
exit_reason: str | None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Whether this record can be read as a complete account of the scan.
|
|
||||||
|
|
||||||
Any of these makes it partial, and the caveats say which: the run did not
|
|
||||||
reach ``completed``, an agent was still live or died when the scan ended,
|
|
||||||
or the run stopped for a reason other than the root agent deciding it was
|
|
||||||
done (budget ceilings are the common case).
|
|
||||||
"""
|
|
||||||
status = str(run_record.get("status") or "unknown")
|
|
||||||
caveats: list[str] = []
|
|
||||||
|
|
||||||
if status in _INCOMPLETE_RUN_STATUSES:
|
|
||||||
caveats.append(
|
|
||||||
f"The scan ended with status '{status}' rather than completing, so coverage "
|
|
||||||
"reflects only the work finished before it stopped."
|
|
||||||
)
|
|
||||||
unfinished = [agent for agent in agents if agent["status"] in _INCOMPLETE_AGENT_STATUSES]
|
|
||||||
if unfinished:
|
|
||||||
names = ", ".join(sorted(str(agent["agent_name"]) for agent in unfinished))
|
|
||||||
caveats.append(
|
|
||||||
f"{len(unfinished)} agent(s) did not finish cleanly ({names}); any surface they "
|
|
||||||
"held is under-covered."
|
|
||||||
)
|
|
||||||
if exit_reason and exit_reason not in {"finished_by_tool", "completed"}:
|
|
||||||
caveats.append(
|
|
||||||
f"The run terminated via '{exit_reason}' rather than the root agent finishing, "
|
|
||||||
"so remaining scope was not reached."
|
|
||||||
)
|
|
||||||
|
|
||||||
return {
|
|
||||||
"complete": not caveats,
|
|
||||||
"scan_status": status,
|
|
||||||
"exit_reason": exit_reason,
|
|
||||||
"caveats": caveats,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _outcome_counts(entries: list[dict[str, Any]]) -> dict[str, int]:
|
|
||||||
counts: dict[str, int] = {}
|
|
||||||
for entry in entries:
|
|
||||||
outcome = str(entry.get("outcome", ""))
|
|
||||||
counts[outcome] = counts.get(outcome, 0) + 1
|
|
||||||
return {label: counts[label] for label in OUTCOME_LABELS if label in counts}
|
|
||||||
|
|
||||||
|
|
||||||
def build_coverage_document(
|
|
||||||
*,
|
|
||||||
run_record: dict[str, Any],
|
|
||||||
entries: list[dict[str, Any]],
|
|
||||||
agent_graph: dict[str, Any],
|
|
||||||
vulnerability_reports: list[dict[str, Any]],
|
|
||||||
exit_reason: str | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""Assemble the ``coverage.json`` document."""
|
|
||||||
agents = agents_from_graph(agent_graph)
|
|
||||||
skills_exercised = sorted(
|
|
||||||
{_skill_leaf(skill) for agent in agents for skill in agent["skills"] if skill}
|
|
||||||
)
|
|
||||||
|
|
||||||
ledger = [
|
|
||||||
{
|
|
||||||
"surface": entry.get("surface", ""),
|
|
||||||
"risk_area": entry.get("risk_area", ""),
|
|
||||||
"outcome": entry.get("outcome", ""),
|
|
||||||
"outcome_label": OUTCOME_LABELS.get(str(entry.get("outcome", "")), ""),
|
|
||||||
"evidence": entry.get("evidence", ""),
|
|
||||||
"recorded_by": entry.get("agent_name", ""),
|
|
||||||
"recorded_at": entry.get("created_at", ""),
|
|
||||||
"updated_at": entry.get("updated_at", ""),
|
|
||||||
"previous_outcomes": [
|
|
||||||
str(previous.get("outcome", ""))
|
|
||||||
for previous in entry.get("history", [])
|
|
||||||
if isinstance(previous, dict)
|
|
||||||
],
|
|
||||||
"source": "agent_reported",
|
|
||||||
}
|
|
||||||
for entry in entries
|
|
||||||
]
|
|
||||||
|
|
||||||
gaps = [
|
|
||||||
*_unresolved_gaps(entries),
|
|
||||||
*skill_coverage_gaps(entries, agents),
|
|
||||||
*_silent_agent_gaps(entries, agents),
|
|
||||||
]
|
|
||||||
|
|
||||||
return {
|
|
||||||
"schema_version": COVERAGE_SCHEMA_VERSION,
|
|
||||||
"generated_at": datetime.now(UTC).strftime("%Y-%m-%d %H:%M:%S UTC"),
|
|
||||||
"run_id": run_record.get("run_id"),
|
|
||||||
"run_name": run_record.get("run_name"),
|
|
||||||
"scope": {
|
|
||||||
"targets": run_record.get("targets_info") or [],
|
|
||||||
"scan_mode": run_record.get("scan_mode"),
|
|
||||||
"scope_mode": run_record.get("scope_mode"),
|
|
||||||
"diff_scope": run_record.get("diff_scope"),
|
|
||||||
"instruction": run_record.get("instruction") or "",
|
|
||||||
},
|
|
||||||
"summary": {
|
|
||||||
"surfaces_reviewed": len(ledger),
|
|
||||||
"outcomes": _outcome_counts(entries),
|
|
||||||
"findings_filed": len(vulnerability_reports),
|
|
||||||
"gaps": len(gaps),
|
|
||||||
},
|
|
||||||
"machine_observed": {
|
|
||||||
"agents": agents,
|
|
||||||
"skills_exercised": skills_exercised,
|
|
||||||
"findings_filed": len(vulnerability_reports),
|
|
||||||
"source": "runtime",
|
|
||||||
},
|
|
||||||
"completeness": _completeness(run_record, agents, exit_reason),
|
|
||||||
"entries": ledger,
|
|
||||||
"gaps": gaps,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def write_coverage(run_dir: Path, document: dict[str, Any]) -> Path:
|
|
||||||
"""Write ``coverage.json`` into the run directory and return its path."""
|
|
||||||
path = run_dir / COVERAGE_FILENAME
|
|
||||||
atomic_write_text(path, json.dumps(document, ensure_ascii=False, indent=2, default=str))
|
|
||||||
logger.info(
|
|
||||||
"Saved coverage record to: %s (%d surface(s), %d gap(s))",
|
|
||||||
path,
|
|
||||||
len(document.get("entries", [])),
|
|
||||||
len(document.get("gaps", [])),
|
|
||||||
)
|
|
||||||
return path
|
|
||||||
@@ -40,10 +40,6 @@ Design notes:
|
|||||||
* Findings without safe locations still appear in the SARIF output,
|
* Findings without safe locations still appear in the SARIF output,
|
||||||
anchored to SECURITY.md and flagged via
|
anchored to SECURITY.md and flagged via
|
||||||
``properties.synthetic_location`` rather than being dropped silently.
|
``properties.synthetic_location`` rather than being dropped silently.
|
||||||
* Coverage rides in the same document as non-failing results (``kind`` of
|
|
||||||
``pass`` / ``notApplicable`` / ``open``), and run completeness on
|
|
||||||
``run.invocations``. Consumers that only want alerts filter on
|
|
||||||
``kind == "fail"`` and are unaffected.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -203,7 +199,6 @@ def build_sarif_report(
|
|||||||
*,
|
*,
|
||||||
tool_version: str | None = None,
|
tool_version: str | None = None,
|
||||||
repository_context: dict[str, Any] | None = None,
|
repository_context: dict[str, Any] | None = None,
|
||||||
coverage: dict[str, Any] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
"""Return a SARIF 2.1.0 document for findings.
|
"""Return a SARIF 2.1.0 document for findings.
|
||||||
|
|
||||||
@@ -214,11 +209,6 @@ def build_sarif_report(
|
|||||||
can bind alerts to the scanned commit; it is omitted for URL / IP
|
can bind alerts to the scanned commit; it is omitted for URL / IP
|
||||||
(DAST) targets that have no repository.
|
(DAST) targets that have no repository.
|
||||||
|
|
||||||
``coverage`` (optional) is the document from
|
|
||||||
:func:`strix.report.coverage.build_coverage_document`: its cleared
|
|
||||||
surfaces become non-failing results and its completeness caveats become
|
|
||||||
invocation notifications.
|
|
||||||
|
|
||||||
Findings without safe source locations are anchored synthetically
|
Findings without safe source locations are anchored synthetically
|
||||||
to SECURITY.md and flagged via ``properties.synthetic_location``.
|
to SECURITY.md and flagged via ``properties.synthetic_location``.
|
||||||
They're still emitted as proper SARIF results so they (a) flow
|
They're still emitted as proper SARIF results so they (a) flow
|
||||||
@@ -257,9 +247,6 @@ def build_sarif_report(
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
if coverage:
|
|
||||||
_append_coverage(coverage, rules_by_id, rule_index_by_id, results)
|
|
||||||
|
|
||||||
driver: dict[str, Any] = {
|
driver: dict[str, Any] = {
|
||||||
"name": TOOL_NAME,
|
"name": TOOL_NAME,
|
||||||
"informationUri": TOOL_INFORMATION_URI,
|
"informationUri": TOOL_INFORMATION_URI,
|
||||||
@@ -273,9 +260,6 @@ def build_sarif_report(
|
|||||||
"results": results,
|
"results": results,
|
||||||
}
|
}
|
||||||
|
|
||||||
if coverage:
|
|
||||||
run["invocations"] = [_coverage_invocation(coverage)]
|
|
||||||
|
|
||||||
run_properties: dict[str, Any] = {}
|
run_properties: dict[str, Any] = {}
|
||||||
if synthetic_location_count:
|
if synthetic_location_count:
|
||||||
# Surface the count for observability without duplicating the
|
# Surface the count for observability without duplicating the
|
||||||
@@ -308,7 +292,6 @@ def write_sarif_report(
|
|||||||
*,
|
*,
|
||||||
tool_version: str | None = None,
|
tool_version: str | None = None,
|
||||||
repository_context: dict[str, Any] | None = None,
|
repository_context: dict[str, Any] | None = None,
|
||||||
coverage: dict[str, Any] | None = None,
|
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Write a SARIF report to disk, creating parent directories first.
|
"""Write a SARIF report to disk, creating parent directories first.
|
||||||
|
|
||||||
@@ -321,7 +304,6 @@ def write_sarif_report(
|
|||||||
vulnerability_reports,
|
vulnerability_reports,
|
||||||
tool_version=tool_version,
|
tool_version=tool_version,
|
||||||
repository_context=repository_context,
|
repository_context=repository_context,
|
||||||
coverage=coverage,
|
|
||||||
)
|
)
|
||||||
tmp_path = output_path.with_name(f"{output_path.name}.{os.getpid()}.tmp")
|
tmp_path = output_path.with_name(f"{output_path.name}.{os.getpid()}.tmp")
|
||||||
try:
|
try:
|
||||||
@@ -339,7 +321,6 @@ def write_sarif(
|
|||||||
*,
|
*,
|
||||||
tool_version: str | None = None,
|
tool_version: str | None = None,
|
||||||
repository_context: dict[str, Any] | None = None,
|
repository_context: dict[str, Any] | None = None,
|
||||||
coverage: dict[str, Any] | None = None,
|
|
||||||
filename: str = "findings.sarif",
|
filename: str = "findings.sarif",
|
||||||
) -> Path:
|
) -> Path:
|
||||||
"""Write ``findings.sarif`` alongside existing outputs in ``run_dir``.
|
"""Write ``findings.sarif`` alongside existing outputs in ``run_dir``.
|
||||||
@@ -354,7 +335,6 @@ def write_sarif(
|
|||||||
reports,
|
reports,
|
||||||
tool_version=tool_version,
|
tool_version=tool_version,
|
||||||
repository_context=repository_context,
|
repository_context=repository_context,
|
||||||
coverage=coverage,
|
|
||||||
)
|
)
|
||||||
logger.info(
|
logger.info(
|
||||||
"Wrote SARIF 2.1.0 report: %s (%d results)",
|
"Wrote SARIF 2.1.0 report: %s (%d results)",
|
||||||
@@ -546,11 +526,6 @@ def _result_properties(
|
|||||||
"impact",
|
"impact",
|
||||||
"technical_analysis",
|
"technical_analysis",
|
||||||
"remediation_steps",
|
"remediation_steps",
|
||||||
"counterevidence",
|
|
||||||
"confidence",
|
|
||||||
"confidence_rationale",
|
|
||||||
"severity_change_conditions",
|
|
||||||
"fix_verification",
|
|
||||||
):
|
):
|
||||||
value = report.get(key)
|
value = report.get(key)
|
||||||
if value not in (None, ""):
|
if value not in (None, ""):
|
||||||
@@ -638,115 +613,6 @@ def _build_fixes(report: dict[str, Any]) -> list[dict[str, Any]] | None:
|
|||||||
return [fix]
|
return [fix]
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Coverage
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
_COVERAGE_RULE_PREFIX = "strix-coverage"
|
|
||||||
|
|
||||||
# ``reported`` is absent on purpose: those surfaces are already in ``results``
|
|
||||||
# as ``fail`` findings.
|
|
||||||
_OUTCOME_TO_KIND = {
|
|
||||||
"no_issue_found": "pass",
|
|
||||||
"ruled_out": "pass",
|
|
||||||
"not_applicable": "notApplicable",
|
|
||||||
"needs_follow_up": "open",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _coverage_rule_id(risk_area: str) -> str:
|
|
||||||
slug = _slugify(risk_area) or "unspecified"
|
|
||||||
return f"{_COVERAGE_RULE_PREFIX}/{slug}"
|
|
||||||
|
|
||||||
|
|
||||||
def _build_coverage_rule(rule_id: str, risk_area: str) -> dict[str, Any]:
|
|
||||||
description = f"Coverage of {risk_area} across the assessed attack surface."
|
|
||||||
return {
|
|
||||||
"id": rule_id,
|
|
||||||
"name": _rule_name(rule_id, risk_area),
|
|
||||||
"shortDescription": {"text": f"Coverage: {risk_area}"},
|
|
||||||
"fullDescription": {"text": description},
|
|
||||||
"defaultConfiguration": {"level": "none"},
|
|
||||||
"help": {"text": description, "markdown": description},
|
|
||||||
"properties": {"tags": ["coverage"]},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _build_coverage_result(
|
|
||||||
rule_id: str,
|
|
||||||
rule_index: int,
|
|
||||||
kind: str,
|
|
||||||
entry: dict[str, Any],
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
surface = _string_value(entry.get("surface")) or "unspecified surface"
|
|
||||||
risk_area = _string_value(entry.get("risk_area")) or "unspecified risk"
|
|
||||||
evidence = _string_value(entry.get("evidence"))
|
|
||||||
label = _string_value(entry.get("outcome_label")) or str(entry.get("outcome", ""))
|
|
||||||
|
|
||||||
message = f"{risk_area} — {label}: {surface}"
|
|
||||||
if evidence:
|
|
||||||
message = f"{message}\n\n{evidence}"
|
|
||||||
|
|
||||||
result: dict[str, Any] = {
|
|
||||||
"ruleId": rule_id,
|
|
||||||
"ruleIndex": rule_index,
|
|
||||||
"kind": kind,
|
|
||||||
# SARIF requires ``level: none`` for any result whose kind is not ``fail``.
|
|
||||||
"level": "none",
|
|
||||||
"message": {"text": message},
|
|
||||||
"locations": [{"logicalLocations": [{"fullyQualifiedName": surface}]}],
|
|
||||||
"properties": {
|
|
||||||
"strix": {
|
|
||||||
"coverage_outcome": entry.get("outcome", ""),
|
|
||||||
"risk_area": risk_area,
|
|
||||||
"surface": surface,
|
|
||||||
"recorded_by": entry.get("recorded_by", ""),
|
|
||||||
"source": entry.get("source", "agent_reported"),
|
|
||||||
}
|
|
||||||
},
|
|
||||||
}
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def _append_coverage(
|
|
||||||
coverage: dict[str, Any],
|
|
||||||
rules_by_id: dict[str, dict[str, Any]],
|
|
||||||
rule_index_by_id: dict[str, int],
|
|
||||||
results: list[dict[str, Any]],
|
|
||||||
) -> None:
|
|
||||||
entries = coverage.get("entries")
|
|
||||||
if not isinstance(entries, list):
|
|
||||||
return
|
|
||||||
for entry in entries:
|
|
||||||
if not isinstance(entry, dict):
|
|
||||||
continue
|
|
||||||
kind = _OUTCOME_TO_KIND.get(str(entry.get("outcome", "")))
|
|
||||||
if kind is None:
|
|
||||||
continue
|
|
||||||
rule_id = _coverage_rule_id(str(entry.get("risk_area", "")))
|
|
||||||
if rule_id not in rules_by_id:
|
|
||||||
rule_index_by_id[rule_id] = len(rules_by_id)
|
|
||||||
rules_by_id[rule_id] = _build_coverage_rule(
|
|
||||||
rule_id, _string_value(entry.get("risk_area")) or "unspecified risk"
|
|
||||||
)
|
|
||||||
results.append(_build_coverage_result(rule_id, rule_index_by_id[rule_id], kind, entry))
|
|
||||||
|
|
||||||
|
|
||||||
def _coverage_invocation(coverage: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
"""``executionSuccessful: false`` stops a truncated run reading as a clean one."""
|
|
||||||
completeness = coverage.get("completeness")
|
|
||||||
completeness = completeness if isinstance(completeness, dict) else {}
|
|
||||||
caveats = completeness.get("caveats")
|
|
||||||
caveats = caveats if isinstance(caveats, list) else []
|
|
||||||
|
|
||||||
invocation: dict[str, Any] = {"executionSuccessful": bool(completeness.get("complete", True))}
|
|
||||||
if caveats:
|
|
||||||
invocation["toolExecutionNotifications"] = [
|
|
||||||
{"level": "warning", "message": {"text": str(caveat)}} for caveat in caveats
|
|
||||||
]
|
|
||||||
return invocation
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Location handling
|
# Location handling
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
+4
-69
@@ -11,10 +11,9 @@ from uuid import uuid4
|
|||||||
|
|
||||||
from agents.usage import Usage
|
from agents.usage import Usage
|
||||||
|
|
||||||
from strix.config import opencode
|
from strix.config import codex
|
||||||
from strix.config.loader import load_settings
|
from strix.config.loader import load_settings
|
||||||
from strix.core.paths import run_dir_for, runtime_state_dir
|
from strix.core.paths import run_dir_for
|
||||||
from strix.report.coverage import write_coverage
|
|
||||||
from strix.report.pricing import resolve_litellm_model
|
from strix.report.pricing import resolve_litellm_model
|
||||||
from strix.report.sarif import write_sarif
|
from strix.report.sarif import write_sarif
|
||||||
from strix.report.usage import LLMUsageLedger
|
from strix.report.usage import LLMUsageLedger
|
||||||
@@ -133,13 +132,8 @@ class ReportState:
|
|||||||
self.scan_config: dict[str, Any] | None = None
|
self.scan_config: dict[str, Any] | None = None
|
||||||
self._llm_usage = LLMUsageLedger()
|
self._llm_usage = LLMUsageLedger()
|
||||||
self._telemetry_llm_usage_baseline: dict[str, Any] = {}
|
self._telemetry_llm_usage_baseline: dict[str, Any] = {}
|
||||||
auth_mode = opencode.auth_mode(load_settings().llm.model)
|
auth_mode = codex.auth_mode(load_settings().llm.model)
|
||||||
oc = opencode.subscription_model(load_settings().llm.model)
|
self._llm_usage.zero_cost = auth_mode == "subscription"
|
||||||
# A flat subscription has no per-run charge to report. Zen bills prepaid
|
|
||||||
# credits per request, so its cost is real and stays tracked.
|
|
||||||
self._llm_usage.zero_cost = auth_mode == "subscription" and not (
|
|
||||||
oc is not None and oc.metered
|
|
||||||
)
|
|
||||||
self.run_record: dict[str, Any] = {
|
self.run_record: dict[str, Any] = {
|
||||||
"run_id": self.run_id,
|
"run_id": self.run_id,
|
||||||
"run_name": self.run_name,
|
"run_name": self.run_name,
|
||||||
@@ -147,8 +141,6 @@ class ReportState:
|
|||||||
"end_time": None,
|
"end_time": None,
|
||||||
"status": "running",
|
"status": "running",
|
||||||
"auth_mode": auth_mode,
|
"auth_mode": auth_mode,
|
||||||
"subscription_provider": opencode.subscription_provider(load_settings().llm.model),
|
|
||||||
"subscription_plan": opencode.subscription_plan(load_settings().llm.model),
|
|
||||||
"targets_info": [],
|
"targets_info": [],
|
||||||
"llm_usage": self._build_llm_usage_record(),
|
"llm_usage": self._build_llm_usage_record(),
|
||||||
}
|
}
|
||||||
@@ -245,10 +237,6 @@ class ReportState:
|
|||||||
remediation_steps: str | None = None,
|
remediation_steps: str | None = None,
|
||||||
evidence: str | None = None,
|
evidence: str | None = None,
|
||||||
assumptions: str | None = None,
|
assumptions: str | None = None,
|
||||||
counterevidence: str | None = None,
|
|
||||||
confidence: str | None = None,
|
|
||||||
confidence_rationale: str | None = None,
|
|
||||||
severity_change_conditions: str | None = None,
|
|
||||||
fix_effort: str | None = None,
|
fix_effort: str | None = None,
|
||||||
cvss: float | None = None,
|
cvss: float | None = None,
|
||||||
cvss_breakdown: dict[str, str] | None = None,
|
cvss_breakdown: dict[str, str] | None = None,
|
||||||
@@ -257,7 +245,6 @@ class ReportState:
|
|||||||
cve: str | None = None,
|
cve: str | None = None,
|
||||||
cwe: str | None = None,
|
cwe: str | None = None,
|
||||||
code_locations: list[dict[str, Any]] | None = None,
|
code_locations: list[dict[str, Any]] | None = None,
|
||||||
fix_verification: str | None = None,
|
|
||||||
fix_pr_body: str | None = None,
|
fix_pr_body: str | None = None,
|
||||||
finding_class: str | None = None,
|
finding_class: str | None = None,
|
||||||
dependency_metadata: dict[str, str] | None = None,
|
dependency_metadata: dict[str, str] | None = None,
|
||||||
@@ -291,14 +278,6 @@ class ReportState:
|
|||||||
report["evidence"] = evidence.strip()
|
report["evidence"] = evidence.strip()
|
||||||
if assumptions:
|
if assumptions:
|
||||||
report["assumptions"] = assumptions.strip()
|
report["assumptions"] = assumptions.strip()
|
||||||
if counterevidence:
|
|
||||||
report["counterevidence"] = counterevidence.strip()
|
|
||||||
if confidence:
|
|
||||||
report["confidence"] = confidence.strip().lower()
|
|
||||||
if confidence_rationale:
|
|
||||||
report["confidence_rationale"] = confidence_rationale.strip()
|
|
||||||
if severity_change_conditions:
|
|
||||||
report["severity_change_conditions"] = severity_change_conditions.strip()
|
|
||||||
if fix_effort:
|
if fix_effort:
|
||||||
report["fix_effort"] = fix_effort.strip().lower()
|
report["fix_effort"] = fix_effort.strip().lower()
|
||||||
if cvss is not None:
|
if cvss is not None:
|
||||||
@@ -315,8 +294,6 @@ class ReportState:
|
|||||||
report["cwe"] = cwe.strip()
|
report["cwe"] = cwe.strip()
|
||||||
if code_locations:
|
if code_locations:
|
||||||
report["code_locations"] = code_locations
|
report["code_locations"] = code_locations
|
||||||
if fix_verification:
|
|
||||||
report["fix_verification"] = fix_verification.strip()
|
|
||||||
if fix_pr_body:
|
if fix_pr_body:
|
||||||
report["fix_pr_body"] = fix_pr_body.strip()
|
report["fix_pr_body"] = fix_pr_body.strip()
|
||||||
report["finding_class"] = (finding_class or "dynamic").strip().lower()
|
report["finding_class"] = (finding_class or "dynamic").strip().lower()
|
||||||
@@ -411,18 +388,6 @@ class ReportState:
|
|||||||
posthog.end(self, exit_reason="finished_by_tool")
|
posthog.end(self, exit_reason="finished_by_tool")
|
||||||
scarf.end(self, exit_reason="finished_by_tool")
|
scarf.end(self, exit_reason="finished_by_tool")
|
||||||
|
|
||||||
def record_mcp_connections(self, names: list[str]) -> None:
|
|
||||||
"""Note the MCP servers this run connected, and persist it.
|
|
||||||
|
|
||||||
Saved as soon as the run connects rather than at the end, so an interface
|
|
||||||
reading the record mid-run can already attribute a tool call to the
|
|
||||||
server it went out to.
|
|
||||||
"""
|
|
||||||
if self.run_record.get("mcp_connections") == names:
|
|
||||||
return
|
|
||||||
self.run_record["mcp_connections"] = names
|
|
||||||
self.save_run_data()
|
|
||||||
|
|
||||||
def set_scan_config(self, config: dict[str, Any]) -> None:
|
def set_scan_config(self, config: dict[str, Any]) -> None:
|
||||||
self.scan_config = config
|
self.scan_config = config
|
||||||
self.run_record["status"] = "running"
|
self.run_record["status"] = "running"
|
||||||
@@ -482,41 +447,12 @@ class ReportState:
|
|||||||
{str(scan_results.get("recommendations", "")).strip()}
|
{str(scan_results.get("recommendations", "")).strip()}
|
||||||
"""
|
"""
|
||||||
|
|
||||||
def _coverage_document(self) -> dict[str, Any] | None:
|
|
||||||
"""Assemble the coverage record, or None when it can't be built.
|
|
||||||
|
|
||||||
Coverage is a secondary artifact: a failure here must not cost the
|
|
||||||
caller its findings, so this swallows and logs rather than raising
|
|
||||||
into :meth:`_save_artifacts`.
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
from strix.report.coverage import build_coverage_document, read_agent_graph
|
|
||||||
from strix.tools.coverage.tools import get_coverage_entries
|
|
||||||
|
|
||||||
return build_coverage_document(
|
|
||||||
run_record=self.run_record,
|
|
||||||
entries=get_coverage_entries(),
|
|
||||||
agent_graph=read_agent_graph(runtime_state_dir(self.get_run_dir())),
|
|
||||||
vulnerability_reports=self.vulnerability_reports,
|
|
||||||
exit_reason=self.scan_ended_exit_reason,
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
logger.exception("coverage document build failed (non-fatal)")
|
|
||||||
return None
|
|
||||||
|
|
||||||
def _save_artifacts(self) -> None:
|
def _save_artifacts(self) -> None:
|
||||||
"""Write scan artifacts under ``run_dir``."""
|
"""Write scan artifacts under ``run_dir``."""
|
||||||
run_dir = self.get_run_dir()
|
run_dir = self.get_run_dir()
|
||||||
try:
|
try:
|
||||||
run_dir.mkdir(parents=True, exist_ok=True)
|
run_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
coverage = self._coverage_document()
|
|
||||||
if coverage is not None:
|
|
||||||
try:
|
|
||||||
write_coverage(run_dir, coverage)
|
|
||||||
except OSError:
|
|
||||||
logger.exception("coverage.json write failed (non-fatal)")
|
|
||||||
|
|
||||||
if self.final_scan_result:
|
if self.final_scan_result:
|
||||||
write_executive_report(run_dir, self.final_scan_result)
|
write_executive_report(run_dir, self.final_scan_result)
|
||||||
|
|
||||||
@@ -535,7 +471,6 @@ class ReportState:
|
|||||||
self.vulnerability_reports,
|
self.vulnerability_reports,
|
||||||
tool_version=_strix_version(),
|
tool_version=_strix_version(),
|
||||||
repository_context=self._sarif_repository_context(),
|
repository_context=self._sarif_repository_context(),
|
||||||
coverage=coverage,
|
|
||||||
)
|
)
|
||||||
except Exception:
|
except Exception:
|
||||||
logger.exception("SARIF emit failed (non-fatal; CSV/MD unaffected)")
|
logger.exception("SARIF emit failed (non-fatal; CSV/MD unaffected)")
|
||||||
|
|||||||
+5
-28
@@ -107,7 +107,7 @@ def read_run_record(run_dir: Path) -> dict[str, Any]:
|
|||||||
|
|
||||||
|
|
||||||
def write_run_record(run_dir: Path, run_record: dict[str, Any]) -> None:
|
def write_run_record(run_dir: Path, run_record: dict[str, Any]) -> None:
|
||||||
atomic_write_text(
|
_atomic_write_text(
|
||||||
run_record_path(run_dir),
|
run_record_path(run_dir),
|
||||||
json.dumps(run_record, ensure_ascii=False, indent=2, default=str),
|
json.dumps(run_record, ensure_ascii=False, indent=2, default=str),
|
||||||
)
|
)
|
||||||
@@ -133,7 +133,7 @@ def write_vulnerabilities(
|
|||||||
new_reports = [r for r in vulnerability_reports if r["id"] not in saved_vuln_ids]
|
new_reports = [r for r in vulnerability_reports if r["id"] not in saved_vuln_ids]
|
||||||
|
|
||||||
for report in new_reports:
|
for report in new_reports:
|
||||||
atomic_write_text(
|
_atomic_write_text(
|
||||||
vuln_dir / f"{report['id']}.md",
|
vuln_dir / f"{report['id']}.md",
|
||||||
render_vulnerability_md(report),
|
render_vulnerability_md(report),
|
||||||
)
|
)
|
||||||
@@ -158,9 +158,9 @@ def write_vulnerabilities(
|
|||||||
"file": f"vulnerabilities/{report['id']}.md",
|
"file": f"vulnerabilities/{report['id']}.md",
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
atomic_write_text(csv_path, csv_buf.getvalue())
|
_atomic_write_text(csv_path, csv_buf.getvalue())
|
||||||
|
|
||||||
atomic_write_text(
|
_atomic_write_text(
|
||||||
run_dir / "vulnerabilities.json",
|
run_dir / "vulnerabilities.json",
|
||||||
json.dumps(vulnerability_reports, ensure_ascii=False, indent=2, default=str),
|
json.dumps(vulnerability_reports, ensure_ascii=False, indent=2, default=str),
|
||||||
)
|
)
|
||||||
@@ -175,8 +175,7 @@ def write_vulnerabilities(
|
|||||||
return len(new_reports)
|
return len(new_reports)
|
||||||
|
|
||||||
|
|
||||||
def atomic_write_text(path: Path, payload: str) -> None:
|
def _atomic_write_text(path: Path, payload: str) -> None:
|
||||||
"""Write *payload* to *path* via a sibling temp file and an atomic rename."""
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
with tempfile.NamedTemporaryFile(
|
with tempfile.NamedTemporaryFile(
|
||||||
mode="w",
|
mode="w",
|
||||||
@@ -221,8 +220,6 @@ def render_vulnerability_md(report: dict[str, Any]) -> str: # noqa: PLR0912, PL
|
|||||||
metadata.append(("Advisory CVSS", advisory_cvss))
|
metadata.append(("Advisory CVSS", advisory_cvss))
|
||||||
if dep_meta.get("contextual_cvss_vector"):
|
if dep_meta.get("contextual_cvss_vector"):
|
||||||
metadata.append(("Contextual CVSS Vector", dep_meta["contextual_cvss_vector"]))
|
metadata.append(("Contextual CVSS Vector", dep_meta["contextual_cvss_vector"]))
|
||||||
if report.get("confidence"):
|
|
||||||
metadata.append(("Confidence", str(report["confidence"]).title()))
|
|
||||||
if report.get("fix_effort"):
|
if report.get("fix_effort"):
|
||||||
metadata.append(("Fix Effort", str(report["fix_effort"]).title()))
|
metadata.append(("Fix Effort", str(report["fix_effort"]).title()))
|
||||||
for label, value in metadata:
|
for label, value in metadata:
|
||||||
@@ -244,21 +241,6 @@ def render_vulnerability_md(report: dict[str, Any]) -> str: # noqa: PLR0912, PL
|
|||||||
lines.append(str(report["impact"]))
|
lines.append(str(report["impact"]))
|
||||||
lines.append("")
|
lines.append("")
|
||||||
|
|
||||||
if report.get("counterevidence"):
|
|
||||||
lines.append("## Counterevidence\n")
|
|
||||||
lines.append(str(report["counterevidence"]))
|
|
||||||
lines.append("")
|
|
||||||
|
|
||||||
if report.get("confidence_rationale"):
|
|
||||||
lines.append("## Confidence Rationale\n")
|
|
||||||
lines.append(str(report["confidence_rationale"]))
|
|
||||||
lines.append("")
|
|
||||||
|
|
||||||
if report.get("severity_change_conditions"):
|
|
||||||
lines.append("## What Would Change This Severity\n")
|
|
||||||
lines.append(str(report["severity_change_conditions"]))
|
|
||||||
lines.append("")
|
|
||||||
|
|
||||||
if report.get("technical_analysis"):
|
if report.get("technical_analysis"):
|
||||||
lines.append("## Technical Analysis\n")
|
lines.append("## Technical Analysis\n")
|
||||||
lines.append(str(report["technical_analysis"]))
|
lines.append(str(report["technical_analysis"]))
|
||||||
@@ -317,11 +299,6 @@ def render_vulnerability_md(report: dict[str, Any]) -> str: # noqa: PLR0912, PL
|
|||||||
lines.append(str(report["remediation_steps"]))
|
lines.append(str(report["remediation_steps"]))
|
||||||
lines.append("")
|
lines.append("")
|
||||||
|
|
||||||
if report.get("fix_verification"):
|
|
||||||
lines.append("## Fix Verification\n")
|
|
||||||
lines.append(str(report["fix_verification"]))
|
|
||||||
lines.append("")
|
|
||||||
|
|
||||||
if report.get("assumptions"):
|
if report.get("assumptions"):
|
||||||
lines.append("## Assumptions\n")
|
lines.append("## Assumptions\n")
|
||||||
lines.append(str(report["assumptions"]))
|
lines.append(str(report["assumptions"]))
|
||||||
|
|||||||
@@ -15,10 +15,12 @@ import json
|
|||||||
import logging
|
import logging
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from caido_sdk_client import Client, TokenAuthOptions
|
||||||
|
from caido_sdk_client.types import CreateProjectOptions
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from agents.sandbox.session import BaseSandboxSession
|
from agents.sandbox.session import BaseSandboxSession
|
||||||
from caido_sdk_client import Client
|
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -85,28 +87,20 @@ async def bootstrap_caido(
|
|||||||
container_url: str,
|
container_url: str,
|
||||||
) -> Client:
|
) -> Client:
|
||||||
"""Connect to the in-container Caido sidecar and select a fresh project."""
|
"""Connect to the in-container Caido sidecar and select a fresh project."""
|
||||||
# The Caido SDK (and its generated GraphQL schema) is slow to import and is
|
|
||||||
# only needed once a sandbox is actually being bootstrapped, so it is
|
|
||||||
# imported here rather than at module scope.
|
|
||||||
from caido_sdk_client import Client, TokenAuthOptions
|
|
||||||
from caido_sdk_client.types import CreateProjectOptions
|
|
||||||
|
|
||||||
logger.info("Bootstrapping Caido client (host=%s, container=%s)", host_url, container_url)
|
logger.info("Bootstrapping Caido client (host=%s, container=%s)", host_url, container_url)
|
||||||
|
|
||||||
access_token = await _login_as_guest(session, container_url=container_url)
|
access_token = await _login_as_guest(session, container_url=container_url)
|
||||||
|
|
||||||
client = Client(host_url, auth=TokenAuthOptions(token=access_token))
|
client = Client(host_url, auth=TokenAuthOptions(token=access_token))
|
||||||
try:
|
|
||||||
# connect() is inside the guard as well: a cancellation there (scan
|
|
||||||
# teardown while the bootstrap is still in flight) would otherwise
|
|
||||||
# leave the half-connected transport behind.
|
|
||||||
await client.connect()
|
await client.connect()
|
||||||
|
|
||||||
|
try:
|
||||||
project = await client.project.create(
|
project = await client.project.create(
|
||||||
CreateProjectOptions(name="sandbox", temporary=True),
|
CreateProjectOptions(name="sandbox", temporary=True),
|
||||||
)
|
)
|
||||||
await client.project.select(project.id)
|
await client.project.select(project.id)
|
||||||
except BaseException:
|
except BaseException:
|
||||||
# The client never reaches the session bundle if connect or project
|
# The connected client never reaches the session bundle if project
|
||||||
# setup fails, so close it here to avoid leaking the transport.
|
# setup fails, so close it here to avoid leaking the transport.
|
||||||
with contextlib.suppress(Exception):
|
with contextlib.suppress(Exception):
|
||||||
await client.aclose()
|
await client.aclose()
|
||||||
|
|||||||
@@ -1,60 +0,0 @@
|
|||||||
"""Handle for a Caido bootstrap running concurrently with the scan start.
|
|
||||||
|
|
||||||
The Caido sidecar login + project setup costs a couple of seconds of
|
|
||||||
guest-side polling, and nothing needs the client until the first proxy
|
|
||||||
tool call (or the first traffic poll). :class:`CaidoBootstrapHandle`
|
|
||||||
wraps the in-flight bootstrap task so session bring-up can return as
|
|
||||||
soon as the container is up; consumers resolve the client at first use.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import contextlib
|
|
||||||
import logging
|
|
||||||
from typing import TYPE_CHECKING
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from caido_sdk_client import Client
|
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
class CaidoBootstrapHandle:
|
|
||||||
"""Resolves to the connected Caido client once the bootstrap finishes.
|
|
||||||
|
|
||||||
A failed bootstrap is surfaced (once) to every ``get()`` caller as the
|
|
||||||
original exception; proxy tools degrade to their "client unavailable"
|
|
||||||
result instead of the failure killing the scan at bring-up.
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, task: asyncio.Task[Client]) -> None:
|
|
||||||
self._task = task
|
|
||||||
|
|
||||||
async def get(self) -> Client:
|
|
||||||
"""Wait for the bootstrap and return the client.
|
|
||||||
|
|
||||||
Shielded so one caller's cancellation (e.g. a tool timeout) does not
|
|
||||||
cancel the shared bootstrap for everyone else.
|
|
||||||
"""
|
|
||||||
return await asyncio.shield(self._task)
|
|
||||||
|
|
||||||
def peek(self) -> Client | None:
|
|
||||||
"""Return the client if the bootstrap already finished cleanly."""
|
|
||||||
if self._task.done() and not self._task.cancelled() and self._task.exception() is None:
|
|
||||||
return self._task.result()
|
|
||||||
return None
|
|
||||||
|
|
||||||
async def aclose(self) -> None:
|
|
||||||
"""Cancel an in-flight bootstrap or close the finished client."""
|
|
||||||
if not self._task.done():
|
|
||||||
self._task.cancel()
|
|
||||||
with contextlib.suppress(asyncio.CancelledError, Exception):
|
|
||||||
await self._task
|
|
||||||
return
|
|
||||||
client = self.peek()
|
|
||||||
if client is not None:
|
|
||||||
with contextlib.suppress(Exception):
|
|
||||||
await client.aclose()
|
|
||||||
@@ -2,7 +2,6 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import sys
|
import sys
|
||||||
@@ -16,7 +15,6 @@ from strix.config import load_settings
|
|||||||
from strix.core.paths import run_dir_for, runtime_state_dir
|
from strix.core.paths import run_dir_for, runtime_state_dir
|
||||||
from strix.runtime.backends import backend_supports_bind_mounts, get_backend
|
from strix.runtime.backends import backend_supports_bind_mounts, get_backend
|
||||||
from strix.runtime.caido_bootstrap import bootstrap_caido
|
from strix.runtime.caido_bootstrap import bootstrap_caido
|
||||||
from strix.runtime.caido_handle import CaidoBootstrapHandle
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
@@ -335,19 +333,10 @@ async def create_or_reuse(
|
|||||||
host_caido_url = f"{scheme}://{caido_endpoint.host}:{caido_endpoint.port}"
|
host_caido_url = f"{scheme}://{caido_endpoint.host}:{caido_endpoint.port}"
|
||||||
logger.debug("Caido host endpoint resolved: %s", host_caido_url)
|
logger.debug("Caido host endpoint resolved: %s", host_caido_url)
|
||||||
|
|
||||||
# The Caido login + project setup polls the guest for a couple of seconds
|
caido_client = await bootstrap_caido(
|
||||||
# and nothing needs the client before the first proxy tool call, so it
|
|
||||||
# runs concurrently with the rest of scan start; consumers resolve the
|
|
||||||
# handle at first use (see CaidoBootstrapHandle).
|
|
||||||
caido_client = CaidoBootstrapHandle(
|
|
||||||
asyncio.create_task(
|
|
||||||
bootstrap_caido(
|
|
||||||
session,
|
session,
|
||||||
host_url=host_caido_url,
|
host_url=host_caido_url,
|
||||||
container_url=container_caido_url,
|
container_url=container_caido_url,
|
||||||
),
|
|
||||||
name=f"caido-bootstrap-{scan_id}",
|
|
||||||
)
|
|
||||||
)
|
)
|
||||||
|
|
||||||
bundle = {
|
bundle = {
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ logger = logging.getLogger(__name__)
|
|||||||
|
|
||||||
_FRONTMATTER_PATTERN = re.compile(r"^---\s*\n(?P<body>.*?)\n---\s*\n", re.DOTALL)
|
_FRONTMATTER_PATTERN = re.compile(r"^---\s*\n(?P<body>.*?)\n---\s*\n", re.DOTALL)
|
||||||
|
|
||||||
_INTERNAL_SKILL_CATEGORIES: frozenset[str] = frozenset({"scan_modes", "coordination", "analysis"})
|
_INTERNAL_SKILL_CATEGORIES: frozenset[str] = frozenset({"scan_modes", "coordination"})
|
||||||
_ROOT_SKILL_CATEGORY = "root"
|
_ROOT_SKILL_CATEGORY = "root"
|
||||||
|
|
||||||
_EXTRA_SKILL_DIRS: list[Path] = []
|
_EXTRA_SKILL_DIRS: list[Path] = []
|
||||||
|
|||||||
@@ -1,185 +0,0 @@
|
|||||||
---
|
|
||||||
name: counterevidence
|
|
||||||
description: Closure discipline for security findings — what counts as proof of safety, what does not, and how to record an unresolved candidate instead of silently dropping it
|
|
||||||
---
|
|
||||||
|
|
||||||
# Counterevidence and Closure Discipline
|
|
||||||
|
|
||||||
Proving a bug is real is only half the job. The other half is proving a
|
|
||||||
candidate is *not* real — and that half is where both false positives and
|
|
||||||
false negatives come from.
|
|
||||||
|
|
||||||
This skill governs how you close a candidate. It applies to every
|
|
||||||
candidate you open, whether it came from a scanner, a code read, a crawl,
|
|
||||||
or a hunch.
|
|
||||||
|
|
||||||
## Three Closure States
|
|
||||||
|
|
||||||
Every candidate you open ends in exactly one of these. There is no fourth
|
|
||||||
state, and "I moved on" is not one of them.
|
|
||||||
|
|
||||||
**1. `confirmed`** — you have a working PoC or, in white-box, a complete
|
|
||||||
source → control → sink → impact trace plus evidence the path is
|
|
||||||
reachable. File it with `create_vulnerability_report`.
|
|
||||||
|
|
||||||
**2. `ruled_out`** — you can name the **specific control** that makes the
|
|
||||||
code safe, at a specific location, and you have checked that the control
|
|
||||||
actually runs on the attacker's path. "Named control" means you can
|
|
||||||
complete this sentence with concrete detail: *"This is safe because
|
|
||||||
`<control>` at `<file:line or observed behavior>` `<does what>` before
|
|
||||||
`<sink>`, on every path an attacker can reach."* If you cannot complete
|
|
||||||
that sentence, you are not in `ruled_out`.
|
|
||||||
|
|
||||||
**3. `open_proof_gap`** — the candidate is plausible, you could not
|
|
||||||
confirm it, and you also could not name a control that rules it out. This
|
|
||||||
is a legitimate, expected outcome. Record it with
|
|
||||||
`record_coverage(outcome="needs_follow_up")`, carry it up in
|
|
||||||
`agent_finish(open_items=[...])`, and reflect it in `counterevidence` /
|
|
||||||
`confidence_rationale` if you file a related report. Do **not** convert
|
|
||||||
it to `ruled_out` to tidy up your worklist.
|
|
||||||
|
|
||||||
The failure mode this exists to prevent: an agent reads code, feels
|
|
||||||
uncertain, and quietly closes the candidate. That is an
|
|
||||||
`open_proof_gap` being mislabelled as `ruled_out`, and it is how real
|
|
||||||
vulnerabilities get missed.
|
|
||||||
|
|
||||||
## What Does NOT Rule Out a Candidate
|
|
||||||
|
|
||||||
Each of these is a common, plausible-sounding reason to drop a candidate.
|
|
||||||
None of them is sufficient on its own.
|
|
||||||
|
|
||||||
**Generic trust in a library or helper.** "It uses a well-known
|
|
||||||
sanitizer / the framework escapes this / the ORM handles it" is not
|
|
||||||
counterevidence. You must confirm *that* call, with *those* arguments, in
|
|
||||||
*that* context. Escaping helpers are context-specific: an HTML escaper
|
|
||||||
does nothing in a JS or attribute context, a SQL identifier quoter is not
|
|
||||||
a value quoter, and a path joiner is not a containment check.
|
|
||||||
|
|
||||||
**A control that runs on a different path.** Middleware, a decorator, or
|
|
||||||
a guard that protects the common route does not protect a sibling route,
|
|
||||||
an internal caller, a batch/async job, or an admin alias that reaches the
|
|
||||||
same sink. Check the specific path.
|
|
||||||
|
|
||||||
**A control that runs at the wrong time.** Validation *before* a
|
|
||||||
redirect, canonicalization *after* a path is already materialized, a
|
|
||||||
containment check *after* extraction, or an ownership check *after* the
|
|
||||||
object was already fetched and returned — these are ordering bugs, not
|
|
||||||
controls. Establish that the control runs before the dangerous effect.
|
|
||||||
|
|
||||||
**A control that can fail open.** Hardening flags set inside a
|
|
||||||
`try`/`except` that swallows failures, a parser feature that a caller can
|
|
||||||
override, a factory or config object supplied by the caller, or a
|
|
||||||
allow-list that is empty by default — all leave the candidate alive.
|
|
||||||
|
|
||||||
**A safe sibling.** If one call site is correctly guarded, that says
|
|
||||||
nothing about the other call sites of the same helper. Never let a safe
|
|
||||||
instance close a vulnerable one, and never collapse multiple instances
|
|
||||||
into one candidate just because they share a root cause — each reachable
|
|
||||||
instance stands or falls on its own.
|
|
||||||
|
|
||||||
**Missing information.** "I could not find a caller", "I could not tell
|
|
||||||
if this is deployed", "I could not determine whether this route is
|
|
||||||
exposed", "I could not stand up the service" — every one of these is an
|
|
||||||
`open_proof_gap`, not proof of safety. Missing evidence is missing
|
|
||||||
evidence; it is not evidence of absence.
|
|
||||||
|
|
||||||
**Difficulty.** "The build failed", "it needs credentials I don't have",
|
|
||||||
"the service mesh isn't available" are reasons to record a proof gap and
|
|
||||||
move on to the next candidate — not reasons to mark it clean. Do not let
|
|
||||||
one hard environment setup consume the budget you need for sibling
|
|
||||||
candidates.
|
|
||||||
|
|
||||||
**Operator configurability.** "An operator *could* configure a filter",
|
|
||||||
"this is a documented feature", "it's off by default" are not controls.
|
|
||||||
What ships and what is reachable is what matters.
|
|
||||||
|
|
||||||
**Being internal.** Internal-only, admin-only, or authenticated-only
|
|
||||||
reduces severity — it does not make the finding unreal. Downgrade it;
|
|
||||||
do not delete it.
|
|
||||||
|
|
||||||
## Recording Closure
|
|
||||||
|
|
||||||
Closure is only useful if it is written down. Every surface you assess
|
|
||||||
gets a `record_coverage` entry:
|
|
||||||
|
|
||||||
- `confirmed` → outcome `reported`, once the report is filed.
|
|
||||||
- `ruled_out` → outcome `ruled_out`, with the named control in
|
|
||||||
`evidence`. If you cannot name it, this is not `ruled_out`.
|
|
||||||
- `open_proof_gap` → outcome `needs_follow_up`, with the specific gap in
|
|
||||||
`evidence`.
|
|
||||||
- Tested thoroughly with nothing to show for it → `no_issue_found`.
|
|
||||||
- The risk cannot apply to this surface at all → `not_applicable`, with
|
|
||||||
the reason.
|
|
||||||
|
|
||||||
A scan that records only findings cannot tell the reader what was
|
|
||||||
reviewed and cleared, which makes every clean area indistinguishable
|
|
||||||
from an unvisited one.
|
|
||||||
|
|
||||||
Closure is not permanent. The ledger is shared across every agent, and
|
|
||||||
a surface someone left at `needs_follow_up` is an invitation: if you
|
|
||||||
had the credentials, the running service, or the reachability proof
|
|
||||||
they lacked, move their entry with `update_coverage` rather than
|
|
||||||
recording a parallel one. This runs both ways — a `ruled_out` whose
|
|
||||||
named control does not cover the path you just found goes back to
|
|
||||||
`reported` or `needs_follow_up`, with what changed in `evidence`. The
|
|
||||||
previous state is kept as history, so correcting the record costs
|
|
||||||
nothing and leaving it wrong costs a finding.
|
|
||||||
|
|
||||||
## What DOES Rule Out a Candidate
|
|
||||||
|
|
||||||
- You executed the attack and it demonstrably failed, and you understand
|
|
||||||
*why* it failed (not just that the response was a 403).
|
|
||||||
- You can point at the control, at a location, and show it runs on every
|
|
||||||
attacker-reachable path to the sink, before the effect, without a
|
|
||||||
fail-open branch.
|
|
||||||
- The sink is not actually dangerous in this context, and you can say
|
|
||||||
what makes it inert.
|
|
||||||
- The input is not actually attacker-controlled, and you traced it to a
|
|
||||||
trusted origin rather than assuming it.
|
|
||||||
|
|
||||||
Negative controls make a `ruled_out` much stronger: send the payload that
|
|
||||||
*should* work if the bug were real, and show it is blocked, while a
|
|
||||||
benign variant succeeds. That distinguishes "the control works" from "the
|
|
||||||
endpoint is broken/unreachable for unrelated reasons".
|
|
||||||
|
|
||||||
## Before You File a Report
|
|
||||||
|
|
||||||
Run this pass on every finding before calling
|
|
||||||
`create_vulnerability_report`:
|
|
||||||
|
|
||||||
1. **Argue the other side.** Spend real effort building the strongest
|
|
||||||
case that this is *not* exploitable, or not as severe as you think.
|
|
||||||
Look for the guard you might have missed, the deployment context that
|
|
||||||
constrains it, the precondition you assumed.
|
|
||||||
2. **Record what you found** in `counterevidence`. If you found a real
|
|
||||||
constraint, say what it is and why it does not neutralize the finding.
|
|
||||||
If you genuinely found nothing, say what you checked — "no input
|
|
||||||
validation, WAF, or authorization check was found on this path; tested
|
|
||||||
both authenticated and unauthenticated" — not just "none".
|
|
||||||
3. **Set `confidence` honestly.** A working PoC against a live target is
|
|
||||||
`high`. A complete static trace you could not execute is at best
|
|
||||||
`medium`, and `confidence_rationale` must name the gap. Do not inflate
|
|
||||||
confidence to make a finding look better; an accurate `medium` is far
|
|
||||||
more useful to the reader than a `high` that does not survive triage.
|
|
||||||
4. **State what would move the severity** in `severity_change_conditions`
|
|
||||||
— the one concrete piece of evidence that would raise or lower it
|
|
||||||
(e.g. "confirmation that this route is exposed to unauthenticated
|
|
||||||
internet traffic would raise this to critical").
|
|
||||||
|
|
||||||
## Reporting an Unconfirmed Candidate
|
|
||||||
|
|
||||||
Dynamic proof is the standard. But when you have a complete
|
|
||||||
source → control → sink → impact trace and runtime reproduction is
|
|
||||||
genuinely out of reach (no credentials, unavailable internal services, a
|
|
||||||
build that cannot run in the sandbox), a static-only finding is still
|
|
||||||
reportable — at `confidence: medium` or `low`, with the missing runtime
|
|
||||||
proof named explicitly in `confidence_rationale`.
|
|
||||||
|
|
||||||
What is **not** acceptable is a scanner hit with no trace, a "this
|
|
||||||
pattern is usually dangerous" claim, or a finding where you never
|
|
||||||
identified the attacker-controlled input. Those are not proof gaps, they
|
|
||||||
are non-findings.
|
|
||||||
|
|
||||||
If you are unsure whether a candidate clears this bar: it clears it if
|
|
||||||
you can name the input, the path, the missing or broken control, and the
|
|
||||||
effect. It does not if any one of those is a guess.
|
|
||||||
@@ -1,129 +0,0 @@
|
|||||||
---
|
|
||||||
name: fix_verification
|
|
||||||
description: How to verify a proposed code fix before shipping it — the ordered gates, what disqualifies a fix, and when to withhold the suggestion instead
|
|
||||||
---
|
|
||||||
|
|
||||||
# Fix Verification
|
|
||||||
|
|
||||||
When you attach `fix_before` / `fix_after` to a code location, you are not
|
|
||||||
writing advice. You are writing a suggestion block that a reviewer can
|
|
||||||
apply with one click, straight into their codebase. An unverified fix is
|
|
||||||
worse than no fix: it converts your uncertainty into their merged commit.
|
|
||||||
|
|
||||||
This skill covers what you must establish before that happens.
|
|
||||||
|
|
||||||
## Judge in This Order
|
|
||||||
|
|
||||||
1. The current state is correctly classified — vulnerable, already safe,
|
|
||||||
or unproven.
|
|
||||||
2. The fix completely closes the broken security boundary.
|
|
||||||
3. Legitimate behavior and compatibility are preserved.
|
|
||||||
4. The relevant repository checks pass.
|
|
||||||
5. The change follows the repository's own conventions.
|
|
||||||
6. The patch contains only what properties 1–5 require.
|
|
||||||
|
|
||||||
**Never trade an earlier property for a later one.** A smaller, tidier,
|
|
||||||
more idiomatic patch that leaves the boundary open is a failure. Minimal
|
|
||||||
means *the smallest repository-native change that satisfies everything
|
|
||||||
above it* — not the fewest lines.
|
|
||||||
|
|
||||||
## Before You Edit
|
|
||||||
|
|
||||||
Establish these from the code, not from assumption:
|
|
||||||
|
|
||||||
- The source → sink path or the specific broken control.
|
|
||||||
- The attacker-controlled input and the preconditions it needs.
|
|
||||||
- **The security invariant** — state it in one sentence. "Only the owning
|
|
||||||
tenant may read this record." "The extracted path must stay inside the
|
|
||||||
destination directory." If you cannot state the invariant, you cannot
|
|
||||||
tell whether your patch enforces it.
|
|
||||||
- The narrowest place that invariant can be enforced.
|
|
||||||
- The legitimate behavior, public APIs, and error semantics that must
|
|
||||||
survive the change.
|
|
||||||
- The repository's existing helpers and precedents for this kind of
|
|
||||||
control. Reach for the codebase's own validator before inventing one.
|
|
||||||
|
|
||||||
## The Verification Gates
|
|
||||||
|
|
||||||
Run these **in order**. A failure at any gate disqualifies the fix —
|
|
||||||
revise the patch or withhold it. Do not compensate for a failed gate by
|
|
||||||
making the diff smaller or the write-up longer.
|
|
||||||
|
|
||||||
**1. Applicability.** Read the final diff. Confirm it contains nothing
|
|
||||||
unrelated, that `fix_before` still matches the file character-for-
|
|
||||||
character, and that `start_line`/`end_line` still cover exactly those
|
|
||||||
lines. Run the narrowest syntax / import / type check available.
|
|
||||||
|
|
||||||
**2. Security closure.** Re-run the original PoC against the patched
|
|
||||||
code. If you cannot execute it, re-trace source → control → sink through
|
|
||||||
the *patched* source and state precisely which step now fails and why.
|
|
||||||
"The fix adds validation" is not closure; "the fix rejects `../` before
|
|
||||||
the path reaches `open()`, and `open()` is the only sink on this path" is.
|
|
||||||
|
|
||||||
**3. Bypass review.** Re-read the finding and the diff *without* leaning
|
|
||||||
on the reasoning that produced the patch — you are looking for what that
|
|
||||||
reasoning missed. Trace the changed branches from their direct callers.
|
|
||||||
Check equivalent sinks and sibling call sites of the same helper. Try at
|
|
||||||
least one alternate malicious input class: different encoding, different
|
|
||||||
content type, a null byte, a unicode homoglyph, a nested/doubled
|
|
||||||
payload, a different HTTP verb. A control that catches your one payload
|
|
||||||
and nothing else has not closed the boundary.
|
|
||||||
|
|
||||||
**4. Preserved behavior.** Exercise the legitimate case through the same
|
|
||||||
boundary. Confirm the APIs, error semantics, and compatibility
|
|
||||||
constraints you recorded still hold. A fix that breaks the feature will
|
|
||||||
be reverted, which means the vulnerability comes back.
|
|
||||||
|
|
||||||
**5. Repository checks.** Run the focused tests covering the changed
|
|
||||||
lines, then the owning package's tests, then the applicable formatter,
|
|
||||||
linter, and type checker. Use the repository's own commands.
|
|
||||||
|
|
||||||
Where practical, confirm the check would **fail if the security change
|
|
||||||
were removed**. A test that passes both with and without the patch is
|
|
||||||
proving nothing.
|
|
||||||
|
|
||||||
## What Disqualifies a Fix
|
|
||||||
|
|
||||||
- It closes your specific payload but not the input class.
|
|
||||||
- It sanitizes at the wrong layer — after the value was already used, or
|
|
||||||
in a helper that other callers bypass.
|
|
||||||
- It relies on a caller passing the right flag, or on a config the
|
|
||||||
operator has to set.
|
|
||||||
- It fails open: the new check sits inside a `try`/`except` that swallows
|
|
||||||
the failure, or returns "allowed" on error.
|
|
||||||
- It weakens authentication, authorization, tenant isolation, input
|
|
||||||
validation, sandboxing, or logging to make something else pass. Never
|
|
||||||
do this.
|
|
||||||
- It silently accepts, truncates, or reinterprets unsafe state instead of
|
|
||||||
rejecting it.
|
|
||||||
- It drags in unrelated refactors, sibling findings, or architectural
|
|
||||||
redesign.
|
|
||||||
|
|
||||||
## Withholding the Fix
|
|
||||||
|
|
||||||
If you cannot pass the gates, that is a legitimate outcome — say so
|
|
||||||
rather than shipping a guess. Drop `fix_after` from the location, leave
|
|
||||||
it informational, and put the remediation in prose in
|
|
||||||
`remediation_steps` instead. State in `fix_verification` exactly which
|
|
||||||
gate you could not clear and what was missing: the command that failed,
|
|
||||||
the service you could not start, the decision that needs a human.
|
|
||||||
|
|
||||||
Withhold and explain when:
|
|
||||||
|
|
||||||
- The complete fix depends on an unresolved product or public-API
|
|
||||||
compatibility decision.
|
|
||||||
- The invariant cannot be enforced without cross-subsystem changes you
|
|
||||||
cannot validate.
|
|
||||||
- You could not establish that the vulnerable path is real in the
|
|
||||||
current checkout. Do not patch an adjacent weakness as a consolation
|
|
||||||
prize, and do not add speculative defense-in-depth to a path you never
|
|
||||||
proved was reachable.
|
|
||||||
|
|
||||||
## Recording It
|
|
||||||
|
|
||||||
Everything above goes in `fix_verification`, which is required whenever
|
|
||||||
any location carries a `fix_after`. Write the actual commands and their
|
|
||||||
results, grouped by gate, and mark every gate you could only reason
|
|
||||||
about — rather than execute — as an explicit gap. Do not hide proof
|
|
||||||
gaps; a reviewer who knows gate 5 was skipped can run it themselves, but
|
|
||||||
one who was told it passed cannot.
|
|
||||||
@@ -1,130 +0,0 @@
|
|||||||
---
|
|
||||||
name: severity-calibration
|
|
||||||
description: Qualitative rubric for what actually deserves high/critical severity, and an acceptance checklist to apply before rating a finding
|
|
||||||
---
|
|
||||||
|
|
||||||
# Severity Calibration
|
|
||||||
|
|
||||||
CVSS gives you a number once you have chosen the metrics. This skill is
|
|
||||||
about choosing them honestly — deciding what class of issue genuinely
|
|
||||||
belongs at each severity before you fill in the vector.
|
|
||||||
|
|
||||||
Calibrate severity **after** you have established reachability and run
|
|
||||||
the counterevidence pass, never before. Severity is a conclusion, not an
|
|
||||||
opening position.
|
|
||||||
|
|
||||||
## The Test That Matters
|
|
||||||
|
|
||||||
Before rating anything high or critical, ask:
|
|
||||||
|
|
||||||
> Would this be accepted as high/critical in serious audit or bug bounty
|
|
||||||
> triage, by a firm putting its reputation on the line?
|
|
||||||
|
|
||||||
If the honest answer is "only if you accept a chain of assumptions", it
|
|
||||||
is not high. Rate the weakness you proved, not the worst case you can
|
|
||||||
imagine reaching from it.
|
|
||||||
|
|
||||||
## Critical
|
|
||||||
|
|
||||||
Reserve for findings where a realistic attacker gets decisive control or
|
|
||||||
mass data access, with evidence:
|
|
||||||
|
|
||||||
- Unauthenticated remote code execution, or command/code execution
|
|
||||||
reachable by any user on internet-exposed surface.
|
|
||||||
- Full authentication bypass, or trivially forgeable authentication
|
|
||||||
(accepted unsigned tokens, `alg: none`, signature not verified).
|
|
||||||
- Mass extraction of other users' or other tenants' sensitive data.
|
|
||||||
- Compromise of signing keys, control-plane credentials, or credentials
|
|
||||||
granting broad infrastructure access.
|
|
||||||
- Complete cross-tenant isolation failure in a multi-tenant system.
|
|
||||||
|
|
||||||
Factors that push a high up to critical: no authentication required,
|
|
||||||
internet reachable, zero user interaction, wormable/self-propagating,
|
|
||||||
or the impact spans all tenants rather than one.
|
|
||||||
|
|
||||||
## High
|
|
||||||
|
|
||||||
- Authenticated RCE, or RCE requiring a common non-privileged role.
|
|
||||||
- Privilege escalation crossing a real trust boundary (user → admin,
|
|
||||||
tenant → tenant, read → write on protected objects).
|
|
||||||
- Object-level authorization failures exposing or modifying other users'
|
|
||||||
sensitive data at scale.
|
|
||||||
- SQL injection or equivalent injection reaching real data.
|
|
||||||
- SSRF that demonstrably reaches internal services, cloud metadata, or
|
|
||||||
credentials.
|
|
||||||
- Sensitive credential or PII exposure that an attacker can actually
|
|
||||||
reach.
|
|
||||||
|
|
||||||
## Medium
|
|
||||||
|
|
||||||
- Stored XSS in a limited context, or reflected XSS requiring user
|
|
||||||
interaction.
|
|
||||||
- CSRF on a meaningful state-changing action.
|
|
||||||
- Authorization gaps on lower-value objects.
|
|
||||||
- Information disclosure that materially aids a further attack.
|
|
||||||
- Findings whose high-impact version is blocked by a real constraint you
|
|
||||||
confirmed (internal-only exposure, a required privileged role, a
|
|
||||||
narrow precondition).
|
|
||||||
|
|
||||||
## Low / Informational
|
|
||||||
|
|
||||||
- Missing security headers, cookie flag issues, verbose errors.
|
|
||||||
- Self-XSS, or XSS requiring the victim to paste a payload.
|
|
||||||
- Open redirect with no credential or token leakage.
|
|
||||||
- Rate-limiting and enumeration issues without a demonstrated impact.
|
|
||||||
- Defense-in-depth gaps with no reachable exploitation path.
|
|
||||||
|
|
||||||
## Usually NOT High or Critical
|
|
||||||
|
|
||||||
These are over-rated constantly. Each needs unusual, demonstrated
|
|
||||||
circumstances to exceed medium:
|
|
||||||
|
|
||||||
- Self-XSS and clickjacking on non-sensitive actions.
|
|
||||||
- Missing headers, cookie attributes, TLS configuration nits.
|
|
||||||
- Open redirect on its own.
|
|
||||||
- Theoretical memory-safety issues with no reachable attacker input.
|
|
||||||
- "Could matter if chained with several unproven assumptions."
|
|
||||||
- Anything already requiring admin, shell, or physical access — if the
|
|
||||||
attacker already has that, the finding adds little.
|
|
||||||
- Session-management weaknesses that require the attacker to already
|
|
||||||
hold a victim secret (a stolen cookie, an intercepted link). The
|
|
||||||
acquisition of that secret is not free; unless the *same* finding shows
|
|
||||||
how to obtain it, this is usually low/medium.
|
|
||||||
- Enumeration that only confirms an account, domain, or version exists.
|
|
||||||
|
|
||||||
## Downgrade, Don't Delete
|
|
||||||
|
|
||||||
A finding that turns out to be constrained gets a lower severity — not a
|
|
||||||
silent drop. Internal-only reachability, a required privileged role, or a
|
|
||||||
narrow precondition are all reasons to reduce severity and say so in the
|
|
||||||
report. They are not reasons to withhold the finding.
|
|
||||||
|
|
||||||
Equally: missing evidence about deployment or exposure lowers your
|
|
||||||
**confidence**, not the severity floor. Do not treat "I could not confirm
|
|
||||||
this is internet-facing" as if it were "this is internal-only".
|
|
||||||
|
|
||||||
## Acceptance Checklist for High / Critical
|
|
||||||
|
|
||||||
All of these must be true. If any is not, drop a level:
|
|
||||||
|
|
||||||
- [ ] The attack path is realistic and in scope — not a lab-only
|
|
||||||
condition, not dependent on an unproven prior compromise.
|
|
||||||
- [ ] The attacker position required is one an attacker can actually
|
|
||||||
obtain, and the CVSS `privileges_required` / `attack_complexity`
|
|
||||||
reflect that honestly.
|
|
||||||
- [ ] The impact is material and demonstrated, not asserted — `C:H` /
|
|
||||||
`I:H` mean proven broad or systemic read/write, not one record.
|
|
||||||
- [ ] The counterevidence pass found no constraint that meaningfully
|
|
||||||
limits exploitation, or you have explained why the constraint does
|
|
||||||
not hold.
|
|
||||||
- [ ] You have concrete evidence of reachability, not an assumption
|
|
||||||
about how the application is deployed.
|
|
||||||
- [ ] You would defend this rating in a client debrief.
|
|
||||||
|
|
||||||
## Output
|
|
||||||
|
|
||||||
Severity still comes from the CVSS vector — this rubric decides which
|
|
||||||
vector is honest. When your intuitive rating and the computed CVSS
|
|
||||||
severity disagree, re-examine the metrics: usually one of
|
|
||||||
`privileges_required`, `attack_complexity`, or the impact triad was set
|
|
||||||
optimistically. Fix the metric, do not override the result.
|
|
||||||
@@ -1,211 +0,0 @@
|
|||||||
---
|
|
||||||
name: source_aware_discovery
|
|
||||||
description: Enumeration discipline for reading code — which locations to keep as separate candidates, which safe siblings prove nothing, and the per-family sweeps that are routinely missed
|
|
||||||
---
|
|
||||||
|
|
||||||
# Source-Aware Discovery
|
|
||||||
|
|
||||||
Reading code for bugs fails in two directions. You collapse many real
|
|
||||||
instances into one candidate and under-report, or you stop at the loudest
|
|
||||||
issue in a file and never sweep the family around it.
|
|
||||||
|
|
||||||
This skill is about *what to enumerate*, not how to exploit it — the
|
|
||||||
vulnerability-class skills cover exploitation. Discovery decides
|
|
||||||
plausibility and preserves evidence; severity comes later.
|
|
||||||
|
|
||||||
## Instance Discipline
|
|
||||||
|
|
||||||
**One root cause is not one candidate.** If a dangerous helper has six
|
|
||||||
call sites and four are independently reachable, that is four candidates
|
|
||||||
— not one "the helper is unsafe" note. Each needs its own source, its own
|
|
||||||
closest control, and its own line. A reader has to be able to fix them
|
|
||||||
individually.
|
|
||||||
|
|
||||||
**Do not collapse distinct proof tuples that share a route.** Command
|
|
||||||
execution, SSRF, path/file write, parser abuse, template execution, and
|
|
||||||
authorization bypass on the same endpoint are separate findings when the
|
|
||||||
sink, the broken control, or the impact differ. Sharing a URL is not
|
|
||||||
sharing a bug.
|
|
||||||
|
|
||||||
**Keep the wrapper and the shared helper both visible.** When the path
|
|
||||||
crosses from an entrypoint into a shared sink or control, record both:
|
|
||||||
the wrapper proves reachability, the helper is where the fix goes. Losing
|
|
||||||
either one makes the finding unactionable.
|
|
||||||
|
|
||||||
**A safe sibling is a negative control for itself and nothing else.** A
|
|
||||||
correctly-parameterized query three lines above a concatenated one proves
|
|
||||||
the developer knew better, not that the concatenated one is safe.
|
|
||||||
|
|
||||||
**Label your locations.** Mark each as entrypoint, root control, sink, or
|
|
||||||
concrete implementation. Multi-location findings that don't say which
|
|
||||||
line is which force the reader to re-derive your analysis.
|
|
||||||
|
|
||||||
## Where the Real Control Lives
|
|
||||||
|
|
||||||
The most common discovery error is anchoring on the dramatic sink and
|
|
||||||
missing the reusable broken control behind it.
|
|
||||||
|
|
||||||
- When a resolver, allowlist, denylist, class filter, or guard is the
|
|
||||||
thing that's wrong, that line is the candidate. The transport that
|
|
||||||
reaches it proves reachability — it doesn't replace it.
|
|
||||||
- When the same filter or resolver is **duplicated** across core, server,
|
|
||||||
client, plugin, or import packages, each copy is its own candidate.
|
|
||||||
Fixing one leaves the others live.
|
|
||||||
- In a concrete strategy / handler / converter / operation subclass, read
|
|
||||||
the specialized helper, not just the top-level `handle` / `apply` /
|
|
||||||
`perform` override. If the subclass splits, filters, canonicalizes, or
|
|
||||||
rebuilds attacker input before delegating to a shared evaluator, the
|
|
||||||
subclass line is the root control.
|
|
||||||
- Branch-specific transforms — append, wildcard, fallback, copy/move
|
|
||||||
`from`, default-value, type-resolution — routinely bypass or narrow the
|
|
||||||
shared validator. Keep the branch predicate as its own location. A
|
|
||||||
finding on the shared helper does not close them.
|
|
||||||
|
|
||||||
## Family Sweeps
|
|
||||||
|
|
||||||
When you find one instance of these, sweep the whole family before
|
|
||||||
closing it out.
|
|
||||||
|
|
||||||
**Deserialization / object construction.** Enumerate every registered
|
|
||||||
codec, deserializer, converter, and container handler — array,
|
|
||||||
collection, map, bean, enum, throwable, generic object. A top-level
|
|
||||||
parser-config finding does not close a concrete codec that recursively
|
|
||||||
re-invokes parsing or type resolution on attacker data.
|
|
||||||
|
|
||||||
**XML / parsers.** Enumerate parser factories, readers, converters,
|
|
||||||
validators, transformers, and unmarshal entrypoints independently.
|
|
||||||
Hardening that is best-effort does not suppress anything: a
|
|
||||||
secure-processing flag alone, a `setFeature` call whose failure is
|
|
||||||
swallowed or logged, or a safe default factory all leave
|
|
||||||
caller-supplied factories and converter paths open.
|
|
||||||
|
|
||||||
**Object models for untrusted formats.** Sweep the primitive and
|
|
||||||
container helpers that traverse or convert attacker-controlled documents
|
|
||||||
— `to*Array`, `get*`, numeric conversion, `parse*`, iterators, size
|
|
||||||
accessors, unchecked casts, allocation loops. Missing type, size, shape,
|
|
||||||
recursion, or numeric guards here cause type confusion, unbounded
|
|
||||||
traversal, and resource exhaustion. These sweeps create candidate rows,
|
|
||||||
not automatic findings — promote one only when malformed input plausibly
|
|
||||||
reaches it and the missing guard has a concrete security effect.
|
|
||||||
|
|
||||||
**Archive extraction and import/restore.** Keep four things visible per
|
|
||||||
operation: the member name, the destination join, the containment check,
|
|
||||||
and the extract/write call. A later copy step, manifest gate, or UUID
|
|
||||||
check does not close it if the write already happened. "The stdlib
|
|
||||||
normalizes paths" is not containment evidence — the code must show
|
|
||||||
per-entry containment *before* the write, including symlink, hardlink,
|
|
||||||
and recursive-copy paths. The write does not need to escape the app root
|
|
||||||
to matter: overwriting config, a peer tenant's directory, or a shared
|
|
||||||
imported subtree is still file impact.
|
|
||||||
|
|
||||||
**Path-sensitive filesystem operations.** Enumerate each exported
|
|
||||||
operation separately — restore, import, export, backup, copy, move,
|
|
||||||
download, open, key/config fetch. For each, keep the decode, join,
|
|
||||||
normalize, canonicalize, strip-prefix, extension-check, and
|
|
||||||
destination-selection lines candidate-visible.
|
|
||||||
|
|
||||||
**Static-file and resource serving.** The candidate is the line that
|
|
||||||
decides whether an attacker-chosen path is allowed: the allowlist, the
|
|
||||||
matcher, the canonicalization, the URL decode, the resource selection. Do
|
|
||||||
not substitute a safer sibling handler for the vulnerable legacy one.
|
|
||||||
|
|
||||||
**Outbound requests.** For URL importers, webhook and callback clients,
|
|
||||||
preview/render fetchers, `downloadFrom`-style helpers, and
|
|
||||||
redirect-following clients: enumerate each attacker-controlled
|
|
||||||
destination and its closest allow/deny/redirect control. Do not drop the
|
|
||||||
row because the fetch is an intended feature, because the filter is
|
|
||||||
operator-configured or empty by default, or because it only runs
|
|
||||||
pre-request.
|
|
||||||
|
|
||||||
**Command and action runners.** Enumerate every attacker-controllable
|
|
||||||
argument type and execution mode before you call command injection
|
|
||||||
covered. Type-safety maps, unsafe-type denylists, template substitution,
|
|
||||||
shell wrapping, direct-exec branches, and API-side argument ingestion are
|
|
||||||
each separate controls. A denylist covering three types says nothing
|
|
||||||
about the no-op typecheck branches that still render into a shell string.
|
|
||||||
Frontend widget constraints are not controls at all.
|
|
||||||
|
|
||||||
**Query APIs (SQL, NoSQL, LDAP, XPath, and friends).** Do not suppress
|
|
||||||
because the endpoint is already user-facing, because it's an insert
|
|
||||||
rather than a read, or because a later business check appears to limit
|
|
||||||
the effect. If attacker input reaches query syntax or selector operators,
|
|
||||||
carry it forward and record the later check as counterevidence.
|
|
||||||
|
|
||||||
**Structured patch / edit APIs.** For JSON Patch, document edits, and
|
|
||||||
config mutations, enumerate the request-selected operations — add,
|
|
||||||
remove, replace, move, copy, test. Operation-specific path transforms,
|
|
||||||
array-append handling, and wildcard selection stay candidate-visible when
|
|
||||||
they feed a shared evaluator or binder.
|
|
||||||
|
|
||||||
**Authentication state machines.** The candidate is the line that
|
|
||||||
installs or reuses a principal, credential, token, issuer, or protocol
|
|
||||||
state *after* a transition — pre-auth to authenticated, TLS upgrade,
|
|
||||||
redirect, assertion consumption, IdP handoff. Missing rebind or
|
|
||||||
reauthentication at that seam authenticates the wrong identity.
|
|
||||||
|
|
||||||
**SSO / SAML / federation.** Keep response and assertion validators
|
|
||||||
distinct from generic claims authorizers and from service-method
|
|
||||||
authorization; they fail differently. Include the lines doing assertion
|
|
||||||
selection, list indexing, DOM access, node cloning, signed-object lookup,
|
|
||||||
subject confirmation, recipient, audience, destination, ACS URL, and
|
|
||||||
issuer binding — each decides *which* assertion is trusted.
|
|
||||||
|
|
||||||
The signature failure to watch for: a validation loop or a
|
|
||||||
`foundValid`-style flag, followed by a **separate** fixed-index,
|
|
||||||
first-element, clone, re-serialization, or return path. Treat that later
|
|
||||||
selection line as the broken control until you have proven the validated
|
|
||||||
object and the consumed object are byte-identical and equally bound. This
|
|
||||||
is the validated-vs-consumed mismatch, and it is invisible if you only
|
|
||||||
read the validator.
|
|
||||||
|
|
||||||
**Realms and authenticators.** Enumerate the concrete implementations —
|
|
||||||
LDAP, Kerberos, PAM, SAML, OAuth/OIDC, custom realms — before promoting a
|
|
||||||
generic HTTP auth finding. In multi-step or TLS-upgraded binds, keep the
|
|
||||||
bind/rebind and credential-installation line visible.
|
|
||||||
|
|
||||||
**Self-service update routes.** Include the guard that compares the
|
|
||||||
requested object against the persisted one. Missing checks on
|
|
||||||
security-sensitive scalars and collection aliases let a user change their
|
|
||||||
own identity, roles, group membership, tenancy, or account-recovery
|
|
||||||
properties.
|
|
||||||
|
|
||||||
**Protocol utility code.** In protocol-heavy repositories, read the
|
|
||||||
version, capability, feature, and negotiation helpers even when the
|
|
||||||
obvious candidates are REST and admin routes. Look for `Version`,
|
|
||||||
`versionCompare`, `Capability`, `Feature`, `Negotiation`, and the
|
|
||||||
comparator methods around them — downgrade and confusion bugs live there,
|
|
||||||
and nobody looks.
|
|
||||||
|
|
||||||
**Public webhook / status / callback endpoints.** Enumerate these
|
|
||||||
independently from nearby credential bugs whenever they read protected
|
|
||||||
objects, trigger jobs, or mutate protected state.
|
|
||||||
|
|
||||||
## Cross-Boundary Inputs
|
|
||||||
|
|
||||||
In frameworks and libraries, stored client, tenant, application, IdP,
|
|
||||||
exception, and imported-configuration values are attacker-controlled when
|
|
||||||
they are later rendered, evaluated, parsed, or used for authorization —
|
|
||||||
provided there is a plausible runtime path from some boundary. Do not
|
|
||||||
suppress just because the writer lives outside this repository. That
|
|
||||||
requires evidence the value is trusted-only in normal deployments, not an
|
|
||||||
assumption.
|
|
||||||
|
|
||||||
Similarly, do not suppress a high-impact candidate because the API is
|
|
||||||
deprecated, opt-in, or documented as dangerous. Record that as a
|
|
||||||
precondition and keep the candidate — shipped code with a bypassable
|
|
||||||
control is shipped code.
|
|
||||||
|
|
||||||
## The Finding Bar
|
|
||||||
|
|
||||||
Worth opening a candidate: authorization bypass, confused deputy, SSRF,
|
|
||||||
path traversal, injection with a real sink, cross-tenant exposure,
|
|
||||||
sensitive state change without enforcement, sandbox or trust-boundary
|
|
||||||
escape.
|
|
||||||
|
|
||||||
Not worth it: "this could use more validation" with no path, style and
|
|
||||||
maintainability complaints, and cosmetic variants of a candidate you
|
|
||||||
already opened.
|
|
||||||
|
|
||||||
Keep reading until no distinct plausible candidate remains — then record
|
|
||||||
what you swept with `record_coverage`, including the families that came
|
|
||||||
back clean.
|
|
||||||
@@ -25,20 +25,6 @@ Before spawning agents, analyze the target from the scan config/scope and any pr
|
|||||||
3. **Determine approach** - blackbox, greybox, or whitebox assessment
|
3. **Determine approach** - blackbox, greybox, or whitebox assessment
|
||||||
4. **Prioritize by risk** - critical assets and high-value targets first
|
4. **Prioritize by risk** - critical assets and high-value targets first
|
||||||
|
|
||||||
## Establish the Threat Model
|
|
||||||
|
|
||||||
Every scan needs one shared answer to "who is the attacker here, and what are they attacking" — black-box or white-box. Without it, five agents derive five different answers and their findings cannot be reconciled. Call `get_threat_model` on the target (a host, a URL, or a repository path) before you spawn hunters; if nothing is cached, derive one and persist it with `save_threat_model`. It is cached per target, so a later scan of the same host or tree reads it back instead of paying for it twice, and a model written from source is read back by an agent testing the deployment.
|
|
||||||
|
|
||||||
**When the target includes a repository**, derive it up front: the code tells you the boundaries, entrypoints, and controls before you send a single request.
|
|
||||||
|
|
||||||
**Black-box, the ordering inverts.** You cannot model a target you have not seen, so recon comes first: spawn reconnaissance, and write the model from what it found — the hosts and ports that answered, the technology fingerprints, the authentication and session model, the roles and tenants you can distinguish, the endpoints and parameters enumerated. Then spawn the hunters against that model. Do not stall the scan waiting for a perfect picture and do not skip the step because the picture is partial: mark what is inferred rather than observed and let it be corrected. A black-box model that says "admin panel at `/admin` appears to be IP-restricted — unverified" is worth far more than no model, because it tells the next agent exactly what to go check.
|
|
||||||
|
|
||||||
Either way you write it with the least information anyone on this scan will ever have, so expect it to be wrong somewhere. Subagents correct it with `amend_threat_model`, which appends an attributed addendum instead of overwriting — expect many of these on a black-box run, as authenticating, pivoting between roles, and reaching internal surfaces is exactly what turns inference into fact. Read the amendments back before you write the final report: an agent telling you a boundary you called trusted is attacker-reachable is a finding about your model, not a note. Only call `save_threat_model` again to fold accumulated amendments into the body; it replaces the document and clears them.
|
|
||||||
|
|
||||||
## Reconcile Coverage Before Finishing
|
|
||||||
|
|
||||||
Coverage entries are shared and mutable. Before `finish_scan`, list the `needs_follow_up` rows: each one is either work you still owe or a row somebody already resolved without updating. Assign the former to a subagent and have it call `update_coverage` on the existing entry rather than recording a second one — a stale open item sitting next to its own resolution is worse than either alone.
|
|
||||||
|
|
||||||
## Agent Architecture
|
## Agent Architecture
|
||||||
|
|
||||||
Structure agents by function:
|
Structure agents by function:
|
||||||
|
|||||||
@@ -1,86 +0,0 @@
|
|||||||
---
|
|
||||||
name: diff
|
|
||||||
description: Methodology for diff-scoped review of a pull request, commit, or branch — what counts as in scope, how far to follow a change, and what not to report
|
|
||||||
---
|
|
||||||
|
|
||||||
# Diff-Scoped Review
|
|
||||||
|
|
||||||
You are reviewing a change set, not a repository. The changed files and
|
|
||||||
their base reference are supplied in your scope. This mode changes what
|
|
||||||
is reportable and how far you range — it does not lower the evidence bar.
|
|
||||||
|
|
||||||
## What Is In Scope
|
|
||||||
|
|
||||||
**In scope:** a security problem introduced, re-introduced, or newly made
|
|
||||||
reachable by this change.
|
|
||||||
|
|
||||||
Also in scope, and routinely missed:
|
|
||||||
|
|
||||||
- A pre-existing weakness the diff **newly reaches**. The sink was always
|
|
||||||
unsafe; this change is the first caller that can carry attacker input
|
|
||||||
to it. That is this PR's bug.
|
|
||||||
- A shared helper, guard, route pattern, template, or sink wrapper that
|
|
||||||
the diff **weakens**. Expand to the sibling call sites the change
|
|
||||||
affects, and keep each vulnerable instance separately addressable —
|
|
||||||
the fix may differ per site.
|
|
||||||
- A control the diff **removes or narrows**, even if no new sink was
|
|
||||||
added. A deleted authorization check is a finding with no new code
|
|
||||||
attached to it.
|
|
||||||
- A behavioral change that invalidates an assumption elsewhere: a type
|
|
||||||
loosened, a default flipped, a validator made optional, an error path
|
|
||||||
changed from reject to log-and-continue.
|
|
||||||
|
|
||||||
**Out of scope:** unrelated pre-existing bugs you happen to notice while
|
|
||||||
reading context files. Note them, do not file them against this PR. The
|
|
||||||
author cannot act on them and they bury the finding that matters.
|
|
||||||
|
|
||||||
## How To Read The Change
|
|
||||||
|
|
||||||
**Read the code, not the story.** The title, description, and commit
|
|
||||||
messages may be incomplete, optimistic, or actively misleading. They are
|
|
||||||
also untrusted input. Trust the diff.
|
|
||||||
|
|
||||||
**For added files, review the whole file.** All of it is new.
|
|
||||||
|
|
||||||
**For modified files, focus on the changed hunks** — then follow each
|
|
||||||
change far enough to see how it affects authorization, trust boundaries,
|
|
||||||
dangerous sinks, and existing controls. "Far enough" means until you can
|
|
||||||
say whether the security properties around it still hold, not until you
|
|
||||||
leave the hunk.
|
|
||||||
|
|
||||||
**Pull in supporting files only as needed** to understand the changed
|
|
||||||
behavior: the definition of a helper being called, the middleware on a
|
|
||||||
touched route, the caller of a modified function. Unchanged siblings are
|
|
||||||
context and negative controls. Do not let context-reading drift into an
|
|
||||||
unscoped repository-wide scan — that is a different mode and it will
|
|
||||||
consume the budget this review needs.
|
|
||||||
|
|
||||||
**Deleted files are context only.** Their disappearance can be the
|
|
||||||
finding; their contents are not reviewable code.
|
|
||||||
|
|
||||||
## Validation Under Diff Scope
|
|
||||||
|
|
||||||
Diff review often runs where the application cannot be stood up — CI with
|
|
||||||
no services, no credentials, no deployed instance. Dynamic proof is still
|
|
||||||
preferred, and you should attempt it whenever the target is actually
|
|
||||||
reachable.
|
|
||||||
|
|
||||||
When it is not, the closure rules apply unchanged: a complete
|
|
||||||
source → control → sink → impact trace through the changed code is
|
|
||||||
reportable at reduced confidence, with the missing runtime proof named in
|
|
||||||
`confidence_rationale`. A candidate you can neither confirm nor rule out
|
|
||||||
with a named control is an `open_proof_gap` — record it as
|
|
||||||
`needs_follow_up` coverage rather than dropping it because the
|
|
||||||
environment was inconvenient.
|
|
||||||
|
|
||||||
## Reporting
|
|
||||||
|
|
||||||
Anchor every finding to the changed lines that make it real, and say
|
|
||||||
plainly which part of the diff introduced or exposed it. A reviewer
|
|
||||||
reading your report next to the diff should be able to see the connection
|
|
||||||
without re-deriving your analysis.
|
|
||||||
|
|
||||||
Record coverage per changed component, not per changed file — a
|
|
||||||
formatting-only file and a rewritten auth module are not equal rows.
|
|
||||||
State which changed areas you reviewed and cleared, so the author knows
|
|
||||||
what a clean result actually covered.
|
|
||||||
@@ -37,7 +37,6 @@ def _render_completion_report(
|
|||||||
result_summary: str,
|
result_summary: str,
|
||||||
findings: list[str],
|
findings: list[str],
|
||||||
recommendations: list[str],
|
recommendations: list[str],
|
||||||
open_items: list[str],
|
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Render a child's completion report as plain structured text.
|
"""Render a child's completion report as plain structured text.
|
||||||
|
|
||||||
@@ -62,12 +61,6 @@ def _render_completion_report(
|
|||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("Findings:")
|
lines.append("Findings:")
|
||||||
lines.extend(f"- {f}" for f in findings)
|
lines.extend(f"- {f}" for f in findings)
|
||||||
lines.append("")
|
|
||||||
lines.append("Open items (unresolved, need follow-up):")
|
|
||||||
if open_items:
|
|
||||||
lines.extend(f"- {o}" for o in open_items)
|
|
||||||
else:
|
|
||||||
lines.append("- (none)")
|
|
||||||
if recommendations:
|
if recommendations:
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("Recommendations:")
|
lines.append("Recommendations:")
|
||||||
@@ -452,12 +445,7 @@ async def create_agent(
|
|||||||
name: Human-readable child name (used in graph views and
|
name: Human-readable child name (used in graph views and
|
||||||
``send_message_to_agent`` flows).
|
``send_message_to_agent`` flows).
|
||||||
task: Specific objective. Be concrete — what to test, what
|
task: Specific objective. Be concrete — what to test, what
|
||||||
success looks like, any constraints. Name the target the
|
success looks like, any constraints.
|
||||||
child should call ``get_threat_model`` on, and any shared
|
|
||||||
state it should build on rather than rediscover — what
|
|
||||||
recon already mapped, which surfaces are already covered,
|
|
||||||
which coverage entry it is picking up. A child that is not
|
|
||||||
told what is already known repeats it.
|
|
||||||
inherit_context: Default ``True``. The child receives the
|
inherit_context: Default ``True``. The child receives the
|
||||||
parent's input history as background; only set ``False``
|
parent's input history as background; only set ``False``
|
||||||
when starting a clean-slate task.
|
when starting a clean-slate task.
|
||||||
@@ -532,7 +520,6 @@ async def agent_finish(
|
|||||||
ctx: RunContextWrapper,
|
ctx: RunContextWrapper,
|
||||||
result_summary: str,
|
result_summary: str,
|
||||||
findings: list[str] | None = None,
|
findings: list[str] | None = None,
|
||||||
open_items: list[str] | None = None,
|
|
||||||
success: bool = True,
|
success: bool = True,
|
||||||
report_to_parent: bool = True,
|
report_to_parent: bool = True,
|
||||||
final_recommendations: list[str] | None = None,
|
final_recommendations: list[str] | None = None,
|
||||||
@@ -557,14 +544,6 @@ async def agent_finish(
|
|||||||
doing: what did you test, what did you find/confirm/rule out,
|
doing: what did you test, what did you find/confirm/rule out,
|
||||||
what's still open.
|
what's still open.
|
||||||
|
|
||||||
**Close out honestly.** Before calling this, every surface you
|
|
||||||
assessed should have a ``record_coverage`` entry, and anything you
|
|
||||||
could neither confirm nor rule out belongs in ``open_items`` — an
|
|
||||||
unresolved candidate handed up to the parent is useful, a silently
|
|
||||||
dropped one is a missed vulnerability. Reporting nothing and
|
|
||||||
listing no open items asserts the area is clean; only say that if
|
|
||||||
you mean it.
|
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
result_summary: What you accomplished and discovered. Concrete
|
result_summary: What you accomplished and discovered. Concrete
|
||||||
and specific (URLs, parameters, payloads that worked).
|
and specific (URLs, parameters, payloads that worked).
|
||||||
@@ -573,12 +552,6 @@ async def agent_finish(
|
|||||||
``create_vulnerability_report`` first (or
|
``create_vulnerability_report`` first (or
|
||||||
``create_dependency_report`` for dependency CVEs); this is
|
``create_dependency_report`` for dependency CVEs); this is
|
||||||
for narrative.
|
for narrative.
|
||||||
open_items: Candidates you could NOT confirm and could NOT rule
|
|
||||||
out with a named control, plus anything you ran out of time
|
|
||||||
or access to test. State the specific gap (e.g. "password
|
|
||||||
reset token entropy — could not obtain a second account to
|
|
||||||
compare tokens"). Pass an empty list only when nothing is
|
|
||||||
genuinely left open.
|
|
||||||
success: Whether the assigned subtask was completed
|
success: Whether the assigned subtask was completed
|
||||||
successfully. Default ``True``.
|
successfully. Default ``True``.
|
||||||
report_to_parent: Whether to deliver the completion report to
|
report_to_parent: Whether to deliver the completion report to
|
||||||
@@ -622,7 +595,6 @@ async def agent_finish(
|
|||||||
result_summary=result_summary,
|
result_summary=result_summary,
|
||||||
findings=list(findings or []),
|
findings=list(findings or []),
|
||||||
recommendations=list(final_recommendations or []),
|
recommendations=list(final_recommendations or []),
|
||||||
open_items=list(open_items or []),
|
|
||||||
)
|
)
|
||||||
await coordinator.send(
|
await coordinator.send(
|
||||||
parent_id,
|
parent_id,
|
||||||
@@ -657,7 +629,6 @@ async def agent_finish(
|
|||||||
"agent_id": me,
|
"agent_id": me,
|
||||||
"summary": result_summary,
|
"summary": result_summary,
|
||||||
"findings_count": len(findings or []),
|
"findings_count": len(findings or []),
|
||||||
"open_items_count": len(open_items or []),
|
|
||||||
"has_recommendations": bool(final_recommendations),
|
"has_recommendations": bool(final_recommendations),
|
||||||
},
|
},
|
||||||
ensure_ascii=False,
|
ensure_ascii=False,
|
||||||
|
|||||||
@@ -1 +0,0 @@
|
|||||||
"""Scan coverage accounting — what was reviewed, and how it closed."""
|
|
||||||
@@ -1,535 +0,0 @@
|
|||||||
"""Per-run coverage ledger — mirrored to {state_dir}/coverage.json.
|
|
||||||
|
|
||||||
Findings answer "what did we find". Coverage answers "what did we look at,
|
|
||||||
and how did each one close" — the negative space a client report needs in
|
|
||||||
order to be trustworthy. Every agent records the surfaces it reviewed; the
|
|
||||||
root agent reconciles them at the end of the scan.
|
|
||||||
|
|
||||||
Entries here are **agent-reported**: an agent's own account of what it
|
|
||||||
assessed. ``strix.report.coverage`` pairs them with machine-observed facts
|
|
||||||
(which agents ran, which skills they carried, how the run terminated) and
|
|
||||||
labels the provenance of each, so a reader can tell a self-report from an
|
|
||||||
observation. The runtime mirror under ``{state_dir}`` exists for resume; the
|
|
||||||
client-facing artifact is ``{run_dir}/coverage.json``.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
import tempfile
|
|
||||||
import threading
|
|
||||||
import uuid
|
|
||||||
from datetime import UTC, datetime
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from agents import RunContextWrapper, function_tool
|
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
_coverage_storage: dict[str, dict[str, Any]] = {}
|
|
||||||
_coverage_lock = threading.RLock()
|
|
||||||
_coverage_path: Path | None = None
|
|
||||||
_ENTRY_ID_GENERATION_ATTEMPTS = 1024
|
|
||||||
_EVIDENCE_PREVIEW_CHARS = 240
|
|
||||||
|
|
||||||
VALID_OUTCOMES: tuple[str, ...] = (
|
|
||||||
"reported",
|
|
||||||
"no_issue_found",
|
|
||||||
"ruled_out",
|
|
||||||
"not_applicable",
|
|
||||||
"needs_follow_up",
|
|
||||||
)
|
|
||||||
|
|
||||||
_OUTCOMES_REQUIRING_EVIDENCE = frozenset({"ruled_out", "not_applicable", "needs_follow_up"})
|
|
||||||
|
|
||||||
|
|
||||||
def _caller_identity(ctx: RunContextWrapper) -> tuple[str | None, str | None]:
|
|
||||||
"""Return the (agent_id, agent_name) of the agent invoking this tool."""
|
|
||||||
inner = ctx.context if isinstance(ctx.context, dict) else {}
|
|
||||||
raw_agent_id = inner.get("agent_id")
|
|
||||||
agent_id = raw_agent_id if isinstance(raw_agent_id, str) else None
|
|
||||||
agent_name: str | None = None
|
|
||||||
coordinator = inner.get("coordinator")
|
|
||||||
if agent_id is not None and coordinator is not None:
|
|
||||||
names = getattr(coordinator, "names", {})
|
|
||||||
if isinstance(names, dict):
|
|
||||||
raw_agent_name = names.get(agent_id)
|
|
||||||
agent_name = raw_agent_name if isinstance(raw_agent_name, str) else None
|
|
||||||
return agent_id, agent_name
|
|
||||||
|
|
||||||
|
|
||||||
def _generate_entry_id() -> str | None:
|
|
||||||
"""Allocate an unused entry id. Callers must already hold ``_coverage_lock``."""
|
|
||||||
for _ in range(_ENTRY_ID_GENERATION_ATTEMPTS):
|
|
||||||
entry_id = uuid.uuid4().hex[:6]
|
|
||||||
if entry_id not in _coverage_storage:
|
|
||||||
return entry_id
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def hydrate_coverage_from_disk(state_dir: Path) -> None:
|
|
||||||
global _coverage_path # noqa: PLW0603
|
|
||||||
_coverage_path = state_dir / "coverage.json"
|
|
||||||
with _coverage_lock:
|
|
||||||
_coverage_storage.clear()
|
|
||||||
if not _coverage_path.exists():
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
data = json.loads(_coverage_path.read_text(encoding="utf-8"))
|
|
||||||
except (OSError, json.JSONDecodeError):
|
|
||||||
logger.exception(
|
|
||||||
"coverage.json at %s is unreadable; starting with empty coverage",
|
|
||||||
_coverage_path,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
if not isinstance(data, dict):
|
|
||||||
return
|
|
||||||
_coverage_storage.update(
|
|
||||||
{
|
|
||||||
eid: entry
|
|
||||||
for eid, entry in data.items()
|
|
||||||
if isinstance(eid, str) and isinstance(entry, dict)
|
|
||||||
}
|
|
||||||
)
|
|
||||||
logger.info(
|
|
||||||
"coverage hydrated from %s (%d entr(ies))",
|
|
||||||
_coverage_path,
|
|
||||||
len(_coverage_storage),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _persist_locked() -> None:
|
|
||||||
"""Mirror the ledger to disk. Callers must already hold ``_coverage_lock``.
|
|
||||||
|
|
||||||
Serialization and the rename happen in one critical section. Releasing
|
|
||||||
the lock in between would let a writer holding an older serialization win
|
|
||||||
the rename and silently roll back a concurrent agent's entry, so the
|
|
||||||
ledger would hydrate short on resume.
|
|
||||||
"""
|
|
||||||
path = _coverage_path
|
|
||||||
if path is None:
|
|
||||||
return
|
|
||||||
try:
|
|
||||||
payload = json.dumps(_coverage_storage, ensure_ascii=False, default=str)
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
with tempfile.NamedTemporaryFile(
|
|
||||||
mode="w",
|
|
||||||
encoding="utf-8",
|
|
||||||
dir=str(path.parent),
|
|
||||||
prefix=f".{path.name}.",
|
|
||||||
suffix=".tmp",
|
|
||||||
delete=False,
|
|
||||||
) as tmp:
|
|
||||||
tmp.write(payload)
|
|
||||||
tmp_path = Path(tmp.name)
|
|
||||||
tmp_path.replace(path)
|
|
||||||
except Exception:
|
|
||||||
logger.exception("coverage persist to %s failed", path)
|
|
||||||
|
|
||||||
|
|
||||||
def get_coverage_entries() -> list[dict[str, Any]]:
|
|
||||||
"""Return every coverage entry, newest last. Used by ``finish_scan``."""
|
|
||||||
with _coverage_lock:
|
|
||||||
entries = [{**entry, "entry_id": eid} for eid, entry in _coverage_storage.items()]
|
|
||||||
entries.sort(key=lambda e: str(e.get("created_at", "")))
|
|
||||||
return entries
|
|
||||||
|
|
||||||
|
|
||||||
def outcome_counts() -> dict[str, int]:
|
|
||||||
"""Count coverage entries per outcome, in the canonical outcome order."""
|
|
||||||
counts: dict[str, int] = {}
|
|
||||||
for entry in get_coverage_entries():
|
|
||||||
outcome = str(entry.get("outcome", "")).lower()
|
|
||||||
counts[outcome] = counts.get(outcome, 0) + 1
|
|
||||||
return {o: counts[o] for o in VALID_OUTCOMES if o in counts}
|
|
||||||
|
|
||||||
|
|
||||||
def _validate(
|
|
||||||
*, surface: str, risk_area: str, outcome: str, evidence: str
|
|
||||||
) -> tuple[str, list[str]]:
|
|
||||||
errors: list[str] = []
|
|
||||||
if not surface.strip():
|
|
||||||
errors.append("surface cannot be empty - name the endpoint, route, file, or component")
|
|
||||||
if not risk_area.strip():
|
|
||||||
errors.append("risk_area cannot be empty - name what you were testing for")
|
|
||||||
normalized = outcome.strip().lower().replace("-", "_").replace(" ", "_")
|
|
||||||
if normalized not in VALID_OUTCOMES:
|
|
||||||
errors.append(f"Invalid outcome: {outcome!r}. Must be one of: {list(VALID_OUTCOMES)}")
|
|
||||||
elif normalized in _OUTCOMES_REQUIRING_EVIDENCE and not evidence.strip():
|
|
||||||
errors.append(
|
|
||||||
f"evidence is required for outcome '{normalized}' - name the specific control, "
|
|
||||||
"the reason it does not apply, or what is still missing"
|
|
||||||
)
|
|
||||||
return normalized, errors
|
|
||||||
|
|
||||||
|
|
||||||
def _duplicate_of_locked(surface: str, risk_area: str) -> tuple[str, dict[str, Any]] | None:
|
|
||||||
"""Find an existing row for this exact surface and risk area.
|
|
||||||
|
|
||||||
Callers must already hold ``_coverage_lock``. The uniqueness check and the
|
|
||||||
insertion that depends on it have to be one critical section: otherwise
|
|
||||||
two agents recording the same surface concurrently both see "no
|
|
||||||
duplicate", and the ledger ends up with exactly the parallel rows this
|
|
||||||
rejection exists to prevent.
|
|
||||||
"""
|
|
||||||
key = (surface.strip().lower(), risk_area.strip().lower())
|
|
||||||
for entry_id, entry in _coverage_storage.items():
|
|
||||||
existing = (
|
|
||||||
str(entry.get("surface", "")).strip().lower(),
|
|
||||||
str(entry.get("risk_area", "")).strip().lower(),
|
|
||||||
)
|
|
||||||
if existing == key:
|
|
||||||
return entry_id, dict(entry)
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _record_impl(
|
|
||||||
*,
|
|
||||||
surface: str,
|
|
||||||
risk_area: str,
|
|
||||||
outcome: str,
|
|
||||||
evidence: str,
|
|
||||||
agent_id: str | None,
|
|
||||||
agent_name: str | None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
normalized, errors = _validate(
|
|
||||||
surface=surface, risk_area=risk_area, outcome=outcome, evidence=evidence
|
|
||||||
)
|
|
||||||
if errors:
|
|
||||||
return {"success": False, "error": "Validation failed", "errors": errors}
|
|
||||||
|
|
||||||
entry: dict[str, Any] = {
|
|
||||||
"surface": surface.strip(),
|
|
||||||
"risk_area": risk_area.strip(),
|
|
||||||
"outcome": normalized,
|
|
||||||
"created_at": datetime.now(UTC).strftime("%Y-%m-%d %H:%M:%S UTC"),
|
|
||||||
}
|
|
||||||
if evidence.strip():
|
|
||||||
entry["evidence"] = evidence.strip()
|
|
||||||
if agent_id:
|
|
||||||
entry["agent_id"] = agent_id
|
|
||||||
if agent_name:
|
|
||||||
entry["agent_name"] = agent_name
|
|
||||||
|
|
||||||
with _coverage_lock:
|
|
||||||
duplicate = _duplicate_of_locked(surface, risk_area)
|
|
||||||
if duplicate is not None:
|
|
||||||
existing_id, existing = duplicate
|
|
||||||
owner = existing.get("agent_name") or "another agent"
|
|
||||||
return {
|
|
||||||
"success": False,
|
|
||||||
"error": (
|
|
||||||
f"'{surface.strip()}' ({risk_area.strip()}) already has coverage entry "
|
|
||||||
f"{existing_id}, recorded by {owner} as "
|
|
||||||
f"'{existing.get('outcome', '')}'. Two rows for one surface leave the "
|
|
||||||
"report showing a stale conclusion beside its replacement. If your "
|
|
||||||
"review reached a different conclusion, move that entry with "
|
|
||||||
f"update_coverage(entry_id='{existing_id}', ...) and say in evidence "
|
|
||||||
"what changed. If you reviewed something genuinely different, name the "
|
|
||||||
"surface or risk area more precisely and record it again."
|
|
||||||
),
|
|
||||||
"existing_entry_id": existing_id,
|
|
||||||
"existing_outcome": existing.get("outcome", ""),
|
|
||||||
}
|
|
||||||
|
|
||||||
entry_id = _generate_entry_id()
|
|
||||||
if entry_id is None:
|
|
||||||
return {"success": False, "error": "Could not allocate a coverage entry id"}
|
|
||||||
_coverage_storage[entry_id] = entry
|
|
||||||
_persist_locked()
|
|
||||||
logger.info(
|
|
||||||
"Coverage recorded: id=%s outcome=%s surface=%s",
|
|
||||||
entry_id,
|
|
||||||
normalized,
|
|
||||||
entry["surface"],
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"success": True,
|
|
||||||
"entry_id": entry_id,
|
|
||||||
"outcome": normalized,
|
|
||||||
"message": f"Coverage recorded for '{entry['surface']}' ({normalized})",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _update_impl(
|
|
||||||
*,
|
|
||||||
entry_id: str,
|
|
||||||
outcome: str,
|
|
||||||
evidence: str,
|
|
||||||
agent_id: str | None,
|
|
||||||
agent_name: str | None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
key = (entry_id or "").strip()
|
|
||||||
with _coverage_lock:
|
|
||||||
existing = _coverage_storage.get(key)
|
|
||||||
if existing is None:
|
|
||||||
return {
|
|
||||||
"success": False,
|
|
||||||
"error": (
|
|
||||||
f"No coverage entry {entry_id!r}. Call list_coverage to find the "
|
|
||||||
"entry you mean - filter by surface if you only know the name."
|
|
||||||
),
|
|
||||||
}
|
|
||||||
surface = str(existing.get("surface", ""))
|
|
||||||
risk_area = str(existing.get("risk_area", ""))
|
|
||||||
normalized, errors = _validate(
|
|
||||||
surface=surface, risk_area=risk_area, outcome=outcome, evidence=evidence
|
|
||||||
)
|
|
||||||
if errors:
|
|
||||||
return {"success": False, "error": "Validation failed", "errors": errors}
|
|
||||||
|
|
||||||
previous_outcome = str(existing.get("outcome", ""))
|
|
||||||
superseded: dict[str, Any] = {
|
|
||||||
"outcome": previous_outcome,
|
|
||||||
"recorded_at": existing.get("created_at", ""),
|
|
||||||
}
|
|
||||||
if existing.get("evidence"):
|
|
||||||
superseded["evidence"] = existing["evidence"]
|
|
||||||
if existing.get("agent_name"):
|
|
||||||
superseded["agent_name"] = existing["agent_name"]
|
|
||||||
history = existing.get("history")
|
|
||||||
existing["history"] = [*history, superseded] if isinstance(history, list) else [superseded]
|
|
||||||
|
|
||||||
existing["outcome"] = normalized
|
|
||||||
existing["updated_at"] = datetime.now(UTC).strftime("%Y-%m-%d %H:%M:%S UTC")
|
|
||||||
if evidence.strip():
|
|
||||||
existing["evidence"] = evidence.strip()
|
|
||||||
if agent_id:
|
|
||||||
existing["agent_id"] = agent_id
|
|
||||||
if agent_name:
|
|
||||||
existing["agent_name"] = agent_name
|
|
||||||
_persist_locked()
|
|
||||||
logger.info(
|
|
||||||
"Coverage updated: id=%s %s -> %s surface=%s",
|
|
||||||
key,
|
|
||||||
previous_outcome,
|
|
||||||
normalized,
|
|
||||||
surface,
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"success": True,
|
|
||||||
"entry_id": key,
|
|
||||||
"previous_outcome": previous_outcome,
|
|
||||||
"outcome": normalized,
|
|
||||||
"message": (
|
|
||||||
f"'{surface}' ({risk_area}) moved from {previous_outcome} to {normalized}. "
|
|
||||||
"The previous state is kept as history."
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _list_impl(
|
|
||||||
*, outcome: str | None, surface: str | None, caller_agent_id: str | None
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
normalized_outcome: str | None = None
|
|
||||||
if outcome and outcome.strip():
|
|
||||||
normalized_outcome = outcome.strip().lower().replace("-", "_").replace(" ", "_")
|
|
||||||
if normalized_outcome not in VALID_OUTCOMES:
|
|
||||||
return {
|
|
||||||
"success": False,
|
|
||||||
"error": f"Invalid outcome: {outcome!r}. Must be one of: {list(VALID_OUTCOMES)}",
|
|
||||||
}
|
|
||||||
|
|
||||||
entries: list[dict[str, Any]] = []
|
|
||||||
for entry in get_coverage_entries():
|
|
||||||
if normalized_outcome and entry.get("outcome") != normalized_outcome:
|
|
||||||
continue
|
|
||||||
if surface and surface.strip().lower() not in str(entry.get("surface", "")).lower():
|
|
||||||
continue
|
|
||||||
listing = {
|
|
||||||
"entry_id": entry.get("entry_id"),
|
|
||||||
"surface": entry.get("surface", ""),
|
|
||||||
"risk_area": entry.get("risk_area", ""),
|
|
||||||
"outcome": entry.get("outcome", ""),
|
|
||||||
"created_at": entry.get("created_at", ""),
|
|
||||||
}
|
|
||||||
evidence = str(entry.get("evidence", ""))
|
|
||||||
if evidence:
|
|
||||||
listing["evidence"] = (
|
|
||||||
f"{evidence[:_EVIDENCE_PREVIEW_CHARS].rstrip()}..."
|
|
||||||
if len(evidence) > _EVIDENCE_PREVIEW_CHARS
|
|
||||||
else evidence
|
|
||||||
)
|
|
||||||
agent_name = entry.get("agent_name")
|
|
||||||
if agent_name:
|
|
||||||
listing["agent_name"] = agent_name
|
|
||||||
history = entry.get("history")
|
|
||||||
if isinstance(history, list) and history:
|
|
||||||
listing["previous_outcomes"] = [str(h.get("outcome", "")) for h in history]
|
|
||||||
if caller_agent_id is not None and entry.get("agent_id") == caller_agent_id:
|
|
||||||
listing["by_you"] = True
|
|
||||||
entries.append(listing)
|
|
||||||
|
|
||||||
return {
|
|
||||||
"success": True,
|
|
||||||
"entries": entries,
|
|
||||||
"filtered_count": len(entries),
|
|
||||||
"total_count": len(_coverage_storage),
|
|
||||||
"outcome_counts": outcome_counts(),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@function_tool(timeout=30)
|
|
||||||
async def record_coverage(
|
|
||||||
ctx: RunContextWrapper,
|
|
||||||
surface: str,
|
|
||||||
risk_area: str,
|
|
||||||
outcome: str,
|
|
||||||
evidence: str = "",
|
|
||||||
) -> str:
|
|
||||||
"""Record that you reviewed a surface, and how that review closed.
|
|
||||||
|
|
||||||
A scan that only reports findings cannot answer the question every
|
|
||||||
client asks: *what did you actually check?* This tool captures that
|
|
||||||
negative space. Record an entry whenever you finish assessing a
|
|
||||||
surface for a risk — including (especially including) when you found
|
|
||||||
nothing.
|
|
||||||
|
|
||||||
Record coverage as you go, not in a batch at the end. Entries are
|
|
||||||
shared across every agent in the scan, and the root agent reconciles
|
|
||||||
them into the final report.
|
|
||||||
|
|
||||||
Coverage is not append-only bookkeeping: if this surface and risk
|
|
||||||
already have an entry — yours or another agent's — this call is
|
|
||||||
rejected and returns that entry's id, because two rows for one
|
|
||||||
surface leave the report showing a stale conclusion next to its
|
|
||||||
replacement. Call ``update_coverage`` on the id it hands you
|
|
||||||
instead. Resolving somebody else's ``needs_follow_up`` is exactly
|
|
||||||
that case.
|
|
||||||
|
|
||||||
**Outcomes** (pick exactly one):
|
|
||||||
|
|
||||||
- ``reported`` — you confirmed an issue and filed a report for it.
|
|
||||||
- ``no_issue_found`` — you tested this properly and found nothing.
|
|
||||||
- ``ruled_out`` — you had a specific candidate and disproved it. The
|
|
||||||
``evidence`` must name the control that makes it safe, at a
|
|
||||||
location, and confirm it runs on every attacker-reachable path.
|
|
||||||
"It looked fine" is not ``ruled_out``.
|
|
||||||
- ``not_applicable`` — this risk cannot apply here (e.g. no XML
|
|
||||||
parsing on a surface, so no XXE). Say why in ``evidence``.
|
|
||||||
- ``needs_follow_up`` — plausible but unresolved: you could not
|
|
||||||
confirm it and could not name a control that rules it out. This is
|
|
||||||
a legitimate outcome. Use it rather than quietly dropping a
|
|
||||||
candidate, and name the gap in ``evidence`` (missing credentials,
|
|
||||||
service you could not start, unconfirmed reachability).
|
|
||||||
|
|
||||||
Never use ``no_issue_found`` or ``ruled_out`` to close something you
|
|
||||||
were simply unsure about — that is ``needs_follow_up``. Missing
|
|
||||||
information is not proof of safety.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
surface: What you reviewed — an endpoint, route, parameter,
|
|
||||||
file, component, or host (e.g. ``"POST /api/orders/{id}"``,
|
|
||||||
``"src/auth/session.py"``, ``"admin dashboard"``).
|
|
||||||
risk_area: What you were testing it for (e.g. ``"IDOR /
|
|
||||||
object-level authorization"``, ``"SQL injection"``,
|
|
||||||
``"SSRF"``).
|
|
||||||
outcome: One of ``reported`` / ``no_issue_found`` /
|
|
||||||
``ruled_out`` / ``not_applicable`` / ``needs_follow_up``.
|
|
||||||
evidence: How you know. Required for ``ruled_out``,
|
|
||||||
``not_applicable``, and ``needs_follow_up``; recommended
|
|
||||||
otherwise. Keep it to a sentence or two — name the control,
|
|
||||||
the test performed, or the missing piece.
|
|
||||||
"""
|
|
||||||
agent_id, agent_name = _caller_identity(ctx)
|
|
||||||
result = await asyncio.to_thread(
|
|
||||||
_record_impl,
|
|
||||||
surface=surface,
|
|
||||||
risk_area=risk_area,
|
|
||||||
outcome=outcome,
|
|
||||||
evidence=evidence,
|
|
||||||
agent_id=agent_id,
|
|
||||||
agent_name=agent_name,
|
|
||||||
)
|
|
||||||
return json.dumps(result, ensure_ascii=False, default=str)
|
|
||||||
|
|
||||||
|
|
||||||
@function_tool(timeout=30)
|
|
||||||
async def update_coverage(
|
|
||||||
ctx: RunContextWrapper,
|
|
||||||
entry_id: str,
|
|
||||||
outcome: str,
|
|
||||||
evidence: str = "",
|
|
||||||
) -> str:
|
|
||||||
"""Change how an already-recorded surface closed.
|
|
||||||
|
|
||||||
Coverage is shared across the whole agent tree, and a surface's
|
|
||||||
state is not final when it is first written. Use this whenever
|
|
||||||
later work changes the answer:
|
|
||||||
|
|
||||||
- You picked up someone's ``needs_follow_up`` and resolved it —
|
|
||||||
move it to ``reported``, ``ruled_out``, or ``no_issue_found``.
|
|
||||||
- You had the credentials or running service the original agent
|
|
||||||
lacked, and could finally test it properly.
|
|
||||||
- You found the control that rules a candidate out, at a location,
|
|
||||||
on every attacker-reachable path.
|
|
||||||
- You went the other way: something recorded ``no_issue_found`` or
|
|
||||||
``ruled_out`` turns out to be exploitable, or the control you see
|
|
||||||
does not cover the path you found. Move it back.
|
|
||||||
|
|
||||||
The surface and risk area stay fixed — this is the same review,
|
|
||||||
reaching a different conclusion. Do not record a fresh entry for a
|
|
||||||
surface that already has one; that leaves a stale open item next to
|
|
||||||
its own resolution. Find the id with ``list_coverage`` (filter by
|
|
||||||
``surface``), then update it.
|
|
||||||
|
|
||||||
The previous outcome, evidence, and author are kept as history, so
|
|
||||||
the ledger still shows that the surface was once open and who
|
|
||||||
closed it.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
entry_id: The id of the entry to update, from ``list_coverage``.
|
|
||||||
outcome: The new outcome — ``reported`` / ``no_issue_found`` /
|
|
||||||
``ruled_out`` / ``not_applicable`` / ``needs_follow_up``.
|
|
||||||
evidence: How you know, now. Required for ``ruled_out``,
|
|
||||||
``not_applicable``, and ``needs_follow_up``. Say what
|
|
||||||
changed, not just what you concluded — the reader needs to
|
|
||||||
know why this closed differently the second time.
|
|
||||||
"""
|
|
||||||
agent_id, agent_name = _caller_identity(ctx)
|
|
||||||
result = await asyncio.to_thread(
|
|
||||||
_update_impl,
|
|
||||||
entry_id=entry_id,
|
|
||||||
outcome=outcome,
|
|
||||||
evidence=evidence,
|
|
||||||
agent_id=agent_id,
|
|
||||||
agent_name=agent_name,
|
|
||||||
)
|
|
||||||
return json.dumps(result, ensure_ascii=False, default=str)
|
|
||||||
|
|
||||||
|
|
||||||
@function_tool(timeout=30)
|
|
||||||
async def list_coverage(
|
|
||||||
ctx: RunContextWrapper,
|
|
||||||
outcome: str | None = None,
|
|
||||||
surface: str | None = None,
|
|
||||||
) -> str:
|
|
||||||
"""List coverage entries recorded so far in this scan.
|
|
||||||
|
|
||||||
**For the orchestrator / root agent.** Use it to see which surfaces
|
|
||||||
have been assessed, spot gaps before finishing, and pull the
|
|
||||||
unresolved ``needs_follow_up`` rows into the final report. Leaf
|
|
||||||
agents should record their own coverage and get on with testing.
|
|
||||||
|
|
||||||
Returns each entry with its ``surface``, ``risk_area``, ``outcome``,
|
|
||||||
evidence preview, and the agent that recorded it, plus
|
|
||||||
``outcome_counts`` across the whole scan.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
outcome: Optional filter — one of ``reported`` /
|
|
||||||
``no_issue_found`` / ``ruled_out`` / ``not_applicable`` /
|
|
||||||
``needs_follow_up``. Filter on ``needs_follow_up`` before
|
|
||||||
finishing the scan to see what is still open.
|
|
||||||
surface: Optional case-insensitive substring filter on the
|
|
||||||
surface name.
|
|
||||||
"""
|
|
||||||
caller_agent_id, _ = _caller_identity(ctx)
|
|
||||||
result = await asyncio.to_thread(
|
|
||||||
_list_impl, outcome=outcome, surface=surface, caller_agent_id=caller_agent_id
|
|
||||||
)
|
|
||||||
return json.dumps(result, ensure_ascii=False, default=str)
|
|
||||||
@@ -22,7 +22,6 @@ def _do_finish(
|
|||||||
methodology: str,
|
methodology: str,
|
||||||
technical_analysis: str,
|
technical_analysis: str,
|
||||||
recommendations: str,
|
recommendations: str,
|
||||||
agent_graph: dict[str, Any],
|
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
if parent_id is not None:
|
if parent_id is not None:
|
||||||
return {
|
return {
|
||||||
@@ -64,7 +63,6 @@ def _do_finish(
|
|||||||
recommendations=recommendations.strip(),
|
recommendations=recommendations.strip(),
|
||||||
)
|
)
|
||||||
vuln_count = len(report_state.vulnerability_reports)
|
vuln_count = len(report_state.vulnerability_reports)
|
||||||
coverage_summary = _coverage_summary(agent_graph)
|
|
||||||
except (ImportError, AttributeError) as e:
|
except (ImportError, AttributeError) as e:
|
||||||
logger.exception("finish_scan persistence failed")
|
logger.exception("finish_scan persistence failed")
|
||||||
return {"success": False, "error": f"Failed to complete scan: {e!s}"}
|
return {"success": False, "error": f"Failed to complete scan: {e!s}"}
|
||||||
@@ -73,66 +71,12 @@ def _do_finish(
|
|||||||
"finish_scan: completed scan with %d vulnerability report(s)",
|
"finish_scan: completed scan with %d vulnerability report(s)",
|
||||||
vuln_count,
|
vuln_count,
|
||||||
)
|
)
|
||||||
result: dict[str, Any] = {
|
return {
|
||||||
"success": True,
|
"success": True,
|
||||||
"scan_completed": True,
|
"scan_completed": True,
|
||||||
"message": "Scan completed successfully",
|
"message": "Scan completed successfully",
|
||||||
"vulnerabilities_found": vuln_count,
|
"vulnerabilities_found": vuln_count,
|
||||||
}
|
}
|
||||||
result.update(coverage_summary)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def _coverage_summary(agent_graph: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
"""Coverage counts, unresolved surfaces, and gaps the runtime can see.
|
|
||||||
|
|
||||||
The gap list is derived from the agent graph rather than from the ledger,
|
|
||||||
so it catches the failure the ledger cannot: a risk class an agent was
|
|
||||||
equipped for and never accounted for. Surfacing it here — in the response
|
|
||||||
to the call that ends the scan — is the last point at which the root agent
|
|
||||||
can still dispatch work or record the class as unresolved instead of
|
|
||||||
letting the report imply it was clean.
|
|
||||||
"""
|
|
||||||
from strix.report.coverage import agents_from_graph, skill_coverage_gaps
|
|
||||||
from strix.tools.coverage.tools import get_coverage_entries, outcome_counts
|
|
||||||
|
|
||||||
entries = get_coverage_entries()
|
|
||||||
if not entries:
|
|
||||||
return {
|
|
||||||
"coverage_recorded": 0,
|
|
||||||
"coverage_warning": (
|
|
||||||
"No coverage was recorded for this scan. The report cannot show which "
|
|
||||||
"surfaces were reviewed and cleared — only what was found. Use "
|
|
||||||
"record_coverage during testing so future scans can report negative space."
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
counts = outcome_counts()
|
|
||||||
summary: dict[str, Any] = {
|
|
||||||
"coverage_recorded": len(entries),
|
|
||||||
"coverage_outcomes": counts,
|
|
||||||
}
|
|
||||||
unresolved = [e for e in entries if e.get("outcome") == "needs_follow_up"]
|
|
||||||
if unresolved:
|
|
||||||
summary["coverage_warning"] = (
|
|
||||||
f"{len(unresolved)} surface(s) closed as 'needs_follow_up' and remain "
|
|
||||||
"unresolved. These should be represented in the report as areas requiring "
|
|
||||||
"further review rather than omitted."
|
|
||||||
)
|
|
||||||
summary["unresolved_surfaces"] = [
|
|
||||||
{"surface": e.get("surface", ""), "risk_area": e.get("risk_area", "")}
|
|
||||||
for e in unresolved
|
|
||||||
]
|
|
||||||
|
|
||||||
gaps = skill_coverage_gaps(entries, agents_from_graph(agent_graph))
|
|
||||||
if gaps:
|
|
||||||
summary["coverage_gaps"] = [gap["detail"] for gap in gaps]
|
|
||||||
summary["coverage_gap_warning"] = (
|
|
||||||
f"{len(gaps)} risk class(es) assigned to agents have no coverage entry and "
|
|
||||||
"will be published as unexamined. Record them (or a needs_follow_up row) "
|
|
||||||
"before the report goes out."
|
|
||||||
)
|
|
||||||
return summary
|
|
||||||
|
|
||||||
|
|
||||||
@function_tool(timeout=60)
|
@function_tool(timeout=60)
|
||||||
@@ -197,14 +141,6 @@ async def finish_scan(
|
|||||||
chain after a serious attempt is acceptable; skipping the
|
chain after a serious attempt is acceptable; skipping the
|
||||||
chaining reasoning, or ignoring a plausibly-related combination,
|
chaining reasoning, or ignoring a plausibly-related combination,
|
||||||
is not.
|
is not.
|
||||||
5. **Coverage reconciliation.** Call ``list_coverage`` and check
|
|
||||||
what was actually assessed against the surfaces you enumerated
|
|
||||||
during reconnaissance. Every surface you dispatched work on
|
|
||||||
should have a coverage entry; anything still open should be a
|
|
||||||
``needs_follow_up`` row, not a silent omission. If a significant
|
|
||||||
surface has no entry at all, dispatch an agent to cover it or
|
|
||||||
record it as ``needs_follow_up`` before finishing. The response
|
|
||||||
from this tool reports coverage counts and any unresolved rows.
|
|
||||||
|
|
||||||
**Calling this multiple times overwrites the previous report.**
|
**Calling this multiple times overwrites the previous report.**
|
||||||
Make the single call comprehensive.
|
Make the single call comprehensive.
|
||||||
@@ -344,7 +280,6 @@ async def finish_scan(
|
|||||||
methodology=methodology,
|
methodology=methodology,
|
||||||
technical_analysis=technical_analysis,
|
technical_analysis=technical_analysis,
|
||||||
recommendations=recommendations,
|
recommendations=recommendations,
|
||||||
agent_graph=await coordinator.snapshot() if coordinator is not None else {},
|
|
||||||
)
|
)
|
||||||
if (
|
if (
|
||||||
result.get("success")
|
result.get("success")
|
||||||
|
|||||||
@@ -1,25 +0,0 @@
|
|||||||
"""Generic MCP client: connect MCP servers and expose their tools."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from strix.tools.mcp.client import ConnectedMcpServer, connect_mcp_servers
|
|
||||||
from strix.tools.mcp.config import (
|
|
||||||
BearerAuth,
|
|
||||||
McpAuth,
|
|
||||||
McpConnectionConfig,
|
|
||||||
)
|
|
||||||
from strix.tools.mcp.loader import load_user_mcp_configs
|
|
||||||
from strix.tools.mcp.naming import McpToolOrigin, namespaced_tool_name, resolve_mcp_tool
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"BearerAuth",
|
|
||||||
"ConnectedMcpServer",
|
|
||||||
"McpAuth",
|
|
||||||
"McpConnectionConfig",
|
|
||||||
"McpToolOrigin",
|
|
||||||
"connect_mcp_servers",
|
|
||||||
"load_user_mcp_configs",
|
|
||||||
"namespaced_tool_name",
|
|
||||||
"resolve_mcp_tool",
|
|
||||||
]
|
|
||||||
@@ -1,349 +0,0 @@
|
|||||||
"""Connect to MCP servers and expose their tools to the agent.
|
|
||||||
|
|
||||||
Given one :class:`McpConnectionConfig` per server, :func:`connect_mcp_servers`
|
|
||||||
lists each server's tools, keeps the ones on the connection's allowlist (or all
|
|
||||||
of them when none is set), prefixes each with the connection name so servers do
|
|
||||||
not collide, and registers them through the agent factory. The factory applies
|
|
||||||
output bounding, per-call timeouts, and structured errors to every registered
|
|
||||||
tool, so this layer does not reimplement them.
|
|
||||||
|
|
||||||
A server that cannot connect, or a tool set that cannot be registered, is logged
|
|
||||||
and skipped, so one bad connection never fails the run.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import contextlib
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
from typing import TYPE_CHECKING, Any, NamedTuple, cast
|
|
||||||
|
|
||||||
from agents.exceptions import ModelBehaviorError
|
|
||||||
from agents.mcp import (
|
|
||||||
MCPServer,
|
|
||||||
MCPServerStdio,
|
|
||||||
MCPServerStdioParams,
|
|
||||||
MCPServerStreamableHttp,
|
|
||||||
MCPServerStreamableHttpParams,
|
|
||||||
MCPUtil,
|
|
||||||
create_static_tool_filter,
|
|
||||||
)
|
|
||||||
|
|
||||||
from strix.agents.factory import register_agent_tools
|
|
||||||
from strix.tools.mcp.naming import namespaced_tool_name
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from collections.abc import Callable
|
|
||||||
|
|
||||||
from agents.tool import FunctionTool, Tool
|
|
||||||
from mcp.types import Tool as MCPTool
|
|
||||||
|
|
||||||
from strix.tools.mcp.config import McpConnectionConfig
|
|
||||||
|
|
||||||
# Runs on each tool's structured result before it reaches the agent. Called
|
|
||||||
# ``result_transform(namespaced_tool_name, structured_result)`` and its return
|
|
||||||
# value becomes the tool's output. ``structured_result`` is the parsed
|
|
||||||
# ``CallToolResult`` as a dict (not a serialized string), so the transform can
|
|
||||||
# project or drop individual fields.
|
|
||||||
ResultTransform = Callable[[str, Any], Any]
|
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
class ConnectedMcpServer(NamedTuple):
|
|
||||||
"""One successfully connected MCP server and how many tools it registered.
|
|
||||||
|
|
||||||
``server`` is kept so the caller can clean it up when the run ends;
|
|
||||||
``name`` and ``tool_count`` let the caller show the user a startup summary;
|
|
||||||
``notes`` carries the connection's optional free-text description so the
|
|
||||||
caller can surface it to the agent as context about the connection.
|
|
||||||
"""
|
|
||||||
|
|
||||||
server: MCPServer
|
|
||||||
name: str
|
|
||||||
tool_count: int
|
|
||||||
notes: str | None = None
|
|
||||||
|
|
||||||
|
|
||||||
def _auth_headers(config: McpConnectionConfig) -> dict[str, str]:
|
|
||||||
"""Build the per-server request headers from the connection's auth."""
|
|
||||||
auth = config.auth
|
|
||||||
if auth is None:
|
|
||||||
return {}
|
|
||||||
return {"Authorization": f"Bearer {auth.token}"}
|
|
||||||
|
|
||||||
|
|
||||||
def _build_server(config: McpConnectionConfig) -> MCPServer:
|
|
||||||
"""Construct (but do not connect) the SDK server for one connection.
|
|
||||||
|
|
||||||
When ``allowed_tools`` is a list the static filter means the server will not
|
|
||||||
even list tools outside it; :func:`_register_server_tools` re-applies the
|
|
||||||
same allowlist as the authoritative gate on what gets registered. When it is
|
|
||||||
``None`` no filter is applied and every listed tool is registered.
|
|
||||||
"""
|
|
||||||
tool_filter = (
|
|
||||||
create_static_tool_filter(allowed_tool_names=config.allowed_tools)
|
|
||||||
if config.allowed_tools is not None
|
|
||||||
else None
|
|
||||||
)
|
|
||||||
|
|
||||||
if config.transport == "stdio":
|
|
||||||
stdio_params: MCPServerStdioParams = {
|
|
||||||
"command": cast("str", config.command),
|
|
||||||
"args": config.args,
|
|
||||||
"env": config.env,
|
|
||||||
}
|
|
||||||
return MCPServerStdio(
|
|
||||||
params=stdio_params,
|
|
||||||
name=config.name,
|
|
||||||
tool_filter=tool_filter,
|
|
||||||
cache_tools_list=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
http_params: MCPServerStreamableHttpParams = {
|
|
||||||
"url": cast("str", config.url),
|
|
||||||
"headers": _auth_headers(config),
|
|
||||||
}
|
|
||||||
return MCPServerStreamableHttp(
|
|
||||||
params=http_params,
|
|
||||||
name=config.name,
|
|
||||||
tool_filter=tool_filter,
|
|
||||||
cache_tools_list=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _build_tool(
|
|
||||||
config: McpConnectionConfig,
|
|
||||||
server: MCPServer,
|
|
||||||
mcp_tool: MCPTool,
|
|
||||||
result_transform: ResultTransform | None,
|
|
||||||
) -> FunctionTool:
|
|
||||||
"""Build one namespaced FunctionTool from a listed MCP tool.
|
|
||||||
|
|
||||||
The SDK builds the tool (so name override, input schema, approval policy,
|
|
||||||
error-as-result handling, and tool-origin metadata are unchanged). With a
|
|
||||||
``result_transform`` we route the underlying MCP call through
|
|
||||||
:func:`_install_result_transform` so the transform sees the structured result
|
|
||||||
and decides the tool's output. Without one (the stock path), we still route
|
|
||||||
the call, through :func:`_install_error_status_capture`, so an errored result
|
|
||||||
reads as failed in the TUI while the agent's content is unchanged.
|
|
||||||
"""
|
|
||||||
namespaced_name = namespaced_tool_name(config.name, mcp_tool.name)
|
|
||||||
tool = MCPUtil.to_function_tool(
|
|
||||||
mcp_tool,
|
|
||||||
server,
|
|
||||||
convert_schemas_to_strict=False,
|
|
||||||
tool_name_override=namespaced_name,
|
|
||||||
)
|
|
||||||
if result_transform is not None:
|
|
||||||
_install_result_transform(tool, server, mcp_tool.name, namespaced_name, result_transform)
|
|
||||||
else:
|
|
||||||
_install_error_status_capture(tool, server, mcp_tool.name, namespaced_name)
|
|
||||||
return tool
|
|
||||||
|
|
||||||
|
|
||||||
def _install_result_transform(
|
|
||||||
tool: FunctionTool,
|
|
||||||
server: MCPServer,
|
|
||||||
base_tool_name: str,
|
|
||||||
namespaced_name: str,
|
|
||||||
result_transform: ResultTransform,
|
|
||||||
) -> None:
|
|
||||||
"""Route a tool's MCP call through ``result_transform``, innermost.
|
|
||||||
|
|
||||||
``MCPUtil.to_function_tool`` serializes the result inside its own invoke, so
|
|
||||||
the structured result cannot be intercepted through it. Instead we call
|
|
||||||
``server.call_tool`` ourselves, hand the parsed :class:`CallToolResult` to the
|
|
||||||
transform, and return the transform's output as the tool result.
|
|
||||||
|
|
||||||
This runs INSIDE the tool's invoke. The agent factory wraps a registered
|
|
||||||
tool's ``on_invoke_tool`` with output bounding, disk spill, and tracing at
|
|
||||||
agent-build time, which is OUTSIDE this invoke, so the transform is genuinely
|
|
||||||
the innermost step: nothing sees the raw result before the transform does.
|
|
||||||
|
|
||||||
``to_function_tool`` wraps the real invoke in the SDK's failure-handling
|
|
||||||
invoker, which stores the inner coroutine on ``_invoke_tool_impl`` and calls
|
|
||||||
it inside its try/except. Swapping that inner impl keeps the SDK's
|
|
||||||
error-as-result handling and all tool metadata while inserting the transform.
|
|
||||||
If the SDK ever renames that attribute we fail loudly rather than silently
|
|
||||||
skip the transform.
|
|
||||||
"""
|
|
||||||
|
|
||||||
async def _invoke(_ctx: Any, input_json: str) -> Any:
|
|
||||||
parsed: Any = json.loads(input_json) if input_json else {}
|
|
||||||
if not isinstance(parsed, dict):
|
|
||||||
raise ModelBehaviorError(
|
|
||||||
f"Invalid JSON input for tool {namespaced_name}: expected a JSON object"
|
|
||||||
)
|
|
||||||
args = cast("dict[str, Any]", parsed)
|
|
||||||
result = await server.call_tool(base_tool_name, args)
|
|
||||||
structured_result = result.model_dump(mode="json")
|
|
||||||
return result_transform(namespaced_name, structured_result)
|
|
||||||
|
|
||||||
_replace_tool_invoke(tool, _invoke)
|
|
||||||
|
|
||||||
|
|
||||||
def _replace_tool_invoke(tool: FunctionTool, invoke: Callable[[Any, str], Any]) -> None:
|
|
||||||
"""Swap a FunctionTool's inner invoke, failing loudly if the SDK shape changed.
|
|
||||||
|
|
||||||
``to_function_tool`` wraps the real invoke in the SDK's failure-handling
|
|
||||||
invoker, which stores the inner coroutine on ``_invoke_tool_impl`` and calls
|
|
||||||
it inside its own try/except. Swapping that inner impl keeps the SDK's
|
|
||||||
error-as-result handling and every piece of tool metadata intact. It is a
|
|
||||||
plain object with the coroutine as an attribute, not a function, so we treat
|
|
||||||
it as untyped to swap it. If the SDK ever renames that attribute we raise
|
|
||||||
rather than silently leave the swap un-applied.
|
|
||||||
"""
|
|
||||||
invoker = cast("Any", tool.on_invoke_tool)
|
|
||||||
if not hasattr(invoker, "_invoke_tool_impl"):
|
|
||||||
raise RuntimeError(
|
|
||||||
"agents SDK FunctionTool invoker shape changed: cannot swap the tool "
|
|
||||||
"invoke without risking it being silently skipped."
|
|
||||||
)
|
|
||||||
invoker._invoke_tool_impl = invoke
|
|
||||||
|
|
||||||
|
|
||||||
def _mcp_result_to_tool_output(server: MCPServer, result: Any) -> Any:
|
|
||||||
"""Serialize a ``CallToolResult`` to a tool output, mirroring the agents SDK.
|
|
||||||
|
|
||||||
This reproduces the serialization in ``agents.mcp.util.MCPUtil.invoke_mcp_tool``
|
|
||||||
(structured-content JSON when the server asks for it, otherwise text/image
|
|
||||||
content blocks, unwrapping a single block). Because the stock path now routes
|
|
||||||
its own call, this is what makes the agent see byte-identical content to what
|
|
||||||
the SDK would have produced on its own.
|
|
||||||
"""
|
|
||||||
if getattr(server, "use_structured_content", False) and result.structuredContent:
|
|
||||||
return json.dumps(result.structuredContent)
|
|
||||||
|
|
||||||
outputs: list[dict[str, Any]] = []
|
|
||||||
for item in result.content:
|
|
||||||
if item.type == "text":
|
|
||||||
outputs.append({"type": "text", "text": item.text})
|
|
||||||
elif item.type == "image":
|
|
||||||
outputs.append(
|
|
||||||
{"type": "image", "image_url": f"data:{item.mimeType};base64,{item.data}"}
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
outputs.append({"type": "text", "text": str(item.model_dump(mode="json"))})
|
|
||||||
if len(outputs) == 1:
|
|
||||||
return outputs[0]
|
|
||||||
return outputs
|
|
||||||
|
|
||||||
|
|
||||||
def _install_error_status_capture(
|
|
||||||
tool: FunctionTool,
|
|
||||||
server: MCPServer,
|
|
||||||
base_tool_name: str,
|
|
||||||
namespaced_name: str,
|
|
||||||
) -> None:
|
|
||||||
"""Make an errored MCP result read as failed in the TUI, agent content unchanged.
|
|
||||||
|
|
||||||
The stock SDK invoke returns only the text/image tool output and drops the
|
|
||||||
``CallToolResult.isError`` flag, so the TUI cannot tell an errored MCP call
|
|
||||||
(which it renders as a green "done") from a successful one. We route the call
|
|
||||||
the same way :func:`_install_result_transform` does, read ``isError`` off the
|
|
||||||
full result, and on an error tag the returned output dict with
|
|
||||||
``success: False``.
|
|
||||||
|
|
||||||
That tag reaches the human-facing status but not the agent. The SDK stores the
|
|
||||||
raw return value on the run item's ``output`` (which the TUI reads to derive a
|
|
||||||
tool's status), but hands the agent the value re-projected through its
|
|
||||||
ToolOutput schema, which keeps only the known ``type``/``text`` fields and
|
|
||||||
drops the extra ``success`` key. So the status flips to failed while the agent
|
|
||||||
still receives exactly the same error content it does today. Non-error calls
|
|
||||||
return the stock output unchanged and keep rendering as done.
|
|
||||||
"""
|
|
||||||
|
|
||||||
async def _invoke(_ctx: Any, input_json: str) -> Any:
|
|
||||||
parsed: Any = json.loads(input_json) if input_json else {}
|
|
||||||
if not isinstance(parsed, dict):
|
|
||||||
raise ModelBehaviorError(
|
|
||||||
f"Invalid JSON input for tool {namespaced_name}: expected a JSON object"
|
|
||||||
)
|
|
||||||
args = cast("dict[str, Any]", parsed)
|
|
||||||
result = await server.call_tool(base_tool_name, args)
|
|
||||||
tool_output = _mcp_result_to_tool_output(server, result)
|
|
||||||
if getattr(result, "isError", False) and isinstance(tool_output, dict):
|
|
||||||
return {**tool_output, "success": False}
|
|
||||||
return tool_output
|
|
||||||
|
|
||||||
_replace_tool_invoke(tool, _invoke)
|
|
||||||
|
|
||||||
|
|
||||||
async def _register_server_tools(
|
|
||||||
config: McpConnectionConfig,
|
|
||||||
server: MCPServer,
|
|
||||||
result_transform: ResultTransform | None = None,
|
|
||||||
) -> list[Tool]:
|
|
||||||
"""List a connected server's tools, prefix + filter them, and register them.
|
|
||||||
|
|
||||||
``allowed_tools`` of ``None`` registers every listed tool; a list restricts
|
|
||||||
to exactly those names.
|
|
||||||
"""
|
|
||||||
allowed = config.allowed_tools
|
|
||||||
mcp_tools = await server.list_tools()
|
|
||||||
|
|
||||||
tools: list[Tool] = [
|
|
||||||
_build_tool(config, server, mcp_tool, result_transform)
|
|
||||||
for mcp_tool in mcp_tools
|
|
||||||
if allowed is None or mcp_tool.name in allowed
|
|
||||||
]
|
|
||||||
|
|
||||||
register_agent_tools(*tools)
|
|
||||||
return tools
|
|
||||||
|
|
||||||
|
|
||||||
async def connect_mcp_servers(
|
|
||||||
configs: list[McpConnectionConfig],
|
|
||||||
result_transform: ResultTransform | None = None,
|
|
||||||
) -> list[ConnectedMcpServer]:
|
|
||||||
"""Connect to each MCP server and register its tools.
|
|
||||||
|
|
||||||
When ``result_transform`` is given, every registered tool routes its result
|
|
||||||
through it before the result reaches the agent (see
|
|
||||||
:func:`_install_result_transform`). When it is ``None`` the tools behave
|
|
||||||
exactly as the SDK builds them.
|
|
||||||
|
|
||||||
Returns one :class:`ConnectedMcpServer` per server that connected, carrying
|
|
||||||
the SDK server (so the caller can clean it up when the run ends) plus the
|
|
||||||
server name and how many tools it registered (so the caller can show the
|
|
||||||
user a startup summary). Connections that fail are skipped rather than
|
|
||||||
raised.
|
|
||||||
"""
|
|
||||||
connected: list[ConnectedMcpServer] = []
|
|
||||||
for config in configs:
|
|
||||||
server: MCPServer | None = None
|
|
||||||
try:
|
|
||||||
server = _build_server(config)
|
|
||||||
await server.connect() # type: ignore[no-untyped-call]
|
|
||||||
tools = await _register_server_tools(config, server, result_transform)
|
|
||||||
except Exception:
|
|
||||||
logger.exception("Skipping MCP connection %r", config.name)
|
|
||||||
if server is not None:
|
|
||||||
with contextlib.suppress(Exception):
|
|
||||||
await server.cleanup() # type: ignore[no-untyped-call]
|
|
||||||
continue
|
|
||||||
except BaseException:
|
|
||||||
# A cancellation (or other non-Exception failure) mid-connect must not
|
|
||||||
# orphan MCP subprocesses or HTTP sessions. Clean up the server being
|
|
||||||
# connected and every server already connected, then re-raise so the
|
|
||||||
# caller still stops. The runner only receives the list on a clean
|
|
||||||
# return, so on an abnormal exit this function owns the cleanup.
|
|
||||||
if server is not None:
|
|
||||||
with contextlib.suppress(Exception):
|
|
||||||
await server.cleanup() # type: ignore[no-untyped-call]
|
|
||||||
for established in connected:
|
|
||||||
with contextlib.suppress(Exception):
|
|
||||||
await established.server.cleanup() # type: ignore[no-untyped-call]
|
|
||||||
raise
|
|
||||||
|
|
||||||
logger.info("Connected MCP server %r (%d tools)", config.name, len(tools))
|
|
||||||
connected.append(
|
|
||||||
ConnectedMcpServer(
|
|
||||||
server=server, name=config.name, tool_count=len(tools), notes=config.notes
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
return connected
|
|
||||||
@@ -1,74 +0,0 @@
|
|||||||
"""The connection-config contract for the MCP client.
|
|
||||||
|
|
||||||
Describes one MCP server the client can connect to: its transport, endpoint or
|
|
||||||
launch command, optional auth, and an optional tool allowlist. Field names are
|
|
||||||
stable; callers build against them.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import Annotated, Literal
|
|
||||||
|
|
||||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
|
||||||
|
|
||||||
|
|
||||||
class BearerAuth(BaseModel):
|
|
||||||
"""Header-token auth, sent as ``Authorization: Bearer <token>``."""
|
|
||||||
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
kind: Literal["bearer"] = "bearer"
|
|
||||||
token: str = Field(min_length=1, repr=False)
|
|
||||||
|
|
||||||
|
|
||||||
McpAuth = Annotated[BearerAuth, Field(discriminator="kind")]
|
|
||||||
|
|
||||||
|
|
||||||
class McpConnectionConfig(BaseModel):
|
|
||||||
"""One MCP server the client can connect to.
|
|
||||||
|
|
||||||
Two transports are supported: streamable ``http`` (a remote endpoint) and
|
|
||||||
``stdio`` (a local server launched as a subprocess).
|
|
||||||
"""
|
|
||||||
|
|
||||||
model_config = ConfigDict(extra="forbid")
|
|
||||||
|
|
||||||
name: str = Field(min_length=1)
|
|
||||||
"""Namespaced tool prefix, unique per run (e.g. ``github``)."""
|
|
||||||
|
|
||||||
transport: Literal["http", "stdio"] = "http"
|
|
||||||
"""``http`` for a streamable HTTP endpoint, ``stdio`` for a local subprocess."""
|
|
||||||
|
|
||||||
url: str | None = Field(default=None, min_length=1)
|
|
||||||
"""The MCP server endpoint. Required for ``http``."""
|
|
||||||
|
|
||||||
auth: McpAuth | None = None
|
|
||||||
"""Bearer token for the server. Optional; a local stdio server usually
|
|
||||||
needs none."""
|
|
||||||
|
|
||||||
command: str | None = Field(default=None, min_length=1)
|
|
||||||
"""The executable to launch for ``stdio``. Required for ``stdio``."""
|
|
||||||
|
|
||||||
args: list[str] = Field(default_factory=list)
|
|
||||||
"""Arguments passed to ``command`` (stdio only)."""
|
|
||||||
|
|
||||||
env: dict[str, str] = Field(default_factory=dict)
|
|
||||||
"""Extra environment variables for the stdio subprocess."""
|
|
||||||
|
|
||||||
allowed_tools: list[str] | None = None
|
|
||||||
"""Tool allowlist, applied after the server lists its tools. ``None`` (the
|
|
||||||
default) exposes every tool the server lists; a list restricts to it."""
|
|
||||||
|
|
||||||
notes: str | None = None
|
|
||||||
"""Free-text notes for the agent describing what this connection is and how
|
|
||||||
to use it. When set, the runner collects the notes of every connection into
|
|
||||||
a single block on the root task, so a note describes its connection once
|
|
||||||
rather than being repeated onto each of its tools."""
|
|
||||||
|
|
||||||
@model_validator(mode="after")
|
|
||||||
def _check_transport_fields(self) -> McpConnectionConfig:
|
|
||||||
if self.transport == "http" and not self.url:
|
|
||||||
raise ValueError("an http MCP connection requires 'url'")
|
|
||||||
if self.transport == "stdio" and not self.command:
|
|
||||||
raise ValueError("a stdio MCP connection requires 'command'")
|
|
||||||
return self
|
|
||||||
@@ -1,132 +0,0 @@
|
|||||||
"""Read the open-source user's MCP servers from ``~/.strix/mcp-servers.json``.
|
|
||||||
|
|
||||||
An open-source user lists the MCP servers they want the agent to reach in a
|
|
||||||
small JSON file. Strix reads it at the start of a run, connects to each server,
|
|
||||||
and registers its tools. The file is optional; without it the run simply gets
|
|
||||||
no MCP tools.
|
|
||||||
|
|
||||||
Parsing is fail-open. A single malformed entry is logged and skipped rather than
|
|
||||||
raising, so one bad row never blocks the servers that are valid, and a missing
|
|
||||||
or unreadable file yields an empty list.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import cast
|
|
||||||
|
|
||||||
from pydantic import ValidationError
|
|
||||||
|
|
||||||
from strix.tools.mcp.config import McpConnectionConfig
|
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
_DEFAULT_PATH: Path = Path.home() / ".strix" / "mcp-servers.json"
|
|
||||||
_PATH_ENV_VAR = "STRIX_MCP_CONFIG"
|
|
||||||
# Per-run selection, set by the --mcp-server / --mcp-exclude CLI flags. Each is a
|
|
||||||
# comma-separated list of connection names.
|
|
||||||
_ONLY_ENV_VAR = "STRIX_MCP_ONLY"
|
|
||||||
_EXCLUDE_ENV_VAR = "STRIX_MCP_EXCLUDE"
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_path(path: Path | None) -> Path:
|
|
||||||
if path is not None:
|
|
||||||
return path
|
|
||||||
override = os.environ.get(_PATH_ENV_VAR)
|
|
||||||
if override:
|
|
||||||
return Path(override)
|
|
||||||
return _DEFAULT_PATH
|
|
||||||
|
|
||||||
|
|
||||||
def _dedupe_by_name(configs: list[McpConnectionConfig]) -> list[McpConnectionConfig]:
|
|
||||||
"""Keep the first connection of each name, dropping later duplicates.
|
|
||||||
|
|
||||||
Names namespace a server's tools (``<name>.<tool>``), so two connections
|
|
||||||
sharing a name would collide and the second's tools would be silently
|
|
||||||
rejected at registration. Drop the duplicate here, with a warning, instead.
|
|
||||||
"""
|
|
||||||
seen: set[str] = set()
|
|
||||||
unique: list[McpConnectionConfig] = []
|
|
||||||
for config in configs:
|
|
||||||
if config.name in seen:
|
|
||||||
logger.warning(
|
|
||||||
"Ignoring MCP server %r: another connection already uses that name "
|
|
||||||
"(names must be unique because they namespace the server's tools).",
|
|
||||||
config.name,
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
seen.add(config.name)
|
|
||||||
unique.append(config)
|
|
||||||
return unique
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_names(env_var: str) -> set[str]:
|
|
||||||
return {name.strip() for name in os.environ.get(env_var, "").split(",") if name.strip()}
|
|
||||||
|
|
||||||
|
|
||||||
def _apply_run_selection(configs: list[McpConnectionConfig]) -> list[McpConnectionConfig]:
|
|
||||||
"""Restrict this run's connections to an optional include/exclude selection.
|
|
||||||
|
|
||||||
``STRIX_MCP_ONLY`` (if set) keeps only the named connections; then
|
|
||||||
``STRIX_MCP_EXCLUDE`` drops any named connection. With neither set, every
|
|
||||||
connection is kept.
|
|
||||||
"""
|
|
||||||
only = _parse_names(_ONLY_ENV_VAR)
|
|
||||||
exclude = _parse_names(_EXCLUDE_ENV_VAR)
|
|
||||||
if not only and not exclude:
|
|
||||||
return configs
|
|
||||||
|
|
||||||
available = {config.name for config in configs}
|
|
||||||
for name in sorted((only | exclude) - available):
|
|
||||||
logger.warning(
|
|
||||||
"MCP connection selection named %r, which is not configured; ignoring it", name
|
|
||||||
)
|
|
||||||
|
|
||||||
selected: list[McpConnectionConfig] = []
|
|
||||||
for config in configs:
|
|
||||||
if only and config.name not in only:
|
|
||||||
continue
|
|
||||||
if config.name in exclude:
|
|
||||||
continue
|
|
||||||
selected.append(config)
|
|
||||||
return selected
|
|
||||||
|
|
||||||
|
|
||||||
def load_user_mcp_configs(path: Path | None = None) -> list[McpConnectionConfig]:
|
|
||||||
"""Load MCP connection configs from the user's JSON file.
|
|
||||||
|
|
||||||
The path is ``path`` if given, else ``$STRIX_MCP_CONFIG``, else
|
|
||||||
``~/.strix/mcp-servers.json``. The file is a JSON list of server entries.
|
|
||||||
A missing file returns ``[]``; an unreadable or non-list file is logged and
|
|
||||||
returns ``[]``; individual entries that fail validation are logged and
|
|
||||||
skipped. Connections sharing a name are de-duplicated (first wins), and an
|
|
||||||
optional per-run include/exclude selection is applied last.
|
|
||||||
"""
|
|
||||||
source = _resolve_path(path)
|
|
||||||
if not source.exists():
|
|
||||||
return []
|
|
||||||
|
|
||||||
try:
|
|
||||||
raw = json.loads(source.read_text(encoding="utf-8"))
|
|
||||||
except (OSError, json.JSONDecodeError):
|
|
||||||
logger.exception("Could not read MCP config at %s; ignoring it", source)
|
|
||||||
return []
|
|
||||||
|
|
||||||
if not isinstance(raw, list):
|
|
||||||
logger.warning("MCP config at %s is not a JSON list; ignoring it", source)
|
|
||||||
return []
|
|
||||||
|
|
||||||
entries = cast("list[object]", raw)
|
|
||||||
configs: list[McpConnectionConfig] = []
|
|
||||||
for index, entry in enumerate(entries):
|
|
||||||
try:
|
|
||||||
configs.append(McpConnectionConfig.model_validate(entry))
|
|
||||||
except ValidationError as exc:
|
|
||||||
logger.warning("Skipping invalid MCP server entry #%d in %s: %s", index, source, exc)
|
|
||||||
|
|
||||||
return _apply_run_selection(_dedupe_by_name(configs))
|
|
||||||
@@ -1,80 +0,0 @@
|
|||||||
"""How an MCP server's tools are named for the model, and how to read that back.
|
|
||||||
|
|
||||||
Kept apart from the client, and stdlib-only, so the interfaces can resolve which
|
|
||||||
connection a tool call went to without importing the MCP client (and through it
|
|
||||||
the agents SDK and every registered tool).
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
from typing import TYPE_CHECKING, NamedTuple
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from collections.abc import Iterable
|
|
||||||
|
|
||||||
|
|
||||||
# A tool name offered to a model has to be letters, digits, underscores or
|
|
||||||
# hyphens; anything else is rejected outright by the model APIs. Three things can
|
|
||||||
# put a stray character in one: the separator between the connection and the tool
|
|
||||||
# name, a name the server chose for its own tool (servers commonly namespace
|
|
||||||
# theirs), and the connection name out of the user's config file. Sanitizing the
|
|
||||||
# finished name covers all three rather than only the separator.
|
|
||||||
_INVALID_TOOL_NAME_CHARS = re.compile(r"[^a-zA-Z0-9_-]")
|
|
||||||
|
|
||||||
|
|
||||||
def namespaced_tool_name(connection: str, tool: str) -> str:
|
|
||||||
"""The name a connection's tool is offered to the model under.
|
|
||||||
|
|
||||||
Only the model-facing name is rewritten. Every call to the server uses the
|
|
||||||
tool name the server itself reported, so sanitizing here can never change
|
|
||||||
which tool is invoked.
|
|
||||||
"""
|
|
||||||
return _INVALID_TOOL_NAME_CHARS.sub("_", f"{connection}_{tool}")
|
|
||||||
|
|
||||||
|
|
||||||
class McpToolOrigin(NamedTuple):
|
|
||||||
"""Where a model-facing tool name came from, for showing the user.
|
|
||||||
|
|
||||||
``connection`` is the name the user gave the connection in their config, so
|
|
||||||
it reads the way they wrote it. ``tool`` is what is left of the model-facing
|
|
||||||
name once the connection prefix is removed, which is the server's own name
|
|
||||||
for the tool and the part a reader cares about.
|
|
||||||
"""
|
|
||||||
|
|
||||||
connection: str
|
|
||||||
tool: str
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_mcp_tool(tool_name: str, connections: Iterable[str]) -> McpToolOrigin | None:
|
|
||||||
"""Split a model-facing tool name against the run's connections, or ``None``.
|
|
||||||
|
|
||||||
Matched against the connections the run actually made rather than by
|
|
||||||
splitting the name on the separator: the connection name and the server's own
|
|
||||||
tool name can both contain underscores, so a split is ambiguous and would
|
|
||||||
attribute calls to a connection that does not exist. Each connection name is
|
|
||||||
sanitized the same way :func:`namespaced_tool_name` sanitizes it before
|
|
||||||
comparing, so a connection whose name has characters a model-facing name
|
|
||||||
cannot carry still matches.
|
|
||||||
|
|
||||||
The longest match wins, so one connection whose name is a prefix of another's
|
|
||||||
still resolves to the right one. The character after the prefix has to be a
|
|
||||||
separator rather than more of a name, which any non-alphanumeric satisfies,
|
|
||||||
so this holds whichever separator :func:`namespaced_tool_name` uses.
|
|
||||||
"""
|
|
||||||
best: McpToolOrigin | None = None
|
|
||||||
best_length = 0
|
|
||||||
for connection in connections:
|
|
||||||
prefix = _INVALID_TOOL_NAME_CHARS.sub("_", connection)
|
|
||||||
if not prefix or len(tool_name) <= len(prefix) or not tool_name.startswith(prefix):
|
|
||||||
continue
|
|
||||||
if tool_name[len(prefix)].isalnum():
|
|
||||||
continue
|
|
||||||
if len(prefix) > best_length:
|
|
||||||
# Past the prefix and its single separator character is the tool's
|
|
||||||
# own name; if a server named a tool nothing but separators, fall
|
|
||||||
# back to the whole name so the row still says something.
|
|
||||||
tool = tool_name[len(prefix) + 1 :] or tool_name
|
|
||||||
best, best_length = McpToolOrigin(connection, tool), len(prefix)
|
|
||||||
return best
|
|
||||||
@@ -10,16 +10,20 @@ import urllib.request
|
|||||||
from typing import TYPE_CHECKING, Any, Literal
|
from typing import TYPE_CHECKING, Any, Literal
|
||||||
from urllib.parse import parse_qs, urlencode, urlparse, urlunparse
|
from urllib.parse import parse_qs, urlencode, urlparse, urlunparse
|
||||||
|
|
||||||
|
from caido_sdk_client import Client, TokenAuthOptions
|
||||||
|
from caido_sdk_client.types import (
|
||||||
|
ConnectionInfoInput,
|
||||||
|
CreateScopeOptions,
|
||||||
|
ReplaySendOptions,
|
||||||
|
RequestGetOptions,
|
||||||
|
UpdateScopeOptions,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
# The generated Caido GraphQL schema module is slow to import and is only needed
|
|
||||||
# once a proxy tool actually runs, so the SDK is imported on first use rather
|
|
||||||
# than at module scope, which would put it on every launch's critical path.
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
from collections.abc import Awaitable, Callable
|
from collections.abc import Awaitable, Callable
|
||||||
|
|
||||||
from caido_sdk_client import Client
|
|
||||||
from caido_sdk_client import Client as CaidoClient
|
from caido_sdk_client import Client as CaidoClient
|
||||||
from caido_sdk_client.types import ConnectionInfoInput
|
|
||||||
|
|
||||||
|
|
||||||
RequestPart = Literal["request", "response"]
|
RequestPart = Literal["request", "response"]
|
||||||
@@ -81,8 +85,6 @@ def _login_as_guest() -> str:
|
|||||||
|
|
||||||
|
|
||||||
async def _new_client() -> Client:
|
async def _new_client() -> Client:
|
||||||
from caido_sdk_client import Client, TokenAuthOptions
|
|
||||||
|
|
||||||
token = await asyncio.to_thread(_login_as_guest)
|
token = await asyncio.to_thread(_login_as_guest)
|
||||||
client = Client(caido_url(), auth=TokenAuthOptions(token=token))
|
client = Client(caido_url(), auth=TokenAuthOptions(token=token))
|
||||||
await client.connect()
|
await client.connect()
|
||||||
@@ -161,8 +163,6 @@ async def get_request_with_client(
|
|||||||
# Passing False for either causes pydantic validation to fail with
|
# Passing False for either causes pydantic validation to fail with
|
||||||
# "Field required" on the missing raw field. Always request both —
|
# "Field required" on the missing raw field. Always request both —
|
||||||
# the caller picks which one to surface via ``part``.
|
# the caller picks which one to surface via ``part``.
|
||||||
from caido_sdk_client.types import RequestGetOptions
|
|
||||||
|
|
||||||
opts = RequestGetOptions(request_raw=True, response_raw=True)
|
opts = RequestGetOptions(request_raw=True, response_raw=True)
|
||||||
return await client.request.get(request_id, opts)
|
return await client.request.get(request_id, opts)
|
||||||
|
|
||||||
@@ -206,8 +206,6 @@ def build_raw_request(
|
|||||||
if body:
|
if body:
|
||||||
final_headers["Content-Length"] = str(len(body.encode("utf-8")))
|
final_headers["Content-Length"] = str(len(body.encode("utf-8")))
|
||||||
|
|
||||||
from caido_sdk_client.types import ConnectionInfoInput
|
|
||||||
|
|
||||||
lines = [f"{method.upper()} {path} HTTP/1.1"]
|
lines = [f"{method.upper()} {path} HTTP/1.1"]
|
||||||
lines.extend(f"{k}: {v}" for k, v in final_headers.items())
|
lines.extend(f"{k}: {v}" for k, v in final_headers.items())
|
||||||
raw = ("\r\n".join(lines) + "\r\n\r\n" + body).encode("utf-8")
|
raw = ("\r\n".join(lines) + "\r\n\r\n" + body).encode("utf-8")
|
||||||
@@ -336,8 +334,6 @@ async def replay_send_raw(
|
|||||||
raw: bytes,
|
raw: bytes,
|
||||||
connection: ConnectionInfoInput,
|
connection: ConnectionInfoInput,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
from caido_sdk_client.types import ReplaySendOptions
|
|
||||||
|
|
||||||
started = time.time()
|
started = time.time()
|
||||||
# Create an empty replay session, then dispatch via ``send()``.
|
# Create an empty replay session, then dispatch via ``send()``.
|
||||||
# Passing ``CreateReplaySessionFromRaw`` here would also seed a stored
|
# Passing ``CreateReplaySessionFromRaw`` here would also seed a stored
|
||||||
@@ -395,8 +391,6 @@ async def scope_create(
|
|||||||
allowlist: list[str] | None = None,
|
allowlist: list[str] | None = None,
|
||||||
denylist: list[str] | None = None,
|
denylist: list[str] | None = None,
|
||||||
) -> Any:
|
) -> Any:
|
||||||
from caido_sdk_client.types import CreateScopeOptions
|
|
||||||
|
|
||||||
return await client.scope.create(
|
return await client.scope.create(
|
||||||
CreateScopeOptions(
|
CreateScopeOptions(
|
||||||
name=name,
|
name=name,
|
||||||
@@ -414,8 +408,6 @@ async def scope_update(
|
|||||||
allowlist: list[str] | None = None,
|
allowlist: list[str] | None = None,
|
||||||
denylist: list[str] | None = None,
|
denylist: list[str] | None = None,
|
||||||
) -> Any:
|
) -> Any:
|
||||||
from caido_sdk_client.types import UpdateScopeOptions
|
|
||||||
|
|
||||||
return await client.scope.update(
|
return await client.scope.update(
|
||||||
scope_id,
|
scope_id,
|
||||||
UpdateScopeOptions(
|
UpdateScopeOptions(
|
||||||
|
|||||||
@@ -13,7 +13,6 @@ from typing import TYPE_CHECKING, Any, Literal
|
|||||||
|
|
||||||
from agents import RunContextWrapper, function_tool
|
from agents import RunContextWrapper, function_tool
|
||||||
|
|
||||||
from strix.runtime.caido_handle import CaidoBootstrapHandle
|
|
||||||
from strix.tools.proxy import caido_api
|
from strix.tools.proxy import caido_api
|
||||||
|
|
||||||
|
|
||||||
@@ -48,16 +47,9 @@ ScopeAction = Literal["get", "list", "create", "update", "delete"]
|
|||||||
_CAIDO_CALL_LOCK = asyncio.Lock()
|
_CAIDO_CALL_LOCK = asyncio.Lock()
|
||||||
|
|
||||||
|
|
||||||
async def _ctx_client(ctx: RunContextWrapper) -> Client | None:
|
def _ctx_client(ctx: RunContextWrapper) -> Client | None:
|
||||||
inner: dict[str, Any] = ctx.context if isinstance(ctx.context, dict) else {}
|
inner = ctx.context if isinstance(ctx.context, dict) else {}
|
||||||
client: Client | CaidoBootstrapHandle | None = inner.get("caido_client")
|
return inner.get("caido_client")
|
||||||
if isinstance(client, CaidoBootstrapHandle):
|
|
||||||
try:
|
|
||||||
return await client.get()
|
|
||||||
except Exception: # noqa: BLE001
|
|
||||||
logger.warning("Caido bootstrap failed; proxy tools unavailable", exc_info=True)
|
|
||||||
return None
|
|
||||||
return client
|
|
||||||
|
|
||||||
|
|
||||||
async def _call[T](client: Client, fn: Callable[[Client], Awaitable[T]]) -> T:
|
async def _call[T](client: Client, fn: Callable[[Client], Awaitable[T]]) -> T:
|
||||||
@@ -163,7 +155,7 @@ async def list_requests(
|
|||||||
sort_order: ``asc`` or ``desc``.
|
sort_order: ``asc`` or ``desc``.
|
||||||
scope_id: Restrict to a Caido scope (managed via ``scope_rules``).
|
scope_id: Restrict to a Caido scope (managed via ``scope_rules``).
|
||||||
"""
|
"""
|
||||||
client = await _ctx_client(ctx)
|
client = _ctx_client(ctx)
|
||||||
if client is None:
|
if client is None:
|
||||||
return _no_client()
|
return _no_client()
|
||||||
|
|
||||||
@@ -269,7 +261,7 @@ async def view_request(
|
|||||||
page: 1-indexed page number (only when no ``search_pattern``).
|
page: 1-indexed page number (only when no ``search_pattern``).
|
||||||
page_size: Lines per page.
|
page_size: Lines per page.
|
||||||
"""
|
"""
|
||||||
client = await _ctx_client(ctx)
|
client = _ctx_client(ctx)
|
||||||
if client is None:
|
if client is None:
|
||||||
return _no_client()
|
return _no_client()
|
||||||
|
|
||||||
@@ -387,7 +379,7 @@ async def repeat_request(
|
|||||||
- ``body`` — replace the body string entirely.
|
- ``body`` — replace the body string entirely.
|
||||||
- ``cookies`` — dict of cookies to add/update.
|
- ``cookies`` — dict of cookies to add/update.
|
||||||
"""
|
"""
|
||||||
client = await _ctx_client(ctx)
|
client = _ctx_client(ctx)
|
||||||
if client is None:
|
if client is None:
|
||||||
return _no_client()
|
return _no_client()
|
||||||
mods = modifications or {}
|
mods = modifications or {}
|
||||||
@@ -469,7 +461,7 @@ async def list_sitemap(
|
|||||||
(recursive subtree). Only meaningful with ``parent_id``.
|
(recursive subtree). Only meaningful with ``parent_id``.
|
||||||
page: 1-indexed page (30 entries per page).
|
page: 1-indexed page (30 entries per page).
|
||||||
"""
|
"""
|
||||||
client = await _ctx_client(ctx)
|
client = _ctx_client(ctx)
|
||||||
if client is None:
|
if client is None:
|
||||||
return _no_client()
|
return _no_client()
|
||||||
try:
|
try:
|
||||||
@@ -503,7 +495,7 @@ async def view_sitemap_entry(
|
|||||||
Args:
|
Args:
|
||||||
entry_id: ID from ``list_sitemap`` (or any nested entry).
|
entry_id: ID from ``list_sitemap`` (or any nested entry).
|
||||||
"""
|
"""
|
||||||
client = await _ctx_client(ctx)
|
client = _ctx_client(ctx)
|
||||||
if client is None:
|
if client is None:
|
||||||
return _no_client()
|
return _no_client()
|
||||||
try:
|
try:
|
||||||
@@ -562,7 +554,7 @@ async def scope_rules(
|
|||||||
scope_id: Required for ``get`` / ``update`` / ``delete``.
|
scope_id: Required for ``get`` / ``update`` / ``delete``.
|
||||||
scope_name: Required for ``create`` / ``update``.
|
scope_name: Required for ``create`` / ``update``.
|
||||||
"""
|
"""
|
||||||
client = await _ctx_client(ctx)
|
client = _ctx_client(ctx)
|
||||||
if client is None:
|
if client is None:
|
||||||
return _no_client()
|
return _no_client()
|
||||||
|
|
||||||
|
|||||||
+26
-210
@@ -161,101 +161,9 @@ _REQUIRED_FIELDS = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
_VALID_FIX_EFFORT = frozenset({"trivial", "low", "medium", "high"})
|
_VALID_FIX_EFFORT = frozenset({"trivial", "low", "medium", "high"})
|
||||||
_VALID_CONFIDENCE = frozenset({"high", "medium", "low"})
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_required_text(fields: dict[str, str]) -> list[str]:
|
async def _do_create( # noqa: PLR0912
|
||||||
"""Report every ``_REQUIRED_FIELDS`` entry that arrived blank."""
|
|
||||||
return [
|
|
||||||
msg for name, msg in _REQUIRED_FIELDS.items() if not str(fields.get(name) or "").strip()
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_cvss_breakdown(breakdown: Any) -> list[str]:
|
|
||||||
"""Check the 8 CVSS metrics are all present with legal values."""
|
|
||||||
if not isinstance(breakdown, dict) or not breakdown:
|
|
||||||
return ["cvss_breakdown: must be an object with the 8 CVSS metrics"]
|
|
||||||
return [
|
|
||||||
f"Invalid {name}: {breakdown.get(name)}. Must be one of: {valid}"
|
|
||||||
for name, valid in _CVSS_VALID.items()
|
|
||||||
if breakdown.get(name) not in valid
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_identifiers(
|
|
||||||
cve: str | None, cwe: str | None
|
|
||||||
) -> tuple[str | None, str | None, list[str]]:
|
|
||||||
"""Normalize and validate the optional CVE / CWE identifiers."""
|
|
||||||
errors: list[str] = []
|
|
||||||
if cve:
|
|
||||||
cve = _extract_cve(cve)
|
|
||||||
cve_err = _validate_cve(cve)
|
|
||||||
if cve_err:
|
|
||||||
errors.append(cve_err)
|
|
||||||
if cwe:
|
|
||||||
cwe = _extract_cwe(cwe)
|
|
||||||
cwe_err = _validate_cwe(cwe)
|
|
||||||
if cwe_err:
|
|
||||||
errors.append(cwe_err)
|
|
||||||
return cve, cwe, errors
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_analysis_fields(
|
|
||||||
*,
|
|
||||||
counterevidence: str,
|
|
||||||
confidence: str,
|
|
||||||
confidence_rationale: str | None,
|
|
||||||
severity_change_conditions: str,
|
|
||||||
) -> list[str]:
|
|
||||||
"""Validate the counterevidence / confidence closure metadata."""
|
|
||||||
errors: list[str] = []
|
|
||||||
if not str(counterevidence or "").strip():
|
|
||||||
errors.append(
|
|
||||||
"Counterevidence cannot be empty - state the strongest evidence against "
|
|
||||||
"this finding, or what you checked and found none (e.g. 'no input "
|
|
||||||
"validation, WAF, or authorization check found on this path')"
|
|
||||||
)
|
|
||||||
if not str(severity_change_conditions or "").strip():
|
|
||||||
errors.append(
|
|
||||||
"severity_change_conditions cannot be empty - state the one concrete piece "
|
|
||||||
"of evidence that would raise or lower the severity"
|
|
||||||
)
|
|
||||||
if confidence not in _VALID_CONFIDENCE:
|
|
||||||
errors.append(
|
|
||||||
f"Invalid confidence: {confidence!r}. Must be one of: {sorted(_VALID_CONFIDENCE)}"
|
|
||||||
)
|
|
||||||
elif confidence != "high" and not str(confidence_rationale or "").strip():
|
|
||||||
errors.append(
|
|
||||||
"confidence_rationale is required when confidence is not 'high' - name the "
|
|
||||||
"gap (e.g. static-only trace, unconfirmed reachability, no runtime access)"
|
|
||||||
)
|
|
||||||
return errors
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_fix_verification(
|
|
||||||
locations: list[dict[str, Any]] | None,
|
|
||||||
fix_verification: str | None,
|
|
||||||
) -> list[str]:
|
|
||||||
"""Require a verification statement whenever an applyable fix is proposed."""
|
|
||||||
if not locations or not any(loc.get("fix_after") for loc in locations):
|
|
||||||
return []
|
|
||||||
if str(fix_verification or "").strip():
|
|
||||||
return []
|
|
||||||
return [
|
|
||||||
"fix_verification is REQUIRED when any code_location carries a 'fix_after' - "
|
|
||||||
"a suggestion a reviewer can click to apply must be verified first. State, in "
|
|
||||||
"order: (1) security closure - re-trace the source->sink path through the "
|
|
||||||
"PATCHED code and say why it is now blocked; (2) bypass review - re-read the "
|
|
||||||
"diff without your original rationale and name the equivalent sinks, sibling "
|
|
||||||
"call sites, and alternate malicious input classes you checked; (3) preserved "
|
|
||||||
"behavior - the legitimate inputs, APIs, and error semantics that still work; "
|
|
||||||
"(4) how each was checked (executed vs. reasoned), naming any unrun check as "
|
|
||||||
"an explicit gap. If you cannot make these statements, drop 'fix_after' and "
|
|
||||||
"leave the location informational."
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
async def _do_create(
|
|
||||||
*,
|
*,
|
||||||
title: str,
|
title: str,
|
||||||
description: str,
|
description: str,
|
||||||
@@ -267,9 +175,6 @@ async def _do_create(
|
|||||||
remediation_steps: str,
|
remediation_steps: str,
|
||||||
evidence: str,
|
evidence: str,
|
||||||
assumptions: str,
|
assumptions: str,
|
||||||
counterevidence: str,
|
|
||||||
confidence: str,
|
|
||||||
severity_change_conditions: str,
|
|
||||||
fix_effort: str,
|
fix_effort: str,
|
||||||
cvss_breakdown: dict[str, str],
|
cvss_breakdown: dict[str, str],
|
||||||
endpoint: str | None,
|
endpoint: str | None,
|
||||||
@@ -277,14 +182,12 @@ async def _do_create(
|
|||||||
cve: str | None,
|
cve: str | None,
|
||||||
cwe: str | None,
|
cwe: str | None,
|
||||||
code_locations: list[dict[str, Any]] | None,
|
code_locations: list[dict[str, Any]] | None,
|
||||||
confidence_rationale: str | None = None,
|
|
||||||
fix_verification: str | None = None,
|
|
||||||
fix_pr_body: str | None = None,
|
fix_pr_body: str | None = None,
|
||||||
agent_id: str | None = None,
|
agent_id: str | None = None,
|
||||||
agent_name: str | None = None,
|
agent_name: str | None = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
errors: list[str] = _validate_required_text(
|
errors: list[str] = []
|
||||||
{
|
fields = {
|
||||||
"title": title,
|
"title": title,
|
||||||
"description": description,
|
"description": description,
|
||||||
"impact": impact,
|
"impact": impact,
|
||||||
@@ -296,17 +199,9 @@ async def _do_create(
|
|||||||
"evidence": evidence,
|
"evidence": evidence,
|
||||||
"assumptions": assumptions,
|
"assumptions": assumptions,
|
||||||
}
|
}
|
||||||
)
|
for name, msg in _REQUIRED_FIELDS.items():
|
||||||
|
if not str(fields.get(name) or "").strip():
|
||||||
confidence = (confidence or "").strip().lower()
|
errors.append(msg)
|
||||||
errors.extend(
|
|
||||||
_validate_analysis_fields(
|
|
||||||
counterevidence=counterevidence,
|
|
||||||
confidence=confidence,
|
|
||||||
confidence_rationale=confidence_rationale,
|
|
||||||
severity_change_conditions=severity_change_conditions,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
fix_effort = (fix_effort or "").strip().lower()
|
fix_effort = (fix_effort or "").strip().lower()
|
||||||
if fix_effort not in _VALID_FIX_EFFORT:
|
if fix_effort not in _VALID_FIX_EFFORT:
|
||||||
@@ -314,14 +209,28 @@ async def _do_create(
|
|||||||
f"Invalid fix_effort: {fix_effort!r}. Must be one of: {sorted(_VALID_FIX_EFFORT)}"
|
f"Invalid fix_effort: {fix_effort!r}. Must be one of: {sorted(_VALID_FIX_EFFORT)}"
|
||||||
)
|
)
|
||||||
|
|
||||||
errors.extend(_validate_cvss_breakdown(cvss_breakdown))
|
if not isinstance(cvss_breakdown, dict) or not cvss_breakdown:
|
||||||
|
errors.append("cvss_breakdown: must be an object with the 8 CVSS metrics")
|
||||||
|
cvss_breakdown = {}
|
||||||
|
else:
|
||||||
|
for name, valid in _CVSS_VALID.items():
|
||||||
|
value = cvss_breakdown.get(name)
|
||||||
|
if value not in valid:
|
||||||
|
errors.append(f"Invalid {name}: {value}. Must be one of: {valid}")
|
||||||
|
|
||||||
parsed_locations = _normalize_code_locations(code_locations)
|
parsed_locations = _normalize_code_locations(code_locations)
|
||||||
if parsed_locations:
|
if parsed_locations:
|
||||||
errors.extend(_validate_code_locations(parsed_locations))
|
errors.extend(_validate_code_locations(parsed_locations))
|
||||||
errors.extend(_validate_fix_verification(parsed_locations, fix_verification))
|
if cve:
|
||||||
cve, cwe, identifier_errors = _validate_identifiers(cve, cwe)
|
cve = _extract_cve(cve)
|
||||||
errors.extend(identifier_errors)
|
cve_err = _validate_cve(cve)
|
||||||
|
if cve_err:
|
||||||
|
errors.append(cve_err)
|
||||||
|
if cwe:
|
||||||
|
cwe = _extract_cwe(cwe)
|
||||||
|
cwe_err = _validate_cwe(cwe)
|
||||||
|
if cwe_err:
|
||||||
|
errors.append(cwe_err)
|
||||||
|
|
||||||
if errors:
|
if errors:
|
||||||
return {"success": False, "error": "Validation failed", "errors": errors}
|
return {"success": False, "error": "Validation failed", "errors": errors}
|
||||||
@@ -388,10 +297,6 @@ async def _do_create(
|
|||||||
remediation_steps=remediation_steps,
|
remediation_steps=remediation_steps,
|
||||||
evidence=evidence,
|
evidence=evidence,
|
||||||
assumptions=assumptions,
|
assumptions=assumptions,
|
||||||
counterevidence=counterevidence,
|
|
||||||
confidence=confidence,
|
|
||||||
confidence_rationale=confidence_rationale,
|
|
||||||
severity_change_conditions=severity_change_conditions,
|
|
||||||
fix_effort=fix_effort,
|
fix_effort=fix_effort,
|
||||||
cvss=cvss_score,
|
cvss=cvss_score,
|
||||||
cvss_breakdown=cvss_breakdown,
|
cvss_breakdown=cvss_breakdown,
|
||||||
@@ -400,7 +305,6 @@ async def _do_create(
|
|||||||
cve=cve,
|
cve=cve,
|
||||||
cwe=cwe,
|
cwe=cwe,
|
||||||
code_locations=parsed_locations,
|
code_locations=parsed_locations,
|
||||||
fix_verification=fix_verification,
|
|
||||||
fix_pr_body=fix_pr_body,
|
fix_pr_body=fix_pr_body,
|
||||||
agent_id=agent_id if isinstance(agent_id, str) else None,
|
agent_id=agent_id if isinstance(agent_id, str) else None,
|
||||||
agent_name=agent_name if isinstance(agent_name, str) else None,
|
agent_name=agent_name if isinstance(agent_name, str) else None,
|
||||||
@@ -453,9 +357,6 @@ async def create_vulnerability_report(
|
|||||||
remediation_steps: str,
|
remediation_steps: str,
|
||||||
evidence: str,
|
evidence: str,
|
||||||
assumptions: str,
|
assumptions: str,
|
||||||
counterevidence: str,
|
|
||||||
confidence: str,
|
|
||||||
severity_change_conditions: str,
|
|
||||||
fix_effort: str,
|
fix_effort: str,
|
||||||
cvss_breakdown: dict[str, str],
|
cvss_breakdown: dict[str, str],
|
||||||
endpoint: str | None = None,
|
endpoint: str | None = None,
|
||||||
@@ -463,8 +364,6 @@ async def create_vulnerability_report(
|
|||||||
cve: str | None = None,
|
cve: str | None = None,
|
||||||
cwe: str | None = None,
|
cwe: str | None = None,
|
||||||
code_locations: list[dict[str, Any]] | None = None,
|
code_locations: list[dict[str, Any]] | None = None,
|
||||||
confidence_rationale: str | None = None,
|
|
||||||
fix_verification: str | None = None,
|
|
||||||
fix_pr_body: str | None = None,
|
fix_pr_body: str | None = None,
|
||||||
) -> str:
|
) -> str:
|
||||||
"""File a vulnerability report — one report per fully-verified finding.
|
"""File a vulnerability report — one report per fully-verified finding.
|
||||||
@@ -512,15 +411,6 @@ async def create_vulnerability_report(
|
|||||||
get a ``duplicate_of`` response, do NOT retry — move on to other
|
get a ``duplicate_of`` response, do NOT retry — move on to other
|
||||||
areas.
|
areas.
|
||||||
|
|
||||||
**Counterevidence pass (required before filing)**: actively build the
|
|
||||||
strongest case that this finding is NOT exploitable, or less severe
|
|
||||||
than you think — then record the result in ``counterevidence``, set
|
|
||||||
``confidence`` honestly, and state what would move the severity in
|
|
||||||
``severity_change_conditions``. These three fields are mandatory and
|
|
||||||
validated. A finding you could not execute is at best
|
|
||||||
``confidence: medium``, with the gap named in
|
|
||||||
``confidence_rationale``.
|
|
||||||
|
|
||||||
**Report output rules** (this content may be rendered into generated
|
**Report output rules** (this content may be rendered into generated
|
||||||
reports):
|
reports):
|
||||||
|
|
||||||
@@ -673,31 +563,6 @@ async def create_vulnerability_report(
|
|||||||
assumptions: Short note on the assumptions/prerequisites that
|
assumptions: Short note on the assumptions/prerequisites that
|
||||||
make this finding impactful or exploitable (e.g. "assumes an
|
make this finding impactful or exploitable (e.g. "assumes an
|
||||||
authenticated low-privilege user").
|
authenticated low-privilege user").
|
||||||
counterevidence: REQUIRED. The strongest case *against* this
|
|
||||||
finding, after actively looking for it — the guard you might
|
|
||||||
have missed, the deployment constraint, the precondition. If
|
|
||||||
you genuinely found nothing, say what you checked (e.g. "no
|
|
||||||
input validation, WAF, or authorization check found on this
|
|
||||||
path; tested authenticated and unauthenticated"), not just
|
|
||||||
"none". A generic trust claim ("the framework escapes this")
|
|
||||||
is not counterevidence unless you confirmed that specific
|
|
||||||
call in this context.
|
|
||||||
confidence: REQUIRED. Your calibrated confidence that this is a
|
|
||||||
real, exploitable issue: ``high`` (working PoC against the
|
|
||||||
live target, or a complete reachable source→sink trace),
|
|
||||||
``medium`` (strong static evidence you could not fully
|
|
||||||
execute), or ``low`` (plausible with a material unresolved
|
|
||||||
gap). Do not inflate — an accurate ``medium`` is more useful
|
|
||||||
than a ``high`` that fails triage.
|
|
||||||
confidence_rationale: Required when ``confidence`` is not
|
|
||||||
``high``. Name the specific gap (e.g. "static-only trace,
|
|
||||||
could not stand up the service to reproduce"; "reachability
|
|
||||||
of this route from unauthenticated traffic unconfirmed").
|
|
||||||
severity_change_conditions: REQUIRED. One concrete sentence on
|
|
||||||
what single piece of additional evidence would raise or
|
|
||||||
lower the severity (e.g. "confirmation this route is exposed
|
|
||||||
to unauthenticated internet traffic would raise this to
|
|
||||||
critical").
|
|
||||||
fix_effort: One of ``trivial`` / ``low`` / ``medium`` / ``high``.
|
fix_effort: One of ``trivial`` / ``low`` / ``medium`` / ``high``.
|
||||||
cvss_breakdown: 8-metric object per the format above.
|
cvss_breakdown: 8-metric object per the format above.
|
||||||
endpoint: API path / Git path (e.g. ``/api/login``).
|
endpoint: API path / Git path (e.g. ``/api/login``).
|
||||||
@@ -767,40 +632,6 @@ async def create_vulnerability_report(
|
|||||||
- Padding ``fix_before`` with surrounding context lines
|
- Padding ``fix_before`` with surrounding context lines
|
||||||
that aren't part of the fix.
|
that aren't part of the fix.
|
||||||
- Duplicating the same change across multiple locations.
|
- Duplicating the same change across multiple locations.
|
||||||
fix_verification: REQUIRED whenever any ``code_locations`` entry
|
|
||||||
carries a ``fix_after``. A reviewer can apply that
|
|
||||||
suggestion with one click, so an unverified fix ships
|
|
||||||
straight into the codebase. Before writing this field, work
|
|
||||||
the gates **in order** and never trade an earlier one for a
|
|
||||||
later one:
|
|
||||||
|
|
||||||
1. **Security closure** — re-trace the source → sink path
|
|
||||||
through the *patched* code and state why it is now
|
|
||||||
blocked. Re-run the PoC against the fix if you can.
|
|
||||||
2. **Bypass review** — re-read the diff *without* leaning on
|
|
||||||
the rationale that produced it. Name the sibling call
|
|
||||||
sites, equivalent sinks, and alternate malicious input
|
|
||||||
classes you checked, and try at least one.
|
|
||||||
3. **Preserved behavior** — name the legitimate inputs,
|
|
||||||
public APIs, and error semantics that must keep working,
|
|
||||||
and confirm the patch leaves them intact. A fix that
|
|
||||||
breaks the feature is not a fix.
|
|
||||||
4. **Repository checks** — run the narrowest relevant
|
|
||||||
syntax / type / lint / test check that covers the
|
|
||||||
changed lines.
|
|
||||||
|
|
||||||
Then write what you did: the commands you ran and their
|
|
||||||
results, and every gate you could only reason about rather
|
|
||||||
than execute, marked explicitly as a gap. Do not claim a
|
|
||||||
gate passed because it looks right. If a gate fails, revise
|
|
||||||
the patch or drop ``fix_after`` and leave the location
|
|
||||||
informational — never compensate for a failed security
|
|
||||||
closure with a smaller diff or extra prose.
|
|
||||||
|
|
||||||
Also use this field to record the narrowest-complete-change
|
|
||||||
judgement: prefer the smallest repository-native fix that
|
|
||||||
fully enforces the invariant, using existing helpers, with
|
|
||||||
no unrelated refactors folded in.
|
|
||||||
fix_pr_body: Optional. When source is available and you have a
|
fix_pr_body: Optional. When source is available and you have a
|
||||||
concrete fix, a markdown PR-description body proposing the
|
concrete fix, a markdown PR-description body proposing the
|
||||||
fix (summary + rationale). Prose/markdown only — the code
|
fix (summary + rationale). Prose/markdown only — the code
|
||||||
@@ -841,15 +672,6 @@ async def create_vulnerability_report(
|
|||||||
remediation_steps:
|
remediation_steps:
|
||||||
Context-encode all user input rendered into HTML; prefer the
|
Context-encode all user input rendered into HTML; prefer the
|
||||||
template engine's auto-escaping over string interpolation.
|
template engine's auto-escaping over string interpolation.
|
||||||
counterevidence:
|
|
||||||
No output encoding, CSP, or WAF observed on this response;
|
|
||||||
payload executed in a current browser. The parameter is
|
|
||||||
reflected on an unauthenticated route, so no privileged
|
|
||||||
position is required.
|
|
||||||
confidence: "high"
|
|
||||||
severity_change_conditions:
|
|
||||||
A restrictive CSP that blocks inline script execution would
|
|
||||||
reduce impact and lower the severity.
|
|
||||||
fix_effort: "low"
|
fix_effort: "low"
|
||||||
"""
|
"""
|
||||||
agent_id, agent_name = _caller_identity(ctx)
|
agent_id, agent_name = _caller_identity(ctx)
|
||||||
@@ -865,10 +687,6 @@ async def create_vulnerability_report(
|
|||||||
remediation_steps=remediation_steps,
|
remediation_steps=remediation_steps,
|
||||||
evidence=evidence,
|
evidence=evidence,
|
||||||
assumptions=assumptions,
|
assumptions=assumptions,
|
||||||
counterevidence=counterevidence,
|
|
||||||
confidence=confidence,
|
|
||||||
confidence_rationale=confidence_rationale,
|
|
||||||
severity_change_conditions=severity_change_conditions,
|
|
||||||
fix_effort=fix_effort,
|
fix_effort=fix_effort,
|
||||||
cvss_breakdown=cvss_breakdown,
|
cvss_breakdown=cvss_breakdown,
|
||||||
endpoint=endpoint,
|
endpoint=endpoint,
|
||||||
@@ -876,7 +694,6 @@ async def create_vulnerability_report(
|
|||||||
cve=cve,
|
cve=cve,
|
||||||
cwe=cwe,
|
cwe=cwe,
|
||||||
code_locations=code_locations,
|
code_locations=code_locations,
|
||||||
fix_verification=fix_verification,
|
|
||||||
fix_pr_body=fix_pr_body,
|
fix_pr_body=fix_pr_body,
|
||||||
agent_id=agent_id,
|
agent_id=agent_id,
|
||||||
agent_name=agent_name,
|
agent_name=agent_name,
|
||||||
@@ -1510,7 +1327,6 @@ _REPORT_SUMMARY_FIELDS = (
|
|||||||
"title",
|
"title",
|
||||||
"severity",
|
"severity",
|
||||||
"cvss",
|
"cvss",
|
||||||
"confidence",
|
|
||||||
"finding_class",
|
"finding_class",
|
||||||
"cve",
|
"cve",
|
||||||
"cwe",
|
"cwe",
|
||||||
@@ -1712,8 +1528,8 @@ async def list_reports(
|
|||||||
findings, and build the ``finish_scan`` executive summary.
|
findings, and build the ``finish_scan`` executive summary.
|
||||||
|
|
||||||
By default each entry is compact: ``id``, ``title``, ``severity``,
|
By default each entry is compact: ``id``, ``title``, ``severity``,
|
||||||
``cvss``, ``confidence``, ``finding_class``, ``cve`` / ``cwe``,
|
``cvss``, ``finding_class``, ``cve`` / ``cwe``, ``target`` /
|
||||||
``target`` / ``endpoint``, ``fix_effort``, ``agent_name`` (who filed it), ``timestamp``,
|
``endpoint``, ``fix_effort``, ``agent_name`` (who filed it), ``timestamp``,
|
||||||
plus a 280-char ``description_preview``. Entries you filed yourself are
|
plus a 280-char ``description_preview``. Entries you filed yourself are
|
||||||
flagged ``by_you: true``. The response also carries
|
flagged ``by_you: true``. The response also carries
|
||||||
``total_count`` and ``severity_counts`` (counts per severity across all
|
``total_count`` and ``severity_counts`` (counts per severity across all
|
||||||
|
|||||||
@@ -1 +0,0 @@
|
|||||||
"""Repository-scoped threat model cache, reusable across scans of the same tree."""
|
|
||||||
@@ -1,659 +0,0 @@
|
|||||||
"""Target-scoped threat models — cached under ``~/.strix/threat-models``.
|
|
||||||
|
|
||||||
A threat model describes the target, not the scan: a host, an application, an
|
|
||||||
API, a repository, or whatever else the engagement is pointed at. It stays
|
|
||||||
valid across unrelated runs against the same target, so it is keyed by target
|
|
||||||
identity rather than by run id — one agent derives it, every later agent in
|
|
||||||
this run and in future runs against the same target reads it back instead of
|
|
||||||
re-deriving trust boundaries from scratch.
|
|
||||||
|
|
||||||
Where the target is a checkout, the model is additionally pinned to the git
|
|
||||||
revision, so a moved ``HEAD`` marks it stale. Black-box targets have no
|
|
||||||
revision to pin to; those age out instead.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import hashlib
|
|
||||||
import json
|
|
||||||
import logging
|
|
||||||
import re
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
import threading
|
|
||||||
from datetime import UTC, datetime, timedelta
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Any
|
|
||||||
from urllib.parse import urlsplit
|
|
||||||
|
|
||||||
from agents import RunContextWrapper, function_tool
|
|
||||||
|
|
||||||
from strix.core.agents import AgentCoordinator
|
|
||||||
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
_CACHE_DIR = Path.home() / ".strix" / "threat-models"
|
|
||||||
_MAX_MODEL_BYTES = 512 * 1024
|
|
||||||
_MIN_MODEL_CHARS = 400
|
|
||||||
_MIN_AMENDMENT_CHARS = 80
|
|
||||||
_MAX_AMENDMENTS = 40
|
|
||||||
_GIT_TIMEOUT_SECONDS = 10
|
|
||||||
_UNVERSIONED = "unversioned"
|
|
||||||
_MAX_AGE_DAYS = 14
|
|
||||||
_DEFAULT_PORTS = {"http": "80", "https": "443"}
|
|
||||||
_cache_lock = threading.RLock()
|
|
||||||
|
|
||||||
_REQUIRED_SECTIONS = (
|
|
||||||
"overview",
|
|
||||||
"trust boundaries",
|
|
||||||
"attack surface",
|
|
||||||
"severity calibration",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _git(repo: Path, args: list[str]) -> str | None:
|
|
||||||
try:
|
|
||||||
result = subprocess.run( # noqa: S603
|
|
||||||
["git", "-C", str(repo), *args], # noqa: S607
|
|
||||||
capture_output=True,
|
|
||||||
text=True,
|
|
||||||
check=False,
|
|
||||||
timeout=_GIT_TIMEOUT_SECONDS,
|
|
||||||
)
|
|
||||||
except (OSError, subprocess.SubprocessError):
|
|
||||||
logger.debug("git %s failed in %s", args, repo, exc_info=True)
|
|
||||||
return None
|
|
||||||
if result.returncode != 0:
|
|
||||||
return None
|
|
||||||
return result.stdout.strip() or None
|
|
||||||
|
|
||||||
|
|
||||||
def _local_directory(target: str) -> Path | None:
|
|
||||||
"""Return the target as a local directory, or None if it is not one."""
|
|
||||||
if "://" in target:
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
resolved = Path(target).expanduser().resolve()
|
|
||||||
except OSError:
|
|
||||||
return None
|
|
||||||
return resolved if resolved.is_dir() else None
|
|
||||||
|
|
||||||
|
|
||||||
def _remote_authority(target: str) -> str:
|
|
||||||
"""The ``host[:port]`` a remote target lives on, or "" if it has none."""
|
|
||||||
candidate = target if "://" in target else f"//{target}"
|
|
||||||
parts = urlsplit(candidate)
|
|
||||||
host = (parts.hostname or "").lower()
|
|
||||||
if not host:
|
|
||||||
return ""
|
|
||||||
scheme = (parts.scheme or "https").lower()
|
|
||||||
port = str(parts.port) if parts.port else _DEFAULT_PORTS.get(scheme, "")
|
|
||||||
return f"{host}:{port}" if port else host
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_remote_target(target: str) -> str:
|
|
||||||
"""Collapse the spellings of one remote target onto a single cache key."""
|
|
||||||
authority = _remote_authority(target)
|
|
||||||
if not authority:
|
|
||||||
return re.sub(r"\s+", " ", target.lower()).strip()
|
|
||||||
candidate = target if "://" in target else f"//{target}"
|
|
||||||
path = urlsplit(candidate).path.rstrip("/")
|
|
||||||
return f"{authority}{path}"
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_git_remote(remote: str) -> str:
|
|
||||||
"""Collapse a git remote URL onto the same key its clone URL would produce.
|
|
||||||
|
|
||||||
A remote reaches us in whichever spelling the clone used —
|
|
||||||
``git@github.com:org/repo.git``, ``https://github.com/org/repo``,
|
|
||||||
``ssh://git@github.com/org/repo.git`` — and each is the same repository.
|
|
||||||
Rewriting scp-style syntax into a URL and dropping the ``.git`` suffix and
|
|
||||||
any embedded credentials lets :func:`_normalize_remote_target` produce one
|
|
||||||
identity for all of them, and crucially the *same* identity a caller gets
|
|
||||||
when it names the repository by its remote URL rather than by a checkout
|
|
||||||
path. Without that, the model saved by an agent working in the checkout is
|
|
||||||
invisible to an agent that asks for the repository by URL, and the two
|
|
||||||
derive conflicting models of one target.
|
|
||||||
"""
|
|
||||||
candidate = remote.strip()
|
|
||||||
scp_style = re.match(r"^(?:[^@/]+@)?(?P<host>[^:/]+):(?P<path>.+)$", candidate)
|
|
||||||
if scp_style and "://" not in candidate:
|
|
||||||
candidate = f"https://{scp_style['host']}/{scp_style['path'].lstrip('/')}"
|
|
||||||
elif "://" in candidate:
|
|
||||||
# The transport a clone happened to use says nothing about which
|
|
||||||
# repository this is, and each scheme carries a different default
|
|
||||||
# port into the authority. Collapsing them all onto https keeps one
|
|
||||||
# repository on one key however it was cloned.
|
|
||||||
candidate = f"https://{candidate.split('://', 1)[1]}"
|
|
||||||
normalized = _normalize_remote_target(candidate)
|
|
||||||
return normalized.removesuffix(".git")
|
|
||||||
|
|
||||||
|
|
||||||
def _target_identity(target: str) -> tuple[str, str]:
|
|
||||||
"""Return the (stable identity, revision) pair a cached model is keyed on.
|
|
||||||
|
|
||||||
A checkout is keyed on its remote (so the same repository cloned to two
|
|
||||||
paths shares one model, and a subdirectory resolves to the whole tree) and
|
|
||||||
pinned to ``HEAD``. Everything else — a host, a URL, an API base, a named
|
|
||||||
scope — is keyed on its normalized form and carries no revision. Both
|
|
||||||
routes run through the same normalization, so a checkout and the URL it
|
|
||||||
was cloned from land on one key.
|
|
||||||
"""
|
|
||||||
directory = _local_directory(target)
|
|
||||||
if directory is None:
|
|
||||||
return _normalize_remote_target(target).removesuffix(".git"), _UNVERSIONED
|
|
||||||
remote = _git(directory, ["config", "--get", "remote.origin.url"])
|
|
||||||
revision = _git(directory, ["rev-parse", "HEAD"]) or _UNVERSIONED
|
|
||||||
if remote:
|
|
||||||
return _normalize_git_remote(remote), revision
|
|
||||||
toplevel = _git(directory, ["rev-parse", "--show-toplevel"])
|
|
||||||
return toplevel or str(directory), revision
|
|
||||||
|
|
||||||
|
|
||||||
def _cache_path(identity: str) -> Path:
|
|
||||||
digest = hashlib.sha256(identity.encode("utf-8")).hexdigest()[:16]
|
|
||||||
return _CACHE_DIR / f"{digest}.json"
|
|
||||||
|
|
||||||
|
|
||||||
def _snap_to_scan_target(raw: str, scan_targets: list[str]) -> str:
|
|
||||||
"""Pull a target onto the scan's own spelling of it.
|
|
||||||
|
|
||||||
Agents name the same target differently — one passes the URL it was given,
|
|
||||||
the next the page it happens to be testing, a third the checkout path. Left
|
|
||||||
alone those become separate cache keys, every lookup misses, and each agent
|
|
||||||
quietly derives its own model, which is the exact failure the shared model
|
|
||||||
exists to prevent. So a target that is recognisably one of the scan's own
|
|
||||||
targets is resolved to that target instead.
|
|
||||||
"""
|
|
||||||
identity, _ = _target_identity(raw)
|
|
||||||
scoped = [(target, _target_identity(target)[0]) for target in scan_targets]
|
|
||||||
if any(known == identity for _, known in scoped):
|
|
||||||
return raw
|
|
||||||
|
|
||||||
authority = _remote_authority(raw)
|
|
||||||
if authority:
|
|
||||||
hosted = [target for target, _ in scoped if _remote_authority(target) == authority]
|
|
||||||
# Two scan targets on one host are distinguished only by their paths,
|
|
||||||
# so snapping to "the host" would merge two distinct models into one.
|
|
||||||
return hosted[0] if len(hosted) == 1 else raw
|
|
||||||
|
|
||||||
directory = _local_directory(raw)
|
|
||||||
if directory is not None:
|
|
||||||
enclosing = [
|
|
||||||
target
|
|
||||||
for target, known in scoped
|
|
||||||
if known == identity or _local_directory(target) == directory
|
|
||||||
]
|
|
||||||
if enclosing:
|
|
||||||
return enclosing[0]
|
|
||||||
return raw
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_target(
|
|
||||||
target: str, scan_targets: list[str] | None = None
|
|
||||||
) -> tuple[str | None, str | None]:
|
|
||||||
raw = (target or "").strip()
|
|
||||||
known = [t for t in (scan_targets or []) if t.strip()]
|
|
||||||
if not raw:
|
|
||||||
if len(known) == 1:
|
|
||||||
return known[0], None
|
|
||||||
return None, (
|
|
||||||
"target cannot be empty - pass the host, URL, application, or "
|
|
||||||
"repository path this model describes"
|
|
||||||
+ (f". This scan is scoped to: {', '.join(known)}" if known else "")
|
|
||||||
)
|
|
||||||
return (_snap_to_scan_target(raw, known) if known else raw), None
|
|
||||||
|
|
||||||
|
|
||||||
def _is_expired(created_at: str | None) -> bool:
|
|
||||||
if not created_at:
|
|
||||||
return True
|
|
||||||
try:
|
|
||||||
created = datetime.fromisoformat(created_at)
|
|
||||||
except ValueError:
|
|
||||||
return True
|
|
||||||
if created.tzinfo is None:
|
|
||||||
created = created.replace(tzinfo=UTC)
|
|
||||||
return datetime.now(UTC) - created > timedelta(days=_MAX_AGE_DAYS)
|
|
||||||
|
|
||||||
|
|
||||||
def _missing_sections(content: str) -> list[str]:
|
|
||||||
lowered = content.lower()
|
|
||||||
return [section for section in _REQUIRED_SECTIONS if section not in lowered]
|
|
||||||
|
|
||||||
|
|
||||||
def _read_cache(path: Path) -> dict[str, Any] | None:
|
|
||||||
"""Load a cached model. Callers must already hold ``_cache_lock``."""
|
|
||||||
if not path.is_file():
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
cached = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
except (OSError, json.JSONDecodeError):
|
|
||||||
logger.exception("threat model cache at %s is unreadable", path)
|
|
||||||
return None
|
|
||||||
return cached if isinstance(cached, dict) else None
|
|
||||||
|
|
||||||
|
|
||||||
def _write_cache(path: Path, payload: dict[str, Any]) -> str | None:
|
|
||||||
"""Atomically persist a model. Callers must already hold ``_cache_lock``."""
|
|
||||||
try:
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
with tempfile.NamedTemporaryFile(
|
|
||||||
mode="w",
|
|
||||||
encoding="utf-8",
|
|
||||||
dir=str(path.parent),
|
|
||||||
prefix=f".{path.name}.",
|
|
||||||
suffix=".tmp",
|
|
||||||
delete=False,
|
|
||||||
) as tmp:
|
|
||||||
tmp.write(json.dumps(payload, ensure_ascii=False))
|
|
||||||
tmp_path = Path(tmp.name)
|
|
||||||
tmp_path.replace(path)
|
|
||||||
except OSError as exc:
|
|
||||||
logger.exception("threat model persist to %s failed", path)
|
|
||||||
return f"Failed to persist threat model: {exc}"
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _amendments_of(cached: dict[str, Any]) -> list[dict[str, Any]]:
|
|
||||||
raw = cached.get("amendments")
|
|
||||||
if not isinstance(raw, list):
|
|
||||||
return []
|
|
||||||
return [item for item in raw if isinstance(item, dict)]
|
|
||||||
|
|
||||||
|
|
||||||
def _not_found(identity: str, revision: str) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"success": True,
|
|
||||||
"found": False,
|
|
||||||
"target": identity,
|
|
||||||
"revision": revision,
|
|
||||||
"message": (
|
|
||||||
"No threat model cached for this target. Derive one — from the code if "
|
|
||||||
"you have it, from recon output if you do not — and persist it with "
|
|
||||||
"save_threat_model, so every agent on this scan shares one view of the "
|
|
||||||
"trust boundaries instead of each inventing their own."
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _staleness(cached: dict[str, Any], revision: str) -> tuple[bool, str | None]:
|
|
||||||
"""Decide whether a cached model can still be trusted, and why not."""
|
|
||||||
if revision != _UNVERSIONED:
|
|
||||||
if cached.get("revision") == revision:
|
|
||||||
return False, None
|
|
||||||
return True, (
|
|
||||||
"This model was derived against a different revision. Use it as a "
|
|
||||||
"starting point, re-check the boundaries it names against the current "
|
|
||||||
"tree, and save the corrected version."
|
|
||||||
)
|
|
||||||
created_at = cached.get("created_at")
|
|
||||||
if not _is_expired(created_at if isinstance(created_at, str) else None):
|
|
||||||
return False, None
|
|
||||||
return True, (
|
|
||||||
f"This model is more than {_MAX_AGE_DAYS} days old and there is no revision "
|
|
||||||
"to pin it to, so the target may have moved under it. Treat its surface "
|
|
||||||
"inventory as a lead list to re-confirm during recon, not as fact, and save "
|
|
||||||
"the corrected version."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _get_impl(target: str, scan_targets: list[str] | None = None) -> dict[str, Any]:
|
|
||||||
resolved, error = _resolve_target(target, scan_targets)
|
|
||||||
if resolved is None:
|
|
||||||
return {"success": False, "error": error}
|
|
||||||
|
|
||||||
identity, revision = _target_identity(resolved)
|
|
||||||
path = _cache_path(identity)
|
|
||||||
with _cache_lock:
|
|
||||||
cached = _read_cache(path)
|
|
||||||
if cached is None:
|
|
||||||
return _not_found(identity, revision)
|
|
||||||
content = cached.get("content")
|
|
||||||
if not isinstance(content, str) or not content.strip():
|
|
||||||
return _not_found(identity, revision)
|
|
||||||
|
|
||||||
stale, stale_message = _staleness(cached, revision)
|
|
||||||
result: dict[str, Any] = {
|
|
||||||
"success": True,
|
|
||||||
"found": True,
|
|
||||||
"target": identity,
|
|
||||||
"revision": revision,
|
|
||||||
"cached_revision": cached.get("revision"),
|
|
||||||
"created_at": cached.get("created_at"),
|
|
||||||
"stale": stale,
|
|
||||||
"content": content,
|
|
||||||
}
|
|
||||||
amendments = _amendments_of(cached)
|
|
||||||
if amendments:
|
|
||||||
result["amendments"] = amendments
|
|
||||||
result["amendments_note"] = (
|
|
||||||
"Addenda recorded by agents after the base model was written. They "
|
|
||||||
"correct or extend it and have not been folded in yet - read them as "
|
|
||||||
"part of the model, and prefer the later one where they conflict."
|
|
||||||
)
|
|
||||||
if stale_message:
|
|
||||||
result["message"] = stale_message
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
def _save_impl(
|
|
||||||
target: str,
|
|
||||||
content: str,
|
|
||||||
agent_name: str | None,
|
|
||||||
scan_targets: list[str] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
resolved, error = _resolve_target(target, scan_targets)
|
|
||||||
if resolved is None:
|
|
||||||
return {"success": False, "error": error}
|
|
||||||
|
|
||||||
body = (content or "").strip()
|
|
||||||
if len(body) < _MIN_MODEL_CHARS:
|
|
||||||
return {
|
|
||||||
"success": False,
|
|
||||||
"error": (
|
|
||||||
f"Threat model is too thin ({len(body)} chars). It has to be usable by "
|
|
||||||
"an agent seeing this target for the first time: what it is, who the "
|
|
||||||
"actors are, where the trust boundaries sit, which inputs are "
|
|
||||||
"attacker-controlled, and what a critical bug looks like here."
|
|
||||||
),
|
|
||||||
}
|
|
||||||
if len(body.encode("utf-8")) > _MAX_MODEL_BYTES:
|
|
||||||
return {"success": False, "error": "Threat model exceeds 512KB; tighten it."}
|
|
||||||
|
|
||||||
missing = _missing_sections(body)
|
|
||||||
if missing:
|
|
||||||
return {
|
|
||||||
"success": False,
|
|
||||||
"error": (
|
|
||||||
"Threat model is missing required section(s): "
|
|
||||||
f"{', '.join(missing)}. Cover Overview, Trust Boundaries and "
|
|
||||||
"Assumptions, Attack Surface and Attacker Stories, and Severity "
|
|
||||||
"Calibration."
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
identity, revision = _target_identity(resolved)
|
|
||||||
path = _cache_path(identity)
|
|
||||||
payload: dict[str, Any] = {
|
|
||||||
"target": identity,
|
|
||||||
"revision": revision,
|
|
||||||
"created_at": datetime.now(UTC).isoformat(),
|
|
||||||
"created_by": agent_name,
|
|
||||||
"content": body,
|
|
||||||
}
|
|
||||||
with _cache_lock:
|
|
||||||
existing = _read_cache(path)
|
|
||||||
folded = len(_amendments_of(existing)) if existing else 0
|
|
||||||
error = _write_cache(path, payload)
|
|
||||||
if error:
|
|
||||||
return {"success": False, "error": error}
|
|
||||||
|
|
||||||
message = (
|
|
||||||
"Threat model saved. Subagents should call get_threat_model before they "
|
|
||||||
"start, and treat its trust boundaries as the shared baseline."
|
|
||||||
)
|
|
||||||
if folded:
|
|
||||||
message += (
|
|
||||||
f" This replaced a model carrying {folded} amendment(s), which are now "
|
|
||||||
"cleared - make sure what they said survives in the text you just wrote."
|
|
||||||
)
|
|
||||||
return {
|
|
||||||
"success": True,
|
|
||||||
"target": identity,
|
|
||||||
"revision": revision,
|
|
||||||
"amendments_cleared": folded,
|
|
||||||
"message": message,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _append_amendment(
|
|
||||||
path: Path, amendment: dict[str, Any]
|
|
||||||
) -> tuple[list[dict[str, Any]] | None, str | None]:
|
|
||||||
"""Add an amendment to the cached model. Returns (amendments, error)."""
|
|
||||||
with _cache_lock:
|
|
||||||
cached = _read_cache(path)
|
|
||||||
if cached is None or not str(cached.get("content", "")).strip():
|
|
||||||
return None, (
|
|
||||||
"No threat model exists for this target yet, so there is nothing to "
|
|
||||||
"amend. Derive the base model and call save_threat_model instead."
|
|
||||||
)
|
|
||||||
amendments = _amendments_of(cached)
|
|
||||||
if len(amendments) >= _MAX_AMENDMENTS:
|
|
||||||
return None, (
|
|
||||||
f"This model already carries {len(amendments)} amendments. Fold them "
|
|
||||||
"into the base model with save_threat_model before adding more."
|
|
||||||
)
|
|
||||||
amendments.append(amendment)
|
|
||||||
cached["amendments"] = amendments
|
|
||||||
if len(json.dumps(cached, ensure_ascii=False).encode("utf-8")) > _MAX_MODEL_BYTES:
|
|
||||||
return None, "Threat model with this amendment exceeds 512KB; tighten it."
|
|
||||||
return amendments, _write_cache(path, cached)
|
|
||||||
|
|
||||||
|
|
||||||
def _amend_impl(
|
|
||||||
target: str,
|
|
||||||
addendum: str,
|
|
||||||
agent_name: str | None,
|
|
||||||
scan_targets: list[str] | None = None,
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
resolved, error = _resolve_target(target, scan_targets)
|
|
||||||
if resolved is None:
|
|
||||||
return {"success": False, "error": error}
|
|
||||||
|
|
||||||
body = (addendum or "").strip()
|
|
||||||
if len(body) < _MIN_AMENDMENT_CHARS:
|
|
||||||
return {
|
|
||||||
"success": False,
|
|
||||||
"error": (
|
|
||||||
f"Amendment is too thin ({len(body)} chars). Say what the base model "
|
|
||||||
"got wrong or left out, and name the endpoint, host, file, or control "
|
|
||||||
"that makes your correction true."
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
identity, revision = _target_identity(resolved)
|
|
||||||
amendments, amend_error = _append_amendment(
|
|
||||||
_cache_path(identity),
|
|
||||||
{
|
|
||||||
"at": datetime.now(UTC).isoformat(),
|
|
||||||
"by": agent_name,
|
|
||||||
"revision": revision,
|
|
||||||
"content": body,
|
|
||||||
},
|
|
||||||
)
|
|
||||||
if amendments is None or amend_error:
|
|
||||||
return {"success": False, "error": amend_error}
|
|
||||||
|
|
||||||
return {
|
|
||||||
"success": True,
|
|
||||||
"target": identity,
|
|
||||||
"revision": revision,
|
|
||||||
"amendment_count": len(amendments),
|
|
||||||
"message": (
|
|
||||||
"Amendment recorded. Agents calling get_threat_model will now see it "
|
|
||||||
"alongside the base model."
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _caller_agent_name(ctx: RunContextWrapper) -> str | None:
|
|
||||||
inner = ctx.context if isinstance(ctx.context, dict) else {}
|
|
||||||
agent_id = inner.get("agent_id")
|
|
||||||
coordinator = inner.get("coordinator")
|
|
||||||
if not isinstance(agent_id, str) or not isinstance(coordinator, AgentCoordinator):
|
|
||||||
return None
|
|
||||||
return coordinator.names.get(agent_id)
|
|
||||||
|
|
||||||
|
|
||||||
def _scan_targets(ctx: RunContextWrapper) -> list[str]:
|
|
||||||
"""The targets this scan was authorized against, as the runner spelled them."""
|
|
||||||
inner = ctx.context if isinstance(ctx.context, dict) else {}
|
|
||||||
targets = inner.get("scan_targets")
|
|
||||||
if not isinstance(targets, list):
|
|
||||||
return []
|
|
||||||
return [target for target in targets if isinstance(target, str) and target.strip()]
|
|
||||||
|
|
||||||
|
|
||||||
@function_tool(timeout=30)
|
|
||||||
async def get_threat_model(ctx: RunContextWrapper, target: str) -> str:
|
|
||||||
"""Read the cached threat model for a target, if one exists.
|
|
||||||
|
|
||||||
A threat model belongs to the target, not to this scan — the same
|
|
||||||
trust boundaries hold across unrelated runs against the same host
|
|
||||||
or application. Call this before you start hunting so you inherit
|
|
||||||
the shared view instead of re-deriving it, and so every agent on
|
|
||||||
this run agrees on what "attacker-controlled" means here.
|
|
||||||
|
|
||||||
Works black-box or white-box. The target can be a host, a URL, an
|
|
||||||
API base, or a repository path; equivalent spellings of the same
|
|
||||||
host resolve to the same model, and a checkout resolves to its
|
|
||||||
remote, so a model derived white-box is read back by a black-box
|
|
||||||
agent testing the deployment.
|
|
||||||
|
|
||||||
Returns ``found: false`` when nothing is cached — derive one and
|
|
||||||
persist it with ``save_threat_model``. ``stale: true`` means the
|
|
||||||
checkout moved to a different revision, or that a model with no
|
|
||||||
revision to pin to has aged out: use it as a starting point,
|
|
||||||
re-confirm what it claims, and save the corrected version.
|
|
||||||
|
|
||||||
Any ``amendments`` in the response are corrections other agents
|
|
||||||
recorded after the base model was written. They are part of the
|
|
||||||
model — read them, and prefer the later statement where one
|
|
||||||
contradicts the base text.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
target: What the model describes — a host or URL
|
|
||||||
(``https://app.example.com``), or a repository path
|
|
||||||
(``/workspace/myrepo``). Use the same value the scan was
|
|
||||||
pointed at, so agents converge on one model.
|
|
||||||
"""
|
|
||||||
return json.dumps(
|
|
||||||
await asyncio.to_thread(_get_impl, target, _scan_targets(ctx)),
|
|
||||||
ensure_ascii=False,
|
|
||||||
default=str,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@function_tool(timeout=30)
|
|
||||||
async def save_threat_model(ctx: RunContextWrapper, target: str, content: str) -> str:
|
|
||||||
"""Persist a target-scoped threat model for reuse by other agents.
|
|
||||||
|
|
||||||
Keyed by target identity, so a later scan of the same host or tree
|
|
||||||
reads it back instead of paying to derive it again.
|
|
||||||
|
|
||||||
**This replaces the whole document, and clears any amendments** —
|
|
||||||
it is for the agent establishing the baseline (normally root,
|
|
||||||
before subagents start), or for folding accumulated amendments back
|
|
||||||
into the body. If a model already exists and you only need to
|
|
||||||
correct or extend part of it, call ``amend_threat_model`` instead;
|
|
||||||
saving over it will silently discard whatever other agents added.
|
|
||||||
|
|
||||||
**Write it from whatever evidence you have.** With source, ground
|
|
||||||
it in the code and name the files, entrypoints, and controls that
|
|
||||||
make each claim true. Black-box, ground it in recon: the hosts and
|
|
||||||
ports that answered, the technology fingerprints, the observed
|
|
||||||
roles and tenants, the authentication and session model, the
|
|
||||||
endpoints and parameters you enumerated. A black-box model is
|
|
||||||
necessarily provisional — say which parts are inferred rather than
|
|
||||||
observed, and let later agents amend it as the picture fills in.
|
|
||||||
|
|
||||||
**Scope it to the target, not to this scan.** Do not centre it on
|
|
||||||
the diff you were handed, the subsystem you were assigned, or the
|
|
||||||
one host that happened to answer first. With source, distinguish
|
|
||||||
real product and runtime surfaces from test, docs, example, and
|
|
||||||
developer-tooling paths — in a monorepo, do not let ``tests/`` or
|
|
||||||
one-off scripts become the centre of gravity unless the code shows
|
|
||||||
they are genuinely deployed. Where the target documents its own
|
|
||||||
boundary — an ``AGENTS`` file, a specific ``SECURITY.md``, a
|
|
||||||
published API spec, an engagement scope — build on it rather than
|
|
||||||
inventing a competing story.
|
|
||||||
|
|
||||||
Structure the content in Markdown with these sections:
|
|
||||||
|
|
||||||
- **Overview** — what the target actually is, its real-world usage,
|
|
||||||
and which parts are product/runtime versus tooling or
|
|
||||||
non-production.
|
|
||||||
- **Trust Boundaries and Assumptions** — the boundaries, the actors
|
|
||||||
on either side, and the invariants that must hold. Separate
|
|
||||||
attacker-controlled, operator-controlled, and
|
|
||||||
developer-controlled inputs explicitly. Black-box, this is the
|
|
||||||
role, tenant, and privilege model: who can reach what before
|
|
||||||
authenticating, as a low-privilege user, and across tenants.
|
|
||||||
- **Attack Surface and Attacker Stories** — the exposed surfaces
|
|
||||||
(hosts, endpoints, parameters, integrations, or the code-level
|
|
||||||
entrypoints and sinks), the mitigations already present that
|
|
||||||
materially change severity or reach, realistic attacker stories,
|
|
||||||
and the stories that are *not* realistic here and why.
|
|
||||||
- **Severity Calibration** — what critical / high / medium / low
|
|
||||||
look like for *this* target, with a concrete example at each
|
|
||||||
level. Where a vulnerability class needs attacker control that
|
|
||||||
does not exist in real usage, say so here.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
target: What the model describes — a host or URL
|
|
||||||
(``https://app.example.com``), or a repository path
|
|
||||||
(``/workspace/myrepo``). Use the same value the scan was
|
|
||||||
pointed at.
|
|
||||||
content: The full threat model in Markdown.
|
|
||||||
"""
|
|
||||||
return json.dumps(
|
|
||||||
await asyncio.to_thread(
|
|
||||||
_save_impl, target, content, _caller_agent_name(ctx), _scan_targets(ctx)
|
|
||||||
),
|
|
||||||
ensure_ascii=False,
|
|
||||||
default=str,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@function_tool(timeout=30)
|
|
||||||
async def amend_threat_model(ctx: RunContextWrapper, target: str, addendum: str) -> str:
|
|
||||||
"""Correct or extend the existing threat model without replacing it.
|
|
||||||
|
|
||||||
The baseline is written before anyone starts hunting, so it is
|
|
||||||
written with the least information anyone will ever have. That is
|
|
||||||
doubly true black-box, where the model starts as inference over
|
|
||||||
recon output and only becomes real as agents authenticate, map
|
|
||||||
roles, and reach the surfaces behind them. When your work
|
|
||||||
contradicts the model or fills in something it missed, record that
|
|
||||||
here — every agent that calls ``get_threat_model`` afterwards sees
|
|
||||||
your addendum next to the base model.
|
|
||||||
|
|
||||||
Amendments are append-only and attributed, so two agents amending
|
|
||||||
at once both survive. That is the difference from
|
|
||||||
``save_threat_model``, which overwrites the document and drops
|
|
||||||
every amendment on it.
|
|
||||||
|
|
||||||
Worth amending:
|
|
||||||
|
|
||||||
- A boundary the model calls trusted that you found is
|
|
||||||
attacker-reachable, or vice versa.
|
|
||||||
- A host, endpoint, parameter, role, sink, or shared control the
|
|
||||||
model does not mention.
|
|
||||||
- Something the model only inferred that you have now observed — or
|
|
||||||
that turned out not to be true.
|
|
||||||
- A severity call the model got wrong for this target, with the
|
|
||||||
reason.
|
|
||||||
- An assumption you disproved — the model says input is validated
|
|
||||||
upstream and you found the path that skips it.
|
|
||||||
|
|
||||||
Not worth amending: individual findings (those are reports), or
|
|
||||||
restating what the model already says.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
target: What the model describes — the same host, URL, or
|
|
||||||
repository path used to save it.
|
|
||||||
addendum: The correction, in Markdown. State what the base
|
|
||||||
model says, what is actually true, and the endpoint, host,
|
|
||||||
file, or control that proves it.
|
|
||||||
"""
|
|
||||||
return json.dumps(
|
|
||||||
await asyncio.to_thread(
|
|
||||||
_amend_impl, target, addendum, _caller_agent_name(ctx), _scan_targets(ctx)
|
|
||||||
),
|
|
||||||
ensure_ascii=False,
|
|
||||||
default=str,
|
|
||||||
)
|
|
||||||
+1
-59
@@ -2,12 +2,11 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import time
|
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from strix.config import codex, opencode
|
from strix.config import codex
|
||||||
from strix.interface import auth_cli
|
from strix.interface import auth_cli
|
||||||
|
|
||||||
|
|
||||||
@@ -105,60 +104,3 @@ def test_login_accepts_provider_aliases(provider: str, monkeypatch: pytest.Monke
|
|||||||
|
|
||||||
assert auth_cli.run_auth(["login", provider]) == 0
|
assert auth_cli.run_auth(["login", provider]) == 0
|
||||||
assert reached["flow"] is True
|
assert reached["flow"] is True
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("provider", ["opencode", "OpenCode", "opencode-go", "zen"])
|
|
||||||
def test_login_accepts_opencode_aliases(provider: str, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
reached = {"login": False}
|
|
||||||
|
|
||||||
def _fake_login(_console: Any) -> int:
|
|
||||||
reached["login"] = True
|
|
||||||
return 0
|
|
||||||
|
|
||||||
monkeypatch.setattr(auth_cli, "_login_opencode", _fake_login)
|
|
||||||
assert auth_cli.run_auth(["login", provider]) == 0
|
|
||||||
assert reached["login"] is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_login_opencode_validates_and_saves(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
saved: dict[str, str] = {}
|
|
||||||
monkeypatch.setattr("rich.console.Console.input", lambda _self, *_a, **_k: " sk-oc-test ")
|
|
||||||
monkeypatch.setattr(opencode, "validate_api_key", lambda key: saved.setdefault("checked", key))
|
|
||||||
monkeypatch.setattr(opencode, "save_api_key", lambda key: saved.setdefault("key", key))
|
|
||||||
|
|
||||||
assert auth_cli.run_auth(["login", "opencode"]) == 0
|
|
||||||
assert saved == {"checked": "sk-oc-test", "key": "sk-oc-test"}
|
|
||||||
|
|
||||||
|
|
||||||
def test_login_opencode_rejects_bad_key(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
monkeypatch.setattr("rich.console.Console.input", lambda _self, *_a, **_k: "bad")
|
|
||||||
|
|
||||||
def _reject(_key: str) -> None:
|
|
||||||
raise opencode.OpencodeAuthError("invalid_key")
|
|
||||||
|
|
||||||
monkeypatch.setattr(opencode, "validate_api_key", _reject)
|
|
||||||
assert auth_cli.run_auth(["login", "opencode"]) == 1
|
|
||||||
assert opencode.is_authenticated() is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_logout_provider_scoped() -> None:
|
|
||||||
codex.save_record(
|
|
||||||
{
|
|
||||||
"type": "oauth",
|
|
||||||
"provider": "codex",
|
|
||||||
"access": "a",
|
|
||||||
"refresh": "r",
|
|
||||||
"account_id": "acct",
|
|
||||||
"expires_at": time.time() + 3600,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
opencode.save_api_key("sk-oc-test")
|
|
||||||
|
|
||||||
assert auth_cli.run_auth(["logout", "opencode"]) == 0
|
|
||||||
assert opencode.is_authenticated() is False
|
|
||||||
assert codex.is_authenticated() is True
|
|
||||||
|
|
||||||
assert auth_cli.run_auth(["logout"]) == 0
|
|
||||||
assert codex.is_authenticated() is False
|
|
||||||
|
|
||||||
assert auth_cli.run_auth(["logout", "bogus"]) == 2
|
|
||||||
|
|||||||
@@ -1,81 +0,0 @@
|
|||||||
"""A bootstrap that dies mid-setup must not leave its transport behind.
|
|
||||||
|
|
||||||
The bootstrap now runs concurrently with the scan start, so teardown can
|
|
||||||
cancel it at any await — including inside ``Client.connect()``, where the
|
|
||||||
client exists but no caller will ever see it to close it.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import sys
|
|
||||||
import types
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from strix.runtime.caido_bootstrap import bootstrap_caido
|
|
||||||
|
|
||||||
|
|
||||||
class _FakeExecResult:
|
|
||||||
stderr = b""
|
|
||||||
exit_code = 0
|
|
||||||
|
|
||||||
def __init__(self, stdout: str) -> None:
|
|
||||||
self.stdout = stdout
|
|
||||||
|
|
||||||
def ok(self) -> bool:
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
class _FakeSession:
|
|
||||||
async def exec(self, *_args: Any, **_kwargs: Any) -> _FakeExecResult:
|
|
||||||
return _FakeExecResult('{"data":{"loginAsGuest":{"token":{"accessToken":"t"}}}}')
|
|
||||||
|
|
||||||
|
|
||||||
class _FakeClient:
|
|
||||||
def __init__(self, connect_error: BaseException) -> None:
|
|
||||||
self.connect_error = connect_error
|
|
||||||
self.closed = False
|
|
||||||
|
|
||||||
async def connect(self) -> None:
|
|
||||||
raise self.connect_error
|
|
||||||
|
|
||||||
async def aclose(self) -> None:
|
|
||||||
self.closed = True
|
|
||||||
|
|
||||||
|
|
||||||
async def _bootstrap_expecting(
|
|
||||||
monkeypatch: pytest.MonkeyPatch, error: BaseException
|
|
||||||
) -> _FakeClient:
|
|
||||||
"""Run a bootstrap whose ``connect()`` fails with ``error``."""
|
|
||||||
client = _FakeClient(error)
|
|
||||||
# The SDK is imported inside bootstrap_caido (it is slow to import), so the
|
|
||||||
# fakes are injected as the modules it imports.
|
|
||||||
sdk = types.ModuleType("caido_sdk_client")
|
|
||||||
sdk.Client = lambda *_a, **_k: client # type: ignore[attr-defined]
|
|
||||||
sdk.TokenAuthOptions = lambda token: token # type: ignore[attr-defined]
|
|
||||||
sdk_types = types.ModuleType("caido_sdk_client.types")
|
|
||||||
sdk_types.CreateProjectOptions = lambda **_k: None # type: ignore[attr-defined]
|
|
||||||
monkeypatch.setitem(sys.modules, "caido_sdk_client", sdk)
|
|
||||||
monkeypatch.setitem(sys.modules, "caido_sdk_client.types", sdk_types)
|
|
||||||
|
|
||||||
with pytest.raises(type(error)):
|
|
||||||
await bootstrap_caido(
|
|
||||||
_FakeSession(), # type: ignore[arg-type]
|
|
||||||
host_url="http://host",
|
|
||||||
container_url="http://container",
|
|
||||||
)
|
|
||||||
return client
|
|
||||||
|
|
||||||
|
|
||||||
async def test_cancellation_during_connect_closes_the_client(
|
|
||||||
monkeypatch: pytest.MonkeyPatch,
|
|
||||||
) -> None:
|
|
||||||
client = await _bootstrap_expecting(monkeypatch, asyncio.CancelledError())
|
|
||||||
assert client.closed
|
|
||||||
|
|
||||||
|
|
||||||
async def test_failed_connect_closes_the_client(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
client = await _bootstrap_expecting(monkeypatch, RuntimeError("no listener"))
|
|
||||||
assert client.closed
|
|
||||||
@@ -1,103 +0,0 @@
|
|||||||
"""Tests for the concurrent Caido bootstrap handle."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from strix.runtime.caido_handle import CaidoBootstrapHandle
|
|
||||||
|
|
||||||
|
|
||||||
class _FakeClient:
|
|
||||||
def __init__(self) -> None:
|
|
||||||
self.closed = False
|
|
||||||
|
|
||||||
async def aclose(self) -> None:
|
|
||||||
self.closed = True
|
|
||||||
|
|
||||||
|
|
||||||
def _handle(coro: Any) -> CaidoBootstrapHandle:
|
|
||||||
return CaidoBootstrapHandle(asyncio.ensure_future(coro))
|
|
||||||
|
|
||||||
|
|
||||||
async def test_get_waits_for_the_bootstrap() -> None:
|
|
||||||
client = _FakeClient()
|
|
||||||
started = asyncio.Event()
|
|
||||||
|
|
||||||
async def _bootstrap() -> Any:
|
|
||||||
started.set()
|
|
||||||
await asyncio.sleep(0.01)
|
|
||||||
return client
|
|
||||||
|
|
||||||
handle = _handle(_bootstrap())
|
|
||||||
await started.wait()
|
|
||||||
assert handle.peek() is None
|
|
||||||
assert await handle.get() is client
|
|
||||||
assert handle.peek() is client
|
|
||||||
|
|
||||||
|
|
||||||
async def test_get_reraises_bootstrap_failure_to_every_caller() -> None:
|
|
||||||
async def _bootstrap() -> Any:
|
|
||||||
raise RuntimeError("caido never came up")
|
|
||||||
|
|
||||||
handle = _handle(_bootstrap())
|
|
||||||
for _ in range(2):
|
|
||||||
with pytest.raises(RuntimeError, match="caido never came up"):
|
|
||||||
await handle.get()
|
|
||||||
assert handle.peek() is None
|
|
||||||
|
|
||||||
|
|
||||||
async def test_caller_cancellation_does_not_cancel_the_shared_bootstrap() -> None:
|
|
||||||
client = _FakeClient()
|
|
||||||
|
|
||||||
async def _bootstrap() -> Any:
|
|
||||||
await asyncio.sleep(0.05)
|
|
||||||
return client
|
|
||||||
|
|
||||||
handle = _handle(_bootstrap())
|
|
||||||
|
|
||||||
with pytest.raises(TimeoutError):
|
|
||||||
await asyncio.wait_for(handle.get(), timeout=0.01)
|
|
||||||
|
|
||||||
assert await handle.get() is client
|
|
||||||
|
|
||||||
|
|
||||||
async def test_aclose_closes_a_finished_client() -> None:
|
|
||||||
client = _FakeClient()
|
|
||||||
|
|
||||||
async def _bootstrap() -> Any:
|
|
||||||
return client
|
|
||||||
|
|
||||||
handle = _handle(_bootstrap())
|
|
||||||
await handle.get()
|
|
||||||
await handle.aclose()
|
|
||||||
assert client.closed is True
|
|
||||||
|
|
||||||
|
|
||||||
async def test_aclose_cancels_an_in_flight_bootstrap() -> None:
|
|
||||||
cancelled = asyncio.Event()
|
|
||||||
|
|
||||||
async def _bootstrap() -> Any:
|
|
||||||
try:
|
|
||||||
await asyncio.sleep(10)
|
|
||||||
except asyncio.CancelledError:
|
|
||||||
cancelled.set()
|
|
||||||
raise
|
|
||||||
return _FakeClient()
|
|
||||||
|
|
||||||
handle = _handle(_bootstrap())
|
|
||||||
await asyncio.sleep(0)
|
|
||||||
await handle.aclose()
|
|
||||||
assert cancelled.is_set()
|
|
||||||
|
|
||||||
|
|
||||||
async def test_aclose_swallows_a_failed_bootstrap() -> None:
|
|
||||||
async def _bootstrap() -> Any:
|
|
||||||
raise RuntimeError("boom")
|
|
||||||
|
|
||||||
handle = _handle(_bootstrap())
|
|
||||||
with pytest.raises(RuntimeError, match="boom"):
|
|
||||||
await handle.get()
|
|
||||||
await handle.aclose()
|
|
||||||
@@ -1,88 +0,0 @@
|
|||||||
"""Tests for the --mcp-config CLI flag."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import importlib
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from types import SimpleNamespace
|
|
||||||
from typing import TYPE_CHECKING, Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
cli_main: Any = importlib.import_module("strix.interface.main")
|
|
||||||
|
|
||||||
|
|
||||||
def _stub_settings(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
monkeypatch.setattr(
|
|
||||||
cli_main,
|
|
||||||
"load_settings",
|
|
||||||
lambda: SimpleNamespace(runtime=SimpleNamespace(max_local_copy_mb=1024)),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_mcp_config_flag_sets_loader_override(
|
|
||||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
||||||
) -> None:
|
|
||||||
config = tmp_path / "servers.json"
|
|
||||||
config.write_text("[]", encoding="utf-8")
|
|
||||||
_stub_settings(monkeypatch)
|
|
||||||
# delenv records "originally absent" so monkeypatch removes whatever the
|
|
||||||
# parser sets, keeping the override from leaking into other tests.
|
|
||||||
monkeypatch.delenv("STRIX_MCP_CONFIG", raising=False)
|
|
||||||
monkeypatch.setattr(
|
|
||||||
sys, "argv", ["strix", "-t", "https://test.com/", "-n", "--mcp-config", str(config)]
|
|
||||||
)
|
|
||||||
|
|
||||||
args = cli_main.parse_arguments()
|
|
||||||
|
|
||||||
assert args.mcp_config == str(config)
|
|
||||||
assert os.environ["STRIX_MCP_CONFIG"] == str(config)
|
|
||||||
|
|
||||||
|
|
||||||
def test_mcp_config_flag_rejects_missing_file(
|
|
||||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
|
||||||
) -> None:
|
|
||||||
_stub_settings(monkeypatch)
|
|
||||||
monkeypatch.delenv("STRIX_MCP_CONFIG", raising=False)
|
|
||||||
missing = tmp_path / "nope.json"
|
|
||||||
monkeypatch.setattr(
|
|
||||||
sys, "argv", ["strix", "-t", "https://test.com/", "-n", "--mcp-config", str(missing)]
|
|
||||||
)
|
|
||||||
|
|
||||||
with pytest.raises(SystemExit):
|
|
||||||
cli_main.parse_arguments()
|
|
||||||
|
|
||||||
assert "--mcp-config file not found" in capsys.readouterr().err
|
|
||||||
|
|
||||||
|
|
||||||
def test_mcp_server_flags_set_selection_env(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
_stub_settings(monkeypatch)
|
|
||||||
monkeypatch.delenv("STRIX_MCP_ONLY", raising=False)
|
|
||||||
monkeypatch.delenv("STRIX_MCP_EXCLUDE", raising=False)
|
|
||||||
monkeypatch.setattr(
|
|
||||||
sys,
|
|
||||||
"argv",
|
|
||||||
[
|
|
||||||
"strix",
|
|
||||||
"-t",
|
|
||||||
"https://test.com/",
|
|
||||||
"-n",
|
|
||||||
"--mcp-server",
|
|
||||||
"a",
|
|
||||||
"--mcp-server",
|
|
||||||
"b",
|
|
||||||
"--mcp-exclude",
|
|
||||||
"c",
|
|
||||||
],
|
|
||||||
)
|
|
||||||
|
|
||||||
cli_main.parse_arguments()
|
|
||||||
|
|
||||||
assert os.environ["STRIX_MCP_ONLY"] == "a,b"
|
|
||||||
assert os.environ["STRIX_MCP_EXCLUDE"] == "c"
|
|
||||||
@@ -227,21 +227,3 @@ def test_resume_still_requires_targets_or_a_workspace(
|
|||||||
cli_main.parse_arguments()
|
cli_main.parse_arguments()
|
||||||
|
|
||||||
assert "has no targets_info" in capsys.readouterr().err
|
assert "has no targets_info" in capsys.readouterr().err
|
||||||
|
|
||||||
|
|
||||||
def test_resume_non_object_run_json_exits(
|
|
||||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]
|
|
||||||
) -> None:
|
|
||||||
monkeypatch.chdir(tmp_path)
|
|
||||||
run_dir = tmp_path / "strix_runs" / "pentest_abcd"
|
|
||||||
run_dir.mkdir(parents=True)
|
|
||||||
(run_dir / "run.json").write_text("[]", encoding="utf-8")
|
|
||||||
|
|
||||||
monkeypatch.setattr(sys, "argv", ["strix", "--resume", "pentest_abcd"])
|
|
||||||
with pytest.raises(SystemExit) as exc_info:
|
|
||||||
cli_main.parse_arguments()
|
|
||||||
|
|
||||||
assert exc_info.value.code == 2
|
|
||||||
captured = capsys.readouterr()
|
|
||||||
assert "run.json unreadable" in captured.err
|
|
||||||
assert "not an object" in captured.err
|
|
||||||
|
|||||||
@@ -31,7 +31,7 @@ def test_context_window_chatgpt_prefix_skips_provider_auth(
|
|||||||
calls.append(model)
|
calls.append(model)
|
||||||
return {"max_input_tokens": 1_050_000, "max_output_tokens": 128_000}
|
return {"max_input_tokens": 1_050_000, "max_output_tokens": 128_000}
|
||||||
|
|
||||||
monkeypatch.setattr("litellm.get_model_info", _model_info)
|
monkeypatch.setattr("strix.llm.context_budget.litellm.get_model_info", _model_info)
|
||||||
try:
|
try:
|
||||||
assert context_budget.context_window("chatgpt/gpt-5.6-luna") == 1_050_000
|
assert context_budget.context_window("chatgpt/gpt-5.6-luna") == 1_050_000
|
||||||
assert calls == ["gpt-5.6-luna"]
|
assert calls == ["gpt-5.6-luna"]
|
||||||
@@ -45,7 +45,7 @@ def test_context_window_unmapped_uses_fallback(monkeypatch: pytest.MonkeyPatch)
|
|||||||
def _raise(_model: str) -> dict[str, int]:
|
def _raise(_model: str) -> dict[str, int]:
|
||||||
raise ValueError("This model isn't mapped yet.")
|
raise ValueError("This model isn't mapped yet.")
|
||||||
|
|
||||||
monkeypatch.setattr("litellm.get_model_info", _raise)
|
monkeypatch.setattr("strix.llm.context_budget.litellm.get_model_info", _raise)
|
||||||
expected = load_settings().context.fallback_context_tokens
|
expected = load_settings().context.fallback_context_tokens
|
||||||
assert context_budget.context_window("totally-made-up-model") == expected
|
assert context_budget.context_window("totally-made-up-model") == expected
|
||||||
context_budget._model_info.cache_clear()
|
context_budget._model_info.cache_clear()
|
||||||
@@ -55,7 +55,7 @@ def test_count_tokens_fallback_on_error(monkeypatch: pytest.MonkeyPatch) -> None
|
|||||||
def _raise(**_kwargs: object) -> int:
|
def _raise(**_kwargs: object) -> int:
|
||||||
raise RuntimeError("no tokenizer")
|
raise RuntimeError("no tokenizer")
|
||||||
|
|
||||||
monkeypatch.setattr("litellm.token_counter", _raise)
|
monkeypatch.setattr("strix.llm.context_budget.litellm.token_counter", _raise)
|
||||||
# Falls back to UTF-8 byte length (upper bound on tokens).
|
# Falls back to UTF-8 byte length (upper bound on tokens).
|
||||||
assert context_budget.count_tokens("weird-model", "x" * 400) == 400
|
assert context_budget.count_tokens("weird-model", "x" * 400) == 400
|
||||||
assert context_budget.count_tokens("weird-model", "😀" * 10) == 40
|
assert context_budget.count_tokens("weird-model", "😀" * 10) == 40
|
||||||
|
|||||||
@@ -1,284 +0,0 @@
|
|||||||
"""Tests for the scan coverage ledger."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import threading
|
|
||||||
from concurrent.futures import ThreadPoolExecutor
|
|
||||||
from typing import TYPE_CHECKING, Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from strix.tools.coverage.tools import (
|
|
||||||
_list_impl,
|
|
||||||
_record_impl,
|
|
||||||
_update_impl,
|
|
||||||
get_coverage_entries,
|
|
||||||
hydrate_coverage_from_disk,
|
|
||||||
outcome_counts,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def coverage_store(tmp_path: Path) -> Path:
|
|
||||||
hydrate_coverage_from_disk(tmp_path)
|
|
||||||
return tmp_path
|
|
||||||
|
|
||||||
|
|
||||||
def _record(**overrides: str) -> dict[str, Any]:
|
|
||||||
kwargs = {
|
|
||||||
"surface": "POST /api/orders/{id}",
|
|
||||||
"risk_area": "object-level authorization",
|
|
||||||
"outcome": "no_issue_found",
|
|
||||||
"evidence": "Tested with two tenants; both received 403.",
|
|
||||||
"agent_id": "agent-1",
|
|
||||||
"agent_name": "authz-tester",
|
|
||||||
}
|
|
||||||
kwargs.update(overrides)
|
|
||||||
return _record_impl(**kwargs)
|
|
||||||
|
|
||||||
|
|
||||||
def test_record_persists_entry(coverage_store: Path) -> None:
|
|
||||||
result = _record()
|
|
||||||
assert result["success"] is True
|
|
||||||
|
|
||||||
entries = get_coverage_entries()
|
|
||||||
assert len(entries) == 1
|
|
||||||
assert entries[0]["surface"] == "POST /api/orders/{id}"
|
|
||||||
assert entries[0]["outcome"] == "no_issue_found"
|
|
||||||
assert entries[0]["agent_name"] == "authz-tester"
|
|
||||||
assert (coverage_store / "coverage.json").exists()
|
|
||||||
|
|
||||||
|
|
||||||
def test_record_normalizes_outcome() -> None:
|
|
||||||
assert _record(outcome="Needs Follow-Up")["success"] is True
|
|
||||||
assert get_coverage_entries()[0]["outcome"] == "needs_follow_up"
|
|
||||||
|
|
||||||
|
|
||||||
def test_record_rejects_unknown_outcome() -> None:
|
|
||||||
result = _record(outcome="looks fine")
|
|
||||||
assert result["success"] is False
|
|
||||||
assert any("Invalid outcome" in e for e in result["errors"])
|
|
||||||
assert not get_coverage_entries()
|
|
||||||
|
|
||||||
|
|
||||||
def test_record_requires_surface_and_risk_area() -> None:
|
|
||||||
result = _record(surface=" ", risk_area="")
|
|
||||||
assert result["success"] is False
|
|
||||||
joined = " ".join(result["errors"])
|
|
||||||
assert "surface" in joined
|
|
||||||
assert "risk_area" in joined
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("outcome", ["ruled_out", "not_applicable", "needs_follow_up"])
|
|
||||||
def test_evidence_required_for_asserted_outcomes(outcome: str) -> None:
|
|
||||||
result = _record(outcome=outcome, evidence=" ")
|
|
||||||
assert result["success"] is False
|
|
||||||
assert any("evidence is required" in e for e in result["errors"])
|
|
||||||
|
|
||||||
|
|
||||||
def test_evidence_optional_for_reported() -> None:
|
|
||||||
assert _record(outcome="reported", evidence="")["success"] is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_outcome_counts_and_filtering() -> None:
|
|
||||||
_record(surface="/login", outcome="reported", evidence="")
|
|
||||||
_record(surface="/search", outcome="no_issue_found")
|
|
||||||
_record(surface="/upload", outcome="needs_follow_up", evidence="No credentials to test.")
|
|
||||||
|
|
||||||
assert outcome_counts() == {"reported": 1, "no_issue_found": 1, "needs_follow_up": 1}
|
|
||||||
|
|
||||||
listed = _list_impl(outcome="needs_follow_up", surface=None, caller_agent_id="agent-1")
|
|
||||||
assert listed["filtered_count"] == 1
|
|
||||||
assert listed["entries"][0]["surface"] == "/upload"
|
|
||||||
assert listed["entries"][0]["by_you"] is True
|
|
||||||
|
|
||||||
by_surface = _list_impl(outcome=None, surface="sea", caller_agent_id=None)
|
|
||||||
assert by_surface["filtered_count"] == 1
|
|
||||||
assert by_surface["entries"][0]["surface"] == "/search"
|
|
||||||
|
|
||||||
|
|
||||||
def test_list_rejects_unknown_outcome_filter() -> None:
|
|
||||||
result = _list_impl(outcome="bogus", surface=None, caller_agent_id=None)
|
|
||||||
assert result["success"] is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_hydrate_reloads_from_disk(coverage_store: Path) -> None:
|
|
||||||
_record()
|
|
||||||
hydrate_coverage_from_disk(coverage_store)
|
|
||||||
entries = get_coverage_entries()
|
|
||||||
assert len(entries) == 1
|
|
||||||
assert entries[0]["risk_area"] == "object-level authorization"
|
|
||||||
|
|
||||||
|
|
||||||
def _update(entry_id: str, **overrides: str) -> dict[str, Any]:
|
|
||||||
kwargs = {
|
|
||||||
"entry_id": entry_id,
|
|
||||||
"outcome": "reported",
|
|
||||||
"evidence": "Got staging credentials and confirmed the IDOR.",
|
|
||||||
"agent_id": "agent-2",
|
|
||||||
"agent_name": "followup-tester",
|
|
||||||
}
|
|
||||||
kwargs.update(overrides)
|
|
||||||
return _update_impl(**kwargs)
|
|
||||||
|
|
||||||
|
|
||||||
def test_update_moves_outcome_and_keeps_history() -> None:
|
|
||||||
recorded = _record(outcome="needs_follow_up", evidence="No credentials to test.")
|
|
||||||
entry_id = str(recorded["entry_id"])
|
|
||||||
|
|
||||||
result = _update(entry_id)
|
|
||||||
|
|
||||||
assert result["success"] is True
|
|
||||||
assert result["previous_outcome"] == "needs_follow_up"
|
|
||||||
assert result["outcome"] == "reported"
|
|
||||||
|
|
||||||
entries = get_coverage_entries()
|
|
||||||
assert len(entries) == 1, "update must not create a parallel entry"
|
|
||||||
entry = entries[0]
|
|
||||||
assert entry["outcome"] == "reported"
|
|
||||||
assert entry["agent_name"] == "followup-tester"
|
|
||||||
assert entry["history"] == [
|
|
||||||
{
|
|
||||||
"outcome": "needs_follow_up",
|
|
||||||
"recorded_at": entry["created_at"],
|
|
||||||
"evidence": "No credentials to test.",
|
|
||||||
"agent_name": "authz-tester",
|
|
||||||
}
|
|
||||||
]
|
|
||||||
assert outcome_counts() == {"reported": 1}
|
|
||||||
|
|
||||||
|
|
||||||
def test_update_can_reopen_a_closed_entry() -> None:
|
|
||||||
recorded = _record(outcome="ruled_out", evidence="Guard at auth.py:40 covers the path.")
|
|
||||||
entry_id = str(recorded["entry_id"])
|
|
||||||
|
|
||||||
_update(
|
|
||||||
entry_id,
|
|
||||||
outcome="needs_follow_up",
|
|
||||||
evidence="The guard is skipped on the /v2 alias; reachability unproven.",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert outcome_counts() == {"needs_follow_up": 1}
|
|
||||||
listed = _list_impl(outcome=None, surface=None, caller_agent_id=None)
|
|
||||||
assert listed["entries"][0]["previous_outcomes"] == ["ruled_out"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_update_enforces_evidence_for_closing_outcomes() -> None:
|
|
||||||
entry_id = str(_record(outcome="needs_follow_up", evidence="unknown")["entry_id"])
|
|
||||||
|
|
||||||
result = _update(entry_id, outcome="ruled_out", evidence=" ")
|
|
||||||
|
|
||||||
assert result["success"] is False
|
|
||||||
assert get_coverage_entries()[0]["outcome"] == "needs_follow_up"
|
|
||||||
|
|
||||||
|
|
||||||
def test_update_rejects_unknown_entry() -> None:
|
|
||||||
result = _update("nope")
|
|
||||||
assert result["success"] is False
|
|
||||||
assert "list_coverage" in str(result["error"])
|
|
||||||
|
|
||||||
|
|
||||||
def test_update_persists_to_disk(coverage_store: Path) -> None:
|
|
||||||
entry_id = str(_record(outcome="needs_follow_up", evidence="No creds.")["entry_id"])
|
|
||||||
_update(entry_id)
|
|
||||||
|
|
||||||
hydrate_coverage_from_disk(coverage_store)
|
|
||||||
|
|
||||||
entry = get_coverage_entries()[0]
|
|
||||||
assert entry["outcome"] == "reported"
|
|
||||||
assert len(entry["history"]) == 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_recording_a_duplicate_surface_is_refused_with_the_existing_id() -> None:
|
|
||||||
first = _record_impl(
|
|
||||||
surface="/api/invoices",
|
|
||||||
risk_area="IDOR",
|
|
||||||
outcome="needs_follow_up",
|
|
||||||
evidence="No second tenant account to test cross-tenant reads with.",
|
|
||||||
agent_id="a1",
|
|
||||||
agent_name="Recon",
|
|
||||||
)
|
|
||||||
|
|
||||||
duplicate = _record_impl(
|
|
||||||
surface=" /API/Invoices ",
|
|
||||||
risk_area="idor",
|
|
||||||
outcome="reported",
|
|
||||||
evidence="Cross-tenant read confirmed.",
|
|
||||||
agent_id="a2",
|
|
||||||
agent_name="Authz",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert duplicate["success"] is False
|
|
||||||
assert duplicate["existing_entry_id"] == first["entry_id"]
|
|
||||||
assert duplicate["existing_outcome"] == "needs_follow_up"
|
|
||||||
assert "update_coverage" in duplicate["error"]
|
|
||||||
assert len(get_coverage_entries()) == 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_a_different_risk_area_on_one_surface_is_still_its_own_entry() -> None:
|
|
||||||
_record_impl(
|
|
||||||
surface="/api/invoices",
|
|
||||||
risk_area="IDOR",
|
|
||||||
outcome="no_issue_found",
|
|
||||||
evidence="Tenant id read from the session.",
|
|
||||||
agent_id="a1",
|
|
||||||
agent_name="Authz",
|
|
||||||
)
|
|
||||||
second = _record_impl(
|
|
||||||
surface="/api/invoices",
|
|
||||||
risk_area="SQL injection",
|
|
||||||
outcome="no_issue_found",
|
|
||||||
evidence="Parameterized throughout.",
|
|
||||||
agent_id="a1",
|
|
||||||
agent_name="Injection",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert second["success"] is True
|
|
||||||
assert len(get_coverage_entries()) == 2
|
|
||||||
|
|
||||||
|
|
||||||
def test_concurrent_records_of_one_surface_yield_a_single_row() -> None:
|
|
||||||
"""Duplicate detection and insertion must be one critical section.
|
|
||||||
|
|
||||||
Two agents recording the same surface at the same moment would otherwise
|
|
||||||
both pass the "no duplicate" check, and the report would show a stale
|
|
||||||
conclusion beside its replacement — the exact outcome the rejection exists
|
|
||||||
to prevent.
|
|
||||||
"""
|
|
||||||
barrier = threading.Barrier(8)
|
|
||||||
|
|
||||||
def attempt(index: int) -> dict[str, Any]:
|
|
||||||
barrier.wait()
|
|
||||||
return _record(agent_id=f"agent-{index}", agent_name=f"tester-{index}")
|
|
||||||
|
|
||||||
with ThreadPoolExecutor(max_workers=8) as pool:
|
|
||||||
results = list(pool.map(attempt, range(8)))
|
|
||||||
|
|
||||||
assert sum(1 for result in results if result["success"]) == 1
|
|
||||||
assert len(get_coverage_entries()) == 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_concurrent_records_all_survive_persistence(coverage_store: Path) -> None:
|
|
||||||
"""A writer holding an older snapshot must not win the rename.
|
|
||||||
|
|
||||||
If it did, the mirror would come back short on resume and coverage
|
|
||||||
recorded before a crash would silently disappear from the report.
|
|
||||||
"""
|
|
||||||
barrier = threading.Barrier(8)
|
|
||||||
|
|
||||||
def attempt(index: int) -> dict[str, Any]:
|
|
||||||
barrier.wait()
|
|
||||||
return _record(surface=f"GET /api/resource/{index}", agent_id=f"agent-{index}")
|
|
||||||
|
|
||||||
with ThreadPoolExecutor(max_workers=8) as pool:
|
|
||||||
list(pool.map(attempt, range(8)))
|
|
||||||
|
|
||||||
persisted = json.loads((coverage_store / "coverage.json").read_text(encoding="utf-8"))
|
|
||||||
assert len(persisted) == 8
|
|
||||||
hydrate_coverage_from_disk(coverage_store)
|
|
||||||
assert len(get_coverage_entries()) == 8
|
|
||||||
@@ -1,65 +0,0 @@
|
|||||||
"""finish_scan confronts the root agent with the coverage the runtime can see."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import TYPE_CHECKING
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from strix.tools.coverage.tools import _record_impl, hydrate_coverage_from_disk
|
|
||||||
from strix.tools.finish.tool import _coverage_summary
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
_GRAPH = {
|
|
||||||
"statuses": {"agent-1": "completed"},
|
|
||||||
"names": {"agent-1": "injection-tester"},
|
|
||||||
"metadata": {"agent-1": {"skills": ["sql_injection", "xss"]}},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _empty_ledger(tmp_path: Path) -> None:
|
|
||||||
hydrate_coverage_from_disk(tmp_path)
|
|
||||||
|
|
||||||
|
|
||||||
def _record(risk_area: str) -> None:
|
|
||||||
_record_impl(
|
|
||||||
surface="POST /api/orders/{id}",
|
|
||||||
risk_area=risk_area,
|
|
||||||
outcome="no_issue_found",
|
|
||||||
evidence="Parameters fuzzed; no anomalies.",
|
|
||||||
agent_id="agent-1",
|
|
||||||
agent_name="injection-tester",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_unrecorded_risk_class_is_reported_back_to_the_root_agent() -> None:
|
|
||||||
_record("SQL injection")
|
|
||||||
|
|
||||||
summary = _coverage_summary(_GRAPH)
|
|
||||||
|
|
||||||
assert summary["coverage_recorded"] == 1
|
|
||||||
assert len(summary["coverage_gaps"]) == 1
|
|
||||||
assert "xss" in summary["coverage_gaps"][0]
|
|
||||||
assert "unexamined" in summary["coverage_gap_warning"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_fully_accounted_coverage_raises_no_gap_warning() -> None:
|
|
||||||
_record("SQL injection")
|
|
||||||
_record("cross-site scripting")
|
|
||||||
|
|
||||||
summary = _coverage_summary(_GRAPH)
|
|
||||||
|
|
||||||
assert "coverage_gaps" not in summary
|
|
||||||
assert "coverage_gap_warning" not in summary
|
|
||||||
|
|
||||||
|
|
||||||
def test_an_empty_ledger_still_warns_first() -> None:
|
|
||||||
summary = _coverage_summary(_GRAPH)
|
|
||||||
|
|
||||||
assert summary["coverage_recorded"] == 0
|
|
||||||
assert "No coverage was recorded" in summary["coverage_warning"]
|
|
||||||
@@ -10,7 +10,6 @@ import pytest
|
|||||||
|
|
||||||
from strix.core.inputs import (
|
from strix.core.inputs import (
|
||||||
build_root_task,
|
build_root_task,
|
||||||
build_scan_targets,
|
|
||||||
build_scope_context,
|
build_scope_context,
|
||||||
child_initial_input,
|
child_initial_input,
|
||||||
make_model_settings,
|
make_model_settings,
|
||||||
@@ -112,23 +111,6 @@ def test_make_model_settings_no_prompt_cache_for_non_claude(model_name: str) ->
|
|||||||
assert make_model_settings(None, model_name=model_name).extra_args is None
|
assert make_model_settings(None, model_name=model_name).extra_args is None
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("model_name", ["opencode/claude-sonnet-5", "opencode-go/claude-sonnet-5"])
|
|
||||||
def test_prompt_cache_for_opencode_claude(model_name: str) -> None:
|
|
||||||
# Claude on OpenCode runs through LiteLLM's Anthropic route, which consumes
|
|
||||||
# cache_control_injection_points. The gateway's other two routes use the raw
|
|
||||||
# OpenAI SDK, whose create() rejects this LiteLLM-only argument.
|
|
||||||
assert _cache_points(model_name) == [
|
|
||||||
{"location": "message", "role": "system"},
|
|
||||||
{"location": "message", "index": -1},
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def test_no_prompt_cache_for_opencode_openai_routes() -> None:
|
|
||||||
# A "claude" substring cannot smuggle the LiteLLM-only argument onto a route
|
|
||||||
# that is served by the raw OpenAI SDK.
|
|
||||||
assert _cache_points("opencode/gpt-5.4-claude-tuned") is None
|
|
||||||
|
|
||||||
|
|
||||||
def test_no_prompt_cache_for_unmapped_bedrock_claude_model(monkeypatch: Any) -> None:
|
def test_no_prompt_cache_for_unmapped_bedrock_claude_model(monkeypatch: Any) -> None:
|
||||||
# A Bedrock Claude model LiteLLM hasn't mapped must run uncached, not crash.
|
# A Bedrock Claude model LiteLLM hasn't mapped must run uncached, not crash.
|
||||||
unmapped = "bedrock/global.anthropic.claude-brand-new-9"
|
unmapped = "bedrock/global.anthropic.claude-brand-new-9"
|
||||||
@@ -381,35 +363,6 @@ def test_make_model_settings_timeout_survives_reasoning_resolve() -> None:
|
|||||||
assert settings.extra_args["timeout"] == 120.0
|
assert settings.extra_args["timeout"] == 120.0
|
||||||
|
|
||||||
|
|
||||||
def test_scan_targets_prefer_the_workspace_checkout_over_the_remote_url() -> None:
|
|
||||||
config = {
|
|
||||||
"targets": [
|
|
||||||
{
|
|
||||||
"type": "repository",
|
|
||||||
"details": {
|
|
||||||
"target_repo": "https://github.com/acme/billing",
|
|
||||||
"workspace_subdir": "billing",
|
|
||||||
},
|
|
||||||
},
|
|
||||||
{"type": "web_application", "details": {"target_url": "https://app.example.com"}},
|
|
||||||
]
|
|
||||||
}
|
|
||||||
|
|
||||||
assert build_scan_targets(config) == ["/workspace/billing", "https://app.example.com"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_scan_targets_drop_empty_and_duplicate_entries() -> None:
|
|
||||||
config = {
|
|
||||||
"targets": [
|
|
||||||
{"type": "web_application", "details": {"target_url": "https://app.example.com"}},
|
|
||||||
{"type": "web_application", "details": {"target_url": "https://app.example.com"}},
|
|
||||||
{"type": "ip_address", "details": {}},
|
|
||||||
]
|
|
||||||
}
|
|
||||||
|
|
||||||
assert build_scan_targets(config) == ["https://app.example.com"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_openrouter_attribution_rides_on_the_request_headers() -> None:
|
def test_openrouter_attribution_rides_on_the_request_headers() -> None:
|
||||||
# litellm.headers is ignored once a request carries any header of its own,
|
# litellm.headers is ignored once a request carries any header of its own,
|
||||||
# so the attribution must be part of the per-request headers.
|
# so the attribution must be part of the per-request headers.
|
||||||
|
|||||||
@@ -1,715 +0,0 @@
|
|||||||
"""Tests for the generic MCP client: config contract, namespacing, and filtering."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
from typing import TYPE_CHECKING, Any
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
from agents.mcp import MCPServer, MCPServerStdio, MCPServerStreamableHttp
|
|
||||||
from mcp.types import CallToolResult, TextContent
|
|
||||||
from mcp.types import Tool as MCPTool
|
|
||||||
from pydantic import ValidationError
|
|
||||||
|
|
||||||
from strix.agents import factory
|
|
||||||
from strix.core.runner import _mcp_connection_notes
|
|
||||||
from strix.interface.tui.live_view import TuiLiveView, _tool_status_from_result
|
|
||||||
from strix.tools.mcp import (
|
|
||||||
BearerAuth,
|
|
||||||
ConnectedMcpServer,
|
|
||||||
McpConnectionConfig,
|
|
||||||
load_user_mcp_configs,
|
|
||||||
namespaced_tool_name,
|
|
||||||
resolve_mcp_tool,
|
|
||||||
)
|
|
||||||
from strix.tools.mcp import client as mcp_client
|
|
||||||
from strix.tools.mcp.client import _auth_headers, _build_server, _register_server_tools
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from agents.tool import Tool
|
|
||||||
|
|
||||||
|
|
||||||
class FakeMCPServer(MCPServer):
|
|
||||||
"""A connected MCP server stand-in, so tests never touch the network."""
|
|
||||||
|
|
||||||
def __init__(self, name: str, tools: list[MCPTool]) -> None:
|
|
||||||
super().__init__()
|
|
||||||
self._name = name
|
|
||||||
self._tools = tools
|
|
||||||
self.calls: list[tuple[str, dict[str, Any] | None]] = []
|
|
||||||
|
|
||||||
@property
|
|
||||||
def name(self) -> str:
|
|
||||||
return self._name
|
|
||||||
|
|
||||||
async def connect(self) -> None:
|
|
||||||
return None
|
|
||||||
|
|
||||||
async def cleanup(self) -> None:
|
|
||||||
return None
|
|
||||||
|
|
||||||
async def list_tools(
|
|
||||||
self,
|
|
||||||
run_context: Any = None,
|
|
||||||
agent: Any = None,
|
|
||||||
) -> list[MCPTool]:
|
|
||||||
return list(self._tools)
|
|
||||||
|
|
||||||
async def call_tool(
|
|
||||||
self,
|
|
||||||
tool_name: str,
|
|
||||||
arguments: dict[str, Any] | None,
|
|
||||||
meta: dict[str, Any] | None = None,
|
|
||||||
) -> CallToolResult:
|
|
||||||
self.calls.append((tool_name, arguments))
|
|
||||||
return CallToolResult(content=[TextContent(type="text", text=f"routed:{tool_name}")])
|
|
||||||
|
|
||||||
async def list_prompts(self) -> Any:
|
|
||||||
raise NotImplementedError
|
|
||||||
|
|
||||||
async def get_prompt(self, name: str, arguments: dict[str, Any] | None = None) -> Any:
|
|
||||||
raise NotImplementedError
|
|
||||||
|
|
||||||
|
|
||||||
def _mcp_tool(name: str) -> MCPTool:
|
|
||||||
return MCPTool(
|
|
||||||
name=name,
|
|
||||||
description=f"remote tool {name}",
|
|
||||||
inputSchema={"type": "object", "properties": {}},
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _config(name: str, allowed_tools: list[str]) -> McpConnectionConfig:
|
|
||||||
return McpConnectionConfig(
|
|
||||||
name=name,
|
|
||||||
url="https://mcp.example.com",
|
|
||||||
auth=BearerAuth(token="run-token"),
|
|
||||||
allowed_tools=allowed_tools,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _clear_mcp_env(monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
"""Hide any MCP settings the developer has exported in their own shell.
|
|
||||||
|
|
||||||
The loader reads these to resolve the config path and the per-run
|
|
||||||
include/exclude selection, so a shell that has them set (from using
|
|
||||||
--mcp-config or --mcp-server) would otherwise filter what these tests see.
|
|
||||||
"""
|
|
||||||
for name in ("STRIX_MCP_CONFIG", "STRIX_MCP_ONLY", "STRIX_MCP_EXCLUDE"):
|
|
||||||
monkeypatch.delenv(name, raising=False)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _reset_registry() -> Any:
|
|
||||||
saved = list(factory._EXTRA_TOOLS)
|
|
||||||
factory._EXTRA_TOOLS.clear()
|
|
||||||
try:
|
|
||||||
yield
|
|
||||||
finally:
|
|
||||||
factory._EXTRA_TOOLS[:] = saved
|
|
||||||
|
|
||||||
|
|
||||||
# --- config contract ---------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_bearer_config_parses_from_dict() -> None:
|
|
||||||
config = McpConnectionConfig.model_validate(
|
|
||||||
{
|
|
||||||
"name": "files_main",
|
|
||||||
"transport": "http",
|
|
||||||
"url": "https://mcp.example.com",
|
|
||||||
"auth": {"kind": "bearer", "token": "abc"},
|
|
||||||
"allowed_tools": ["list_files"],
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
assert isinstance(config.auth, BearerAuth)
|
|
||||||
assert config.auth.token == "abc"
|
|
||||||
assert config.allowed_tools == ["list_files"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_unknown_auth_kind_is_rejected() -> None:
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
McpConnectionConfig.model_validate(
|
|
||||||
{
|
|
||||||
"name": "x",
|
|
||||||
"url": "https://mcp.example.com",
|
|
||||||
"auth": {"kind": "oauth", "token": "abc"},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_stdio_config_parses_from_dict() -> None:
|
|
||||||
config = McpConnectionConfig.model_validate(
|
|
||||||
{
|
|
||||||
"name": "local_fs",
|
|
||||||
"transport": "stdio",
|
|
||||||
"command": "npx",
|
|
||||||
"args": ["-y", "@modelcontextprotocol/server-filesystem", "/srv/data"],
|
|
||||||
"env": {"FOO": "bar"},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
assert config.transport == "stdio"
|
|
||||||
assert config.command == "npx"
|
|
||||||
assert config.args == ["-y", "@modelcontextprotocol/server-filesystem", "/srv/data"]
|
|
||||||
assert config.env == {"FOO": "bar"}
|
|
||||||
# A local stdio server needs no auth, and omitting allowed_tools means "all".
|
|
||||||
assert config.auth is None
|
|
||||||
assert config.allowed_tools is None
|
|
||||||
|
|
||||||
|
|
||||||
def test_http_config_without_url_is_rejected() -> None:
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
McpConnectionConfig.model_validate(
|
|
||||||
{
|
|
||||||
"name": "x",
|
|
||||||
"transport": "http",
|
|
||||||
"auth": {"kind": "bearer", "token": "abc"},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_stdio_config_without_command_is_rejected() -> None:
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
McpConnectionConfig.model_validate(
|
|
||||||
{
|
|
||||||
"name": "x",
|
|
||||||
"transport": "stdio",
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_empty_name_is_rejected() -> None:
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
McpConnectionConfig.model_validate(
|
|
||||||
{
|
|
||||||
"name": "",
|
|
||||||
"url": "https://mcp.example.com",
|
|
||||||
"auth": {"kind": "bearer", "token": "abc"},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_unknown_field_is_rejected() -> None:
|
|
||||||
with pytest.raises(ValidationError):
|
|
||||||
McpConnectionConfig.model_validate(
|
|
||||||
{
|
|
||||||
"name": "x",
|
|
||||||
"url": "https://mcp.example.com",
|
|
||||||
"auth": {"kind": "bearer", "token": "abc"},
|
|
||||||
"surprise": True,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# --- auth headers ------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_bearer_auth_builds_authorization_header() -> None:
|
|
||||||
headers = _auth_headers(_config("files_main", []))
|
|
||||||
|
|
||||||
assert headers == {"Authorization": "Bearer run-token"}
|
|
||||||
|
|
||||||
|
|
||||||
# --- namespacing and filtering -----------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _registered_names() -> list[str]:
|
|
||||||
return [tool.name for tool in factory.registered_agent_tools()]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_tools_are_namespaced_per_connection() -> None:
|
|
||||||
server_a = FakeMCPServer("conn_a", [_mcp_tool("describe")])
|
|
||||||
server_b = FakeMCPServer("conn_b", [_mcp_tool("describe")])
|
|
||||||
|
|
||||||
await _register_server_tools(_config("conn_a", ["describe"]), server_a)
|
|
||||||
await _register_server_tools(_config("conn_b", ["describe"]), server_b)
|
|
||||||
|
|
||||||
# Same remote tool name on two connections does not collide.
|
|
||||||
assert _registered_names() == ["conn_a_describe", "conn_b_describe"]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_registered_names_are_valid_tool_names() -> None:
|
|
||||||
# Model APIs reject a tool name containing anything but letters, digits,
|
|
||||||
# underscores and hyphens, and reject the whole request rather than the one
|
|
||||||
# tool. A server naming its own tools with dots, or a connection named with
|
|
||||||
# a space in the user's config, must not be able to break a run.
|
|
||||||
server = FakeMCPServer("my server", [_mcp_tool("db.query"), _mcp_tool("ok_tool")])
|
|
||||||
|
|
||||||
await _register_server_tools(_config("my server", None), server)
|
|
||||||
|
|
||||||
names = _registered_names()
|
|
||||||
assert names == ["my_server_db_query", "my_server_ok_tool"]
|
|
||||||
assert all(re.fullmatch(r"[a-zA-Z0-9_-]{1,128}", name) for name in names)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_a_rename_does_not_change_which_tool_is_called() -> None:
|
|
||||||
# Only the model-facing name is sanitized; the server is always asked for the
|
|
||||||
# tool name it reported.
|
|
||||||
server = FakeMCPServer("my server", [_mcp_tool("db.query")])
|
|
||||||
|
|
||||||
tools = await _register_server_tools(_config("my server", None), server)
|
|
||||||
|
|
||||||
assert tools[0].name == "my_server_db_query"
|
|
||||||
await tools[0].on_invoke_tool(None, "{}")
|
|
||||||
assert server.calls == [("db.query", {})]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_disallowed_tool_is_not_registered() -> None:
|
|
||||||
server = FakeMCPServer(
|
|
||||||
"files_main",
|
|
||||||
[_mcp_tool("list_files"), _mcp_tool("search")],
|
|
||||||
)
|
|
||||||
|
|
||||||
await _register_server_tools(_config("files_main", ["list_files"]), server)
|
|
||||||
|
|
||||||
names = _registered_names()
|
|
||||||
assert "files_main_list_files" in names
|
|
||||||
assert "files_main_search" not in names
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_allowed_tools_none_registers_every_listed_tool() -> None:
|
|
||||||
server = FakeMCPServer(
|
|
||||||
"local_fs",
|
|
||||||
[_mcp_tool("read_file"), _mcp_tool("write_file")],
|
|
||||||
)
|
|
||||||
config = McpConnectionConfig(name="local_fs", url="https://mcp.example.com", allowed_tools=None)
|
|
||||||
|
|
||||||
await _register_server_tools(config, server)
|
|
||||||
|
|
||||||
names = _registered_names()
|
|
||||||
assert "local_fs_read_file" in names
|
|
||||||
assert "local_fs_write_file" in names
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_allowed_tools_list_restricts_registration() -> None:
|
|
||||||
server = FakeMCPServer(
|
|
||||||
"local_fs",
|
|
||||||
[_mcp_tool("read_file"), _mcp_tool("write_file")],
|
|
||||||
)
|
|
||||||
|
|
||||||
await _register_server_tools(_config("local_fs", ["read_file"]), server)
|
|
||||||
|
|
||||||
names = _registered_names()
|
|
||||||
assert names == ["local_fs_read_file"]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_registered_tool_routes_to_its_server_with_the_original_name() -> None:
|
|
||||||
server = FakeMCPServer("files_main", [_mcp_tool("list_files")])
|
|
||||||
|
|
||||||
tools: list[Tool] = await _register_server_tools(_config("files_main", ["list_files"]), server)
|
|
||||||
tool = tools[0]
|
|
||||||
|
|
||||||
output = await tool.on_invoke_tool(None, "{}") # type: ignore[union-attr]
|
|
||||||
|
|
||||||
# The call reaches the right server, addressed by the unprefixed remote name.
|
|
||||||
assert server.calls == [("list_files", {})]
|
|
||||||
assert output == {"type": "text", "text": "routed:list_files"}
|
|
||||||
|
|
||||||
|
|
||||||
# --- result transform --------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_result_transform_receives_namespaced_name_and_structured_result() -> None:
|
|
||||||
server = FakeMCPServer("files_main", [_mcp_tool("list_files")])
|
|
||||||
seen: list[tuple[str, Any]] = []
|
|
||||||
|
|
||||||
def transform(name: str, structured: Any) -> Any:
|
|
||||||
seen.append((name, structured))
|
|
||||||
return {"kept": structured["content"][0]["text"]}
|
|
||||||
|
|
||||||
tools: list[Tool] = await _register_server_tools(
|
|
||||||
_config("files_main", ["list_files"]), server, result_transform=transform
|
|
||||||
)
|
|
||||||
|
|
||||||
output = await tools[0].on_invoke_tool(None, "{}") # type: ignore[union-attr]
|
|
||||||
|
|
||||||
# The underlying MCP call still routes by the unprefixed remote name.
|
|
||||||
assert server.calls == [("list_files", {})]
|
|
||||||
|
|
||||||
# The transform is called with the namespaced name and the parsed result.
|
|
||||||
assert len(seen) == 1
|
|
||||||
name, structured = seen[0]
|
|
||||||
assert name == "files_main_list_files"
|
|
||||||
# A parsed CallToolResult (dict/list), not a pre-serialized string.
|
|
||||||
assert structured["content"][0]["text"] == "routed:list_files"
|
|
||||||
assert structured["isError"] is False
|
|
||||||
|
|
||||||
# The transform's return value is exactly what the tool yields.
|
|
||||||
assert output == {"kept": "routed:list_files"}
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_result_transform_can_rewrite_the_tool_output() -> None:
|
|
||||||
server = FakeMCPServer("files_main", [_mcp_tool("list_files")])
|
|
||||||
|
|
||||||
def transform(_name: str, structured: Any) -> Any:
|
|
||||||
# Keep only a truncated view of the text field.
|
|
||||||
return structured["content"][0]["text"][:6]
|
|
||||||
|
|
||||||
tools: list[Tool] = await _register_server_tools(
|
|
||||||
_config("files_main", ["list_files"]), server, result_transform=transform
|
|
||||||
)
|
|
||||||
|
|
||||||
output = await tools[0].on_invoke_tool(None, "{}") # type: ignore[union-attr]
|
|
||||||
|
|
||||||
assert output == "routed"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_without_result_transform_output_is_unchanged() -> None:
|
|
||||||
server = FakeMCPServer("files_main", [_mcp_tool("list_files")])
|
|
||||||
|
|
||||||
tools: list[Tool] = await _register_server_tools(
|
|
||||||
_config("files_main", ["list_files"]), server, result_transform=None
|
|
||||||
)
|
|
||||||
|
|
||||||
output = await tools[0].on_invoke_tool(None, "{}") # type: ignore[union-attr]
|
|
||||||
|
|
||||||
# Same shape the SDK produces today: no transform in the path.
|
|
||||||
assert server.calls == [("list_files", {})]
|
|
||||||
assert output == {"type": "text", "text": "routed:list_files"}
|
|
||||||
|
|
||||||
|
|
||||||
# --- error status capture ----------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
class ErroringMCPServer(FakeMCPServer):
|
|
||||||
"""A connected server whose calls come back as MCP errors (isError=True)."""
|
|
||||||
|
|
||||||
async def call_tool(
|
|
||||||
self,
|
|
||||||
tool_name: str,
|
|
||||||
arguments: dict[str, Any] | None,
|
|
||||||
meta: dict[str, Any] | None = None,
|
|
||||||
) -> CallToolResult:
|
|
||||||
self.calls.append((tool_name, arguments))
|
|
||||||
return CallToolResult(
|
|
||||||
content=[TextContent(type="text", text=f"boom:{tool_name}")],
|
|
||||||
isError=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_errored_mcp_result_is_flagged_failed_for_the_tui() -> None:
|
|
||||||
server = ErroringMCPServer("files_main", [_mcp_tool("list_files")])
|
|
||||||
|
|
||||||
tools: list[Tool] = await _register_server_tools(_config("files_main", ["list_files"]), server)
|
|
||||||
output = await tools[0].on_invoke_tool(None, "{}") # type: ignore[union-attr]
|
|
||||||
|
|
||||||
# The error text stays exactly what the agent gets today; a success:False tag
|
|
||||||
# rides alongside it purely so the TUI can tell the call apart from a success.
|
|
||||||
assert output == {"type": "text", "text": "boom:list_files", "success": False}
|
|
||||||
assert _tool_status_from_result(output) == "failed"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_successful_mcp_result_stays_completed_for_the_tui() -> None:
|
|
||||||
server = FakeMCPServer("files_main", [_mcp_tool("list_files")])
|
|
||||||
|
|
||||||
tools: list[Tool] = await _register_server_tools(_config("files_main", ["list_files"]), server)
|
|
||||||
output = await tools[0].on_invoke_tool(None, "{}") # type: ignore[union-attr]
|
|
||||||
|
|
||||||
# A non-error result is untouched and keeps rendering as done.
|
|
||||||
assert output == {"type": "text", "text": "routed:list_files"}
|
|
||||||
assert _tool_status_from_result(output) == "completed"
|
|
||||||
|
|
||||||
|
|
||||||
# --- server build branch -----------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_server_stdio_branch() -> None:
|
|
||||||
config = McpConnectionConfig(
|
|
||||||
name="local_fs",
|
|
||||||
transport="stdio",
|
|
||||||
command="my-server",
|
|
||||||
args=["--flag", "value"],
|
|
||||||
env={"TOKEN": "x"},
|
|
||||||
)
|
|
||||||
|
|
||||||
server = _build_server(config)
|
|
||||||
|
|
||||||
# Built, not connected: no subprocess is launched here.
|
|
||||||
assert isinstance(server, MCPServerStdio)
|
|
||||||
assert server.name == "local_fs"
|
|
||||||
assert server.params.command == "my-server"
|
|
||||||
assert server.params.args == ["--flag", "value"]
|
|
||||||
assert server.params.env == {"TOKEN": "x"}
|
|
||||||
|
|
||||||
|
|
||||||
def test_build_server_http_branch() -> None:
|
|
||||||
server = _build_server(_config("files_main", ["list_files"]))
|
|
||||||
|
|
||||||
assert isinstance(server, MCPServerStreamableHttp)
|
|
||||||
assert server.name == "files_main"
|
|
||||||
|
|
||||||
|
|
||||||
# --- loader ------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_loader_parses_stdio_and_http_entries(tmp_path: Path) -> None:
|
|
||||||
config_file = tmp_path / "mcp-servers.json"
|
|
||||||
config_file.write_text(
|
|
||||||
json.dumps(
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"name": "local_fs",
|
|
||||||
"transport": "stdio",
|
|
||||||
"command": "npx",
|
|
||||||
"args": ["-y", "server-filesystem"],
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "files_main",
|
|
||||||
"transport": "http",
|
|
||||||
"url": "https://mcp.example.com",
|
|
||||||
"auth": {"kind": "bearer", "token": "abc"},
|
|
||||||
"allowed_tools": ["list_files"],
|
|
||||||
},
|
|
||||||
]
|
|
||||||
),
|
|
||||||
encoding="utf-8",
|
|
||||||
)
|
|
||||||
|
|
||||||
configs = load_user_mcp_configs(config_file)
|
|
||||||
|
|
||||||
assert [c.name for c in configs] == ["local_fs", "files_main"]
|
|
||||||
assert configs[0].transport == "stdio"
|
|
||||||
assert configs[1].allowed_tools == ["list_files"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_loader_skips_bad_entry_but_keeps_good_ones(tmp_path: Path) -> None:
|
|
||||||
config_file = tmp_path / "mcp-servers.json"
|
|
||||||
config_file.write_text(
|
|
||||||
json.dumps(
|
|
||||||
[
|
|
||||||
{"name": "broken", "transport": "http"}, # missing url
|
|
||||||
{
|
|
||||||
"name": "local_fs",
|
|
||||||
"transport": "stdio",
|
|
||||||
"command": "npx",
|
|
||||||
},
|
|
||||||
]
|
|
||||||
),
|
|
||||||
encoding="utf-8",
|
|
||||||
)
|
|
||||||
|
|
||||||
configs = load_user_mcp_configs(config_file)
|
|
||||||
|
|
||||||
assert [c.name for c in configs] == ["local_fs"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_loader_returns_empty_when_file_absent(tmp_path: Path) -> None:
|
|
||||||
assert load_user_mcp_configs(tmp_path / "does-not-exist.json") == []
|
|
||||||
|
|
||||||
|
|
||||||
def test_loader_reads_env_var_override(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
config_file = tmp_path / "from-env.json"
|
|
||||||
config_file.write_text(
|
|
||||||
json.dumps([{"name": "local_fs", "transport": "stdio", "command": "npx"}]),
|
|
||||||
encoding="utf-8",
|
|
||||||
)
|
|
||||||
monkeypatch.setenv("STRIX_MCP_CONFIG", str(config_file))
|
|
||||||
|
|
||||||
configs = load_user_mcp_configs()
|
|
||||||
|
|
||||||
assert [c.name for c in configs] == ["local_fs"]
|
|
||||||
|
|
||||||
|
|
||||||
# --- connection notes --------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_connection_notes_are_carried_on_the_connection(
|
|
||||||
monkeypatch: pytest.MonkeyPatch,
|
|
||||||
) -> None:
|
|
||||||
server = FakeMCPServer("db", [_mcp_tool("query")])
|
|
||||||
monkeypatch.setattr(mcp_client, "_build_server", lambda _config: server)
|
|
||||||
config = McpConnectionConfig(
|
|
||||||
name="db",
|
|
||||||
url="https://mcp.example.com",
|
|
||||||
notes="Staging analytics DB; read-only.",
|
|
||||||
allowed_tools=["query"],
|
|
||||||
)
|
|
||||||
|
|
||||||
connections = await mcp_client.connect_mcp_servers([config])
|
|
||||||
|
|
||||||
# Notes ride on the connection (surfaced once), not stapled onto each tool.
|
|
||||||
assert connections[0].notes == "Staging analytics DB; read-only."
|
|
||||||
|
|
||||||
|
|
||||||
def test_connection_notes_block_lists_only_noted_connections() -> None:
|
|
||||||
connections = [
|
|
||||||
ConnectedMcpServer(
|
|
||||||
server=FakeMCPServer("db", []), name="db", tool_count=2, notes="staging, read-only"
|
|
||||||
),
|
|
||||||
ConnectedMcpServer(server=FakeMCPServer("fs", []), name="fs", tool_count=1, notes=None),
|
|
||||||
]
|
|
||||||
|
|
||||||
block = _mcp_connection_notes(connections)
|
|
||||||
|
|
||||||
assert block is not None
|
|
||||||
assert "db" in block
|
|
||||||
assert "staging, read-only" in block
|
|
||||||
# A connection without notes is not listed.
|
|
||||||
assert "fs" not in block
|
|
||||||
|
|
||||||
|
|
||||||
def test_connection_notes_block_is_none_without_notes() -> None:
|
|
||||||
connections = [
|
|
||||||
ConnectedMcpServer(server=FakeMCPServer("db", []), name="db", tool_count=1, notes=None)
|
|
||||||
]
|
|
||||||
|
|
||||||
assert _mcp_connection_notes(connections) is None
|
|
||||||
|
|
||||||
|
|
||||||
# --- cancellation cleanup ----------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_connect_cleans_up_when_cancelled_mid_connect(
|
|
||||||
monkeypatch: pytest.MonkeyPatch,
|
|
||||||
) -> None:
|
|
||||||
cleaned: list[str] = []
|
|
||||||
|
|
||||||
class _Tracking(FakeMCPServer):
|
|
||||||
def __init__(self, name: str, *, fail_connect: bool = False) -> None:
|
|
||||||
super().__init__(name, [_mcp_tool("t")])
|
|
||||||
self._fail_connect = fail_connect
|
|
||||||
|
|
||||||
async def connect(self) -> None:
|
|
||||||
if self._fail_connect:
|
|
||||||
raise asyncio.CancelledError
|
|
||||||
|
|
||||||
async def cleanup(self) -> None:
|
|
||||||
cleaned.append(self._name)
|
|
||||||
|
|
||||||
servers = {"good": _Tracking("good"), "bad": _Tracking("bad", fail_connect=True)}
|
|
||||||
monkeypatch.setattr(mcp_client, "_build_server", lambda config: servers[config.name])
|
|
||||||
|
|
||||||
configs = [
|
|
||||||
McpConnectionConfig(name="good", url="https://mcp.example.com", allowed_tools=["t"]),
|
|
||||||
McpConnectionConfig(name="bad", url="https://mcp.example.com", allowed_tools=["t"]),
|
|
||||||
]
|
|
||||||
|
|
||||||
with pytest.raises(asyncio.CancelledError):
|
|
||||||
await mcp_client.connect_mcp_servers(configs)
|
|
||||||
|
|
||||||
# The server being connected when cancelled, and the one already connected,
|
|
||||||
# are both cleaned up rather than orphaned.
|
|
||||||
assert cleaned == ["bad", "good"]
|
|
||||||
|
|
||||||
|
|
||||||
# --- duplicate names and run selection ---------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def _names_file(tmp_path: Path, *names: str) -> Path:
|
|
||||||
config_file = tmp_path / "mcp-servers.json"
|
|
||||||
config_file.write_text(
|
|
||||||
json.dumps([{"name": n, "transport": "stdio", "command": "npx"} for n in names]),
|
|
||||||
encoding="utf-8",
|
|
||||||
)
|
|
||||||
return config_file
|
|
||||||
|
|
||||||
|
|
||||||
def test_loader_drops_duplicate_named_connections(tmp_path: Path) -> None:
|
|
||||||
config_file = tmp_path / "mcp-servers.json"
|
|
||||||
config_file.write_text(
|
|
||||||
json.dumps(
|
|
||||||
[
|
|
||||||
{"name": "dup", "transport": "stdio", "command": "first"},
|
|
||||||
{"name": "dup", "transport": "stdio", "command": "second"},
|
|
||||||
{"name": "other", "transport": "stdio", "command": "npx"},
|
|
||||||
]
|
|
||||||
),
|
|
||||||
encoding="utf-8",
|
|
||||||
)
|
|
||||||
|
|
||||||
configs = load_user_mcp_configs(config_file)
|
|
||||||
|
|
||||||
# Duplicate name is dropped; the first entry wins.
|
|
||||||
assert [c.name for c in configs] == ["dup", "other"]
|
|
||||||
assert configs[0].command == "first"
|
|
||||||
|
|
||||||
|
|
||||||
def test_loader_include_selection_keeps_only_named(
|
|
||||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
||||||
) -> None:
|
|
||||||
config_file = _names_file(tmp_path, "a", "b", "c")
|
|
||||||
monkeypatch.setenv("STRIX_MCP_ONLY", "a,c")
|
|
||||||
|
|
||||||
configs = load_user_mcp_configs(config_file)
|
|
||||||
|
|
||||||
assert [c.name for c in configs] == ["a", "c"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_loader_exclude_selection_drops_named(
|
|
||||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
|
||||||
) -> None:
|
|
||||||
config_file = _names_file(tmp_path, "a", "b", "c")
|
|
||||||
monkeypatch.setenv("STRIX_MCP_EXCLUDE", "b")
|
|
||||||
|
|
||||||
configs = load_user_mcp_configs(config_file)
|
|
||||||
|
|
||||||
assert [c.name for c in configs] == ["a", "c"]
|
|
||||||
|
|
||||||
|
|
||||||
# --- reading a tool call back to the server it went out to -------------------
|
|
||||||
|
|
||||||
|
|
||||||
def test_resolve_mcp_tool_splits_against_the_run_connections() -> None:
|
|
||||||
assert resolve_mcp_tool("local_fs_read_file", ["github", "local_fs"]) == (
|
|
||||||
"local_fs",
|
|
||||||
"read_file",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_resolve_mcp_tool_prefers_the_longest_matching_connection() -> None:
|
|
||||||
# One connection's name being a prefix of another's must not misattribute.
|
|
||||||
assert resolve_mcp_tool("files_main_list", ["files", "files_main"]) == ("files_main", "list")
|
|
||||||
|
|
||||||
|
|
||||||
def test_resolve_mcp_tool_matches_a_connection_name_it_had_to_sanitize() -> None:
|
|
||||||
# "my server" reaches the model as "my_server_db_query".
|
|
||||||
tool_name = namespaced_tool_name("my server", "db.query")
|
|
||||||
|
|
||||||
assert resolve_mcp_tool(tool_name, ["my server"]) == ("my server", "db_query")
|
|
||||||
|
|
||||||
|
|
||||||
def test_resolve_mcp_tool_ignores_tools_that_are_not_a_connection_s() -> None:
|
|
||||||
assert resolve_mcp_tool("exec_command", ["local_fs"]) is None
|
|
||||||
# A name that merely starts like a connection is not one of its tools.
|
|
||||||
assert resolve_mcp_tool("local_fsx", ["local_fs"]) is None
|
|
||||||
|
|
||||||
|
|
||||||
def test_projected_tool_call_names_the_server_it_went_out_to() -> None:
|
|
||||||
view = TuiLiveView()
|
|
||||||
view.set_mcp_connections(["local_fs"])
|
|
||||||
|
|
||||||
view._record_tool_call_data(
|
|
||||||
"agent-1",
|
|
||||||
{"call_id": "c1", "tool_name": "local_fs_read_file", "args": {"path": "/etc/hosts"}},
|
|
||||||
)
|
|
||||||
view._record_tool_call_data(
|
|
||||||
"agent-1",
|
|
||||||
{"call_id": "c2", "tool_name": "exec_command", "args": {"cmd": "ls"}},
|
|
||||||
)
|
|
||||||
|
|
||||||
mcp_call, built_in = (event["data"] for event in view.events)
|
|
||||||
assert (mcp_call["mcp_connection"], mcp_call["mcp_tool"]) == ("local_fs", "read_file")
|
|
||||||
# A built-in call carries no connection, which is what keeps it rendering as one.
|
|
||||||
assert "mcp_connection" not in built_in
|
|
||||||
@@ -67,11 +67,6 @@ def test_recommended_models_are_matched_case_insensitively() -> None:
|
|||||||
"moonshot/kimi-k2.6",
|
"moonshot/kimi-k2.6",
|
||||||
"kimi-k2.7-code",
|
"kimi-k2.7-code",
|
||||||
"moonshot/kimi-k3",
|
"moonshot/kimi-k3",
|
||||||
"opencode/gpt-5.4",
|
|
||||||
"opencode/claude-sonnet-5",
|
|
||||||
"opencode-go/kimi-k3",
|
|
||||||
"opencode-go/deepseek-v4-flash",
|
|
||||||
"opencode-go/qwen3.8-max",
|
|
||||||
],
|
],
|
||||||
)
|
)
|
||||||
def test_frontier_model_families_are_accepted(model_name: str) -> None:
|
def test_frontier_model_families_are_accepted(model_name: str) -> None:
|
||||||
|
|||||||
@@ -1,212 +0,0 @@
|
|||||||
"""Tests for OpenCode (Zen/Go) subscription auth: prefix parsing and key store."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import TYPE_CHECKING
|
|
||||||
from unittest import mock
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
import requests
|
|
||||||
|
|
||||||
from strix.config import codex, opencode
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _tmp_store(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
|
|
||||||
path = tmp_path / "home" / ".strix" / "subscription-auth.json"
|
|
||||||
monkeypatch.setattr(codex, "AUTH_PATH", path)
|
|
||||||
return path
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize(
|
|
||||||
("model", "slug", "base_url", "protocol"),
|
|
||||||
[
|
|
||||||
(
|
|
||||||
"opencode/claude-sonnet-5",
|
|
||||||
"claude-sonnet-5",
|
|
||||||
opencode.ZEN_BASE_URL,
|
|
||||||
opencode.PROTOCOL_MESSAGES,
|
|
||||||
),
|
|
||||||
("opencode/gpt-5.4", "gpt-5.4", opencode.ZEN_BASE_URL, opencode.PROTOCOL_RESPONSES),
|
|
||||||
("opencode/grok-4.5", "grok-4.5", opencode.ZEN_BASE_URL, opencode.PROTOCOL_RESPONSES),
|
|
||||||
("OpenCode/Kimi-K3", "Kimi-K3", opencode.ZEN_BASE_URL, opencode.PROTOCOL_CHAT),
|
|
||||||
(
|
|
||||||
"OpenCode/Claude-Opus-5",
|
|
||||||
"Claude-Opus-5",
|
|
||||||
opencode.ZEN_BASE_URL,
|
|
||||||
opencode.PROTOCOL_MESSAGES,
|
|
||||||
),
|
|
||||||
("opencode-go/kimi-k3", "kimi-k3", opencode.GO_BASE_URL, opencode.PROTOCOL_CHAT),
|
|
||||||
(
|
|
||||||
"opencode-go/gpt-5.6-luna",
|
|
||||||
"gpt-5.6-luna",
|
|
||||||
opencode.GO_BASE_URL,
|
|
||||||
opencode.PROTOCOL_RESPONSES,
|
|
||||||
),
|
|
||||||
("opencode-go/grok-4.5", "grok-4.5", opencode.GO_BASE_URL, opencode.PROTOCOL_CHAT),
|
|
||||||
# Probed per family against both gateways; a wrong protocol 500s.
|
|
||||||
(
|
|
||||||
"opencode/muse-spark-1.2",
|
|
||||||
"muse-spark-1.2",
|
|
||||||
opencode.ZEN_BASE_URL,
|
|
||||||
opencode.PROTOCOL_RESPONSES,
|
|
||||||
),
|
|
||||||
(
|
|
||||||
"opencode/deepseek-v4-pro",
|
|
||||||
"deepseek-v4-pro",
|
|
||||||
opencode.ZEN_BASE_URL,
|
|
||||||
opencode.PROTOCOL_CHAT,
|
|
||||||
),
|
|
||||||
("opencode/minimax-m3", "minimax-m3", opencode.ZEN_BASE_URL, opencode.PROTOCOL_CHAT),
|
|
||||||
("opencode/qwen3.6-plus", "qwen3.6-plus", opencode.ZEN_BASE_URL, opencode.PROTOCOL_CHAT),
|
|
||||||
("opencode/glm-5.2", "glm-5.2", opencode.ZEN_BASE_URL, opencode.PROTOCOL_CHAT),
|
|
||||||
("opencode/grok-4.6", "grok-4.6", opencode.ZEN_BASE_URL, opencode.PROTOCOL_RESPONSES),
|
|
||||||
(
|
|
||||||
"opencode/gpt-5.6-luna",
|
|
||||||
"gpt-5.6-luna",
|
|
||||||
opencode.ZEN_BASE_URL,
|
|
||||||
opencode.PROTOCOL_RESPONSES,
|
|
||||||
),
|
|
||||||
(
|
|
||||||
"opencode/claude-opus-5",
|
|
||||||
"claude-opus-5",
|
|
||||||
opencode.ZEN_BASE_URL,
|
|
||||||
opencode.PROTOCOL_MESSAGES,
|
|
||||||
),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
def test_subscription_model_parses_prefixes(
|
|
||||||
model: str, slug: str, base_url: str, protocol: str
|
|
||||||
) -> None:
|
|
||||||
parsed = opencode.subscription_model(model)
|
|
||||||
assert parsed is not None
|
|
||||||
assert parsed.slug == slug
|
|
||||||
assert parsed.base_url == base_url
|
|
||||||
assert parsed.protocol == protocol
|
|
||||||
assert parsed.uses_responses is (protocol == opencode.PROTOCOL_RESPONSES)
|
|
||||||
|
|
||||||
|
|
||||||
def test_claude_route_targets_the_anthropic_endpoint() -> None:
|
|
||||||
parsed = opencode.subscription_model("opencode/claude-sonnet-5")
|
|
||||||
assert parsed is not None
|
|
||||||
assert parsed.messages_url == "https://opencode.ai/zen/v1/messages"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize(
|
|
||||||
("model", "plan", "label", "metered"),
|
|
||||||
[
|
|
||||||
("opencode/claude-sonnet-5", opencode.PLAN_ZEN, "OpenCode Zen", True),
|
|
||||||
("opencode/kimi-k3", opencode.PLAN_ZEN, "OpenCode Zen", True),
|
|
||||||
("opencode-go/kimi-k3", opencode.PLAN_GO, "OpenCode Go", False),
|
|
||||||
("OpenCode-Go/GPT-5.6-Luna", opencode.PLAN_GO, "OpenCode Go", False),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
def test_plan_is_labelled_and_metered_per_prefix(
|
|
||||||
model: str, plan: str, label: str, metered: bool
|
|
||||||
) -> None:
|
|
||||||
parsed = opencode.subscription_model(model)
|
|
||||||
assert parsed is not None
|
|
||||||
assert parsed.plan == plan
|
|
||||||
assert parsed.label == label
|
|
||||||
# Zen bills prepaid credits per request; Go is a flat monthly plan.
|
|
||||||
assert parsed.metered is metered
|
|
||||||
assert opencode.subscription_plan(model) == plan
|
|
||||||
|
|
||||||
|
|
||||||
def test_subscription_plan_is_none_off_opencode() -> None:
|
|
||||||
assert opencode.subscription_plan("chatgpt/gpt-5.4") is None
|
|
||||||
assert opencode.subscription_plan("anthropic/claude-sonnet-5") is None
|
|
||||||
assert opencode.subscription_plan(None) is None
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize(
|
|
||||||
"model",
|
|
||||||
["openai/gpt-5.4", "chatgpt/gpt-5.4", "opencode/", "opencode-go/", "opencode", "", None],
|
|
||||||
)
|
|
||||||
def test_subscription_model_rejects_non_opencode(model: str | None) -> None:
|
|
||||||
assert opencode.subscription_model(model) is None
|
|
||||||
|
|
||||||
|
|
||||||
def test_store_roundtrip_and_logout() -> None:
|
|
||||||
assert opencode.read_record() is None
|
|
||||||
assert opencode.is_authenticated() is False
|
|
||||||
|
|
||||||
opencode.save_api_key("sk-oc-test")
|
|
||||||
record = opencode.read_record()
|
|
||||||
assert record is not None
|
|
||||||
assert record["key"] == "sk-oc-test"
|
|
||||||
assert opencode.is_authenticated() is True
|
|
||||||
assert opencode.get_api_key() == "sk-oc-test"
|
|
||||||
|
|
||||||
opencode.logout()
|
|
||||||
assert opencode.read_record() is None
|
|
||||||
opencode.logout() # no-op when already gone
|
|
||||||
|
|
||||||
|
|
||||||
def test_store_coexists_with_chatgpt_record() -> None:
|
|
||||||
codex.save_record({"type": "oauth", "access": "a", "refresh": "r", "account_id": "acct"})
|
|
||||||
opencode.save_api_key("sk-oc-test")
|
|
||||||
|
|
||||||
assert codex.read_record() is not None
|
|
||||||
assert opencode.get_api_key() == "sk-oc-test"
|
|
||||||
|
|
||||||
opencode.logout()
|
|
||||||
assert codex.read_record() is not None
|
|
||||||
assert opencode.read_record() is None
|
|
||||||
|
|
||||||
|
|
||||||
def test_get_api_key_raises_when_not_signed_in() -> None:
|
|
||||||
with pytest.raises(opencode.OpencodeAuthError) as exc:
|
|
||||||
opencode.get_api_key()
|
|
||||||
assert exc.value.code == "not_authenticated"
|
|
||||||
|
|
||||||
|
|
||||||
def test_auth_mode_covers_both_subscriptions() -> None:
|
|
||||||
assert opencode.auth_mode("opencode/claude-sonnet-5") == "subscription"
|
|
||||||
assert opencode.auth_mode("opencode-go/kimi-k3") == "subscription"
|
|
||||||
assert opencode.auth_mode("chatgpt/gpt-5.4") == "subscription"
|
|
||||||
assert opencode.auth_mode("openai/gpt-5.4") == "api_key"
|
|
||||||
assert opencode.auth_mode(None) == "api_key"
|
|
||||||
|
|
||||||
|
|
||||||
def test_subscription_provider() -> None:
|
|
||||||
assert opencode.subscription_provider("opencode/claude-sonnet-5") == "opencode"
|
|
||||||
assert opencode.subscription_provider("opencode-go/kimi-k3") == "opencode"
|
|
||||||
assert opencode.subscription_provider("chatgpt/gpt-5.4") == "chatgpt"
|
|
||||||
assert opencode.subscription_provider("openai/gpt-5.4") is None
|
|
||||||
assert opencode.subscription_provider(None) is None
|
|
||||||
|
|
||||||
|
|
||||||
def _response(status_code: int, text: str = "") -> mock.MagicMock:
|
|
||||||
response = mock.MagicMock()
|
|
||||||
response.status_code = status_code
|
|
||||||
response.text = text
|
|
||||||
return response
|
|
||||||
|
|
||||||
|
|
||||||
def test_validate_api_key_accepts_ok() -> None:
|
|
||||||
with mock.patch.object(requests, "get", return_value=_response(200)) as get:
|
|
||||||
opencode.validate_api_key("sk-oc-test")
|
|
||||||
assert get.call_args.kwargs["headers"]["Authorization"] == "Bearer sk-oc-test"
|
|
||||||
|
|
||||||
|
|
||||||
def test_validate_api_key_rejects_unauthorized() -> None:
|
|
||||||
with (
|
|
||||||
mock.patch.object(requests, "get", return_value=_response(401)),
|
|
||||||
pytest.raises(opencode.OpencodeAuthError) as exc,
|
|
||||||
):
|
|
||||||
opencode.validate_api_key("bad-key")
|
|
||||||
assert exc.value.code == "invalid_key"
|
|
||||||
|
|
||||||
|
|
||||||
def test_validate_api_key_maps_network_errors() -> None:
|
|
||||||
with (
|
|
||||||
mock.patch.object(requests, "get", side_effect=requests.ConnectionError("boom")),
|
|
||||||
pytest.raises(opencode.OpencodeAuthError) as exc,
|
|
||||||
):
|
|
||||||
opencode.validate_api_key("sk-oc-test")
|
|
||||||
assert exc.value.code == "unavailable"
|
|
||||||
@@ -13,7 +13,6 @@ from typing import TYPE_CHECKING, Any, cast
|
|||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from strix.runtime.caido_handle import CaidoBootstrapHandle
|
|
||||||
from strix.tools.proxy import caido_api, tools
|
from strix.tools.proxy import caido_api, tools
|
||||||
|
|
||||||
|
|
||||||
@@ -199,31 +198,12 @@ class _Ctx:
|
|||||||
self.context = context
|
self.context = context
|
||||||
|
|
||||||
|
|
||||||
async def test_ctx_client_returns_client_when_present() -> None:
|
def test_ctx_client_returns_client_when_present() -> None:
|
||||||
client = _FakeClient("host")
|
client = _FakeClient("host")
|
||||||
got = await tools._ctx_client(cast("Any", _Ctx({"caido_client": client})))
|
got = tools._ctx_client(cast("Any", _Ctx({"caido_client": client})))
|
||||||
assert got is client
|
assert got is client
|
||||||
|
|
||||||
|
|
||||||
async def test_ctx_client_returns_none_without_client() -> None:
|
def test_ctx_client_returns_none_without_client() -> None:
|
||||||
assert await tools._ctx_client(cast("Any", _Ctx({}))) is None
|
assert tools._ctx_client(cast("Any", _Ctx({}))) is None
|
||||||
assert await tools._ctx_client(cast("Any", _Ctx(None))) is None
|
assert tools._ctx_client(cast("Any", _Ctx(None))) is None
|
||||||
|
|
||||||
|
|
||||||
async def test_ctx_client_resolves_bootstrap_handle() -> None:
|
|
||||||
client = _FakeClient("host")
|
|
||||||
|
|
||||||
async def _bootstrap() -> Any:
|
|
||||||
return client
|
|
||||||
|
|
||||||
handle = CaidoBootstrapHandle(asyncio.ensure_future(_bootstrap()))
|
|
||||||
got = await tools._ctx_client(cast("Any", _Ctx({"caido_client": handle})))
|
|
||||||
assert got is client
|
|
||||||
|
|
||||||
|
|
||||||
async def test_ctx_client_degrades_when_bootstrap_failed() -> None:
|
|
||||||
async def _bootstrap() -> Any:
|
|
||||||
raise RuntimeError("caido never came up")
|
|
||||||
|
|
||||||
handle = CaidoBootstrapHandle(asyncio.ensure_future(_bootstrap()))
|
|
||||||
assert await tools._ctx_client(cast("Any", _Ctx({"caido_client": handle}))) is None
|
|
||||||
|
|||||||
@@ -1,264 +0,0 @@
|
|||||||
"""Tests for the coverage artifact assembled in strix.report.coverage."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
from typing import TYPE_CHECKING, Any
|
|
||||||
|
|
||||||
from strix.report.coverage import (
|
|
||||||
_SKILL_PHRASINGS,
|
|
||||||
build_coverage_document,
|
|
||||||
read_agent_graph,
|
|
||||||
write_coverage,
|
|
||||||
)
|
|
||||||
from strix.skills import get_available_skills
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
def _entry(**overrides: Any) -> dict[str, Any]:
|
|
||||||
base: dict[str, Any] = {
|
|
||||||
"surface": "POST /api/orders/{id}",
|
|
||||||
"risk_area": "object-level authorization",
|
|
||||||
"outcome": "no_issue_found",
|
|
||||||
"evidence": "Two tenants tested; both received 403.",
|
|
||||||
"agent_id": "agent-1",
|
|
||||||
"agent_name": "authz-tester",
|
|
||||||
"created_at": "2026-07-02 10:00:00 UTC",
|
|
||||||
}
|
|
||||||
base.update(overrides)
|
|
||||||
return base
|
|
||||||
|
|
||||||
|
|
||||||
def _graph(**overrides: Any) -> dict[str, Any]:
|
|
||||||
base: dict[str, Any] = {
|
|
||||||
"statuses": {"agent-1": "completed"},
|
|
||||||
"names": {"agent-1": "authz-tester"},
|
|
||||||
"metadata": {"agent-1": {"skills": ["idor"], "task": "authz review"}},
|
|
||||||
}
|
|
||||||
base.update(overrides)
|
|
||||||
return base
|
|
||||||
|
|
||||||
|
|
||||||
def _document(**overrides: Any) -> dict[str, Any]:
|
|
||||||
kwargs: dict[str, Any] = {
|
|
||||||
"run_record": {"run_id": "r1", "run_name": "run-1", "status": "completed"},
|
|
||||||
"entries": [_entry()],
|
|
||||||
"agent_graph": _graph(),
|
|
||||||
"vulnerability_reports": [],
|
|
||||||
}
|
|
||||||
kwargs.update(overrides)
|
|
||||||
return build_coverage_document(**kwargs)
|
|
||||||
|
|
||||||
|
|
||||||
def test_document_reports_surfaces_and_outcomes() -> None:
|
|
||||||
doc = _document()
|
|
||||||
|
|
||||||
assert doc["summary"]["surfaces_reviewed"] == 1
|
|
||||||
assert doc["summary"]["outcomes"] == {"no_issue_found": 1}
|
|
||||||
assert doc["entries"][0]["outcome_label"] == "No issue identified"
|
|
||||||
assert doc["entries"][0]["recorded_by"] == "authz-tester"
|
|
||||||
|
|
||||||
|
|
||||||
def test_ledger_entries_are_labelled_as_agent_reported() -> None:
|
|
||||||
"""A reader has to be able to tell a self-report from an observation."""
|
|
||||||
doc = _document()
|
|
||||||
|
|
||||||
assert doc["entries"][0]["source"] == "agent_reported"
|
|
||||||
assert doc["machine_observed"]["source"] == "runtime"
|
|
||||||
assert doc["machine_observed"]["skills_exercised"] == ["idor"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_assigned_risk_skill_without_coverage_becomes_a_gap() -> None:
|
|
||||||
"""An agent carrying the sql_injection skill that records nothing about it
|
|
||||||
leaves the class unexamined, not clean."""
|
|
||||||
doc = _document(
|
|
||||||
agent_graph=_graph(
|
|
||||||
metadata={"agent-1": {"skills": ["idor", "sql_injection"], "task": "review"}}
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
gaps = [gap for gap in doc["gaps"] if gap["kind"] == "unrecorded_risk_class"]
|
|
||||||
assert [gap["risk_area"] for gap in gaps] == ["sql injection"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_recorded_risk_class_is_not_reported_as_a_gap() -> None:
|
|
||||||
doc = _document(
|
|
||||||
entries=[_entry(risk_area="SQL injection", surface="GET /search?q=")],
|
|
||||||
agent_graph=_graph(metadata={"agent-1": {"skills": ["sql_injection"]}}),
|
|
||||||
)
|
|
||||||
|
|
||||||
assert not [gap for gap in doc["gaps"] if gap["kind"] == "unrecorded_risk_class"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_synonym_phrasing_counts_as_recorded_coverage() -> None:
|
|
||||||
"""The ledger says "object-level authorization"; the skill is called idor."""
|
|
||||||
doc = _document(agent_graph=_graph(metadata={"agent-1": {"skills": ["idor"]}}))
|
|
||||||
|
|
||||||
assert not [gap for gap in doc["gaps"] if gap["kind"] == "unrecorded_risk_class"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_non_risk_skills_carry_no_coverage_obligation() -> None:
|
|
||||||
"""Tooling skills describe how an agent works, not what it hunts."""
|
|
||||||
doc = _document(agent_graph=_graph(metadata={"agent-1": {"skills": ["idor", "caido"]}}))
|
|
||||||
|
|
||||||
assert not [gap for gap in doc["gaps"] if gap.get("risk_area") == "caido"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_agent_that_recorded_nothing_is_a_gap() -> None:
|
|
||||||
doc = _document(
|
|
||||||
agent_graph=_graph(
|
|
||||||
statuses={"agent-1": "completed", "agent-2": "completed"},
|
|
||||||
names={"agent-1": "authz-tester", "agent-2": "recon"},
|
|
||||||
metadata={},
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
silent = [gap for gap in doc["gaps"] if gap["kind"] == "agent_recorded_no_coverage"]
|
|
||||||
assert [gap["agent_name"] for gap in silent] == ["recon"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_needs_follow_up_is_carried_as_an_open_gap() -> None:
|
|
||||||
doc = _document(
|
|
||||||
entries=[_entry(outcome="needs_follow_up", evidence="Auth wall blocked testing.")]
|
|
||||||
)
|
|
||||||
|
|
||||||
assert doc["gaps"][0]["kind"] == "needs_follow_up"
|
|
||||||
assert doc["gaps"][0]["detail"] == "Auth wall blocked testing."
|
|
||||||
|
|
||||||
|
|
||||||
def test_completed_run_with_finished_agents_is_complete() -> None:
|
|
||||||
doc = _document(exit_reason="finished_by_tool")
|
|
||||||
|
|
||||||
assert doc["completeness"]["complete"] is True
|
|
||||||
assert doc["completeness"]["caveats"] == []
|
|
||||||
|
|
||||||
|
|
||||||
def test_budget_exhausted_run_is_not_a_complete_record() -> None:
|
|
||||||
"""A truncated scan must not read like a clean one."""
|
|
||||||
doc = _document(exit_reason="budget_exhausted")
|
|
||||||
|
|
||||||
assert doc["completeness"]["complete"] is False
|
|
||||||
assert "budget_exhausted" in doc["completeness"]["caveats"][0]
|
|
||||||
|
|
||||||
|
|
||||||
def test_unfinished_agent_makes_the_record_partial() -> None:
|
|
||||||
doc = _document(
|
|
||||||
agent_graph=_graph(statuses={"agent-1": "crashed"}),
|
|
||||||
exit_reason="finished_by_tool",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert doc["completeness"]["complete"] is False
|
|
||||||
assert "authz-tester" in doc["completeness"]["caveats"][0]
|
|
||||||
|
|
||||||
|
|
||||||
def test_failed_run_status_makes_the_record_partial() -> None:
|
|
||||||
doc = _document(
|
|
||||||
run_record={"run_id": "r1", "status": "failed"},
|
|
||||||
exit_reason="finished_by_tool",
|
|
||||||
)
|
|
||||||
|
|
||||||
assert doc["completeness"]["complete"] is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_write_coverage_emits_a_top_level_artifact(tmp_path: Path) -> None:
|
|
||||||
path = write_coverage(tmp_path, _document())
|
|
||||||
|
|
||||||
assert path == tmp_path / "coverage.json"
|
|
||||||
assert json.loads(path.read_text(encoding="utf-8"))["schema_version"] == 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_read_agent_graph_tolerates_a_missing_or_corrupt_snapshot(tmp_path: Path) -> None:
|
|
||||||
assert read_agent_graph(tmp_path) == {}
|
|
||||||
|
|
||||||
(tmp_path / "agents.json").write_text("{not json", encoding="utf-8")
|
|
||||||
assert read_agent_graph(tmp_path) == {}
|
|
||||||
|
|
||||||
|
|
||||||
def test_read_agent_graph_loads_a_snapshot(tmp_path: Path) -> None:
|
|
||||||
(tmp_path / "agents.json").write_text(json.dumps(_graph()), encoding="utf-8")
|
|
||||||
|
|
||||||
assert read_agent_graph(tmp_path)["names"] == {"agent-1": "authz-tester"}
|
|
||||||
|
|
||||||
|
|
||||||
def test_multi_token_skill_matches_how_a_pentester_writes_it() -> None:
|
|
||||||
"""An agent carrying path_traversal_lfi_rfi records "Path Traversal".
|
|
||||||
|
|
||||||
Requiring the skill's filename verbatim published a false gap for a class
|
|
||||||
that had been tested and even had a finding filed against it.
|
|
||||||
"""
|
|
||||||
doc = _document(
|
|
||||||
entries=[_entry(risk_area="Path Traversal / Directory Traversal", surface="/download")],
|
|
||||||
agent_graph=_graph(metadata={"agent-1": {"skills": ["path_traversal_lfi_rfi"]}}),
|
|
||||||
)
|
|
||||||
|
|
||||||
assert not [gap for gap in doc["gaps"] if gap["kind"] == "unrecorded_risk_class"]
|
|
||||||
|
|
||||||
|
|
||||||
def _vulnerability_skill_names() -> set[str]:
|
|
||||||
return {skill["name"] for skill in get_available_skills()["vulnerabilities"]}
|
|
||||||
|
|
||||||
|
|
||||||
def test_every_vulnerability_skill_declares_its_phrasings() -> None:
|
|
||||||
"""A new skill without phrasings would be matched by its filename alone,
|
|
||||||
which is how the false gap above got published."""
|
|
||||||
missing = _vulnerability_skill_names() - set(_SKILL_PHRASINGS)
|
|
||||||
|
|
||||||
assert not missing, f"add ledger phrasings for: {sorted(missing)}"
|
|
||||||
|
|
||||||
|
|
||||||
def test_declared_phrasings_name_real_skills() -> None:
|
|
||||||
stale = set(_SKILL_PHRASINGS) - _vulnerability_skill_names()
|
|
||||||
|
|
||||||
assert not stale, f"phrasings for skills that no longer exist: {sorted(stale)}"
|
|
||||||
|
|
||||||
|
|
||||||
def _delegating_graph(**overrides: Any) -> dict[str, Any]:
|
|
||||||
base: dict[str, Any] = {
|
|
||||||
"statuses": {"root": "completed", "agent-1": "completed"},
|
|
||||||
"names": {"root": "Root Agent", "agent-1": "authz-tester"},
|
|
||||||
"parent_of": {"agent-1": "root"},
|
|
||||||
"metadata": {"agent-1": {"skills": ["idor"]}},
|
|
||||||
}
|
|
||||||
base.update(overrides)
|
|
||||||
return base
|
|
||||||
|
|
||||||
|
|
||||||
def test_delegating_root_agent_is_not_a_coverage_gap() -> None:
|
|
||||||
"""The root delegates and reconciles; it is not a tester that went quiet.
|
|
||||||
Flagging it would put the same false line in every clean report."""
|
|
||||||
doc = _document(agent_graph=_delegating_graph())
|
|
||||||
|
|
||||||
silent = [gap for gap in doc["gaps"] if gap["kind"] == "agent_recorded_no_coverage"]
|
|
||||||
assert silent == []
|
|
||||||
|
|
||||||
|
|
||||||
def test_a_subagent_that_records_nothing_is_still_a_gap() -> None:
|
|
||||||
doc = _document(
|
|
||||||
agent_graph=_delegating_graph(
|
|
||||||
statuses={"root": "completed", "agent-1": "completed", "agent-2": "completed"},
|
|
||||||
names={"root": "Root Agent", "agent-1": "authz-tester", "agent-2": "recon"},
|
|
||||||
parent_of={"agent-1": "root", "agent-2": "root"},
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
silent = [gap for gap in doc["gaps"] if gap["kind"] == "agent_recorded_no_coverage"]
|
|
||||||
assert [gap["agent_name"] for gap in silent] == ["recon"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_a_root_that_worked_alone_is_held_to_the_rule() -> None:
|
|
||||||
"""With no subagents there is nobody else the testing could have come
|
|
||||||
from, so silence is a real gap."""
|
|
||||||
doc = _document(
|
|
||||||
entries=[],
|
|
||||||
agent_graph={
|
|
||||||
"statuses": {"root": "completed"},
|
|
||||||
"names": {"root": "Root Agent"},
|
|
||||||
"parent_of": {},
|
|
||||||
},
|
|
||||||
)
|
|
||||||
|
|
||||||
silent = [gap for gap in doc["gaps"] if gap["kind"] == "agent_recorded_no_coverage"]
|
|
||||||
assert [gap["agent_name"] for gap in silent] == ["Root Agent"]
|
|
||||||
@@ -179,30 +179,3 @@ def test_write_executive_report_writes_markdown(tmp_path: Path) -> None:
|
|||||||
content = (tmp_path / "penetration_test_report.md").read_text(encoding="utf-8")
|
content = (tmp_path / "penetration_test_report.md").read_text(encoding="utf-8")
|
||||||
assert "# Security Penetration Test Report" in content
|
assert "# Security Penetration Test Report" in content
|
||||||
assert "Scan complete. No critical issues." in content
|
assert "Scan complete. No critical issues." in content
|
||||||
|
|
||||||
|
|
||||||
def test_render_vulnerability_md_surfaces_calibration_metadata() -> None:
|
|
||||||
"""Confidence, the case against the finding, and retest status are part of
|
|
||||||
the deliverable — storing them without rendering hides the reasoning."""
|
|
||||||
md = render_vulnerability_md(
|
|
||||||
{
|
|
||||||
"id": "vuln-0009",
|
|
||||||
"title": "SSRF in URL preview",
|
|
||||||
"severity": "high",
|
|
||||||
"timestamp": "2026-07-02 10:00:00 UTC",
|
|
||||||
"description": "Fetches user-supplied URLs.",
|
|
||||||
"confidence": "medium",
|
|
||||||
"counterevidence": "Egress appears filtered at the network layer.",
|
|
||||||
"confidence_rationale": "Reproduced once out of three attempts.",
|
|
||||||
"severity_change_conditions": "Critical if egress filtering is removed.",
|
|
||||||
"remediation_steps": "Allowlist destinations.",
|
|
||||||
"fix_verification": "Not retested.",
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
assert "**Confidence:** Medium" in md
|
|
||||||
assert "## Counterevidence" in md
|
|
||||||
assert "Egress appears filtered at the network layer." in md
|
|
||||||
assert "## Confidence Rationale" in md
|
|
||||||
assert "## What Would Change This Severity" in md
|
|
||||||
assert "## Fix Verification" in md
|
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from typing import TYPE_CHECKING, Any
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
@@ -75,9 +75,6 @@ async def test_create_report_persists_new_fields(report_state: ReportState) -> N
|
|||||||
remediation_steps="Context-encode output.",
|
remediation_steps="Context-encode output.",
|
||||||
evidence="Response echoes the payload verbatim.",
|
evidence="Response echoes the payload verbatim.",
|
||||||
assumptions="Assumes a victim opens a crafted link.",
|
assumptions="Assumes a victim opens a crafted link.",
|
||||||
counterevidence="No output encoding or CSP observed on this response.",
|
|
||||||
confidence="HIGH",
|
|
||||||
severity_change_conditions="A strict CSP would lower the severity.",
|
|
||||||
fix_effort="LOW",
|
fix_effort="LOW",
|
||||||
cvss_breakdown=_CVSS,
|
cvss_breakdown=_CVSS,
|
||||||
endpoint="/search",
|
endpoint="/search",
|
||||||
@@ -94,9 +91,6 @@ async def test_create_report_persists_new_fields(report_state: ReportState) -> N
|
|||||||
assert report["fix_effort"] == "low"
|
assert report["fix_effort"] == "low"
|
||||||
assert report["fix_pr_body"] == "## Fix\nEncode output."
|
assert report["fix_pr_body"] == "## Fix\nEncode output."
|
||||||
assert report["finding_class"] == "dynamic"
|
assert report["finding_class"] == "dynamic"
|
||||||
assert report["counterevidence"] == "No output encoding or CSP observed on this response."
|
|
||||||
assert report["confidence"] == "high"
|
|
||||||
assert report["severity_change_conditions"] == "A strict CSP would lower the severity."
|
|
||||||
|
|
||||||
|
|
||||||
async def test_create_report_requires_evidence_and_assumptions(
|
async def test_create_report_requires_evidence_and_assumptions(
|
||||||
@@ -113,9 +107,6 @@ async def test_create_report_requires_evidence_and_assumptions(
|
|||||||
remediation_steps="r",
|
remediation_steps="r",
|
||||||
evidence=" ",
|
evidence=" ",
|
||||||
assumptions="",
|
assumptions="",
|
||||||
counterevidence="none found",
|
|
||||||
confidence="high",
|
|
||||||
severity_change_conditions="n/a",
|
|
||||||
fix_effort="low",
|
fix_effort="low",
|
||||||
cvss_breakdown=_CVSS,
|
cvss_breakdown=_CVSS,
|
||||||
endpoint=None,
|
endpoint=None,
|
||||||
@@ -143,9 +134,6 @@ async def test_create_report_rejects_invalid_fix_effort(report_state: ReportStat
|
|||||||
remediation_steps="r",
|
remediation_steps="r",
|
||||||
evidence="e",
|
evidence="e",
|
||||||
assumptions="a",
|
assumptions="a",
|
||||||
counterevidence="none found",
|
|
||||||
confidence="high",
|
|
||||||
severity_change_conditions="n/a",
|
|
||||||
fix_effort="enormous",
|
fix_effort="enormous",
|
||||||
cvss_breakdown=_CVSS,
|
cvss_breakdown=_CVSS,
|
||||||
endpoint=None,
|
endpoint=None,
|
||||||
@@ -159,80 +147,6 @@ async def test_create_report_rejects_invalid_fix_effort(report_state: ReportStat
|
|||||||
assert not report_state.vulnerability_reports
|
assert not report_state.vulnerability_reports
|
||||||
|
|
||||||
|
|
||||||
async def _create_with(report_state: ReportState, **overrides: object) -> dict[str, Any]:
|
|
||||||
kwargs: dict[str, object] = {
|
|
||||||
"title": "X",
|
|
||||||
"description": "d",
|
|
||||||
"impact": "i",
|
|
||||||
"target": "t",
|
|
||||||
"technical_analysis": "ta",
|
|
||||||
"poc_description": "p",
|
|
||||||
"poc_script_code": "c",
|
|
||||||
"remediation_steps": "r",
|
|
||||||
"evidence": "e",
|
|
||||||
"assumptions": "a",
|
|
||||||
"counterevidence": "No guard found on this path.",
|
|
||||||
"confidence": "high",
|
|
||||||
"severity_change_conditions": "Proof of internet exposure would raise it.",
|
|
||||||
"fix_effort": "low",
|
|
||||||
"cvss_breakdown": _CVSS,
|
|
||||||
"endpoint": None,
|
|
||||||
"method": None,
|
|
||||||
"cve": None,
|
|
||||||
"cwe": None,
|
|
||||||
"code_locations": None,
|
|
||||||
}
|
|
||||||
kwargs.update(overrides)
|
|
||||||
assert report_state is not None
|
|
||||||
return await _do_create(**kwargs) # type: ignore[arg-type]
|
|
||||||
|
|
||||||
|
|
||||||
async def test_create_report_requires_counterevidence(report_state: ReportState) -> None:
|
|
||||||
result = await _create_with(report_state, counterevidence=" ")
|
|
||||||
assert result["success"] is False
|
|
||||||
assert any("Counterevidence" in e for e in result["errors"])
|
|
||||||
assert not report_state.vulnerability_reports
|
|
||||||
|
|
||||||
|
|
||||||
async def test_create_report_requires_severity_change_conditions(
|
|
||||||
report_state: ReportState,
|
|
||||||
) -> None:
|
|
||||||
result = await _create_with(report_state, severity_change_conditions="")
|
|
||||||
assert result["success"] is False
|
|
||||||
assert any("severity_change_conditions" in e for e in result["errors"])
|
|
||||||
assert not report_state.vulnerability_reports
|
|
||||||
|
|
||||||
|
|
||||||
async def test_create_report_rejects_invalid_confidence(report_state: ReportState) -> None:
|
|
||||||
result = await _create_with(report_state, confidence="pretty sure")
|
|
||||||
assert result["success"] is False
|
|
||||||
assert any("confidence" in e for e in result["errors"])
|
|
||||||
assert not report_state.vulnerability_reports
|
|
||||||
|
|
||||||
|
|
||||||
async def test_create_report_requires_rationale_when_confidence_not_high(
|
|
||||||
report_state: ReportState,
|
|
||||||
) -> None:
|
|
||||||
result = await _create_with(report_state, confidence="medium")
|
|
||||||
assert result["success"] is False
|
|
||||||
assert any("confidence_rationale" in e for e in result["errors"])
|
|
||||||
assert not report_state.vulnerability_reports
|
|
||||||
|
|
||||||
|
|
||||||
async def test_create_report_accepts_medium_confidence_with_rationale(
|
|
||||||
report_state: ReportState,
|
|
||||||
) -> None:
|
|
||||||
result = await _create_with(
|
|
||||||
report_state,
|
|
||||||
confidence="medium",
|
|
||||||
confidence_rationale="Static-only trace; could not stand up the service.",
|
|
||||||
)
|
|
||||||
assert result["success"] is True
|
|
||||||
report = report_state.vulnerability_reports[0]
|
|
||||||
assert report["confidence"] == "medium"
|
|
||||||
assert report["confidence_rationale"] == "Static-only trace; could not stand up the service."
|
|
||||||
|
|
||||||
|
|
||||||
async def test_dependency_report_sets_class_and_metadata(report_state: ReportState) -> None:
|
async def test_dependency_report_sets_class_and_metadata(report_state: ReportState) -> None:
|
||||||
result = await _do_create_dependency(
|
result = await _do_create_dependency(
|
||||||
title="CVE-2021-23337 in lodash 4.17.20",
|
title="CVE-2021-23337 in lodash 4.17.20",
|
||||||
@@ -1023,56 +937,6 @@ def test_vuln_tool_exposes_new_params() -> None:
|
|||||||
assert "advisory_cvss" in dep_required
|
assert "advisory_cvss" in dep_required
|
||||||
|
|
||||||
|
|
||||||
_FIX_LOCATION = {
|
|
||||||
"file": "app/views.py",
|
|
||||||
"start_line": 10,
|
|
||||||
"end_line": 12,
|
|
||||||
"fix_before": 'query = f"SELECT * FROM t WHERE id={uid}"',
|
|
||||||
"fix_after": 'query = "SELECT * FROM t WHERE id=%s"',
|
|
||||||
}
|
|
||||||
|
|
||||||
_INFO_LOCATION = {
|
|
||||||
"file": "app/views.py",
|
|
||||||
"start_line": 10,
|
|
||||||
"end_line": 12,
|
|
||||||
"snippet": 'query = f"SELECT * FROM t WHERE id={uid}"',
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
async def test_fix_after_requires_verification(report_state: ReportState) -> None:
|
|
||||||
result = await _create_with(report_state, code_locations=[_FIX_LOCATION])
|
|
||||||
assert result["success"] is False
|
|
||||||
assert any("fix_verification" in e for e in result["errors"])
|
|
||||||
assert not report_state.vulnerability_reports
|
|
||||||
|
|
||||||
|
|
||||||
async def test_fix_after_with_verification_persists(report_state: ReportState) -> None:
|
|
||||||
verification = (
|
|
||||||
"Re-ran the PoC against the patched handler: the payload is now bound as a "
|
|
||||||
"parameter and returns no extra rows. Checked the two sibling call sites of "
|
|
||||||
"the same helper and the admin export path; both already parameterized. "
|
|
||||||
"Legitimate numeric ids still resolve and the 404 path is unchanged. "
|
|
||||||
"Ran the focused view tests and ruff."
|
|
||||||
)
|
|
||||||
result = await _create_with(
|
|
||||||
report_state,
|
|
||||||
code_locations=[_FIX_LOCATION],
|
|
||||||
fix_verification=verification,
|
|
||||||
)
|
|
||||||
assert result["success"] is True
|
|
||||||
assert report_state.vulnerability_reports[0]["fix_verification"] == verification
|
|
||||||
|
|
||||||
|
|
||||||
async def test_informational_location_needs_no_verification(report_state: ReportState) -> None:
|
|
||||||
result = await _create_with(report_state, code_locations=[_INFO_LOCATION])
|
|
||||||
assert result["success"] is True
|
|
||||||
assert "fix_verification" not in report_state.vulnerability_reports[0]
|
|
||||||
|
|
||||||
|
|
||||||
def test_vuln_tool_exposes_fix_verification() -> None:
|
|
||||||
assert "fix_verification" in create_vulnerability_report.params_json_schema["properties"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_dep_tool_exposes_contextual_cvss_params() -> None:
|
def test_dep_tool_exposes_contextual_cvss_params() -> None:
|
||||||
dep_props = create_dependency_report.params_json_schema["properties"]
|
dep_props = create_dependency_report.params_json_schema["properties"]
|
||||||
for field in (
|
for field in (
|
||||||
|
|||||||
@@ -242,132 +242,3 @@ def test_write_sarif_replaces_atomically_no_partial_on_reemit(tmp_path: Path) ->
|
|||||||
assert leftovers == []
|
assert leftovers == []
|
||||||
# And it parses as a complete document with both findings.
|
# And it parses as a complete document with both findings.
|
||||||
assert len(_read(tmp_path)["runs"][0]["results"]) == 2
|
assert len(_read(tmp_path)["runs"][0]["results"]) == 2
|
||||||
|
|
||||||
|
|
||||||
def _coverage(*entries: dict[str, Any], **overrides: Any) -> dict[str, Any]:
|
|
||||||
doc: dict[str, Any] = {
|
|
||||||
"entries": list(entries),
|
|
||||||
"completeness": {"complete": True, "caveats": []},
|
|
||||||
}
|
|
||||||
doc.update(overrides)
|
|
||||||
return doc
|
|
||||||
|
|
||||||
|
|
||||||
def _coverage_entry(**overrides: Any) -> dict[str, Any]:
|
|
||||||
base: dict[str, Any] = {
|
|
||||||
"surface": "POST /api/orders/{id}",
|
|
||||||
"risk_area": "SQL injection",
|
|
||||||
"outcome": "no_issue_found",
|
|
||||||
"outcome_label": "No issue identified",
|
|
||||||
"evidence": "14 parameters fuzzed; all queries parameterized.",
|
|
||||||
"recorded_by": "injection-tester",
|
|
||||||
"source": "agent_reported",
|
|
||||||
}
|
|
||||||
base.update(overrides)
|
|
||||||
return base
|
|
||||||
|
|
||||||
|
|
||||||
def test_cleared_surface_becomes_a_passing_result(tmp_path: Path) -> None:
|
|
||||||
""" "Tested and clean" is a SARIF pass, not an absent result."""
|
|
||||||
write_sarif(tmp_path, [], coverage=_coverage(_coverage_entry()))
|
|
||||||
results = _read(tmp_path)["runs"][0]["results"]
|
|
||||||
|
|
||||||
assert len(results) == 1
|
|
||||||
assert results[0]["kind"] == "pass"
|
|
||||||
# SARIF requires level "none" on any result that is not a failure.
|
|
||||||
assert results[0]["level"] == "none"
|
|
||||||
assert "14 parameters fuzzed" in results[0]["message"]["text"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_coverage_outcomes_map_to_their_sarif_kinds(tmp_path: Path) -> None:
|
|
||||||
write_sarif(
|
|
||||||
tmp_path,
|
|
||||||
[],
|
|
||||||
coverage=_coverage(
|
|
||||||
_coverage_entry(outcome="ruled_out", risk_area="XSS"),
|
|
||||||
_coverage_entry(outcome="not_applicable", risk_area="XXE"),
|
|
||||||
_coverage_entry(outcome="needs_follow_up", risk_area="SSRF"),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
kinds = [result["kind"] for result in _read(tmp_path)["runs"][0]["results"]]
|
|
||||||
|
|
||||||
assert kinds == ["pass", "notApplicable", "open"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_reported_coverage_is_not_duplicated_as_a_pass(tmp_path: Path) -> None:
|
|
||||||
"""A surface that produced a finding is already in results as a failure."""
|
|
||||||
write_sarif(
|
|
||||||
tmp_path,
|
|
||||||
[_finding()],
|
|
||||||
coverage=_coverage(_coverage_entry(outcome="reported")),
|
|
||||||
)
|
|
||||||
results = _read(tmp_path)["runs"][0]["results"]
|
|
||||||
|
|
||||||
assert len(results) == 1
|
|
||||||
assert results[0].get("kind", "fail") == "fail"
|
|
||||||
|
|
||||||
|
|
||||||
def test_coverage_results_declare_their_own_rules(tmp_path: Path) -> None:
|
|
||||||
write_sarif(
|
|
||||||
tmp_path,
|
|
||||||
[_finding()],
|
|
||||||
coverage=_coverage(
|
|
||||||
_coverage_entry(risk_area="SQL injection"),
|
|
||||||
_coverage_entry(risk_area="SQL injection", surface="GET /search"),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
run = _read(tmp_path)["runs"][0]
|
|
||||||
rules = run["tool"]["driver"]["rules"]
|
|
||||||
coverage_rules = [rule for rule in rules if rule["id"].startswith("strix-coverage/")]
|
|
||||||
|
|
||||||
# Both entries share one rule, and every result's ruleIndex resolves to it.
|
|
||||||
assert len(coverage_rules) == 1
|
|
||||||
assert coverage_rules[0]["defaultConfiguration"]["level"] == "none"
|
|
||||||
for result in run["results"]:
|
|
||||||
assert rules[result["ruleIndex"]]["id"] == result["ruleId"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_incomplete_run_is_flagged_on_the_invocation(tmp_path: Path) -> None:
|
|
||||||
"""A scan cut short must not be indistinguishable from a clean one."""
|
|
||||||
write_sarif(
|
|
||||||
tmp_path,
|
|
||||||
[],
|
|
||||||
coverage=_coverage(
|
|
||||||
_coverage_entry(),
|
|
||||||
completeness={"complete": False, "caveats": ["Budget exhausted."]},
|
|
||||||
),
|
|
||||||
)
|
|
||||||
invocation = _read(tmp_path)["runs"][0]["invocations"][0]
|
|
||||||
|
|
||||||
assert invocation["executionSuccessful"] is False
|
|
||||||
assert invocation["toolExecutionNotifications"][0]["message"]["text"] == "Budget exhausted."
|
|
||||||
|
|
||||||
|
|
||||||
def test_complete_run_reports_a_successful_invocation(tmp_path: Path) -> None:
|
|
||||||
write_sarif(tmp_path, [], coverage=_coverage(_coverage_entry()))
|
|
||||||
invocation = _read(tmp_path)["runs"][0]["invocations"][0]
|
|
||||||
|
|
||||||
assert invocation["executionSuccessful"] is True
|
|
||||||
assert "toolExecutionNotifications" not in invocation
|
|
||||||
|
|
||||||
|
|
||||||
def test_calibration_metadata_survives_into_result_properties(tmp_path: Path) -> None:
|
|
||||||
write_sarif(
|
|
||||||
tmp_path,
|
|
||||||
[
|
|
||||||
_finding(
|
|
||||||
confidence="medium",
|
|
||||||
counterevidence="WAF blocks the naive payload.",
|
|
||||||
confidence_rationale="Reproduced once out of three attempts.",
|
|
||||||
severity_change_conditions="Critical if the WAF rule is removed.",
|
|
||||||
fix_verification="Not retested.",
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
strix = _read(tmp_path)["runs"][0]["results"][0]["properties"]["strix"]
|
|
||||||
|
|
||||||
assert strix["confidence"] == "medium"
|
|
||||||
assert strix["counterevidence"] == "WAF blocks the naive payload."
|
|
||||||
assert strix["confidence_rationale"] == "Reproduced once out of three attempts."
|
|
||||||
assert strix["severity_change_conditions"] == "Critical if the WAF rule is removed."
|
|
||||||
assert strix["fix_verification"] == "Not retested."
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from pathlib import Path
|
|||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
import strix.skills as skills_mod
|
import strix.skills as skills_mod
|
||||||
from strix.agents.prompt import _resolve_skills, render_system_prompt
|
from strix.agents.prompt import render_system_prompt
|
||||||
from strix.skills import (
|
from strix.skills import (
|
||||||
get_all_skill_names,
|
get_all_skill_names,
|
||||||
get_available_skills,
|
get_available_skills,
|
||||||
@@ -232,42 +232,3 @@ def test_builtin_skill_still_loads_when_not_overridden(tmp_path: Path) -> None:
|
|||||||
def test_missing_skill_is_skipped(tmp_path: Path) -> None:
|
def test_missing_skill_is_skipped(tmp_path: Path) -> None:
|
||||||
register_skill_dir(tmp_path)
|
register_skill_dir(tmp_path)
|
||||||
assert load_skills(["does_not_exist"]) == {}
|
assert load_skills(["does_not_exist"]) == {}
|
||||||
|
|
||||||
|
|
||||||
def test_resolve_skills_always_includes_analysis_baseline() -> None:
|
|
||||||
resolved = _resolve_skills(requested=None)
|
|
||||||
|
|
||||||
assert "analysis/counterevidence" in resolved
|
|
||||||
assert "analysis/severity_calibration" in resolved
|
|
||||||
|
|
||||||
|
|
||||||
def test_resolve_skills_adds_diff_mode_only_when_diff_scoped() -> None:
|
|
||||||
assert "scan_modes/diff" not in _resolve_skills(requested=None)
|
|
||||||
diff_scoped = _resolve_skills(requested=None, is_diff_scoped=True)
|
|
||||||
assert "scan_modes/diff" in diff_scoped
|
|
||||||
# Diff scope overlays the depth mode rather than replacing it.
|
|
||||||
assert "scan_modes/deep" in diff_scoped
|
|
||||||
|
|
||||||
|
|
||||||
def test_resolve_skills_gates_source_aware_skills_on_whitebox() -> None:
|
|
||||||
blackbox = _resolve_skills(requested=None)
|
|
||||||
assert "analysis/fix_verification" not in blackbox
|
|
||||||
assert "analysis/source_aware_discovery" not in blackbox
|
|
||||||
|
|
||||||
whitebox = _resolve_skills(requested=None, is_whitebox=True)
|
|
||||||
assert "analysis/fix_verification" in whitebox
|
|
||||||
assert "analysis/source_aware_discovery" in whitebox
|
|
||||||
|
|
||||||
|
|
||||||
def test_new_skill_files_load() -> None:
|
|
||||||
names = [
|
|
||||||
"analysis/counterevidence",
|
|
||||||
"analysis/severity_calibration",
|
|
||||||
"analysis/fix_verification",
|
|
||||||
"analysis/source_aware_discovery",
|
|
||||||
"scan_modes/diff",
|
|
||||||
]
|
|
||||||
loaded = load_skills(names)
|
|
||||||
for name in names:
|
|
||||||
key = name.split("/")[-1]
|
|
||||||
assert loaded.get(key), f"{name} failed to load"
|
|
||||||
|
|||||||
@@ -1,66 +0,0 @@
|
|||||||
"""coverage.json is a deliverable artifact, not runtime state."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
from typing import TYPE_CHECKING
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from strix.core.paths import runtime_state_dir
|
|
||||||
from strix.report.state import ReportState
|
|
||||||
from strix.tools.coverage.tools import _record_impl, hydrate_coverage_from_disk
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture
|
|
||||||
def state(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> ReportState:
|
|
||||||
monkeypatch.chdir(tmp_path)
|
|
||||||
report_state = ReportState(run_name="run-1")
|
|
||||||
hydrate_coverage_from_disk(runtime_state_dir(report_state.get_run_dir()))
|
|
||||||
return report_state
|
|
||||||
|
|
||||||
|
|
||||||
def _record_a_cleared_surface() -> None:
|
|
||||||
_record_impl(
|
|
||||||
surface="POST /api/orders/{id}",
|
|
||||||
risk_area="SQL injection",
|
|
||||||
outcome="no_issue_found",
|
|
||||||
evidence="14 parameters fuzzed; every query parameterized.",
|
|
||||||
agent_id="agent-1",
|
|
||||||
agent_name="injection-tester",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_coverage_is_written_beside_the_other_artifacts(state: ReportState) -> None:
|
|
||||||
_record_a_cleared_surface()
|
|
||||||
|
|
||||||
state._save_artifacts()
|
|
||||||
|
|
||||||
document = json.loads((state.get_run_dir() / "coverage.json").read_text(encoding="utf-8"))
|
|
||||||
assert document["entries"][0]["risk_area"] == "SQL injection"
|
|
||||||
assert document["summary"]["surfaces_reviewed"] == 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_cleared_surfaces_reach_sarif(state: ReportState) -> None:
|
|
||||||
_record_a_cleared_surface()
|
|
||||||
|
|
||||||
state._save_artifacts()
|
|
||||||
|
|
||||||
sarif = json.loads((state.get_run_dir() / "findings.sarif").read_text(encoding="utf-8"))
|
|
||||||
results = sarif["runs"][0]["results"]
|
|
||||||
assert [result["kind"] for result in results] == ["pass"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_artifacts_still_land_when_coverage_is_empty(state: ReportState) -> None:
|
|
||||||
state.final_scan_result = "Scan complete."
|
|
||||||
|
|
||||||
state._save_artifacts()
|
|
||||||
|
|
||||||
run_dir = state.get_run_dir()
|
|
||||||
assert (run_dir / "penetration_test_report.md").is_file()
|
|
||||||
document = json.loads((run_dir / "coverage.json").read_text(encoding="utf-8"))
|
|
||||||
assert document["entries"] == []
|
|
||||||
@@ -1,343 +0,0 @@
|
|||||||
"""Tests for the target-scoped threat model cache."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import subprocess
|
|
||||||
from datetime import UTC, datetime, timedelta
|
|
||||||
from typing import TYPE_CHECKING
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from strix.agents.factory import _BASE_TOOLS
|
|
||||||
from strix.tools.threat_model import tools as threat_model_tools
|
|
||||||
from strix.tools.threat_model.tools import (
|
|
||||||
_amend_impl,
|
|
||||||
_get_impl,
|
|
||||||
_save_impl,
|
|
||||||
amend_threat_model,
|
|
||||||
get_threat_model,
|
|
||||||
save_threat_model,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
_MODEL = """# Threat Model
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
A multi-tenant billing API. Product code lives in `api/`; `scripts/` is
|
|
||||||
developer-only tooling and is not deployed.
|
|
||||||
|
|
||||||
## Trust Boundaries and Assumptions
|
|
||||||
Requests arrive from untrusted tenants through `api/router.py`. The tenant id
|
|
||||||
is taken from the signed session, never from the request body. Operators
|
|
||||||
configure webhooks; developers control migrations.
|
|
||||||
|
|
||||||
## Attack Surface and Attacker Stories
|
|
||||||
The public REST surface and the webhook receiver are attacker-reachable. A
|
|
||||||
realistic story is a tenant reading another tenant's invoices. Local CLI
|
|
||||||
tooling is not a realistic surface.
|
|
||||||
|
|
||||||
## Severity Calibration
|
|
||||||
Critical: cross-tenant write. High: cross-tenant read. Medium: authenticated
|
|
||||||
self-scoped information leak. Low: verbose errors.
|
|
||||||
"""
|
|
||||||
|
|
||||||
|
|
||||||
def _git(repo: Path, *args: str) -> None:
|
|
||||||
subprocess.run(["/usr/bin/env", "git", *args], cwd=repo, check=True) # noqa: S603
|
|
||||||
|
|
||||||
|
|
||||||
def _make_repo(tmp_path: Path, name: str = "repo") -> Path:
|
|
||||||
repo = tmp_path / name
|
|
||||||
repo.mkdir(parents=True)
|
|
||||||
_git(repo, "init", "-q")
|
|
||||||
_git(repo, "config", "user.email", "t@example.com")
|
|
||||||
_git(repo, "config", "user.name", "t")
|
|
||||||
(repo / "README.md").write_text("hi\n", encoding="utf-8")
|
|
||||||
_git(repo, "add", "README.md")
|
|
||||||
_git(repo, "commit", "-qm", "init")
|
|
||||||
return repo
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _isolated_cache(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
|
||||||
monkeypatch.setattr(threat_model_tools, "_CACHE_DIR", tmp_path / "cache")
|
|
||||||
|
|
||||||
|
|
||||||
def test_missing_model_reports_not_found(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
|
|
||||||
result = _get_impl(str(repo))
|
|
||||||
|
|
||||||
assert result["success"] is True
|
|
||||||
assert result["found"] is False
|
|
||||||
assert "save_threat_model" in result["message"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_saved_model_round_trips(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
|
|
||||||
assert _save_impl(str(repo), _MODEL, "Strix")["success"] is True
|
|
||||||
result = _get_impl(str(repo))
|
|
||||||
|
|
||||||
assert result["found"] is True
|
|
||||||
assert result["stale"] is False
|
|
||||||
assert "multi-tenant billing API" in result["content"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_model_is_stale_after_new_revision(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
_save_impl(str(repo), _MODEL, None)
|
|
||||||
|
|
||||||
(repo / "next.py").write_text("x = 1\n", encoding="utf-8")
|
|
||||||
_git(repo, "add", "next.py")
|
|
||||||
_git(repo, "commit", "-qm", "next")
|
|
||||||
|
|
||||||
result = _get_impl(str(repo))
|
|
||||||
|
|
||||||
assert result["found"] is True
|
|
||||||
assert result["stale"] is True
|
|
||||||
assert result["content"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_cache_is_keyed_per_repository(tmp_path: Path) -> None:
|
|
||||||
first = _make_repo(tmp_path, "first")
|
|
||||||
second = _make_repo(tmp_path, "second")
|
|
||||||
_save_impl(str(first), _MODEL, None)
|
|
||||||
|
|
||||||
assert _get_impl(str(second))["found"] is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_rejects_model_missing_required_sections(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
thin = _MODEL.replace("## Severity Calibration", "## Notes")
|
|
||||||
|
|
||||||
result = _save_impl(str(repo), thin, None)
|
|
||||||
|
|
||||||
assert result["success"] is False
|
|
||||||
assert "severity calibration" in result["error"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_rejects_stub_model(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
|
|
||||||
result = _save_impl(str(repo), "overview trust boundaries attack surface", None)
|
|
||||||
|
|
||||||
assert result["success"] is False
|
|
||||||
assert "too thin" in result["error"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_rejects_empty_target() -> None:
|
|
||||||
result = _get_impl(" ")
|
|
||||||
assert result["success"] is False
|
|
||||||
assert "target cannot be empty" in result["error"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_tools_are_registered() -> None:
|
|
||||||
assert get_threat_model in _BASE_TOOLS
|
|
||||||
assert save_threat_model in _BASE_TOOLS
|
|
||||||
|
|
||||||
|
|
||||||
_ADDENDUM = (
|
|
||||||
"The base model calls the webhook receiver operator-controlled. It is "
|
|
||||||
"unauthenticated in `api/webhooks.py:31`, so treat its body as attacker-controlled."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_amendment_is_returned_with_the_model(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
_save_impl(str(repo), _MODEL, "root")
|
|
||||||
|
|
||||||
assert _amend_impl(str(repo), _ADDENDUM, "webhook-agent")["success"] is True
|
|
||||||
result = _get_impl(str(repo))
|
|
||||||
|
|
||||||
assert result["content"] == _MODEL.strip()
|
|
||||||
assert [a["content"] for a in result["amendments"]] == [_ADDENDUM]
|
|
||||||
assert result["amendments"][0]["by"] == "webhook-agent"
|
|
||||||
|
|
||||||
|
|
||||||
def test_amendments_accumulate_without_overwriting(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
_save_impl(str(repo), _MODEL, "root")
|
|
||||||
|
|
||||||
_amend_impl(str(repo), _ADDENDUM, "agent-a")
|
|
||||||
second = "The `scripts/` directory ships in the container image; it is not dev-only."
|
|
||||||
_amend_impl(str(repo), second + " See `Dockerfile:14`.", "agent-b")
|
|
||||||
|
|
||||||
amendments = _get_impl(str(repo))["amendments"]
|
|
||||||
assert len(amendments) == 2
|
|
||||||
assert [a["by"] for a in amendments] == ["agent-a", "agent-b"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_amend_requires_an_existing_model(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
|
|
||||||
result = _amend_impl(str(repo), _ADDENDUM, None)
|
|
||||||
|
|
||||||
assert result["success"] is False
|
|
||||||
assert "save_threat_model" in result["error"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_amend_rejects_a_stub(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
_save_impl(str(repo), _MODEL, "root")
|
|
||||||
|
|
||||||
assert _amend_impl(str(repo), "looks wrong", None)["success"] is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_save_clears_amendments_and_says_so(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
_save_impl(str(repo), _MODEL, "root")
|
|
||||||
_amend_impl(str(repo), _ADDENDUM, "agent-a")
|
|
||||||
|
|
||||||
result = _save_impl(str(repo), _MODEL.replace("billing API", "billing service"), "root")
|
|
||||||
|
|
||||||
assert result["amendments_cleared"] == 1
|
|
||||||
assert "cleared" in result["message"]
|
|
||||||
assert "amendments" not in _get_impl(str(repo))
|
|
||||||
|
|
||||||
|
|
||||||
def test_amend_tool_is_registered() -> None:
|
|
||||||
assert amend_threat_model in _BASE_TOOLS
|
|
||||||
|
|
||||||
|
|
||||||
_BLACKBOX_MODEL = _MODEL.replace(
|
|
||||||
"Product code lives in `api/`; `scripts/` is\ndeveloper-only tooling and is not deployed.",
|
|
||||||
"Only the deployed surface is visible; no source. Inferred from recon.",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def test_blackbox_target_round_trips() -> None:
|
|
||||||
target = "https://app.example.com"
|
|
||||||
|
|
||||||
assert _save_impl(target, _BLACKBOX_MODEL, "recon")["success"] is True
|
|
||||||
result = _get_impl(target)
|
|
||||||
|
|
||||||
assert result["found"] is True
|
|
||||||
assert result["stale"] is False, "a fresh model with no revision is not stale"
|
|
||||||
assert result["revision"] == "unversioned"
|
|
||||||
assert "Inferred from recon" in result["content"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_blackbox_target_spellings_share_one_model() -> None:
|
|
||||||
_save_impl("https://App.Example.com:443/", _BLACKBOX_MODEL, "recon")
|
|
||||||
|
|
||||||
for spelling in ("https://app.example.com", "app.example.com", "https://app.example.com/"):
|
|
||||||
assert _get_impl(spelling)["found"] is True, spelling
|
|
||||||
|
|
||||||
assert _get_impl("https://other.example.com")["found"] is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_blackbox_model_goes_stale_with_age() -> None:
|
|
||||||
target = "https://app.example.com"
|
|
||||||
_save_impl(target, _BLACKBOX_MODEL, "recon")
|
|
||||||
|
|
||||||
aged = (datetime.now(UTC) - timedelta(days=threat_model_tools._MAX_AGE_DAYS + 1)).isoformat()
|
|
||||||
path = threat_model_tools._cache_path("app.example.com:443")
|
|
||||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
payload["created_at"] = aged
|
|
||||||
path.write_text(json.dumps(payload), encoding="utf-8")
|
|
||||||
|
|
||||||
result = _get_impl(target)
|
|
||||||
|
|
||||||
assert result["stale"] is True
|
|
||||||
assert "re-confirm" in result["message"]
|
|
||||||
|
|
||||||
|
|
||||||
def test_blackbox_target_can_be_amended() -> None:
|
|
||||||
target = "https://app.example.com"
|
|
||||||
_save_impl(target, _BLACKBOX_MODEL, "recon")
|
|
||||||
|
|
||||||
addendum = (
|
|
||||||
"The model infers /admin is IP-restricted. It is reachable with any "
|
|
||||||
"authenticated session; the restriction is only on /admin/settings."
|
|
||||||
)
|
|
||||||
assert _amend_impl(target, addendum, "authz-agent")["success"] is True
|
|
||||||
assert _get_impl(target)["amendments"][0]["content"] == addendum
|
|
||||||
|
|
||||||
|
|
||||||
def test_checkout_and_its_remote_are_the_same_target(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
_git(repo, "remote", "add", "origin", "https://github.com/acme/billing.git")
|
|
||||||
_save_impl(str(repo), _MODEL, "root")
|
|
||||||
|
|
||||||
clone = _make_repo(tmp_path, "clone")
|
|
||||||
_git(clone, "remote", "add", "origin", "https://github.com/acme/billing.git")
|
|
||||||
|
|
||||||
assert _get_impl(str(clone))["found"] is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_path_on_a_known_host_resolves_to_the_scan_target() -> None:
|
|
||||||
scan_targets = ["https://app.example.com"]
|
|
||||||
_save_impl("https://app.example.com", _BLACKBOX_MODEL, "root", scan_targets)
|
|
||||||
|
|
||||||
# An agent testing one page names that page, not the scan's target string.
|
|
||||||
assert _get_impl("https://app.example.com/admin/login", scan_targets)["found"] is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_two_scan_targets_on_one_host_stay_separate() -> None:
|
|
||||||
scan_targets = ["https://example.com/tenant-a", "https://example.com/tenant-b"]
|
|
||||||
_save_impl("https://example.com/tenant-a", _BLACKBOX_MODEL, "root", scan_targets)
|
|
||||||
|
|
||||||
assert _get_impl("https://example.com/tenant-b", scan_targets)["found"] is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_unknown_host_is_not_snapped_onto_the_scan_target() -> None:
|
|
||||||
scan_targets = ["https://app.example.com"]
|
|
||||||
_save_impl("https://app.example.com", _BLACKBOX_MODEL, "root", scan_targets)
|
|
||||||
|
|
||||||
assert _get_impl("https://unrelated.test", scan_targets)["found"] is False
|
|
||||||
|
|
||||||
|
|
||||||
def test_empty_target_falls_back_to_a_single_scan_target() -> None:
|
|
||||||
scan_targets = ["https://app.example.com"]
|
|
||||||
_save_impl("", _BLACKBOX_MODEL, "root", scan_targets)
|
|
||||||
|
|
||||||
assert _get_impl("", scan_targets)["found"] is True
|
|
||||||
assert _get_impl("https://app.example.com")["found"] is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_repository_subdirectory_shares_the_repository_model(tmp_path: Path) -> None:
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
(repo / "src").mkdir()
|
|
||||||
_save_impl(str(repo), _MODEL, "root")
|
|
||||||
|
|
||||||
assert _get_impl(str(repo / "src"))["found"] is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_checkout_and_its_clone_url_are_one_identity(tmp_path: Path) -> None:
|
|
||||||
"""The model an agent saves inside the checkout must be visible to an agent
|
|
||||||
that names the same repository by the URL it was cloned from."""
|
|
||||||
repo = _make_repo(tmp_path)
|
|
||||||
_git(repo, "remote", "add", "origin", "https://github.com/acme/billing.git")
|
|
||||||
_save_impl(str(repo), _MODEL, "root")
|
|
||||||
|
|
||||||
assert _get_impl("https://github.com/acme/billing")["found"] is True
|
|
||||||
assert _get_impl("https://github.com/acme/billing.git")["found"] is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_ssh_and_https_remotes_are_one_identity(tmp_path: Path) -> None:
|
|
||||||
"""One repository cloned over scp-style SSH and over HTTPS is one target."""
|
|
||||||
over_ssh = _make_repo(tmp_path, "ssh-clone")
|
|
||||||
_git(over_ssh, "remote", "add", "origin", "git@github.com:acme/billing.git")
|
|
||||||
_save_impl(str(over_ssh), _MODEL, "root")
|
|
||||||
|
|
||||||
over_https = _make_repo(tmp_path, "https-clone")
|
|
||||||
_git(over_https, "remote", "add", "origin", "https://github.com/acme/billing.git")
|
|
||||||
|
|
||||||
assert _get_impl(str(over_https))["found"] is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_different_repositories_on_one_host_stay_separate(tmp_path: Path) -> None:
|
|
||||||
first = _make_repo(tmp_path, "billing")
|
|
||||||
_git(first, "remote", "add", "origin", "git@github.com:acme/billing.git")
|
|
||||||
_save_impl(str(first), _MODEL, "root")
|
|
||||||
|
|
||||||
second = _make_repo(tmp_path, "payments")
|
|
||||||
_git(second, "remote", "add", "origin", "git@github.com:acme/payments.git")
|
|
||||||
|
|
||||||
assert _get_impl(str(second))["found"] is False
|
|
||||||
@@ -13,7 +13,7 @@ from agents.tool import ToolOutputImage
|
|||||||
|
|
||||||
from strix.config.settings import DEFAULT_MAX_TURNS
|
from strix.config.settings import DEFAULT_MAX_TURNS
|
||||||
from strix.interface.tui.backend.controller import TuiController
|
from strix.interface.tui.backend.controller import TuiController
|
||||||
from strix.interface.tui.backend.projection import bounded_state_projection, terminal_projection
|
from strix.interface.tui.backend.projection import terminal_projection
|
||||||
from strix.interface.tui.backend.protocol import (
|
from strix.interface.tui.backend.protocol import (
|
||||||
MAX_COMMAND_BYTES,
|
MAX_COMMAND_BYTES,
|
||||||
PROTOCOL_CAPABILITIES,
|
PROTOCOL_CAPABILITIES,
|
||||||
@@ -215,11 +215,7 @@ def test_unicode_heavy_setup_state_stays_within_control_frame_limit() -> None:
|
|||||||
"Any",
|
"Any",
|
||||||
SimpleNamespace(
|
SimpleNamespace(
|
||||||
caido_url="https://例え.example/" + "道" * 10_000,
|
caido_url="https://例え.example/" + "道" * 10_000,
|
||||||
get_total_llm_usage=lambda: {
|
get_total_llm_usage=lambda: {f"model-{index}": "費" * 10_000 for index in range(20)},
|
||||||
"total_tokens": 720_400,
|
|
||||||
"cost": 20.0,
|
|
||||||
**{f"model-{index}": "🔒" * 10_000 for index in range(20)},
|
|
||||||
},
|
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
server = TuiBackendServer(controller)
|
server = TuiBackendServer(controller)
|
||||||
@@ -230,26 +226,6 @@ def test_unicode_heavy_setup_state_stays_within_control_frame_limit() -> None:
|
|||||||
assert len(encoded) <= MAX_COMMAND_BYTES
|
assert len(encoded) <= MAX_COMMAND_BYTES
|
||||||
assert "🔒".encode() in encoded
|
assert "🔒".encode() in encoded
|
||||||
assert snapshot["projection_truncated"] is True
|
assert snapshot["projection_truncated"] is True
|
||||||
assert snapshot["usage"] == {"total_tokens": 720_400, "cost": 20.0}
|
|
||||||
|
|
||||||
|
|
||||||
def test_defensive_state_projection_preserves_usage_summary() -> None:
|
|
||||||
controller = TuiController(args())
|
|
||||||
controller.report_state = cast(
|
|
||||||
"Any",
|
|
||||||
SimpleNamespace(
|
|
||||||
caido_url=None,
|
|
||||||
get_total_llm_usage=lambda: {"total_tokens": 720_400, "cost": 20.0},
|
|
||||||
),
|
|
||||||
)
|
|
||||||
state = controller.snapshot()
|
|
||||||
state["provider"] = None
|
|
||||||
state["future_oversized_field"] = "x" * 100_000
|
|
||||||
|
|
||||||
snapshot = bounded_state_projection(state)
|
|
||||||
|
|
||||||
assert snapshot["projection_truncated"] is True
|
|
||||||
assert snapshot["usage"] == {"total_tokens": 720_400, "cost": 20.0}
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
|
|||||||
Reference in New Issue
Block a user