mirror of
https://github.com/usestrix/strix.git
synced 2026-08-16 09:26:39 +02:00
Compare commits
15
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5ca7bc2092 | ||
|
|
cb3fa2db51 | ||
|
|
5e8a773f3e | ||
|
|
13b7374d54 | ||
|
|
5ad2c96f9b | ||
|
|
067597cfdb | ||
|
|
b79483c6b7 | ||
|
|
2bc86902a4 | ||
|
|
c8b83d91db | ||
|
|
8ca0c4a9b8 | ||
|
|
7cc9fa9faa | ||
|
|
174c16fa26 | ||
|
|
94a2586aaa | ||
|
|
372e27fa17 | ||
|
|
ad727edd66 |
@@ -118,6 +118,20 @@ ENV AGENT_BROWSER_USER_AGENT="Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36
|
||||
ENV AGENT_BROWSER_ARGS="--disable-blink-features=AutomationControlled,--no-first-run,--no-default-browser-check,--lang=en-US"
|
||||
ENV AGENT_BROWSER_SCREENSHOT_DIR=/workspace/.agent-browser-screenshots
|
||||
ENV AGENT_BROWSER_IDLE_TIMEOUT_MS=180000
|
||||
USER root
|
||||
RUN set -eu; \
|
||||
{ \
|
||||
for var in AGENT_BROWSER_EXECUTABLE_PATH AGENT_BROWSER_USER_AGENT \
|
||||
AGENT_BROWSER_ARGS AGENT_BROWSER_SCREENSHOT_DIR \
|
||||
AGENT_BROWSER_IDLE_TIMEOUT_MS; do \
|
||||
eval "value=\${$var}"; \
|
||||
printf 'export %s="${%s:-%s}"\n' "$var" "$var" "$value"; \
|
||||
done; \
|
||||
} > /tmp/agent-browser.sh; \
|
||||
install -m 0644 /tmp/agent-browser.sh /etc/profile.d/agent-browser.sh; \
|
||||
rm /tmp/agent-browser.sh; \
|
||||
env -i bash -lc 'test "${AGENT_BROWSER_IDLE_TIMEOUT_MS}" = "180000"'
|
||||
USER pentester
|
||||
RUN /home/pentester/.npm-global/bin/agent-browser doctor --offline --quick
|
||||
|
||||
RUN set -eux; \
|
||||
|
||||
@@ -8,7 +8,7 @@ Configure Strix using environment variables or a config file.
|
||||
## LLM Configuration
|
||||
|
||||
<ParamField path="STRIX_LLM" type="string" required>
|
||||
Model name in LiteLLM format (e.g., `openai/gpt-5.4`, `anthropic/claude-sonnet-4-6`).
|
||||
Model name in LiteLLM format, such as `openai/gpt-5.4` or `anthropic/claude-sonnet-4-6`.
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="LLM_API_KEY" type="string">
|
||||
@@ -20,8 +20,8 @@ Configure Strix using environment variables or a config file.
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="LLM_EXTRA_HEADERS" type="string">
|
||||
Extra HTTP headers sent on every LLM request, as a JSON object (e.g.
|
||||
`{"X-Feature-Key":"value","X-Tenant":"acme"}`). Useful for OpenAI-compatible
|
||||
Extra HTTP headers sent on every LLM request as a JSON object, such as
|
||||
`{"X-Feature-Key":"value","X-Tenant":"acme"}`. These headers help OpenAI-compatible
|
||||
gateways that require attribution or routing headers in addition to the bearer
|
||||
token. The bearer token itself still comes from `LLM_API_KEY`. Applies to both
|
||||
the LiteLLM and native OpenAI routing paths.
|
||||
@@ -65,8 +65,8 @@ affecting the agents that do the actual testing.
|
||||
|
||||
<ParamField path="DEDUPE_LLM_EXTRA_HEADERS" type="string">
|
||||
Optional JSON object of extra HTTP headers sent on every deduplication-model
|
||||
request, e.g. `{"X-Feature-Key":"value"}`. A dedicated dedupe model never
|
||||
inherits `LLM_EXTRA_HEADERS`; set this when its endpoint needs custom headers.
|
||||
request, such as `{"X-Feature-Key":"value"}`. A dedicated dedupe model never
|
||||
inherits `LLM_EXTRA_HEADERS`. Set this variable when its endpoint needs custom headers.
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="STRIX_DEDUPE_REASONING_EFFORT" type="string">
|
||||
@@ -81,7 +81,7 @@ affecting the agents that do the actual testing.
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="POSTMAN_API_KEY" type="string">
|
||||
Postman API key (`PMAK-…`). Enables fetching Postman collections by id as a target (`postman://<collection-uid>`), and Postman environments (`postman://<collection-uid>?env=<environment-uid>`) to resolve collection variables. Not needed when passing a local collection export file.
|
||||
Postman API key (`PMAK-…`). Enables fetching Postman collections by ID as a target (`postman://<collection-uid>`), and Postman environments (`postman://<collection-uid>?env=<environment-uid>`) to resolve collection variables. Not needed when passing a local collection export file.
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="STRIX_TELEMETRY" default="1" type="string">
|
||||
|
||||
@@ -3,11 +3,11 @@ title: "Skills"
|
||||
description: "Specialized knowledge packages that enhance agent capabilities"
|
||||
---
|
||||
|
||||
Skills are structured knowledge packages that give Strix agents deep expertise in specific vulnerability types, technologies, and testing methodologies.
|
||||
Skills are structured knowledge packages that give Strix agents deep expertise in specific vulnerability types, technologies, and testing methods.
|
||||
|
||||
## The Idea
|
||||
|
||||
LLMs have broad but shallow security knowledge. They know _about_ SQL injection, but lack the nuanced techniques that experienced pentesters use—parser quirks, bypass methods, validation tricks, and chain attacks.
|
||||
LLMs have broad but shallow security knowledge. They know _about_ SQL injection but lack the nuanced techniques that experienced pentesters use, such as parser quirks, bypass methods, validation tricks, and chain attacks.
|
||||
|
||||
Skills inject this deep, specialized knowledge directly into the agent's context, transforming it from a generalist into a specialist for the task at hand.
|
||||
|
||||
@@ -25,9 +25,9 @@ create_agent(
|
||||
|
||||
The skills are injected into the agent's system prompt, giving it access to:
|
||||
|
||||
- **Advanced techniques** — Non-obvious methods beyond standard testing
|
||||
- **Working payloads** — Practical examples with variations
|
||||
- **Validation methods** — How to confirm findings and avoid false positives
|
||||
- **Advanced techniques:** Non-obvious methods beyond standard testing
|
||||
- **Working payloads:** Practical examples with variations
|
||||
- **Validation methods:** How to confirm findings and avoid false positives
|
||||
|
||||
## Skill Categories
|
||||
|
||||
@@ -138,7 +138,7 @@ How to confirm findings and avoid false positives.
|
||||
|
||||
Community contributions are welcome. Create a `.md` file in the appropriate category with YAML frontmatter (`name` and `description` fields). Good skills include:
|
||||
|
||||
1. **Real-world techniques** — Methods that work in practice
|
||||
2. **Practical payloads** — Working examples with variations
|
||||
3. **Validation steps** — How to confirm without false positives
|
||||
4. **Context awareness** — Version/environment-specific behavior
|
||||
1. **Real-world techniques:** Methods that work in practice
|
||||
2. **Practical payloads:** Working examples with variations
|
||||
3. **Validation steps:** How to confirm without false positives
|
||||
4. **Context awareness:** Version/environment-specific behavior
|
||||
|
||||
@@ -24,10 +24,10 @@ Skip the setup. Run Strix in the cloud at [app.strix.ai](https://app.strix.ai).
|
||||
|
||||
## What You Get
|
||||
|
||||
- **Penetration test reports** — Validated findings with PoCs
|
||||
- **Shareable dashboards** — Collaborate with your team
|
||||
- **CI/CD integration** — Block risky changes automatically
|
||||
- **Continuous monitoring** — Catch new vulnerabilities quickly
|
||||
- **Penetration test reports:** Validated findings with PoCs
|
||||
- **Shareable dashboards:** Collaborate with your team
|
||||
- **CI/CD integration:** Block risky changes automatically
|
||||
- **Continuous monitoring:** Catch new vulnerabilities quickly
|
||||
|
||||
## Getting Started
|
||||
|
||||
|
||||
@@ -52,20 +52,20 @@ Skills are specialized knowledge packages that enhance agent capabilities. They
|
||||
|
||||
1. Choose the right category
|
||||
2. Create a `.md` file with YAML frontmatter (`name` and `description` fields)
|
||||
3. Include practical examples—working payloads, commands, test cases
|
||||
3. Include practical examples, such as working payloads, commands, and test cases
|
||||
4. Provide validation methods to confirm findings
|
||||
5. Submit via PR
|
||||
5. Submit a pull request
|
||||
|
||||
## Contributing Code
|
||||
|
||||
### Pull Request Process
|
||||
|
||||
1. **Create an issue first** — Describe the problem or feature
|
||||
2. **Fork and branch** — Work from `main`
|
||||
3. **Make changes** — Follow existing code style
|
||||
4. **Write tests** — Ensure coverage for new features
|
||||
5. **Run checks** — `make check-all` should pass
|
||||
6. **Submit PR** — Link to issue and provide context
|
||||
1. **Create an issue first:** Describe the problem or feature
|
||||
2. **Fork and branch:** Work from `main`
|
||||
3. **Make changes:** Follow existing code style
|
||||
4. **Write tests:** Ensure coverage for new features
|
||||
5. **Run checks:** `make check-all` should pass
|
||||
6. **Submit a pull request:** Link to issue and provide context
|
||||
|
||||
### Code Style
|
||||
|
||||
@@ -77,7 +77,7 @@ Skills are specialized knowledge packages that enhance agent capabilities. They
|
||||
|
||||
## Package Builds
|
||||
|
||||
Editable installs do not require Go; they run the TUI from source (`go run`).
|
||||
Editable installs do not require Go. They run the TUI from source (`go run`).
|
||||
|
||||
Wheels are intentionally strict: they always bundle the matching Go sidecar and
|
||||
are platform-specific.
|
||||
|
||||
+12
-12
@@ -3,7 +3,7 @@ title: "Introduction"
|
||||
description: "Open-source AI hackers to secure your apps"
|
||||
---
|
||||
|
||||
Strix are autonomous AI agents that act like real hackers—they run your code dynamically, find vulnerabilities, and validate them with proof-of-concepts. Built for developers and security teams who need fast, accurate security testing without the overhead of manual pentesting or the false positives of static analysis tools.
|
||||
Strix agents are autonomous and act like real hackers. They run your code dynamically, find vulnerabilities, and validate each vulnerability with a proof of concept. Strix helps developers and security teams that need fast and accurate security testing. Strix does not have the overhead of a manual pentest or the false positives of a static analysis tool.
|
||||
|
||||
<Frame>
|
||||
<img src="/images/screenshot.png" alt="Strix Demo" />
|
||||
@@ -26,17 +26,17 @@ Strix are autonomous AI agents that act like real hackers—they run your code d
|
||||
|
||||
## Use Cases
|
||||
|
||||
- **Application Security Testing** — Detect and validate critical vulnerabilities in your applications
|
||||
- **Rapid Penetration Testing** — Get penetration tests done in hours, not weeks
|
||||
- **Bug Bounty Automation** — Automate research and generate PoCs for faster reporting
|
||||
- **CI/CD Integration** — Block vulnerabilities before they reach production
|
||||
- **Application Security Testing:** Detect and validate critical vulnerabilities in your applications
|
||||
- **Rapid Penetration Testing:** Get penetration tests done in hours, not weeks
|
||||
- **Bug Bounty Automation:** Automate research and generate PoCs for faster reporting
|
||||
- **CI/CD Integration:** Block vulnerabilities before they reach production
|
||||
|
||||
## Key Capabilities
|
||||
|
||||
- **Full hacker toolkit** — Browser automation, HTTP proxy, terminal, Python runtime
|
||||
- **Real validation** — PoCs, not false positives
|
||||
- **Multi-agent orchestration** — Specialized agents collaborate on complex targets
|
||||
- **Developer-first CLI** — Interactive TUI or headless mode for automation
|
||||
- **Full hacker toolkit:** Browser automation, HTTP proxy, terminal, Python runtime
|
||||
- **Real validation:** PoCs, not false positives
|
||||
- **Multi-agent orchestration:** Specialized agents collaborate on complex targets
|
||||
- **Developer-first CLI:** Interactive TUI or headless mode for automation
|
||||
|
||||
## Security Tools
|
||||
|
||||
@@ -67,9 +67,9 @@ Strix agents come equipped with a comprehensive toolkit:
|
||||
|
||||
Strix uses a graph of specialized agents for comprehensive security testing:
|
||||
|
||||
- **Distributed Workflows** — Specialized agents for different attacks and assets
|
||||
- **Scalable Testing** — Parallel execution for fast comprehensive coverage
|
||||
- **Dynamic Coordination** — Agents collaborate and share discoveries
|
||||
- **Distributed Workflows:** Specialized agents for different attacks and assets
|
||||
- **Scalable Testing:** Parallel execution for fast comprehensive coverage
|
||||
- **Dynamic Coordination:** Agents collaborate and share discoveries
|
||||
|
||||
## Quick Example
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ Strix is built to be driven by AI coding agents. Install the official agent skil
|
||||
|
||||
## Install the Skills
|
||||
|
||||
Works with any agent that supports the open [SKILL.md standard](https://agentskills.io) — Claude Code, Cursor, Codex, Gemini CLI, OpenCode, and dozens more:
|
||||
Works with any agent that supports the open [SKILL.md standard](https://agentskills.io), including Claude Code, Cursor, Codex, Gemini CLI, OpenCode, and dozens more:
|
||||
|
||||
```bash
|
||||
npx skills add usestrix/strix
|
||||
@@ -15,8 +15,8 @@ npx skills add usestrix/strix
|
||||
|
||||
| Skill | What your agent learns |
|
||||
|-------|------------------------|
|
||||
| `penetration-testing-with-strix` | Run headless scans against code, URLs, domains, or IPs — self-hosted CLI or managed cloud — with budget caps, and read the results |
|
||||
| `managed-pentesting-with-strix` | Drive the managed [app.strix.ai](https://app.strix.ai) platform over REST — no local Docker or LLM key needed |
|
||||
| `penetration-testing-with-strix` | Run headless scans against code, URLs, domains, or IPs with the self-hosted CLI or the managed cloud, apply budget caps, and read the results |
|
||||
| `managed-pentesting-with-strix` | Drive the managed [app.strix.ai](https://app.strix.ai) platform over REST. No local Docker or LLM key needed |
|
||||
| `fix-security-vulnerabilities-with-strix` | Triage findings, fix root causes, and re-run Strix to verify each fix |
|
||||
| `ci-security-scanning-with-strix` | Add PR security scanning to GitHub Actions or any CI (self-hosted CLI or managed app) |
|
||||
|
||||
@@ -26,23 +26,25 @@ Install a single skill with `npx skills add usestrix/strix --skill penetration-t
|
||||
npx skills use usestrix/strix@penetration-testing-with-strix | claude
|
||||
```
|
||||
|
||||
## Two ways to run — self-hosted or managed
|
||||
## Two ways to run: self-hosted or managed
|
||||
|
||||
Both use the same engine and produce the same validated findings and SARIF, so agents can pick per situation or combine them:
|
||||
Both use the same engine and produce the same validated findings and SARIF. Agents can pick per situation or combine them.
|
||||
|
||||
- **Open-source CLI (self-hosted)** — runs locally in a Docker sandbox with your own LLM key. Free, fully local, air-gap capable. Best for local dev loops and full control.
|
||||
- **Managed cloud** — runs on Strix's infrastructure via the [app.strix.ai REST API](https://docs.app.strix.ai). No Docker, no LLM key, no local install; adds team dashboards, scheduling, PR reviews, and downloadable PDF/DOCX reports (Enterprise plan). Best in sandboxed/CI environments and for teams. Create an API token under **Settings → API Access**; the `managed-pentesting-with-strix` skill has the full flow.
|
||||
- **Open-source CLI (self-hosted):** Runs locally in a Docker sandbox with your own LLM key. It is free, fully local, and air-gap capable. It suits local development loops and full control.
|
||||
- **Managed cloud:** Runs on Strix infrastructure through the [app.strix.ai REST API](https://docs.app.strix.ai). It needs no Docker, LLM key, or local installation. The Enterprise plan adds team dashboards, scheduling, pull request reviews, and downloadable PDF or DOCX reports. It suits sandboxed or CI environments and teams.
|
||||
|
||||
Create a managed API token under **Settings → API Access**. The `managed-pentesting-with-strix` skill documents the full flow.
|
||||
|
||||
## Agent-Friendly Interfaces
|
||||
|
||||
Everything an agent needs is machine-readable:
|
||||
|
||||
- **Headless CLI** — `strix -n` runs without the TUI and exits with `0` (clean), `1` (error), or `2` (vulnerabilities found).
|
||||
- **REST API** — the managed platform exposes a documented [OpenAPI](https://docs.app.strix.ai/openapi.json) at `https://app.strix.ai/api/v1` (scans, vulnerabilities, assets, PR reviews, schedules, webhooks) with bearer tokens and scopes.
|
||||
- **Structured results** — every run writes `vulnerabilities.json`, `vulnerabilities.csv`, `findings.sarif` (SARIF 2.1.0), and per-finding Markdown under `strix_runs/<run-name>/`; the cloud exposes the same as JSON plus SARIF export.
|
||||
- **Budget controls** — `--max-budget` and `--max-turns` give agents hard cost/time caps.
|
||||
- **`AGENTS.md`** — the [repository's agent guide](https://github.com/usestrix/strix/blob/main/AGENTS.md) with a quick reference.
|
||||
- **`llms.txt`** — this documentation is indexed at [docs.strix.ai/llms.txt](https://docs.strix.ai/llms.txt) and fully exported at [docs.strix.ai/llms-full.txt](https://docs.strix.ai/llms-full.txt); every page is also available as Markdown by appending `.md` to its URL.
|
||||
- **Headless CLI:** `strix -n` runs without the TUI. It exits with `0` for a clean scan, `1` for an error, or `2` for vulnerabilities.
|
||||
- **REST API:** The managed platform exposes a documented [OpenAPI](https://docs.app.strix.ai/openapi.json) at `https://app.strix.ai/api/v1`. It supports scans, vulnerabilities, assets, pull request reviews, schedules, and webhooks. The API uses bearer tokens and scopes.
|
||||
- **Structured results:** Each self-hosted run writes `vulnerabilities.json`, `vulnerabilities.csv`, and `findings.sarif` in SARIF 2.1.0 format. It also writes per-finding Markdown under `strix_runs/<run-name>/`. The cloud exposes the same data as JSON and provides SARIF export.
|
||||
- **Budget controls:** `--max-budget` and `--max-turns` set cost and turn limits.
|
||||
- **`AGENTS.md`:** The [repository's agent guide](https://github.com/usestrix/strix/blob/main/AGENTS.md) provides a quick reference.
|
||||
- **`llms.txt`:** The index is available at [docs.strix.ai/llms.txt](https://docs.strix.ai/llms.txt). The full export is available at [docs.strix.ai/llms-full.txt](https://docs.strix.ai/llms-full.txt). Every page is also available as Markdown by appending `.md` to its URL.
|
||||
|
||||
## Example Prompts
|
||||
|
||||
|
||||
@@ -37,7 +37,7 @@ Add these secrets to your repository:
|
||||
|
||||
| Secret | Description |
|
||||
|--------|-------------|
|
||||
| `STRIX_LLM` | Model name (e.g., `openai/gpt-5.4`) |
|
||||
| `STRIX_LLM` | Model name, such as `openai/gpt-5.4` |
|
||||
| `LLM_API_KEY` | API key for your LLM provider |
|
||||
|
||||
## Exit Codes
|
||||
@@ -46,8 +46,8 @@ The workflow fails when vulnerabilities are found:
|
||||
|
||||
| Code | Result |
|
||||
|------|--------|
|
||||
| 0 | Pass — No vulnerabilities |
|
||||
| 2 | Fail — Vulnerabilities found |
|
||||
| 0 | Pass. No vulnerabilities found |
|
||||
| 2 | Fail. Vulnerabilities found |
|
||||
|
||||
## Scan Modes for CI
|
||||
|
||||
@@ -62,5 +62,5 @@ Use `quick` mode for PRs to keep feedback fast. Schedule `deep` scans nightly.
|
||||
</Tip>
|
||||
|
||||
<Note>
|
||||
For pull_request workflows, Strix automatically uses changed-files diff-scope in CI/headless runs. If diff resolution fails, ensure full history is fetched (`fetch-depth: 0`) or set `--diff-base`.
|
||||
For `pull_request` workflows, Strix automatically uses changed-files diff-scope in CI/headless runs. If diff resolution fails, fetch full history with `fetch-depth: 0` or set `--diff-base`.
|
||||
</Note>
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: "Azure OpenAI"
|
||||
description: "Configure Strix with OpenAI models via Azure"
|
||||
description: "Configure Strix with OpenAI models through Azure"
|
||||
---
|
||||
|
||||
## Setup
|
||||
@@ -19,7 +19,7 @@ export AZURE_API_VERSION="2025-11-01-preview"
|
||||
| `STRIX_LLM` | `azure/<your-deployment-name>` |
|
||||
| `AZURE_API_KEY` | Your Azure OpenAI API key |
|
||||
| `AZURE_API_BASE` | Your Azure OpenAI endpoint URL |
|
||||
| `AZURE_API_VERSION` | API version (e.g., `2025-11-01-preview`) |
|
||||
| `AZURE_API_VERSION` | API version, such as `2025-11-01-preview` |
|
||||
|
||||
## Example
|
||||
|
||||
@@ -33,5 +33,5 @@ export AZURE_API_VERSION="2025-11-01-preview"
|
||||
## Prerequisites
|
||||
|
||||
1. Create an Azure OpenAI resource
|
||||
2. Deploy a model (e.g., GPT-5.4)
|
||||
2. Deploy a model, such as GPT-5.4
|
||||
3. Get the endpoint URL and API key from the Azure portal
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: "AWS Bedrock"
|
||||
description: "Configure Strix with models via AWS Bedrock"
|
||||
description: "Configure Strix with models through AWS Bedrock"
|
||||
---
|
||||
|
||||
## Installation
|
||||
@@ -17,7 +17,7 @@ pipx install "strix-agent[bedrock]"
|
||||
export STRIX_LLM="bedrock/anthropic.claude-4-5-sonnet-20251022-v1:0"
|
||||
```
|
||||
|
||||
No API key required—uses AWS credentials from environment.
|
||||
Strix does not require an API key. Strix uses AWS credentials from the environment.
|
||||
|
||||
## Authentication
|
||||
|
||||
|
||||
@@ -17,7 +17,7 @@ Running Strix with local models allows for completely offline, privacy-first sec
|
||||
<Warning>
|
||||
**Compatibility Note**: Strix relies on advanced agentic capabilities (tool use, multi-step planning, self-correction). Most local models, especially those under 70B parameters, struggle with these complex tasks.
|
||||
|
||||
For critical assessments, we strongly recommend using state-of-the-art cloud models like **Claude 4.5 Sonnet** or **GPT-5**. Use local models only when privacy is the absolute priority.
|
||||
For critical assessments, use state-of-the-art cloud models such as **Claude 4.5 Sonnet** or **GPT-5**. Use local models only when privacy is the absolute priority.
|
||||
</Warning>
|
||||
|
||||
## Ollama
|
||||
@@ -40,8 +40,6 @@ For critical assessments, we strongly recommend using state-of-the-art cloud mod
|
||||
### Recommended Models
|
||||
|
||||
We recommend these models for the best balance of reasoning and tool use:
|
||||
|
||||
**Recommended models:**
|
||||
- **Qwen3 VL** (`ollama pull qwen3-vl`)
|
||||
- **DeepSeek V3.1** (`ollama pull deepseek-v3.1`)
|
||||
- **Devstral 2** (`ollama pull devstral-2`)
|
||||
@@ -59,7 +57,7 @@ export LLM_API_BASE="http://localhost:1234/v1" # Adjust port as needed
|
||||
|
||||
Some OpenAI-compatible gateways require extra HTTP headers (for attribution or
|
||||
tenant routing) alongside the bearer token. Set them with `LLM_EXTRA_HEADERS` as
|
||||
a JSON object — they are sent on every request:
|
||||
a JSON object. Strix sends these headers on every request:
|
||||
|
||||
```bash
|
||||
export STRIX_LLM="openai/your-model"
|
||||
@@ -69,12 +67,12 @@ export LLM_EXTRA_HEADERS='{"X-Feature-Key":"value","X-Tenant":"acme"}'
|
||||
```
|
||||
|
||||
For endpoints behind a private CA, point Strix at your certificate bundle with
|
||||
the standard `SSL_CERT_FILE=/path/to/ca-bundle.pem` — never disable TLS
|
||||
the standard `SSL_CERT_FILE=/path/to/ca-bundle.pem`. Do not disable TLS
|
||||
verification against a real endpoint.
|
||||
|
||||
## Tool calling must return structured `tool_calls`
|
||||
|
||||
Strix is entirely tool-driven: every working turn must be a **native** function/tool call. If your inference server returns the tool call as plain assistant text instead of a structured `tool_calls` field, Strix never sees a call it can execute, so the agent makes no real progress — it re-prompts the model for a tool call and gives up once its recovery attempts are exhausted.
|
||||
Strix is entirely tool-driven. Every working turn must be a **native** function or tool call. If the inference server returns the call as plain assistant text, Strix never sees a call it can execute. The agent makes no real progress and gives up after its recovery attempts end.
|
||||
|
||||
This is almost always an **inference-server configuration** problem, not a model or Strix problem. Common symptoms are the model printing a call as text such as:
|
||||
|
||||
@@ -84,19 +82,19 @@ exec_command(cmd="nmap ...", timeout=180)
|
||||
{"action": "exec_command", "params": {"cmd": "nmap ..."}}
|
||||
```
|
||||
|
||||
The fix belongs on the inference server: it must be configured to parse the model's tool tokens into structured `tool_calls`. A correctly configured endpoint either returns a structured call or rejects the request outright — it never leaks the call as text.
|
||||
The fix belongs on the inference server. It must be configured to parse the model's tool tokens into structured `tool_calls`. A correctly configured endpoint either returns a structured call or rejects the request outright. It never leaks the call as text.
|
||||
|
||||
### Fixes by server
|
||||
|
||||
**llama.cpp (`llama-server`)**
|
||||
- Run with `--jinja` and a correct tool-use chat template (`--chat-template` / `--chat-template-file` matching the model). Recent builds enable `--jinja` by default — **upgrade** if yours doesn't.
|
||||
- For thinking models, align or disable reasoning (`--reasoning-format`, `-rea off`) so it doesn't break tool-call parsing.
|
||||
- A low temperature (e.g. `--temp 0.2`) improves tool-call reliability.
|
||||
- Run with `--jinja` and a correct tool-use chat template (`--chat-template` or `--chat-template-file` matching the model). Recent builds enable `--jinja` by default. Upgrade if yours does not.
|
||||
- For thinking models, align or disable reasoning (`--reasoning-format` or `-rea off`) so it does not break tool-call parsing.
|
||||
- A low temperature, such as `--temp 0.2`, improves tool-call reliability.
|
||||
|
||||
**Ollama**
|
||||
- Use a recent Ollama and a model whose template wires tools. Modern Ollama refuses tools (`tools param requires --jinja flag`) if the template lacks tool support.
|
||||
- For reasoning models (e.g. qwen3), disable the model's **thinking** mode — thinking left on frequently pushes the tool call into the text `content` instead of the structured `tool_calls` field. Turn it off on the Ollama side (a non-thinking model variant, or `think: false` in the model's parameters / `Modelfile`).
|
||||
- Raise **`num_ctx`** to at least 16k–32k. Strix sends a large system prompt plus many tool schemas; at Ollama's small default context the tool definitions are truncated out of the prompt and the model stops emitting valid calls. A short test prompt can look fine while a real scan fails, so set this explicitly rather than inferring it from a quick check.
|
||||
- Use a recent Ollama and a model whose template wires tools. Modern Ollama returns `tools param requires --jinja flag` if the template lacks tool support.
|
||||
- For reasoning models such as qwen3, disable the model's **thinking** mode. Thinking left on can push the tool call into `content` instead of the structured `tool_calls` field. Turn it off on the Ollama side with a non-thinking model variant, `think: false` in the model's parameters, or `Modelfile`.
|
||||
- Raise **`num_ctx`** to at least 16k to 32k. Strix sends a large system prompt plus many tool schemas. At Ollama's small default context, the tool definitions can be truncated from the prompt. The model can then stop emitting valid calls. A short test prompt can look fine while a real scan fails. Set this explicitly instead of inferring it from a quick check.
|
||||
|
||||
**vLLM**
|
||||
- Start with `--enable-auto-tool-choice`, a matching `--tool-call-parser` (`hermes`, `qwen3_xml`, or `llama3_json`), and a matching `--reasoning-parser` for reasoning models.
|
||||
|
||||
@@ -3,7 +3,7 @@ title: "Novita AI"
|
||||
description: "Configure Strix with Novita AI models"
|
||||
---
|
||||
|
||||
[Novita AI](https://novita.ai) provides fast, cost-efficient inference for open-source models via an OpenAI-compatible API.
|
||||
[Novita AI](https://novita.ai) provides fast, cost-efficient inference for open-source models through an OpenAI-compatible API.
|
||||
|
||||
## Setup
|
||||
|
||||
@@ -29,7 +29,7 @@ export LLM_API_BASE="https://api.novita.ai/openai"
|
||||
|
||||
## Benefits
|
||||
|
||||
- **Cost-efficient** — Competitive pricing with per-token billing
|
||||
- **OpenAI-compatible** — Drop-in replacement using `LLM_API_BASE`
|
||||
- **Large context** — Models support up to 262k token context windows
|
||||
- **Function calling** — All listed models support tool/function calling
|
||||
- **Cost-efficient:** Competitive pricing with per-token billing
|
||||
- **OpenAI-compatible:** Drop-in replacement using `LLM_API_BASE`
|
||||
- **Large context:** Models support up to 262k token context windows
|
||||
- **Function calling:** All listed models support tool/function calling
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: "OpenRouter"
|
||||
description: "Configure Strix with models via OpenRouter"
|
||||
description: "Configure Strix with models through OpenRouter"
|
||||
---
|
||||
|
||||
[OpenRouter](https://openrouter.ai) provides access to 100+ models from multiple providers through a single API.
|
||||
@@ -31,7 +31,7 @@ Access any model on OpenRouter using the format `openrouter/<provider>/<model>`:
|
||||
|
||||
## Benefits
|
||||
|
||||
- **Single API** — Access models from OpenAI, Anthropic, Google, Meta, and more
|
||||
- **Fallback routing** — Automatic failover between providers
|
||||
- **Cost tracking** — Monitor usage across all models
|
||||
- **Higher rate limits** — OpenRouter handles provider limits for you
|
||||
- **Single API:** Access models from OpenAI, Anthropic, Google, Meta, and more
|
||||
- **Fallback routing:** Automatic failover between providers
|
||||
- **Cost tracking:** Monitor usage across all models
|
||||
- **Higher rate limits:** OpenRouter handles provider limits for you
|
||||
|
||||
@@ -44,13 +44,13 @@ See the [Local Models guide](/llm-providers/local) for setup instructions and re
|
||||
Access 100+ models through a single API.
|
||||
</Card>
|
||||
<Card title="Google Vertex AI" href="/llm-providers/vertex">
|
||||
Gemini 3 models via Google Cloud.
|
||||
Gemini 3 models through Google Cloud.
|
||||
</Card>
|
||||
<Card title="AWS Bedrock" href="/llm-providers/bedrock">
|
||||
Claude and Titan models via AWS.
|
||||
Claude and Titan models through AWS.
|
||||
</Card>
|
||||
<Card title="Azure OpenAI" href="/llm-providers/azure">
|
||||
GPT-5.4 via Azure.
|
||||
GPT-5.4 through Azure.
|
||||
</Card>
|
||||
<Card title="Local Models" href="/llm-providers/local">
|
||||
Llama 4, Mistral, and self-hosted models.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
title: "Google Vertex AI"
|
||||
description: "Configure Strix with Gemini models via Google Cloud"
|
||||
description: "Configure Strix with Gemini models through Google Cloud"
|
||||
---
|
||||
|
||||
## Installation
|
||||
@@ -17,7 +17,7 @@ pipx install "strix-agent[vertex]"
|
||||
export STRIX_LLM="vertex_ai/gemini-3-pro-preview"
|
||||
```
|
||||
|
||||
No API key required—uses Google Cloud Application Default Credentials.
|
||||
Strix does not require an API key. Strix uses Google Cloud Application Default Credentials.
|
||||
|
||||
## Authentication
|
||||
|
||||
|
||||
+3
-2
@@ -6,7 +6,8 @@ description: "Install Strix and run your first security scan"
|
||||
## Prerequisites
|
||||
|
||||
- Docker (running)
|
||||
- An LLM API key from any [supported provider](/llm-providers/overview) (OpenAI, Anthropic, Google, etc.)
|
||||
- Access to a [supported LLM provider](/llm-providers/overview), such as OpenAI, Anthropic, or Google
|
||||
- Most providers require an API key. Vertex and Bedrock use cloud credentials.
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -43,7 +44,7 @@ strix --target ./your-app
|
||||
```
|
||||
|
||||
<Note>
|
||||
First run pulls the Docker sandbox image automatically. Results are saved to `strix_runs/<run-name>`.
|
||||
The first run pulls the Docker sandbox image automatically. Strix saves results to `strix_runs/<run-name>`.
|
||||
</Note>
|
||||
|
||||
## Target Types
|
||||
|
||||
@@ -3,7 +3,7 @@ title: "Browser"
|
||||
description: "Playwright-powered Chrome for web application testing"
|
||||
---
|
||||
|
||||
Strix uses a headless Chrome browser via Playwright to interact with web applications exactly like a real user would.
|
||||
Strix uses a headless Chrome browser through Playwright to interact with web applications exactly like a real user would.
|
||||
|
||||
## How It Works
|
||||
|
||||
|
||||
@@ -28,6 +28,6 @@ Strix agents use specialized tools to test your applications like a real penetra
|
||||
| -------------- | ---------------------------------------- |
|
||||
| Python Runtime | Write and execute custom exploit scripts |
|
||||
| File Editor | Read and modify source code |
|
||||
| Web Search | Real-time OSINT via Perplexity |
|
||||
| Web Search | Real-time OSINT through Perplexity |
|
||||
| Notes | Document findings during the scan |
|
||||
| Reporting | Generate vulnerability reports with PoCs |
|
||||
|
||||
+11
-12
@@ -70,10 +70,9 @@ asyncio.run(main())
|
||||
| `view_sitemap_entry()` | Inspect one sitemap entry + its related requests |
|
||||
| `scope_rules()` | Manage proxy scope (allowlist/denylist) |
|
||||
|
||||
For one-off arbitrary requests, use shell tooling like `curl` — the
|
||||
sandbox's `HTTP_PROXY` env routes the traffic through Caido
|
||||
automatically, so it lands in `list_requests` and can be replayed via
|
||||
`repeat_request`.
|
||||
For one-off arbitrary requests, use shell tools such as `curl`.
|
||||
The sandbox routes traffic through Caido with the `HTTP_PROXY` variable.
|
||||
Caido then adds each request to `list_requests` for replay through `repeat_request`.
|
||||
|
||||
### Example: Automated IDOR Testing
|
||||
|
||||
@@ -106,24 +105,24 @@ asyncio.run(main())
|
||||
|
||||
## Human-in-the-Loop
|
||||
|
||||
Strix exposes the Caido proxy to your host machine, so you can interact with it alongside the automated scan. When the sandbox starts, the Caido URL is displayed in the TUI sidebar — click it to copy, then open it in Caido Desktop.
|
||||
Strix exposes the Caido proxy to your host machine, so you can interact with it alongside the automated scan. When the sandbox starts, the Caido URL is displayed in the TUI sidebar. Click the URL to copy it, then open it in Caido Desktop.
|
||||
|
||||
### Accessing Caido
|
||||
|
||||
1. Start a scan as usual
|
||||
2. Look for the **Caido** URL in the sidebar stats panel (e.g. `localhost:52341`)
|
||||
2. Find the **Caido** URL in the sidebar stats panel, such as `localhost:52341`
|
||||
3. Open the URL in Caido Desktop
|
||||
4. Click **Continue as guest** to access the instance
|
||||
|
||||
### What You Can Do
|
||||
|
||||
- **Inspect traffic** — Browse all HTTP/HTTPS requests the agent is making in real time
|
||||
- **Replay requests** — Take any captured request and resend it with your own modifications
|
||||
- **Intercept and modify** — Pause requests mid-flight, edit them, then forward
|
||||
- **Explore the sitemap** — See the full attack surface the agent has discovered
|
||||
- **Manual testing** — Use Caido's tools to test findings the agent reports, or explore areas it hasn't reached
|
||||
- **Inspect traffic:** Browse all HTTP/HTTPS requests the agent is making in real time
|
||||
- **Replay requests:** Take any captured request and resend it with your own modifications
|
||||
- **Intercept and modify:** Pause requests mid-flight, edit them, then forward
|
||||
- **Explore the sitemap:** See the full attack surface the agent has discovered
|
||||
- **Manual testing:** Use Caido's tools to test findings the agent reports, or explore areas it has not reached
|
||||
|
||||
This turns Strix from a fully automated scanner into a collaborative tool — the agent handles the heavy lifting while you focus on the interesting parts.
|
||||
Strix is a collaborative tool, not only a fully automated scanner. The agent handles the heavy lifting while you focus on the interesting parts.
|
||||
|
||||
## Scope
|
||||
|
||||
|
||||
+7
-7
@@ -14,14 +14,14 @@ strix (--target <target> | --target-list <path>) [options]
|
||||
<ParamField path="--target, -t" type="string">
|
||||
Target to test. Accepts URLs, repositories, local directories, domains, IP addresses, API spec files (OpenAPI/Swagger `.json`/`.yaml`, a Postman collection export), or a live Postman collection by id (`postman://<collection-uuid>`). Can be specified multiple times. Fresh runs require at least one target source: `--target` or `--target-list`.
|
||||
|
||||
When the target is an API spec, Strix copies it into the agent's workspace and authorizes the base URLs it declares (including those resolved from a Postman environment) as in-scope hosts - so the agent reads the contract and tests the full declared surface instead of discovering endpoints by crawling. Pair the spec with the deployed base URL (e.g. `--target ./openapi.yaml --target https://api.example.com`) so the agent has a reachable host to attack.
|
||||
When the target is an API spec, Strix copies it into the agent workspace and authorizes its declared base URLs as in-scope hosts. Strix also authorizes base URLs that it resolves from a Postman environment. The agent then reads the contract and tests the full declared surface instead of finding endpoints by crawling. Pair the spec with the deployed base URL, such as `--target ./openapi.yaml --target https://api.example.com`, so the agent has a reachable host to attack.
|
||||
|
||||
<Note>
|
||||
A local directory is mounted into the sandbox live and **writable**, so the agent edits your real files (`.git` excepted). Commit or stash first.
|
||||
</Note>
|
||||
|
||||
<Note>
|
||||
Fetching a Postman collection by id requires `POSTMAN_API_KEY`. Add `?env=<environment-uuid>` to also pull a Postman environment, which resolves `{{baseUrl}}` / token variables the collection references (e.g. `postman://<collection-uuid>?env=<environment-uid>`).
|
||||
Fetching a Postman collection by ID requires `POSTMAN_API_KEY`. Add `?env=<environment-uuid>` to also fetch a Postman environment, which resolves the `{{baseUrl}}` and token variables that the collection references. Use a target such as `postman://<collection-uuid>?env=<environment-uid>`.
|
||||
</Note>
|
||||
</ParamField>
|
||||
|
||||
@@ -46,7 +46,7 @@ strix (--target <target> | --target-list <path>) [options]
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="--diff-base" type="string">
|
||||
Target branch or commit to compare against (e.g., `origin/main`). Defaults to the repository's default branch.
|
||||
Target branch or commit to compare against, such as `origin/main`. Defaults to the repository's default branch.
|
||||
</ParamField>
|
||||
|
||||
<ParamField path="--non-interactive, -n" type="boolean">
|
||||
@@ -88,8 +88,8 @@ strix (--target <target> | --target-list <path>) [options]
|
||||
slightly overshoot the limit by any calls already in flight when the
|
||||
threshold is crossed (most relevant with several child agents running
|
||||
concurrently).
|
||||
- Cost is a best-effort estimate derived from token usage and model pricing;
|
||||
providers that do not expose priced usage may under-count.
|
||||
- Cost is a best-effort estimate derived from token usage and model pricing.
|
||||
Providers that do not expose priced usage may under-count.
|
||||
- For LiteLLM-routed models, Strix enables streaming success callbacks to
|
||||
capture provider-reported cost. Message content remains excluded, but
|
||||
third-party LiteLLM callbacks configured in the same process can receive
|
||||
@@ -148,6 +148,6 @@ strix --target-list ./targets.txt
|
||||
|
||||
| Code | Meaning |
|
||||
|------|---------|
|
||||
| 0 | Scan completed successfully (interactive mode always exits `0`; in headless mode, `0` means no vulnerabilities were found) |
|
||||
| 1 | A fatal error occurred before or during the scan (e.g. missing environment variables, Docker unavailable, invalid config file, diff-scope resolution failure, or an unhandled error) |
|
||||
| 0 | Interactive mode always exits with `0`. In headless mode, `0` means that no vulnerabilities were found. |
|
||||
| 1 | A fatal error occurred before or during the scan. Causes include missing environment variables, unavailable Docker, an invalid config file, diff-scope resolution failure, or an unhandled error. |
|
||||
| 2 | Vulnerabilities found (headless mode only) |
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "strix-agent"
|
||||
version = "1.5.2"
|
||||
version = "1.5.3"
|
||||
description = "Open-source AI Hackers for your apps"
|
||||
readme = "README.md"
|
||||
license = "Apache-2.0"
|
||||
|
||||
@@ -652,27 +652,31 @@ def _install_openrouter_stream_cost_capture() -> None:
|
||||
litellm.OpenrouterConfig = _StrixOpenrouterConfig # type: ignore[misc]
|
||||
|
||||
|
||||
_OPENROUTER_ATTRIBUTION_HEADERS = {
|
||||
OPENROUTER_ATTRIBUTION_HEADERS = {
|
||||
"HTTP-Referer": "https://strix.ai",
|
||||
"X-Title": "Strix",
|
||||
"X-OpenRouter-Categories": "cli-agent",
|
||||
}
|
||||
|
||||
|
||||
def is_openrouter_model(model_name: str | None) -> bool:
|
||||
return bool(model_name) and "openrouter/" in (model_name or "").strip().lower()
|
||||
|
||||
|
||||
def _configure_openrouter_attribution(model_name: str | None) -> None:
|
||||
import litellm
|
||||
|
||||
current: object = litellm.headers
|
||||
existing: dict[str, str] = current if isinstance(current, dict) else {}
|
||||
if not model_name or "openrouter/" not in model_name.strip().lower():
|
||||
if any(key in existing for key in _OPENROUTER_ATTRIBUTION_HEADERS):
|
||||
if not is_openrouter_model(model_name):
|
||||
if any(key in existing for key in OPENROUTER_ATTRIBUTION_HEADERS):
|
||||
remaining = {
|
||||
k: v for k, v in existing.items() if k not in _OPENROUTER_ATTRIBUTION_HEADERS
|
||||
k: v for k, v in existing.items() if k not in OPENROUTER_ATTRIBUTION_HEADERS
|
||||
}
|
||||
litellm.headers = remaining or None # type: ignore[assignment]
|
||||
return
|
||||
|
||||
litellm.headers = {**existing, **_OPENROUTER_ATTRIBUTION_HEADERS} # type: ignore[assignment]
|
||||
litellm.headers = {**existing, **OPENROUTER_ATTRIBUTION_HEADERS} # type: ignore[assignment]
|
||||
|
||||
|
||||
def _configure_extra_headers(llm: LlmSettings) -> None:
|
||||
|
||||
+15
-1
@@ -10,10 +10,12 @@ from openai.types.shared import Reasoning
|
||||
|
||||
from strix.config.models import (
|
||||
DEFAULT_MODEL_RETRY,
|
||||
OPENROUTER_ATTRIBUTION_HEADERS,
|
||||
bedrock_route_supports_prompt_caching,
|
||||
is_bedrock_route,
|
||||
is_claude_model,
|
||||
is_known_openai_bare_model,
|
||||
is_openrouter_model,
|
||||
model_supports_reasoning,
|
||||
request_timeout_extra_args,
|
||||
)
|
||||
@@ -203,12 +205,13 @@ def make_model_settings(
|
||||
extra_headers: dict[str, str] | None = None,
|
||||
has_tools: bool = True,
|
||||
) -> ModelSettings:
|
||||
headers = _request_headers(model_name, extra_headers)
|
||||
model_settings = ModelSettings(
|
||||
parallel_tool_calls=False if has_tools else None,
|
||||
retry=DEFAULT_MODEL_RETRY,
|
||||
include_usage=True,
|
||||
extra_args=request_timeout_extra_args(request_timeout),
|
||||
extra_headers=dict(extra_headers) if extra_headers else None,
|
||||
extra_headers=headers,
|
||||
)
|
||||
if (
|
||||
reasoning_effort is not None
|
||||
@@ -231,6 +234,17 @@ def make_model_settings(
|
||||
return model_settings
|
||||
|
||||
|
||||
def _request_headers(
|
||||
model_name: str, extra_headers: dict[str, str] | None
|
||||
) -> dict[str, str] | None:
|
||||
headers: dict[str, str] = {}
|
||||
if is_openrouter_model(model_name):
|
||||
headers.update(OPENROUTER_ATTRIBUTION_HEADERS)
|
||||
if extra_headers:
|
||||
headers.update(extra_headers)
|
||||
return headers or None
|
||||
|
||||
|
||||
def _reasoning_settings(
|
||||
effort: ReasoningEffort,
|
||||
extra_args: dict[str, Any] | None,
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
"""LiteLLM model-name resolution for local cost estimates."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from functools import lru_cache
|
||||
from typing import Any, cast
|
||||
|
||||
|
||||
@lru_cache(maxsize=512)
|
||||
def resolve_litellm_model(model: str) -> str | None:
|
||||
"""Return a provider-qualified model name that LiteLLM can price."""
|
||||
try:
|
||||
import litellm
|
||||
|
||||
normalized = model.strip()
|
||||
for prefix in ("litellm/", "any-llm/", "openai/"):
|
||||
if normalized.startswith(prefix):
|
||||
normalized = normalized.removeprefix(prefix)
|
||||
break
|
||||
if not normalized:
|
||||
return None
|
||||
|
||||
model_cost = cast(
|
||||
"dict[str, dict[str, Any]]",
|
||||
getattr(litellm, "model_cost"), # noqa: B009
|
||||
)
|
||||
bare_entry = model_cost.get(normalized)
|
||||
if "/" not in normalized and isinstance(bare_entry, dict):
|
||||
provider = bare_entry.get("litellm_provider")
|
||||
if isinstance(provider, str) and provider:
|
||||
return f"{provider}/{normalized}"
|
||||
if "/" in normalized and isinstance(bare_entry, dict):
|
||||
return normalized
|
||||
|
||||
names = [normalized]
|
||||
if "/" in normalized:
|
||||
names.append(normalized.rsplit("/", 1)[-1])
|
||||
for name in names:
|
||||
matches = sorted(key for key in model_cost if key.endswith(f"/{name}"))
|
||||
if not matches:
|
||||
continue
|
||||
prices = {
|
||||
(
|
||||
model_cost[key].get("input_cost_per_token"),
|
||||
model_cost[key].get("output_cost_per_token"),
|
||||
)
|
||||
for key in matches
|
||||
if isinstance(model_cost.get(key), dict)
|
||||
}
|
||||
if len(matches) == 1 or len(prices) == 1:
|
||||
return matches[0]
|
||||
return None # noqa: TRY300
|
||||
except Exception: # noqa: BLE001
|
||||
return None
|
||||
@@ -14,6 +14,7 @@ from agents.usage import Usage
|
||||
from strix.config import codex
|
||||
from strix.config.loader import load_settings
|
||||
from strix.core.paths import run_dir_for
|
||||
from strix.report.pricing import resolve_litellm_model
|
||||
from strix.report.sarif import write_sarif
|
||||
from strix.report.usage import LLMUsageLedger
|
||||
from strix.report.writer import (
|
||||
@@ -696,10 +697,13 @@ def _estimate_response_cost(kwargs: Any, completion_response: Any) -> float | No
|
||||
candidates.append(model.rsplit("/", 1)[-1])
|
||||
|
||||
for candidate in candidates:
|
||||
resolved = resolve_litellm_model(candidate)
|
||||
if not resolved:
|
||||
continue
|
||||
try:
|
||||
value = completion_cost(
|
||||
completion_response={"model": candidate, "usage": usage_payload},
|
||||
model=candidate,
|
||||
completion_response={"model": resolved, "usage": usage_payload},
|
||||
model=resolved,
|
||||
)
|
||||
except Exception: # nosec B112 # noqa: BLE001, S112
|
||||
continue
|
||||
|
||||
+30
-29
@@ -7,6 +7,8 @@ from typing import Any
|
||||
|
||||
from agents.usage import Usage, deserialize_usage, serialize_usage
|
||||
|
||||
from strix.report.pricing import resolve_litellm_model
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -18,7 +20,9 @@ class LLMUsageLedger:
|
||||
self._total_usage = Usage()
|
||||
self._agent_usage: dict[str, Usage] = {}
|
||||
self._agent_metadata: dict[str, dict[str, str]] = {}
|
||||
self._total_cost = 0.0
|
||||
self._observed_cost = 0.0
|
||||
self._estimated_cost = 0.0
|
||||
self._has_observed_cost = False
|
||||
# When True, tokens are still tracked but cost stays $0 — the run is on a
|
||||
# model subscription, so there is no metered per-token charge to report.
|
||||
self.zero_cost = False
|
||||
@@ -44,10 +48,10 @@ class LLMUsageLedger:
|
||||
if model:
|
||||
metadata["model"] = model
|
||||
|
||||
if not self.zero_cost and not _is_litellm_routed(model):
|
||||
if not self.zero_cost:
|
||||
estimated = _estimate_litellm_cost(usage, model)
|
||||
if estimated:
|
||||
self._total_cost += estimated
|
||||
self._estimated_cost += estimated
|
||||
|
||||
return True
|
||||
|
||||
@@ -55,15 +59,18 @@ class LLMUsageLedger:
|
||||
if self.zero_cost:
|
||||
return
|
||||
if isinstance(cost, int | float) and cost > 0:
|
||||
self._total_cost += float(cost)
|
||||
self._observed_cost += float(cost)
|
||||
self._has_observed_cost = True
|
||||
|
||||
@property
|
||||
def total_cost(self) -> float:
|
||||
return _round_cost(self._total_cost)
|
||||
if self.zero_cost:
|
||||
return 0.0
|
||||
return _round_cost(self._observed_cost if self._has_observed_cost else self._estimated_cost)
|
||||
|
||||
def to_record(self) -> dict[str, Any]:
|
||||
record = serialize_usage(self._total_usage)
|
||||
record["cost"] = _round_cost(self._total_cost)
|
||||
record["cost"] = self.total_cost
|
||||
record["agents"] = []
|
||||
|
||||
agent_tokens = {aid: _resolve_total_tokens(u) for aid, u in self._agent_usage.items()}
|
||||
@@ -72,7 +79,7 @@ class LLMUsageLedger:
|
||||
usage = self._agent_usage[agent_id]
|
||||
metadata = self._agent_metadata.get(agent_id, {})
|
||||
agent_cost = (
|
||||
self._total_cost * (agent_tokens[agent_id] / total_tokens) if total_tokens else 0.0
|
||||
self.total_cost * (agent_tokens[agent_id] / total_tokens) if total_tokens else 0.0
|
||||
)
|
||||
|
||||
agent_record = serialize_usage(usage)
|
||||
@@ -92,7 +99,9 @@ class LLMUsageLedger:
|
||||
self._total_usage = Usage()
|
||||
self._agent_usage.clear()
|
||||
self._agent_metadata.clear()
|
||||
self._total_cost = 0.0
|
||||
self._observed_cost = 0.0
|
||||
self._estimated_cost = 0.0
|
||||
self._has_observed_cost = False
|
||||
|
||||
if not isinstance(raw_usage, dict):
|
||||
return
|
||||
@@ -103,7 +112,9 @@ class LLMUsageLedger:
|
||||
logger.exception("Failed to hydrate aggregate llm_usage from run.json")
|
||||
self._total_usage = Usage()
|
||||
|
||||
self._total_cost = _float_or_zero(raw_usage.get("cost"))
|
||||
persisted_cost = _float_or_zero(raw_usage.get("cost"))
|
||||
self._observed_cost = persisted_cost
|
||||
self._estimated_cost = persisted_cost
|
||||
|
||||
for raw_agent in raw_usage.get("agents") or []:
|
||||
if not isinstance(raw_agent, dict):
|
||||
@@ -136,15 +147,6 @@ def _resolve_total_tokens(usage: Usage) -> int:
|
||||
return prompt + completion
|
||||
|
||||
|
||||
def _is_litellm_routed(model: str | None) -> bool:
|
||||
if not model:
|
||||
return False
|
||||
name = model.strip().lower()
|
||||
if "/" not in name:
|
||||
return False
|
||||
return not name.startswith("openai/")
|
||||
|
||||
|
||||
def _usage_has_activity(usage: Usage) -> bool:
|
||||
return bool(
|
||||
usage.requests
|
||||
@@ -201,24 +203,23 @@ def _estimate_litellm_entry_cost(entry: Any, model: str) -> float | None:
|
||||
|
||||
candidates = [model]
|
||||
if "/" in model:
|
||||
candidates.append(model.split("/", 1)[-1])
|
||||
candidates.append(model.rsplit("/", 1)[-1])
|
||||
|
||||
cost: Any = None
|
||||
for candidate in candidates:
|
||||
resolved = resolve_litellm_model(candidate)
|
||||
if not resolved:
|
||||
continue
|
||||
try:
|
||||
cost = completion_cost(
|
||||
completion_response={"model": candidate, "usage": usage_payload},
|
||||
model=model,
|
||||
completion_response={"model": resolved, "usage": usage_payload},
|
||||
model=resolved,
|
||||
)
|
||||
break
|
||||
except Exception: # nosec B112 # noqa: BLE001, S112
|
||||
continue
|
||||
|
||||
if cost is None:
|
||||
logger.debug("LiteLLM cost estimate unavailable for model %s", model)
|
||||
return None
|
||||
|
||||
return cost if isinstance(cost, int | float) and cost >= 0 else None
|
||||
if cost > 0:
|
||||
return float(cost)
|
||||
logger.debug("LiteLLM cost estimate unavailable for model %s", model)
|
||||
return None
|
||||
|
||||
|
||||
def _litellm_model_name(model: str | None) -> str | None:
|
||||
|
||||
@@ -143,7 +143,7 @@ def test_cost_callback_estimates_cost_with_bare_model_fallback() -> None:
|
||||
}
|
||||
|
||||
def fake_completion_cost(**kwargs: object) -> float:
|
||||
if kwargs["model"] == "gpt-4o-mini":
|
||||
if kwargs["model"] == "openai/gpt-4o-mini":
|
||||
return 0.025
|
||||
raise ValueError(kwargs["model"])
|
||||
|
||||
|
||||
@@ -361,3 +361,32 @@ def test_make_model_settings_timeout_survives_reasoning_resolve() -> None:
|
||||
|
||||
assert settings.extra_args is not None
|
||||
assert settings.extra_args["timeout"] == 120.0
|
||||
|
||||
|
||||
def test_openrouter_attribution_rides_on_the_request_headers() -> None:
|
||||
# litellm.headers is ignored once a request carries any header of its own,
|
||||
# so the attribution must be part of the per-request headers.
|
||||
headers = make_model_settings(
|
||||
None, model_name="openrouter/anthropic/claude-sonnet-4-5"
|
||||
).extra_headers
|
||||
assert headers == {
|
||||
"HTTP-Referer": "https://strix.ai",
|
||||
"X-Title": "Strix",
|
||||
"X-OpenRouter-Categories": "cli-agent",
|
||||
}
|
||||
|
||||
|
||||
def test_openrouter_attribution_absent_for_other_providers() -> None:
|
||||
assert make_model_settings(None, model_name="anthropic/claude-sonnet-4-5").extra_headers is None
|
||||
|
||||
|
||||
def test_user_headers_override_openrouter_attribution() -> None:
|
||||
headers = make_model_settings(
|
||||
None,
|
||||
model_name="openrouter/anthropic/claude-sonnet-4-5",
|
||||
extra_headers={"X-Title": "Custom", "X-Tenant": "acme"},
|
||||
).extra_headers
|
||||
assert headers is not None
|
||||
assert headers["X-Title"] == "Custom"
|
||||
assert headers["X-Tenant"] == "acme"
|
||||
assert headers["HTTP-Referer"] == "https://strix.ai"
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import litellm
|
||||
from agents.usage import Usage
|
||||
|
||||
from strix.report.pricing import resolve_litellm_model
|
||||
from strix.report.usage import LLMUsageLedger
|
||||
|
||||
|
||||
def test_resolves_common_bare_model_names() -> None:
|
||||
resolve_litellm_model.cache_clear()
|
||||
assert resolve_litellm_model("deepseek-v4-flash") == "deepseek/deepseek-v4-flash"
|
||||
assert resolve_litellm_model("openai/deepseek-v4-flash") == "deepseek/deepseek-v4-flash"
|
||||
assert resolve_litellm_model("grok-4.5") == "xai/grok-4.5"
|
||||
assert resolve_litellm_model("MiniMax-M3") == "minimax/MiniMax-M3"
|
||||
|
||||
|
||||
def test_resolver_returns_none_for_unresolvable_model() -> None:
|
||||
resolve_litellm_model.cache_clear()
|
||||
assert resolve_litellm_model("provider/not-a-real-model") is None
|
||||
|
||||
|
||||
def test_ledger_uses_estimate_when_routed_provider_reports_no_cost() -> None:
|
||||
usage = Usage()
|
||||
usage.requests = 1
|
||||
usage.input_tokens = 1000
|
||||
usage.output_tokens = 200
|
||||
usage.total_tokens = 1200
|
||||
ledger = LLMUsageLedger()
|
||||
|
||||
with patch("litellm.completion_cost", return_value=0.42):
|
||||
ledger.record(agent_id="a", usage=usage, model="openai/deepseek-v4-flash")
|
||||
|
||||
assert ledger.total_cost == 0.42
|
||||
|
||||
|
||||
def test_ledger_prefers_observed_cost_over_estimate() -> None:
|
||||
usage = Usage()
|
||||
usage.requests = 1
|
||||
usage.input_tokens = 1000
|
||||
usage.output_tokens = 200
|
||||
usage.total_tokens = 1200
|
||||
ledger = LLMUsageLedger()
|
||||
|
||||
with patch("litellm.completion_cost", return_value=0.42):
|
||||
ledger.record(agent_id="a", usage=usage, model="openai/deepseek-v4-flash")
|
||||
ledger.record_observed_cost(0.17)
|
||||
|
||||
assert ledger.total_cost == 0.17
|
||||
|
||||
|
||||
def test_hydrated_estimate_continues_accumulating_new_estimates() -> None:
|
||||
usage = Usage()
|
||||
usage.requests = 1
|
||||
usage.input_tokens = 1000
|
||||
usage.output_tokens = 200
|
||||
usage.total_tokens = 1200
|
||||
ledger = LLMUsageLedger()
|
||||
ledger.hydrate({"cost": 0.42})
|
||||
|
||||
with patch("litellm.completion_cost", return_value=0.17):
|
||||
ledger.record(agent_id="a", usage=usage, model="openai/deepseek-v4-flash")
|
||||
|
||||
assert ledger.total_cost == 0.59
|
||||
|
||||
|
||||
def test_zero_cost_disables_both_observed_and_estimated_costs() -> None:
|
||||
usage = Usage()
|
||||
usage.requests = 1
|
||||
usage.input_tokens = 1000
|
||||
usage.output_tokens = 200
|
||||
usage.total_tokens = 1200
|
||||
ledger = LLMUsageLedger()
|
||||
ledger.zero_cost = True
|
||||
|
||||
with patch("litellm.completion_cost", return_value=0.42) as estimate:
|
||||
ledger.record(agent_id="a", usage=usage, model="deepseek-v4-flash")
|
||||
ledger.record_observed_cost(1.0)
|
||||
|
||||
estimate.assert_not_called()
|
||||
assert ledger.total_cost == 0.0
|
||||
|
||||
|
||||
def test_resolver_uses_provider_when_bare_entry_has_one() -> None:
|
||||
original = litellm.model_cost
|
||||
litellm.model_cost = {
|
||||
"example": {
|
||||
"litellm_provider": "example-provider",
|
||||
"input_cost_per_token": 1.0,
|
||||
"output_cost_per_token": 2.0,
|
||||
}
|
||||
}
|
||||
try:
|
||||
resolve_litellm_model.cache_clear()
|
||||
assert resolve_litellm_model("example") == "example-provider/example"
|
||||
finally:
|
||||
litellm.model_cost = original
|
||||
resolve_litellm_model.cache_clear()
|
||||
|
||||
|
||||
def test_resolver_does_not_guess_between_differently_priced_providers() -> None:
|
||||
original = litellm.model_cost
|
||||
litellm.model_cost = {
|
||||
"provider-a/example": {
|
||||
"input_cost_per_token": 1.0,
|
||||
"output_cost_per_token": 2.0,
|
||||
},
|
||||
"provider-b/example": {
|
||||
"input_cost_per_token": 3.0,
|
||||
"output_cost_per_token": 4.0,
|
||||
},
|
||||
}
|
||||
try:
|
||||
resolve_litellm_model.cache_clear()
|
||||
assert resolve_litellm_model("example") is None
|
||||
finally:
|
||||
litellm.model_cost = original
|
||||
resolve_litellm_model.cache_clear()
|
||||
Reference in New Issue
Block a user