mirror of
https://github.com/usestrix/strix.git
synced 2026-08-21 02:45:31 +02:00
Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
857b24454d | ||
|
|
7a736da465 |
@@ -15,14 +15,6 @@ npx skills add usestrix/strix
|
||||
- `fix-security-vulnerabilities-with-strix` — remediate findings and re-run Strix to verify
|
||||
- `ci-security-scanning-with-strix` — add PR scanning to CI/CD (self-hosted CLI or managed app)
|
||||
|
||||
Target-specific workflows built on the same engine:
|
||||
|
||||
- `application-security-testing` — whole-product AppSec review: pick the right test per asset, then rank the results
|
||||
- `web-app-penetration-testing` — black-box pentest of a live web app or staging site
|
||||
- `api-security-testing` — REST/GraphQL APIs and the OWASP API Security Top 10 (BOLA/IDOR, authz)
|
||||
- `owasp-top-10-testing` — systematic OWASP Top 10 assessment with honest per-category coverage
|
||||
- `find-security-vulnerabilities-in-code` — white-box review of a repo or working tree
|
||||
|
||||
**Two ways to run, same engine — pick per situation:**
|
||||
|
||||
- **Open-source CLI (self-hosted):** free, fully local, BYO LLM key, needs Docker. Best for local dev loops, air-gapped/offline, and full control.
|
||||
|
||||
@@ -116,7 +116,7 @@ Strix is agent-ready. Give Claude Code, Cursor, Codex, or any [SKILL.md-compatib
|
||||
npx skills add usestrix/strix
|
||||
```
|
||||
|
||||
This installs nine skills: **penetration-testing-with-strix** (run headless scans and read results), **managed-pentesting-with-strix** (drive the managed [app.strix.ai](https://app.strix.ai) platform via REST — no local Docker or LLM key), **fix-security-vulnerabilities-with-strix** (remediate + re-scan to verify), **ci-security-scanning-with-strix** (PR scanning in CI), plus target-specific workflows: **application-security-testing**, **web-app-penetration-testing**, **api-security-testing**, **owasp-top-10-testing**, and **find-security-vulnerabilities-in-code**. Agents can run Strix two ways with the same engine — the open-source CLI locally, or the managed cloud when there's no local infra — and read [`AGENTS.md`](AGENTS.md) for a quick reference, [docs.strix.ai/llms.txt](https://docs.strix.ai/llms.txt) for the CLI docs, and [docs.app.strix.ai](https://docs.app.strix.ai) for the API.
|
||||
This installs four skills: **penetration-testing-with-strix** (run headless scans and read results), **managed-pentesting-with-strix** (drive the managed [app.strix.ai](https://app.strix.ai) platform via REST — no local Docker or LLM key), **fix-security-vulnerabilities-with-strix** (remediate + re-scan to verify), and **ci-security-scanning-with-strix** (PR scanning in CI). Agents can run Strix two ways with the same engine — the open-source CLI locally, or the managed cloud when there's no local infra — and read [`AGENTS.md`](AGENTS.md) for a quick reference, [docs.strix.ai/llms.txt](https://docs.strix.ai/llms.txt) for the CLI docs, and [docs.app.strix.ai](https://docs.app.strix.ai) for the API.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -19,11 +19,6 @@ npx skills add usestrix/strix
|
||||
| `managed-pentesting-with-strix` | Drive the managed [app.strix.ai](https://app.strix.ai) platform over REST — no local Docker or LLM key needed |
|
||||
| `fix-security-vulnerabilities-with-strix` | Triage findings, fix root causes, and re-run Strix to verify each fix |
|
||||
| `ci-security-scanning-with-strix` | Add PR security scanning to GitHub Actions or any CI (self-hosted CLI or managed app) |
|
||||
| `application-security-testing` | Assess a whole product: choose the right test for each asset, then rank the findings into one remediation plan |
|
||||
| `web-app-penetration-testing` | Black-box pentest of a live web app or staging site — scope, credentials, and multi-account access-control testing |
|
||||
| `api-security-testing` | Test a REST/GraphQL API against the OWASP API Security Top 10 — schema-driven enumeration, BOLA/IDOR, authz |
|
||||
| `owasp-top-10-testing` | Systematic OWASP Top 10 assessment with honest per-category coverage |
|
||||
| `find-security-vulnerabilities-in-code` | White-box security review of a repo or working tree, with exploits to confirm findings |
|
||||
|
||||
Install a single skill with `npx skills add usestrix/strix --skill penetration-testing-with-strix`, or use one without installing:
|
||||
|
||||
|
||||
@@ -1,61 +0,0 @@
|
||||
---
|
||||
name: api-security-testing
|
||||
description: Security-test a REST, GraphQL, or gRPC API with Strix — autonomous agents that enumerate endpoints from an OpenAPI/GraphQL schema (or by crawling), then actually exploit the API-specific vulnerability classes in the OWASP API Security Top 10 (2023) — broken object-level authorization (BOLA/IDOR), broken object property level authorization (excessive data exposure and mass assignment), broken function-level authorization, unrestricted resource consumption, SSRF, injection, and auth/token flaws. Every finding comes with a working proof-of-concept request. Use when the user asks to pentest, security-test, audit, or find vulnerabilities in an API, endpoint, or backend service.
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
author: usestrix
|
||||
homepage: https://docs.strix.ai
|
||||
---
|
||||
|
||||
# Security-test an API
|
||||
|
||||
APIs fail differently from web UIs: there is no rendered surface to crawl, the interesting bugs are authorization-shaped rather than injection-shaped, and the same endpoint behaves differently per token. This workflow targets those specifics with Strix's autonomous agents, using the current [OWASP API Security Top 10 (2023)](https://owasp.org/API-Security/editions/2023/en/0x11-t10/) as the coverage checklist. For the web-app equivalent, the current edition is the OWASP Top 10:2025 — see **owasp-top-10-testing**.
|
||||
|
||||
Install, LLM setup, full CLI flags, and the managed-cloud path are in the **penetration-testing-with-strix** skill. Read it if `strix --version` fails or the target is not an API.
|
||||
|
||||
## 1. Gather what the agents need
|
||||
|
||||
APIs are near-impossible to test blind, so collect first:
|
||||
|
||||
| Input | Why it matters |
|
||||
|---|---|
|
||||
| **Schema** — OpenAPI/Swagger file, Postman collection, GraphQL endpoint (introspection), or a gRPC `.proto` | Turns guesswork into full endpoint enumeration. Biggest single win in coverage. An OpenAPI/Swagger or Postman spec (`.json`/`.yaml`/`.yml`) is a target Strix takes directly; a `.proto` is not, so pass it with `--workspace-file`. |
|
||||
| **Two sets of credentials/tokens**, ideally in different tenants | BOLA/IDOR — API1:2023, still the #1 API risk — can only be *proven* by accessing tenant A's objects with tenant B's token. |
|
||||
| **A low-privilege and a high-privilege token** | Required to prove broken function-level authorization (API5:2023 — a `user` calling admin-only routes). |
|
||||
| **Example object IDs** | Lets agents test ID tampering immediately instead of hunting for valid identifiers. |
|
||||
| **Out-of-scope routes** | Payments, mass notification, destructive admin endpoints. |
|
||||
| **Rate limits / WAF** in front of the API | Avoids agents burning budget on throttled requests; mention them so testing adapts. |
|
||||
|
||||
Ask the user for anything missing — do not fabricate tokens or scan an API they do not own.
|
||||
|
||||
## 2. Run the scan
|
||||
|
||||
Pass the spec as a **target**, not as prose in the instruction — Strix parses OpenAPI/Swagger (`.json`/`.yaml`) and Postman collection exports directly, so the agents start from the real endpoint list:
|
||||
|
||||
```bash
|
||||
strix -n -t ./openapi.yaml -t https://api.staging.example.com --max-budget 20 \
|
||||
--instruction "Tenant A token: <tokenA> (org 1111, user id 11, order id 501).
|
||||
Tenant B token: <tokenB> (org 2222, user id 22).
|
||||
Admin token: <tokenAdmin>.
|
||||
Focus: BOLA across orgs (API1), function-level authz on /admin/* (API5), object property level authz on PATCH /users/{id} — both mass assignment and over-exposed fields in list responses (API3), unrestricted resource consumption (API4).
|
||||
Out of scope: POST /billing/*, POST /notifications/broadcast."
|
||||
```
|
||||
|
||||
- **Postman instead of OpenAPI:** a collection export works as a target (`-t ./collection.postman_collection.json`), or pull one live with `-t postman://<collection-uuid>` (optionally `"postman://<collection-uuid>?env=<environment-uuid>"`), which needs `POSTMAN_API_KEY` in the environment.
|
||||
- **Many services at once:** put one target per line in a file and pass `--target-list ./targets.txt`, repeatable and combinable with `-t`.
|
||||
- **Add the backend source for depth:** `-t ./services/api -t https://api.staging.example.com`. With code access the agents can reason about authorization checks and object ownership rather than inferring them from responses.
|
||||
- **gRPC:** target the endpoint and pass the definition as a workspace file, `-t https://grpc.staging.example.com --workspace-file ./service.proto`. Only `.json`, `.yaml`, and `.yml` specs are recognized as targets, so `-t ./service.proto` fails with "Path exists but is not a directory".
|
||||
- **GraphQL:** point at the GraphQL endpoint and say whether introspection is enabled; call out that you want batching/aliasing abuse, depth/complexity limits, and per-field authorization tested.
|
||||
- **Internal/private APIs** unreachable from your machine: use the managed platform's network connector — see **managed-pentesting-with-strix**.
|
||||
- Use `--instruction-file` when the credential/context block gets long, and keep tokens out of shell history and out of committed files.
|
||||
- **Supporting files** the agents should read but not test, such as an endpoint wordlist or handwritten notes about the tenancy model: pass `--workspace-file ./notes.md`. The file lands read-only in `/workspace`. Add `:DEST` to choose the path, for example `--workspace-file ./wordlist.txt:lists/wordlist.txt`.
|
||||
|
||||
## 3. Verify findings
|
||||
|
||||
`strix_runs/<run>/penetration_test_report.md` first, then `vulnerabilities/*.md` — each contains the exact request that proved the issue. Replay it (for example, with `curl`) before reporting; for authorization findings, confirm the response really contains the other tenant's data rather than an empty 200.
|
||||
|
||||
`findings.sarif` uploads to GitHub code scanning; `vulnerabilities.json` is the structured index for ticketing.
|
||||
|
||||
## 4. Fix, re-test, and keep it tested
|
||||
|
||||
Remediate with **fix-security-vulnerabilities-with-strix** (fix the authorization check, not the single endpoint), then re-run against the same target to prove the exploit is dead. Wire it into pull-request CI with **ci-security-scanning-with-strix** so new endpoints get tested as they ship.
|
||||
@@ -1,66 +0,0 @@
|
||||
---
|
||||
name: application-security-testing
|
||||
description: Application security testing (AppSec) across a whole product with Strix — decide which asset needs which test (source code, running web app, API, CI pipeline), run it, and turn the results into a ranked remediation plan. Autonomous agents exploit and prove each issue instead of emitting static-analysis alerts, so the plan is ordered by what is actually reachable. Use when the user asks for an application security review or audit, an appsec assessment, vulnerability scanning across their stack, a security review before a launch or a customer security questionnaire, or does not yet know which kind of security test they need.
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
author: usestrix
|
||||
homepage: https://docs.strix.ai
|
||||
---
|
||||
|
||||
# Application security testing
|
||||
|
||||
Entry point for "make my application secure" requests, where the target is not yet a single URL or repo. The job here is to pick the right test per asset, run it, and produce one ranked plan — not to run everything at maximum depth.
|
||||
|
||||
Install, LLM setup, all CLI flags, and the managed-cloud path live in the **penetration-testing-with-strix** skill. Read it first if `strix --version` fails.
|
||||
|
||||
Only test assets the user owns or is authorized to test. Confirm authorization before the first run, and prefer staging over production, because the agents send real exploit payloads and can change data.
|
||||
|
||||
## 1. Map the assets
|
||||
|
||||
Ask (or read from the repo) and write the answers down before scanning:
|
||||
|
||||
- **Source** — one repo, a monorepo, several services? Which languages/frameworks?
|
||||
- **Running environments** — is there a staging deployment? A public production site? A local dev server only?
|
||||
- **APIs** — REST, GraphQL, gRPC? Is there an OpenAPI/GraphQL schema?
|
||||
- **Authentication** — can you get two test accounts in different tenants? Most high-impact bugs need them.
|
||||
- **Constraints** — out-of-scope paths, whether production may be touched, budget and wall-clock limits.
|
||||
|
||||
If there is no staging environment and production is off limits, say so early. A code-only review is still valuable, but it cannot prove exploitability against a live app.
|
||||
|
||||
## 2. Pick the right test per asset
|
||||
|
||||
| Asset | Skill to use |
|
||||
| --- | --- |
|
||||
| Repository or working tree | **find-security-vulnerabilities-in-code** |
|
||||
| Live web app or staging site | **web-app-penetration-testing** |
|
||||
| REST/GraphQL/gRPC API | **api-security-testing** |
|
||||
| Assessment mapped to OWASP categories | **owasp-top-10-testing** |
|
||||
| Every pull request, continuously | **ci-security-scanning-with-strix** |
|
||||
| No Docker, no LLM key, or a report an auditor will accept | **managed-pentesting-with-strix** |
|
||||
|
||||
Those skills carry the flags, credential handling, and result-reading details. Do not duplicate their instructions here.
|
||||
|
||||
Sequence for a first assessment:
|
||||
|
||||
1. Review the code. It is the cheapest run and it maps the authorization model.
|
||||
2. Pentest staging with credentials, and pass the repo as a second target so the agents keep source context.
|
||||
3. Add CI scanning, so later regressions are caught without another manual pass.
|
||||
|
||||
Run one asset at a time and read each report before starting the next. Findings from the code review make the live run sharper.
|
||||
|
||||
## 3. Consolidate into one plan
|
||||
|
||||
Findings arrive per run in `strix_runs/<run>/`. Merge them into a single list and rank by **proven impact**, not by scanner severity:
|
||||
|
||||
1. Validated exploits reachable without authentication.
|
||||
2. Validated cross-tenant or privilege-escalation issues.
|
||||
3. Validated issues needing an authenticated account.
|
||||
4. Unproven observations (configuration, dependency, and hardening notes) — flag as such, and never present them as confirmed vulnerabilities.
|
||||
|
||||
Deduplicate: the same root cause often surfaces in both the code review and the live pentest.
|
||||
|
||||
## 4. Be honest about coverage
|
||||
|
||||
State plainly what was *not* tested — assets with no staging environment, categories a black-box run cannot reach (logging and alerting, supply-chain integrity, insecure design), and any run that hit its budget or turn cap before finishing. Check `run.json` status and cost against `--max-budget` for each run. An empty result set from a truncated scan is not a clean bill of health.
|
||||
|
||||
Then remediate with **fix-security-vulnerabilities-with-strix**, which re-runs Strix against each fix to prove the exploit no longer works.
|
||||
@@ -12,7 +12,7 @@ metadata:
|
||||
You can gate PRs two ways — pick based on the environment, or combine them:
|
||||
|
||||
- **Managed platform (recommended for most teams)** — connect the GitHub/GitLab/Bitbucket app once and Strix reviews every PR with **no workflow file, no runner, no Docker, and no LLM key**. Results post as PR comments and land in the team dashboard. Best when you want zero CI maintenance, central tracking, or your runners lack Docker. See "Managed platform" below and the **managed-pentesting-with-strix** skill.
|
||||
- **Self-hosted OSS CLI in your runner** — run a diff-scoped scan as a pipeline step. Fully in your infra, free (BYO LLM key), no external account. Requires Docker on the runner. Best for air-gapped/self-hosted CI or when you do not want scans leaving your environment.
|
||||
- **Self-hosted OSS CLI in your runner** — run a diff-scoped scan as a pipeline step. Fully in your infra, free (BYO LLM key), no external account. Requires Docker on the runner. Best for air-gapped/self-hosted CI or when you don't want scans leaving your environment.
|
||||
|
||||
Both fail the build on validated findings and both emit SARIF 2.1.0, so you can start with one and add the other later.
|
||||
|
||||
@@ -63,13 +63,13 @@ jobs:
|
||||
fi
|
||||
```
|
||||
|
||||
Then tell the user to add two repository secrets: `STRIX_LLM` (model id, for example `openai/gpt-5.4`) and `LLM_API_KEY` (the provider key). Do not create these values yourself.
|
||||
Then tell the user to add two repository secrets: `STRIX_LLM` (model id, e.g. `openai/gpt-5.4`) and `LLM_API_KEY` (the provider key). Do not create these values yourself.
|
||||
|
||||
Notes:
|
||||
- In CI/headless runs Strix automatically scopes to the PR's changed files (`--scope-mode auto`). If diff resolution fails, keep `fetch-depth: 0` or set `--diff-base` to the PR's actual base branch — use `origin/${{ github.base_ref }}` in GitHub Actions rather than a hard-coded `origin/main`, since repos use different default branches.
|
||||
- Exit codes: `0` pass, `2` vulnerabilities found (fails the job), `1` setup error.
|
||||
- The runner needs Docker (default GitHub-hosted Ubuntu runners have it).
|
||||
- **Size the budget so the scan completes — do not let it fail open.** A `0` exit means "no validated vulnerabilities in what was analyzed"; if `--max-budget` is hit before the diff is fully covered, the scan wraps up early and can still exit `0`. The "Fail unless the scan completed" step above narrows the gap: `strix_runs/<run>/run.json` is `"stopped"` when the scan was cut off at the hard budget limit without a final report. It is not a complete guard — the agents get graduated wrap-up warnings before that limit, and a run that wraps up on a warning still calls `finish_scan` and records `"completed"` with partial coverage. So keep that step in any pipeline that gates merges **and** give the scan real headroom (compare `run.json`'s `llm_usage.cost` against `--max-budget`; if it ran right up to the cap, raise it). For a `quick` diff-scoped PR scan `--max-budget 10` is usually ample, raise it for large diffs.
|
||||
- **Size the budget so the scan completes — don't let it fail open.** A `0` exit means "no validated vulnerabilities in what was analyzed"; if `--max-budget` is hit before the diff is fully covered, the scan wraps up early and can still exit `0`. The "Fail unless the scan completed" step above narrows the gap: `strix_runs/<run>/run.json` is `"stopped"` when the scan was cut off at the hard budget limit without a final report. It is not a complete guard — the agents get graduated wrap-up warnings before that limit, and a run that wraps up on a warning still calls `finish_scan` and records `"completed"` with partial coverage. So keep that step in any pipeline that gates merges **and** give the scan real headroom (compare `run.json`'s `llm_usage.cost` against `--max-budget`; if it ran right up to the cap, raise it). For a `quick` diff-scoped PR scan `--max-budget 10` is usually ample, raise it for large diffs.
|
||||
|
||||
### Optional: upload findings to GitHub code scanning
|
||||
|
||||
@@ -90,7 +90,7 @@ Any pipeline works the same way — install, set the two env vars, run headless:
|
||||
```bash
|
||||
curl -sSL https://strix.ai/install | bash
|
||||
# Resolve the PR's base branch robustly (use your CI's base-branch variable if it
|
||||
# has one, for example GitHub Actions: origin/${{ github.base_ref }}). Avoid piping the
|
||||
# has one, e.g. GitHub Actions: origin/${{ github.base_ref }}). Avoid piping the
|
||||
# git lookup into another command — a failed lookup would otherwise be masked.
|
||||
BASE_BRANCH="${CI_MERGE_REQUEST_TARGET_BRANCH_NAME:-}" # GitLab MR target
|
||||
if [ -z "$BASE_BRANCH" ]; then
|
||||
@@ -98,7 +98,7 @@ if [ -z "$BASE_BRANCH" ]; then
|
||||
BASE_BRANCH="${BASE_BRANCH#origin/}"
|
||||
fi
|
||||
DIFF_BASE="origin/${BASE_BRANCH:-main}"
|
||||
# Fail loudly rather than silently narrowing scope (for example, to HEAD~1, which on a
|
||||
# Fail loudly rather than silently narrowing scope (e.g. to HEAD~1, which on a
|
||||
# multi-commit branch would scan only the last commit and let earlier ones pass).
|
||||
if ! git rev-parse --verify --quiet "$DIFF_BASE" >/dev/null; then
|
||||
echo "Cannot resolve diff base '$DIFF_BASE'. Fetch the base branch (git fetch origin <base>) or set --diff-base explicitly." >&2
|
||||
|
||||
@@ -1,62 +0,0 @@
|
||||
---
|
||||
name: find-security-vulnerabilities-in-code
|
||||
description: Find security vulnerabilities in a codebase or repository with Strix — a white-box AI security review that reads your source, reasons about the actual data flow and authorization model, then exploits what it finds in a live sandbox so every reported issue has a working proof-of-concept instead of a noisy static-analysis alert. Covers injection, XSS, SSRF, broken access control and IDOR, insecure deserialization, secrets in code, unsafe dependencies, and business-logic flaws. Use when the user asks to security-scan, security-review, or audit their code, repo, or pull request for vulnerabilities.
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
author: usestrix
|
||||
homepage: https://docs.strix.ai
|
||||
---
|
||||
|
||||
# Find security vulnerabilities in code
|
||||
|
||||
White-box security review with Strix: the agents read the source to build a model of routes, sinks, and authorization checks, then attempt real exploitation. Findings come with a proof-of-concept, so the output is a short list of proven issues rather than the hundreds of "potential" hits a pattern-matching scanner produces.
|
||||
|
||||
Install, LLM setup, all flags, and the managed-cloud path are in the **penetration-testing-with-strix** skill.
|
||||
|
||||
## Run it
|
||||
|
||||
```bash
|
||||
# Local working tree
|
||||
strix -n -t ./ --scan-mode standard --max-budget 15
|
||||
|
||||
# A GitHub repo directly
|
||||
strix -n -t https://github.com/org/app --max-budget 15
|
||||
|
||||
# Monorepo: point at the service that matters, not the whole tree
|
||||
strix -n -t ./services/checkout --max-budget 20
|
||||
|
||||
# Only what a branch changed (whole-repo review is wasteful on a large repo)
|
||||
strix -n -t ./ --scope-mode diff --diff-base origin/main --max-budget 10
|
||||
```
|
||||
|
||||
A local path is mounted into the sandbox **writable**, so the agents can modify it. Run against a clean checkout.
|
||||
|
||||
Two things sharply improve results:
|
||||
|
||||
1. **Add a running instance of the app.** `-t ./ -t http://host.docker.internal:3000` lets the agents confirm exploitability against live behavior instead of reasoning about it statically — this is the difference between "this looks unsafe" and a validated finding. If nothing is running, static-only findings should be described as unconfirmed.
|
||||
2. **Scope the review.** Point at the risky subtree and say what matters:
|
||||
```bash
|
||||
strix -n -t ./services/api --max-budget 15 \
|
||||
--instruction "Focus on the authorization layer in src/auth and every route under src/routes/admin. Multi-tenant app: tenant id comes from the JWT. Flag any query that filters by object id without also filtering by tenant."
|
||||
```
|
||||
Tenancy model, trust boundaries, and which inputs are attacker-controlled are things the agents cannot infer reliably — tell them.
|
||||
|
||||
## Reviewing a pull request instead of the whole repo
|
||||
|
||||
For diff-scoped review of a branch or PR (and blocking merges on findings), use **ci-security-scanning-with-strix** — it covers diff scoping, PR comments, and SARIF upload to GitHub code scanning. The managed platform can also review PRs directly via API (**managed-pentesting-with-strix**).
|
||||
|
||||
## Read the results
|
||||
|
||||
In `strix_runs/<run>/`: `penetration_test_report.md` (start here), `vulnerabilities/*.md` (one per finding, with PoC and remediation), `vulnerabilities.json` / `.csv`, `findings.sarif` (upload to code scanning), `run.json`.
|
||||
|
||||
Before reporting to the user, open each finding and check the PoC actually demonstrates impact. Report file and line alongside the exploit so the fix is obvious.
|
||||
|
||||
Exit `0` means nothing exploitable was proven in what was analyzed — not that the codebase is clean. Check `run.json` status and cost against `--max-budget`, and note which paths went unreviewed if the run was capped.
|
||||
|
||||
## Complementary tooling
|
||||
|
||||
This is exploit-validated review, not an exhaustive inventory. Keep a dependency scanner (SCA) and secret scanning in place for complete coverage of known-CVE dependencies and committed credentials; use this for the logic, authorization, and injection bugs those tools structurally cannot find.
|
||||
|
||||
## Fix and verify
|
||||
|
||||
Hand results to **fix-security-vulnerabilities-with-strix**: patch the root cause (the shared authorization helper, not the one route), then re-run Strix to prove the exploit no longer works.
|
||||
@@ -27,7 +27,7 @@ Order work by severity: critical → high → medium → low. Every Strix findin
|
||||
For each finding:
|
||||
|
||||
1. Reproduce it with the PoC from the finding file when feasible.
|
||||
2. Fix the root cause, not the specific payload (parameterize every query instead of blocking one string, and enforce authorization in the handler instead of hiding the endpoint).
|
||||
2. Fix the root cause, not the specific payload (e.g. parameterize all queries, don't blocklist one string; enforce authorization in the handler, don't hide the endpoint).
|
||||
3. Prefer the framework's built-in defense (ORM parameterization, template auto-escaping, CSRF middleware, centralized authz) over ad-hoc sanitization.
|
||||
4. Keep the diff minimal and apply the repo's existing patterns. Finding files often include `fix_before`/`fix_after` snippets — use them as a starting point, not verbatim.
|
||||
|
||||
@@ -70,7 +70,7 @@ new_id=$(curl -sS "$BASE/scans/$scan_id/rerun" "${auth[@]}" -X POST | jq -r .sca
|
||||
Or, if the cloud scan came from a repo/PR, trigger a fresh PR review on the fix branch (`POST /pr-reviews/start`). The platform also retests a single finding directly: `POST /api/v1/vulnerabilities/{vulnerabilityId}/retest`.
|
||||
|
||||
- Also re-run the PoC manually when it is a simple request/script — fastest signal.
|
||||
- Run the project's own test suite to make sure the fix does not break behavior.
|
||||
- Run the project's own test suite to make sure the fix doesn't break behavior.
|
||||
|
||||
## 4. Report
|
||||
|
||||
|
||||
@@ -80,7 +80,7 @@ Useful `CreateScanRequest` fields:
|
||||
| `domain_ids` / `repository_ids` / `internal_targets` | targets (at least one) |
|
||||
| `domain_paths` / `repository_branches` | narrow to specific paths / branches |
|
||||
| `credentials` | authenticated scanning, incl. `mfa_method` (`totp`/`email_otp`/…) + `totp_secret` |
|
||||
| `headers` | extra HTTP headers (API keys, for example) for the target |
|
||||
| `headers` | extra HTTP headers (e.g. API keys) for the target |
|
||||
| `focus` / `concerns` / `context` | steer the agents |
|
||||
| `upload_ids` | attach uploaded source/docs archives for white-box context |
|
||||
| `notify_on_completion` / `notification_emails` | email when done |
|
||||
@@ -89,7 +89,7 @@ Response is `{ scan_id, title, status }` with `status` = `pending`.
|
||||
|
||||
## 3. Poll to completion
|
||||
|
||||
`GET /scans/{scanId}` (`scans:read`). Status flow: `pending → running → completed` (or `failed` / `cancelled`). Poll on an interval — scans take minutes to hours. Do not block.
|
||||
`GET /scans/{scanId}` (`scans:read`). Status flow: `pending → running → completed` (or `failed` / `cancelled`). Poll on an interval — scans take minutes to hours; don't block.
|
||||
|
||||
```bash
|
||||
while :; do
|
||||
@@ -143,10 +143,10 @@ List/inspect via `GET /pr-reviews` and `GET /pr-reviews/{id}`. Repo-level PR-rev
|
||||
## 7. Continuous testing (schedules & webhooks)
|
||||
|
||||
- **Schedules** (`schedules:write`, Pro plan): create recurring scans and trigger them on demand — the managed equivalent of a cron-driven CLI loop.
|
||||
- **Webhooks** (`webhooks:write`): subscribe to pentest/vulnerability lifecycle events such as `scan.completed` and `vulnerability.created` to push results into Slack, ticketing, or your own pipeline instead of polling.
|
||||
- **Webhooks** (`webhooks:write`): subscribe to pentest/vulnerability lifecycle events (e.g. `scan.completed`, `vulnerability.created`) to push results into Slack, ticketing, or your own pipeline instead of polling.
|
||||
|
||||
See the schedules and webhooks sections at [docs.app.strix.ai](https://docs.app.strix.ai) for payloads.
|
||||
|
||||
## Safety
|
||||
|
||||
Only scan assets the user's organization owns or is authorized to test. External domain scans require verification (DNS/file/meta-tag) enforced by the platform — do not try to bypass it.
|
||||
Only scan assets the user's organization owns or is authorized to test. External domain scans require verification (DNS/file/meta-tag) enforced by the platform — don't try to bypass it.
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
---
|
||||
name: owasp-top-10-testing
|
||||
description: Test an application against the OWASP Top 10 with Strix — autonomous AI agents that attempt real exploits for each category of the current OWASP Top 10:2025 (broken access control including SSRF, security misconfiguration, software supply chain failures, cryptographic failures, injection, insecure design, authentication failures, integrity failures, logging and alerting failures, mishandling of exceptional conditions) and report only what they could actually prove, mapped back to the category with a proof-of-concept. Also covers the OWASP API Security Top 10 (2023). Use when the user asks for an OWASP Top 10 assessment, OWASP compliance testing, or a security review mapped to OWASP categories.
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
author: usestrix
|
||||
homepage: https://docs.strix.ai
|
||||
---
|
||||
|
||||
# Test against the OWASP Top 10
|
||||
|
||||
The OWASP Top 10 is a taxonomy of risk categories, not a test suite — "OWASP Top 10 testing" means exercising each category against the real application and reporting what's actually exploitable. Strix's agents do the exploitation; this skill covers running it category-by-category and reporting coverage honestly.
|
||||
|
||||
**Use the current edition: [OWASP Top 10:2025](https://owasp.org/Top10/)** (8th installment, superseding 2021). Ask the user before targeting an older edition — some compliance checklists still reference 2021, and a report labelled with the wrong edition is misleading. Key differences from 2021: **SSRF is folded into A01**, **A03 Software Supply Chain Failures** expands the old "Vulnerable and Outdated Components", and **A10 Mishandling of Exceptional Conditions** is new; A02 Security Misconfiguration moved 5→2.
|
||||
|
||||
Install, LLM setup, and the managed-cloud alternative: **penetration-testing-with-strix**.
|
||||
|
||||
## What is and is not testable by an agent
|
||||
|
||||
Be straight with the user about this — claiming a clean sweep of all ten is misleading.
|
||||
|
||||
| Category (2025) | Coverage |
|
||||
|---|---|
|
||||
| A01 Broken Access Control (incl. SSRF) | **Strong** — cross-user/tenant access, privilege escalation, IDOR, and SSRF (including blind, via out-of-band callbacks) are all exploit-validated. Needs two accounts plus a privileged one to prove the authorization half. |
|
||||
| A02 Security Misconfiguration | **Strong** — debug endpoints, verbose errors, permissive CORS, missing hardening, default credentials, exposed admin surfaces. |
|
||||
| A03 Software Supply Chain Failures | **Partial** — version fingerprinting, and vulnerable/outdated dependency review when source is supplied. Build-system and distribution-infrastructure compromise (the broader half of this category) is out of scope for a runtime scan — pair with SCA plus build-provenance controls. |
|
||||
| A04 Cryptographic Failures | **Partial** — transport config, unencrypted data in transit, secrets and tokens leaked in responses. At-rest crypto and key management need source or infra review. |
|
||||
| A05 Injection | **Strong** — SQL/NoSQL/command/template injection and XSS, exploit-validated. |
|
||||
| A06 Insecure Design | **Partial** — business-logic abuse (price/quantity tampering, workflow skipping, race conditions) is found where reachable; design intent still needs human review and threat modelling. |
|
||||
| A07 Authentication Failures | **Strong** — auth bypass, weak session/token handling, password-reset and MFA flaws. |
|
||||
| A08 Software or Data Integrity Failures | **Partial** — insecure deserialization and unsigned-update paths where reachable; CI/CD trust boundaries are not runtime-testable. |
|
||||
| A09 Security Logging & Alerting Failures | **Not testable from outside** — requires reviewing the logging and alerting pipeline. State this rather than reporting it as passed. |
|
||||
| A10 Mishandling of Exceptional Conditions | **Partial** — agents actively probe error handling and fail-open behavior (malformed input, forced errors, race and timeout conditions) and report what leaks or bypasses a control; exhaustive coverage of internal error paths needs source review. |
|
||||
|
||||
For APIs, run the same exercise against the **OWASP API Security Top 10 (2023)** — API1 BOLA, API3 Broken Object Property Level Authorization (2019's excessive data exposure + mass assignment merged), API5 broken function-level authorization — using the **api-security-testing** skill.
|
||||
|
||||
## Run it
|
||||
|
||||
Maximum category coverage comes from giving the agents both the source and a running instance, plus credentials at two privilege levels:
|
||||
|
||||
```bash
|
||||
strix -n \
|
||||
-t https://github.com/org/app \
|
||||
-t https://staging.example.com \
|
||||
--scan-mode deep --max-budget 30 \
|
||||
--instruction "OWASP Top 10:2025 assessment. Cover every category systematically and map each finding to its 2025 category id.
|
||||
Accounts: userA@example.com/<pw> (org 1), userB@example.com/<pw> (org 2), admin@example.com/<pw>.
|
||||
Prioritise A01 (cross-org access, privilege escalation, SSRF), A02, A05, A07, A10.
|
||||
Out of scope: /billing/*, outbound email."
|
||||
```
|
||||
|
||||
- `--scan-mode deep` matters here: systematically walking ten categories is not a quick scan.
|
||||
- Without a second account, A01 results are structurally incomplete — say so in the report rather than leaving it implied.
|
||||
- Need an auditor-facing PDF? Run it through the managed platform and pull the technical report (**managed-pentesting-with-strix**).
|
||||
|
||||
## Report honestly
|
||||
|
||||
From `strix_runs/<run>/`, group `vulnerabilities/*.md` by category and state, per category: what was attempted, what was proven, and what could not be assessed (A09 always; A03/A04/A06/A08/A10 partially). Label the report with the edition used. Verify each PoC yourself before it goes in front of the user.
|
||||
|
||||
A `0` exit code means nothing exploitable was proven **in what was analyzed** — check `run.json` status and cost against `--max-budget`; a budget-capped run is not a completed assessment.
|
||||
|
||||
## Then fix and re-test
|
||||
|
||||
Remediate with **fix-security-vulnerabilities-with-strix** and re-run to prove each exploit is closed. For ongoing coverage as the app changes, gate pull requests using **ci-security-scanning-with-strix**.
|
||||
@@ -14,14 +14,14 @@ Strix runs autonomous AI pentesting agents that dynamically exploit a target and
|
||||
- **Open-source CLI** (self-hosted) — runs on your machine in a Docker sandbox with your own LLM key. Free, fully local, BYO-LLM, air-gap capable. Docs: [docs.strix.ai](https://docs.strix.ai).
|
||||
- **Cloud API** (managed) — runs on Strix's infrastructure via `https://app.strix.ai/api/v1`. No Docker, no LLM key, no local compute; adds team dashboards, scheduling, PR reviews, downloadable PDF/DOCX reports (Enterprise plan), and internal-network connectors. Docs: [docs.app.strix.ai](https://docs.app.strix.ai). Full workflow in the **managed-pentesting-with-strix** skill.
|
||||
|
||||
## Which one? (decide, do not default)
|
||||
## Which one? (decide, don't default)
|
||||
|
||||
Choose honestly based on the situation — neither is "better":
|
||||
|
||||
| Situation | Prefer |
|
||||
|---|---|
|
||||
| No Docker available, or a sandboxed/hosted agent/CI environment | **Cloud** |
|
||||
| User has no LLM key / does not want to pay per-token or manage models | **Cloud** |
|
||||
| User has no LLM key / doesn't want to pay per-token or manage models | **Cloud** |
|
||||
| Team visibility, shareable dashboard, scheduled/continuous scans, PR reviews, downloadable PDF/DOCX report (Enterprise) | **Cloud** |
|
||||
| Scanning internal/private infrastructure not reachable from your machine | **Cloud** (network connector) |
|
||||
| Source must never leave local infra (privacy/air-gap), or fully offline | **OSS CLI** |
|
||||
@@ -30,7 +30,7 @@ Choose honestly based on the situation — neither is "better":
|
||||
| CI: runner already has Docker and you want a self-contained gate | **OSS CLI** |
|
||||
| CI: no Docker, or you want results tracked centrally | **Cloud** |
|
||||
|
||||
**Mix them:** use the OSS CLI for the fast local dev-loop while writing/fixing code, and the Cloud for the authoritative, team-visible scan + report + tracking; or gate PRs with the OSS CLI in CI while the Cloud runs scheduled deep scans and PR reviews across the org. Both emit the same SARIF 2.1.0, so findings line up across environments.
|
||||
**Mix them:** e.g. use the OSS CLI for the fast local dev-loop while writing/fixing code, and the Cloud for the authoritative, team-visible scan + report + tracking; or gate PRs with the OSS CLI in CI while the Cloud runs scheduled deep scans and PR reviews across the org. Both emit the same SARIF 2.1.0, so findings line up across environments.
|
||||
|
||||
If unsure and the user has (or will create) an app.strix.ai account, prefer **Cloud** — it avoids all local-infra friction. If they want zero signup / full local control, use the **OSS CLI**.
|
||||
|
||||
@@ -70,33 +70,21 @@ strix -n -t https://github.com/org/app -t https://staging.example.com
|
||||
strix -n -t https://app.example.com \
|
||||
--instruction "Use credentials user@example.com:pass123. Focus on IDOR and auth bypass."
|
||||
|
||||
# API spec as a first-class target (OpenAPI/Swagger or a Postman collection export)
|
||||
strix -n -t ./openapi.yaml -t https://api.staging.example.com
|
||||
|
||||
# Many targets from a file, one per line
|
||||
strix -n --target-list ./targets.txt --max-budget 30
|
||||
|
||||
# Give the agents a file to work with (wordlist, spec, notes) without making it a target
|
||||
strix -n -t https://staging.example.com --workspace-file ./wordlist.txt --max-budget 20
|
||||
# Large monorepo: bind-mount instead of copying
|
||||
strix -n --mount ./huge-monorepo
|
||||
```
|
||||
|
||||
A local path passed with `-t` is mounted into the sandbox **writable** — the agents can read and modify it, so point at a clean checkout, not uncommitted work you care about.
|
||||
|
||||
Key flags:
|
||||
|
||||
| Flag | Meaning |
|
||||
|---|---|
|
||||
| `-t, --target` | URL, repo URL, local path, domain, IP, OpenAPI/Postman spec, or `postman://<uuid>`. Repeatable. |
|
||||
| `--target-list PATH` | File of targets, one per line (`#` comments allowed). Repeatable, combines with `-t`. |
|
||||
| `-t, --target` | URL, repo URL, local path, domain, or IP. Repeatable. |
|
||||
| `-n, --non-interactive` | Headless, exits on completion. Required for agents. |
|
||||
| `-m, --scan-mode` | `quick` (minutes) / `standard` (~30 min) / `deep` (hours, default). |
|
||||
| `--instruction` / `--instruction-file` | Credentials, focus areas, scope rules. |
|
||||
| `--workspace-file PATH[:DEST]` | Place a file from this machine into `/workspace` read-only before the scan, for a wordlist, a spec, or notes. Repeatable. |
|
||||
| `--max-budget USD` | Hard LLM spend cap; scan wraps up cleanly at the limit. |
|
||||
| `--max-turns N` | Per-agent turn cap (default 500). |
|
||||
| `--resume RUN_NAME` | Resume a prior run from `strix_runs/`, with its agent history and targets. Cannot be combined with `-t`. |
|
||||
| `--scope-mode` | For code targets: `auto` (diff-scope in CI/headless), `diff` (force changed files only), `full` (whole tree). |
|
||||
| `--diff-base REF` | Branch or commit that `diff` scope compares against. Defaults to the repo's default branch. |
|
||||
| `--resume RUN_NAME` | Resume a prior run from `strix_runs/`. |
|
||||
|
||||
Scans take minutes (`quick`) to hours (`deep`). Run them in the background and poll for completion rather than blocking.
|
||||
|
||||
@@ -142,7 +130,7 @@ curl -sS "$BASE/scans/$scan_id" -H "Authorization: Bearer $STRIX_API_TOKEN" | jq
|
||||
curl -sS "$BASE/scans/$scan_id/sarif" -H "Authorization: Bearer $STRIX_API_TOKEN" -o findings.sarif
|
||||
```
|
||||
|
||||
Ask the user to create the token (and register the target as a domain/repository asset) if they have not. If Docker/local prerequisites are not already satisfied, use this path instead of trying to install infra.
|
||||
Ask the user to create the token (and register the target as a domain/repository asset) if they haven't. If Docker/local prerequisites aren't already satisfied, use this path instead of trying to install infra.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
---
|
||||
name: web-app-penetration-testing
|
||||
description: Pentest a web app or website end to end — black-box testing of a live URL, staging environment, or local dev server that finds and exploits real vulnerabilities (auth bypass, broken access control, IDOR, injection, XSS, SSRF, business logic) and proves each one with a working proof-of-concept instead of a signature match. Runs with Strix, either the self-hosted open-source CLI or the managed app.strix.ai cloud. Use when the user asks to pentest, hack, security-test, or audit their web app, website, web application, or staging site.
|
||||
license: Apache-2.0
|
||||
metadata:
|
||||
author: usestrix
|
||||
homepage: https://docs.strix.ai
|
||||
---
|
||||
|
||||
# Pentest a web application
|
||||
|
||||
Black-box (and optionally source-assisted) penetration testing of a running web app with Strix's autonomous agents. Every reported finding is validated with a working exploit, so there are no signature-based false positives to triage.
|
||||
|
||||
Install, LLM setup, all CLI flags, and the managed-cloud alternative are covered in the **penetration-testing-with-strix** skill — read it if the target is not a running web app, or if `strix --version` fails. This skill is the web-app-specific workflow.
|
||||
|
||||
## 1. Confirm authorization and scope
|
||||
|
||||
Before running anything, establish:
|
||||
|
||||
- **The target is the user's** (or they are explicitly authorized to test it). Never pentest a third-party site on a hunch.
|
||||
- **Which environment.** Prefer staging over production; agents send real exploit payloads and will create/modify data.
|
||||
- **Out-of-scope paths** — payment flows, mass-email endpoints, admin destructive actions, third-party SSO providers.
|
||||
- **Credentials.** Most real vulnerabilities live behind login. Without a test account, the agents only ever see the marketing surface.
|
||||
|
||||
Ask for anything missing rather than guessing.
|
||||
|
||||
## 2. Run the scan
|
||||
|
||||
```bash
|
||||
strix -n -t https://staging.example.com --max-budget 20 \
|
||||
--instruction "Test account: qa@example.com / <password>. In scope: /app/*, /api/*. Do not touch /billing or send email. Focus on access control between the two seeded orgs."
|
||||
```
|
||||
|
||||
Notes that matter for web apps specifically:
|
||||
|
||||
- **Give it credentials via `--instruction`** (or `--instruction-file` for anything long), including how to log in if the flow is unusual (magic link, SSO, MFA-exempt test user).
|
||||
- **Two accounts beat one.** Multi-tenant IDOR and broken-access-control bugs — consistently the highest-impact class in web apps — can only be proven when the agent can attempt cross-account access.
|
||||
- **Add the repo for white-box depth** when you have the source: `-t https://github.com/org/app -t https://staging.example.com` (or a local path). Source access materially improves coverage of business-logic and authorization flaws.
|
||||
- **Localhost works.** Point at `http://host.docker.internal:3000` (Docker Desktop) so the sandbox can reach a dev server on the host.
|
||||
- `--scan-mode quick` for a fast dev-loop pass, `standard` (~30 min) for a normal review, `deep` for pre-release assurance. Always set `--max-budget`.
|
||||
|
||||
For a hosted run with no Docker/LLM key, or when the user wants a shareable dashboard and an auditor-ready PDF, use the cloud path in **managed-pentesting-with-strix** instead — same engine, same findings.
|
||||
|
||||
## 3. Review results
|
||||
|
||||
Read `strix_runs/<run>/penetration_test_report.md` first, then per-finding files in `vulnerabilities/`. Each contains the PoC — re-run it yourself to confirm before reporting to the user.
|
||||
|
||||
Exit codes: `0` no validated vulns in what was analyzed, `2` vulnerabilities found, `1` fatal error. A `0` is not proof of full coverage — if the budget or turn cap was hit the scan wraps up early, so check `run.json` status and cost against `--max-budget` before calling the app clean.
|
||||
|
||||
## 4. Fix and verify
|
||||
|
||||
Hand findings to the **fix-security-vulnerabilities-with-strix** skill: patch the root cause, then re-run Strix against the same target to prove the exploit no longer works. Re-testing is the only reliable confirmation a fix landed.
|
||||
|
||||
To keep the app tested on every change rather than once, wire Strix into CI with **ci-security-scanning-with-strix**.
|
||||
+10
-47
@@ -2,7 +2,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import inspect
|
||||
import json
|
||||
import logging
|
||||
@@ -223,17 +222,6 @@ def _with_coerced_arguments(tool: FunctionTool) -> FunctionTool:
|
||||
return tool
|
||||
|
||||
|
||||
def _with_strictness(tool: FunctionTool, strict_schemas: bool) -> FunctionTool:
|
||||
"""Drop strict JSON-schema mode when the route can't take it (see
|
||||
``supports_strict_tool_schemas``); the tool stays functionally identical.
|
||||
|
||||
Returns a copy so the shared tool singletons keep their declared mode.
|
||||
"""
|
||||
if strict_schemas or not tool.strict_json_schema:
|
||||
return tool
|
||||
return dataclasses.replace(tool, strict_json_schema=False)
|
||||
|
||||
|
||||
def _function_tool_with_error_result(tool: FunctionTool) -> FunctionTool:
|
||||
invoke_tool = tool.on_invoke_tool
|
||||
|
||||
@@ -297,38 +285,24 @@ def _bound_custom_tool(tool: CustomTool) -> CustomTool:
|
||||
return tool
|
||||
|
||||
|
||||
def _configure_filesystem_tools(
|
||||
toolset: Any, *, chat_completions: bool, strict_schemas: bool = True
|
||||
) -> None:
|
||||
def _configure_filesystem_tools(toolset: Any, *, chat_completions: bool) -> None:
|
||||
for name, tool in vars(toolset).items():
|
||||
if chat_completions:
|
||||
if isinstance(tool, CustomTool):
|
||||
setattr(toolset, name, _custom_tool_as_function_tool(tool))
|
||||
elif isinstance(tool, FunctionTool):
|
||||
setattr(
|
||||
toolset,
|
||||
name,
|
||||
_function_tool_with_error_result(
|
||||
_with_strictness(_with_coerced_arguments(tool), strict_schemas)
|
||||
),
|
||||
toolset, name, _function_tool_with_error_result(_with_coerced_arguments(tool))
|
||||
)
|
||||
elif isinstance(tool, CustomTool):
|
||||
setattr(toolset, name, _bound_custom_tool(tool))
|
||||
elif isinstance(tool, FunctionTool):
|
||||
setattr(
|
||||
toolset,
|
||||
name,
|
||||
_with_bounded_result(
|
||||
_with_strictness(_with_coerced_arguments(tool), strict_schemas)
|
||||
),
|
||||
)
|
||||
setattr(toolset, name, _with_bounded_result(_with_coerced_arguments(tool)))
|
||||
|
||||
|
||||
def _make_filesystem_configurator(*, chat_completions: bool, strict_schemas: bool) -> Any:
|
||||
def _make_filesystem_configurator(*, chat_completions: bool) -> Any:
|
||||
def configure(toolset: Any) -> None:
|
||||
_configure_filesystem_tools(
|
||||
toolset, chat_completions=chat_completions, strict_schemas=strict_schemas
|
||||
)
|
||||
_configure_filesystem_tools(toolset, chat_completions=chat_completions)
|
||||
|
||||
return configure
|
||||
|
||||
@@ -432,13 +406,11 @@ def _wrap_write_stdin(tool: FunctionTool) -> FunctionTool:
|
||||
return tool
|
||||
|
||||
|
||||
def _configure_shell_tools(
|
||||
toolset: Any, *, chat_completions: bool, strict_schemas: bool = True
|
||||
) -> None:
|
||||
def _configure_shell_tools(toolset: Any, *, chat_completions: bool) -> None:
|
||||
for name, tool in vars(toolset).items():
|
||||
if not isinstance(tool, FunctionTool):
|
||||
continue
|
||||
wrapped = _with_strictness(_with_coerced_arguments(tool), strict_schemas)
|
||||
wrapped = _with_coerced_arguments(tool)
|
||||
if tool.name == "exec_command":
|
||||
wrapped = _wrap_exec_command(wrapped)
|
||||
elif tool.name == "write_stdin":
|
||||
@@ -448,11 +420,9 @@ def _configure_shell_tools(
|
||||
setattr(toolset, name, wrapped)
|
||||
|
||||
|
||||
def _make_shell_configurator(*, chat_completions: bool, strict_schemas: bool) -> Any:
|
||||
def _make_shell_configurator(*, chat_completions: bool) -> Any:
|
||||
def configure(toolset: Any) -> None:
|
||||
_configure_shell_tools(
|
||||
toolset, chat_completions=chat_completions, strict_schemas=strict_schemas
|
||||
)
|
||||
_configure_shell_tools(toolset, chat_completions=chat_completions)
|
||||
|
||||
return configure
|
||||
|
||||
@@ -598,7 +568,6 @@ def build_strix_agent(
|
||||
is_whitebox: bool = False,
|
||||
interactive: bool = False,
|
||||
chat_completions_tools: bool = False,
|
||||
strict_tool_schemas: bool = True,
|
||||
system_prompt_context: dict[str, Any] | None = None,
|
||||
extra_tools: Sequence[Tool] | None = None,
|
||||
instructions_override: str | None = None,
|
||||
@@ -608,8 +577,6 @@ def build_strix_agent(
|
||||
Args:
|
||||
chat_completions_tools: Wrap SDK custom tools as function tools
|
||||
when the selected backend cannot accept Responses custom tools.
|
||||
strict_tool_schemas: Send function tools as strict-schema tools. Off
|
||||
for routes that reject a toolset this size as strict.
|
||||
extra_tools: Additional tools for this scan agent only, on top of any
|
||||
registered via ``register_agent_tools``.
|
||||
instructions_override: Use this verbatim as the system prompt instead
|
||||
@@ -637,7 +604,7 @@ def build_strix_agent(
|
||||
tools = [*_BASE_TOOLS, *agent_tools, agent_finish]
|
||||
_ensure_unique_tool_names(tools)
|
||||
tools = [
|
||||
_with_bounded_result(_with_strictness(_with_coerced_arguments(tool), strict_tool_schemas))
|
||||
_with_bounded_result(_with_coerced_arguments(tool))
|
||||
if isinstance(tool, FunctionTool)
|
||||
else tool
|
||||
for tool in tools
|
||||
@@ -663,13 +630,11 @@ def build_strix_agent(
|
||||
Filesystem(
|
||||
configure_tools=_make_filesystem_configurator(
|
||||
chat_completions=chat_completions_tools,
|
||||
strict_schemas=strict_tool_schemas,
|
||||
),
|
||||
),
|
||||
Shell(
|
||||
configure_tools=_make_shell_configurator(
|
||||
chat_completions=chat_completions_tools,
|
||||
strict_schemas=strict_tool_schemas,
|
||||
),
|
||||
),
|
||||
],
|
||||
@@ -682,7 +647,6 @@ def make_child_factory(
|
||||
is_whitebox: bool = False,
|
||||
interactive: bool = False,
|
||||
chat_completions_tools: bool = False,
|
||||
strict_tool_schemas: bool = True,
|
||||
system_prompt_context: dict[str, Any] | None = None,
|
||||
) -> Any:
|
||||
"""Return the runner-owned builder used by ``spawn_child_agent``.
|
||||
@@ -701,7 +665,6 @@ def make_child_factory(
|
||||
is_whitebox=is_whitebox,
|
||||
interactive=interactive,
|
||||
chat_completions_tools=chat_completions_tools,
|
||||
strict_tool_schemas=strict_tool_schemas,
|
||||
system_prompt_context=system_prompt_context,
|
||||
)
|
||||
|
||||
|
||||
+24
-2
@@ -54,12 +54,16 @@ def apply_config_override(path: Path) -> None:
|
||||
|
||||
|
||||
def persist_current() -> None:
|
||||
"""Write currently-set env vars to the active config file (0o600)."""
|
||||
"""Merge currently-set env vars into the active config file (0o600).
|
||||
|
||||
Keys already on disk are preserved (including keys unknown to this
|
||||
version's schema); env vars win on conflicts.
|
||||
"""
|
||||
s = load_settings()
|
||||
target = _override or _DEFAULT_PATH
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
env_block: dict[str, str] = {}
|
||||
env_block: dict[str, Any] = _read_persisted_env(target)
|
||||
for sub_name in s.model_fields:
|
||||
sub_model = getattr(s, sub_name)
|
||||
if not isinstance(sub_model, BaseModel):
|
||||
@@ -74,6 +78,24 @@ def persist_current() -> None:
|
||||
write_secret_text(target, json.dumps({"env": env_block}, indent=2))
|
||||
|
||||
|
||||
def _read_persisted_env(path: Path) -> dict[str, Any]:
|
||||
"""Read the ``{"env": {...}}`` block already stored at ``path``."""
|
||||
if not path.exists():
|
||||
return {}
|
||||
try:
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
except (json.JSONDecodeError, OSError):
|
||||
return {}
|
||||
env_block = data.get("env", {}) if isinstance(data, dict) else {}
|
||||
if not isinstance(env_block, dict):
|
||||
return {}
|
||||
return {
|
||||
str(key).upper(): value
|
||||
for key, value in env_block.items()
|
||||
if value is not None and value != ""
|
||||
}
|
||||
|
||||
|
||||
def _aliases_for(finfo: FieldInfo) -> list[str]:
|
||||
"""Collect every env-var name that should populate ``finfo``."""
|
||||
aliases: list[str] = []
|
||||
|
||||
@@ -749,18 +749,6 @@ def uses_chat_completions_tool_schema(model_name: str, settings: Settings) -> bo
|
||||
return not model_supports_reasoning(model_name)
|
||||
|
||||
|
||||
def supports_strict_tool_schemas(model_name: str) -> bool:
|
||||
"""Return whether the route accepts strict tool schemas for Strix's toolset.
|
||||
|
||||
Claude caps a request at 20 strict tools and 16 union-typed parameters
|
||||
across all strict schemas. Strix ships ~30 tools and the strict dialect
|
||||
turns every optional parameter into a nullable union, so both caps are
|
||||
exceeded and the request is rejected outright.
|
||||
"""
|
||||
name = model_name.strip().lower()
|
||||
return not any(marker in name for marker in _ANTHROPIC_MODEL_MARKERS)
|
||||
|
||||
|
||||
def model_supports_reasoning(model_name: str) -> bool:
|
||||
import litellm
|
||||
|
||||
@@ -857,9 +845,6 @@ def is_known_openai_bare_model(model_name: str) -> bool:
|
||||
return bool(entry and entry.get("litellm_provider") == "openai")
|
||||
|
||||
|
||||
_ANTHROPIC_MODEL_MARKERS = ("anthropic", "claude", "sonnet", "opus", "haiku")
|
||||
|
||||
|
||||
def is_claude_model(model_name: str) -> bool:
|
||||
return "claude" in (model_name or "").strip().lower()
|
||||
|
||||
|
||||
@@ -22,7 +22,6 @@ from strix.config import load_settings
|
||||
from strix.config.models import (
|
||||
StrixProvider,
|
||||
configure_sdk_model_defaults,
|
||||
supports_strict_tool_schemas,
|
||||
uses_chat_completions_tool_schema,
|
||||
)
|
||||
from strix.config.settings import DEFAULT_MAX_TURNS
|
||||
@@ -176,9 +175,6 @@ async def run_strix_scan(
|
||||
)
|
||||
logger.info("LLM model resolved: %s", resolved_model)
|
||||
chat_completions_tools = uses_chat_completions_tool_schema(resolved_model, settings)
|
||||
strict_tool_schemas = supports_strict_tool_schemas(resolved_model)
|
||||
if not strict_tool_schemas:
|
||||
logger.info("Sending non-strict tool schemas: %s caps strict tools", resolved_model)
|
||||
|
||||
if coordinator is None:
|
||||
coordinator = AgentCoordinator()
|
||||
@@ -310,7 +306,6 @@ async def run_strix_scan(
|
||||
is_whitebox=is_whitebox,
|
||||
interactive=interactive,
|
||||
chat_completions_tools=chat_completions_tools,
|
||||
strict_tool_schemas=strict_tool_schemas,
|
||||
system_prompt_context=root_context,
|
||||
instructions_override=root_instructions,
|
||||
)
|
||||
@@ -329,7 +324,6 @@ async def run_strix_scan(
|
||||
is_whitebox=is_whitebox,
|
||||
interactive=interactive,
|
||||
chat_completions_tools=chat_completions_tools,
|
||||
strict_tool_schemas=strict_tool_schemas,
|
||||
system_prompt_context=scope_context,
|
||||
)
|
||||
|
||||
|
||||
@@ -346,7 +346,7 @@ def _load_resume_state(args: argparse.Namespace, parser: argparse.ArgumentParser
|
||||
)
|
||||
try:
|
||||
state = read_run_record(run_dir)
|
||||
except (RuntimeError, TypeError) as exc:
|
||||
except RuntimeError as exc:
|
||||
parser.error(f"--resume {args.resume}: run.json unreadable: {exc}")
|
||||
|
||||
args.targets_info = state.get("targets_info") or []
|
||||
|
||||
@@ -146,9 +146,7 @@ def bounded_state_projection(state: dict[str, Any]) -> dict[str, Any]:
|
||||
}
|
||||
for message in state["messages"][-5:]
|
||||
]
|
||||
state["usage"] = {
|
||||
key: state["usage"][key] for key in ("total_tokens", "cost") if key in state["usage"]
|
||||
}
|
||||
state["usage"] = {}
|
||||
state["error"] = terminal_projection(state["error"], max_string=512)
|
||||
state["model_warning"] = terminal_projection(state["model_warning"], max_string=256)
|
||||
state["caido_url"] = terminal_projection(state["caido_url"], max_string=256)
|
||||
@@ -175,7 +173,7 @@ def bounded_state_projection(state: dict[str, Any]) -> dict[str, Any]:
|
||||
"model_warning": "",
|
||||
"caido_url": None,
|
||||
"messages": [],
|
||||
"usage": state["usage"],
|
||||
"usage": {},
|
||||
"subscription": state["subscription"],
|
||||
"viewer_status": state["viewer_status"],
|
||||
"viewer_url": None,
|
||||
|
||||
@@ -100,7 +100,7 @@ func applyMarkdownStyles(text string) string {
|
||||
case strings.HasPrefix(line, "- "), strings.HasPrefix(line, "* "):
|
||||
out.WriteString(Col(Green).Render("• ") + inlineFormat(line[2:]))
|
||||
case len(line) > 2 && line[0] >= '0' && line[0] <= '9' && (line[1:3] == ". " || line[1:3] == ") "):
|
||||
out.WriteString(Col(Green).Render(line[:2]+" ") + inlineFormat(line[3:]))
|
||||
out.WriteString(Col(Green).Render(string(line[0])+". ") + inlineFormat(line[2:]))
|
||||
case line == "---" || line == "***" || line == "___":
|
||||
out.WriteString(Col(Green).Render(strings.Repeat("─", 40)))
|
||||
default:
|
||||
|
||||
@@ -72,19 +72,6 @@ func TestNonTablePipeLinesAreLeftAlone(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestMarkdownOrderedListsUseSingleSpaceAfterMarker(t *testing.T) {
|
||||
out := renderAssistantMarkdown("1. hello\n2) world")
|
||||
plain := ansi.Strip(out)
|
||||
for _, want := range []string{"1. hello", "2) world"} {
|
||||
if !strings.Contains(plain, want) {
|
||||
t.Fatalf("ordered list item %q missing: %q", want, plain)
|
||||
}
|
||||
}
|
||||
if strings.Contains(plain, "1. hello") || strings.Contains(plain, "2) world") {
|
||||
t.Fatalf("double space after the list marker: %q", plain)
|
||||
}
|
||||
}
|
||||
|
||||
func TestInlineFormatKeepsNonEmphasisMarkers(t *testing.T) {
|
||||
literal := []string{
|
||||
"ls *.py *.go",
|
||||
|
||||
@@ -112,19 +112,3 @@ def test_wait_for_agents_is_available_in_both_modes() -> None:
|
||||
for interactive in (True, False):
|
||||
agent = factory.build_strix_agent(is_root=True, interactive=interactive)
|
||||
assert "wait_for_agents" in [t.name for t in agent.tools]
|
||||
|
||||
|
||||
def test_strict_tool_schemas_can_be_disabled_per_route() -> None:
|
||||
"""Claude routes cap strict tools; the toolset must be sendable without strict."""
|
||||
agent = factory.build_strix_agent(is_root=True, strict_tool_schemas=False)
|
||||
|
||||
function_tools = [t for t in agent.tools if isinstance(t, FunctionTool)]
|
||||
assert function_tools
|
||||
assert not any(t.strict_json_schema for t in function_tools)
|
||||
|
||||
|
||||
def test_disabling_strict_leaves_shared_tools_untouched() -> None:
|
||||
factory.build_strix_agent(is_root=True, strict_tool_schemas=False)
|
||||
agent = factory.build_strix_agent(is_root=True)
|
||||
|
||||
assert any(t.strict_json_schema for t in agent.tools if isinstance(t, FunctionTool))
|
||||
|
||||
@@ -227,18 +227,3 @@ def test_resume_still_requires_targets_or_a_workspace(
|
||||
cli_main.parse_arguments()
|
||||
|
||||
assert "has no targets_info" in capsys.readouterr().err
|
||||
|
||||
def test_resume_non_object_run_json_exits(tmp_path: Path, monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None:
|
||||
monkeypatch.chdir(tmp_path)
|
||||
run_dir = tmp_path / "strix_runs" / "pentest_abcd"
|
||||
run_dir.mkdir(parents=True)
|
||||
(run_dir / "run.json").write_text("[]", encoding="utf-8")
|
||||
|
||||
monkeypatch.setattr(sys, "argv", ["strix", "--resume", "pentest_abcd"])
|
||||
with pytest.raises(SystemExit) as exc_info:
|
||||
cli_main.parse_arguments()
|
||||
|
||||
assert exc_info.value.code == 2
|
||||
captured = capsys.readouterr()
|
||||
assert "run.json unreadable" in captured.err
|
||||
assert "not an object" in captured.err
|
||||
|
||||
@@ -208,6 +208,69 @@ def test_persist_current_writes_env_block(tmp_path: Path, monkeypatch: pytest.Mo
|
||||
}
|
||||
|
||||
|
||||
def test_persist_current_preserves_keys_missing_from_env(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
monkeypatch.delenv("STRIX_LLM", raising=False)
|
||||
monkeypatch.delenv("LLM_API_KEY", raising=False)
|
||||
monkeypatch.setenv("OPENAI_API_KEY", "env-key")
|
||||
target = tmp_path / "cli-config.json"
|
||||
target.write_text(
|
||||
json.dumps({"env": {"STRIX_LLM": "file-model", "STRIX_FUTURE_KEY": "keep-me"}}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
loader.apply_config_override(target)
|
||||
|
||||
loader.persist_current()
|
||||
|
||||
assert json.loads(target.read_text(encoding="utf-8")) == {
|
||||
"env": {
|
||||
"STRIX_LLM": "file-model",
|
||||
"STRIX_FUTURE_KEY": "keep-me",
|
||||
"OPENAI_API_KEY": "env-key",
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def test_persist_current_env_wins_over_persisted_value(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
monkeypatch.setenv("STRIX_LLM", "env-model")
|
||||
target = tmp_path / "cli-config.json"
|
||||
target.write_text(json.dumps({"env": {"STRIX_LLM": "file-model"}}), encoding="utf-8")
|
||||
loader.apply_config_override(target)
|
||||
|
||||
loader.persist_current()
|
||||
|
||||
assert json.loads(target.read_text(encoding="utf-8"))["env"]["STRIX_LLM"] == "env-model"
|
||||
|
||||
|
||||
def test_persist_current_preserves_dict_valued_key(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
monkeypatch.delenv("STRIX_LLM", raising=False)
|
||||
monkeypatch.delenv("LLM_EXTRA_HEADERS", raising=False)
|
||||
target = tmp_path / "cli-config.json"
|
||||
target.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"env": {
|
||||
"STRIX_LLM": "file-model",
|
||||
"LLM_EXTRA_HEADERS": {"X-Foo": "bar"},
|
||||
}
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
loader.apply_config_override(target)
|
||||
|
||||
loader.persist_current()
|
||||
|
||||
persisted_env = json.loads(target.read_text(encoding="utf-8"))["env"]
|
||||
assert persisted_env["STRIX_LLM"] == "file-model"
|
||||
assert persisted_env["LLM_EXTRA_HEADERS"] == {"X-Foo": "bar"}
|
||||
|
||||
|
||||
def test_persist_current_sets_0600_mode(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setenv("STRIX_LLM", "persisted-model")
|
||||
target = tmp_path / "cli-config.json"
|
||||
|
||||
@@ -9,7 +9,6 @@ from strix.config.models import (
|
||||
RECOMMENDED_MODEL_NAMES,
|
||||
is_recommended_or_frontier_model,
|
||||
request_timeout_extra_args,
|
||||
supports_strict_tool_schemas,
|
||||
)
|
||||
|
||||
|
||||
@@ -91,24 +90,3 @@ def test_frontier_model_families_are_accepted(model_name: str) -> None:
|
||||
)
|
||||
def test_non_frontier_models_are_rejected(model_name: str) -> None:
|
||||
assert not is_recommended_or_frontier_model(model_name)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"model_name",
|
||||
[
|
||||
"anthropic/claude-sonnet-4-6",
|
||||
"bedrock/anthropic.claude-opus-4-8-v1:0",
|
||||
"vertex_ai/claude-sonnet-5",
|
||||
"Sonnet-5",
|
||||
],
|
||||
)
|
||||
def test_claude_routes_reject_strict_tool_schemas(model_name: str) -> None:
|
||||
assert not supports_strict_tool_schemas(model_name)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"model_name",
|
||||
["openai/gpt-5.4", "gpt-5.4", "gemini/gemini-3.1-pro-preview", "deepseek/deepseek-v4"],
|
||||
)
|
||||
def test_other_routes_keep_strict_tool_schemas(model_name: str) -> None:
|
||||
assert supports_strict_tool_schemas(model_name)
|
||||
|
||||
@@ -13,7 +13,7 @@ from agents.tool import ToolOutputImage
|
||||
|
||||
from strix.config.settings import DEFAULT_MAX_TURNS
|
||||
from strix.interface.tui.backend.controller import TuiController
|
||||
from strix.interface.tui.backend.projection import bounded_state_projection, terminal_projection
|
||||
from strix.interface.tui.backend.projection import terminal_projection
|
||||
from strix.interface.tui.backend.protocol import (
|
||||
MAX_COMMAND_BYTES,
|
||||
PROTOCOL_CAPABILITIES,
|
||||
@@ -215,11 +215,7 @@ def test_unicode_heavy_setup_state_stays_within_control_frame_limit() -> None:
|
||||
"Any",
|
||||
SimpleNamespace(
|
||||
caido_url="https://例え.example/" + "道" * 10_000,
|
||||
get_total_llm_usage=lambda: {
|
||||
"total_tokens": 720_400,
|
||||
"cost": 20.0,
|
||||
**{f"model-{index}": "🔒" * 10_000 for index in range(20)},
|
||||
},
|
||||
get_total_llm_usage=lambda: {f"model-{index}": "費" * 10_000 for index in range(20)},
|
||||
),
|
||||
)
|
||||
server = TuiBackendServer(controller)
|
||||
@@ -230,26 +226,6 @@ def test_unicode_heavy_setup_state_stays_within_control_frame_limit() -> None:
|
||||
assert len(encoded) <= MAX_COMMAND_BYTES
|
||||
assert "🔒".encode() in encoded
|
||||
assert snapshot["projection_truncated"] is True
|
||||
assert snapshot["usage"] == {"total_tokens": 720_400, "cost": 20.0}
|
||||
|
||||
|
||||
def test_defensive_state_projection_preserves_usage_summary() -> None:
|
||||
controller = TuiController(args())
|
||||
controller.report_state = cast(
|
||||
"Any",
|
||||
SimpleNamespace(
|
||||
caido_url=None,
|
||||
get_total_llm_usage=lambda: {"total_tokens": 720_400, "cost": 20.0},
|
||||
),
|
||||
)
|
||||
state = controller.snapshot()
|
||||
state["provider"] = None
|
||||
state["future_oversized_field"] = "x" * 100_000
|
||||
|
||||
snapshot = bounded_state_projection(state)
|
||||
|
||||
assert snapshot["projection_truncated"] is True
|
||||
assert snapshot["usage"] == {"total_tokens": 720_400, "cost": 20.0}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
||||
Reference in New Issue
Block a user