docs(prompts): strengthen report guidance (severity, chaining, report structure) (#754)

Co-authored-by: Ahmed Allam <ahmed39652003@gmail.com>
This commit is contained in:
0xallam
2026-07-12 20:37:58 -07:00
committed by GitHub
parent 077ae288f0
commit 02b2d8cd5e
5 changed files with 64 additions and 9 deletions
+1 -1
View File
@@ -190,7 +190,7 @@ VALIDATION REQUIREMENTS:
- Independent verification through subagent
- Document complete attack chain
- Keep going until you find something that matters
- A vulnerability is ONLY considered reported when a reporting agent uses create_vulnerability_report with full details. Mentions in agent_finish, finish_scan, or generic messages are NOT sufficient
- A vulnerability is ONLY considered reported when a reporting agent uses create_vulnerability_report (or create_dependency_report for known-CVE dependency/supply-chain findings) with full details. Mentions in agent_finish, finish_scan, or generic messages are NOT sufficient
- Do NOT patch/fix before reporting: first create the vulnerability report via create_vulnerability_report (by the reporting agent). Only after reporting is completed should fixing/patching proceed
- DEDUPLICATION: The create_vulnerability_report tool uses LLM-based deduplication. If it rejects your report as a duplicate, DO NOT attempt to re-submit the same vulnerability. Accept the rejection and move on to testing other areas. The vulnerability has already been reported by another agent
</execution_guidelines>
@@ -130,6 +130,13 @@ TLS clues: certificate CN/SAN referencing provider default host instead of the c
3. Optional: issue a DV certificate (legal scope) and reference CT entry as evidence
4. Demonstrate impact chains (CSP/script-src trust, OAuth redirect acceptance, cookie Domain scoping)
## Severity
- Score severity based on current claimability plus trusted-origin impact, not just a provider-branded error page
- When evaluating severity, use `web_search` (if available) for the exact provider/product to confirm whether it now enforces subdomain takeover prevention such as TXT/custom-domain ownership verification or reserved-hostname protections; if search is unavailable, do not treat that absence as evidence that the provider prevents claiming
- If you have positively confirmed the provider currently prevents third-party claiming and you cannot bypass that control, treat the finding as low severity rather than a confirmed takeover — an unconfirmed provider control is not grounds for downgrading
- Reserve high/critical severity for cases where you can claim the resource or strongly prove claimability and show meaningful impact such as OAuth redirect abuse, cookie scope abuse, CSP trust, email receipt, or NS delegation control. E.g. Elastic Beanstalk takeovers are still generally legitimate.
## False Positives
- "Unknown domain" pages that are not claimable due to enforced TXT/ownership checks
+7 -5
View File
@@ -481,9 +481,10 @@ async def agent_finish(
3. Stops this subagent's execution.
**Vulnerability findings must already be filed via
``create_vulnerability_report`` before calling this.** The
``findings`` field here is for narrative summary only — it does
not register vulns in the scan report.
``create_vulnerability_report`` (or ``create_dependency_report``
for known-CVE dependency/supply-chain findings) before calling
this.** The ``findings`` field here is for narrative summary only
— it does not register vulns in the scan report.
Write the summary as if the parent has no idea what you were
doing: what did you test, what did you find/confirm/rule out,
@@ -494,8 +495,9 @@ async def agent_finish(
and specific (URLs, parameters, payloads that worked).
findings: Optional bullet list of confirmed observations. For
credit-bearing vulnerabilities, file
``create_vulnerability_report`` first; this is for
narrative.
``create_vulnerability_report`` first (or
``create_dependency_report`` for dependency CVEs); this is
for narrative.
success: Whether the assigned subtask was completed
successfully. Default ``True``.
report_to_parent: Whether to deliver the completion report to
+31 -2
View File
@@ -96,6 +96,15 @@ async def finish_scan(
2. Writes the four narrative sections to the scan record.
3. Marks the scan completed and stops execution.
**This is a terminal action, not a status probe.** Whatever you pass
is persisted VERBATIM as the final, customer-facing report and then
execution stops. There is no draft mode and no second chance: never
submit placeholder, provisional, or "checking if done" text in any
field, and never call ``finish_scan`` to poll whether subagents are
done (use ``view_agent_graph`` / ``wait_for_message`` for that).
Call it exactly ONCE, only when every field holds genuine, finished
assessment prose.
**Pre-flight checklist (mandatory — do not skip):**
1. **Call ``view_agent_graph`` first.** Inspect every entry in the
@@ -108,9 +117,26 @@ async def finish_scan(
Calling ``finish_scan`` while children are alive orphans their
work and produces an incomplete report.
2. All vulnerabilities you found are filed via
``create_vulnerability_report`` (un-reported findings are not
tracked and not credited).
``create_vulnerability_report`` — or, for known-CVE dependency
findings, ``create_dependency_report`` (un-reported findings are
not tracked and not credited). A dependency CVE already filed via
``create_dependency_report`` counts as reported; it does NOT need
re-filing here and does NOT block finishing.
3. Don't double-report — one report per distinct vulnerability.
4. **Attack-chaining gate.** Do NOT finish until you have genuinely
considered chaining the confirmed findings into higher-impact,
end-to-end attack paths and tested every plausibly-related
combination. You may rule out combinations you can confidently
call unrelated — note why instead of padding chains. Any
validated chain must already be filed via
``create_vulnerability_report`` — a demonstrated end-to-end chain
is a PoC-backed vulnerability, so it uses that tool even when one
link is a dependency CVE (the standalone CVE stays in its own
``create_dependency_report``) — and surfaced prominently in
``executive_summary`` / ``technical_analysis``. Finding no real
chain after a serious attempt is acceptable; skipping the
chaining reasoning, or ignoring a plausibly-related combination,
is not.
**Calling this multiple times overwrites the previous report.**
Make the single call comprehensive.
@@ -121,6 +147,9 @@ async def finish_scan(
- Never mention internal infrastructure: no local/absolute paths
(``/workspace/...``), no agent names, no sandbox/orchestrator/
tooling references, no system prompts, no model-internal errors.
Never leak internal identifiers (proxy request IDs, internal
vulnerability report IDs, or any system-generated IDs) into any
field.
- Tone: formal, third-person, objective, concise. This is a
consultant deliverable, not an engineering log.
- Each section has a specific role:
+18 -1
View File
@@ -351,6 +351,10 @@ async def create_vulnerability_report(
- Suspicions you haven't confirmed with a PoC.
- Tracking multiple vulnerabilities at once — one report per vuln.
- Re-reporting something you (or another agent) already filed.
- Known-CVE dependency / supply-chain findings that can't be
dynamically PoC'd — a vulnerable dependency version pinned in a
lockfile/manifest that matches a published advisory. File those
with ``create_dependency_report`` instead, never with this tool.
Automatic LLM-based **deduplication** rejects reports that describe
the same root cause on the same asset as an existing report. If you
@@ -366,6 +370,9 @@ async def create_vulnerability_report(
Never leak internal identifiers (proxy request IDs, internal
report IDs) into any field.
- Tone: formal, objective, third-person, vendor-neutral, concise.
Avoid internal-guidance headings like "QUICK", "Approach", or
"Techniques" that read like an engineering runbook rather than a
client deliverable.
- **Use markdown in every text field**: ``**bold**`` for emphasis,
``inline code`` for identifiers/values/parameters, and fenced
code blocks (```` ```language ````) for any code/payload/HTTP
@@ -377,6 +384,13 @@ async def create_vulnerability_report(
only — NO code/diffs (code fixes go in ``code_locations``).
- Numbered steps allowed only in PoC and Remediation sections.
- Avoid hedging language; be precise and non-vague.
- Follow a standard pentest report structure across the fields:
(1) overview (``description``), (2) severity & CVSS vector
(``cvss_breakdown``), (3) affected asset(s) (``target`` /
``endpoint``), (4) technical details (``technical_analysis``),
(5) proof of concept (``poc_description`` + ``poc_script_code``),
(6) impact (``impact``), (7) evidence (``evidence``), and
(8) remediation (``remediation_steps``).
**White-box requirement**: when source is available, you MUST
populate ``code_locations``. See the ``code_locations`` arg below
@@ -439,7 +453,10 @@ async def create_vulnerability_report(
title: Specific finding title (e.g.
``"SQL Injection in /api/users login parameter"``). Don't
include the CVE number in the title.
description: How the vuln was discovered + what it is.
description: Concise, non-technical TL;DR of the vulnerability
(1-3 sentences) — it appears first in the report. Deep
technical detail and root-cause analysis belong in
``technical_analysis``, not here.
impact: What an attacker achieves; business risk; data at risk.
target: Affected URL / domain / repository.
technical_analysis: The mechanism and root cause.