fix(proxy,tooling): serialize+reconnect Caido client, actionable HTTPQL errors, sandbox tool guidance (#794)

* fix(proxy,tooling): serialize+reconnect Caido client, actionable HTTPQL errors, sandbox tool guidance

Addresses the top recurring agent tool-call failures observed in telemetry:

- proxy: the shared Caido client had no locking or reconnect, so concurrent
  agent calls raced ("Transport is already connected") and a dead transport
  poisoned the rest of the run ("Connector is closed"/"Server disconnected").
  Add an asyncio lock + bounded reconnect in caido_api.call_with_client (sandbox
  path) and a scan-wide caido_lock in the run context that host-side proxy tools
  hold around every call. Deterministic errors are not retried.
- proxy: list_requests now returns Caido's exact parser message, echoes the
  offending query, and includes a corrected-syntax hint so agents self-correct
  instead of retrying a broken HTTPQL filter.
- shell/prompt: document that write_stdin requires a process started with
  tty=true; nudge toward writing Python to a file over deeply-nested one-liners;
  note the venv pre-installs common libs.
- agent-browser: distinguish daemon/connection failures (run doctor, don't loop)
  from malformed commands; invoke directly (no sh -c wrapper).
- containers: use POSIX '.' instead of the bashism 'source' in generated rc
  files (fixes 'sh: source: not found'); add file + xxd and pre-install
  requests/httpx/beautifulsoup4/lxml/pyjwt/cryptography in the sandbox venv.
- tests: cover proxy serialization/reconnect/no-retry and HTTPQL errors.

* fix(proxy): host-side reconnect, close stale clients, don't retry mutations

Addresses Greptile review on the reconnect logic:

- Host path had no reconnect: a dead shared context client (Caido restart /
  network blip) previously disabled proxy tools for the rest of the scan. Add
  SharedCaidoClient, a serialized reconnect-safe holder stored once per scan in
  the run context and shared across agents. On a dead transport it rebuilds via
  reconnect_caido, which re-selects the SAME Caido project (preserving captured
  traffic) instead of creating a new empty one.
- Don't repeat completed mutations: call_with_client / SharedCaidoClient.call
  take idempotent=. Reads retry once on reconnect; replay + scope
  create/update/delete heal the client but re-raise instead of risking a
  double-apply.
- Don't leak replaced clients: the stale client is aclose()d (best-effort) on
  every reconnect.
- Extend tests to cover close-on-reconnect, non-idempotent re-raise, and the
  SharedCaidoClient holder.

* fix(proxy): close replacement Caido client when project.select fails

Addresses Greptile P1: in reconnect_caido (and bootstrap_caido) a successful
connect() followed by a failing project.select()/create() discarded the
connected client without closing it, so a missing/unavailable project could
leak a transport on every retry. Close the client before re-raising.

---------

Co-authored-by: Alex Schapiro <bearsyankees@gmail.com>
This commit is contained in:
devin-ai-integration[bot]
2026-07-17 13:31:57 -04:00
committed by GitHub
co-authored by Alex Schapiro
parent df97c86f8f
commit e4548cb28c
12 changed files with 784 additions and 109 deletions
+17 -2
View File
@@ -169,8 +169,23 @@ EFFICIENCY TACTICS:
- Run multiple scans in parallel when possible
- Load the most relevant skill before starting a specialized testing workflow if doing so will improve accuracy, speed, or tool usage
- Use `exec_command` for Python code: write reusable scripts to a file and
run them with `python3`. For one-off snippets, `python3 -c` or a
here-document is acceptable.
run them with `python3 script.py`. For one-off snippets, `python3 -c` or a
here-document is acceptable, but avoid deeply nested quotes/parentheses — if
a snippet needs complex quoting or is more than a few lines, write it to a
file first to prevent syntax errors.
- Before importing a third-party Python library, make sure it is installed. The
sandbox's `python3` runs inside a preconfigured virtualenv that ships
`requests`, `httpx`, `beautifulsoup4` (bs4), `lxml`, `pyjwt`, and
`cryptography`; for anything else prefer the stdlib or run `pip install <pkg>`
(it installs into that active venv) before importing, rather than letting the
script fail with `ModuleNotFoundError`.
- `exec_command` runs each command in a fresh non-interactive shell (plain
pipes, no TTY). To drive an interactive or long-running process with
`write_stdin` — REPLs, `ssh`/`nc`/`ftp`, `msfconsole`, or to send Ctrl-C —
you MUST start it with `exec_command(cmd="...", tty=true)` and then
`write_stdin(session_id=<id>, chars="...")`. Calling `write_stdin` on a
default (non-TTY) command or on a process that has already exited fails with
"stdin is not available".
- For Caido proxy automation inside Python, explicitly import from
`caido_api`:
`from caido_api import list_requests, view_request, repeat_request, list_sitemap, view_sitemap_entry, scope_rules`