Commit Graph
27 Commits
Author SHA1 Message Date
0xallam 640868a525 refactor: nuke `events.jsonl` pipeline and the unused PII sanitizer
The JSONL trace sink was never read — TUI consumes ``Tracer`` state
directly (chat_messages, agents, tool_executions, vulnerability_reports,
LLM stats), and SQLiteSession owns the conversation history. The whole
``StrixTracingProcessor`` → ``_emit_event`` → ``append_jsonl_record``
pipeline was producing files nothing opens.

Deleted:
- ``strix/telemetry/strix_processor.py`` (the SDK ``TracingProcessor``).
- ``strix/telemetry/utils.py`` — ``TelemetrySanitizer`` (no remaining
  callers), ``append_jsonl_record``, ``get_events_write_lock``,
  ``reset_events_write_locks``.
- ``strix/telemetry/flags.py`` — ``is_telemetry_enabled`` /
  ``is_posthog_enabled`` collapsed into a 4-line check inside
  ``posthog._is_enabled`` (its only caller).
- ``Tracer._emit_event`` and every event-emit call inside the tracer
  (``run.started``, ``run.configured``, ``run.completed``,
  ``finding.created``, ``finding.reviewed``, ``chat.message``).
- ``Tracer._enrich_actor`` (only used by ``_emit_event``).
- ``Tracer._sanitize_data`` + ``_sanitizer`` field (PII scrub only ran
  on JSONL events).
- ``Tracer.events_file_path`` property and the ``_events_file_path`` /
  ``_telemetry_enabled`` / ``_run_completed_emitted`` /
  ``_next_execution_id`` fields.
- ``Tracer._calculate_duration`` (one caller in posthog — inlined).
- ``add_trace_processor(StrixTracingProcessor(run_dir))`` from
  ``entry.py``.

The ``Tracer`` class is now ~275 LoC of pure runtime state for the TUI
+ vulnerability artifact writer (markdown / CSV / pentest report).
Conversation history goes to ``SQLiteSession``; SDK trace events are
not persisted.
2026-04-25 13:47:37 -07:00
0xallam 0acadd4135 refactor: lean on SDK for tracing + native session resume; nuke OTEL/Traceloop
The SDK ships its own tracing pipeline (``agents.tracing``) plus
``SQLiteSession`` for native conversation persistence. Strix's custom
OTEL bootstrap + Traceloop integration was dead weight — the SDK does
not bridge to OpenTelemetry, so all of our adapter code was solving a
problem we didn't actually need solved.

Telemetry purge:
- Drop the ``traceloop-sdk`` and
  ``opentelemetry-exporter-otlp-proto-http`` runtime deps. ``uv sync``
  uninstalls ~30 transitive packages (the OTEL family,
  ``traceloop-sdk``, ``protobuf``, ``opentelemetry-exporter-otlp-*``,
  ``deprecated``, ``wrapt``, ``backoff``, etc.) — about 1000 lines off
  ``uv.lock``.
- Delete ``bootstrap_otel`` and ``JsonlSpanExporter`` from
  ``telemetry/utils.py``; strip the OTEL pruning helpers,
  ``parse_traceloop_headers``, ``default_resource_attributes``,
  ``format_trace_id`` / ``format_span_id`` / ``iso_from_unix_ns``.
  Keep only the sanitizer + JSONL writer + write-lock registry.
- Strip ``Tracer._setup_telemetry``, ``_otel_tracer``,
  ``_remote_export_enabled``, ``_active_events_file_path``,
  ``_active_run_metadata``, ``_get_events_write_lock``,
  ``_set_association_properties``. ``_emit_event`` now generates
  trace/span ids from ``uuid4`` directly.
- Drop the ``traceloop_base_url`` / ``traceloop_api_key`` /
  ``traceloop_headers`` / ``strix_otel_telemetry`` config knobs.
- Rename ``is_otel_enabled`` → ``is_telemetry_enabled`` (the gate now
  controls JSONL emission only).

Native session resume:
- ``entry.py`` now constructs an ``agents.memory.SQLiteSession`` keyed
  by ``scan_id`` and persists conversation history at
  ``strix_runs/<scan_id>/session.db``. A second call to
  ``run_strix_scan`` with the same ``scan_id`` resumes from where the
  prior run left off — no manual state plumbing needed.

Tracer.agents fix (TUI agent tree was silently empty):
- ``StrixOrchestrationHooks.on_agent_start`` now mirrors bus state
  into ``tracer.agents`` (id / name / parent_id / status), and
  ``on_agent_end`` flips the entry to ``completed`` / ``crashed``.
  The TUI now actually shows the agent tree during scans.

Tooling:
- Drop ``pylint`` from dev deps; ``ruff`` covers everything we used
  it for. Strip the ``make lint`` pylint step.
2026-04-25 13:18:21 -07:00
0xallam 7bcbd55dba refactor: collapse dual stat buckets, prune unused params, kill dead helpers
Tracer:
- Collapse the ``live`` / ``completed`` LLM stat buckets into one
  flat dict. The ``completed`` bucket was only ever written by tests
  — production never moved stats across, and ``get_total_llm_stats``
  always summed both for display.
- Drop ``record_llm_usage(agent_id=...)``: argument was unused, and
  the per-call ``bucket=`` knob is gone with the buckets.

run_config_factory:
- Drop unused ``parallel_tool_calls``, ``tool_choice`` parameters
  from ``make_run_config`` — no caller ever overrode them.
- Drop ``agent_name`` from ``make_agent_context`` — set into the
  context dict but no consumer ever read it; the bus's ``names`` map
  is the source of truth.

Wire reasoning_effort through:
- ``Config.get("strix_reasoning_effort")`` is now actually plumbed
  to ``make_run_config`` from ``entry.py``. Previously the env var
  was advertised but never consumed.

Multi-agent graph tools:
- Replace six copies of
  ``inner = ctx.context if isinstance(ctx.context, dict) else {}``
  with a single ``_ctx(ctx)`` helper.

Todo tools:
- Lift the duplicated ``priority_order`` / ``status_order`` dicts
  to module-level ``_PRIORITY_RANK`` / ``_STATUS_RANK`` and replace
  both inline sort lambdas with ``_todo_sort_key``.

Notes tools:
- Delete ``append_note_content`` (and its test): docstring claimed
  it was for an "agents-graph wiki-update hook on agent_finish" that
  was never wired up. Pure dead public API.

Style:
- Drop the ``del ctx`` no-ops from notes / reporting / web_search
  tools. ``ARG001`` is already silenced project-wide for tool
  modules; the ``del`` was cargo-culted.
2026-04-25 12:44:48 -07:00
0xallam ecd59f6b95 refactor: delete orphaned dirs, dead streaming infra, unused session/compressor
Orphaned files/dirs:
- ``strix/agents/StrixAgent/`` — empty, only ``__pycache__``.
- ``strix/tools/browser/litellm/`` — empty, only ``__pycache__``.
- ``strix/strix_runs/`` — runtime output left in the working tree.
- ``strix/prompts/`` — single Jinja template that nothing renders.

Dead streaming pipeline (was never wired in the SDK migration):
- Delete ``strix/interface/streaming_parser.py`` (XML tool-call parser
  for an output format the SDK doesn't produce).
- Strip ``streaming_content`` / ``interrupted_content`` dicts and
  five unused methods from ``Tracer``.
- Strip the streaming-render path + ``interrupted`` branch from TUI.
- Trim ``strix/llm/utils.py``: drop ``normalize_tool_format``,
  ``parse_tool_invocations``, ``format_tool_call``,
  ``fix_incomplete_tool_call`` and the XML-stripping in
  ``clean_content``. Keep only the inter-agent-XML scrub.

Unwired session compression:
- Delete ``strix/llm/strix_session.py`` and
  ``strix/llm/memory_compressor.py``. ``Runner.run`` was never called
  with a ``session=``, so the compressor never ran. Drop the matching
  test file and the ``strix_memory_compressor_timeout`` config knob.

Tracer cleanup:
- Remove ``log_agent_creation``, ``log_tool_execution_start``,
  ``update_tool_execution``, ``update_agent_status``,
  ``get_agent_tools`` — none had production callers.
- Rewrite the redaction + correlation tests against
  ``log_chat_message`` (which still emits events).
2026-04-25 12:21:59 -07:00
0xallam c996f5efd5 refactor: scrub migration scars, dead code, and unused helpers
- Strip PLAYBOOK / AUDIT / Phase-N / C-numbered references from
  module docstrings across 16 files; rename
  ``_PHASE1_PARALLEL_DEFAULT`` → ``_PARALLEL_TOOL_CALLS_DEFAULT``.
- Delete unused exception classes: ``SandboxInitializationError``,
  ``ImplementedInClientSideOnlyError``.
- Delete the no-op ``on_handoff`` hook (we don't use SDK handoffs).
- Delete the unreachable backward-compat tab-delimited fallback in
  ``_parse_git_diff_output``.
- Delete orphaned ``strix/tools/load_skill/`` (dir contained only a
  pycache) and stale pycache files.
- Rewrite ``strix/skills/__init__.py``: 168 → 56 LoC. Drop seven
  helper functions (``get_available_skills``, ``get_all_skill_names``,
  ``validate_skill_names``, ``parse_skill_list``,
  ``validate_requested_skills``, ``generate_skills_description``,
  ``_get_all_categories``) — none had external callers; only
  ``load_skills`` is used.
- Drop the stale ``strix/agents/sdk_factory.py`` per-file ruff ignore
  (file no longer exists).
2026-04-25 12:05:24 -07:00
0xallam d47b4374d1 refactor: nuke legacy harness, drop sdk_ prefixes
The SDK harness is the only path now; legacy host-side code is gone.
File names no longer carry the ``sdk_`` distinction.

Deleted legacy host-side modules:
- strix/agents/StrixAgent/ (template moved to strix/agents/prompts/)
- strix/agents/base_agent.py, state.py
- strix/llm/llm.py, config.py
- strix/runtime/docker_runtime.py, runtime.py
- strix/tools/executor.py, agents_graph/agents_graph_actions.py
- strix/interface/sdk_dispatch.py + the env-flag dispatch in cli.py

Renamed (drop ``sdk_`` prefix):
- strix/sdk_entry.py → strix/entry.py
- strix/agents/sdk_factory.py → strix/agents/factory.py
- strix/agents/sdk_prompt.py → strix/agents/prompt.py
- strix/tools/<x>/<x>_sdk_tool[s].py → strix/tools/<x>/tool[s].py
- strix/tools/_legacy_adapter.py → strix/tools/_state_adapter.py
- ``_legacy`` aliases inside the wrappers → ``_impl``

CLI + TUI now call ``run_strix_scan`` directly — they build the
sandbox image / sources_path locally and rely on
``session_manager.cleanup`` (called inside ``run_strix_scan``'s finally)
for teardown. Three TUI handlers that reached into legacy multi-agent
globals (``_agent_instances``, ``send_user_message_to_agent``,
``stop_agent``) are now no-ops with a TODO; reconnecting them to the
``AgentMessageBus`` is a follow-up.

Tracer.get_total_llm_stats no longer reaches into the deleted
``agents_graph_actions`` globals — the orchestration hooks now feed the
tracer via ``Tracer.record_llm_usage`` (live + completed buckets).
finish_scan's ``_check_active_agents`` and load_skill's runtime
``_agent_instances`` reach-in are no-op stubs; the
``AgentMessageBus`` is the source of truth post-migration.

llm/utils.py rewritten to keep only the streaming-parser helpers
(``normalize_tool_format``, ``parse_tool_invocations``,
``fix_incomplete_tool_call``, ``format_tool_call``, ``clean_content``).
``STRIX_MODEL_MAP`` moved to ``llm/multi_provider_setup.py`` (its only
remaining caller).

Per-file ruff ignores added for legacy interface modules (TUI / main /
CLI / utils / streaming_parser / tool_components) and tracer.py —
pre-existing PLC0415/BLE001/PLR0915 patterns are out of scope.

Tests: 287/287 passing. Renamed test files to drop ``sdk_`` prefix.
``test_tracer.py::test_get_total_llm_stats_aggregates_live_and_completed``
rewritten to feed ``Tracer.record_llm_usage`` instead of legacy globals.
Test file annotations added so pre-commit's strict mypy passes.
2026-04-25 09:30:23 -07:00
0xallam a093b5e3be feat(migration): phase 1 — Session + Tracer + RunConfig factory
Three foundation modules per PLAYBOOK §2.8 / §2.9 / §2.10 with all
relevant R2/R3 corrections (C7, C10, C11, C16, C21):

  strix/llm/strix_session.py            SessionABC wrapper around the
                                        legacy MemoryCompressor; on any
                                        compression failure, returns
                                        uncompressed history and
                                        permanently disables compression
                                        for the rest of the run (C10 +
                                        Round 3.4 W5/E2).

  strix/telemetry/strix_processor.py    SDK TracingProcessor that writes
                                        events.jsonl in our schema. All
                                        hooks SYNC per ABC (F3); writes
                                        protected by per-path
                                        threading.Lock (C7); OSError
                                        swallowed and logged (C16); PII
                                        scrubbed via the existing
                                        TelemetrySanitizer.

  strix/run_config_factory.py           make_run_config() with our
                                        defaults: parallel_tool_calls=
                                        False (C1 Phase-1 safe default),
                                        retry policy explicitly excludes
                                        401/403/400 (C11), reasoning
                                        effort + model_settings_override
                                        merge path (C21).
                                        make_agent_context() returns the
                                        canonical per-agent dict
                                        including is_whitebox/diff_scope/
                                        run_id (C21).

32 new smoke tests (197/197 total). mypy strict + ruff clean. Per-file
ignores added for tests/** S105/PT018 and for the two new src modules'
intentional broad-Exception catches (BLE001).
2026-04-25 00:01:05 -07:00
alex s 995059cd87 fix: ensure LLM stats tracking is accurate by including completed subagents (#441) 2026-04-13 00:09:13 -04:00
STJ db459db6e3 feat: Migrate from Poetry to uv (#379) 2026-03-31 17:20:41 -07:00
alex sand0xallam 508e473633 Add OpenTelemetry observability with local JSONL traces (#347)
Co-authored-by: 0xallam <ahmed39652003@gmail.com>
2026-03-09 01:11:24 -07:00
0xallam ae8f740c4a feat: Expose Caido proxy port to host for human-in-the-loop interaction
Users can now access the Caido web UI from their browser to inspect traffic,
replay requests, and perform manual testing alongside the automated scan.

- Map Caido port (48080) to a random host port in DockerRuntime
- Add caido_port to SandboxInfo and track across container lifecycle
- Display Caido URL in TUI sidebar stats panel with selectable text
- Bind Caido to 0.0.0.0 in entrypoint (requires image rebuild)
- Bump sandbox image to 0.1.12
- Restore discord link in exit screen
2026-02-23 18:37:25 -08:00
0xallam d4793567c1 fix: Remove indentation prefix from diff code block markers for syntax highlighting 2026-02-15 17:25:59 -08:00
0xallam 4e2fe786f3 feat: Redesign vulnerability reporting with nested XML code locations and CVSS
Replace 12 flat parameters (code_file, code_before, code_after, code_diff,
and 8 CVSS fields) with structured nested XML fields: code_locations with
co-located fix_before/fix_after per location, cvss_breakdown, and cwe.

This enables multi-file vulnerability locations, per-location fixes with
precise line numbers, data flow representation (source/sink), CWE
classification, and compatibility with GitHub/GitLab PR review APIs.
2026-02-15 17:25:59 -08:00
0xallam 2052d66e27 fix(agent): fix agent loop hanging and simplify LLM module
- Fix agent loop getting stuck by adding hard stop mechanism
- Add _force_stop flag for immediate task cancellation across threads
- Use thread-safe loop.call_soon_threadsafe for cross-thread cancellation
- Remove request_queue.py (eliminated threading/queue complexity causing hangs)
- Simplify llm.py: direct acompletion calls, cleaner streaming
- Reduce retry wait times to prevent long hangs during retries
- Make timeouts configurable (llm_max_retries, memory_compressor_timeout, sandbox_execution_timeout)
- Keep essential token tracking (input/output/cached tokens, cost, requests)
- Maintain Anthropic prompt caching for system messages
2026-01-14 18:54:45 -08:00
0xallam c0ee00c89c fix: correct GitHub repository URL in README 2026-01-10 15:53:10 -08:00
0xallam 543b785d1a feat: add centralized Config class with auto-save to ~/.strix/cli-config.json
- Add Config class with all env var defaults in one place
- Auto-load saved config on startup (env vars take precedence)
- Auto-save config after successful LLM warm-up
- Replace scattered os.getenv() calls with Config.get()
2026-01-10 15:49:03 -08:00
Ahmed Allam 34a3e57ff3 Remove title from README 2026-01-10 02:35:20 +04:00
0xallam 6e7a77cf78 Add PostHog integration for analytics and error debugging 2026-01-09 14:24:04 -08:00
0xallam 3cb150d1d0 feat(reporting): improve vulnerability display and reporting format 2026-01-07 19:51:41 -08:00
0xallam ea1bb0b0ed feat(reporting): add LLM-based vulnerability deduplication
- Add dedupe.py with XML-based LLM deduplication using direct litellm calls
- Integrate deduplication check in create_vulnerability_report tool
- Add get_existing_vulnerabilities() method to tracer for fetching reports
- Update schema and system prompt with deduplication guidelines
2026-01-07 19:32:45 -08:00
0xallam 7930db963b feat(reporting): enhance vulnerability reporting with detailed fields and CVSS calculation 2026-01-07 17:50:32 -08:00
0xallam 013806f4d2 feat(agent): implement user interruption handling in agent execution 2026-01-06 16:44:22 -08:00
0xallam 82ec0f8266 feat(tui): add real-time streaming LLM output with full content display
- Convert LiteLLM requests to streaming mode with stream_request()
- Add streaming parser to handle live LLM output segments
- Update TUI for real-time streaming content rendering
- Add tracer methods for streaming content tracking
- Clean function tags from streamed content to prevent display
- Remove all truncation from tool renderers for full content visibility
2026-01-06 16:44:22 -08:00
Vincent550102 05cfdd7337 fix: Convert dictionary views to lists for stable iteration over agents and tool executions. 2026-01-02 14:17:32 -08:00
Ahmed Allam e920585e5e feat: implement incremental pentest data persistence 2025-11-22 22:54:07 +04:00
Ahmed Allam 23ace6aac2 feat: implement multi-target scanning 2025-11-01 02:38:37 +02:00
Ahmed Allam ec45205fce refactor: Migrate tracer to new telemetry module 2025-10-31 21:07:21 +02:00