forkless/strix
1
0
Fork 0
mirror of https://github.com/usestrix/strix.git synced 2026-08-16 09:26:39 +02:00
Code Issues Packages Projects Releases Wiki Activity

feat(llm): opt-in LLM_DISABLE_STREAMING for non-streaming OpenAI-compatible endpoints

Browse Source 
Some OpenAI-compatible gateways don't support Server-Sent Events (or
deliver them unreliably), but the SDK run loop Strix uses only issues
streamed requests, so such a gateway fails every turn. Add an opt-in
LLM_DISABLE_STREAMING setting that wraps the resolved model in
_NonStreamingModel: each turn makes one non-streaming get_response and
replays the completed result as a single terminal stream event, so tool
calls, usage, and the rest of the agent loop are unchanged. Subscription
(ChatGPT) models are always streamed and are not wrapped.
This commit is contained in:
Ahmed Allam
2026-07-30 08:30:06 +03:00
committed by Ahmed Allam
parent d4e58b2cd0
commit 980216860e
5 changed files with 404 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1
View File
@@ -262,6 +262,7 @@ export LLM_API_KEY="your-api-key"
export LLM_API_BASE="your-api-base-url" # if using a local model, e.g. Ollama, LMStudio
export PERPLEXITY_API_KEY="your-api-key" # for search capabilities
export STRIX_REASONING_EFFORT="high" # control thinking effort (default: high, quick scan: medium)
export LLM_DISABLE_STREAMING="true" # for OpenAI-compatible endpoints that don't support streaming
```
> [!NOTE]