Files
mattermore/DESIGN.md
T

662 lines
26 KiB
Markdown

# Design: Mattermore — Mattermost AI Chat Agent for Ollama
> **Status:** Draft / Pre-implementation
> **Last updated:** 2026-06-16
> **Assumed direction:** AI chat agent plugin connecting to a remote Ollama instance.
> **Open questions are marked** `TODO(design):` throughout. This is a living template — change
> assumptions as the product direction firms up.
---
## 1. Purpose & Scope
### 1.1 What It Is
Mattermore is a Mattermost (Community Edition) plugin that lets users chat
with remote LLMs through an Ollama backend. Users address the bot by
**@-mentioning** it in any channel (e.g. `@mattermore write a poem`). The
bot responds **only when addressed** and disengages naturally — it never
reads every message in the room. A slash command (`/ai`) remains available
as an alternative entry point.
### 1.2 What It Is Not (Explicit Out-of-Scope)
| Out of scope | Rationale |
|---|---|
| Hosting its own LLM inference | Ollama is the sole backend; the plugin is a thin proxy |
| Multi-backend abstraction (OpenAI, Anthropic, etc.) | If needed later, abstract behind an `LLMProvider` interface — not now |
| Training / fine-tuning | Ollama handles that |
| **Channel whisper mode** — replying to every message in a channel | The agent only activates on @-mention or thread reply. It never reads unprompted messages. |
### 1.3 Target Audience
- Teams self-hosting Mattermost who want AI assistant access without data
leaving their infrastructure
- Users comfortable with @-mentioning a bot the same way they @-mention a
human teammate
---
## 2. Architecture Overview
### 2.1 Component Diagram (text)
```
┌──────────────────────────────────────────────────────────────┐
│ Mattermost Server (Community Edition) │
│ │
│ ┌───────────────────────────────────────────────────────┐ │
│ │ Mattermore Plugin │ │
│ │ │ │
│ │ ┌──────────────────┐ ┌────────────────────────┐ │ │
│ │ │ MessageHasBeen │──▶│ Plugin (Go) │ │ │
│ │ │ Posted (primary) │ │ │ │ │
│ │ └──────────────────┘ │ ┌──────────────────┐ │ │ │
│ │ ┌──────────────────┐ │ │ OllamaClient │──┼───┼───┼──▶ Ollama
│ │ │ SlashCommand │──▶│ └──────────────────┘ │ │ │ (remote)
│ │ │ /ai (alt.) │ │ ┌──────────────────┐ │ │ │
│ │ └──────────────────┘ │ │ ConversationStore│ │ │ │
│ │ ┌──────────────────┐ │ └──────────────────┘ │ │ │
│ │ │ EngagementEngine │ │ ┌──────────────────┐ │ │ │
│ │ │ wake / sleep │──▶│ │ RateLimiter │ │ │ │
│ │ └──────────────────┘ │ └──────────────────┘ │ │ │
│ │ ┌──────────────────┐ │ ┌──────────────────┐ │ │ │
│ │ │ Bot Account │ │ │ Config (KV) │ │ │ │
│ │ │ (auto-created) │ │ └──────────────────┘ │ │ │
│ │ └──────────────────┘ │ │ │ │
│ │ ┌──────────────────┐ │ │ │ │
│ │ │ Webapp (none) │ │ │ │ │
│ │ └──────────────────┘ │ │ │ │
│ └──────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
```
### 2.2 Request Flow (happy path)
```
User types "@mattermore write a release note for v2.1"
Mattermore MessageHasBeenPosted fires
EngagementEngine.Wake(userId, channelId) → mark session Active
@-mention detected → strip "@mattermore " prefix from prompt
RateLimiter.Allow(userId) → 200 or 429
ConversationStore.GetSession(userId, channelId, threadId) → previous messages for context
OllamaClient.ChatCompletion(messages, stream=true, model=defaultModel)
Ollama returns ndjson stream → plugin creates root post with live-updating thread
Post created in channel → user sees response in thread
ConversationStore.Append(userId, channelId, threadId, userMsg, assistantMsg)
```
#### Thread Reply Flow (continues the conversation)
```
User replies in thread → "Can you make it more formal?"
Mattermore MessageHasBeenPosted fires (same thread as bot post)
EngagementEngine.IsActive(userId, threadId) → true
Not an @-mention, but active thread → treat as continuation
ConversationStore.GetSession(userId, channelId, threadId) → full context so far
OllamaClient.ChatCompletion(...) → stream response into same thread
ConversationStore.Append(userId, channelId, userMsg, assistantMsg)
```
### 2.3 Error Flow
| Failure point | Behaviour |
|---|---|
| Ollama unreachable | Ephemeral post: "Ollama at <url> unreachable. Check configuration." |
| Rate limit hit | Ephemeral post: "Rate limit exceeded. Try again in N seconds." |
| Invalid model name | Ephemeral post: "Model 'xyz' not found on Ollama server." |
| Timeout (>30 s) | Abort stream, post partial response + "Response truncated (timeout)." |
| Input too long | Reject with context window limit message; suggest `@mattermore new` to reset. |
---
## 3. Plugin Manifest & Identity
### 3.1 `plugin.json`
```json
{
"id": "com.forkless.mattermore",
"name": "Mattermore",
"description": "AI chat agent powered by Ollama.",
"version": "0.1.0",
"server": {
"executables": {
"linux-amd64": "server/dist/plugin-linux-amd64",
"linux-arm64": "server/dist/plugin-linux-arm64",
"darwin-amd64": "server/dist/plugin-darwin-amd64"
}
},
"webapp": {},
"settings_schema": {
"header": "Configure the connection to your Ollama instance.",
"footer": "",
"settings": [
{
"key": "OllamaURL",
"display_name": "Ollama Server URL",
"type": "text",
"help_text": "Base URL of the Ollama API, e.g. http://10.0.0.5:11434",
"placeholder": "http://localhost:11434",
"default": "http://localhost:11434"
},
{
"key": "DefaultModel",
"display_name": "Default Model",
"type": "text",
"help_text": "Model name to use when none is specified.",
"placeholder": "llama3:latest",
"default": "llama3:latest"
},
{
"key": "SystemPrompt",
"display_name": "System Prompt",
"type": "text",
"help_text": "System prompt prepended to every conversation.",
"placeholder": "You are a helpful assistant.",
"default": "You are a helpful assistant."
},
{
"key": "MaxTokens",
"display_name": "Max Response Tokens",
"type": "number",
"help_text": "Maximum tokens per response (0 = model default).",
"default": 2048
},
{
"key": "RateLimitPerMinute",
"display_name": "Rate Limit (requests/minute/user)",
"type": "number",
"help_text": "Maximum requests per minute per user. 0 = unlimited.",
"default": 10
},
{
"key": "AllowedUserIDs",
"display_name": "Allowed User IDs (comma-separated)",
"type": "text",
"help_text": "Restrict to specific user IDs. Empty = all users.",
"default": ""
},
{
"key": "StopWords",
"display_name": "Stop Words (comma-separated)",
"type": "text",
"help_text": "When the user says one of these, the bot responds briefly and disengages.",
"placeholder": "thanks, thank you, bye, goodbye, that's all, done",
"default": "thanks, thank you, bye, goodbye, see you, that's all, that's it, done, all done, stop, quit, end"
}
]
}
}
```
`TODO(design):` Should `AllowedUserIDs` be a team/channel whitelist instead?
For now, keep user-level granularity — easy to widen later.
### 3.2 Versioning
Follow `release-workflow` skill: `v0.1.0`, `v0.2.0`, etc. Bump PATCH for
bugs, MINOR for features. MAJOR after 1.0 for breaking config schema changes.
---
## 4. Server-Side Components
### 4.1 Plugin Lifecycle Hooks
| Hook | Purpose |
|---|---|
| `OnActivate()` | Register slash command, create bot account via `API.CreateBot()`, initialise `EngagementEngine` |
| `OnConfigurationChange()` | Re-read System Console settings, validate OllamaURL, rebuild clients, rebuild stop-word trie |
| `MessageHasBeenPosted()` | **Primary entry point** — detect @-mentions and thread continuations, delegate to `EngagementEngine` |
| `ExecuteCommand()` | **Alternative entry point** — intercept `/ai` and subcommands |
| `OnDeactivate()` | Clean up goroutines, deactivate bot account, close idle HTTP connections |
### 4.2 Interaction Model
The plugin has two entry points, but **@-mention is the primary UX**.
#### 4.2.1 @-mention (primary)
User types `@mattermore <prompt>` in any channel. The plugin:
1. Detects the @-mention via `MessageHasBeenPosted``model.Post.Mentions`
2. Strips the bot username from the prompt
3. Wakes the `EngagementEngine` for this user+thread
4. Creates a **thread root post** with the streamed response
5. Monitors thread replies for continuation — as long as the user keeps
replying in the thread, the bot auto-responds (no @-mention needed for
follow-ups in the same thread)
**When the bot does NOT respond:**
- Messages without `@mattermore` while the EngagementEngine is Sleeping
- Messages in threads the bot didn't start
- Messages from other bot accounts (infinite-loop guard)
#### 4.2.2 Slash command (alternative)
`/ai <prompt>` works as a fallback for users who prefer it or when
@-mention is inconvenient. Behaviour is identical — creates a thread,
activates the EngagementEngine, continues on thread replies.
**Full syntax:**
```
/ai <prompt>
— Send a prompt using the default model (current session context).
/ai model <name> <prompt>
— Send a prompt using a specific model.
/ai new
— Reset conversation context for the current user+channel.
/ai model list
— List available models from the Ollama server.
/ai help
— Show usage help.
```
### 4.3 OllamaClient
Interface (in `server/ollama/client.go`):
```go
type Client interface {
ChatCompletion(ctx context.Context, req *ChatRequest) (<-chan ChatStreamEvent, error)
ListModels(ctx context.Context) ([]Model, error)
Ping(ctx context.Context) error
}
type ChatRequest struct {
Model string `json:"model"`
Messages []ChatMessage `json:"messages"`
Stream bool `json:"stream"`
Options map[string]any `json:"options,omitempty"`
}
type ChatStreamEvent struct {
Token string // delta content
Done bool
Error error
}
```
`TODO(design):` Should we use Ollama's `/api/chat` (chat format) or
`/api/generate` (raw prompt)? `/api/chat` is preferred — it maps cleanly to
the conversational UX and supports the messages array natively.
### 4.4 ConversationStore
Store session context in Mattermost's KV store (plugin's `API.KVSet` /
`KVGet`).
| Key pattern | Value | TTL |
|---|---|---|
| `ctx_{userID}_{channelID}` | `[]Message` (last N turns) | 1 hour |
| `ctx_{userID}_{channelID}_model` | Model name override | 1 hour |
`TODO(design):`
- How many turns to keep? **Proposed:** last 10 messages (5 user + 5 assistant).
- TTL of 1 hour — resets on activity. Configurable?
- Should context be channel-scoped or user-global? Channel-scoped means a
user gets different context in different channels.
### 4.5 RateLimiter
Token-bucket per user. Configurable requests/minute from System Console.
Resets on config change. Stored in memory, not KV (ephemeral state).
### 4.6 Engagement Lifecycle
The **EngagementEngine** is the core mechanism that prevents the bot from
replying to everything in the channel. It manages whether the bot should
respond to a given message.
#### States
| State | Behaviour | Entered when |
|---|---|---|
| **Sleeping** | Ignores all messages | Agent started, disengage triggered, timeout expired |
| **Active** | Responds to @-mentions and thread replies in the active session | User @-mentions bot or sends `/ai` |
| **Disengaging** | Responds to current message with a closing note, then sleeps | Stop word detected or user sends stop command |
```
┌─────────────────┐
│ Sleeping │
└────────┬────────┘
│ @-mention or /ai
┌─────────────────┐
┌──────│ Active │◄──── thread reply continues
│ └────────┬────────┘
│ │ stop word / stop command / timeout
│ ▼
│ ┌─────────────────┐
│ │ Disengaging │── responds once, then...
│ └────────┬────────┘
│ │
└───────────────┘
```
#### Disengagement Triggers
| Trigger | Example | Behaviour |
|---|---|---|
| Stop word | "thanks", "thank you", "bye", "goodbye", "that's all", "that's it" | Bot responds with a brief closing message, then sleeps |
| Stop command | `@mattermore stop` | Bot sleeps immediately, no closing message |
| Thread timeout | No reply in thread for 15 minutes | Bot sleeps silently — next @-mention starts a fresh session |
| Model EOS signal | Model outputs `[EOS]` naturally | Same as stop word — clean disengagement from the AI's side |
#### Stop word matching
Stop words are configurable via System Console (`StopWords`). The plugin
builds a case-insensitive trie at config load. Matching is done on the
**user's raw prompt** (after stripping the @-mention), before sending to
Ollama — this avoids the latency and cost of a round-trip just to detect
a goodbye.
The system prompt also instructs the model to append `[EOS]` when it
believes the conversation is naturally complete. The EngagementEngine
watches the response stream for this token as an additional trigger.
#### Thread timeout
15 minutes of inactivity in the thread → bot sleeps. The `ConversationStore`
TTL naturally handles cleanup. On wake, a new thread is created rather than
resuming the old one.
`TODO(design):` Should the timeout be configurable via System Console?
**Proposed:** yes — `EngagementTimeoutMinutes` setting, default 15.
#### Infinite-loop guard
The plugin never processes messages from:
- Its own bot account (checked via `post.UserId == botUserID`)
- Any other bot account (`post.UserId` cross-referenced against `model.Bot`)
---
## 5. Ollama Integration
### 5.1 API Mapping
| Plugin Action | Ollama Endpoint | Method |
|---|---|---|
| Chat completion (streaming) | `/api/chat` | POST |
| List models | `/api/tags` | GET |
| Health check | `/api/tags` (or HEAD `/`) | GET |
| Pull model | `/api/pull` | POST |
`TODO(design):` Should the plugin auto-pull a model if it's not present?
Risk: user mistypes name → long pull. **Proposed:** return error with list of
available models instead.
### 5.2 Streaming Strategy
1. Send POST to `/api/chat` with `"stream": true`.
2. Read ndjson response body line by line (each line is `{"message":{"role":"assistant","content":"..."},"done":false}`).
3. On each `done:false` event, update the thread root post via
`API.UpdatePost()` with accumulated content.
4. On `done:true`, finalise the post with full content, usage stats
(token count, duration).
`TODO(design):` Post-update on every token may be too chatty. **Proposed:**
batch updates every ~200 ms or every 3 tokens, whichever comes first.
### 5.3 Context Window Management
- Token counting: use Ollama's `num_ctx` parameter or a Go tokenizer
(tiktoken-go).
- If input exceeds configured limit (default: 4096 tokens), trim oldest
messages until within limit, or reject if a single message overflows.
- Display a warning: "Context window trimmed (oldest messages removed)."
`TODO(design):` Should the user be able to set `num_ctx` per-request?
`@mattermore --context 8192 ...`
### 5.4 Model Selection Precedence
1. Per-request: `@mattermore model gemma3:12b write a poem`
2. Session override: `/ai model gemma3:12b` (stored in KV)
3. Default from System Console: `DefaultModel`
---
## 6. Configuration
### 6.1 System Console Settings
Defined in `plugin.json` `settings_schema` (see §3.1). The System Console
provides the UI; the plugin reads via `OnConfigurationChange()`.
### 6.2 Dynamic Config Reload
`OnConfigurationChange()` is called on every save. The plugin must:
1. Re-read the config struct from `API.GetConfig()` / `API.GetPluginConfig()`.
2. Validate `OllamaURL` (parse as URL, reject non-HTTP(S) schemes).
3. If URL changed, create a new `http.Client` (with configurable timeout).
4. If `RateLimitPerMinute` changed, rebuild the token buckets.
5. If `StopWords` changed, rebuild the stop-word trie.
6. Log the change at debug level.
---
## 7. Data Flow — Detailed
### 7.1 Full Lifecycle
```
Step User / System Component
──── ────────────────────────────── ──────────────────
1 User types "@mattermore explain TCP" Mattermost webapp
2 MessageHasBeenPosted fires Mattermore plugin
3 EngagementEngine.Wake() EngagementEngine
3a Check not a bot post (loop guard)
3b Check @-mention or active thread
3c Transition to Active
4 Strip "@mattermore " → "explain TCP"
5 Check stop-word trie (no match)
6 RateLimiter.Allow(userId) RateLimiter
7 ConversationStore.GetSession(...) ConversationStore
8 Build []ChatMessage (system + history + new prompt)
9 OllamaClient.ChatCompletion(...) OllamaClient
10 Ollama streams ndjson tokens Ollama (remote)
11 Plugin accumulates tokens, updates Mattermore plugin
thread post every ~200ms
12 On done:true → finalise post Mattermore plugin
13 Check response for [EOS] signal EngagementEngine
14 ConversationStore.Append(...) ConversationStore
```
### 7.2 Disengagement Flow
```
Step User / System Component
──── ────────────────────────────── ──────────────────
1 User replies "thanks that's all"
2 MessageHasBeenPosted fires
3 EngagementEngine.IsActive → true
4 Strip @-mention if present
5 Stop-word trie matches "that's all" StopWordTrie
6 EngagementEngine.Transition(Disengaging)
7 OllamaClient.ChatCompletion(lastMsg) (brief closing response)
8 Post closing message in thread
9 EngagementEngine.Transition(Sleeping)
```
### 7.3 Thread Participation
Thread replies continue the conversation without requiring a new @-mention.
The EngagementEngine tracks which threads it owns (via `ownerThread_{threadID}`
in KV). For posts in owned threads:
- If the EngagementEngine is Active for this user+thread → respond
- If Sleeping → respond once to wake, then continue
- If Disengaging → respond with closing note, then sleep
---
## 8. Security Model
### 8.1 Input Sanitization
- Strip control characters / zero-width Unicode from prompts before sending
to Ollama.
- Limit prompt length to 4096 characters (configurable via System Console).
- `TODO(design):` Should we block prompts that look like prompt injections?
("Ignore previous instructions...") Hard to do reliably — document risk
instead.
### 8.2 Output Safety
- Cap response tokens in the Ollama request (`options.num_predict`).
- `TODO(design):` Add a "Report this response" feedback mechanism? Defer.
### 8.3 Network Security
- `OllamaURL` must start with `http://` or `https://` — validate at config
load.
- Reject internal IP ranges unless explicitly enabled by a
`AllowPrivateNetworks` toggle (default: off for production).
- TLS verify enabled by default; optional `SkipTLSVerify` toggle (logged as
a warning).
### 8.4 Authorization
- If `AllowedUserIDs` is non-empty, reject requests from other users with a
"not authorized" ephemeral message.
- All plugin API calls are already scoped to authenticated Mattermost users.
### 8.5 Rate Limiting
- Per-user token bucket, replenish rate set via System Console.
- Default: 10 requests/minute. 0 = unlimited (logged as a warning).
- Burst: 1 (enforce smooth spacing).
---
## 9. Observability
### 9.1 Logging
| Level | When |
|---|---|
| `ERROR` | Ollama unreachable, config validation failure, KV store error |
| `WARN` | Rate limit hit, context window trimmed, TLS skipped |
| `INFO` | Plugin activated/deactivated, config changed, first request |
| `DEBUG` | Every request/response round-trip (with userID, model, token count, latency, engagement state) |
### 9.2 Metrics (future)
`TODO(design):` Expose Prometheus-style counters via a `/metrics` endpoint
if the Mattermost server has plugin metrics support:
- `mattermore_requests_total{user,model,status}`
- `mattermore_latency_seconds`
- `mattermore_tokens_total{direction="input|output"}`
- `mattermore_rate_limit_hits_total`
### 9.3 Error Reporting
Ephemeral posts with user-facing messages (not raw Go errors). Internal
errors logged at `ERROR` with stack trace.
---
## 10. Open Design Decisions
These are the unresolved questions that need answers before v0.1.0 ships.
| # | Question | Proposed | Alternatives | Decided? |
|---|---|---|---|---|
| D1 | Reply style | **Thread posts** (not ephemeral) — community sees the interaction | Ephemeral, channel-wide | ✅ → DECISIONS.md |
| D2 | Context scope | Per-channel per-user — each channel gets its own conversation | Per-user global, per-channel only, no context | ❌ |
| D3 | Context turns | Last 5 exchanges (10 messages) | Last N tokens, everything in TTL, sliding window | ❌ |
| D4 | Webapp UI | **None** — @-mention + thread replies cover all interaction | Channel header button, RHS panel | ✅ → DECISIONS.md |
| D5 | Model governance | All models available | Admin-restricted model list, least-privilege Ollama API key | ❌ |
| D6 | Thread replies | **Core feature** — the EngagementEngine auto-follows thread replies | Slash-command-only, ephemeral | ✅ → DECISIONS.md |
| D7 | `num_ctx` per request | Via `--context N` flag | System Console default only | ❌ |
| D8 | Auto-pull models | No (error + list available) | Yes (convenient but slow) | ❌ |
| D9 | Prompt injection guard | Defer (document risk) | Regex blocklist, LLM-as-judge pre-filter | ❌ |
| D10 | Metrics endpoint | Defer | `/metrics` via plugin HTTP handler | ❌ |
| D11 | Engagement scope | Per-user+channel | Per-user global, per-channel only | ❌ |
| D12 | Thread timeout | 15 minutes, configurable | Fixed 30min, no timeout | ❌ |
Each decision should be recorded in `DECISIONS.md` once resolved.
---
## 11. Deployment & Lifecycle
### 11.1 Build
```bash
make dist
```
Produces `dist/com.forkless.mattermore-0.1.0.tar.gz` containing the server
binaries and `plugin.json`.
### 11.2 Upload
Via Mattermost System Console → Plugins → Upload Plugin, or via REST API:
```bash
curl -X POST $MM_URL/api/v4/plugins \
-H "Authorization: Bearer $MM_TOKEN" \
-F "plugin=@dist/com.forkless.mattermore-0.1.0.tar.gz"
```
### 11.3 Upgrade
1. Upload new `.tar.gz` — Mattermost replaces the plugin binaries.
2. Plugin's `OnConfigurationChange()` fires (config preserved).
3. Bot account is re-fetched (not re-created) — idempotent.
4. If the KV schema changed, a migration in `OnActivate()` handles it.
### 11.4 Rollback
Re-upload the previous version's `.tar.gz`. No data migration needed if the
KV key schema hasn't changed. Bot account survives rollback. If it has,
restore from backup or `DECISIONS.md` records the migration path.
---
## 12. Future Considerations (v0.2+)
- **Multi-modal inputs** — Ollama supports images via `/api/chat` with
`images` field. Could allow image attachment in thread.
- **Multi-model routing** — Different models for different channel categories
(e.g. code-review channel uses codellama).
- **Tool calling** — Ollama supports tool definitions. Could let the
plugin fetch data from external APIs (weather, Jira, etc.).
- **Streaming into message attachments** — Render structured data (tables,
JSON, code blocks) with better formatting.
- **Conversation export** — Allow users to export an AI thread as markdown.