buildWikiContext enumerated an agent's bound knowledge-base pages into
the system prompt capped only by maxContextChars (default 10000, sized
for large cloud models). On a small-context model a large KB therefore
consumed a big fixed slice of the window on every turn — the "tool token
estimate fills the context" report in #521 (the growth lands in the
system-prompt bucket, not the tool-schema bucket; wiki tool schemas are
fixed-size and do not scale with file count).
Add a budgeted buildWikiContext(agentId, budgetTokens) overload mirroring
buildRelevantContext: the page enumeration also stops once the estimated
token total exceeds the budget, appending the existing
'... and more (use wiki_list_pages)' hint. AgentGraphBuilder passes the
same prefix budget it already applies to the memory block; the legacy
Integer.MAX_VALUE path keeps chars-only behavior for large models.
Tests cover null-budget (all pages), token-budget truncation, and
zero-budget skip.
Wire MCP standard notifications/progress into the existing SSE stream so long-running MCP tool calls surface live progress instead of a bare spinner. A per-call progressToken maps back to (conversationId, toolCallId); ProgressAwareMcpToolCallback injects it into tools/call _meta and calls McpSyncClient directly (falling back to the delegate on error, and applying identity forwarding first). Progress events skip the ring buffer and are replayed from a latest-value snapshot on SSE reconnect. Frontend renders a gradient progress bar in ToolCallSegment when a running tool reports progress.
- ProgressLedgerService.upsert: the reserved-prefix guard rejects the write; the
comment said 'strip the prefix and continue', which no longer matches. Rewrite
it to describe the actual reject behavior.
- ActionNode: translate the class javadoc (including the B2 pinned-constraints and
B5 auto-backfill notes) to English per code style.
The identity forwarded to opt-in MCP servers was a one-dimensional string
(ChatOrigin.requesterId): a MateClaw username for web logins, but a webchat
visitorId for visitors and an IM sender id for IM — indistinguishable to the
REST backend. The signed-token mode (d204b702) made this worse: an RS256
signature over an unauthenticated visitorId reads as "MateClaw authenticated
this user" to any backend that trusts the signature.
Introduce an identity-typing dimension at McpIdentityForwardService:
- classify() branches on ChatOrigin: authenticated (web login, sub=immutable
userId), anonymous (webchat visitor, trust=anonymous), external (IM sender,
trust=external), or none (cron/system → nothing injected, fail-closed).
- mint() adds `trust` and `channel_type` claims; plaintext value is prefixed
`trust:subject` so backends can tell the kinds apart without a JWT.
The immutable userId reaches resolve() without coupling it to the user store:
JwtAuthFilter stamps user.id into auth.setDetails() (both JWT and PAT paths),
and ChatController.memoryOrigin carries it on a new ChatOrigin.requesterUserId
field (only-add, per the record's evolution rule).
Resolves the webchat semantic mismatch raised in #459 and the "sub should be
an immutable user id" follow-up. 82 tests green (4 identity classes covered
with claim assertions + full ChatOrigin/MCP regression).
(cherry picked from commit b5d2cfbf98b39848d7139c743a0b81fea71e8ffe)
Lets an employee pin an ordered fallback chain of (provider, model) entries; the same provider may appear multiple times with different models. Build-time dedup keys on exact (provider, model).
- resolveAgentBasePath: the relative-override branch now normalizes the
resolved path and rejects values that escape the workspace root via "../"
(the absolute branch already did this), keeping attachment/media/tool I/O
contained when an agent's workspaceBasePath is a relative override.
- cleanAttachmentFiles: return early on a null/blank conversationId so a bare
upload root can never be walked and deleted wholesale.
- Translate the chat-upload Javadoc/comments to English (cleanAttachmentFiles,
BaseAgent image-path resolver) per code style.
- Add a resolver test for the relative-override escape fallback.
Strict OpenAI-compatible providers reject the /chat/completions request
with HTTP 400 when an assistant message in history carries a tool call
whose function.arguments is not parseable JSON. Normalize blank or
non-JSON arguments to "{}" at the send chokepoint so streaming,
history-replay, and older-persisted tool calls all stay well-formed.
execute_code (bash/sh/shell) bypassed the workspace boundary guard, so shell
code run through it could read/write/delete paths outside the workspace sandbox
(e.g. cat /etc/passwd) while the same paths were blocked for read_file and the
shell tools. Bring execute_code under the guard (scan only shell-language code,
report the code param), and trust the tool-result spill roots so a legitimate
spilled result stays readable. Adds regression tests.
- Remove mateclaw-ui/package-lock.json and yarn.lock. This is a pnpm
monorepo where pnpm-lock.yaml is the only lockfile; the npm/yarn locks
were stray duplicates. Add a .gitignore rule so they are not committed
again by mistake.
- SourceEvidenceLedger: reference Pattern/Matcher by their imported
simple names instead of inline fully-qualified names.
Backend (SourceEvidenceLedger):
- appendWikiSourceTable now normalizes existing source lines in-place to
canonical "[N] Title - section - page N" format instead of skipping them
- Added replaceSourceLine helper that matches a full source line by regex
and replaces it with the canonical form
- When source lines exist without a "来源：" header, automatically insert
one so the frontend preprocessor can locate the source table
Frontend (useMarkdownRenderer):
- Added data-citation-index / data-citation-title to DOMPurify whitelist
- Added preprocessWikiCitations preprocessor: parses the canonical source
table to build an index-to-title map, replaces [n] markers in the answer
body with clickable <a> links, and wraps entire source-table rows so the
full line is clickable
- Integrated into the render pipeline after wikilink substitution and
before Marked parsing
Frontend (useGlobalWikilinkClick):
- Extended the click delegation selector to match both .wiki-link and
.wiki-citation elements
- Title extraction falls back: data-citation-title || data-wiki-title
Tests: added three test cases for source-line normalization, idempotency,
and automatic header insertion
Live lifecycle board (grid<->board toggle) plus an assignee-swimlane plan
board that groups follow-up re-runs of one goal into a single xN card.
Custom right-side detail/goal panels with markdown output. Fixes plans
being persisted under the per-run trace id so the board actually populates.
Closes#385
Issue #304. Operators who want an agent with NO knowledge base had no
way to express it: leaving the KB picker empty fell through to "inherit
workspace-wide" (every KB visible), so the agent ended up ingesting
every KB's context. This adds the same opt-out toggle that
skills_disabled (V126) / tools_disabled already provide.
Backend:
- V154 migration (h2 + mysql + kingbase): mate_agent.wiki_disabled
BOOLEAN/TINYINT/SMALLINT NOT NULL DEFAULT FALSE. Legacy agents stay
bit-identical.
- AgentEntity.wikiDisabled: Boolean field, @TableField("wiki_disabled").
- AgentBindingService.getBoundKbIds: short-circuit at the top —
wiki_disabled=true returns Set.of() regardless of binding rows. Mirrors
the precedence contract of getBoundSkillIds vs skills_disabled.
- AgentBindingService.setKbBindings: a non-empty save auto-clears a
stale wiki_disabled flag (same contract as setSkillBindings /
setToolBindings on their respective flags). Empty saves leave the flag
untouched — the UI toggle owns the bit, not the binding writer.
- AgentBindingServiceWikiDisabledTest: 5 cases covering all three
return states + the stale-flag auto-clear + empty-save no-op.
Frontend:
- Agents.vue KB picker: add the "此智能体不使用任何知识库" /
"This agent uses no knowledge bases" toggle, mirroring the skills /
tools picker layout. Tab badge shows "Off" when the toggle is on.
- types/index.ts: add Agent.wikiDisabled?: boolean.
- Save logic: when wikiDisabled is on, send an empty KB list (the
setKbs contract then leaves the flag alone server-side, exactly as
setSkills / setTools behave for their opt-out flags).
- i18n (zh + en): new strings for toggle label, hint, badge, and the
scope description shown when the toggle is on.
Stacked on top of #382 (which introduced AgentBindingResolver
.getBoundKbIds). No agent-runtime changes — wiki tools already degrade
cleanly when getBoundKbIds returns Set.of().
Add GET /api/v1/channels/webchat/wiki/pages mirroring /skills, so
downstream integrators can build a [[slug]] picker UI that points the
LLM at specific wiki pages. The picker token format is the universal
Obsidian/Wikipedia wikilink convention; the LLM consumes [[slug]] via
the existing wiki_read_page(slug=...) tool, so no agent-runtime changes
are needed.
- AgentBindingResolver.getBoundKbIds(agentId): three-state mirror of
getBoundSkillIds. null = no rows (fall through to workspace-wide KBs),
Set.of() = explicitly scoped to zero KBs, non-empty = explicit scope.
- WebChatController.listWikiPages: API Key + visitorToken auth chain,
agentId workspace anti-escalation, visibility excludes pageType=
synthesis (LLM intermediate artifacts), 100-page cap forces keyword
filter, response carries only display-level metadata.
- WebChatWikiPageView DTO: kbId/kbName/slug/title/summary/pageType;
content/embedding/sourceRawIds deliberately stay admin-console-only.
- WikiTool.wiki_read_page @Tool description: document the [[slug]]
convention so the LLM treats each token as a wiki-page reference.
- WebChatWikiPageListTest: 8 cases covering happy path, keyword filter,
synthesis exclusion, anti-escalation, auth failures, cap behavior,
and the no-binding → workspace-wide fallback.
Closes#381.
Turn a single natural-language requirement into a ready-to-review
employee: the model proposes name, persona, runtime type and a
validated set of skills/tools/knowledge base, which the user confirms
or tweaks before the agent is created.
- backend: POST /api/v1/agents/generate builds a draft from the
workspace's real capability catalog; every suggested tool/skill/KB is
re-validated against the catalog so nothing hallucinated is offered
- frontend: 3-step wizard at /agents/create reusing the existing
create + binding endpoints; reusable capability picker shows selected
items as compact chips with an on-demand searchable catalog
- goal: continue (not skip) on max-iterations and evidence-insufficient turns.
A max-iterations turn grants a fresh iteration budget ("hard continuation"),
bounded per run and sized into the graph recursion ceiling, so a task too big
for one budget keeps going instead of stalling until the next user message.
- plan-execute: re-plan the remaining work on a step exception, and on a
signature-based stall (repeated failures / identical results / no usable
result) instead of advancing dependent steps with junk; bounded by a per-run
re-plan cap, with a graduated change-strategy nudge before the hard stop.
- plan-execute: auto-derive a goal from a genuine multi-step plan, seeding the
acceptance criteria from the plan steps, so the goal subsystem engages without
the model calling setGoal; broadcast goal_created so the UI hydrates.
- react: refund the iteration for setup-only rounds (load_skill / enable_tool)
so a tight budget is not eaten by the load-then-use two-step.
- ui: re-fetch the active goal when a turn finishes so a goal created or mutated
mid-conversation surfaces without depending on an SSE event.
- streaming: make retry backoff / total-time budget instance fields with a
test-only seam; clarify that the wall-clock budget (not max-retries) bounds a
sustained SERVER_ERROR loop to ~8 attempts, fixing the slow/flaky retry test.
The knowledge-base trust verification recorded wiki citations from every
non-readFile tool response by sniffing its JSON for a top-level title /
pages / chunks field. Tools like getGoalStatus return a top-level title,
which falsely populated the citation set and then forced [n] citations on
the final answer (otherwise flagged EVIDENCE_INSUFFICIENT). Gate citation
mining on the wiki_* tool name instead.
Likewise, the grounded answer contract (cite-or-refuse) was appended to
every ReasoningNode call unconditionally, degrading general agents that
have no knowledge base. Append it only when the agent has a wiki_* tool
bound, scoping the strict regime to KB-grounded scenarios.
Adds a regression test asserting a non-wiki tool with a top-level title
creates no wiki citations.
Add ErrorClassificationTest regression cases for the AI-gateway retry
hardening: 5xx-before-4xx ordering (a proxy 502 whose body says
"bad request" stays retryable), Chinese / numeric provider billing
patterns, and DNS / TLS infrastructure-fatal detection.
Fix the cert-trust pattern while adding its test: Java's ValidatorException
emits "PKIX path building failed" with an uppercase PKIX and the error
chain is not lower-cased, so the previous lowercase pattern never matched
— an untrusted/expired cert chain fell through to the retryable
SERVER_ERROR bucket and was retried in vain instead of failing over.
- Fail closed to a global fallback sandbox root when a conversation has no
per-workspace base path, instead of leaving file/shell tools unconstrained
- Refuse shell commands that delete the workspace root directory itself
- Block workspace-boundary escapes at the policy layer before the approval
prompt, not only at execution time
- Approval bar now shows the actual command / target path being approved
Closes#289 — after an MCP server (re)connects, chat queries kept replying
"from memory" instead of calling MCP tools.
Root cause: agents snapshot their tool set at build time and are cached in
AgentService.agentInstances, but MCP server lifecycle changes never
invalidated that cache (unlike model-config / tool-guard changes which do).
A stale, tool-less agent graph survived until process restart.
Changes:
- Add McpServerChangedEvent; McpServerService publishes it on connect /
disconnect / reconnect / delete / (re)connect-failure / batch refresh /
startup init. AgentService listens and calls refreshAllAgents(), so the
next turn rebuilds against the live MCP tool set. Also closes the boot
race where the web server accepts requests before the @Order(200) MCP
init runner finishes.
- Make create/update/toggle connect asynchronously on a dedicated pool
("mcp-connect") so a slow/unreachable server can no longer freeze the
admin request; status returns immediately as "connecting".
- UI: render the new "connecting" status (pulsing amber dot), show a
friendly "connecting in background" toast, and poll until the status
settles (window widened to ~40s to outlast the default connect timeout).
- UI: MCP config modal no longer closes on outside/backdrop click — only
the × and Cancel buttons close it, so an accidental click can't discard
unsaved config.
Verified E2E: ckjia-shopping (参考价) MCP server connected at runtime with
no backend restart; the cached 通用助手 agent immediately enabled and called
ckjia_shopping_recommend, returning real product cards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an execute_code built-in tool that runs python/bash/node code the agent
writes on the fly, so a documentation-only skill (a SKILL.md with no bundled
scripts) can be acted on. Scoped runs inject the skill's secrets and run in
the skill directory; otherwise a private scratch directory is used. Host
secret env vars are scrubbed from the subprocess. execute_code is an
agent-wide capability, registered in the tool catalog (V143), and screened
by the tool guard with a dedicated set of destructive-pattern rules.
Tests cover python/bash/node execution, scratch-dir fallback, env scrubbing,
argument decoding, and guard gating.
Add 5-arg buildContextMessage overload that emits [system-context] Model:
for every origin (web/cron/IM). Legacy 3/4-arg overloads delegate to the
new one with null model args, keeping their output byte-identical.
Also fix pre-existing FeishuMentionTest compile error caused by removed
mentionMatchesAnyAlias/collectMentionIdentifiers methods.
Gate MarkdownNormalizer behind mate.agent.markdown-normalize-enabled (default
true) so operators can disable the rewrite verbatim if a normalization edge
case ever mangles a legitimate answer.
LLMs routinely emit malformed Markdown (missing heading spaces, glued `---`, unaligned table pipes) that prompt rules cannot reliably prevent. Add a zero-token, regex-only MarkdownNormalizer applied on the FinalAnswerNode convergence path before persistence / channel delivery. It is code-fence aware, idempotent, and conservative (em-dash `---`, `#5`-style refs, stray prose pipes are left untouched). RETURN_DIRECT verbatim output and approval-wait paths return earlier and are unaffected.
Closes#274
Streaming chat ran render tools on an async thread with no bound request,
so download links lost their host and arrived without a domain. Resolve the
host on the request thread and carry it through ChatOrigin/ToolContext;
falls back to a configurable public-base-url, then a relative path.
Agents now have a per-agent primary wiki KB stored on
mate_agent.primary_kb_id. KBs remain workspace-shared — selecting one in
the agent editor only chooses the default wiki target for that agent, it
does not change the KB's ownership or visibility.
Backend
- AgentEntity: add primary_kb_id field (FieldStrategy.ALWAYS so the UI
can clear it back to "no primary")
- AgentController#update: switch body to Map<String, Object> so we can
tell "field missing" apart from "explicit null" via containsKey, then
convertValue back to AgentEntity
- WikiKnowledgeBaseService:
- new resolvePrimaryKb(agentId): prefers agent.primary_kb_id when it
points to a workspace-visible KB; falls back to legacy
kb.agent_id marker, then to most-recently-updated workspace KB
- listByAgentId now returns the full workspace set (KBs are
workspace-shared under the new model)
- update(id, name, description) no longer touches agent_id
- WikiController: new GET /knowledge-bases/bindable for the UI picker;
PUT /knowledge-bases/{id} no longer reads agentId
- WikiKnowledgeBaseEntity: add FieldStrategy.ALWAYS on embeddingModelId
and configContent so explicit nulls actually unbind/clear instead of
being silently skipped by MyBatis-Plus's NOT_NULL default
- Migrations V129 (H2 + MySQL): add primary_kb_id column + index, backfill
from legacy kb.agent_id, MySQL uses INFORMATION_SCHEMA guard +
PREPARE/EXECUTE for idempotency
- WikiKnowledgeBaseServiceTest: 13 cases, all passing
Frontend
- Agents.vue: new "Knowledge Base" tab, radio-select bindable KBs
- API: listBindableKBs() + Agent.primaryKbId typed string | number | null
- IDs handled as strings throughout (Snowflake-safe)
- i18n keys for the new tab in zh-CN and en-US
Some OpenAI-compatible providers (LM Studio's built-in server, certain
strict-mode vLLM / SGLang deployments) reject 400 "System message must
be at the beginning" when SystemMessages appear after user / assistant
/ tool messages. The reasoning loop currently emits four SystemMessage
segments — main prompt at index 0, skill catalog inserted at index 1,
progress-ledger snapshot and stale-reminder appended at the end of
nonHistoryPrefix after the runtime-context UserMessage. The latter two
violate the strict shape, so conversations on LM Studio 400 on the
first turn (reported in #218).
Add MessageNormalizer: collects every SystemMessage in the outbound
prompt regardless of position, joins their text with a blank-line
separator, and emits a single SystemMessage at index 0. Non-system
messages keep their relative order, so AssistantMessage(tool_calls) ↔
ToolResponseMessage adjacency is preserved verbatim (required by strict
pair validators).
Wire it into doStreamCall as the first pre-egress step so every node
(reasoning, step-execution, summarizing, plan-generation, limit-exceeded)
inherits the fix without per-node changes, and any future node that
emits multiple SystemMessages stays compliant.
The transformation is semantically equivalent on permissive providers
(OpenAI, DashScope, Ollama, DeepSeek, Kimi, Doubao, GLM) — the merged
token sequence matches what they would have seen across N SystemMessages
— and safe on non-OpenAI protocols (Anthropic, Vertex / Gemini), whose
adapters already extract SystemMessages into a top-level system field
and receive an identical payload.
Kill switch: -Dmateclaw.llm.message-normalizer.enabled=false reverts to
the prior behavior for emergency rollback.
Tests: 11 unit tests on MessageNormalizer cover empty / no-system /
canonical / mid-list / tail / blanks / tool-pair preservation / Prompt
option-reference preservation / kill switch. 1 wiring test pins the
call site in doStreamCall. Full vip.mate.agent.** suite (504 tests)
stays green.
Closes#218.
1. Relative parent traversal in shell commands (HIGH)
validateShellCommand only scanned absolute path tokens, so commands
like `cat ../mateclaw/CLAUDE.md`, `cd .. && cat foo`, or
`ln -sf ../bar breakout` had no absolute path to trip the check.
From a workspace cwd that's a real escape — `..` segments resolve
against the JVM cwd at file-tool time and reach anywhere the user
can read.
Add a second pass: any token containing `..` as a path segment is
resolved against the workspace root via root.resolve(token).
normalize(); reject when the result falls outside. In-workspace
traversal like `subdir/../sibling` normalizes back inside and
passes. Identifiers without slashes (e.g. version strings with
`1.2..3`) are not treated as paths.
2. Shell validation and process working directory used different
context sources (MEDIUM)
execute_shell_command validated with the explicit ToolContext, but
buildShellProcess called WorkspacePathGuard.getWorkingDirectory()
(no-arg), which only sees the ThreadLocal fallback. Today the
ToolExecutionExecutor sets both so the discrepancy is latent, but
a future direct Spring AI invocation passing only ToolContext would
validate against one basePath and exec against another. Thread ctx
through buildShellProcess and call getWorkingDirectory(ctx) so
validation and execution agree on a single source of truth.
3. Absolute agent override could disable workspace scoping (MEDIUM)
resolveAgentBasePath accepted an absolute override verbatim, even
when it pointed outside the workspace root. An admin (or any
account with agent-edit permission) could set workspaceBasePath="/"
or another team's repo and bypass workspace boundaries entirely.
When a workspace has its own basePath, require absolute overrides
to sit underneath it. The caller in build() catches the rejection,
logs WARN, and falls back to the workspace basePath so chat stays
available rather than crashing agent construction. When the
workspace has no basePath there's no boundary to enforce, so legacy
behavior is preserved.
Test coverage: WorkspacePathGuardShellTest grows from 17 to 23 (six
new cases for `cd ..`, relative parent traversal, relative symlink
escape, deeper traversal, in-workspace normalization, and the
identifier false-positive guard). AgentGraphBuilderBasePathResolutionTest
grows from 7 to 10 (three new cases for in-workspace absolute,
outside-workspace absolute rejection, and no-workspace legacy
behavior). All 45 sandbox-area tests pass with no regressions.
* feat(agent): optional agent-level workspace basePath override
Add workspaceBasePath field to AgentEntity that optionally overrides
the workspace-level basePath. When set, the agent uses its own directory;
when null, it inherits the workspace's basePath (existing behavior).
- AgentEntity: new workspaceBasePath field with ALWAYS update strategy
- AgentGraphBuilder: agent-level override takes priority over workspace
- Flyway migration V121 for H2 and MySQL
- UI: form input in basic tab with i18n (zh-CN, en-US)
* fix(agent): rename migration V121→V125 to avoid Flyway conflict with upstream
Upstream already has V121__tool_disclosure_tier.sql. Rename our
migration to V125 (next available after V124).
* fix(agent): make MySQL V125 migration idempotent
Use INFORMATION_SCHEMA check before ADD COLUMN to avoid
"Duplicate column name" error on re-deploy.
* feat(tool): add send_file tool for sending existing server files as IM attachments
Adds a new built-in tool that reads a file from the server and stashes it
in GeneratedFileCache so the channel adapter (Feishu, DingTalk, etc.)
automatically sends it as a native attachment. This fills the gap where
agents had no way to send existing server files to users — ReadFileTool
only reads text, and render tools only generate new files.
- New SendFileTool with path validation, MIME detection, 20MB limit
- Added "send_file" to tool allowlist in AgentBindingService
- Added i18n error messages (zh-CN + en-US)
* fix(tool): send_file returns URL in scrubber-detectable format
The previous JSON return format caused the LLM to reply with just
"status: sent" without echoing the /api/v1/files/generated/{id} URL.
GeneratedFileScrubber only scans the LLM's final text output, so the
file was never delivered as a native attachment.
Changed to match GeneratedFileLink's format: returns a markdown link
with explicit instructions for the LLM to echo the URL verbatim.
Some providers (notably SiliconFlow) return "network connection error" in the response body when their backend is overloaded or the upstream model connection is disrupted. classifyError() had no pattern for this string, so it fell through to UNKNOWN (non-retryable), surfacing the raw error to the user on the first failure instead of running the exponential-backoff recovery. Adds the pattern to the SERVER_ERROR classifier and a friendly message mapping in extractUserFriendlyError(); bumps MAX_RETRIES from 5 to 10 so sustained wiki batch load can ride out provider flaps without surfacing an error to the channel user.
Closes#178
