The knowledge-base trust verification recorded wiki citations from every
non-readFile tool response by sniffing its JSON for a top-level title /
pages / chunks field. Tools like getGoalStatus return a top-level title,
which falsely populated the citation set and then forced [n] citations on
the final answer (otherwise flagged EVIDENCE_INSUFFICIENT). Gate citation
mining on the wiki_* tool name instead.
Likewise, the grounded answer contract (cite-or-refuse) was appended to
every ReasoningNode call unconditionally, degrading general agents that
have no knowledge base. Append it only when the agent has a wiki_* tool
bound, scoping the strict regime to KB-grounded scenarios.
Adds a regression test asserting a non-wiki tool with a top-level title
creates no wiki citations.
Add ErrorClassificationTest regression cases for the AI-gateway retry
hardening: 5xx-before-4xx ordering (a proxy 502 whose body says
"bad request" stays retryable), Chinese / numeric provider billing
patterns, and DNS / TLS infrastructure-fatal detection.
Fix the cert-trust pattern while adding its test: Java's ValidatorException
emits "PKIX path building failed" with an uppercase PKIX and the error
chain is not lower-cased, so the previous lowercase pattern never matched
— an untrusted/expired cert chain fell through to the retryable
SERVER_ERROR bucket and was retried in vain instead of failing over.
- Fail closed to a global fallback sandbox root when a conversation has no
per-workspace base path, instead of leaving file/shell tools unconstrained
- Refuse shell commands that delete the workspace root directory itself
- Block workspace-boundary escapes at the policy layer before the approval
prompt, not only at execution time
- Approval bar now shows the actual command / target path being approved
Closes#289 — after an MCP server (re)connects, chat queries kept replying
"from memory" instead of calling MCP tools.
Root cause: agents snapshot their tool set at build time and are cached in
AgentService.agentInstances, but MCP server lifecycle changes never
invalidated that cache (unlike model-config / tool-guard changes which do).
A stale, tool-less agent graph survived until process restart.
Changes:
- Add McpServerChangedEvent; McpServerService publishes it on connect /
disconnect / reconnect / delete / (re)connect-failure / batch refresh /
startup init. AgentService listens and calls refreshAllAgents(), so the
next turn rebuilds against the live MCP tool set. Also closes the boot
race where the web server accepts requests before the @Order(200) MCP
init runner finishes.
- Make create/update/toggle connect asynchronously on a dedicated pool
("mcp-connect") so a slow/unreachable server can no longer freeze the
admin request; status returns immediately as "connecting".
- UI: render the new "connecting" status (pulsing amber dot), show a
friendly "connecting in background" toast, and poll until the status
settles (window widened to ~40s to outlast the default connect timeout).
- UI: MCP config modal no longer closes on outside/backdrop click — only
the × and Cancel buttons close it, so an accidental click can't discard
unsaved config.
Verified E2E: ckjia-shopping (参考价) MCP server connected at runtime with
no backend restart; the cached 通用助手 agent immediately enabled and called
ckjia_shopping_recommend, returning real product cards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an execute_code built-in tool that runs python/bash/node code the agent
writes on the fly, so a documentation-only skill (a SKILL.md with no bundled
scripts) can be acted on. Scoped runs inject the skill's secrets and run in
the skill directory; otherwise a private scratch directory is used. Host
secret env vars are scrubbed from the subprocess. execute_code is an
agent-wide capability, registered in the tool catalog (V143), and screened
by the tool guard with a dedicated set of destructive-pattern rules.
Tests cover python/bash/node execution, scratch-dir fallback, env scrubbing,
argument decoding, and guard gating.
Add 5-arg buildContextMessage overload that emits [system-context] Model:
for every origin (web/cron/IM). Legacy 3/4-arg overloads delegate to the
new one with null model args, keeping their output byte-identical.
Also fix pre-existing FeishuMentionTest compile error caused by removed
mentionMatchesAnyAlias/collectMentionIdentifiers methods.
Gate MarkdownNormalizer behind mate.agent.markdown-normalize-enabled (default
true) so operators can disable the rewrite verbatim if a normalization edge
case ever mangles a legitimate answer.
LLMs routinely emit malformed Markdown (missing heading spaces, glued `---`, unaligned table pipes) that prompt rules cannot reliably prevent. Add a zero-token, regex-only MarkdownNormalizer applied on the FinalAnswerNode convergence path before persistence / channel delivery. It is code-fence aware, idempotent, and conservative (em-dash `---`, `#5`-style refs, stray prose pipes are left untouched). RETURN_DIRECT verbatim output and approval-wait paths return earlier and are unaffected.
Closes#274
Streaming chat ran render tools on an async thread with no bound request,
so download links lost their host and arrived without a domain. Resolve the
host on the request thread and carry it through ChatOrigin/ToolContext;
falls back to a configurable public-base-url, then a relative path.
Agents now have a per-agent primary wiki KB stored on
mate_agent.primary_kb_id. KBs remain workspace-shared — selecting one in
the agent editor only chooses the default wiki target for that agent, it
does not change the KB's ownership or visibility.
Backend
- AgentEntity: add primary_kb_id field (FieldStrategy.ALWAYS so the UI
can clear it back to "no primary")
- AgentController#update: switch body to Map<String, Object> so we can
tell "field missing" apart from "explicit null" via containsKey, then
convertValue back to AgentEntity
- WikiKnowledgeBaseService:
- new resolvePrimaryKb(agentId): prefers agent.primary_kb_id when it
points to a workspace-visible KB; falls back to legacy
kb.agent_id marker, then to most-recently-updated workspace KB
- listByAgentId now returns the full workspace set (KBs are
workspace-shared under the new model)
- update(id, name, description) no longer touches agent_id
- WikiController: new GET /knowledge-bases/bindable for the UI picker;
PUT /knowledge-bases/{id} no longer reads agentId
- WikiKnowledgeBaseEntity: add FieldStrategy.ALWAYS on embeddingModelId
and configContent so explicit nulls actually unbind/clear instead of
being silently skipped by MyBatis-Plus's NOT_NULL default
- Migrations V129 (H2 + MySQL): add primary_kb_id column + index, backfill
from legacy kb.agent_id, MySQL uses INFORMATION_SCHEMA guard +
PREPARE/EXECUTE for idempotency
- WikiKnowledgeBaseServiceTest: 13 cases, all passing
Frontend
- Agents.vue: new "Knowledge Base" tab, radio-select bindable KBs
- API: listBindableKBs() + Agent.primaryKbId typed string | number | null
- IDs handled as strings throughout (Snowflake-safe)
- i18n keys for the new tab in zh-CN and en-US
Some OpenAI-compatible providers (LM Studio's built-in server, certain
strict-mode vLLM / SGLang deployments) reject 400 "System message must
be at the beginning" when SystemMessages appear after user / assistant
/ tool messages. The reasoning loop currently emits four SystemMessage
segments — main prompt at index 0, skill catalog inserted at index 1,
progress-ledger snapshot and stale-reminder appended at the end of
nonHistoryPrefix after the runtime-context UserMessage. The latter two
violate the strict shape, so conversations on LM Studio 400 on the
first turn (reported in #218).
Add MessageNormalizer: collects every SystemMessage in the outbound
prompt regardless of position, joins their text with a blank-line
separator, and emits a single SystemMessage at index 0. Non-system
messages keep their relative order, so AssistantMessage(tool_calls) ↔
ToolResponseMessage adjacency is preserved verbatim (required by strict
pair validators).
Wire it into doStreamCall as the first pre-egress step so every node
(reasoning, step-execution, summarizing, plan-generation, limit-exceeded)
inherits the fix without per-node changes, and any future node that
emits multiple SystemMessages stays compliant.
The transformation is semantically equivalent on permissive providers
(OpenAI, DashScope, Ollama, DeepSeek, Kimi, Doubao, GLM) — the merged
token sequence matches what they would have seen across N SystemMessages
— and safe on non-OpenAI protocols (Anthropic, Vertex / Gemini), whose
adapters already extract SystemMessages into a top-level system field
and receive an identical payload.
Kill switch: -Dmateclaw.llm.message-normalizer.enabled=false reverts to
the prior behavior for emergency rollback.
Tests: 11 unit tests on MessageNormalizer cover empty / no-system /
canonical / mid-list / tail / blanks / tool-pair preservation / Prompt
option-reference preservation / kill switch. 1 wiring test pins the
call site in doStreamCall. Full vip.mate.agent.** suite (504 tests)
stays green.
Closes#218.
1. Relative parent traversal in shell commands (HIGH)
validateShellCommand only scanned absolute path tokens, so commands
like `cat ../mateclaw/CLAUDE.md`, `cd .. && cat foo`, or
`ln -sf ../bar breakout` had no absolute path to trip the check.
From a workspace cwd that's a real escape — `..` segments resolve
against the JVM cwd at file-tool time and reach anywhere the user
can read.
Add a second pass: any token containing `..` as a path segment is
resolved against the workspace root via root.resolve(token).
normalize(); reject when the result falls outside. In-workspace
traversal like `subdir/../sibling` normalizes back inside and
passes. Identifiers without slashes (e.g. version strings with
`1.2..3`) are not treated as paths.
2. Shell validation and process working directory used different
context sources (MEDIUM)
execute_shell_command validated with the explicit ToolContext, but
buildShellProcess called WorkspacePathGuard.getWorkingDirectory()
(no-arg), which only sees the ThreadLocal fallback. Today the
ToolExecutionExecutor sets both so the discrepancy is latent, but
a future direct Spring AI invocation passing only ToolContext would
validate against one basePath and exec against another. Thread ctx
through buildShellProcess and call getWorkingDirectory(ctx) so
validation and execution agree on a single source of truth.
3. Absolute agent override could disable workspace scoping (MEDIUM)
resolveAgentBasePath accepted an absolute override verbatim, even
when it pointed outside the workspace root. An admin (or any
account with agent-edit permission) could set workspaceBasePath="/"
or another team's repo and bypass workspace boundaries entirely.
When a workspace has its own basePath, require absolute overrides
to sit underneath it. The caller in build() catches the rejection,
logs WARN, and falls back to the workspace basePath so chat stays
available rather than crashing agent construction. When the
workspace has no basePath there's no boundary to enforce, so legacy
behavior is preserved.
Test coverage: WorkspacePathGuardShellTest grows from 17 to 23 (six
new cases for `cd ..`, relative parent traversal, relative symlink
escape, deeper traversal, in-workspace normalization, and the
identifier false-positive guard). AgentGraphBuilderBasePathResolutionTest
grows from 7 to 10 (three new cases for in-workspace absolute,
outside-workspace absolute rejection, and no-workspace legacy
behavior). All 45 sandbox-area tests pass with no regressions.
* feat(agent): optional agent-level workspace basePath override
Add workspaceBasePath field to AgentEntity that optionally overrides
the workspace-level basePath. When set, the agent uses its own directory;
when null, it inherits the workspace's basePath (existing behavior).
- AgentEntity: new workspaceBasePath field with ALWAYS update strategy
- AgentGraphBuilder: agent-level override takes priority over workspace
- Flyway migration V121 for H2 and MySQL
- UI: form input in basic tab with i18n (zh-CN, en-US)
* fix(agent): rename migration V121→V125 to avoid Flyway conflict with upstream
Upstream already has V121__tool_disclosure_tier.sql. Rename our
migration to V125 (next available after V124).
* fix(agent): make MySQL V125 migration idempotent
Use INFORMATION_SCHEMA check before ADD COLUMN to avoid
"Duplicate column name" error on re-deploy.
* feat(tool): add send_file tool for sending existing server files as IM attachments
Adds a new built-in tool that reads a file from the server and stashes it
in GeneratedFileCache so the channel adapter (Feishu, DingTalk, etc.)
automatically sends it as a native attachment. This fills the gap where
agents had no way to send existing server files to users — ReadFileTool
only reads text, and render tools only generate new files.
- New SendFileTool with path validation, MIME detection, 20MB limit
- Added "send_file" to tool allowlist in AgentBindingService
- Added i18n error messages (zh-CN + en-US)
* fix(tool): send_file returns URL in scrubber-detectable format
The previous JSON return format caused the LLM to reply with just
"status: sent" without echoing the /api/v1/files/generated/{id} URL.
GeneratedFileScrubber only scans the LLM's final text output, so the
file was never delivered as a native attachment.
Changed to match GeneratedFileLink's format: returns a markdown link
with explicit instructions for the LLM to echo the URL verbatim.
Some providers (notably SiliconFlow) return "network connection error" in the response body when their backend is overloaded or the upstream model connection is disrupted. classifyError() had no pattern for this string, so it fell through to UNKNOWN (non-retryable), surfacing the raw error to the user on the first failure instead of running the exponential-backoff recovery. Adds the pattern to the SERVER_ERROR classifier and a friendly message mapping in extractUserFriendlyError(); bumps MAX_RETRIES from 5 to 10 so sustained wiki batch load can ride out provider flaps without surfacing an error to the channel user.
Closes#178
When an agent had any skill bound, the runtime tool gate was silently
hiding @Tool beans that aren't declared in any skill manifest, even
though the global system prompts (SOUL.md / "Web Search Capability" /
"File Reading Guidelines") explicitly tell the LLM these tools are
available. Result: the model would call search / renderDocx / read_file
/ etc., hit "Tool not found", then either give up or fall back to
unhelpful behaviour (e.g. dumping markdown text instead of producing a
.docx download).
This commit:
- Adds universally-promised, agent-wide tools to SYSTEM_LEVEL_TOOLS so
they bypass the manifest restriction: document/media generation
(renderDocx*, image_generate, music_generate, video_generate),
global capability tools the system prompt mentions (search,
browser_use, read_file / write_file / edit_file /
execute_shell_command, detect_file_type, extract_*_text,
readMateClawDoc), skill discovery siblings (listSkillFiles,
listAvailableSkills), and the delegate triplet (delegateToAgent,
delegateParallel, listAvailableAgents).
- Fixes 5 entries in the prior whitelist whose names did not match
any real @Tool bean and were therefore silently dead:
read_workspace_file -> read_workspace_memory_file
write_workspace_file -> write_workspace_memory_file
list_workspace_files -> list_workspace_memory_files
delegate_agent -> delegateToAgent
datetime -> getCurrentDate / getCurrentDateTime / getCurrentTime
Also adds the missing edit_workspace_memory_file.
- In the chat markdown renderer, strips any hallucinated
https?://<host> prefix from /api/v1/files/generated/<id> download
links before building the <a href>. Multiple LLMs have been
observed prepending bogus hosts when echoing tool-returned download
URLs back to the user, breaking the click. One-line defensive
normalization independent of which model is in use.
Verified end-to-end on a previously-broken agent: search / browser_use
/ execute_shell_command / renderDocx all dispatch correctly now and
the final markdown link is a clean same-origin path. 36 whitelist
entries cross-checked against real @Tool method names.
AgentBindingServiceTest green.
When a user-installed skill (e.g. RedisOps) was bound to an agent, the
model frequently called the skill name directly as a tool, hit
"Tool not found: RedisOps", and either gave up or fell back to shell
guessing. Two compounding causes:
1. The system prompt block injected by SkillRuntimeService listed each
skill as `- **RedisOps** — desc`, which is the same format used for
tool catalogs and primed the model to call the names directly. The
"how to use" instructions referenced `read_skill_file` /
`run_skill_script` — names that don't exist in the tool registry,
so even a compliant LLM couldn't follow them.
2. ToolExecutionExecutor's `callback == null` branches returned a bare
"Tool not found: <name>" string. The model had no recovery signal
and no hint that the name it called was actually a skill.
Fix is two-layered:
- Prompt rewrite (SkillRuntimeService.buildSkillPromptEnhancement): lead
with an explicit warning that skills are NOT directly callable, use the
correct camelCase tool names (readSkillFile / runSkillScript), include
a concrete worked example anchored to the first enabled skill, and
render the listing as a markdown table so it stops looking like a
callable tool list. listAvailableSkills tool description and output
follow the same pattern.
- Runtime safety net (ToolExecutionExecutor): when toolCallbackMap.get
misses, check if the requested name (case-insensitive) matches an
active skill. If so, return a precise hint telling the LLM the right
invocation pattern instead of the bare error. Wired through both the
main execute path and the pre-approved replay path. SkillRuntimeService
is attached via a setter from AgentGraphBuilder so the executor's many
legacy constructors stay untouched, and it's nullable so isolated
tests still work.
Adds 5 unit tests covering: skill match -> hint, case-insensitive match,
no-match -> bare error, no SkillRuntimeService wired -> bare error,
pre-approved replay path -> hint.
Reported and reproduced by @pipima9950-glitch in issue #46.
Three layers landed together because they share the same routing /
lifecycle plumbing:
1. Cron output unification
- New CronConversationResolver routes web-origin jobs to the per-workspace
tasks_<wsId> conversation; IM-bound jobs go to the channel session
conversation when one exists (matched by senderId then targetId);
legacy cron_<id> remains as the fallback.
- CronJobLifecycleService inserts a system-role header divider when a
run starts so users browsing the unified tasks_<wsId> view can tell
which job started a run. BaseAgent.sanitizeForLlm filters these
headers so they never reach the model.
- WorkspaceService seeds tasks_<wsId> on workspace creation; V65
migration backfills existing workspaces.
- DeliveryConfig gains a userId field so IM session lookup can match
by senderId (replyToken-based targetId is not stable across runs).
- ConversationVO recognizes tasks_/cron_ underscore prefix as cron
source. MessageList renders the system header as a labeled divider.
- ChatConsole pins tasks_* conversations and tracks per-conversation
read state so new cron output gets a visible unread dot.
2. Reminder task type
- New task_type='reminder' in CronJobEntity + service validation.
- CronJobRunner short-circuits 'reminder' jobs: hands trigger_message
to finishRunAndPublish verbatim, no LLM call. Fixes a regression
where reminders were rephrased into echoed wrappers.
- New create_reminder tool alongside create_cron_job, with descriptions
tightened so the model picks the right one (verbatim push vs LLM
query that needs computation).
- CronJobs.vue gets a third radio option + dedicated reminder field.
3. In-flight progress placeholder
- Cron uses non-streaming chat()/execute(); tool-heavy ReAct loops
can run 1-5 minutes between start and finish with no visible
state, looking hung.
- New GET /api/v1/cron-jobs/active-runs returns runs in status=running
for a conversation. ChatConsole polls it on the existing 4s tick
(and on conversation switch) and shows a spinner bar with elapsed
time. When run count drops to zero, it refetches messages so the
assistant bubble appears within ~1s of finish.
Add a callout above the existing intro to make the wedge explicit:
multi-user workspaces, approval-gated sensitive actions, full audit trail,
production-grade health monitoring, per-channel error isolation.
One JAR on your own machine, zero data egress.
