Add an execute_code built-in tool that runs python/bash/node code the agent
writes on the fly, so a documentation-only skill (a SKILL.md with no bundled
scripts) can be acted on. Scoped runs inject the skill's secrets and run in
the skill directory; otherwise a private scratch directory is used. Host
secret env vars are scrubbed from the subprocess. execute_code is an
agent-wide capability, registered in the tool catalog (V143), and screened
by the tool guard with a dedicated set of destructive-pattern rules.
Tests cover python/bash/node execution, scratch-dir fallback, env scrubbing,
argument decoding, and guard gating.
- WikiPageTypeProfile: normalise pageType keys to lowercase on set, so a
user-authored profile with an uppercase key still matches the
case-insensitive hasPageType/get lookups.
- WikiDirectoryScanService: normalise the symlink-resolved glob base to
forward slashes so directory-scan globs work on Windows paths.
- Agents roster tag filter: keep selected tags that no longer exist on any
agent visible and deselectable (and show the filter bar when only such
orphan selections remain) instead of silently filtering with no way to clear.
A second POST to /reclassify on the same KB spawned an independent pass over
the same pages, doubling LLM spend and racing the first pass's page-type
writes. Add a per-KB in-flight guard that rejects a concurrent run with a
friendly message (409 via R.fail rather than a generic 500), released in a
finally once the async pass completes. Also count and broadcast per-page
failures so an all-failing run is visible instead of reporting changed=0, and
type the api modelId param as string|number per the snowflake ID convention.
The resilient connect() override started the child process into a local
variable and never set the parent's private process field, so the parent's
closeGracefully() logged "Process not started" and left the child running on
every reconnect/disable. Retain the started process and override
closeGracefully() to destroy it and interrupt the spawned I/O threads.
Also make the inboundSink/errorSink reflection fail loud (the transport is
useless without them — a silent null yielded a connected transport that timed
out on every call), force UTF-8 on the stdio reader/writer, honour a
cooperative closing flag in the reader loops, and drop duplicate PATH entries.
Add a backfill path so pages created before a KB's pageType profile changed
can be migrated into newly-added types. A per-page classify-only LLM call
(title + summary in, single page_type out) is normalised through the profile
and written back via a partial update that never touches page content.
Exposed as POST /knowledge-bases/{id}/reclassify (admin) and a "re-classify
existing pages" action in the Wiki advanced panel.
Wiki page classification was only profile-aware in the main ingest pipeline.
Transformation outputs hard-coded "synthesis", agent-created pages were left
untyped, and the frontend hard-coded the built-in ten types for ordering,
colouring and labels — so custom/synthesis types sank to the bottom, rendered
grey and showed raw keys.
Backend:
- Add nullable target_page_type column to mate_wiki_transformation (V142,
mysql + h2) plus the entity field and CRUD normalization (blank = use
profile fallbackType; membership validated at save time, not edit time).
- Route transformation single-run + KB-aggregate page saves through
WikiPageTypeProfileService.normalizePageType so output joins the KB
classification; agent wiki_create_page now lands on the profile fallbackType
instead of an untyped page.
Frontend:
- Load + parse the KB pageType profile into the wiki store (order, labels,
fallbackType) on KB select / refresh.
- New useWikiPageType composable: profile-driven label (3-tier fallback) and
colour (built-in fixed + deterministic hash palette for custom types).
- Sidebar grouping order, graph colouring, node panel, graph filter and the
page header badge now follow the profile; transformation editor gains a
target-type dropdown sourced from the profile when output target is a page.
Refs #292
Add 5-arg buildContextMessage overload that emits [system-context] Model:
for every origin (web/cron/IM). Legacy 3/4-arg overloads delegate to the
new one with null model args, keeping their output byte-identical.
Also fix pre-existing FeishuMentionTest compile error caused by removed
mentionMatchesAnyAlias/collectMentionIdentifiers methods.
A silent (no-op) cron run still makes a full LLM call, but the marker
message was persisted via the token-free saveMessage overload, so the
settings-page Token total (which aggregates MessageEntity token columns)
under-counted cron spend by exactly the silent runs.
Carry chatResult's prompt/completion tokens onto the marker message,
matching the non-silent branch. Closes#284.
The base directory is canonicalized via toRealPath before walking, so the
walked files carry the symlink-resolved prefix. The PathMatcher was built
from the literal pattern, so a symlinked base never matched and files were
silently dropped. Rebuild the glob against the resolved scan root, escaping
glob metacharacters in the base so a real directory name containing */?/{}/[]
is treated literally.
The upstream StdioClientTransport breaks out of its inbound read loop on
any JSON parse error, permanently killing the reader thread. Some MCP
servers write non-JSON debug output to stdout (e.g.
"=== Document parser messages ==="), which triggers this and causes all
subsequent valid JSON-RPC responses to be lost → 30s timeout → agent stuck.
Override connect() in CwdAwareStdioClientTransport to skip non-JSON lines
(log at DEBUG) instead of breaking, keeping the inbound thread alive.
Closes#226
The #254 fix changed resolveConventionPath to {name}-{hashCode} and folded
hyphens to underscores, which re-pathed every existing skill (browser-cdp ->
browser_cdp-<hash>) with no migration, orphaning already-created workspaces and
breaking SkillWorkspaceManagerApplyBundleTest. Drop the hash suffix and keep
hyphens: the bare Unicode-preserving sanitized name already prevents the
non-ASCII collision (distinct CJK names map to distinct dirs) and leaves ASCII
kebab-case paths identical to the legacy scheme. Add path regression tests.
Skills with non-ASCII (e.g. Chinese) names collapsed to underscores in the workspace path, so two such skills resolved to the same directory and overwrote each other. Preserve Unicode letters/digits when sanitizing the path so distinct names map to distinct directories. Also write SKILL.md with CREATE_NEW to avoid a TOCTOU race on concurrent uploads, and mount the skills workspace as a named Docker volume so packages survive container restarts.
Closes#254
updateTextContentFromScan / updateBinaryFileFromScan are only reached via
self-invocation from the ingest* methods, so the proxy-based @Transactional
never applied. Each runs a single atomic updateById; remove the misleading
annotation and document why.
Directory-scan ingestion deduped only by content hash, so when a file at a known path changed, the new hash missed the existing raw and a second row was inserted for the same source_path — both rows then generated wiki pages, accumulating duplicates. Make source path the primary dedup key: same path + same hash skips, same path + changed hash updates the existing raw in place (reset to pending, re-process), falling back to the content-hash check only for genuine copies at new paths. Reprocessing reuses the same rawId, so deleteExclusiveBySourceRawId cleans the old pages before regeneration — no duplicate rows and no duplicate pages. findBySourcePath gains LIMIT 1 to tolerate pre-existing duplicates; docs/fix-duplicate-raws.sql remediates existing data.
Closes#271
Gate MarkdownNormalizer behind mate.agent.markdown-normalize-enabled (default
true) so operators can disable the rewrite verbatim if a normalization edge
case ever mangles a legitimate answer.
LLMs routinely emit malformed Markdown (missing heading spaces, glued `---`, unaligned table pipes) that prompt rules cannot reliably prevent. Add a zero-token, regex-only MarkdownNormalizer applied on the FinalAnswerNode convergence path before persistence / channel delivery. It is code-fence aware, idempotent, and conservative (em-dash `---`, `#5`-style refs, stray prose pipes are left untouched). RETURN_DIRECT verbatim output and approval-wait paths return earlier and are unaffected.
Closes#274
Same-message attachment reads failed on Feishu: the content part carried the sandbox-external ~/.mateclaw/media/ path, so read_file/extract_document_text were rejected by WorkspacePathGuard and ChatUploadResolver's basename fallback missed (media names {messageId}_{key} vs chat-uploads {millis}_{fileName}). cacheRecentFile already copies the attachment into the per-conversation data/chat-uploads/ dir; return that absolute path and stamp it onto the current message's image/file/audio/media parts so the path surfaced to the LLM is resolver-reachable. Reuses the existing copy — no extra I/O, no new state. Rich-text post multi-image and other channels sharing the media dir are noted as follow-ups.
Closes#278
Allow knowledge base source paths to be configured as a newline-separated
list of absolute paths or glob patterns rather than a single directory.
- Each non-blank, non-# line is treated as one path or glob pattern
- Plain paths (no wildcards) retain the existing recursive-scan behaviour
filtered by SUPPORTED_EXTENSIONS
- Glob patterns (e.g. /data/ocr/**/*.txt) walk from the fixed-prefix base
and apply Java's PathMatcher against each candidate's absolute path
- Patterns whose filename segment explicitly specifies an extension
(*.txt, *.{xlsx,csv}) skip the SUPPORTED_EXTENSIONS secondary filter,
respecting the user's explicit choice (key for OCR pipelines that
produce .txt output and should ignore the original PDF scans)
- Candidates collected across multiple patterns are deduplicated by
resolved absolute path so overlapping patterns don't double-count
- Symlink-escape check uses each pattern's own validated scan root
- parseSourcePatterns / extractBasePath moved into WikiSourcePathValidator
to eliminate a static circular reference between the two services
- Frontend watcher panel: single-line <input> replaced with <textarea>
supporting multiline editing; i18n updated with example patterns
- No DB schema change; fully backward-compatible with existing single-path
configs stored in sourceDirectory
Closes #(pending issue)
finishRunAndPublish updated the run row with status and finished_at but dropped the LLM token usage, so mate_cron_job_run.token_usage stayed NULL and the scheduler history always showed 0. Write the prompt+completion token total (already computed for the assistant message row) to the run row, guarded so non-LLM paths (reminder direct-push with a null ChatResult) leave the column untouched.
Closes#262
The skill meta-tools (listAvailableSkills, readSkillFile, listSkillFiles, load_skill, runSkillScript) queried the full skill catalog without checking the calling agent's bindings, so an agent scoped to a subset of skills could still read, load, or execute any skill via direct tool calls. Resolve the agent's bound skill ids from the tool context and filter/deny accordingly: a null binding set means no restriction (backward compatible), a non-null set (including empty) restricts access to that set — matching the system-prompt catalog filtering.
Closes#264
Surface directory-scan failures to the user via toast and render ScanResult.errors[]; expose MATE_WIKI_ALLOWED_SOURCE_ROOTS as a Docker env entry with blank-entry filtering in the path validator; set C.UTF-8 locale in the runtime image so non-ASCII file names decode correctly during scans.
Fixes#259
Add a single global-proxy switch that routes the backend's outbound traffic
through a configured HTTP/HTTPS/SOCKS proxy, for deployments that cannot reach
overseas APIs directly or must use a unified egress.
- ProxyManager installs the proxy via a default ProxySelector (honored by
java.net.http and HttpURLConnection), the proxy system properties, and a
--proxy-server arg for the browser tool; restores direct egress when
disabled. One switch covers LLM, web search, media generation, channels,
MCP and the browser.
- SOCKS applies to the HttpURLConnection-based egress only; the java.net.http
LLM/streaming path uses an HTTP proxy, and the UI states this.
- New Settings -> Network Proxy page: enable toggle, address, bypass list,
test-connection, and a coverage summary. Config stored as key/value in
mate_system_setting (no migration).
refs #109
Streaming chat ran render tools on an async thread with no bound request,
so download links lost their host and arrived without a domain. Resolve the
host on the request thread and carry it through ChatOrigin/ToolContext;
falls back to a configurable public-base-url, then a relative path.
