Turn a single natural-language requirement into a ready-to-review
employee: the model proposes name, persona, runtime type and a
validated set of skills/tools/knowledge base, which the user confirms
or tweaks before the agent is created.
- backend: POST /api/v1/agents/generate builds a draft from the
workspace's real capability catalog; every suggested tool/skill/KB is
re-validated against the catalog so nothing hallucinated is offered
- frontend: 3-step wizard at /agents/create reusing the existing
create + binding endpoints; reusable capability picker shows selected
items as compact chips with an on-demand searchable catalog
Add an opt-in named-entity extraction pass so the wiki knowledge graph
captures fine-grained entities (people, organizations, locations, ...)
and their relations, not just page-level link relations.
- new tables mate_wiki_entity / _mention / _relation (h2/mysql/kingbase)
- structured LLM extraction per chunk with entity resolution
(normalized-key dedup + embedding near-merge), mention/relation
persistence and page linking via chunk citations
- per-KB opt-in toggle (off by default); async dispatch after embedding
- read API: entity list, KB graph, entity ego-graph, manual extract
- UI: entity-layer toggle in the graph view + KB config toggle
- replace inline fully-qualified class names with imports in WikiProcessingService
Closes#336
Always-on memory (structured user/feedback blocks, PROFILE.md, MEMORY.md) is injected into every system prompt but only ever grew, inflating per-turn context over time. This adds deterministic size control across all always-on sources:
- Injection budget: cap the always-on structured block by total chars and per-type entry count, keeping the most-recently-updated entries (LRU by Updated date) and disclosing how many were omitted
- Nightly consolidation: a dedicated scheduled pass merges duplicate/stale user & feedback entries via the LLM, preserving each entry's original Updated date; runs per owner bucket (shared + personal) with a per-run cap and a never-grow safety guard
- File ceilings: deterministic backstop truncates PROFILE.md / MEMORY.md at a section boundary when a rewrite overruns its budget
- Manual trigger endpoint for the consolidation maintenance task
All knobs under mate.memory.*; covered by unit tests.
- goal: continue (not skip) on max-iterations and evidence-insufficient turns.
A max-iterations turn grants a fresh iteration budget ("hard continuation"),
bounded per run and sized into the graph recursion ceiling, so a task too big
for one budget keeps going instead of stalling until the next user message.
- plan-execute: re-plan the remaining work on a step exception, and on a
signature-based stall (repeated failures / identical results / no usable
result) instead of advancing dependent steps with junk; bounded by a per-run
re-plan cap, with a graduated change-strategy nudge before the hard stop.
- plan-execute: auto-derive a goal from a genuine multi-step plan, seeding the
acceptance criteria from the plan steps, so the goal subsystem engages without
the model calling setGoal; broadcast goal_created so the UI hydrates.
- react: refund the iteration for setup-only rounds (load_skill / enable_tool)
so a tight budget is not eaten by the load-then-use two-step.
- ui: re-fetch the active goal when a turn finishes so a goal created or mutated
mid-conversation surfaces without depending on an SSE event.
- streaming: make retry backoff / total-time budget instance fields with a
test-only seam; clarify that the wall-clock budget (not max-retries) bounds a
sustained SERVER_ERROR loop to ~8 attempts, fixing the slow/flaky retry test.
The knowledge-base trust verification recorded wiki citations from every
non-readFile tool response by sniffing its JSON for a top-level title /
pages / chunks field. Tools like getGoalStatus return a top-level title,
which falsely populated the citation set and then forced [n] citations on
the final answer (otherwise flagged EVIDENCE_INSUFFICIENT). Gate citation
mining on the wiki_* tool name instead.
Likewise, the grounded answer contract (cite-or-refuse) was appended to
every ReasoningNode call unconditionally, degrading general agents that
have no knowledge base. Append it only when the agent has a wiki_* tool
bound, scoping the strict regime to KB-grounded scenarios.
Adds a regression test asserting a non-wiki tool with a top-level title
creates no wiki citations.
- rescan: keep the KB id as a string end to end so the 19-digit snowflake id
isn't truncated past Number.MAX_SAFE_INTEGER (rescan no longer 404s)
- lint: resolve [[...]] targets against page slugs AND titles like the viewer,
so a title reference to an existing page is no longer reported broken
- ingest: derive the slug deterministically from the title (no inconsistent
romanization), auto-recompute broken links once a KB finishes importing,
and reconcile dangling [[concept]] links — redirect to the covering page via
declared aliases, or demote to plain text when uncovered
- add the page aliases column migration for h2 / mysql / kingbase
Surface the connected database product as a subtle chip in the Dashboard
header. SystemHealthService now reports a database label on /system/health
(reused by the front-end — no extra request), derived from a new
DatabaseBootstrapRunner.getDatabaseLabel() that reads the JDBC product name
once and normalizes it to a canonical label (MySQL / MariaDB / PostgreSQL /
H2, and 人大金仓 for the KingbaseES family), collapsing driver version noise.
Add ErrorClassificationTest regression cases for the AI-gateway retry
hardening: 5xx-before-4xx ordering (a proxy 502 whose body says
"bad request" stays retryable), Chinese / numeric provider billing
patterns, and DNS / TLS infrastructure-fatal detection.
Fix the cert-trust pattern while adding its test: Java's ValidatorException
emits "PKIX path building failed" with an uppercase PKIX and the error
chain is not lower-cased, so the previous lowercase pattern never matched
— an untrusted/expired cert chain fell through to the retryable
SERVER_ERROR bucket and was retried in vain instead of failing over.
The disk fallback's catch block logged at debug, so a failed scan
silently dropped recovered files — reproducing the same 'bot can't see
the file' symptom the fallback was added to fix. Promote to warn with
the full stack trace, matching cacheRecentFile's logging.
openAiCompatibleClientBuilder() was creating a new java.net.http.HttpClient
per request. Each instance spawns a selector thread and connection pool that
are never closed, exhausting the OS thread limit under frequent model-test
calls (e.g. DeepSeek provider).
Elevate the HttpClient to a static singleton so all OpenAI-compatible
provider requests share one connection pool and one selector thread.
Closesmatevip/mateclaw#328
loadRecentFilesFromDisk now filters out files older than RECENT_FILE_TTL_MINUTES
(60 min) so the disk fallback matches the Caffeine cache TTL and does not inject
stale attachments into future conversations.
Testability refactoring:
- recentFileCache: private → package-private (tests can seed the cache directly)
- chatUploadsRoot: new package-private Path field (tests redirect to @TempDir)
- loadRecentFilesFromDisk: add (Path dir, long cutoffMs) package-private overload;
private (String) wrapper delegates to it
- injectRecentFiles: private → package-private
New test class FeishuRecentFileCacheTest (14 cases):
- loadRecentFilesFromDisk: non-existent dir, empty dir, fresh files sorted
newest-first, stale files excluded by TTL, mixed fresh+stale, >5 files capped,
timestamp-prefix stripping, MIME guessing from extension
- injectRecentFiles: Caffeine cache hit, cache-miss disk fallback, empty disk,
duplicate-path dedup, image vs file part typing, null textContent guard
Relates to #325
The per-chat recent file cache (Caffeine, 60 min TTL) is purely
in-memory. After a process restart, GC eviction, or TTL expiry the
cache is empty, but the staged copies under data/chat-uploads/ survive
on disk. A follow-up text message that should have seen the cached file
instead found nothing — the bot replied as if no file was ever sent.
Changes:
- injectRecentFiles(): fall back to scanning data/chat-uploads/{id}/
when the Caffeine cache misses, sorted by last-modified time, capped
at RECENT_FILE_MAX_PER_CHAT (5).
- cacheRecentFile(): promote catch log from debug → warn with full
stack trace so silent download failures are visible in production
logs. Add entry-level info log for correlation.
- New helper loadRecentFilesFromDisk() + guessContentType().
Closes#325
Relates to #201
The KingbaseES change removed usingDbTime() unconditionally, which made
every deployment (MySQL/H2/PostgreSQL) fall back to app-server time for
distributed lock timing — reintroducing node clock-drift risk in
multi-instance setups. Re-enable usingDbTime() for databases in ShedLock's
built-in dialect map and skip it only for KingbaseES, which is not covered
and would otherwise throw at lock acquisition.
Add KingbaseES support as an opt-in profile: dedicated migration tree, bilingual seed data, runtime DbType detection (KINGBASE_ES / POSTGRE_SQL), and JDBC URL handling in the datasource manager.
* feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync
The raw-material directory scan and the Advanced "source watcher" sub-tab were
the same engine (same kb.sourceDirectory, same WikiDirectoryScanService) split
across two surfaces with two editable directory inputs. Merge them into one
"Sources" tab (upload / paste / directory manual scan + auto-sync toggle +
the raw-material list) and drop the watcher sub-tab from Advanced.
Auto-sync is now per-KB opt-in: a new watcher_enabled column (V146) gates the
periodic scan per knowledge base. The server-global mate.wiki.watcher-enabled
stays as an ops master switch — a KB is auto-scanned only when both are on
(AND). Manual scans are unaffected. Scan interval stays global for now
(tracked separately).
Closesmatevip/mateclaw#314
* docs(wiki): document source-watcher global switch env vars
Expose MATE_WIKI_WATCHER_ENABLED / MATE_WIKI_WATCHER_INTERVAL_MS as
explicit placeholders in application-mysql.yml, .env.example and
docker-compose.yml, mirroring MATE_WIKI_ALLOWED_SOURCE_ROOTS. Notes the
AND semantics (global ops gate + per-KB toggle) so operators know the
global switch alone is not sufficient.
- Fail closed to a global fallback sandbox root when a conversation has no
per-workspace base path, instead of leaving file/shell tools unconstrained
- Refuse shell commands that delete the workspace root directory itself
- Block workspace-boundary escapes at the policy layer before the approval
prompt, not only at execution time
- Approval bar now shows the actual command / target path being approved
Closes#289 — after an MCP server (re)connects, chat queries kept replying
"from memory" instead of calling MCP tools.
Root cause: agents snapshot their tool set at build time and are cached in
AgentService.agentInstances, but MCP server lifecycle changes never
invalidated that cache (unlike model-config / tool-guard changes which do).
A stale, tool-less agent graph survived until process restart.
Changes:
- Add McpServerChangedEvent; McpServerService publishes it on connect /
disconnect / reconnect / delete / (re)connect-failure / batch refresh /
startup init. AgentService listens and calls refreshAllAgents(), so the
next turn rebuilds against the live MCP tool set. Also closes the boot
race where the web server accepts requests before the @Order(200) MCP
init runner finishes.
- Make create/update/toggle connect asynchronously on a dedicated pool
("mcp-connect") so a slow/unreachable server can no longer freeze the
admin request; status returns immediately as "connecting".
- UI: render the new "connecting" status (pulsing amber dot), show a
friendly "connecting in background" toast, and poll until the status
settles (window widened to ~40s to outlast the default connect timeout).
- UI: MCP config modal no longer closes on outside/backdrop click — only
the × and Cancel buttons close it, so an accidental click can't discard
unsaved config.
Verified E2E: ckjia-shopping (参考价) MCP server connected at runtime with
no backend restart; the cached 通用助手 agent immediately enabled and called
ckjia_shopping_recommend, returning real product cards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The webchat conversationId (webchat:<key8>:<visitorId>[:<sessionId>]) and the
derived username (webchat:<visitorId>) are written to VARCHAR(64) columns, but
visitorId had no validation and sessionId allows 64 chars — so a long visitorId,
or a legitimate 64-char sessionId, overflows the column and the getOrCreateConversation
INSERT throws (500 on /stream). Validate visitorId (charset + blank->UUID) and
fold the variable part into a stable hash when the derived id/username would
exceed 64 chars, keeping short ids byte-identical (backward compatible). Also
make listSessions filter on exact owner username, not just the conversationId
prefix, so system-owned rows can never leak via a crafted visitorId. Adds
boundary regression tests.
Add per-visitor session management for the WebChat Web/API access mode:
list a visitor's conversation threads, fetch a thread's messages, and
delete a thread.
Authorization: visitorId is a client-asserted request param, so it cannot
be trusted on its own — deriving conversationId from it and then checking
ownership against it is tautological (any caller passes). Instead, /stream
issues a per-visitor token = HMAC-SHA256(jwtSecret, channelId:visitorId),
returned in the meta event; the management endpoints require it back via
the X-MC-Visitor-Token header and verify it in constant time. The signing
secret is server-only (unlike the public channel API key) and the channelId
in the payload makes tokens non-portable across channels.
Includes regression tests for token issuance/verification semantics
(forged visitorId rejected, cross-visitor and cross-channel tokens rejected,
tampered tokens rejected).
The WebChat stream endpoint resolves agentId only when the
(visitorId + sessionId) conversation is first created; later requests
with a different agentId reuse the existing conversation's agent and
silently ignore the new value. Document this on WebChatRequest.agentId
so integrators know to use a new sessionId to reach another agent.
The vision/video sidecar dropdown filtered candidates through the built-in
capability heuristics, so provider-compatible models whose custom names match
no known prefix (and carry no declared modalities) were hidden and could not be
selected as a sidecar — even when they natively support the modality.
- listByType now returns every enabled chat model, annotating each row with a
transient modalityCapable flag and sorting known-capable rows first, instead
of hard-filtering recognized models only.
- MultimodalRouter honours an explicitly configured sidecar model instead of
dropping it when the heuristics don't recognize it; a wrong pick degrades
gracefully through the caption path rather than silently disabling routing.
- ModelPicker gains an optional capability badge; the sidecar UI tags
recognized vision models while keeping every enabled model selectable.
The mention alias-learning fed every identifier of every mention in a
delivery into the per-chat alias cache. A single delivery of "@bot @alice"
matched the bot by its global id and then learned alice's openId as a bot
alias, so every later "@alice" message was misdetected as @bot and the agent
replied to messages never addressed to it.
Only single-mention deliveries are unambiguous bot identities, so restrict
alias learning to them — a multi-mention delivery mixes the bot with
co-mentioned humans, and Feishu's dual-delivery alias form is itself a single
mention, so this is safe and keeps the learning feature working. Also cap the
per-chat alias set size. Adds a [bot, human] co-mention regression test.
Rename the webchat channel from "embed widget" to "Web / API access" (key
unchanged, docs/i18n only) and extend the backend SSE endpoint for pure
backend integration:
- WebChatRequest gains optional agentId (route one Key to multiple agents;
rejected unless the agent shares the channel's workspace) and sessionId
(one visitor, multiple isolated threads).
- sessionId is validated ([A-Za-z0-9_-]{1,64}) and only composed into the
server-derived conversationId — raw conversationIds are never accepted, so
the key+visitor namespace still bounds every thread.
- A `meta` SSE event echoes the effective sessionId/conversationId at stream
start so callers can persist and re-address a thread.
- Memory stays attributed per visitor (api:<visitorId>), shared across that
visitor's sessions.
All new fields are optional; omitting them reproduces the prior behaviour
byte-for-byte. Refs matevip/mateclaw#295.
ChannelMessageEventBridge.onChannelMessage() ran synchronously on the Lark
SDK WebSocket dispatch thread; when the DB pool was saturated its ingest
query blocked that thread and subsequent messages were silently dropped.
- @Async moves trigger ingest onto the (vthread + SecurityContext-propagating)
async executor, freeing the WS dispatch thread.
- Add a 10s timeout to the refreshTenantAccessToken() HTTP request, which
previously had none.
Scope narrowed per review to only fix#208: the @mention alias learning and
session-id changes are dropped (to be raised as separate PRs), and dev's
existing message dedup is left untouched.
Closes#208
Add an execute_code built-in tool that runs python/bash/node code the agent
writes on the fly, so a documentation-only skill (a SKILL.md with no bundled
scripts) can be acted on. Scoped runs inject the skill's secrets and run in
the skill directory; otherwise a private scratch directory is used. Host
secret env vars are scrubbed from the subprocess. execute_code is an
agent-wide capability, registered in the tool catalog (V143), and screened
by the tool guard with a dedicated set of destructive-pattern rules.
Tests cover python/bash/node execution, scratch-dir fallback, env scrubbing,
argument decoding, and guard gating.
- WikiPageTypeProfile: normalise pageType keys to lowercase on set, so a
user-authored profile with an uppercase key still matches the
case-insensitive hasPageType/get lookups.
- WikiDirectoryScanService: normalise the symlink-resolved glob base to
forward slashes so directory-scan globs work on Windows paths.
- Agents roster tag filter: keep selected tags that no longer exist on any
agent visible and deselectable (and show the filter bar when only such
orphan selections remain) instead of silently filtering with no way to clear.
A second POST to /reclassify on the same KB spawned an independent pass over
the same pages, doubling LLM spend and racing the first pass's page-type
writes. Add a per-KB in-flight guard that rejects a concurrent run with a
friendly message (409 via R.fail rather than a generic 500), released in a
finally once the async pass completes. Also count and broadcast per-page
failures so an all-failing run is visible instead of reporting changed=0, and
type the api modelId param as string|number per the snowflake ID convention.
The resilient connect() override started the child process into a local
variable and never set the parent's private process field, so the parent's
closeGracefully() logged "Process not started" and left the child running on
every reconnect/disable. Retain the started process and override
closeGracefully() to destroy it and interrupt the spawned I/O threads.
Also make the inboundSink/errorSink reflection fail loud (the transport is
useless without them — a silent null yielded a connected transport that timed
out on every call), force UTF-8 on the stdio reader/writer, honour a
cooperative closing flag in the reader loops, and drop duplicate PATH entries.
Add a backfill path so pages created before a KB's pageType profile changed
can be migrated into newly-added types. A per-page classify-only LLM call
(title + summary in, single page_type out) is normalised through the profile
and written back via a partial update that never touches page content.
Exposed as POST /knowledge-bases/{id}/reclassify (admin) and a "re-classify
existing pages" action in the Wiki advanced panel.
Wiki page classification was only profile-aware in the main ingest pipeline.
Transformation outputs hard-coded "synthesis", agent-created pages were left
untyped, and the frontend hard-coded the built-in ten types for ordering,
colouring and labels — so custom/synthesis types sank to the bottom, rendered
grey and showed raw keys.
Backend:
- Add nullable target_page_type column to mate_wiki_transformation (V142,
mysql + h2) plus the entity field and CRUD normalization (blank = use
profile fallbackType; membership validated at save time, not edit time).
- Route transformation single-run + KB-aggregate page saves through
WikiPageTypeProfileService.normalizePageType so output joins the KB
classification; agent wiki_create_page now lands on the profile fallbackType
instead of an untyped page.
Frontend:
- Load + parse the KB pageType profile into the wiki store (order, labels,
fallbackType) on KB select / refresh.
- New useWikiPageType composable: profile-driven label (3-tier fallback) and
colour (built-in fixed + deterministic hash palette for custom types).
- Sidebar grouping order, graph colouring, node panel, graph filter and the
page header badge now follow the profile; transformation editor gains a
target-type dropdown sourced from the profile when output target is a page.
Refs #292
Add 5-arg buildContextMessage overload that emits [system-context] Model:
for every origin (web/cron/IM). Legacy 3/4-arg overloads delegate to the
new one with null model args, keeping their output byte-identical.
Also fix pre-existing FeishuMentionTest compile error caused by removed
mentionMatchesAnyAlias/collectMentionIdentifiers methods.
