The disk fallback's catch block logged at debug, so a failed scan
silently dropped recovered files — reproducing the same 'bot can't see
the file' symptom the fallback was added to fix. Promote to warn with
the full stack trace, matching cacheRecentFile's logging.
openAiCompatibleClientBuilder() was creating a new java.net.http.HttpClient
per request. Each instance spawns a selector thread and connection pool that
are never closed, exhausting the OS thread limit under frequent model-test
calls (e.g. DeepSeek provider).
Elevate the HttpClient to a static singleton so all OpenAI-compatible
provider requests share one connection pool and one selector thread.
Closesmatevip/mateclaw#328
loadRecentFilesFromDisk now filters out files older than RECENT_FILE_TTL_MINUTES
(60 min) so the disk fallback matches the Caffeine cache TTL and does not inject
stale attachments into future conversations.
Testability refactoring:
- recentFileCache: private → package-private (tests can seed the cache directly)
- chatUploadsRoot: new package-private Path field (tests redirect to @TempDir)
- loadRecentFilesFromDisk: add (Path dir, long cutoffMs) package-private overload;
private (String) wrapper delegates to it
- injectRecentFiles: private → package-private
New test class FeishuRecentFileCacheTest (14 cases):
- loadRecentFilesFromDisk: non-existent dir, empty dir, fresh files sorted
newest-first, stale files excluded by TTL, mixed fresh+stale, >5 files capped,
timestamp-prefix stripping, MIME guessing from extension
- injectRecentFiles: Caffeine cache hit, cache-miss disk fallback, empty disk,
duplicate-path dedup, image vs file part typing, null textContent guard
Relates to #325
The per-chat recent file cache (Caffeine, 60 min TTL) is purely
in-memory. After a process restart, GC eviction, or TTL expiry the
cache is empty, but the staged copies under data/chat-uploads/ survive
on disk. A follow-up text message that should have seen the cached file
instead found nothing — the bot replied as if no file was ever sent.
Changes:
- injectRecentFiles(): fall back to scanning data/chat-uploads/{id}/
when the Caffeine cache misses, sorted by last-modified time, capped
at RECENT_FILE_MAX_PER_CHAT (5).
- cacheRecentFile(): promote catch log from debug → warn with full
stack trace so silent download failures are visible in production
logs. Add entry-level info log for correlation.
- New helper loadRecentFilesFromDisk() + guessContentType().
Closes#325
Relates to #201
The KingbaseES change removed usingDbTime() unconditionally, which made
every deployment (MySQL/H2/PostgreSQL) fall back to app-server time for
distributed lock timing — reintroducing node clock-drift risk in
multi-instance setups. Re-enable usingDbTime() for databases in ShedLock's
built-in dialect map and skip it only for KingbaseES, which is not covered
and would otherwise throw at lock acquisition.
The KingbaseES JDBC driver is not on Maven Central; declaring it as a
required runtime dependency broke the default build for anyone without
the proprietary jar. Move it into an opt-in `kingbase` Maven profile
(build with `mvn package -Pkingbase`). No Java code imports the driver
classes — it is loaded at runtime via driver-class-name only, so the
default build no longer needs it.
Also drop `mateclaw.browser.ssrf-check-enabled: false` from the default
application.yml: the code default is true, and disabling the SSRF guard
globally is unrelated to KingbaseES support.
Add KingbaseES support as an opt-in profile: dedicated migration tree, bilingual seed data, runtime DbType detection (KINGBASE_ES / POSTGRE_SQL), and JDBC URL handling in the datasource manager.
The raw-materials surface was renamed to "Sources" when upload, paste,
directory scan and per-KB auto-sync were unified into one tab. The read-only
viewers' reading-toggle segment still carried the old "Raw materials" label,
so managers saw "Sources" while read-only viewers saw "Raw materials" for the
same panel. Point the segment at the same i18n key for a consistent name.
* feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync
The raw-material directory scan and the Advanced "source watcher" sub-tab were
the same engine (same kb.sourceDirectory, same WikiDirectoryScanService) split
across two surfaces with two editable directory inputs. Merge them into one
"Sources" tab (upload / paste / directory manual scan + auto-sync toggle +
the raw-material list) and drop the watcher sub-tab from Advanced.
Auto-sync is now per-KB opt-in: a new watcher_enabled column (V146) gates the
periodic scan per knowledge base. The server-global mate.wiki.watcher-enabled
stays as an ops master switch — a KB is auto-scanned only when both are on
(AND). Manual scans are unaffected. Scan interval stays global for now
(tracked separately).
Closesmatevip/mateclaw#314
* docs(wiki): document source-watcher global switch env vars
Expose MATE_WIKI_WATCHER_ENABLED / MATE_WIKI_WATCHER_INTERVAL_MS as
explicit placeholders in application-mysql.yml, .env.example and
docker-compose.yml, mirroring MATE_WIKI_ALLOWED_SOURCE_ROOTS. Notes the
AND semantics (global ops gate + per-KB toggle) so operators know the
global switch alone is not sufficient.
The KB workspace split into a reading view (pages + graph) and a manage view
(raw materials, config, transformations, advanced, recent-activity snapshot)
gated behind manage:wiki. That moved the raw-materials and recent-activity
surfaces — which read-only viewers (view:wiki without manage:wiki) could
previously browse — entirely behind the management gate, silently dropping
their access.
Re-surface both in the reading-view segmented toggle for viewers who lack a
management view. Managers keep the focused pages/graph toggle and still reach
these surfaces through the management view, so nothing is duplicated for them.
The content panels already render by activeTab, so this only widens the
reading toggle and the activeTab/readingTab types.
- Fail closed to a global fallback sandbox root when a conversation has no
per-workspace base path, instead of leaving file/shell tools unconstrained
- Refuse shell commands that delete the workspace root directory itself
- Block workspace-boundary escapes at the policy layer before the approval
prompt, not only at execution time
- Approval bar now shows the actual command / target path being approved
The pageType profile could only be edited as a raw JSON string. Add a
structured form editor (default) that builds the profile without writing JSON:
profile-level settings (fallbackType / allowAdditionalFields), an ordered page
type list (add via a small wizard, remove, reorder), and a per-type form for
label / description / layer / field schema, with stage prompts (route/create/
merge) and the markdown template folded into an "advanced" section that carries
inline descriptions and examples. A form/JSON toggle keeps the JSON view as the
final review surface; serialization preserves unknown keys for forward-compat,
and save/validate/reset reuse the existing endpoints (no backend change).
Closesmatevip/mateclaw#310
The KB workspace previously stacked all seven surfaces in one tab strip.
Split them by intent: a gear on each library card opens the management view
(raw materials, config, transformations, advanced, hot cache), while clicking
the card body opens the reading view (pages + graph). The reading view drives
page/graph via a header segmented control with the page tree shown only for
the page viewer; the two views share loaded data and toggle without refetch.
Closesmatevip/mateclaw#308
The V85 seed shipped a dev/test placeholder (sse + http://localhost:8085/sse
+ "Bearer ${CKJIA_MCP_KEY}") that can never connect, so the 参考价 price-
comparison skill was unusable until an admin hand-edited the row.
Add V144 (h2 + mysql) to rewrite the seed to the real CKJIA SaaS endpoint:
streamable_http + https://m.ckjia.com/api/ai/mcp, no auth header, connect/read
timeouts raised to 60s. Conditional on the row still carrying the dev
placeholder URL, so an admin who already pointed it at a private deployment is
left untouched; idempotent and leaves `enabled` opt-in.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes#289 — after an MCP server (re)connects, chat queries kept replying
"from memory" instead of calling MCP tools.
Root cause: agents snapshot their tool set at build time and are cached in
AgentService.agentInstances, but MCP server lifecycle changes never
invalidated that cache (unlike model-config / tool-guard changes which do).
A stale, tool-less agent graph survived until process restart.
Changes:
- Add McpServerChangedEvent; McpServerService publishes it on connect /
disconnect / reconnect / delete / (re)connect-failure / batch refresh /
startup init. AgentService listens and calls refreshAllAgents(), so the
next turn rebuilds against the live MCP tool set. Also closes the boot
race where the web server accepts requests before the @Order(200) MCP
init runner finishes.
- Make create/update/toggle connect asynchronously on a dedicated pool
("mcp-connect") so a slow/unreachable server can no longer freeze the
admin request; status returns immediately as "connecting".
- UI: render the new "connecting" status (pulsing amber dot), show a
friendly "connecting in background" toast, and poll until the status
settles (window widened to ~40s to outlast the default connect timeout).
- UI: MCP config modal no longer closes on outside/backdrop click — only
the × and Cancel buttons close it, so an accidental click can't discard
unsaved config.
Verified E2E: ckjia-shopping (参考价) MCP server connected at runtime with
no backend restart; the cached 通用助手 agent immediately enabled and called
ckjia_shopping_recommend, returning real product cards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The webchat conversationId (webchat:<key8>:<visitorId>[:<sessionId>]) and the
derived username (webchat:<visitorId>) are written to VARCHAR(64) columns, but
visitorId had no validation and sessionId allows 64 chars — so a long visitorId,
or a legitimate 64-char sessionId, overflows the column and the getOrCreateConversation
INSERT throws (500 on /stream). Validate visitorId (charset + blank->UUID) and
fold the variable part into a stable hash when the derived id/username would
exceed 64 chars, keeping short ids byte-identical (backward compatible). Also
make listSessions filter on exact owner username, not just the conversationId
prefix, so system-owned rows can never leak via a crafted visitorId. Adds
boundary regression tests.
Add per-visitor session management for the WebChat Web/API access mode:
list a visitor's conversation threads, fetch a thread's messages, and
delete a thread.
Authorization: visitorId is a client-asserted request param, so it cannot
be trusted on its own — deriving conversationId from it and then checking
ownership against it is tautological (any caller passes). Instead, /stream
issues a per-visitor token = HMAC-SHA256(jwtSecret, channelId:visitorId),
returned in the meta event; the management endpoints require it back via
the X-MC-Visitor-Token header and verify it in constant time. The signing
secret is server-only (unlike the public channel API key) and the channelId
in the payload makes tokens non-portable across channels.
Includes regression tests for token issuance/verification semantics
(forged visitorId rejected, cross-visitor and cross-channel tokens rejected,
tampered tokens rejected).
The WebChat stream endpoint resolves agentId only when the
(visitorId + sessionId) conversation is first created; later requests
with a different agentId reuse the existing conversation's agent and
silently ignore the new value. Document this on WebChatRequest.agentId
so integrators know to use a new sessionId to reach another agent.
The multimodal sidecar selector change made resolveSidecar honour an explicit
sidecar selection even when the built-in capability heuristics don't recognize
the model (it now logs a diagnostic and returns the model instead of rejecting
to NONE). The test still asserted the old reject->NONE path. Update it to assert
SIDECAR and the honoured model, matching the current production behavior.
The vision/video sidecar dropdown filtered candidates through the built-in
capability heuristics, so provider-compatible models whose custom names match
no known prefix (and carry no declared modalities) were hidden and could not be
selected as a sidecar — even when they natively support the modality.
- listByType now returns every enabled chat model, annotating each row with a
transient modalityCapable flag and sorting known-capable rows first, instead
of hard-filtering recognized models only.
- MultimodalRouter honours an explicitly configured sidecar model instead of
dropping it when the heuristics don't recognize it; a wrong pick degrades
gracefully through the caption path rather than silently disabling routing.
- ModelPicker gains an optional capability badge; the sidecar UI tags
recognized vision models while keeping every enabled model selectable.
Both failed on dev independently of the recent merges (confirmed against a
pre-merge baseline):
- MemorySummarizationStructuredRoutingTest reflected applyStructuredEntries by
its old (Long, JsonNode) signature; owner-isolation added a trailing ownerKey
param. Update the reflective lookup to (Long, JsonNode, String) and the
remember() verifications to the 6-arg overload.
- ToolGuardCardHandlerTest still asserted the old 'system-owned pending accepts
any clicker' behavior, but the handler now rejects a group click on a
system/cron-owned approval fail-closed (no human requester to match), routing
it to the admin console. Assert no synthetic injection + the unauthorized card
render instead.
Decoupling the channel-message event bridge onto an @Async listener means the
downstream workflow run is produced off the event-publishing thread. The test
read the run table synchronously right after publishEvent, racing the listener
— the positive cases failed and the negative cases passed for the wrong reason.
Poll briefly for the run (positive) / give the listener time then assert none
(negative) so the test reflects the async dispatch semantics.
The mention alias-learning fed every identifier of every mention in a
delivery into the per-chat alias cache. A single delivery of "@bot @alice"
matched the bot by its global id and then learned alice's openId as a bot
alias, so every later "@alice" message was misdetected as @bot and the agent
replied to messages never addressed to it.
Only single-mention deliveries are unambiguous bot identities, so restrict
alias learning to them — a multi-mention delivery mixes the bot with
co-mentioned humans, and Feishu's dual-delivery alias form is itself a single
mention, so this is safe and keeps the learning feature working. Also cap the
per-chat alias set size. Adds a [bot, human] co-mention regression test.
Rename the webchat channel from "embed widget" to "Web / API access" (key
unchanged, docs/i18n only) and extend the backend SSE endpoint for pure
backend integration:
- WebChatRequest gains optional agentId (route one Key to multiple agents;
rejected unless the agent shares the channel's workspace) and sessionId
(one visitor, multiple isolated threads).
- sessionId is validated ([A-Za-z0-9_-]{1,64}) and only composed into the
server-derived conversationId — raw conversationIds are never accepted, so
the key+visitor namespace still bounds every thread.
- A `meta` SSE event echoes the effective sessionId/conversationId at stream
start so callers can persist and re-address a thread.
- Memory stays attributed per visitor (api:<visitorId>), shared across that
visitor's sessions.
All new fields are optional; omitting them reproduces the prior behaviour
byte-for-byte. Refs matevip/mateclaw#295.
ChannelMessageEventBridge.onChannelMessage() ran synchronously on the Lark
SDK WebSocket dispatch thread; when the DB pool was saturated its ingest
query blocked that thread and subsequent messages were silently dropped.
- @Async moves trigger ingest onto the (vthread + SecurityContext-propagating)
async executor, freeing the WS dispatch thread.
- Add a 10s timeout to the refreshTenantAccessToken() HTTP request, which
previously had none.
Scope narrowed per review to only fix#208: the @mention alias learning and
session-id changes are dropped (to be raised as separate PRs), and dev's
existing message dedup is left untouched.
Closes#208
Add an execute_code built-in tool that runs python/bash/node code the agent
writes on the fly, so a documentation-only skill (a SKILL.md with no bundled
scripts) can be acted on. Scoped runs inject the skill's secrets and run in
the skill directory; otherwise a private scratch directory is used. Host
secret env vars are scrubbed from the subprocess. execute_code is an
agent-wide capability, registered in the tool catalog (V143), and screened
by the tool guard with a dedicated set of destructive-pattern rules.
Tests cover python/bash/node execution, scratch-dir fallback, env scrubbing,
argument decoding, and guard gating.
- WikiPageTypeProfile: normalise pageType keys to lowercase on set, so a
user-authored profile with an uppercase key still matches the
case-insensitive hasPageType/get lookups.
- WikiDirectoryScanService: normalise the symlink-resolved glob base to
forward slashes so directory-scan globs work on Windows paths.
- Agents roster tag filter: keep selected tags that no longer exist on any
agent visible and deselectable (and show the filter bar when only such
orphan selections remain) instead of silently filtering with no way to clear.
