Bring the webchat visitor session API closer to the admin console's:
- GET /sessions/messages gains beforeId + limit. With a limit it returns
{messages, hasMore} (latest N, then pull-up for older) using the
external path-stripped view; without it, the full list as before.
- GET /sessions/page paginates + keyword-searches a visitor's threads
(in-memory: a visitor's thread set is bounded to its own namespace).
- PUT /sessions/title renames a thread (1-100 chars).
All keep the webchat auth model (API key + visitor token, server-derived
conversationId, ownership guard). Message views go through the shared
toExternalMessageViews helper so the paginated path is sanitized too.
Refs matevip/mateclaw#346
WebChat had no file support: the /stream body carried only text, and
agent-produced files had no visitor-reachable download path (the JWT
/chat/files endpoint is unreachable for API-key visitors).
Add webchat-authenticated file transfer, reusing MessageContentPart +
the existing upload dir + agent multimodal injection:
- WebChatFileService: validate (size cap, extension whitelist, filename
sanitize), store under the conversation's upload dir, stage by opaque
fileId, traversal-safe resolve. Untrusted-uploader hardening lives here.
- POST /upload (multipart) and GET /files, both authed by API key +
visitor token with a server-derived conversationId (never client paths).
Downloads send non-images as attachment + X-Content-Type-Options:nosniff.
- /stream gains attachmentIds; the server resolves each id from the
staging registry (client metadata is never trusted), builds parts, and
persists them on the user message so the agent's multimodal/file tools
pick them up from history — same path as the JWT web chat.
- Strip server-side file paths from the visitor-facing message view
(listMessageViewsExternal + includePath flag) so the filesystem layout
is not disclosed.
Refs matevip/mateclaw#342
The disk fallback's catch block logged at debug, so a failed scan
silently dropped recovered files — reproducing the same 'bot can't see
the file' symptom the fallback was added to fix. Promote to warn with
the full stack trace, matching cacheRecentFile's logging.
loadRecentFilesFromDisk now filters out files older than RECENT_FILE_TTL_MINUTES
(60 min) so the disk fallback matches the Caffeine cache TTL and does not inject
stale attachments into future conversations.
Testability refactoring:
- recentFileCache: private → package-private (tests can seed the cache directly)
- chatUploadsRoot: new package-private Path field (tests redirect to @TempDir)
- loadRecentFilesFromDisk: add (Path dir, long cutoffMs) package-private overload;
private (String) wrapper delegates to it
- injectRecentFiles: private → package-private
New test class FeishuRecentFileCacheTest (14 cases):
- loadRecentFilesFromDisk: non-existent dir, empty dir, fresh files sorted
newest-first, stale files excluded by TTL, mixed fresh+stale, >5 files capped,
timestamp-prefix stripping, MIME guessing from extension
- injectRecentFiles: Caffeine cache hit, cache-miss disk fallback, empty disk,
duplicate-path dedup, image vs file part typing, null textContent guard
Relates to #325
The per-chat recent file cache (Caffeine, 60 min TTL) is purely
in-memory. After a process restart, GC eviction, or TTL expiry the
cache is empty, but the staged copies under data/chat-uploads/ survive
on disk. A follow-up text message that should have seen the cached file
instead found nothing — the bot replied as if no file was ever sent.
Changes:
- injectRecentFiles(): fall back to scanning data/chat-uploads/{id}/
when the Caffeine cache misses, sorted by last-modified time, capped
at RECENT_FILE_MAX_PER_CHAT (5).
- cacheRecentFile(): promote catch log from debug → warn with full
stack trace so silent download failures are visible in production
logs. Add entry-level info log for correlation.
- New helper loadRecentFilesFromDisk() + guessContentType().
Closes#325
Relates to #201
Add KingbaseES support as an opt-in profile: dedicated migration tree, bilingual seed data, runtime DbType detection (KINGBASE_ES / POSTGRE_SQL), and JDBC URL handling in the datasource manager.
The webchat conversationId (webchat:<key8>:<visitorId>[:<sessionId>]) and the
derived username (webchat:<visitorId>) are written to VARCHAR(64) columns, but
visitorId had no validation and sessionId allows 64 chars — so a long visitorId,
or a legitimate 64-char sessionId, overflows the column and the getOrCreateConversation
INSERT throws (500 on /stream). Validate visitorId (charset + blank->UUID) and
fold the variable part into a stable hash when the derived id/username would
exceed 64 chars, keeping short ids byte-identical (backward compatible). Also
make listSessions filter on exact owner username, not just the conversationId
prefix, so system-owned rows can never leak via a crafted visitorId. Adds
boundary regression tests.
Add per-visitor session management for the WebChat Web/API access mode:
list a visitor's conversation threads, fetch a thread's messages, and
delete a thread.
Authorization: visitorId is a client-asserted request param, so it cannot
be trusted on its own — deriving conversationId from it and then checking
ownership against it is tautological (any caller passes). Instead, /stream
issues a per-visitor token = HMAC-SHA256(jwtSecret, channelId:visitorId),
returned in the meta event; the management endpoints require it back via
the X-MC-Visitor-Token header and verify it in constant time. The signing
secret is server-only (unlike the public channel API key) and the channelId
in the payload makes tokens non-portable across channels.
Includes regression tests for token issuance/verification semantics
(forged visitorId rejected, cross-visitor and cross-channel tokens rejected,
tampered tokens rejected).
The WebChat stream endpoint resolves agentId only when the
(visitorId + sessionId) conversation is first created; later requests
with a different agentId reuse the existing conversation's agent and
silently ignore the new value. Document this on WebChatRequest.agentId
so integrators know to use a new sessionId to reach another agent.
The mention alias-learning fed every identifier of every mention in a
delivery into the per-chat alias cache. A single delivery of "@bot @alice"
matched the bot by its global id and then learned alice's openId as a bot
alias, so every later "@alice" message was misdetected as @bot and the agent
replied to messages never addressed to it.
Only single-mention deliveries are unambiguous bot identities, so restrict
alias learning to them — a multi-mention delivery mixes the bot with
co-mentioned humans, and Feishu's dual-delivery alias form is itself a single
mention, so this is safe and keeps the learning feature working. Also cap the
per-chat alias set size. Adds a [bot, human] co-mention regression test.
Rename the webchat channel from "embed widget" to "Web / API access" (key
unchanged, docs/i18n only) and extend the backend SSE endpoint for pure
backend integration:
- WebChatRequest gains optional agentId (route one Key to multiple agents;
rejected unless the agent shares the channel's workspace) and sessionId
(one visitor, multiple isolated threads).
- sessionId is validated ([A-Za-z0-9_-]{1,64}) and only composed into the
server-derived conversationId — raw conversationIds are never accepted, so
the key+visitor namespace still bounds every thread.
- A `meta` SSE event echoes the effective sessionId/conversationId at stream
start so callers can persist and re-address a thread.
- Memory stays attributed per visitor (api:<visitorId>), shared across that
visitor's sessions.
All new fields are optional; omitting them reproduces the prior behaviour
byte-for-byte. Refs matevip/mateclaw#295.
ChannelMessageEventBridge.onChannelMessage() ran synchronously on the Lark
SDK WebSocket dispatch thread; when the DB pool was saturated its ingest
query blocked that thread and subsequent messages were silently dropped.
- @Async moves trigger ingest onto the (vthread + SecurityContext-propagating)
async executor, freeing the WS dispatch thread.
- Add a 10s timeout to the refreshTenantAccessToken() HTTP request, which
previously had none.
Scope narrowed per review to only fix#208: the @mention alias learning and
session-id changes are dropped (to be raised as separate PRs), and dev's
existing message dedup is left untouched.
Closes#208
Same-message attachment reads failed on Feishu: the content part carried the sandbox-external ~/.mateclaw/media/ path, so read_file/extract_document_text were rejected by WorkspacePathGuard and ChatUploadResolver's basename fallback missed (media names {messageId}_{key} vs chat-uploads {millis}_{fileName}). cacheRecentFile already copies the attachment into the per-conversation data/chat-uploads/ dir; return that absolute path and stamp it onto the current message's image/file/audio/media parts so the path surfaced to the LLM is resolver-reachable. Reuses the existing copy — no extra I/O, no new state. Rich-text post multi-image and other channels sharing the media dir are noted as follow-ups.
Closes#278
Streaming chat ran render tools on an async thread with no bound request,
so download links lost their host and arrived without a domain. Resolve the
host on the request thread and carry it through ChatOrigin/ToolContext;
falls back to a configurable public-base-url, then a relative path.
stopWebSocket() only nullified the wsClient reference without calling
disconnect() on the SDK client. This left the old WebSocket connection's
pingLoop thread and ExecutorService running, leaking file descriptors
and threads on each reconnect. Over time, accumulated leaks prevented
new connections from being established, causing the Feishu channel to
silently stop receiving messages.
Fix: use reflection to access the SDK's protected `conn` field and call
close(1000) on the OkHttp WebSocket, triggering the SDK's onClosed →
disconnect() cleanup chain.
Note: oapi-sdk 2.7.1 adds a public close() method that would make this
reflection unnecessary. Consider upgrading as a follow-up.
Closes#220
IM channels (Feishu, DingTalk, WeCom, etc.) and the WebChat widget were
calling saveMessage without token usage parameters, causing promptTokens
and completionTokens to default to 0. This made the Token Statistics
module report significantly lower numbers than actual usage.
Root cause: the _usage_final event (containing promptTokens /
completionTokens) emitted by the agent graph at stream end was not being
captured in these paths, unlike ChatController's StreamAccumulator which
already handles it correctly.
Fix: capture _usage_final events in doOnNext handlers for:
- ChannelMessageRouter sync path (non-streaming IM adapters)
- ChannelMessageRouter streaming path (DingTalk, etc.)
- WebChatController SSE stream
Refs #214 (remaining String-API paths covered by follow-up).
The Lark SDK throws HandlerNotFoundException for any event type without
a registered handler. This exception is caught internally by the SDK's
WebSocket client, which then sends a 500 response to the Feishu server.
The server may close the connection as a result, and the exception is
swallowed — never reaching the application layer.
Added empty handlers for all remaining IM event types:
- P2MessageReadV1 (read receipts)
- P2MessageRecalledV1 (message recall)
- P2ChatMemberBotDeletedV1 (bot removed from chat)
- P2ChatMemberUserAddedV1 / UserDeletedV1 / UserWithdrawnV1
- P2ChatUpdatedV1 (chat info update)
- P2ChatDisbandedV1 (chat disbanded)
- P2ChatAccessEventBotP2pChatEnteredV1 (bot entered p2p chat)
Also added explicit logback config for com.lark.oapi at WARN level
to ensure SDK internal errors are not silently filtered.
Refs: larksuite/oapi-sdk-java#185
