Commit Graph

817 Commits

Author SHA1 Message Date
倪程伟
db3644dc8d refactor(webchat): centralise error codes + dedupe /sessions/page auth (epic #355 PR 6)
Two cleanups promised in the plan, kept narrow to avoid cascading churn:

1. New WebChatErrors enum — single source of truth for the visitor-facing
   HTTP error codes + messages. All future R.fail() calls can reference
   WebChatErrors.INVALID_API_KEY etc. instead of bare literals. This PR
   doesn't migrate every existing call site (that's a noisy sweep better
   done in a follow-up); the enum just needs to exist so audit/OpenAPI
   work in PR 7/8 can quote canonical messages.

2. pageSessions now delegates auth to listSessions instead of duplicating
   the resolveChannel + verifyVisitorToken block. Same external behavior;
   -15 lines of duplication. The pagination/keyword logic stays where it
   is (it's specific to the /page variant and doesn't belong in
   listSessions).

Visitor-token `required=true` migration from plan §6 was dropped: changing
it would flip missing-token responses from 401 to 400, which violates the
current error-code contract that visitors and tests rely on. The
`required=false` + explicit-verify pattern stays.

Regression: 7 webchat test classes, 42/42 green.

Part of epic #355.
2026-06-18 06:33:17 +08:00
倪程伟
f04d57a483 feat(webchat): regenerate last assistant reply (epic #355 PR 4)
New endpoint POST /api/v1/channels/webchat/sessions/regenerate. Behavior:

1. Auth (API Key + visitorToken + ownsConversation — same chain as the
   other session mutations).
2. streamTracker.requestStop() — kill any in-flight stream first so its
   doOnComplete doesn't race the delete/save below.
3. Find last role=user message (seed) and last role=assistant message
   (target).
4. Delete the last assistant message if present.
5. Reuse chatStream by handing it a synthetic WebChatRequest whose
   message is the seed user content. chatStream saves a fresh user
   message (new id, same content) and starts the agent turn.

Trade-off: chatStream saves a new user message rather than replaying the
existing one in place, so the user-side message count grows by 1 per
regenerate. Acceptable — the alternative (refactoring chatStream into
reusable chunks) is a 4-hour distraction from the actual feature, and the
extra row is harmless (history still reads naturally: user, asst, user,
asst, user, asst instead of user, asst, asst).

ConversationService gains findLastMessageByRole() and deleteMessageById()
helpers; both are scoped exactly to what regenerate needs.

WebChatRegenerateTest (@SpringBootTest, 5 cases):
- empty thread (no user message) → error event, no DB change
- deletes the last assistant reply (count strictly decreases)
- bad token → no DB change (auth fails before mutation)
- unknown sessionId → returns emitter without throwing
- seeds from the LAST user message when multiple exist

Tests don't assert on the actual LLM stream content — that's left for
PR 5's WebChatStreamE2ETest, which mocks the chat model.

Part of epic #355.
2026-06-18 06:33:17 +08:00
倪程伟
961ecad7f1 feat(webchat): pin + archive endpoints (epic #355 PR 3)
Two new session-state mutations, both following the rename endpoint's
shape (PUT + {flag: true|false} body + visitorId/sessionId query):

- PUT /api/v1/channels/webchat/sessions/pinned — flips mate_conversation.pinned
- PUT /api/v1/channels/webchat/sessions/archive — flips mate_conversation.archived

Archive complements delete as a "soft-close" — the thread stays on disk
(history preserved, addressable, downloadable) but is hidden from the
default /sessions listing. Pin makes a thread sort first in the visitor's
listing, mirroring the admin-console behavior.

Archive dominates pin: an archived+pinned thread is still hidden by
default. Callers opt back in via includeArchived=true (added in PR 1).

ConversationService gains setArchived(), mirroring the existing
setPinned() pattern.

WebChatArchivePinTest (@SpringBootTest, 6 cases):
- pin flips column + view reflects pinned=1
- archive hides from default listing, includeArchived=true shows it,
  un-archive restores
- archived+pinned still hidden (archive dominates)
- malformed body / wrong type → 400
- unknown sessionId → 404
- bad token → 401

Part of epic #355.
2026-06-18 06:33:17 +08:00
倪程伟
bccc5767ed feat(webchat): visitorToken revocation + 7-day expiry + Caffeine cache (epic #355 PR 2)
Closes the "no way to ban a single visitor without burning the global
JWT secret" gap from the epic. Two changes:

1. Token format: HMAC payload now includes exp, format is
   `<base64sig>.<expEpochSec>`. Default TTL 7 days (VISITOR_TOKEN_TTL_SECONDS).
   Expiry participates in the HMAC, so bumping it client-side invalidates
   the signature. /stream still mints fresh tokens on first contact — a
   revoked visitor can start a new /stream (gets a new token), they just
   can't use the old one on management endpoints.

2. WebChatTokenRevocationService — DB-backed registry (webchat_revoked_visitor
   table from V148) with a 5-minute Caffeine cache in front. revoke() /
   unrevoke() / isRevoked(). The cache accepts up to 10min eventual
   consistency across instances — webchat is low-volume, and a fresh node
   sees revocations immediately on cold cache. DB remains source of truth.

WebChatController.verifyVisitorToken becomes an instance method that chains
verifyVisitorTokenSignature (static, HMAC + exp) with isRevoked (instance,
DB + cache). All 9 management endpoints now check revocation transitively.

Admin endpoint: POST /api/v1/admin/webchat/revoked-visitor (and DELETE to
un-revoke). Mounted under /api/v1/admin/** so it requires a MateClaw JWT
— visitors can't reach it. Records an audit row (action=webchat.revoke-
visitor, resourceType=CHANNEL) via AuditEventService.

Tests:
- WebChatTokenRevocationTest (@SpringBootTest, 7 cases): revoke blocks
  /sessions with 401, un-revoke restores, double-revoke idempotent,
  expired token rejected even without revocation, /stream unaffected
  (signature still verifies), admin endpoint inserts row + audit.
- WebChatVisitorTokenTest extended to 16 cases — added expired-token,
  tampered-exp, and "differs when exp differs" coverage; existing
  verify_* cases moved to verifyVisitorTokenSignature (the static half).

Regression: WebChatSchemaFieldsTest (5/5), WebChatCreateSessionTest (9/9),
WebChatSessionManagementTest (5/5), WebChatStopStreamTest (5/5).

Part of epic #355.
2026-06-18 06:33:17 +08:00
倪程伟
332f3339a9 feat(webchat): archive flag + revoked-visitor table + view fields (epic #355 PR 1)
Schema foundations for the rest of the epic. Three additions:

1. mate_conversation.archived — INT default 0. Lets a visitor "soft-close"
   a thread: stays on disk (history preserved, addressable, downloadable)
   but excluded from default /sessions listing. Pinned/archived are
   orthogonal; archive dominates (archived+pinned still hidden by default).

2. webchat_revoked_visitor — registry table consumed in PR 2 by the
   WebChatTokenRevocationService. Unique on (channel_id, visitor_id, deleted)
   so re-revoke is idempotent and deleted=1 un-revokes. Migration written
   for all three DBs (h2 IF NOT EXISTS, MySQL INFORMATION_SCHEMA guard,
   KingbaseES native IF NOT EXISTS) following the V147 pattern.

3. WebChatSessionView gains pinned/archived/streamStatus so the visitor-
   side listing surfaces the same state the admin console sees. Closes
   the "field exposure" gap from the epic.

loadVisitorSessions gains an includeArchived flag (default false) — the
default hides archived threads and excludes them from the empty-session
quota, since the visitor already declared they're done with them.
GET /sessions and GET /sessions/page thread an `includeArchived=true`
query param through.

Tests (WebChatSchemaFieldsTest, @SpringBootTest + H2 + V148):
- revoked-visitor table is queryable
- archived column is read/write
- view exposes pinned/archived/streamStatus
- archived hidden by default, visible with includeArchived=true
- archived empty threads don't saturate the 5-thread quota

Regression: WebChatCreateSessionTest (9/9), WebChatSessionManagementTest
(5/5) — both updated for the new includeArchived param.

Part of epic #355.
2026-06-18 06:33:17 +08:00
倪程伟
23c1d49241 feat(webchat): stop an in-flight session stream (POST /sessions/stop)
Until now webchat had no way to actually interrupt a running stream —
ChatController's /api/v1/chat/{id}/stop was technically permitAll'd but
silently no-op'd on webchat streams because WebChatController.chatStream
dropped the subscribe() return value, so ChatStreamTracker.requestStop
had no Disposable to dispose. Visitors could only "stop" client-side by
closing the SSE connection; the server-side LLM call kept running,
burning tokens and firing any side-effecting tools to completion.

Two changes (issue #353):

1. WebChatController.chatStream: keep the Disposable and register it
   with streamTracker.setDisposable, mirroring ChatController#chatStream
   line 495. Now requestStop actually disposes the Flux.

2. New endpoint POST /api/v1/channels/webchat/sessions/stop:
   - Auth mirrors the other session-management endpoints: X-MC-Key +
     X-MC-Visitor-Token + ownsConversation (404 on unknown sessionId,
     so callers can't probe the namespace).
   - Returns {stopped: true|false}; false means no active stream
     (idempotent, not an error).
   - No approval sweep — webchat has no MateClaw username and exposes
     no approval UI today; defer until that surfaces.

WebChatStopStreamTest (@SpringBootTest, H2, V147) — 5 cases:
- stopActiveStream registers a real Flux.never() Disposable on the
  tracker and asserts both stopped=true AND disposable.isDisposed(),
  proving the chatStream wiring change is what makes the endpoint work.
- noActiveStreamReturnsFalse — idempotent path.
- bad token / bad API key → 401.
- unknown sessionId → 404.
2026-06-18 06:33:17 +08:00
倪程伟
d84be668fa feat(webchat): explicit empty-session creation endpoint POST /sessions
Complements the implicit getOrCreate in /stream: lets a caller pre-create
an empty thread (message_count = 0) and receive sessionId / conversationId /
visitorToken up front, then decide when to send the first message via
/stream. Mirrors how downstream CRM/ticketing systems model "create the
conversation object first, message later".

Auth is the visitor's first touch — only X-MC-Key is required (no
X-MC-Visitor-Token, which the visitor can't have yet); the server signs
and returns a fresh visitorToken the caller must echo back on subsequent
GET/PUT/DELETE.

Behavior (issue #351):
- Idempotent on sessionId collision → returns the existing thread 200,
  does NOT clobber title.
- Empty-session quota ≤ 5 per (channel, visitor); 409 with a clear
  message when exceeded. Existing rows are exempt (re-create is idempotent).
- Caller-supplied title (1-100 chars) is persisted; absent title leaves
  the default "新对话" so the first /stream user message still derives
  it. getOrCreateWebchatConversation now accepts an optional title and
  only writes it on insert (existing rows untouched).
- agentId override mirrors /stream's workspace check.

ConversationService.getOrCreateWebchatConversation gains a title-aware
overload; the original 5-arg signature delegates with title = null.

End-to-end coverage in WebChatCreateSessionTest (@SpringBootTest, H2
with V147 migration): happy path, caller-title survives first user
message, default-title still derived, idempotent collision, quota 409,
bad API key 401, illegal sessionId/title 400, listed after creation.
2026-06-17 23:21:08 +08:00
倪程伟
bdda9c7357 perf(webchat): scope session listing to the visitor; cap upload disk use
Two webchat hardening fixes:

- Session listing no longer pulls every system-owned conversation into
  memory. listSessions/pageSessions went through listConversations(owner)
  whose `username IN (owner, system)` loaded all IM/cron rows just to show
  one visitor's handful of threads. New listWebchatConversations(username)
  queries only the visitor's own rows; the channel prefix is matched
  in-memory with a literal startsWith (so a '_'/'%' in the api key's first
  8 chars can't act as a LIKE wildcard).
- Upload now enforces a per-conversation quota (file count + total bytes,
  both configurable) so a visitor can't fill the disk with many
  individually-under-cap files. Pairs with the existing staging TTL sweep.
2026-06-17 23:21:08 +08:00
倪程伟
ee3e391977 fix(webchat): list sessions whose conversationId hashed (long ids)
When webchat:<key8>:<visitorId>:<sessionId> exceeds 64 chars the
conversationId folds visitorId+sessionId into an unrecoverable hash, so
the thread fell outside listSessions' conversationId-prefix filter and its
sessionId could not be recovered — the thread was invisible and
unaddressable (common with a UUID visitorId + a >10-char sessionId).

Persist the sessionId on creation (new nullable webchat_session_id column)
and enumerate by username + channel prefix (webchat:<key8>:), which also
matches the hashed form. sessionId is read from the column, falling back to
parsing the conversationId only for legacy rows.

Adds a @SpringBootTest covering listing (incl. the hashed thread), message
pagination, session paging/search, rename, and token rejection end-to-end.

Refs matevip/mateclaw#346
2026-06-17 23:21:08 +08:00
倪程伟
4842a2208a feat(webchat): message pagination, session list paging/search, rename
Bring the webchat visitor session API closer to the admin console's:

- GET /sessions/messages gains beforeId + limit. With a limit it returns
  {messages, hasMore} (latest N, then pull-up for older) using the
  external path-stripped view; without it, the full list as before.
- GET /sessions/page paginates + keyword-searches a visitor's threads
  (in-memory: a visitor's thread set is bounded to its own namespace).
- PUT /sessions/title renames a thread (1-100 chars).

All keep the webchat auth model (API key + visitor token, server-derived
conversationId, ownership guard). Message views go through the shared
toExternalMessageViews helper so the paginated path is sanitized too.

Refs matevip/mateclaw#346
2026-06-17 23:21:08 +08:00
倪程伟
594880bd64 feat(webchat): support inbound file upload and outbound download
WebChat had no file support: the /stream body carried only text, and
agent-produced files had no visitor-reachable download path (the JWT
/chat/files endpoint is unreachable for API-key visitors).

Add webchat-authenticated file transfer, reusing MessageContentPart +
the existing upload dir + agent multimodal injection:

- WebChatFileService: validate (size cap, extension whitelist, filename
  sanitize), store under the conversation's upload dir, stage by opaque
  fileId, traversal-safe resolve. Untrusted-uploader hardening lives here.
- POST /upload (multipart) and GET /files, both authed by API key +
  visitor token with a server-derived conversationId (never client paths).
  Downloads send non-images as attachment + X-Content-Type-Options:nosniff.
- /stream gains attachmentIds; the server resolves each id from the
  staging registry (client metadata is never trusted), builds parts, and
  persists them on the user message so the agent's multimodal/file tools
  pick them up from history — same path as the JWT web chat.
- Strip server-side file paths from the visitor-facing message view
  (listMessageViewsExternal + includePath flag) so the filesystem layout
  is not disclosed.

Refs matevip/mateclaw#342
2026-06-17 23:21:08 +08:00
倪程伟
8e339f083a docs(security): correct generated-file TTL comment (7 days, not 10 min)
The permitAll comment claimed a 10-minute TTL, but GeneratedFileCache.TTL
is 7 days. The stale figure could mislead future security reasoning about
how long an unauthenticated capability URL stays live. Align the comment
with the actual value; the unguessable UUID remains the access guard.

Refs matevip/mateclaw#344
2026-06-17 23:21:08 +08:00
倪程伟
7f4c62c3d6 fix(conversation): surface webchat visitor sessions in the admin console
WebChat conversations are owned by an external visitor principal
(webchat:<visitorId>) so each visitor's threads stay isolated for the
self-service session API. But the console list/page/owner-check only
recognized the current user + system owners, so these conversations were
invisible in the sidebar and the Sessions page — and would 403 on open
even if surfaced.

Treat webchat: owners like system owners for the console: include them in
the lenient list/page queries and in isConversationOwner. The strict
listConversations overload (used by the visitor self-service path) is
unchanged, so a visitor's own access is not widened.

Refs matevip/mateclaw#340
2026-06-17 23:21:08 +08:00
matevip
1522009aec feat(agent): one-sentence AI employee creation wizard
Turn a single natural-language requirement into a ready-to-review
employee: the model proposes name, persona, runtime type and a
validated set of skills/tools/knowledge base, which the user confirms
or tweaks before the agent is created.

- backend: POST /api/v1/agents/generate builds a draft from the
  workspace's real capability catalog; every suggested tool/skill/KB is
  re-validated against the catalog so nothing hallucinated is offered
- frontend: 3-step wizard at /agents/create reusing the existing
  create + binding endpoints; reusable capability picker shows selected
  items as compact chips with an on-demand searchable catalog
2026-06-17 17:38:42 +08:00
matevip
6e7c137154 feat(wiki): entity-level knowledge graph extraction (#336)
Add an opt-in named-entity extraction pass so the wiki knowledge graph
captures fine-grained entities (people, organizations, locations, ...)
and their relations, not just page-level link relations.

- new tables mate_wiki_entity / _mention / _relation (h2/mysql/kingbase)
- structured LLM extraction per chunk with entity resolution
  (normalized-key dedup + embedding near-merge), mention/relation
  persistence and page linking via chunk citations
- per-KB opt-in toggle (off by default); async dispatch after embedding
- read API: entity list, KB graph, entity ego-graph, manual extract
- UI: entity-layer toggle in the graph view + KB config toggle
- replace inline fully-qualified class names with imports in WikiProcessingService

Closes #336
2026-06-17 14:17:54 +08:00
matevip
fe68f22aa8 feat(memory): bound always-on memory growth with injection budget, consolidation, and file ceilings
Always-on memory (structured user/feedback blocks, PROFILE.md, MEMORY.md) is injected into every system prompt but only ever grew, inflating per-turn context over time. This adds deterministic size control across all always-on sources:

- Injection budget: cap the always-on structured block by total chars and per-type entry count, keeping the most-recently-updated entries (LRU by Updated date) and disclosing how many were omitted
- Nightly consolidation: a dedicated scheduled pass merges duplicate/stale user & feedback entries via the LLM, preserving each entry's original Updated date; runs per owner bucket (shared + personal) with a per-run cap and a never-grow safety guard
- File ceilings: deterministic backstop truncates PROFILE.md / MEMORY.md at a section boundary when a rewrite overruns its budget
- Manual trigger endpoint for the consolidation maintenance task

All knobs under mate.memory.*; covered by unit tests.
2026-06-17 11:04:19 +08:00
matevip
1affbd7b82 feat(agent): loop-engineering robustness — goal continuation, plan re-plan, stall detection
- goal: continue (not skip) on max-iterations and evidence-insufficient turns.
  A max-iterations turn grants a fresh iteration budget ("hard continuation"),
  bounded per run and sized into the graph recursion ceiling, so a task too big
  for one budget keeps going instead of stalling until the next user message.
- plan-execute: re-plan the remaining work on a step exception, and on a
  signature-based stall (repeated failures / identical results / no usable
  result) instead of advancing dependent steps with junk; bounded by a per-run
  re-plan cap, with a graduated change-strategy nudge before the hard stop.
- plan-execute: auto-derive a goal from a genuine multi-step plan, seeding the
  acceptance criteria from the plan steps, so the goal subsystem engages without
  the model calling setGoal; broadcast goal_created so the UI hydrates.
- react: refund the iteration for setup-only rounds (load_skill / enable_tool)
  so a tight budget is not eaten by the load-then-use two-step.
- ui: re-fetch the active goal when a turn finishes so a goal created or mutated
  mid-conversation surfaces without depending on an SSE event.
- streaming: make retry backoff / total-time budget instance fields with a
  test-only seam; clarify that the wall-clock budget (not max-retries) bounds a
  sustained SERVER_ERROR loop to ~8 attempts, fixing the slow/flaky retry test.
2026-06-17 06:37:08 +08:00
matevip
85ceafa055 fix(agent): scope KB grounding to wiki-equipped agents and wiki tools
The knowledge-base trust verification recorded wiki citations from every
non-readFile tool response by sniffing its JSON for a top-level title /
pages / chunks field. Tools like getGoalStatus return a top-level title,
which falsely populated the citation set and then forced [n] citations on
the final answer (otherwise flagged EVIDENCE_INSUFFICIENT). Gate citation
mining on the wiki_* tool name instead.

Likewise, the grounded answer contract (cite-or-refuse) was appended to
every ReasoningNode call unconditionally, degrading general agents that
have no knowledge base. Append it only when the agent has a wiki_* tool
bound, scoping the strict regime to KB-grounded scenarios.

Adds a regression test asserting a non-wiki tool with a top-level title
creates no wiki citations.
2026-06-16 07:48:04 +08:00
jack
88be1f748a
[#305] [Feature] Add knowledge base trust verification (#334)
Co-authored-by: SuperCoderMan521 <SuperCoderManqq.com>
2026-06-16 07:47:04 +08:00
matevip
d9a9d07704 fix(wiki): broken-link rescan precision, slug/title resolution, and dangling-link reconcile (#333)
- rescan: keep the KB id as a string end to end so the 19-digit snowflake id
  isn't truncated past Number.MAX_SAFE_INTEGER (rescan no longer 404s)
- lint: resolve [[...]] targets against page slugs AND titles like the viewer,
  so a title reference to an existing page is no longer reported broken
- ingest: derive the slug deterministically from the title (no inconsistent
  romanization), auto-recompute broken links once a KB finishes importing,
  and reconcile dangling [[concept]] links — redirect to the covering page via
  declared aliases, or demote to plain text when uncovered
- add the page aliases column migration for h2 / mysql / kingbase
2026-06-15 16:14:25 +08:00
matevip
4cbd2b50f3 feat(dashboard): show the connected database on the dashboard
Surface the connected database product as a subtle chip in the Dashboard
header. SystemHealthService now reports a database label on /system/health
(reused by the front-end — no extra request), derived from a new
DatabaseBootstrapRunner.getDatabaseLabel() that reads the JDBC product name
once and normalizes it to a canonical label (MySQL / MariaDB / PostgreSQL /
H2, and 人大金仓 for the KingbaseES family), collapsing driver version noise.
2026-06-15 08:47:53 +08:00
matevip
710b756281 test(chat): cover gateway-resilience error classification; fix PKIX casing
Add ErrorClassificationTest regression cases for the AI-gateway retry
hardening: 5xx-before-4xx ordering (a proxy 502 whose body says
"bad request" stays retryable), Chinese / numeric provider billing
patterns, and DNS / TLS infrastructure-fatal detection.

Fix the cert-trust pattern while adding its test: Java's ValidatorException
emits "PKIX path building failed" with an uppercase PKIX and the error
chain is not lower-cased, so the previous lowercase pattern never matched
— an untrusted/expired cert chain fell through to the retryable
SERVER_ERROR bucket and was retried in vain instead of failing over.
2026-06-15 08:09:59 +08:00
MIST
42f1d5b685 fix(chat): resilient retry for transient AI gateway errors 2026-06-15 08:02:02 +08:00
倪程伟
7c4380a116
feat(docs): expose bundled help docs via in-app viewer
Closes #330
2026-06-15 07:48:49 +08:00
matevip
6bebfed07c fix(feishu): surface recent-file disk-scan failures at warn level
The disk fallback's catch block logged at debug, so a failed scan
silently dropped recovered files — reproducing the same 'bot can't see
the file' symptom the fallback was added to fix. Promote to warn with
the full stack trace, matching cacheRecentFile's logging.
2026-06-14 17:25:26 +08:00
倪程伟
f361e0e917 fix: reuse shared HttpClient to prevent thread-leak OOM on model test
openAiCompatibleClientBuilder() was creating a new java.net.http.HttpClient
per request. Each instance spawns a selector thread and connection pool that
are never closed, exhausting the OS thread limit under frequent model-test
calls (e.g. DeepSeek provider).

Elevate the HttpClient to a static singleton so all OpenAI-compatible
provider requests share one connection pool and one selector thread.

Closes matevip/mateclaw#328
2026-06-14 16:57:13 +08:00
倪程伟
515cba88ee test(feishu): add TTL filter and unit tests for recent-file disk fallback
loadRecentFilesFromDisk now filters out files older than RECENT_FILE_TTL_MINUTES
(60 min) so the disk fallback matches the Caffeine cache TTL and does not inject
stale attachments into future conversations.

Testability refactoring:
- recentFileCache: private → package-private (tests can seed the cache directly)
- chatUploadsRoot: new package-private Path field (tests redirect to @TempDir)
- loadRecentFilesFromDisk: add (Path dir, long cutoffMs) package-private overload;
  private (String) wrapper delegates to it
- injectRecentFiles: private → package-private

New test class FeishuRecentFileCacheTest (14 cases):
- loadRecentFilesFromDisk: non-existent dir, empty dir, fresh files sorted
  newest-first, stale files excluded by TTL, mixed fresh+stale, >5 files capped,
  timestamp-prefix stripping, MIME guessing from extension
- injectRecentFiles: Caffeine cache hit, cache-miss disk fallback, empty disk,
  duplicate-path dedup, image vs file part typing, null textContent guard

Relates to #325
2026-06-14 16:57:13 +08:00
倪程伟
39a55db65f fix(feishu): recover recent files from disk when in-memory cache misses
The per-chat recent file cache (Caffeine, 60 min TTL) is purely
in-memory.  After a process restart, GC eviction, or TTL expiry the
cache is empty, but the staged copies under data/chat-uploads/ survive
on disk.  A follow-up text message that should have seen the cached file
instead found nothing — the bot replied as if no file was ever sent.

Changes:
- injectRecentFiles(): fall back to scanning data/chat-uploads/{id}/
  when the Caffeine cache misses, sorted by last-modified time, capped
  at RECENT_FILE_MAX_PER_CHAT (5).
- cacheRecentFile(): promote catch log from debug → warn with full
  stack trace so silent download failures are visible in production
  logs.  Add entry-level info log for correlation.
- New helper loadRecentFilesFromDisk() + guessContentType().

Closes #325
Relates to #201
2026-06-14 16:57:13 +08:00
matevip
07da1d610b feat(db): add PostgreSQL Spring profile 2026-06-14 16:47:02 +08:00
matevip
ed6eac310a fix(cron): restore ShedLock DB-time for dialects that support it
The KingbaseES change removed usingDbTime() unconditionally, which made
every deployment (MySQL/H2/PostgreSQL) fall back to app-server time for
distributed lock timing — reintroducing node clock-drift risk in
multi-instance setups. Re-enable usingDbTime() for databases in ShedLock's
built-in dialect map and skip it only for KingbaseES, which is not covered
and would otherwise throw at lock acquisition.
2026-06-14 10:44:52 +08:00
铭萱
446f34b6b5
feat(db): support KingbaseES (人大金仓) domestic database (#324)
Add KingbaseES support as an opt-in profile: dedicated migration tree, bilingual seed data, runtime DbType detection (KINGBASE_ES / POSTGRE_SQL), and JDBC URL handling in the datasource manager.
2026-06-14 10:30:09 +08:00
matevip
936c8621ed fix(wiki): extract uploaded documents via a sandbox-exempt path (#323) 2026-06-13 07:29:39 +08:00
matevip
48024e4a83 fix(wiki): dedup pages by title and bound route prompt growth (#321) 2026-06-12 15:17:32 +08:00
matevip
c1691e466c fix(agent): stop ProgressLedger from pinning virtual-thread carriers under parallel progress_update 2026-06-12 11:07:55 +08:00
matevip
e6f8da9606 fix(agent): recover tool-call follow-up on interleaved-thinking models (#256) 2026-06-12 11:07:55 +08:00
matevip
5235e30138 fix(tool): keep Snowflake ids precise across the tool boundary (#319) 2026-06-11 15:17:51 +08:00
matevip
bad216d686 feat(tool): add image_analyze for on-demand image re-analysis (#303) 2026-06-11 13:53:56 +08:00
matevip
aaf06b262c feat(agent): retain image context across turns for follow-ups (#303) 2026-06-11 13:53:56 +08:00
matevip
dfc8e4c786 fix(channel): warn when wecom inbound image is stored URL-only (#303) 2026-06-11 13:53:56 +08:00
matevip
4a3d4cf568 fix(mcp): auto-heal stale MCP connections after server restart (#317) 2026-06-11 09:59:25 +08:00
倪程伟
18daad79b2
feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync (#316)
* feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync

The raw-material directory scan and the Advanced "source watcher" sub-tab were
the same engine (same kb.sourceDirectory, same WikiDirectoryScanService) split
across two surfaces with two editable directory inputs. Merge them into one
"Sources" tab (upload / paste / directory manual scan + auto-sync toggle +
the raw-material list) and drop the watcher sub-tab from Advanced.

Auto-sync is now per-KB opt-in: a new watcher_enabled column (V146) gates the
periodic scan per knowledge base. The server-global mate.wiki.watcher-enabled
stays as an ops master switch — a KB is auto-scanned only when both are on
(AND). Manual scans are unaffected. Scan interval stays global for now
(tracked separately).

Closes matevip/mateclaw#314

* docs(wiki): document source-watcher global switch env vars

Expose MATE_WIKI_WATCHER_ENABLED / MATE_WIKI_WATCHER_INTERVAL_MS as
explicit placeholders in application-mysql.yml, .env.example and
docker-compose.yml, mirroring MATE_WIKI_ALLOWED_SOURCE_ROOTS. Notes the
AND semantics (global ops gate + per-KB toggle) so operators know the
global switch alone is not sufficient.
2026-06-11 09:29:26 +08:00
matevip
83615593cd fix(tool/guard): harden workspace filesystem sandbox (#313)
- Fail closed to a global fallback sandbox root when a conversation has no
  per-workspace base path, instead of leaving file/shell tools unconstrained
- Refuse shell commands that delete the workspace root directory itself
- Block workspace-boundary escapes at the policy layer before the approval
  prompt, not only at execution time
- Approval bar now shows the actual command / target path being approved
2026-06-10 17:17:07 +08:00
matevip
7478491652 feat(llm): add Claude Fable 5 support 2026-06-10 10:14:51 +08:00
matevip
c1fd39a1e2 fix(agent/plan): 分流器证据闸门防止复杂任务不执行就停止(v2 剥离 memory-context) 2026-06-09 17:55:26 +08:00
matevip
64b8e167d5 fix(mcp): refresh agent cache on MCP connection change + non-blocking connect (#289)
Closes #289 — after an MCP server (re)connects, chat queries kept replying
"from memory" instead of calling MCP tools.

Root cause: agents snapshot their tool set at build time and are cached in
AgentService.agentInstances, but MCP server lifecycle changes never
invalidated that cache (unlike model-config / tool-guard changes which do).
A stale, tool-less agent graph survived until process restart.

Changes:
- Add McpServerChangedEvent; McpServerService publishes it on connect /
  disconnect / reconnect / delete / (re)connect-failure / batch refresh /
  startup init. AgentService listens and calls refreshAllAgents(), so the
  next turn rebuilds against the live MCP tool set. Also closes the boot
  race where the web server accepts requests before the @Order(200) MCP
  init runner finishes.
- Make create/update/toggle connect asynchronously on a dedicated pool
  ("mcp-connect") so a slow/unreachable server can no longer freeze the
  admin request; status returns immediately as "connecting".
- UI: render the new "connecting" status (pulsing amber dot), show a
  friendly "connecting in background" toast, and poll until the status
  settles (window widened to ~40s to outlast the default connect timeout).
- UI: MCP config modal no longer closes on outside/backdrop click — only
  the × and Cancel buttons close it, so an accidental click can't discard
  unsaved config.

Verified E2E: ckjia-shopping (参考价) MCP server connected at runtime with
no backend restart; the cached 通用助手 agent immediately enabled and called
ckjia_shopping_recommend, returning real product cards.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 14:14:08 +08:00
matevip
846c1c31ca fix(channel): bound webchat conversationId/username to prevent VARCHAR(64) overflow
The webchat conversationId (webchat:<key8>:<visitorId>[:<sessionId>]) and the
derived username (webchat:<visitorId>) are written to VARCHAR(64) columns, but
visitorId had no validation and sessionId allows 64 chars — so a long visitorId,
or a legitimate 64-char sessionId, overflows the column and the getOrCreateConversation
INSERT throws (500 on /stream). Validate visitorId (charset + blank->UUID) and
fold the variable part into a stable hash when the derived id/username would
exceed 64 chars, keeping short ids byte-identical (backward compatible). Also
make listSessions filter on exact owner username, not just the conversationId
prefix, so system-owned rows can never leak via a crafted visitorId. Adds
boundary regression tests.
2026-06-09 11:36:41 +08:00
倪程伟
77b6baeccc feat(channel): webchat session-management endpoints (list / messages / delete)
Add per-visitor session management for the WebChat Web/API access mode:
list a visitor's conversation threads, fetch a thread's messages, and
delete a thread.

Authorization: visitorId is a client-asserted request param, so it cannot
be trusted on its own — deriving conversationId from it and then checking
ownership against it is tautological (any caller passes). Instead, /stream
issues a per-visitor token = HMAC-SHA256(jwtSecret, channelId:visitorId),
returned in the meta event; the management endpoints require it back via
the X-MC-Visitor-Token header and verify it in constant time. The signing
secret is server-only (unlike the public channel API key) and the channelId
in the payload makes tokens non-portable across channels.

Includes regression tests for token issuance/verification semantics
(forged visitorId rejected, cross-visitor and cross-channel tokens rejected,
tampered tokens rejected).
2026-06-09 11:10:13 +08:00
倪程伟
12c24651a2 docs(webchat): clarify agentId is pinned at conversation creation
The WebChat stream endpoint resolves agentId only when the
(visitorId + sessionId) conversation is first created; later requests
with a different agentId reuse the existing conversation's agent and
silently ignore the new value. Document this on WebChatRequest.agentId
so integrators know to use a new sessionId to reach another agent.
2026-06-09 11:10:11 +08:00
倪程伟
a40868eff2 refactor(feishu): 群会话 ID 改用完整 chatId 避免后缀碰撞
旧实现群会话 conversationId = feishu:{appId后4}_{chatId后8},截断后缀
存在碰撞风险:不同群 chatId 后 8 位相同 → 消息落进同一会话、上下文串台。
群会话改用完整 chatId(feishu:{chatId})消除碰撞。

存量迁移(读时别名回退,不重写存量行):
- 旧后缀不可逆推完整 chatId,故不做一次性回填;
- 但每条入站群消息都带完整 chatId + appId,可在路由前重算 legacy key;
- 新群 / 已迁移群 → 用 feishu:{chatId};存量群(canonical 无、legacy 有)
  → 沿用 legacy key,历史无缝延续,零停机、零破坏性写。

- ChannelMessageRouter 增只读 conversationExists(id)(委托 findByConversationId)。
- 私聊不受影响(DM 经 buildConversationId 直接用完整 senderOpenId,后缀本就不参与)。

Closes #299
2026-06-09 11:10:09 +08:00
matevip
a9cf3cbd55 fix(llm): show all enabled chat models in multimodal sidecar selector
The vision/video sidecar dropdown filtered candidates through the built-in
capability heuristics, so provider-compatible models whose custom names match
no known prefix (and carry no declared modalities) were hidden and could not be
selected as a sidecar — even when they natively support the modality.

- listByType now returns every enabled chat model, annotating each row with a
  transient modalityCapable flag and sorting known-capable rows first, instead
  of hard-filtering recognized models only.
- MultimodalRouter honours an explicitly configured sidecar model instead of
  dropping it when the heuristics don't recognize it; a wrong pick degrades
  gracefully through the caption path rather than silently disabling routing.
- ModelPicker gains an optional capability badge; the sidecar UI tags
  recognized vision models while keeping every enabled model selectable.
2026-06-09 10:30:37 +08:00