Follow-up to #447. The generated-file link extraction accepted any
non-')' text before the path, so a paren-free javascript:/data: URL
embedding /api/v1/files/generated/<id> could be captured and bound to an
<a href>, enabling XSS on click. Adopt the scheme-restricted pattern
already used by SegmentSupersedeDetector and the channel adapters, on
both backend (ChatController) and frontend (useChat). Also replace the
inline fully-qualified Pattern/Matcher with imports and drop an unused
run-overview i18n key.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extract generated-file download links from tool results — on the backend (persisted to message metadata for history) and on the frontend (live during SSE) — de-duplicate by URL, and render them as a Generated Files section with file-type icons and a rail badge.
Propagates structured error codes through the KB processing pipeline, surfaces silent sub-step warnings as a non-failure warning state, and adds a cross-KB failure center for aggregated visibility.
Lets an employee pin an ordered fallback chain of (provider, model) entries; the same provider may appear multiple times with different models. Build-time dedup keys on exact (provider, model).
Two compounding causes made the management view jump from the config
tab back to 'raw' a few seconds after the user selected it:
1. The tab-snap watcher used a single getter returning a new array
(`() => [currentKB?.id, workspaceMode]`). Vue compares the returned
value with Object.is, so a fresh array reference reports a change on
every re-evaluation — including background refreshCurrentKB() calls
that reassign the KB object with the same id. That re-ran the snap and
forced activeTab back to 'raw'. Switch to an array of getters so each
source is compared individually and the callback fires only on a real
id/mode change.
2. RawMaterialPanel's onBeforeUnmount cleared the SSE stream and the 60s
fallback timer but not the per-raw jobPoller setTimeout chain. While a
raw was still processing, leaving the sources tab left that 3s poller
running, calling refreshCurrentKB() indefinitely. Clear jobPoller on
unmount as well.
The config tab pane (.tab-content--config) was set to overflow:hidden,
mirroring the graph pane, but its inner .wiki-config has no bounded height
so its own overflow-y:auto never triggers. Tall config content (model
strategy / processing rules / search-preview cards) overflowed off-screen
with no scrollbar.
Switch the pane to overflow-y:auto like the generic .tab-content. The
existing <=980px media query (overflow:visible) keeps mobile page-scroll
intact. Pure CSS, no logic change.
The chat composables (useStickToBottom / useStream / useMessages / useTyping)
carried '参考 @agentscope-ai/chat …' attribution comments. That package is not
a dependency and is never imported — the lines were pure citation. Rewrite them
as objective functional descriptions so shipped code does not name external
projects.
- useChat.ts reconnectStream: cast the reused assistant message id to string
when calling updateMessage; the id is optional in the Message type, so the
raw value broke the vue-tsc build (TS2345, undefined not assignable).
- useStickToBottom.ts handleScroll: restore the block's indentation (it had
drifted to 1/3-space) and add a comment for the scroll-up release branch.
Verified: vue-tsc --noEmit passes; snowflake precision check clean.
* feat(sso): feishu OAuth2 single sign-on (ISSUE #405 P0)
Implements the SSO design (ISSUE #405) with feishu as the first IdP
and a generic OAuth2 provider abstraction for future dingtalk/wecom
extensions. SSO is disabled by default — existing deployments are
unaffected until mateclaw.sso.enabled=true.
Backend:
- SsoProvider interface + SsoUserInfo record: generic IdP abstraction
- FeishuSsoProvider: OAuth2 authorization-code flow (app_access_token
with Caffeine cache → user_access_token → user info). apiBase switches
between feishu.cn / larksuite.com by domain config.
- SsoProviderRegistry: conditional registration, lists enabled providers
- SsoStateService: HMAC-signed OAuth2 state + self-contained bind_token
JWT, both persisted to sso_state DB table for multi-node correctness.
State is one-time-consumable (conditional UPDATE), bind_token jti
anti-replay via PK insert. Hourly ShedLock purge (LambdaQuery + Java
time, works on all 3 dialects).
- SsoService: authorize/callback/bind, user mapping (union_id first →
external_id fallback), auto-create with concurrent idempotency
(DuplicateKeyException → rollback orphan user → re-query), link-only
mode issues bind_token for existing-account binding.
- SsoController: 4 endpoints (/providers, /authorize, /callback, /bind)
all permitAll.
- V159 migration (h2/mysql/kingbase): mate_user_external_identity,
sso_state, ALTER mate_user.password NULL (SSO-only users).
- AuthService: generateToken promoted to public; login() guards
password=null (SSO-only users cannot password-login).
- SecurityConfig: /auth/sso/** added to permitAll whitelist.
- LoginRateLimitFilter: expanded to cover /auth/sso/bind (brute-force
surface equivalent to /auth/login).
- application.yml: mateclaw.sso.* config block (all env-var driven).
Frontend:
- Login.vue: dynamic SSO buttons (only shown when providers configured),
OAuth2 callback detection (?sso=callback), link-only bind dialog,
shared applyLogin flow (localStorage + workspace + route).
- api/index.ts: ssoApi (providers, authorize, callback, bind).
Tests: SsoStateServiceTest (11) — state issue/verify/replay/tamper,
bind_token issue/verify/anti-replay/garbage. Regression: PAT (23) +
Approval resolve (13) all green.
Not in scope (P1/P2): link-only bind/unbind management endpoints,
user enable/disable endpoint, dingtalk/wecom providers, admin SSO
config page. Workspace assignment for auto-created users remains a
product decision (design doc §12 item 2).
* fix(sso): self-review fixes — P0 security + P1 quality
P0-1 BindRequired serialization: replaced the R.fail(200, Map.toString())
hack with a structured SsoCallbackResponse record. Controller no longer
catches an exception for a non-error path; frontend reads bindRequired
flag directly instead of regex-parsing a stringified map.
P0-2 createSsoUser unbounded recursion: added a retry flag — second
DuplicateKeyException (extreme race where identity was concurrently
deleted) now throws a 503 instead of recursing to stack overflow.
P0-3 state TTL not enforced: verifyState's conditional UPDATE now
includes created_at > cutoff, so a state unused for 5+ min is rejected
at consumption time, not just at the 1h purge. Without this the 5-min
window was advisory only.
P1-5 SsoStateService unused ObjectMapper: removed dead injection.
P1-6 audit JSON string concat: replaced with ObjectMapper serialization
(provider/externalId no longer risk breaking the JSON structure).
P1-7 LoginRateLimitFilter shared counter: documented the intentional
decision that login + bind share a per-IP counter (same brute-force
surface) with guidance on switching to per-path if finer isolation
is needed.
Add a floating back-to-bottom control to the chat message list that
appears when the user scrolls up away from the live bottom. The button
auto-docks to the right edge after 15s of inactivity (with a subtle
breathing pulse) and un-docks on mouseenter, keeping it unobtrusive
while reading history.
- End key jumps to the bottom, ignored when focus is in an input,
textarea, or contentEditable field.
- Explicit jump (button click or End) forces past the stick-to-bottom
escape lock and clears it so sticky auto-scroll resumes following new
content; automatic scrolls still respect the escape lock so they do
not fight the user reading history.
- Larger thumb-reach hit area and lower placement on mobile.
- New i18n key chat.scrollToBottom (zh-CN / en-US).
Add an async export feature on the Dashboard page -- global admins can
generate and download a multi-sheet operational data report (.xlsx
packaged as .zip). The export covers 9 sheets:
1. Overview - interval KPIs, system snapshot, 7-day trend, period comparison,
model details (configured providers only), agent activity ranking top 10
2. Token Usage - daily breakdown by runtime_provider with avg tokens/msg
3. Skill Stats - skill list with usage count, last-call time, bound agents
4. User Stats - per-(workspace, user) aggregated tokens, duration, last active
5. User Conversations - detail rows pairing user-asst messages
6. Security and Audit - unified view across 6 sources (guard rules, audit logs,
approvals, grants, config, business audit events)
7. Channel Stats - per-channel conversation count, tokens, unique users
8. Model Config - enabled plus API-key-configured models with parameters
9. Cron Jobs - execution records with duration and token usage
Backend highlights:
- generate/progress/download endpoints guarded by PreAuthorize hasRole ADMIN
- single AtomicBoolean lock (409 when busy), 90-day frontend cap, 5-min deadline
- metadata-based tool-call counting, deleted=0 filtering everywhere
- value label mapping (chat to dialogue, TRUE to enabled, etc.)
- one-time downloadToken, file auto-cleanup after 24h or download
Frontend highlights:
- SVG ring progress bar with smooth dashoffset transition plus slow rotation
- visibility gated by workspaceStore.isGlobalAdmin (v-if on button)
- 1-second polling driving progress state machine (idle/generating/done)
- Element Plus date-picker (30-day default, 90-day max)
- Remove mateclaw-ui/package-lock.json and yarn.lock. This is a pnpm
monorepo where pnpm-lock.yaml is the only lockfile; the npm/yarn locks
were stray duplicates. Add a .gitignore rule so they are not committed
again by mistake.
- SourceEvidenceLedger: reference Pattern/Matcher by their imported
simple names instead of inline fully-qualified names.
Backend (SourceEvidenceLedger):
- appendWikiSourceTable now normalizes existing source lines in-place to
canonical "[N] Title - section - page N" format instead of skipping them
- Added replaceSourceLine helper that matches a full source line by regex
and replaces it with the canonical form
- When source lines exist without a "来源：" header, automatically insert
one so the frontend preprocessor can locate the source table
Frontend (useMarkdownRenderer):
- Added data-citation-index / data-citation-title to DOMPurify whitelist
- Added preprocessWikiCitations preprocessor: parses the canonical source
table to build an index-to-title map, replaces [n] markers in the answer
body with clickable <a> links, and wraps entire source-table rows so the
full line is clickable
- Integrated into the render pipeline after wikilink substitution and
before Marked parsing
Frontend (useGlobalWikilinkClick):
- Extended the click delegation selector to match both .wiki-link and
.wiki-citation elements
- Title extraction falls back: data-citation-title || data-wiki-title
Tests: added three test cases for source-line normalization, idempotency,
and automatic header insertion
Live lifecycle board (grid<->board toggle) plus an assignee-swimlane plan
board that groups follow-up re-runs of one goal into a single xN card.
Custom right-side detail/goal panels with markdown output. Fixes plans
being persisted under the per-run trace id so the board actually populates.
Closes#385
Issue #304. Operators who want an agent with NO knowledge base had no
way to express it: leaving the KB picker empty fell through to "inherit
workspace-wide" (every KB visible), so the agent ended up ingesting
every KB's context. This adds the same opt-out toggle that
skills_disabled (V126) / tools_disabled already provide.
Backend:
- V154 migration (h2 + mysql + kingbase): mate_agent.wiki_disabled
BOOLEAN/TINYINT/SMALLINT NOT NULL DEFAULT FALSE. Legacy agents stay
bit-identical.
- AgentEntity.wikiDisabled: Boolean field, @TableField("wiki_disabled").
- AgentBindingService.getBoundKbIds: short-circuit at the top —
wiki_disabled=true returns Set.of() regardless of binding rows. Mirrors
the precedence contract of getBoundSkillIds vs skills_disabled.
- AgentBindingService.setKbBindings: a non-empty save auto-clears a
stale wiki_disabled flag (same contract as setSkillBindings /
setToolBindings on their respective flags). Empty saves leave the flag
untouched — the UI toggle owns the bit, not the binding writer.
- AgentBindingServiceWikiDisabledTest: 5 cases covering all three
return states + the stale-flag auto-clear + empty-save no-op.
Frontend:
- Agents.vue KB picker: add the "此智能体不使用任何知识库" /
"This agent uses no knowledge bases" toggle, mirroring the skills /
tools picker layout. Tab badge shows "Off" when the toggle is on.
- types/index.ts: add Agent.wikiDisabled?: boolean.
- Save logic: when wikiDisabled is on, send an empty KB list (the
setKbs contract then leaves the flag alone server-side, exactly as
setSkills / setTools behave for their opt-out flags).
- i18n (zh + en): new strings for toggle label, hint, badge, and the
scope description shown when the toggle is on.
Stacked on top of #382 (which introduced AgentBindingResolver
.getBoundKbIds). No agent-runtime changes — wiki tools already degrade
cleanly when getBoundKbIds returns Set.of().
- per-KB entity-type whitelist (config UI + persistence; empty = built-in defaults)
- entity graph: legend grouped by type with click-to-filter; nodes colored by type
- always show entity names on graph nodes (not only on hover)
- earthy categorical palette aligned to the app theme, shared by entity & page graphs
- theme-aware graph label color (resolve CSS var for canvas, light/dark correct)
- manual extract = full rebuild: idempotent force re-extraction + orphan pruning,
guarded against data loss on a fully-failed run
- regression test for force re-extraction; zh/en i18n
When a webchat visitorId + sessionId pair exceeds the conversation_id
column width, WebChatController#deriveConversationId folds the variable
part into a SHA-256 hash prefixed with `#`:
webchat:<key8>:#<sha256[0..40]>
That `#` is the URL fragment delimiter. Every URL the admin console
builds by interpolating the conversationId into a path — message list,
status, rename, pin, model, delete, goals/by-conversation, chat/stop,
chat/pending-approvals — gets truncated at the `#` before reaching the
server. Symptom: opening one of these conversations in the console
surfaces as 405 (GET landing on @DeleteMapping("/{conversationId}"))
and 403 (owner check on the truncated id).
Add an `encId` helper (encodeURIComponent) and apply it to every
conversationId path segment. The server's @PathVariable decoder already
handles the percent-encoded form transparently, so this is purely a
client-side fix that recovers every existing hashed-id row in addition
to any future ones.
Issue: #372
The Web/API (webchat) onboarding wizard marked api_key as required while
it is also readOnly and platform-generated on save. The readOnly field
could never be filled during creation, so canSubmitConfig never passed
and "Continue" stayed disabled.
Exclude readOnly fields from the wizard's required/optional field sets so
they neither gate "Continue" nor render as fillable inputs. The api_key
still appears as required + readOnly in the edit modal once a value
exists.
Refs matevip/mateclaw#338
Turn a single natural-language requirement into a ready-to-review
employee: the model proposes name, persona, runtime type and a
validated set of skills/tools/knowledge base, which the user confirms
or tweaks before the agent is created.
- backend: POST /api/v1/agents/generate builds a draft from the
workspace's real capability catalog; every suggested tool/skill/KB is
re-validated against the catalog so nothing hallucinated is offered
- frontend: 3-step wizard at /agents/create reusing the existing
create + binding endpoints; reusable capability picker shows selected
items as compact chips with an on-demand searchable catalog
Add an opt-in named-entity extraction pass so the wiki knowledge graph
captures fine-grained entities (people, organizations, locations, ...)
and their relations, not just page-level link relations.
- new tables mate_wiki_entity / _mention / _relation (h2/mysql/kingbase)
- structured LLM extraction per chunk with entity resolution
(normalized-key dedup + embedding near-merge), mention/relation
persistence and page linking via chunk citations
- per-KB opt-in toggle (off by default); async dispatch after embedding
- read API: entity list, KB graph, entity ego-graph, manual extract
- UI: entity-layer toggle in the graph view + KB config toggle
- replace inline fully-qualified class names with imports in WikiProcessingService
Closes#336
Always-on memory (structured user/feedback blocks, PROFILE.md, MEMORY.md) is injected into every system prompt but only ever grew, inflating per-turn context over time. This adds deterministic size control across all always-on sources:
- Injection budget: cap the always-on structured block by total chars and per-type entry count, keeping the most-recently-updated entries (LRU by Updated date) and disclosing how many were omitted
- Nightly consolidation: a dedicated scheduled pass merges duplicate/stale user & feedback entries via the LLM, preserving each entry's original Updated date; runs per owner bucket (shared + personal) with a per-run cap and a never-grow safety guard
- File ceilings: deterministic backstop truncates PROFILE.md / MEMORY.md at a section boundary when a rewrite overruns its budget
- Manual trigger endpoint for the consolidation maintenance task
All knobs under mate.memory.*; covered by unit tests.
- goal: continue (not skip) on max-iterations and evidence-insufficient turns.
A max-iterations turn grants a fresh iteration budget ("hard continuation"),
bounded per run and sized into the graph recursion ceiling, so a task too big
for one budget keeps going instead of stalling until the next user message.
- plan-execute: re-plan the remaining work on a step exception, and on a
signature-based stall (repeated failures / identical results / no usable
result) instead of advancing dependent steps with junk; bounded by a per-run
re-plan cap, with a graduated change-strategy nudge before the hard stop.
- plan-execute: auto-derive a goal from a genuine multi-step plan, seeding the
acceptance criteria from the plan steps, so the goal subsystem engages without
the model calling setGoal; broadcast goal_created so the UI hydrates.
- react: refund the iteration for setup-only rounds (load_skill / enable_tool)
so a tight budget is not eaten by the load-then-use two-step.
- ui: re-fetch the active goal when a turn finishes so a goal created or mutated
mid-conversation surfaces without depending on an SSE event.
- streaming: make retry backoff / total-time budget instance fields with a
test-only seam; clarify that the wall-clock budget (not max-retries) bounds a
sustained SERVER_ERROR loop to ~8 attempts, fixing the slow/flaky retry test.
- rescan: keep the KB id as a string end to end so the 19-digit snowflake id
isn't truncated past Number.MAX_SAFE_INTEGER (rescan no longer 404s)
- lint: resolve [[...]] targets against page slugs AND titles like the viewer,
so a title reference to an existing page is no longer reported broken
- ingest: derive the slug deterministically from the title (no inconsistent
romanization), auto-recompute broken links once a KB finishes importing,
and reconcile dangling [[concept]] links — redirect to the covering page via
declared aliases, or demote to plain text when uncovered
- add the page aliases column migration for h2 / mysql / kingbase
- add useStreamingMarkdown: cap mid-stream markdown re-render to ~140ms,
full-fidelity render once the segment completes
- skip code-block language auto-detection while streaming (escaped plain
text), restore full highlighting on the final render
- defer echarts/mermaid blocks to a lightweight loading placeholder while
streaming so their parsers never run on truncated source
- bypass the render cache for streaming-mode output
- wire into ContentSegment and MessageBubble (content + thinking)
Surface the connected database product as a subtle chip in the Dashboard
header. SystemHealthService now reports a database label on /system/health
(reused by the front-end — no extra request), derived from a new
DatabaseBootstrapRunner.getDatabaseLabel() that reads the JDBC product name
once and normalizes it to a canonical label (MySQL / MariaDB / PostgreSQL /
H2, and 人大金仓 for the KingbaseES family), collapsing driver version noise.
The raw-materials surface was renamed to "Sources" when upload, paste,
directory scan and per-KB auto-sync were unified into one tab. The read-only
viewers' reading-toggle segment still carried the old "Raw materials" label,
so managers saw "Sources" while read-only viewers saw "Raw materials" for the
same panel. Point the segment at the same i18n key for a consistent name.
* feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync
The raw-material directory scan and the Advanced "source watcher" sub-tab were
the same engine (same kb.sourceDirectory, same WikiDirectoryScanService) split
across two surfaces with two editable directory inputs. Merge them into one
"Sources" tab (upload / paste / directory manual scan + auto-sync toggle +
the raw-material list) and drop the watcher sub-tab from Advanced.
Auto-sync is now per-KB opt-in: a new watcher_enabled column (V146) gates the
periodic scan per knowledge base. The server-global mate.wiki.watcher-enabled
stays as an ops master switch — a KB is auto-scanned only when both are on
(AND). Manual scans are unaffected. Scan interval stays global for now
(tracked separately).
Closesmatevip/mateclaw#314
* docs(wiki): document source-watcher global switch env vars
Expose MATE_WIKI_WATCHER_ENABLED / MATE_WIKI_WATCHER_INTERVAL_MS as
explicit placeholders in application-mysql.yml, .env.example and
docker-compose.yml, mirroring MATE_WIKI_ALLOWED_SOURCE_ROOTS. Notes the
AND semantics (global ops gate + per-KB toggle) so operators know the
global switch alone is not sufficient.
The KB workspace split into a reading view (pages + graph) and a manage view
(raw materials, config, transformations, advanced, recent-activity snapshot)
gated behind manage:wiki. That moved the raw-materials and recent-activity
surfaces — which read-only viewers (view:wiki without manage:wiki) could
previously browse — entirely behind the management gate, silently dropping
their access.
Re-surface both in the reading-view segmented toggle for viewers who lack a
management view. Managers keep the focused pages/graph toggle and still reach
these surfaces through the management view, so nothing is duplicated for them.
The content panels already render by activeTab, so this only widens the
reading toggle and the activeTab/readingTab types.
- Fail closed to a global fallback sandbox root when a conversation has no
per-workspace base path, instead of leaving file/shell tools unconstrained
- Refuse shell commands that delete the workspace root directory itself
- Block workspace-boundary escapes at the policy layer before the approval
prompt, not only at execution time
- Approval bar now shows the actual command / target path being approved
The pageType profile could only be edited as a raw JSON string. Add a
structured form editor (default) that builds the profile without writing JSON:
profile-level settings (fallbackType / allowAdditionalFields), an ordered page
type list (add via a small wizard, remove, reorder), and a per-type form for
label / description / layer / field schema, with stage prompts (route/create/
merge) and the markdown template folded into an "advanced" section that carries
inline descriptions and examples. A form/JSON toggle keeps the JSON view as the
final review surface; serialization preserves unknown keys for forward-compat,
and save/validate/reset reuse the existing endpoints (no backend change).
Closesmatevip/mateclaw#310
The KB workspace previously stacked all seven surfaces in one tab strip.
Split them by intent: a gear on each library card opens the management view
(raw materials, config, transformations, advanced, hot cache), while clicking
the card body opens the reading view (pages + graph). The reading view drives
page/graph via a header segmented control with the page tree shown only for
the page viewer; the two views share loaded data and toggle without refetch.
Closesmatevip/mateclaw#308
Closes#289 — after an MCP server (re)connects, chat queries kept replying
"from memory" instead of calling MCP tools.
Root cause: agents snapshot their tool set at build time and are cached in
AgentService.agentInstances, but MCP server lifecycle changes never
invalidated that cache (unlike model-config / tool-guard changes which do).
A stale, tool-less agent graph survived until process restart.
Changes:
- Add McpServerChangedEvent; McpServerService publishes it on connect /
disconnect / reconnect / delete / (re)connect-failure / batch refresh /
startup init. AgentService listens and calls refreshAllAgents(), so the
next turn rebuilds against the live MCP tool set. Also closes the boot
race where the web server accepts requests before the @Order(200) MCP
init runner finishes.
- Make create/update/toggle connect asynchronously on a dedicated pool
("mcp-connect") so a slow/unreachable server can no longer freeze the
admin request; status returns immediately as "connecting".
- UI: render the new "connecting" status (pulsing amber dot), show a
friendly "connecting in background" toast, and poll until the status
settles (window widened to ~40s to outlast the default connect timeout).
- UI: MCP config modal no longer closes on outside/backdrop click — only
the × and Cancel buttons close it, so an accidental click can't discard
unsaved config.
Verified E2E: ckjia-shopping (参考价) MCP server connected at runtime with
no backend restart; the cached 通用助手 agent immediately enabled and called
ckjia_shopping_recommend, returning real product cards.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
