- explicit SecurityConfig authorization for /swagger-ui*, /v3/api-docs*, /webjars/**
- public for local/default profile; admin-only (ROLE_ADMIN) by default in production DB profiles
- override via MATECLAW_OPENAPI_EXPOSE_UI; add RANDOM_PORT integration tests and docs
Live lifecycle board (grid<->board toggle) plus an assignee-swimlane plan
board that groups follow-up re-runs of one goal into a single xN card.
Custom right-side detail/goal panels with markdown output. Fixes plans
being persisted under the per-run trace id so the board actually populates.
Closes#385
The cherry-picked V154 used 'ALTER TABLE ... ADD COLUMN IF NOT EXISTS' for
MySQL — invalid on MySQL 8.0.x (a MariaDB-only extension) which aborts Flyway
at startup. Switch to the INFORMATION_SCHEMA + PREPARE guard the other MySQL
migrations use. Also change the KingbaseES column from SMALLINT to BOOLEAN to
match the Java 'Boolean wikiDisabled' field and the existing skills_disabled /
tools_disabled flags (vanilla PostgreSQL is strict about boolean vs smallint).
H2 (already BOOLEAN) is unchanged.
Issue #304. Operators who want an agent with NO knowledge base had no
way to express it: leaving the KB picker empty fell through to "inherit
workspace-wide" (every KB visible), so the agent ended up ingesting
every KB's context. This adds the same opt-out toggle that
skills_disabled (V126) / tools_disabled already provide.
Backend:
- V154 migration (h2 + mysql + kingbase): mate_agent.wiki_disabled
BOOLEAN/TINYINT/SMALLINT NOT NULL DEFAULT FALSE. Legacy agents stay
bit-identical.
- AgentEntity.wikiDisabled: Boolean field, @TableField("wiki_disabled").
- AgentBindingService.getBoundKbIds: short-circuit at the top —
wiki_disabled=true returns Set.of() regardless of binding rows. Mirrors
the precedence contract of getBoundSkillIds vs skills_disabled.
- AgentBindingService.setKbBindings: a non-empty save auto-clears a
stale wiki_disabled flag (same contract as setSkillBindings /
setToolBindings on their respective flags). Empty saves leave the flag
untouched — the UI toggle owns the bit, not the binding writer.
- AgentBindingServiceWikiDisabledTest: 5 cases covering all three
return states + the stale-flag auto-clear + empty-save no-op.
Frontend:
- Agents.vue KB picker: add the "此智能体不使用任何知识库" /
"This agent uses no knowledge bases" toggle, mirroring the skills /
tools picker layout. Tab badge shows "Off" when the toggle is on.
- types/index.ts: add Agent.wikiDisabled?: boolean.
- Save logic: when wikiDisabled is on, send an empty KB list (the
setKbs contract then leaves the flag alone server-side, exactly as
setSkills / setTools behave for their opt-out flags).
- i18n (zh + en): new strings for toggle label, hint, badge, and the
scope description shown when the toggle is on.
Stacked on top of #382 (which introduced AgentBindingResolver
.getBoundKbIds). No agent-runtime changes — wiki tools already degrade
cleanly when getBoundKbIds returns Set.of().
Add /wiki/pages row to endpoint table and a new "Wiki knowledge-base
reference ([[slug]] picker)" section explaining the directive-text
mechanism, query parameters, visibility rules (synthesis excluded,
100-page cap, KB-scope fallback), and curl examples (zh + en).
Follow-up docs for the wiki picker endpoint shipped in this PR.
Add /skills row to endpoint list table, note optional agentId on /stream,
and add a new "Skill invocation (slash picker)" section explaining the
directive-text mechanism with curl examples (zh + en).
Follow-up polish for the /skills endpoint shipped via PR #374.
The webchat session-id (was V147) and archive/revocation (was V148) migrations
collided with the wiki migrations already occupying V147 (wiki_page_aliases)
and V148 (wiki_entity); wiki migrations run up to V150. Two files sharing a
version makes Flyway abort at startup with "Found more than one migration with
version N", so the app failed to boot on a fresh database. Renumber the webchat
migrations to the next free slots (V151, V152) across all three dialects. Table
names are unchanged, so entities and idempotent column guards are unaffected.
Previously WebChatController.chatStream silently dropped every agent
lifecycle event except _usage_final (and content_delta / thinking_delta
derived from delta.payload). Visitors sat with nothing between the
meta event and the first content chunk — typically 3–10s when the
agent plans / recalls memory / runs tools, longer when the agent
chained multiple tool calls. The JWT chat path (ChatController) had
this wiring; webchat did not.
Curated 4-event subset (per design review):
- phase — high-level phase transition (planning / generating /
summarizing / ...). SDK shows a "AI is thinking..."
typing indicator before the first token.
- tool_start — agent invoked a tool. SDK shows a localized badge
("Searching...", "Reading file.pdf", ...).
- tool_end — tool completed. SDK clears the badge.
- plan — Plan-Execute agents expose their step list. SDK can
render a checklist.
Deliberately NOT forwarded (internal noise / leak risk):
- _usage_final, _routing_decision — consumed internally
- finish_reason — implicit in `done`
- feedback_event — visitor can't retry/regenerate anyway
- perf_summary, iteration_* — internal metrics
- plan_step_started/completed — too granular; the plan event covers
the visitor's needs
Critical safety constraint: tool_start / tool_end carry ONLY the tool
name. Tool arguments and results are dropped — agent tool calls can
contain PII (file paths, user queries, credentials), and relaying
those to a 3rd-party website frontend is a data leak. The SDK maps
tool name → localized label via its own lookup.
Backward compat: existing clients ignore unknown event types per the
SSE spec, so adding these is non-breaking.
Tests: 5 new cases in WebChatStreamE2ETest covering each event type
+ a regression case asserting internal events are silently dropped.
85/85 webchat tests green.
Docs: docs/zh/webchat.md gains a "实时进度事件" subsection.
Stack: feat/webchat-attachment-e2e → feat/webchat-stream-phase-events
Follow-up to epic #355.
docs/zh/webchat.md — single source of truth for downstream integrators.
Covers everything needed to embed MateClaw webchat into a third-party
site without reading source:
- Base URL, auth model (API Key + visitorToken), R<T> response wrap
- Endpoint table (14 visitor-facing + 2 admin)
- Auth flow diagram (how visitorToken gets minted on /stream, reused
on management endpoints)
- Error code table (400/401/404/409 with semantics)
- SSE event protocol (meta / content_delta / thinking_delta / done /
error)
- File upload + download flow (visitor-attached vs agent-generated;
/api/v1/files/generated/<uuid> is permitAll + 7d TTL)
- visitorToken revocation admin endpoint
- Three end-to-end curl examples (first message / list sessions /
upload-then-send)
- Limits (5 empty-session quota, upload caps, 7d expirations,
single-instance constraint today)
@Operation / @Parameter / @ApiResponse / @ExampleObject polish on
WebChatController is intentionally deferred — it's noisy mechanical
work that deserves its own focused PR rather than getting rushed in
here. The doc is the canonical reference now; the swagger annotations
can quote it.
Part of epic #355.
Schema foundations for the rest of the epic. Three additions:
1. mate_conversation.archived — INT default 0. Lets a visitor "soft-close"
a thread: stays on disk (history preserved, addressable, downloadable)
but excluded from default /sessions listing. Pinned/archived are
orthogonal; archive dominates (archived+pinned still hidden by default).
2. webchat_revoked_visitor — registry table consumed in PR 2 by the
WebChatTokenRevocationService. Unique on (channel_id, visitor_id, deleted)
so re-revoke is idempotent and deleted=1 un-revokes. Migration written
for all three DBs (h2 IF NOT EXISTS, MySQL INFORMATION_SCHEMA guard,
KingbaseES native IF NOT EXISTS) following the V147 pattern.
3. WebChatSessionView gains pinned/archived/streamStatus so the visitor-
side listing surfaces the same state the admin console sees. Closes
the "field exposure" gap from the epic.
loadVisitorSessions gains an includeArchived flag (default false) — the
default hides archived threads and excludes them from the empty-session
quota, since the visitor already declared they're done with them.
GET /sessions and GET /sessions/page thread an `includeArchived=true`
query param through.
Tests (WebChatSchemaFieldsTest, @SpringBootTest + H2 + V148):
- revoked-visitor table is queryable
- archived column is read/write
- view exposes pinned/archived/streamStatus
- archived hidden by default, visible with includeArchived=true
- archived empty threads don't saturate the 5-thread quota
Regression: WebChatCreateSessionTest (9/9), WebChatSessionManagementTest
(5/5) — both updated for the new includeArchived param.
Part of epic #355.
When webchat:<key8>:<visitorId>:<sessionId> exceeds 64 chars the
conversationId folds visitorId+sessionId into an unrecoverable hash, so
the thread fell outside listSessions' conversationId-prefix filter and its
sessionId could not be recovered — the thread was invisible and
unaddressable (common with a UUID visitorId + a >10-char sessionId).
Persist the sessionId on creation (new nullable webchat_session_id column)
and enumerate by username + channel prefix (webchat:<key8>:), which also
matches the hashed form. sessionId is read from the column, falling back to
parsing the conversationId only for legacy rows.
Adds a @SpringBootTest covering listing (incl. the hashed thread), message
pagination, session paging/search, rename, and token rejection end-to-end.
Refs matevip/mateclaw#346
Add an opt-in named-entity extraction pass so the wiki knowledge graph
captures fine-grained entities (people, organizations, locations, ...)
and their relations, not just page-level link relations.
- new tables mate_wiki_entity / _mention / _relation (h2/mysql/kingbase)
- structured LLM extraction per chunk with entity resolution
(normalized-key dedup + embedding near-merge), mention/relation
persistence and page linking via chunk citations
- per-KB opt-in toggle (off by default); async dispatch after embedding
- read API: entity list, KB graph, entity ego-graph, manual extract
- UI: entity-layer toggle in the graph view + KB config toggle
- replace inline fully-qualified class names with imports in WikiProcessingService
Closes#336
Always-on memory (structured user/feedback blocks, PROFILE.md, MEMORY.md) is injected into every system prompt but only ever grew, inflating per-turn context over time. This adds deterministic size control across all always-on sources:
- Injection budget: cap the always-on structured block by total chars and per-type entry count, keeping the most-recently-updated entries (LRU by Updated date) and disclosing how many were omitted
- Nightly consolidation: a dedicated scheduled pass merges duplicate/stale user & feedback entries via the LLM, preserving each entry's original Updated date; runs per owner bucket (shared + personal) with a per-run cap and a never-grow safety guard
- File ceilings: deterministic backstop truncates PROFILE.md / MEMORY.md at a section boundary when a rewrite overruns its budget
- Manual trigger endpoint for the consolidation maintenance task
All knobs under mate.memory.*; covered by unit tests.
- rescan: keep the KB id as a string end to end so the 19-digit snowflake id
isn't truncated past Number.MAX_SAFE_INTEGER (rescan no longer 404s)
- lint: resolve [[...]] targets against page slugs AND titles like the viewer,
so a title reference to an existing page is no longer reported broken
- ingest: derive the slug deterministically from the title (no inconsistent
romanization), auto-recompute broken links once a KB finishes importing,
and reconcile dangling [[concept]] links — redirect to the covering page via
declared aliases, or demote to plain text when uncovered
- add the page aliases column migration for h2 / mysql / kingbase
The blanket SMALLINT->BOOLEAN flag-column conversion over-reached: six wiki
columns map to Integer (1/0) entity fields, not Boolean. On vanilla PostgreSQL,
reading a BOOLEAN into a JDBC int throws 'Bad value for type int : f', breaking
every wiki KB list / SSE chat. Revert only those six back to SMALLINT (V133/V134/
V135/V136/V146 in the PostgreSQL-family tree) with guard comments; genuine
Boolean-entity columns stay BOOLEAN.
The KingbaseES JDBC driver is not on Maven Central; declaring it as a
required runtime dependency broke the default build for anyone without
the proprietary jar. Move it into an opt-in `kingbase` Maven profile
(build with `mvn package -Pkingbase`). No Java code imports the driver
classes — it is loaded at runtime via driver-class-name only, so the
default build no longer needs it.
Also drop `mateclaw.browser.ssrf-check-enabled: false` from the default
application.yml: the code default is true, and disabling the SSRF guard
globally is unrelated to KingbaseES support.
Add KingbaseES support as an opt-in profile: dedicated migration tree, bilingual seed data, runtime DbType detection (KINGBASE_ES / POSTGRE_SQL), and JDBC URL handling in the datasource manager.
* feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync
The raw-material directory scan and the Advanced "source watcher" sub-tab were
the same engine (same kb.sourceDirectory, same WikiDirectoryScanService) split
across two surfaces with two editable directory inputs. Merge them into one
"Sources" tab (upload / paste / directory manual scan + auto-sync toggle +
the raw-material list) and drop the watcher sub-tab from Advanced.
Auto-sync is now per-KB opt-in: a new watcher_enabled column (V146) gates the
periodic scan per knowledge base. The server-global mate.wiki.watcher-enabled
stays as an ops master switch — a KB is auto-scanned only when both are on
(AND). Manual scans are unaffected. Scan interval stays global for now
(tracked separately).
Closesmatevip/mateclaw#314
* docs(wiki): document source-watcher global switch env vars
Expose MATE_WIKI_WATCHER_ENABLED / MATE_WIKI_WATCHER_INTERVAL_MS as
explicit placeholders in application-mysql.yml, .env.example and
docker-compose.yml, mirroring MATE_WIKI_ALLOWED_SOURCE_ROOTS. Notes the
AND semantics (global ops gate + per-KB toggle) so operators know the
global switch alone is not sufficient.
- Fail closed to a global fallback sandbox root when a conversation has no
per-workspace base path, instead of leaving file/shell tools unconstrained
- Refuse shell commands that delete the workspace root directory itself
- Block workspace-boundary escapes at the policy layer before the approval
prompt, not only at execution time
- Approval bar now shows the actual command / target path being approved
The V85 seed shipped a dev/test placeholder (sse + http://localhost:8085/sse
+ "Bearer ${CKJIA_MCP_KEY}") that can never connect, so the 参考价 price-
comparison skill was unusable until an admin hand-edited the row.
Add V144 (h2 + mysql) to rewrite the seed to the real CKJIA SaaS endpoint:
streamable_http + https://m.ckjia.com/api/ai/mcp, no auth header, connect/read
timeouts raised to 60s. Conditional on the row still carrying the dev
placeholder URL, so an admin who already pointed it at a private deployment is
left untouched; idempotent and leaves `enabled` opt-in.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an execute_code built-in tool that runs python/bash/node code the agent
writes on the fly, so a documentation-only skill (a SKILL.md with no bundled
scripts) can be acted on. Scoped runs inject the skill's secrets and run in
the skill directory; otherwise a private scratch directory is used. Host
secret env vars are scrubbed from the subprocess. execute_code is an
agent-wide capability, registered in the tool catalog (V143), and screened
by the tool guard with a dedicated set of destructive-pattern rules.
Tests cover python/bash/node execution, scratch-dir fallback, env scrubbing,
argument decoding, and guard gating.
Add a backfill path so pages created before a KB's pageType profile changed
can be migrated into newly-added types. A per-page classify-only LLM call
(title + summary in, single page_type out) is normalised through the profile
and written back via a partial update that never touches page content.
Exposed as POST /knowledge-bases/{id}/reclassify (admin) and a "re-classify
existing pages" action in the Wiki advanced panel.
Wiki page classification was only profile-aware in the main ingest pipeline.
Transformation outputs hard-coded "synthesis", agent-created pages were left
untyped, and the frontend hard-coded the built-in ten types for ordering,
colouring and labels — so custom/synthesis types sank to the bottom, rendered
grey and showed raw keys.
Backend:
- Add nullable target_page_type column to mate_wiki_transformation (V142,
mysql + h2) plus the entity field and CRUD normalization (blank = use
profile fallbackType; membership validated at save time, not edit time).
- Route transformation single-run + KB-aggregate page saves through
WikiPageTypeProfileService.normalizePageType so output joins the KB
classification; agent wiki_create_page now lands on the profile fallbackType
instead of an untyped page.
Frontend:
- Load + parse the KB pageType profile into the wiki store (order, labels,
fallbackType) on KB select / refresh.
- New useWikiPageType composable: profile-driven label (3-tier fallback) and
colour (built-in fixed + deterministic hash palette for custom types).
- Sidebar grouping order, graph colouring, node panel, graph filter and the
page header badge now follow the profile; transformation editor gains a
target-type dropdown sourced from the profile when output target is a page.
Refs #292
Add 5-arg buildContextMessage overload that emits [system-context] Model:
for every origin (web/cron/IM). Legacy 3/4-arg overloads delegate to the
new one with null model args, keeping their output byte-identical.
Also fix pre-existing FeishuMentionTest compile error caused by removed
mentionMatchesAnyAlias/collectMentionIdentifiers methods.
Gate MarkdownNormalizer behind mate.agent.markdown-normalize-enabled (default
true) so operators can disable the rewrite verbatim if a normalization edge
case ever mangles a legitimate answer.
Surface directory-scan failures to the user via toast and render ScanResult.errors[]; expose MATE_WIKI_ALLOWED_SOURCE_ROOTS as a Docker env entry with blank-entry filtering in the path validator; set C.UTF-8 locale in the runtime image so non-ASCII file names decode correctly during scans.
Fixes#259
Streaming chat ran render tools on an async thread with no bound request,
so download links lost their host and arrived without a domain. Resolve the
host on the request thread and carry it through ChatOrigin/ToolContext;
falls back to a configurable public-base-url, then a relative path.
Add the user-facing and API-reference docs that explain how an employee
can declare a primary knowledge base, what the runtime fallback chain
looks like, and how the binding is driven from the API.
agents.md (zh + en)
- New section "Knowledge base binding (per-agent primary KB)" right after
the tool-binding section, with a 1.5.0 New In badge
- Lays out the design intent explicitly: KBs stay workspace-shared, the
binding only chooses a default target, multiple agents may pick the
same KB as primary
- Documents the runtime resolution order used by wiki tools: explicit
kbName/kbId, agent.primaryKbId, most-recently-updated workspace KB
- Migration note about the legacy kb.agent_id to agent.primary_kb_id
backfill and the visibility change for anyone who relied on the old
one-to-one isolation
api.md (zh + en)
- Under Agents: document the primaryKbId field with the three-state PUT
semantics (omit / null / value) and example curl calls
- Under LLM Wiki: document GET /api/v1/wiki/knowledge-bases/bindable and
the explicit warning that PUT /api/v1/wiki/knowledge-bases/{id} no
longer processes the agentId field
PR #237 introduced V129__agent_primary_kb.sql while dev also has
V129__wiki_page_broken_links.sql shipped from the broken-link lint work.
Flyway rejects duplicate version numbers at startup, so the agent
primary_kb migration moves to V130 across both H2 and MySQL dialects.
No content change beyond the rename — the column add, index, and
backfill SQL are identical to the V129 originals from #237.
Agents now have a per-agent primary wiki KB stored on
mate_agent.primary_kb_id. KBs remain workspace-shared — selecting one in
the agent editor only chooses the default wiki target for that agent, it
does not change the KB's ownership or visibility.
Backend
- AgentEntity: add primary_kb_id field (FieldStrategy.ALWAYS so the UI
can clear it back to "no primary")
- AgentController#update: switch body to Map<String, Object> so we can
tell "field missing" apart from "explicit null" via containsKey, then
convertValue back to AgentEntity
- WikiKnowledgeBaseService:
- new resolvePrimaryKb(agentId): prefers agent.primary_kb_id when it
points to a workspace-visible KB; falls back to legacy
kb.agent_id marker, then to most-recently-updated workspace KB
- listByAgentId now returns the full workspace set (KBs are
workspace-shared under the new model)
- update(id, name, description) no longer touches agent_id
- WikiController: new GET /knowledge-bases/bindable for the UI picker;
PUT /knowledge-bases/{id} no longer reads agentId
- WikiKnowledgeBaseEntity: add FieldStrategy.ALWAYS on embeddingModelId
and configContent so explicit nulls actually unbind/clear instead of
being silently skipped by MyBatis-Plus's NOT_NULL default
- Migrations V129 (H2 + MySQL): add primary_kb_id column + index, backfill
from legacy kb.agent_id, MySQL uses INFORMATION_SCHEMA guard +
PREPARE/EXECUTE for idempotency
- WikiKnowledgeBaseServiceTest: 13 cases, all passing
Frontend
- Agents.vue: new "Knowledge Base" tab, radio-select bindable KBs
- API: listBindableKBs() + Agent.primaryKbId typed string | number | null
- IDs handled as strings throughout (Snowflake-safe)
- i18n keys for the new tab in zh-CN and en-US
* feat(agent): optional agent-level workspace basePath override
Add workspaceBasePath field to AgentEntity that optionally overrides
the workspace-level basePath. When set, the agent uses its own directory;
when null, it inherits the workspace's basePath (existing behavior).
- AgentEntity: new workspaceBasePath field with ALWAYS update strategy
- AgentGraphBuilder: agent-level override takes priority over workspace
- Flyway migration V121 for H2 and MySQL
- UI: form input in basic tab with i18n (zh-CN, en-US)
* fix(agent): rename migration V121→V125 to avoid Flyway conflict with upstream
Upstream already has V121__tool_disclosure_tier.sql. Rename our
migration to V125 (next available after V124).
* fix(agent): make MySQL V125 migration idempotent
Use INFORMATION_SCHEMA check before ADD COLUMN to avoid
"Duplicate column name" error on re-deploy.
* feat(tool): add send_file tool for sending existing server files as IM attachments
Adds a new built-in tool that reads a file from the server and stashes it
in GeneratedFileCache so the channel adapter (Feishu, DingTalk, etc.)
automatically sends it as a native attachment. This fills the gap where
agents had no way to send existing server files to users — ReadFileTool
only reads text, and render tools only generate new files.
- New SendFileTool with path validation, MIME detection, 20MB limit
- Added "send_file" to tool allowlist in AgentBindingService
- Added i18n error messages (zh-CN + en-US)
* fix(tool): send_file returns URL in scrubber-detectable format
The previous JSON return format caused the LLM to reply with just
"status: sent" without echoing the /api/v1/files/generated/{id} URL.
GeneratedFileScrubber only scans the LLM's final text output, so the
file was never delivered as a native attachment.
Changed to match GeneratedFileLink's format: returns a markdown link
with explicit instructions for the LLM to echo the URL verbatim.
The Lark SDK throws HandlerNotFoundException for any event type without
a registered handler. This exception is caught internally by the SDK's
WebSocket client, which then sends a 500 response to the Feishu server.
The server may close the connection as a result, and the exception is
swallowed — never reaching the application layer.
Added empty handlers for all remaining IM event types:
- P2MessageReadV1 (read receipts)
- P2MessageRecalledV1 (message recall)
- P2ChatMemberBotDeletedV1 (bot removed from chat)
- P2ChatMemberUserAddedV1 / UserDeletedV1 / UserWithdrawnV1
- P2ChatUpdatedV1 (chat info update)
- P2ChatDisbandedV1 (chat disbanded)
- P2ChatAccessEventBotP2pChatEnteredV1 (bot entered p2p chat)
Also added explicit logback config for com.lark.oapi at WARN level
to ensure SDK internal errors are not silently filtered.
Refs: larksuite/oapi-sdk-java#185
