Add an opt-in named-entity extraction pass so the wiki knowledge graph
captures fine-grained entities (people, organizations, locations, ...)
and their relations, not just page-level link relations.
- new tables mate_wiki_entity / _mention / _relation (h2/mysql/kingbase)
- structured LLM extraction per chunk with entity resolution
(normalized-key dedup + embedding near-merge), mention/relation
persistence and page linking via chunk citations
- per-KB opt-in toggle (off by default); async dispatch after embedding
- read API: entity list, KB graph, entity ego-graph, manual extract
- UI: entity-layer toggle in the graph view + KB config toggle
- replace inline fully-qualified class names with imports in WikiProcessingService
Closes#336
Always-on memory (structured user/feedback blocks, PROFILE.md, MEMORY.md) is injected into every system prompt but only ever grew, inflating per-turn context over time. This adds deterministic size control across all always-on sources:
- Injection budget: cap the always-on structured block by total chars and per-type entry count, keeping the most-recently-updated entries (LRU by Updated date) and disclosing how many were omitted
- Nightly consolidation: a dedicated scheduled pass merges duplicate/stale user & feedback entries via the LLM, preserving each entry's original Updated date; runs per owner bucket (shared + personal) with a per-run cap and a never-grow safety guard
- File ceilings: deterministic backstop truncates PROFILE.md / MEMORY.md at a section boundary when a rewrite overruns its budget
- Manual trigger endpoint for the consolidation maintenance task
All knobs under mate.memory.*; covered by unit tests.
- rescan: keep the KB id as a string end to end so the 19-digit snowflake id
isn't truncated past Number.MAX_SAFE_INTEGER (rescan no longer 404s)
- lint: resolve [[...]] targets against page slugs AND titles like the viewer,
so a title reference to an existing page is no longer reported broken
- ingest: derive the slug deterministically from the title (no inconsistent
romanization), auto-recompute broken links once a KB finishes importing,
and reconcile dangling [[concept]] links — redirect to the covering page via
declared aliases, or demote to plain text when uncovered
- add the page aliases column migration for h2 / mysql / kingbase
The blanket SMALLINT->BOOLEAN flag-column conversion over-reached: six wiki
columns map to Integer (1/0) entity fields, not Boolean. On vanilla PostgreSQL,
reading a BOOLEAN into a JDBC int throws 'Bad value for type int : f', breaking
every wiki KB list / SSE chat. Revert only those six back to SMALLINT (V133/V134/
V135/V136/V146 in the PostgreSQL-family tree) with guard comments; genuine
Boolean-entity columns stay BOOLEAN.
The KingbaseES JDBC driver is not on Maven Central; declaring it as a
required runtime dependency broke the default build for anyone without
the proprietary jar. Move it into an opt-in `kingbase` Maven profile
(build with `mvn package -Pkingbase`). No Java code imports the driver
classes — it is loaded at runtime via driver-class-name only, so the
default build no longer needs it.
Also drop `mateclaw.browser.ssrf-check-enabled: false` from the default
application.yml: the code default is true, and disabling the SSRF guard
globally is unrelated to KingbaseES support.
Add KingbaseES support as an opt-in profile: dedicated migration tree, bilingual seed data, runtime DbType detection (KINGBASE_ES / POSTGRE_SQL), and JDBC URL handling in the datasource manager.
* feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync
The raw-material directory scan and the Advanced "source watcher" sub-tab were
the same engine (same kb.sourceDirectory, same WikiDirectoryScanService) split
across two surfaces with two editable directory inputs. Merge them into one
"Sources" tab (upload / paste / directory manual scan + auto-sync toggle +
the raw-material list) and drop the watcher sub-tab from Advanced.
Auto-sync is now per-KB opt-in: a new watcher_enabled column (V146) gates the
periodic scan per knowledge base. The server-global mate.wiki.watcher-enabled
stays as an ops master switch — a KB is auto-scanned only when both are on
(AND). Manual scans are unaffected. Scan interval stays global for now
(tracked separately).
Closesmatevip/mateclaw#314
* docs(wiki): document source-watcher global switch env vars
Expose MATE_WIKI_WATCHER_ENABLED / MATE_WIKI_WATCHER_INTERVAL_MS as
explicit placeholders in application-mysql.yml, .env.example and
docker-compose.yml, mirroring MATE_WIKI_ALLOWED_SOURCE_ROOTS. Notes the
AND semantics (global ops gate + per-KB toggle) so operators know the
global switch alone is not sufficient.
- Fail closed to a global fallback sandbox root when a conversation has no
per-workspace base path, instead of leaving file/shell tools unconstrained
- Refuse shell commands that delete the workspace root directory itself
- Block workspace-boundary escapes at the policy layer before the approval
prompt, not only at execution time
- Approval bar now shows the actual command / target path being approved
The V85 seed shipped a dev/test placeholder (sse + http://localhost:8085/sse
+ "Bearer ${CKJIA_MCP_KEY}") that can never connect, so the 参考价 price-
comparison skill was unusable until an admin hand-edited the row.
Add V144 (h2 + mysql) to rewrite the seed to the real CKJIA SaaS endpoint:
streamable_http + https://m.ckjia.com/api/ai/mcp, no auth header, connect/read
timeouts raised to 60s. Conditional on the row still carrying the dev
placeholder URL, so an admin who already pointed it at a private deployment is
left untouched; idempotent and leaves `enabled` opt-in.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an execute_code built-in tool that runs python/bash/node code the agent
writes on the fly, so a documentation-only skill (a SKILL.md with no bundled
scripts) can be acted on. Scoped runs inject the skill's secrets and run in
the skill directory; otherwise a private scratch directory is used. Host
secret env vars are scrubbed from the subprocess. execute_code is an
agent-wide capability, registered in the tool catalog (V143), and screened
by the tool guard with a dedicated set of destructive-pattern rules.
Tests cover python/bash/node execution, scratch-dir fallback, env scrubbing,
argument decoding, and guard gating.
Add a backfill path so pages created before a KB's pageType profile changed
can be migrated into newly-added types. A per-page classify-only LLM call
(title + summary in, single page_type out) is normalised through the profile
and written back via a partial update that never touches page content.
Exposed as POST /knowledge-bases/{id}/reclassify (admin) and a "re-classify
existing pages" action in the Wiki advanced panel.
Wiki page classification was only profile-aware in the main ingest pipeline.
Transformation outputs hard-coded "synthesis", agent-created pages were left
untyped, and the frontend hard-coded the built-in ten types for ordering,
colouring and labels — so custom/synthesis types sank to the bottom, rendered
grey and showed raw keys.
Backend:
- Add nullable target_page_type column to mate_wiki_transformation (V142,
mysql + h2) plus the entity field and CRUD normalization (blank = use
profile fallbackType; membership validated at save time, not edit time).
- Route transformation single-run + KB-aggregate page saves through
WikiPageTypeProfileService.normalizePageType so output joins the KB
classification; agent wiki_create_page now lands on the profile fallbackType
instead of an untyped page.
Frontend:
- Load + parse the KB pageType profile into the wiki store (order, labels,
fallbackType) on KB select / refresh.
- New useWikiPageType composable: profile-driven label (3-tier fallback) and
colour (built-in fixed + deterministic hash palette for custom types).
- Sidebar grouping order, graph colouring, node panel, graph filter and the
page header badge now follow the profile; transformation editor gains a
target-type dropdown sourced from the profile when output target is a page.
Refs #292
Add 5-arg buildContextMessage overload that emits [system-context] Model:
for every origin (web/cron/IM). Legacy 3/4-arg overloads delegate to the
new one with null model args, keeping their output byte-identical.
Also fix pre-existing FeishuMentionTest compile error caused by removed
mentionMatchesAnyAlias/collectMentionIdentifiers methods.
Gate MarkdownNormalizer behind mate.agent.markdown-normalize-enabled (default
true) so operators can disable the rewrite verbatim if a normalization edge
case ever mangles a legitimate answer.
Surface directory-scan failures to the user via toast and render ScanResult.errors[]; expose MATE_WIKI_ALLOWED_SOURCE_ROOTS as a Docker env entry with blank-entry filtering in the path validator; set C.UTF-8 locale in the runtime image so non-ASCII file names decode correctly during scans.
Fixes#259
Streaming chat ran render tools on an async thread with no bound request,
so download links lost their host and arrived without a domain. Resolve the
host on the request thread and carry it through ChatOrigin/ToolContext;
falls back to a configurable public-base-url, then a relative path.
Add the user-facing and API-reference docs that explain how an employee
can declare a primary knowledge base, what the runtime fallback chain
looks like, and how the binding is driven from the API.
agents.md (zh + en)
- New section "Knowledge base binding (per-agent primary KB)" right after
the tool-binding section, with a 1.5.0 New In badge
- Lays out the design intent explicitly: KBs stay workspace-shared, the
binding only chooses a default target, multiple agents may pick the
same KB as primary
- Documents the runtime resolution order used by wiki tools: explicit
kbName/kbId, agent.primaryKbId, most-recently-updated workspace KB
- Migration note about the legacy kb.agent_id to agent.primary_kb_id
backfill and the visibility change for anyone who relied on the old
one-to-one isolation
api.md (zh + en)
- Under Agents: document the primaryKbId field with the three-state PUT
semantics (omit / null / value) and example curl calls
- Under LLM Wiki: document GET /api/v1/wiki/knowledge-bases/bindable and
the explicit warning that PUT /api/v1/wiki/knowledge-bases/{id} no
longer processes the agentId field
PR #237 introduced V129__agent_primary_kb.sql while dev also has
V129__wiki_page_broken_links.sql shipped from the broken-link lint work.
Flyway rejects duplicate version numbers at startup, so the agent
primary_kb migration moves to V130 across both H2 and MySQL dialects.
No content change beyond the rename — the column add, index, and
backfill SQL are identical to the V129 originals from #237.
Agents now have a per-agent primary wiki KB stored on
mate_agent.primary_kb_id. KBs remain workspace-shared — selecting one in
the agent editor only chooses the default wiki target for that agent, it
does not change the KB's ownership or visibility.
Backend
- AgentEntity: add primary_kb_id field (FieldStrategy.ALWAYS so the UI
can clear it back to "no primary")
- AgentController#update: switch body to Map<String, Object> so we can
tell "field missing" apart from "explicit null" via containsKey, then
convertValue back to AgentEntity
- WikiKnowledgeBaseService:
- new resolvePrimaryKb(agentId): prefers agent.primary_kb_id when it
points to a workspace-visible KB; falls back to legacy
kb.agent_id marker, then to most-recently-updated workspace KB
- listByAgentId now returns the full workspace set (KBs are
workspace-shared under the new model)
- update(id, name, description) no longer touches agent_id
- WikiController: new GET /knowledge-bases/bindable for the UI picker;
PUT /knowledge-bases/{id} no longer reads agentId
- WikiKnowledgeBaseEntity: add FieldStrategy.ALWAYS on embeddingModelId
and configContent so explicit nulls actually unbind/clear instead of
being silently skipped by MyBatis-Plus's NOT_NULL default
- Migrations V129 (H2 + MySQL): add primary_kb_id column + index, backfill
from legacy kb.agent_id, MySQL uses INFORMATION_SCHEMA guard +
PREPARE/EXECUTE for idempotency
- WikiKnowledgeBaseServiceTest: 13 cases, all passing
Frontend
- Agents.vue: new "Knowledge Base" tab, radio-select bindable KBs
- API: listBindableKBs() + Agent.primaryKbId typed string | number | null
- IDs handled as strings throughout (Snowflake-safe)
- i18n keys for the new tab in zh-CN and en-US
* feat(agent): optional agent-level workspace basePath override
Add workspaceBasePath field to AgentEntity that optionally overrides
the workspace-level basePath. When set, the agent uses its own directory;
when null, it inherits the workspace's basePath (existing behavior).
- AgentEntity: new workspaceBasePath field with ALWAYS update strategy
- AgentGraphBuilder: agent-level override takes priority over workspace
- Flyway migration V121 for H2 and MySQL
- UI: form input in basic tab with i18n (zh-CN, en-US)
* fix(agent): rename migration V121→V125 to avoid Flyway conflict with upstream
Upstream already has V121__tool_disclosure_tier.sql. Rename our
migration to V125 (next available after V124).
* fix(agent): make MySQL V125 migration idempotent
Use INFORMATION_SCHEMA check before ADD COLUMN to avoid
"Duplicate column name" error on re-deploy.
* feat(tool): add send_file tool for sending existing server files as IM attachments
Adds a new built-in tool that reads a file from the server and stashes it
in GeneratedFileCache so the channel adapter (Feishu, DingTalk, etc.)
automatically sends it as a native attachment. This fills the gap where
agents had no way to send existing server files to users — ReadFileTool
only reads text, and render tools only generate new files.
- New SendFileTool with path validation, MIME detection, 20MB limit
- Added "send_file" to tool allowlist in AgentBindingService
- Added i18n error messages (zh-CN + en-US)
* fix(tool): send_file returns URL in scrubber-detectable format
The previous JSON return format caused the LLM to reply with just
"status: sent" without echoing the /api/v1/files/generated/{id} URL.
GeneratedFileScrubber only scans the LLM's final text output, so the
file was never delivered as a native attachment.
Changed to match GeneratedFileLink's format: returns a markdown link
with explicit instructions for the LLM to echo the URL verbatim.
The Lark SDK throws HandlerNotFoundException for any event type without
a registered handler. This exception is caught internally by the SDK's
WebSocket client, which then sends a 500 response to the Feishu server.
The server may close the connection as a result, and the exception is
swallowed — never reaching the application layer.
Added empty handlers for all remaining IM event types:
- P2MessageReadV1 (read receipts)
- P2MessageRecalledV1 (message recall)
- P2ChatMemberBotDeletedV1 (bot removed from chat)
- P2ChatMemberUserAddedV1 / UserDeletedV1 / UserWithdrawnV1
- P2ChatUpdatedV1 (chat info update)
- P2ChatDisbandedV1 (chat disbanded)
- P2ChatAccessEventBotP2pChatEnteredV1 (bot entered p2p chat)
Also added explicit logback config for com.lark.oapi at WARN level
to ensure SDK internal errors are not silently filtered.
Refs: larksuite/oapi-sdk-java#185
