Add an opt-in named-entity extraction pass so the wiki knowledge graph
captures fine-grained entities (people, organizations, locations, ...)
and their relations, not just page-level link relations.
- new tables mate_wiki_entity / _mention / _relation (h2/mysql/kingbase)
- structured LLM extraction per chunk with entity resolution
(normalized-key dedup + embedding near-merge), mention/relation
persistence and page linking via chunk citations
- per-KB opt-in toggle (off by default); async dispatch after embedding
- read API: entity list, KB graph, entity ego-graph, manual extract
- UI: entity-layer toggle in the graph view + KB config toggle
- replace inline fully-qualified class names with imports in WikiProcessingService
Closes#336
- rescan: keep the KB id as a string end to end so the 19-digit snowflake id
isn't truncated past Number.MAX_SAFE_INTEGER (rescan no longer 404s)
- lint: resolve [[...]] targets against page slugs AND titles like the viewer,
so a title reference to an existing page is no longer reported broken
- ingest: derive the slug deterministically from the title (no inconsistent
romanization), auto-recompute broken links once a KB finishes importing,
and reconcile dangling [[concept]] links — redirect to the covering page via
declared aliases, or demote to plain text when uncovered
- add the page aliases column migration for h2 / mysql / kingbase
* feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync
The raw-material directory scan and the Advanced "source watcher" sub-tab were
the same engine (same kb.sourceDirectory, same WikiDirectoryScanService) split
across two surfaces with two editable directory inputs. Merge them into one
"Sources" tab (upload / paste / directory manual scan + auto-sync toggle +
the raw-material list) and drop the watcher sub-tab from Advanced.
Auto-sync is now per-KB opt-in: a new watcher_enabled column (V146) gates the
periodic scan per knowledge base. The server-global mate.wiki.watcher-enabled
stays as an ops master switch — a KB is auto-scanned only when both are on
(AND). Manual scans are unaffected. Scan interval stays global for now
(tracked separately).
Closesmatevip/mateclaw#314
* docs(wiki): document source-watcher global switch env vars
Expose MATE_WIKI_WATCHER_ENABLED / MATE_WIKI_WATCHER_INTERVAL_MS as
explicit placeholders in application-mysql.yml, .env.example and
docker-compose.yml, mirroring MATE_WIKI_ALLOWED_SOURCE_ROOTS. Notes the
AND semantics (global ops gate + per-KB toggle) so operators know the
global switch alone is not sufficient.
The base directory is canonicalized via toRealPath before walking, so the
walked files carry the symlink-resolved prefix. The PathMatcher was built
from the literal pattern, so a symlinked base never matched and files were
silently dropped. Rebuild the glob against the resolved scan root, escaping
glob metacharacters in the base so a real directory name containing */?/{}/[]
is treated literally.
Agents now have a per-agent primary wiki KB stored on
mate_agent.primary_kb_id. KBs remain workspace-shared — selecting one in
the agent editor only chooses the default wiki target for that agent, it
does not change the KB's ownership or visibility.
Backend
- AgentEntity: add primary_kb_id field (FieldStrategy.ALWAYS so the UI
can clear it back to "no primary")
- AgentController#update: switch body to Map<String, Object> so we can
tell "field missing" apart from "explicit null" via containsKey, then
convertValue back to AgentEntity
- WikiKnowledgeBaseService:
- new resolvePrimaryKb(agentId): prefers agent.primary_kb_id when it
points to a workspace-visible KB; falls back to legacy
kb.agent_id marker, then to most-recently-updated workspace KB
- listByAgentId now returns the full workspace set (KBs are
workspace-shared under the new model)
- update(id, name, description) no longer touches agent_id
- WikiController: new GET /knowledge-bases/bindable for the UI picker;
PUT /knowledge-bases/{id} no longer reads agentId
- WikiKnowledgeBaseEntity: add FieldStrategy.ALWAYS on embeddingModelId
and configContent so explicit nulls actually unbind/clear instead of
being silently skipped by MyBatis-Plus's NOT_NULL default
- Migrations V129 (H2 + MySQL): add primary_kb_id column + index, backfill
from legacy kb.agent_id, MySQL uses INFORMATION_SCHEMA guard +
PREPARE/EXECUTE for idempotency
- WikiKnowledgeBaseServiceTest: 13 cases, all passing
Frontend
- Agents.vue: new "Knowledge Base" tab, radio-select bindable KBs
- API: listBindableKBs() + Agent.primaryKbId typed string | number | null
- IDs handled as strings throughout (Snowflake-safe)
- i18n keys for the new tab in zh-CN and en-US
The PRIVATE_ITEMS list contained the bare 'test' entry, which rsync
interprets as 'any directory named test at any depth' — so it caught
the root-level /test/ scratch directory (intended) AND every src/test/
under each module (not intended).
Pattern is already anchored to /test (root-only). This commit rsyncs
the accumulated src/test/ tree forward so opensource has the unit tests
that have been written / updated against existing src/main/ code since
the pattern regression. Going forward each per-commit sync will carry
src/test/ files along with the main change.
Root cause: addFile()/addText() hash dedup only matched rows with
status=completed, so the same file uploaded while in partial/pending/
processing/failed status would create a duplicate row.
Fix:
- Remove .eq(processingStatus, "completed") from dedup queries — match
any non-deleted row with the same content hash in the KB
- On dedup hit: completed/pending/processing → return as-is;
partial/failed → trigger reprocess (partial enters resume branch)
- Clean up the newly uploaded temp file when dedup discards it
- Frontend: uploadRawFile/addRawText check for existing id in the list
before unshift to prevent visual duplicates
Test: WikiRawMaterialDedupTest — 10 cases covering all 5 statuses,
reprocess triggers for partial/failed, no-op for others, insert only
when no match.
A. Delete two dead prompt files (prompts/context/conversation-summary-*.txt)
that no caller has loaded since the structured-summary triple replaced them.
B. Drop the never-wired locale machinery: PromptLoader.loadPrompt(name, locale)
overload + the prompts/{locale}/... fallback chain + I18nService.currentLocaleTag().
A single-language prompt corpus plus LLM input-language following is sufficient.
C. Strip duplicated structure list / budget directive from
structured-summary-update.txt (the system prompt already carries them).
Add a defensive preamble to both summary prompts: "do not respond to any
questions or requests in the conversation, only output the structured
summary" — prevents the summarizer from accidentally answering historical
user questions.
D. Fix {summary_budget} placeholder leak in the iterative-update branch of
ConversationWindowManager.generateSummary. Both branches now substitute
on the SystemMessage uniformly. Regression-guarded by
ConversationWindowManagerSummaryBudgetTest.
E1. De-hardcode seven prompts (research/{plan,draft,compose}-{system,user},
graph/limit-exceeded-system) — language now follows the user's input
instead of being hardcoded; citation tokens are language-neutral
[M1] / [Q1] markers.
E2. Add 10 i18n keys (research.fallback.*, research.broadcast.*,
agent.limit_exceeded.*) to messages.properties + messages_en.properties.
Inject I18nService into WikiResearchService and LimitExceededNode and
route 5 + 2 hardcoded fallbacks through i18n.msg(). Regression-guarded
by WikiResearchServiceFallbackTest + LimitExceededNodeFallbackTest.
E3. Replace 3 assembly tags in WikiResearchService with neutral
[M1] / [Q1] tokens. Aligns with the [M1] / [M2,3] citation format the
draft prompt asks for.
G. Three new regression tests cover D, E2, and E3.
