Entity extraction previously constrained entity types but let the
model freely invent any relation between entities, producing noise
that diluted the entities a knowledge base actually cares about.
Adds an optional per-KB relation schema (subjectType/predicate/
objectType triples): when set, the extraction prompt is scoped to
only those relations, and a hard filter drops anything that slips
through before it is persisted. Empty/unset keeps the existing
open-vocabulary behaviour.
Give raw-material uploads a dedicated five-minute timeout and process file-picker and drag/drop uploads through a shared two-worker queue, so constrained uplinks no longer abort multipart requests at the global 30-second deadline.
Switch wiki processing jobs and page citations to application-assigned IDs: the PostgreSQL/Kingbase migrations define plain BIGINT primary keys without identity defaults, so database-generated keys fail on insert.
Use const for a never-reassigned local, drop an unused v-for index,
add a default branch to an exhaustive switch the linter cannot prove,
and remove eslint-disable directives that no longer match enabled rules.
The lint script referenced eslint with --ext flags but the repo never had
an ESLint config file, so pnpm lint always failed. Add a flat config
(typescript-eslint recommended + vue essential) with legacy-code rules
downgraded to warnings, drop the flat-config-incompatible --ext flags,
and move pnpm build approvals from the no-longer-read
pnpm.onlyBuiltDependencies field to pnpm-workspace.yaml allowBuilds.
Give raw-material uploads a dedicated five-minute timeout and process file-picker and drag/drop uploads through a shared two-worker queue, so constrained uplinks no longer abort multipart requests at the global 30-second deadline.
Switch wiki processing jobs and page citations to application-assigned IDs: the PostgreSQL/Kingbase migrations define plain BIGINT primary keys without identity defaults, so database-generated keys fail on insert.
Propagates structured error codes through the KB processing pipeline, surfaces silent sub-step warnings as a non-failure warning state, and adds a cross-KB failure center for aggregated visibility.
Two compounding causes made the management view jump from the config
tab back to 'raw' a few seconds after the user selected it:
1. The tab-snap watcher used a single getter returning a new array
(`() => [currentKB?.id, workspaceMode]`). Vue compares the returned
value with Object.is, so a fresh array reference reports a change on
every re-evaluation — including background refreshCurrentKB() calls
that reassign the KB object with the same id. That re-ran the snap and
forced activeTab back to 'raw'. Switch to an array of getters so each
source is compared individually and the callback fires only on a real
id/mode change.
2. RawMaterialPanel's onBeforeUnmount cleared the SSE stream and the 60s
fallback timer but not the per-raw jobPoller setTimeout chain. While a
raw was still processing, leaving the sources tab left that 3s poller
running, calling refreshCurrentKB() indefinitely. Clear jobPoller on
unmount as well.
The config tab pane (.tab-content--config) was set to overflow:hidden,
mirroring the graph pane, but its inner .wiki-config has no bounded height
so its own overflow-y:auto never triggers. Tall config content (model
strategy / processing rules / search-preview cards) overflowed off-screen
with no scrollbar.
Switch the pane to overflow-y:auto like the generic .tab-content. The
existing <=980px media query (overflow:visible) keeps mobile page-scroll
intact. Pure CSS, no logic change.
- per-KB entity-type whitelist (config UI + persistence; empty = built-in defaults)
- entity graph: legend grouped by type with click-to-filter; nodes colored by type
- always show entity names on graph nodes (not only on hover)
- earthy categorical palette aligned to the app theme, shared by entity & page graphs
- theme-aware graph label color (resolve CSS var for canvas, light/dark correct)
- manual extract = full rebuild: idempotent force re-extraction + orphan pruning,
guarded against data loss on a fully-failed run
- regression test for force re-extraction; zh/en i18n
Add an opt-in named-entity extraction pass so the wiki knowledge graph
captures fine-grained entities (people, organizations, locations, ...)
and their relations, not just page-level link relations.
- new tables mate_wiki_entity / _mention / _relation (h2/mysql/kingbase)
- structured LLM extraction per chunk with entity resolution
(normalized-key dedup + embedding near-merge), mention/relation
persistence and page linking via chunk citations
- per-KB opt-in toggle (off by default); async dispatch after embedding
- read API: entity list, KB graph, entity ego-graph, manual extract
- UI: entity-layer toggle in the graph view + KB config toggle
- replace inline fully-qualified class names with imports in WikiProcessingService
Closes#336
- rescan: keep the KB id as a string end to end so the 19-digit snowflake id
isn't truncated past Number.MAX_SAFE_INTEGER (rescan no longer 404s)
- lint: resolve [[...]] targets against page slugs AND titles like the viewer,
so a title reference to an existing page is no longer reported broken
- ingest: derive the slug deterministically from the title (no inconsistent
romanization), auto-recompute broken links once a KB finishes importing,
and reconcile dangling [[concept]] links — redirect to the covering page via
declared aliases, or demote to plain text when uncovered
- add the page aliases column migration for h2 / mysql / kingbase
The raw-materials surface was renamed to "Sources" when upload, paste,
directory scan and per-KB auto-sync were unified into one tab. The read-only
viewers' reading-toggle segment still carried the old "Raw materials" label,
so managers saw "Sources" while read-only viewers saw "Raw materials" for the
same panel. Point the segment at the same i18n key for a consistent name.
* feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync
The raw-material directory scan and the Advanced "source watcher" sub-tab were
the same engine (same kb.sourceDirectory, same WikiDirectoryScanService) split
across two surfaces with two editable directory inputs. Merge them into one
"Sources" tab (upload / paste / directory manual scan + auto-sync toggle +
the raw-material list) and drop the watcher sub-tab from Advanced.
Auto-sync is now per-KB opt-in: a new watcher_enabled column (V146) gates the
periodic scan per knowledge base. The server-global mate.wiki.watcher-enabled
stays as an ops master switch — a KB is auto-scanned only when both are on
(AND). Manual scans are unaffected. Scan interval stays global for now
(tracked separately).
Closesmatevip/mateclaw#314
* docs(wiki): document source-watcher global switch env vars
Expose MATE_WIKI_WATCHER_ENABLED / MATE_WIKI_WATCHER_INTERVAL_MS as
explicit placeholders in application-mysql.yml, .env.example and
docker-compose.yml, mirroring MATE_WIKI_ALLOWED_SOURCE_ROOTS. Notes the
AND semantics (global ops gate + per-KB toggle) so operators know the
global switch alone is not sufficient.
The KB workspace split into a reading view (pages + graph) and a manage view
(raw materials, config, transformations, advanced, recent-activity snapshot)
gated behind manage:wiki. That moved the raw-materials and recent-activity
surfaces — which read-only viewers (view:wiki without manage:wiki) could
previously browse — entirely behind the management gate, silently dropping
their access.
Re-surface both in the reading-view segmented toggle for viewers who lack a
management view. Managers keep the focused pages/graph toggle and still reach
these surfaces through the management view, so nothing is duplicated for them.
The content panels already render by activeTab, so this only widens the
reading toggle and the activeTab/readingTab types.
The pageType profile could only be edited as a raw JSON string. Add a
structured form editor (default) that builds the profile without writing JSON:
profile-level settings (fallbackType / allowAdditionalFields), an ordered page
type list (add via a small wizard, remove, reorder), and a per-type form for
label / description / layer / field schema, with stage prompts (route/create/
merge) and the markdown template folded into an "advanced" section that carries
inline descriptions and examples. A form/JSON toggle keeps the JSON view as the
final review surface; serialization preserves unknown keys for forward-compat,
and save/validate/reset reuse the existing endpoints (no backend change).
Closesmatevip/mateclaw#310
The KB workspace previously stacked all seven surfaces in one tab strip.
Split them by intent: a gear on each library card opens the management view
(raw materials, config, transformations, advanced, hot cache), while clicking
the card body opens the reading view (pages + graph). The reading view drives
page/graph via a header segmented control with the page tree shown only for
the page viewer; the two views share loaded data and toggle without refetch.
Closesmatevip/mateclaw#308
Add a backfill path so pages created before a KB's pageType profile changed
can be migrated into newly-added types. A per-page classify-only LLM call
(title + summary in, single page_type out) is normalised through the profile
and written back via a partial update that never touches page content.
Exposed as POST /knowledge-bases/{id}/reclassify (admin) and a "re-classify
existing pages" action in the Wiki advanced panel.
Wiki page classification was only profile-aware in the main ingest pipeline.
Transformation outputs hard-coded "synthesis", agent-created pages were left
untyped, and the frontend hard-coded the built-in ten types for ordering,
colouring and labels — so custom/synthesis types sank to the bottom, rendered
grey and showed raw keys.
Backend:
- Add nullable target_page_type column to mate_wiki_transformation (V142,
mysql + h2) plus the entity field and CRUD normalization (blank = use
profile fallbackType; membership validated at save time, not edit time).
- Route transformation single-run + KB-aggregate page saves through
WikiPageTypeProfileService.normalizePageType so output joins the KB
classification; agent wiki_create_page now lands on the profile fallbackType
instead of an untyped page.
Frontend:
- Load + parse the KB pageType profile into the wiki store (order, labels,
fallbackType) on KB select / refresh.
- New useWikiPageType composable: profile-driven label (3-tier fallback) and
colour (built-in fixed + deterministic hash palette for custom types).
- Sidebar grouping order, graph colouring, node panel, graph filter and the
page header badge now follow the profile; transformation editor gains a
target-type dropdown sourced from the profile when output target is a page.
Refs #292
Allow knowledge base source paths to be configured as a newline-separated
list of absolute paths or glob patterns rather than a single directory.
- Each non-blank, non-# line is treated as one path or glob pattern
- Plain paths (no wildcards) retain the existing recursive-scan behaviour
filtered by SUPPORTED_EXTENSIONS
- Glob patterns (e.g. /data/ocr/**/*.txt) walk from the fixed-prefix base
and apply Java's PathMatcher against each candidate's absolute path
- Patterns whose filename segment explicitly specifies an extension
(*.txt, *.{xlsx,csv}) skip the SUPPORTED_EXTENSIONS secondary filter,
respecting the user's explicit choice (key for OCR pipelines that
produce .txt output and should ignore the original PDF scans)
- Candidates collected across multiple patterns are deduplicated by
resolved absolute path so overlapping patterns don't double-count
- Symlink-escape check uses each pattern's own validated scan root
- parseSourcePatterns / extractBasePath moved into WikiSourcePathValidator
to eliminate a static circular reference between the two services
- Frontend watcher panel: single-line <input> replaced with <textarea>
supporting multiline editing; i18n updated with example patterns
- No DB schema change; fully backward-compatible with existing single-path
configs stored in sourceDirectory
Closes #(pending issue)
Surface directory-scan failures to the user via toast and render ScanResult.errors[]; expose MATE_WIKI_ALLOWED_SOURCE_ROOTS as a Docker env entry with blank-entry filtering in the path validator; set C.UTF-8 locale in the runtime image so non-ASCII file names decode correctly during scans.
Fixes#259
