opensource/mateclaw
2
0
Fork 0
mirror of https://gitee.com/mateos/mateclaw.git synced 2026-09-15 03:55:09 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,005 Commits 2 Branches 16 Tags 35 MiB
4c56df2ed3
Commit Graph

151 Commits

Author SHA1 Message Date
matevip
465a727be2 feat(wiki): record per-run token usage so operators can see what each template burns 2026-05-12 14:35:17 +08:00
matevip
e911af2192 feat(wiki): JSON output mode — structured transformation output for programmatic downstream 2026-05-12 14:11:02 +08:00
matevip
90936dba4f feat(wiki): cross-material transformation aggregator — map-reduce all runs of a template into one KB page 2026-05-12 14:10:54 +08:00
matevip
a5b952050c feat(wiki): reverse-citation extractor — bind synthesis pages to the exact source chunks they cite 2026-05-12 14:10:47 +08:00
matevip
e1759e0e3c feat(wiki): page-input transformations — run a template against an existing page 2026-05-12 14:10:40 +08:00
matevip
bee39cc420 feat(wiki): page-level embedding so synthesis pages enter semantic search 2026-05-12 14:10:23 +08:00
matevip
62c9776b01 feat(wiki): clean conversational preamble and code fences from transformation output 2026-05-12 11:25:50 +08:00
matevip
f04ea58c8d feat(wiki): cancel a running transformation and re-run any past run 2026-05-12 11:25:44 +08:00
matevip
ef6ad1b91a feat(wiki): expose per-template model picker in the transformations modal 2026-05-12 11:25:38 +08:00
matevip
4098346fb7 fix(wiki): strip file extensions when deriving transformation page slugs 2026-05-12 11:25:32 +08:00
matevip
bae9f68261 feat(wiki): user-defined transformation templates with optional auto-save to synthesis pages 2026-05-12 10:49:58 +08:00
matevip
27f614d48a feat(wiki): enrich chunk embedding input with raw title and structural metadata 2026-05-12 08:44:26 +08:00
matevip
f7c1af80a4 fix(wiki): preserve user-owned files when deleting directory-scanned raws (#84) 2026-05-09 17:15:57 +08:00
matevip
91e93e8831 feat(wiki): circuit-breaker for failing embedding provider (#72) 2026-05-08 15:03:19 +08:00
matevip
0357c9891d feat(wiki): user-initiated cancel for in-progress raw material processing (#72) 2026-05-08 15:03:10 +08:00
matevip
96840b0126 chore: bump version to 1.3.0-SNAPSHOT 2026-05-08 15:02:08 +08:00
matevip
b63d83a596 feat(wiki): circuit-breaker for failing embedding provider (#72) 2026-05-07 16:39:39 +08:00
matevip
76f5b3ca94 feat(wiki): user-initiated cancel for in-progress raw material processing (#72) 2026-05-07 16:32:35 +08:00
matevip
eb64556636 fix(wiki): skip per-turn retrieval for continuation messages + drop weak tail hits 2026-05-04 20:54:44 +08:00
matevip
03a82ece93 fix(wiki): retry route LLM call once with strict-JSON correction when first response isn't JSON 2026-05-03 09:12:51 +08:00
matevip
912a6336d3 fix(wiki): also remove the upload file from disk when a raw material is deleted 2026-05-03 09:12:44 +08:00
matevip
b0ca4c06d4 fix(wiki): cascade-delete pages + chunks when a raw material is removed 2026-05-03 09:12:32 +08:00
matevip
41ed929882 fix(wiki): deterministic, health-aware chat-LLM fallback 2026-05-02 23:31:48 +08:00
matevip
29d2d49d25 fix(wiki): widen mime_type to VARCHAR(255) so Office uploads don't blow the column 2026-05-02 23:11:40 +08:00
matevip
c60fc4cf83 feat(wiki): one-hop chat-LLM fallback when the primary provider is wedged 2026-05-02 21:46:22 +08:00
matevip
2ae8758cd1 feat(wiki): admin endpoints for hot cache — get / regenerate / reset 2026-05-02 21:46:02 +08:00
matevip
5e936fd50a feat(wiki): hot cache rebuild engine — LLM updater, scheduler, conversation-end listener 2026-05-02 21:45:55 +08:00
matevip
93807d85ef feat(wiki): inject KB hot cache into agent system prompt via MemoryProvider SPI 2026-05-02 21:45:49 +08:00
matevip
a8f0998c24 feat(wiki): hot cache storage skeleton — table, entity, value object, read service 2026-05-02 21:45:43 +08:00
matevip
44ea8c784b fix(wiki): skip vision pipeline entirely when wiki.ocr.enabled is off 2026-05-02 19:05:05 +08:00
matevip
f910d762a3 feat(wiki): feature-flag toggle UI + defer extracted_text cache when vision unavailable 2026-05-02 19:04:58 +08:00
matevip
bcefe43234 feat(wiki): inline image refs in search results + click-to-zoom lightbox 2026-05-02 19:04:51 +08:00
matevip
51995275bd feat(wiki): caption inline PDF images via the vision-in pipeline 2026-05-02 19:04:45 +08:00
matevip
4dc280a3d1 feat(wiki): wire image uploads to vision-in pipeline 2026-05-02 19:04:39 +08:00
matevip
b26a4ee4d1 feat(wiki): add SHA-256 keyed image caption cache 2026-05-02 19:04:25 +08:00
matevip
6f0ead162e feat(wiki): instrument compile / relation / retrieval with WikiMetrics 2026-05-02 19:04:19 +08:00
matevip
d0706239ea feat(wiki): add mate_wiki_relation cache table for page-to-page edges 2026-05-02 19:04:06 +08:00
matevip
19b82cdc2a feat(wiki): add WikiMetrics gateway for compile/relation/retrieval/vision 2026-05-02 19:03:59 +08:00
matevip
4a15027a98 feat(wiki): download original raw material file 2026-04-26 20:46:52 +08:00
matevip
0d78beb44f fix(wiki): batch-create per-slug retry — recover unparseable JSON, bump to 2 attempts 2026-04-26 20:37:09 +08:00
matevip
30e7e67bb6 feat(wiki): LLM-narrated overview section with debounced regen + Recent Updates list + scaffold self-heal 2026-04-26 19:53:12 +08:00
matevip
e2df16893e fix(wiki): smaller batch-create + resume button for partial generation 2026-04-26 18:42:27 +08:00
matevip
0b55d5a227 feat(agent): Utf8SseEmitter + returnDirect end-to-end chain test 2026-04-26 08:32:44 +08:00
matevip
6474d0e6be feat(wiki): normalized relation boost + reason in search results 2026-04-25 19:02:34 +08:00
matevip
a1e40d6eae feat(wiki): enrich batch — N pages per LLM call 2026-04-25 19:02:34 +08:00
matevip
ca4c447250 feat(wiki): enrich prompt knows what's already linked 2026-04-25 19:02:34 +08:00
matevip
8ef523f953 feat(wiki): archived pages drawer in Wiki UI 2026-04-25 19:02:33 +08:00
matevip
51566a47a4 chore(wiki): externalize compile prompts; doc archive tools + admin endpoints 2026-04-25 19:02:33 +08:00
matevip
58b49f6e20 feat(wiki): structured no-evidence compile + ops admin endpoints 2026-04-25 19:02:33 +08:00
matevip
5206d65be7 fix(wiki): eager 0-pages = partial when chunks indexed; per-KB structured route; archived filter completion 2026-04-25 19:02:33 +08:00
matevip
3f25064ef9 fix(wiki): cancel in-flight LLM work when raw is deleted 2026-04-25 19:02:32 +08:00
matevip
1e62dbad47 feat(wiki): PR-7 archived soft-archive 2026-04-25 19:02:32 +08:00
matevip
850d59c04f feat(wiki): PR-6b structured-output route phase (opt-in) 2026-04-25 19:02:32 +08:00
matevip
c8af19cae2 feat(wiki): PR-2b/2c overview rebuilder + activity log 2026-04-25 19:02:31 +08:00
matevip
f4d4e973df feat(wiki): PR-5b enrichment via replacement plan 2026-04-25 19:02:31 +08:00
matevip
9c59092d9c feat(wiki): PR-6 skeleton — DTOs for structured eager output 2026-04-25 09:56:19 +08:00
matevip
ec0aaf7da6 feat(wiki): PR-5 wikilink alias parsing + relation seed filter 2026-04-25 09:56:19 +08:00
matevip
b41496ed46 feat(wiki): PR-4 on-demand compile + multi-page read tools 2026-04-25 09:56:19 +08:00
matevip
e4818a8ee2 feat(wiki): PR-3 eager pipeline honors per-step model config 2026-04-25 09:56:19 +08:00
matevip
1281153aa8 feat(wiki): PR-2 system pages — overview/log scaffold + locked + filters 2026-04-25 09:56:18 +08:00
matevip
9746271ea5 feat(wiki): PR-1c preprocessor + chunk metadata + search exposure 2026-04-25 09:56:18 +08:00
matevip
80725c9ac7 feat(wiki): PR-1b lazy ingest — chunk+embed, no page generation 2026-04-25 09:56:18 +08:00
matevip
50d9ff2b3d feat(wiki): PR-1a infra — content hash split, chunk metadata columns, kb-default model 2026-04-25 09:56:17 +08:00
matevip
27c4c3e5e2 feat(wiki): config UI overhaul — model strategy, search preview modal, graph fullscreen 2026-04-24 10:03:45 +08:00
matevip
4f67e31887 fix(wiki): eliminate per-chunk duplicate updates and fix token overflow 2026-04-24 06:55:21 +08:00
matevip
d04d90dfc5 feat(wiki): grouped page list, pageCount on raw materials, frosted-glass UI 2026-04-24 06:55:21 +08:00
matevip
c50977785a fix(wiki): internal link navigation and source citation guidance 2026-04-24 06:55:20 +08:00
matevip
4be867a3e6 feat(wiki): ingest optimization — BatchCreate, document analysis, retry 2026-04-24 06:55:20 +08:00
matevip
1073890e64 feat(skill): BuiltinSkillSeedService — close SQL/SKILL.md double-write 2026-04-24 06:55:18 +08:00
matevip
f8c7e5271b fix(embedding): skip unconfigured provider in embedding model resolution 2026-04-22 10:12:57 +08:00
matevip
11fa7487d0 fix(delegate): reliability patches for multi-agent delegation 2026-04-22 10:12:48 +08:00
matevip
d8f008e427 fix(embedding): skip unconfigured provider in embedding model resolution 2026-04-22 05:08:21 +08:00
matevip
0301d5628e fix(wiki): JobStageBar stuck at queued — add job stage transitions 
Root cause: processRawMaterial() created a job record at queued stage
but never called jobService.transition() during processing. The job row
stayed at queued forever, so the stage bar never advanced.

Backend (WikiProcessingService):
- Transition job to ROUTING immediately after creation
- Transition to PHASE_A_RUNNING before chunk processing begins
- Transition to COMPLETED/PARTIAL/FAILED at the end based on finalStatus
- Transition to FAILED in the catch block on unhandled exceptions

Backend (WikiProcessingJobService.transition):
- Handle FAILED, PARTIAL terminal stages (set finishedAt + status)
- Handle non-terminal intermediate stages (set status to running)

Frontend (JobStageBar.vue):
- Add stageMapping for backend stages not shown as dots: phase_a_done →
  phase_b_running, failed/partial/cancelled → completed position
- Guard stageIndex() against -1 (unknown stages default to all-pending)
- Terminal failure states show red failed dot instead of pulsing active
 2026-04-19 20:53:35 +08:00
matevip
a171f2ac0e fix(wiki): recover raw materials stuck in processing on server restart 
Root cause: recoverOnStartup() only reset mate_wiki_processing_job rows,
not mate_wiki_raw_material. claimForProcessing() only accepts pending,
so restart-orphaned processing rows were permanently stuck — frontend
showed "preparing..." forever.

Fix:
- Add WikiRawMaterialService.recoverStuckRawMaterialsOnStartup():
  resets processing→pending, clears progress fields, fires
  WikiProcessingEvent when autoProcessOnUpload is enabled
- WikiAutoConfiguration: call raw recovery after job recovery
- Execution order: job table first (queued), then raw table (pending)

Test: WikiRawMaterialRecoveryTest — 4 cases: reset + events, reset
without events (autoProcess=false), noop on empty.
 2026-04-19 19:37:50 +08:00
matevip
cbcb7229b6 fix(wiki): citation drawer shows real data instead of empty fallbacks 
Root cause: PageCitationWithRaw record only had id/pageId/chunkId/rawId/
paragraphIdx/anchorText/confidence — missing rawTitle, chunkOrdinal,
startOffset, endOffset, snippet. Frontend displayed Source + Chunk ?
for every card.

Backend:
- Extend PageCitationWithRaw with rawTitle, chunkOrdinal, startOffset,
  endOffset, snippet fields
- Expand listWithRawByPageId SQL to LEFT JOIN mate_wiki_raw_material for
  title, JOIN mate_wiki_chunk for ordinal/offsets/content snippet (first
  200 chars via SUBSTRING)

Frontend:
- CitationDrawer: replace hardcoded Source / Chunk ? / offset with
  i18n keys
- Add 4 new i18n keys: citationUnknownSource, citationChunkN,
  citationChunkUnknown, citationOffset (zh-CN + en-US)
 2026-04-19 19:36:57 +08:00
matevip
4700d0312d fix(wiki): deduplicate raw material uploads across all processing statuses 
Root cause: addFile()/addText() hash dedup only matched rows with
status=completed, so the same file uploaded while in partial/pending/
processing/failed status would create a duplicate row.

Fix:
- Remove .eq(processingStatus, "completed") from dedup queries — match
  any non-deleted row with the same content hash in the KB
- On dedup hit: completed/pending/processing → return as-is;
  partial/failed → trigger reprocess (partial enters resume branch)
- Clean up the newly uploaded temp file when dedup discards it
- Frontend: uploadRawFile/addRawText check for existing id in the list
  before unshift to prevent visual duplicates

Test: WikiRawMaterialDedupTest — 10 cases covering all 5 statuses,
reprocess triggers for partial/failed, no-op for others, insert only
when no match.
 2026-04-19 19:36:13 +08:00
matevip
3d213eb281 chore: sync multiple commits from private dev 
Covers 15 upstream commits (private mirror → public):

Multi-provider failover (RFC-009):
- PR-0: extract ChatModelBuilder strategy seam
- PR-1a: AvailableProviderPool data structure
- PR-1b: startup provider liveness probe + 4 protocol strategies
- PR-1c: wire AvailableProviderPool into runtime chat-model selection
- PR-1d: provider pool REST endpoint + UI badges
- PR-1e: manual reprobe trigger + auto-reprobe on provider config change
- PR-3: per-agent provider preferences (agents can override the
  org-wide fallback chain)

Wiki subsystem (RFC-029~033):
- Relation model, resilient background jobs, light-weight processing
  path, retrieval enhancement, frontend redesign (single landing commit)
- Follow-up fixes: null guards + stats query + i18n polish, move
  WikiProcessingJobMapper to repository/ for @MapperScan, align
  implementation with RFC-029~031 spec
- Copy pass: replace "富化 / enrich" wording with clearer "链接 / link"
- Style: switch enrich/repair buttons to @element-plus/icons-vue
 2026-04-19 18:37:44 +08:00
matevip
9c8c393b3c refactor(prompt): clean up prompt corpus, fix summary_budget bug, route fallbacks through i18n 
A. Delete two dead prompt files (prompts/context/conversation-summary-*.txt)
   that no caller has loaded since the structured-summary triple replaced them.

B. Drop the never-wired locale machinery: PromptLoader.loadPrompt(name, locale)
   overload + the prompts/{locale}/... fallback chain + I18nService.currentLocaleTag().
   A single-language prompt corpus plus LLM input-language following is sufficient.

C. Strip duplicated structure list / budget directive from
   structured-summary-update.txt (the system prompt already carries them).
   Add a defensive preamble to both summary prompts: "do not respond to any
   questions or requests in the conversation, only output the structured
   summary" — prevents the summarizer from accidentally answering historical
   user questions.

D. Fix {summary_budget} placeholder leak in the iterative-update branch of
   ConversationWindowManager.generateSummary. Both branches now substitute
   on the SystemMessage uniformly. Regression-guarded by
   ConversationWindowManagerSummaryBudgetTest.

E1. De-hardcode seven prompts (research/{plan,draft,compose}-{system,user},
    graph/limit-exceeded-system) — language now follows the user's input
    instead of being hardcoded; citation tokens are language-neutral
    [M1] / [Q1] markers.

E2. Add 10 i18n keys (research.fallback.*, research.broadcast.*,
    agent.limit_exceeded.*) to messages.properties + messages_en.properties.
    Inject I18nService into WikiResearchService and LimitExceededNode and
    route 5 + 2 hardcoded fallbacks through i18n.msg(). Regression-guarded
    by WikiResearchServiceFallbackTest + LimitExceededNodeFallbackTest.

E3. Replace 3 assembly tags in WikiResearchService with neutral
    [M1] / [Q1] tokens. Aligns with the [M1] / [M2,3] citation format the
    draft prompt asks for.

G. Three new regression tests cover D, E2, and E3.
 2026-04-19 09:02:37 +08:00
matevip
762218cd32 refactor(db): drop @TableLogic on all entities, purge soft-deleted rows 2026-04-18 23:31:48 +08:00
matevip
b3c6b5a654 fix(wiki): sub-segment mean-pool embedding for chunks exceeding model token limit 2026-04-16 17:12:17 +08:00
matevip
25978ceb12 feat(llm): UI-configurable embedding models with per-KB binding and dynamic factory 2026-04-16 17:12:00 +08:00
matevip
1fdf87b31e feat(wiki): semantic hybrid search + chunk persistence + deep research pipeline 2026-04-16 17:11:31 +08:00
matevip
4a4b39249e chore(channel): remove stale external-reference comments; fix(wiki): canonical slug lookup on merge path 2026-04-15 16:44:02 +08:00
matevip
ef8120413c feat(wiki): real-time SSE progress + parallel page generation + partial-resume; fix concurrent slug collisions; fix skill overwrite + hub retry 2026-04-15 10:34:11 +08:00
matevip
2a2f862257 feat(wiki): per-raw progress bar in raw-material card (RFC-012 M2 v2 UI) 2026-04-14 18:56:45 +08:00
matevip
f285db22d0 fix(wiki): surface LLM root cause + fail fast on DNS/TLS/refused (RFC-012 M1 follow-up v2) 2026-04-14 18:24:49 +08:00
matevip
62ac1fb79c fix(wiki): route emits metadata only; per-page create + merge in phase B (RFC-012 M2 v2) 2026-04-14 18:11:35 +08:00
matevip
4fb6a83eb6 feat(wiki): two-phase digest — route + per-page merge (RFC-012 M2) 2026-04-14 17:39:50 +08:00
matevip
a369e8055d fix(wiki): bound LLM retry + http read timeout (RFC-012 M1 follow-up) 2026-04-14 16:05:37 +08:00
matevip
b9ed8219ba feat(wiki): parallel processing + resilient LLM retry + hash-based skip (RFC-012 M1) 2026-04-14 15:18:03 +08:00
matevip
d5e3502829 fix(wiki): backend hardening + batch delete UI 2026-04-13 15:37:12 +08:00
matevip
e4679796b8 feat(wiki): performance + quality + delete redesign (RFC-008) 2026-04-13 09:35:26 +08:00
matevip
408ad980dd feat(i18n): complete remaining i18n — DefaultToolGuard, LLM/Datasource services, Channels UI 2026-04-11 18:02:03 +08:00
matevip
cc28f665f1 feat(db): introduce Flyway migration framework and unify H2/MySQL schemas 
- Add Flyway baseline migration (V1) for both H2 and MySQL
- Remove 5 legacy SchemaMigration ApplicationRunner classes
- Add WorkspacePathGuard for file tool sandbox enforcement
- Inject workspace basePath context into agent graph state and tool executor
- Add workspace basePath config UI in frontend
- Fix workspace slug preservation on update
 2026-04-11 16:34:09 +08:00
matevip
5124842526 feat(platform): workspace isolation, info architecture, and UI redesign 2026-04-09 16:29:02 +08:00
matevip
d44ba0cdd8 fix(security): close workspace isolation gaps and audit context loss 2026-04-09 14:19:51 +08:00
matevip
3506c55bc8 feat(platform): Phase 3 Sprint 1-3 — permission, agent binding, dashboard 2026-04-09 11:58:20 +08:00
matevip
3d58a48eae feat(platform): add workspace foundation and channel execution upgrades 2026-04-09 10:29:16 +08:00
matevip
c568b4faa8 fix(db): unify schema and seed data across H2/MySQL, en/zh 2026-04-08 16:30:33 +08:00
matevip
f6102c4733 fix(wiki): auto-resolve kbId, improve search and processing robustness 2026-04-08 16:09:22 +08:00