Commit Graph

42 Commits

Author SHA1 Message Date
matevip
32dd6e7911 fix(channel): sync-path IM channels stop relaying pre-tool rehearsals verbatim 2026-08-04 22:50:42 -04:00
mateaix
7fb6ba7461 sync: v2.0.0 dev + 落盘文件按天分目录 (#554)
- 附件 / 生成媒体写入 {conversationDir}/yyyy-MM-dd/{storedName},serve URL 保持平铺
- 读取路径同时探测平铺与日期两种布局,可随时开关 mateclaw.chat.upload.date-folders
- 读取侧强制 storedName 为纯文件名;原始文件名回退取最近修改的匹配
- 修正 Linux / macOS / Windows 的路径与时间戳精度差异
2026-08-01 18:33:24 +08:00
matevip
aaae3bf122 fix(channel): claim each inbound message once so redeliveries stop answering twice 2026-07-29 05:43:01 -04:00
matevip
e294b32542 feat(channel): scope conversation ids by channel to stop cross-workspace bleed 2026-07-23 18:50:10 +08:00
matevip
e2747df2c5 feat(channel): /model magic command for per-conversation model switching (restore) 2026-07-23 11:49:28 +08:00
matevip
1bb81234d0 feat(approval): classify auto-approve misses and persist the outcome on guard audit rows 2026-07-23 11:44:21 +08:00
mateaix
a4ee953d31 feat(channel): /model magic command for per-conversation model switching 2026-07-22 22:27:34 +08:00
matevip
a183519d69 feat(channel): extract the per-turn stream accumulator and share it between web SSE and IM sync paths 2026-07-22 18:09:59 +08:00
matevip
4ac72a90a0 feat(skill): bundle file management — view and edit scripts, references, and templates from the console
The skill detail drawer could only show and edit SKILL.md; the bundle
files under scripts/ and references/ had no console surface, and
templates/ was readable by agents but absent from the canonical
store's bucket set.

- admin endpoints on /api/v1/skills/{id}/files: list (self-heals an
  empty canonical store from on-disk files), read, upsert, delete.
  Writes update the canonical row, materialize the workspace cache,
  and re-resolve the skill so agents pick changes up immediately.
  Path envelope enforces the three buckets and blocks traversal;
  builtin skill files stay read-only; virtual skills own no files.
- templates/ becomes a first-class DB-persisted bucket shared across
  the syncer, the workspace write/delete envelope, and prune guards.
- the agent-facing write_file action now mirrors into the canonical
  store and re-resolves instead of writing only the local filesystem.
- SkillMarket detail drawer gains a Files tab: grouped list, viewer,
  inline editor, create and delete, refetched on every entry.
2026-07-22 15:39:59 +08:00
matevip
8a55bdd367 feat(channel): relay per-stage narration as standalone messages on the sync IM path 2026-07-22 14:08:55 +08:00
matevip
beb1a8c243 feat(channel): extensible magic commands (/new /help /status /stop) 2026-07-21 10:28:41 +08:00
jack
1e2b7bbc2c
fix(weixin): send generated files through weixin channel (#543)
Send agent-generated files as native WeChat attachments via the iLink upload flow, and fix the wire protocol for file uploads: dedicated wire ObjectMapper (bypasses the global Long-to-String serializer), md5/len fields and encrypt_type on media items, channel_version 1.0.2, and explicit business-error handling on ret != 0. The weixin adapter now routes generated-file URLs through GeneratedFileScrubber, matching WeCom/Feishu behavior.

Fixes #307
2026-07-20 20:34:59 +08:00
matevip
d10ed9dd06 feat(channel): add clear magic command 2026-07-20 18:21:46 +08:00
matevip
e6c35ecfc7 fix(channel): sanitize conversationId as a filesystem path segment — fixes wecom/dingtalk/feishu attachment upload on Windows (#507) 2026-07-10 16:54:05 +08:00
matevip
25737495e5 feat(chat): per-turn token usage breakdown with cache hit/miss/write and reasoning split (#474) 2026-07-03 11:28:00 +08:00
倪程伟
fcb488c567
feat(workspace): chat-uploads 上传目录工作空间/Agent 感知化 (#422)
* feat(workspace): chat-uploads 上传目录工作空间/Agent 感知化 (#421)

把硬编码的 data/chat-uploads/{conversationId}/ 改为按工作空间/Agent 解析,
解析优先级:Agent workspaceBasePath → Workspace basePath → 可配置默认目录
(新配置 mateclaw.chat.upload.base-dir,默认 data/chat-uploads,保持现网零行为变化)。

- 新增 ChatUploadLocationResolver 中央解析器:写路径返回唯一根,读/清理
  路径返回候选根列表(工作空间根 + 默认根)做双重查找,保证迁移前旧附件
  仍可解析/清理;conversationId→ConversationEntity 查询带 5min 缓存。
- 新增 ChatUploadProperties + ChatUploadAutoConfiguration(启动建目录)。
- 复用 AgentGraphBuilder.resolveAgentBasePath(提升为 public)的优先级与
  安全规则(相对路径在 workspace 根下解析,绝对路径逃逸被拒)。
- 所有写入/读取点改为走 resolver;读取点走双重查找。
- 解决 Spring 循环依赖(resolver → agentService → ... → conversationService
  → resolver):resolver 的 AgentService 注入加 @Lazy。

向后兼容:默认目录不变;双重查找覆盖历史消息里的相对路径;
服务端点 URL 契约不变,前端无需改动。

测试:新增 ChatUploadLocationResolverTest(8 用例);修复受影响的现有测试构造。

* refactor(workspace): address review findings on chat-uploads resolver (#421)

应用 code review 的 4 项修复:

1. (correctness) ChatUploadLocationResolver 缓存新增 ConversationDeletedEvent
   监听器,删除会话时立即失效 conversationId→ConversationEntity 映射。
   否则备份恢复后用相同 id 重建会话,会继承最长 5 分钟的过期 workspace/agent
   映射,导致 cleanAttachmentFiles 走错(过期的)上传目录。复用既有
   @EventListener-on-bean 模式(与 AsyncTaskService / WorkspaceLookupCache 一致)。

2. 收紧 resolveWorkspaceScopedRoot 里 3 个过宽的 catch(Exception) →
   MateClawException + warn,让真正的 bug(NPE / DataAccessException)暴露
   而非被静默降级为 debug 日志。

3. 更新 ChatController.upload 过期注释:会话尚未创建时附件暂存默认目录,
   会话创建后读取走双重查找仍能命中。

4. 移除不可达分支(agentWorkspaceId != workspaceId)—— 会话的 agent 必然
   归属会话的 workspace(创建时强约束),直接用会话 workspace 即可,
   少一次冗余 DB 查询与一层推测性逻辑。

测试:ChatUploadLocationResolverTest (8) + ConversationServiceCleanAttachmentFilesTest (2) 全绿。
2026-06-26 14:20:54 +08:00
倪程伟
a40868eff2 refactor(feishu): 群会话 ID 改用完整 chatId 避免后缀碰撞
旧实现群会话 conversationId = feishu:{appId后4}_{chatId后8},截断后缀
存在碰撞风险:不同群 chatId 后 8 位相同 → 消息落进同一会话、上下文串台。
群会话改用完整 chatId(feishu:{chatId})消除碰撞。

存量迁移(读时别名回退,不重写存量行):
- 旧后缀不可逆推完整 chatId,故不做一次性回填;
- 但每条入站群消息都带完整 chatId + appId,可在路由前重算 legacy key;
- 新群 / 已迁移群 → 用 feishu:{chatId};存量群(canonical 无、legacy 有)
  → 沿用 legacy key,历史无缝延续,零停机、零破坏性写。

- ChannelMessageRouter 增只读 conversationExists(id)(委托 findByConversationId)。
- 私聊不受影响(DM 经 buildConversationId 直接用完整 senderOpenId,后缀本就不参与)。

Closes #299
2026-06-09 11:10:09 +08:00
matevip
48611a6f4d feat(memory): per-owner memory isolation with owner_key + visibility scope (#235) 2026-06-02 17:04:06 +08:00
matevip
d2b23c049c fix(wiki): let agents reach every visible KB via kbName + wiki_list_kbs (#224) 2026-05-26 13:53:32 +08:00
matevip
3ae4498f38 fix(channel,agent,chat): unify channel binding / conversation agent / model pin state sources 2026-05-26 09:40:49 +08:00
matevip
a01f0354eb feat(conversation): introduce ChatResult to carry token usage through sync chat paths 2026-05-25 21:15:53 +08:00
matevip
8d01396130 fix(conversation): capture runtime model/provider with token usage in IM and webchat paths 2026-05-25 20:31:09 +08:00
倪程伟
727373f67c
fix(conversation): capture token usage in IM channel and webchat paths (#217)
IM channels (Feishu, DingTalk, WeCom, etc.) and the WebChat widget were
calling saveMessage without token usage parameters, causing promptTokens
and completionTokens to default to 0. This made the Token Statistics
module report significantly lower numbers than actual usage.

Root cause: the _usage_final event (containing promptTokens /
completionTokens) emitted by the agent graph at stream end was not being
captured in these paths, unlike ChatController's StreamAccumulator which
already handles it correctly.

Fix: capture _usage_final events in doOnNext handlers for:
- ChannelMessageRouter sync path (non-streaming IM adapters)
- ChannelMessageRouter streaming path (DingTalk, etc.)
- WebChatController SSE stream

Refs #214 (remaining String-API paths covered by follow-up).
2026-05-25 20:25:22 +08:00
matevip
a9c2d45790 Harden goal approval and workspace flows 2026-05-23 22:55:16 +08:00
matevip
6a4318c268 fix(channel): IM conversations respect per-conversation model selection (#183) 2026-05-20 17:49:03 +08:00
matevip
71e08b015e sync: Feishu approval card 5-chain hotfix — verified end-to-end in production 2026-05-20 15:24:47 +08:00
matevip
3554da8dbc sync: inject sender context into agent prompt + Feishu DONE ack hook 2026-05-20 11:51:58 +08:00
matevip
48510c9751 feat(channel): per-sender attribution + debounce boundary for group chats 2026-05-10 19:15:51 +08:00
matevip
2e0a4674e5 feat(channel): IM channel media + paste-merge + native file delivery
Inbound (WeCom):
- Save uploaded media under data/chat-uploads/{conversationId}/ with full
  fileName/path/fileUrl/storedName/fileSize on the content part. Web mirrors
  of an IM conversation now show real thumbnails instead of "未命名".
- Magic-byte sniff (PDF / PNG / JPEG / GIF / Office / ODF / archives /
  audio / video) recovers a real extension when the platform omits filename
  for forwarded files — no more PDFs labelled "file.bin".
- ZIP container peek distinguishes DOCX / XLSX / PPTX / VSDX / ODT / ODS /
  ODP / EPUB / JAR from a plain zip via discriminator paths and the OASIS
  mimetype entry.

Outbound (WeCom):
- Chunk upload field name corrected so server-side actually stores the
  bytes — file messages used to arrive with correct filename/size but
  empty content, breaking every PDF / DOCX / PPTX recipient.
- Scan agent text for served-file URLs in both the text-reply and
  content-parts paths; fetch bytes from the in-memory generated-file
  cache and dispatch through the native chunk upload + media message
  protocol so users receive a tappable file card instead of an
  unopenable markdown link. Cache miss surfaces a clear retry hint.

Async tool result forwarding:
- New AsyncTaskMediaDispatcher routes generation completions (image,
  video, music, 3D model) to whichever IM channel the conversation is
  bound to via ChannelSessionStore + ChannelManager. Web / webchat
  conversations are intentionally skipped — their SSE stream already
  renders the result.
- Wired into all four generation services so IM users actually receive
  generated media as native attachments. Each part now carries an
  absolute disk path so adapters read bytes locally instead of round-
  tripping through an authenticated served URL.

Slack native file upload:
- SlackChannelAdapter overrides the content-parts dispatch. Image /
  audio / video / file / model3d parts ride filesUploadV2 so users see a
  file card with preview thumbnail bound to the same thread as the
  originating message. Text parts continue through chat.postMessage.
- Resolves bytes from the part's local path, falls back to an HTTP fetch
  of fully-qualified URLs.

IM approval hint visibility:
- IM-driven approve / deny / auto-cancel / replay-error hints now go
  through saveMessage + tracker broadcast in addition to the channel
  adapter, so a Web mirror viewing the same conversationId sees the
  resolution. Previously hints reached only the IM channel; the Web
  admin console had no record of the outcome.

Adaptive paste-merge debounce:
- WeCom and other IM clients silently split long pasted prompts into
  fragments that arrive 0.5-2 seconds apart, missing the existing 500ms
  merge window. The agent then saw torn context and emitted multiple
  conflicting replies.
- When the merged buffer crosses a content-length threshold, extend the
  debounce window so subsequent fragments arrive in time. Default
  500ms unchanged for normal short messages.
2026-05-10 09:00:22 +08:00
matevip
134fa1a975 fix(channel): wecom file pipeline + IM approval hint visibility 2026-05-10 08:28:14 +08:00
matevip
225d14026b feat(channel): wecom approval card + keepalive + chunk dedup 2026-05-10 08:28:07 +08:00
matevip
dae74cf26c feat(channel,trigger): wire channel_message + content_match event sources 2026-05-08 15:07:11 +08:00
matevip
92a7508dac fix(channel): mirror plan-execute events to web sse for im-routed conversations 2026-04-30 08:54:32 +08:00
matevip
e759ad4a1b sync: settings UI polish, channel reliability fixes, DeepSeek cross-turn fix
- Settings → Models: inline API key, frosted drawer, dark-mode polish, provider icons, i18n sweep
- WeChat Work channel: rebuild HttpClient on reconnect, dedup failure signals, route auth_succeed errcode!=0 through failure handler
- Channel framework: per-adapter error isolation, QR auth SPI, health indicators
- Agent: patch cross-turn assistants for DeepSeek thinking-mode
- GitHub: bilingual issue templates with required fields
2026-04-29 11:22:47 +08:00
matevip
4011050ceb feat(cron): channel delivery via ChatOrigin + Spring AI ToolContext
Replaces the prior ThreadLocal context plumbing with explicit Spring AI
ToolContext threading carried by an immutable ChatOrigin value object,
so a cron created from inside WeChat (or any IM channel) delivers its
results back to the originating channel.

Architecture
- ChatOrigin / ChannelTarget value objects + per-entry-point factories
  (ChannelChatOriginFactory in vip.mate.channel, CronChatOriginFactory
  in vip.mate.cron — symmetric, no cyclic deps).
- LocaleAwareToolCallback now forwards call(String, ToolContext) and
  getToolMetadata so the decorator chain cannot silently drop the origin.
- AgentService 6-method overhaul + ChatOriginHolder bridge into
  StateGraph buildInitialState which writes CHAT_ORIGIN; ActionNode +
  StepExecutionNode forward it to ToolExecutionExecutor.
- ToolExecutionExecutor builds ToolContext per call; 8/8 tools migrated
  (CronJobTool, WorkspacePathGuard, Video/Image/Browser/ReadFile/Music,
  DelegateAgentTool with parent-origin inheritance).
- CronJobRunner + CronJobLifecycleService 3-segment REQUIRES_NEW model
  (T1 startRun / no-tx runAgent / T2 finishRunAndPublish); ArchUnit
  pins CronJobRunner as @Transactional-free.
- CronResultDelivery Strategy + AbstractCronResultDelivery Template
  with SQL CAS idempotency on mate_cron_job_run.delivery_status —
  replaces the prior process-local Caffeine TTL, cluster-safe.
- CronJobCompletedEvent + @Async @TransactionalEventListener(AFTER_COMMIT);
  cronDeliveryExecutor (core=2, max=4, queue=1000, AbortPolicy + audit).
- CronRunStaleCleanup @Scheduled(5min) sweeps PENDING-15min and
  status='running'-30min in one query each.
- CronJobRunner.wrapWithDeliveryGuard prepends a system note for
  channel-bound crons to suppress hallucinated 'install CLI to send
  WeChat' suggestions.
- ApprovalWorkflowService Memento: persist ChatOrigin snapshot on
  create, restore on replay so cross-restart approvals keep channel
  binding; ChannelMessageRouter + ChatController web-replay both prefer
  the Memento and fall back to fresh-build.
- ChannelManager.sendToChannel 4-arg DeliveryOptions overload;
  ChannelAdapter#proactiveSend default 4-arg pass-through; Slack
  overrides for thread_ts and Telegram overrides for message_thread_id.
- CronJobs UI: read-only 'last delivery' badge driven by
  CronJobMapper.selectListWithDeliveryStatus subquery.

Schema migrations V57/V58/V59 (V56 was already taken by an unrelated
provider migration — Flyway processes versions in order regardless of
gaps):
- V57: mate_cron_job_run delivery_status / target / error + composite
       index (delivery_status, started_at) covering the cleanup sweep.
- V58: mate_cron_job channel_id (indexed) + delivery_config TEXT (JSON
       via MyBatis Plus JacksonTypeHandler).
- V59: mate_tool_approval chat_origin TEXT (Memento).
All idempotent in both H2 (IF NOT EXISTS) and MySQL (INFORMATION_SCHEMA
guard + PREPARE).

ArchUnit guards (test scope, archunit-junit5 1.3.0):
- every concrete vip.mate.* ToolCallback must override
  call(String, ToolContext) — pins the decorator-forward fix.
- CronJobRunner must NOT carry @Transactional on the class or any
  method — pins the 3-segment lifecycle rule.

Tests: 32 new unit tests + 21 regression tests in touched areas, all
53 green:
- ChatOriginTest (6) — value-object invariants + JSON round-trip.
- LocaleAwareToolCallbackToolContextTest (2) — decorator forward.
- DeliveryConfigTest (4) — Jackson round-trip + forward-compat.
- ToolCallbackToolContextForwardArchTest (2) — both ArchUnit guards.
- CronJobRunnerDeliveryGuardTest (3) — channel-cron prefix injection.
- AbstractCronResultDeliveryTest (4) — claim CAS + concurrent CAS.
- ChannelCronResultDeliveryTest (6) — supports / doDeliver / errors.
- ApprovalReplayContinuityTest (5) — Memento round-trip + corrupt
  payload fallback + unknown-field tolerance.

Refs: #25, #16
2026-04-28 21:43:58 +08:00
matevip
2f93d53737 refactor(approval): unify state machine across DB/metadata/memory
Foundation for the ghost-approval root-cause fix.

Adds ResolveOutcome / MetadataDecision; rewrites ApprovalWorkflowService so
every resolve / consume / timeout / supersede transitions through one
two-phase contract: snapshot → DB UPDATE conditional on status=PENDING →
metadata reconciliation → afterCommit memory mutation. ChatController,
ChannelMessageRouter, and ApprovalController all switch to the workflow;
ApprovalService.resolve / resolveAndConsume / consumeApproved /
cancelStalePending / denyAllByConversation are physically removed so
DB-bypass is no longer reachable at compile time.

Specific fixes:
- recoverFromDb preserves DB pendingId + createdAt (was generating fresh
  random ids, breaking every later DB sync)
- effectiveExpireAt = expireAt ?? createdAt + PENDING_TTL: legacy rows
  with NULL expireAt no longer resurrect as live PENDING after restart
- markPendingApprovalsResolved flips pendingApproval.status + currentPhase
  + MessageEntity.status atomically (was only flipping the first field;
  message.status uses existing completed/stopped, not approved/denied,
  to stay within the frontend Message.status union)
- GC scheduler moves to ApprovalWorkflowService; timeouts and overflow
  evictions now sync DB + metadata + memory through markTimeout
- DB UPDATE rows=0 returns alreadyResolved (concurrent-resolve safe);
  exception propagates so @Transactional rolls back; memory stays untouched
- expireRecoveredRow gates metadata write on DB success (was writing
  metadata even when DB update failed, producing the worst-case ghost)
- Mockito JDK 21 agent attach fixed via maven-dependency-plugin properties
  + surefire argLine (no more flaky self-attach across machines)

Tests: 34 new across 4 classes (recovery, resolve, GC, metadata sync).
Full suite: 788 / 788.
2026-04-27 19:30:52 +08:00
matevip
869e0c47e6 refactor(memory): unify ConversationCompletedEvent publish 2026-04-23 08:09:48 +08:00
matevip
a3e6724cf4 fix(chat): channel conversation sync + running indicator
- ChannelMessageRouter: include assistantMessageId in message_complete
  and done broadcasts so ChatConsole observers can reconcile the
  streaming placeholder to the persisted DB row by id instead of
  falling back to the FIFO 'claim' heuristic (which occasionally
  dropped the assistant bubble on external channel conversations).
  Capture the id from ConversationService.saveMessage in both the
  sync agentService.chat path and the streaming processWithStreaming
  path; switch to HashMap since Map.of rejects null values when save
  is skipped (e.g. under approval).

- ChatConsole: add a 'running' indicator on the sidebar so users can
  tell which conversations have an in-flight agent run. Pulsing amber
  dot on the channel icon (both expanded and collapsed modes) plus a
  '生成中…' / 'Generating…' pill in expanded mode.

- ChatConsole: don't cancel the previous conversation's streaming run
  when switching conversations — let it keep running in the background
  and reconcile when the user comes back.
2026-04-17 10:03:56 +08:00
matevip
b35c29767c feat(chat): realtime sync for external channel conversations 2026-04-17 08:25:32 +08:00
matevip
d5e3502829 fix(wiki): backend hardening + batch delete UI 2026-04-13 15:37:12 +08:00
matevip
5124842526 feat(platform): workspace isolation, info architecture, and UI redesign 2026-04-09 16:29:02 +08:00
matevip
579d60125b Initial commit: MateClaw — Java + Vue 3 AI Assistant System
Full-stack AI assistant built on Spring AI Alibaba.
Features: ReAct Agent, Plan-and-Execute, MCP Protocol, Multi-Model, Multi-Channel.

Apache-2.0 License
2026-04-04 19:03:49 +08:00