Commit Graph

93 Commits

Author SHA1 Message Date
matevip
26150b68e8 feat(team): add controlled worker intervention 2026-09-04 04:18:41 -04:00
matevip
7eb44731e7 fix(chat): stabilize long-task input recovery 2026-08-27 05:21:59 -04:00
matevip
01ed4a4fcd feat(goal): make long tasks recoverable and pace continuation dispatch 2026-08-27 03:49:15 -04:00
taobig
2fa2e60170 feat(goal): persist continuous execution across bounded turns
- Add a database-backed supervisor with durable scheduling, fenced leases,
  cooldowns, bounded worker concurrency and expired-lease restart recovery.
- Default new goals to persistent execution with zero meaning unlimited
  cumulative budget; preserve legacy goals and explicit positive limits.
- Yield bounded graph segments to the supervisor instead of ending unfinished
  goals at graph-local continuation limits. Require persisted checklist
  evidence before accepting completion, including concurrent criterion edits.
- Share conversation admission across interactive and background execution;
  preserve partial replies, usage and queued user input during interruption.
- Persist Stop and missing-input pauses, respect approval boundaries, and
  commit resume and approval-denial transitions with correct transactions.
- Retry identifiable transient failures with backoff; retain visible pauses
  for budget limits and errors that require review instead of replaying tools.
- Expose owner-authorized execution status and reconnectable scheduling events;
  add H2, MySQL and Kingbase migrations, API types and bilingual documentation.

Validation: 298 focused backend tests passed, including persistence, restart
scheduling, approval races, cancellation, admission and existing runtime tests.
Frontend type checking, bundled-doc parity and ID precision checks passed.
V188 is registered and all three dialects have unique migration versions;
the migration-map audit still reports 95 pre-existing missing registrations.

Scope: single-backend native runtime. Recovery checks existing state before
repeating effects; this does not promise exactly-once external tool execution.
2026-08-26 05:46:15 -04:00
mateaix
dd7e561e48 fix(agent): harden tool completion and team retries (#606) 2026-08-23 21:43:35 +08:00
matevip
a8a0b75bfa fix(dsh): complete cancellable runtime stream lifecycle; make JSON-RPC streaming asynchronous; terminate process trees and inherited pipes; handle cancellation races; propagate usage telemetry; add regression coverage 2026-08-19 04:30:06 -04:00
matevip
9a3b95df53 fix(team): unify Team Run delivery experience (#596) 2026-08-15 00:37:08 -04:00
matevip
dc3506f748 feat(team): unify team run experience (#596) 2026-08-13 07:00:24 -04:00
倪程伟
0ae8552f5a
fix(webchat): harden stream cleanup and timeout handling
Normalize stream idle timeout overrides, make WebChat orphan cleanup use exact run handles so stale callbacks cannot mutate replacement runs, and preserve approval replay usage metadata with regression coverage.
2026-08-12 02:08:59 -04:00
matevip
071fff76ab fix(agent): enforce tool-backed action completion 2026-08-11 05:31:44 -04:00
倪程伟
55cf53b9b4
fix(webchat): detach SSE subscribers and reclaim orphan runs
Use detach rather than complete for WebChat SSE disconnect callbacks, add an orphan-run grace policy for subscriberless active runs, and emergency-save partial assistant output when reclaimed.
2026-08-11 03:42:43 -04:00
倪程伟
e9c08edb3a
fix(webchat): close SSE streams on done or error
Close WebChat subscriber SSE connections when the logical stream reaches done/error, make the emitter timeout configurable, and ensure stale-run eviction closes subscribers instead of leaving clients waiting.
2026-08-11 03:39:56 -04:00
mateaix
197e697173 fix(stt): repair DashScope audio decoding (#580) 2026-08-10 21:43:02 +08:00
mateaix
54eb77d7c7 feat(feishu): show streaming execution progress 2026-08-09 20:34:54 +08:00
mateaix
a204e8eec8 fix(self-evolution): harden lifecycle and plan loops 2026-08-09 18:13:24 +08:00
matevip
22c49f3a07 fix(chat): keep every iteration reasoning and render the timeline in emission order 2026-08-06 05:13:09 -04:00
matevip
32dd6e7911 fix(channel): sync-path IM channels stop relaying pre-tool rehearsals verbatim 2026-08-04 22:50:42 -04:00
matevip
8a7a04ee1f refactor(channel): shared provisional-narration tracker as the single supersede authority 2026-08-04 22:50:12 -04:00
matevip
547adb30ee feat(agent): producer-assigned content-kind tag on stream deltas 2026-08-04 22:49:34 -04:00
matevip
54a2c3c0a3 fix(wecom): drop pre-tool rehearsal narrations from permanent bubbles 2026-08-04 06:14:55 -04:00
matevip
29a1edc3b1 fix(chat): supersede pre-tool content segments by structure, not text patterns 2026-08-04 03:46:32 -04:00
mateaix
7fb6ba7461 sync: v2.0.0 dev + 落盘文件按天分目录 (#554)
- 附件 / 生成媒体写入 {conversationDir}/yyyy-MM-dd/{storedName},serve URL 保持平铺
- 读取路径同时探测平铺与日期两种布局,可随时开关 mateclaw.chat.upload.date-folders
- 读取侧强制 storedName 为纯文件名;原始文件名回退取最近修改的匹配
- 修正 Linux / macOS / Windows 的路径与时间戳精度差异
2026-08-01 18:33:24 +08:00
matevip
ad6b0728e9 fix(channel): stop streaming replies from showing the same text twice 2026-07-29 21:29:48 -04:00
matevip
aaae3bf122 fix(channel): claim each inbound message once so redeliveries stop answering twice 2026-07-29 05:43:01 -04:00
matevip
9bc4741aa7 fix(channel): apply message filters on card streaming paths 2026-07-28 06:31:01 -04:00
matevip
e294b32542 feat(channel): scope conversation ids by channel to stop cross-workspace bleed 2026-07-23 18:50:10 +08:00
matevip
e2747df2c5 feat(channel): /model magic command for per-conversation model switching (restore) 2026-07-23 11:49:28 +08:00
matevip
1bb81234d0 feat(approval): classify auto-approve misses and persist the outcome on guard audit rows 2026-07-23 11:44:21 +08:00
mateaix
a4ee953d31 feat(channel): /model magic command for per-conversation model switching 2026-07-22 22:27:34 +08:00
matevip
8f2d76965b test(channel): cover execution metadata persisted on the IM sync path 2026-07-22 18:10:45 +08:00
matevip
a183519d69 feat(channel): extract the per-turn stream accumulator and share it between web SSE and IM sync paths 2026-07-22 18:09:59 +08:00
matevip
4fde4c2b7a fix(channel): deliver magic-command confirmations via renderAndSend so pre-posted thinking placeholders get consumed 2026-07-22 15:47:03 +08:00
matevip
8a55bdd367 feat(channel): relay per-stage narration as standalone messages on the sync IM path 2026-07-22 14:08:55 +08:00
matevip
8749c915cd feat(wecom): roll progress bubble per stage narration 2026-07-22 13:40:22 +08:00
matevip
3781275e59 feat(llm): smarter failover — overload-aware retries, provider retry windows, provider pool auto-recovery 2026-07-22 11:12:47 +08:00
mateaix
be836b6f0a fix(wecom): harden streaming reply management 2026-07-21 21:48:30 +08:00
matevip
bf6bed5511 feat(wecom): event-driven progress bubble with live tool trace 2026-07-21 10:29:00 +08:00
matevip
beb1a8c243 feat(channel): extensible magic commands (/new /help /status /stop) 2026-07-21 10:28:41 +08:00
jack
1e2b7bbc2c
fix(weixin): send generated files through weixin channel (#543)
Send agent-generated files as native WeChat attachments via the iLink upload flow, and fix the wire protocol for file uploads: dedicated wire ObjectMapper (bypasses the global Long-to-String serializer), md5/len fields and encrypt_type on media items, channel_version 1.0.2, and explicit business-error handling on ret != 0. The weixin adapter now routes generated-file URLs through GeneratedFileScrubber, matching WeCom/Feishu behavior.

Fixes #307
2026-07-20 20:34:59 +08:00
matevip
d10ed9dd06 feat(channel): add clear magic command 2026-07-20 18:21:46 +08:00
matevip
c9cc5b4f6f feat(chat): glass-themed preview for uploaded & AI-generated docx/xlsx/pdf (#513) 2026-07-13 18:00:54 +08:00
matevip
e6c35ecfc7 fix(channel): sanitize conversationId as a filesystem path segment — fixes wecom/dingtalk/feishu attachment upload on Windows (#507) 2026-07-10 16:54:05 +08:00
倪程伟
9d4041714f
fix(chat): store chat-upload path as root-relative, not absolute server path (#455)
After the workspace-aware chat-uploads change, the upload root became
absolute (the resolver normalizes via toAbsolutePath/normalize, and the
autoconfiguration rewrites baseDir to an absolute path). ChatController.upload
then set ChatUploadResponse.path to that absolute path — despite the inline
comment promising a relative path "to avoid exposing the server's absolute
path". The field is rendered into the LLM prompt ("附件: foo (path)") and
returned to the client, so this leaked the server filesystem layout into both
the prompt and the response, and broke portability if the deploy dir moves.

Extract toRelativeUploadPath(uploadRoot, convId, storedName) which makes the
path relative to the upload root's parent (preserving the trailing sub-dir
name, e.g. chat-uploads/{convId}/{storedName}) and normalizes separators to
'/'. Retrieval is unaffected: it goes through the basename-based
ChatUploadResolver and the /api/v1/chat/files/... URL, not this field.

Adds ChatControllerUploadPathTest (default root, absolute workspace-scoped
root, custom base-dir name) asserting the result is relative and leak-free.

Addresses the blocker item in #452.
2026-07-02 17:40:59 +08:00
倪程伟
fcb488c567
feat(workspace): chat-uploads 上传目录工作空间/Agent 感知化 (#422)
* feat(workspace): chat-uploads 上传目录工作空间/Agent 感知化 (#421)

把硬编码的 data/chat-uploads/{conversationId}/ 改为按工作空间/Agent 解析,
解析优先级:Agent workspaceBasePath → Workspace basePath → 可配置默认目录
(新配置 mateclaw.chat.upload.base-dir,默认 data/chat-uploads,保持现网零行为变化)。

- 新增 ChatUploadLocationResolver 中央解析器:写路径返回唯一根,读/清理
  路径返回候选根列表(工作空间根 + 默认根)做双重查找,保证迁移前旧附件
  仍可解析/清理;conversationId→ConversationEntity 查询带 5min 缓存。
- 新增 ChatUploadProperties + ChatUploadAutoConfiguration(启动建目录)。
- 复用 AgentGraphBuilder.resolveAgentBasePath(提升为 public)的优先级与
  安全规则(相对路径在 workspace 根下解析,绝对路径逃逸被拒)。
- 所有写入/读取点改为走 resolver;读取点走双重查找。
- 解决 Spring 循环依赖(resolver → agentService → ... → conversationService
  → resolver):resolver 的 AgentService 注入加 @Lazy。

向后兼容:默认目录不变;双重查找覆盖历史消息里的相对路径;
服务端点 URL 契约不变,前端无需改动。

测试:新增 ChatUploadLocationResolverTest(8 用例);修复受影响的现有测试构造。

* refactor(workspace): address review findings on chat-uploads resolver (#421)

应用 code review 的 4 项修复:

1. (correctness) ChatUploadLocationResolver 缓存新增 ConversationDeletedEvent
   监听器,删除会话时立即失效 conversationId→ConversationEntity 映射。
   否则备份恢复后用相同 id 重建会话,会继承最长 5 分钟的过期 workspace/agent
   映射,导致 cleanAttachmentFiles 走错(过期的)上传目录。复用既有
   @EventListener-on-bean 模式(与 AsyncTaskService / WorkspaceLookupCache 一致)。

2. 收紧 resolveWorkspaceScopedRoot 里 3 个过宽的 catch(Exception) →
   MateClawException + warn,让真正的 bug(NPE / DataAccessException)暴露
   而非被静默降级为 debug 日志。

3. 更新 ChatController.upload 过期注释:会话尚未创建时附件暂存默认目录,
   会话创建后读取走双重查找仍能命中。

4. 移除不可达分支(agentWorkspaceId != workspaceId)—— 会话的 agent 必然
   归属会话的 workspace(创建时强约束),直接用会话 workspace 即可,
   少一次冗余 DB 查询与一层推测性逻辑。

测试:ChatUploadLocationResolverTest (8) + ConversationServiceCleanAttachmentFilesTest (2) 全绿。
2026-06-26 14:20:54 +08:00
倪程伟
2f12c269f4
feat(webchat): API-Key 渠道补齐审批 resolve + replay (ISSUE #413 P1) (#415)
* feat(webchat): add approval resolve + replay for API-Key channel (ISSUE #413 P1)

Before this, a WebChat (API-Key) channel that hit a ToolGuard-protected
tool parked the turn in a pending approval the visitor could never
clear — it hung for 30 min until the GC timeout and the turn was
wasted. This PR closes the loop, mirroring the web ChatController.

A1 — no code change. tool_approval_requested already reaches the SDK
via ToolExecutionGuardHelper's streamTracker.broadcastObject (direct
SSE push, bypassing the StreamDelta path). Adding it to
forwardVisitorEvent would double-deliver; the default-drop is correct.

A2 — new /sessions/approve and /sessions/deny REST endpoints. Auth is
the existing visitorToken + conversationId ownership guard; the actor
is webchatUsername(visitorId), which resolves the 'no MateClaw
username' blocker noted in the old stopSession javadoc. Both broadcast
tool_approval_resolved so the SDK clears its banner in real time.

A3 — approve returns an SSE stream: resolveAndConsume (atomic DB +
metadata + memory), restoreChatOrigin (recovers the webchat origin
captured at createPending), then chatWithReplayStream replays the
tool call and continues the turn. Replay may re-trigger approvals,
which the existing tool_approval_requested direct push handles.

A4 — stopSession now sweeps pending approvals (denyAllByConversation)
and broadcasts each resolution, so stopping a stream no longer leaves
approvals lingering for the GC.

Tests: WebChatApprovalInteractionTest (7) — deny resolves + broadcasts,
deny auth/ownership guards, idempotent unknown-pending, stop sweep
clears pending, stop no-op when nothing pending.

Regression: WebChatStopStreamTest (5), WebChatArchivePinTest (6),
WebChatSchemaFieldsTest (5), WebChatWikiPageListTest (8),
ApprovalWorkflowServiceResolveTest (13), GcTest (7), RecoveryTest (7).

* fix(webchat): IDOR guard + SSE hang fix (PR #415 review)

Addresses all review feedback from mateaix:

P0 IDOR (security): /sessions/approve and /sessions/deny accepted a
client-supplied pendingId without cross-checking it belonged to the
caller's conversation. A visitor could resolve / replay another
visitor's guarded tool call. Fix: getPending(pendingId) then assert
conversationId matches before resolving. Added getPending delegate on
ApprovalWorkflowService so the webchat controller (which holds the
workflow facade) can do the precise lookup.

SSE hang: approveSession's already-resolved / error branches broadcast
'done' before streamTracker.register/attach, so the event had no
subscriber and the SSE hung to the 10-min timeout. Fix: register+attach
first, then resolveAndConsume. Removed the now-duplicate register/attach
in the replay branch.

Tests: +2 IDOR cases (cross-visitor pendingId rejected 404; mismatched
pendingId rejected 404). denyResolvesPending now asserts via getPending
(findPendingByConversation returns the earliest pending, polluted by
cross-test map state). denyUnknownPendingIsSafe updated to expect 404
(no longer leaks pendingId existence). Isolated IDOR victim/attacker
visitor IDs to avoid cross-test conversationId collisions.

Regression: WebChatStopStreamTest (5), WebChatArchivePinTest (6),
ApprovalWorkflowServiceResolveTest (13), GcTest (7), RecoveryTest (7).

* style(webchat): use simple ChatOrigin name in approveSession (PR #415 review)

Reviewer flagged fully-qualified inline types (ResolveOutcome was fixed
in the prior commit; ChatOrigin was missed). Add the import and switch
the 3 FQN references in approveSession to the simple name, matching the
ResolveOutcome cleanup. chatStream's pre-existing FQN usages are out of
this PR's scope and left untouched.
2026-06-25 11:18:52 +08:00
倪程伟
b478eef78c
feat(im): resolve workflow approvals via feishu/wecom card clicks (ISSUE #413 P2-B3) (#416)
Before this, a workflow await_approval step whose approverChannels
pointed at feishu/wecom was effectively dead for IM interaction. Even
after PR #414 (B1) pushed the notice to the IM group, clicking the
card's Approve/Deny buttons did nothing useful:

- Identity check (requester==clicker) failed-closed: wf- approvals
  have userId=null (system-initiated), so every click was rejected.
- Even if it passed, the synthetic /approve injection was a dead end:
  the router routes by conversationId, but wf- ids use a synthetic
  workflow:run:{runId} key that no IM conversation matches, so
  findPendingByConversation returned null and the /approve was fed
  to the LLM as plain text.

B3 fix: both ToolGuardCardHandlers now detect the wf- prefix and
resolve inline (approvalService.resolve), bypassing the synthetic
injection entirely. The WorkflowApprovalResolvedEvent published
inside resolve is picked up by ApprovalResumeBridge (activated in
PR #414 B2), which resumes the paused run. This mirrors the Web /
WebChat resolve path (PR #415).

Identity policy: any audience member may resolve a wf- approval.
The card only reaches channels declared in await_approval's
approverChannels, so whoever sees it is a designated approver.
Regular tool approvals keep the strict requester==clicker guard.

Tests:
- wecom ToolGuardCardHandlerTest: +2 wf- cases (inline resolve, no
  synthetic injection; already-resolved renders expired). Existing 6
  cases updated for the new 3-arg constructor.
- feishu FeishuCardDispatcherTest: updated for the new factory
  constructor signature.

Regression: ApprovalWorkflowServiceResolveTest (13), GcTest (7),
RecoveryTest (7), feishu dispatcher (4), button value (7),
renderer (3+3) — all green.
2026-06-25 09:56:36 +08:00
倪程伟
f6156f6093 feat(webchat): expose agent-bound wiki pages to API-Key callers
Add GET /api/v1/channels/webchat/wiki/pages mirroring /skills, so
downstream integrators can build a [[slug]] picker UI that points the
LLM at specific wiki pages. The picker token format is the universal
Obsidian/Wikipedia wikilink convention; the LLM consumes [[slug]] via
the existing wiki_read_page(slug=...) tool, so no agent-runtime changes
are needed.

- AgentBindingResolver.getBoundKbIds(agentId): three-state mirror of
  getBoundSkillIds. null = no rows (fall through to workspace-wide KBs),
  Set.of() = explicitly scoped to zero KBs, non-empty = explicit scope.
- WebChatController.listWikiPages: API Key + visitorToken auth chain,
  agentId workspace anti-escalation, visibility excludes pageType=
  synthesis (LLM intermediate artifacts), 100-page cap forces keyword
  filter, response carries only display-level metadata.
- WebChatWikiPageView DTO: kbId/kbName/slug/title/summary/pageType;
  content/embedding/sourceRawIds deliberately stay admin-console-only.
- WikiTool.wiki_read_page @Tool description: document the [[slug]]
  convention so the LLM treats each token as a wiki-page reference.
- WebChatWikiPageListTest: 8 cases covering happy path, keyword filter,
  synthesis exclusion, anti-escalation, auth failures, cap behavior,
  and the no-binding → workspace-wide fallback.

Closes #381.
2026-06-20 07:21:07 +08:00
倪程伟
31c98e923d feat(webchat): expose agent-bound skill list to API-Key callers
GET /api/v1/channels/webchat/skills?agentId=<optional>&visitorId=<required>
Headers: X-MC-Key + X-MC-Visitor-Token

Downstream systems integrating via the webchat SSE endpoint have no way
today to enumerate the skills a visitor can invoke — the existing
GET /api/v1/skills is JWT + workspace-role gated, unreachable from the
API-Key-authenticated webchat channel. Without a list, integrators
can't render a slash picker UI; visitors have to know skill slugs by
heart.

The new endpoint mirrors the /stream auth chain (resolveChannel +
verifyVisitorToken) and reuses AgentBindingResolver.getBoundSkillIds
to scope visibility. Only enabled skills explicitly bound to the agent
surface; agents with no explicit bindings return an empty list rather
than inheriting the global pool (the agent config stays the source of
truth for what surfaces in visitor UI). The agentId anti-escalation
guard from /stream is reused verbatim — an explicit agentId must
belong to the channel's workspace.

Returns WebChatSkillView (id / name / nameZh / nameEn / description /
icon). Deliberately omits SKILL.md content, configJson and
securityScanResult: those never leave the admin console.

Issue: #373
2026-06-19 06:20:59 +08:00
倪程伟
6bbb6489f4 feat(webchat): expose phase / tool_start / tool_end / plan as SSE events
Previously WebChatController.chatStream silently dropped every agent
lifecycle event except _usage_final (and content_delta / thinking_delta
derived from delta.payload). Visitors sat with nothing between the
meta event and the first content chunk — typically 3–10s when the
agent plans / recalls memory / runs tools, longer when the agent
chained multiple tool calls. The JWT chat path (ChatController) had
this wiring; webchat did not.

Curated 4-event subset (per design review):
- phase        — high-level phase transition (planning / generating /
                 summarizing / ...). SDK shows a "AI is thinking..."
                 typing indicator before the first token.
- tool_start   — agent invoked a tool. SDK shows a localized badge
                 ("Searching...", "Reading file.pdf", ...).
- tool_end     — tool completed. SDK clears the badge.
- plan         — Plan-Execute agents expose their step list. SDK can
                 render a checklist.

Deliberately NOT forwarded (internal noise / leak risk):
- _usage_final, _routing_decision — consumed internally
- finish_reason                  — implicit in `done`
- feedback_event                 — visitor can't retry/regenerate anyway
- perf_summary, iteration_*      — internal metrics
- plan_step_started/completed    — too granular; the plan event covers
                                   the visitor's needs

Critical safety constraint: tool_start / tool_end carry ONLY the tool
name. Tool arguments and results are dropped — agent tool calls can
contain PII (file paths, user queries, credentials), and relaying
those to a 3rd-party website frontend is a data leak. The SDK maps
tool name → localized label via its own lookup.

Backward compat: existing clients ignore unknown event types per the
SSE spec, so adding these is non-breaking.

Tests: 5 new cases in WebChatStreamE2ETest covering each event type
+ a regression case asserting internal events are silently dropped.
85/85 webchat tests green.

Docs: docs/zh/webchat.md gains a "实时进度事件" subsection.

Stack: feat/webchat-attachment-e2e → feat/webchat-stream-phase-events
Follow-up to epic #355.
2026-06-18 06:33:17 +08:00
倪程伟
a0598fb0b8 test(webchat): HTTP e2e coverage for attachment upload/stream/download
Adds WebChatAttachmentE2ETest — second HTTP-level test in the webchat
suite. Boots RANDOM_PORT, drives Spring's multipart parser for real,
and verifies the cross-endpoint wiring that turns an upload into an
agent-addressable attachment.

Coverage (7 tests):
- upload + /stream round-trip: persisted user message's content_parts
  carries the file part with a server path that points into the
  conversation's upload dir; bytes on disk match what was uploaded
- unknown attachmentId → silently dropped (no error, text-only parts)
- foreign visitor cannot reference another visitor's fileId
  (consume() is conversation-scoped)
- upload without visitorToken → HTTP 401
- upload with disallowed extension → HTTP 400
- GET /files streams back the uploaded bytes
- GET /files without visitorToken → HTTP 401

AgentService is @MockBean'd so /stream returns instantly; what we
assert is the persisted user-message shape (DB row content_parts),
not the agent's actual file consumption (which would need a real
agent + tool runtime — out of scope for the wire-format focus).

Worth noting: RHttpStatusAdvice maps R.fail(401/400) to the matching
HTTP status, so 4xx assertions are on the HTTP status, not the body.

Stack: feat/webchat-stream-e2e-test → feat/webchat-attachment-e2e
Follow-up to PR #363.
2026-06-18 06:33:17 +08:00