Commit Graph

1020 Commits

Author SHA1 Message Date
matevip
b6cca3edd0 fix(skill): make skill ZIP size caps configurable (#467) 2026-07-03 14:58:48 +08:00
matevip
bb946685b3 fix(chat): render generated-file download links with the file name, not the raw id URL (#466) 2026-07-03 14:34:35 +08:00
matevip
25737495e5 feat(chat): per-turn token usage breakdown with cache hit/miss/write and reasoning split (#474) 2026-07-03 11:28:00 +08:00
MIST
fa5d406118
fix(wiki): agent 通过 wiki_create_page 写入的页面缺少 raw/chunks/embeddings/citations,界面无法识别与操作 (#475)
## 背景

Agent 通过 `wiki_create_page` 工具写入知识库的报告、分析结果等页面,虽然能在"Wiki 页面"列表中看到,但:

- **识别不到**:不出现在"原始材料"面板
- **不可操作**:"查看引用"按钮消失(`sourceRawIds` 为空)
- **不可处理**:无 raw 可 reprocess、无 chunks 导致语义检索查不到
- **不可下载**:无 raw 行,下载端点无数据

## 根因

`wiki_create_page` 只调用 `WikiPageService.createPage()` 写了一张 `mate_wiki_page` 表(`sourceRawIds=null`),跳过了 UI 上传文本路径的全部"消化副产物"——raw material 创建、chunks 切片、embeddings 生成、citations 构建、lineage 血缘、`WikiPageCreatedEvent` 事件发布。

## 解决方案

让 `wiki_create_page` 在落 page 之后,同步创建 raw material 并补齐全部消化副产物,但**不重跑 LLM 页面生成**(agent 已提供最终内容)。`WikiTool.wiki_create_page` 的同步返回(`ok / pageId / slug`)不受影响。

### 改动(4 个文件,+190 / -1)

| 文件 | 改动 |
|---|---|
| `WikiRawMaterialService.java` | 新增 `addAgentAuthored(kbId, title, content)`:创建 `sourceType="text"` 的 raw 行,状态置 `processing`(不发布 `WikiProcessingEvent`,避免触发 LLM 重消化),按 content hash 去重 |
| `WikiProcessingService.java` | 新增 `linkAgentPageToRaw(pageId, kbId, rawId, rawTitle, pageType)`:编排 ①`mergeSourceLineage`(血缘)②`deriveKnowledgeLayer` ③`persistChunks`(切片)④`embedMissingChunks` + `embedPage` ⑤`buildCitationsAsync` ⑥发布 `WikiPageCreatedEvent` ⑦raw 置 `completed` + `lastProcessedHash`。每步独立 try/catch,单点失败不阻塞其它 |
| `WikiTool.java` | `wiki_create_page` 在 `createPage` 后调用 `addAgentAuthored` + `linkAgentPageToRaw`。`processingService` 为可选注入(`@Autowired(required = false)`),测试上下文中为 null 时退化为旧行为 |
| `WikiPageService.java` | `deleteExclusiveBySourceRawId` 新增跳过 `lastUpdatedBy = "ai"` 的页面:agent 直接创作的页面不应在 reprocess 时被自动清理 |

### 修复后效果

| 能力 | 修复前 | 修复后 |
|---|---|---|
| Wiki 页面列表可见 |  |  |
| 原始材料面板可见(可识别) |  |  |
| 下载按钮 |  |  |
| 查看引用按钮(可操作) |  隐藏 |  |
| CitationDrawer 有内容 |  空 |  |
| 可 reprocess(可处理) |  |  |
| 语义检索可命中 |  |  chunks+embeddings |
| pipeline 触发器评估 |  |  |
| 页面内容 = agent 原文 |  | (不重跑 LLM) |
| UI "AI 生成"标记 |  | (`lastUpdatedBy = "ai"`) |

## 验证

全量 wiki 模块测试通过:`Tests run: 474, Failures: 0, Errors: 0, Skipped: 0`

## 风险

- `processingService` 为可选注入,测试/轻量上下文下退化为旧行为,**向后兼容无破坏**
- 新增两处数据库写入(raw + chunks),每次 `wiki_create_page` 增加 1 条 raw + N 条 chunk 行
- reprocess 行为:reprocess 时 `lastUpdatedBy = "ai"` 的页面被保留不清理,LLM 管线会从 raw content 重新消化生成额外概念页
2026-07-03 10:37:16 +08:00
matevip
2e3dd071a5 fix(kb-open): assert session belongs to path kbId + cleanup
- requireSessionOwnership now also checks session.kbId() == path kbId (404 on
  mismatch), so a research session started under one KB cannot be addressed via
  another KB path even when the caller's key is bound to both — defense-in-depth
  on top of the keyId ownership check.
- Drop internal "R7" / "review #446" markers from the controller Javadoc in
  favour of functional wording.
- Import Set/Map/concurrent types and static any() instead of inline FQNs in the
  new kb-open research/auth tests, per code style.
2026-07-02 17:52:25 +08:00
倪程伟
20c681a7c8
feat(kb-open): Deep Research 开放 API(start/SSE/status/cancel) (#446)
* feat(kb-open): Deep Research open API (start/SSE/status/cancel)

Implements the async Deep Research endpoint for the KB Open API (#443).
Research is a multi-step LLM pipeline (plan → retrieve+draft → compose)
that runs asynchronously and broadcasts progress via SSE.

Endpoints:
- POST /{kbId}/research                      start (returns sessionId + streamUrl)
- GET  /{kbId}/research/{id}/stream          SSE progress (?token= for EventSource)
- GET  /{kbId}/research/{id}/status          query status / final report
- POST /{kbId}/research/{id}/cancel          cancel running session

Components:
- KbOpenResearchController: 4 endpoints, @RequireKbScope("kb:search")
- KbResearchSessionRegistry: in-memory session tracking with keyId
  ownership (a caller can only query/cancel their own sessions)

Security:
- R7: SSE uses ?token= query param (KbOpenApiAuthFilter already supports
  this fallback for EventSource which can't set Authorization headers)
- Session ownership: status/cancel/stream all verify keyId match
- Cancel checks session is RUNNING (409 otherwise)

Reuses existing WikiResearchService.research() + ChatStreamTracker for
the actual research pipeline and SSE broadcasting.

Tests (6 new, all green):
- KbResearchSessionRegistryTest: register/complete/fail/cancel lifecycle,
  cancel-on-completed no-op, unknown session returns empty

Closes #443

* fix(kb-open-research): cooperative cancel, sticky terminal, TTL, concurrency cap

Review #446 — address all 4 job-lifecycle/cost blockers + nits:

1. Cooperative cancellation (was: cancel only flipped status, pipeline ran
   to completion). Cancel endpoint now calls streamTracker.requestStop();
   WikiResearchService.ensureNotCancelled() checks isStopRequested at each
   stage boundary (plan→draft, draft→compose) and inside the parallel draft
   fan-out — so cancel actually halts the expensive LLM calls, not just the
   SSE stream. Throws ResearchCancelledException (caught locally, no error
   broadcast).

2. Sticky CANCELLED terminal. complete()/fail() now no-op on a CANCELLED
   session, so a user who cancelled never sees a COMPLETED report surface
   via /status.

3. Session registry TTL. Terminal sessions get an updatedAt timestamp and
   are evicted by a @Scheduled sweep after
   mate.kbopen.research.session-ttl (default 30m). RUNNING sessions are
   never evicted. Prevents unbounded memory growth.

4. Per-key concurrency cap. startIfAllowed() rejects new research when a
   key already has mate.kbopen.research.max-concurrent-per-key (default 3)
   RUNNING sessions → 429. Stops one key from spawning ~60 parallel
   multi-step LLM pipelines per minute under the per-min rate limiter.

5. Inline FQN → import (controller LinkedHashMap, test List.of).

Nits (inherited from P0-A rebase):
- V162→V164, prefix VARCHAR(12), design doc moved to rfcs/.
- Design doc: kb:search scope row now documents it covers /research/**.

31 tests pass (12 registry incl. sticky-cancel/concurrency/TTL +
13 service + 4 rate limiter + 4 controller + ...).

* fix(kb-open): scope-limited ?token= SSE auth fallback in KbOpenApiAuthFilter

R7: the SSE progress stream (/research/{id}/stream) is consumed by browser
EventSource, which cannot set an Authorization header. The filter's
extractBearerToken() never read ?token= (still a TODO), so the SSE endpoint
was unreachable from the browser — the headline use case got 401.

Fix: accept ?token= ONLY on SSE stream paths (isSseStreamPath, suffix
/stream), reject it everywhere else so the API key does not leak into
access/proxy logs for normal calls (R5). Matches the JwtAuthFilter convention
(getRequestURI logs carry no query string).

Also bypass the per-minute rate limiter on the SSE path: EventSource
reconnects/heartbeats would otherwise burn the key's window and 429 its own
POST /research start. Rate limiting belongs on the cost-producing endpoints.

Tests (6 new, KbOpenApiAuthFilterTest):
- non-SSE: header passes, ?token= rejected (no authenticate call)
- SSE:     ?token= authenticates, missing token → 401
- SSE:     bypasses rate limiter; non-SSE still hits it

* fix(kb-open-research): make per-key concurrency cap atomic (no check-then-act race)

startIfAllowed() did stream-and-count then put() — not atomic. Two
concurrent starts for the same key could both pass the count check (both
see < cap) and both put, admitting more sessions than the cap. On the
virtual-thread start endpoint this is a real DoS/cost-bypass path.

Fix: maintain a per-key AtomicInteger running counter (runningPerKey),
incremented atomically on start (incrementAndGet + rollback on overflow)
and decremented on each RUNNING→terminal transition (complete/fail/cancel).
The counter is kept in lock-step with status==RUNNING; since terminal
states are sticky, each session decrements exactly once.

cancel() also rewritten to capture the pre-transition state cleanly (the
old return check relied on Map.computeIfPresent returning the new value,
which worked but read as 'before.status==CANCELLED').

Tests (+2): cancelled/failed release slot (counter consistency), and a
concurrent-start test (12 virtual threads, cap=3) asserting exactly cap
admits — would be flaky/fail under the old impl.

* refactor(kb-open-research): remove unused register() back-compat method

register() was left over from the initial impl — it bypassed the per-key
concurrency cap (no startIfAllowed check) and, after the atomic-counter fix,
incremented runningPerKey without any overflow rollback. With no production
caller (the start endpoint uses startIfAllowed), it only existed for tests to
set up a RUNNING session. Drop it and route the tests through startIfAllowed
so nothing can accidentally ship a path that ignores the cap.
2026-07-02 17:47:24 +08:00
倪程伟
9d4041714f
fix(chat): store chat-upload path as root-relative, not absolute server path (#455)
After the workspace-aware chat-uploads change, the upload root became
absolute (the resolver normalizes via toAbsolutePath/normalize, and the
autoconfiguration rewrites baseDir to an absolute path). ChatController.upload
then set ChatUploadResponse.path to that absolute path — despite the inline
comment promising a relative path "to avoid exposing the server's absolute
path". The field is rendered into the LLM prompt ("附件: foo (path)") and
returned to the client, so this leaked the server filesystem layout into both
the prompt and the response, and broke portability if the deploy dir moves.

Extract toRelativeUploadPath(uploadRoot, convId, storedName) which makes the
path relative to the upload root's parent (preserving the trailing sub-dir
name, e.g. chat-uploads/{convId}/{storedName}) and normalizes separators to
'/'. Retrieval is unaffected: it goes through the basename-based
ChatUploadResolver and the /api/v1/chat/files/... URL, not this field.

Adds ChatControllerUploadPathTest (default root, absolute workspace-scoped
root, custom base-dir name) asserting the result is relative and leak-free.

Addresses the blocker item in #452.
2026-07-02 17:40:59 +08:00
倪程伟
91a7842393
fix(mcp): fail-closed on unknown channel + signing-key self-heal (#471)
Adversarial review of PR #464 found that classify() promoted an absent
channelType to the 'authenticated' trust branch, stamping an untrusted
ThreadLocal username (e.g. stale value on a reused thread, or internal
tasks like SkillConsolidation/Reflection that carry no channel) with
authenticated trust — contradicting the fail-closed contract the service
documents.

- classify(): channel==null/blank now resolves to NONE (no injection);
  only the explicit 'web' channel may yield authenticated. Unrecognised
  non-web channels downgrade to external, never authenticated.
- signingKey(): replace the one-shot keyParseAttempted latch with
  lastAttemptedPem so a corrected/hot-reloaded PEM re-parses on the next
  call without an app restart. Still fail-closed when PEM is unchanged.
- Tests: 4 new cases lock the regression (null+dirty-ThreadLocal->NONE,
  blank->NONE, novel channel->external, self-heal after config fix).
- .gitignore: exclude local .codebase-memory/ agent index.

MCP+identity suite: 93/93 green.
2026-07-02 11:13:27 +08:00
matevip
c3da1ce817 fix(browser): actually block redirect targets in SSRF interceptor
The first cut of the per-request interceptor called route.resume() for every
request. Playwright follows server-side 3xx redirects internally on resume()
WITHOUT re-invoking the route handler, so a public page that 302s to a
metadata IP still reached it — verified via runtime E2E (the handler only ever
saw the httpbin.org URLs, never the 169.254.169.254 redirect target).

Fix: for navigation requests, fetch with maxRedirects=0 and validate the
Location of each hop through UrlSafetyChecker before fulfilling; abort when a
hop resolves to a blocked host. Subresources/fetches keep the direct per-URL
check + resume path. Non-navigation and non-http(s) requests are unaffected.

Runtime-verified: httpbin.org 302 -> 169.254.169.254 is now aborted
(net::ERR_FAILED; log "blocked redirect ... cloud-metadata endpoint"), while
example.com and wikipedia.org (rich subresources) still load with no false
blocks.
2026-07-02 10:31:35 +08:00
matevip
c95df54949 harden(browser): re-check SSRF on every request + make metadata block unbypassable
Two SSRF hardenings on top of the private-network deployment mode:

1. Redirect / subresource re-validation. The SSRF guard previously ran only on
   the initial navigation URL in the tool layer, so a public page that 302s to
   169.254.169.254 (or a script fetch / img to a metadata IP) reached the target
   unchecked — worse now that private-network mode exists. Install a per-context
   request interceptor (BrowserLauncher.applyContextDefaults) that re-runs
   UrlSafetyChecker on every http(s) request and aborts blocked ones. Non-network
   schemes (data:/blob:/about:) pass through; unexpected checker faults fail open
   so a transient error cannot wedge the page (the initial URL was already checked).

2. Allowlist can no longer open a cloud-metadata endpoint. Metadata hostnames and
   IPs are now checked BEFORE the allowlist short-circuits, so an operator entry
   like 169.254.0.0/16 or metadata.google.internal can never expose instance
   metadata. Ordinary private-host allowlisting is unaffected (regression-tested).

Also correct the 192.0.0.192 comment (Oracle Cloud IMDS, not Azure).
2026-07-02 09:43:35 +08:00
matevip
b9f01db6f7 fix(browser): scope per-context TLS bypass to LAN mode
The per-context setIgnoreHTTPSErrors was gated on ignoreHttpsErrors alone,
while the Chromium command-line cert flags require both ignoreHttpsErrors AND
allowPrivateNetwork. Setting only PLAYWRIGHT_IGNORE_HTTPS_ERRORS therefore
disabled certificate validation for all browser traffic, including the public
internet (MITM exposure). Gate the per-context bypass on allowPrivateNetwork
too, so ignoring HTTPS errors is scoped to LAN deployments — matching the
command-line path and the documented intent.

Also correct a comment: 192.0.0.192 is Oracle Cloud's IMDS address, not Azure.
2026-07-02 09:27:11 +08:00
MIST
9e33782b7d
feat(browser): 放开内网服务访问限制,新增局域网部署模式开关 (#472)
* feat(browser): 放开内网服务访问限制,新增局域网部署模式开关

### 背景
局域网部署时,Agent 用浏览器工具访问 http://192.168.x.x:port 等内网服务会被默认 SSRF 严格模式拦截。

### 方案
新增两个 .env 开关(默认 false,行为与改动前完全一致):

- PLAYWRIGHT_ALLOW_PRIVATE_NETWORK=true :放行本地回环和私有 IP,云元数据端点仍拦截
- PLAYWRIGHT_IGNORE_HTTPS_ERRORS=true :忽略 HTTPS 证书错误(自签证书场景)
顺带修复 IPv6 AWS IMDS 网段 fd00:ec2::/64 在严格模式下漏网的问题。

### 验证
24 个单元测试全通过(UrlSafetyChecker 21 + BrowserProperties 3),覆盖严格/豁免两模式 + 4 个 check 重载 + IPv6 IMDS 网段。

### 风险
开关仅作用于浏览器工具;公网部署务必保持 false。

* feat(browser): Playwright 超时可配 + snapshot 智能截断与 selector 作用域

### 背景
Agent 用浏览器工具访问慢链路或大页面(超大表格)时遇到两类问题:

1. Playwright 默认 30s 超时不够用,且无法配置
2. snapshot 全量抓取页面文本,硬截断在 20000 字符处会切在元素中间,LLM 拿到残缺数据且不知道有截断
### 方案
新增三个 .env 开关(默认值与改动前完全一致):

开关 作用 PLAYWRIGHT_DEFAULT_TIMEOUT_SECONDS 单次操作超时(click / fill / waitForLoadState) PLAYWRIGHT_NAVIGATION_TIMEOUT_SECONDS 导航超时(page.navigate / load-state) PLAYWRIGHT_SNAPSHOT_MAX_LENGTH snapshot 文本截断长度

snapshot 改造:

- 支持 selector 参数作用域到子树(之前只用于 click/type)
- budget 机制按元素边界智能截断,不再切在 <td> 中间
- 返回 truncated:true + hint 引导 LLM 用 selector 重抓
- JSON 字段顺序优化:truncated/hint 放 content 前,确保框架 spill preview(head 800 chars)能切到
### 验证
- 28 个单元测试全通过(BrowserPropertiesTest 7 + UrlSafetyCheckerTest 21)
- IDE 诊断 0 错误
- 覆盖:默认值不变 + setter 往返 + 严格/豁免两模式
### 兼容性
- 默认值保持 30s / 30s / 20000,行为与改动前完全一致
- selector 参数本就是 @ToolParam(required=false) ,LLM schema 无变化,只是描述更新引导 snapshot 场景也能用
- 不影响 webhook / image download 等其他 SSRF 守卫

## 改动汇总
文件 改动 BrowserProperties.java +3 字段: defaultTimeoutSeconds / defaultNavigationTimeoutSeconds / snapshotMaxLength (默认 30/30/20000) BrowserLauncher.java 抽 applyContextDefaults(context) 在三处 context 创建点调用;补全 setIgnoreHTTPSErrors 在 wrapLocalBrowser 落地 BrowserUseTool.java 工具描述 + selector 描述引导 LLM 在 snapshot 场景用 selector;doSnapshot 改造支持 selector 参数 + budget 智能截断 + JSON 字段顺序(truncated/hint 放 content 前确保 spill preview 能切到) docker-compose.yml +3 开关: PLAYWRIGHT_DEFAULT_TIMEOUT_SECONDS / PLAYWRIGHT_NAVIGATION_TIMEOUT_SECONDS / PLAYWRIGHT_SNAPSHOT_MAX_LENGTH .env.example +3 开关,简短说明 BrowserPropertiesTest.java +4 测试覆盖新字段默认值和 setter

## 测试结果
## 数据流论证的关键决策
决策 依据 truncated:true 和 hint 放在 JSON content 字段之前 ToolResultStorage.buildPreview 会把 >8000 chars 的结果截到 head 800 chars,放在前面确保 LLM 看到 selector 参数描述从 "for click/type" 改为明确说 "OPTIONAL for snapshot" LLM 读 JSON schema 时按描述判断参数用途,原描述误导 LLM 不在 snapshot 用 selector 截断从硬 substring(0, N) 改为 JS budget 机制递归累计 避免切在 <td>订单号 ABC 中间,截断发生在 TEXT_NODE 完整段或下一个子元素开始之前 用 ElementHandle.querySelector + evaluate 替代字符串拼接 selector 进 JS 防止 selector 注入(selector 含特殊字符如引号、反斜杠) 三处 context 创建点统一调 applyContextDefaults 确保 CDP / external-CDP / 本地 launch 三条路径都应用配置的 timeout

## 临时改动还原
文件 改动 还原状态 mateclaw-server/pom.xml 临时加 maven-compiler-plugin + Lombok annotation processor  已删除,恢复原始状态

## 未测项
项 原因 doSnapshot 的 JS budget 逻辑 需启动真 Playwright + 大页面,单元测试范围外 BrowserLauncher.applyContextDefaults 是否真的影响 page.click 行为 同上,集成测试范围 LLM 是否真的会按 hint 用 selector 重调 取决于 LLM 推理能力,需端到端测试
2026-07-02 09:25:16 +08:00
倪程伟
a1221ac02d feat(mcp): type on-behalf-of identity by channel/trust (#459)
The identity forwarded to opt-in MCP servers was a one-dimensional string
(ChatOrigin.requesterId): a MateClaw username for web logins, but a webchat
visitorId for visitors and an IM sender id for IM — indistinguishable to the
REST backend. The signed-token mode (d204b702) made this worse: an RS256
signature over an unauthenticated visitorId reads as "MateClaw authenticated
this user" to any backend that trusts the signature.

Introduce an identity-typing dimension at McpIdentityForwardService:

- classify() branches on ChatOrigin: authenticated (web login, sub=immutable
  userId), anonymous (webchat visitor, trust=anonymous), external (IM sender,
  trust=external), or none (cron/system → nothing injected, fail-closed).
- mint() adds `trust` and `channel_type` claims; plaintext value is prefixed
  `trust:subject` so backends can tell the kinds apart without a JWT.

The immutable userId reaches resolve() without coupling it to the user store:
JwtAuthFilter stamps user.id into auth.setDetails() (both JWT and PAT paths),
and ChatController.memoryOrigin carries it on a new ChatOrigin.requesterUserId
field (only-add, per the record's evolution rule).

Resolves the webchat semantic mismatch raised in #459 and the "sub should be
an immutable user id" follow-up. 82 tests green (4 identity classes covered
with claim assertions + full ChatOrigin/MCP regression).

(cherry picked from commit b5d2cfbf98b39848d7139c743a0b81fea71e8ffe)
2026-07-01 19:02:49 +08:00
matevip
fcd682e4b4 test(mcp): import Set/Map instead of inline FQN in identity-forward tests
Replace java.util.Set.of / java.util.Map.of inline fully-qualified calls with
top-of-file imports per code style (test sources sync to the open-source repo).
2026-07-01 18:53:40 +08:00
倪程伟
758bdbb94b
feat(mcp): STDIO MCP server 透传认证用户身份(opt-in per server) (#460)
* feat(mcp): forward authenticated user identity to opt-in STDIO MCP servers

A STDIO MCP server is one shared subprocess per configuration; its env is fixed
at spawn and STDIO has no per-request header channel, so per-user identity must
travel in-band with each tool call. Previously nothing carried it, so an MCP
server could not call its downstream REST backend on behalf of the acting user.

Inject the authenticated username (from ToolExecutionContext) into each tool
call's JSON arguments under the reserved key `__mateclaw_user__`, for servers an
operator explicitly opts in via `mateclaw.mcp.identity-forward.servers` (by name
or id). The MCP server reads/strips it and forwards on-behalf-of alongside its
own backend API key.

- McpIdentityForwardProperties: per-server opt-in allowlist (name or id).
- IdentityForwardingToolCallback: wraps an MCP callback, merges the username
  into the args; injected by trusted code, overwrites any LLM-supplied value
  (no spoofing); forwards unchanged when there is no user or args aren't an
  object/are malformed.
- McpClientManager: captures server names; wraps opt-in servers' callbacks
  inside the prefix wrapper (so name-prefixing / return-direct still see the raw
  delegate). Non-opt-in servers are untouched — username never leaks to them.
- Tests: injection, LLM-value overwrite, empty/non-object/malformed inputs,
  no-user passthrough, opt-in matching by id/name.
- Docs (zh/en mcp.md): opt-in config, `__mateclaw_user__` contract, FastMCP
  Python skeleton, trust model.

Default off (empty allowlist) — zero behavior change for existing servers.
Plaintext username suits a trusted-network REST backend keyed by an API key;
a signed short-lived token is noted as the stronger-isolation follow-up.

* feat(mcp): add signed-token trust model for MCP identity forwarding

Plaintext username forwarding makes the REST backend trust an unverifiable
assertion from the (shared, LLM-adjacent) MCP service — a confused-deputy model.
Add an opt-in signed-token mode so identity crosses the trust boundary as a
short-lived RS256 JWT the backend can verify with a public key.

- McpIdentityForwardProperties: nested `token` config (enabled, issuer,
  ttl-seconds, key-id, private-key-pem, audiences) + USER_ARG/TOKEN_ARG keys.
- McpIdentityForwardService: resolves the injection — plaintext username
  (__mateclaw_user__) when token mode off, else a minted RS256 JWT
  (__mateclaw_token__) with sub=user, aud=server, short exp, jti. Lazy key
  parse; fail-closed when token mode is on but the key is missing/unparseable
  (no silent downgrade to plaintext). Signs with MateClaw's private key so the
  backend only needs the public key (cannot mint/impersonate).
- IdentityForwardingToolCallback: now delegates the what-to-inject decision to
  the service (keyed by per-server audience); static withClaim() keeps the
  JSON-merge logic (overwrites LLM-supplied key, leaves non-object/malformed
  args untouched).
- McpClientManager: injects the service; passes service + audience through the
  wrap path for opt-in servers only.
- Tests: token mint+verify (with an in-test RSA keypair, asserting sub/aud/iss/
  exp/jti), plaintext mode, no-user and no-key fail-closed, audience resolution.
- Docs (zh/en): token config, key generation, claims, REST-side verification
  example, public-key distribution + JWKS-endpoint follow-up.

Default unchanged: token.enabled=false → plaintext (back-compat); whole feature
still opt-in per server and off by default.
2026-07-01 18:51:08 +08:00
倪程伟
3ac73623ee
fix(memory): bound mate_memory_recall.filename to VARCHAR(256) (#461) (#463)
mate_memory_recall.filename is VARCHAR(256), but the snippet-level recall
tracker assembles the key as `path + '#' + H2-heading-slug`. When the LLM
writes an over-long daily-note heading (the summarize prompt placed no
length cap on the `##` title), the CJK-preserving slug pushes the filename
past the column, and writes fail with Data too long / string too long.

Three layers of defence, root cause + hard caps:

1. prompt (source) — summarize-system.txt now asks for short (≤30 chars)
   `##` titles; details go in the body, not the heading.
2. slug cap (close to source) — MemoryRecallTracker.sanitizeSectionKey
   caps the slug at MAX_SECTION_SLUG=200, leaving path+'#' well under 256.
3. write-side cap (catches every path) — MemoryRecallService.recordRecall
   truncates filename to MAX_FILENAME_LENGTH=255 at the entry point, so
   the select/insert/update branches share one value and the dup-key
   concurrency fallback still matches. Covers trackActiveRetrieval too,
   which bypasses sanitizeSectionKey.

Tests: MemoryRecallFilenameTruncationTest covers both caps (over-long CJK
heading, normal heading untouched, ascii slug, date prefix survives) plus
an end-to-end assertion that the stored value fits VARCHAR(256). Existing
memory-suite unit tests still green.
2026-07-01 18:42:20 +08:00
matevip
fa4e7018a0 feat(delegation): 本轮 token 总量页脚 + 子 Agent 用量向上滚加
- 新增 DelegatedUsageAccumulator:按根会话累加每个完成子 Agent 用量,根 Agent
  在 _usage_final 处一次性 drain 整棵子树、中间层得 0,每个子只计一次、无重复计数
- runSingleChild 增 accumulateToParent:同步/并行/计划步骤委派计入父轮,游离异步不计
- ReAct / Plan-Execute 在 _usage_final 处 drain 并加进 token + 附委派分解字段,
  doFinally 清理防泄漏;该事件同时驱动实时 SSE 与 mate_message 落库,实时/刷新一致
- 前端消息底部新增 Σ<total> tok 徽标(tooltip 含委派分解),总量仅取 message 用量
- 测试:补 DelegatedUsageAccumulator 接线,委派全套 59 绿
2026-06-30 17:21:10 +08:00
matevip
6854f8cc44 feat(tool): surface execute_shell_command artifacts; fix download filename 2026-06-30 14:01:48 +08:00
matevip
5cf1c46dd4 feat(tool): surface files written by execute_code as one-click downloads 2026-06-30 11:31:14 +08:00
matevip
d390935763 feat(delegation): 子 Agent 成本透出 + 单任务/计划步骤委派结构化
- 子执行改走 chatWithUsage,捕获并透出每个子 Agent 的 prompt/completion token
  (单任务回复、并行机读头+逐行、delegation_end/child_complete/单路 broadcastEnd 事件)
- 新增 delegateByAgentIdStructured 返回结构化 ChildResult;计划步骤委派改按
  success()/isBlank() 判定成败,替掉脆弱的错误前缀匹配
- 前端委派段与嵌套节点显示紧凑成本后缀/徽标
- 测试:子执行 stub 迁移到 chatWithUsage + 新增 token 回归用例
2026-06-30 11:06:19 +08:00
matevip
fb811b9dc1 test(kb-open): use static import for assertThat in KbOpenApiControllerTest
Replace the single inline org.assertj.core.api.Assertions.assertThat call
with the static import already used for assertThatThrownBy, per code style
(test sources sync to the open-source repo).
2026-06-30 09:40:50 +08:00
倪程伟
9d292a9893
feat(kb-open): P0-B 9 个开放 API 端点 (#445)
* feat(kb-open): P0-B 9 open API endpoints

Implements the 9 read-only KB Open API endpoints on top of the P0-A
auth skeleton (#441). Each returns an explicit DTO (A5: never raw
entities) and delegates assembly to service-layer methods that return
pure DTOs (A6: no HTTP coupling, MCP-ready).

Endpoints:
- GET  /pages/{slug}        entity card (mode=summary/full/section:{heading})
- POST /search              hybrid retrieval (granularity=entity/chunk)
- POST /search/chunks       chunk-level semantic search
- POST /pages/{slug}/traverse  entity relation graph (depth ≤ 2)
- GET  /pages/{slug}/trace  provenance (page → chunk → raw)
- GET  /taxonomy            pageType/entityType/relationType enumeration
- GET  /whats-new           recent changes + stale pages
- GET  /stats               KB statistics
- GET  /pages               lightweight page list

Components:
- KbOpenApiController: 9 endpoints, each @RequireKbScope annotated
- KbOpenApiService: assembly layer (card, traverse BFS, metadata parsing)
- KbOpenApiDtos: all response DTOs as records (PageCard, TraceResult,
  TaxonomyResult, KbStats, WhatsNewResult, TraverseResult, PageList)

Traverse (pragmatic version):
- depth ≤ 2 with explosion guard, predicate LIKE matching
- slug → pageId → mention → primaryEntity (salience-highest)
- neighbor nodes echo slug when available (R11)
- edge sourceHandle via evidenceChunkId → citing page

Tests (4 new, all green):
- KbOpenApiControllerTest: 404 on missing page/slug, delegation to service

Closes #442

* fix(kb-open): address review feedback on #445

BLOCKERS:
- stats.pagesWithLinks always returned 0 because listByKbId() nulls out
  content. Switch to listByKbIdWithContent() so [[wiki link]] detection works.
- Test file: replace inline java.util.List.of() FQN with import + simple name
  (sync-opensource would expose the unidiomatic style).

NITS (inherited from P0-A rebase):
- V162→V164, prefix VARCHAR(12), FQN imports, parseScopes trim, ?token=
  fallback removal, design doc moved to rfcs/ — all now in ancestor commit
  6fd62440.

EXTRA:
- whatsNew staleReason: hardcoded Chinese "上游 fact 页面变更" → English
  "Upstream fact page changed" (external-facing API response).

* chore(wiki): drop RFC-012 prefix from progress field Javadocs (#449 nit)

Per #449 review (4825113234): the internal RFC-012 reference should not
appear in code. progressPhase/progressTotal/progressDone Javadocs still
carried the "RFC-012 M2 v2 UI:" prefix after #449's English translation
pass — drop it now that these lines are touched.

Zero behavior change.

* chore(kb-open): drop inline FQN in parseScopes (#444 nit)

Per #444 review (4825157096): parseScopes used
`.collect(java.util.stream.Collectors.toUnmodifiableSet())` while
`Collectors` is already imported at the top of the file. Use the simple
name. Zero behavior change.
2026-06-30 09:36:55 +08:00
matevip
bf86b1f737 test(memory): regression test for session_search concurrent-session isolation
@SpringBootTest + H2 coverage asserting that both listRecent and search exclude
a still-running sibling conversation (stream_status='running') and the caller's
own current conversation, so concurrent sessions of the same agent cannot leak
into each other's session_search results.
2026-06-30 09:30:57 +08:00
matevip
0c7ea8d563 docs(memory): clarify session_search conversation-id source in English
Translate the inline comment on the ToolContext-derived conversation id to
English per code style; no behavior change.
2026-06-30 09:25:29 +08:00
MIST
dcc8c9aed3
修复同一 Agent 多并发会话记忆混乱问题 (#458)
### 问题现象
同一 agent 开多个并发会话时(如 A1=查南京天气、A2=查北京天气),A2 在多轮 ReAct 执行中会"突然去查南京天气",表现为 A1 会话的上下文泄漏到 A2 会话。

### 根因4:Agent 实例共享 + state 覆盖(确认,仅状态显示问题)
确认点:
- AgentService.java:83-90 agentInstances 按 (agentId, modelKey) 缓存,不含 conversationId
- AgentService.java:598-624 getOrBuildAgentForConversation 只按 (agentId, provider, model) 解析,不按 conversationId
- AgentService.java:530-545 withLifecycleFlux 无锁 ,A/B/C 完全并发
影响: A 完成设 IDLE → B 仍在运行但显示 IDLE → 状态显示错乱。 不会直接导致记忆串台 ,但对用户可见。

## 二、根因与症状匹配度总结
根因 匹配度 触发条件 串台通道 1. SessionSearchTool ★★★★★ LLM 多轮遇到困难时主动调用 session_search 返回并发兄弟会话消息 2. 审批重放无过滤 ★★★☆☆ Plan-Execute + 审批 + 并发 awaiting_approval 误取兄弟会话计划 3. 结构化记忆共享 ★★★☆☆ A 会话 LLM 主动 remember_structured 写入 prefetch 注入到 B 会话 4. Agent 实例共享 ★★☆☆☆ 任意并发 state 显示错乱(非记忆串台)

用户描述的"B突然去查南京天气"最可能是根因1 ——因为 system prompt 明确引导 LLM 在遇到困难时用 session_search 回忆历史,而 SQL 会返回并发兄弟会话的"南京天气"内容。

## 三、修复方案(按优先级排序)
### 方案1:修复 SessionSearchTool(最优先,直接命中症状)
改动点 A — SessionSearchTool 增加 ToolContext 参数,强制读取真实 conversationId:

SessionSearchTool.java:37-44

改动点 B — SessionSearchService 增加运行状态过滤,排除并发兄弟会话:

SessionSearchService.java:60-73 SQL 增加:

或更保守:只返回 status = 'completed' 的会话,排除 running / awaiting_approval 的并发会话。

风险评估: 改动 SQL 查询条件,不影响写入逻辑。 completed 会话才是真正的"历史对话", running 会话是"正在进行"不应被搜索。功能上合理。

### 方案2:修复审批重放跨会话取计划(确凿 bug,必须修)
改动点 A — PlanningService.findAwaitingApprovalContext 增加 conversationId 参数:

PlanningService.java:228-232

改动点 B — 调用方传入 conversationId:

StateGraphPlanExecuteAgent.java:100

风险评估: 需确认 PlanEntity 有 conversationId 字段(从之前排查看应存在)。改动最小,仅加查询过滤,不影响其他逻辑。

### 方案3:结构化记忆引入会话级隔离(改动较大,需评估)
问题: 当前 ownerKey = user:<requesterId> ,3 会话共享。如果改为 conversation:<conversationId> ,会破坏"用户长期记忆跨会话共享"的设计意图(用户画像、偏好等应跨会话)。

建议方案: 不改 ownerKey 机制,而是在 StructuredMemoryTool.remember_structured 的 system prompt 说明中 明确限制 只记住"长期有效的事实",临时任务结果(如天气查询)不应写入。或在 type 枚举中新增 transient 类型,该类型按 conversationId 隔离,会话结束自动清除。

风险评估: 改动较大,涉及记忆分层设计。建议作为中长期优化,本次先修方案1和2。

### 方案4:Agent 实例 state 按 conversationId 隔离(可选)
改动点: BaseAgent.java:33 AtomicReference<AgentState> state 改为 Map<String, AtomicReference<AgentState>> (按 conversationId)。

风险评估: 影响所有 getState() / setState() 调用点,改动面广。且这只是状态显示问题,不影响记忆串台。建议暂不修,或在前端按 conversationId 单独查询状态。

--------------------------------------------
本次完成bug1、2修复;3、4未动
2026-06-30 09:23:26 +08:00
matevip
181e81a236 fix(mcp): cascade-delete agent-tool bindings when an MCP server is removed 2026-06-29 15:03:09 +08:00
matevip
421fd3cd61 fix(llm): stop assuming DeepSeek is vision-capable 2026-06-29 14:48:44 +08:00
matevip
07d6f01b56 fix(wiki): make built-in transformation starter pack visible in every workspace 2026-06-29 14:35:09 +08:00
matevip
64b5587f56 feat(wiki): route cheap ingest steps to a configurable light model 2026-06-29 14:10:57 +08:00
matevip
d512643960 feat(tool): configurable SSRF allowlist for outbound HTTP guards 2026-06-29 10:33:38 +08:00
mateaix
83660893a6 fix(chat): restrict generated-file link regex to http(s)/relative URLs
Follow-up to #447. The generated-file link extraction accepted any
non-')' text before the path, so a paren-free javascript:/data: URL
embedding /api/v1/files/generated/<id> could be captured and bound to an
<a href>, enabling XSS on click. Adopt the scheme-restricted pattern
already used by SegmentSupersedeDetector and the channel adapters, on
both backend (ChatController) and frontend (useChat). Also replace the
inline fully-qualified Pattern/Matcher with imports and drop an unused
run-overview i18n key.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-28 16:25:11 +08:00
jack
3c6c765e51
feat(chat): surface generated-file artifacts in the run-overview rail
Extract generated-file download links from tool results — on the backend (persisted to message metadata for history) and on the frontend (live during SSE) — de-duplicate by URL, and render them as a Generated Files section with file-type icons and a rail badge.
2026-06-28 16:06:06 +08:00
倪程伟
2d04ce92ef
feat(kb-open): P0-A open-API auth — API keys, rate limit, centralized authorization
Hashed API-key auth (SHA-256, plaintext shown once), per-key sliding-window rate limit, and a fail-closed filter + scope/KB-binding interceptor enforcing empty-binding=zero-access. Admin CRUD for key lifecycle. Migration V164 across h2/mysql/kingbase.
2026-06-28 14:45:53 +08:00
倪程伟
2f46619e5b
fix(wiki): close IDOR in WikiRelationController & WikiEntityController (cross-KB id binding)
Every endpoint now binds its independent id param to an authorized KB: rawId/chunkId resolve-then-workspace-check, pageId is asserted to belong to the path kbId, and slugs stay kbId-scoped. Adds unit tests for same-KB/cross-KB/unknown cases.
2026-06-28 14:41:19 +08:00
倪程伟
04197d7ba9
chore(wiki): address #437 review nits (import convention + English Javadoc)
Pure style cleanup, zero behavior change: replace inline FQN return type in WikiRawMaterialService.listFailures with an import + simple name, and translate the new WikiRawMaterialEntity field Javadocs to English.
2026-06-28 14:24:19 +08:00
倪程伟
fbbd1218e8
feat(wiki): KB processing-failure visibility (error-code chain + silent sub-step alerts + cross-KB failure center)
Propagates structured error codes through the KB processing pipeline, surfaces silent sub-step warnings as a non-failure warning state, and adds a cross-KB failure center for aggregated visibility.
2026-06-28 13:07:34 +08:00
倪程伟
7be8f81353
feat(agent): per-employee model-chain preference (provider + model, repeatable provider)
Lets an employee pin an ordered fallback chain of (provider, model) entries; the same provider may appear multiple times with different models. Build-time dedup keys on exact (provider, model).
2026-06-28 13:05:48 +08:00
matevip
8522aa7591 feat(tool): desktop local file/shell tools via WebSocket tunnel 2026-06-26 18:24:25 +08:00
matevip
40cb39fb3b test(wiki): cover StageInstructions string/object deserialization
Add unit coverage for the StageInstructions custom deserializer: plain-string
shorthand, full object with instructions+template, unknown-field skipping, and
both forms coexisting on one WikiPageTypeDef (backward compatibility).
2026-06-26 14:34:32 +08:00
Sharon
4e82185be7
fix(wiki): replace broken @JsonCreator with custom StdDeserializer for StageInstructions (#424)
Jackson treated the @JsonCreator factory method as a properties creator
(matching the 'instructions' parameter name to the JSON field), not a
string/delegating creator, so plain-string values still failed at runtime
with "no String-argument constructor/factory method".

Replace with @JsonDeserialize + StdDeserializer that explicitly checks
VALUE_STRING vs START_OBJECT tokens, handling both shorthand strings and
full {instructions, template} objects.
2026-06-26 14:33:32 +08:00
matevip
26dd8a37d5 fix(workspace): harden chat-upload base-path containment + cleanup guard
- resolveAgentBasePath: the relative-override branch now normalizes the
  resolved path and rejects values that escape the workspace root via "../"
  (the absolute branch already did this), keeping attachment/media/tool I/O
  contained when an agent's workspaceBasePath is a relative override.
- cleanAttachmentFiles: return early on a null/blank conversationId so a bare
  upload root can never be walked and deleted wholesale.
- Translate the chat-upload Javadoc/comments to English (cleanAttachmentFiles,
  BaseAgent image-path resolver) per code style.
- Add a resolver test for the relative-override escape fallback.
2026-06-26 14:25:56 +08:00
倪程伟
fcb488c567
feat(workspace): chat-uploads 上传目录工作空间/Agent 感知化 (#422)
* feat(workspace): chat-uploads 上传目录工作空间/Agent 感知化 (#421)

把硬编码的 data/chat-uploads/{conversationId}/ 改为按工作空间/Agent 解析,
解析优先级:Agent workspaceBasePath → Workspace basePath → 可配置默认目录
(新配置 mateclaw.chat.upload.base-dir,默认 data/chat-uploads,保持现网零行为变化)。

- 新增 ChatUploadLocationResolver 中央解析器:写路径返回唯一根,读/清理
  路径返回候选根列表(工作空间根 + 默认根)做双重查找,保证迁移前旧附件
  仍可解析/清理;conversationId→ConversationEntity 查询带 5min 缓存。
- 新增 ChatUploadProperties + ChatUploadAutoConfiguration(启动建目录)。
- 复用 AgentGraphBuilder.resolveAgentBasePath(提升为 public)的优先级与
  安全规则(相对路径在 workspace 根下解析,绝对路径逃逸被拒)。
- 所有写入/读取点改为走 resolver;读取点走双重查找。
- 解决 Spring 循环依赖(resolver → agentService → ... → conversationService
  → resolver):resolver 的 AgentService 注入加 @Lazy。

向后兼容:默认目录不变;双重查找覆盖历史消息里的相对路径;
服务端点 URL 契约不变,前端无需改动。

测试:新增 ChatUploadLocationResolverTest(8 用例);修复受影响的现有测试构造。

* refactor(workspace): address review findings on chat-uploads resolver (#421)

应用 code review 的 4 项修复:

1. (correctness) ChatUploadLocationResolver 缓存新增 ConversationDeletedEvent
   监听器,删除会话时立即失效 conversationId→ConversationEntity 映射。
   否则备份恢复后用相同 id 重建会话,会继承最长 5 分钟的过期 workspace/agent
   映射,导致 cleanAttachmentFiles 走错(过期的)上传目录。复用既有
   @EventListener-on-bean 模式(与 AsyncTaskService / WorkspaceLookupCache 一致)。

2. 收紧 resolveWorkspaceScopedRoot 里 3 个过宽的 catch(Exception) →
   MateClawException + warn,让真正的 bug(NPE / DataAccessException)暴露
   而非被静默降级为 debug 日志。

3. 更新 ChatController.upload 过期注释:会话尚未创建时附件暂存默认目录,
   会话创建后读取走双重查找仍能命中。

4. 移除不可达分支(agentWorkspaceId != workspaceId)—— 会话的 agent 必然
   归属会话的 workspace(创建时强约束),直接用会话 workspace 即可,
   少一次冗余 DB 查询与一层推测性逻辑。

测试:ChatUploadLocationResolverTest (8) + ConversationServiceCleanAttachmentFilesTest (2) 全绿。
2026-06-26 14:20:54 +08:00
matevip
7705778903 fix(agent): normalize replayed tool-call arguments to valid JSON (#410)
Strict OpenAI-compatible providers reject the /chat/completions request
with HTTP 400 when an assistant message in history carries a tool call
whose function.arguments is not parseable JSON. Normalize blank or
non-JSON arguments to "{}" at the send chokepoint so streaming,
history-replay, and older-persisted tool calls all stay well-formed.
2026-06-26 11:19:19 +08:00
matevip
e0ce5e2ea7 fix(sso): gate SSO runtime beans on mateclaw.sso.enabled
SsoService / SsoStateService / SsoController / SsoProviderRegistry were
unconditional component-scanned beans, but their configuration
(SsoProperties) is only registered by the conditional auto-configuration.
With SSO disabled (the default) the services were still instantiated and
startup failed: "required a bean of type SsoProperties that could not be
found". Gate the four beans on the same mateclaw.sso.enabled=true
condition so the SSO stack loads as a unit — disabled = no beans and no
exposed endpoints; enabled = the auto-configuration provides SsoProperties
and everything wires.

Verified: backend starts clean with SSO disabled (default).
2026-06-26 10:27:16 +08:00
matevip
b048718298 fix(sso): inline FQN→import + harden auto-create orphan rollback
- Replace inline fully-qualified names with top-of-file imports across
  SsoService / SsoStateService / FeishuSsoProvider (ObjectMapper, Autowired,
  Map.of, Date, DuplicateKeyException, Mac, URLEncoder) per code style.
- createSsoUser: roll back the freshly inserted user on any non-duplicate
  identity-insert failure, preventing passwordless orphan accounts. The two
  inserts share no transaction — the method is self-invoked and the enclosing
  callback performs a network call, so a method-level @Transactional would not
  apply; an explicit rollback in the catch is the correct guard here.
2026-06-26 10:09:43 +08:00
倪程伟
03a6d61131
feat(sso): 飞书 OAuth2 单点登录 (ISSUE #405 P0) (#419)
* feat(sso): feishu OAuth2 single sign-on (ISSUE #405 P0)

Implements the SSO design (ISSUE #405) with feishu as the first IdP
and a generic OAuth2 provider abstraction for future dingtalk/wecom
extensions. SSO is disabled by default — existing deployments are
unaffected until mateclaw.sso.enabled=true.

Backend:
- SsoProvider interface + SsoUserInfo record: generic IdP abstraction
- FeishuSsoProvider: OAuth2 authorization-code flow (app_access_token
  with Caffeine cache → user_access_token → user info). apiBase switches
  between feishu.cn / larksuite.com by domain config.
- SsoProviderRegistry: conditional registration, lists enabled providers
- SsoStateService: HMAC-signed OAuth2 state + self-contained bind_token
  JWT, both persisted to sso_state DB table for multi-node correctness.
  State is one-time-consumable (conditional UPDATE), bind_token jti
  anti-replay via PK insert. Hourly ShedLock purge (LambdaQuery + Java
  time, works on all 3 dialects).
- SsoService: authorize/callback/bind, user mapping (union_id first →
  external_id fallback), auto-create with concurrent idempotency
  (DuplicateKeyException → rollback orphan user → re-query), link-only
  mode issues bind_token for existing-account binding.
- SsoController: 4 endpoints (/providers, /authorize, /callback, /bind)
  all permitAll.
- V159 migration (h2/mysql/kingbase): mate_user_external_identity,
  sso_state, ALTER mate_user.password NULL (SSO-only users).
- AuthService: generateToken promoted to public; login() guards
  password=null (SSO-only users cannot password-login).
- SecurityConfig: /auth/sso/** added to permitAll whitelist.
- LoginRateLimitFilter: expanded to cover /auth/sso/bind (brute-force
  surface equivalent to /auth/login).
- application.yml: mateclaw.sso.* config block (all env-var driven).

Frontend:
- Login.vue: dynamic SSO buttons (only shown when providers configured),
  OAuth2 callback detection (?sso=callback), link-only bind dialog,
  shared applyLogin flow (localStorage + workspace + route).
- api/index.ts: ssoApi (providers, authorize, callback, bind).

Tests: SsoStateServiceTest (11) — state issue/verify/replay/tamper,
bind_token issue/verify/anti-replay/garbage. Regression: PAT (23) +
Approval resolve (13) all green.

Not in scope (P1/P2): link-only bind/unbind management endpoints,
user enable/disable endpoint, dingtalk/wecom providers, admin SSO
config page. Workspace assignment for auto-created users remains a
product decision (design doc §12 item 2).

* fix(sso): self-review fixes — P0 security + P1 quality

P0-1 BindRequired serialization: replaced the R.fail(200, Map.toString())
hack with a structured SsoCallbackResponse record. Controller no longer
catches an exception for a non-error path; frontend reads bindRequired
flag directly instead of regex-parsing a stringified map.

P0-2 createSsoUser unbounded recursion: added a retry flag — second
DuplicateKeyException (extreme race where identity was concurrently
deleted) now throws a 503 instead of recursing to stack overflow.

P0-3 state TTL not enforced: verifyState's conditional UPDATE now
includes created_at > cutoff, so a state unused for 5+ min is rejected
at consumption time, not just at the 1h purge. Without this the 5-min
window was advisory only.

P1-5 SsoStateService unused ObjectMapper: removed dead injection.

P1-6 audit JSON string concat: replaced with ObjectMapper serialization
(provider/externalId no longer risk breaking the JSON structure).

P1-7 LoginRateLimitFilter shared counter: documented the intentional
decision that login + bind share a per-IP counter (same brute-force
surface) with guidance on switching to per-path if finer isolation
is needed.
2026-06-26 10:00:31 +08:00
matevip
d7d409245b docs: feature-page coverage for 1.7.0 (operational export, desktop remote, webchat approval, run overview, workflow notify) 2026-06-25 17:26:33 +08:00
matevip
6ecec63098 docs(release): add v1.7.0 changelog entry 2026-06-25 17:07:11 +08:00
倪程伟
2f12c269f4
feat(webchat): API-Key 渠道补齐审批 resolve + replay (ISSUE #413 P1) (#415)
* feat(webchat): add approval resolve + replay for API-Key channel (ISSUE #413 P1)

Before this, a WebChat (API-Key) channel that hit a ToolGuard-protected
tool parked the turn in a pending approval the visitor could never
clear — it hung for 30 min until the GC timeout and the turn was
wasted. This PR closes the loop, mirroring the web ChatController.

A1 — no code change. tool_approval_requested already reaches the SDK
via ToolExecutionGuardHelper's streamTracker.broadcastObject (direct
SSE push, bypassing the StreamDelta path). Adding it to
forwardVisitorEvent would double-deliver; the default-drop is correct.

A2 — new /sessions/approve and /sessions/deny REST endpoints. Auth is
the existing visitorToken + conversationId ownership guard; the actor
is webchatUsername(visitorId), which resolves the 'no MateClaw
username' blocker noted in the old stopSession javadoc. Both broadcast
tool_approval_resolved so the SDK clears its banner in real time.

A3 — approve returns an SSE stream: resolveAndConsume (atomic DB +
metadata + memory), restoreChatOrigin (recovers the webchat origin
captured at createPending), then chatWithReplayStream replays the
tool call and continues the turn. Replay may re-trigger approvals,
which the existing tool_approval_requested direct push handles.

A4 — stopSession now sweeps pending approvals (denyAllByConversation)
and broadcasts each resolution, so stopping a stream no longer leaves
approvals lingering for the GC.

Tests: WebChatApprovalInteractionTest (7) — deny resolves + broadcasts,
deny auth/ownership guards, idempotent unknown-pending, stop sweep
clears pending, stop no-op when nothing pending.

Regression: WebChatStopStreamTest (5), WebChatArchivePinTest (6),
WebChatSchemaFieldsTest (5), WebChatWikiPageListTest (8),
ApprovalWorkflowServiceResolveTest (13), GcTest (7), RecoveryTest (7).

* fix(webchat): IDOR guard + SSE hang fix (PR #415 review)

Addresses all review feedback from mateaix:

P0 IDOR (security): /sessions/approve and /sessions/deny accepted a
client-supplied pendingId without cross-checking it belonged to the
caller's conversation. A visitor could resolve / replay another
visitor's guarded tool call. Fix: getPending(pendingId) then assert
conversationId matches before resolving. Added getPending delegate on
ApprovalWorkflowService so the webchat controller (which holds the
workflow facade) can do the precise lookup.

SSE hang: approveSession's already-resolved / error branches broadcast
'done' before streamTracker.register/attach, so the event had no
subscriber and the SSE hung to the 10-min timeout. Fix: register+attach
first, then resolveAndConsume. Removed the now-duplicate register/attach
in the replay branch.

Tests: +2 IDOR cases (cross-visitor pendingId rejected 404; mismatched
pendingId rejected 404). denyResolvesPending now asserts via getPending
(findPendingByConversation returns the earliest pending, polluted by
cross-test map state). denyUnknownPendingIsSafe updated to expect 404
(no longer leaks pendingId existence). Isolated IDOR victim/attacker
visitor IDs to avoid cross-test conversationId collisions.

Regression: WebChatStopStreamTest (5), WebChatArchivePinTest (6),
ApprovalWorkflowServiceResolveTest (13), GcTest (7), RecoveryTest (7).

* style(webchat): use simple ChatOrigin name in approveSession (PR #415 review)

Reviewer flagged fully-qualified inline types (ResolveOutcome was fixed
in the prior commit; ChatOrigin was missed). Add the import and switch
the 3 FQN references in approveSession to the simple name, matching the
ResolveOutcome cleanup. chatStream's pre-existing FQN usages are out of
this PR's scope and left untouched.
2026-06-25 11:18:52 +08:00
倪程伟
b478eef78c
feat(im): resolve workflow approvals via feishu/wecom card clicks (ISSUE #413 P2-B3) (#416)
Before this, a workflow await_approval step whose approverChannels
pointed at feishu/wecom was effectively dead for IM interaction. Even
after PR #414 (B1) pushed the notice to the IM group, clicking the
card's Approve/Deny buttons did nothing useful:

- Identity check (requester==clicker) failed-closed: wf- approvals
  have userId=null (system-initiated), so every click was rejected.
- Even if it passed, the synthetic /approve injection was a dead end:
  the router routes by conversationId, but wf- ids use a synthetic
  workflow:run:{runId} key that no IM conversation matches, so
  findPendingByConversation returned null and the /approve was fed
  to the LLM as plain text.

B3 fix: both ToolGuardCardHandlers now detect the wf- prefix and
resolve inline (approvalService.resolve), bypassing the synthetic
injection entirely. The WorkflowApprovalResolvedEvent published
inside resolve is picked up by ApprovalResumeBridge (activated in
PR #414 B2), which resumes the paused run. This mirrors the Web /
WebChat resolve path (PR #415).

Identity policy: any audience member may resolve a wf- approval.
The card only reaches channels declared in await_approval's
approverChannels, so whoever sees it is a designated approver.
Regular tool approvals keep the strict requester==clicker guard.

Tests:
- wecom ToolGuardCardHandlerTest: +2 wf- cases (inline resolve, no
  synthetic injection; already-resolved renders expired). Existing 6
  cases updated for the new 3-arg constructor.
- feishu FeishuCardDispatcherTest: updated for the new factory
  constructor signature.

Regression: ApprovalWorkflowServiceResolveTest (13), GcTest (7),
RecoveryTest (7), feishu dispatcher (4), button value (7),
renderer (3+3) — all green.
2026-06-25 09:56:36 +08:00