Commit Graph

16 Commits

Author SHA1 Message Date
mateaix
81c6f4aece fix(chat): gracefully terminate active tool runs
Stopping a conversation previously disposed the outer reactive stream without reliably cancelling synchronous tool callbacks running on worker threads. Shell commands, skill scripts, and Playwright sessions could therefore outlive the visible chat turn.

Propagate cancellation through run-scoped hooks, interrupt active tool execution and parallel batches, terminate subprocess trees, close per-conversation browser sessions, and wait briefly for final stream persistence before acknowledging Stop.

Expose an explicit interrupting state in the chat input to block duplicate stop clicks and show progress, with regression coverage for cancelling an in-flight synchronous tool callback.
2026-08-16 16:26:10 +08:00
matevip
ba0e4506b7 feat(browser): harden browser automation reliability 2026-08-12 06:02:36 -04:00
mateaix
60ea00dede feat(tool/browser): 无障碍树 ref 快照与按 ref 交互 + 真实浏览器隐私护栏 + 受控 CDP 逃生舱 2026-07-12 14:56:02 +08:00
MIST
9e33782b7d
feat(browser): 放开内网服务访问限制,新增局域网部署模式开关 (#472)
* feat(browser): 放开内网服务访问限制,新增局域网部署模式开关

### 背景
局域网部署时,Agent 用浏览器工具访问 http://192.168.x.x:port 等内网服务会被默认 SSRF 严格模式拦截。

### 方案
新增两个 .env 开关(默认 false,行为与改动前完全一致):

- PLAYWRIGHT_ALLOW_PRIVATE_NETWORK=true :放行本地回环和私有 IP,云元数据端点仍拦截
- PLAYWRIGHT_IGNORE_HTTPS_ERRORS=true :忽略 HTTPS 证书错误(自签证书场景)
顺带修复 IPv6 AWS IMDS 网段 fd00:ec2::/64 在严格模式下漏网的问题。

### 验证
24 个单元测试全通过(UrlSafetyChecker 21 + BrowserProperties 3),覆盖严格/豁免两模式 + 4 个 check 重载 + IPv6 IMDS 网段。

### 风险
开关仅作用于浏览器工具;公网部署务必保持 false。

* feat(browser): Playwright 超时可配 + snapshot 智能截断与 selector 作用域

### 背景
Agent 用浏览器工具访问慢链路或大页面(超大表格)时遇到两类问题:

1. Playwright 默认 30s 超时不够用,且无法配置
2. snapshot 全量抓取页面文本,硬截断在 20000 字符处会切在元素中间,LLM 拿到残缺数据且不知道有截断
### 方案
新增三个 .env 开关(默认值与改动前完全一致):

开关 作用 PLAYWRIGHT_DEFAULT_TIMEOUT_SECONDS 单次操作超时(click / fill / waitForLoadState) PLAYWRIGHT_NAVIGATION_TIMEOUT_SECONDS 导航超时(page.navigate / load-state) PLAYWRIGHT_SNAPSHOT_MAX_LENGTH snapshot 文本截断长度

snapshot 改造:

- 支持 selector 参数作用域到子树(之前只用于 click/type)
- budget 机制按元素边界智能截断,不再切在 <td> 中间
- 返回 truncated:true + hint 引导 LLM 用 selector 重抓
- JSON 字段顺序优化:truncated/hint 放 content 前,确保框架 spill preview(head 800 chars)能切到
### 验证
- 28 个单元测试全通过(BrowserPropertiesTest 7 + UrlSafetyCheckerTest 21)
- IDE 诊断 0 错误
- 覆盖:默认值不变 + setter 往返 + 严格/豁免两模式
### 兼容性
- 默认值保持 30s / 30s / 20000,行为与改动前完全一致
- selector 参数本就是 @ToolParam(required=false) ,LLM schema 无变化,只是描述更新引导 snapshot 场景也能用
- 不影响 webhook / image download 等其他 SSRF 守卫

## 改动汇总
文件 改动 BrowserProperties.java +3 字段: defaultTimeoutSeconds / defaultNavigationTimeoutSeconds / snapshotMaxLength (默认 30/30/20000) BrowserLauncher.java 抽 applyContextDefaults(context) 在三处 context 创建点调用;补全 setIgnoreHTTPSErrors 在 wrapLocalBrowser 落地 BrowserUseTool.java 工具描述 + selector 描述引导 LLM 在 snapshot 场景用 selector;doSnapshot 改造支持 selector 参数 + budget 智能截断 + JSON 字段顺序(truncated/hint 放 content 前确保 spill preview 能切到) docker-compose.yml +3 开关: PLAYWRIGHT_DEFAULT_TIMEOUT_SECONDS / PLAYWRIGHT_NAVIGATION_TIMEOUT_SECONDS / PLAYWRIGHT_SNAPSHOT_MAX_LENGTH .env.example +3 开关,简短说明 BrowserPropertiesTest.java +4 测试覆盖新字段默认值和 setter

## 测试结果
## 数据流论证的关键决策
决策 依据 truncated:true 和 hint 放在 JSON content 字段之前 ToolResultStorage.buildPreview 会把 >8000 chars 的结果截到 head 800 chars,放在前面确保 LLM 看到 selector 参数描述从 "for click/type" 改为明确说 "OPTIONAL for snapshot" LLM 读 JSON schema 时按描述判断参数用途,原描述误导 LLM 不在 snapshot 用 selector 截断从硬 substring(0, N) 改为 JS budget 机制递归累计 避免切在 <td>订单号 ABC 中间,截断发生在 TEXT_NODE 完整段或下一个子元素开始之前 用 ElementHandle.querySelector + evaluate 替代字符串拼接 selector 进 JS 防止 selector 注入(selector 含特殊字符如引号、反斜杠) 三处 context 创建点统一调 applyContextDefaults 确保 CDP / external-CDP / 本地 launch 三条路径都应用配置的 timeout

## 临时改动还原
文件 改动 还原状态 mateclaw-server/pom.xml 临时加 maven-compiler-plugin + Lombok annotation processor  已删除,恢复原始状态

## 未测项
项 原因 doSnapshot 的 JS budget 逻辑 需启动真 Playwright + 大页面,单元测试范围外 BrowserLauncher.applyContextDefaults 是否真的影响 page.click 行为 同上,集成测试范围 LLM 是否真的会按 hint 用 selector 重调 取决于 LLM 推理能力,需端到端测试
2026-07-02 09:25:16 +08:00
matevip
d512643960 feat(tool): configurable SSRF allowlist for outbound HTTP guards 2026-06-29 10:33:38 +08:00
matevip
68effa3f90 fix(tool): retry browser_use eval with async IIFE on top-level return 2026-05-24 22:57:30 +08:00
matevip
d1b7e76f5b fix(tool): support top-level await in the browser eval action 2026-05-19 16:53:18 +08:00
matevip
b40cbfb0a1 fix(tool): clean up EXTERNAL_CDP profile dir + Chrome subprocesses on session stop 2026-04-30 10:25:00 +08:00
matevip
1864801c90 fix(tool): browser_use Windows compat + stop LLM treating it as web search 2026-04-30 08:54:15 +08:00
matevip
4011050ceb feat(cron): channel delivery via ChatOrigin + Spring AI ToolContext
Replaces the prior ThreadLocal context plumbing with explicit Spring AI
ToolContext threading carried by an immutable ChatOrigin value object,
so a cron created from inside WeChat (or any IM channel) delivers its
results back to the originating channel.

Architecture
- ChatOrigin / ChannelTarget value objects + per-entry-point factories
  (ChannelChatOriginFactory in vip.mate.channel, CronChatOriginFactory
  in vip.mate.cron — symmetric, no cyclic deps).
- LocaleAwareToolCallback now forwards call(String, ToolContext) and
  getToolMetadata so the decorator chain cannot silently drop the origin.
- AgentService 6-method overhaul + ChatOriginHolder bridge into
  StateGraph buildInitialState which writes CHAT_ORIGIN; ActionNode +
  StepExecutionNode forward it to ToolExecutionExecutor.
- ToolExecutionExecutor builds ToolContext per call; 8/8 tools migrated
  (CronJobTool, WorkspacePathGuard, Video/Image/Browser/ReadFile/Music,
  DelegateAgentTool with parent-origin inheritance).
- CronJobRunner + CronJobLifecycleService 3-segment REQUIRES_NEW model
  (T1 startRun / no-tx runAgent / T2 finishRunAndPublish); ArchUnit
  pins CronJobRunner as @Transactional-free.
- CronResultDelivery Strategy + AbstractCronResultDelivery Template
  with SQL CAS idempotency on mate_cron_job_run.delivery_status —
  replaces the prior process-local Caffeine TTL, cluster-safe.
- CronJobCompletedEvent + @Async @TransactionalEventListener(AFTER_COMMIT);
  cronDeliveryExecutor (core=2, max=4, queue=1000, AbortPolicy + audit).
- CronRunStaleCleanup @Scheduled(5min) sweeps PENDING-15min and
  status='running'-30min in one query each.
- CronJobRunner.wrapWithDeliveryGuard prepends a system note for
  channel-bound crons to suppress hallucinated 'install CLI to send
  WeChat' suggestions.
- ApprovalWorkflowService Memento: persist ChatOrigin snapshot on
  create, restore on replay so cross-restart approvals keep channel
  binding; ChannelMessageRouter + ChatController web-replay both prefer
  the Memento and fall back to fresh-build.
- ChannelManager.sendToChannel 4-arg DeliveryOptions overload;
  ChannelAdapter#proactiveSend default 4-arg pass-through; Slack
  overrides for thread_ts and Telegram overrides for message_thread_id.
- CronJobs UI: read-only 'last delivery' badge driven by
  CronJobMapper.selectListWithDeliveryStatus subquery.

Schema migrations V57/V58/V59 (V56 was already taken by an unrelated
provider migration — Flyway processes versions in order regardless of
gaps):
- V57: mate_cron_job_run delivery_status / target / error + composite
       index (delivery_status, started_at) covering the cleanup sweep.
- V58: mate_cron_job channel_id (indexed) + delivery_config TEXT (JSON
       via MyBatis Plus JacksonTypeHandler).
- V59: mate_tool_approval chat_origin TEXT (Memento).
All idempotent in both H2 (IF NOT EXISTS) and MySQL (INFORMATION_SCHEMA
guard + PREPARE).

ArchUnit guards (test scope, archunit-junit5 1.3.0):
- every concrete vip.mate.* ToolCallback must override
  call(String, ToolContext) — pins the decorator-forward fix.
- CronJobRunner must NOT carry @Transactional on the class or any
  method — pins the 3-segment lifecycle rule.

Tests: 32 new unit tests + 21 regression tests in touched areas, all
53 green:
- ChatOriginTest (6) — value-object invariants + JSON round-trip.
- LocaleAwareToolCallbackToolContextTest (2) — decorator forward.
- DeliveryConfigTest (4) — Jackson round-trip + forward-compat.
- ToolCallbackToolContextForwardArchTest (2) — both ArchUnit guards.
- CronJobRunnerDeliveryGuardTest (3) — channel-cron prefix injection.
- AbstractCronResultDeliveryTest (4) — claim CAS + concurrent CAS.
- ChannelCronResultDeliveryTest (6) — supports / doDeliver / errors.
- ApprovalReplayContinuityTest (5) — Memento round-trip + corrupt
  payload fallback + unknown-field tolerance.

Refs: #25, #16
2026-04-28 21:43:58 +08:00
matevip
d17b06f454 fix(browser): serialize diagnose findings manually; Hutool cannot reflect on records 2026-04-24 21:38:25 +08:00
matevip
83567e95f0 fix(browser): multi-strategy launcher + self-diagnostics for win/linux 2026-04-24 21:37:46 +08:00
matevip
3d58a48eae feat(platform): add workspace foundation and channel execution upgrades 2026-04-09 10:29:16 +08:00
matevip
4a99382004 fix(browser): fix idle watchdog leak on repeated start/stop cycles 2026-04-06 12:01:09 +08:00
matevip
6756577428 fix(browser): add Windows and Docker Chromium launch args 2026-04-06 11:47:10 +08:00
matevip
579d60125b Initial commit: MateClaw — Java + Vue 3 AI Assistant System
Full-stack AI assistant built on Spring AI Alibaba.
Features: ReAct Agent, Plan-and-Execute, MCP Protocol, Multi-Model, Multi-Channel.

Apache-2.0 License
2026-04-04 19:03:49 +08:00