Commit Graph

10 Commits

Author SHA1 Message Date
mateaix
60ea00dede feat(tool/browser): 无障碍树 ref 快照与按 ref 交互 + 真实浏览器隐私护栏 + 受控 CDP 逃生舱 2026-07-12 14:56:02 +08:00
matevip
c3da1ce817 fix(browser): actually block redirect targets in SSRF interceptor
The first cut of the per-request interceptor called route.resume() for every
request. Playwright follows server-side 3xx redirects internally on resume()
WITHOUT re-invoking the route handler, so a public page that 302s to a
metadata IP still reached it — verified via runtime E2E (the handler only ever
saw the httpbin.org URLs, never the 169.254.169.254 redirect target).

Fix: for navigation requests, fetch with maxRedirects=0 and validate the
Location of each hop through UrlSafetyChecker before fulfilling; abort when a
hop resolves to a blocked host. Subresources/fetches keep the direct per-URL
check + resume path. Non-navigation and non-http(s) requests are unaffected.

Runtime-verified: httpbin.org 302 -> 169.254.169.254 is now aborted
(net::ERR_FAILED; log "blocked redirect ... cloud-metadata endpoint"), while
example.com and wikipedia.org (rich subresources) still load with no false
blocks.
2026-07-02 10:31:35 +08:00
matevip
c95df54949 harden(browser): re-check SSRF on every request + make metadata block unbypassable
Two SSRF hardenings on top of the private-network deployment mode:

1. Redirect / subresource re-validation. The SSRF guard previously ran only on
   the initial navigation URL in the tool layer, so a public page that 302s to
   169.254.169.254 (or a script fetch / img to a metadata IP) reached the target
   unchecked — worse now that private-network mode exists. Install a per-context
   request interceptor (BrowserLauncher.applyContextDefaults) that re-runs
   UrlSafetyChecker on every http(s) request and aborts blocked ones. Non-network
   schemes (data:/blob:/about:) pass through; unexpected checker faults fail open
   so a transient error cannot wedge the page (the initial URL was already checked).

2. Allowlist can no longer open a cloud-metadata endpoint. Metadata hostnames and
   IPs are now checked BEFORE the allowlist short-circuits, so an operator entry
   like 169.254.0.0/16 or metadata.google.internal can never expose instance
   metadata. Ordinary private-host allowlisting is unaffected (regression-tested).

Also correct the 192.0.0.192 comment (Oracle Cloud IMDS, not Azure).
2026-07-02 09:43:35 +08:00
matevip
b9f01db6f7 fix(browser): scope per-context TLS bypass to LAN mode
The per-context setIgnoreHTTPSErrors was gated on ignoreHttpsErrors alone,
while the Chromium command-line cert flags require both ignoreHttpsErrors AND
allowPrivateNetwork. Setting only PLAYWRIGHT_IGNORE_HTTPS_ERRORS therefore
disabled certificate validation for all browser traffic, including the public
internet (MITM exposure). Gate the per-context bypass on allowPrivateNetwork
too, so ignoring HTTPS errors is scoped to LAN deployments — matching the
command-line path and the documented intent.

Also correct a comment: 192.0.0.192 is Oracle Cloud's IMDS address, not Azure.
2026-07-02 09:27:11 +08:00
MIST
9e33782b7d
feat(browser): 放开内网服务访问限制,新增局域网部署模式开关 (#472)
* feat(browser): 放开内网服务访问限制,新增局域网部署模式开关

### 背景
局域网部署时,Agent 用浏览器工具访问 http://192.168.x.x:port 等内网服务会被默认 SSRF 严格模式拦截。

### 方案
新增两个 .env 开关(默认 false,行为与改动前完全一致):

- PLAYWRIGHT_ALLOW_PRIVATE_NETWORK=true :放行本地回环和私有 IP,云元数据端点仍拦截
- PLAYWRIGHT_IGNORE_HTTPS_ERRORS=true :忽略 HTTPS 证书错误(自签证书场景)
顺带修复 IPv6 AWS IMDS 网段 fd00:ec2::/64 在严格模式下漏网的问题。

### 验证
24 个单元测试全通过(UrlSafetyChecker 21 + BrowserProperties 3),覆盖严格/豁免两模式 + 4 个 check 重载 + IPv6 IMDS 网段。

### 风险
开关仅作用于浏览器工具;公网部署务必保持 false。

* feat(browser): Playwright 超时可配 + snapshot 智能截断与 selector 作用域

### 背景
Agent 用浏览器工具访问慢链路或大页面(超大表格)时遇到两类问题:

1. Playwright 默认 30s 超时不够用,且无法配置
2. snapshot 全量抓取页面文本,硬截断在 20000 字符处会切在元素中间,LLM 拿到残缺数据且不知道有截断
### 方案
新增三个 .env 开关(默认值与改动前完全一致):

开关 作用 PLAYWRIGHT_DEFAULT_TIMEOUT_SECONDS 单次操作超时(click / fill / waitForLoadState) PLAYWRIGHT_NAVIGATION_TIMEOUT_SECONDS 导航超时(page.navigate / load-state) PLAYWRIGHT_SNAPSHOT_MAX_LENGTH snapshot 文本截断长度

snapshot 改造:

- 支持 selector 参数作用域到子树(之前只用于 click/type)
- budget 机制按元素边界智能截断,不再切在 <td> 中间
- 返回 truncated:true + hint 引导 LLM 用 selector 重抓
- JSON 字段顺序优化:truncated/hint 放 content 前,确保框架 spill preview(head 800 chars)能切到
### 验证
- 28 个单元测试全通过(BrowserPropertiesTest 7 + UrlSafetyCheckerTest 21)
- IDE 诊断 0 错误
- 覆盖:默认值不变 + setter 往返 + 严格/豁免两模式
### 兼容性
- 默认值保持 30s / 30s / 20000,行为与改动前完全一致
- selector 参数本就是 @ToolParam(required=false) ,LLM schema 无变化,只是描述更新引导 snapshot 场景也能用
- 不影响 webhook / image download 等其他 SSRF 守卫

## 改动汇总
文件 改动 BrowserProperties.java +3 字段: defaultTimeoutSeconds / defaultNavigationTimeoutSeconds / snapshotMaxLength (默认 30/30/20000) BrowserLauncher.java 抽 applyContextDefaults(context) 在三处 context 创建点调用;补全 setIgnoreHTTPSErrors 在 wrapLocalBrowser 落地 BrowserUseTool.java 工具描述 + selector 描述引导 LLM 在 snapshot 场景用 selector;doSnapshot 改造支持 selector 参数 + budget 智能截断 + JSON 字段顺序(truncated/hint 放 content 前确保 spill preview 能切到) docker-compose.yml +3 开关: PLAYWRIGHT_DEFAULT_TIMEOUT_SECONDS / PLAYWRIGHT_NAVIGATION_TIMEOUT_SECONDS / PLAYWRIGHT_SNAPSHOT_MAX_LENGTH .env.example +3 开关,简短说明 BrowserPropertiesTest.java +4 测试覆盖新字段默认值和 setter

## 测试结果
## 数据流论证的关键决策
决策 依据 truncated:true 和 hint 放在 JSON content 字段之前 ToolResultStorage.buildPreview 会把 >8000 chars 的结果截到 head 800 chars,放在前面确保 LLM 看到 selector 参数描述从 "for click/type" 改为明确说 "OPTIONAL for snapshot" LLM 读 JSON schema 时按描述判断参数用途,原描述误导 LLM 不在 snapshot 用 selector 截断从硬 substring(0, N) 改为 JS budget 机制递归累计 避免切在 <td>订单号 ABC 中间,截断发生在 TEXT_NODE 完整段或下一个子元素开始之前 用 ElementHandle.querySelector + evaluate 替代字符串拼接 selector 进 JS 防止 selector 注入(selector 含特殊字符如引号、反斜杠) 三处 context 创建点统一调 applyContextDefaults 确保 CDP / external-CDP / 本地 launch 三条路径都应用配置的 timeout

## 临时改动还原
文件 改动 还原状态 mateclaw-server/pom.xml 临时加 maven-compiler-plugin + Lombok annotation processor  已删除,恢复原始状态

## 未测项
项 原因 doSnapshot 的 JS budget 逻辑 需启动真 Playwright + 大页面,单元测试范围外 BrowserLauncher.applyContextDefaults 是否真的影响 page.click 行为 同上,集成测试范围 LLM 是否真的会按 hint 用 selector 重调 取决于 LLM 推理能力,需端到端测试
2026-07-02 09:25:16 +08:00
matevip
d512643960 feat(tool): configurable SSRF allowlist for outbound HTTP guards 2026-06-29 10:33:38 +08:00
matevip
808047d723 feat(proxy): global outbound HTTP/SOCKS proxy with settings page (#109)
Add a single global-proxy switch that routes the backend's outbound traffic
through a configured HTTP/HTTPS/SOCKS proxy, for deployments that cannot reach
overseas APIs directly or must use a unified egress.

- ProxyManager installs the proxy via a default ProxySelector (honored by
  java.net.http and HttpURLConnection), the proxy system properties, and a
  --proxy-server arg for the browser tool; restores direct egress when
  disabled. One switch covers LLM, web search, media generation, channels,
  MCP and the browser.
- SOCKS applies to the HttpURLConnection-based egress only; the java.net.http
  LLM/streaming path uses an HTTP proxy, and the UI states this.
- New Settings -> Network Proxy page: enable toggle, address, bypass list,
  test-connection, and a coverage summary. Config stored as key/value in
  mate_system_setting (no migration).

refs #109
2026-06-07 16:12:02 +08:00
matevip
b40cbfb0a1 fix(tool): clean up EXTERNAL_CDP profile dir + Chrome subprocesses on session stop 2026-04-30 10:25:00 +08:00
matevip
1864801c90 fix(tool): browser_use Windows compat + stop LLM treating it as web search 2026-04-30 08:54:15 +08:00
matevip
83567e95f0 fix(browser): multi-strategy launcher + self-diagnostics for win/linux 2026-04-24 21:37:46 +08:00