Commit Graph

15 Commits

Author SHA1 Message Date
MIST
11fa2b0a03
feat(skill): configurable pip index for skill Python scripts (mirror & private LAN sources)
Let skill Python scripts install packages from a configurable pip index instead of the default PyPI. docker-compose passes PIP_INDEX_URL / PIP_TRUSTED_HOST into the container; for the desktop app (host JVM, no Docker env) SkillScriptExecutionService falls back to mateclaw.pip.index-url / trusted-host Spring config and injects them into the subprocess, auto-deriving the trusted host for plain-HTTP LAN mirrors. The runtime image gains pip and a build toolchain (with the PEP 668 marker removed so on-the-fly installs work), and the script timeout ceiling is raised to accommodate large installs.
2026-07-10 14:48:56 +08:00
matevip
09d9fba1ff chore(deploy): switch public Docker stack to PostgreSQL 16 (#491) 2026-07-06 14:21:53 +08:00
MIST
9e33782b7d
feat(browser): 放开内网服务访问限制,新增局域网部署模式开关 (#472)
* feat(browser): 放开内网服务访问限制,新增局域网部署模式开关

### 背景
局域网部署时,Agent 用浏览器工具访问 http://192.168.x.x:port 等内网服务会被默认 SSRF 严格模式拦截。

### 方案
新增两个 .env 开关(默认 false,行为与改动前完全一致):

- PLAYWRIGHT_ALLOW_PRIVATE_NETWORK=true :放行本地回环和私有 IP,云元数据端点仍拦截
- PLAYWRIGHT_IGNORE_HTTPS_ERRORS=true :忽略 HTTPS 证书错误(自签证书场景)
顺带修复 IPv6 AWS IMDS 网段 fd00:ec2::/64 在严格模式下漏网的问题。

### 验证
24 个单元测试全通过(UrlSafetyChecker 21 + BrowserProperties 3),覆盖严格/豁免两模式 + 4 个 check 重载 + IPv6 IMDS 网段。

### 风险
开关仅作用于浏览器工具;公网部署务必保持 false。

* feat(browser): Playwright 超时可配 + snapshot 智能截断与 selector 作用域

### 背景
Agent 用浏览器工具访问慢链路或大页面(超大表格)时遇到两类问题:

1. Playwright 默认 30s 超时不够用,且无法配置
2. snapshot 全量抓取页面文本,硬截断在 20000 字符处会切在元素中间,LLM 拿到残缺数据且不知道有截断
### 方案
新增三个 .env 开关(默认值与改动前完全一致):

开关 作用 PLAYWRIGHT_DEFAULT_TIMEOUT_SECONDS 单次操作超时(click / fill / waitForLoadState) PLAYWRIGHT_NAVIGATION_TIMEOUT_SECONDS 导航超时(page.navigate / load-state) PLAYWRIGHT_SNAPSHOT_MAX_LENGTH snapshot 文本截断长度

snapshot 改造:

- 支持 selector 参数作用域到子树(之前只用于 click/type)
- budget 机制按元素边界智能截断,不再切在 <td> 中间
- 返回 truncated:true + hint 引导 LLM 用 selector 重抓
- JSON 字段顺序优化:truncated/hint 放 content 前,确保框架 spill preview(head 800 chars)能切到
### 验证
- 28 个单元测试全通过(BrowserPropertiesTest 7 + UrlSafetyCheckerTest 21)
- IDE 诊断 0 错误
- 覆盖:默认值不变 + setter 往返 + 严格/豁免两模式
### 兼容性
- 默认值保持 30s / 30s / 20000,行为与改动前完全一致
- selector 参数本就是 @ToolParam(required=false) ,LLM schema 无变化,只是描述更新引导 snapshot 场景也能用
- 不影响 webhook / image download 等其他 SSRF 守卫

## 改动汇总
文件 改动 BrowserProperties.java +3 字段: defaultTimeoutSeconds / defaultNavigationTimeoutSeconds / snapshotMaxLength (默认 30/30/20000) BrowserLauncher.java 抽 applyContextDefaults(context) 在三处 context 创建点调用;补全 setIgnoreHTTPSErrors 在 wrapLocalBrowser 落地 BrowserUseTool.java 工具描述 + selector 描述引导 LLM 在 snapshot 场景用 selector;doSnapshot 改造支持 selector 参数 + budget 智能截断 + JSON 字段顺序(truncated/hint 放 content 前确保 spill preview 能切到) docker-compose.yml +3 开关: PLAYWRIGHT_DEFAULT_TIMEOUT_SECONDS / PLAYWRIGHT_NAVIGATION_TIMEOUT_SECONDS / PLAYWRIGHT_SNAPSHOT_MAX_LENGTH .env.example +3 开关,简短说明 BrowserPropertiesTest.java +4 测试覆盖新字段默认值和 setter

## 测试结果
## 数据流论证的关键决策
决策 依据 truncated:true 和 hint 放在 JSON content 字段之前 ToolResultStorage.buildPreview 会把 >8000 chars 的结果截到 head 800 chars,放在前面确保 LLM 看到 selector 参数描述从 "for click/type" 改为明确说 "OPTIONAL for snapshot" LLM 读 JSON schema 时按描述判断参数用途,原描述误导 LLM 不在 snapshot 用 selector 截断从硬 substring(0, N) 改为 JS budget 机制递归累计 避免切在 <td>订单号 ABC 中间,截断发生在 TEXT_NODE 完整段或下一个子元素开始之前 用 ElementHandle.querySelector + evaluate 替代字符串拼接 selector 进 JS 防止 selector 注入(selector 含特殊字符如引号、反斜杠) 三处 context 创建点统一调 applyContextDefaults 确保 CDP / external-CDP / 本地 launch 三条路径都应用配置的 timeout

## 临时改动还原
文件 改动 还原状态 mateclaw-server/pom.xml 临时加 maven-compiler-plugin + Lombok annotation processor  已删除,恢复原始状态

## 未测项
项 原因 doSnapshot 的 JS budget 逻辑 需启动真 Playwright + 大页面,单元测试范围外 BrowserLauncher.applyContextDefaults 是否真的影响 page.click 行为 同上,集成测试范围 LLM 是否真的会按 hint 用 selector 重调 取决于 LLM 推理能力,需端到端测试
2026-07-02 09:25:16 +08:00
matevip
982c6c048c feat(security): gate Swagger/OpenAPI UI behind mateclaw.openapi.expose-ui flag
- explicit SecurityConfig authorization for /swagger-ui*, /v3/api-docs*, /webjars/**
- public for local/default profile; admin-only (ROLE_ADMIN) by default in production DB profiles
- override via MATECLAW_OPENAPI_EXPOSE_UI; add RANDOM_PORT integration tests and docs
2026-06-24 10:42:46 +08:00
倪程伟
18daad79b2
feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync (#316)
* feat(wiki): unify raw materials & source watcher into a Sources tab with per-KB auto-sync

The raw-material directory scan and the Advanced "source watcher" sub-tab were
the same engine (same kb.sourceDirectory, same WikiDirectoryScanService) split
across two surfaces with two editable directory inputs. Merge them into one
"Sources" tab (upload / paste / directory manual scan + auto-sync toggle +
the raw-material list) and drop the watcher sub-tab from Advanced.

Auto-sync is now per-KB opt-in: a new watcher_enabled column (V146) gates the
periodic scan per knowledge base. The server-global mate.wiki.watcher-enabled
stays as an ops master switch — a KB is auto-scanned only when both are on
(AND). Manual scans are unaffected. Scan interval stays global for now
(tracked separately).

Closes matevip/mateclaw#314

* docs(wiki): document source-watcher global switch env vars

Expose MATE_WIKI_WATCHER_ENABLED / MATE_WIKI_WATCHER_INTERVAL_MS as
explicit placeholders in application-mysql.yml, .env.example and
docker-compose.yml, mirroring MATE_WIKI_ALLOWED_SOURCE_ROOTS. Notes the
AND semantics (global ops gate + per-KB toggle) so operators know the
global switch alone is not sufficient.
2026-06-11 09:29:26 +08:00
matevip
be7bbd9644 fix(skill): persist skill workspace on the existing data volume; skip binary entries in ZIP packages (#273) 2026-06-07 23:09:42 +08:00
倪程伟
29fcfb5572
fix(wiki): surface scan errors and fix Chinese path handling in Docker (#260)
Surface directory-scan failures to the user via toast and render ScanResult.errors[]; expose MATE_WIKI_ALLOWED_SOURCE_ROOTS as a Docker env entry with blank-entry filtering in the path validator; set C.UTF-8 locale in the runtime image so non-ASCII file names decode correctly during scans.

Fixes #259
2026-06-07 16:18:49 +08:00
matevip
7894a50067 fix(tool): emit absolute download URLs for streaming-generated files (#164)
Streaming chat ran render tools on an async thread with no bound request,
so download links lost their host and arrived without a domain. Resolve the
host on the request thread and carry it through ChatOrigin/ToolContext;
falls back to a configurable public-base-url, then a relative path.
2026-06-07 15:03:48 +08:00
srant1
5f77434953
fix(llm): activate default model after OpenAI OAuth and gate provider checks by enabled
OAuth token save now promotes the first available chat model when no usable default exists. Default-model resolution and provider availability checks require Provider.enabled=true alongside credentials, so disabled Providers no longer return stale defaults. /models/enabled drops the single-Provider hard-code so OpenAI OAuth and other enabled chat models surface in selectors. Docker exposes the 1455 PKCE callback via MATECLAW_OAUTH_OPENAI_CALLBACK_BIND_HOST (default 127.0.0.1) and a port mapping; deployment mode stays Host-driven.
2026-05-12 09:58:30 +08:00
matevip
53f54fe05d build(docker): pass MAVEN_FLAGS build-arg to support aliyun-first profile in CN builds 2026-04-25 10:32:23 +08:00
matevip
52a9a785c1 fix(search): bundled SearXNG sidecar actually works out of the box 2026-04-24 22:08:42 +08:00
matevip
178b9306d9 build(docker): use Playwright runtime image so browser works out of the box 2026-04-24 21:57:11 +08:00
matevip
5d5eb7e2e6 fix(docker): add Node.js frontend build stage and clean up deploy config
- Add Node 22/pnpm stage to mateclaw-server/Dockerfile so docker compose
  up -d --build produces a fully working image (the Vue SPA is now built
  inside the image and copied into the JAR's classpath/static).
- Remove DASHSCOPE_API_KEY from .env.example and root docker-compose.yml;
  LLM API keys are configured post-startup via the model management UI.
2026-04-24 11:16:52 +08:00
matevip
7f88a6ddd4 release: v1.1.0 2026-04-17 19:41:03 +08:00
matevip
579d60125b Initial commit: MateClaw — Java + Vue 3 AI Assistant System
Full-stack AI assistant built on Spring AI Alibaba.
Features: ReAct Agent, Plan-and-Execute, MCP Protocol, Multi-Model, Multi-Channel.

Apache-2.0 License
2026-04-04 19:03:49 +08:00