diff --git a/CHANGELOG.md b/CHANGELOG.md index 516adf1b..3a09f6f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,8 @@ - Native DashScope chat/embedding adapters; native speech SDK retained. - Owned HTTP cancellation, guarded tool batches and durable Goal checkpoints prevent blind side-effect replay. - Existing frontend interaction contracts and conversation history remain supported. +- Shared prompt budgets inherit model defaults and count effective tool schemas plus one output reserve; AgentScope remains on stable 2.0.4. +- Latest budget regression: 342 focused tests and 6,509 full-suite tests (50 skipped) with no failures/errors. Real-model acceptance: 18/20 after manual review; tool argument and progress-count failures remain documented in the detailed notes. [English details](mateclaw-server/src/main/resources/docs/en/changelog.md) · [中文说明](mateclaw-server/src/main/resources/docs/zh/changelog.md) diff --git a/README.md b/README.md index 6a12d115..add03f08 100644 --- a/README.md +++ b/README.md @@ -51,6 +51,17 @@ Most AI tools die when their vendor has a bad day. Most forget you the moment th --- +## What changes in 3.0.0-SNAPSHOT + +3.0.0 is in development and includes the unpublished 2.4.0 development cycle. Stable downloads remain the builds listed in Releases. + +- **Native execution:** AgentScope Java **2.0.4** runs ReAct and the tool loop within each Plan-Execute step. MateClaw retains plan scheduling, tool approvals, workspace permissions, cancellation, Goal acceptance and persistence. This path does not require a beta AgentScope Harness API. +- **Consistent context budgets:** request preparation, compaction retries and summary calls account for messages, effective tool schemas and one output reserve. Partial request options inherit model defaults; an explicitly empty tool list disables inherited callbacks. Estimates are conservative, rather than provider billing counts. +- **Governed decisions and recovery:** typed decisions default to SHADOW mode; existing domain constraints and Goal budgets still apply. Durable checkpoints block blind replay when side effects are uncertain. +- **Everyday improvements:** opt-in strict member file isolation, managed DSH SDK upgrades, voice selection/preview, structured MCP results and stricter model probes. + +Use Java 21 for source builds. Back up the database, workspace, plugins and configuration before migration; use that backup for rollback. [Architecture and upgrade notes](mateclaw-server/src/main/resources/docs/en/changelog.md) · [Model budget rules](mateclaw-server/src/main/resources/docs/en/models.md) + ## Three things that make it different ### 1 · Your AI doesn't die when a model does @@ -167,6 +178,11 @@ Same "whole widget" philosophy. Different center of gravity. ```bash # Backend +# Java 21 and Maven required. Create a local JWT secret once and reuse it on restart. +[ -f .env.local ] || (umask 077; printf 'JWT_SECRET=%s\n' "$(openssl rand -hex 48)" > .env.local) +set -a +source .env.local +set +a # From the repository root; use its configured models and workspace SPRING_DATASOURCE_URL="jdbc:h2:file:$PWD/data/mateclaw;MODE=MySQL;DATABASE_TO_LOWER=TRUE;CASE_INSENSITIVE_IDENTIFIERS=TRUE" \ MATECLAW_WORKSPACE_SANDBOX_ROOT="$PWD/data/workspace" \ @@ -177,7 +193,7 @@ cd mateclaw-ui npm install && npm run dev -- --host 127.0.0.1 # http://localhost:5173 ``` -Login: `admin` / `admin123` +On a fresh database, log in with `admin` / `admin123`, change the password, then configure a model in **Settings → Models**. Existing model settings are read from the root `data/` database; starting against `mateclaw-server/data/` opens a different database. Keep `.env.local` private and set a non-empty `JWT_SECRET` of at least 32 UTF-8 bytes. ### Docker @@ -327,8 +343,8 @@ Full story in the [v1.7.0 release notes](https://claw.mate.vip/docs/en/releases/ ```bash git clone https://github.com/mateaix/mateclaw.git cd mateclaw -cd mateclaw-server && mvn clean compile -cd ../mateclaw-ui && npm install && npm run dev -- --host 127.0.0.1 +mvn -pl mateclaw-server -am clean compile +cd mateclaw-ui && npm install && npm run dev -- --host 127.0.0.1 ``` --- diff --git a/README_zh.md b/README_zh.md index cdbe6a42..bee2bbe3 100644 --- a/README_zh.md +++ b/README_zh.md @@ -51,6 +51,17 @@ --- +## 3.0.0-SNAPSHOT 更新概览 + +3.0.0 仍在开发，包含未单独正式发布的 2.4.0 开发周期更新。稳定版下载以 Releases 中的发行包为准。 + +- **原生执行循环**：AgentScope Java **2.0.4** 驱动 ReAct 与 Plan-Execute 每个步骤内部的工具循环。计划调度、工具审批、工作空间权限、取消、Goal 验收和持久化继续由 MateClaw 管理；这条路径不依赖 beta 版 AgentScope Harness API。 +- **统一上下文预算**：请求准备、压缩重试和摘要调用统一计算消息、实际生效的工具 schema 与一份输出预留。请求只覆盖部分选项时继承模型默认值，显式空工具列表关闭默认回调。估算偏保守，不等同于供应商计费 token。 +- **受治理的决策与恢复**：类型化决策默认 SHADOW 模式，继续遵守领域约束和 Goal 预算；持久检查点在副作用结果不确定时阻止盲目重放。 +- **日常能力完善**：可选严格成员文件隔离、受管理的 DSH SDK 升级、音色选择与试听、MCP 结构化展示及更严格的模型探测。 + +源码构建使用 Java 21。升级前备份数据库、workspace、插件与配置，回滚使用升级前备份。[架构与升级说明](mateclaw-server/src/main/resources/docs/zh/changelog.md) · [模型预算规则](mateclaw-server/src/main/resources/docs/zh/models.md) + ## 三件让它与众不同的事 ### 1 · 模型挂了，AI 不挂 @@ -167,6 +178,11 @@ RBAC + JWT。**Personal Access Token** 给无人值守脚本和 CI 使用。**We ```bash # 后端 +# 需要 Java 21 与 Maven。首次生成本地 JWT 密钥，重启时复用。 +[ -f .env.local ] || (umask 077; printf 'JWT_SECRET=%s\n' "$(openssl rand -hex 48)" > .env.local) +set -a +source .env.local +set +a # 从仓库根目录运行，使用根目录下已配置的模型数据库与工作空间 SPRING_DATASOURCE_URL="jdbc:h2:file:$PWD/data/mateclaw;MODE=MySQL;DATABASE_TO_LOWER=TRUE;CASE_INSENSITIVE_IDENTIFIERS=TRUE" \ MATECLAW_WORKSPACE_SANDBOX_ROOT="$PWD/data/workspace" \ @@ -177,7 +193,7 @@ cd mateclaw-ui npm install && npm run dev -- --host 127.0.0.1 # http://localhost:5173 ``` -默认登录：`admin` / `admin123` +新数据库默认登录：`admin` / `admin123`。登录后修改密码，并在 **设置 → 模型** 配置模型。已有模型配置从根目录 `data/` 数据库读取；使用 `mateclaw-server/data/` 会打开另一份数据库。不要提交 `.env.local`，其中 `JWT_SECRET` 必须非空且至少包含 32 个 UTF-8 字节。 ### Docker 部署 @@ -327,8 +343,8 @@ mateclaw/ ```bash git clone https://github.com/mateaix/mateclaw.git cd mateclaw -cd mateclaw-server && mvn clean compile -cd ../mateclaw-ui && npm install && npm run dev -- --host 127.0.0.1 +mvn -pl mateclaw-server -am clean compile +cd mateclaw-ui && npm install && npm run dev -- --host 127.0.0.1 ``` --- diff --git a/mateclaw-server/src/main/resources/docs/en/architecture.md b/mateclaw-server/src/main/resources/docs/en/architecture.md index 79391dfb..1215d543 100644 --- a/mateclaw-server/src/main/resources/docs/en/architecture.md +++ b/mateclaw-server/src/main/resources/docs/en/architecture.md @@ -128,6 +128,10 @@ The backend is a **single modular monolith**. Other projects are independent pac The backend uses Java 21, Spring Boot 4.1.1, Spring AI 2.0.1 and AgentScope Java 2.0.4. `NativePolicyDriver` creates a fresh native `ReActAgent` for each invocation; the SDK owns reasoning/acting iteration. Application policies retain prompt preparation, approvals, observations, summaries and goal budgets. +### Request and budget ownership + +AgentScope 2.0.4 owns loop orchestration, while Spring AI/provider adapters prepare actual model requests. `PromptBudget` shares effective-option estimates across reasoning preflight and compaction-model calls; it does not replace provider serialization. Tool approvals, workspace guards, Goal budgets and completion gates remain application responsibilities. [Budget details](./models#context-budgets-and-request-preparation-3-0-0) + ### Key pieces - `agent/runtime/agentscope/NativePolicyDriver.java` — native loop and application policy callbacks diff --git a/mateclaw-server/src/main/resources/docs/en/changelog.md b/mateclaw-server/src/main/resources/docs/en/changelog.md index f61dba07..d590e733 100644 --- a/mateclaw-server/src/main/resources/docs/en/changelog.md +++ b/mateclaw-server/src/main/resources/docs/en/changelog.md @@ -55,3 +55,25 @@ ApplicationGuardedToolkit delegates tool batches to the application executor, re Stop the old process and back up the database, workspace, plugins and configuration before upgrading. Build with Java 21 and check third-party plugin compatibility. Rollback uses the pre-upgrade backup; downgrading an upgraded database is not verified. Validation covers the Maven reactor, frontend build, PostgreSQL/MySQL migrations and transactional pagination, approvals/cancellation, process-crash recovery and six-hour sustained execution. A real Kingbase instance was not tested. Existing frontend interaction contracts remain compatible; this migration does not rewrite the chat UI. + +## Context-budget consistency on AgentScope 2.0.4 + +The native execution path stays on the stable AgentScope Java 2.0.4 API; it does not adopt a beta Harness dependency. Spring AI's native option combination resolves partial runtime options against model defaults. Shared request estimates cover messages, effective tool callback schemas and one output reserve, including OpenAI completion limits. Explicitly empty runtime callbacks suppress default tools, and long totals prevent integer overflow. Reasoning preparation, compaction retries and summary-model guards use the shared estimate while retaining call budgets, permissions, approvals and Goal acceptance. + +These checks estimate request size; they do not guarantee provider acceptance, exact billing or correct tool arguments. See [model budget rules](./models#context-budgets-and-request-preparation-3-0-0). + +## Upgrade checklist + +1. Stop the previous backend. Back up the database together with workspace files, plugin files and configuration, including the JWT secret. Retain the previous application package for rollback. +2. Build with Java 21 and check plugins against the Boot 4 / Spring AI 2 interfaces. Preserve the database profile and paths. In a source checkout, start from the repository root with explicit root `data/` database and workspace paths; `mateclaw-server/data/` is a separate database. +3. Keep Flyway validation enabled. If migration validation fails, inspect the SQL and schema before considering an explicitly authorized history repair; do not enable automatic repair as an upgrade default. See [database operations](./config). +4. After startup, check health and login, confirm existing model configurations, then exercise a real model response, tool approval/rejection, cancellation and an existing conversation. For Goal recovery, inspect checkpoints and acceptance evidence before resuming work with side effects. +5. If rollback is required, stop the new process and restore the pre-upgrade database, files and configuration alongside the old application. Do not assume the old binary can safely open an upgraded database. + +## Latest budget regression evidence (2026-10-20) + +The budget change passed 342 focused backend tests and the full backend suite: 6,509 tests, zero failures/errors, 50 skipped. These counts describe this verification run rather than every feature listed above. + +After restarting the backend against the configured root database and the frontend on port 5173, 20 real-model rounds ran through the frontend API proxy with DeepSeek V4 Flash. Numeric results and streamed/persisted answers matched in all 20 rounds. The original strict audit passed 17/20; manual review accepted 18/20 after identifying a keyword-only false positive in round 15. Original audit flags were retained. + +Two rounds remain failed: round 6 placed `language` outside the tool's nested arguments and triggered a schema rejection; round 20 reported 15 progress updates where receipts showed 41. Task-level recovery did not demonstrate a successful retry of the deliberately rejected original `print(1)` call. These results do not establish a fully passing 20-round acceptance run or an efficiency improvement. diff --git a/mateclaw-server/src/main/resources/docs/en/models.md b/mateclaw-server/src/main/resources/docs/en/models.md index 450a40ae..53098fe1 100644 --- a/mateclaw-server/src/main/resources/docs/en/models.md +++ b/mateclaw-server/src/main/resources/docs/en/models.md @@ -530,3 +530,16 @@ The OpenAI-compatible path also preserves `integer` / `number` in tool JSON Sche ## Model probes and cancellation (3.0.0) A probe requires a valid completed response; connection success or empty output does not prove model availability. Stopping generation cancels HTTP calls owned by the request and prevents subsequent failover. + +## Context budgets and request preparation (3.0.0) + +The native AgentScope runtime continues to call the existing Spring AI/provider adapters. `PromptBudget` estimates the effective request without changing its prompt or options, using Spring AI's native option combination to inherit defaults when runtime options are partial. + +- Count messages (including structured tool-call content), effective tool callback schemas and one output reserve. +- A non-null runtime callback list replaces default callbacks; an explicitly empty list contributes no inherited tool schemas. +- Reserve the declared output maximum, including OpenAI `maxCompletionTokens`; retain a provider-specific runtime maximum as a conservative floor when generic options cannot represent it. Use the caller's fallback only when no maximum is available. +- Use long arithmetic for totals so a large output limit cannot overflow into an apparently safe request. + +Initial preparation, structured-compaction retries and empty-response retries share this estimate. Summary-model calls check the same components before claiming their call budget or reaching the provider. Existing call limits, usage accounting, cancellation and Goal protections remain in force. + +These are local safety estimates, not exact tokenizer or billing measurements. A provider can still reject a request, and fitting a context window does not establish that the answer is correct. diff --git a/mateclaw-server/src/main/resources/docs/zh/architecture.md b/mateclaw-server/src/main/resources/docs/zh/architecture.md index c3dfdb96..5535a927 100644 --- a/mateclaw-server/src/main/resources/docs/zh/architecture.md +++ b/mateclaw-server/src/main/resources/docs/zh/architecture.md @@ -128,6 +128,10 @@ mateclaw/ 后端使用 Java 21、Spring Boot 4.1.1、Spring AI 2.0.1 和 AgentScope Java 2.0.4。`NativePolicyDriver` 为每次调用创建独立的原生 `ReActAgent`，由 SDK 管理推理/行动循环。应用策略继续管理提示词、审批、观察、总结与目标预算。 +### 请求与预算归属 + +AgentScope 2.0.4 管理循环编排，Spring AI/provider 适配器负责实际模型请求准备。`PromptBudget` 在推理预检与压缩模型调用之间共享生效选项的预算估算，不替代供应商请求序列化。工具审批、工作空间保护、Goal 预算和完成验收门仍由应用负责。[预算说明](./models#上下文预算与请求准备-3-0-0) + ### 关键文件 - `agent/runtime/agentscope/NativePolicyDriver.java`——原生循环与应用策略回调 diff --git a/mateclaw-server/src/main/resources/docs/zh/changelog.md b/mateclaw-server/src/main/resources/docs/zh/changelog.md index 796e4002..e7aba853 100644 --- a/mateclaw-server/src/main/resources/docs/zh/changelog.md +++ b/mateclaw-server/src/main/resources/docs/zh/changelog.md @@ -55,3 +55,25 @@ Java 21，Spring Boot 4.1.1，Spring AI 2.0.1，AgentScope Java 2.0.4。移除 S 升级前停止旧进程并备份数据库、workspace、插件及配置。使用 Java 21 构建，检查第三方插件与新版依赖的兼容性；恢复旧版本时使用升级前备份，不能把升级后的数据库降级兼容视为已验证。 本轮完成 Maven reactor、前端构建、PostgreSQL/MySQL 迁移与事务分页、工具审批/取消、进程崩溃恢复及六小时持续运行验证。Kingbase 实机验证尚未覆盖。前端交互接口保持兼容，本轮没有重写聊天 UI。 + +## AgentScope 2.0.4 上的上下文预算一致性 + +原生执行路径继续使用稳定的 AgentScope Java 2.0.4 API，不引入 beta Harness 依赖。通过 Spring AI 原生选项合并，将部分运行时选项与模型默认配置组合。共享请求估算覆盖消息、实际生效的工具回调 schema 与一份输出预留，包括 OpenAI completion 上限；显式空回调列表关闭默认工具，总量使用 long 防止整数溢出。推理准备、压缩重试与摘要模型保护共用估算，同时保留调用预算、权限、审批和 Goal 验收。 + +这些检查用于估算请求大小，不保证供应商接受请求、精确计费或工具参数正确。参见[模型预算规则](./models#上下文预算与请求准备-3-0-0)。 + +## 升级检查清单 + +1. 停止旧后端，将数据库、workspace 文件、插件文件及配置一并备份，保留 JWT 密钥和旧版应用包，以便回滚。 +2. 使用 Java 21 构建，检查插件与 Boot 4 / Spring AI 2 接口的兼容性。保留数据库 profile 和路径；源码环境从仓库根目录启动，显式指定根目录 `data/` 数据库和工作空间。`mateclaw-server/data/` 是另一份数据库。 +3. 保持 Flyway 校验开启。校验失败时先核对 SQL 和表结构，再考虑显式授权的历史修复，不将自动 repair 作为升级默认值。参见[数据库运维说明](./config)。 +4. 启动后检查健康状态和登录，确认原有模型配置，再验证真实模型响应、工具批准/拒绝、取消及已有会话。恢复 Goal 前先检查检查点和验收证据，再继续有副作用的工作。 +5. 如需回滚，停止新进程，将升级前数据库、文件及配置与旧应用一起恢复，不假定旧二进制能安全打开升级后的数据库。 + +## 最新预算回归证据（2026-10-20） + +预算改动通过 342 项定向后端测试；完整后端测试共 6,509 项，零失败、零错误、50 项跳过。上述数量对应本次验证，不代表前述每项功能的独立验收。 + +后端使用已配置的根目录数据库、前端使用 5173 端口重启后，经前端 API 代理完成 DeepSeek V4 Flash 的 20 轮真实模型测试。20 轮数值结果正确，流式输出与落库回答一致。原始严格审计通过 17/20；人工复核第 15 轮的纯关键词误判后，验收为 18/20，原始审计标记保留。 + +仍有两轮失败：第 6 轮将 `language` 放在工具嵌套参数之外，触发 schema 拒绝；第 20 轮将实际 41 次进度更新写成 15 次。任务层恢复没有证明故意拒绝的原始 `print(1)` 调用已成功重试。因此不能将本次结果表述为 20 轮全部通过，也没有据此宣称效率提升。 diff --git a/mateclaw-server/src/main/resources/docs/zh/models.md b/mateclaw-server/src/main/resources/docs/zh/models.md index fdf95545..a2c20f50 100644 --- a/mateclaw-server/src/main/resources/docs/zh/models.md +++ b/mateclaw-server/src/main/resources/docs/zh/models.md @@ -531,3 +531,16 @@ OpenAI 兼容链路同时保留工具 JSON Schema 中的 `integer` / `number`。 ## 模型探测与取消（3.0.0） 探测必须拿到有效的完成响应，连接成功或空响应不能代表模型可用。停止生成会取消本次请求所属的 HTTP 调用，停止后不再切换备用模型。 + +## 上下文预算与请求准备（3.0.0） + +原生 AgentScope 运行时继续通过现有 Spring AI/provider 适配器调用模型。`PromptBudget` 不修改请求提示词或选项，使用 Spring AI 原生选项合并，在运行时只覆盖部分选项时继承默认配置。 + +- 计算消息（包括结构化工具调用内容）、实际生效的工具回调 schema，以及一份输出预留。 +- 非 null 的运行时工具回调列表替换默认列表；显式空列表不计入默认工具 schema。 +- 预留声明的输出上限，包括 OpenAI `maxCompletionTokens`；通用选项无法表达供应商专属字段时，仍将运行时声明的输出上限作为保守下限。没有可用上限时才采用调用方的回退值。 +- 总量使用 long 运算，避免大输出上限溢出后被误判为安全请求。 + +首次准备、结构化压缩重试和空响应重试共用上述估算。摘要模型调用也在占用调用次数预算及访问供应商前检查相同组成项。原有调用次数限制、用量记录、取消和 Goal 保护继续生效。 + +这是本地安全估算，不是精确分词或计费结果。供应商仍可能拒绝请求，能够装入上下文窗口也不代表回答正确。