release: v1.6.0

This commit is contained in:
matevip 2026-06-22 15:17:06 +08:00
parent 84375da3c5
commit 63c1b59045
574 changed files with 39361 additions and 7714 deletions

View File

@ -29,6 +29,11 @@ JWT_SECRET=
# 若留空,服务器会允许所有 origin 并在启动日志里 WARN。生产部署务必设置。
MATECLAW_CORS_ALLOWED_ORIGINS=
# 公开访问基址(如 https://mateclaw.example.com。用于把智能体生成文件的下载
# 链接拼成绝对地址,便于在 Web 之外IM 消息、复制链接、外部下载)直接打开。
# 留空时回退到当前请求的 host再退回相对路径。反代后部署建议显式设置。
MATECLAW_PUBLIC_BASE_URL=
# SearXNG 会话密钥(容器内部用,留空会用开发默认值)。生产部署请设成 32+ 位随机串。
# openssl rand -hex 32
SEARXNG_SECRET=
@ -69,6 +74,36 @@ MATECLAW_BROWSER_CHANNEL=
MATECLAW_OAUTH_OPENAI_DEPLOYMENT_MODE=
MATECLAW_OAUTH_OPENAI_CALLBACK_BIND_HOST=
# ==================== Wiki 知识库目录白名单Docker 模式,可选)====================
#
# Docker 生产部署开启了路径安全校验fail-closed
# 知识库使用「目录扫描」功能时,扫描路径必须在此白名单内,否则返回 400 错误。
# 多个路径用英文逗号分隔;留空则禁止所有目录扫描。
#
# 示例MATE_WIKI_ALLOWED_SOURCE_ROOTS=/data/wiki,/opt/docs
#
# 同时在 docker-compose.yml 的 volumes 里把宿主机目录挂进容器,例如:
# volumes:
# - /your/host/path:/data/wiki
MATE_WIKI_ALLOWED_SOURCE_ROOTS=
# ── Wiki 知识源自动同步(变更监测)总开关 ────────────────────────
# 定时扫描各知识库的源目录、自动消化新文件。默认关闭,运维主动开启。
# AND 语义:全局这个开关开 *且* 某知识库自己的「自动同步」开关也开,
# 该库才会被定时扫描;手动「立即扫描」不受此开关影响。
# 间隔单位毫秒,默认 5 分钟(目前为全局,暂不支持按库配置)。
MATE_WIKI_WATCHER_ENABLED=false
MATE_WIKI_WATCHER_INTERVAL_MS=300000
# ── Skill 工作区目录 ─────────────────────────────────────────────
# 已安装的 skill、运行时积累的 LESSONS.md、skill 运行产物都落在这个目录。
# 默认(容器内)已指向 /app/data/skills由 docker-compose 的 server_data 卷
# 持久化,容器重启不丢,无需额外挂卷。一般无需修改。
# 内置 skill 由 JAR classpath 每次启动现场释放,挂空卷也不会丢内置文件。
# 仅当你想把 skill 目录放到别处(如独立的 bind mount时才覆盖此项
# 并记得在 docker-compose.yml 的 volumes 里把对应宿主机目录挂进容器。
MATECLAW_SKILL_WORKSPACE_ROOT=
# ── Maven 镜像(国内加速)─────────────────────────────────────────
# 在中国大陆构建时取消注释,将 Aliyun 仓库优先级提前,大幅提速 mvn 拉包。
# 空值(默认)使用 US Maven Central → Google CDN → Aliyun 的顺序。

5
.gitignore vendored
View File

@ -111,3 +111,8 @@ scripts/.*-sync-state.json
# Sandbox / external client work that lives in this directory
# but should not ship in the repo.
outputs/
# This is a pnpm monorepo — pnpm-lock.yaml is the only lockfile we track.
# Ignore stray npm/yarn lockfiles so they are not committed by mistake.
package-lock.json
yarn.lock

View File

@ -217,9 +217,20 @@ Full docs at **[claw.mate.vip/docs](https://claw.mate.vip/docs)** — setup, arc
## Roadmap
**v1.3.0 (shipped 2026-05-13)** — Workflow engine · 6-pattern trigger system · Wiki transformations · per-agent MCP binding · multimodal sidecar routing · four JVM-native document generation tools · image edit. See the [v1.3.0 release notes](https://claw.mate.vip/docs/en/releases/1.3.0) for the full story.
**v1.5.0 (shipped 2026-06-04)** — Goal checklists (fuzzy score → ticked boxes) · self-maintaining Wiki (`[[wikilinks]]` · fact/experience layers · pageType profiles & permissions · KB pipelines · local-directory ingest) · per-owner memory isolation (`owner_key` + visibility scope + `endUserId` passthrough) · per-agent primary knowledge base · provider-preference model routing. Full story in the [v1.5.0 release notes](https://claw.mate.vip/docs/en/releases/1.5.0).
**Next** — Drag-to-edit workflow canvas · run replay timeline · `loop` and `invoke_skill` step modes · trigger priorities and event replay · industry scenario marketplace · more ACP upstream integrations.
**v1.4.0 (shipped 2026-05-23)** — Persistent Goals (lock a goal, self-evaluate every turn) · subagent delegation tree (3 levels deep · sync / parallel / async · one-sentence team builder) · progressive tool/skill disclosure · Workspace RBAC (Owner / Admin / Member / Viewer) · Feishu first-class (interactive / approval / streaming cards · channel-native tools). See the [v1.4.0 release notes](https://claw.mate.vip/docs/en/releases/1.4.0).
**v1.3.0 (shipped 2026-05-13)** — Workflow engine · 6-pattern trigger system · Wiki transformations · per-agent MCP binding · multimodal sidecar routing · four JVM-native document-generation tools · image edit. See the [v1.3.0 release notes](https://claw.mate.vip/docs/en/releases/1.3.0).
**v1.6.0 (in progress)** — make the autonomous employee *fast, sharp-eyed, and embeddable*:
- **Faster first token** — two-stage skill loading (base skills resident, scenario skills retrieved on demand by a relevance scorer) plus prefix compression, cutting the cold-start payload that used to blow past a million characters
- **Native code execution**`execute_code` lets an employee write and run sandboxed code to compute, transform data, and assemble multi-format reports, all JVM-side
- **Vision that persists** — images stay in context across turns; `image_analyze` re-reads an attachment on demand, so "zoom into that chart" follow-ups work without re-uploading
- **Embeddable & headless** — the webchat widget becomes a Web/API surface with multi-session support and per-end-user identity (`endUserId`), isolating memory per end user
- **A Wiki you actually read** — reading split from management, a unified Sources tab with per-KB auto-sync, and clickable cross-KB `[[wikilinks]]`
- **Steadier under load** — self-healing MCP connections · tool-call recovery on interleaved-thinking models · evidence-gated plan execution
## Contributing

View File

@ -217,9 +217,20 @@ mateclaw/
## 路线图
**v1.3.02026-05-13 发布)** — 工作流引擎 · 6 种 pattern 触发器 · Wiki 加工器 · 每员工独立 MCP 绑定 · 多模态旁路路由 · 4 个 JVM 原生文档生成工具 · 图像编辑。完整故事见 [v1.3.0 release notes](https://claw.mate.vip/docs/zh/releases/1.3.0)。
**v1.5.02026-06-04 发布)** — Goal 可勾选清单(模糊评分 → 逐项打勾)· Wiki 自维护(`[[wikilinks]]` · 事实层/经验层 · pageType 模板与权限 · 知识库流水线 · 本地目录接入)· 按拥有者隔离记忆(`owner_key` + 可见域 + `endUserId` 透传)· 每员工绑定主知识库 · 偏好 provider 驱动选型。完整故事见 [v1.5.0 release notes](https://claw.mate.vip/docs/zh/releases/1.5.0)。
**下一步** — 工作流画布可拖拉编辑 · 运行回放时间线 · `loop` / `invoke_skill` step mode · 触发器优先级 + 事件回放 · 行业场景应用市场 · 更多 ACP 上游集成。
**v1.4.02026-05-23 发布)** — 持续目标(锁定目标,每轮自评)· 子员工委派树(最深 3 层 · 同步 / 并行 / 异步 · 一句话组队)· 工具/技能渐进式披露 · 工作空间 RBACOwner / Admin / Member / Viewer· 飞书一等公民(交互卡 / 审批卡 / 流式卡 · 渠道原生工具)。详见 [v1.4.0 release notes](https://claw.mate.vip/docs/zh/releases/1.4.0)。
**v1.3.02026-05-13 发布)** — 工作流引擎 · 6 种 pattern 触发器 · Wiki 加工器 · 每员工独立 MCP 绑定 · 多模态旁路路由 · 4 个 JVM 原生文档生成工具 · 图像编辑。详见 [v1.3.0 release notes](https://claw.mate.vip/docs/zh/releases/1.3.0)。
**v1.6.0(开发中)** — 让自驱的数字员工*更快、更会看、更易嵌入*
- **首字节更快** — 技能两段式载入(基础技能常驻,场景技能由相关性评分器按需检索)+ prefix 压缩,砍掉过去单请求动辄上百万字符的冷启动负载
- **原生代码执行**`execute_code` 让员工自己写、自己跑沙箱代码,完成计算、数据加工与多格式报告生成,全程在 JVM 内
- **能记住图的视觉** — 图片跨轮次保留在上下文里;`image_analyze` 按需重新解析某张附件,"放大看那张图表"这类追问无需重新上传
- **可嵌入、可无头** — webchat 组件升级为 Web/API 接入面,支持多会话与按终端用户身份(`endUserId`)隔离记忆
- **真正可读的 Wiki** — 阅读与管理分离、统一的 Sources 标签页(按知识库自动同步)、可点击的跨库 `[[wikilinks]]`
- **高负载更稳** — MCP 连接自愈 · interleaved-thinking 模型的工具调用恢复 · 计划执行的证据闸门
## 参与贡献

View File

@ -96,6 +96,19 @@ services:
# 本机 Docker 若要强制使用 localhost:1455 回调,可在 .env 显式设为 local。
MATECLAW_OAUTH_OPENAI_DEPLOYMENT_MODE: ${MATECLAW_OAUTH_OPENAI_DEPLOYMENT_MODE:-}
MATECLAW_OAUTH_OPENAI_CALLBACK_BIND_HOST: ${MATECLAW_OAUTH_OPENAI_CALLBACK_BIND_HOST:-0.0.0.0}
# Wiki 知识库目录扫描白名单(逗号分隔,留空则禁止所有目录扫描)。
# 示例MATE_WIKI_ALLOWED_SOURCE_ROOTS=/data/wiki,/opt/docs
# 记得同步在 volumes 里把宿主机路径挂进容器。
MATE_WIKI_ALLOWED_SOURCE_ROOTS: ${MATE_WIKI_ALLOWED_SOURCE_ROOTS:-}
# Wiki 知识源自动同步总开关运维总闸默认关。AND 语义:全局开关与
# 每个知识库自己的「自动同步」开关都开,该库才会被定时扫描。
# 间隔单位毫秒,默认 5 分钟。
MATE_WIKI_WATCHER_ENABLED: ${MATE_WIKI_WATCHER_ENABLED:-false}
MATE_WIKI_WATCHER_INTERVAL_MS: ${MATE_WIKI_WATCHER_INTERVAL_MS:-300000}
# Skill 工作区根目录。放在 /app/data 下,让现有的 server_data 卷一并持久化
# 已安装的 skill、运行时积累的 LESSONS.md 以及 skill 运行产物,容器重启不丢。
# 内置 skill 仍由 JAR classpath 每次启动现场释放,空卷不会丢内置文件。
MATECLAW_SKILL_WORKSPACE_ROOT: ${MATECLAW_SKILL_WORKSPACE_ROOT:-/app/data/skills}
# Chromium needs a real /dev/shm. Docker defaults to 64MB which causes
# SIGBUS / "Target page closed" errors under load. 2GB is the usual
# recommendation for Playwright / headless chrome.
@ -104,6 +117,9 @@ services:
- "18080:18088" # host:container — app listens on 18088 inside the container
- "1455:1455"
volumes:
# server_data covers /app/data — H2 DB, wiki-uploads, AND the skill
# workspace (MATECLAW_SKILL_WORKSPACE_ROOT=/app/data/skills above), so a
# single volume persists everything. No separate skills volume needed.
- server_data:/app/data
volumes:

View File

@ -105,9 +105,18 @@ RUN apt-get update \
# BrowserLauncher's BUNDLED strategy will then succeed without extra config.
ENV PLAYWRIGHT_BROWSERS_PATH=/ms-playwright \
TZ=Asia/Shanghai \
JAVA_TOOL_OPTIONS="-Duser.timezone=Asia/Shanghai"
LANG=C.UTF-8 \
LC_ALL=C.UTF-8 \
JAVA_TOOL_OPTIONS="-Duser.timezone=Asia/Shanghai -Dsun.jnu.encoding=UTF-8"
# Default DB profile, overridable by the SPRING_PROFILES_ACTIVE env var
# (compose sets it explicitly: mysql / postgres / kingbase). It must be an ENV,
# not a -D system property on the ENTRYPOINT: a hardcoded
# -Dspring.profiles.active outranks the SPRING_PROFILES_ACTIVE env var and would
# silently pin the profile regardless of what compose passes.
ENV SPRING_PROFILES_ACTIVE=mysql
COPY --from=builder /build/mateclaw-server/target/*.jar app.jar
EXPOSE 18088
EXPOSE 1455
ENTRYPOINT ["java", "-jar", "-Dspring.profiles.active=mysql", "app.jar"]
ENTRYPOINT ["java", "-jar", "app.jar"]

View File

@ -337,6 +337,30 @@
<groupId>org.flywaydb</groupId>
<artifactId>flyway-mysql</artifactId>
</dependency>
<!-- Flyway PostgreSQL support (used by KingbaseES as well since KingbaseES is PostgreSQL-compatible) -->
<dependency>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-database-postgresql</artifactId>
</dependency>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>42.7.7</version>
<scope>runtime</scope>
</dependency>
<!--
KingbaseES (人大金仓) JDBC driver is NOT on Maven Central, so it is
declared in the opt-in `kingbase` Maven profile instead of here.
The default build never resolves it. To build with KingbaseES:
1. install the driver: mvn install:install-file \
-Dfile=${KINGBASE_HOME}/Interface/jdbc/kingbase8-8.6.0.jar \
-DgroupId=com.kingbase8 -DartifactId=kingbase8 \
-Dversion=8.6.0 -Dpackaging=jar
2. build with the profile: mvn package -Pkingbase
No Java code imports com.kingbase8.* — the driver is loaded at
runtime via spring.datasource.driver-class-name only.
-->
<!-- ===== Spring Boot Test ===== -->
<dependency>
@ -470,5 +494,24 @@
</plugins>
</build>
</profile>
<!--
Profile: KingbaseES (人大金仓) JDBC driver.
The driver is not published to Maven Central, so it is kept out of the
default build to keep `mvn package` resolvable for everyone. Install the
driver into the local repository, then build with `mvn package -Pkingbase`.
Runtime selection is via the `kingbase` Spring profile (application-kingbase.yml).
-->
<profile>
<id>kingbase</id>
<dependencies>
<dependency>
<groupId>com.kingbase8</groupId>
<artifactId>kingbase8</artifactId>
<version>8.6.0</version>
<scope>runtime</scope>
</dependency>
</dependencies>
</profile>
</profiles>
</project>

View File

@ -1,19 +1,29 @@
package vip.mate;
import com.baomidou.mybatisplus.annotation.DbType;
import com.baomidou.mybatisplus.extension.plugins.MybatisPlusInterceptor;
import com.baomidou.mybatisplus.extension.plugins.inner.PaginationInnerInterceptor;
import jakarta.annotation.PostConstruct;
import lombok.extern.slf4j.Slf4j;
import org.mybatis.spring.annotation.MapperScan;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.annotation.Bean;
import org.springframework.context.event.EventListener;
import org.springframework.scheduling.annotation.EnableScheduling;
import javax.sql.DataSource;
import java.sql.Connection;
/**
* MateClaw - Personal AI Assistant
* Powered by Spring AI Alibaba
*
* @author MateClaw Team
*/
@Slf4j
@SpringBootApplication(exclude = {
// Disable Spring AI MCP Client auto-configuration (lifecycle owned by McpClientManager).
org.springframework.ai.mcp.client.common.autoconfigure.McpClientAutoConfiguration.class,
@ -33,22 +43,80 @@ import org.springframework.scheduling.annotation.EnableScheduling;
@MapperScan("vip.mate.**.repository")
public class MateClawApplication {
@Autowired
private DataSource dataSource;
/** Cached DbType for the PaginationInnerInterceptor. */
private volatile DbType resolvedDbType;
public static void main(String[] args) {
SpringApplication.run(MateClawApplication.class, args);
}
/**
* Detect the actual database type from the live DataSource so the
* {@link PaginationInnerInterceptor} always uses the correct dialect,
* even when the JDBC URL is wrapped by a proxy (HikariCP, P6Spy, etc.).
*
* <p>DbType is cached after the first successful detection; a failure
* falls back to the value set in {@code mybatis-plus.global-config.db-config.db-type},
* or eventually to {@link DbType#MYSQL} but by then the connection
* pool would already have failed.
*/
@PostConstruct
void detectDbType() {
try (Connection conn = dataSource.getConnection()) {
String productName = conn.getMetaData().getDatabaseProductName().toLowerCase();
if (productName.contains("kingbase")) {
resolvedDbType = DbType.KINGBASE_ES;
} else if (productName.contains("postgresql")) {
resolvedDbType = DbType.POSTGRE_SQL;
} else if (productName.contains("mysql") || productName.contains("mariadb")) {
resolvedDbType = DbType.MYSQL;
} else if (productName.contains("h2")) {
resolvedDbType = DbType.H2;
} else {
// Let the PaginationInnerInterceptor auto-detect at query time
resolvedDbType = null;
}
if (resolvedDbType != null) {
log.info("Detected database type: {} (product={})", resolvedDbType, productName);
}
} catch (Exception e) {
log.warn("Could not detect database type — PaginationInnerInterceptor will auto-detect on first query: {}",
e.getMessage());
}
}
/**
* MyBatis Plus pagination plugin.
*
* <p>DbType is auto-detected from the JDBC connection at runtime rather
* than hardcoded. Hardcoding H2 here meant the MySQL deployment used
* the H2 dialect for the count query, which silently returned 0
* frontends saw records but total=0 and couldn't paginate (RFC-042 P0).
* <p>When {@code resolvedDbType} is available the interceptor uses it directly;
* otherwise it falls back to JDBC-URL auto-detection, which works for
* {@code jdbc:kingbase8://} but not for proxied DataSources (RFC-042 P0).
*/
@Bean
public MybatisPlusInterceptor mybatisPlusInterceptor() {
MybatisPlusInterceptor interceptor = new MybatisPlusInterceptor();
interceptor.addInnerInterceptor(new PaginationInnerInterceptor());
PaginationInnerInterceptor pagination = resolvedDbType != null
? new PaginationInnerInterceptor(resolvedDbType)
: new PaginationInnerInterceptor();
interceptor.addInnerInterceptor(pagination);
return interceptor;
}
/**
* Print a clear "READY" banner after all post-startup initialization,
* so operators can tell at a glance when the application is ready to serve.
*/
@EventListener(ApplicationReadyEvent.class)
public void onReady() {
log.info("");
log.info("╔══════════════════════════════════════════════════════════════════════╗");
log.info("║ MateClaw is READY ✓ ║");
log.info("║ Web UI → http://localhost:18088 ║");
log.info("║ Swagger → http://localhost:18088/swagger-ui.html ║");
log.info("╚══════════════════════════════════════════════════════════════════════╝");
log.info("");
}
}

View File

@ -88,6 +88,11 @@ public class AgentGraphBuilder {
@org.springframework.beans.factory.annotation.Value(
"${mateclaw.skill.disclosure.load-skill-tool.enabled:true}")
private boolean loadSkillToolEnabled;
/** Escape hatch: when false, the final answer is sent verbatim without Markdown normalization. */
@org.springframework.beans.factory.annotation.Value(
"${mate.agent.markdown-normalize-enabled:true}")
private boolean markdownNormalizeEnabled;
private final ConversationService conversationService;
private final ModelConfigService modelConfigService;
private final ModelProviderService modelProviderService;
@ -152,6 +157,30 @@ public class AgentGraphBuilder {
this.auditEventService = s;
}
/**
* Optional per-step delegation dependencies for the Plan-Execute graph.
* Setter injection (like {@link #auditEventService}) breaks the
* {@code AgentService AgentGraphBuilder} construction cycle. Null when not
* wired (legacy / test) per-step delegation is then simply disabled.
*/
private AgentService agentService;
// @Lazy on the injection point: inject a lazy-resolution proxy so the
// AgentService AgentGraphBuilder cycle is broken at bean-creation time
// (the real bean is resolved on first use, when the graph is built).
@org.springframework.beans.factory.annotation.Autowired(required = false)
public void setAgentService(@org.springframework.context.annotation.Lazy AgentService agentService) {
this.agentService = agentService;
}
private vip.mate.tool.builtin.DelegateAgentTool delegateAgentTool;
@org.springframework.beans.factory.annotation.Autowired(required = false)
public void setDelegateAgentTool(
@org.springframework.context.annotation.Lazy vip.mate.tool.builtin.DelegateAgentTool delegateAgentTool) {
this.delegateAgentTool = delegateAgentTool;
}
/**
* 根据 AgentEntity 构建完整的 Agent 实例沿用 Agent / 全局默认模型
*/
@ -549,8 +578,11 @@ public class AgentGraphBuilder {
if (auditEventService != null) {
executor.setAuditEventService(auditEventService);
}
PlanGenerationNode planGenerationNode = new PlanGenerationNode(chatModel, planningService, streamingHelper, conversationWindowManager, toolSet);
PlanGenerationNode planGenerationNode = new PlanGenerationNode(chatModel, planningService, streamingHelper, conversationWindowManager, toolSet, goalService, goalProperties, agentService);
StepExecutionNode stepExecutionNode = new StepExecutionNode(chatModel, toolSet, executor, planningService, streamTracker, reasoningEffort, streamingHelper, conversationWindowManager, skillCatalogRenderer);
// Per-step delegation: route a step assigned to a specialist agent
// through DelegateAgentTool (null when delegation deps aren't wired).
stepExecutionNode.setDelegateAgentTool(delegateAgentTool);
PlanSummaryNode planSummaryNode = new PlanSummaryNode(chatModel, planningService, streamingHelper);
DirectAnswerNode directAnswerNode = new DirectAnswerNode();
@ -574,6 +606,7 @@ public class AgentGraphBuilder {
.addStrategy(PlanStateKeys.CURRENT_STEP_TITLE, KeyStrategy.REPLACE)
.addStrategy(PlanStateKeys.CURRENT_STEP_RESULT, KeyStrategy.REPLACE)
.addStrategy(PlanStateKeys.COMPLETED_RESULTS, KeyStrategy.APPEND)
.addStrategy(PlanStateKeys.PLAN_REPLAN_COUNT, KeyStrategy.REPLACE)
.addStrategy(PlanStateKeys.FINAL_SUMMARY, KeyStrategy.REPLACE)
.addStrategy(PlanStateKeys.DIRECT_ANSWER, KeyStrategy.REPLACE)
// 工作上下文REPLACE 策略每次重新生成
@ -638,6 +671,7 @@ public class AgentGraphBuilder {
.addStrategy(MateClawStateKeys.GOAL_EVALUATED_THIS_RUN, KeyStrategy.REPLACE)
.addStrategy(MateClawStateKeys.GOAL_FOLLOWUP_COUNT, KeyStrategy.REPLACE)
.addStrategy(MateClawStateKeys.GOAL_ACCOUNTED_LLM_CALL_COUNT, KeyStrategy.REPLACE)
.addStrategy(MateClawStateKeys.GOAL_HARD_CONTINUATION_COUNT, KeyStrategy.REPLACE)
// Skill progressive disclosure pinned skills loaded this
// run. Registered in BOTH graphs so the read-merge-write in
// ActionNode is not dropped on multi-node merges.
@ -652,6 +686,7 @@ public class AgentGraphBuilder {
// DIRECT_ANSWER_NODE END
// STEP_EXECUTION (StepProgressDispatcher)
// STEP_EXECUTION (loop)
// PLAN_GENERATION (re-plan on step failure, bounded by PLAN_REPLAN_COUNT)
// PLAN_SUMMARY (active goal?)
// GOAL_EVALUATION (followup?)
// PLAN_GENERATION (re-plan)
@ -685,11 +720,18 @@ public class AgentGraphBuilder {
Map.of(
PlanStateKeys.STEP_EXECUTION_NODE, PlanStateKeys.STEP_EXECUTION_NODE,
PlanStateKeys.PLAN_SUMMARY_NODE, PlanStateKeys.PLAN_SUMMARY_NODE,
// Step-failure recovery: re-plan the remaining work
// (StepProgressDispatcher returns this on phase=plan_replan).
PlanStateKeys.PLAN_GENERATION_NODE, PlanStateKeys.PLAN_GENERATION_NODE,
StateGraph.END, StateGraph.END))
.addConditionalEdges(PlanStateKeys.PLAN_SUMMARY_NODE,
AsyncEdgeAction.edge_async(state -> {
MateClawStateAccessor a = new MateClawStateAccessor(state);
boolean hasGoal = a.hasActiveGoal();
// Same-turn activation: fall back to a DB lookup (gated on the
// feature flag) so a goal the agent set THIS turn is evaluated now,
// not only from the next message. See GoalEvaluationNode.resolveActiveGoal.
boolean hasGoal = goalProperties.isEnabled()
&& GoalEvaluationNode.resolveActiveGoal(state, goalService).isPresent();
boolean already = a.goalEvaluatedThisRun();
return (hasGoal && !already)
? MateClawStateKeys.GOAL_EVALUATION_NODE
@ -714,7 +756,11 @@ public class AgentGraphBuilder {
.addConditionalEdges(PlanStateKeys.DIRECT_ANSWER_NODE,
AsyncEdgeAction.edge_async(state -> {
MateClawStateAccessor a = new MateClawStateAccessor(state);
boolean hasGoal = a.hasActiveGoal();
// Same-turn activation: fall back to a DB lookup (gated on the
// feature flag) so a goal the agent set THIS turn is evaluated now,
// not only from the next message. See GoalEvaluationNode.resolveActiveGoal.
boolean hasGoal = goalProperties.isEnabled()
&& GoalEvaluationNode.resolveActiveGoal(state, goalService).isPresent();
boolean already = a.goalEvaluatedThisRun();
return (hasGoal && !already)
? MateClawStateKeys.GOAL_EVALUATION_NODE
@ -751,9 +797,17 @@ public class AgentGraphBuilder {
* and tool-result chunking. Decoupled from the per-agent value so a small
* {@code max_iterations} can never accidentally re-introduce the silent
* killer.
* <p>
* The base segment budget is further multiplied to cover goal-driven "hard
* continuations" — each grants a fresh full iteration budget after a
* max-iterations turn (see {@code GoalEvaluationNode}). One run can perform
* up to {@link vip.mate.goal.config.GoalProperties#MAX_HARD_CONTINUATIONS_CEILING} of them, so
* the ceiling is sized for {@code (1 + CEILING)} segments to keep the
* recursion guard from tripping before the soft caps do.
*/
private static int frameworkRecursionLimit() {
return (BaseAgent.MAX_ITERATIONS_HARD_CEILING + 5) * 4 + 100;
int perSegment = (BaseAgent.MAX_ITERATIONS_HARD_CEILING + 5) * 4;
return perSegment * (1 + vip.mate.goal.config.GoalProperties.MAX_HARD_CONTINUATIONS_CEILING) + 100;
}
CompiledGraph buildReActGraph(AgentToolSet toolSet, ChatModel chatModel, int maxIterations, String reasoningEffort) {
@ -809,7 +863,7 @@ public class AgentGraphBuilder {
SummarizingNode summarizingNode = new SummarizingNode(chatModel, streamingHelper, streamTracker);
LimitExceededNode limitExceededNode = new LimitExceededNode(
chatModel, observationProcessor, streamingHelper, i18nService, progressLedgerService);
FinalAnswerNode finalAnswerNode = new FinalAnswerNode(generatedFileCache);
FinalAnswerNode finalAnswerNode = new FinalAnswerNode(generatedFileCache, markdownNormalizeEnabled);
KeyStrategyFactory keyStrategyFactory = KeyStrategy.builder()
// 输入字段
@ -821,6 +875,7 @@ public class AgentGraphBuilder {
.addStrategy(MateClawStateKeys.MESSAGES, KeyStrategy.APPEND)
// 迭代控制
.addStrategy(MateClawStateKeys.CURRENT_ITERATION, KeyStrategy.REPLACE)
.addStrategy(MateClawStateKeys.ITERATION_REFUND_COUNT, KeyStrategy.REPLACE)
.addStrategy(MateClawStateKeys.MAX_ITERATIONS, KeyStrategy.REPLACE)
// 工具调用
.addStrategy(MateClawStateKeys.TOOL_CALLS, KeyStrategy.REPLACE)
@ -902,6 +957,7 @@ public class AgentGraphBuilder {
.addStrategy(MateClawStateKeys.GOAL_EVALUATED_THIS_RUN, KeyStrategy.REPLACE)
.addStrategy(MateClawStateKeys.GOAL_FOLLOWUP_COUNT, KeyStrategy.REPLACE)
.addStrategy(MateClawStateKeys.GOAL_ACCOUNTED_LLM_CALL_COUNT, KeyStrategy.REPLACE)
.addStrategy(MateClawStateKeys.GOAL_HARD_CONTINUATION_COUNT, KeyStrategy.REPLACE)
// Skill progressive disclosure pinned skills loaded this
// run. Registered in BOTH graphs so the read-merge-write in
// ActionNode is not dropped on multi-node merges.
@ -951,7 +1007,11 @@ public class AgentGraphBuilder {
.addConditionalEdges(MateClawStateKeys.FINAL_ANSWER_NODE,
AsyncEdgeAction.edge_async(state -> {
MateClawStateAccessor a = new MateClawStateAccessor(state);
boolean hasGoal = a.hasActiveGoal();
// Same-turn activation: fall back to a DB lookup (gated on the
// feature flag) so a goal the agent set THIS turn is evaluated now,
// not only from the next message. See GoalEvaluationNode.resolveActiveGoal.
boolean hasGoal = goalProperties.isEnabled()
&& GoalEvaluationNode.resolveActiveGoal(state, goalService).isPresent();
boolean already = a.goalEvaluatedThisRun();
return (hasGoal && !already)
? MateClawStateKeys.GOAL_EVALUATION_NODE
@ -1315,6 +1375,23 @@ public class AgentGraphBuilder {
// ==================== Prompt 构建 ====================
/**
* Cache-stable platform identity, appended to every agent's system
* prompt. Answers "who are you / what are you based on". The volatile
* "which model right now" fact is injected per-turn by
* {@link vip.mate.agent.context.RuntimeContextInjector} instead, to
* keep this prefix's prompt-cache hash stable.
*/
static final String ABOUT_YOU_BLOCK = """
## About You
You are powered by MateClaw a multi-user AI Agent platform built on
Spring Boot 3.5 and Spring AI Alibaba Graph. You are reachable through
WebChat and 8+ IM channels (DingTalk, Feishu, WeCom, WeChat, Telegram,
Discord, QQ, Slack). If asked who you are or what you are based on,
answer with MateClaw and the technology stack above.
""";
private String buildEnhancedPrompt(AgentEntity entity, boolean builtinSearchEnabled) {
// The agent's own systemPrompt encodes its identity (role / goal /
// backstory). The memory block from workspace files (AGENTS.md, SOUL.md,
@ -1470,7 +1547,7 @@ public class AgentGraphBuilder {
// Wiki 知识库上下文注入
String wikiContext = wikiContextService.buildWikiContext(entity.getId());
return basePrompt + toolGuidance + searchGuidance + wikiContext;
return basePrompt + ABOUT_YOU_BLOCK + toolGuidance + searchGuidance + wikiContext;
}
/**

View File

@ -505,6 +505,19 @@ public class AgentService {
log.info("Agent caches refreshed after tool guard config change (denied tools may have changed)");
}
/**
* Issue #289: an MCP server connecting / disconnecting / reconnecting
* changes the live tool set, but cached agents snapshot their tools at
* build time. Clear the cache so the next turn rebuilds against the
* current MCP tools instead of replying "from memory" with a stale,
* tool-less graph.
*/
@EventListener
public void onMcpServerChanged(vip.mate.tool.mcp.event.McpServerChangedEvent event) {
refreshAllAgents();
log.info("Agent caches refreshed after MCP server change: {}", event.reason());
}
// ==================== Lifecycle helpers ====================
/**

View File

@ -955,6 +955,11 @@ public abstract class BaseAgent {
if (decision.strategy() == MultimodalRoutingDecision.Strategy.SIDECAR
&& mediaCaptionService != null
&& decision.sidecarModel() != null) {
// The user's actual question (text parts only, excluding media markers)
// so the vision model tailors its description to what was asked rather
// than emitting a generic caption.
String userQuestion = extractUserQuestion(parts);
boolean captionPersisted = false;
for (MessageContentPart part : parts) {
if (part == null) continue;
String contentType = part.getContentType();
@ -963,13 +968,23 @@ public abstract class BaseAgent {
&& !contentType.contains("svg");
if (!isImage) continue;
MediaCaptionService.CaptionResult result = mediaCaptionService.caption(
decision.sidecarModel(), part, userLocale);
decision.sidecarModel(), part, userLocale, userQuestion);
if (result.isFailure()) {
log.warn("[{}] Sidecar caption failed for {}: {}",
agentName, part.getFileName(), result.failure().getMessage());
textBuilder.append("\n\n[系统提示] 视觉模型未能解析附件 ")
.append(part.getFileName())
.append(",请稍后重试或在「设置 → 模型」检查视觉模型配置。");
if (isRemoteOnlyAttachment(part)) {
// The image was never downloaded locally (only a remote
// channel URL survives) for WeCom/aibot that URL points
// at short-lived AES-encrypted bytes, so captioning can
// never succeed until media download is enabled.
textBuilder.append("\n\n[系统提示] 图片 ")
.append(part.getFileName())
.append(" 未下载到本地,无法识别;请在「设置 → 渠道」开启该渠道的媒体下载。");
} else {
textBuilder.append("\n\n[系统提示] 视觉模型未能解析附件 ")
.append(part.getFileName())
.append(",请稍后重试或在「设置 → 模型」检查视觉模型配置。");
}
continue;
}
textBuilder.append("\n\n[图片附件描述: ")
@ -977,9 +992,17 @@ public abstract class BaseAgent {
.append("]\n")
.append(result.description())
.append("\n[/图片附件描述]");
// Persist the caption onto the part so later turns retain the image
// content: history user messages replay as text only, and without a
// stored caption every follow-up question loses the attachment.
part.setCaption(result.description());
captionPersisted = true;
String identifier = identifyPart(part);
if (identifier != null) sidecarHandledIdentifiers.add(identifier);
}
if (captionPersisted && conversationService != null) {
conversationService.updateMessageParts(message, parts);
}
}
List<Media> mediaList = new ArrayList<>();
@ -1048,7 +1071,7 @@ public abstract class BaseAgent {
if (mediaPath == null) {
log.warn("[{}] {} file not found for attachment: {}, path: {}, mediaId: {}",
agentName, isVideo ? "Video" : "Image", part.getFileName(), part.getPath(), part.getMediaId());
skippedAttachments.add(part.getFileName() + "(文件未找到)");
skippedAttachments.add(part.getFileName() + unresolvedAttachmentReason(part));
continue;
}
try {
@ -1083,6 +1106,52 @@ public abstract class BaseAgent {
return new CurrentTurnUserMessage(built, decision);
}
/**
* Concatenate the text parts of a message into the user's question, dropping
* image/file/media parts. Returns {@code null} when there is no usable text
* (e.g. an image-only IM message), which makes the caption fall back to the
* generic full-description prompt.
*/
private static String extractUserQuestion(List<MessageContentPart> parts) {
if (parts == null || parts.isEmpty()) return null;
StringBuilder sb = new StringBuilder();
for (MessageContentPart part : parts) {
if (part == null || !"text".equals(part.getType())) continue;
String text = part.getText();
if (text == null || text.isBlank()) continue;
if (sb.length() > 0) sb.append('\n');
sb.append(text.trim());
}
String question = sb.toString().trim();
return question.isEmpty() ? null : question;
}
/**
* True when an attachment has no resolvable local file and its only locator
* is a remote http(s) URL i.e. the IM channel never downloaded it locally.
* For WeCom/aibot images that URL points at short-lived AES-encrypted bytes,
* so it is unusable as-is. Lets callers turn a generic "file not found" into
* an actionable hint instead of a dead end.
*/
private static boolean isRemoteOnlyAttachment(MessageContentPart part) {
if (part == null) return false;
if (part.getPath() != null && !part.getPath().isBlank()) return false;
String locator = part.getMediaId();
if (locator == null || locator.isBlank()) locator = part.getFileUrl();
return locator != null && (locator.startsWith("http://") || locator.startsWith("https://"));
}
/**
* Reason string appended to a skipped attachment whose local file could not
* be resolved distinguishes "never downloaded" (channel media download
* off) from a genuine missing-file so the user gets an actionable message.
*/
private static String unresolvedAttachmentReason(MessageContentPart part) {
return isRemoteOnlyAttachment(part)
? "(图片未下载到本地,无法识别;请在「设置 → 渠道」开启该渠道的媒体下载)"
: "(文件未找到)";
}
/**
* Stable identifier for de-duplicating parts already handled by the sidecar
* pass. Falls back across {@code path mediaId fileName} since not every
@ -1194,7 +1263,14 @@ public abstract class BaseAgent {
if ("user".equals(msg.getRole())) {
// DB 中的实际内容可能包含 contentParts不用传入的 text
String content = conversationService.renderMessageContent(msg);
return buildUserMessageForCurrentTurn(msg, content != null && !content.isBlank() ? content : userMessageText);
CurrentTurnUserMessage built = buildUserMessageForCurrentTurn(
msg, content != null && !content.isBlank() ? content : userMessageText);
// Vision-capable models replay history as text only, so a
// follow-up question about an earlier image would otherwise be
// answered blind. Re-attach the most recent image to this turn
// so the model actually re-sees it. (Text-only models instead
// rely on the persisted sidecar caption see buildUserMessageInternal.)
return maybeCarryRecentImage(history, i, msg, built);
}
}
} catch (Exception e) {
@ -1204,6 +1280,85 @@ public abstract class BaseAgent {
return new CurrentTurnUserMessage(new UserMessage(userMessageText), null);
}
/** How far back (in messages) to look for an image to carry into a follow-up turn. */
private static final int CARRY_IMAGE_LOOKBACK = 8;
/**
* For a vision-capable model, re-attach the most recent image from a recent
* earlier turn to the current user message when the current turn carries no
* image of its own. History is replayed as text only (see {@link #toSpringMessage}),
* so without this a follow-up like "what's in the top-left of that photo?" is
* answered blind. Bounded to a single image within {@link #CARRY_IMAGE_LOOKBACK}
* messages so a long conversation doesn't re-send pixels on every turn.
*
* <p>No-op (returns {@code built} unchanged) when: the model can't see images,
* the current turn already has an image, no recent image exists, or the recent
* image has no resolvable local file (e.g. an undownloaded channel URL).
*/
private CurrentTurnUserMessage maybeCarryRecentImage(List<MessageEntity> history, int currentIdx,
MessageEntity currentMsg, CurrentTurnUserMessage built) {
try {
if (built == null || !modelSupportsVision()) return built;
if (messageHasImagePart(currentMsg)) return built; // current turn already carries an image
int from = Math.max(0, currentIdx - CARRY_IMAGE_LOOKBACK);
for (int j = currentIdx - 1; j >= from; j--) {
MessageEntity m = history.get(j);
if (m == null || !"user".equals(m.getRole())) continue;
List<MessageContentPart> parts = conversationService.parseMessageParts(m);
for (int k = parts.size() - 1; k >= 0; k--) {
MessageContentPart part = parts.get(k);
if (!isResolvableImagePart(part)) continue;
Path imgPath = resolveImagePath(part.getPath());
if (imgPath == null && part.getMediaId() != null) imgPath = resolveImagePath(part.getMediaId());
if (imgPath == null) continue;
String contentType = part.getContentType();
if (contentType == null || "image/*".equals(contentType)) contentType = "image/jpeg";
try {
Media carried = new Media(MimeType.valueOf(contentType), new FileSystemResource(imgPath));
UserMessage orig = built.userMessage();
String name = part.getFileName() == null ? "image" : part.getFileName();
String text = (orig.getText() == null ? "" : orig.getText())
+ "\n\n[系统提示] 以下图片是用户本次对话中较早发送的「" + name
+ "」,当前问题很可能与它相关。请直接查看该图片作答,不要凭记忆猜测。";
List<Media> media = new ArrayList<>();
if (orig.getMedia() != null) media.addAll(orig.getMedia());
media.add(carried);
log.debug("[{}] Carried recent image {} into follow-up turn for vision model",
agentName, name);
return new CurrentTurnUserMessage(
UserMessage.builder().text(text).media(media).build(),
built.routingDecision());
} catch (Exception e) {
log.debug("[{}] Failed to carry recent image {}: {}",
agentName, part.getFileName(), e.getMessage());
return built;
}
}
}
} catch (Exception e) {
log.debug("[{}] maybeCarryRecentImage failed: {}", agentName, e.getMessage());
}
return built;
}
private boolean messageHasImagePart(MessageEntity message) {
for (MessageContentPart part : conversationService.parseMessageParts(message)) {
if (isResolvableImagePart(part)) return true;
}
return false;
}
/** An image part (not SVG) — the raster kind a multimodal API can ingest. */
private static boolean isResolvableImagePart(MessageContentPart part) {
if (part == null) return false;
String type = part.getType();
String contentType = part.getContentType();
boolean isImage = ("image".equals(type) || "file".equals(type))
&& contentType != null && contentType.startsWith("image/");
return isImage && !contentType.contains("svg");
}
protected Path resolveImagePath(String relativePath) {
if (relativePath == null || relativePath.isBlank()) {
return null;

View File

@ -8,6 +8,7 @@ import vip.mate.agent.AgentService;
import vip.mate.agent.binding.model.AgentProviderPreference;
import vip.mate.agent.binding.model.AgentSkillBinding;
import vip.mate.agent.binding.model.AgentToolBinding;
import vip.mate.agent.binding.model.AgentWikiKbBinding;
import vip.mate.agent.binding.service.AgentBindingService;
import vip.mate.agent.model.AgentEntity;
import vip.mate.audit.service.AuditEventService;
@ -136,6 +137,32 @@ public class AgentBindingController {
return R.ok();
}
// ==================== Knowledge Base Access Scope ====================
@Operation(summary = "获取 Agent 的知识库访问范围")
@GetMapping("/kbs")
@RequireWorkspaceRole("viewer")
public R<List<AgentWikiKbBinding>> listKbs(@PathVariable Long agentId,
@RequestHeader(value = "X-Workspace-Id", required = false) Long workspaceId) {
verifyAgentWorkspace(agentId, workspaceId);
return R.ok(bindingService.listKbBindings(agentId));
}
@Operation(summary = "批量设置 Agent 的知识库访问范围(替换模式,空表示不限制)")
@PutMapping("/kbs")
@RequireWorkspaceRole("member")
public R<Void> setKbs(@PathVariable Long agentId, @RequestBody List<Long> kbIds,
@RequestHeader(value = "X-Workspace-Id", required = false) Long workspaceId) {
verifyAgentWorkspace(agentId, workspaceId);
bindingService.setKbBindings(agentId, kbIds);
agentService.invalidateAgentCache(agentId);
// A non-Vue caller can POST a bare `null`; the service tolerates it.
int count = kbIds == null ? 0 : kbIds.size();
auditEventService.record("UPDATE", "AGENT_WIKI_KB", String.valueOf(agentId),
"kbs=" + count, null);
return R.ok();
}
// ==================== Workspace Verification ====================
private void verifyAgentWorkspace(Long agentId, Long headerWorkspaceId) {

View File

@ -0,0 +1,27 @@
package vip.mate.agent.binding.model;
import com.baomidou.mybatisplus.annotation.*;
import lombok.Data;
import java.time.LocalDateTime;
/**
* Agent knowledge base access scope row.
* <p>
* Each enabled row whitelists one KB for one agent. When an agent has at
* least one row the wiki tools restrict their visible KB set to the bound
* ones; an agent with no rows stays workspace-wide (legacy behavior).
*/
@Data
@TableName("mate_agent_wiki_kb")
public class AgentWikiKbBinding {
@TableId(type = IdType.ASSIGN_ID)
private Long id;
private Long agentId;
private Long kbId;
private Boolean enabled;
@TableField(fill = FieldFill.INSERT)
private LocalDateTime createTime;
@TableField(fill = FieldFill.INSERT_UPDATE)
private LocalDateTime updateTime;
private Integer deleted;
}

View File

@ -0,0 +1,9 @@
package vip.mate.agent.binding.repository;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.apache.ibatis.annotations.Mapper;
import vip.mate.agent.binding.model.AgentWikiKbBinding;
@Mapper
public interface AgentWikiKbBindingMapper extends BaseMapper<AgentWikiKbBinding> {
}

View File

@ -9,9 +9,11 @@ import org.springframework.stereotype.Service;
import vip.mate.agent.binding.model.AgentProviderPreference;
import vip.mate.agent.binding.model.AgentSkillBinding;
import vip.mate.agent.binding.model.AgentToolBinding;
import vip.mate.agent.binding.model.AgentWikiKbBinding;
import vip.mate.agent.binding.repository.AgentProviderPreferenceMapper;
import vip.mate.agent.binding.repository.AgentSkillBindingMapper;
import vip.mate.agent.binding.repository.AgentToolBindingMapper;
import vip.mate.agent.binding.repository.AgentWikiKbBindingMapper;
import vip.mate.agent.model.AgentEntity;
import vip.mate.agent.repository.AgentMapper;
import vip.mate.exception.MateClawException;
@ -26,6 +28,8 @@ import vip.mate.skill.runtime.SkillRuntimeService;
import vip.mate.skill.runtime.model.ResolvedSkill;
import vip.mate.tool.model.AvailableToolDTO;
import vip.mate.tool.service.AvailableToolService;
import vip.mate.wiki.model.WikiKnowledgeBaseEntity;
import vip.mate.wiki.repository.WikiKnowledgeBaseMapper;
import java.time.Duration;
import java.time.LocalDateTime;
@ -55,6 +59,17 @@ public class AgentBindingService implements AgentBindingResolver {
private final AgentSkillBindingMapper skillBindingMapper;
private final AgentToolBindingMapper toolBindingMapper;
private final AgentProviderPreferenceMapper providerPreferenceMapper;
/**
* Agent KB access-scope rows. Plain mapper (no transitive deps), so it
* is safe to wire directly here without risking a boot-time cycle.
*/
private final AgentWikiKbBindingMapper kbBindingMapper;
/**
* Used only to verify a KB lives in the agent's workspace before pinning
* it. Like {@link #agentMapper}, a bare mapper avoids pulling the wiki
* service layer (and its dependency on agent binding) into this bean.
*/
private final WikiKnowledgeBaseMapper kbMapper;
/**
* {@code @Lazy} SkillRuntimeService and AgentBindingService both sit
* near the agent boot path; the lazy proxy avoids a circular bean
@ -93,6 +108,8 @@ public class AgentBindingService implements AgentBindingResolver {
public AgentBindingService(AgentSkillBindingMapper skillBindingMapper,
AgentToolBindingMapper toolBindingMapper,
AgentProviderPreferenceMapper providerPreferenceMapper,
AgentWikiKbBindingMapper kbBindingMapper,
WikiKnowledgeBaseMapper kbMapper,
@Lazy SkillRuntimeService skillRuntimeService,
AvailableToolService availableToolService,
AgentMapper agentMapper,
@ -101,6 +118,8 @@ public class AgentBindingService implements AgentBindingResolver {
this.skillBindingMapper = skillBindingMapper;
this.toolBindingMapper = toolBindingMapper;
this.providerPreferenceMapper = providerPreferenceMapper;
this.kbBindingMapper = kbBindingMapper;
this.kbMapper = kbMapper;
this.skillRuntimeService = skillRuntimeService;
this.availableToolService = availableToolService;
this.agentMapper = agentMapper;
@ -725,6 +744,11 @@ public class AgentBindingService implements AgentBindingResolver {
"write_file",
"edit_file",
"execute_shell_command",
// Inline code execution an agent-wide capability alongside shell.
// Lets any agent act on a documentation-only skill (a SKILL.md with
// no scripts) by writing and running the code its instructions
// describe. Dangerous code is screened by the same tool guard.
"execute_code",
"detect_file_type",
"extract_document_text",
"extract_pdf_text",
@ -941,6 +965,123 @@ public class AgentBindingService implements AgentBindingResolver {
}
}
// ==================== Knowledge base access scope ====================
/** Raw scope rows for the agent edit form, oldest first. */
public List<AgentWikiKbBinding> listKbBindings(Long agentId) {
return kbBindingMapper.selectList(
new LambdaQueryWrapper<AgentWikiKbBinding>()
.eq(AgentWikiKbBinding::getAgentId, agentId)
.orderByAsc(AgentWikiKbBinding::getCreateTime));
}
/**
* Effective KB ids the agent may see. Three states (mirror
* {@link #getBoundSkillIds}):
*
* <ul>
* <li>{@code null} {@code wiki_disabled=false} AND no binding rows.
* Caller treats this as "no agent-level restriction; inherit every
* KB in the agent's workspace" (the default wiki-tool behavior).</li>
* <li>{@code Set.of()} either {@code wiki_disabled=true}, or binding
* rows exist but none are {@code enabled=true}. Caller treats this
* as "explicitly scoped to zero KBs" wiki tools degrade with
* their standard "no knowledge base" message.</li>
* <li>non-empty set the explicit allowlist.</li>
* </ul>
*
* <p>The {@code wiki_disabled} flag takes precedence over row count, so
* a stale (flag + leftover rows) combination still surfaces as "no KBs".
* Mirrors how {@code skills_disabled} interacts with
* {@link #getBoundSkillIds}.
*/
@Override
public Set<Long> getBoundKbIds(Long agentId) {
if (isWikiDisabled(agentId)) {
return Set.of();
}
List<AgentWikiKbBinding> bindings = listKbBindings(agentId);
if (bindings.isEmpty()) {
return null;
}
return bindings.stream()
.filter(b -> Boolean.TRUE.equals(b.getEnabled()))
.map(AgentWikiKbBinding::getKbId)
.filter(Objects::nonNull)
.collect(Collectors.toSet());
}
/**
* Replace the agent's KB access scope. An empty / null list clears the
* scope, returning the agent to workspace-wide (unrestricted) access.
* Every incoming KB must live in the agent's workspace pinning a KB
* from another tenancy is refused (403).
*/
public void setKbBindings(Long agentId, List<Long> kbIds) {
// De-dup defensively: the unique index is (agent_id, kb_id, deleted),
// so two identical ids in the incoming list would collide on insert.
Set<Long> distinct = new LinkedHashSet<>();
if (kbIds != null) {
for (Long kbId : kbIds) {
if (kbId != null) {
distinct.add(kbId);
}
}
}
// Validate the whole set BEFORE deleting anything, so a rejected id
// can't leave the agent half-scoped.
for (Long kbId : distinct) {
requireKbInAgentWorkspace(agentId, kbId);
}
// Auto-clear wiki_disabled on a non-empty save same contract as
// setSkillBindings: a concrete KB commitment contradicts an opt-out
// flag, so the data layer must never hold both states at once.
if (!distinct.isEmpty()) {
clearWikiDisabledFlag(agentId);
}
kbBindingMapper.delete(
new LambdaQueryWrapper<AgentWikiKbBinding>()
.eq(AgentWikiKbBinding::getAgentId, agentId));
for (Long kbId : distinct) {
AgentWikiKbBinding row = new AgentWikiKbBinding();
row.setAgentId(agentId);
row.setKbId(kbId);
row.setEnabled(true);
kbBindingMapper.insert(row);
}
}
/**
* Refuse to scope an agent to a KB outside its workspace. KBs are
* workspace-shared artifacts ({@code mate_wiki_knowledge_base.workspace_id});
* letting workspace A's agent pin workspace B's KB would cross the
* tenancy boundary the same way a cross-workspace skill binding would.
* A {@code null} workspace on either side is normalized to the default
* workspace (1) to match the rest of the codebase.
*/
private void requireKbInAgentWorkspace(Long agentId, Long kbId) {
if (agentId == null) {
throw new MateClawException("err.agent.not_found", 404, "Agent ID is required");
}
AgentEntity agent = agentMapper.selectById(agentId);
if (agent == null) {
throw new MateClawException("err.agent.not_found", 404, "Agent 不存在: " + agentId);
}
WikiKnowledgeBaseEntity kb = kbMapper.selectById(kbId);
if (kb == null) {
throw new MateClawException("err.wiki.kb_not_found", 404,
"Knowledge base 不存在: " + kbId);
}
long agentWs = agent.getWorkspaceId() == null ? 1L : agent.getWorkspaceId();
long kbWs = kb.getWorkspaceId() == null ? 1L : kb.getWorkspaceId();
if (agentWs != kbWs) {
throw new MateClawException("err.wiki.cross_workspace_kb_binding", 403,
"Knowledge base " + kbId + " (workspace=" + kbWs
+ ") cannot be scoped to Agent " + agentId
+ " (workspace=" + agentWs + ")");
}
}
// ==================== Binding-mode flags (V126) ====================
/**
@ -993,4 +1134,28 @@ public class AgentBindingService implements AgentBindingResolver {
update.setToolsDisabled(false);
agentMapper.updateById(update);
}
/** Mirror of {@link #isSkillsDisabled} for the wiki/knowledge-base opt-out toggle. */
private boolean isWikiDisabled(Long agentId) {
if (agentId == null) return false;
AgentEntity agent = agentMapper.selectById(agentId);
return agent != null && Boolean.TRUE.equals(agent.getWikiDisabled());
}
/**
* Mirror of {@link #clearSkillsDisabledFlag} for the wiki toggle. Used as
* an auto-clear step in {@link #setKbBindings} so a concrete KB commitment
* always wins over a stale opt-out flag.
*/
private void clearWikiDisabledFlag(Long agentId) {
if (agentId == null) return;
AgentEntity agent = agentMapper.selectById(agentId);
if (agent == null || !Boolean.TRUE.equals(agent.getWikiDisabled())) {
return;
}
AgentEntity update = new AgentEntity();
update.setId(agentId);
update.setWikiDisabled(false);
agentMapper.updateById(update);
}
}

View File

@ -58,7 +58,15 @@ public record ChatOrigin(
* vs. group conversations. Null for 1:1 chats. Distinct from
* {@link #channelTarget()} (which targets cron / proactive sends).
*/
@Nullable String chatId
@Nullable String chatId,
/**
* Public base URL ({@code scheme://host[:port][/contextPath]}) resolved
* from the inbound HTTP request on the request thread. Carried here so
* tools running on async/streaming threads where no request is bound
* can still mint absolute download links. Null for IM/cron origins, which
* have no request host; those rely on {@code mateclaw.server.public-base-url}.
*/
@Nullable String baseUrl
) {
/** Key used when this origin is wrapped into a Spring AI {@link ToolContext}. */
@ -66,7 +74,7 @@ public record ChatOrigin(
/** Sentinel used by AgentService default overloads where no origin is supplied. */
public static final ChatOrigin EMPTY =
new ChatOrigin(null, null, "", null, null, null, null, false, null, null, null);
new ChatOrigin(null, null, "", null, null, null, null, false, null, null, null, null);
// ---------------- Factories per entry point ----------------
@ -74,9 +82,17 @@ public record ChatOrigin(
@Nullable String requesterId,
@Nullable Long workspaceId,
@Nullable String workspaceBasePath) {
return web(conversationId, requesterId, workspaceId, workspaceBasePath, null);
}
public static ChatOrigin web(@Nullable String conversationId,
@Nullable String requesterId,
@Nullable Long workspaceId,
@Nullable String workspaceBasePath,
@Nullable String baseUrl) {
return new ChatOrigin(null, conversationId,
requesterId != null ? requesterId : "",
workspaceId, workspaceBasePath, null, null, false, null, "web", null);
workspaceId, workspaceBasePath, null, null, false, null, "web", null, baseUrl);
}
public static ChatOrigin cron(@Nullable String conversationId,
@ -85,7 +101,7 @@ public record ChatOrigin(
@Nullable Long channelId,
@Nullable ChannelTarget target) {
return new ChatOrigin(null, conversationId, "system",
workspaceId, workspaceBasePath, channelId, target, true, null, null, null);
workspaceId, workspaceBasePath, channelId, target, true, null, null, null, null);
}
// ---------------- Wither-style updates ----------------
@ -93,20 +109,27 @@ public record ChatOrigin(
public ChatOrigin withAgent(@Nullable Long newAgentId) {
return new ChatOrigin(newAgentId, conversationId, requesterId,
workspaceId, workspaceBasePath, channelId, channelTarget, cronOrigin,
senderName, channelType, chatId);
senderName, channelType, chatId, baseUrl);
}
public ChatOrigin withWorkspace(@Nullable Long newWorkspaceId,
@Nullable String newWorkspaceBasePath) {
return new ChatOrigin(agentId, conversationId, requesterId,
newWorkspaceId, newWorkspaceBasePath, channelId, channelTarget, cronOrigin,
senderName, channelType, chatId);
senderName, channelType, chatId, baseUrl);
}
public ChatOrigin withConversationId(@Nullable String newConversationId) {
return new ChatOrigin(agentId, newConversationId, requesterId,
workspaceId, workspaceBasePath, channelId, channelTarget, cronOrigin,
senderName, channelType, chatId);
senderName, channelType, chatId, baseUrl);
}
/** Carry a request-derived public base URL (see {@link #baseUrl()}). */
public ChatOrigin withBaseUrl(@Nullable String newBaseUrl) {
return new ChatOrigin(agentId, conversationId, requesterId,
workspaceId, workspaceBasePath, channelId, channelTarget, cronOrigin,
senderName, channelType, chatId, newBaseUrl);
}
/**
@ -120,7 +143,7 @@ public record ChatOrigin(
@Nullable String newChatId) {
return new ChatOrigin(agentId, conversationId, requesterId,
workspaceId, workspaceBasePath, channelId, channelTarget, cronOrigin,
newSenderName, newChannelType, newChatId);
newSenderName, newChannelType, newChatId, baseUrl);
}
// ---------------- Spring AI ToolContext interop ----------------

View File

@ -67,6 +67,24 @@ public final class RuntimeContextInjector {
public static String buildContextMessage(String workspaceBasePath,
vip.mate.i18n.I18nService i18n,
ChatOrigin origin) {
return buildContextMessage(workspaceBasePath, i18n, origin, null, null);
}
/**
* Full overload that also renders the agent's runtime model identity.
* The model line is emitted for EVERY origin (web / cron / IM / null)
* because it describes the agent, not the caller only the sender
* block stays IM-only. {@code modelName}/{@code providerId} come from
* graph state ({@code RUNTIME_MODEL_NAME}/{@code RUNTIME_PROVIDER_ID}),
* i.e. the model selected at run start (mid-run failover is not
* reflected accepted trade-off). Stays well under the 1024-char
* spring-ai user-cache threshold.
*/
public static String buildContextMessage(String workspaceBasePath,
vip.mate.i18n.I18nService i18n,
ChatOrigin origin,
String modelName,
String providerId) {
LocalDateTime now = LocalDateTime.now(ZONE);
String dateStr = now.format(DATE_FMT);
String timeStr = now.format(TIME_FMT);
@ -91,6 +109,7 @@ public final class RuntimeContextInjector {
}
appendSenderBlockIfPresent(sb, origin);
appendModelLineIfPresent(sb, modelName, providerId, i18n);
return sb.toString();
}
@ -121,6 +140,33 @@ public final class RuntimeContextInjector {
}
}
/**
* Append the agent's runtime model identity. Emitted for all origins
* (it's an agent fact, not a sender fact). Skipped when modelName is
* blank. Provider parenthetical is omitted when providerId is blank.
*/
private static void appendModelLineIfPresent(StringBuilder sb, String modelName,
String providerId,
vip.mate.i18n.I18nService i18n) {
if (modelName == null || modelName.isBlank()) return;
String model = modelName.trim();
sb.append("\n");
if (i18n != null) {
sb.append(i18n.msg("context.model_identity", model));
} else {
sb.append("[system-context] Model: ").append(model);
}
if (providerId != null && !providerId.isBlank()) {
sb.append(" (provider: ").append(providerId.trim()).append(')');
}
sb.append("\n");
if (i18n != null) {
sb.append(i18n.msg("context.model_identity_hint"));
} else {
sb.append("If asked which model you are using, answer with this value for the current run.");
}
}
/**
* Append a sender / channel / chat block when the origin carries
* meaningful IM context. Format is intentionally one line per

View File

@ -12,7 +12,9 @@ import vip.mate.channel.web.Utf8SseEmitter;
import vip.mate.agent.AgentService;
import vip.mate.agent.AgentState;
import vip.mate.agent.model.AgentEntity;
import vip.mate.agent.service.AgentGenerationService;
import vip.mate.agent.vo.AgentCapabilitiesVO;
import vip.mate.agent.vo.AgentDraftVO;
import vip.mate.audit.service.AuditEventService;
import vip.mate.llm.model.ModelConfigEntity;
import vip.mate.llm.service.ModelCapabilityService;
@ -51,6 +53,7 @@ public class AgentController {
private final ModelConfigService modelConfigService;
private final ModelCapabilityService modelCapabilityService;
private final SystemSettingService systemSettingService;
private final AgentGenerationService agentGenerationService;
private final ObjectMapper objectMapper;
private final ExecutorService sseExecutor = Executors.newCachedThreadPool();
@ -128,6 +131,16 @@ public class AgentController {
}
}
@Operation(summary = "根据一句话需求生成员工草稿(不落库)")
@PostMapping("/generate")
@RequireWorkspaceRole("member")
public R<AgentDraftVO> generate(
@RequestHeader(value = "X-Workspace-Id", required = false) Long workspaceId,
@RequestBody GenerateRequest request) {
long wsId = workspaceId != null ? workspaceId : 1L;
return R.ok(agentGenerationService.generateDraft(request.getRequirement(), wsId));
}
@Operation(summary = "创建Agent")
@PostMapping
@RequireWorkspaceRole("member")
@ -270,6 +283,11 @@ public class AgentController {
private String conversationId = "default";
}
@lombok.Data
public static class GenerateRequest {
private String requirement;
}
/**
* 校验目标资源实际归属的 workspace 与请求 header 一致
* 防止 "在 workspace A 鉴权,操作 workspace B 资源" 的跨域攻击

View File

@ -350,25 +350,57 @@ public class NodeStreamingChatHelper {
// provider during a rate-limit window wastes time without recovery.
// SERVER_ERROR keeps MAX_RETRIES (upstream flaps often self-heal).
static final int MAX_RETRIES_RATE_LIMIT = 2;
private static final long BACKOFF_BASE_MS = 3000;
private static final long BACKOFF_CAP_MS = 60_000;
// EMPTY_RESPONSE: transient gateway blip often resolves on same-model
// retry (e.g., proxy timeout returns HTTP 200 with empty body). Keep
// the cap low if it truly takes 4+ attempts, the provider is sick.
static final int MAX_RETRIES_EMPTY_RESPONSE = 3;
// UNKNOWN: conservative retry cap. Defensive: retry what we can't
// classify, but with a smaller budget than SERVER_ERROR (5 vs 10) to
// avoid masking truly fatal errors. MAX_TOTAL_DURATION_MS is the
// ultimate safety net.
static final int MAX_RETRIES_UNKNOWN = 5;
// Hard time budget for the primary retry loop (3 min). Prevents
// retries from stalling a single conversation turn indefinitely.
// Aligned with WikiProcessingService.llmMaxTotalDurationMs.
//
// Because the backoff grows exponentially (3s, 6s, 12s, 24s, 48s, then
// capped at 60s), this wall-clock budget not MAX_RETRIES is what
// actually bounds a sustained SERVER_ERROR loop: only ~8 of the 10
// retries fit inside 3 minutes before the elapsed-time check in
// streamCallInternal breaks to the fallback chain.
//
// These three values are instance fields seeded from the DEFAULT_*
// constants (rather than compile-time constants) so tests can shrink
// them to exercise the full retry path in milliseconds instead of
// minutes. Production wiring never overrides them see
// setRetryTimingForTest.
private static final long DEFAULT_MAX_TOTAL_DURATION_MS = 3 * 60 * 1000L;
private static final long DEFAULT_BACKOFF_BASE_MS = 3000;
private static final long DEFAULT_BACKOFF_CAP_MS = 60_000;
private static final ObjectMapper TOOL_ARG_JSON_MAPPER = new ObjectMapper();
private long maxTotalDurationMs = DEFAULT_MAX_TOTAL_DURATION_MS;
private long backoffBaseMs = DEFAULT_BACKOFF_BASE_MS;
private long backoffCapMs = DEFAULT_BACKOFF_CAP_MS;
/**
* 判断错误是否可重试基于状态码/异常类型
* Test-only seam to shrink the retry backoff and total-time budget so the
* full {@link #MAX_RETRIES} path (or the time-budget cut-off) can be
* exercised in milliseconds instead of minutes. Package-private and never
* invoked from production wiring, which always keeps the {@code DEFAULT_*}
* timings.
*
* @param backoffBaseMs base backoff for the first retry (doubles each attempt)
* @param backoffCapMs per-attempt backoff ceiling
* @param maxTotalDurationMs hard wall-clock budget for the whole primary retry loop
*/
private static boolean isRetryable(Throwable error) {
String msg = extractFullErrorChain(error);
// Kimi engine_overloaded / 标准 HTTP 错误 / 速率限制
return msg.contains("engine_overloaded")
|| msg.contains("rate_limit") || msg.contains("RateLimitError")
|| msg.contains("429") || msg.contains("Too Many Requests")
|| msg.contains("500") || msg.contains("502") || msg.contains("503") || msg.contains("504")
|| msg.contains("APITimeoutError") || msg.contains("APIConnectionError")
|| msg.contains("Connection reset") || msg.contains("Connection refused");
void setRetryTimingForTest(long backoffBaseMs, long backoffCapMs, long maxTotalDurationMs) {
this.backoffBaseMs = backoffBaseMs;
this.backoffCapMs = backoffCapMs;
this.maxTotalDurationMs = maxTotalDurationMs;
}
private static final ObjectMapper TOOL_ARG_JSON_MAPPER = new ObjectMapper();
/**
* 分类错误类型用于分级重试和上层 Node 决策
*/
@ -384,7 +416,27 @@ public class NodeStreamingChatHelper {
|| msg.contains("请求体中的 input tokens 总数超出了模型允许")) {
return ErrorType.PROMPT_TOO_LONG;
}
// Auth errors
// Auth errors keys, certs, DNS, TLS infrastructure. These will not
// self-heal on retry (a bad API key / expired cert / wrong host won't
// suddenly become valid), so classify as AUTH_ERROR to terminate the
// retry loop and hand off to the fallback chain.
// Infrastructure-level permanent failures checked first:
// DNS resolution (UnknownHostException) misconfigured endpoint
// TLS certificate (CertificateException, SSLPeerUnverifiedException,
// pkix path building failed, certificate verify failed) expired
// or untrusted certs that cannot recover without human intervention
if (msg.contains("UnknownHostException")
|| msg.contains("CertificateException")
|| msg.contains("SSLPeerUnverifiedException")
// Java's ValidatorException emits "PKIX path building failed" with an
// uppercase PKIX, and the error chain is not lower-cased the pattern
// must match the real casing, otherwise the fatal cert failure falls
// through to the retryable SERVER_ERROR bucket and is retried in vain.
|| msg.contains("PKIX path building failed")
|| msg.contains("certificate verify failed")
|| msg.contains("certificate_unknown")) {
return ErrorType.AUTH_ERROR;
}
if (msg.contains("401") || msg.contains("Unauthorized") || msg.contains("Invalid API Key")
|| msg.contains("authentication") || msg.contains("AuthenticationError")) {
return ErrorType.AUTH_ERROR;
@ -404,11 +456,17 @@ public class NodeStreamingChatHelper {
// a different provider may have credits, so we should fall back instead of
// terminating the call. Both OpenAI ("insufficient_quota") and Anthropic
// ("credit balance is too low") use these phrases in 402-class responses.
// Chinese provider patterns (Zhipu 1113, DashScope, general) same hard
// failure semantics: retrying the same provider won't refill the balance.
if (msg.contains("402") || msg.contains("insufficient_quota")
|| msg.contains("credit balance is too low")
|| msg.contains("billing_error") || msg.contains("billing_hard_limit_reached")
|| msg.contains("You exceeded your current quota")
|| msg.contains("quota exceeded") || msg.contains("Quota exceeded")) {
|| msg.contains("quota exceeded") || msg.contains("Quota exceeded")
|| msg.contains("余额不足") || msg.contains("请充值")
|| msg.contains("\"code\":\"1113\"") || msg.contains("\"code\":1113")
|| msg.contains("AccountBalanceNotEnough")
|| msg.contains("balance not enough")) {
return ErrorType.BILLING;
}
// RFC-009 P3.2: MODEL_NOT_FOUND provider rejects the requested model id.
@ -436,19 +494,14 @@ public class NodeStreamingChatHelper {
|| msg.contains("InvalidEndpointOrModel")) {
return ErrorType.MODEL_NOT_FOUND;
}
// Client errors (400 Bad Request unsupported format, invalid params, etc.) NOT retryable.
// DashScope's remaining "InvalidParameter" responses are request-shape bugs, e.g. a reserved
// or illegal tool name ("Tool names are not allowed to be [search]") or an unsupported
// parameter. These fail identically on every provider, so classifying them as CLIENT_ERROR
// (rather than MODEL_NOT_FOUND) keeps the model in the failover pool and surfaces the real
// cause instead of a misleading "model not available" message.
if (msg.contains("400") || msg.contains("Bad Request")
|| msg.contains("invalid_request_error") || msg.contains("unsupported")
|| msg.contains("Tool names are not allowed")
|| msg.contains("InvalidParameter")) {
return ErrorType.CLIENT_ERROR;
}
// Server errors and transient TLS / socket-level network hiccups.
// MUST be checked BEFORE CLIENT_ERROR. 5xx patterns (502/503/504) are
// transient gateway failures that self-heal on retry. If the error chain
// carries BOTH 5xx and 4xx-like keywords (a proxy 502 whose response body
// happens to say "bad request"), the 5xx is the root cause and should win
// retrying a true 400 wastes seconds, but NOT retrying a transient 502
// loses the user's entire conversation turn. MAX_TOTAL_DURATION_MS
// provides the ultimate safety net against unbounded retry.
// Without the TLS-specific patterns, a single SSL fatal alert
// (e.g. bad_record_mac during long-running streams) falls through to
// UNKNOWN non-retryable so one transient handshake glitch surfaces
@ -477,9 +530,27 @@ public class NodeStreamingChatHelper {
// in the response body when their backend is under high load or the
// upstream connection to the model server is disrupted. This is a
// transient server-side failure classify as retryable.
|| msg.contains("network connection error")) {
|| msg.contains("network connection error")
// AI gateway / reverse-proxy rewrites: upstream 5xx (502/503/504)
// surfaced as HTTP 400 with a body that describes the upstream
// outage. These are transient server-side failures retryable.
|| msg.contains("temporarily unavailable")
|| msg.contains("service unavailable")
|| msg.contains("model is overloaded")) {
return ErrorType.SERVER_ERROR;
}
// Client errors (400 Bad Request unsupported format, invalid params, etc.) NOT retryable.
// DashScope's remaining "InvalidParameter" responses are request-shape bugs, e.g. a reserved
// or illegal tool name ("Tool names are not allowed to be [search]") or an unsupported
// parameter. These fail identically on every provider, so classifying them as CLIENT_ERROR
// (rather than MODEL_NOT_FOUND) keeps the model in the failover pool and surfaces the real
// cause instead of a misleading "model not available" message.
if (msg.contains("400") || msg.contains("Bad Request")
|| msg.contains("invalid_request_error") || msg.contains("unsupported")
|| msg.contains("Tool names are not allowed")
|| msg.contains("InvalidParameter")) {
return ErrorType.CLIENT_ERROR;
}
return ErrorType.UNKNOWN;
}
@ -550,6 +621,15 @@ public class NodeStreamingChatHelper {
// 主模型重试循环
StreamResult lastResult = null;
if (!primarySkipped) for (int attempt = 0; attempt <= MAX_RETRIES; attempt++) {
// Time budget check: prevent retries from stalling a single
// conversation turn indefinitely (e.g., a provider that stays
// at 503 for minutes). Aligned with Wiki's maxTotalDurationMs.
long elapsedMs = System.currentTimeMillis() - callStartMs;
if (elapsedMs >= maxTotalDurationMs) {
log.warn("[{}] Primary retry time budget exhausted ({}ms), handing off to fallback chain",
phase, elapsedMs);
break;
}
llmCallCount++;
if (attempt > 0) retryCount++;
lastResult = doStreamCall(chatModel, prompt, conversationId, phase, broadcast, attempt);
@ -601,11 +681,18 @@ public class NodeStreamingChatHelper {
if (lastResult.errorType() == ErrorType.THINKING_BLOCK_ERROR) {
return lastResult; // 已经重试过了
}
// RFC-009: EMPTY_RESPONSE break the primary-retry loop and fall through to
// the fallback chain. Retrying the same model that returned nothing is rarely
// productive; a different provider has a better chance of succeeding.
// EMPTY_RESPONSE transient gateway blip often resolves on same-model
// retry (e.g., proxy timeout returns HTTP 200 with empty body).
// Retry up to MAX_RETRIES_EMPTY_RESPONSE before handing off to the
// fallback chain. A different provider has a better chance of
// succeeding if the same model repeatedly returns nothing.
if (lastResult.errorType() == ErrorType.EMPTY_RESPONSE) {
log.warn("[{}] Primary returned empty response — skipping same-model retries, handing off to fallback chain", phase);
if (attempt < MAX_RETRIES_EMPTY_RESPONSE) {
log.warn("[{}] Primary returned empty response (attempt {}/{}), retrying same model...",
phase, attempt + 1, MAX_RETRIES_EMPTY_RESPONSE + 1);
continue;
}
log.warn("[{}] Primary exhausted empty-response retries — handing off to fallback chain", phase);
recordPrimary(false);
break;
}
@ -616,25 +703,33 @@ public class NodeStreamingChatHelper {
logPerfSummary(phase, conversationId, callStartMs, llmCallCount, retryCount, failoverCount);
return lastResult;
}
// RATE_LIMIT / SERVER_ERROR past their retry budget are provider-level
// failures: the same model will not recover within this turn, but a
// different provider can. Break to the fallback chain instead of
// returning recordPrimary(false) runs once at the post-loop provider
// health check below, and if every fallback also fails the chain
// walker re-surfaces this same error to the caller.
// RATE_LIMIT / SERVER_ERROR / UNKNOWN past their retry budget are
// provider-level failures: the same model will not recover within
// this turn, but a different provider can. Break to the fallback
// chain instead of returning recordPrimary(false) runs once at
// the post-loop provider health check below, and if every fallback
// also fails the chain walker re-surfaces this same error to the
// caller.
// UNKNOWN errors are included defensively: an error we can't
// classify may be a transient (mis-classified by our keyword
// patterns) or a fatal (truly new error shape). Retrying with a
// smaller budget (MAX_RETRIES_UNKNOWN=5 vs MAX_RETRIES=10) is
// safer than immediate termination MAX_TOTAL_DURATION_MS provides
// the ultimate safety net.
if (lastResult.errorType() == ErrorType.RATE_LIMIT
|| lastResult.errorType() == ErrorType.SERVER_ERROR) {
|| lastResult.errorType() == ErrorType.SERVER_ERROR
|| lastResult.errorType() == ErrorType.UNKNOWN) {
log.warn("[{}] Primary exhausted retries (type={}) — handing off to fallback chain",
phase, lastResult.errorType());
break;
}
// Any other non-null errored result (e.g. UNKNOWN) that doStreamCall
// chose NOT to retry must exit otherwise we silently spin through
// attempts and waste seconds per turn on unrecoverable errors like
// DashScope's "url error" / unknown model.
// Any truly unhandled error type safety net. Prefer falling back
// over terminating the entire call. If this branch is ever hit in
// production, the type should be added explicitly above.
recordPrimary(false);
logPerfSummary(phase, conversationId, callStartMs, llmCallCount, retryCount, failoverCount);
return lastResult;
log.warn("[{}] Primary returned unhandled error type={} — handing off to fallback chain",
phase, lastResult.errorType());
break;
}
// lastResult == null 表示需要重试
}
@ -813,10 +908,10 @@ public class NodeStreamingChatHelper {
String conversationId, String phase,
boolean broadcast, int attempt) {
if (attempt > 0) {
long delay = Math.min(BACKOFF_BASE_MS * (1L << (attempt - 1)), BACKOFF_CAP_MS);
long delay = Math.min(backoffBaseMs * (1L << (attempt - 1)), backoffCapMs);
// 加入 jitter 防止雷群效应
delay += ThreadLocalRandom.current().nextLong(0, Math.max(1, delay / 2));
delay = Math.min(delay, BACKOFF_CAP_MS);
delay = Math.min(delay, backoffCapMs);
log.warn("[{}] Retry attempt {}/{} after {}ms for conversation {}",
phase, attempt, MAX_RETRIES, delay, conversationId);
// 广播给前端用户可见的重试倒计时
@ -1155,12 +1250,20 @@ public class NodeStreamingChatHelper {
conversationId, phase, errorType);
}
// Rate limit / Server error: retryable, but with different budgets.
// Rate limit / Server error / Unknown: retryable, but with different budgets.
// RATE_LIMIT: cap at 2 retries then failover (RFC 06 D-2).
// SERVER_ERROR: keep full MAX_RETRIES upstream flaps often self-heal.
if (errorType == ErrorType.RATE_LIMIT || errorType == ErrorType.SERVER_ERROR) {
int effectiveMaxRetries = (errorType == ErrorType.RATE_LIMIT)
? MAX_RETRIES_RATE_LIMIT : MAX_RETRIES;
// UNKNOWN: conservative cap (5 vs 10). Defensive: retry what we can't
// classify, but with a smaller budget to avoid masking truly fatal
// errors. MAX_TOTAL_DURATION_MS provides the ultimate safety net.
if (errorType == ErrorType.RATE_LIMIT
|| errorType == ErrorType.SERVER_ERROR
|| errorType == ErrorType.UNKNOWN) {
int effectiveMaxRetries = switch (errorType) {
case RATE_LIMIT -> MAX_RETRIES_RATE_LIMIT;
case UNKNOWN -> MAX_RETRIES_UNKNOWN;
default -> MAX_RETRIES;
};
if (attempt < effectiveMaxRetries) {
log.warn("[{}] Retryable error (attempt {}/{}, type={}): {}",
phase, attempt, effectiveMaxRetries, errorType, error.getMessage());

View File

@ -184,6 +184,44 @@ public class ToolExecutionExecutor {
return StructuredTruncator.truncate(result, headLen, tailLen, marker);
}
/**
* Appended to the cross-platform shopping recommendation tool's result so the
* model reliably renders products as chat cards instead of a markdown table.
* The tool's MCP schema is globally callable, so a model can invoke it without
* ever loading the skill's instructions this puts the rendering contract
* where the model always sees it: in the tool result itself.
*/
private static final String PRODUCT_CARD_RENDER_DIRECTIVE =
"\n\n[Rendering directive] Render these recommendations for the user as a"
+ " fenced code block whose language tag is exactly `product-cards`, containing a"
+ " JSON array. Each element: {name, url, imageUrl, price, originalPrice, lowestPrice,"
+ " platformLabel, shopName, purchaseAdvice}. Copy `url` and `imageUrl` verbatim from"
+ " the result above (never invent or alter them). The chat UI turns this block into"
+ " clickable product cards with a buy button. Do NOT use a markdown table or inline"
+ " image markdown for these products. You may add a short intro sentence and purchase"
+ " tips around the block.";
/**
* Returns {@code true} when the tool is the cross-platform shopping
* recommendation tool and its result actually carries product records
* (so timeouts / empty results fall through to the model's own fallback).
*/
static boolean shouldAppendProductCardDirective(String toolName, String result) {
if (toolName == null || result == null) return false;
if (!toolName.contains("ckjia_shopping_recom")) return false;
return result.contains("recommendations")
|| result.contains("imageUrl")
|| result.contains("priceTag")
|| result.contains("markdownLink");
}
/** Appends {@link #PRODUCT_CARD_RENDER_DIRECTIVE} when applicable, else returns the result unchanged. */
static String withProductCardDirective(String toolName, String result) {
return shouldAppendProductCardDirective(toolName, result)
? result + PRODUCT_CARD_RENDER_DIRECTIVE
: result;
}
private final Map<String, ToolCallback> toolCallbackMap;
/**
* Maps a normalized tool name (lowercase snake_case, with `_tool`/`_function`
@ -675,8 +713,10 @@ public class ToolExecutionExecutor {
log.info("[ToolExecutor] Pre-approved tool {} returned {} chars{}", toolName, rawLen,
result != null && result.length() < rawLen ? " (now " + result.length() + " after spill/truncate)" : "");
events.add(GraphEventPublisher.toolComplete(toolCall.id(), toolName, result, true));
// Append the card-rendering directive to the LLM-facing response only,
// leaving the broadcast tool-result panel unchanged.
return new ToolResponseMessage.ToolResponse(
toolCall.id(), toolName, result != null ? result : "");
toolCall.id(), toolName, withProductCardDirective(toolName, result != null ? result : ""));
} catch (Exception e) {
log.error("[ToolExecutor] Pre-approved tool {} failed: {}", toolName, e.getMessage());
String safeError = isReturnDirect(callback)
@ -905,8 +945,10 @@ public class ToolExecutionExecutor {
GraphEventPublisher.toolComplete(pc.toolCall.id(), toolName, result, true).data());
streamTracker.updateRunningTool(pc.conversationId, null);
}
// Append the card-rendering directive to the LLM-facing response only,
// leaving the broadcast tool-result panel unchanged.
return new ToolResponseMessage.ToolResponse(
pc.toolCall.id(), toolName, result != null ? result : "");
pc.toolCall.id(), toolName, withProductCardDirective(toolName, result != null ? result : ""));
} catch (Exception e) {
log.error("[ToolExecutor] Tool {} execution failed: {}", toolName, e.getMessage(), e);
// RFC-052: for returnDirect tools, even the error message is
@ -946,7 +988,10 @@ public class ToolExecutionExecutor {
ToolInvocationContext guardCtx = ToolInvocationContext.of(
toolName, java.util.Map.of(), arguments,
conversationId, agentId,
/*channelType*/ null, requesterId, workspaceId);
/*channelType*/ null, requesterId, workspaceId)
// Carry the active workspace base path so a guardian can enforce
// the filesystem boundary before approval (issue #313).
.withWorkspaceBasePath(origin != null ? origin.workspaceBasePath() : null);
if (toolGuardService != null) {
GuardEvaluation evaluation = toolGuardService.evaluate(guardCtx);

View File

@ -8,6 +8,7 @@ import vip.mate.agent.graph.state.DirectToolOutput;
import vip.mate.agent.graph.state.FinishReason;
import vip.mate.agent.graph.state.MateClawStateAccessor;
import vip.mate.agent.graph.state.SourceEvidenceLedger;
import vip.mate.common.text.MarkdownNormalizer;
import vip.mate.tool.document.GeneratedFileCache;
import java.util.List;
@ -39,12 +40,25 @@ public class FinalAnswerNode implements NodeAction {
*/
private final GeneratedFileCache generatedFileCache;
/**
* Kill-switch for the deterministic Markdown cleanup applied to the answer
* body. {@code true} (default) runs {@link MarkdownNormalizer}; set to
* {@code false} to surface model output verbatim if a normalization edge
* case ever mangles a legitimate answer in production.
*/
private final boolean markdownNormalizeEnabled;
public FinalAnswerNode() {
this(null);
this(null, true);
}
public FinalAnswerNode(GeneratedFileCache generatedFileCache) {
this(generatedFileCache, true);
}
public FinalAnswerNode(GeneratedFileCache generatedFileCache, boolean markdownNormalizeEnabled) {
this.generatedFileCache = generatedFileCache;
this.markdownNormalizeEnabled = markdownNormalizeEnabled;
}
@Override
@ -161,6 +175,7 @@ public class FinalAnswerNode implements NodeAction {
// validation so the validator sees the user-visible warning rather
// than treating the fake link as a "reference".
finalAnswer = scrubFakeUrls(finalAnswer);
finalAnswer = accessor.sourceEvidenceLedger().appendWikiSourceTable(finalAnswer);
SourceEvidenceLedger.Validation validation = accessor.sourceEvidenceLedger().validateAnswer(finalAnswer);
if (finishReason == FinishReason.NORMAL && !validation.valid()) {
@ -170,6 +185,17 @@ public class FinalAnswerNode implements NodeAction {
validation.unsupportedReferences());
}
// Deterministic Markdown cleanup on the model-generated answer body. LLMs
// routinely emit malformed Markdown (missing heading spaces, glued `---`,
// unaligned table pipes) that prompt rules fail to prevent; this fixes the
// mechanical defects before the answer is persisted / sent to channels.
// Verbatim tool output (RETURN_DIRECT) and approval-wait paths return early
// above and are intentionally left untouched. Gated so operators can turn
// the rewrite off (mate.agent.markdown-normalize-enabled=false).
if (markdownNormalizeEnabled) {
finalAnswer = MarkdownNormalizer.normalize(finalAnswer);
}
// Build the event list. Always carries the finish_reason event so
// downstream consumers (memory gate, channel accumulator, message
// metadata persistence) see a machine-readable status. When the
@ -211,9 +237,9 @@ public class FinalAnswerNode implements NodeAction {
}
private static String appendEvidenceWarning(String answer, List<String> unsupportedReferences) {
return answer + "\n\n[证据不足] 以下源码引用未出现在已读取/搜索到的工具证据中:"
return answer + "\n\n[证据不足] 以下引用未出现在本轮已读取/搜索到的工具证据中,或缺少有效来源标注"
+ String.join(", ", unsupportedReferences)
+ "。请继续读取相关文件后再下结论。";
+ "。请继续检索/读取相关证据后再下结论。";
}
/**

View File

@ -78,7 +78,12 @@ public class GoalEvaluationNode implements NodeAction {
MateClawStateAccessor accessor = new MateClawStateAccessor(state);
Optional<Object> goalOpt = accessor.activeGoal();
// Resolve the active goal from the turn-start snapshot, falling back to
// a conversation lookup. The fallback is what makes a goal created
// MID-TURN (the agent calls setGoal, which only writes the DB) get
// evaluated on the very turn it was set otherwise ACTIVE_GOAL is empty
// in this run's state and the goal would sit inert until the next message.
Optional<GoalEntity> goalOpt = resolveActiveGoal(state, goalService);
if (goalOpt.isEmpty()) {
return Map.of();
}
@ -94,26 +99,32 @@ public class GoalEvaluationNode implements NodeAction {
// chat composable's `message_complete` handler optimistically sets
// evaluating=true; without a balancing event the ring would stay
// in that state forever after e.g. a max-iterations turn.
Long goalIdForEvents = (goalOpt.get() instanceof GoalEntity ge) ? ge.getId() : null;
Long goalIdForEvents = goalOpt.get().getId();
// ReAct path: FinalAnswerNode wrote a canonical finishReason that
// determines whether this turn counts. Plan-Execute usually doesn't
// set finishReason on the happy path, so we only enforce these
// exit conditions in REACT mode + the universal awaiting_approval
// gate that both flavors share.
// A turn that hit the ReAct iteration cap. Continuing it needs a FRESH
// iteration budget (a "hard continuation"), handled in the follow-up
// branch below; capture it here while finishReason is still authoritative.
boolean reactIterationCapReached = flavor == GraphFlavor.REACT
&& isIterationCapReached(accessor.finishReason());
if (flavor == GraphFlavor.REACT) {
String fr = accessor.finishReason();
if (FinishReason.EVIDENCE_INSUFFICIENT.getValue().equals(fr)
|| FinishReason.STOPPED.getValue().equals(fr)
|| FinishReason.ERROR_FALLBACK.getValue().equals(fr)
|| FinishReason.RETURN_DIRECT.getValue().equals(fr)
|| FinishReason.MAX_ITERATIONS_REACHED.getValue().equals(fr)) {
if (isHardSkipFinishReason(fr)) {
log.debug("[GoalEvaluationNode] skipping evaluation (REACT finishReason={})", fr);
return MateClawStateAccessor.output()
.goalEvaluatedThisRun(true)
.events(List.of(skippedEvent(goalIdForEvents, "react_finish_reason:" + fr)))
.build();
}
// MAX_ITERATIONS_REACHED and EVIDENCE_INSUFFICIENT intentionally
// fall through: both mean "answer produced but the goal is likely
// unmet", which is exactly when a corrective follow-up helps.
// Max-iterations additionally needs a fresh budget (see below).
}
if (accessor.awaitingApproval()) {
return MateClawStateAccessor.output()
@ -122,14 +133,7 @@ public class GoalEvaluationNode implements NodeAction {
.build();
}
Object goalObj = goalOpt.get();
if (!(goalObj instanceof GoalEntity goal)) {
log.warn("[GoalEvaluationNode] ACTIVE_GOAL is not a GoalEntity: {}", goalObj.getClass());
return MateClawStateAccessor.output()
.goalEvaluatedThisRun(true)
.events(List.of(skippedEvent(null, "non_goal_entity")))
.build();
}
GoalEntity goal = goalOpt.get();
String terminal = accessor.terminalAnswer();
if (terminal.isEmpty()) {
@ -226,6 +230,9 @@ public class GoalEvaluationNode implements NodeAction {
}
int followupCountThisRun = accessor.goalFollowupCount();
int hardContinuationCount = accessor.goalHardContinuationCount();
int hardCap = Math.min(properties.getMaxHardContinuationsPerRun(),
GoalProperties.MAX_HARD_CONTINUATIONS_CEILING);
Optional<String> followup;
try {
followup = followupService.maybeBuildFollowup(refreshed, result);
@ -241,11 +248,19 @@ public class GoalEvaluationNode implements NodeAction {
// active and the cross-message turn / LLM budget (or the user) carries
// it on.
boolean perRunCapReached = followupCountThisRun >= properties.getMaxFollowupsPerRun();
if (followup.isPresent() && perRunCapReached) {
log.info("[GoalEvaluationNode] per-run followup cap reached ({}/{}) for goal={}; ending this run",
followupCountThisRun, properties.getMaxFollowupsPerRun(), refreshed.getId());
// A max-iterations continuation re-runs a FULL fresh ReAct segment
// (iteration budget reset), so it carries a tighter, dedicated cap on
// top of the per-run follow-up cap and is sized into the graph
// recursion ceiling. hardCap==0 keeps the legacy behaviour (a
// max-iterations turn simply ends the run).
boolean hardCapReached = reactIterationCapReached && hardContinuationCount >= hardCap;
if (followup.isPresent() && (perRunCapReached || hardCapReached)) {
log.info("[GoalEvaluationNode] follow-up suppressed for goal={} " +
"(followups {}/{}, hardContinuations {}/{}, iterationCapReached={}); ending this run",
refreshed.getId(), followupCountThisRun, properties.getMaxFollowupsPerRun(),
hardContinuationCount, hardCap, reactIterationCapReached);
}
if (followup.isPresent() && !perRunCapReached) {
if (followup.isPresent() && !perRunCapReached && !hardCapReached) {
try {
goalService.recordFollowupInjected(refreshed.getId(), followup.get());
} catch (Throwable t) {
@ -283,6 +298,20 @@ public class GoalEvaluationNode implements NodeAction {
out.clearFinalAnswer()
.clearFinishReason()
.messages(List.of((Message) new UserMessage(followup.get())));
if (reactIterationCapReached) {
// Hard continuation: the run's iteration budget is spent, so
// grant a brand-new ReAct segment. Reset the counter, clear
// the stale limit-exceeded draft/flag (FinalAnswerNode prefers
// the draft over a freshly reasoned answer) and any latched
// error, and advance the dedicated hard-continuation counter.
out.iterationCount(0)
.clearLimitExceededDraft()
.error("")
.goalHardContinuationCount(hardContinuationCount + 1);
log.info("[GoalEvaluationNode] hard continuation {}/{} for goal={} " +
"(fresh ReAct iteration budget after max-iterations turn)",
hardContinuationCount + 1, hardCap, refreshed.getId());
}
} else {
// Plan-Execute: wipe the wider mid-pass + terminal state.
// WORKING_CONTEXT and PlanStateKeys.GOAL are intentionally
@ -318,6 +347,70 @@ public class GoalEvaluationNode implements NodeAction {
.build();
}
/**
* Resolve the active goal for this run: prefer the turn-start
* {@code ACTIVE_GOAL} snapshot; if absent, fall back to a conversation
* lookup so a goal created mid-turn (via {@code setGoal}, which only writes
* the DB) is still evaluated on the turn it was set.
*
* <p>Shared by {@link #apply} and the {@code FinalAnswer/PlanSummary
* GoalEvaluation} routing edges so both agree on whether a goal is active.
* The DB fallback costs one indexed lookup per terminal turn whose snapshot
* is empty; callers should additionally gate on {@code properties.isEnabled()}
* to skip it when the feature is off.
*/
public static Optional<GoalEntity> resolveActiveGoal(OverAllState state, GoalService goalService) {
MateClawStateAccessor a = new MateClawStateAccessor(state);
Optional<Object> snapshot = a.activeGoal();
if (snapshot.isPresent() && snapshot.get() instanceof GoalEntity ge) {
return Optional.of(ge);
}
if (goalService == null) {
return Optional.empty();
}
String conversationId = a.conversationId();
if (conversationId == null || conversationId.isBlank()) {
return Optional.empty();
}
try {
return Optional.ofNullable(goalService.findActiveByConversation(conversationId));
} catch (Throwable t) {
log.warn("[GoalEvaluationNode] active-goal fallback lookup failed for conversation={}: {}",
conversationId, t.toString());
return Optional.empty();
}
}
/**
* REACT-mode finish reasons that should neither count toward the goal nor
* trigger a continuation:
* <ul>
* <li>{@code STOPPED} the user halted the run; don't fight them.</li>
* <li>{@code RETURN_DIRECT} a tool produced the answer verbatim; this
* is not goal-progress reasoning work to evaluate or continue.</li>
* <li>{@code ERROR_FALLBACK} a fatal error already failed the turn;
* re-running immediately would just re-fail.</li>
* </ul>
* Other terminal reasons notably {@code MAX_ITERATIONS_REACHED} and
* {@code EVIDENCE_INSUFFICIENT} mean "answer produced but the goal is
* likely unmet", which is exactly when a corrective follow-up helps, so
* they are deliberately NOT skipped.
*/
static boolean isHardSkipFinishReason(String finishReason) {
return FinishReason.STOPPED.getValue().equals(finishReason)
|| FinishReason.RETURN_DIRECT.getValue().equals(finishReason)
|| FinishReason.ERROR_FALLBACK.getValue().equals(finishReason);
}
/**
* True when the terminal turn hit the ReAct iteration cap. Continuing such
* a turn requires a fresh iteration budget (a "hard continuation"), because
* the run's shared budget is already exhausted.
*/
static boolean isIterationCapReached(String finishReason) {
return FinishReason.MAX_ITERATIONS_REACHED.getValue().equals(finishReason);
}
/** Stand-in for a missing {@code GraphEventPublisher.custom()} factory. */
private static GraphEventPublisher.GraphEvent goalEvent(String type, Map<String, Object> data) {
return new GraphEventPublisher.GraphEvent(type, Map.copyOf(data), System.currentTimeMillis());

View File

@ -32,6 +32,18 @@ public class ObservationNode implements NodeAction {
private final ObservationProcessor observationProcessor;
private final vip.mate.channel.web.ChatStreamTracker streamTracker;
/**
* Progressive-disclosure meta-tools that perform setup, not real work. A
* round whose entire batch is one of these is refunded its iteration (see
* {@link MateClawStateKeys#ITERATION_REFUND_COUNT}). Mirrors the authoritative
* set in {@code DefaultToolDisclosureService.ALWAYS_CORE}.
*/
private static final java.util.Set<String> DISCLOSURE_TOOLS =
java.util.Set.of("load_skill", "enable_tool");
/** Per-run cap on iteration refunds — keeps a load-skill-only model from looping forever. */
private static final int MAX_ITERATION_REFUNDS_PER_RUN = 3;
public ObservationNode(ObservationProcessor observationProcessor) {
this(observationProcessor, null);
}
@ -56,14 +68,29 @@ public class ObservationNode implements NodeAction {
int currentIteration = accessor.iterationCount();
int maxIterations = accessor.maxIterations();
int nextIteration = currentIteration + 1;
log.info("[ObservationNode] Iteration {}/{}", nextIteration, maxIterations);
// 提取最新的工具结果并处理
List<ToolResponseMessage.ToolResponse> toolResults =
state.<List<ToolResponseMessage.ToolResponse>>value(TOOL_RESULTS).orElse(List.of());
// Iteration refund: a round whose entire batch was progressive-disclosure
// setup (load_skill / enable_tool) did no real work, so don't charge it an
// iteration otherwise a tight budget loses a step to the load-then-use
// two-step. Bounded by MAX_ITERATION_REFUNDS_PER_RUN so a model that only
// ever loads skills can't dodge the budget forever.
int refundCount = accessor.iterationRefundCount();
boolean setupOnlyRound = !toolResults.isEmpty()
&& toolResults.stream().allMatch(tr -> DISCLOSURE_TOOLS.contains(tr.name()));
boolean refundIteration = setupOnlyRound && refundCount < MAX_ITERATION_REFUNDS_PER_RUN;
int nextIteration = refundIteration ? currentIteration : currentIteration + 1;
if (refundIteration) {
log.info("[ObservationNode] Iteration refunded (setup-only round, refunds {}/{}); staying at {}/{}",
refundCount + 1, MAX_ITERATION_REFUNDS_PER_RUN, nextIteration, maxIterations);
} else {
log.info("[ObservationNode] Iteration {}/{}", nextIteration, maxIterations);
}
// 将每个工具结果通过 ObservationProcessor 标准化和截断
List<String> processedObservations = toolResults.stream()
.map(tr -> observationProcessor.process(tr.name(), tr.responseData()))
@ -121,6 +148,10 @@ public class ObservationNode implements NodeAction {
.shouldSummarize(shouldSummarize)
.toolCallCount(newToolCallCount);
if (refundIteration) {
builder.iterationRefundCount(refundCount + 1);
}
// Close out the iteration we just observed. We use currentIteration
// (not nextIteration) so the index pairs with whatever
// iteration_start the ReasoningNode emitted at the top of this turn.

View File

@ -141,19 +141,96 @@ public class ReasoningNode implements NodeAction {
+ "required step is already done, output the final answer to the user now.";
/**
* A turn carrying no tool call, no content, and no thinking is not a usable
* answer it would route to the final-answer branch as an empty string and
* terminate the run. Fatal / prompt-too-long / partial results are handled by
* their own branches and must not be misread as "empty".
* Continuation nudge for the most common premature-stop pattern: an empty
* turn immediately after a successful tool call. The tool result is already
* in context but the model stopped before writing the user-facing answer
* (e.g. a download URL produced by a send-file tool). Anchoring the nudge to
* the tool result recovers the answer in the same run instead of leaving the
* user to send another message to resume.
*/
static boolean isEmptyCompletion(NodeStreamingChatHelper.StreamResult result) {
private static final String POST_TOOL_EMPTY_NUDGE =
"上一步工具已成功返回(结果在上文)。请基于工具结果直接给出面向用户的最终答复"
+ "(例如下载地址 / 执行结论),不要停在思考阶段,也不要只描述\"接下来要做什么\"";
/**
* Continuation nudge for a turn that carries reasoning/thinking but no
* visible content and no tool call. Interleaved-thinking models sometimes
* "decide" the task is done in their reasoning yet never emit the answer
* text; this re-prompts them to write it (or call the next tool).
*/
private static final String THINKING_ONLY_NUDGE =
"你已完成思考但还没有输出正文。请现在把面向用户的最终答案写出来;"
+ "如果还有未完成的步骤,则立即调用对应工具。";
/**
* Why a no-tool-call reasoning turn cannot yet be accepted as a final answer.
* Both non-FINAL states route a turn into the bounded continuation-nudge loop
* instead of letting the final-answer branch emit an empty string and end the
* run prematurely.
*/
enum ContinuationIntent {
/**
* Real user-facing content present, or a tool call, or a failure owned by
* another branch (fatal / prompt-too-long / partial) finalize normally.
*/
FINAL,
/**
* Reasoning/thinking present but no visible content and no tool call. The
* model "thought it was done" without writing the answer common on
* interleaved-thinking models after a tool result. Nudge it to emit it.
*/
THINKING_ONLY,
/** No content, no thinking, no tool call — a fully blank turn. Nudge to continue. */
BLANK,
}
/**
* Classify a no-tool-call turn's continuation intent. Fatal / prompt-too-long
* / partial results are handled by their own branches and must not be misread
* as needing a nudge.
*/
static ContinuationIntent classifyContinuation(NodeStreamingChatHelper.StreamResult result) {
if (result == null || result.hasToolCalls() || result.hasFatalError()
|| result.isPromptTooLong() || result.partial()) {
return false;
return ContinuationIntent.FINAL;
}
boolean noContent = result.text() == null || result.text().isBlank();
if (!noContent) {
return ContinuationIntent.FINAL;
}
boolean noThinking = result.thinking() == null || result.thinking().isBlank();
return noContent && noThinking;
return noThinking ? ContinuationIntent.BLANK : ContinuationIntent.THINKING_ONLY;
}
/**
* A fully blank turn (no tool call, no content, no thinking) is not a usable
* answer it would route to the final-answer branch as an empty string and
* terminate the run. Retained as a thin predicate over {@link #classifyContinuation}.
*/
static boolean isEmptyCompletion(NodeStreamingChatHelper.StreamResult result) {
return classifyContinuation(result) == ContinuationIntent.BLANK;
}
/**
* True when the newest conversational turn in the model input is a tool
* response i.e. the model is about to reason over a fresh tool result. Used
* to pick the result-anchored continuation nudge for the common "empty turn
* right after a tool call" stop pattern.
*/
static boolean lastTurnIsToolResponse(List<Message> messages) {
if (messages == null) {
return false;
}
for (int i = messages.size() - 1; i >= 0; i--) {
Message m = messages.get(i);
if (m instanceof ToolResponseMessage) {
return true;
}
if (m instanceof UserMessage || m instanceof AssistantMessage) {
return false;
}
}
return false;
}
/**
@ -194,6 +271,40 @@ public class ReasoningNode implements NodeAction {
+ " · ledger snapshot 永远显示初始状态,对你毫无帮助\n\n"
+ "**例外**:单一问题、简单问答、不可拆解的请求 — 不需要用。\n";
private static final String GROUNDED_CONTRACT = "\n\n"
+ "## 回答来源约束(强制规则)\n\n"
+ "**核心原则**:你的回答必须完全基于工具返回的信息(证据),不得使用内部知识编造内容。\n\n"
+ "**必须遵守**\n"
+ "1. **仅据证据作答**:如果工具返回的信息不足以回答问题,必须明确说明\"根据现有信息无法回答此问题\"\n"
+ "2. **标记引用来源**:回答中引用的每个事实性陈述都必须用方括号数字标记来源,例如 [1]、[2]。\n"
+ "3. **文末列出来源**:在回答末尾列出所有引用的来源列表,格式为:\n"
+ " [1] 页面标题 - 章节(如有)\n"
+ " [2] 页面标题 - 章节(如有)\n"
+ "4. **禁止捏造来源**:不得引用未在本次对话中通过工具获取的页面或文件。\n"
+ "5. **内容忠实**:必须准确反映证据内容,不得歪曲、编造或过度推断。\n\n"
+ "**违规后果**:未按规则引用来源或使用未验证的信息将导致回答被拒绝。\n";
private static String buildGroundedSystemPrompt(String basePrompt, boolean groundingEnforced) {
String prompt = basePrompt + TOOL_USE_ENFORCEMENT;
return groundingEnforced ? prompt + GROUNDED_CONTRACT : prompt;
}
/**
* The grounded-answer contract (cite-or-refuse) only fits agents that retrieve
* from a knowledge base. Detecting a bound {@code wiki_*} tool scopes the strict
* regime to those scenarios instead of degrading every agent a casual agent
* with no KB should not be forced to refuse or emit [n] citations.
*/
private boolean hasWikiTool() {
if (toolCallbacks == null) {
return false;
}
return toolCallbacks.stream().anyMatch(cb -> {
String name = cb.getToolDefinition().name();
return name != null && name.toLowerCase(Locale.ROOT).replace("-", "_").startsWith("wiki_");
});
}
private final ChatModel chatModel;
private final List<ToolCallback> toolCallbacks;
/**
@ -448,17 +559,16 @@ public class ReasoningNode implements NodeAction {
// ======= 构建 Prompt =======
String systemPrompt = accessor.systemPrompt();
// Append a tool-use enforcement clause to every ReasoningNode call.
// Without it, some models (notably DeepSeek thinking and Claude Opus)
// tend to "narrate" emit a final_answer like "现在直接生成立项材料
// docx" instead of actually calling renderDocx, which makes the
// graph silently terminate at final_answer_node with the narration
// as the user-facing reply.
// Tool-use enforcement is always appended: without it some models tend to
// "narrate" instead of calling tools. The grounded contract (cite-or-refuse)
// is appended only when the agent has a knowledge-base (wiki_*) tool bound,
// so KB-grounded scenarios get strict source attribution while general
// agents keep their normal answering behaviour.
//
// Appended at runtime rather than woven into the AgentEntity-stored
// prompt so it stays out of the user-editable agent UI but is still
// always-on for the runtime LLM.
systemPrompt = systemPrompt + TOOL_USE_ENFORCEMENT;
systemPrompt = buildGroundedSystemPrompt(systemPrompt, hasWikiTool());
List<Message> messages = accessor.messages();
// Per-loop budget: bound the working message list a single Reasoning
@ -509,6 +619,8 @@ public class ReasoningNode implements NodeAction {
String workspaceBasePath = state.value(vip.mate.agent.graph.state.MateClawStateKeys.WORKSPACE_BASE_PATH, "");
String agentIdStr = state.value(MateClawStateKeys.AGENT_ID, "");
String userMsg = state.value(MateClawStateKeys.USER_MESSAGE, "");
String runtimeModelName = state.value(MateClawStateKeys.RUNTIME_MODEL_NAME, "");
String runtimeProviderId = state.value(MateClawStateKeys.RUNTIME_PROVIDER_ID, "");
// Build the non-history prefix ONCE. The PTL retry branch below
// reuses this list verbatim so the retried prompt has exactly the
@ -517,7 +629,7 @@ public class ReasoningNode implements NodeAction {
// segment which led to "answer regressed after compaction"
// complaints on long sessions.
List<Message> nonHistoryPrefix = buildNonHistoryPrefix(systemPrompt, workspaceBasePath, agentIdStr, userMsg,
accessor.chatOrigin());
accessor.chatOrigin(), runtimeModelName, runtimeProviderId);
// Append the runtime-rendered skill catalog as a SEPARATE SystemMessage
// right after the skeleton system prompt. Keeping it out of the baked
@ -675,22 +787,40 @@ public class ReasoningNode implements NodeAction {
}
}
// Empty-completion guard: a turn with no tool call, no content, and
// no thinking is not a real answer. Under heavy message-window
// trimming on long multi-step tasks the model occasionally emits a
// blank turn; the final-answer branch would then treat it as "done"
// (finalAnswer="") and end the run prematurely (observed: a 10-item
// research task stopping at item 2). Re-prompt it to continue
// bounded, so a model that genuinely has nothing left still
// terminates cleanly through the normal empty-answer path below.
// Continuation guard: a no-tool-call turn with no visible content is
// not a real answer, whether it is fully blank or carries only
// reasoning. The final-answer branch would otherwise treat it as
// "done" (finalAnswer="") and end the run prematurely. Two shapes:
// BLANK no content, no thinking, no tool call. Seen under
// heavy message-window trimming on long multi-step
// tasks (a 10-item research task stopping at item 2).
// THINKING_ONLY reasoning present but no content and no tool call.
// Interleaved-thinking models "decide" they are done
// in their reasoning yet never emit the answer text;
// most often right after a tool result (e.g. a
// send-file tool succeeds but the download URL is
// never written, so the user has to send another
// message to resume).
// Re-prompt to continue bounded, so a model that genuinely has
// nothing left still terminates cleanly through the normal
// empty-answer path below. When the newest turn is a tool result, an
// answer-anchored nudge recovers the user-facing reply in the same run.
int emptyRetries = 0;
while (emptyRetries < MAX_EMPTY_COMPLETION_RETRIES && isEmptyCompletion(result)) {
for (ContinuationIntent intent = classifyContinuation(result);
emptyRetries < MAX_EMPTY_COMPLETION_RETRIES && intent != ContinuationIntent.FINAL;
intent = classifyContinuation(result)) {
emptyRetries++;
log.warn("[ReasoningNode] Empty LLM completion (no tool call / content / thinking); "
+ "nudging to continue (retry {}/{}), conv={}",
emptyRetries, MAX_EMPTY_COMPLETION_RETRIES, conversationId);
boolean afterTool = lastTurnIsToolResponse(promptMessages);
String nudge = afterTool
? POST_TOOL_EMPTY_NUDGE
: (intent == ContinuationIntent.THINKING_ONLY
? THINKING_ONLY_NUDGE
: EMPTY_COMPLETION_NUDGE);
log.warn("[ReasoningNode] {} completion (afterTool={}); nudging to continue "
+ "(retry {}/{}), conv={}",
intent, afterTool, emptyRetries, MAX_EMPTY_COMPLETION_RETRIES, conversationId);
List<Message> nudgedMessages = new ArrayList<>(promptMessages);
nudgedMessages.add(new UserMessage(EMPTY_COMPLETION_NUDGE));
nudgedMessages.add(new UserMessage(nudge));
Prompt nudgePrompt = new Prompt(nudgedMessages, options);
nextLlmCallCount++;
result = streamingHelper.streamCall(
@ -863,12 +993,14 @@ public class ReasoningNode implements NodeAction {
"iteration", accessor.iterationCount(),
"answerChars", content != null ? content.length() : 0
));
String answerWithSources = accessor.sourceEvidenceLedger()
.appendWikiSourceTable(content != null ? content : "");
SourceEvidenceLedger.Validation validation =
accessor.sourceEvidenceLedger().validateAnswer(content != null ? content : "");
accessor.sourceEvidenceLedger().validateAnswer(answerWithSources);
boolean evidenceInsufficient = !validation.valid();
String finalAnswer = evidenceInsufficient
? evidenceWarning(validation.unsupportedReferences())
: (content != null ? content : "");
: answerWithSources;
if (evidenceInsufficient) {
log.warn("[ReasoningNode] Evidence insufficient for final answer, unsupportedReferences={}",
validation.unsupportedReferences());
@ -891,7 +1023,7 @@ public class ReasoningNode implements NodeAction {
.currentPhase("reasoning")
.streamedContent(evidenceInsufficient ? (content != null ? content : "") : "")
.finishReason(evidenceInsufficient ? FinishReason.EVIDENCE_INSUFFICIENT : FinishReason.NORMAL)
.contentStreamed(!evidenceInsufficient)
.contentStreamed(!evidenceInsufficient && Objects.equals(answerWithSources, content != null ? content : ""))
.thinkingStreamed(!result.thinking().isEmpty())
.llmCallCount(nextLlmCallCount)
.mergeUsage(state, result)
@ -901,9 +1033,9 @@ public class ReasoningNode implements NodeAction {
}
private static String evidenceWarning(List<String> unsupportedReferences) {
return "\n\n[证据不足] 以下源码引用未出现在已读取/搜索到的工具证据中:"
return "\n\n[证据不足] 以下引用未出现在本轮已读取/搜索到的工具证据中,或缺少有效来源标注"
+ String.join(", ", unsupportedReferences)
+ "。请继续读取相关文件后再下结论。";
+ "。请继续检索/读取相关证据后再下结论。";
}
private AssistantMessage.ToolCall deserializeToolCall(String json) {
@ -967,10 +1099,13 @@ public class ReasoningNode implements NodeAction {
String workspaceBasePath,
String agentIdStr,
String userMsg,
vip.mate.agent.context.ChatOrigin chatOrigin) {
vip.mate.agent.context.ChatOrigin chatOrigin,
String runtimeModelName,
String runtimeProviderId) {
List<Message> prefix = new ArrayList<>();
prefix.add(new SystemMessage(systemPrompt));
prefix.add(new UserMessage(RuntimeContextInjector.buildContextMessage(workspaceBasePath, null, chatOrigin)));
prefix.add(new UserMessage(RuntimeContextInjector.buildContextMessage(
workspaceBasePath, null, chatOrigin, runtimeModelName, runtimeProviderId)));
// When this turn already recalled the user's own current project from
// structured memory, skip auto-injecting knowledge-base reference context.
// Otherwise the KB pages (reference material, possibly about unrelated

View File

@ -30,6 +30,12 @@ public class StepProgressDispatcher implements EdgeAction {
if ("awaiting_approval".equals(currentPhase) || "plan_aborted".equals(currentPhase)) {
return StateGraph.END;
}
// Step-failure recovery: a failed step requested a re-plan of the
// remaining work. Route back to PlanGeneration instead of aborting;
// PLAN_REPLAN_COUNT (set by StepExecutionNode) bounds the loop.
if ("plan_replan".equals(currentPhase)) {
return PlanStateKeys.PLAN_GENERATION_NODE;
}
int currentIndex = state.value(PlanStateKeys.CURRENT_STEP_INDEX, 0);
List<String> steps = state.<List<String>>value(PlanStateKeys.PLAN_STEPS).orElse(List.of());

View File

@ -10,17 +10,27 @@ import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.converter.BeanOutputConverter;
import org.springframework.util.StringUtils;
import vip.mate.agent.AgentService;
import vip.mate.agent.AgentToolSet;
import vip.mate.agent.GraphEventPublisher;
import vip.mate.agent.model.AgentEntity;
import vip.mate.agent.graph.NodeStreamingChatHelper;
import vip.mate.agent.graph.plan.state.PlanStateAccessor;
import vip.mate.agent.graph.plan.state.PlanStateKeys;
import vip.mate.agent.graph.state.MateClawStateKeys;
import vip.mate.agent.context.ChatOrigin;
import vip.mate.agent.context.ConversationWindowManager;
import vip.mate.agent.context.RuntimeContextInjector;
import vip.mate.goal.config.GoalProperties;
import vip.mate.goal.model.GoalCreateRequest;
import vip.mate.goal.model.GoalCriterion;
import vip.mate.goal.model.GoalEntity;
import vip.mate.goal.service.GoalService;
import vip.mate.planning.service.PlanningService;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.stream.Collectors;
@ -52,6 +62,17 @@ public class PlanGenerationNode implements NodeAction {
private final NodeStreamingChatHelper streamingHelper;
private final ConversationWindowManager conversationWindowManager;
private final AgentToolSet toolSet;
/** Optional — auto-derive a goal from the plan. Null disables the feature (legacy/test). */
private final GoalService goalService;
private final GoalProperties goalProperties;
/** Optional advertise delegatable specialist agents to the planner and
* resolve per-step assignments. Null disables per-step delegation (legacy/test). */
private final AgentService agentService;
/** Plan steps below this size are trivial tool tasks, not goal-worthy. */
private static final int MIN_STEPS_FOR_AUTO_GOAL = 2;
/** Cap the auto-derived goal title; the full request rides in the description. */
private static final int AUTO_GOAL_TITLE_MAX = 80;
/**
* Structured triage result field names use @JsonProperty to match the
@ -61,7 +82,12 @@ public class PlanGenerationNode implements NodeAction {
@JsonProperty("needs_planning") boolean needsPlanning,
@JsonProperty("direct_answer") String directAnswer,
@JsonProperty("plan_type") String planType,
@JsonProperty("steps") List<String> steps
@JsonProperty("steps") List<String> steps,
// Optional per-step delegation: agent names parallel to steps (same
// order). An empty string / missing entry means "run with the parent
// agent". Only populated when delegatable specialist agents are
// advertised to the planner; absent for backward compatibility.
@JsonProperty("step_agents") List<String> stepAgents
) {}
private static final String PLANNING_PROMPT = """
@ -96,15 +122,104 @@ public class PlanGenerationNode implements NodeAction {
- 多部分多阶段需要逐步推进的目标走(C)真正单一原子动作走(B)只有简单一问一答才用(A)
""";
/**
* Evidence gate action signals in the USER GOAL. When triage returns
* direct_answer (A) but the goal contains any of these, the model almost
* certainly mis-routed a tool-requiring task; accepting the direct answer
* would end the turn without ever executing a tool ("复杂任务不执行就停止").
* <p>
* The gate is deliberately biased toward executing: a false positive only
* costs one extra executor pass (which still produces the answer, with or
* without tools), whereas a false negative silently drops the whole task.
* Intentionally excludes very common bare temporal words (现在/当前/最新)
* to avoid downgrading genuine knowledge Q&A on every occurrence.
*/
private static final java.util.regex.Pattern GOAL_REQUIRES_EXECUTION = java.util.regex.Pattern.compile(
"读取|读一下|读一份|打开文件|查一下|检索|搜索|联网|下载|上传|抓取"
+ "|记住|记一下|录入|保存|写入|存储|更新|删除|新建|创建|生成|画一[张幅]|画个"
+ "|运行|执行|调用|跑一下|发送|发给|安排|提醒|预约"
+ "|我的(记忆|文件|知识库|偏好|笔记|日程|目标)"
+ "|你(现在|目前)?(挂载|加载|有哪些|支持哪些)|挂载了哪些|你的(技能|工具|MCP|插件)"
+ "|帮我(做|改|查|建|写|发|跑|算|订|定|生成|整理|安排)"
+ "|\\.(java|py|ts|js|vue|md|json|ya?ml|sql|csv|xml|txt|sh)\\b",
java.util.regex.Pattern.CASE_INSENSITIVE);
/**
* Evidence gate execution-promise phrasing in the direct answer itself.
* The model says it WILL act ("我先去读取…", "接下来调用…") rather than
* actually answering, which means the "direct answer" is really a plan
* preamble that would terminate before the action runs. Scoped to a verb
* whitelist so a normal narrative opener like "我来介绍一下杭州" is NOT caught.
*/
private static final java.util.regex.Pattern ANSWER_PROMISES_ACTION = java.util.regex.Pattern.compile(
"(我(先|这就|马上|稍后|接下来|现在)?(去|来)?|让我(先|来)?|接下来(我)?(会|要|将|需要)?|正在)"
+ "(读取|读一下|查一下|查询|检索|搜索|联网|调用|执行|运行|获取|访问|查看一下"
+ "|保存|记住|记录|写入|录入|创建|新建|生成|下载|上传|发送)");
/**
* Returns true when a triage {@code direct_answer} (A) should be overridden
* and routed through the executor as a single-step plan instead. Package-
* private and side-effect free so the gate's regex behavior is unit-testable
* without mocking the whole node.
*
* @param goal the user goal
* @param directAnswer the answer the triage model produced (may be null)
*/
static boolean shouldOverrideDirectAnswer(String goal, String directAnswer) {
String userAsk = stripInjectedContext(goal);
boolean goalNeedsExecution = userAsk != null && GOAL_REQUIRES_EXECUTION.matcher(userAsk).find();
boolean answerPromisesAction = directAnswer != null && ANSWER_PROMISES_ACTION.matcher(directAnswer).find();
return goalNeedsExecution || answerPromisesAction;
}
/**
* Strips the injected {@code <memory-context></memory-context>} wrapper that
* RuntimeContextInjector prepends to every goal, returning just the user's
* actual ask. Without this the gate matches on the injected memory/profile
* text (which contains filenames like {@code user.md} and memory keywords),
* firing on essentially every task and defeating the direct-answer fast path.
*/
static String stripInjectedContext(String goal) {
if (goal == null) {
return null;
}
int end = goal.lastIndexOf("</memory-context>");
if (end >= 0) {
return goal.substring(end + "</memory-context>".length()).trim();
}
return goal;
}
public PlanGenerationNode(ChatModel chatModel, PlanningService planningService,
NodeStreamingChatHelper streamingHelper,
ConversationWindowManager conversationWindowManager,
AgentToolSet toolSet) {
this(chatModel, planningService, streamingHelper, conversationWindowManager, toolSet, null, null);
}
public PlanGenerationNode(ChatModel chatModel, PlanningService planningService,
NodeStreamingChatHelper streamingHelper,
ConversationWindowManager conversationWindowManager,
AgentToolSet toolSet,
GoalService goalService, GoalProperties goalProperties) {
this(chatModel, planningService, streamingHelper, conversationWindowManager, toolSet,
goalService, goalProperties, null);
}
public PlanGenerationNode(ChatModel chatModel, PlanningService planningService,
NodeStreamingChatHelper streamingHelper,
ConversationWindowManager conversationWindowManager,
AgentToolSet toolSet,
GoalService goalService, GoalProperties goalProperties,
AgentService agentService) {
this.chatModel = chatModel;
this.planningService = planningService;
this.streamingHelper = streamingHelper;
this.conversationWindowManager = conversationWindowManager;
this.toolSet = toolSet;
this.goalService = goalService;
this.goalProperties = goalProperties;
this.agentService = agentService;
}
/**
@ -112,7 +227,119 @@ public class PlanGenerationNode implements NodeAction {
*/
@Deprecated
public PlanGenerationNode(ChatModel chatModel, PlanningService planningService) {
this(chatModel, planningService, null, null, null);
this(chatModel, planningService, null, null, null, null, null);
}
/**
* Auto-derive a goal from a freshly-generated multi-step plan so the
* Plan-Execute path engages the goal subsystem (the planner / step executor
* never call {@code setGoal} themselves). The plan steps become the goal's
* acceptance criteria the plan IS the decomposition so the first
* evaluation skips the bootstrap round and judges those criteria directly.
*
* <p>Returns the created goal (to inject into {@code ACTIVE_GOAL} so THIS
* run's GoalEvaluationNode picks it up) or {@code null} when not applicable:
* feature off, fewer than {@link #MIN_STEPS_FOR_AUTO_GOAL} steps, no channel
* context, or the conversation already has an active goal. Best-effort
* any failure is swallowed so planning is never blocked by goal bookkeeping.
*/
GoalEntity maybeAutoCreateGoal(PlanStateAccessor accessor, List<String> steps) {
if (goalService == null || goalProperties == null
|| !goalProperties.isEnabled() || !goalProperties.isAutoGoalFromPlan()) {
return null;
}
if (steps == null || steps.size() < MIN_STEPS_FOR_AUTO_GOAL) {
return null;
}
ChatOrigin origin = accessor.chatOrigin();
String convId = origin.conversationId();
if (convId == null || convId.isBlank() || origin.agentId() == null) {
return null;
}
try {
if (goalService.findActiveByConversation(convId) != null) {
return null; // respect an existing goal (incl. re-plan passes)
}
String request = stripInjectedContext(accessor.goal()).strip();
GoalCreateRequest req = new GoalCreateRequest();
req.setConversationId(convId);
req.setAgentId(origin.agentId());
req.setWorkspaceId(origin.workspaceId() != null ? origin.workspaceId() : 1L);
req.setTitle(request.isEmpty() ? "多步任务"
: request.length() > AUTO_GOAL_TITLE_MAX
? request.substring(0, AUTO_GOAL_TITLE_MAX) : request);
req.setDescription(request);
List<GoalCriterion> criteria = steps.stream()
.filter(s -> s != null && !s.isBlank())
.map(s -> new GoalCriterion("", s.strip(), false, ""))
.collect(Collectors.toList());
if (!criteria.isEmpty()) {
req.setCriteria(criteria);
}
String username = origin.requesterId() != null && !origin.requesterId().isBlank()
? origin.requesterId() : "system";
GoalEntity created = goalService.create(req, username);
log.info("[PlanGeneration] Auto-derived goal {} from plan ({} criteria) for conversation {}",
created.getId(), criteria.size(), convId);
return created;
} catch (Exception e) {
log.warn("[PlanGeneration] Auto-goal-from-plan skipped (non-fatal): {}", e.toString());
return null;
}
}
/**
* Enabled agents in the given workspace, excluding the parent (plan) agent
* itself these are the agents a step can be delegated to. Empty when
* delegation is unavailable (no {@link AgentService}) or no peers exist.
*/
private List<AgentEntity> listDelegatableAgents(Long workspaceId, String parentAgentId) {
if (agentService == null || workspaceId == null) {
return List.of();
}
try {
return agentService.listAgentsByWorkspace(workspaceId, true).stream()
.filter(a -> a.getId() != null && !String.valueOf(a.getId()).equals(parentAgentId))
.collect(Collectors.toList());
} catch (Exception e) {
log.warn("[PlanGeneration] Failed to list delegatable agents (non-fatal): {}", e.toString());
return List.of();
}
}
/**
* Map the planner's {@code step_agents} (agent names, parallel to steps) to
* agent ids. Returns {@code null} when nothing is delegated so {@code createPlan}
* stays on the legacy path. Names are matched case-insensitively against the
* delegatable agents; blank / unknown / parent-agent names resolve to {@code null}
* (that step runs with the parent agent).
*/
private List<Long> resolveStepAgents(List<String> steps, List<String> stepAgents,
Long workspaceId, String parentAgentId) {
if (stepAgents == null || stepAgents.isEmpty() || steps == null || steps.isEmpty()) {
return null;
}
List<AgentEntity> delegatable = listDelegatableAgents(workspaceId, parentAgentId);
if (delegatable.isEmpty()) {
return null;
}
Map<String, Long> byName = new HashMap<>();
for (AgentEntity a : delegatable) {
if (a.getName() != null) {
byName.put(a.getName().trim().toLowerCase(), a.getId());
}
}
List<Long> ids = new ArrayList<>();
boolean any = false;
for (int i = 0; i < steps.size(); i++) {
String name = i < stepAgents.size() ? stepAgents.get(i) : null;
Long id = (name == null || name.isBlank()) ? null : byName.get(name.trim().toLowerCase());
if (id != null) {
any = true;
}
ids.add(id);
}
return any ? ids : null;
}
@Override
@ -134,7 +361,11 @@ public class PlanGenerationNode implements NodeAction {
}
String systemPrompt = accessor.systemPrompt();
String agentId = state.value(MateClawStateKeys.TRACE_ID, "unknown");
// Persist plans under the real agent id (the same key StepExecutionNode
// reads), NOT the per-run trace id otherwise mate_plan.agent_id holds a
// random trace string and listByAgent never matches, leaving the Plan
// board permanently empty even after plans are generated.
String agentId = state.value(MateClawStateKeys.AGENT_ID, "");
String conversationId = accessor.conversationId();
log.info("[PlanGeneration] Evaluating goal: {}", goal.length() > 100 ? goal.substring(0, 100) + "..." : goal);
@ -169,8 +400,11 @@ public class PlanGenerationNode implements NodeAction {
vip.mate.agent.context.ChatOrigin chatOrigin =
state.<vip.mate.agent.context.ChatOrigin>value(MateClawStateKeys.CHAT_ORIGIN)
.orElse(vip.mate.agent.context.ChatOrigin.EMPTY);
String runtimeModelName = state.value(MateClawStateKeys.RUNTIME_MODEL_NAME, "");
String runtimeProviderId = state.value(MateClawStateKeys.RUNTIME_PROVIDER_ID, "");
promptMessages.add(new UserMessage(
RuntimeContextInjector.buildContextMessage(workspaceBasePath, null, chatOrigin)));
RuntimeContextInjector.buildContextMessage(
workspaceBasePath, null, chatOrigin, runtimeModelName, runtimeProviderId)));
// Advertise available tools so the LLM can recognize when an action is possible,
// but do NOT force "any tool usage implies multi-step" single-hop tool use
@ -184,6 +418,24 @@ public class PlanGenerationNode implements NodeAction {
+ "\n单次工具调用应归为单步B不要拆成多步。"));
}
// Advertise delegatable specialist agents so the planner can assign a
// multi-step plan's step to a dedicated agent (e.g. a test step to a
// QA agent, a UI step to a frontend agent). Only fills the step's
// step_agents slot; unassigned steps stay with the parent agent.
// Skipped entirely when no peer agents exist in the workspace.
List<AgentEntity> delegatable = listDelegatableAgents(chatOrigin.workspaceId(), agentId);
if (!delegatable.isEmpty()) {
String agentLines = delegatable.stream()
.map(a -> "- " + a.getName()
+ (StringUtils.hasText(a.getDescription()) ? "" + a.getDescription() : ""))
.collect(Collectors.joining("\n"));
promptMessages.add(new UserMessage(
"可委派的专职 Agent仅当某步骤明显属于其专长时才指派否则该步骤留空、由你自己执行\n"
+ agentLines
+ "\n若要委派在 step_agents 数组对应位置填写 Agent 名称(与 steps 同序、等长);"
+ "不委派的步骤填空字符串。多数步骤通常不需要委派。"));
}
// Inject working context (rolling conversation summary) so triage respects
// prior constraints without re-reading full history.
String workingContext = accessor.workingContext();
@ -247,6 +499,31 @@ public class PlanGenerationNode implements NodeAction {
// Category (A): direct answer push to client and terminate via DirectAnswerNode.
String directAnswer = triage != null && triage.directAnswer() != null
? triage.directAnswer() : llmResponse;
// Evidence gate: catch a mis-routed A that actually needs tools.
// Downgrading to a single-step plan keeps tool access; the cost of
// a false positive is one extra executor pass, while a missed
// misroute drops the whole task silently.
if (shouldOverrideDirectAnswer(goal, directAnswer)) {
log.warn("[PlanGeneration] Evidence gate overrode direct-answer route; "
+ "downgrading to single-step plan so tools can execute (goal: {})",
goal.length() > 60 ? goal.substring(0, 60) + "..." : goal);
List<String> gatedSteps = List.of(goal);
var gatedPlan = planningService.createPlan(agentId, conversationId, goal, gatedSteps);
events.add(GraphEventPublisher.planCreated(gatedPlan.getId(), gatedSteps));
return PlanStateAccessor.output()
.needsPlanning(true)
.planId(gatedPlan.getId())
.planSteps(gatedSteps)
.planValid(true)
.currentStepIndex(0)
.currentPhase("plan_generated")
.thinkingStreamed(!result.thinking().isEmpty())
.mergeUsage(state, result)
.events(events)
.build();
}
log.info("[PlanGeneration] Direct-answer route taken (no tools, no planning)");
streamingHelper.broadcastContent(conversationId, directAnswer);
@ -273,13 +550,34 @@ public class PlanGenerationNode implements NodeAction {
steps = List.of(goal);
}
var plan = planningService.createPlan(agentId, goal, steps);
log.info("[PlanGeneration] Plan created: id={}, steps={} ({})",
plan.getId(), steps.size(), steps.size() == 1 ? "single-step" : "multi-step");
// Resolve any per-step agent delegation the planner asked for. Null
// when nothing is delegated, keeping createPlan on the legacy path.
List<Long> stepAgentIds = resolveStepAgents(steps,
triage != null ? triage.stepAgents() : null,
chatOrigin.workspaceId(), agentId);
var plan = planningService.createPlan(agentId, conversationId, goal, steps, stepAgentIds);
log.info("[PlanGeneration] Plan created: id={}, steps={} ({}){}",
plan.getId(), steps.size(), steps.size() == 1 ? "single-step" : "multi-step",
stepAgentIds != null ? ", per-step delegation=" + stepAgentIds : "");
events.add(GraphEventPublisher.planCreated(plan.getId(), steps));
return PlanStateAccessor.output()
// Auto-derive a goal from a genuine multi-step plan so the
// Plan-Execute path engages the goal subsystem. Injected into
// ACTIVE_GOAL so this same run's GoalEvaluationNode evaluates it.
GoalEntity autoGoal = maybeAutoCreateGoal(accessor, steps);
if (autoGoal != null && goalService != null) {
// Surface it to the UI exactly like the setGoal tool does
// ({goalId, conversationId, goal}) so the goal panel hydrates
// even though the user never called setGoal. Same SSE event the
// frontend goal store already listens for.
events.add(new GraphEventPublisher.GraphEvent("goal_created", Map.of(
"goalId", String.valueOf(autoGoal.getId()),
"conversationId", conversationId,
"goal", goalService.toResponse(autoGoal)), System.currentTimeMillis()));
}
PlanStateAccessor.OutputBuilder planOut = PlanStateAccessor.output()
.needsPlanning(true)
.planId(plan.getId())
.planSteps(steps)
@ -289,8 +587,11 @@ public class PlanGenerationNode implements NodeAction {
.contentStreamed(true)
.thinkingStreamed(!result.thinking().isEmpty())
.mergeUsage(state, result)
.events(events)
.build();
.events(events);
if (autoGoal != null) {
planOut.put(MateClawStateKeys.ACTIVE_GOAL, autoGoal);
}
return planOut.build();
} catch (Exception e) {
log.error("[PlanGeneration] Triage failed, falling back to single-step plan: {}", e.getMessage(), e);
@ -299,7 +600,7 @@ public class PlanGenerationNode implements NodeAction {
// answer. This preserves tool access on the failure path; the previous
// "direct answer" fallback silently degraded tool-requiring tasks.
try {
var plan = planningService.createPlan(agentId, goal, List.of(goal));
var plan = planningService.createPlan(agentId, conversationId, goal, List.of(goal));
events.add(GraphEventPublisher.planCreated(plan.getId(), List.of(goal)));
return PlanStateAccessor.output()
.needsPlanning(true)

View File

@ -27,11 +27,16 @@ import vip.mate.agent.context.RuntimeContextInjector;
import vip.mate.agent.graph.executor.ToolExecutionExecutor;
import vip.mate.channel.web.ChatStreamTracker;
import vip.mate.planning.service.PlanningService;
import vip.mate.agent.context.ChatOrigin;
import vip.mate.skill.runtime.SkillCatalogRenderer;
import vip.mate.tool.builtin.DelegateAgentTool;
import vip.mate.tool.builtin.DelegationContext;
import vip.mate.tool.builtin.ToolExecutionContext;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Set;
/**
* 步骤执行节点
@ -68,6 +73,17 @@ public class StepExecutionNode implements NodeAction {
*/
private final SkillCatalogRenderer skillCatalogRenderer;
/**
* Optional per-step delegation executor. Set after construction (this node is
* built by AgentGraphBuilder, not Spring) so a plan step assigned to a
* specialist agent runs on that agent. Null disables per-step delegation.
*/
private DelegateAgentTool delegateAgentTool;
public void setDelegateAgentTool(DelegateAgentTool delegateAgentTool) {
this.delegateAgentTool = delegateAgentTool;
}
/**
* Per-step tool-call ceiling, aligned with {@code BaseAgent.MAX_ITERATIONS_HARD_CEILING}.
* Matching the agent-level cap means this constant is never the bottleneck
@ -89,6 +105,16 @@ public class StepExecutionNode implements NodeAction {
* pathological cases where the agent appears frozen to the user.
*/
private static final long STEP_WALL_CLOCK_TIMEOUT_MS = 10 * 60 * 1000L;
/**
* Max re-plans per graph run. When a step throws, the executor re-plans the
* remaining work around the failure instead of aborting the whole plan a
* single transient tool error or one badly-scoped step no longer kills the
* task. Bounded so a step that fails every attempt can't re-plan forever;
* once exhausted the plan aborts as before. Kept small (the recursion
* ceiling already accommodates it) raise with care.
*/
private static final int MAX_REPLANS_PER_RUN = 1;
private static final ObjectMapper MAPPER = new ObjectMapper();
public StepExecutionNode(ChatModel chatModel, AgentToolSet toolSet,
@ -166,6 +192,8 @@ public class StepExecutionNode implements NodeAction {
vip.mate.agent.context.ChatOrigin chatOrigin =
state.<vip.mate.agent.context.ChatOrigin>value(MateClawStateKeys.CHAT_ORIGIN)
.orElse(vip.mate.agent.context.ChatOrigin.EMPTY);
String runtimeModelName = state.value(MateClawStateKeys.RUNTIME_MODEL_NAME, "");
String runtimeProviderId = state.value(MateClawStateKeys.RUNTIME_PROVIDER_ID, "");
if (stepIndex >= steps.size()) {
log.warn("[StepExecution] stepIndex {} >= steps.size() {}, skipping", stepIndex, steps.size());
@ -186,6 +214,19 @@ public class StepExecutionNode implements NodeAction {
// "react_step" / "first_turn" markers when both stream into the
// same SSE feed.
boolean iterationEventsOn = streamTracker == null || streamTracker.isIterationEventsEnabled();
// Per-step delegation: when this step is assigned to a different
// specialist agent, run it on that agent (as an isolated child) and use
// its reply as the step result, instead of executing locally with the
// parent agent's tools. assignedAgentId comes from the DB so it survives
// replay / approval-resume.
Long assignedAgentId = planningService.getStepAssignedAgent(planId, stepIndex);
if (delegateAgentTool != null && assignedAgentId != null
&& !assignedAgentId.equals(parseLongOrNull(agentId))) {
return executeDelegatedStep(accessor, stepIndex, step, planId, assignedAgentId,
conversationId, chatOrigin, events, iterationEventsOn);
}
if (iterationEventsOn) {
events.add(GraphEventPublisher.iterationStart(stepIndex, "plan_step", "parent", null));
}
@ -195,7 +236,8 @@ public class StepExecutionNode implements NodeAction {
planningService.updateSubPlanStatus(planId, stepIndex, "running");
// 构建消息列表
List<Message> messages = buildStepMessages(accessor, step, systemPrompt, workspaceBasePath);
List<Message> messages = buildStepMessages(accessor, step, systemPrompt, workspaceBasePath,
runtimeModelName, runtimeProviderId);
// 显式工具执行循环
String finalResult = null;
@ -219,6 +261,12 @@ public class StepExecutionNode implements NodeAction {
long stepStartedAtMs = System.currentTimeMillis();
boolean wallClockExceeded = false;
// Signature-based progress detector: nudges the model to change strategy
// when a round stalls (repeated failures / identical results), and flags
// the step as stuck past a hard threshold so we re-plan instead of
// burning the whole tool-call budget and advancing with junk.
StepProgressTracker progressTracker = new StepProgressTracker();
try {
while (toolCallCount < MAX_TOOL_CALLS_PER_STEP) {
long elapsedMs = System.currentTimeMillis() - stepStartedAtMs;
@ -350,6 +398,30 @@ public class StepExecutionNode implements NodeAction {
break;
}
// Progress tracking: feed this round's tool results to the
// detector. A WARN-level stall injects a one-shot "change
// strategy" SystemMessage the model sees on its next call; a
// HALT-level stall stops the inner loop so the post-loop logic
// re-plans instead of spinning to the tool-call ceiling.
java.util.Map<String, String> idToArgs = new java.util.HashMap<>();
for (AssistantMessage.ToolCall tc : allToolCalls) {
if (tc != null && tc.id() != null) {
idToArgs.put(tc.id(), tc.arguments());
}
}
for (ToolResponseMessage.ToolResponse tr : toolResponses) {
var nudge = progressTracker.record(
tr.name(), idToArgs.getOrDefault(tr.id(), ""), tr.responseData());
if (nudge.isPresent()) {
messages.add(new SystemMessage(nudge.get()));
}
}
if (progressTracker.isStuck()) {
log.warn("[StepExecution] Step {} stalled ({}); stopping inner loop to re-plan",
stepIndex, progressTracker.haltReason());
break;
}
// RFC-052: returnDirect short-circuit. Any direct tool in this
// step ends the plan immediately; the dispatcher routes via
// currentPhase=plan_aborted so no further LLM call happens.
@ -415,6 +487,56 @@ public class StepExecutionNode implements NodeAction {
.build();
}
// Step-failure recovery WITHOUT an exception: the inner loop ended
// with no usable result stalled (repeated failures / identical
// results, flagged by the progress tracker), hit the wall-clock or
// tool-call ceiling, or returned an empty answer. Re-plan the
// remaining work around it instead of advancing dependent steps with
// junk. Shares PLAN_REPLAN_COUNT with the exception path; once the
// budget is spent we fall through to the legacy "complete with a
// failure note" path below so the plan still terminates.
boolean noUsableResult = progressTracker.isStuck()
|| finalResult == null || finalResult.isBlank();
int noProgressReplanCount = accessor.replanCount();
if (noUsableResult && noProgressReplanCount < MAX_REPLANS_PER_RUN) {
String reason = progressTracker.isStuck()
? "本步骤陷入停滞(" + progressTracker.haltReason() + "),未取得有效结果"
: wallClockExceeded
? "本步骤超过最大耗时限制,未取得有效结果"
: finalResult == null
? "本步骤超过最大工具调用次数,未取得有效结果"
: "本步骤未产出有效结果";
planningService.updateSubPlanFailure(planId, stepIndex, reason);
planningService.markPlanFailed(planId, "步骤" + (stepIndex + 1) + "" + reason);
events.add(GraphEventPublisher.stepCompleted(stepIndex, reason));
if (iterationEventsOn) {
events.add(GraphEventPublisher.iterationEnd(stepIndex, "parent", null, reason.length(), 0));
}
events.add(new GraphEventPublisher.GraphEvent("plan_replan", Map.of(
"failedStepIndex", stepIndex,
"attempt", noProgressReplanCount + 1,
"maxReplans", MAX_REPLANS_PER_RUN,
"reason", reason), System.currentTimeMillis()));
log.warn("[StepExecution] Step {} produced no usable result ({}); re-planning (attempt {}/{})",
stepIndex + 1, reason, noProgressReplanCount + 1, MAX_REPLANS_PER_RUN);
return PlanStateAccessor.output()
.workingContext(buildReplanContext(accessor, stepIndex, reason))
.currentPhase("plan_replan")
.replanCount(noProgressReplanCount + 1)
.planId(null)
.planSteps(List.of())
.planValid(false)
.needsPlanning(true)
.currentStepIndex(0)
.currentStepTitle("")
.currentStepResult("")
.contentStreamed(false)
.put(MateClawStateKeys.PROMPT_TOKENS, state.value(MateClawStateKeys.PROMPT_TOKENS, 0) + stepPromptTokens)
.put(MateClawStateKeys.COMPLETION_TOKENS, state.value(MateClawStateKeys.COMPLETION_TOKENS, 0) + stepCompletionTokens)
.events(events)
.build();
}
if (finalResult == null) {
if (wallClockExceeded) {
finalResult = "步骤执行超过最大耗时限制("
@ -435,6 +557,46 @@ public class StepExecutionNode implements NodeAction {
events.add(GraphEventPublisher.iterationEnd(stepIndex, "parent", null,
shortError != null ? shortError.length() : 0, 0));
}
// Step-failure recovery: rather than aborting the whole plan on a
// single failed step, re-plan the remaining work around the failure
// (up to MAX_REPLANS_PER_RUN). Completed steps are preserved in
// WORKING_CONTEXT, so the next PlanGeneration pass can skip them and
// route around (or retry) what broke. The mid-pass plan state is
// cleared so a fresh plan is derived; PLAN_REPLAN_COUNT bounds the loop.
int replanCount = accessor.replanCount();
if (replanCount < MAX_REPLANS_PER_RUN) {
String replanContext = buildReplanContext(accessor, stepIndex, shortError);
events.add(new GraphEventPublisher.GraphEvent("plan_replan", Map.of(
"failedStepIndex", stepIndex,
"attempt", replanCount + 1,
"maxReplans", MAX_REPLANS_PER_RUN,
"error", shortError == null ? "" : shortError),
System.currentTimeMillis()));
log.warn("[StepExecution] Step {} failed; re-planning remaining work (attempt {}/{})",
stepIndex + 1, replanCount + 1, MAX_REPLANS_PER_RUN);
return PlanStateAccessor.output()
.workingContext(replanContext)
.currentPhase("plan_replan")
.replanCount(replanCount + 1)
// Wipe mid-pass plan state so PlanGenerationNode re-derives
// a fresh plan from goal + (failure-augmented) context.
.planId(null)
.planSteps(List.of())
.planValid(false)
.needsPlanning(true)
.currentStepIndex(0)
.currentStepTitle("")
.currentStepResult("")
.contentStreamed(false)
.put(MateClawStateKeys.PROMPT_TOKENS, state.value(MateClawStateKeys.PROMPT_TOKENS, 0) + stepPromptTokens)
.put(MateClawStateKeys.COMPLETION_TOKENS, state.value(MateClawStateKeys.COMPLETION_TOKENS, 0) + stepCompletionTokens)
.events(events)
.build();
}
log.warn("[StepExecution] Step {} failed and re-plan budget exhausted ({}); aborting plan",
stepIndex + 1, MAX_REPLANS_PER_RUN);
return PlanStateAccessor.output()
.currentStepResult(shortError)
.currentPhase("plan_aborted")
@ -489,6 +651,98 @@ public class StepExecutionNode implements NodeAction {
.build();
}
/**
* Execute a step by delegating it to its assigned specialist agent. The
* delegated agent runs the step description as a self-contained goal and its
* reply becomes the step result. Mirrors the success/failure bookkeeping of
* the local execution path (sub-plan status, completed-results accumulation,
* incremental working-context update) so the rest of the plan graph is
* unaffected by where the step ran.
*/
private Map<String, Object> executeDelegatedStep(
PlanStateAccessor accessor, int stepIndex, String step, Long planId,
Long assignedAgentId, String conversationId, ChatOrigin chatOrigin,
List<GraphEventPublisher.GraphEvent> events, boolean iterationEventsOn) {
if (iterationEventsOn) {
events.add(GraphEventPublisher.iterationStart(stepIndex, "plan_step", "parent", null));
}
events.add(GraphEventPublisher.stepStarted(stepIndex, step));
events.add(GraphEventPublisher.phase("executing", Map.of(
"stepIndex", stepIndex, "stepTitle", step,
"delegatedAgentId", String.valueOf(assignedAgentId))));
planningService.updateSubPlanStatus(planId, stepIndex, "running");
log.info("[StepExecution] Delegating step {} to agent {}", stepIndex + 1, assignedAgentId);
// Seed the delegation context with the plan's REAL conversation id (from
// graph state) so the delegated child conversation is parented to it and
// stays hidden from the user's conversation list. The ChatOrigin in the
// plan-execute path carries no conversationId, so delegateByAgentId can't
// derive the parent on its own we provide it here.
boolean seeded = false;
if (conversationId != null && !conversationId.isBlank()
&& DelegationContext.parentConversationId() == null
&& ToolExecutionContext.conversationId() == null) {
DelegationContext.enter(conversationId, Set.of(), conversationId, null, 0);
seeded = true;
}
String result;
try {
result = delegateAgentTool.delegateByAgentId(assignedAgentId, step, chatOrigin);
} catch (Exception e) {
log.error("[StepExecution] Delegated step {} threw: {}", stepIndex, e.getMessage(), e);
result = "[错误] 委派执行异常:" + e.getMessage();
} finally {
if (seeded) {
DelegationContext.exit();
}
}
String finalResult = result != null ? result : "";
boolean failed = finalResult.isEmpty() || finalResult.startsWith("[错误]");
if (failed) {
planningService.updateSubPlanFailure(planId, stepIndex, finalResult);
} else {
planningService.updateSubPlanResult(planId, stepIndex, finalResult);
}
events.add(GraphEventPublisher.stepCompleted(stepIndex, finalResult));
if (iterationEventsOn) {
events.add(GraphEventPublisher.iterationEnd(stepIndex, "parent", null, finalResult.length(), 0));
}
// Keep the rolling working-context in sync exactly like the local path
// so later steps see this delegated step's result.
String prevWorkingContext = accessor.workingContext();
String formattedNewStep = formatStepResult(stepIndex, finalResult);
String updatedWorkingContext = prevWorkingContext.isEmpty()
? rebuildWorkingContext(accessor, appendOne(accessor.completedResults(), formattedNewStep))
: appendStepIncremental(prevWorkingContext, formattedNewStep);
return PlanStateAccessor.output()
.currentStepResult(finalResult)
.completedResults(formattedNewStep)
.currentStepIndex(stepIndex + 1)
.workingContext(updatedWorkingContext)
.currentPhase("step_completed")
.contentStreamed(true)
.events(events)
.build();
}
/** Parse a string id to Long, or null when blank / non-numeric. */
private static Long parseLongOrNull(String s) {
if (s == null || s.isBlank()) {
return null;
}
try {
return Long.parseLong(s.trim());
} catch (NumberFormatException e) {
return null;
}
}
/**
* RFC-052: assemble the final answer text from direct tool outputs in this
* step. Mirrors {@code FinalAnswerNode#assembleDirectAnswer} so the user
@ -509,7 +763,8 @@ public class StepExecutionNode implements NodeAction {
return sb.toString();
}
private List<Message> buildStepMessages(PlanStateAccessor accessor, String step, String systemPrompt, String workspaceBasePath) {
private List<Message> buildStepMessages(PlanStateAccessor accessor, String step, String systemPrompt,
String workspaceBasePath, String runtimeModelName, String runtimeProviderId) {
List<Message> messages = new ArrayList<>();
// Layer 1: System prompt增强指令
@ -537,9 +792,10 @@ public class StepExecutionNode implements NodeAction {
messages.add(new SystemMessage(skillCatalog));
}
}
// 注入运行时上下文当前时间 + 工作目录 + 发起者上下文
// 注入运行时上下文当前时间 + 工作目录 + 发起者上下文 + 模型身份
messages.add(new UserMessage(
RuntimeContextInjector.buildContextMessage(workspaceBasePath, null, accessor.chatOrigin())));
RuntimeContextInjector.buildContextMessage(
workspaceBasePath, null, accessor.chatOrigin(), runtimeModelName, runtimeProviderId)));
// Layer 2: Working context对话历史 + 步骤结果的受控长度摘要
String workingContext = accessor.workingContext();
@ -607,6 +863,30 @@ public class StepExecutionNode implements NodeAction {
}
}
/**
* Augment the working context with a note about the failed step so the next
* PlanGeneration pass re-plans around it. The completed-step results are
* already encoded in {@code WORKING_CONTEXT}; this appends only the failure
* so the planner can skip what's done, retry differently, or route around
* the broken step. The note is an internal LLM prompt (Chinese, matching the
* surrounding planning/execution prompts).
*/
static String buildReplanContext(PlanStateAccessor accessor, int failedStepIndex, String error) {
List<String> steps = accessor.planSteps();
String failedTitle = (failedStepIndex >= 0 && failedStepIndex < steps.size())
? steps.get(failedStepIndex) : ("步骤 " + (failedStepIndex + 1));
StringBuilder sb = new StringBuilder(accessor.workingContext());
if (sb.length() > 0) {
sb.append("\n\n");
}
sb.append("【上一轮计划执行失败】步骤 ").append(failedStepIndex + 1)
.append("").append(failedTitle).append(")执行失败:")
.append(error == null ? "未知错误" : error)
.append("\n请基于上面已完成的工作重新规划达成总目标所需的剩余步骤")
.append("绕开或换一种方式完成失败的部分,不要重复已经完成的步骤。");
return sb.toString();
}
/**
* 将异常转换为简短的错误摘要避免将完整异常体尤其是 429 JSON写入后续 prompt
* <ul>

View File

@ -0,0 +1,152 @@
package vip.mate.agent.graph.plan.node;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
/**
* Per-step progress detector for the Plan-Execute executor.
*
* <p>A plan step runs its own inner tool-calling loop. Without progress
* tracking, a step can spin repeatedly calling the same tool, or hammering
* different variants that all fail / return nothing until it hits the
* tool-call ceiling, then "complete" with an empty result and let the plan
* plow into dependent steps that have no real input.
*
* <p>This tracker watches the tool results of each round and recognises two
* signature-based stall patterns:
* <ul>
* <li><b>repeated failure</b> the same call (tool name + canonical args)
* keeps failing, or the same tool keeps failing with different args;</li>
* <li><b>no progress</b> a call keeps returning the <em>same</em> result,
* so re-issuing it yields nothing new.</li>
* </ul>
*
* <p>Detection is graduated: at the WARN threshold it emits a one-shot nudge
* (injected back into the step's messages so the model changes strategy);
* past the HALT threshold it flags the step as stuck so the executor can stop
* the inner loop and re-plan instead of advancing with junk. Thresholds are
* deliberately low the goal is to break a stall early, before the whole
* tool-call budget is burned.
*
* <p>Not thread-safe; create one per step.
*/
public final class StepProgressTracker {
/** Same exact call (tool + args) failing: nudge / halt thresholds. */
static final int SAME_CALL_FAIL_WARN = 2;
static final int SAME_CALL_FAIL_HALT = 4;
/** Same tool failing across different args: nudge / halt thresholds. */
static final int SAME_TOOL_FAIL_WARN = 3;
static final int SAME_TOOL_FAIL_HALT = 6;
/** Same call returning identical output (no new information): nudge / halt. */
static final int NO_PROGRESS_WARN = 2;
static final int NO_PROGRESS_HALT = 4;
/**
* Lower-cased markers that identify a tool result as a failure / empty
* outcome. Kept intentionally small and language-mixed: tool errors in this
* codebase surface as English exception text, while a few common "not
* found" phrasings also appear in Chinese tool output.
*/
private static final String[] FAILURE_MARKERS = {
"execution failed", "error:", "exception", "timeout", "timed out",
"enoent", "no such file", "not found", "authentication failed",
"permission denied", "failed to", "未找到", "不存在", "没有找到", "执行失败", "无法"
};
private final Map<String, Integer> sameCallFail = new HashMap<>();
private final Map<String, Integer> sameToolFail = new HashMap<>();
private final Map<String, Integer> resultRepeat = new HashMap<>();
private final Set<String> warnedKeys = new HashSet<>();
private boolean stuck = false;
private String haltReason = null;
/**
* Record one tool result from the current round.
*
* @param toolName the invoked tool's name (never null)
* @param argsJson the raw arguments JSON (may be empty when unresolved)
* @param resultText the tool's result text (may be null/empty)
* @return a nudge to inject into the step's messages when a WARN threshold
* was freshly crossed, otherwise empty. Each distinct warning fires
* at most once.
*/
public Optional<String> record(String toolName, String argsJson, String resultText) {
String name = toolName == null ? "tool" : toolName;
String args = argsJson == null ? "" : argsJson;
String result = resultText == null ? "" : resultText;
boolean failure = looksLikeFailure(result);
String callSig = name + "::" + args.hashCode();
String resultKey = callSig + "##" + result.trim().hashCode();
// No-progress: identical result for the same call, regardless of success.
int repeats = resultRepeat.merge(resultKey, 1, Integer::sum);
if (repeats >= NO_PROGRESS_HALT) {
markStuck("no_progress:" + name);
}
Optional<String> nudge = maybeWarn(repeats >= NO_PROGRESS_WARN, "np:" + resultKey,
"工具 " + name + " 已连续 " + repeats + " 次返回相同结果。不要重复同样的调用——"
+ "改用已有结果、换查询/换工具,或直接基于现有信息给出本步骤结论。");
if (failure) {
int callFails = sameCallFail.merge(callSig, 1, Integer::sum);
int toolFails = sameToolFail.merge(name, 1, Integer::sum);
if (callFails >= SAME_CALL_FAIL_HALT || toolFails >= SAME_TOOL_FAIL_HALT) {
markStuck("repeated_failure:" + name);
}
if (nudge.isEmpty()) {
nudge = maybeWarn(callFails >= SAME_CALL_FAIL_WARN, "cf:" + callSig,
"工具 " + name + " 用相同参数已失败 " + callFails + " 次,像是死循环。"
+ "先看错误原因再换一种方式,不要原样重试。");
}
if (nudge.isEmpty()) {
nudge = maybeWarn(toolFails >= SAME_TOOL_FAIL_WARN, "tf:" + name,
"工具 " + name + " 本步骤已失败 " + toolFails + " 次。停止在同一条失败路径上重试,"
+ "换工具或换思路完成本步骤。");
}
}
return nudge;
}
/** True once a HALT threshold was crossed — the step should stop and re-plan. */
public boolean isStuck() {
return stuck;
}
/** Machine-readable reason for the halt, or null when not stuck. */
public String haltReason() {
return haltReason;
}
private Optional<String> maybeWarn(boolean crossed, String key, String message) {
if (crossed && warnedKeys.add(key)) {
return Optional.of(message);
}
return Optional.empty();
}
private void markStuck(String reason) {
if (!stuck) {
stuck = true;
haltReason = reason;
}
}
static boolean looksLikeFailure(String result) {
if (result == null || result.isBlank()) {
return true; // an empty result is no progress either
}
String lower = result.toLowerCase();
for (String marker : FAILURE_MARKERS) {
if (lower.contains(marker)) {
return true;
}
}
return false;
}
}

View File

@ -73,6 +73,11 @@ public final class PlanStateAccessor {
return state.<List<String>>value(COMPLETED_RESULTS).orElse(List.of());
}
/** Re-plans already performed this run (0 at run start). */
public int replanCount() {
return state.value(PLAN_REPLAN_COUNT, 0);
}
// ===== 终止 =====
public String finalSummary() {
@ -187,6 +192,10 @@ public final class PlanStateAccessor {
return put(CURRENT_STEP_INDEX, index);
}
public OutputBuilder replanCount(int count) {
return put(PLAN_REPLAN_COUNT, count);
}
public OutputBuilder currentStepTitle(String title) {
return put(CURRENT_STEP_TITLE, title);
}

View File

@ -27,6 +27,15 @@ public final class PlanStateKeys {
public static final String CURRENT_STEP_RESULT = "current_step_result";
public static final String COMPLETED_RESULTS = "completed_results"; // APPEND 策略
/**
* Number of re-plans performed in THIS graph run (REPLACE strategy). When a
* step throws, the executor re-plans the remaining work around the failure
* (carried in {@link #WORKING_CONTEXT}) instead of aborting outright, up to
* a small bound this counter enforces that bound so a pathological failure
* loop can't re-plan forever. Implicitly 0 at run start.
*/
public static final String PLAN_REPLAN_COUNT = "plan_replan_count";
// ===== 终止 =====
public static final String FINAL_SUMMARY = "final_summary";
public static final String DIRECT_ANSWER = "direct_answer"; // 简单问答的直接回答

View File

@ -80,6 +80,11 @@ public final class MateClawStateAccessor {
return state.value(LLM_CALL_COUNT, 0);
}
/** Iterations refunded this run for setup-only (progressive-disclosure) rounds (0 at run start). */
public int iterationRefundCount() {
return state.value(ITERATION_REFUND_COUNT, 0);
}
// ===== 观察历史 =====
@SuppressWarnings("unchecked")
@ -303,6 +308,11 @@ public final class MateClawStateAccessor {
return state.value(GOAL_ACCOUNTED_LLM_CALL_COUNT, 0);
}
/** Hard continuations (fresh-budget ReAct segments) performed this run (0 at run start). */
public int goalHardContinuationCount() {
return state.value(GOAL_HARD_CONTINUATION_COUNT, 0);
}
/**
* Bridge across ReAct and Plan-Execute: ReAct writes the terminal text
* to {@link MateClawStateKeys#FINAL_ANSWER} via FinalAnswerNode;
@ -368,6 +378,10 @@ public final class MateClawStateAccessor {
return put(NEEDS_TOOL_CALL, needs);
}
public OutputBuilder iterationRefundCount(int count) {
return put(ITERATION_REFUND_COUNT, count);
}
// ---- 消息 ----
public OutputBuilder messages(List<Message> msgs) {
return put(MESSAGES, msgs);
@ -552,6 +566,10 @@ public final class MateClawStateAccessor {
return put(GOAL_ACCOUNTED_LLM_CALL_COUNT, n);
}
public OutputBuilder goalHardContinuationCount(int n) {
return put(GOAL_HARD_CONTINUATION_COUNT, n);
}
/** Wipe FINAL_ANSWER on follow-up so the next graph pass doesn't
* immediately re-terminate via the existing final text. */
public OutputBuilder clearFinalAnswer() {
@ -563,6 +581,18 @@ public final class MateClawStateAccessor {
return put(FINISH_REASON, "");
}
/**
* Wipe the limit-exceeded draft + flag. Required before a hard
* continuation re-enters the ReAct loop: FinalAnswerNode prefers
* FINAL_ANSWER_DRAFT over a freshly reasoned answer, so a stale draft
* left by LimitExceededNode would otherwise resurface as the next
* segment's answer.
*/
public OutputBuilder clearLimitExceededDraft() {
put(FINAL_ANSWER_DRAFT, "");
return put(LIMIT_EXCEEDED, false);
}
/** Plan-Execute follow-up: clear the terminal-side plan summary so
* the next PlanGeneration pass starts clean. Identifier is the
* string literal "final_summary" to avoid a compile-time link to

View File

@ -30,6 +30,16 @@ public final class MateClawStateKeys {
public static final String CURRENT_ITERATION = "current_iteration";
public static final String MAX_ITERATIONS = "max_iterations";
/**
* Iterations refunded this run because a reasoning round did no real work
* its whole tool batch was progressive-disclosure setup ({@code load_skill}
* / {@code enable_tool}). ObservationNode skips the iteration increment for
* such rounds so a tight budget isn't eaten by the load-then-use two-step;
* this counter bounds the refunds so a model that only ever loads skills
* still terminates. Implicitly 0 at run start. REPLACE strategy.
*/
public static final String ITERATION_REFUND_COUNT = "iteration_refund_count";
// ===== 工具调用REPLACE 策略=====
public static final String TOOL_CALLS = "tool_calls";
public static final String TOOL_RESULTS = "tool_results";
@ -228,6 +238,22 @@ public final class MateClawStateKeys {
*/
public static final String GOAL_ACCOUNTED_LLM_CALL_COUNT = "goal_accounted_llm_call_count";
/**
* Number of "hard continuations" already performed in THIS graph run a
* hard continuation is a goal follow-up that re-enters the ReAct loop with
* a FRESH iteration budget (CURRENT_ITERATION reset to 0) after a turn that
* ended in {@link FinishReason#MAX_ITERATIONS_REACHED}. Unlike a normal
* follow-up (which shares the run's single iteration budget), a hard
* continuation grants the goal a brand-new ReAct segment so a task too big
* for one budget can keep going autonomously instead of stalling until the
* user sends another message. Because each such segment costs up to a full
* {@code maxIterations} worth of node visits, it is bounded by a dedicated,
* tighter cap ({@code mateclaw.goal.max-hard-continuations-per-run}, clamped
* to {@link vip.mate.goal.config.GoalProperties#MAX_HARD_CONTINUATIONS_CEILING})
* and sized into the graph recursion ceiling. Implicitly 0 at run start.
*/
public static final String GOAL_HARD_CONTINUATION_COUNT = "goal_hard_continuation_count";
/** Graph-node identifier for the GoalEvaluationNode. */
public static final String GOAL_EVALUATION_NODE = "goal_evaluation";

View File

@ -20,7 +20,10 @@ import java.util.regex.Pattern;
public record SourceEvidenceLedger(
Set<String> sourcePaths,
Set<String> sourceSymbols,
Set<String> failedPaths
Set<String> failedPaths,
Set<String> wikiPageTitles,
Set<String> wikiChunkIds,
Set<SourceEvidenceLedger.WikiCitation> wikiCitations
) implements Serializable {
private static final ObjectMapper MAPPER = new ObjectMapper();
@ -31,15 +34,19 @@ public record SourceEvidenceLedger(
"\\b[A-Z][A-Za-z0-9_]*(?:Controller|Service|ServiceImpl|Node|Tool|Parser|Resolver|Manager|Syncer|Mapper|Entity|Repository|Dispatcher|Executor|Accessor|Builder|Policy|Guard)\\b");
private static final Pattern DECLARED_TYPE = Pattern.compile(
"\\b(?:class|interface|enum|record)\\s+([A-Z][A-Za-z0-9_]*)\\b");
private static final Pattern CITATION_MARKER = Pattern.compile("\\[(\\d+)\\]");
public SourceEvidenceLedger {
sourcePaths = Set.copyOf(sourcePaths == null ? Set.of() : sourcePaths);
sourceSymbols = Set.copyOf(sourceSymbols == null ? Set.of() : sourceSymbols);
failedPaths = Set.copyOf(failedPaths == null ? Set.of() : failedPaths);
wikiPageTitles = Set.copyOf(wikiPageTitles == null ? Set.of() : wikiPageTitles);
wikiChunkIds = Set.copyOf(wikiChunkIds == null ? Set.of() : wikiChunkIds);
wikiCitations = Set.copyOf(wikiCitations == null ? Set.of() : wikiCitations);
}
public static SourceEvidenceLedger empty() {
return new SourceEvidenceLedger(Set.of(), Set.of(), Set.of());
return new SourceEvidenceLedger(Set.of(), Set.of(), Set.of(), Set.of(), Set.of(), Set.of());
}
public static SourceEvidenceLedger fromToolResponses(List<ToolResponseMessage.ToolResponse> responses) {
@ -56,6 +63,14 @@ public record SourceEvidenceLedger(
recordReadFile(data, builder);
} else {
recordPlainTextEvidence(data, builder);
// Only mine wiki citations from wiki retrieval tools. Sniffing every
// tool's JSON for a top-level title/pages/chunks field would let
// unrelated tools (e.g. getGoalStatus, which returns a top-level
// "title") populate the citation set and falsely force [n] citation
// enforcement on the final answer.
if (isWikiTool(response.name())) {
recordWikiEvidence(data, builder);
}
}
}
return builder.build();
@ -69,9 +84,15 @@ public record SourceEvidenceLedger(
sourcePaths.forEach(builder::sourcePath);
sourceSymbols.forEach(builder::symbol);
failedPaths.forEach(builder::failedPath);
wikiPageTitles.forEach(builder::wikiPageTitle);
wikiChunkIds.forEach(builder::wikiChunkId);
wikiCitations.forEach(builder::wikiCitation);
other.sourcePaths.forEach(builder::sourcePath);
other.sourceSymbols.forEach(builder::symbol);
other.failedPaths.forEach(builder::failedPath);
other.wikiPageTitles.forEach(builder::wikiPageTitle);
other.wikiChunkIds.forEach(builder::wikiChunkId);
other.wikiCitations.forEach(builder::wikiCitation);
return builder.build();
}
@ -80,12 +101,61 @@ public record SourceEvidenceLedger(
sourcePaths.forEach(builder::sourcePath);
sourceSymbols.forEach(builder::symbol);
failedPaths.forEach(builder::failedPath);
wikiPageTitles.forEach(builder::wikiPageTitle);
wikiChunkIds.forEach(builder::wikiChunkId);
wikiCitations.forEach(builder::wikiCitation);
builder.sourcePath(path);
return builder.build();
}
public SourceEvidenceLedger withWikiPageTitle(String title) {
Builder builder = new Builder();
sourcePaths.forEach(builder::sourcePath);
sourceSymbols.forEach(builder::symbol);
failedPaths.forEach(builder::failedPath);
wikiPageTitles.forEach(builder::wikiPageTitle);
wikiChunkIds.forEach(builder::wikiChunkId);
wikiCitations.forEach(builder::wikiCitation);
builder.wikiPageTitle(title);
return builder.build();
}
public SourceEvidenceLedger withWikiChunkId(String chunkId) {
Builder builder = new Builder();
sourcePaths.forEach(builder::sourcePath);
sourceSymbols.forEach(builder::symbol);
failedPaths.forEach(builder::failedPath);
wikiPageTitles.forEach(builder::wikiPageTitle);
wikiChunkIds.forEach(builder::wikiChunkId);
wikiCitations.forEach(builder::wikiCitation);
builder.wikiChunkId(chunkId);
return builder.build();
}
public boolean hasEvidence() {
return !sourcePaths.isEmpty() || !sourceSymbols.isEmpty() || !failedPaths.isEmpty();
return !sourcePaths.isEmpty() || !sourceSymbols.isEmpty() || !failedPaths.isEmpty()
|| hasWikiEvidence();
}
public boolean hasWikiEvidence() {
return !wikiPageTitles.isEmpty() || !wikiChunkIds.isEmpty() || !wikiCitations.isEmpty();
}
public boolean hasWikiPageTitle(String title) {
if (title == null || title.isBlank()) {
return false;
}
String normalized = title.trim();
return wikiPageTitles.contains(normalized)
|| wikiPageTitles.stream().anyMatch(t -> t.equalsIgnoreCase(normalized));
}
public boolean hasWikiChunkId(String chunkId) {
return chunkId != null && wikiChunkIds.contains(chunkId);
}
public boolean hasWikiCitationIndex(int index) {
return wikiCitations.stream().anyMatch(c -> c.index() == index);
}
public boolean hasPath(String path) {
@ -103,6 +173,7 @@ public record SourceEvidenceLedger(
}
LinkedHashSet<String> unsupported = new LinkedHashSet<>();
LinkedHashSet<String> unsupportedFileStems = new LinkedHashSet<>();
Matcher fileMatcher = JAVA_FILE_REF.matcher(answer);
while (fileMatcher.find()) {
String ref = fileMatcher.group();
@ -111,6 +182,7 @@ public record SourceEvidenceLedger(
unsupportedFileStems.add(ref.substring(0, ref.length() - ".java".length()));
}
}
Matcher symbolMatcher = JAVA_SYMBOL_REF.matcher(answer);
while (symbolMatcher.find()) {
String ref = symbolMatcher.group();
@ -118,9 +190,122 @@ public record SourceEvidenceLedger(
unsupported.add(ref);
}
}
validateWikiCitations(answer, unsupported);
return unsupported.isEmpty() ? Validation.ok() : new Validation(false, List.copyOf(unsupported));
}
public String appendWikiSourceTable(String answer) {
if (answer == null || answer.isBlank() || wikiCitations.isEmpty()) {
return answer;
}
LinkedHashSet<Integer> used = citationIndexesIn(answer);
if (used.isEmpty()) {
return answer;
}
String result = answer;
StringBuilder additions = new StringBuilder();
for (Integer index : used) {
WikiCitation citation = wikiCitation(index);
if (citation == null) {
continue;
}
String canonicalLine = citation.sourceLine();
if (sourceLineFor(result, index) != null) {
// Normalize in-place: replace the existing (possibly
// non-canonical) source line with the standard format so
// the frontend can reliably parse the source table.
result = replaceSourceLine(result, index, canonicalLine);
} else {
if (additions.isEmpty()) {
additions.append("\n\n来源");
}
additions.append("\n").append(canonicalLine);
}
}
// If source lines were normalized in-place but no 来源 header
// exists, insert one before the first source line so the frontend
// preprocessWikiCitations() can locate the source table.
if (additions.isEmpty() && !result.contains("来源:")) {
Matcher firstSource = Pattern
.compile("(?m)^\\[")
.matcher(result);
if (firstSource.find()) {
int pos = firstSource.start();
result = result.substring(0, pos) + "\n\n来源\n" + result.substring(pos);
}
}
return additions.isEmpty() ? result : result + additions;
}
private void validateWikiCitations(String answer, LinkedHashSet<String> unsupported) {
if (wikiCitations.isEmpty()) {
return;
}
LinkedHashSet<Integer> indexes = citationIndexesIn(answer);
if (indexes.isEmpty()) {
unsupported.add("missing wiki citation [n]");
return;
}
for (Integer index : indexes) {
WikiCitation citation = wikiCitation(index);
if (citation == null) {
unsupported.add("wiki citation [" + index + "]");
continue;
}
String sourceLine = sourceLineFor(answer, index);
if (sourceLine == null) {
unsupported.add("wiki source table [" + index + "]");
} else if (!citation.matchesSourceLine(sourceLine)) {
unsupported.add("wiki source title for [" + index + "]");
}
}
}
private LinkedHashSet<Integer> citationIndexesIn(String answer) {
LinkedHashSet<Integer> indexes = new LinkedHashSet<>();
Matcher citationMatcher = CITATION_MARKER.matcher(answer);
while (citationMatcher.find()) {
try {
indexes.add(Integer.parseInt(citationMatcher.group(1)));
} catch (NumberFormatException ignored) {
}
}
return indexes;
}
private WikiCitation wikiCitation(int index) {
return wikiCitations.stream()
.filter(c -> c.index() == index)
.findFirst()
.orElse(null);
}
private static String sourceLineFor(String answer, int index) {
Pattern pattern = Pattern.compile("(?m)^\\s*\\[" + index + "\\]\\s+(.+)$");
Matcher matcher = pattern.matcher(answer);
return matcher.find() ? matcher.group(1).trim() : null;
}
/**
* Replace the existing source line for {@code index} with the canonical
* form. Matches a full line starting with optional whitespace, {@code [N]},
* then any content, and replaces it in-place so the frontend can reliably
* parse the source table to build a citation index title map.
*/
private static String replaceSourceLine(String answer, int index, String canonicalLine) {
Pattern pattern = Pattern.compile("(?m)^\\s*\\[" + index + "\\]\\s+.+$");
return pattern.matcher(answer).replaceFirst(
Matcher.quoteReplacement(canonicalLine));
}
private boolean hasFileName(String fileName) {
String normalized = normalizePath(fileName);
return sourcePaths.stream().anyMatch(p -> p.equals(normalized) || p.endsWith("/" + normalized));
@ -134,6 +319,13 @@ public record SourceEvidenceLedger(
return normalized.equals("read_file");
}
private static boolean isWikiTool(String name) {
if (name == null) {
return false;
}
return name.toLowerCase(Locale.ROOT).replace("-", "_").startsWith("wiki_");
}
private static void recordReadFile(String data, Builder builder) {
try {
JsonNode root = MAPPER.readTree(data);
@ -158,6 +350,56 @@ public record SourceEvidenceLedger(
recordSymbols(text, builder);
}
private static void recordWikiEvidence(String text, Builder builder) {
if (text == null || text.isBlank()) {
return;
}
try {
JsonNode root = MAPPER.readTree(text);
recordWikiArray(root.path("chunks"), builder);
recordWikiArray(root.path("pages"), builder);
String title = root.path("title").asText("");
String rawTitle = root.path("rawTitle").asText("");
if (!title.isBlank()) {
builder.wikiPageTitle(title);
builder.wikiCitation(new WikiCitation(1, "", title, "", null));
}
if (!rawTitle.isBlank()) {
builder.wikiPageTitle(rawTitle);
builder.wikiCitation(new WikiCitation(1, "", rawTitle, "", null));
}
} catch (Exception ignored) {
}
}
private static void recordWikiArray(JsonNode nodes, Builder builder) {
if (!nodes.isArray()) {
return;
}
int ordinal = 1;
for (JsonNode node : nodes) {
int index = node.path("index").isInt() ? node.path("index").asInt() : ordinal;
String title = firstNonBlank(node.path("rawTitle").asText(""), node.path("title").asText(""));
String chunkId = node.path("chunkId").asText("");
String section = node.path("section").asText("");
Integer pageNumber = node.hasNonNull("pageNumber") ? node.path("pageNumber").asInt() : null;
if (!title.isBlank()) {
builder.wikiPageTitle(title);
}
if (!chunkId.isBlank()) {
builder.wikiChunkId(chunkId);
}
if (!title.isBlank() || !chunkId.isBlank()) {
builder.wikiCitation(new WikiCitation(index, chunkId, title, section, pageNumber));
}
ordinal++;
}
}
private static String firstNonBlank(String first, String second) {
return first != null && !first.isBlank() ? first : (second == null ? "" : second);
}
private static void recordSymbols(String text, Builder builder) {
Matcher matcher = DECLARED_TYPE.matcher(text);
while (matcher.find()) {
@ -180,6 +422,9 @@ public record SourceEvidenceLedger(
private final LinkedHashSet<String> sourcePaths = new LinkedHashSet<>();
private final LinkedHashSet<String> sourceSymbols = new LinkedHashSet<>();
private final LinkedHashSet<String> failedPaths = new LinkedHashSet<>();
private final LinkedHashSet<String> wikiPageTitles = new LinkedHashSet<>();
private final LinkedHashSet<String> wikiChunkIds = new LinkedHashSet<>();
private final LinkedHashSet<WikiCitation> wikiCitations = new LinkedHashSet<>();
void sourcePath(String path) {
String normalized = normalizePath(path);
@ -207,8 +452,63 @@ public record SourceEvidenceLedger(
}
}
void wikiPageTitle(String title) {
if (title != null && !title.isBlank()) {
wikiPageTitles.add(title.trim());
}
}
void wikiChunkId(String chunkId) {
if (chunkId != null && !chunkId.isBlank()) {
wikiChunkIds.add(chunkId.trim());
}
}
void wikiCitation(WikiCitation citation) {
if (citation == null || citation.index() < 1) {
return;
}
wikiCitations.removeIf(existing -> existing.index() == citation.index());
wikiCitations.add(citation.normalized());
}
SourceEvidenceLedger build() {
return new SourceEvidenceLedger(sourcePaths, sourceSymbols, failedPaths);
return new SourceEvidenceLedger(sourcePaths, sourceSymbols, failedPaths,
wikiPageTitles, wikiChunkIds, wikiCitations);
}
}
public record WikiCitation(int index, String chunkId, String title,
String section, Integer pageNumber) implements Serializable {
WikiCitation normalized() {
return new WikiCitation(index,
chunkId == null ? "" : chunkId.trim(),
title == null ? "" : title.trim(),
section == null ? "" : section.trim(),
pageNumber);
}
String sourceLine() {
StringBuilder sb = new StringBuilder();
sb.append("[").append(index).append("] ");
sb.append(title == null || title.isBlank() ? "chunkId=" + chunkId : title);
if (section != null && !section.isBlank()) {
sb.append(" - ").append(section);
}
if (pageNumber != null) {
sb.append(" - page ").append(pageNumber);
}
return sb.toString();
}
boolean matchesSourceLine(String line) {
if (line == null || line.isBlank()) {
return false;
}
if (title != null && !title.isBlank() && line.contains(title)) {
return true;
}
return chunkId != null && !chunkId.isBlank() && line.contains(chunkId);
}
}

View File

@ -131,6 +131,22 @@ public class AgentEntity {
@TableField(value = "tools_disabled")
private Boolean toolsDisabled;
/**
* Explicit opt-out from every knowledge base. When {@code true},
* {@code AgentBindingService.getBoundKbIds} returns
* {@link java.util.Collections#emptySet()} and the wiki tools degrade with
* their standard "no knowledge base" message; the webchat
* {@code /wiki/pages} picker endpoint returns an empty list. Without this
* flag, leaving the KB picker empty means "inherit workspace-wide" every
* KB visible which is the right default but leaves no way to express
* "this agent intentionally uses no KB" (issue #304).
*
* <p>Same defaulting / auto-clear / update strategy contract as
* {@link #skillsDisabled}.
*/
@TableField(value = "wiki_disabled")
private Boolean wikiDisabled;
@TableField(fill = FieldFill.INSERT)
private LocalDateTime createTime;

View File

@ -14,6 +14,7 @@ import java.time.Instant;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.locks.ReentrantLock;
/**
* Loader / writer for the per-conversation progress ledger persisted as a
@ -38,7 +39,7 @@ public class ProgressLedgerService {
new TypeReference<>() {};
/**
* Per-conversation mutex for the load-mutate-save sequence inside
* Per-conversation lock for the load-mutate-save sequence inside
* {@link #upsert}. Without this guard, a single agent turn that issues
* N parallel {@code progress_update} tool calls (observed: 12 calls in
* one batch when the model pre-registered every step at task start)
@ -46,13 +47,26 @@ public class ProgressLedgerService {
* the whole point of the ledger. Different conversations stay
* uncontended; only intra-conversation writes serialise.
*
* <p>Must be a {@link ReentrantLock}, not an intrinsic {@code synchronized}
* monitor. Tool calls execute on virtual threads, and the critical section
* spans blocking JDBC I/O (load + persist). A virtual thread that blocks
* whether on the DB call or while waiting to enter the lock pins its
* carrier when the lock is an intrinsic monitor. A turn that fires dozens
* of parallel {@code progress_update} calls on the same conversation then
* pins every carrier in the pool at once: the holder cannot be rescheduled
* to release its connection and exit, JDBC connections are held past the
* leak-detection threshold, and the whole server stops servicing requests.
* {@code ReentrantLock} parks via {@code LockSupport}, which unmounts the
* virtual thread and frees the carrier, so contention costs a park instead
* of a pinned platform thread.
*
* <p>Entries are computed on demand and never explicitly removed; even
* with thousands of long-running conversations the map stays bounded by
* the active conversation set, and any leak is a {@code Object} per
* conversation id small enough to ignore relative to the rest of the
* per-conv state already held in memory.
* the active conversation set, and any leak is one lock per conversation
* id small enough to ignore relative to the rest of the per-conv state
* already held in memory.
*/
private final ConcurrentHashMap<String, Object> upsertLocks = new ConcurrentHashMap<>();
private final ConcurrentHashMap<String, ReentrantLock> upsertLocks = new ConcurrentHashMap<>();
private final ConversationMapper conversationMapper;
private final ObjectMapper objectMapper;
@ -116,8 +130,9 @@ public class ProgressLedgerService {
// last save() drops the other's entry. Observed in production: a
// 12-entry pre-registration collapsed to 8 because four sibling
// tool calls landed in the same window.
Object mutex = upsertLocks.computeIfAbsent(conversationId, k -> new Object());
synchronized (mutex) {
ReentrantLock lock = upsertLocks.computeIfAbsent(conversationId, k -> new ReentrantLock());
lock.lock();
try {
ProgressLedger ledger = load(conversationId);
Map<String, ProgressEntry> map = ledger.asMap();
ProgressEntry existing = map.get(key);
@ -127,6 +142,8 @@ public class ProgressLedgerService {
map.put(key, new ProgressEntry(key, effectiveLabel, status, note, Instant.now()));
persist(conversationId, map);
return new ProgressLedger(map);
} finally {
lock.unlock();
}
}

View File

@ -0,0 +1,357 @@
package vip.mate.agent.service;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.chat.messages.SystemMessage;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.stereotype.Service;
import vip.mate.agent.AgentGraphBuilder;
import vip.mate.agent.vo.AgentDraftVO;
import vip.mate.exception.MateClawException;
import vip.mate.llm.model.ModelConfigEntity;
import vip.mate.llm.service.ModelConfigService;
import vip.mate.skill.model.SkillEntity;
import vip.mate.skill.service.SkillService;
import vip.mate.tool.model.AvailableToolDTO;
import vip.mate.tool.service.AvailableToolService;
import vip.mate.wiki.model.WikiKnowledgeBaseEntity;
import vip.mate.wiki.service.WikiKnowledgeBaseService;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
/**
* Turns a single natural-language requirement into a ready-to-review employee
* draft. The model is given the workspace's real capability catalog (tools,
* skills, knowledge bases) and asked to pick from it, so the resulting draft
* proposes a name, persona, type and a coherent set of capabilities in one
* shot. Every suggested capability is re-validated against the catalog before
* it leaves this service, so a hallucinated tool name or skill id never
* reaches the wizard.
*
* <p>The draft is intentionally not persisted here. The wizard renders it for
* review and edits, then commits through the existing agent-create and
* capability-binding endpoints reusing their tested persistence and audit
* paths rather than duplicating them.
*/
@Slf4j
@Service
@RequiredArgsConstructor
public class AgentGenerationService {
private final ModelConfigService modelConfigService;
private final AgentGraphBuilder agentGraphBuilder;
private final ObjectMapper objectMapper;
private final AvailableToolService availableToolService;
private final SkillService skillService;
private final WikiKnowledgeBaseService wikiKnowledgeBaseService;
/** Bound the catalog we feed the model so the prompt stays compact. */
private static final int MAX_TOOLS = 60;
private static final int MAX_SKILLS = 40;
private static final int MAX_KBS = 20;
public AgentDraftVO generateDraft(String requirement, Long workspaceId) {
if (requirement == null || requirement.isBlank()) {
throw new MateClawException("err.agent.generate_empty", 400,
"Please describe the employee you want to create");
}
long wsId = workspaceId != null ? workspaceId : 1L;
ModelConfigEntity defaultModel = modelConfigService.getDefaultModel();
if (defaultModel == null) {
throw new MateClawException("err.agent.generate_no_model", 400,
"No default model is configured yet");
}
// Build the capability catalog the model is allowed to pick from.
List<AvailableToolDTO> tools = bindableTools();
List<SkillEntity> skills = workspaceSkills(wsId);
List<WikiKnowledgeBaseEntity> kbs = workspaceKbs(wsId);
String systemPrompt = buildSystemPrompt();
String userPrompt = buildUserPrompt(requirement.trim(), tools, skills, kbs);
String raw;
try {
ChatModel chatModel = agentGraphBuilder.buildRuntimeChatModel(defaultModel);
ChatResponse response = chatModel.call(new Prompt(List.of(
new SystemMessage(systemPrompt),
new UserMessage(userPrompt))));
raw = response != null && response.getResult() != null
&& response.getResult().getOutput() != null
? response.getResult().getOutput().getText() : null;
} catch (Exception e) {
log.warn("[AgentGen] LLM call failed: {}", e.getMessage());
throw new MateClawException("err.agent.generate_failed", 500,
"Failed to generate employee draft");
}
JsonNode root = parseJson(raw);
if (root == null || !root.isObject()) {
throw new MateClawException("err.agent.generate_failed", 500,
"Model returned an unexpected response");
}
return toDraft(root, tools, skills, kbs);
}
// ==================== Catalog ====================
private List<AvailableToolDTO> bindableTools() {
List<AvailableToolDTO> all;
try {
all = availableToolService.listAvailable();
} catch (Exception e) {
log.warn("[AgentGen] failed to list tools: {}", e.getMessage());
return List.of();
}
List<AvailableToolDTO> out = new ArrayList<>();
for (AvailableToolDTO t : all) {
// Only offer tools that are currently bindable and reachable; a
// stale MCP tool would resolve to nothing at chat time.
if (t != null && t.isAvailable() && !t.isStale()
&& t.getName() != null && !t.getName().isBlank()) {
out.add(t);
if (out.size() >= MAX_TOOLS) break;
}
}
return out;
}
private List<SkillEntity> workspaceSkills(long wsId) {
try {
List<SkillEntity> skills = skillService.listEnabledSkills(wsId);
return skills.size() > MAX_SKILLS ? skills.subList(0, MAX_SKILLS) : skills;
} catch (Exception e) {
log.warn("[AgentGen] failed to list skills: {}", e.getMessage());
return List.of();
}
}
private List<WikiKnowledgeBaseEntity> workspaceKbs(long wsId) {
try {
List<WikiKnowledgeBaseEntity> kbs = wikiKnowledgeBaseService.listByWorkspace(wsId);
return kbs.size() > MAX_KBS ? kbs.subList(0, MAX_KBS) : kbs;
} catch (Exception e) {
log.warn("[AgentGen] failed to list knowledge bases: {}", e.getMessage());
return List.of();
}
}
// ==================== Prompt ====================
private String buildSystemPrompt() {
return """
You are an employee (AI agent) configuration generator for an agent platform.
Given a one-sentence requirement, output a single JSON object describing one
ready-to-use employee. Respond in the SAME language as the requirement.
Output ONLY the JSON object, no prose, no markdown fences. Schema:
{
"name": "short display name, no instruction words",
"icon": "a single emoji matching the role",
"description": "one concise sentence shown on the roster card",
"agentType": "react | plan_execute",
"role": "short role label",
"goal": "one short sentence on what this employee achieves",
"systemPrompt": "the full persona prompt: who it is, how it works, constraints",
"tags": ["1-3 short tags"],
"recommendedQuestions": ["2-4 starter questions a user might ask first"],
"tools": ["tool names chosen ONLY from the provided tool catalog"],
"skillIds": ["skill ids chosen ONLY from the provided skill catalog, as strings"],
"primaryKbId": "one knowledge base id from the catalog, or null"
}
Rules:
- Use agentType "plan_execute" only for multi-step / long-horizon work; otherwise "react".
- Pick tools, skillIds and primaryKbId ONLY from the catalogs given below. Never invent
names or ids. If nothing fits, return an empty array (or null for primaryKbId).
- Prefer the smallest capability set that satisfies the requirement.
- Skills already bundle their own tools, so do not also list a tool a chosen skill provides.
""";
}
private String buildUserPrompt(String requirement, List<AvailableToolDTO> tools,
List<SkillEntity> skills, List<WikiKnowledgeBaseEntity> kbs) {
StringBuilder sb = new StringBuilder();
sb.append("Requirement:\n").append(requirement).append("\n\n");
sb.append("Tool catalog (name — description):\n");
if (tools.isEmpty()) {
sb.append("(none)\n");
} else {
for (AvailableToolDTO t : tools) {
sb.append("- ").append(t.getName());
if (t.getDescription() != null && !t.getDescription().isBlank()) {
sb.append("").append(trim(t.getDescription(), 120));
}
sb.append('\n');
}
}
sb.append("\nSkill catalog (id — name — description):\n");
if (skills.isEmpty()) {
sb.append("(none)\n");
} else {
for (SkillEntity s : skills) {
sb.append("- ").append(s.getId()).append("").append(s.getName());
if (s.getDescription() != null && !s.getDescription().isBlank()) {
sb.append("").append(trim(s.getDescription(), 120));
}
sb.append('\n');
}
}
sb.append("\nKnowledge base catalog (id — name — description):\n");
if (kbs.isEmpty()) {
sb.append("(none)\n");
} else {
for (WikiKnowledgeBaseEntity kb : kbs) {
sb.append("- ").append(kb.getId()).append("").append(kb.getName());
if (kb.getDescription() != null && !kb.getDescription().isBlank()) {
sb.append("").append(trim(kb.getDescription(), 120));
}
sb.append('\n');
}
}
return sb.toString();
}
// ==================== Parse + validate ====================
private AgentDraftVO toDraft(JsonNode root, List<AvailableToolDTO> tools,
List<SkillEntity> skills, List<WikiKnowledgeBaseEntity> kbs) {
String name = text(root, "name");
if (name.isBlank()) {
name = "New employee";
}
String agentType = text(root, "agentType");
if (!"plan_execute".equals(agentType)) {
agentType = "react";
}
return AgentDraftVO.builder()
.name(trim(name, 60))
.icon(firstEmoji(text(root, "icon")))
.description(trim(text(root, "description"), 200))
.agentType(agentType)
.role(trim(text(root, "role"), 60))
.goal(trim(text(root, "goal"), 120))
.systemPrompt(text(root, "systemPrompt"))
.tags(stringList(root.get("tags"), 5))
.recommendedQuestions(stringList(root.get("recommendedQuestions"), 4))
.tools(validTools(root.get("tools"), tools))
.skillIds(validSkillIds(root.get("skillIds"), skills))
.primaryKbId(validKbId(root.get("primaryKbId"), kbs))
.build();
}
private List<String> validTools(JsonNode node, List<AvailableToolDTO> catalog) {
Set<String> allowed = new LinkedHashSet<>();
for (AvailableToolDTO t : catalog) allowed.add(t.getName());
List<String> out = new ArrayList<>();
if (node != null && node.isArray()) {
for (JsonNode n : node) {
String v = n.asText("");
if (allowed.contains(v) && !out.contains(v)) out.add(v);
}
}
return out;
}
private List<Long> validSkillIds(JsonNode node, List<SkillEntity> catalog) {
Map<Long, Boolean> allowed = new LinkedHashMap<>();
for (SkillEntity s : catalog) allowed.put(s.getId(), Boolean.TRUE);
List<Long> out = new ArrayList<>();
if (node != null && node.isArray()) {
for (JsonNode n : node) {
Long id = asLong(n);
if (id != null && allowed.containsKey(id) && !out.contains(id)) out.add(id);
}
}
return out;
}
private Long validKbId(JsonNode node, List<WikiKnowledgeBaseEntity> catalog) {
Long id = asLong(node);
if (id == null) return null;
for (WikiKnowledgeBaseEntity kb : catalog) {
if (kb.getId().equals(id)) return id;
}
return null;
}
// ==================== Helpers ====================
private JsonNode parseJson(String response) {
if (response == null || response.isBlank()) return null;
String cleaned = response.trim();
if (cleaned.startsWith("```json")) cleaned = cleaned.substring(7);
else if (cleaned.startsWith("```")) cleaned = cleaned.substring(3);
if (cleaned.endsWith("```")) cleaned = cleaned.substring(0, cleaned.length() - 3);
cleaned = cleaned.trim();
try {
return objectMapper.readTree(cleaned);
} catch (Exception e) {
log.debug("[AgentGen] JSON parse failed: {}", e.getMessage());
return null;
}
}
/** Accept both numeric and textual ids — textual is preferred to preserve precision. */
private Long asLong(JsonNode node) {
if (node == null || node.isNull()) return null;
try {
if (node.isTextual()) {
String v = node.asText().trim();
return v.isEmpty() ? null : Long.parseLong(v);
}
if (node.isNumber()) return node.asLong();
} catch (NumberFormatException ignored) {
// fall through
}
return null;
}
private static String text(JsonNode root, String field) {
JsonNode n = root.get(field);
return n == null || n.isNull() ? "" : n.asText("").trim();
}
private static List<String> stringList(JsonNode node, int max) {
List<String> out = new ArrayList<>();
if (node != null && node.isArray()) {
for (JsonNode n : node) {
String v = n.asText("").trim();
if (!v.isEmpty() && !out.contains(v)) {
out.add(v);
if (out.size() >= max) break;
}
}
}
return out;
}
private static String trim(String s, int max) {
if (s == null) return "";
String t = s.trim();
return t.length() > max ? t.substring(0, max) : t;
}
/** Keep only the first emoji-ish glyph so the icon column never holds a sentence. */
private static String firstEmoji(String s) {
if (s == null || s.isBlank()) return "🤖";
String t = s.trim();
int end = t.offsetByCodePoints(0, Math.min(t.codePointCount(0, t.length()), 1));
return t.substring(0, end);
}
}

View File

@ -0,0 +1,65 @@
package vip.mate.agent.vo;
import com.fasterxml.jackson.databind.annotation.JsonSerialize;
import com.fasterxml.jackson.databind.ser.std.ToStringSerializer;
import lombok.Builder;
import lombok.Data;
import java.util.List;
/**
* An AI-generated employee draft produced from a single natural-language
* requirement. The draft is never persisted on its own the create wizard
* shows it for review, lets the user tweak any field, then commits it through
* the normal agent-create and capability-binding endpoints.
*
* <p>Every suggested capability ({@link #tools}, {@link #skillIds},
* {@link #primaryKbId}) is validated against the workspace catalog before the
* draft is returned, so the wizard never offers a tool name or knowledge base
* that does not actually exist.
*/
@Data
@Builder
public class AgentDraftVO {
/** Display name for the new employee. */
private String name;
/** Emoji icon chosen to match the role. */
private String icon;
/** One-line description shown on the roster card. */
private String description;
/** Runtime kind: {@code react} or {@code plan_execute}. */
private String agentType;
/** Assembled persona / system prompt, editable before commit. */
private String systemPrompt;
/** Short role label, used for the card tagline preview. */
private String role;
/** Short goal statement, used for the card tagline preview. */
private String goal;
/** Suggested tags. */
private List<String> tags;
/** A few starter questions to seed the first conversation. */
private List<String> recommendedQuestions;
/**
* Tool names to bind, drawn from the workspace's available tool catalog
* (built-in and MCP). Hallucinated names are dropped during validation.
*/
private List<String> tools;
/** Skill ids to bind, validated against the workspace's enabled skills. */
@JsonSerialize(contentUsing = ToStringSerializer.class)
private List<Long> skillIds;
/** Primary knowledge base id to attach, or null when none fits. */
@JsonSerialize(using = ToStringSerializer.class)
private Long primaryKbId;
}

View File

@ -63,6 +63,37 @@ public class AuditEventService {
}
}
/**
* 异步记录审计事件,显式指定 actor(而非从 SecurityContext 推导)
* <p>用于非 MateClaw 用户的写操作 当前主要是 webchat 访客actor 形如
* {@code "webchat:<channelId>:<visitorId>"},{@code userId} 0(访客没有 MateClaw 账户)
* IP / User-Agent 仍尽量从当前请求抓取(webEnvironment=NONE 下为 null,可接受)
*/
public void recordAs(String actor, Long workspaceId, String action, String resourceType,
String resourceId, String resourceName, String detailJson) {
AuditEventEntity event = new AuditEventEntity();
event.setUsername(actor != null ? actor : "system");
event.setUserId(0L);
event.setAction(action);
event.setResourceType(resourceType);
event.setResourceId(resourceId);
event.setResourceName(resourceName);
event.setDetailJson(detailJson);
event.setWorkspaceId(workspaceId);
event.setCreateTime(LocalDateTime.now());
try {
ServletRequestAttributes attrs = (ServletRequestAttributes) RequestContextHolder.getRequestAttributes();
if (attrs != null) {
HttpServletRequest request = attrs.getRequest();
event.setIpAddress(getClientIp(request));
event.setUserAgent(truncate(request.getHeader("User-Agent"), 256));
}
} catch (Exception ignored) {
// 异步或非 web 上下文跳过 IP/UA
}
insertAsync(event);
}
@Async
void insertAsync(AuditEventEntity event) {
try {

View File

@ -43,7 +43,8 @@ public class ChannelChatOriginFactory {
/* channelType */ message.getChannelType() != null
? message.getChannelType()
: channel.getChannelType(),
/* chatId */ message.getChatId());
/* chatId */ message.getChatId(),
/* baseUrl */ null); // IM origins have no request host; rely on public-base-url config
}
/**

View File

@ -6,6 +6,7 @@ import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.event.EventListener;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Component;
import vip.mate.channel.dingtalk.DingTalkChannelAdapter;
import vip.mate.channel.discord.DiscordChannelAdapter;
@ -230,9 +231,12 @@ public class ChannelManager {
);
/**
* 应用启动完成后自动加载并启动所有已启用的渠道
* 使用 ApplicationReadyEvent 确保数据库 schema/data 初始化完成
* 应用启动完成后自动加载并启动所有已启用的渠道
* 使用 ApplicationReadyEvent 确保数据库 schema/data 初始化完成
* {@code @Async} 渠道适配器的网络建连 Discord WebSocket / Telegram webhook
* 可能因外部网络不可达而阻塞数分钟异步启动避免卡住主线程
*/
@Async
@EventListener(ApplicationReadyEvent.class)
public void init() {
log.info("Initializing ChannelManager...");

View File

@ -1310,6 +1310,15 @@ public class ChannelMessageRouter {
return message.getChannelType() + ":" + identifier;
}
/**
* Read-only existence check for a conversation by its logical id. Used by
* adapters that need to alias a legacy conversationId scheme to a new one
* without rewriting stored rows (e.g. Feishu group session-id migration).
*/
public boolean conversationExists(String conversationId) {
return conversationService.findByConversationId(conversationId) != null;
}
/**
* Build a sender-attribution tag for group messages. Returns
* {@code [@senderName]} when the message is from a multi-user channel

View File

@ -707,7 +707,7 @@ public class DingTalkChannelAdapter extends AbstractChannelAdapter implements St
}
private static final java.util.regex.Pattern GENERATED_URL_PATTERN =
java.util.regex.Pattern.compile("/api/v1/files/generated/([a-zA-Z0-9-]+)");
java.util.regex.Pattern.compile("(?:https?://[^/\\s)\\]]+)?/api/v1/files/generated/([a-zA-Z0-9-]+)");
private static boolean isImageMime(String mimeType) {
return mimeType != null && mimeType.toLowerCase().startsWith("image/");

View File

@ -86,6 +86,38 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
/** 消息去重:最近处理过的 message_id */
private final Set<String> processedMessageIds = ConcurrentHashMap.newKeySet();
/**
* 群内 bot 别名缓存chatId 学到的别名集合openId / unionId / userId / name
* <p>飞书 SDK 投递的 mention bot 的标识可能是群内自定义别名{@code ou_357e...} / 自定义名称
* 而不是 {@code /bot/v3/info} 返回的全局 openId / app_name我们在双投递场景下
* 机会性地学习这些别名后续单事件投递的消息就能命中缓存
*/
private final ConcurrentHashMap<String, Set<String>> chatBotAliases = new ConcurrentHashMap<>();
/** Max learned aliases retained per chat, to bound memory on busy groups. */
private static final int CHAT_ALIAS_MAX = 64;
/**
* Per-messageId mention tracker TTL
* <p>飞书 SDK 经常对同一条消息双投递一份 mentions bot <em>全局身份</em>来自 /bot/v3/info
* 另一份含 bot <em>群内别名</em>我们累积同一 messageId <em> mention</em>投递看到的标识
* 一旦其中任何一份被识别为 @bot就把累积的标识写入 {@link #chatBotAliases}
* <p>只累积单 mention 投递是有意为之 mention 投递 {@code @bot @某人}会把 bot
* 被同时 @ 的人混在一起无法区分若整体学习会把人误学成 bot 别名导致之后 @ 该人的消息
* 被误判为 @bot而飞书双投递里 bot 别名那一份本身就是单 mention所以这样既安全又不丢功能
*/
private final ConcurrentHashMap<String, MentionTrack> mentionTracker = new ConcurrentHashMap<>();
/** mention tracker 条目 TTL60s 远大于双投递的真实间隔,几个 ms 级别)。 */
private static final long MENTION_TRACK_TTL_MS = 60_000L;
/** Package-private for testing. */
static final class MentionTrack {
final Set<String> seenIds = ConcurrentHashMap.newKeySet();
final long createdAtMs = System.currentTimeMillis();
volatile boolean matched = false;
}
/** 昵称缓存open_id → 显示名称 */
private final ConcurrentHashMap<String, String> nicknameCache = new ConcurrentHashMap<>();
private static final int NICKNAME_CACHE_MAX = 500;
@ -114,6 +146,9 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
/** Bot's own open_id, fetched once from /open-apis/bot/v3/info and cached. */
private volatile String botOpenId;
/** Bot's display name (app_name), fetched alongside open_id. Used for name-based mention matching. */
private volatile String botName;
/** Serializes lazy bot-open-id fetches so concurrent group messages share one API roundtrip. */
private final Object botOpenIdLock = new Object();
@ -187,11 +222,15 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
record RecentFileEntry(String fileName, String path, String fileUrl, String contentType) {}
private final Cache<String, List<RecentFileEntry>> recentFileCache = Caffeine.newBuilder()
// Package-private for testing: seed the cache directly to verify injection paths.
final Cache<String, List<RecentFileEntry>> recentFileCache = Caffeine.newBuilder()
.expireAfterWrite(RECENT_FILE_TTL_MINUTES, TimeUnit.MINUTES)
.maximumSize(200)
.build();
// Package-private for testing: redirect to a temp directory without touching real disk.
Path chatUploadsRoot = Path.of("data", "chat-uploads");
public FeishuChannelAdapter(ChannelEntity channelEntity,
ChannelMessageRouter messageRouter,
ObjectMapper objectMapper) {
@ -331,9 +370,12 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
// a torn write that could re-cache a stale id.
synchronized (botOpenIdLock) {
this.botOpenId = null;
this.botName = null;
this.botOpenIdLastFailureMs = 0L;
}
this.processedMessageIds.clear();
this.chatBotAliases.clear();
this.mentionTracker.clear();
this.nicknameCache.clear();
this.quotedMessageCache.clear();
log.info("[feishu] Feishu channel stopped");
@ -643,14 +685,109 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
senderOpenId = sender.getSenderId().getOpenId();
}
boolean isBotMentioned = isBotMentionedInEvent(message.getMentions());
com.lark.oapi.service.im.v1.model.MentionEvent[] mentions = message.getMentions();
boolean isBotMentioned = detectBotMentionWithLearning(mentions, chatId, messageId);
handleFeishuMessage(messageId, messageType, contentStr, chatId, chatType, senderOpenId, parentId, isBotMentioned, event);
}
// ==================== @提及检测 ====================
private boolean isBotMentionedInEvent(com.lark.oapi.service.im.v1.model.MentionEvent[] mentions) {
return eventMentionsContainBot(mentions, getBotOpenId());
/**
* 判断本次事件是否 @ bot并机会性地学习"群内 bot 别名"
*
* <p>飞书 SDK 在群内对同一条 @bot 的消息会双投递两个事件两次的 mentions 数据形态不同
* <ul>
* <li>一份带 bot <em>全局身份</em> {@code /open-apis/bot/v3/info} 返回的 openId / app_name 一致</li>
* <li>一份带 bot <em>群内别名</em>用户给 bot 起的 chat-scope openId 也是另一套</li>
* </ul>
* 重启后第一条消息能命中"全局身份"那一份直接匹配后续消息往往只来一份"群内别名"
* 本方法在双投递可见时把两份的所有标识聚合到 {@link #chatBotAliases}后续单事件投递就能命中缓存放行
*
* <p>识别顺序
* <ol>
* <li>直接匹配 {@code /bot/v3/info} 拿到的 botOpenId / botName</li>
* <li> {@link #chatBotAliases} 缓存里学到的群内别名</li>
* <li>双投递推断同一 messageId 之前的事件已被识别 本事件的 mentions 也是 bot 的别名</li>
* </ol>
*/
private boolean detectBotMentionWithLearning(com.lark.oapi.service.im.v1.model.MentionEvent[] mentions,
String chatId, String messageId) {
return detectBotMentionWithLearning(mentions, chatId, messageId, getBotOpenId(), botName);
}
/** Package-private for testing: 纯有状态核心bot 身份由调用方显式传入(避免触发 /bot/v3/info HTTP。 */
boolean detectBotMentionWithLearning(com.lark.oapi.service.im.v1.model.MentionEvent[] mentions,
String chatId, String messageId,
String botOpenId, String botName) {
if (mentions == null || mentions.length == 0) {
return false;
}
cleanupMentionTracker();
// 把本次事件看到的所有标识累积到 per-messageId tracker 即使本次匹配不上
// 后到的事件如果匹配成功learnFromTrack 会把它们一起 cache
MentionTrack track = null;
if (messageId != null) {
track = mentionTracker.computeIfAbsent(messageId, k -> new MentionTrack());
// Only single-mention deliveries are unambiguous bot identities. A
// multi-mention delivery (e.g. @bot @alice) mixes the bot with
// co-mentioned humans that must NOT be learned as aliases; Feishu's
// dual-delivery alias form is itself a single mention, so this is safe.
if (mentions.length == 1) {
collectMentionIdentifiers(mentions, track.seenIds);
}
}
// 1. 直接匹配 bot 的全局身份
if (eventMentionsContainBot(mentions, botOpenId, botName)) {
learnFromTrack(chatId, track);
if (track != null) track.matched = true;
return true;
}
// 2. 群内已学习别名命中
if (chatId != null) {
Set<String> learned = chatBotAliases.get(chatId);
if (learned != null && mentionMatchesAnyAlias(mentions, learned)) {
log.info("[feishu] @bot matched via learned chat alias: chatId={}, messageId={}", chatId, messageId);
learnFromTrack(chatId, track);
if (track != null) track.matched = true;
return true;
}
}
// 3. 双投递学习 messageId 的另一次投递已被识别 本事件 mentions bot 别名
if (track != null && track.matched) {
log.info("[feishu] @bot inferred via dual-delivery learning: chatId={}, messageId={}", chatId, messageId);
learnFromTrack(chatId, track);
return true;
}
return false;
}
private void learnFromTrack(String chatId, MentionTrack track) {
if (chatId == null || track == null || track.seenIds.isEmpty()) return;
Set<String> aliases = chatBotAliases.computeIfAbsent(chatId, k -> ConcurrentHashMap.newKeySet());
if (aliases.size() >= CHAT_ALIAS_MAX) return;
int before = aliases.size();
aliases.addAll(track.seenIds);
int added = aliases.size() - before;
if (added > 0) {
log.info("[feishu] Learned {} new bot alias(es) for chat={} (cache size={})",
added, chatId, aliases.size());
}
}
private void cleanupMentionTracker() {
evictStaleTracks(mentionTracker, System.currentTimeMillis(), MENTION_TRACK_TTL_MS);
}
/** Package-private for testing: 按 TTL 淘汰 mention tracker 中的过期项({@code nowMs} 显式传入便于测试)。 */
static void evictStaleTracks(Map<String, MentionTrack> tracker, long nowMs, long ttlMs) {
long cutoff = nowMs - ttlMs;
tracker.entrySet().removeIf(e -> e.getValue().createdAtMs < cutoff);
}
private boolean isBotMentionedInWebhookMessage(Map<String, Object> message) {
@ -659,12 +796,51 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
return webhookMentionsContainBot(list, getBotOpenId());
}
/** Package-private for testing: 判断 SDK mentions 数组中是否包含指定 open_id */
/** Package-private for testing: 判断 SDK mentions 数组中是否包含指定 bot按 openId / unionId / userId / name 命中) */
static boolean eventMentionsContainBot(com.lark.oapi.service.im.v1.model.MentionEvent[] mentions,
String botOpenId) {
if (mentions == null || mentions.length == 0 || botOpenId == null) return false;
String botOpenId, String botName) {
if (mentions == null || mentions.length == 0) return false;
for (var mention : mentions) {
if (mention.getId() != null && botOpenId.equals(mention.getId().getOpenId())) return true;
var id = mention.getId();
// 匹配 openIdunionIduserId 中的任意一个
if (id != null && botOpenId != null) {
if (botOpenId.equals(id.getOpenId())) return true;
if (botOpenId.equals(id.getUnionId())) return true;
if (botOpenId.equals(id.getUserId())) return true;
}
// 飞书 SDK bot mention 可能使用不同 ID 体系fallback name 匹配
if (botName != null && botName.equals(mention.getName())) return true;
}
return false;
}
/** Package-private for testing: 把 mentions 中每个非空 openId / unionId / userId / name 灌入 sink。 */
static void collectMentionIdentifiers(com.lark.oapi.service.im.v1.model.MentionEvent[] mentions,
Set<String> sink) {
if (mentions == null || sink == null) return;
for (var mention : mentions) {
var id = mention.getId();
if (id != null) {
if (id.getOpenId() != null) sink.add(id.getOpenId());
if (id.getUnionId() != null) sink.add(id.getUnionId());
if (id.getUserId() != null) sink.add(id.getUserId());
}
if (mention.getName() != null) sink.add(mention.getName());
}
}
/** Package-private for testing: mentions 中是否有任意 openId / unionId / userId / name 命中 aliases 集合。 */
static boolean mentionMatchesAnyAlias(com.lark.oapi.service.im.v1.model.MentionEvent[] mentions,
Set<String> aliases) {
if (mentions == null || mentions.length == 0 || aliases == null || aliases.isEmpty()) return false;
for (var mention : mentions) {
var id = mention.getId();
if (id != null) {
if (id.getOpenId() != null && aliases.contains(id.getOpenId())) return true;
if (id.getUnionId() != null && aliases.contains(id.getUnionId())) return true;
if (id.getUserId() != null && aliases.contains(id.getUserId())) return true;
}
if (mention.getName() != null && aliases.contains(mention.getName())) return true;
}
return false;
}
@ -734,7 +910,10 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
Map<?, ?> bot = (Map<?, ?>) body.get("bot");
if (bot != null && bot.get("open_id") instanceof String openId && !openId.isBlank()) {
botOpenId = openId;
log.info("[feishu] Bot open_id fetched and cached: {}", openId);
if (bot.get("app_name") instanceof String name && !name.isBlank()) {
botName = name;
}
log.info("[feishu] Bot info fetched: open_id={}, name={}", openId, botName);
return openId;
}
// 2xx with no bot.open_id field treat as transient failure.
@ -794,6 +973,7 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
HttpRequest request = HttpRequest.newBuilder()
.uri(URI.create(apiBase + "/open-apis/auth/v3/tenant_access_token/internal"))
.header("Content-Type", "application/json; charset=utf-8")
.timeout(Duration.ofSeconds(10))
.POST(HttpRequest.BodyPublishers.ofString(jsonBody))
.build();
@ -940,10 +1120,16 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
// prompt only exposes the file name (not its path) to the model so if this id does
// not match, ReadFileTool / DocumentExtractTool cannot find the cached file.
String shortSuffix = generateShortSessionSuffix(chatId, senderOpenId, isGroup);
// 群会话改用完整 chatId但存量旧会话仍在 legacy 后缀下读时别名回退
// 让升级前已存在的群沿用旧 conversationId 延续不重写存量行
if (isGroup && chatId != null) {
shortSuffix = resolveGroupSessionSuffix(chatId);
}
String conversationId = buildConversationId(shortSuffix, senderOpenId, isGroup);
String stagedUploadPath = null;
if (isFileMessage) {
cacheRecentFile(messageId, messageType, contentStr, conversationId);
stagedUploadPath = cacheRecentFile(messageId, messageType, contentStr, conversationId);
}
// require_mention 群聊过滤群聊中必须 @机器人才响应
@ -979,7 +1165,7 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
// 解析消息内容
List<MessageContentPart> contentParts = new ArrayList<>();
String textContent = extractContentParts(messageId, messageType, contentStr, contentParts);
String textContent = extractContentParts(messageId, messageType, contentStr, contentParts, stagedUploadPath);
if (contentParts.isEmpty() && (textContent == null || textContent.isBlank())) {
log.debug("[feishu] Empty message content, ignoring");
@ -1379,19 +1565,22 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
return null;
}
// ==================== 会话 ID 优化 ====================
// ==================== 会话 ID ====================
/**
* 生成更短的会话标识后缀
* - 群聊app_id 4 + "_" + chat_id 8
* - 私聊open_id 12
* 生成会话标识后缀
* <ul>
* <li>群聊直接使用完整 {@code chatId}全局唯一旧实现用 {@code {appId后4}_{chatId后8}}
* 截断后缀不同群的 {@code chatId} 8 位可能相同 会话串台改用完整 chatId 消除碰撞
* 存量旧会话不重写 {@link #resolveGroupSessionSuffix} 做读时别名回退</li>
* <li>私聊保持原状 {@code openId} 12 注意私聊路径下该后缀实际不参与
* conversationId{@link #buildConversationId} DM 直接用完整 {@code senderOpenId}
* 故私聊会话 ID 不受本次改动影响</li>
* </ul>
*/
private String generateShortSessionSuffix(String chatId, String openId, boolean isGroup) {
if (isGroup && chatId != null) {
String appId = getConfigString("app_id", "");
String appSuffix = appId.length() >= 4 ? appId.substring(appId.length() - 4) : appId;
String chatSuffix = chatId.length() >= 8 ? chatId.substring(chatId.length() - 8) : chatId;
return appSuffix + "_" + chatSuffix;
return chatId;
}
if (openId != null) {
return openId.length() >= 12 ? openId.substring(openId.length() - 12) : openId;
@ -1402,6 +1591,49 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
return null;
}
/**
* 旧群会话后缀算法{@code {appId后4}_{chatId后8}}仅用于读时别名回退
* 在不重写存量行的前提下让升级前已存在的群会话沿用旧 conversationId 无缝延续
*/
// Package-private for testing.
String legacyGroupSuffix(String chatId) {
if (chatId == null) return null;
String appId = getConfigString("app_id", "");
String appSuffix = appId.length() >= 4 ? appId.substring(appId.length() - 4) : appId;
String chatSuffix = chatId.length() >= 8 ? chatId.substring(chatId.length() - 8) : chatId;
return appSuffix + "_" + chatSuffix;
}
/**
* 群会话后缀的读时别名回退不重写存量
* <ul>
* <li>新群两个 key 都无会话 用完整 chatId canonical key</li>
* <li>已迁移群canonical key 已有会话 canonical</li>
* <li>存量群canonical legacy 沿用 legacy key历史无缝延续</li>
* </ul>
*/
// Package-private for testing.
String resolveGroupSessionSuffix(String chatId) {
String canonical = chatId;
String legacy = legacyGroupSuffix(chatId);
if (legacy == null || legacy.equals(canonical)) return canonical;
boolean canonicalExists = messageRouter.conversationExists(CHANNEL_TYPE + ":" + canonical);
boolean legacyExists = !canonicalExists
&& messageRouter.conversationExists(CHANNEL_TYPE + ":" + legacy);
String picked = pickGroupSessionSuffix(canonical, legacy, canonicalExists, legacyExists);
if (picked.equals(legacy)) {
log.info("[feishu] Reusing legacy group session id for chat={} (read-time alias, no migration write)", chatId);
}
return picked;
}
/** Package-private for testing: 纯选择逻辑——存量 legacy 会话存在且尚未迁移时沿用 legacy否则用 canonical。 */
static String pickGroupSessionSuffix(String canonical, String legacy,
boolean canonicalExists, boolean legacyExists) {
if (!canonicalExists && legacyExists) return legacy;
return canonical;
}
/**
* Compute the conversationId that {@link ChannelMessageRouter} would
* derive for this chat, so we can save inbound files to the matching
@ -1431,10 +1663,17 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
* ({@code ReadFileTool}, {@code DocumentExtractTool}) can find it
* via {@code ChatUploadResolver}, and it gets cleaned up when the
* conversation is deleted.
*
* @return absolute path of the staged {@code data/chat-uploads/} copy, or
* {@code null} when nothing was cached (unsupported type, missing
* file key, download failure). Callers stamp this path onto the
* current-message content part so the path surfaced to the LLM is
* resolver-reachable instead of the sandbox-external media path.
*/
private void cacheRecentFile(String messageId, String messageType, String contentStr,
private String cacheRecentFile(String messageId, String messageType, String contentStr,
String conversationId) {
try {
log.info("[feishu] cacheRecentFile: type={}, conversationId={}, messageId={}", messageType, conversationId, messageId);
Map<String, Object> contentObj = objectMapper.readValue(contentStr, Map.class);
String fileKey = null;
@ -1461,20 +1700,20 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
type = "file";
}
default -> {
return;
return null;
}
}
if (fileKey == null) return;
if (fileKey == null) return null;
// Download file bytes
DownloadedResource dl = "image".equals(messageType)
? maybeDownloadImage(messageId, fileKey)
: maybeDownloadResource(messageId, fileKey, type, fileName);
if (dl == null) return;
if (dl == null) return null;
// Save to data/chat-uploads/{conversationId}/
Path uploadDir = Path.of("data", "chat-uploads", conversationId);
Path uploadDir = chatUploadsRoot.resolve(conversationId);
Files.createDirectories(uploadDir);
String rawName = (dl.fileName() != null && !dl.fileName().isBlank())
? dl.fileName() : fileKey;
@ -1502,8 +1741,10 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
log.info("[feishu] Cached recent file for conversation={}: {} ({} bytes, {})",
conversationId, entry.fileName(), Files.size(dest), contentType);
return dest.toAbsolutePath().toString();
} catch (Exception e) {
log.debug("[feishu] Failed to cache recent file: {}", e.getMessage());
log.warn("[feishu] Failed to cache recent file for conversation={}: {}", conversationId, e.getMessage(), e);
return null;
}
}
@ -1514,9 +1755,17 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
*
* @return updated textContent with file descriptions appended
*/
private String injectRecentFiles(String conversationId, List<MessageContentPart> parts, String textContent) {
// Package-private for testing.
String injectRecentFiles(String conversationId, List<MessageContentPart> parts, String textContent) {
List<RecentFileEntry> recent = recentFileCache.getIfPresent(conversationId);
if (recent == null || recent.isEmpty()) return textContent;
if (recent == null || recent.isEmpty()) {
// Fallback: scan data/chat-uploads/{conversationId}/ on disk.
// Survives process restart / Caffeine TTL expiry / GC eviction.
recent = loadRecentFilesFromDisk(conversationId);
if (recent.isEmpty()) return textContent;
log.info("[feishu] injectRecentFiles: cache miss, recovered {} file(s) from disk for conversation={}",
recent.size(), conversationId);
}
// Collect paths already in parts to avoid duplicates
Set<String> existingPaths = new java.util.HashSet<>();
@ -1546,20 +1795,111 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
return text.toString();
}
/**
* Scan {@code data/chat-uploads/{conversationId}/} on disk and return
* the most recent files as {@link RecentFileEntry}s. Used as a
* fallback when the in-memory Caffeine cache has been evicted
* (process restart, TTL expiry, GC pressure) but the staged copies
* are still on disk.
*/
private List<RecentFileEntry> loadRecentFilesFromDisk(String conversationId) {
long cutoff = System.currentTimeMillis() - RECENT_FILE_TTL_MINUTES * 60_000L;
return loadRecentFilesFromDisk(chatUploadsRoot.resolve(conversationId), cutoff);
}
/**
* Package-private for testing: explicit {@code dir} and {@code cutoffMs} make the
* test deterministic without temp-directory path construction or time mocking.
* Production callers go through {@link #loadRecentFilesFromDisk(String)}.
*/
List<RecentFileEntry> loadRecentFilesFromDisk(Path dir, long cutoffMs) {
if (!Files.isDirectory(dir)) return List.of();
try (var stream = Files.list(dir)) {
return stream
.filter(Files::isRegularFile)
.filter(p -> {
try {
return Files.getLastModifiedTime(p).toMillis() >= cutoffMs;
} catch (Exception e) {
return true;
}
})
.sorted((a, b) -> {
try {
return Long.compare(
Files.getLastModifiedTime(b).toMillis(),
Files.getLastModifiedTime(a).toMillis());
} catch (Exception e) {
return 0;
}
})
.limit(RECENT_FILE_MAX_PER_CHAT)
.map(p -> {
String fileName = p.getFileName().toString();
// Strip timestamp prefix (e.g. "1777391026594_report.pdf" "report.pdf")
int sep = fileName.indexOf('_');
String display = (sep > 0 && sep < 20) ? fileName.substring(sep + 1) : fileName;
String contentType = guessContentType(p);
return new RecentFileEntry(display, p.toAbsolutePath().toString(), null, contentType);
})
.toList();
} catch (Exception e) {
// warn (not debug): a failed disk scan silently drops recovered files, which
// reproduces the exact "bot can't see the file" symptom this fallback fixes.
log.warn("[feishu] Failed to scan disk for recent files in {}: {}", dir, e.getMessage(), e);
return List.of();
}
}
/** Best-effort content type from file extension. */
private static String guessContentType(Path p) {
String name = p.getFileName().toString().toLowerCase();
if (name.endsWith(".pdf")) return "application/pdf";
if (name.endsWith(".docx")) return "application/vnd.openxmlformats-officedocument.wordprocessingml.document";
if (name.endsWith(".xlsx")) return "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
if (name.endsWith(".pptx")) return "application/vnd.openxmlformats-officedocument.presentationml.presentation";
if (name.endsWith(".doc")) return "application/msword";
if (name.endsWith(".xls")) return "application/vnd.ms-excel";
if (name.endsWith(".ppt")) return "application/vnd.ms-powerpoint";
if (name.endsWith(".txt")) return "text/plain";
if (name.endsWith(".csv")) return "text/csv";
if (name.endsWith(".json")) return "application/json";
if (name.endsWith(".xml")) return "application/xml";
if (name.endsWith(".md")) return "text/markdown";
if (name.endsWith(".png")) return "image/png";
if (name.endsWith(".jpg") || name.endsWith(".jpeg")) return "image/jpeg";
if (name.endsWith(".gif")) return "image/gif";
if (name.endsWith(".webp")) return "image/webp";
if (name.endsWith(".mp3")) return "audio/mpeg";
if (name.endsWith(".ogg")) return "audio/ogg";
if (name.endsWith(".opus")) return "audio/opus";
if (name.endsWith(".wav")) return "audio/wav";
if (name.endsWith(".mp4")) return "video/mp4";
if (name.endsWith(".mov")) return "video/quicktime";
if (name.endsWith(".zip")) return "application/zip";
if (name.endsWith(".rar")) return "application/x-rar-compressed";
if (name.endsWith(".7z")) return "application/x-7z-compressed";
return "application/octet-stream";
}
// ==================== 消息内容解析 ====================
/**
* 解析飞书消息内容为 contentParts
*
* @param messageId 消息 ID用于媒体下载
* @param messageType 消息类型
* @param contentStr 消息内容 JSON 字符串
* @param parts 输出的 content parts
* @param messageId 消息 ID用于媒体下载
* @param messageType 消息类型
* @param contentStr 消息内容 JSON 字符串
* @param parts 输出的 content parts
* @param stagedUploadPath {@code cacheRecentFile} 复制到 {@code data/chat-uploads/}
* 的绝对路径可空非空时覆盖各附件 part path使其指向
* 沙箱可达 {@code ChatUploadResolver}的那一份而不是
* 沙箱外的 {@code ~/.mateclaw/media/} 路径
* @return 纯文本摘要
*/
@SuppressWarnings("unchecked")
private String extractContentParts(String messageId, String messageType, String contentStr,
List<MessageContentPart> parts) {
List<MessageContentPart> parts, String stagedUploadPath) {
if (contentStr == null) return null;
try {
@ -1588,6 +1928,7 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
DownloadedResource dl = maybeDownloadImage(messageId, imageKey);
MessageContentPart part = MessageContentPart.image(imageKey, null);
applyDownload(part, dl);
if (stagedUploadPath != null) part.setPath(stagedUploadPath);
parts.add(part);
}
yield "[图片]";
@ -1599,6 +1940,7 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
DownloadedResource dl = maybeDownloadResource(messageId, fileKey, "file", fileName);
MessageContentPart part = MessageContentPart.file(fileKey, fileName, null);
applyDownload(part, dl);
if (stagedUploadPath != null) part.setPath(stagedUploadPath);
parts.add(part);
}
yield "[文件: " + (fileName != null ? fileName : "") + "]";
@ -1612,6 +1954,7 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
DownloadedResource dl = maybeDownloadResource(messageId, fileKey, "file", "voice.opus");
MessageContentPart part = MessageContentPart.audio(fileKey, null);
applyDownload(part, dl);
if (stagedUploadPath != null) part.setPath(stagedUploadPath);
// STT hop: inject the transcript as a sibling text part BEFORE
// the audio part so ChannelMessageRouter.buildPromptFromParts
// sees real content instead of just "[音频]". WeCom / DingTalk
@ -1632,6 +1975,7 @@ public class FeishuChannelAdapter extends AbstractChannelAdapter implements Stre
DownloadedResource dl = maybeDownloadResource(messageId, fileKey, "file", fileName);
MessageContentPart part = MessageContentPart.video(fileKey, fileName);
applyDownload(part, dl);
if (stagedUploadPath != null) part.setPath(stagedUploadPath);
parts.add(part);
}
yield "[视频]";

View File

@ -85,6 +85,14 @@ public class ChatController {
// RFC-058 PR-1: Utf8SseEmitter 显式声明 charset=UTF-8防止中文在 Windows 中文 Chrome / 部分代理处乱码
SseEmitter emitter = new Utf8SseEmitter(10 * 60 * 1000L);
// Resolve the public base URL on THIS (request) thread. Every agent run
// below is dispatched to sseExecutor / reactive callbacks that run off
// the request thread, where the request is no longer bound and
// ServletUriComponentsBuilder would yield null. Capturing it here lets
// tool-generated download links carry an absolute host on the streaming,
// approval-replay, and queued-message paths alike.
final String requestBaseUrl = resolveRequestBaseUrl();
// ---- 分支 A断线重连 ----
if (Boolean.TRUE.equals(request.getReconnect())) {
String reconnectUser = auth != null ? auth.getName() : "anonymous";
@ -258,7 +266,7 @@ public class ChatController {
// deny 是正常 turn 终结用户可能在 awaiting_approval 阶段排了消息
ChatStreamTracker.CompletionResult denyCr = streamTracker.completeAndConsumeIfLast(conversationId);
if (denyCr.allDone() && denyCr.queuedInput() != null) {
startQueuedMessage(conversationId, emitter, approvalEmitterDone, denyCr.queuedInput(), username);
startQueuedMessage(conversationId, emitter, approvalEmitterDone, denyCr.queuedInput(), username, requestBaseUrl);
} else {
completeEmitterQuietly(emitter, approvalEmitterDone);
}
@ -272,7 +280,7 @@ public class ChatController {
// 审批记录被另一个请求消费但用户可能在等待期间排了消息
ChatStreamTracker.CompletionResult consumedNullCr = streamTracker.completeAndConsumeIfLast(conversationId);
if (consumedNullCr.allDone() && consumedNullCr.queuedInput() != null) {
startQueuedMessage(conversationId, emitter, approvalEmitterDone, consumedNullCr.queuedInput(), username);
startQueuedMessage(conversationId, emitter, approvalEmitterDone, consumedNullCr.queuedInput(), username, requestBaseUrl);
} else {
completeEmitterQuietly(emitter, approvalEmitterDone);
}
@ -298,6 +306,9 @@ public class ChatController {
replayOrigin = vip.mate.agent.context.ChatOrigin.web(
conversationId, username, workspaceId, null);
}
// Carry the request-thread base URL so any file a replayed
// tool generates gets an absolute download link.
replayOrigin = replayOrigin.withBaseUrl(requestBaseUrl);
Disposable disposable = agentService.chatWithReplayStream(
replayAgentId, replayPrompt, conversationId, finalConsumed.getToolCallPayload(), username, replayOrigin)
.doOnNext(delta -> {
@ -371,7 +382,7 @@ public class ChatController {
ChatStreamTracker.CompletionResult cr = streamTracker.completeAndConsumeIfLast(conversationId);
if (cr.allDone()) {
if (cr.queuedInput() != null) {
startQueuedMessage(conversationId, emitter, approvalEmitterDone, cr.queuedInput(), username);
startQueuedMessage(conversationId, emitter, approvalEmitterDone, cr.queuedInput(), username, requestBaseUrl);
} else {
conversationService.updateStreamStatus(conversationId, "idle");
completeEmitterQuietly(emitter, approvalEmitterDone);
@ -469,7 +480,7 @@ public class ChatController {
ChatStreamTracker.CompletionResult cr = streamTracker.completeAndConsumeIfLast(conversationId);
if (cr.allDone()) {
if (cr.queuedInput() != null) {
startQueuedMessage(conversationId, emitter, approvalEmitterDone, cr.queuedInput(), username);
startQueuedMessage(conversationId, emitter, approvalEmitterDone, cr.queuedInput(), username, requestBaseUrl);
} else {
conversationService.updateStreamStatus(conversationId, "idle");
completeEmitterQuietly(emitter, approvalEmitterDone);
@ -544,7 +555,8 @@ public class ChatController {
// tools that need a workspace path read it from the agent (origin
// is enriched with workspaceBasePath in StateGraph buildInitialState).
vip.mate.agent.context.ChatOrigin webOrigin =
memoryOrigin(conversationId, username, workspaceId, request.getEndUserId());
memoryOrigin(conversationId, username, workspaceId, request.getEndUserId())
.withBaseUrl(requestBaseUrl);
Disposable disposable = agentService.chatStructuredStream(agentId, promptText, conversationId, username, request.getThinkingLevel(), webOrigin)
.doOnNext(delta -> {
if (emitterDone.get()) return;
@ -680,7 +692,7 @@ public class ChatController {
// genuinely doesn't want continuation, no message would
// have been in messageQueue to begin with.
if (cr.queuedInput() != null) {
startQueuedMessage(conversationId, emitter, emitterDone, cr.queuedInput(), username);
startQueuedMessage(conversationId, emitter, emitterDone, cr.queuedInput(), username, requestBaseUrl);
} else {
conversationService.updateStreamStatus(conversationId, "idle");
// 延迟关闭 emitter确保最后的事件都已发送
@ -771,7 +783,7 @@ public class ChatController {
if (cr.allDone()) {
if (cr.queuedInput() != null) {
// 无论中断类型都消费排队消息修复 Disposable 不可用时队列被丢弃的 bug
startQueuedMessage(conversationId, emitter, emitterDone, cr.queuedInput(), username);
startQueuedMessage(conversationId, emitter, emitterDone, cr.queuedInput(), username, requestBaseUrl);
} else {
conversationService.updateStreamStatus(conversationId, "idle");
completeEmitterQuietly(emitter, emitterDone);
@ -891,7 +903,7 @@ public class ChatController {
// just run it. Aligns with doOnComplete and the 4 other
// queue-launch sites in this controller.
if (cr.queuedInput() != null) {
startQueuedMessage(conversationId, emitter, emitterDone, cr.queuedInput(), username);
startQueuedMessage(conversationId, emitter, emitterDone, cr.queuedInput(), username, requestBaseUrl);
} else {
conversationService.updateStreamStatus(conversationId, "idle");
completeEmitterQuietly(emitter, emitterDone);
@ -1143,12 +1155,30 @@ public class ChatController {
*/
private vip.mate.agent.context.ChatOrigin memoryOrigin(String conversationId, String username,
Long workspaceId, String endUserId) {
// Resolve the public base URL here, on the request thread, so it can ride
// the origin into async tool execution where no request is bound. Tools
// then mint absolute download links without operator config.
String baseUrl = resolveRequestBaseUrl();
if (endUserId != null && !endUserId.isBlank()) {
return vip.mate.agent.context.ChatOrigin
.web(conversationId, endUserId.trim(), workspaceId, null)
.web(conversationId, endUserId.trim(), workspaceId, null, baseUrl)
.withSender(null, "api", null);
}
return vip.mate.agent.context.ChatOrigin.web(conversationId, username, workspaceId, null);
return vip.mate.agent.context.ChatOrigin.web(conversationId, username, workspaceId, null, baseUrl);
}
/**
* Resolve {@code scheme://host[:port][/contextPath]} from the current request,
* honouring {@code X-Forwarded-*} when a {@code ForwardedHeaderFilter} is active.
* Returns null off the request thread (caller falls back to config / relative).
*/
private String resolveRequestBaseUrl() {
try {
return org.springframework.web.servlet.support.ServletUriComponentsBuilder
.fromCurrentContextPath().build().toUriString();
} catch (Exception e) {
return null;
}
}
@lombok.Data
@ -1218,7 +1248,8 @@ public class ChatController {
* 支持链式续跑queued stream 自身完成时也通过 completeAndConsumeIfLast 检查并递归调用
*/
private void startQueuedMessage(String conversationId, SseEmitter emitter, AtomicBoolean emitterDone,
ChatStreamTracker.QueuedInput preConsumedInput, String requesterId) {
ChatStreamTracker.QueuedInput preConsumedInput, String requesterId,
String baseUrl) {
if (preConsumedInput == null) {
conversationService.updateStreamStatus(conversationId, "idle");
completeEmitterQuietly(emitter, emitterDone);
@ -1279,7 +1310,8 @@ public class ChatController {
// a web-origin ChatOrigin so any cron job created during the queued
// turn keeps a consistent (null-channel) binding.
vip.mate.agent.context.ChatOrigin queuedOrigin =
vip.mate.agent.context.ChatOrigin.web(conversationId, requesterId, null, null);
vip.mate.agent.context.ChatOrigin.web(conversationId, requesterId, null, null)
.withBaseUrl(baseUrl);
Disposable disposable = agentService.chatStructuredStream(agentId, queuedMessage, conversationId, requesterId, null, queuedOrigin)
.doOnNext(delta -> {
if (emitterDone.get()) return;
@ -1339,7 +1371,7 @@ public class ChatController {
if (cr.allDone()) {
if (cr.queuedInput() != null) {
// 链式续跑queued stream 期间又排了新消息
startQueuedMessage(conversationId, emitter, emitterDone, cr.queuedInput(), requesterId);
startQueuedMessage(conversationId, emitter, emitterDone, cr.queuedInput(), requesterId, baseUrl);
} else {
conversationService.updateStreamStatus(conversationId, "idle");
sseExecutor.execute(() -> {
@ -1381,7 +1413,7 @@ public class ChatController {
ChatStreamTracker.CompletionResult cr = streamTracker.completeAndConsumeIfLast(conversationId);
if (cr.allDone()) {
if (cr.queuedInput() != null) {
startQueuedMessage(conversationId, emitter, emitterDone, cr.queuedInput(), requesterId);
startQueuedMessage(conversationId, emitter, emitterDone, cr.queuedInput(), requesterId, baseUrl);
} else {
conversationService.updateStreamStatus(conversationId, "idle");
completeEmitterQuietly(emitter, emitterDone);

View File

@ -14,7 +14,7 @@ final class SegmentSupersedeDetector {
static final String REASON_TOOL_RESULT_REPLACED_MODEL_CLAIM = "tool_result_replaced_model_claim";
private static final Pattern GENERATED_FILE_URL =
Pattern.compile("/api/v1/files/generated/[A-Za-z0-9-]+");
Pattern.compile("(?:https?://[^/\\s)\\]]+)?/api/v1/files/generated/[A-Za-z0-9-]+");
private static final Pattern BYTE_COUNT =
Pattern.compile("\\d+\\s*字节");
private static final Pattern REPLACEMENT_COUNT =

View File

@ -0,0 +1,104 @@
package vip.mate.channel.webchat;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.security.core.Authentication;
import org.springframework.web.bind.annotation.DeleteMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import vip.mate.audit.service.AuditEventService;
import vip.mate.channel.model.ChannelEntity;
import vip.mate.channel.service.ChannelService;
import vip.mate.common.result.R;
import java.util.Map;
/**
* Admin-facing webchat operations. Mounted under {@code /api/v1/admin/webchat/**}
* (outside the {@code /api/v1/channels/webchat/**} permitAll block) so it
* requires a regular MateClaw JWT visitors cannot reach these endpoints.
*
* <p>Currently only manages visitor-token revocation. Audit-recorded via
* {@link AuditEventService}; actor is the JWT-authenticated admin username,
* not the visitor.
*
* @author MateClaw Team
*/
@Tag(name = "WebChat 管理(管理员)")
@Slf4j
@RestController
@RequestMapping("/api/v1/admin/webchat")
@RequiredArgsConstructor
public class WebChatAdminController {
private final ChannelService channelService;
private final WebChatTokenRevocationService revocationService;
private final AuditEventService auditService;
@Operation(summary = "撤销访客的 visitorToken(ban 该 visitor 在管理端点的所有调用)")
@PostMapping("/revoked-visitor")
public R<Void> revokeVisitor(
@RequestBody RevokeVisitorRequest request,
Authentication auth) {
if (request == null || request.getChannelId() == null
|| request.getVisitorId() == null || request.getVisitorId().isBlank()) {
return R.fail(400, "channelId and visitorId are required");
}
ChannelEntity channel = channelService.getChannel(request.getChannelId());
if (channel == null || !"webchat".equals(channel.getChannelType())) {
return R.fail(404, "webchat channel not found");
}
revocationService.revoke(channel.getId(), request.getVisitorId().trim(),
request.getReason());
String adminUser = auth != null ? auth.getName() : "system";
auditService.record(
"webchat.revoke-visitor",
"CHANNEL",
String.valueOf(channel.getId()),
channel.getName(),
"{\"visitorId\":\"" + request.getVisitorId().trim()
+ "\",\"reason\":\"" + (request.getReason() != null ? request.getReason() : "")
+ "\",\"admin\":\"" + adminUser + "\"}",
channel.getWorkspaceId());
return R.ok();
}
@Operation(summary = "取消撤销访客(un-ban)")
@DeleteMapping("/revoked-visitor")
public R<Void> unrevokeVisitor(
@RequestBody Map<String, Object> body,
Authentication auth) {
Long channelId = body.get("channelId") instanceof Number n ? n.longValue() : null;
Object rawChannelId = body.get("channelId");
if (rawChannelId instanceof String s && !s.isBlank()) {
try { channelId = Long.parseLong(s); } catch (NumberFormatException ignored) { }
}
String visitorId = body.get("visitorId") instanceof String s ? s.trim() : null;
if (channelId == null || visitorId == null || visitorId.isEmpty()) {
return R.fail(400, "channelId and visitorId are required");
}
revocationService.unrevoke(channelId, visitorId);
String adminUser = auth != null ? auth.getName() : "system";
auditService.record(
"webchat.unrevoke-visitor",
"CHANNEL",
String.valueOf(channelId),
null,
"{\"visitorId\":\"" + visitorId + "\",\"admin\":\"" + adminUser + "\"}",
null);
return R.ok();
}
@lombok.Data
public static class RevokeVisitorRequest {
private Long channelId;
private String visitorId;
private String reason;
}
}

View File

@ -0,0 +1,52 @@
package vip.mate.channel.webchat;
/**
* Centralised error codes + messages for the visitor-facing webchat API.
* <p>
* Every {@code R.fail(...)} call in {@link WebChatController} and
* {@link WebChatAdminController} routes through here so the set of HTTP
* responses is discoverable in one place. Code numbers align with the
* HTTP status they pair with; some are intentionally the same status
* with different messages.
*
* @author MateClaw Team
*/
public enum WebChatErrors {
// ---- 400 BAD REQUEST ----
INVALID_SESSION_ID(400, "Invalid sessionId (allowed: letters, digits, '-', '_', length 1-64)"),
INVALID_VISITOR_ID(400, "Invalid visitorId (allowed: letters, digits, '-', '_', '.', ':', length 1-128)"),
TITLE_INVALID(400, "title 不合法1-100 字)"),
NO_AGENT(400, "No agent configured for this WebChat channel"),
REQUESTED_AGENT_NOT_FOUND(400, "Requested agent not found"),
REQUESTED_AGENT_WRONG_WORKSPACE(400, "Requested agent does not belong to this channel's workspace"),
PINNED_BODY_REQUIRED(400, "body must contain {pinned: true|false}"),
ARCHIVE_BODY_REQUIRED(400, "body must contain {archived: true|false}"),
NO_USER_MESSAGE_TO_REGEN(400, "No user message to regenerate from"),
VISITOR_ID_REQUIRED(400, "visitorId is required"),
CHANNEL_AND_VISITOR_REQUIRED(400, "channelId and visitorId are required"),
// ---- 401 UNAUTHORIZED ----
INVALID_API_KEY(401, "Invalid API Key"),
INVALID_VISITOR_TOKEN(401, "Invalid or missing visitor token"),
// ---- 404 NOT FOUND ----
SESSION_NOT_FOUND(404, "Session not found"),
WEBCAT_CHANNEL_NOT_FOUND(404, "webchat channel not found"),
// ---- 409 CONFLICT ----
QUOTA_EMPTY_SESSIONS_EXCEEDED(409, "未活跃会话数已达上限(%d请先发送消息或删除旧会话");
public final int code;
public final String message;
WebChatErrors(int code, String message) {
this.code = code;
this.message = message;
}
/** Apply an int substitution to messages using {@code %d}. */
public String with(int arg) {
return String.format(message, arg);
}
}

View File

@ -0,0 +1,255 @@
package vip.mate.channel.webchat;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Service;
import org.springframework.web.multipart.MultipartFile;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.Locale;
import java.util.Optional;
import java.util.Set;
import java.util.UUID;
import java.util.concurrent.ConcurrentHashMap;
import java.util.stream.Collectors;
import java.util.stream.Stream;
/**
* Storage + validation for files exchanged over the WebChat channel.
*
* <p>WebChat is reached by untrusted external visitors (API key + visitor
* token, no JWT), so uploads are hardened here: size cap, extension allow-list,
* filename sanitization, and a server-issued stored name. Every path is derived
* from the server-computed {@code conversationId} never from a client-supplied
* path and download resolution is traversal-guarded.
*
* <p>Uploads are staged in an in-memory registry keyed by an opaque file id.
* Only when the visitor references that id on the next {@code /stream} call does
* the file become a real conversation attachment (the bytes already live under
* the conversation's upload dir, so cleanup rides the existing
* {@code cleanAttachmentFiles} cascade). Unreferenced staged files are swept
* after {@link #STAGING_TTL_MS}.
*/
@Slf4j
@Service
public class WebChatFileService {
/** Shared with the JWT chat upload dir so deleteConversation cleanup applies. */
private static final Path UPLOAD_ROOT = Paths.get("data", "chat-uploads");
/** How long an uploaded-but-unreferenced file lingers before the sweep removes it. */
private static final long STAGING_TTL_MS = 60 * 60 * 1000L; // 1 hour
private final boolean enabled;
private final long maxSizeBytes;
private final Set<String> allowedExtensions;
private final int maxFilesPerConversation;
private final long maxTotalBytesPerConversation;
/** fileId (== storedName) -> staged metadata, pending a /stream reference. */
private final ConcurrentHashMap<String, StagedFile> staged = new ConcurrentHashMap<>();
public WebChatFileService(
@Value("${mateclaw.webchat.upload.enabled:true}") boolean enabled,
@Value("${mateclaw.webchat.upload.max-size-mb:20}") long maxSizeMb,
@Value("${mateclaw.webchat.upload.allowed-extensions:"
+ "png,jpg,jpeg,gif,webp,bmp,pdf,txt,md,csv,json,log,"
+ "doc,docx,xls,xlsx,ppt,pptx,zip,mp3,wav,m4a,mp4,mov,webm}") String allowedExtensionsCsv,
@Value("${mateclaw.webchat.upload.max-files-per-conversation:50}") int maxFilesPerConversation,
@Value("${mateclaw.webchat.upload.max-total-mb-per-conversation:200}") long maxTotalMbPerConversation) {
this.enabled = enabled;
this.maxSizeBytes = maxSizeMb * 1024 * 1024;
this.allowedExtensions = Arrays.stream(allowedExtensionsCsv.split(","))
.map(s -> s.trim().toLowerCase(Locale.ROOT))
.filter(s -> !s.isEmpty())
.collect(Collectors.toUnmodifiableSet());
this.maxFilesPerConversation = maxFilesPerConversation;
this.maxTotalBytesPerConversation = maxTotalMbPerConversation * 1024 * 1024;
}
/** Metadata for a staged upload. */
public record StagedFile(String conversationId, String storedName, String originalName,
String contentType, long size, long expireAt) {
boolean expired() {
return System.currentTimeMillis() > expireAt;
}
}
/** Thrown on any validation failure; the controller maps it to a 4xx. */
public static class UploadRejectedException extends RuntimeException {
public UploadRejectedException(String message) {
super(message);
}
}
public boolean isEnabled() {
return enabled;
}
/**
* Validate and store an uploaded file under the conversation's upload dir,
* returning a staged record whose {@code storedName} doubles as the opaque
* file id the visitor references on the next /stream call.
*
* @param conversationId server-derived conversation id (never client-supplied)
*/
public StagedFile store(String conversationId, MultipartFile file) throws IOException {
if (!enabled) {
throw new UploadRejectedException("WebChat file upload is disabled");
}
if (file == null || file.isEmpty()) {
throw new UploadRejectedException("Empty file");
}
if (file.getSize() > maxSizeBytes) {
throw new UploadRejectedException("File too large (max " + (maxSizeBytes / 1024 / 1024) + " MB)");
}
String originalName = file.getOriginalFilename() != null ? file.getOriginalFilename() : "file";
// Strip any directory components, then collapse to a safe charset.
String baseName = Paths.get(originalName).getFileName().toString();
String ext = extensionOf(baseName);
if (ext.isEmpty() || !allowedExtensions.contains(ext)) {
throw new UploadRejectedException("File type not allowed: ." + ext);
}
String safeName = baseName.replaceAll("[^a-zA-Z0-9._-]", "_");
String storedName = UUID.randomUUID() + "_" + safeName;
Path dir = UPLOAD_ROOT.resolve(conversationId).normalize();
if (!dir.startsWith(UPLOAD_ROOT.normalize())) {
// conversationId is server-derived, so this should never happen; fail closed if it does.
throw new UploadRejectedException("Invalid conversation");
}
Files.createDirectories(dir);
enforceConversationQuota(dir, file.getSize());
Path target = dir.resolve(storedName);
file.transferTo(target.toAbsolutePath());
String contentType = Optional.ofNullable(file.getContentType())
.filter(ct -> !ct.isBlank())
.orElseGet(() -> probe(target));
StagedFile entry = new StagedFile(conversationId, storedName, baseName, contentType,
file.getSize(), System.currentTimeMillis() + STAGING_TTL_MS);
staged.put(storedName, entry);
log.info("[webchat-file] Stored upload conv={} stored={} type={} size={}",
conversationId, storedName, contentType, file.getSize());
return entry;
}
/**
* Resolve a staged file id into its metadata, asserting it belongs to this
* conversation and has not expired. Consuming it removes the staging entry
* (the bytes remain as a committed conversation attachment). Returns empty
* if the id is unknown, expired, or belongs to another conversation.
*/
public Optional<StagedFile> consume(String conversationId, String fileId) {
if (fileId == null) {
return Optional.empty();
}
StagedFile entry = staged.get(fileId);
if (entry == null || entry.expired() || !entry.conversationId().equals(conversationId)) {
return Optional.empty();
}
staged.remove(fileId);
return Optional.of(entry);
}
/**
* Traversal-safe resolution of a stored file under the conversation's dir.
* Both the dir and the final path are derived from the server-computed
* conversationId; the client-supplied {@code storedName} is confined by the
* {@code startsWith} guard. Returns empty if missing or escaping the dir.
*/
public Optional<Path> resolve(String conversationId, String storedName) {
if (storedName == null || storedName.isBlank()) {
return Optional.empty();
}
Path base = UPLOAD_ROOT.resolve(conversationId).normalize();
Path file = base.resolve(storedName).normalize();
if (!file.startsWith(base) || !Files.exists(file) || !Files.isRegularFile(file)) {
return Optional.empty();
}
return Optional.of(file);
}
/** Map a content type to the MessageContentPart type the agent/UI understands. */
public static String partTypeFor(String contentType) {
if (contentType == null) {
return "file";
}
String ct = contentType.toLowerCase(Locale.ROOT);
if (ct.startsWith("image/")) return "image";
if (ct.startsWith("video/")) return "video";
if (ct.startsWith("audio/")) return "audio";
return "file";
}
/** Periodically drop staged files the visitor never referenced. */
@Scheduled(fixedDelay = 15 * 60 * 1000L)
public void sweepExpired() {
staged.values().removeIf(entry -> {
if (!entry.expired()) {
return false;
}
resolve(entry.conversationId(), entry.storedName()).ifPresent(p -> {
try {
Files.deleteIfExists(p);
} catch (IOException e) {
log.warn("[webchat-file] Failed to delete expired staged file {}: {}",
p, e.getMessage());
}
});
return true;
});
}
/**
* Bound a conversation's disk footprint: reject when the dir already holds
* the max file count, or when adding {@code incomingSize} would push the
* total over the cap. Cheap dir scan (these dirs hold at most a few dozen
* files); pairs with the staging TTL sweep that reclaims unreferenced files.
*/
private void enforceConversationQuota(Path dir, long incomingSize) throws IOException {
int count = 0;
long total = 0;
try (Stream<Path> files = Files.list(dir)) {
for (Path p : (Iterable<Path>) files::iterator) {
if (Files.isRegularFile(p)) {
count++;
total += Files.size(p);
}
}
}
if (count >= maxFilesPerConversation) {
throw new UploadRejectedException(
"Too many files in this conversation (max " + maxFilesPerConversation + ")");
}
if (total + incomingSize > maxTotalBytesPerConversation) {
throw new UploadRejectedException(
"Conversation upload quota exceeded (max "
+ (maxTotalBytesPerConversation / 1024 / 1024) + " MB)");
}
}
private static String extensionOf(String name) {
int dot = name.lastIndexOf('.');
if (dot < 0 || dot == name.length() - 1) {
return "";
}
return name.substring(dot + 1).toLowerCase(Locale.ROOT);
}
private static String probe(Path path) {
try {
String ct = Files.probeContentType(path);
return ct != null ? ct : "application/octet-stream";
} catch (IOException e) {
return "application/octet-stream";
}
}
}

View File

@ -0,0 +1,46 @@
package vip.mate.channel.webchat;
import com.baomidou.mybatisplus.annotation.IdType;
import com.baomidou.mybatisplus.annotation.TableId;
import com.baomidou.mybatisplus.annotation.TableName;
import lombok.Data;
import java.time.LocalDateTime;
/**
* Persistent registry of visitors whose {@code visitorToken} HMAC is no longer
* accepted on management endpoints (list/messages/title/delete/stop/upload/
* regenerate). Created by V148. The unique constraint on
* {@code (channel_id, visitor_id, deleted)} makes re-revoke idempotent;
* setting {@code deleted = 1} un-revokes.
* <p>
* {@code POST /stream} is intentionally NOT bound by this a revoked visitor
* can still start a fresh {@code /stream}, which mints a new token; the
* revocation applies to the old token presented on management endpoints.
*
* @author MateClaw Team
*/
@Data
@TableName("webchat_revoked_visitor")
public class WebChatRevokedVisitorEntity {
@TableId(type = IdType.ASSIGN_ID)
private Long id;
private Long channelId;
/** Visitor identifier in the same charset as {@code WebChatController.normalizeVisitorId}. */
private String visitorId;
private LocalDateTime revokedAt;
/** Free-form reason (admin-supplied). Nullable. */
private String reason;
private LocalDateTime createTime;
private LocalDateTime updateTime;
/** 0 = active revocation; 1 = un-revoked (tombstoned). */
private Integer deleted;
}

View File

@ -0,0 +1,134 @@
package vip.mate.channel.webchat;
import com.baomidou.mybatisplus.core.conditions.query.LambdaQueryWrapper;
import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.stereotype.Service;
import vip.mate.channel.webchat.repository.WebChatRevokedVisitorMapper;
import java.time.Duration;
import java.time.LocalDateTime;
/**
* Visitor-token revocation lookup with a process-local Caffeine cache in front
* of the {@code webchat_revoked_visitor} table.
*
* <p>The cache is best-effort: a revoked visitor may take up to
* {@link #CACHE_TTL} to become effectively revoked on a node that has the
* un-revoked entry cached. We accept that window webchat is low-volume
* rather than pay a DB round-trip on every management endpoint call. For
* multi-instance deployments the same eventual-consistency applies
* independently per node; the DB remains the source of truth and a fresh
* node sees revocations immediately on cold cache.
*
* <p>All operations are idempotent: revoking an already-revoked visitor is a
* no-op (the row's {@code revokedAt} is updated for record-keeping);
* un-revoking an un-revoked one is also a no-op.
*
* @author MateClaw Team
*/
@Slf4j
@Service
@RequiredArgsConstructor
public class WebChatTokenRevocationService {
static final Duration CACHE_TTL = Duration.ofMinutes(5);
private static final int CACHE_MAX_SIZE = 10_000;
private final WebChatRevokedVisitorMapper revokedVisitorMapper;
/**
* Keyed by "{channelId}:{visitorId}". Value is true when an active
* revocation row exists, false otherwise. absence means "not cached";
* callers must treat null as "fall through to DB".
*/
private final Cache<String, Boolean> revocationCache = Caffeine.newBuilder()
.expireAfterWrite(CACHE_TTL)
.maximumSize(CACHE_MAX_SIZE)
.build();
/**
* True if this visitor is currently revoked on this channel. Pads the
* cache miss with a single DB lookup; the result is then cached for
* {@link #CACHE_TTL}.
*/
public boolean isRevoked(Long channelId, String visitorId) {
if (channelId == null || visitorId == null) {
return false;
}
String key = channelId + ":" + visitorId;
Boolean cached = revocationCache.getIfPresent(key);
if (cached != null) {
return cached;
}
boolean revoked = lookupRevoked(channelId, visitorId);
revocationCache.put(key, revoked);
return revoked;
}
/**
* Record a revocation. Idempotent: re-revoking an already-revoked visitor
* refreshes {@code revokedAt} + {@code reason} on the existing row.
* Flushes the cache so the change is visible immediately on this node.
*/
public void revoke(Long channelId, String visitorId, String reason) {
WebChatRevokedVisitorEntity existing = findActive(channelId, visitorId);
LocalDateTime now = LocalDateTime.now();
if (existing == null) {
WebChatRevokedVisitorEntity row = new WebChatRevokedVisitorEntity();
row.setChannelId(channelId);
row.setVisitorId(visitorId);
row.setReason(reason);
row.setRevokedAt(now);
row.setCreateTime(now);
row.setUpdateTime(now);
row.setDeleted(0);
revokedVisitorMapper.insert(row);
} else {
existing.setReason(reason);
existing.setRevokedAt(now);
existing.setUpdateTime(now);
revokedVisitorMapper.updateById(existing);
}
revocationCache.put(channelId + ":" + visitorId, true);
log.info("[WebChat] visitor revoked: channelId={}, visitorId={}, reason={}",
channelId, visitorId, reason);
}
/**
* Lift a revocation. Idempotent. Removes the cache entry so subsequent
* {@link #isRevoked} calls re-query the DB (and find nothing).
*/
public void unrevoke(Long channelId, String visitorId) {
WebChatRevokedVisitorEntity existing = findActive(channelId, visitorId);
if (existing != null) {
existing.setDeleted(1);
existing.setUpdateTime(LocalDateTime.now());
revokedVisitorMapper.updateById(existing);
}
// Invalidate rather than put(false): the un-revoke may race with a
// concurrent revoke on another node. Forcing a DB re-lookup is safer.
revocationCache.invalidate(channelId + ":" + visitorId);
log.info("[WebChat] visitor un-revoked: channelId={}, visitorId={}",
channelId, visitorId);
}
private boolean lookupRevoked(Long channelId, String visitorId) {
return findActive(channelId, visitorId) != null;
}
private WebChatRevokedVisitorEntity findActive(Long channelId, String visitorId) {
return revokedVisitorMapper.selectOne(new LambdaQueryWrapper<WebChatRevokedVisitorEntity>()
.eq(WebChatRevokedVisitorEntity::getChannelId, channelId)
.eq(WebChatRevokedVisitorEntity::getVisitorId, visitorId)
.eq(WebChatRevokedVisitorEntity::getDeleted, 0)
.last("LIMIT 1"));
}
/** Test-only: drop every cached entry so the next isRevoked() falls through to DB. */
void invalidateCacheForTest() {
revocationCache.invalidateAll();
}
}

View File

@ -0,0 +1,13 @@
package vip.mate.channel.webchat.repository;
import com.baomidou.mybatisplus.core.mapper.BaseMapper;
import org.apache.ibatis.annotations.Mapper;
import vip.mate.channel.webchat.WebChatRevokedVisitorEntity;
/**
* Mapper for {@link WebChatRevokedVisitorEntity}. Lives under {@code repository}
* so the application-wide {@code @MapperScan("vip.mate.**.repository")} picks it up.
*/
@Mapper
public interface WebChatRevokedVisitorMapper extends BaseMapper<WebChatRevokedVisitorEntity> {
}

View File

@ -1502,7 +1502,7 @@ public class WeComChannelAdapter extends AbstractChannelAdapter {
* each adapter rewrites the URL to a channel-native attachment.
*/
private static final java.util.regex.Pattern GENERATED_URL_PATTERN =
java.util.regex.Pattern.compile("/api/v1/files/generated/([a-zA-Z0-9-]+)");
java.util.regex.Pattern.compile("(?:https?://[^/\\s)\\]]+)?/api/v1/files/generated/([a-zA-Z0-9-]+)");
/**
* Scan the agent's text for {@code /api/v1/files/generated/{id}} URLs;
@ -2870,6 +2870,16 @@ public class WeComChannelAdapter extends AbstractChannelAdapter {
// Fallback: download disabled or failed. Browser preview will be broken
// because the WeCom CDN URL carries a short-lived signature, but at
// least the bubble shows "image.jpg" instead of "未命名 / unknown".
// The image is also unusable by the model: the URL points at AES-encrypted
// bytes (when aeskey is present) and expires in ~5 minutes, so without a
// local download the agent can only report "file not found". Warn loudly so
// operators know to enable media download rather than chase a phantom bug.
boolean encrypted = aesKey != null && !aesKey.isBlank();
log.warn("[wecom] Inbound image '{}' stored URL-only ({} download). The model "
+ "cannot read it — enable 'media_download_enabled' on this channel. "
+ "url={}", fileNameHint,
getConfigBoolean("media_download_enabled", true) ? "failed" : "disabled",
encrypted ? "<encrypted COS URL>" : url);
MessageContentPart part = new MessageContentPart();
part.setType("image");
part.setFileName(fileNameHint);

View File

@ -0,0 +1,293 @@
package vip.mate.common.text;
import java.util.ArrayList;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
/**
* 确定性 Markdown 规范化工具
*
* <p>用于在 Agent 最终答案落库 / 发渠道前修复 LLM 原始输出中常见的机械排版缺陷纯本地正则处理
* 不调用任何模型 token设计目标是修畸形而不改语义因此遵循以下原则</p>
*
* <ul>
* <li><b>代码块感知</b>先按 ``` / ~~~ 围栏切分围栏内的内容原样保留避免破坏代码里的
* {@code #} / {@code |} / {@code ---}</li>
* <li><b>幂等</b>{@code normalize(normalize(x)).equals(normalize(x))}</li>
* <li><b>保守</b>只在能高置信判断为畸形时才改写散文中的散落管道符行内 {@code #} 不动</li>
* </ul>
*
* <p>覆盖的修复ATX 标题补空格{@code ---} 与后续内容粘连时拆行含行首与 mid-line 后接标题两种
* 表格块单元格与分隔行对齐标题与表格粘连时拆行标题/表格块边界补空行</p>
*
* <p>不在范围内属语义判断正则无法安全自动化保留在提示词约束Emoji 位置代码块语言标注补全</p>
*/
public final class MarkdownNormalizer {
private MarkdownNormalizer() {}
/** 行首 ATX 标题但紧跟非空格、非 # 字符(缺少标题空格)。 */
private static final Pattern HEADING_NO_SPACE = Pattern.compile("^(#{1,6})([^#\\s].*)$");
/** 已规范的标题行:#{1,6} + 空白。用于块边界判断。 */
private static final Pattern HEADING_LINE = Pattern.compile("^#{1,6}\\s.*$");
/** 主题分隔线与后续内容粘连:行首 3+ 短横,后面紧跟非短横的可见内容。 */
private static final Pattern HR_GLUED = Pattern.compile("^(-{3,})([^-\\s].*)$");
/**
* 主题分隔线粘连在行内容之后mid-line且后面紧跟一个 ATX 标题
* 形如 {@code *来源2026-06-01*---### } {@code **90%**---### 综合判断}
* 仅在 {@code ---} 后紧跟 {@code #} 标题时才拆避免误伤散文里 em-dash 风格的 {@code ---}
*/
private static final Pattern HR_MID_HEADING =
Pattern.compile("^(.*?\\S)\\s*(-{3,})\\s*(#{1,6}.*)$");
/** 标题行尾粘连了表格:#{1,6} 标题文字(不含管道符)+ 管道符起始的尾部。 */
private static final Pattern HEADING_TABLE = Pattern.compile("^(#{1,6}[^|\\n]*?)\\s*(\\|.+)$");
/** GFM 表格分隔行:由短横/冒号组成的单元格,用管道符分隔(必须同时含 - 与 |)。 */
private static final Pattern SEPARATOR_ROW =
Pattern.compile("^\\s*\\|?\\s*:?-{1,}:?\\s*(\\|\\s*:?-{1,}:?\\s*)*\\|?\\s*$");
/**
* 规范化 Markdown 文本{@code null} 或空串原样返回
*/
public static String normalize(String md) {
if (md == null || md.isEmpty()) {
return md;
}
String normalized = md.replace("\r\n", "\n").replace("\r", "\n");
String[] lines = normalized.split("\n", -1);
List<String> out = new ArrayList<>();
List<String> textBuf = new ArrayList<>();
boolean inFence = false;
String fenceMarker = null;
for (String line : lines) {
String lead = line.stripLeading();
if (!inFence && (lead.startsWith("```") || lead.startsWith("~~~"))) {
flushText(textBuf, out);
textBuf.clear();
inFence = true;
fenceMarker = lead.startsWith("```") ? "```" : "~~~";
out.add(line);
} else if (inFence && lead.startsWith(fenceMarker)) {
inFence = false;
fenceMarker = null;
out.add(line);
} else if (inFence) {
out.add(line);
} else {
textBuf.add(line);
}
}
flushText(textBuf, out);
String result = String.join("\n", out);
// 仅清理文档首尾多余空行不触碰代码块内部
return result.replaceAll("^\\n+", "").replaceAll("\\n+$", "");
}
// ==================== 非代码段处理 ====================
private static void flushText(List<String> textLines, List<String> out) {
if (textLines.isEmpty()) {
return;
}
// 1. 行级展开HR 粘连拆行标题粘表格拆行标题补空格
List<String> expanded = new ArrayList<>();
for (String l : textLines) {
expanded.addAll(expandLine(l));
}
// 2. 表格块识别与规范化
List<String> normalizedLines = new ArrayList<>();
List<Boolean> isTable = new ArrayList<>();
normalizeTables(expanded, normalizedLines, isTable);
// 3. 标题/表格块边界补空行 + 折叠多余空行
out.addAll(insertBoundaryBlanks(normalizedLines, isTable));
}
private static List<String> expandLine(String line) {
List<String> result = new ArrayList<>();
Matcher hrMid = HR_MID_HEADING.matcher(line);
if (hrMid.matches()) {
result.addAll(expandLine(hrMid.group(1)));
result.add("");
result.add("---");
result.add("");
result.addAll(expandLine(hrMid.group(3)));
return result;
}
Matcher hr = HR_GLUED.matcher(line);
if (hr.matches()) {
result.add("---");
result.add("");
result.addAll(expandLine(hr.group(2)));
return result;
}
Matcher ht = HEADING_TABLE.matcher(line);
if (ht.matches() && countPipes(ht.group(2)) >= 2) {
result.add(fixHeadingSpace(ht.group(1).strip()));
result.add("");
result.add(ht.group(2).strip());
return result;
}
result.add(fixHeadingSpace(line));
return result;
}
/**
* 行首 ATX 标题缺空格时补一个空格为避免误伤 {@code #5}{@code #1} 这类引用
* 紧跟数字的不处理
*/
private static String fixHeadingSpace(String line) {
Matcher m = HEADING_NO_SPACE.matcher(line);
if (m.matches()) {
String hashes = m.group(1);
String rest = m.group(2);
if (!Character.isDigit(rest.charAt(0))) {
return hashes + " " + rest;
}
}
return line;
}
// ==================== 表格规范化 ====================
private static void normalizeTables(List<String> lines, List<String> out, List<Boolean> isTable) {
int n = lines.size();
boolean[] tbl = new boolean[n];
for (int i = 0; i < n; i++) {
if (isSeparatorRow(lines.get(i)) && i > 0 && containsPipe(lines.get(i - 1))) {
int start = i - 1;
int end = i;
int j = i + 1;
while (j < n && !lines.get(j).isBlank() && containsPipe(lines.get(j))
&& !HEADING_LINE.matcher(lines.get(j)).matches()) {
end = j;
j++;
}
for (int k = start; k <= end; k++) {
tbl[k] = true;
}
i = end;
}
}
for (int i = 0; i < n; i++) {
if (tbl[i]) {
out.add(normalizeTableRow(lines.get(i), isSeparatorRow(lines.get(i))));
} else {
out.add(lines.get(i));
}
isTable.add(tbl[i]);
}
}
private static String normalizeTableRow(String line, boolean separator) {
String s = line.strip();
if (s.startsWith("|")) {
s = s.substring(1);
}
if (s.endsWith("|")) {
s = s.substring(0, s.length() - 1);
}
String[] cells = s.split("(?<!\\\\)\\|", -1);
StringBuilder sb = new StringBuilder("|");
for (String cell : cells) {
String c = cell.strip();
if (separator) {
c = normalizeDelimiterCell(c);
}
sb.append(' ').append(c).append(" |");
}
return sb.toString();
}
private static String normalizeDelimiterCell(String cell) {
boolean left = cell.startsWith(":");
boolean right = cell.endsWith(":");
return (left ? ":" : "") + "---" + (right ? ":" : "");
}
private static boolean isSeparatorRow(String line) {
return line.indexOf('-') >= 0 && line.indexOf('|') >= 0
&& SEPARATOR_ROW.matcher(line).matches();
}
private static boolean containsPipe(String line) {
return line.indexOf('|') >= 0;
}
private static int countPipes(String s) {
int count = 0;
for (int i = 0; i < s.length(); i++) {
if (s.charAt(i) == '|') {
count++;
}
}
return count;
}
// ==================== 块边界空行 ====================
private static final int BLANK = 0;
private static final int HEADING = 1;
private static final int TABLE = 2;
private static final int OTHER = 3;
private static final int NONE = -1;
private static List<String> insertBoundaryBlanks(List<String> lines, List<Boolean> isTable) {
List<String> res = new ArrayList<>();
int lastType = NONE;
for (int i = 0; i < lines.size(); i++) {
String cur = lines.get(i);
int curType = classify(cur, isTable.get(i));
if (curType == BLANK) {
if (lastType == BLANK || lastType == NONE) {
continue; // 折叠连续空行 / 去掉前导空行
}
res.add(cur);
lastType = BLANK;
continue;
}
if (lastType != NONE && lastType != BLANK && needsBlankBetween(lastType, curType)) {
res.add("");
}
res.add(cur);
lastType = curType;
}
return res;
}
private static boolean needsBlankBetween(int prev, int cur) {
if (cur == HEADING || prev == HEADING) {
return true;
}
if (cur == TABLE && prev != TABLE) {
return true;
}
return prev == TABLE && cur != TABLE;
}
private static int classify(String line, boolean tableFlag) {
if (line.isBlank()) {
return BLANK;
}
if (tableFlag) {
return TABLE;
}
if (HEADING_LINE.matcher(line).matches()) {
return HEADING;
}
return OTHER;
}
}

View File

@ -14,6 +14,12 @@ import java.util.concurrent.Executor;
* The inner delegate uses virtual threads (JDK 21); the outer
* {@link DelegatingSecurityContextTaskExecutor} wrapper propagates the caller's
* SecurityContext (JWT identity, audit permissions) to every @Async invocation.
* <p>
* A concurrency limit is set to prevent runaway virtual-thread creation
* from exhausting the HikariCP pool (typical pattern: a stampede of
* {@code @Async} tasks all trying to acquire DB connections at the same
* minute boundary). Excess tasks are rejected immediately so the
* scheduler threads never block on submission.
*
* @author MateClaw Team
*/
@ -21,6 +27,13 @@ import java.util.concurrent.Executor;
@EnableAsync
public class AsyncSecurityConfig implements AsyncConfigurer {
/**
* Cap in-flight async tasks. Well below HikariCP maximum-pool-size (30)
* so async tasks never saturate the pool on their own non-async paths
* (HTTP requests, SSE, channel adapters) always have headroom.
*/
private static final int ASYNC_CONCURRENCY_LIMIT = 24;
@Override
public Executor getAsyncExecutor() {
// Keep DelegatingSecurityContextTaskExecutor so SecurityContext
@ -28,6 +41,7 @@ public class AsyncSecurityConfig implements AsyncConfigurer {
// Replace the inner platform-thread pool with a virtual-thread executor.
var delegate = new SimpleAsyncTaskExecutorBuilder()
.virtualThreads(true)
.concurrencyLimit(ASYNC_CONCURRENCY_LIMIT)
.threadNamePrefix("async-vt-")
.build();
return new DelegatingSecurityContextTaskExecutor(delegate);

View File

@ -38,9 +38,18 @@ public class DatabaseBootstrapRunner implements ApplicationRunner {
private final DataSource dataSource;
private final JdbcTemplate jdbcTemplate;
/** Cached flag: true when running on MySQL/MariaDB, false for H2. */
/** Cached flag: true when running on MySQL/MariaDB, false for H2/Kingbase. */
private volatile Boolean isMySQL;
/** Cached flag: true when running on KingbaseES. */
private volatile Boolean isKingbase;
/** Cached flag: true when running on PostgreSQL. */
private volatile Boolean isPostgres;
/** Cached human-readable label of the connected database, e.g. "MySQL" / "H2" / "PostgreSQL". */
private volatile String databaseLabel;
/**
* When true, wait for Desktop splash screen to call /setup/init with chosen language.
* When false (default), auto-initialize immediately on startup.
@ -111,6 +120,10 @@ public class DatabaseBootstrapRunner implements ApplicationRunner {
String scriptName;
if (isMySQL()) {
scriptName = "en-US".equals(locale) ? "db/data-mysql-en.sql" : "db/data-mysql-zh.sql";
} else if (isKingbase() || isPostgres()) {
// PostgreSQL-family seed (covers both PostgreSQL and KingbaseES,
// which share the same ON CONFLICT / SERIAL-free DDL dialect).
scriptName = "en-US".equals(locale) ? "db/data-kingbase-en.sql" : "db/data-kingbase-zh.sql";
} else {
scriptName = "en-US".equals(locale) ? "db/data-en.sql" : "db/data-zh.sql";
}
@ -140,6 +153,57 @@ public class DatabaseBootstrapRunner implements ApplicationRunner {
}
}
/**
* Friendly product name of the currently connected database, e.g. {@code "MySQL"},
* {@code "PostgreSQL"}, {@code "H2"} or {@code "KingbaseES"}. Read once from JDBC
* metadata and cached the connected database never changes at runtime.
*
* @return the product name, or {@code "Unknown"} if metadata is unavailable.
*/
public String getDatabaseLabel() {
if (databaseLabel == null) {
try (Connection connection = dataSource.getConnection()) {
databaseLabel = normalizeDatabaseLabel(connection.getMetaData().getDatabaseProductName());
} catch (Exception e) {
log.debug("Failed to read database product name: {}", e.getMessage());
databaseLabel = "Unknown";
}
}
return databaseLabel;
}
/**
* Maps a raw JDBC product name to a clean, canonical label. Some drivers append
* version noise to the product name (e.g. KingbaseES reports "KingbaseES V008R006");
* collapsing on a keyword keeps the displayed label stable across driver versions
* and consistent with the dialect this runner detects for DDL.
*
* @return a canonical label, or {@code "Unknown"} when the product name is absent.
*/
static String normalizeDatabaseLabel(String product) {
if (product == null || product.isBlank()) {
return "Unknown";
}
String lower = product.toLowerCase();
if (lower.contains("kingbase")) {
// KingbaseES is the product name; show the vendor's Chinese brand name.
return "人大金仓";
}
if (lower.contains("mariadb")) {
return "MariaDB";
}
if (lower.contains("mysql")) {
return "MySQL";
}
if (lower.contains("postgresql")) {
return "PostgreSQL";
}
if (lower.contains("h2")) {
return "H2";
}
return product.trim();
}
private boolean tableExists(String tableName) throws Exception {
try (Connection connection = dataSource.getConnection()) {
DatabaseMetaData metaData = connection.getMetaData();
@ -159,15 +223,43 @@ public class DatabaseBootstrapRunner implements ApplicationRunner {
try (Connection connection = dataSource.getConnection()) {
String dbProduct = connection.getMetaData().getDatabaseProductName().toLowerCase();
isMySQL = dbProduct.contains("mysql") || dbProduct.contains("mariadb");
log.info("Detected database: {} (MySQL mode: {})", dbProduct, isMySQL);
isKingbase = dbProduct.contains("kingbase");
// KingbaseES reports its own product name ("KingbaseES"), so the
// postgres check stays mutually exclusive with the kingbase one.
isPostgres = dbProduct.contains("postgresql") && !isKingbase;
if (isKingbase) {
log.info("Detected database: {} (KingbaseES mode)", dbProduct);
} else if (isPostgres) {
log.info("Detected database: {} (PostgreSQL mode)", dbProduct);
} else {
log.info("Detected database: {} (MySQL mode: {})", dbProduct, isMySQL);
}
} catch (Exception e) {
log.warn("Failed to detect database type, falling back to H2 mode", e);
isMySQL = false;
isKingbase = false;
isPostgres = false;
}
}
return isMySQL;
}
private boolean isKingbase() {
if (isKingbase == null) {
// Trigger detection
isMySQL();
}
return isKingbase != null && isKingbase;
}
private boolean isPostgres() {
if (isPostgres == null) {
// Trigger detection
isMySQL();
}
return isPostgres != null && isPostgres;
}
private void runScript(String path) {
ResourceDatabasePopulator populator = new ResourceDatabasePopulator();
populator.setContinueOnError(false);

View File

@ -0,0 +1,68 @@
package vip.mate.config;
import com.zaxxer.hikari.HikariDataSource;
import com.zaxxer.hikari.HikariPoolMXBean;
import lombok.extern.slf4j.Slf4j;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import javax.sql.DataSource;
/**
* Periodically logs HikariCP pool metrics so operators can detect
* connection exhaustion / leak patterns before the application stalls.
* <p>
* Runs every 30s frequent enough to catch a pool drain within 1-2
* ticks, cheap enough to never become a problem itself.
*/
@Slf4j
@Component
public class HikariPoolMonitor {
private final HikariDataSource hikari;
public HikariPoolMonitor(DataSource dataSource) {
if (dataSource instanceof HikariDataSource hds) {
this.hikari = hds;
} else if (dataSource instanceof org.springframework.jdbc.datasource.DelegatingDataSource dds
&& dds.getTargetDataSource() instanceof HikariDataSource hds) {
// Some auto-configurations wrap Hikari in a delegating DS.
this.hikari = hds;
} else {
this.hikari = null;
log.info("[HikariMonitor] DataSource is not HikariCP — pool monitor disabled");
}
}
@Scheduled(fixedDelay = 30_000, initialDelay = 60_000)
public void logPoolStats() {
if (hikari == null) return;
HikariPoolMXBean pool = hikari.getHikariPoolMXBean();
if (pool == null) return;
int active = pool.getActiveConnections();
int idle = pool.getIdleConnections();
int total = pool.getTotalConnections();
int waiting = pool.getThreadsAwaitingConnection();
// Normal log at debug so it doesn't spam the console.
log.debug("[HikariMonitor] pool: active={}, idle={}, total={}, max={}, waiting={}",
active, idle, total, hikari.getMaximumPoolSize(), waiting);
// Warning threshold: more than 80 % of the pool is active AND
// threads are queued waiting for a connection.
int maxPool = hikari.getMaximumPoolSize();
if (active > maxPool * 0.8 && waiting > 0) {
log.warn("[HikariMonitor] Pool pressure detected — active={}, idle={}, "
+ "total={}, max={}, waiting={}", active, idle, total, maxPool, waiting);
}
// Critical: pool is fully saturated AND threads are waiting.
if (active >= maxPool && waiting > 0) {
log.error("[HikariMonitor] Pool EXHAUSTED — active={}, max={}, waiting={}. "
+ "Application will appear frozen until connections are released.",
active, maxPool, waiting);
}
}
}

View File

@ -0,0 +1,56 @@
package vip.mate.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.scheduling.TaskScheduler;
import org.springframework.scheduling.concurrent.ThreadPoolTaskScheduler;
import org.springframework.scheduling.annotation.SchedulingConfigurer;
import org.springframework.scheduling.config.ScheduledTaskRegistrar;
/**
* Scheduled-task thread-pool configuration.
* <p>
* Spring Boot's auto-configured {@code TaskScheduler} defaults to
* <b>pool-size = 1</b>, which serializes every {@code @Scheduled}
* method across the entire application. With 15+ scheduled beans
* (health checks, trigger sync, fact rebuild, feature-flag refresh,
* etc.) and several firing on the same cron tick, a single-thread
* pool causes back-pressure that makes the application appear "frozen"
* when any task blocks briefly on database I/O or lock acquisition.
* <p>
* This config sets a pool large enough to absorb simultaneous
* minute/half-hour boundaries without head-of-line blocking, while
* keeping thread count low so the scheduler does not contend with
* HikariCP or the async virtual-thread pool.
* <p>
* <b>IMPORTANT:</b> {@code @EnableScheduling} is declared once on
* {@link vip.mate.MateClawApplication}. Declaring it here as well
* creates a <em>second</em> {@code ScheduledAnnotationBeanPostProcessor}
* that competes with the first, resulting in some tasks unknowingly
* scheduled on the default single-thread executor.
*/
@Configuration
public class SchedulingConfig implements SchedulingConfigurer {
/** Pool threads — enough for concurrent ticks but kept moderate. */
private static final int POOL_SIZE = 4;
@Bean
public TaskScheduler taskScheduler() {
ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler();
scheduler.setPoolSize(POOL_SIZE);
scheduler.setThreadNamePrefix("sched-");
scheduler.setRemoveOnCancelPolicy(true);
scheduler.setAwaitTerminationSeconds(30);
scheduler.setWaitForTasksToCompleteOnShutdown(true);
return scheduler;
}
@Override
public void configureTasks(ScheduledTaskRegistrar registrar) {
// Re-use the singleton TaskScheduler bean so we never create two
// separate thread-pool instances (the @Bean above is the single
// source of truth).
registrar.setTaskScheduler(taskScheduler());
}
}

View File

@ -62,7 +62,9 @@ public class SecurityConfig {
"/api/v1/channels/webhook/**",
"/api/v1/channels/webchat/**",
"/api/v1/talk/ws",
// RFC-045: tool-generated files served via unguessable UUID + 10-min TTL
// RFC-045: tool-generated files served via unguessable UUID; entries
// expire after GeneratedFileCache.TTL (7 days) delayed access (e.g. an
// IM-delivered link opened later) is intentional, the UUID is the guard.
"/api/v1/files/generated/**"
).permitAll()
// 所有其他 API 接口需要认证

View File

@ -9,6 +9,7 @@ import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.core.JdbcTemplate;
import javax.sql.DataSource;
import java.sql.Connection;
/**
* RFC-03 Lane G2 distributed lock provider for the cron scheduler.
@ -37,13 +38,38 @@ public class ShedLockConfig {
@Bean
public LockProvider lockProvider(DataSource dataSource) {
log.info("[ShedLock] Initializing JDBC LockProvider for cron scheduling");
return new JdbcTemplateLockProvider(
boolean useDbTime = supportsDbTime(dataSource);
log.info("[ShedLock] Initializing JDBC LockProvider for cron scheduling (usingDbTime={})", useDbTime);
JdbcTemplateLockProvider.Configuration.Builder builder =
JdbcTemplateLockProvider.Configuration.builder()
.withJdbcTemplate(new JdbcTemplate(dataSource))
.withTableName("shedlock")
.usingDbTime() // server-side NOW() avoids node clock drift
.build()
);
.withTableName("shedlock");
// Server-side DB time (NOW()) avoids node clock drift across a
// multi-instance deployment. ShedLock's built-in db-time dialect map
// covers MySQL/MariaDB, PostgreSQL, H2, etc. KingbaseES is not in that
// map, so usingDbTime() would throw there fall back to app-server
// time for it, which is safe given lockAtMostFor=PT30M.
if (useDbTime) {
builder.usingDbTime();
}
return new JdbcTemplateLockProvider(builder.build());
}
/**
* Returns true when the DataSource's database is covered by ShedLock's
* built-in db-time dialect map. Only KingbaseES is excluded; on any
* detection failure we conservatively return false so the lock provider
* never throws at acquisition time.
*/
private boolean supportsDbTime(DataSource dataSource) {
try (Connection connection = dataSource.getConnection()) {
String product = connection.getMetaData().getDatabaseProductName().toLowerCase();
return !product.contains("kingbase");
} catch (Exception e) {
log.warn("[ShedLock] Could not detect database product; using app-server time: {}", e.getMessage());
return false;
}
}
}

View File

@ -196,10 +196,13 @@ public class CronJobLifecycleService {
String convId = conversationId != null ? conversationId : run.getConversationId();
String text = result != null && result.getText() != null ? result.getText() : "";
int totalTokens = chatResult != null
? chatResult.promptTokens() + chatResult.completionTokens() : 0;
runMapper.update(null, new LambdaUpdateWrapper<CronJobRunEntity>()
.eq(CronJobRunEntity::getId, run.getId())
.set(CronJobRunEntity::getStatus, "succeeded")
.set(CronJobRunEntity::getFinishedAt, LocalDateTime.now()));
.set(CronJobRunEntity::getFinishedAt, LocalDateTime.now())
.set(totalTokens > 0, CronJobRunEntity::getTokenUsage, totalTokens));
if (silent) {
// No-op run: persist a short marker so the tasks_<wsId>
@ -208,7 +211,17 @@ public class CronJobLifecycleService {
// real content to deliver or to learn from.
String marker = i18n != null ? i18n.msg("cron.run.silent")
: "(本次定时任务无新内容,已跳过)";
conversationService.saveMessage(convId, "assistant", marker);
// A silent run still made a full LLM call, so carry its token usage
// onto the marker message otherwise the settings-page total (which
// aggregates MessageEntity token columns) under-counts cron spend.
if (chatResult != null
&& (chatResult.promptTokens() > 0 || chatResult.completionTokens() > 0)) {
conversationService.saveMessage(convId, "assistant", marker, null, "completed",
chatResult.promptTokens(), chatResult.completionTokens(),
chatResult.runtimeModel(), chatResult.runtimeProvider());
} else {
conversationService.saveMessage(convId, "assistant", marker);
}
return;
}

View File

@ -118,6 +118,14 @@ public class DatasourceConnectionManager implements DisposableBean {
extra = (extra == null || extra.isBlank()) ? schemaParam : extra + "&" + schemaParam;
}
break;
case "kingbase":
case "kingbasees":
baseUrl = String.format("jdbc:kingbase8://%s:%d/%s", host, port, dbName);
if (entity.getSchemaName() != null && !entity.getSchemaName().isBlank()) {
String schemaParam = "currentSchema=" + entity.getSchemaName();
extra = (extra == null || extra.isBlank()) ? schemaParam : extra + "&" + schemaParam;
}
break;
case "clickhouse":
baseUrl = String.format("jdbc:clickhouse://%s:%d/%s", host, port, dbName);
break;

View File

@ -0,0 +1,73 @@
package vip.mate.doc;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import vip.mate.common.result.R;
import vip.mate.tool.builtin.MateClawDocService;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
/**
* 内置帮助文档的只读接口供前端文档查看器消费
*
* <p>文档本体打包在 classpath:docs/{zh,en}/ 与给智能体用的
* {@link MateClawDocService} 共享同一套扫描/校验逻辑
*/
@Slf4j
@Tag(name = "Docs")
@RestController
@RequestMapping("/api/v1/docs")
@RequiredArgsConstructor
public class DocController {
private final MateClawDocService docService;
@Operation(summary = "列出某语言下的全部帮助文档slug + 标题)")
@GetMapping
public R<List<MateClawDocService.DocMeta>> list(
@RequestParam(defaultValue = "zh") String lang) {
return R.ok(docService.list(normalizeLang(lang)));
}
@Operation(summary = "读取单篇帮助文档正文(已剥离 frontmatter")
@GetMapping("/content")
public R<Map<String, Object>> content(
@RequestParam(defaultValue = "zh") String lang,
@RequestParam String slug) {
String normLang = normalizeLang(lang);
String body = docService.read(normLang, slug);
if (body == null) {
return R.fail(404, "Document not found");
}
String title = docService.list(normLang).stream()
.filter(d -> d.slug().equals(slug))
.map(MateClawDocService.DocMeta::title)
.findFirst()
.orElse(slug);
Map<String, Object> payload = new LinkedHashMap<>();
payload.put("slug", slug);
payload.put("title", title);
payload.put("content", body);
return R.ok(payload);
}
private String normalizeLang(String lang) {
if (lang == null) {
return "zh";
}
String l = lang.toLowerCase();
// 前端 locale 形如 zh-CN / en-US取主语言段
if (l.startsWith("en")) {
return "en";
}
return "zh";
}
}

View File

@ -9,6 +9,7 @@ import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.BindException;
import org.springframework.web.bind.annotation.ExceptionHandler;
import org.springframework.web.HttpRequestMethodNotSupportedException;
import org.springframework.web.bind.annotation.RestControllerAdvice;
import org.springframework.web.context.request.async.AsyncRequestTimeoutException;
import org.springframework.web.method.annotation.MethodArgumentTypeMismatchException;
@ -121,6 +122,21 @@ public class GlobalExceptionHandler {
return ResponseEntity.status(HttpStatus.NOT_FOUND).body(R.fail(404, "Resource not found"));
}
/**
* Spring's default lets this escape to the catch-all below, surfacing as a
* 500 with a stack trace. Return a clean 405 so the client gets a
* structured body and the log stays at WARN. Doubles as a defence when a
* malformed path segment makes a reverse proxy strip the trailing path
* (e.g. a conversationId ending in ":" lands a GET on a @DeleteMapping).
*/
@ExceptionHandler(HttpRequestMethodNotSupportedException.class)
public ResponseEntity<R<Void>> handleMethodNotSupported(HttpRequestMethodNotSupportedException e,
HttpServletRequest request) {
log.warn("Method not supported: {} {} (supported: {})",
request.getMethod(), request.getRequestURI(), e.getSupportedHttpMethods());
return ResponseEntity.status(HttpStatus.METHOD_NOT_ALLOWED).body(R.fail(405, "Method not allowed"));
}
@ExceptionHandler(Exception.class)
public ResponseEntity<R<Void>> handleException(Exception e,
HttpServletRequest request,

View File

@ -16,6 +16,16 @@ import org.springframework.stereotype.Component;
@ConfigurationProperties(prefix = "mateclaw.goal")
public class GoalProperties {
/**
* Compile-time ceiling for {@link #maxHardContinuationsPerRun}. The graph
* recursion limit is sized statically to accommodate this many extra
* fresh-budget ReAct segments per run, so the runtime value is clamped to
* it an operator cannot push hard continuations past what the recursion
* backstop was sized for. Raising this requires re-sizing the recursion
* ceiling in {@code AgentGraphBuilder.frameworkRecursionLimit()}.
*/
public static final int MAX_HARD_CONTINUATIONS_CEILING = 3;
/**
* Master switch when off, the graph never invokes GoalEvaluationNode
* (the conditional edge sees no active goal, so the node is unreachable).
@ -39,6 +49,19 @@ public class GoalProperties {
*/
private boolean allowAutoFollowup = true;
/**
* Auto-derive a goal from a multi-step Plan-Execute plan. The Plan-Execute
* planner decomposes the request into steps and the step executor is a
* narrow "task runner" neither calls {@code setGoal}, so without this a
* Plan-Execute run never engages the goal subsystem. When enabled, a goal is
* created server-side at plan generation (title = request, acceptance
* criteria seeded from the plan steps), so the already-wired
* GoalEvaluationNode tracks completion. Gated by {@link #enabled}; only
* fires for genuine multi-step plans and when the conversation has no active
* goal yet. Set to {@code false} to keep Plan-Execute goal-free.
*/
private boolean autoGoalFromPlan = true;
/** Default turn budget when the user doesn't override. */
private int defaultTurnBudget = 20;
@ -57,6 +80,20 @@ public class GoalProperties {
*/
private int maxFollowupsPerRun = 8;
/**
* Max "hard continuations" per single graph run. A hard continuation is a
* goal follow-up that re-enters the ReAct loop with a FRESH iteration
* budget after a turn that hit {@code MAX_ITERATIONS_REACHED} letting a
* task too large for one budget keep going autonomously instead of stalling
* until the user sends another message. Each one costs up to a full
* {@code maxIterations} worth of node visits, so this is a dedicated cap on
* top of {@link #maxFollowupsPerRun}, clamped to
* {@link #MAX_HARD_CONTINUATIONS_CEILING} and sized into the graph recursion
* ceiling. The goal's cross-turn turn / LLM-call budgets still apply. Set to
* 0 to keep the previous behaviour (max-iterations turns end the run).
*/
private int maxHardContinuationsPerRun = 1;
/**
* Provider/model id for the evaluator. Empty string means "use the
* same model as the chat agent" — convenient for dev, expensive in

View File

@ -137,11 +137,27 @@ public class AnthropicChatModelBuilder implements ChatModelBuilder {
}
/**
* True for any Claude 4.7+ model the family that drops temperature /
* top_p / top_k and exposes the "xhigh" thinking tier between high and max.
* Detect the Claude Fable model line (e.g. {@code claude-fable-5}).
* Fable is a reasoning-first family that follows the same strict API
* contract as Claude 4.7+: temperature / top_p / top_k must be unset
* (any non-default value returns HTTP 400) and the "xhigh" adaptive
* thinking tier is available. Matching on the {@code claude-fable} token
* covers the direct-API id, the OpenRouter-prefixed form
* ({@code anthropic/claude-fable-5}), and future {@code claude-fable-N}
* revisions without a per-version code change.
*/
static boolean isClaudeFable(String modelName) {
if (modelName == null) return false;
return modelName.toLowerCase().contains("claude-fable");
}
/**
* True for any modern Claude model that drops temperature / top_p / top_k
* and exposes the "xhigh" thinking tier between high and max the Claude
* 4.7 / 4.8 generations and the Fable reasoning line.
*/
static boolean isClaude47OrLater(String modelName) {
return isClaude47(modelName) || isClaude48(modelName);
return isClaude47(modelName) || isClaude48(modelName) || isClaudeFable(modelName);
}
AnthropicChatOptions buildAnthropicOptions(ModelConfigEntity runtimeModel) {

View File

@ -79,6 +79,7 @@ public class ProviderInitProbe {
this.strategies = map;
}
@Async
@EventListener(ApplicationReadyEvent.class)
public void onApplicationReady() {
probeAllConfigured();

View File

@ -77,6 +77,16 @@ public class ModelConfigEntity {
*/
private String modalities;
/**
* Transient, request-scoped flag set by {@link vip.mate.llm.service.ModelConfigService#listByType}
* when a modality filter is supplied: {@code true} when this row's declared or
* heuristically-resolved capabilities cover the requested modality. Lets the
* sidecar selector list every enabled chat model while still highlighting the
* ones already known to support the modality. Never persisted.
*/
@TableField(exist = false)
private Boolean modalityCapable;
@TableField(fill = FieldFill.INSERT)
private LocalDateTime createTime;

View File

@ -4,8 +4,9 @@ import java.util.List;
import java.util.Set;
/**
* Read access to an agent's skill / provider bindings, as needed by
* {@link ProviderRouter} for capability-aware routing.
* Read access to an agent's skill / provider / wiki-kb bindings, as needed by
* {@link ProviderRouter} for capability-aware routing and by webchat
* endpoints that need to enumerate an agent's visible catalog.
*
* <p>Declared in the {@code llm} layer so the routing code depends only on
* this abstraction. The {@code agent} layer supplies the implementation,
@ -23,4 +24,14 @@ public interface AgentBindingResolver {
* Provider ids the agent prefers, in priority order; empty when none.
*/
List<String> getPreferredProviderIds(Long agentId);
/**
* Wiki knowledge-base ids bound to the agent, or {@code null} when the
* agent has no explicit KB scope (meaning "workspace-wide — every KB
* in the agent's workspace is visible"). Mirrors the three-state
* contract of {@link #getBoundSkillIds}: {@code null} = inherit
* default, {@code Set.of()} = explicitly scoped to nothing, non-empty
* = the explicit allowlist.
*/
Set<Long> getBoundKbIds(Long agentId);
}

View File

@ -42,6 +42,18 @@ public class MediaCaptionService {
private final RetryTemplate retryTemplate;
public CaptionResult caption(ModelConfigEntity visionModel, MessageContentPart imagePart, Locale locale) {
return caption(visionModel, imagePart, locale, null);
}
/**
* Caption an image, optionally tailored to a user question. When
* {@code userQuestion} is non-blank the vision model is asked to answer it
* directly (in addition to describing the image), so multi-turn follow-ups
* get an answer rather than a generic description. When blank, falls back to
* the factual full-description prompt.
*/
public CaptionResult caption(ModelConfigEntity visionModel, MessageContentPart imagePart, Locale locale,
String userQuestion) {
if (visionModel == null || imagePart == null) {
return CaptionResult.failure(0, new IllegalArgumentException("vision model or image part is null"));
}
@ -59,7 +71,7 @@ public class MediaCaptionService {
ChatModel chatModel = chatModelFactory.buildFor(visionModel, retryTemplate);
ChatClient client = ChatClient.create(chatModel);
UserMessage userMessage = UserMessage.builder()
.text(buildPrompt(locale, imagePart.getFileName()))
.text(buildPrompt(locale, imagePart.getFileName(), userQuestion))
.media(List.of(new Media(MimeType.valueOf(contentType), new FileSystemResource(mediaPath))))
.build();
String description = client.prompt()
@ -87,9 +99,27 @@ public class MediaCaptionService {
* (matches the primary user base) but switches to English so vision-model output
* matches the chat language and avoids polluting English-only contexts.
*/
private String buildPrompt(Locale locale, String fileName) {
private String buildPrompt(Locale locale, String fileName, String userQuestion) {
boolean english = locale != null && Locale.ENGLISH.getLanguage().equalsIgnoreCase(locale.getLanguage());
String fileHint = (fileName == null || fileName.isBlank()) ? "" : " (" + fileName + ")";
boolean hasQuestion = userQuestion != null && !userQuestion.isBlank();
if (hasQuestion) {
String question = userQuestion.trim();
// Question-aware: extract everything relevant to the user's ask, then
// answer it. Answering in the question's own language keeps the caption
// consistent with the chat regardless of the configured locale.
if (english) {
return "Look at this image" + fileHint + " and answer the user's question. "
+ "First note any details relevant to the question (objects, scene, visible text/OCR, "
+ "numbers, layout), then answer directly. Reply in the same language as the question. "
+ "Question: " + question;
}
return "请仔细查看这张图片" + fileHint + ",并回答用户的问题。"
+ "先指出与问题相关的细节(物体、场景、画面文字/OCR、数字、排版等再直接作答。"
+ "用与问题相同的语言回复。问题:" + question;
}
if (english) {
return "Describe this image" + fileHint
+ " concisely: list the main objects, scene, any visible text (OCR), "

View File

@ -125,11 +125,29 @@ public class MultimodalRouter {
}
/**
* Resolve the configured sidecar model for a modality. Returns null when:
* Resolve the configured default vision model, or {@code null} if none is set
* or the referenced model is missing/disabled. Exposed so tools (e.g. an
* on-demand image-analysis tool) can reuse the exact same model the automatic
* sidecar uses, keeping behaviour consistent across the auto and tool paths.
*/
public ModelConfigEntity resolveVisionSidecar() {
return resolveSidecar(Modality.VISION);
}
/**
* Resolve the configured sidecar model for a modality. Returns null only when:
* - the setting is empty / blank;
* - the referenced row no longer exists or has been disabled;
* - the row's resolved capability set does not actually contain the modality.
* - the referenced row no longer exists or has been disabled.
* The caller treats null as "ask the user to configure one."
* <p>
* An explicit sidecar selection is treated as the user's own capability
* declaration: a provider-compatible model can be vision-capable in practice
* even when the built-in heuristics don't recognize its name and it carries no
* declared {@code modalities}. Rejecting such a model here made it impossible to
* use a perfectly good compatible-mode vision model as the sidecar. We therefore
* honour the explicit choice and only emit a diagnostic when the heuristics
* can't confirm it a wrong pick degrades gracefully (the caption call fails and
* the attachment is reported as un-processed) rather than being silently ignored.
*/
private ModelConfigEntity resolveSidecar(Modality modality) {
SystemSettingsDTO settings = systemSettingService.getSettings();
@ -148,9 +166,9 @@ public class MultimodalRouter {
}
if (model == null || !Boolean.TRUE.equals(model.getEnabled())) return null;
if (!capabilityService.supports(model.getModelName(), model.getModalities(), modality)) {
log.warn("Configured sidecar model {}/{} does not actually support {} — ignoring",
model.getProvider(), model.getModelName(), modality);
return null;
log.info("Configured sidecar model {}/{} is not recognized as {}-capable by the "
+ "built-in heuristics; honouring the explicit selection anyway",
model.getProvider(), model.getModelName(), modality.name().toLowerCase());
}
return model;
}

View File

@ -86,6 +86,7 @@ public class ModelCapabilityService {
// ===== Anthropic Claude =====
// Vision yes (image), native video no Anthropic's API only accepts images.
m.put("claude-fable", EnumSet.of(Modality.VISION));
m.put("claude-4.7", EnumSet.of(Modality.VISION));
m.put("claude-4.5", EnumSet.of(Modality.VISION));
m.put("claude-4", EnumSet.of(Modality.VISION));

View File

@ -11,6 +11,7 @@ import vip.mate.llm.event.ModelConfigChangedEvent;
import vip.mate.llm.model.ModelConfigEntity;
import vip.mate.llm.repository.ModelConfigMapper;
import java.util.Comparator;
import java.util.List;
import org.springframework.context.ApplicationEventPublisher;
@ -73,9 +74,18 @@ public class ModelConfigService {
/**
* Optional modality filter (case-insensitive: {@code "vision" / "video" / "audio"}).
* When non-null, only enabled rows whose resolved capability set contains the
* requested modality survive used by the multimodal sidecar settings UI to
* populate "default vision model" / "default video model" dropdowns.
* Used by the multimodal sidecar settings UI to populate "default vision model" /
* "default video model" dropdowns.
* <p>
* The filter does <b>not</b> hide models the built-in heuristics fail to recognize:
* a provider-compatible model (e.g. a DashScope OpenAI-compatible vision model with
* a custom name) is vision-capable in practice even though its name matches no
* built-in prefix and it carries no declared {@code modalities}. Hard-filtering
* those out left them un-selectable as a sidecar. Instead every <em>enabled</em>
* chat model is returned; each row's transient {@link ModelConfigEntity#getModalityCapable()}
* flag records whether its declared / heuristic capabilities already cover the
* requested modality, and known-capable rows are sorted to the top so the UI can
* highlight them while still letting the user pick any model.
*/
public List<ModelConfigEntity> listByType(String modelType, String modality) {
List<ModelConfigEntity> rows;
@ -100,7 +110,12 @@ public class ModelConfigService {
}
return rows.stream()
.filter(m -> Boolean.TRUE.equals(m.getEnabled()))
.filter(m -> modelCapabilityService.supports(m.getModelName(), m.getModalities(), required))
.peek(m -> m.setModalityCapable(
modelCapabilityService.supports(m.getModelName(), m.getModalities(), required)))
// Known-capable models first; preserve the existing default-then-name
// order within each group.
.sorted(Comparator.comparing(
(ModelConfigEntity m) -> Boolean.TRUE.equals(m.getModalityCapable())).reversed())
.toList();
}

View File

@ -66,6 +66,13 @@ public class ModelDiscoveryService {
private static final Duration TIMEOUT = Duration.ofSeconds(10);
// Shared HttpClient for OpenAI-compatible providers avoids creating a new
// native thread + connection pool per request (was causing thread-leak OOM).
private static final HttpClient SHARED_HTTP_CLIENT = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_1_1)
.connectTimeout(Duration.ofSeconds(30))
.build();
// Virtual-thread executor for parallel model probing (lightweight, short-lived)
private static final ExecutorService PROBE_EXECUTOR = Executors.newVirtualThreadPerTaskExecutor();
@ -962,10 +969,7 @@ public class ModelDiscoveryService {
* the upgrade negotiation.
*/
private RestClient.Builder openAiCompatibleClientBuilder() {
HttpClient httpClient = HttpClient.newBuilder()
.version(HttpClient.Version.HTTP_1_1)
.build();
return RestClient.builder().requestFactory(new JdkClientHttpRequestFactory(httpClient));
return RestClient.builder().requestFactory(new JdkClientHttpRequestFactory(SHARED_HTTP_CLIENT));
}
@SuppressWarnings("unchecked")

View File

@ -81,6 +81,66 @@ public class MemoryProperties {
/** 禁用的 MemoryProvider ID 集合(例如 "structured", "session_search" */
private Set<String> disabledProviders = new HashSet<>();
// ==================== Always-on injection budget ====================
/**
* Character budget for the always-on structured memory block injected into
* every system prompt (user + feedback entries). When exceeded, the
* most-recently-updated entries are kept and older ones are omitted, so
* accumulated memory cannot grow the per-turn context without bound.
* 0 = unlimited (legacy behavior).
*/
private int systemBlockMaxChars = 4000;
/**
* Hard cap on the number of entries injected per structured type in the
* always-on block. Keeps the newest entries per type even when the global
* character budget would otherwise admit more from a single type.
* 0 = unlimited.
*/
private int systemBlockMaxEntriesPerType = 40;
/**
* Enable the nightly consolidation pass over always-on structured memory
* (user/feedback): an LLM merges near-duplicate and stale entries so the
* files shrink on disk rather than only being trimmed at injection time.
*/
private boolean structuredConsolidationEnabled = true;
/**
* Minimum entry count before a structured type is consolidated. Below this
* the file is already small, so the LLM call is skipped.
*/
private int structuredConsolidationMinEntries = 8;
/**
* Cron expression for the structured-memory consolidation maintenance task.
* Independent of {@code dreamingCron} so the two passes can be scheduled
* (and gated) separately. Default: 3:30 AM daily, after nightly emergence.
*/
private String structuredConsolidationCron = "0 30 3 * * ?";
/**
* Maximum number of owner buckets (shared + personal) consolidated per agent
* per run. Bounds LLM cost on agents with many per-owner memory buckets;
* remaining owners are picked up on subsequent runs. 0 = unlimited.
*/
private int structuredConsolidationMaxOwnersPerRun = 50;
/**
* Deterministic character ceiling for PROFILE.md, the always-on user-profile
* file. The summarization pass rewrites it and is asked to stay concise; this
* is the hard backstop that truncates at a section boundary if it overruns.
* 0 = unlimited.
*/
private int profileMaxChars = 4000;
/**
* Deterministic character ceiling for MEMORY.md, the always-on long-term
* memory file rewritten by summarization and emergence. 0 = unlimited.
*/
private int memoryMdMaxChars = 8000;
// ==================== Dream v2 Feature Flags ====================
// --- Phase 1: Lifecycle mediator wiring ---

View File

@ -37,6 +37,30 @@ public class MemoryController {
private final MemoryProperties memoryProperties;
private final DreamingScheduler dreamingScheduler;
private final WorkspaceFileService workspaceFileService;
private final StructuredMemoryConsolidationService structuredConsolidationService;
@Operation(summary = "手动触发 always-on 结构化记忆整合user/feedback合并去重过时条目")
@PostMapping("/{agentId}/structured-consolidation")
@RequireWorkspaceRole("member")
public R<Map<String, Object>> triggerStructuredConsolidation(@PathVariable Long agentId) {
try {
StructuredMemoryConsolidationService.ConsolidationStats s =
structuredConsolidationService.consolidateAgent(agentId);
Map<String, Object> out = new LinkedHashMap<>();
out.put("ownersConsolidated", s.ownersConsolidated);
out.put("updated", s.updated);
out.put("skippedSmall", s.skippedSmall);
out.put("skippedOverCap", s.skippedOverCap);
out.put("failed", s.failed);
out.put("entriesBefore", s.entriesBefore);
out.put("entriesAfter", s.entriesAfter);
return R.ok(out);
} catch (Exception e) {
log.error("[Memory] Manual structured consolidation failed for agent={}: {}",
agentId, e.getMessage(), e);
return R.fail("结构化记忆整合失败: " + e.getMessage());
}
}
@Operation(summary = "手动触发记忆整合daily notes → MEMORY.mdNIGHTLY 模式)")
@PostMapping("/{agentId}/emergence")

View File

@ -2,6 +2,7 @@ package vip.mate.memory.fact.projection;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.scheduling.annotation.Async;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import vip.mate.agent.AgentService;
@ -14,6 +15,9 @@ import java.util.List;
* Scheduled full rebuild of the fact projection for all active agents.
* Cron expression configured via mate.memory.fact.projection-rebuild-cron.
* Only runs when projection-enabled=true.
* <p>
* {@code @Async} keeps the scheduler-thread pool free the actual DB
* work runs on the virtual-thread async executor.
*
* @author MateClaw Team
*/
@ -26,6 +30,7 @@ public class FactProjectionScheduler {
private final FactProjectionBuilder projectionBuilder;
private final MemoryProperties properties;
@Async
@Scheduled(cron = "${mate.memory.fact.projection-rebuild-cron:0 */30 * * * ?}")
public void rebuildAll() {
if (!properties.getFact().isProjectionEnabled()) {

View File

@ -0,0 +1,72 @@
package vip.mate.memory.scheduler;
import lombok.Getter;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import vip.mate.agent.AgentService;
import vip.mate.agent.model.AgentEntity;
import vip.mate.memory.MemoryProperties;
import vip.mate.memory.service.StructuredMemoryConsolidationService;
import vip.mate.memory.service.StructuredMemoryConsolidationService.ConsolidationStats;
import java.time.LocalDateTime;
/**
* Scheduled maintenance for always-on structured memory.
* <p>
* Runs the consolidation pass that merges duplicate / stale user & feedback
* entries so the always-on block shrinks at the storage level. Kept separate from
* {@link DreamingScheduler} so it has its own cron and enable flag disabling
* nightly dreaming must not silently disable structured consolidation, and vice
* versa.
*
* @author MateClaw Team
*/
@Slf4j
@Component
@RequiredArgsConstructor
public class StructuredMemoryMaintenanceScheduler {
private final AgentService agentService;
private final StructuredMemoryConsolidationService consolidationService;
private final MemoryProperties properties;
/** Last run time, for the status API. */
@Getter
private volatile LocalDateTime lastRunTime;
@Scheduled(cron = "${mate.memory.structured-consolidation-cron:0 30 3 * * ?}")
public void runConsolidation() {
if (!properties.isStructuredConsolidationEnabled()) {
log.debug("[StructuredConsolidation] Disabled, skipping");
return;
}
log.info("[StructuredConsolidation] Starting maintenance cycle");
ConsolidationStats total = new ConsolidationStats();
int agents = 0;
for (AgentEntity agent : agentService.listAgents()) {
if (!Boolean.TRUE.equals(agent.getEnabled())) {
continue;
}
agents++;
try {
ConsolidationStats agentStats = consolidationService.consolidateAgent(agent.getId());
total.add(agentStats);
} catch (Exception e) {
log.warn("[StructuredConsolidation] Failed for agent={} ({}): {}",
agent.getId(), agent.getName(), e.getMessage());
}
}
lastRunTime = LocalDateTime.now();
log.info("[StructuredConsolidation] Cycle done: agents={}, buckets={}, updated={}, "
+ "skipped(small)={}, skipped(cap)={}, failed={}, entries {}->{}",
agents, total.ownersConsolidated, total.updated,
total.skippedSmall, total.skippedOverCap, total.failed,
total.entriesBefore, total.entriesAfter);
}
}

View File

@ -0,0 +1,42 @@
package vip.mate.memory.service;
/**
* Deterministic character-budget backstop for always-on memory files
* (PROFILE.md / MEMORY.md) that are injected into every system prompt.
* <p>
* These files are rewritten wholesale by the summarization and emergence passes,
* which are instructed to stay concise but have no hard ceiling so on their own
* they can grow the per-turn context without bound. This enforces a deterministic
* cap: when content exceeds the budget it is truncated at a Markdown section
* boundary (keeping the head, where the core/principle sections live) and a marker
* is appended. The LLM rewrite remains the primary, content-aware compressor; this
* is the last-resort guarantee that the file stays bounded.
*
* @author MateClaw Team
*/
final class AlwaysOnFileBudget {
/** Appended when content is truncated; user-facing note kept in the file. */
static final String MARKER = "\n\n> ⚠️ 后续内容已截断以控制注入体积。";
private AlwaysOnFileBudget() {
}
/**
* Truncate {@code content} to at most {@code maxChars} characters, cutting at
* the last {@code "## "} section boundary that fits so a section is never split
* mid-way. Returns the input unchanged when it already fits or when budgeting
* is disabled ({@code maxChars <= 0}).
*/
static String enforce(String content, int maxChars) {
if (content == null || maxChars <= 0 || content.length() <= maxChars) {
return content;
}
int limit = Math.max(0, maxChars - MARKER.length());
// Prefer cutting at a section boundary within the budget; "\n## " keeps the
// leading section header intact. Fall back to a hard cut when none fits.
int boundary = content.lastIndexOf("\n## ", limit);
String head = boundary > 0 ? content.substring(0, boundary) : content.substring(0, limit);
return head.stripTrailing() + MARKER;
}
}

View File

@ -159,6 +159,9 @@ public class MemoryEmergenceService {
return buildSkippedReport(agentId, mode, topic, triggerSource, startedAt, "empty memory_content");
}
// Deterministic backstop so the always-on MEMORY.md stays bounded even
// if the LLM rewrite ignores the "keep concise" instruction.
newContent = AlwaysOnFileBudget.enforce(newContent, properties.getMemoryMdMaxChars());
workspaceFileService.saveFile(agentId, "MEMORY.md", newContent);
eventPublisher.publishEvent(new MemoryWriteEvent(agentId, "MEMORY.md", "consolidate", newContent));
String llmReason = root.path("reason").asText("");

View File

@ -328,6 +328,7 @@ public class MemorySummarizationService {
* (TEAM) file when there is no real owner (cron / system).
*/
private void saveMemory(Long agentId, String filename, String content, String ownerKey) {
content = capAlwaysOnFile(filename, content);
if (isPersonal(ownerKey)) {
workspaceFileService.saveMemoryFile(agentId, filename, content, ownerKey);
} else {
@ -335,6 +336,21 @@ public class MemorySummarizationService {
}
}
/**
* Enforce the deterministic size ceiling on the always-on profile/memory files
* so they cannot grow the per-turn system prompt without bound. Daily notes and
* other files (only recalled on demand) are left untouched.
*/
private String capAlwaysOnFile(String filename, String content) {
if ("PROFILE.md".equals(filename)) {
return AlwaysOnFileBudget.enforce(content, properties.getProfileMaxChars());
}
if ("MEMORY.md".equals(filename)) {
return AlwaysOnFileBudget.enforce(content, properties.getMemoryMdMaxChars());
}
return content;
}
/** A real, isolatable owner — i.e. not null/blank and not the system bucket. */
private boolean isPersonal(String ownerKey) {
return ownerKey != null && !ownerKey.isBlank()

View File

@ -0,0 +1,184 @@
package vip.mate.memory.service;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.chat.messages.SystemMessage;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.converter.BeanOutputConverter;
import org.springframework.stereotype.Service;
import vip.mate.agent.AgentGraphBuilder;
import vip.mate.agent.prompt.PromptLoader;
import vip.mate.llm.model.ModelConfigEntity;
import vip.mate.llm.service.ModelConfigService;
import vip.mate.memory.MemoryProperties;
import java.time.LocalDate;
import java.util.LinkedHashMap;
import java.util.List;
/**
* Nightly consolidation of always-on structured memory (user / feedback).
* <p>
* These typed files are injected into every system prompt and only ever grow:
* nudge and post-conversation summarization append entries, key-exact dedup lets
* paraphrased keys through, and no pass ever merges them. Over time the always-on
* block inflates per-turn context. This service periodically rewrites each
* always-on type file into a smaller, deduplicated, non-stale set via the LLM, so
* accumulated memory shrinks at the storage level rather than only being trimmed
* at injection time.
* <p>
* Consolidation runs per bucket: the shared (TEAM/GLOBAL) file plus every personal
* owner's file, because most growth accumulates in per-owner buckets that the
* always-on prefetch injects each turn. The number of buckets processed per agent
* per run is capped to bound LLM cost.
*
* @author MateClaw Team
*/
@Slf4j
@Service
@RequiredArgsConstructor
public class StructuredMemoryConsolidationService {
private final StructuredMemoryService structuredMemoryService;
private final ModelConfigService modelConfigService;
private final AgentGraphBuilder agentGraphBuilder;
private final MemoryProperties properties;
/** Typed LLM output for a single consolidation call. */
public record ConsolidationResult(boolean shouldUpdate, List<Entry> entries, String reason) {
public record Entry(String key, String content) {}
}
/** Aggregate counters for one consolidation run, for observability. */
public static class ConsolidationStats {
public int ownersConsolidated; // buckets that passed the gate and called the LLM
public int updated; // buckets actually rewritten
public int skippedSmall; // buckets below the min-entries gate
public int skippedOverCap; // buckets deferred to a later run by the per-run cap
public int failed; // buckets whose LLM/parse/write raised
public int entriesBefore;
public int entriesAfter;
public void add(ConsolidationStats o) {
ownersConsolidated += o.ownersConsolidated;
updated += o.updated;
skippedSmall += o.skippedSmall;
skippedOverCap += o.skippedOverCap;
failed += o.failed;
entriesBefore += o.entriesBefore;
entriesAfter += o.entriesAfter;
}
}
/** Consolidate every always-on structured bucket (shared + personal) for an agent. */
public ConsolidationStats consolidateAgent(Long agentId) {
ConsolidationStats stats = new ConsolidationStats();
if (!properties.isStructuredConsolidationEnabled()) {
return stats;
}
int cap = properties.getStructuredConsolidationMaxOwnersPerRun();
int remaining = cap > 0 ? cap : Integer.MAX_VALUE;
for (String type : structuredMemoryService.alwaysOnTypes()) {
for (String ownerKey : structuredMemoryService.consolidatableOwnerKeys(agentId, type)) {
String content = structuredMemoryService.readTypeRaw(agentId, type, ownerKey);
int count = structuredMemoryService.countEntries(content);
if (count < properties.getStructuredConsolidationMinEntries()) {
stats.skippedSmall++;
continue;
}
if (remaining <= 0) {
stats.skippedOverCap++;
continue;
}
remaining--;
stats.ownersConsolidated++;
stats.entriesBefore += count;
try {
int after = consolidateBucket(agentId, type, ownerKey, content, count);
if (after >= 0) {
stats.updated++;
stats.entriesAfter += after;
} else {
stats.entriesAfter += count; // no write bucket unchanged
}
} catch (Exception e) {
stats.failed++;
stats.entriesAfter += count;
log.warn("[StructuredConsolidation] agent={} type={} owner={} failed: {}",
agentId, type, ownerKey, e.getMessage());
}
}
}
return stats;
}
/**
* Consolidate one bucket. Returns the new entry count when the file was
* rewritten, or {@code -1} when nothing was written (LLM declined, produced
* unparseable output, or would not have reduced the entry count).
*/
private int consolidateBucket(Long agentId, String type, String ownerKey, String content, int count) {
BeanOutputConverter<ConsolidationResult> converter = new BeanOutputConverter<>(ConsolidationResult.class);
String systemPrompt = PromptLoader.loadPrompt("memory/consolidate-structured-system");
String userPrompt = PromptLoader.loadPrompt("memory/consolidate-structured-user")
.replace("{type}", type)
.replace("{today}", LocalDate.now().toString())
.replace("{count}", String.valueOf(count))
.replace("{content}", content);
ChatModel chatModel = buildChatModel();
ChatResponse resp = chatModel.call(new Prompt(List.of(
new SystemMessage(systemPrompt),
new UserMessage(userPrompt),
new UserMessage(converter.getFormat()))));
String text = resp.getResult().getOutput().getText();
ConsolidationResult result;
try {
result = converter.convert(text);
} catch (Exception e) {
log.warn("[StructuredConsolidation] Unparseable LLM output agent={} type={} owner={}: {}",
agentId, type, ownerKey, e.getMessage());
return -1;
}
if (result == null || !result.shouldUpdate()
|| result.entries() == null || result.entries().isEmpty()) {
return -1;
}
LinkedHashMap<String, String> consolidated = new LinkedHashMap<>();
for (ConsolidationResult.Entry e : result.entries()) {
if (e == null) continue;
String key = e.key() == null ? "" : e.key().trim();
String value = e.content() == null ? "" : e.content().trim();
if (!key.isEmpty() && !value.isEmpty()) {
consolidated.put(key, value);
}
}
if (consolidated.isEmpty()) {
return -1;
}
// Safety invariant: consolidation must never grow the entry count. A model
// that hallucinates extra entries would otherwise make the bloat worse.
if (consolidated.size() > count) {
log.debug("[StructuredConsolidation] agent={} type={} owner={} produced {} > {} entries; skipping write",
agentId, type, ownerKey, consolidated.size(), count);
return -1;
}
structuredMemoryService.replaceTypeEntries(agentId, type, ownerKey, consolidated, "consolidation");
log.info("[StructuredConsolidation] agent={} type={} owner={} consolidated {} -> {} entries",
agentId, type, ownerKey, count, consolidated.size());
return consolidated.size();
}
private ChatModel buildChatModel() {
ModelConfigEntity defaultModel = modelConfigService.getDefaultModel();
return agentGraphBuilder.buildRuntimeChatModel(defaultModel);
}
}

View File

@ -105,6 +105,7 @@ public class StructuredMemoryService {
private final WorkspaceFileService workspaceFileService;
private final ApplicationEventPublisher eventPublisher;
private final vip.mate.memory.MemoryProperties properties;
/** Per-file lock to prevent concurrent read-modify-write on the same file */
private final ConcurrentHashMap<String, ReentrantLock> fileLocks = new ConcurrentHashMap<>();
@ -240,35 +241,133 @@ public class StructuredMemoryService {
return buildMemoryBlock(agentId, null);
}
/** Reserve left for structural overhead (headers, blank lines) when truncating a single oversized entry. */
private static final int BLOCK_STRUCTURE_RESERVE = 120;
/** Owner-scoped variant of {@link #buildMemoryBlock(Long)}. */
public String buildMemoryBlock(Long agentId, String ownerKey) {
StringBuilder sb = new StringBuilder();
boolean hasContent = false;
int maxChars = Math.max(0, properties.getSystemBlockMaxChars());
int maxPerType = Math.max(0, properties.getSystemBlockMaxEntriesPerType());
// 1. Collect every always-on entry with its update date and a global
// insertion index (file order, types in SYSTEM_PROMPT_TYPES order).
// Truncate any single entry larger than the whole budget so it can
// never blow the cap on its own.
int contentCap = maxChars > 0 ? Math.max(1, maxChars - BLOCK_STRUCTURE_RESERVE) : -1;
List<BlockEntry> all = new ArrayList<>();
int globalIndex = 0;
for (String type : SYSTEM_PROMPT_TYPES) {
String fileContent = readFileSafe(agentId, toFilename(type), ownerKey);
if (fileContent.isBlank()) continue;
for (Map.Entry<String, String> entry : parseSections(fileContent).entrySet()) {
String content = extractContentOnly(entry.getValue());
if (content.isBlank()) continue;
if (contentCap >= 0 && content.length() > contentCap) {
content = content.substring(0, contentCap) + "";
}
all.add(new BlockEntry(type, entry.getKey(), content,
extractUpdated(entry.getValue()), globalIndex++));
}
}
if (all.isEmpty()) return "";
Map<String, String> sections = parseSections(fileContent);
if (sections.isEmpty()) continue;
// 2. Enforce the always-on budget against the TRUE rendered length
// (headers, blank lines, and the omission note all counted), keeping
// the most-recently-updated entries so accumulated memory cannot grow
// the per-turn context without bound.
Set<BlockEntry> kept = selectWithinBudget(all, maxChars, maxPerType);
int omitted = all.size() - kept.size();
return renderBlock(all, kept, omitted);
}
/** A candidate entry for the always-on block, with budget metadata. */
private record BlockEntry(String type, String key, String content,
String updated, int index) {}
/**
* Render the always-on block: survivors grouped by type, in original file
* order (stable ordering keeps the system prefix cacheable), followed by an
* omission note when entries were dropped.
*/
private String renderBlock(List<BlockEntry> all, Set<BlockEntry> kept, int omitted) {
StringBuilder sb = new StringBuilder();
boolean hasContent = false;
for (String type : SYSTEM_PROMPT_TYPES) {
List<BlockEntry> typeEntries = all.stream()
.filter(e -> e.type().equals(type) && kept.contains(e))
.sorted(Comparator.comparingInt(BlockEntry::index))
.toList();
if (typeEntries.isEmpty()) continue;
if (!hasContent) {
sb.append("## Structured Memory\n\n");
hasContent = true;
}
sb.append("### ").append(typeDisplayName(type)).append("\n");
for (Map.Entry<String, String> entry : sections.entrySet()) {
// Extract just the content line (skip metadata)
String content = extractContentOnly(entry.getValue());
sb.append("- **").append(entry.getKey()).append("**: ").append(content).append("\n");
for (BlockEntry e : typeEntries) {
sb.append("- **").append(e.key()).append("**: ").append(e.content()).append("\n");
}
sb.append("\n");
}
if (omitted > 0) {
sb.append("> ").append(omitted)
.append(" older memory entries omitted to bound context size.\n");
}
return sb.toString().trim();
}
/**
* Select the entries that fit the always-on injection budget, preferring the
* most-recently-updated ones. Applies a per-type entry cap first, then a
* global character budget measured against the actual rendered block (not
* just bullet lengths). Newer entries (later update date, then later
* insertion order) win; ties and missing dates fall back to insertion order.
*/
private Set<BlockEntry> selectWithinBudget(List<BlockEntry> all, int maxChars, int maxPerType) {
// Keep-priority: most recent update first, then most recently inserted.
Comparator<BlockEntry> newestFirst = Comparator
.comparing(BlockEntry::updated, Comparator.nullsFirst(Comparator.naturalOrder()))
.thenComparingInt(BlockEntry::index)
.reversed();
// Per-type cap: drop the oldest entries beyond the cap.
List<BlockEntry> survivors = new ArrayList<>(all);
if (maxPerType > 0) {
Set<BlockEntry> overflow = new HashSet<>();
for (String type : SYSTEM_PROMPT_TYPES) {
List<BlockEntry> ofType = survivors.stream()
.filter(e -> e.type().equals(type))
.sorted(newestFirst)
.toList();
if (ofType.size() > maxPerType) {
overflow.addAll(ofType.subList(maxPerType, ofType.size()));
}
}
survivors.removeAll(overflow);
}
if (maxChars <= 0) {
return new HashSet<>(survivors);
}
// Global character budget: admit newest entries while the fully rendered
// block stays within budget. Measuring the real render (including the
// omission note) makes the cap exact; once an entry no longer fits, every
// remaining entry is older and is dropped too.
List<BlockEntry> ordered = survivors.stream().sorted(newestFirst).toList();
Set<BlockEntry> picked = new HashSet<>();
for (BlockEntry e : ordered) {
Set<BlockEntry> trial = new HashSet<>(picked);
trial.add(e);
int omittedIfStop = all.size() - trial.size();
if (renderBlock(all, trial, Math.max(0, omittedIfStop)).length() > maxChars) {
break;
}
picked.add(e);
}
return picked;
}
/**
* Build a query-conditioned memory block for per-turn prefetch injection.
* Scores {@link #PREFETCH_TYPES} entries against the user's question and returns
@ -400,6 +499,97 @@ public class StructuredMemoryService {
return "structured/" + type + ".md";
}
// ==================== Consolidation support ====================
/** The always-on structured types injected into every system prompt. */
public List<String> alwaysOnTypes() {
return SYSTEM_PROMPT_TYPES;
}
/** Read the raw Markdown of a structured type file (owner-scoped when personal). */
public String readTypeRaw(Long agentId, String type, String ownerKey) {
validateType(type);
return readFileSafe(agentId, toFilename(type), ownerKey);
}
/** Count the {@code ## key} entries in a structured file's raw Markdown. */
public int countEntries(String rawContent) {
return (rawContent == null || rawContent.isBlank()) ? 0 : parseSections(rawContent).size();
}
/**
* Distinct buckets that hold entries for a structured type and are eligible
* for consolidation: the shared bucket (returned as {@code null}) plus each
* personal owner that has its own row. Lets the nightly maintenance pass
* consolidate per-owner memory, where most growth actually accumulates.
*/
public List<String> consolidatableOwnerKeys(Long agentId, String type) {
validateType(type);
String filename = toFilename(type);
List<String> owners = new ArrayList<>();
owners.add(null); // shared (TEAM/GLOBAL) bucket
for (WorkspaceFileEntity f : workspaceFileService.listFiles(agentId)) {
if (filename.equals(f.getFilename()) && isPersonal(f.getOwnerKey())
&& !owners.contains(f.getOwnerKey())) {
owners.add(f.getOwnerKey());
}
}
return owners;
}
/**
* Atomically replace all entries of a structured type with a consolidated set,
* re-serialized in the canonical {@code ## key / content / > Source | Updated}
* format. Used by the nightly consolidation pass to shrink always-on memory.
* Insertion order of {@code entries} is preserved.
* <p>
* Update dates are preserved per key: an entry whose key already existed keeps
* its original {@code Updated} date, and a newly-merged key inherits the newest
* date among the existing entries. This keeps recency/LRU semantics intact
* consolidation must not make a batch of old facts look freshly written.
*/
public void replaceTypeEntries(Long agentId, String type, String ownerKey,
LinkedHashMap<String, String> entries, String source) {
validateType(type);
String filename = toFilename(type);
String lockKey = agentId + ":" + (ownerKey == null ? "" : ownerKey) + ":" + filename;
ReentrantLock lock = fileLocks.computeIfAbsent(lockKey, k -> new ReentrantLock());
lock.lock();
try {
// Derive prior update dates so consolidation preserves provenance.
Map<String, String> keyToDate = new HashMap<>();
String newestDate = "";
for (Map.Entry<String, String> s : parseSections(readFileSafe(agentId, filename, ownerKey)).entrySet()) {
String d = extractUpdated(s.getValue());
if (!d.isEmpty()) {
keyToDate.put(s.getKey(), d);
if (d.compareTo(newestDate) > 0) newestDate = d;
}
}
String fallbackDate = newestDate.isEmpty() ? LocalDate.now().toString() : newestDate;
String src = source != null ? source : "consolidation";
StringBuilder sb = new StringBuilder();
for (Map.Entry<String, String> e : entries.entrySet()) {
if (e.getKey() == null || e.getKey().isBlank()
|| e.getValue() == null || e.getValue().isBlank()) {
continue;
}
String key = e.getKey().trim();
String date = keyToDate.getOrDefault(key, fallbackDate);
if (sb.length() > 0) sb.append("\n\n");
sb.append("## ").append(key).append("\n")
.append(e.getValue().trim())
.append("\n> Source: ").append(src).append(" | Updated: ").append(date);
}
saveStructured(agentId, filename, sb.toString(), ownerKey);
log.info("[StructuredMemory] Replaced {} entries in '{}' for agent={} owner={} (source={})",
entries.size(), filename, agentId, ownerKey, src);
} finally {
lock.unlock();
}
}
private void validateType(String type) {
if (!VALID_TYPES.contains(type)) {
throw new IllegalArgumentException("Invalid memory type: " + type

View File

@ -23,10 +23,14 @@ public class PlanningController {
private final PlanningService planningService;
@Operation(summary = "获取 Agent 的计划列表")
@Operation(summary = "获取计划列表(带 agentId 则按员工,否则跨员工取最近 N 条)")
@GetMapping
public R<List<PlanEntity>> listByAgent(@RequestParam String agentId) {
return R.ok(planningService.listPlansByAgent(agentId));
public R<List<PlanEntity>> list(@RequestParam(required = false) String agentId,
@RequestParam(required = false, defaultValue = "100") int limit) {
if (agentId != null && !agentId.isBlank()) {
return R.ok(planningService.listPlansByAgent(agentId));
}
return R.ok(planningService.listRecentPlans(limit));
}
@Operation(summary = "获取计划详情(含步骤)")

View File

@ -21,6 +21,9 @@ public class PlanEntity {
/** 关联的 Agent ID字符串 */
private String agentId;
/** 产生该计划的对话/运行 ID可空历史行为 null。用于把计划绑定到具体运行、支持跨员工/协同分组。 */
private String conversationId;
/** 任务目标 */
private String goal;

View File

@ -29,6 +29,14 @@ public class SubPlanEntity {
/** 步骤状态pending / running / completed / failed */
private String status;
/**
* Delegated agent id for this step. When non-null, the executor routes this
* step to that specialist agent instead of the parent (plan) agent. Null =
* run with the parent agent (original behavior). The frontend resolves the
* display name from this id via the agent store.
*/
private Long assignedAgentId;
/** 步骤执行结果 */
@TableField(value = "result", updateStrategy = FieldStrategy.ALWAYS)
private String result;

View File

@ -34,10 +34,36 @@ public class PlanningService {
*/
@Transactional
public PlanEntity createPlan(String agentId, String goal, List<String> steps) {
return createPlan(agentId, null, goal, steps);
}
/**
* 创建执行计划并绑定到产生它的对话/运行
* conversationId 可空历史调用方便于把计划归到某次运行支撑跨员工/协同看板
*/
@Transactional
public PlanEntity createPlan(String agentId, String conversationId, String goal, List<String> steps) {
return createPlan(agentId, conversationId, goal, steps, null);
}
/**
* 创建执行计划并为每个步骤可选地指派专职子 agent
* stepAgentIds steps 等长同序按位置对应某位为 null 表示该步骤由父 agent 执行
* stepAgentIds 整体可空无委派的历史调用方
*/
@Transactional
public PlanEntity createPlan(String agentId, String conversationId, String goal,
List<String> steps, List<Long> stepAgentIds) {
PlanEntity plan = new PlanEntity();
plan.setAgentId(agentId);
plan.setConversationId(conversationId);
plan.setGoal(goal);
plan.setStatus("running");
// A freshly generated plan is queued, not yet running: it sits in the
// board's "pending" column until its first step actually starts
// (updateSubPlanStatus promotes the plan to "running" then). This makes
// the board's pending column meaningful for queued / approval-gated
// plans instead of being perpetually empty.
plan.setStatus("pending");
plan.setTotalSteps(steps.size());
plan.setCompletedSteps(0);
planMapper.insert(plan);
@ -48,6 +74,11 @@ public class PlanningService {
sub.setStepIndex(i);
sub.setDescription(steps.get(i));
sub.setStatus("pending");
// Per-step delegation: only set when an assignment exists for this
// index; otherwise the step runs with the parent agent.
if (stepAgentIds != null && i < stepAgentIds.size()) {
sub.setAssignedAgentId(stepAgentIds.get(i));
}
subPlanMapper.insert(sub);
});
@ -64,6 +95,13 @@ public class PlanningService {
sub.setStatus(status);
if ("running".equals(status)) {
sub.setStartTime(LocalDateTime.now());
// Promote the parent plan out of the "pending" (queued) column
// the moment its first step actually starts executing.
PlanEntity plan = planMapper.selectById(planId);
if (plan != null && "pending".equals(plan.getStatus())) {
plan.setStatus("running");
planMapper.updateById(plan);
}
}
subPlanMapper.updateById(sub);
}
@ -111,6 +149,17 @@ public class PlanningService {
.orderByDesc(PlanEntity::getCreateTime));
}
/**
* 跨员工获取最近的计划列表用于团队/泳道看板
* 按创建时间倒序limit 兜底防止全表拉取
*/
public List<PlanEntity> listRecentPlans(int limit) {
int capped = limit <= 0 ? 100 : Math.min(limit, 500);
return planMapper.selectList(new LambdaQueryWrapper<PlanEntity>()
.orderByDesc(PlanEntity::getCreateTime)
.last("LIMIT " + capped));
}
/**
* 获取计划详情含子计划
*/
@ -135,6 +184,17 @@ public class PlanningService {
.orderByAsc(SubPlanEntity::getStepIndex));
}
/**
* The agent delegated to run a given step, or {@code null} when the step
* runs with the parent (plan) agent. Read by the executor to route a step to
* its specialist agent. Sourced from the DB (not graph state) so it survives
* replay / approval-resume.
*/
public Long getStepAssignedAgent(Long planId, int stepIndex) {
SubPlanEntity sub = getSubPlan(planId, stepIndex);
return sub != null ? sub.getAssignedAgentId() : null;
}
/**
* 标记计划失败
*/

View File

@ -7,6 +7,7 @@ import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.event.EventListener;
import org.springframework.core.annotation.Order;
import org.springframework.scheduling.annotation.Async;
import org.springframework.stereotype.Component;
import vip.mate.channel.ChannelManager;
import vip.mate.llm.service.ModelProviderService;
@ -65,7 +66,9 @@ public class PluginManager {
* Load all plugins on application startup.
* Scans three paths in priority order: workspace > user-global.
* Higher priority plugins shadow lower priority ones with the same name.
* {@code @Async} 文件系统扫描和 JAR 类加载不阻塞主启动线程
*/
@Async
@EventListener(ApplicationReadyEvent.class)
@Order(250)
public void loadAllPlugins() {

View File

@ -29,6 +29,8 @@ import java.util.zip.ZipInputStream;
* <li>Zip Slip path traversal</li>
* <li>Per-file 1MB, total 50MB</li>
* <li>Only SKILL.md / references/ / scripts/ entries are kept</li>
* <li>Binary entries are skipped with a WARN bundle storage is text-only,
* so decoding them as text would persist corrupted content</li>
* </ul>
*
* <p>Extraction is two-pass: the entire archive is buffered in memory first
@ -229,6 +231,23 @@ public class ZipSkillFetcher {
throw new IOException("Total extracted size exceeds 50MB limit");
}
// Skill bundles persist file contents as text (mate_skill_file
// is a TEXT column; SkillBundle carries Map<String,String>).
// Decoding a binary entry (.png/.woff/.zip/compiled helper, )
// as text replaces every invalid byte with U+FFFD, so the file
// would be stored permanently corrupted and "restored" broken
// on every sync. Binary resources are not supported in a bundle
// today, so skip them with a clear WARN instead of silently
// mangling them matches how unknown root-level files are
// already handled below. (Root-level binaries were already
// dropped; this also covers binaries nested in scripts/ and
// references/, which previously slipped through corrupted.)
if (isLikelyBinary(bytes)) {
log.warn("[ZipSkillFetcher] Skipping binary entry (not supported in skill bundles): {}", entryName);
zis.closeEntry();
continue;
}
String content = new String(bytes, charset);
String normalizedName = entryPath.toString().replace('\\', '/');
String fileName = entryPath.getFileName().toString();
@ -289,6 +308,27 @@ public class ZipSkillFetcher {
return new ExtractedSkill(skillMdContent, references, scripts);
}
/**
* Heuristic binary detector: an entry is treated as binary if a NUL byte
* (0x00) appears within the inspected prefix. UTF-8 and GBK text never
* contain a NUL, while virtually every binary format (PNG/WOFF/ZIP/class/
* native executable) carries one near the start this is the same cheap,
* reliable test git uses to decide "is this a text file". Inspecting only a
* prefix keeps it O(1) for large entries.
*/
private static boolean isLikelyBinary(byte[] bytes) {
if (bytes == null || bytes.length == 0) {
return false;
}
int limit = Math.min(bytes.length, 8000);
for (int i = 0; i < limit; i++) {
if (bytes[i] == 0x00) {
return true;
}
}
return false;
}
/**
* Classify a root-level file (sibling of SKILL.md, no directory prefix)
* by extension. Returns {@code "scripts"} / {@code "references"} for

View File

@ -495,7 +495,10 @@ public class SkillRuntimeService {
sb.append("To read a skill's reference or script files, use ");
sb.append("`readSkillFile(skillName=<name>, filePath=\"references/...\")`. ");
sb.append("Skills with a `scripts/` directory expose `runSkillScript`; ");
sb.append("SKILL.md will name the script when needed.\n\n");
sb.append("SKILL.md will name the script when needed. ");
sb.append("If a skill describes steps but ships no runnable script, write the code ");
sb.append("its instructions describe and run it with ");
sb.append("`execute_code(language=<python|bash|node>, code=..., skillName=<name>)`.\n\n");
sb.append("| Skill | Status | Description |\n");
sb.append("|-------|--------|-------------|\n");
for (ResolvedSkill skill : selected) {

View File

@ -28,10 +28,22 @@ import java.util.concurrent.TimeUnit;
public class SkillScriptExecutionService {
private static final long DEFAULT_TIMEOUT_SECONDS = 30;
private static final long MAX_TIMEOUT_SECONDS = 300;
private static final int MAX_OUTPUT_BYTES = 50_000;
private static final boolean IS_WINDOWS = System.getProperty("os.name", "")
.toLowerCase(Locale.ROOT).contains("win");
/** Supported inline-code languages mapped to the temp-file extension. */
private static final Map<String, String> LANGUAGE_EXTENSIONS = Map.of(
"python", ".py",
"py", ".py",
"bash", ".sh",
"sh", ".sh",
"shell", ".sh",
"node", ".js",
"javascript", ".js",
"js", ".js");
/**
* 执行脚本兼容签名 不注入额外 env vars
*
@ -56,6 +68,85 @@ public class SkillScriptExecutionService {
* @return 执行结果
*/
public ScriptResult execute(Path scriptPath, List<String> args, Map<String, String> envVars) {
return executeResolved(scriptPath, args, envVars, DEFAULT_TIMEOUT_SECONDS, false);
}
/**
* Execute LLM-generated source code inline, without a pre-existing script file.
* <p>
* Materializes {@code code} into a temporary file (extension chosen from
* {@code language}) inside {@code workingDir}, runs it through the same
* interpreter-selection + timeout + output-capping + env-injection path as
* {@link #execute(Path, List, Map)}, then deletes the temp file.
*
* <p>This is what makes a documentation-only skill (a SKILL.md with no
* {@code scripts:} entries) runnable: the agent reads the instructions,
* generates code, and runs it here.
*
* @param language one of python / bash / node (and aliases); selects the interpreter
* @param code the source code to run; must be non-blank
* @param workingDir directory the temp file is written to and the process cwd. When {@code null}
* a private temp scratch directory is created and removed afterward; when
* non-null it must be an existing directory (e.g. a skill or workspace dir)
* @param args optional positional arguments passed to the program
* @param envVars optional env vars injected into the subprocess (e.g. decrypted skill secrets)
* @param timeoutSeconds optional timeout override; clamped to (0, {@value #MAX_TIMEOUT_SECONDS}], defaults to {@value #DEFAULT_TIMEOUT_SECONDS}
* @return execution result
*/
public ScriptResult executeCode(String language, String code, Path workingDir,
List<String> args, Map<String, String> envVars, Long timeoutSeconds) {
if (code == null || code.isBlank()) {
return ScriptResult.error(-1, "No code supplied");
}
String ext = language == null ? null
: LANGUAGE_EXTENSIONS.get(language.trim().toLowerCase(Locale.ROOT));
if (ext == null) {
return ScriptResult.error(-1, "Unsupported language: " + language
+ ". Supported: python, bash, node");
}
if (workingDir != null && !Files.isDirectory(workingDir)) {
return ScriptResult.error(-1, "Working directory does not exist: " + workingDir);
}
long timeout = DEFAULT_TIMEOUT_SECONDS;
if (timeoutSeconds != null && timeoutSeconds > 0) {
timeout = Math.min(timeoutSeconds, MAX_TIMEOUT_SECONDS);
}
// No caller-supplied directory (e.g. an agent with no workspace base path):
// run in a private scratch directory and remove it afterward. Mirrors the
// shell tool tolerating a null working directory rather than failing.
Path scratchDir = null;
Path codeFile = null;
try {
Path dir = workingDir;
if (dir == null) {
scratchDir = Files.createTempDirectory("mc_code_ws_");
dir = scratchDir;
}
// Write the code into the working dir so the process cwd matches the
// file location relative paths in the generated code resolve as the
// author expects, and skill-scoped runs stay inside the skill dir.
codeFile = Files.createTempFile(dir, "mc_code_", ext);
Files.writeString(codeFile, code, StandardCharsets.UTF_8);
if (!IS_WINDOWS && ext.equals(".sh")) {
codeFile.toFile().setExecutable(true);
}
// Scrub sensitive host env vars: the code is LLM-authored, so it must
// not inherit the server's API keys / tokens. Skill secrets, when
// supplied via envVars, are re-added on top.
return executeResolved(codeFile, args, envVars, timeout, true);
} catch (IOException e) {
log.error("Failed to materialize inline code: {}", e.getMessage());
return ScriptResult.error(-1, "Failed to write code file: " + e.getMessage());
} finally {
deleteQuietly(codeFile);
deleteDirQuietly(scratchDir);
}
}
private ScriptResult executeResolved(Path scriptPath, List<String> args, Map<String, String> envVars,
long timeoutSeconds, boolean scrubSensitiveEnv) {
if (!Files.exists(scriptPath) || !Files.isRegularFile(scriptPath)) {
return ScriptResult.error(-1, "Script not found: " + scriptPath);
}
@ -113,7 +204,15 @@ public class SkillScriptExecutionService {
pb.directory(scriptPath.getParent().toFile());
pb.redirectOutput(stdoutFile.toFile());
pb.redirectError(stderrFile.toFile());
// RFC-091: inject per-skill secrets / settings as env vars.
// Strip secrets from the inherited environment before injecting the
// caller's own env. Used for LLM-authored inline code so it never
// sees the server's API keys / tokens via process inheritance.
if (scrubSensitiveEnv) {
pb.environment().keySet().removeIf(key ->
key.contains("KEY") || key.contains("SECRET") || key.contains("TOKEN")
|| key.contains("PASSWORD") || key.contains("CREDENTIAL"));
}
// Inject per-skill secrets / settings as env vars.
// pb.environment() inherits the parent process env; putAll
// OVERRIDES same-named entries with the supplied values.
// Null / blank values are skipped to avoid clearing
@ -129,12 +228,12 @@ public class SkillScriptExecutionService {
Process process = pb.start();
boolean finished = process.waitFor(DEFAULT_TIMEOUT_SECONDS, TimeUnit.SECONDS);
boolean finished = process.waitFor(timeoutSeconds, TimeUnit.SECONDS);
if (!finished) {
killProcess(process);
String stdout = readFileTruncated(stdoutFile, MAX_OUTPUT_BYTES);
String stderr = readFileTruncated(stderrFile, MAX_OUTPUT_BYTES);
String timeoutMsg = "[timeout after " + DEFAULT_TIMEOUT_SECONDS + "s]";
String timeoutMsg = "[timeout after " + timeoutSeconds + "s]";
stderr = stderr.isEmpty() ? timeoutMsg : stderr + "\n" + timeoutMsg;
return new ScriptResult(-1, stdout, stderr);
}
@ -199,6 +298,16 @@ public class SkillScriptExecutionService {
}
}
/** Recursively remove a scratch directory created for an inline-code run. */
private static void deleteDirQuietly(Path dir) {
if (dir == null) return;
try (var paths = Files.walk(dir)) {
paths.sorted(java.util.Comparator.reverseOrder()).forEach(p -> {
try { Files.deleteIfExists(p); } catch (IOException ignored) {}
});
} catch (IOException ignored) {}
}
@lombok.Data
@lombok.AllArgsConstructor
public static class ScriptResult {

View File

@ -47,10 +47,36 @@ public class SkillWorkspaceManager {
}
/**
* 按约定解析 skill 工作区路径{root}/{skillName}/
* Resolve the conventional skill workspace path: {@code {root}/{sanitizedName}}.
* <p>
* Deterministic in {@code skillName} alone (no filesystem-state dependency). The
* non-ASCII collision fixed in #254 comes from {@link #sanitizeNameForFs} preserving
* Unicode letters/digits, so distinct names already map to distinct directories. No
* {@code -hash} suffix is appended: skill names are charset-constrained, so two names
* sanitizing to the same string is not a real case, and keeping the bare name avoids
* changing the path scheme for every existing skill (which would orphan already-created
* workspaces with no migration).
*/
public Path resolveConventionPath(String skillName) {
return getWorkspaceRoot().resolve(sanitizeName(skillName));
return getWorkspaceRoot().resolve(sanitizeNameForFs(skillName));
}
/**
* Sanitize a skill name into a filesystem-safe directory segment.
* <p>
* Keeps the same charset as the legacy {@code [a-zA-Z0-9_.-]} rule but additionally
* preserves Unicode letters/digits ({@code \p{L}\p{N}}), so non-ASCII names no longer
* collapse to underscores and collide (#254). Everything else path separators,
* control characters, whitespace becomes {@code _}. Hyphens are kept (not folded to
* {@code _}) so existing kebab-case skill paths (e.g. {@code browser-cdp}) are unchanged
* across upgrades; only the Unicode handling differs from the legacy behavior.
*/
private String sanitizeNameForFs(String name) {
if (name == null || name.isBlank()) {
return "unnamed";
}
String cleaned = name.strip().replaceAll("[^\\p{L}\\p{N}_.\\-]", "_");
return cleaned.isEmpty() ? "unnamed" : cleaned;
}
/**
@ -116,11 +142,18 @@ public class SkillWorkspaceManager {
Files.createDirectories(workspaceDir.resolve("scripts"));
Path skillMd = workspaceDir.resolve("SKILL.md");
if (overwrite || !Files.exists(skillMd)) {
String content = (initialContent != null && !initialContent.isBlank())
? initialContent
: buildDefaultSkillMd(skillName);
String content = (initialContent != null && !initialContent.isBlank())
? initialContent
: buildDefaultSkillMd(skillName);
if (overwrite) {
Files.writeString(skillMd, content);
} else {
// 原子创建避免并发上传时的 TOCTOU 竞态
try {
Files.writeString(skillMd, content, StandardOpenOption.CREATE_NEW);
} catch (FileAlreadyExistsException e) {
// 另一线程已创建跳过写入
}
}
log.info("Initialized skill workspace: {} (overwrite={})", workspaceDir, overwrite);

View File

@ -0,0 +1,84 @@
package vip.mate.system.controller;
import io.swagger.v3.oas.annotations.Operation;
import io.swagger.v3.oas.annotations.tags.Tag;
import lombok.Data;
import lombok.RequiredArgsConstructor;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import vip.mate.common.result.R;
import vip.mate.system.proxy.ProxyManager;
import vip.mate.system.proxy.ProxySettings;
import vip.mate.workspace.core.annotation.RequireGlobalAdmin;
/**
* Global outbound-proxy configuration. System-wide, so reads require an admin
* and writes require the global admin.
*/
@Tag(name = "网络代理")
@RestController
@RequestMapping("/api/v1/settings/proxy")
@RequiredArgsConstructor
public class ProxyController {
private final ProxyManager proxyManager;
@Operation(summary = "获取全局代理配置")
@GetMapping
@RequireGlobalAdmin
public R<ProxyConfigResponse> get() {
return R.ok(toResponse(proxyManager.currentSettings()));
}
@Operation(summary = "保存全局代理配置")
@PutMapping
@RequireGlobalAdmin
public R<ProxyConfigResponse> save(@RequestBody ProxyConfigRequest req) {
ProxySettings saved = proxyManager.save(
Boolean.TRUE.equals(req.getEnabled()),
req.getUrl(),
req.getNonProxyHosts());
if (saved.enabled() && !saved.valid()) {
return R.fail(saved.error());
}
return R.ok(toResponse(saved));
}
@Operation(summary = "测试代理连通性")
@PostMapping("/test")
@RequireGlobalAdmin
public R<ProxyManager.ProbeResult> test(@RequestBody ProxyConfigRequest req) {
ProxyManager.ProbeResult result = proxyManager.test(req.getUrl());
return R.ok(result);
}
private ProxyConfigResponse toResponse(ProxySettings s) {
ProxyConfigResponse resp = new ProxyConfigResponse();
resp.setEnabled(s.enabled());
resp.setUrl(s.url());
resp.setNonProxyHosts(s.nonProxyHosts());
resp.setValid(s.valid());
resp.setError(s.error());
return resp;
}
@Data
public static class ProxyConfigRequest {
private Boolean enabled;
private String url;
private String nonProxyHosts;
}
@Data
public static class ProxyConfigResponse {
private boolean enabled;
private String url;
private String nonProxyHosts;
private boolean valid;
private String error;
}
}

View File

@ -0,0 +1,291 @@
package vip.mate.system.proxy;
import lombok.extern.slf4j.Slf4j;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.context.event.EventListener;
import org.springframework.stereotype.Component;
import org.springframework.util.StringUtils;
import vip.mate.system.service.SystemSettingService;
import java.io.IOException;
import java.net.Authenticator;
import java.net.InetSocketAddress;
import java.net.PasswordAuthentication;
import java.net.Proxy;
import java.net.ProxySelector;
import java.net.Socket;
import java.net.SocketAddress;
import java.net.URI;
import java.util.List;
import java.util.concurrent.atomic.AtomicReference;
/**
* Installs and refreshes the process-wide outbound proxy from the global
* {@code proxy.*} system settings.
*
* <p>When enabled, the configured proxy is applied through three mechanisms so
* it reaches every egress style in the backend with a single switch, instead of
* patching dozens of scattered HTTP-client construction sites:
* <ol>
* <li>A default {@link ProxySelector} honored by every
* {@link java.net.http.HttpClient} built without an explicit proxy (LLM
* calls, model probes, most channel adapters, MCP, STT/TTS) and by
* {@code HttpURLConnection} (the Hutool-based search / media tools).</li>
* <li>{@code http(s).proxyHost}/{@code socksProxyHost} system properties for
* libraries that read them directly.</li>
* <li>A static accessor ({@link #chromeProxyServer()}) the browser launcher
* reads to add {@code --proxy-server}, since Chromium does not honor the
* JVM proxy.</li>
* </ol>
*
* <p>SOCKS proxies are honored by {@code HttpURLConnection} but silently ignored
* by {@code java.net.http.HttpClient}; see {@link ProxySettings} for the
* resulting coverage boundary.
*
* <p>Adapters that carry their own per-channel proxy (Telegram / Discord set an
* explicit {@code .proxy()}) override the default selector, so this global proxy
* never fights a more specific one.
*/
@Slf4j
@Component
public class ProxyManager {
static final String KEY_ENABLED = "proxy.enabled";
static final String KEY_URL = "proxy.url";
static final String KEY_NON_PROXY_HOSTS = "proxy.nonProxyHosts";
private static final String[] MANAGED_SYSTEM_PROPS = {
"http.proxyHost", "http.proxyPort",
"https.proxyHost", "https.proxyPort",
"http.nonProxyHosts",
"socksProxyHost", "socksProxyPort",
};
/**
* Current {@code scheme://host:port} for Chromium's {@code --proxy-server},
* or {@code null} when no proxy is active. Static so the static browser
* launch-arg builder can read it without a bean reference.
*/
private static volatile String chromeProxyServer;
private final SystemSettingService settings;
/** The default selector present before we ever overrode it, for restore. */
private final ProxySelector originalDefaultSelector;
private final Authenticator originalDefaultAuthenticator;
private final AtomicReference<ProxySettings> current = new AtomicReference<>();
private volatile boolean authenticatorInstalled;
public ProxyManager(SystemSettingService settings) {
this.settings = settings;
this.originalDefaultSelector = ProxySelector.getDefault();
this.originalDefaultAuthenticator = Authenticator.getDefault();
}
/** Chromium {@code --proxy-server} value, or {@code null} when inactive. */
public static String chromeProxyServer() {
return chromeProxyServer;
}
@EventListener(ApplicationReadyEvent.class)
public void onReady() {
try {
apply(readSettings());
} catch (Exception e) {
log.warn("Failed to apply proxy settings at startup: {}", e.getMessage());
}
}
/** Re-read persisted settings and re-apply. Called after a config save. */
public synchronized void refresh() {
apply(readSettings());
}
public ProxySettings currentSettings() {
ProxySettings s = current.get();
return s != null ? s : readSettings();
}
public ProxySettings readSettings() {
boolean enabled = settings.getBool(KEY_ENABLED, false);
String url = settings.getString(KEY_URL, "");
String nph = settings.getString(KEY_NON_PROXY_HOSTS, "");
return ProxySettings.parse(enabled, url, nph);
}
/** Persist new config and apply it immediately. Returns the parsed result. */
public synchronized ProxySettings save(boolean enabled, String url, String nonProxyHosts) {
settings.saveBool(KEY_ENABLED, enabled, "Global outbound proxy enabled");
settings.saveString(KEY_URL, url == null ? "" : url.trim(), "Global outbound proxy url");
settings.saveString(KEY_NON_PROXY_HOSTS,
nonProxyHosts == null ? "" : nonProxyHosts.trim(),
"Global proxy bypass list (| separated)");
ProxySettings parsed = readSettings();
apply(parsed);
return parsed;
}
private synchronized void apply(ProxySettings s) {
current.set(s);
clearManagedSystemProps();
if (!s.active()) {
ProxySelector.setDefault(originalDefaultSelector);
uninstallAuthenticator();
chromeProxyServer = null;
if (s.enabled() && !s.valid()) {
log.warn("Global proxy is enabled but the url is invalid ({}); running without a proxy",
s.error());
} else {
log.info("Global proxy disabled; outbound traffic goes direct");
}
return;
}
ProxySelector.setDefault(new GlobalProxySelector(s, originalDefaultSelector));
setSystemProps(s);
installAuthenticatorIfNeeded(s);
chromeProxyServer = s.chromeProxyServer();
log.info("Global proxy active: {} {}:{}{} (bypass: {})",
s.isSocks() ? "SOCKS" : "HTTP", s.host(), s.port(),
s.hasCredentials() ? " (auth)" : "", s.nonProxyHosts());
if (s.isSocks()) {
log.warn("SOCKS proxy applies to HttpURLConnection-based egress (search/media) only; "
+ "the java.net.http LLM/streaming path does not support SOCKS and will go direct");
}
}
private void setSystemProps(ProxySettings s) {
if (s.isSocks()) {
System.setProperty("socksProxyHost", s.host());
System.setProperty("socksProxyPort", String.valueOf(s.port()));
} else {
System.setProperty("http.proxyHost", s.host());
System.setProperty("http.proxyPort", String.valueOf(s.port()));
System.setProperty("https.proxyHost", s.host());
System.setProperty("https.proxyPort", String.valueOf(s.port()));
// JVM uses '|'-separated patterns here same format we persist.
if (StringUtils.hasText(s.nonProxyHosts())) {
System.setProperty("http.nonProxyHosts", s.nonProxyHosts());
}
}
}
private void clearManagedSystemProps() {
for (String prop : MANAGED_SYSTEM_PROPS) {
System.clearProperty(prop);
}
}
private void installAuthenticatorIfNeeded(ProxySettings s) {
if (!s.hasCredentials()) {
uninstallAuthenticator();
return;
}
final String user = s.username();
final char[] pass = s.password() == null ? new char[0] : s.password().toCharArray();
// Allow Basic proxy auth over an HTTPS CONNECT tunnel (disabled by default since JDK 8u111).
System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");
System.setProperty("jdk.http.auth.proxying.disabledSchemes", "");
Authenticator.setDefault(new Authenticator() {
@Override
protected PasswordAuthentication getPasswordAuthentication() {
if (getRequestorType() == RequestorType.PROXY) {
return new PasswordAuthentication(user, pass);
}
return null;
}
});
authenticatorInstalled = true;
}
private void uninstallAuthenticator() {
if (authenticatorInstalled) {
Authenticator.setDefault(originalDefaultAuthenticator);
authenticatorInstalled = false;
}
}
/**
* Probe reachability of the configured proxy by opening a TCP connection to
* its host:port. Confirms the proxy endpoint is listening (the common
* failure mode proxy app not running / wrong port) without depending on
* upstream internet connectivity. Returns latency in milliseconds.
*/
public ProbeResult test(String url) {
ProxySettings s = ProxySettings.parse(true, url, null);
if (!s.valid()) {
return new ProbeResult(false, 0, s.error());
}
long start = System.nanoTime();
try (Socket socket = new Socket()) {
socket.connect(new InetSocketAddress(s.host(), s.port()), 4000);
long ms = (System.nanoTime() - start) / 1_000_000;
return new ProbeResult(true, ms, null);
} catch (IOException e) {
return new ProbeResult(false, 0, e.getClass().getSimpleName() + ": " + e.getMessage());
}
}
/** Result of {@link #test(String)}. */
public record ProbeResult(boolean ok, long latencyMs, String error) {
}
/**
* Routes through the configured proxy unless the target host matches the
* bypass list, in which case it defers to the selector that was the default
* before we took over (preserving any pre-existing direct/proxy behavior).
*/
private static final class GlobalProxySelector extends ProxySelector {
private final List<Proxy> proxyList;
private final List<String> bypass;
private final ProxySelector fallback;
GlobalProxySelector(ProxySettings s, ProxySelector fallback) {
this.proxyList = List.of(s.toProxy());
this.bypass = s.bypassPatterns();
this.fallback = fallback;
}
@Override
public List<Proxy> select(URI uri) {
String host = uri == null ? null : uri.getHost();
if (host == null || matchesBypass(host)) {
return fallback != null ? fallback.select(uri) : List.of(Proxy.NO_PROXY);
}
return proxyList;
}
@Override
public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
if (fallback != null) {
fallback.connectFailed(uri, sa, ioe);
}
}
private boolean matchesBypass(String host) {
for (String pattern : bypass) {
if (matches(pattern, host)) {
return true;
}
}
return false;
}
/** Glob match with leading/trailing {@code *}, matching JVM nonProxyHosts semantics. */
private static boolean matches(String pattern, String host) {
if (pattern.equalsIgnoreCase(host)) {
return true;
}
if (pattern.startsWith("*") && pattern.endsWith("*") && pattern.length() > 2) {
return host.toLowerCase().contains(pattern.substring(1, pattern.length() - 1).toLowerCase());
}
if (pattern.startsWith("*")) {
return host.toLowerCase().endsWith(pattern.substring(1).toLowerCase());
}
if (pattern.endsWith("*")) {
return host.toLowerCase().startsWith(pattern.substring(0, pattern.length() - 1).toLowerCase());
}
return false;
}
}
}

View File

@ -0,0 +1,194 @@
package vip.mate.system.proxy;
import org.springframework.util.StringUtils;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.net.URI;
import java.util.ArrayList;
import java.util.List;
/**
* Parsed, validated form of the global outbound-proxy configuration.
*
* <p>The raw config is a single proxy URL {@code http://127.0.0.1:7890},
* {@code https://host:443}, or {@code socks5://127.0.0.1:1080} with optional
* {@code user:pass@} credentials. The scheme selects the proxy type:
* {@code socks}/{@code socks5}/{@code socks4} map to a SOCKS proxy, everything
* else to an HTTP proxy.
*
* <p>HTTP/HTTPS proxies are honored across the whole backend (the JDK
* {@link java.net.http.HttpClient} and {@code HttpURLConnection} both respect a
* default {@link java.net.ProxySelector}). SOCKS is only honored by the
* {@code HttpURLConnection}-based egress (search / media tools); the
* {@code java.net.http} client silently ignores a SOCKS proxy, so SOCKS does not
* cover the LLM / streaming path callers must use an HTTP proxy for that.
*/
public final class ProxySettings {
/** Default bypass list applied when the user leaves the field blank. */
public static final String DEFAULT_NON_PROXY_HOSTS =
"localhost|127.*|[::1]|10.*|172.16.*|172.17.*|172.18.*|172.19.*|"
+ "172.2*|172.30.*|172.31.*|192.168.*|*.local";
private final boolean enabled;
private final String url;
private final String nonProxyHosts;
// Derived (only meaningful when valid()).
private final Proxy.Type type;
private final String host;
private final int port;
private final String username;
private final String password;
private final boolean valid;
private final String error;
private ProxySettings(boolean enabled, String url, String nonProxyHosts,
Proxy.Type type, String host, int port,
String username, String password, boolean valid, String error) {
this.enabled = enabled;
this.url = url;
this.nonProxyHosts = nonProxyHosts;
this.type = type;
this.host = host;
this.port = port;
this.username = username;
this.password = password;
this.valid = valid;
this.error = error;
}
/**
* Parse persisted values into a validated settings object. Never throws
* an unparseable URL yields {@code valid() == false} with {@link #error()}
* populated, so a bad row can't crash startup.
*/
public static ProxySettings parse(boolean enabled, String url, String nonProxyHosts) {
String nph = StringUtils.hasText(nonProxyHosts) ? nonProxyHosts.trim() : DEFAULT_NON_PROXY_HOSTS;
if (!StringUtils.hasText(url)) {
return new ProxySettings(enabled, url, nph, null, null, -1, null, null, false,
"proxy url is empty");
}
String trimmed = url.trim();
URI uri;
try {
uri = new URI(trimmed);
} catch (Exception e) {
return new ProxySettings(enabled, url, nph, null, null, -1, null, null, false,
"malformed proxy url: " + e.getMessage());
}
String scheme = uri.getScheme() == null ? "" : uri.getScheme().toLowerCase();
Proxy.Type proxyType = switch (scheme) {
case "socks", "socks5", "socks4" -> Proxy.Type.SOCKS;
case "http", "https" -> Proxy.Type.HTTP;
default -> null;
};
if (proxyType == null) {
return new ProxySettings(enabled, url, nph, null, null, -1, null, null, false,
"unsupported proxy scheme: " + scheme + " (use http/https/socks5)");
}
String h = uri.getHost();
int p = uri.getPort();
if (!StringUtils.hasText(h) || p <= 0) {
return new ProxySettings(enabled, url, nph, null, null, -1, null, null, false,
"proxy url must include host and port, e.g. http://127.0.0.1:7890");
}
String user = null;
String pass = null;
String userInfo = uri.getUserInfo();
if (StringUtils.hasText(userInfo)) {
int idx = userInfo.indexOf(':');
if (idx >= 0) {
user = userInfo.substring(0, idx);
pass = userInfo.substring(idx + 1);
} else {
user = userInfo;
}
}
return new ProxySettings(enabled, trimmed, nph, proxyType, h, p, user, pass, true, null);
}
public boolean enabled() {
return enabled;
}
/** True when the proxy should actually be installed: enabled AND parseable. */
public boolean active() {
return enabled && valid;
}
public boolean valid() {
return valid;
}
public String error() {
return error;
}
public String url() {
return url;
}
public String nonProxyHosts() {
return nonProxyHosts;
}
public Proxy.Type type() {
return type;
}
public boolean isSocks() {
return type == Proxy.Type.SOCKS;
}
public String host() {
return host;
}
public int port() {
return port;
}
public String username() {
return username;
}
public String password() {
return password;
}
public boolean hasCredentials() {
return StringUtils.hasText(username);
}
/** A {@link Proxy} instance for explicit injection where needed. */
public Proxy toProxy() {
return new Proxy(type, new InetSocketAddress(host, port));
}
/**
* The {@code --proxy-server} value for Chromium. Chromium accepts
* {@code scheme://host:port} but not embedded credentials, so userinfo is
* dropped here.
*/
public String chromeProxyServer() {
String scheme = isSocks() ? "socks5" : "http";
return scheme + "://" + host + ":" + port;
}
/** Split the {@code |}-separated bypass list into individual patterns. */
public List<String> bypassPatterns() {
List<String> out = new ArrayList<>();
if (!StringUtils.hasText(nonProxyHosts)) {
return out;
}
for (String part : nonProxyHosts.split("\\|")) {
String t = part.trim();
if (!t.isEmpty()) {
out.add(t);
}
}
return out;
}
}

View File

@ -67,7 +67,7 @@ public class SystemHealthService {
}
}
return new HealthResponse(overall, checks);
return new HealthResponse(overall, checks, bootstrapRunner.getDatabaseLabel());
}
private HealthCheck checkDefaultModel() {
@ -190,7 +190,7 @@ public class SystemHealthService {
// ==================== Response Records ====================
public record HealthResponse(String overall, List<HealthCheck> checks) {}
public record HealthResponse(String overall, List<HealthCheck> checks, String database) {}
public record HealthCheck(String name, String status, String message, HealthAction action) {}

View File

@ -365,6 +365,14 @@ public class BrowserLauncher {
if (IS_WINDOWS) {
args.add("--disable-gpu");
}
// Chromium does not honor the JVM proxy selector / system properties, so
// route it explicitly when a global proxy is active. Bypass loopback so
// the local CDP endpoint and local services stay reachable.
String proxyServer = vip.mate.system.proxy.ProxyManager.chromeProxyServer();
if (proxyServer != null && !proxyServer.isBlank()) {
args.add("--proxy-server=" + proxyServer);
args.add("--proxy-bypass-list=<-loopback>");
}
return args;
}

View File

@ -0,0 +1,222 @@
package vip.mate.tool.builtin;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyDescription;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.chat.model.ToolContext;
import org.springframework.ai.tool.annotation.Tool;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Lazy;
import org.springframework.lang.Nullable;
import org.springframework.stereotype.Component;
import vip.mate.agent.context.ChatOrigin;
import vip.mate.llm.routing.AgentBindingResolver;
import vip.mate.skill.runtime.SkillRuntimeService;
import vip.mate.skill.runtime.SkillScriptExecutionService;
import vip.mate.skill.runtime.model.ResolvedSkill;
import vip.mate.skill.secret.SkillSecretService;
import vip.mate.tool.guard.WorkspacePathGuard;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.Set;
/**
* Built-in tool: execute LLM-generated source code inline.
* <p>
* Unlike {@code runSkillScript}, which runs a pre-existing file under a skill's
* {@code scripts/} directory, this tool accepts the code as text and runs it on
* the fly. It lets a documentation-only skill (a SKILL.md with no {@code scripts:}
* entries) be acted on: the agent reads the instructions, writes the code those
* instructions describe, and runs it here.
*
* <p>Safety:
* <ul>
* <li>Dangerous patterns in the code trigger ToolGuard approval/blocking the
* tool name is registered as a shell-equivalent guarded tool.</li>
* <li>The subprocess does not inherit the server's secret env vars; only a
* bound skill's own declared secrets are injected.</li>
* <li>When {@code skillName} is given, the calling agent must be bound to that
* skill, and execution is scoped to the skill directory.</li>
* <li>Timeout defaults to 30s, hard-capped at 300s; output is truncated.</li>
* </ul>
*/
@Slf4j
@Component
@RequiredArgsConstructor
public class CodeExecuteTool {
private final SkillRuntimeService runtimeService;
private final SkillScriptExecutionService executionService;
private final SkillSecretService skillSecretService;
private final ObjectMapper objectMapper;
@Lazy
@Autowired
private AgentBindingResolver agentBindingResolver;
@vip.mate.tool.ConcurrencyUnsafe("code execution can have arbitrary side effects on the host process and filesystem")
@Tool(name = "execute_code", description = """
Execute a snippet of code you write, in python, bash, or node.
Use this to act on a skill whose SKILL.md describes steps but ships no runnable script:
read the instructions, write the code they describe, and run it here.
Parameters:
- language: one of "python", "bash", "node"
- code: the full source code to run
- skillName: optional. When set, the code runs inside that skill's directory
(so it can read the skill's reference/template files by relative path)
and the skill's stored secrets are injected as environment variables.
- args: optional positional arguments, given as ONE JSON-encoded string:
a JSON array for multiple args, or plain text for a single argument.
- timeoutSeconds: optional, default 30, max 300.
Returns: JSON with exitCode, stdout, stderr.
Security: dangerous operations trigger security approval. The server's own
secret environment variables are not exposed to the code.
""")
public String execute_code(
@JsonProperty(required = true)
@JsonPropertyDescription("Language: python, bash, or node")
String language,
@JsonProperty(required = true)
@JsonPropertyDescription("The full source code to run")
String code,
@JsonProperty(required = false)
@JsonPropertyDescription("Optional skill name to scope execution to and inject secrets from")
String skillName,
@JsonProperty(required = false)
@JsonPropertyDescription("Optional positional arguments as ONE JSON-encoded string: a JSON array for multiple args, or plain text for one literal argument.")
String args,
@JsonProperty(required = false)
@JsonPropertyDescription("Timeout in seconds, default 30, max 300")
Integer timeoutSeconds,
@Nullable ToolContext ctx
) {
log.info("[CodeExecute] language={}, skill={}, codeChars={}",
language, skillName, code == null ? 0 : code.length());
Path workingDir;
Map<String, String> envVars = Collections.emptyMap();
if (skillName != null && !skillName.isBlank()) {
// Skill-scoped run: validate binding + resolve the skill directory.
ResolvedSkill skill = runtimeService.findActiveSkill(skillName);
if (skill == null) {
return formatError("Skill '" + skillName + "' not found or not enabled");
}
Long agentId = ChatOrigin.from(ctx).agentId();
if (agentId != null) {
Set<Long> boundSkillIds = agentBindingResolver.getBoundSkillIds(agentId);
if (boundSkillIds != null && (skill.getId() == null || !boundSkillIds.contains(skill.getId()))) {
return formatError("Skill '" + skillName + "' is not available for this agent.");
}
}
// Directory-backed skills run inside their own directory so the code
// can read the skill's reference/template files by relative path. A
// database-backed skill (no directory) is still runnable: fall through
// with a null working dir so executeCode uses a private scratch dir.
// Either way the skill's stored secrets are injected.
workingDir = skill.getSkillDir();
if (skill.getId() != null) {
envVars = skillSecretService.getDecrypted(skill.getId());
}
} else {
// Workspace-scoped run: prefer the agent's workspace base path so any
// files the code produces land where the user expects. When the agent
// has no workspace dir, pass null executeCode then runs in a private
// temp scratch dir (same tolerance as the shell tool).
workingDir = WorkspacePathGuard.getWorkingDirectory(ctx);
if (workingDir != null && !Files.isDirectory(workingDir)) {
workingDir = null;
}
}
Long timeout = timeoutSeconds != null ? timeoutSeconds.longValue() : null;
List<String> argList = normalizeArgs(args);
try {
SkillScriptExecutionService.ScriptResult result =
executionService.executeCode(language, code, workingDir, argList, envVars, timeout);
return formatResult(result);
} catch (Exception e) {
log.error("[CodeExecute] Execution failed: {}", e.getMessage());
return formatError("Execution failed: " + e.getMessage());
}
}
/**
* Decode the JSON-encoded {@code args} string into a positional argument list,
* mirroring {@code SkillScriptTool.normalizeArgs}: a JSON array becomes one
* argument per element, anything else is forwarded verbatim as a single
* argument (so a bare date / version string is never mangled by JSON parsing).
*/
List<String> normalizeArgs(String args) {
if (args == null) {
return null;
}
String trimmed = args.trim();
if (trimmed.isEmpty()) {
return null;
}
char lead = trimmed.charAt(0);
if (lead == '[') {
try {
JsonNode node = objectMapper.reader()
.with(DeserializationFeature.FAIL_ON_TRAILING_TOKENS)
.readTree(trimmed);
if (node != null && node.isArray()) {
List<String> out = new ArrayList<>(node.size());
for (JsonNode el : node) {
out.add(el.isTextual() ? el.asText() : el.toString());
}
return out.isEmpty() ? null : out;
}
} catch (Exception e) {
log.debug("execute_code: args not valid JSON array, forwarding verbatim: {}", e.getMessage());
}
}
return List.of(trimmed);
}
private String formatResult(SkillScriptExecutionService.ScriptResult result) {
return String.format(
"{\n \"exitCode\": %d,\n \"stdout\": %s,\n \"stderr\": %s\n}",
result.getExitCode(),
jsonEscape(result.getStdout()),
jsonEscape(result.getStderr())
);
}
private String formatError(String message) {
return String.format(
"{\n \"exitCode\": -1,\n \"stdout\": \"\",\n \"stderr\": %s\n}",
jsonEscape(message)
);
}
private String jsonEscape(String str) {
if (str == null || str.isEmpty()) {
return "\"\"";
}
return "\"" + str.replace("\\", "\\\\")
.replace("\"", "\\\"")
.replace("\n", "\\n")
.replace("\r", "\\r")
.replace("\t", "\\t") + "\"";
}
}

View File

@ -1,8 +1,6 @@
package vip.mate.tool.builtin;
import cn.hutool.json.JSONArray;
import cn.hutool.json.JSONObject;
import cn.hutool.json.JSONUtil;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.chat.model.ToolContext;
@ -14,7 +12,10 @@ import vip.mate.agent.context.ChatOrigin;
import vip.mate.cron.model.CronJobDTO;
import vip.mate.cron.service.CronJobService;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
/**
* Built-in tool: scheduled task (cron job) management via chat.
@ -32,6 +33,14 @@ import java.util.List;
public class CronJobTool {
private final CronJobService cronJobService;
/**
* The application ObjectMapper serializes every {@code Long} as a JSON string
* (see {@code JacksonConfig}). Tool output goes through it so a 19-digit
* Snowflake {@code jobId} reaches the model as a string never a JSON number
* that loses its low digits in a double / JS Number round-trip on the way
* back into toggle_cron_job / delete_cron_job.
*/
private final ObjectMapper objectMapper;
@vip.mate.tool.ConcurrencyUnsafe("cron job creation persists to mate_cron_job; concurrent creates can race on name")
@Tool(description = "Create a scheduled task that asks the agent to do something at a specific time — "
@ -94,15 +103,15 @@ public class CronJobTool {
Long workspaceId = origin.workspaceId() != null ? origin.workspaceId() : 1L;
CronJobDTO created = cronJobService.create(dto, workspaceId);
JSONObject result = new JSONObject();
result.set("success", true);
result.set("jobId", created.getId());
result.set("name", created.getName());
result.set("cronExpression", created.getCronExpression());
result.set("timezone", created.getTimezone());
result.set("nextRunTime", created.getNextRunTime() != null ? created.getNextRunTime().toString() : "");
result.set("enabled", created.getEnabled());
return JSONUtil.toJsonPrettyStr(result);
Map<String, Object> result = new LinkedHashMap<>();
result.put("success", true);
result.put("jobId", created.getId());
result.put("name", created.getName());
result.put("cronExpression", created.getCronExpression());
result.put("timezone", created.getTimezone());
result.put("nextRunTime", created.getNextRunTime() != null ? created.getNextRunTime().toString() : "");
result.put("enabled", created.getEnabled());
return writeJson(result);
} catch (Exception e) {
log.error("[CronJobTool] create failed: {}", e.getMessage());
@ -154,16 +163,16 @@ public class CronJobTool {
Long workspaceId = origin.workspaceId() != null ? origin.workspaceId() : 1L;
CronJobDTO created = cronJobService.create(dto, workspaceId);
JSONObject result = new JSONObject();
result.set("success", true);
result.set("jobId", created.getId());
result.set("name", created.getName());
result.set("taskType", "reminder");
result.set("cronExpression", created.getCronExpression());
result.set("timezone", created.getTimezone());
result.set("nextRunTime", created.getNextRunTime() != null ? created.getNextRunTime().toString() : "");
result.set("enabled", created.getEnabled());
return JSONUtil.toJsonPrettyStr(result);
Map<String, Object> result = new LinkedHashMap<>();
result.put("success", true);
result.put("jobId", created.getId());
result.put("name", created.getName());
result.put("taskType", "reminder");
result.put("cronExpression", created.getCronExpression());
result.put("timezone", created.getTimezone());
result.put("nextRunTime", created.getNextRunTime() != null ? created.getNextRunTime().toString() : "");
result.put("enabled", created.getEnabled());
return writeJson(result);
} catch (Exception e) {
log.error("[CronJobTool] create_reminder failed: {}", e.getMessage());
@ -179,23 +188,23 @@ public class CronJobTool {
// sees the cron jobs of the workspace it's running in.
Long workspaceId = workspaceFromContext(ctx);
List<CronJobDTO> jobs = cronJobService.list(workspaceId);
JSONArray arr = new JSONArray();
List<Map<String, Object>> arr = new ArrayList<>();
for (CronJobDTO job : jobs) {
JSONObject obj = new JSONObject();
obj.set("jobId", job.getId());
obj.set("name", job.getName());
obj.set("cronExpression", job.getCronExpression());
obj.set("timezone", job.getTimezone());
obj.set("enabled", job.getEnabled());
obj.set("nextRunTime", job.getNextRunTime() != null ? job.getNextRunTime().toString() : "");
obj.set("lastRunTime", job.getLastRunTime() != null ? job.getLastRunTime().toString() : "");
obj.set("agentName", job.getAgentName());
Map<String, Object> obj = new LinkedHashMap<>();
obj.put("jobId", job.getId());
obj.put("name", job.getName());
obj.put("cronExpression", job.getCronExpression());
obj.put("timezone", job.getTimezone());
obj.put("enabled", job.getEnabled());
obj.put("nextRunTime", job.getNextRunTime() != null ? job.getNextRunTime().toString() : "");
obj.put("lastRunTime", job.getLastRunTime() != null ? job.getLastRunTime().toString() : "");
obj.put("agentName", job.getAgentName());
arr.add(obj);
}
JSONObject result = new JSONObject();
result.set("totalJobs", jobs.size());
result.set("jobs", arr);
return JSONUtil.toJsonPrettyStr(result);
Map<String, Object> result = new LinkedHashMap<>();
result.put("totalJobs", jobs.size());
result.put("jobs", arr);
return writeJson(result);
} catch (Exception e) {
log.error("[CronJobTool] list failed: {}", e.getMessage());
return errorResult("Failed to list cron jobs: " + e.getMessage());
@ -214,13 +223,13 @@ public class CronJobTool {
Long workspaceId = workspaceFromContext(ctx);
cronJobService.toggle(jobId, enabled, workspaceId);
CronJobDTO updated = cronJobService.getById(jobId, workspaceId);
JSONObject result = new JSONObject();
result.set("success", true);
result.set("jobId", jobId);
result.set("name", updated.getName());
result.set("enabled", updated.getEnabled());
result.set("nextRunTime", updated.getNextRunTime() != null ? updated.getNextRunTime().toString() : "");
return JSONUtil.toJsonPrettyStr(result);
Map<String, Object> result = new LinkedHashMap<>();
result.put("success", true);
result.put("jobId", jobId);
result.put("name", updated.getName());
result.put("enabled", updated.getEnabled());
result.put("nextRunTime", updated.getNextRunTime() != null ? updated.getNextRunTime().toString() : "");
return writeJson(result);
} catch (Exception e) {
log.error("[CronJobTool] toggle failed: {}", e.getMessage());
return errorResult("Failed to toggle cron job: " + e.getMessage());
@ -239,10 +248,10 @@ public class CronJobTool {
CronJobDTO job = cronJobService.getById(jobId, workspaceId);
String jobName = job.getName();
cronJobService.delete(jobId, workspaceId);
JSONObject result = new JSONObject();
result.set("success", true);
result.set("deleted", jobName);
return JSONUtil.toJsonPrettyStr(result);
Map<String, Object> result = new LinkedHashMap<>();
result.put("success", true);
result.put("deleted", jobName);
return writeJson(result);
} catch (Exception e) {
log.error("[CronJobTool] delete failed: {}", e.getMessage());
return errorResult("Failed to delete cron job: " + e.getMessage());
@ -250,10 +259,25 @@ public class CronJobTool {
}
private String errorResult(String message) {
JSONObject result = new JSONObject();
result.set("success", false);
result.set("error", message);
return JSONUtil.toJsonPrettyStr(result);
Map<String, Object> result = new LinkedHashMap<>();
result.put("success", false);
result.put("error", message);
return writeJson(result);
}
/**
* Serialize tool output through the id-safe application ObjectMapper so every
* {@code Long} (notably {@code jobId}) is rendered as a string. Falls back to
* a minimal literal on the rare serialization failure rather than throwing
* out of a tool call.
*/
private String writeJson(Object value) {
try {
return objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(value);
} catch (Exception e) {
log.error("[CronJobTool] result serialization failed: {}", e.getMessage());
return "{\"success\":false,\"error\":\"result serialization failed\"}";
}
}
/**

View File

@ -1,8 +1,8 @@
package vip.mate.tool.builtin;
import cn.hutool.json.JSONArray;
import cn.hutool.json.JSONObject;
import cn.hutool.json.JSONUtil;
import com.fasterxml.jackson.databind.ObjectMapper;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.ai.tool.annotation.Tool;
@ -14,7 +14,9 @@ import vip.mate.datasource.service.DatasourceService;
import java.sql.*;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Map;
import java.util.regex.Pattern;
/**
@ -32,6 +34,14 @@ public class DatasourceTool {
private final DatasourceService datasourceService;
private final DatasourceConnectionManager connectionManager;
/**
* The application ObjectMapper, which serializes every {@code Long} as a JSON
* string (see {@code JacksonConfig}). Tool output goes through it so 19-digit
* Snowflake ids reach the model as strings exactly like the HTTP API and
* never as JSON numbers that lose their low digits in a double/JS-number
* round-trip on the way back into a tool call.
*/
private final ObjectMapper objectMapper;
/** SQL identifier whitelist: letters, digits, underscore, dot, hyphen only */
private static final Pattern SAFE_IDENTIFIER = Pattern.compile("^[a-zA-Z0-9_][a-zA-Z0-9_.\\-]{0,127}$");
@ -67,22 +77,25 @@ public class DatasourceTool {
}
}
private String listDatasources() {
private String listDatasources() throws Exception {
List<DatasourceEntity> list = datasourceService.listEnabled();
JSONArray arr = new JSONArray();
List<Map<String, Object>> rows = new ArrayList<>();
for (DatasourceEntity ds : list) {
JSONObject obj = new JSONObject();
obj.set("id", ds.getId());
obj.set("name", ds.getName());
obj.set("dbType", ds.getDbType());
obj.set("databaseName", ds.getDatabaseName());
obj.set("description", ds.getDescription());
arr.add(obj);
Map<String, Object> obj = new LinkedHashMap<>();
// ds.getId() is a Long; the shared ObjectMapper renders it as a JSON
// string so the model copies an exact id back into list_tables /
// execute_sql / describe_table calls.
obj.put("id", ds.getId());
obj.put("name", ds.getName());
obj.put("dbType", ds.getDbType());
obj.put("databaseName", ds.getDatabaseName());
obj.put("description", ds.getDescription());
rows.add(obj);
}
JSONObject result = new JSONObject();
result.set("datasources", arr);
result.set("count", arr.size());
return result.toStringPretty();
Map<String, Object> result = new LinkedHashMap<>();
result.put("datasources", rows);
result.put("count", rows.size());
return objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(result);
}
private String listTables(Long datasourceId) throws SQLException {

Some files were not shown because too many files have changed in this diff Show More