diff --git a/UPGRADING.md b/UPGRADING.md deleted file mode 100644 index 5fed302d..00000000 --- a/UPGRADING.md +++ /dev/null @@ -1,159 +0,0 @@ -# Upgrading MateClaw - -## 1.0.x → 1.1.0 - -**TL;DR** — Most users have nothing to do. Restart with 1.1.0, Flyway's built-in repair heals known checksum drift, Ollama auto-discovery rewrites the bad `:latest` defaults, and everything else self-converges. Docker Compose deployments need a one-time `.env` update. - -See `docs/en/releases/1.1.0.md` for the feature changelog. - ---- - -## For everyone - -### ⚠️ What happens automatically (no action) - -- **Flyway migration self-heal** — 1.1.0 rewrote all MySQL migrations V2–V14 to replace unsupported `ADD COLUMN IF NOT EXISTS` syntax (Gitee #IIYHLJ). `FlywayRepairConfig` runs `flyway.repair()` on every boot, so the new checksums auto-accept and migration resumes from wherever your schema is. -- **Ollama default model** — if your 1.0.x run auto-picked a model tag Ollama no longer has (commonly `deepseek-r1:latest`), on 1.1.0 restart `OllamaAutoDiscoveryRunner` detects the broken default and re-picks a tag-capable model (e.g. `deepseek-r1:7b`, `qwen3:latest`), preferring one that supports function calling. -- **Stale `mate_model_config` rows** — idempotent seed data reconciles on each startup. - -### 📋 Recommended pre-upgrade steps - -1. Back up your database — `mateclaw` schema on MySQL, or `data/mateclaw.mv.db` on H2. -2. Back up `data/` directory (skill workspaces, uploaded files, memory files). -3. Note your current default model in Settings → Models in case you want to switch back. - -### 🚀 Upgrade - -```bash -git pull -cd mateclaw-server -mvn clean package -DskipTests -# then restart your service per your deployment method -``` - -Or for Desktop app users: just update to 1.1.0 via the in-app updater or re-download. - ---- - -## For Docker Compose deployments - -**One-time migration step required** — 1.1.0 refuses to start with default hardcoded passwords. - -### 1. Copy-paste merge the new `.env.example` keys - -```bash -cp .env .env.backup -# open .env.example — it has new required keys: -# DB_PASSWORD= (was default 'mateclaw123', now MUST be overridden) -# DB_ROOT_PASSWORD= (new, required for MySQL root) -# JWT_SECRET= (new, strongly recommended) -# MATECLAW_CORS_ALLOWED_ORIGINS= (new, strongly recommended for prod) -``` - -### 2. Set strong values in your `.env` - -```env -# STRONG passwords — at least 16 chars, mixed case + digits + symbols -DB_PASSWORD= -DB_ROOT_PASSWORD= - -# 32+ char random string — generate with: openssl rand -base64 48 -JWT_SECRET= - -# Production CORS allowlist — comma-separated, no wildcards -MATECLAW_CORS_ALLOWED_ORIGINS=https://mateclaw.example.com -``` - -If any of `DB_PASSWORD` / `DB_ROOT_PASSWORD` / `DASHSCOPE_API_KEY` is missing, `docker compose up` will fail fast with a clear error — this is intentional. - -### 3. Existing MySQL volume compatibility - -If you already ran 1.0.x with the old default password (`mateclaw123`), **your existing MySQL volume still has the old root password inside**. You have two options: - -**Option A — keep existing password** (fastest, least secure): -Set `DB_ROOT_PASSWORD=mateclaw123` and `DB_PASSWORD=mateclaw123` in `.env` to match. Upgrade works. Then rotate after upgrade using `ALTER USER ... IDENTIFIED BY ...` inside the MySQL container. - -**Option B — fresh volume with new password** (cleanest, loses DB if not backed up): -```bash -docker compose down -v # ⚠️ deletes mysql_data volume; back up first -# edit .env with new strong password -docker compose up -d -``` -Then re-import your backup if you kept one. - -### 4. Restart - -```bash -docker compose up -d -docker compose logs -f mateclaw-server # watch for "Flyway Successfully applied N migrations" -``` - -Expected log lines during boot: -- `Flyway Successfully applied N migrations to schema mateclaw` -- `Ollama: auto-activated default model ''` (if you use Ollama — should NOT say `:latest` any more) -- `[Security] Using default JWT secret!` → means you forgot to set `JWT_SECRET` — fix and restart - ---- - -## For local dev / H2 deployments - -No action required. `mvn spring-boot:run` picks up the latest migrations on next start, Flyway repair handles checksum drift, H2 file at `data/mateclaw.mv.db` is preserved. - ---- - -## Known migration quirks - -### 1. If you manually fiddled with `flyway_schema_history` - -In 1.0.x some users hit Flyway version collisions (V8/V9 and V9/V10) which 1.1.0 fixes by renumbering. If you manually deleted rows from `flyway_schema_history` you may see `Validate failed` on 1.1.0 startup — run: - -```sql --- MySQL -DELETE FROM flyway_schema_history WHERE success = 0; -``` - -Then restart. `FlywayRepairConfig` will rebuild history from current schema state. - -### 2. If your Ollama models are all in the no-tools family - -After upgrade, agents that require tool calling will log a warning on first invocation: - -``` -Ollama: auto-activated default model '...' but its family does not support tool calling -``` - -Fix — pull a tool-capable model, or switch default in Settings → Models: - -```bash -ollama pull qwen3 -# or -ollama pull llama3.1:8b -# or -ollama pull mistral-nemo -``` - -### 3. If you had custom tools using `extract_document_text` / wiki tools - -Wiki chunk schema changed (new `embedding` + `embedding_model` columns on `mate_wiki_chunk`). Your existing wiki pages work unchanged; only semantic search is new and requires an embedding model to be configured in Settings → Models (a default DashScope embedding is seeded). - ---- - -## Rolling back to 1.0.x - -Not recommended (some new tables / columns don't exist in 1.0.x), but possible if you backed up the DB before upgrade: - -```bash -git checkout v1.0.418 -# restore DB backup -docker compose up -d # or mvn spring-boot:run -``` - -If you need to keep the new data but downgrade the app, you're in unsupported territory — open a Gitee issue. - ---- - -## Getting help - -- **Logs first**: `mateclaw-server/logs/mateclaw.log` + `mateclaw-error.log` have everything. Flyway decisions are at INFO level in main log. -- **Doctor tab**: in-app Settings → Doctor runs basic health checks -- **Gitee**: https://gitee.com/matevip_admin/mateclaw/issues — include your upgrade path (1.0.?? → 1.1.0), profile (H2 / MySQL), and the last 100 lines of startup log diff --git a/docs/compatibility/feishu-websocket-lifecycle.md b/docs/compatibility/feishu-websocket-lifecycle.md new file mode 100644 index 00000000..f1bd9321 --- /dev/null +++ b/docs/compatibility/feishu-websocket-lifecycle.md @@ -0,0 +1,52 @@ +# Feishu WebSocket lifecycle (oapi-sdk 2.7.1) + +The adapter owns reconnect scheduling. SDK `autoReconnect` stays false. SDK +`start()` initiates a handshake and returns; only `awaitReady` success makes the +adapter CONNECTED. Startup failures and actual socket failures enter the existing +single exponential-backoff loop. A 15-second probe observes the SDK socket; +OkHttp protocol ping/pong (20 seconds) detects half-open transports without +business messages. A quiet channel does not imply a broken connection. + +`FeishuWebSocketClient` is a deliberately narrow, version-bound compatibility +bridge. SDK 2.7.1 has a private constructor and no public resource-disposal or +transport-status APIs. Its `close()` does not shut down the executor. The bridge +validates `executor`, `httpClient`, and `conn` field types. Reads of the non-volatile +`conn` field acquire the SDK monitor used by `disconnect()`. It configures the dedicated +HTTP client before use, and releases SDK/HTTP executors and idle connections on +permanent close. It cancels the underlying socket after SDK close (which only +enqueues a graceful close), outside the SDK monitor to allow failure callbacks. +It cancels HTTP calls and interrupts the starter before taking +the lifecycle lock, so a pending handshake cannot prevent shutdown. Do not remove +these contract tests or upgrade the SDK without rechecking its lifecycle. + +Feishu opts out of message-age staleness with Duration.ZERO. Other adapters retain +their existing thresholds. +ERROR recovery remains active; long-poll adapters retain their existing positive +thresholds. The legacy Feishu `silent_disconnect_threshold_seconds` setting no +longer triggers reconnects. A transport reconnect reuses the HTTP token client; +token refresh already has its own schedule. Permanent stop shuts down that client. + +Regression tests use a loopback-only HTTP/WebSocket peer and synthetic credentials, +with no model calls or live Feishu messages. They cover delayed/rejected handshakes, +TCP resets, peer close, missing pongs, stop during startup, close-before-start, +close between handshake and success, and 25 real SDK start/stop cycles. Each old +SDK and HTTP executor must terminate; quiet channels, ERROR cooldown, and Weixin +five-minute stale recovery are verified separately. + +## Validation + +Run the lifecycle and existing channel regressions on Java 21: + +```sh +JAVA_TOOL_OPTIONS=-Djdk.httpclient.allowRestrictedHeaders=connection \ + mvn -pl mateclaw-server -am \ + -Dtest='Feishu*Test,ChannelHealthMonitor*Test,ChannelAdapterStalenessTest' \ + -Dsurefire.failIfNoSpecifiedTests=false test +``` + +The JVM option matches the application startup HTTP setting; the broader existing +Feishu HTTP tests also run outside the application main method. The lifecycle +tests do not need credentials or Internet access. The native SDK +resource bridge is scoped to the pinned oapi-sdk 2.7.1; review the reflected field +contracts before changing that dependency. Local socket tests do not establish +real-user delivery or multi-day reliability. diff --git a/docs/fix-duplicate-raws.sql b/docs/fix-duplicate-raws.sql new file mode 100644 index 00000000..9e420007 --- /dev/null +++ b/docs/fix-duplicate-raws.sql @@ -0,0 +1,117 @@ +-- ============================================================ +-- 修复：知识库原始材料重复入库 +-- 适用：MySQL 8.0+（使用 JSON 函数处理 source_raw_ids） +-- 说明：同一 (kb_id, source_path) 可能因文件内容变更 +-- 被多次 INSERT 而形成多行。本脚本保留最新行， +-- 并级联清理其关联的 chunk、citation、page。 +-- ============================================================ + +-- ────────────────────────────────────────────────────────── +-- STEP 0：预览（只读，不改数据，先跑这一步确认影响范围） +-- ────────────────────────────────────────────────────────── + +-- 0-A：查看所有重复组（按 kb_id + source_path 分组，count > 1） +SELECT + kb_id, + source_path, + COUNT(*) AS duplicate_count, + MAX(id) AS keep_id, + GROUP_CONCAT(id ORDER BY id DESC) AS all_ids +FROM mate_wiki_raw_material +WHERE source_path IS NOT NULL +GROUP BY kb_id, source_path +HAVING COUNT(*) > 1; + +-- 0-B：查看待删除的具体行（排除每组最新的那一行） +SELECT + r.id, r.kb_id, r.source_path, + r.content_hash, r.processing_status, r.create_time +FROM mate_wiki_raw_material r +WHERE r.source_path IS NOT NULL + AND r.id NOT IN ( + SELECT MAX(id) + FROM mate_wiki_raw_material + WHERE source_path IS NOT NULL + GROUP BY kb_id, source_path + ) +ORDER BY r.kb_id, r.source_path, r.id; + + +-- ────────────────────────────────────────────────────────── +-- STEP 1：开事务，执行清理（确认 STEP 0 结果后再运行） +-- ────────────────────────────────────────────────────────── + +START TRANSACTION; + +-- 1-A：把待删除的 raw id 暂存到临时表，后续步骤复用 +CREATE TEMPORARY TABLE IF NOT EXISTS _stale_raw_ids AS +SELECT id AS raw_id, kb_id +FROM mate_wiki_raw_material +WHERE source_path IS NOT NULL + AND id NOT IN ( + SELECT MAX(id) + FROM mate_wiki_raw_material + WHERE source_path IS NOT NULL + GROUP BY kb_id, source_path + ); + +-- 1-B：删除这些 raw 产生的 citation（通过 chunk_id 关联） +DELETE c +FROM mate_wiki_page_citation c + INNER JOIN mate_wiki_chunk ch ON c.chunk_id = ch.id + INNER JOIN _stale_raw_ids s ON ch.raw_id = s.raw_id; + +-- 1-C：删除 chunk +DELETE ch +FROM mate_wiki_chunk ch + INNER JOIN _stale_raw_ids s ON ch.raw_id = s.raw_id; + +-- 1-D：删除仅由该 raw 派生的 page（source_raw_ids 数组长度为 1） +-- 使用 JSON_CONTAINS 判断 page 是否引用了待删 raw +DELETE p +FROM mate_wiki_page p +WHERE JSON_LENGTH(p.source_raw_ids) = 1 + AND EXISTS ( + SELECT 1 + FROM _stale_raw_ids s + WHERE JSON_CONTAINS(p.source_raw_ids, CAST(s.raw_id AS CHAR)) + ); + +-- 1-E：对多来源 page，将待删 raw 从 source_raw_ids 中移除 +-- 通过 JSON_TABLE 把数组展开再重组，排除掉 stale raw id +UPDATE mate_wiki_page p +SET p.source_raw_ids = ( + SELECT JSON_ARRAYAGG(jt.v) + FROM JSON_TABLE(p.source_raw_ids, '$[*]' COLUMNS (v BIGINT PATH '$')) jt + WHERE jt.v NOT IN (SELECT raw_id FROM _stale_raw_ids) +) +WHERE JSON_LENGTH(p.source_raw_ids) > 1 + AND EXISTS ( + SELECT 1 + FROM _stale_raw_ids s + WHERE JSON_CONTAINS(p.source_raw_ids, CAST(s.raw_id AS CHAR)) + ); + +-- 1-F：删除 stale raw 行 +DELETE r +FROM mate_wiki_raw_material r + INNER JOIN _stale_raw_ids s ON r.id = s.raw_id; + +-- 1-G：确认结果 +SELECT + 'stale raws deleted' AS action, + ROW_COUNT() AS affected_rows; + +SELECT + 'remaining duplicates' AS check_item, + COUNT(*) AS count +FROM mate_wiki_raw_material +WHERE source_path IS NOT NULL +GROUP BY kb_id, source_path +HAVING COUNT(*) > 1; + +-- 确认无误后提交；如有问题改为 ROLLBACK +COMMIT; +-- ROLLBACK; + +DROP TEMPORARY TABLE IF EXISTS _stale_raw_ids; diff --git a/docs/superpowers/plans/2026-07-03-plugin-search-provider-pr1.md b/docs/superpowers/plans/2026-07-03-plugin-search-provider-pr1.md new file mode 100644 index 00000000..5841e2df --- /dev/null +++ b/docs/superpowers/plans/2026-07-03-plugin-search-provider-pr1.md @@ -0,0 +1,1288 @@ +# 插件化搜索 Provider（PR-1：SDK + 桥接 + Registry）实施计划 + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** 第三方以独立 jar 实现 `PluginSearchProvider` 并丢进插件目录，即可为 `web_search` 工具新增搜索源，mateclaw-server 源码零改动。 + +**Architecture:** 在 `mateclaw-plugin-api` 新增 SEARCH 插件类型与自包含 SDK 接口；server 侧用 `PluginSearchBridge` 把插件接口适配成核心 `SearchProvider`，`SearchProviderRegistry` 从 immutable 改为"Spring bean 基底 + 插件区合并视图"；生命周期（注册/disable 反注册/加载失败 rollback）与现有 TOOL/CHANNEL/MEMORY/PROVIDER 四类完全对称。 + +**Tech Stack:** Java 21 / Spring Boot 3.5 / Maven 多模块 reactor（`mateclaw-plugin-api` → `mateclaw-server` → samples）/ JUnit 5 + Mockito。 + +**上游 issue:** https://github.com/mateaix/mateclaw/issues/477（已获认可后动工） +**设计文档:** `docs/superpowers/specs/2026-07-03-plugin-search-provider-design.md` + +**背景速览（给零上下文的执行者）：** +- 核心接口 `vip.mate.tool.search.SearchProvider`（server 模块）：`id()/label()/requiresCredential()/autoDetectOrder()/isAvailable(SystemSettingsDTO)/search(SearchQuery, SystemSettingsDTO)`。内置 4 个实现（serper/tavily/searxng/duckduckgo）是 `@Component`。 +- `SearchProviderRegistry`（`mateclaw-server/src/main/java/vip/mate/tool/search/SearchProviderRegistry.java`）构造器注入 `List`，`resolve(config)` 三分支：显式配置 → 按 `autoDetectOrder` 找有 credential 的 → keyless 兜底。 +- 插件系统：`PluginManager`（`@RequiredArgsConstructor`，字段即依赖）从文件系统加载 jar（`URLClassLoader`），`PluginContextImpl` 是插件看到的平台 API，`LoadedPlugin` 记录插件注册了什么以便 disable 时反注册。 +- 构建命令均从仓库根执行；server 单测在 `mateclaw-server/` 下执行。 + +--- + +### Task 1: 建分支并提交设计文档 + +**Files:** +- 已存在（未提交）: `docs/superpowers/specs/2026-07-03-plugin-search-provider-design.md` +- 本文件: `docs/superpowers/plans/2026-07-03-plugin-search-provider-pr1.md` + +- [ ] **Step 1: 从 dev 建分支** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw +git fetch upstream dev 2>/dev/null || git fetch origin dev +git checkout -b feat/plugin-search-provider $(git rev-parse --verify upstream/dev 2>/dev/null || git rev-parse origin/dev) +``` + +预期：新分支 `feat/plugin-search-provider`，基于最新 dev。 + +- [ ] **Step 2: 提交 spec 与 plan** + +```bash +git add docs/superpowers/ +git commit -m "docs: add plugin search provider design spec and plan (#477)" +``` + +--- + +### Task 2: SDK — PluginType.SEARCH + search 包（纯新增，无行为变化） + +**Files:** +- Modify: `mateclaw-plugin-api/src/main/java/vip/mate/plugin/api/PluginType.java` +- Create: `mateclaw-plugin-api/src/main/java/vip/mate/plugin/api/search/PluginSearchQuery.java` +- Create: `mateclaw-plugin-api/src/main/java/vip/mate/plugin/api/search/PluginSearchResult.java` +- Create: `mateclaw-plugin-api/src/main/java/vip/mate/plugin/api/search/PluginSearchProvider.java` + +注意：**本任务不改 `PluginContext`**（接口加方法会导致 server 模块的 `PluginContextImpl` 编译失败；那一步放在 Task 5 与实现同 commit，保证每个 commit 可编译）。 + +- [ ] **Step 1: PluginType 加枚举值** + +`PluginType.java` 的 `MEMORY` 之后追加： + +```java + /** Register new memory providers */ + MEMORY, + + /** Register new web-search providers for the web_search tool */ + SEARCH +``` + +（`PluginManifest.validate()` 用 `PluginType.valueOf(type.toUpperCase())` 校验，枚举加值后 manifest `"type": "search"` 自动合法，无需改 validate。） + +- [ ] **Step 2: 新建 PluginSearchQuery** + +```java +package vip.mate.plugin.api.search; + +/** + * Search query passed from the platform to a plugin search provider. + *

+ * Self-contained SDK type — must not depend on any mateclaw-server class, + * because plugin JARs are compiled only against mateclaw-plugin-api. + * + * @param query search keywords (never null/blank) + * @param freshness time-range filter: day / week / month / year (nullable) + * @param language language preference, e.g. zh-CN / en (nullable) + * @param count max results 1-10, already clamped by the platform (never null) + * + * @author MateClaw Team + */ +public record PluginSearchQuery( + String query, + String freshness, + String language, + Integer count +) { +} +``` + +- [ ] **Step 3: 新建 PluginSearchResult** + +```java +package vip.mate.plugin.api.search; + +/** + * A single search result returned by a plugin search provider. + *

+ * Self-contained SDK type — mirrors the platform's internal SearchResult + * (title/url/snippet/source/date) without depending on server classes. + * + * @param title result title + * @param url result link + * @param snippet short excerpt + * @param source source domain, e.g. "reuters.com" (nullable) + * @param date published date as raw string (nullable) + * + * @author MateClaw Team + */ +public record PluginSearchResult( + String title, + String url, + String snippet, + String source, + String date +) { +} +``` + +- [ ] **Step 4: 新建 PluginSearchProvider** + +```java +package vip.mate.plugin.api.search; + +import java.util.List; + +/** + * SPI for plugin-provided web-search providers. + *

+ * Implementations are registered via {@code PluginContext#registerSearchProvider} + * and appear in the platform's search provider chain alongside the built-in + * providers (serper / tavily / searxng / duckduckgo). + *

+ * Configuration (API keys, base URLs, ...) is NOT passed in — plugins read their + * own config declared in {@code mateclaw-plugin.json} via + * {@code PluginContext#getConfig(String, Class)}. + * + * @author MateClaw Team + */ +public interface PluginSearchProvider { + + /** Globally unique provider id, e.g. "my-search". Must not clash with built-in ids. */ + String id(); + + /** Human-readable display name. */ + String label(); + + /** Whether this provider needs a credential (affects auto-detect priority). */ + default boolean requiresCredential() { + return true; + } + + /** + * Auto-detect ordering (ascending). Built-in providers occupy 50-400; + * plugin providers default to 500 (after built-ins) but may override. + */ + default int autoDetectOrder() { + return 500; + } + + /** + * Whether the provider is currently usable — typically: required config present. + * Called on every provider resolution; keep it cheap (no network I/O). + */ + boolean isAvailable(); + + /** + * Execute the search. + * + * @param query the query (never null) + * @return results; empty list if nothing found. Must not return null. + * Throw on failure — the platform falls back to the next provider. + */ + List search(PluginSearchQuery query); +} +``` + +- [ ] **Step 5: 编译 plugin-api 并安装到本地仓库** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw +mvn -q -pl mateclaw-plugin-api install -DskipTests +``` + +预期：BUILD SUCCESS。 + +- [ ] **Step 6: Commit** + +```bash +git add mateclaw-plugin-api/ +git commit -m "feat(plugin-api): add SEARCH plugin type and PluginSearchProvider SPI (#477)" +``` + +--- + +### Task 3: SearchProviderRegistry 可变化（TDD） + +**Files:** +- Test: `mateclaw-server/src/test/java/vip/mate/tool/search/SearchProviderRegistryPluginTest.java`（新建，目录也新建） +- Modify: `mateclaw-server/src/main/java/vip/mate/tool/search/SearchProviderRegistry.java` + +- [ ] **Step 1: 写失败测试** + +```java +package vip.mate.tool.search; + +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import vip.mate.system.model.SystemSettingsDTO; + +import java.util.List; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertSame; +import static org.junit.jupiter.api.Assertions.assertThrows; + +/** + * Plugin-provider mutability of {@link SearchProviderRegistry} (issue #477): + * plugin JARs register/unregister providers at runtime; the registry must merge + * them with the Spring-injected built-ins and reject id conflicts. + */ +class SearchProviderRegistryPluginTest { + + /** Minimal stub standing in for both built-in and plugin-bridged providers. */ + private static SearchProvider stub(String id, int order, boolean credentialed, boolean available) { + return new SearchProvider() { + @Override public String id() { return id; } + @Override public String label() { return id; } + @Override public boolean requiresCredential() { return credentialed; } + @Override public int autoDetectOrder() { return order; } + @Override public boolean isAvailable(SystemSettingsDTO config) { return available; } + @Override public List search(String query, SystemSettingsDTO config) { return List.of(); } + }; + } + + private static SearchProviderRegistry registryWithBuiltins() { + // Mirrors the real built-in landscape: one credentialed, one keyless. + return new SearchProviderRegistry(List.of( + stub("serper", 300, true, false), // credentialed but NOT configured + stub("duckduckgo", 100, false, true) // keyless, available + )); + } + + @Test + @DisplayName("registered plugin provider shows up in allSorted, ordered by autoDetectOrder") + void pluginProviderAppearsInMergedSortedView() { + SearchProviderRegistry registry = registryWithBuiltins(); + SearchProvider plugin = stub("my-search", 500, true, true); + + registry.registerPluginProvider(plugin); + + List all = registry.allSorted(); + assertEquals(3, all.size()); + assertEquals("duckduckgo", all.get(0).id()); // order 100 + assertEquals("serper", all.get(1).id()); // order 300 + assertSame(plugin, all.get(2)); // order 500 + } + + @Test + @DisplayName("getById finds plugin providers") + void getByIdFindsPluginProvider() { + SearchProviderRegistry registry = registryWithBuiltins(); + SearchProvider plugin = stub("my-search", 500, true, true); + registry.registerPluginProvider(plugin); + + assertSame(plugin, registry.getById("my-search")); + } + + @Test + @DisplayName("plugin id clashing with a built-in id is rejected") + void builtinIdConflictRejected() { + SearchProviderRegistry registry = registryWithBuiltins(); + + assertThrows(IllegalArgumentException.class, + () -> registry.registerPluginProvider(stub("serper", 500, true, true))); + } + + @Test + @DisplayName("plugin id clashing with an already-registered plugin id is rejected") + void pluginIdConflictRejected() { + SearchProviderRegistry registry = registryWithBuiltins(); + registry.registerPluginProvider(stub("my-search", 500, true, true)); + + assertThrows(IllegalArgumentException.class, + () -> registry.registerPluginProvider(stub("my-search", 501, true, true))); + } + + @Test + @DisplayName("resolve honours an explicitly configured plugin provider") + void resolvePicksConfiguredPluginProvider() { + SearchProviderRegistry registry = registryWithBuiltins(); + SearchProvider plugin = stub("my-search", 500, true, true); + registry.registerPluginProvider(plugin); + + SystemSettingsDTO config = new SystemSettingsDTO(); + config.setSearchProvider("my-search"); + + SearchProviderRegistry.ResolvedProvider resolved = registry.resolve(config); + assertSame(plugin, resolved.provider()); + assertEquals("configured", resolved.source()); + } + + @Test + @DisplayName("resolve auto-detects an available credentialed plugin provider") + void resolveAutoDetectsPluginProvider() { + SearchProviderRegistry registry = registryWithBuiltins(); + SearchProvider plugin = stub("my-search", 500, true, true); + registry.registerPluginProvider(plugin); + + // No explicit provider configured; serper (credentialed) is unavailable, + // so auto-detect must reach the plugin provider before keyless fallback. + SearchProviderRegistry.ResolvedProvider resolved = registry.resolve(new SystemSettingsDTO()); + assertSame(plugin, resolved.provider()); + assertEquals("auto-detect", resolved.source()); + } + + @Test + @DisplayName("after unregister, an explicitly configured plugin id falls back to auto-detect") + void unregisteredConfiguredProviderFallsBackToAutoDetect() { + SearchProviderRegistry registry = registryWithBuiltins(); + registry.registerPluginProvider(stub("my-search", 500, true, true)); + registry.unregisterPluginProvider("my-search"); + + assertNull(registry.getById("my-search")); + + SystemSettingsDTO config = new SystemSettingsDTO(); + config.setSearchProvider("my-search"); + SearchProviderRegistry.ResolvedProvider resolved = registry.resolve(config); + // Plugin gone; keyless duckduckgo is the only available provider left. + assertEquals("duckduckgo", resolved.provider().id()); + assertEquals("keyless-fallback", resolved.source()); + } +} +``` + +- [ ] **Step 2: 跑测试确认失败** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw/mateclaw-server +mvn test -Dtest=SearchProviderRegistryPluginTest +``` + +预期：编译失败，`registerPluginProvider`/`unregisterPluginProvider` 方法不存在。 + +- [ ] **Step 3: 修改 SearchProviderRegistry** + +对 `SearchProviderRegistry.java` 做如下修改（保留现有 javadoc 与日志，只列出变化）： + +新增 import： + +```java +import java.util.ArrayList; +import java.util.concurrent.ConcurrentHashMap; +``` + +新增字段（`providerMap` 之后）： + +```java + /** 插件注册的 provider（运行时可变），与 Spring 注入的内置 provider 合并成完整视图 */ + private final ConcurrentHashMap pluginProviders = new ConcurrentHashMap<>(); +``` + +新增方法（`getById` 之前）： + +```java + /** + * 注册一个插件提供的 provider（issue #477）。 + * + * @throws IllegalArgumentException id 为空，或与内置/已注册插件 provider 冲突 + */ + public void registerPluginProvider(SearchProvider provider) { + String id = provider.id(); + if (id == null || id.isBlank()) { + throw new IllegalArgumentException("Search provider id must not be blank"); + } + if (providerMap.containsKey(id)) { + throw new IllegalArgumentException( + "Search provider id conflicts with a built-in provider: " + id); + } + if (pluginProviders.putIfAbsent(id, provider) != null) { + throw new IllegalArgumentException( + "Search provider id already registered by another plugin: " + id); + } + log.info("插件搜索提供商已注册: {} (order={})", id, provider.autoDetectOrder()); + } + + /** 反注册插件 provider（disable / rollback 路径调用；id 不存在时静默） */ + public void unregisterPluginProvider(String id) { + if (pluginProviders.remove(id) != null) { + log.info("插件搜索提供商已反注册: {}", id); + } + } +``` + +`getById` 改为查合并视图： + +```java + /** 按 ID 获取指定 provider（内置优先，其次插件注册区） */ + public SearchProvider getById(String id) { + SearchProvider builtin = providerMap.get(id); + return builtin != null ? builtin : pluginProviders.get(id); + } +``` + +`allSorted` 改为合并视图（provider 总数 <10，每次读时排序无性能顾虑）： + +```java + /** 获取按 autoDetectOrder 排序的全部 provider（内置 + 插件） */ + public List allSorted() { + if (pluginProviders.isEmpty()) { + return sortedProviders; + } + List merged = new ArrayList<>(sortedProviders); + merged.addAll(pluginProviders.values()); + merged.sort(Comparator.comparingInt(SearchProvider::autoDetectOrder)); + return merged; + } +``` + +`resolve` 内部两处改为走合并视图：第 1 分支 `providerMap.get(configuredId)` → `getById(configuredId)`；第 2 分支 `for (SearchProvider p : sortedProviders)` → `for (SearchProvider p : allSorted())`。 + +- [ ] **Step 4: 跑测试确认通过** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw/mateclaw-server +mvn test -Dtest=SearchProviderRegistryPluginTest +``` + +预期：7 个测试全 PASS。 + +- [ ] **Step 5: Commit** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw +git add mateclaw-server/src/main/java/vip/mate/tool/search/SearchProviderRegistry.java \ + mateclaw-server/src/test/java/vip/mate/tool/search/SearchProviderRegistryPluginTest.java +git commit -m "feat(search): make SearchProviderRegistry accept runtime plugin providers (#477)" +``` + +--- + +### Task 4: PluginSearchBridge（TDD） + +**Files:** +- Test: `mateclaw-server/src/test/java/vip/mate/plugin/bridge/PluginSearchBridgeTest.java`（新建，目录也新建） +- Create: `mateclaw-server/src/main/java/vip/mate/plugin/bridge/PluginSearchBridge.java` + +- [ ] **Step 1: 写失败测试** + +```java +package vip.mate.plugin.bridge; + +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import vip.mate.plugin.api.search.PluginSearchProvider; +import vip.mate.plugin.api.search.PluginSearchQuery; +import vip.mate.plugin.api.search.PluginSearchResult; +import vip.mate.system.model.SystemSettingsDTO; +import vip.mate.tool.search.SearchQuery; +import vip.mate.tool.search.SearchResult; + +import java.util.List; +import java.util.concurrent.atomic.AtomicReference; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * {@link PluginSearchBridge} adapts the self-contained plugin SPI + * ({@code PluginSearchProvider}) to the platform's {@code SearchProvider} + * without leaking server types into plugin land. + */ +class PluginSearchBridgeTest { + + @Test + @DisplayName("query fields pass through and results are converted with the plugin's providerId") + void convertsQueryAndResults() { + AtomicReference received = new AtomicReference<>(); + PluginSearchProvider plugin = new PluginSearchProvider() { + @Override public String id() { return "my-search"; } + @Override public String label() { return "My Search"; } + @Override public boolean isAvailable() { return true; } + @Override public List search(PluginSearchQuery query) { + received.set(query); + return List.of(new PluginSearchResult( + "T1", "https://example.com/a", "snippet-1", "example.com", "2026-07-01")); + } + }; + + PluginSearchBridge bridge = new PluginSearchBridge(plugin); + List results = bridge.search( + new SearchQuery("kw", "week", "zh-CN", 3), new SystemSettingsDTO()); + + assertEquals("kw", received.get().query()); + assertEquals("week", received.get().freshness()); + assertEquals("zh-CN", received.get().language()); + assertEquals(3, received.get().count()); + + assertEquals(1, results.size()); + SearchResult r = results.get(0); + assertEquals("T1", r.getTitle()); + assertEquals("https://example.com/a", r.getUrl()); + assertEquals("snippet-1", r.getSnippet()); + assertEquals("example.com", r.getSource()); + assertEquals("2026-07-01", r.getDate()); + assertEquals("my-search", r.getProviderId()); + } + + @Test + @DisplayName("count is clamped via SearchQuery.resolvedCount before reaching the plugin") + void countIsClamped() { + AtomicReference received = new AtomicReference<>(); + PluginSearchBridge bridge = new PluginSearchBridge(stub(q -> { + received.set(q); + return List.of(); + })); + + bridge.search(new SearchQuery("kw", null, null, 99), new SystemSettingsDTO()); + assertEquals(10, received.get().count()); // MAX_COUNT + + bridge.search(new SearchQuery("kw", null, null, null), new SystemSettingsDTO()); + assertEquals(5, received.get().count()); // DEFAULT_COUNT + } + + @Test + @DisplayName("delegates id/label/order/credential and maps isAvailable() ignoring the DTO") + void delegatesMetadata() { + PluginSearchBridge bridge = new PluginSearchBridge(stub(q -> List.of())); + assertEquals("stub-search", bridge.id()); + assertEquals("Stub Search", bridge.label()); + assertTrue(bridge.requiresCredential()); + assertEquals(500, bridge.autoDetectOrder()); + assertTrue(bridge.isAvailable(new SystemSettingsDTO())); + } + + @Test + @DisplayName("a null result list from a sloppy plugin is normalised to empty") + void nullResultListNormalised() { + PluginSearchBridge bridge = new PluginSearchBridge(stub(q -> null)); + List results = bridge.search(SearchQuery.of("kw"), new SystemSettingsDTO()); + assertTrue(results.isEmpty()); + } + + @Test + @DisplayName("plugin exceptions propagate so WebSearchService's fallback chain can react") + void exceptionsPropagate() { + PluginSearchBridge bridge = new PluginSearchBridge(stub(q -> { + throw new IllegalStateException("plugin boom"); + })); + assertThrows(IllegalStateException.class, + () -> bridge.search(SearchQuery.of("kw"), new SystemSettingsDTO())); + } + + // ---- helpers ---- + + private interface SearchFn { + List apply(PluginSearchQuery q); + } + + private static PluginSearchProvider stub(SearchFn fn) { + return new PluginSearchProvider() { + @Override public String id() { return "stub-search"; } + @Override public String label() { return "Stub Search"; } + @Override public boolean isAvailable() { return true; } + @Override public List search(PluginSearchQuery query) { + return fn.apply(query); + } + }; + } +} +``` + +- [ ] **Step 2: 跑测试确认失败** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw/mateclaw-server +mvn test -Dtest=PluginSearchBridgeTest +``` + +预期：编译失败，`PluginSearchBridge` 不存在。 + +- [ ] **Step 3: 实现 PluginSearchBridge** + +```java +package vip.mate.plugin.bridge; + +import vip.mate.plugin.api.search.PluginSearchProvider; +import vip.mate.plugin.api.search.PluginSearchQuery; +import vip.mate.plugin.api.search.PluginSearchResult; +import vip.mate.system.model.SystemSettingsDTO; +import vip.mate.tool.search.SearchProvider; +import vip.mate.tool.search.SearchQuery; +import vip.mate.tool.search.SearchResult; + +import java.util.ArrayList; +import java.util.List; + +/** + * Bridge that wraps a plugin's {@link PluginSearchProvider} into the platform's + * internal {@link SearchProvider} interface (issue #477). + *

+ * The platform-side {@link SystemSettingsDTO} is intentionally ignored — plugin + * providers read their own config via {@code PluginContext#getConfig}, keeping + * the SDK free of server types. + * + * @author MateClaw Team + */ +public class PluginSearchBridge implements SearchProvider { + + private final PluginSearchProvider delegate; + + public PluginSearchBridge(PluginSearchProvider delegate) { + this.delegate = delegate; + } + + @Override + public String id() { + return delegate.id(); + } + + @Override + public String label() { + return delegate.label(); + } + + @Override + public boolean requiresCredential() { + return delegate.requiresCredential(); + } + + @Override + public int autoDetectOrder() { + return delegate.autoDetectOrder(); + } + + @Override + public boolean isAvailable(SystemSettingsDTO config) { + return delegate.isAvailable(); + } + + @Override + public List search(String query, SystemSettingsDTO config) { + return search(SearchQuery.of(query), config); + } + + @Override + public List search(SearchQuery searchQuery, SystemSettingsDTO config) { + PluginSearchQuery pluginQuery = new PluginSearchQuery( + searchQuery.query(), + searchQuery.freshness(), + searchQuery.language(), + searchQuery.resolvedCount() + ); + List pluginResults = delegate.search(pluginQuery); + if (pluginResults == null) { + return List.of(); + } + List results = new ArrayList<>(pluginResults.size()); + for (PluginSearchResult r : pluginResults) { + results.add(SearchResult.builder() + .title(r.title()) + .url(r.url()) + .snippet(r.snippet()) + .source(r.source()) + .date(r.date()) + .providerId(delegate.id()) + .build()); + } + return results; + } +} +``` + +- [ ] **Step 4: 跑测试确认通过** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw/mateclaw-server +mvn test -Dtest=PluginSearchBridgeTest +``` + +预期：5 个测试全 PASS。 + +- [ ] **Step 5: Commit** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw +git add mateclaw-server/src/main/java/vip/mate/plugin/bridge/PluginSearchBridge.java \ + mateclaw-server/src/test/java/vip/mate/plugin/bridge/PluginSearchBridgeTest.java +git commit -m "feat(plugin): bridge PluginSearchProvider to the core SearchProvider chain (#477)" +``` + +--- + +### Task 5: 注册入口 + 生命周期（PluginContext / PluginContextImpl / LoadedPlugin / PluginManager / PluginInfo） + +**Files:** +- Modify: `mateclaw-plugin-api/src/main/java/vip/mate/plugin/api/PluginContext.java` +- Modify: `mateclaw-server/src/main/java/vip/mate/plugin/PluginContextImpl.java` +- Modify: `mateclaw-server/src/main/java/vip/mate/plugin/LoadedPlugin.java` +- Modify: `mateclaw-server/src/main/java/vip/mate/plugin/PluginManager.java`（字段 + 构造点 + rollback + disable + listPlugins） +- Modify: `mateclaw-server/src/main/java/vip/mate/plugin/model/PluginInfo.java` +- Test: `mateclaw-server/src/test/java/vip/mate/plugin/PluginContextImplSearchTest.java`（新建） + +接口加方法会让 `PluginContextImpl` 编译失败，因此本任务的接口与实现**必须同一 commit**。 + +- [ ] **Step 1: 写失败测试** + +```java +package vip.mate.plugin; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import vip.mate.channel.ChannelManager; +import vip.mate.llm.service.ModelProviderService; +import vip.mate.memory.spi.MemoryManager; +import vip.mate.plugin.api.PluginException; +import vip.mate.plugin.api.PluginManifest; +import vip.mate.plugin.api.MateClawPlugin; +import vip.mate.plugin.api.PluginContext; +import vip.mate.plugin.api.search.PluginSearchProvider; +import vip.mate.plugin.api.search.PluginSearchQuery; +import vip.mate.plugin.api.search.PluginSearchResult; +import vip.mate.tool.ToolRegistry; +import vip.mate.tool.search.SearchProviderRegistry; + +import java.net.URL; +import java.net.URLClassLoader; +import java.util.List; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.mockito.Mockito.mock; + +/** + * PluginContextImpl#registerSearchProvider: wraps the plugin SPI in a bridge, + * registers it into SearchProviderRegistry, and records the id on LoadedPlugin + * so disable/rollback can unregister it (issue #477). + */ +class PluginContextImplSearchTest { + + private SearchProviderRegistry registry; + private PluginContextImpl context; + private LoadedPlugin loadedPlugin; + + @BeforeEach + void setUp() { + registry = new SearchProviderRegistry(List.of()); + + PluginManifest manifest = new PluginManifest(); + manifest.setName("test-plugin"); + manifest.setVersion("1.0.0"); + manifest.setType("search"); + manifest.setEntrypoint("x.Y"); + + MateClawPlugin plugin = new MateClawPlugin() { + @Override public void onLoad(PluginContext ctx) { } + @Override public void onEnable() { } + @Override public void onDisable() { } + }; + loadedPlugin = new LoadedPlugin(manifest, plugin, + new URLClassLoader(new URL[0], getClass().getClassLoader())); + + context = new PluginContextImpl( + loadedPlugin, manifest, + mock(ToolRegistry.class), mock(ChannelManager.class), + mock(MemoryManager.class), mock(ModelProviderService.class), + registry, + null); + } + + private static PluginSearchProvider provider(String id) { + return new PluginSearchProvider() { + @Override public String id() { return id; } + @Override public String label() { return id; } + @Override public boolean isAvailable() { return true; } + @Override public List search(PluginSearchQuery query) { + return List.of(); + } + }; + } + + @Test + @DisplayName("registers into the registry and records the id on LoadedPlugin") + void registersAndRecords() { + context.registerSearchProvider(provider("my-search")); + + assertNotNull(registry.getById("my-search")); + assertEquals(List.of("my-search"), loadedPlugin.getRegisteredSearchProviders()); + } + + @Test + @DisplayName("id conflict surfaces as PluginException and is not recorded") + void conflictBecomesPluginException() { + context.registerSearchProvider(provider("my-search")); + + assertThrows(PluginException.class, + () -> context.registerSearchProvider(provider("my-search"))); + assertEquals(1, loadedPlugin.getRegisteredSearchProviders().size()); + } + + @Test + @DisplayName("blank id is rejected with PluginException") + void blankIdRejected() { + assertThrows(PluginException.class, + () -> context.registerSearchProvider(provider(" "))); + assertTrue(loadedPlugin.getRegisteredSearchProviders().isEmpty()); + } +} +``` + +- [ ] **Step 2: 跑测试确认失败** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw/mateclaw-server +mvn test -Dtest=PluginContextImplSearchTest +``` + +预期：编译失败（`registerSearchProvider` / `getRegisteredSearchProviders` / 7 参构造器不存在）。 + +- [ ] **Step 3: PluginContext 接口加方法** + +`PluginContext.java` 新增 import `vip.mate.plugin.api.search.PluginSearchProvider;`，在 `registerMemoryProvider` 之后加： + +```java + /** + * Register a web-search provider that joins the platform's search provider + * chain used by the {@code web_search} tool. + *

+ * The provider id must be globally unique — registration fails with a + * {@link PluginException} if it clashes with a built-in provider + * (serper / tavily / searxng / duckduckgo) or another plugin's provider. + * + * @param provider the search provider + * @throws PluginException if the id is blank or already taken + */ + void registerSearchProvider(PluginSearchProvider provider); +``` + +- [ ] **Step 4: LoadedPlugin 加记录字段** + +`LoadedPlugin.java` 在 `registeredChannels` 之后加： + +```java + /** Search provider ids registered by this plugin */ + private final List registeredSearchProviders = new ArrayList<>(); +``` + +- [ ] **Step 5: PluginContextImpl 实现** + +新增 import： + +```java +import vip.mate.plugin.api.search.PluginSearchProvider; +import vip.mate.plugin.bridge.PluginSearchBridge; +import vip.mate.tool.search.SearchProviderRegistry; +``` + +字段区加 `private final SearchProviderRegistry searchProviderRegistry;`，构造器在 `modelProviderService` 参数后加 `SearchProviderRegistry searchProviderRegistry` 并赋值（保持 `configJson` 为最后一个参数）。 + +`registerMemoryProvider` 之后加实现： + +```java + @Override + public void registerSearchProvider(PluginSearchProvider provider) { + if (provider == null || provider.id() == null || provider.id().isBlank()) { + throw new PluginException("Search provider id must not be blank"); + } + try { + searchProviderRegistry.registerPluginProvider(new PluginSearchBridge(provider)); + } catch (IllegalArgumentException e) { + throw new PluginException(e.getMessage()); + } + loadedPlugin.getRegisteredSearchProviders().add(provider.id()); + } +``` + +- [ ] **Step 6: PluginManager 注入 registry + 生命周期反注册** + +新增 import `vip.mate.tool.search.SearchProviderRegistry;`；字段区（`modelProviderService` 之后）加： + +```java + private final SearchProviderRegistry searchProviderRegistry; +``` + +（`@RequiredArgsConstructor` 自动进构造器。） + +`new PluginContextImpl(...)` 构造点（约 L211）在 `modelProviderService` 后传入 `searchProviderRegistry`。 + +`rollbackRegistrations`（约 L251）末尾加： + +```java + for (String searchId : loaded.getRegisteredSearchProviders()) { + try { searchProviderRegistry.unregisterPluginProvider(searchId); } catch (Exception e) { /* best effort */ } + } +``` + +`disablePlugin`（约 L269）在 memory/provider 反注册段之后、`loaded.setEnabled(false)` 之前加： + +```java + for (String searchId : loaded.getRegisteredSearchProviders()) { + searchProviderRegistry.unregisterPluginProvider(searchId); + } + int searchRemoved = loaded.getRegisteredSearchProviders().size(); +``` + +并把结尾的 log.info 扩展一个占位： + +```java + log.info("Plugin disabled: {} (tools={}, channels={}, provider={}, memory={}, search={})", + name, toolsRemoved, channelsRemoved, + providerRemoved != null ? providerRemoved : "none", + memoryRemoved != null ? memoryRemoved : "none", + searchRemoved); +``` + +`listPlugins()` 内存分支 builder（约 L355）加 `.registeredSearchProviders(List.copyOf(loaded.getRegisteredSearchProviders()))`；DB-only 分支（约 L377）加 `.registeredSearchProviders(List.of())`。 + +- [ ] **Step 7: PluginInfo 加字段** + +`PluginInfo.java` 在 `registeredMemoryProvider` 之后加： + +```java + /** Search provider ids registered by this plugin */ + private List registeredSearchProviders; +``` + +- [ ] **Step 8: 全链编译 + 跑测试确认通过** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw +mvn -q -pl mateclaw-plugin-api install -DskipTests +cd mateclaw-server +mvn test -Dtest='PluginContextImplSearchTest,SearchProviderRegistryPluginTest,PluginSearchBridgeTest' +``` + +预期：全 PASS。 + +- [ ] **Step 9: Commit** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw +git add mateclaw-plugin-api/src/main/java/vip/mate/plugin/api/PluginContext.java \ + mateclaw-server/src/main/java/vip/mate/plugin/ \ + mateclaw-server/src/test/java/vip/mate/plugin/PluginContextImplSearchTest.java +git commit -m "feat(plugin): registerSearchProvider lifecycle — register, disable, rollback (#477)" +``` + +--- + +### Task 6: 参考实现 — mateclaw-plugin-search-sample 模块 + +**Files:** +- Modify: `pom.xml`（根，`` 加一行） +- Create: `mateclaw-plugin-search-sample/pom.xml` +- Create: `mateclaw-plugin-search-sample/src/main/resources/mateclaw-plugin.json` +- Create: `mateclaw-plugin-search-sample/src/main/java/vip/mate/plugin/sample/search/SimpleSearchPlugin.java` + +设计说明：spec §3.6 说"sample 模块增加参考实现"，但一个插件 jar 只有一个 manifest（一个 type/entrypoint），往现有 `mateclaw-plugin-sample`（type=tool 的 HelloPlugin）里塞 search 会破坏它的演示语义，因此新建独立 sample 模块。实现刻意最小：调一个可配 `baseUrl` 的 JSON 搜索端点（约定响应 `{"results":[{"title","url","snippet"}]}`），JSON 解析用 Jackson（provided，平台父 ClassLoader 提供），HTTP 用 JDK `java.net.http`，零额外依赖。 + +- [ ] **Step 1: 根 pom 加 module** + +`pom.xml` 的 `` 中 `mateclaw-plugin-sample` 之后加： + +```xml + mateclaw-plugin-search-sample +``` + +- [ ] **Step 2: 模块 pom** + +```xml + + + 4.0.0 + + + vip.mate + mateclaw + ${revision} + ../pom.xml + + + mateclaw-plugin-search-sample + jar + + MateClaw Search Provider Sample Plugin + Sample plugin registering a custom web-search provider via the MateClaw Plugin SDK + + + + + vip.mate + mateclaw-plugin-api + provided + + + + + org.springframework.ai + spring-ai-model + provided + + + + + com.fasterxml.jackson.core + jackson-databind + provided + + + + + org.slf4j + slf4j-api + provided + + + +``` + +- [ ] **Step 3: manifest（含 config schema，驱动 PR-2 的配置表单）** + +`src/main/resources/mateclaw-plugin.json`: + +```json +{ + "name": "mateclaw-plugin-search-demo", + "version": "1.0.0", + "type": "search", + "displayName": "Demo Search Provider", + "description": "Registers a custom web-search provider backed by a configurable JSON search endpoint.", + "entrypoint": "vip.mate.plugin.sample.search.SimpleSearchPlugin", + "minPlatformVersion": "1.1.0", + "author": "MateClaw Team", + "config": { + "baseUrl": { + "type": "string", + "required": true, + "secret": false, + "description": "Search endpoint returning {\"results\":[{\"title\",\"url\",\"snippet\"}]}" + }, + "apiKey": { + "type": "string", + "required": false, + "secret": true, + "description": "Optional bearer token sent as Authorization header" + } + } +} +``` + +- [ ] **Step 4: 插件实现** + +```java +package vip.mate.plugin.sample.search; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import org.slf4j.Logger; +import vip.mate.plugin.api.MateClawPlugin; +import vip.mate.plugin.api.PluginContext; +import vip.mate.plugin.api.search.PluginSearchProvider; +import vip.mate.plugin.api.search.PluginSearchQuery; +import vip.mate.plugin.api.search.PluginSearchResult; + +import java.net.URI; +import java.net.URLEncoder; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.util.ArrayList; +import java.util.List; + +/** + * Sample plugin demonstrating {@code PluginType.SEARCH}: registers a search + * provider that queries a configurable JSON endpoint. Expected response shape: + * {@code {"results":[{"title":"...","url":"...","snippet":"..."}]}} + * + * @author MateClaw Team + */ +public class SimpleSearchPlugin implements MateClawPlugin { + + private Logger log; + + @Override + public void onLoad(PluginContext context) { + this.log = context.getLogger(); + context.registerSearchProvider(new DemoSearchProvider(context)); + log.info("SimpleSearchPlugin loaded, search provider registered"); + } + + @Override + public void onEnable() { + if (log != null) log.info("SimpleSearchPlugin enabled"); + } + + @Override + public void onDisable() { + if (log != null) log.info("SimpleSearchPlugin disabled"); + } + + static class DemoSearchProvider implements PluginSearchProvider { + + private static final Duration TIMEOUT = Duration.ofSeconds(15); + + private final PluginContext context; + private final HttpClient http = HttpClient.newBuilder().connectTimeout(TIMEOUT).build(); + private final ObjectMapper objectMapper = new ObjectMapper(); + + DemoSearchProvider(PluginContext context) { + this.context = context; + } + + @Override + public String id() { + return "demo-search"; + } + + @Override + public String label() { + return "Demo Search"; + } + + @Override + public boolean isAvailable() { + String baseUrl = context.getConfig("baseUrl", String.class); + return baseUrl != null && !baseUrl.isBlank(); + } + + @Override + public List search(PluginSearchQuery query) { + String baseUrl = context.getConfig("baseUrl", String.class); + String apiKey = context.getConfig("apiKey", String.class); + + String url = baseUrl + (baseUrl.contains("?") ? "&" : "?") + + "q=" + URLEncoder.encode(query.query(), StandardCharsets.UTF_8) + + "&count=" + query.count(); + + HttpRequest.Builder req = HttpRequest.newBuilder(URI.create(url)) + .timeout(TIMEOUT) + .GET(); + if (apiKey != null && !apiKey.isBlank()) { + req.header("Authorization", "Bearer " + apiKey); + } + + try { + HttpResponse resp = http.send(req.build(), HttpResponse.BodyHandlers.ofString()); + if (resp.statusCode() != 200) { + throw new IllegalStateException("Search endpoint returned HTTP " + resp.statusCode()); + } + return parse(resp.body()); + } catch (IllegalStateException e) { + throw e; + } catch (Exception e) { + throw new IllegalStateException("Search request failed: " + e.getMessage(), e); + } + } + + private List parse(String body) throws Exception { + List results = new ArrayList<>(); + JsonNode items = objectMapper.readTree(body).path("results"); + for (JsonNode item : items) { + results.add(new PluginSearchResult( + item.path("title").asText(null), + item.path("url").asText(null), + item.path("snippet").asText(null), + null, + null)); + } + return results; + } + } +} +``` + +- [ ] **Step 5: 全量构建（reactor 三模块 + 新 sample）** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw +mvn -q -pl mateclaw-plugin-search-sample -am package -DskipTests +ls mateclaw-plugin-search-sample/target/*.jar +``` + +预期：BUILD SUCCESS，产出 `mateclaw-plugin-search-sample-1.7.0-SNAPSHOT.jar`。 + +- [ ] **Step 6: Commit** + +```bash +git add pom.xml mateclaw-plugin-search-sample/ +git commit -m "feat(plugin): add search provider sample plugin module (#477)" +``` + +--- + +### Task 7: 回归 + 手工端到端验证 + +- [ ] **Step 1: server 全量测试回归** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw/mateclaw-server +mvn test +``` + +预期：全绿（重点确认 `WebSearchTool`/`WebSearchService`/registry 相关既有测试无回归）。 + +- [ ] **Step 2: 手工端到端（jar 落盘加载）** + +```bash +mkdir -p ~/.mateclaw/plugins +cp /Users/connor/workspace/ai-lab/mateclaw/mateclaw-plugin-search-sample/target/mateclaw-plugin-search-sample-*.jar ~/.mateclaw/plugins/ +cd /Users/connor/workspace/ai-lab/mateclaw/mateclaw-server +mvn spring-boot:run +``` + +启动日志验证三点： +1. `Plugin loaded: mateclaw-plugin-search-demo v1.0.0 (type=search, ...)` +2. `插件搜索提供商已注册: demo-search (order=500)` +3. 无 id 冲突/rollback 报错。 + +再验证 disable 路径：登录（admin/admin123）→ 插件页关掉该插件 → 日志出现 `插件搜索提供商已反注册: demo-search` 与 `Plugin disabled: ... search=1`。 + +验证完清理： + +```bash +rm ~/.mateclaw/plugins/mateclaw-plugin-search-sample-*.jar +``` + +- [ ] **Step 3: 若有问题修复后补 commit；全部通过则进入 Task 8** + +--- + +### Task 8: 推分支 + 开 PR + +- [ ] **Step 1: 推到 origin（ncw1992120/mateclaw）** + +```bash +cd /Users/connor/workspace/ai-lab/mateclaw +git push -u origin feat/plugin-search-provider +``` + +- [ ] **Step 2: 开 PR 到 mateaix:dev** + +```bash +gh pr create --repo mateaix/mateclaw --base dev \ + --title "feat(plugin): 插件化搜索 Provider — PluginType.SEARCH + PluginSearchProvider SPI (#477)" \ + --body "$(cat <<'EOF' +Closes 部分 #477（PR-1：SDK + 桥接 + registry；PR-2 的 catalog 接口与设置页重构另行提交）。 + +## 改动 +- `mateclaw-plugin-api`: `PluginType.SEARCH` + `vip.mate.plugin.api.search` 包（`PluginSearchProvider`/`PluginSearchQuery`/`PluginSearchResult`，自包含、不依赖 server 类）+ `PluginContext.registerSearchProvider()` +- `mateclaw-server`: `PluginSearchBridge`（适配核心 `SearchProvider`）；`SearchProviderRegistry` 支持运行时注册/反注册插件 provider（合并视图，id 冲突拒绝）；`PluginManager` disable/rollback 反注册与现有四类对称；`PluginInfo` 暴露 `registeredSearchProviders` +- 新模块 `mateclaw-plugin-search-sample`：type=search 的最小参考实现（可配 baseUrl/apiKey 的 JSON 端点） + +## 测试 +- `SearchProviderRegistryPluginTest`（7）：合并排序 / getById / 内置与插件 id 冲突 / resolve 三分支含插件项 / 反注册回退 +- `PluginSearchBridgeTest`（5）：query/result 转换、count 钳制、null 归一、异常透传 +- `PluginContextImplSearchTest`（3）：注册记录、冲突转 PluginException、空 id 拒绝 +- 手工端到端：sample jar 落盘 → 启动加载注册 → disable 反注册，日志均符合预期 + +无破坏性改动：接口新增方法/枚举值对存量插件透明；`GET/PUT /api/v1/settings` 未动。 +EOF +)" +``` + +- [ ] **Step 2 后**：PR 链接贴回 issue #477。 + +--- + +## 范围外（本计划不做） + +- PR-2：`GET /api/v1/settings/search-providers` catalog 接口、搜索设置页分组折叠、`Plugins.vue` schema 配置表单 —— PR-1 合并后单独出计划。 +- 面向插件作者的开发文档（仓库内暂无 plugin dev docs 先例，是否新增由上游在 PR 评审中定夺）。 diff --git a/docs/superpowers/specs/2026-07-03-plugin-search-provider-design.md b/docs/superpowers/specs/2026-07-03-plugin-search-provider-design.md new file mode 100644 index 00000000..0afd5b14 --- /dev/null +++ b/docs/superpowers/specs/2026-07-03-plugin-search-provider-design.md @@ -0,0 +1,144 @@ +# 插件化搜索 Provider + 搜索设置页重构 设计文档 + +日期：2026-07-03 +状态：待评审 +相关：`vip.mate.tool.search`（现有搜索 provider 链）、`mateclaw-plugin-api`（插件 SDK）、`/settings/system` 搜索设置区块 + +## 1. 背景与问题 + +### 1.1 自定义搜索 provider 没有插件化路径 + +当前 `SearchProviderRegistry` 通过 Spring 构造器注入 `List` 收集 provider，只认同一 `ApplicationContext` 里的 bean。要新增一个搜索源，唯一办法是**在 `vip.mate.tool.search` 源码树里加 `@Component` 类并重新编译部署整个 server**。 + +而项目已有一套真正的运行时插件系统（`mateclaw-plugin-api` + `PluginManager`）：独立 jar 丢进 `~/.mateclaw/plugins/` 或工作区 `plugins/`，`URLClassLoader` 隔离加载，支持运行时 enable/disable，配置走 manifest 声明的 schema（`mateclaw-plugin.json` 的 `config` 字段）+ `plugin` 表 `config_json` 持久化 + `PUT /api/v1/plugins/{name}/config` 接口。但 `PluginType` 只有 `TOOL / PROVIDER(LLM) / CHANNEL / MEMORY` 四类，**没有 SEARCH**，`PluginContext` 也没有对应注册方法。 + +LLM provider 已有"内置 `@Component` 链 + 插件注册表"双轨并存的先例（`ModelProviderService.pluginChatModels`），搜索 provider 缺的就是同构的第二轨。 + +### 1.2 搜索设置 UI 平铺、下拉菜单硬编码 + +`/settings/system` 的搜索区块把 4 个 provider 的开关/key/url 共 9 个配置项拍平在一个列表里；主 provider 下拉菜单是写死的两个 `