Commit Graph

40 Commits

Author SHA1 Message Date
mateaix
8dc6c64683 perf(agent): add progressive tool bridge 2026-08-09 19:39:45 +08:00
matevip
ed7f6f5c19 feat(chat): add reasoning retention controls and a linear trajectory export 2026-08-06 05:48:28 -04:00
matevip
bf0d64e46a feat(tool): 工具 schema 预算阈值门——超出窗口预算时按使用频度自动降级到扩展目录,enable_tool 可找回 2026-07-03 18:52:45 +08:00
matevip
56737e197f feat(agent): prefix 注入块统一 token 预算——记忆/Wiki 注入随模型有效窗口缩放,身份 prompt 超大告警 2026-07-03 18:51:32 +08:00
倪程伟
a1221ac02d feat(mcp): type on-behalf-of identity by channel/trust (#459)
The identity forwarded to opt-in MCP servers was a one-dimensional string
(ChatOrigin.requesterId): a MateClaw username for web logins, but a webchat
visitorId for visitors and an IM sender id for IM — indistinguishable to the
REST backend. The signed-token mode (d204b702) made this worse: an RS256
signature over an unauthenticated visitorId reads as "MateClaw authenticated
this user" to any backend that trusts the signature.

Introduce an identity-typing dimension at McpIdentityForwardService:

- classify() branches on ChatOrigin: authenticated (web login, sub=immutable
  userId), anonymous (webchat visitor, trust=anonymous), external (IM sender,
  trust=external), or none (cron/system → nothing injected, fail-closed).
- mint() adds `trust` and `channel_type` claims; plaintext value is prefixed
  `trust:subject` so backends can tell the kinds apart without a JWT.

The immutable userId reaches resolve() without coupling it to the user store:
JwtAuthFilter stamps user.id into auth.setDetails() (both JWT and PAT paths),
and ChatController.memoryOrigin carries it on a new ChatOrigin.requesterUserId
field (only-add, per the record's evolution rule).

Resolves the webchat semantic mismatch raised in #459 and the "sub should be
an immutable user id" follow-up. 82 tests green (4 identity classes covered
with claim assertions + full ChatOrigin/MCP regression).

(cherry picked from commit b5d2cfbf98b39848d7139c743a0b81fea71e8ffe)
2026-07-01 19:02:49 +08:00
倪程伟
2d04ce92ef
feat(kb-open): P0-A open-API auth — API keys, rate limit, centralized authorization
Hashed API-key auth (SHA-256, plaintext shown once), per-key sliding-window rate limit, and a fail-closed filter + scope/KB-binding interceptor enforcing empty-binding=zero-access. Admin CRUD for key lifecycle. Migration V164 across h2/mysql/kingbase.
2026-06-28 14:45:53 +08:00
matevip
8522aa7591 feat(tool): desktop local file/shell tools via WebSocket tunnel 2026-06-26 18:24:25 +08:00
倪程伟
03a6d61131
feat(sso): 飞书 OAuth2 单点登录 (ISSUE #405 P0) (#419)
* feat(sso): feishu OAuth2 single sign-on (ISSUE #405 P0)

Implements the SSO design (ISSUE #405) with feishu as the first IdP
and a generic OAuth2 provider abstraction for future dingtalk/wecom
extensions. SSO is disabled by default — existing deployments are
unaffected until mateclaw.sso.enabled=true.

Backend:
- SsoProvider interface + SsoUserInfo record: generic IdP abstraction
- FeishuSsoProvider: OAuth2 authorization-code flow (app_access_token
  with Caffeine cache → user_access_token → user info). apiBase switches
  between feishu.cn / larksuite.com by domain config.
- SsoProviderRegistry: conditional registration, lists enabled providers
- SsoStateService: HMAC-signed OAuth2 state + self-contained bind_token
  JWT, both persisted to sso_state DB table for multi-node correctness.
  State is one-time-consumable (conditional UPDATE), bind_token jti
  anti-replay via PK insert. Hourly ShedLock purge (LambdaQuery + Java
  time, works on all 3 dialects).
- SsoService: authorize/callback/bind, user mapping (union_id first →
  external_id fallback), auto-create with concurrent idempotency
  (DuplicateKeyException → rollback orphan user → re-query), link-only
  mode issues bind_token for existing-account binding.
- SsoController: 4 endpoints (/providers, /authorize, /callback, /bind)
  all permitAll.
- V159 migration (h2/mysql/kingbase): mate_user_external_identity,
  sso_state, ALTER mate_user.password NULL (SSO-only users).
- AuthService: generateToken promoted to public; login() guards
  password=null (SSO-only users cannot password-login).
- SecurityConfig: /auth/sso/** added to permitAll whitelist.
- LoginRateLimitFilter: expanded to cover /auth/sso/bind (brute-force
  surface equivalent to /auth/login).
- application.yml: mateclaw.sso.* config block (all env-var driven).

Frontend:
- Login.vue: dynamic SSO buttons (only shown when providers configured),
  OAuth2 callback detection (?sso=callback), link-only bind dialog,
  shared applyLogin flow (localStorage + workspace + route).
- api/index.ts: ssoApi (providers, authorize, callback, bind).

Tests: SsoStateServiceTest (11) — state issue/verify/replay/tamper,
bind_token issue/verify/anti-replay/garbage. Regression: PAT (23) +
Approval resolve (13) all green.

Not in scope (P1/P2): link-only bind/unbind management endpoints,
user enable/disable endpoint, dingtalk/wecom providers, admin SSO
config page. Workspace assignment for auto-created users remains a
product decision (design doc §12 item 2).

* fix(sso): self-review fixes — P0 security + P1 quality

P0-1 BindRequired serialization: replaced the R.fail(200, Map.toString())
hack with a structured SsoCallbackResponse record. Controller no longer
catches an exception for a non-error path; frontend reads bindRequired
flag directly instead of regex-parsing a stringified map.

P0-2 createSsoUser unbounded recursion: added a retry flag — second
DuplicateKeyException (extreme race where identity was concurrently
deleted) now throws a 503 instead of recursing to stack overflow.

P0-3 state TTL not enforced: verifyState's conditional UPDATE now
includes created_at > cutoff, so a state unused for 5+ min is rejected
at consumption time, not just at the 1h purge. Without this the 5-min
window was advisory only.

P1-5 SsoStateService unused ObjectMapper: removed dead injection.

P1-6 audit JSON string concat: replaced with ObjectMapper serialization
(provider/externalId no longer risk breaking the JSON structure).

P1-7 LoginRateLimitFilter shared counter: documented the intentional
decision that login + bind share a per-IP counter (same brute-force
surface) with guidance on switching to per-path if finer isolation
is needed.
2026-06-26 10:00:31 +08:00
matevip
982c6c048c feat(security): gate Swagger/OpenAPI UI behind mateclaw.openapi.expose-ui flag
- explicit SecurityConfig authorization for /swagger-ui*, /v3/api-docs*, /webjars/**
- public for local/default profile; admin-only (ROLE_ADMIN) by default in production DB profiles
- override via MATECLAW_OPENAPI_EXPOSE_UI; add RANDOM_PORT integration tests and docs
2026-06-24 10:42:46 +08:00
倪程伟
865513a3b6
docs(api): 完善 WebAPI 文档与 OpenAPI / Swagger 配置 (#407)
Closes #406

- 新增 OpenApiConfig 全局配置 Bean:标题/描述/服务器 + bearerAuth 安全方案
  (覆盖 JWT 与 mc_ PAT,对齐 JwtAuthFilter 前缀分发)
- application.yml 增 springdoc default-flat-param-object + mateclaw.openapi.* 外置项
- api.md 中英双语补全「通用约定」(R<T> 信封、ResultCode、错误模型、IPage 分页、
  ID 约定、三态认证、X-Workspace-Id 机制)+ 9 个旗舰端点完整参考
- 新增 openapi.md 中英双语 Swagger 使用指南
2026-06-24 10:07:11 +08:00
倪程伟
8e339f083a docs(security): correct generated-file TTL comment (7 days, not 10 min)
The permitAll comment claimed a 10-minute TTL, but GeneratedFileCache.TTL
is 7 days. The stale figure could mislead future security reasoning about
how long an unauthenticated capability URL stays live. Align the comment
with the actual value; the unguessable UUID remains the access guard.

Refs matevip/mateclaw#344
2026-06-17 23:21:08 +08:00
matevip
4cbd2b50f3 feat(dashboard): show the connected database on the dashboard
Surface the connected database product as a subtle chip in the Dashboard
header. SystemHealthService now reports a database label on /system/health
(reused by the front-end — no extra request), derived from a new
DatabaseBootstrapRunner.getDatabaseLabel() that reads the JDBC product name
once and normalizes it to a canonical label (MySQL / MariaDB / PostgreSQL /
H2, and 人大金仓 for the KingbaseES family), collapsing driver version noise.
2026-06-15 08:47:53 +08:00
matevip
07da1d610b feat(db): add PostgreSQL Spring profile 2026-06-14 16:47:02 +08:00
铭萱
446f34b6b5
feat(db): support KingbaseES (人大金仓) domestic database (#324)
Add KingbaseES support as an opt-in profile: dedicated migration tree, bilingual seed data, runtime DbType detection (KINGBASE_ES / POSTGRE_SQL), and JDBC URL handling in the datasource manager.
2026-06-14 10:30:09 +08:00
matevip
03e68d3c74 fix(agent): structure-aware truncation to stop mid-JSON cuts inducing hallucination (#187) 2026-05-21 15:13:07 +08:00
matevip
db16ff02a5 chore: drop external project name references from code comments 2026-05-20 08:09:43 +08:00
matevip
32d633ae4b fix(workspace): verify a path-bound agent belongs to the request workspace 2026-05-19 20:07:02 +08:00
matevip
a4b8c00eba fix(orm): make updateTime auto-fill unconditional so rows actually move on update 2026-05-15 15:21:23 +08:00
matevip
3ba22b99f3 feat(workspace,system): add @RequireGlobalAdmin and gate admin-only controllers 2026-05-15 10:18:10 +08:00
matevip
7015d1513c feat(context): anchor the first user message after compaction so long tasks remember the original goal 2026-05-13 08:50:15 +08:00
matevip
37894a6978 fix(context): never compact across a split tool_call ↔ tool_response pair 2026-05-13 08:50:08 +08:00
matevip
c07048f258 feat(skill,mcp): ckjia-shopping skill bundle + same-name dedup against bridged MCP/ACP virtual skills 2026-05-03 17:14:19 +08:00
matevip
3c33422a08 feat(auth): personal access tokens for headless / CI integration 2026-05-02 15:41:34 +08:00
matevip
349f4d7d3c refactor(bootstrap): drop legacy tools-sync.sql in favor of per-tool Flyway migrations
The two tools-sync scripts ran on every startup and used H2 MERGE INTO
... KEY(id), which overwrites every column on existing rows. That
silently reverted UI-toggled `enabled` and was the proximate cause of
a recent WriteFileTool/EditFileTool outage.

They were also a strict subset of the fresh-install seed (data-zh.sql /
data-en.sql register all 19 builtins; the sync scripts only 16) and out
of date. Per-tool Flyway migrations (V3, V31) are already the canonical
'register a new builtin' path, so the sync layer was duplicated and
error-prone.

Delete both files and the runToolSyncScript() loader. Tool descriptions
shown to the LLM come from @Tool annotations in code, not the DB row,
so removing per-startup metadata refresh has no functional impact.
2026-04-27 14:00:08 +08:00
matevip
0476447ab6 fix(agent): persist mid-turn narrative, queue follow-ups without dispose, flush on shutdown
A bundle of stability fixes that all surfaced together while running
the same long-form generation task across multiple turns. Each one
addresses a distinct way the previous behavior silently dropped
content the user had already seen on screen.

1. Mid-turn narrative persistence (StateGraphReActAgent +
   SummarizingNode). Intermediate ReasoningNode rounds and
   SummarizingNode broadcast their content_delta directly to the
   SSE channel for live display, but the StreamAccumulator only
   received the final answer. After refresh the assistant message
   showed only tool_call cards with no body text.
   StateGraphReActAgent now also forwards STREAMED_CONTENT (already
   set per round) as a persistOnly StreamDelta whenever it changes,
   so every narrative chunk lands in the accumulator's content
   buffer and gets written to mate_message. SummarizingNode now
   writes its summary into the same key so summarize narratives
   persist too.

2. Follow-up message queue, not dispose (ChatController#interruptStream).
   Sending a new message while a turn was running called
   requestInterrupt, which dispose()d the active Reactor chain mid
   LLM call. That cancelled the in-flight generation, lost partial
   tokens, and left the user staring at a half-finished bubble.
   The endpoint now uses enqueueMessage in all paths, matching
   the "wait for current turn, then run" behavior. The old
   requestInterrupt API is kept for any future force-replace UI
   but no caller routes to it.

3. Queued user message ordering (ChatStreamTracker.QueuedInput +
   ChatController.startQueuedMessage). interruptStream used to save
   the queued user message immediately, before the in-flight
   assistant message finalized in doOnError. listMessages orders
   by create_time ASC, so the queued user message ended up above
   the assistant reply it was supposed to follow. QueuedInput now
   carries contentParts; persistence is delayed to startQueuedMessage,
   which runs only after Asst-N is on disk.

4. JVM shutdown flush (ChatStreamTracker @PreDestroy +
   emergencySaveAccumulator). A mvn spring-boot:run restart used to
   wipe in-flight turns: SSE emitter timed out, ShutdownHook fired,
   HikariPool closed before doOnError could save. ChatStreamTracker
   now exposes an emergency-save callback per RunState; ChatController
   registers one per stream that snapshots the accumulator and
   writes status="interrupted_shutdown". @PreDestroy walks active
   runs, invokes the callback, then disposes. Spring's reverse-order
   bean teardown keeps ConversationService and Hikari alive long
   enough for the save to complete.

5. Observation thresholds for summarize (GraphObservationProperties +
   application.yml). The previous total-chars threshold of 12 KB
   triggered summarize after one or two RFC reads, costing a 40 to
   80 second compaction LLM call per loop. Tuned to: total 200 KB,
   single 16 KB, large-result 32 KB, rounds safety net 25. Java
   field defaults reverted to the conservative original values so
   application.yml stays the source of truth.

6. Frontend thinking segmentation (useChat.ts thinking_delta +
   phase). Multi-round ReAct turns merged every reasoning + summarize
   round's thinking into one segment, accumulating to 9 KB+ in a
   single bubble. thinking_delta now uses findLast(running) so a
   tool_call_started or phase transition closes the previous segment
   and the next delta opens a fresh one. phase event also closes
   running thinking/content segments.

7. Other small things bundled: removed a debug metadata-keys log
   that flooded the log file with one line per stream chunk; fixed
   three stale tests that didn't compile after earlier constructor
   changes (WikiLogServiceTest, WikiOverviewSpliceTest,
   WikiProcessingServiceLazyTest); added rfc-066 documenting the
   unified message queue + priority refactor as the next logical
   step on top of these stabilizations.

Verified end-to-end with multiple full sessions: a four-minute
generation that produced the expected docx and a follow-up enqueue
that ran cleanly after the previous turn naturally completed,
without the old "Disposable unavailable" interrupt path.
2026-04-27 07:51:49 +08:00
matevip
b7c911f01d feat(stt): DashScope realtime voice + language-aware routing + TalkMode polish
- DashScope paraformer-realtime-v2 WebSocket streaming
- Language-aware provider routing: Whisper for English, Paraformer for Chinese
- PCM WAV recording replaces WebM (provider filename bug + diagnostics)
- TalkMode push-to-talk fixes (audio drop, WS connecting race)
- Vite dev proxy WebSocket upgrade fix
- WebSocket binary buffer 8KB → 8MB (Tomcat default truncated voice clips)
- Audio chunk pacing at 100ms (DashScope returned 0 chars otherwise)
- Resolved language hint propagation + raw frame logging
- V46 seed idempotency fix on UI-toggled STT row
- Diagnostic cleanup after debugging session
2026-04-26 16:37:55 +08:00
matevip
edaf762878 feat(tool): native Java DocxRender tool — eliminate Node.js subprocess 2026-04-23 16:31:13 +08:00
matevip
aed905efb7 feat(agent): implement Lane E — JDK 21 virtual threads, Spring AI observability, BeanOutputConverter 2026-04-22 21:00:40 +08:00
matevip
379422101c refactor(ui): minimalist login redesign, MCP fixes, Flyway upgrade compatibility 2026-04-11 23:01:08 +08:00
matevip
d3b72929c7 fix(security): login rate limiting, SQL injection fix, error boundary 2026-04-11 18:31:51 +08:00
matevip
fe83b72d67 fix(security): harden JWT, CORS, and H2 Console for production 2026-04-11 18:21:17 +08:00
matevip
cc28f665f1 feat(db): introduce Flyway migration framework and unify H2/MySQL schemas
- Add Flyway baseline migration (V1) for both H2 and MySQL
- Remove 5 legacy SchemaMigration ApplicationRunner classes
- Add WorkspacePathGuard for file tool sandbox enforcement
- Inject workspace basePath context into agent graph state and tool executor
- Add workspace basePath config UI in frontend
- Fix workspace slug preservation on update
2026-04-11 16:34:09 +08:00
matevip
ec7ea038e7 feat(agent): context compression upgrade, 429 retry, iteration limit & repetition fixes 2026-04-10 00:52:26 +08:00
matevip
5124842526 feat(platform): workspace isolation, info architecture, and UI redesign 2026-04-09 16:29:02 +08:00
matevip
3506c55bc8 feat(platform): Phase 3 Sprint 1-3 — permission, agent binding, dashboard 2026-04-09 11:58:20 +08:00
matevip
3d58a48eae feat(platform): add workspace foundation and channel execution upgrades 2026-04-09 10:29:16 +08:00
matevip
5fc60ec513 feat(agent): smart truncation, stale stream cleanup, configurable tool timeouts, and new indexes 2026-04-07 06:39:46 +08:00
matevip
732bb13f47 fix(security): resolve SPA frontend route refresh returning 401 2026-04-05 18:20:40 +08:00
matevip
4f60043549 refactor(server): replace Knife4j with SpringDoc OpenAPI 2.8.16 2026-04-05 07:24:00 +08:00
matevip
579d60125b Initial commit: MateClaw — Java + Vue 3 AI Assistant System
Full-stack AI assistant built on Spring AI Alibaba.
Features: ReAct Agent, Plan-and-Execute, MCP Protocol, Multi-Model, Multi-Channel.

Apache-2.0 License
2026-04-04 19:03:49 +08:00