mirror of
https://gitee.com/mateos/mateclaw.git
synced 2026-09-14 03:33:43 +08:00
1bb81234d0
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
matevip
|
2e3dd071a5 |
fix(kb-open): assert session belongs to path kbId + cleanup
- requireSessionOwnership now also checks session.kbId() == path kbId (404 on mismatch), so a research session started under one KB cannot be addressed via another KB path even when the caller's key is bound to both — defense-in-depth on top of the keyId ownership check. - Drop internal "R7" / "review #446" markers from the controller Javadoc in favour of functional wording. - Import Set/Map/concurrent types and static any() instead of inline FQNs in the new kb-open research/auth tests, per code style. |
||
|
倪程伟
|
20c681a7c8
|
feat(kb-open): Deep Research 开放 API（start/SSE/status/cancel） (#446)
* feat(kb-open): Deep Research open API (start/SSE/status/cancel) Implements the async Deep Research endpoint for the KB Open API (#443). Research is a multi-step LLM pipeline (plan → retrieve+draft → compose) that runs asynchronously and broadcasts progress via SSE. Endpoints: - POST /{kbId}/research start (returns sessionId + streamUrl) - GET /{kbId}/research/{id}/stream SSE progress (?token= for EventSource) - GET /{kbId}/research/{id}/status query status / final report - POST /{kbId}/research/{id}/cancel cancel running session Components: - KbOpenResearchController: 4 endpoints, @RequireKbScope("kb:search") - KbResearchSessionRegistry: in-memory session tracking with keyId ownership (a caller can only query/cancel their own sessions) Security: - R7: SSE uses ?token= query param (KbOpenApiAuthFilter already supports this fallback for EventSource which can't set Authorization headers) - Session ownership: status/cancel/stream all verify keyId match - Cancel checks session is RUNNING (409 otherwise) Reuses existing WikiResearchService.research() + ChatStreamTracker for the actual research pipeline and SSE broadcasting. Tests (6 new, all green): - KbResearchSessionRegistryTest: register/complete/fail/cancel lifecycle, cancel-on-completed no-op, unknown session returns empty Closes #443 * fix(kb-open-research): cooperative cancel, sticky terminal, TTL, concurrency cap Review #446 — address all 4 job-lifecycle/cost blockers + nits: 1. Cooperative cancellation (was: cancel only flipped status, pipeline ran to completion). Cancel endpoint now calls streamTracker.requestStop(); WikiResearchService.ensureNotCancelled() checks isStopRequested at each stage boundary (plan→draft, draft→compose) and inside the parallel draft fan-out — so cancel actually halts the expensive LLM calls, not just the SSE stream. Throws ResearchCancelledException (caught locally, no error broadcast). 2. Sticky CANCELLED terminal. complete()/fail() now no-op on a CANCELLED session, so a user who cancelled never sees a COMPLETED report surface via /status. 3. Session registry TTL. Terminal sessions get an updatedAt timestamp and are evicted by a @Scheduled sweep after mate.kbopen.research.session-ttl (default 30m). RUNNING sessions are never evicted. Prevents unbounded memory growth. 4. Per-key concurrency cap. startIfAllowed() rejects new research when a key already has mate.kbopen.research.max-concurrent-per-key (default 3) RUNNING sessions → 429. Stops one key from spawning ~60 parallel multi-step LLM pipelines per minute under the per-min rate limiter. 5. Inline FQN → import (controller LinkedHashMap, test List.of). Nits (inherited from P0-A rebase): - V162→V164, prefix VARCHAR(12), design doc moved to rfcs/. - Design doc: kb:search scope row now documents it covers /research/**. 31 tests pass (12 registry incl. sticky-cancel/concurrency/TTL + 13 service + 4 rate limiter + 4 controller + ...). * fix(kb-open): scope-limited ?token= SSE auth fallback in KbOpenApiAuthFilter R7: the SSE progress stream (/research/{id}/stream) is consumed by browser EventSource, which cannot set an Authorization header. The filter's extractBearerToken() never read ?token= (still a TODO), so the SSE endpoint was unreachable from the browser — the headline use case got 401. Fix: accept ?token= ONLY on SSE stream paths (isSseStreamPath, suffix /stream), reject it everywhere else so the API key does not leak into access/proxy logs for normal calls (R5). Matches the JwtAuthFilter convention (getRequestURI logs carry no query string). Also bypass the per-minute rate limiter on the SSE path: EventSource reconnects/heartbeats would otherwise burn the key's window and 429 its own POST /research start. Rate limiting belongs on the cost-producing endpoints. Tests (6 new, KbOpenApiAuthFilterTest): - non-SSE: header passes, ?token= rejected (no authenticate call) - SSE: ?token= authenticates, missing token → 401 - SSE: bypasses rate limiter; non-SSE still hits it * fix(kb-open-research): make per-key concurrency cap atomic (no check-then-act race) startIfAllowed() did stream-and-count then put() — not atomic. Two concurrent starts for the same key could both pass the count check (both see < cap) and both put, admitting more sessions than the cap. On the virtual-thread start endpoint this is a real DoS/cost-bypass path. Fix: maintain a per-key AtomicInteger running counter (runningPerKey), incremented atomically on start (incrementAndGet + rollback on overflow) and decremented on each RUNNING→terminal transition (complete/fail/cancel). The counter is kept in lock-step with status==RUNNING; since terminal states are sticky, each session decrements exactly once. cancel() also rewritten to capture the pre-transition state cleanly (the old return check relied on Map.computeIfPresent returning the new value, which worked but read as 'before.status==CANCELLED'). Tests (+2): cancelled/failed release slot (counter consistency), and a concurrent-start test (12 virtual threads, cap=3) asserting exactly cap admits — would be flaky/fail under the old impl. * refactor(kb-open-research): remove unused register() back-compat method register() was left over from the initial impl — it bypassed the per-key concurrency cap (no startIfAllowed check) and, after the atomic-counter fix, incremented runningPerKey without any overflow rollback. With no production caller (the start endpoint uses startIfAllowed), it only existed for tests to set up a RUNNING session. Drop it and route the tests through startIfAllowed so nothing can accidentally ship a path that ignores the cap. |
||
|
倪程伟
|
9d292a9893
|
feat(kb-open): P0-B 9 个开放 API 端点 (#445)
* feat(kb-open): P0-B 9 open API endpoints Implements the 9 read-only KB Open API endpoints on top of the P0-A auth skeleton (#441). Each returns an explicit DTO (A5: never raw entities) and delegates assembly to service-layer methods that return pure DTOs (A6: no HTTP coupling, MCP-ready). Endpoints: - GET /pages/{slug} entity card (mode=summary/full/section:{heading}) - POST /search hybrid retrieval (granularity=entity/chunk) - POST /search/chunks chunk-level semantic search - POST /pages/{slug}/traverse entity relation graph (depth ≤ 2) - GET /pages/{slug}/trace provenance (page → chunk → raw) - GET /taxonomy pageType/entityType/relationType enumeration - GET /whats-new recent changes + stale pages - GET /stats KB statistics - GET /pages lightweight page list Components: - KbOpenApiController: 9 endpoints, each @RequireKbScope annotated - KbOpenApiService: assembly layer (card, traverse BFS, metadata parsing) - KbOpenApiDtos: all response DTOs as records (PageCard, TraceResult, TaxonomyResult, KbStats, WhatsNewResult, TraverseResult, PageList) Traverse (pragmatic version): - depth ≤ 2 with explosion guard, predicate LIKE matching - slug → pageId → mention → primaryEntity (salience-highest) - neighbor nodes echo slug when available (R11) - edge sourceHandle via evidenceChunkId → citing page Tests (4 new, all green): - KbOpenApiControllerTest: 404 on missing page/slug, delegation to service Closes #442 * fix(kb-open): address review feedback on #445 BLOCKERS: - stats.pagesWithLinks always returned 0 because listByKbId() nulls out content. Switch to listByKbIdWithContent() so [[wiki link]] detection works. - Test file: replace inline java.util.List.of() FQN with import + simple name (sync-opensource would expose the unidiomatic style). NITS (inherited from P0-A rebase): - V162→V164, prefix VARCHAR(12), FQN imports, parseScopes trim, ?token= fallback removal, design doc moved to rfcs/ — all now in ancestor commit 6fd62440. EXTRA: - whatsNew staleReason: hardcoded Chinese "上游 fact 页面变更" → English "Upstream fact page changed" (external-facing API response). * chore(wiki): drop RFC-012 prefix from progress field Javadocs (#449 nit) Per #449 review (4825113234): the internal RFC-012 reference should not appear in code. progressPhase/progressTotal/progressDone Javadocs still carried the "RFC-012 M2 v2 UI:" prefix after #449's English translation pass — drop it now that these lines are touched. Zero behavior change. * chore(kb-open): drop inline FQN in parseScopes (#444 nit) Per #444 review (4825157096): parseScopes used `.collect(java.util.stream.Collectors.toUnmodifiableSet())` while `Collectors` is already imported at the top of the file. Use the simple name. Zero behavior change. |
||
|
倪程伟
|
2d04ce92ef
|
feat(kb-open): P0-A open-API auth — API keys, rate limit, centralized authorization
Hashed API-key auth (SHA-256, plaintext shown once), per-key sliding-window rate limit, and a fail-closed filter + scope/KB-binding interceptor enforcing empty-binding=zero-access. Admin CRUD for key lifecycle. Migration V164 across h2/mysql/kingbase. |