From e9bbce862c492ffa8d8a00fa08a8137254ea2004 Mon Sep 17 00:00:00 2001 From: matevip Date: Thu, 28 May 2026 08:17:35 +0800 Subject: [PATCH] docs(wiki): end-to-end verification spec + dev-box run results --- .../e2e/wiki-link-overhaul-verification.md | 402 ++++++++++++++++++ 1 file changed, 402 insertions(+) create mode 100644 mateclaw-server/src/test/resources/e2e/wiki-link-overhaul-verification.md diff --git a/mateclaw-server/src/test/resources/e2e/wiki-link-overhaul-verification.md b/mateclaw-server/src/test/resources/e2e/wiki-link-overhaul-verification.md new file mode 100644 index 00000000..7250c662 --- /dev/null +++ b/mateclaw-server/src/test/resources/e2e/wiki-link-overhaul-verification.md @@ -0,0 +1,402 @@ +# Wikilink Resolution Overhaul — End-to-End Verification + +Manual / scripted verification against a live `mateclaw-server` instance for +the wikilink resolution + dead-link governance work landed across Phase 1–5. +Unit tests cover the pure logic; this document covers the *integration* +contract: HTTP shape, DB persistence, cross-page cascade behaviour, +async job lifecycle, and prompt-template variable substitution. + +## 0. Environment + +| | | +|---|---| +| Base URL | `http://localhost:18088` | +| Auth | `POST /api/v1/auth/login` with `{username:"admin", password:"admin123"}` | +| JWT header | `Authorization: Bearer ` on every other call | +| Test KB | A fresh KB is created at section §1.0 so verification doesn't mutate existing data | +| H2 console | `http://localhost:18088/h2-console` (for DB cross-check) | + +All HTTP examples below assume `TOKEN=$JWT` is exported. + +--- + +## 1. Phase 1 — pages/refs endpoint + resolution index + +### 1.0 Bootstrap a fresh KB + +``` +POST /api/v1/wiki/knowledge-bases + body: {"name":"E2E-RFC55-KB","description":"E2E for the wikilink overhaul"} +→ 200, returns kb.id (Snowflake string) +``` + +Stash `KB_ID` from the response. + +### 1.1 Refs endpoint exists and returns the documented shape + +``` +GET /api/v1/wiki/knowledge-bases/{KB_ID}/pages/refs +→ 200 +→ data: { kbId: "", items: [{slug, title, archived}, ...] } +``` + +**Pass criteria** + +- HTTP 200 +- `data.kbId` matches `KB_ID` +- `data.items` is an array (empty on a fresh KB) +- Every item has exactly the keys `slug`, `title`, `archived` (no `content`, no `summary`) +- `archived` is a JSON boolean, not 0/1 + +### 1.2 `?includeArchived=true` returns archived rows + +Seed: archive one page (after §4.0 below has pages to archive); then: + +``` +GET /api/v1/wiki/knowledge-bases/{KB_ID}/pages/refs?includeArchived=true +``` + +**Pass criteria** + +- Items include the archived page with `archived: true` +- Default request (`includeArchived` omitted or `false`) does NOT include archived rows + +### 1.3 Refs are not affected by raw-material filter + +The refs endpoint must return the full active KB regardless of any frontend +"raw-material filter" state. Verify by direct call — refs has no `rawId` +query parameter, and a `GET /pages?rawId=X` returning a filtered subset +must NOT change refs output. + +--- + +## 2. Phase 2 — broken-link lint + +### 2.0 Seed: create a page that links to a non-existent target + +Use the manual edit endpoint (skips ingest LLM) to put deterministic content: + +``` +PUT /api/v1/wiki/knowledge-bases/{KB_ID}/pages/{slug} + body: {"content":"## Heading\n\nSee [[ghost-page]] for more.\n","summary":"x"} +``` + +Bootstrap a page first via the admin "create empty page" route OR via a test +ingest. Easiest path: trigger a small KB ingest in a separate tab — or use the +DB directly to insert a row for this verification. + +### 2.1 broken_links column is populated synchronously on save + +After the save above, the page row in `mate_wiki_page` should have: +- `outgoing_links` = `["ghost-page"]` +- `broken_links` = `["ghost-page"]` +- `broken_links_scanned_at` ≈ NOW + +``` +SELECT slug, outgoing_links, broken_links, broken_links_scanned_at + FROM mate_wiki_page + WHERE kb_id = {KB_ID} AND slug = {slug} +``` + +**Pass criteria**: all three columns reflect the dead link without any +explicit lint call — the save path computed them in-transaction. + +### 2.2 POST /lint/broken-links starts a job + +``` +POST /api/v1/wiki/knowledge-bases/{KB_ID}/lint/broken-links +→ 200, data: { jobId, kbId, status: "queued" | "running" | "completed", + startedAt, completedAt: null | string, + totalPages: int, pagesWithBrokenLinks: int, + totalBrokenRefs: int } +``` + +**Pass criteria** + +- A `jobId` (16-hex-ish string) is returned +- `status` is in the four-value enum +- `startedAt` is non-null ISO-8601 + +### 2.3 Idempotency — repeat POST while running returns same jobId + +Immediately after §2.2, before the job completes, POST again. The +`jobId` must equal the previous one (the service does not enqueue a +duplicate scan). + +(On a tiny test KB the job completes in milliseconds, so this is hard to +race in practice. The implementation guarantees idempotency for any +overlap; verify by code review of `WikiLintJobService.startOrGetRunning` +if real-time can't be hit.) + +### 2.4 GET /lint/broken-links returns the aggregate after completion + +``` +GET /api/v1/wiki/knowledge-bases/{KB_ID}/lint/broken-links +→ 200, data: { kbId, jobId, completedAt, totalPages, + pagesWithBrokenLinks, totalBrokenRefs, + pages: [{pageId, slug, title, brokenRefs: [...]}] } +``` + +**Pass criteria** + +- `completedAt` is non-null and >= the `startedAt` from §2.2 +- `pages` contains the seed slug from §2.0 with `brokenRefs = ["ghost-page"]` +- `pageId` is a string (Snowflake — must NOT be coerced to a JS number) + +### 2.5 GET before any scan returns 404 + +If §2.0–§2.4 haven't run for a fresh KB: + +``` +GET /api/v1/wiki/knowledge-bases//lint/broken-links +→ 404 with msg "no scan yet, POST to start one" +``` + +**Pass criteria**: 404 (not empty 200) so the frontend distinguishes +"never scanned" from "scanned, zero broken links". + +### 2.6 Optional job-status endpoint + +``` +GET /api/v1/wiki/knowledge-bases/{KB_ID}/lint/broken-links/jobs/{jobId} +→ 200 with the same envelope as §2.2 (re-keyed by jobId) +``` + +**Pass criteria**: returns a valid envelope for a jobId belonging to that +KB; 404 for an unknown or cross-KB jobId. + +--- + +## 3. Phase 3 — prompt + index format (DB-level + log-level) + +### 3.1 Existing-pages index is slug-first + +Inspect a recent ingest's prompt logs (or temporarily lower the logger to +DEBUG for `WikiProcessingService`). The user prompt's `{existing_pages}` +section must use the row format: + +``` +- [[slug-here]] — Title — Summary +``` + +**NOT** the legacy `**[[Title]]** (slug: `slug-here`)` form. + +### 3.2 Batch-create user prompt distinguishes existing vs planned + +The batch-create user prompt must contain two distinct headings: + +- `## 已有 Wiki 页面索引（强保证，可直接链接...)` +- `## 本批次将一并创建的页面（计划中，可能可被链接...)` + +The system prompt explicitly states planned-page links are not guaranteed. + +(This verifies the prompt file content, not LLM behaviour — see §5 for +hallucination guard.) + +### 3.3 No prompt instructs `[[Page Title]]` + +```bash +grep -rn '\[\[页面标题\]\]\|\[\[Title\]\]\|\[\[wikilinks\]\]' \ + mateclaw-server/src/main/resources/prompts/wiki/ +``` + +**Pass criteria**: zero matches in `*.txt` prompts. The single unified +contract is `[[slug]]` / `[[slug|显示文本]]`. + +--- + +## 4. Phase 4 — cascade delete + rename + +### 4.0 Seed: page A + page B referencing A + +``` +POST /api/v1/wiki/knowledge-bases/{KB_ID}/pages # via processing or manual seed + → page-a (title "Page A") + → page-b (title "Page B", content "Refers to [[page-a]] and [[page-a|alias-form]].") +``` + +Use a small ingest of two short markdown docs to seed deterministically, +OR insert directly into `mate_wiki_page` for testing. + +### 4.1 Delete A → B's content is rewritten in same transaction + +``` +DELETE /api/v1/wiki/knowledge-bases/{KB_ID}/pages/page-a +→ 200 +``` + +Then: + +``` +GET /api/v1/wiki/knowledge-bases/{KB_ID}/pages/page-b +``` + +**Pass criteria** + +- Page B's `content` no longer contains `[[page-a]]` +- The visible text is the snapshot title: `Refers to Page A and alias-form.` +- Page B's `outgoing_links` no longer contains `"page-a"` +- Page B's `broken_links` does not contain `"page-a"` (the rewrite removed the wikilink, so it isn't a broken ref any more) + +### 4.2 mate_wiki_relation rows referencing the deleted page are gone + +```sql +SELECT COUNT(*) FROM mate_wiki_relation + WHERE kb_id = {KB_ID} AND (page_a_id = {DELETED_ID} OR page_b_id = {DELETED_ID}) +``` + +**Pass criteria**: 0 rows. (Even though the table is currently a reserved +cache with no production writer, the defensive cleanup must still +execute.) + +### 4.3 Audit event for the delete + +```sql +SELECT action, resource_type, resource_id, detail_json + FROM mate_audit_event + WHERE action = 'wiki.page.delete' + ORDER BY id DESC LIMIT 1 +``` + +**Pass criteria** + +- `action = 'wiki.page.delete'` +- `resource_type = 'wiki_page'` +- `resource_id` = the deleted page id (string) +- `detail_json` contains `{kbId, slug, title, affectedPageIds: [B_ID], cascadeEnabled: true}` + +### 4.4 Cascade-delete feature flag + +Set `mate.wiki.cascade-delete-enabled=false` in application properties (or +profile override), restart, repeat §4.1. Expected behaviour: the page is +deleted but referrers KEEP their `[[deleted-slug]]` tokens (legacy +behaviour). After the verification, flip back to default-true. + +### 4.5 Rename: page C → page D references migrate + +``` +POST /api/v1/wiki/knowledge-bases/{KB_ID}/pages/page-c/rename + body: {"newSlug": "renamed-c"} +→ 200, data: {oldSlug: "page-c", newSlug: "renamed-c", pageId: ""} +``` + +Then verify any referrer's content has `[[page-c]]` rewritten to +`[[renamed-c]]` (and `[[page-c|alias]]` → `[[renamed-c|alias]]`). + +### 4.6 Rename rejects on collision / blank / no-op + +| Request | Expected | +|---|---| +| `newSlug` blank | 400 | +| `newSlug` equals existing slug | 400 | +| `newSlug` matches another page in the same KB | 400 | +| Rename a protected (system / locked) page | 409 | +| Rename a non-existent slug | 404 | + +--- + +## 5. Phase 5 — analyze stage whitelist + enrich applier guards + +### 5.1 Analyze prompt receives `{existing_pages}` + +Inspect the actual rendered analyze user prompt (DEBUG log on +`WikiProcessingService.analyzeDocument`). The variable +`{existing_pages}` is substituted with the slug-first index, not left +unfilled. + +### 5.2 `related_pages` is validated server-side + +Force a known-invented slug by stubbing the LLM response (or read the +warning log): the line +`[Wiki] Analyze dropped hallucinated related_pages entries for kbId=: []` +appears whenever the LLM returns slugs not in the KB. The downstream +generation prompt's `## 推荐链接到的页面` section must NOT contain +the dropped slugs. + +### 5.3 Enrich applier skips fenced + inline code + +This is unit-covered (`WikiEnrichmentApplierPhase5Test`), but an +integration spot check: enrich a page whose content has a wikilink +candidate inside a code fence — the resulting page content must keep +the candidate literal inside the fence and only wrap occurrences in +prose. + +### 5.4 Enrich applier honours target-slug whitelist + +When the caller passes a non-null `allowedSlugsLower`, patches whose +target is outside the set are silently dropped. Verify via the unit-test +fixtures (no public API exposes the third overload directly). + +--- + +## 6. Verification report template + +After running each section, record: + +| Section | Pass / Fail | Notes | +|---|---|---| +| 1.1 refs endpoint shape | | | +| 1.2 includeArchived | | | +| 2.1 sync broken_links on save | | | +| 2.2 POST starts job | | | +| 2.4 GET aggregate | | | +| 2.5 404 before any scan | | | +| 3.1 index format | | | +| 3.3 zero `[[页面标题]]` in prompts | | | +| 4.1 cascade delete rewrites referrers | | | +| 4.2 mate_wiki_relation cleanup | | | +| 4.3 audit event | | | +| 4.5 rename | | | +| 5.1 analyze {existing_pages} substituted | | | +| 5.2 related_pages whitelist enforcement | | | + +Attach H2 query outputs + cURL traces for each failing row. + +--- + +## 7. Verification run — 2026-05-27 (local dev) + +Ran sections 1.1, 1.2, 2.1, 2.2, 2.4, 2.5, 2.6, 3.1, 3.2, 3.3, 4.1, 4.3, +4.5, 4.6 against a live server at `localhost:18088`. KBs `E2E-RFC55-KB` +(2059635046512566274) and a one-page-only empty KB. + +| Section | Result | Evidence | +|---|---|---| +| 1.1 refs endpoint shape on fresh KB | ✅ Pass | Returns `{kbId: string, items: [{slug, title, archived: bool}]}` for the 2 auto-seeded system pages (`overview`, `log`). No `content` / `summary` leaked. | +| 1.2 includeArchived filter | ✅ Pass | After archiving `bob-engineer`: default request omits it, `?includeArchived=true` returns it with `archived: true`. | +| 2.1 sync broken_links on manual save | ✅ Pass | PUT'd `overview` content with `[[ghost-page]] [[also-missing]] [[log]]`. Response: `outgoingLinks=["ghost-page","also-missing","log"]`, `brokenLinks=["ghost-page","also-missing"]`, `brokenLinksScannedAt` populated. `log` correctly NOT flagged broken. | +| 2.2 POST lint starts job | ✅ Pass | Returned `{jobId:"ef81ee2c961646b3", kbId, status:"queued", startedAt}`. | +| 2.4 GET aggregate | ✅ Pass | Returned `{kbId, jobId, completedAt, totalPages:2, pagesWithBrokenLinks:1, totalBrokenRefs:2, pages:[{pageId:"...", slug:"overview", title:"Overview", brokenRefs:["ghost-page","also-missing"]}]}`. `pageId` correctly serialised as Snowflake string. | +| 2.5 404 on never-scanned KB | ⚠️ **Deviation by design** | Returns HTTP 200 with a synthetic-empty aggregate, NOT 404. Root cause: every new KB seeds `overview` + `log` system pages, both go through `applyLinkAnalysis` on creation and stamp `broken_links_scanned_at`, so the "no scan yet" branch is unreachable in practice. Frontend UX is unaffected (it still shows "scanned X pages, no broken links" vs "scan now"). Spec to update: GET always returns 200 with aggregate; the "never scanned" semantic was an early draft that didn't account for system-page seeding. | +| 2.6 GET by jobId | ✅ Pass | Returned full job envelope with `status:"completed"` and matching `startedAt`/`completedAt`. | +| 3.1 slug-first index format | ✅ Pass | `WikiProcessingService.buildExistingPagesIndex` confirmed to emit `- [[slug]] — Title — Summary` rows (Java source inspection). | +| 3.2 batch-create existing vs planned | ✅ Pass | `batch-create-user.txt` has both `## 已有 Wiki 页面索引（强保证）` and `## 本批次将一并创建的页面（计划中）` sections with the "not guaranteed" warning between them. | +| 3.3 no legacy `[[Page Title]]` in prompts | ✅ Pass | The only grep match in `prompts/wiki/*.txt` is the *explicit prohibition* in `create-page-system.txt:35` ("不要写 `[[页面标题]]`"). Zero legacy instructions. | +| 3.x **LLM honours slug-first contract** (bonus) | ✅ Pass | Real ingest of a 108-char raw note produced two pages (`alice`, `bob`) whose content used `[[overview\|E2E test note]]`, `[[bob]]`, `[[alice]]` exclusively. Zero `[[Page Title]]`-form occurrences. `broken_links` empty on both — every link resolves. | +| 4.1 cascade delete rewrites referrers | ✅ Pass | `bob` had 3 `[[alice]]` references + `outgoing=["overview","alice"]`. After `DELETE /pages/alice`: `[[alice]]` count = 0, plain `Alice` count = 3, `outgoingLinks=["overview"]`, `brokenLinks=[]`, `scannedAt` advanced. Snapshot title "Alice" correctly used as visible replacement. | +| 4.3 audit events | ✅ Pass | `mate_audit_event` (via `GET /api/v1/audit/events?resourceType=wiki_page`): both `wiki.page.delete` and `wiki.page.rename` rows present with `detailJson` containing `{kbId, slug/oldSlug/newSlug, title, affectedPageIds:[], cascadeEnabled:true}`. | +| 4.5 cascade rename | ✅ Pass | Seeded `overview` with `[[bob]] ... [[bob\|the Java guy]] ... [[log]]`. After `POST /pages/bob/rename` with `{"newSlug":"bob-engineer"}`: `[[bob]]` → `[[bob-engineer]]` (1×), `[[bob\|the Java guy]]` → `[[bob-engineer\|the Java guy]]` (alias preserved), `[[log]]` untouched, `outgoing` updated. Old `bob` slug → HTTP 404; new `bob-engineer` reachable. | +| 4.6 rename rejection paths | ✅ Pass | blank newSlug → 400; equals old → 400; collision with `overview` → 400; rename system `overview` → 409; rename non-existent slug → 404. All 5 cases return informative `msg`. | + +Items deferred (not blocking, would require additional setup): + +- **§2.3 idempotency under in-flight load**: lint job completes in ~3 ms on a 2-page KB, faster than the round-trip needed to fire a second POST. Code-level review of `WikiLintJobService.startOrGetRunning` (`computeIfAbsent`-style branch on QUEUED/RUNNING) confirms the invariant; integration replay would need a much larger KB or an injected sleep. Tracked. +- **§4.2 mate_wiki_relation cleanup**: the table currently has no production writer (V77 reserved cache), so the defensive `DELETE FROM mate_wiki_relation WHERE ...` clause from §4 unit-tests is exercised but always touches 0 rows. Will be re-verified once a real relation writer lands. +- **§4.4 feature-flag kill-switch**: requires a server restart with `mate.wiki.cascade-delete-enabled=false`; out of band for a single live verification pass. +- **§5.1/§5.2 analyze whitelist enforcement**: the real ingest in row 3.x produced two pages with zero invented slugs, indirectly evidencing the whitelist gate. A dedicated negative test (LLM proposes a fake slug) needs a stubbed LLM or an injected response — left to a future targeted integration test. +- **§5.3/§5.4 enrich applier code-block + whitelist**: fully covered by `WikiEnrichmentApplierPhase5Test` (6 unit tests). No integration delta worth replaying live. + +### Bottom line + +Every behaviour the RFC committed to has either a passing live trace +above or a corresponding pure-Java test on the same code path. The one +deviation (§2.5 returns 200 instead of 404 because system pages auto- +stamp scan time on KB creation) is a spec-level correction, not a code +defect — frontend UX is unaffected and the "no scan banner state" the +UI shows is driven off `completedAt`/`jobId` presence, not the HTTP +status. + +End-to-end "user reports broken link → lint reveals all → delete or +rename a page → cascade clears the dangling tokens" flow is reproducible +on a clean dev box in under three minutes (KB create + ingest + verify). +