docs(wiki): end-to-end verification spec + dev-box run results

2026-09-14 03:33:43 +08:00 · 2026-05-28 08:17:35 +08:00 · 2026-05-28 08:17:35 +08:00 · e9bbce862c
commit e9bbce862c
parent a71c49d374
1 changed files with 402 additions and 0 deletions
--- a/mateclaw-server/src/test/resources/e2e/wiki-link-overhaul-verification.md
+++ b/mateclaw-server/src/test/resources/e2e/wiki-link-overhaul-verification.md
@ -0,0 +1,402 @@
 # Wikilink Resolution Overhaul — End-to-End Verification
 Manual / scripted verification against a live `mateclaw-server` instance for
 the wikilink resolution + dead-link governance work landed across Phase 1–5.
 Unit tests cover the pure logic; this document covers the *integration*
 contract: HTTP shape, DB persistence, cross-page cascade behaviour,
 async job lifecycle, and prompt-template variable substitution.
 ## 0. Environment
 | | |
 |---|---|
 | Base URL | `http://localhost:18088` |
 | Auth | `POST /api/v1/auth/login` with `{username:"admin", password:"admin123"}` |
 | JWT header | `Authorization: Bearer <token>` on every other call |
 | Test KB | A fresh KB is created at section §1.0 so verification doesn't mutate existing data |
 | H2 console | `http://localhost:18088/h2-console` (for DB cross-check) |
 All HTTP examples below assume `TOKEN=$JWT` is exported.
 ---
 ## 1. Phase 1 — pages/refs endpoint + resolution index
 ### 1.0 Bootstrap a fresh KB
 ```
 POST /api/v1/wiki/knowledge-bases
  body: {"name":"E2E-RFC55-KB","description":"E2E for the wikilink overhaul"}
 → 200, returns kb.id (Snowflake string)
 ```
 Stash `KB_ID` from the response.
 ### 1.1 Refs endpoint exists and returns the documented shape
 ```
 GET /api/v1/wiki/knowledge-bases/{KB_ID}/pages/refs
 → 200
 → data: { kbId: "<id>", items: [{slug, title, archived}, ...] }
 ```
 **Pass criteria**
 - HTTP 200
 - `data.kbId` matches `KB_ID`
 - `data.items` is an array (empty on a fresh KB)
 - Every item has exactly the keys `slug`, `title`, `archived` (no `content`, no `summary`)
 - `archived` is a JSON boolean, not 0/1
 ### 1.2 `?includeArchived=true` returns archived rows
 Seed: archive one page (after §4.0 below has pages to archive); then:
 ```
 GET /api/v1/wiki/knowledge-bases/{KB_ID}/pages/refs?includeArchived=true
 ```
 **Pass criteria**
 - Items include the archived page with `archived: true`
 - Default request (`includeArchived` omitted or `false`) does NOT include archived rows
 ### 1.3 Refs are not affected by raw-material filter
 The refs endpoint must return the full active KB regardless of any frontend
 "raw-material filter" state. Verify by direct call — refs has no `rawId`
 query parameter, and a `GET /pages?rawId=X` returning a filtered subset
 must NOT change refs output.
 ---
 ## 2. Phase 2 — broken-link lint
 ### 2.0 Seed: create a page that links to a non-existent target
 Use the manual edit endpoint (skips ingest LLM) to put deterministic content:
 ```
 PUT /api/v1/wiki/knowledge-bases/{KB_ID}/pages/{slug}
  body: {"content":"## Heading\n\nSee [[ghost-page]] for more.\n","summary":"x"}
 ```
 Bootstrap a page first via the admin "create empty page" route OR via a test
 ingest. Easiest path: trigger a small KB ingest in a separate tab — or use the
 DB directly to insert a row for this verification.
 ### 2.1 broken_links column is populated synchronously on save
 After the save above, the page row in `mate_wiki_page` should have:
 - `outgoing_links` = `["ghost-page"]`
 - `broken_links` = `["ghost-page"]`
 - `broken_links_scanned_at` ≈ NOW
 ```
 SELECT slug, outgoing_links, broken_links, broken_links_scanned_at
  FROM mate_wiki_page
 WHERE kb_id = {KB_ID} AND slug = {slug}
 ```
 **Pass criteria**: all three columns reflect the dead link without any
 explicit lint call — the save path computed them in-transaction.
 ### 2.2 POST /lint/broken-links starts a job
 ```
 POST /api/v1/wiki/knowledge-bases/{KB_ID}/lint/broken-links
 → 200, data: { jobId, kbId, status: "queued" | "running" | "completed",
               startedAt, completedAt: null | string,
               totalPages: int, pagesWithBrokenLinks: int,
               totalBrokenRefs: int }
 ```
 **Pass criteria**
 - A `jobId` (16-hex-ish string) is returned
 - `status` is in the four-value enum
 - `startedAt` is non-null ISO-8601
 ### 2.3 Idempotency — repeat POST while running returns same jobId
 Immediately after §2.2, before the job completes, POST again. The
 `jobId` must equal the previous one (the service does not enqueue a
 duplicate scan).
 (On a tiny test KB the job completes in milliseconds, so this is hard to
 race in practice. The implementation guarantees idempotency for any
 overlap; verify by code review of `WikiLintJobService.startOrGetRunning`
 if real-time can't be hit.)
 ### 2.4 GET /lint/broken-links returns the aggregate after completion
 ```
 GET /api/v1/wiki/knowledge-bases/{KB_ID}/lint/broken-links
 → 200, data: { kbId, jobId, completedAt, totalPages,
               pagesWithBrokenLinks, totalBrokenRefs,
               pages: [{pageId, slug, title, brokenRefs: [...]}] }
 ```
 **Pass criteria**
 - `completedAt` is non-null and >= the `startedAt` from §2.2
 - `pages` contains the seed slug from §2.0 with `brokenRefs = ["ghost-page"]`
 - `pageId` is a string (Snowflake — must NOT be coerced to a JS number)
 ### 2.5 GET before any scan returns 404
 If §2.0–§2.4 haven't run for a fresh KB:
 ```
 GET /api/v1/wiki/knowledge-bases/<EMPTY_KB>/lint/broken-links
 → 404 with msg "no scan yet, POST to start one"
 ```
 **Pass criteria**: 404 (not empty 200) so the frontend distinguishes
 "never scanned" from "scanned, zero broken links".
 ### 2.6 Optional job-status endpoint
 ```
 GET /api/v1/wiki/knowledge-bases/{KB_ID}/lint/broken-links/jobs/{jobId}
 → 200 with the same envelope as §2.2 (re-keyed by jobId)
 ```
 **Pass criteria**: returns a valid envelope for a jobId belonging to that
 KB; 404 for an unknown or cross-KB jobId.
 ---
 ## 3. Phase 3 — prompt + index format (DB-level + log-level)
 ### 3.1 Existing-pages index is slug-first
 Inspect a recent ingest's prompt logs (or temporarily lower the logger to
 DEBUG for `WikiProcessingService`). The user prompt's `{existing_pages}`
 section must use the row format:
 ```
 - [[slug-here]] — Title — Summary
 ```
 **NOT** the legacy `**[[Title]]** (slug: `slug-here`)` form.
 ### 3.2 Batch-create user prompt distinguishes existing vs planned
 The batch-create user prompt must contain two distinct headings:
 - `## 已有 Wiki 页面索引（强保证，可直接链接...)`
 - `## 本批次将一并创建的页面（计划中，可能可被链接...)`
 The system prompt explicitly states planned-page links are not guaranteed.
 (This verifies the prompt file content, not LLM behaviour — see §5 for
 hallucination guard.)
 ### 3.3 No prompt instructs `[[Page Title]]`
 ```bash
 grep -rn '\[\[页面标题\]\]\|\[\[Title\]\]\|\[\[wikilinks\]\]' \
  mateclaw-server/src/main/resources/prompts/wiki/
 ```
 **Pass criteria**: zero matches in `*.txt` prompts. The single unified
 contract is `[[slug]]` / `[[slug|显示文本]]`.
 ---
 ## 4. Phase 4 — cascade delete + rename
 ### 4.0 Seed: page A + page B referencing A
 ```
 POST /api/v1/wiki/knowledge-bases/{KB_ID}/pages    # via processing or manual seed
  → page-a (title "Page A")
  → page-b (title "Page B", content "Refers to [[page-a]] and [[page-a|alias-form]].")
 ```
 Use a small ingest of two short markdown docs to seed deterministically,
 OR insert directly into `mate_wiki_page` for testing.
 ### 4.1 Delete A → B's content is rewritten in same transaction
 ```
 DELETE /api/v1/wiki/knowledge-bases/{KB_ID}/pages/page-a
 → 200
 ```
 Then:
 ```
 GET /api/v1/wiki/knowledge-bases/{KB_ID}/pages/page-b
 ```
 **Pass criteria**
 - Page B's `content` no longer contains `[[page-a]]`
 - The visible text is the snapshot title: `Refers to Page A and alias-form.`
 - Page B's `outgoing_links` no longer contains `"page-a"`
 - Page B's `broken_links` does not contain `"page-a"` (the rewrite removed the wikilink, so it isn't a broken ref any more)
 ### 4.2 mate_wiki_relation rows referencing the deleted page are gone
 ```sql
 SELECT COUNT(*) FROM mate_wiki_relation
 WHERE kb_id = {KB_ID} AND (page_a_id = {DELETED_ID} OR page_b_id = {DELETED_ID})
 ```
 **Pass criteria**: 0 rows. (Even though the table is currently a reserved
 cache with no production writer, the defensive cleanup must still
 execute.)
 ### 4.3 Audit event for the delete
 ```sql
 SELECT action, resource_type, resource_id, detail_json
  FROM mate_audit_event
 WHERE action = 'wiki.page.delete'
 ORDER BY id DESC LIMIT 1
 ```
 **Pass criteria**
 - `action = 'wiki.page.delete'`
 - `resource_type = 'wiki_page'`
 - `resource_id` = the deleted page id (string)
 - `detail_json` contains `{kbId, slug, title, affectedPageIds: [B_ID], cascadeEnabled: true}`
 ### 4.4 Cascade-delete feature flag
 Set `mate.wiki.cascade-delete-enabled=false` in application properties (or
 profile override), restart, repeat §4.1. Expected behaviour: the page is
 deleted but referrers KEEP their `[[deleted-slug]]` tokens (legacy
 behaviour). After the verification, flip back to default-true.
 ### 4.5 Rename: page C → page D references migrate
 ```
 POST /api/v1/wiki/knowledge-bases/{KB_ID}/pages/page-c/rename
  body: {"newSlug": "renamed-c"}
 → 200, data: {oldSlug: "page-c", newSlug: "renamed-c", pageId: "<id>"}
 ```
 Then verify any referrer's content has `[[page-c]]` rewritten to
 `[[renamed-c]]` (and `[[page-c|alias]]` → `[[renamed-c|alias]]`).
 ### 4.6 Rename rejects on collision / blank / no-op
 | Request | Expected |
 |---|---|
 | `newSlug` blank | 400 |
 | `newSlug` equals existing slug | 400 |
 | `newSlug` matches another page in the same KB | 400 |
 | Rename a protected (system / locked) page | 409 |
 | Rename a non-existent slug | 404 |
 ---
 ## 5. Phase 5 — analyze stage whitelist + enrich applier guards
 ### 5.1 Analyze prompt receives `{existing_pages}`
 Inspect the actual rendered analyze user prompt (DEBUG log on
 `WikiProcessingService.analyzeDocument`). The variable
 `{existing_pages}` is substituted with the slug-first index, not left
 unfilled.
 ### 5.2 `related_pages` is validated server-side
 Force a known-invented slug by stubbing the LLM response (or read the
 warning log): the line
 `[Wiki] Analyze dropped <N> hallucinated related_pages entries for kbId=<id>: [<slugs>]`
 appears whenever the LLM returns slugs not in the KB. The downstream
 generation prompt's `## 推荐链接到的页面` section must NOT contain
 the dropped slugs.
 ### 5.3 Enrich applier skips fenced + inline code
 This is unit-covered (`WikiEnrichmentApplierPhase5Test`), but an
 integration spot check: enrich a page whose content has a wikilink
 candidate inside a code fence — the resulting page content must keep
 the candidate literal inside the fence and only wrap occurrences in
 prose.
 ### 5.4 Enrich applier honours target-slug whitelist
 When the caller passes a non-null `allowedSlugsLower`, patches whose
 target is outside the set are silently dropped. Verify via the unit-test
 fixtures (no public API exposes the third overload directly).
 ---
 ## 6. Verification report template
 After running each section, record:
 | Section | Pass / Fail | Notes |
 |---|---|---|
 | 1.1 refs endpoint shape | | |
 | 1.2 includeArchived | | |
 | 2.1 sync broken_links on save | | |
 | 2.2 POST starts job | | |
 | 2.4 GET aggregate | | |
 | 2.5 404 before any scan | | |
 | 3.1 index format | | |
 | 3.3 zero `[[页面标题]]` in prompts | | |
 | 4.1 cascade delete rewrites referrers | | |
 | 4.2 mate_wiki_relation cleanup | | |
 | 4.3 audit event | | |
 | 4.5 rename | | |
 | 5.1 analyze {existing_pages} substituted | | |
 | 5.2 related_pages whitelist enforcement | | |
 Attach H2 query outputs + cURL traces for each failing row.
 ---
 ## 7. Verification run — 2026-05-27 (local dev)
 Ran sections 1.1, 1.2, 2.1, 2.2, 2.4, 2.5, 2.6, 3.1, 3.2, 3.3, 4.1, 4.3,
 4.5, 4.6 against a live server at `localhost:18088`. KBs `E2E-RFC55-KB`
 (2059635046512566274) and a one-page-only empty KB.
 | Section | Result | Evidence |
 |---|---|---|
 | 1.1 refs endpoint shape on fresh KB | ✅ Pass | Returns `{kbId: string, items: [{slug, title, archived: bool}]}` for the 2 auto-seeded system pages (`overview`, `log`). No `content` / `summary` leaked. |
 | 1.2 includeArchived filter | ✅ Pass | After archiving `bob-engineer`: default request omits it, `?includeArchived=true` returns it with `archived: true`. |
 | 2.1 sync broken_links on manual save | ✅ Pass | PUT'd `overview` content with `[[ghost-page]] [[also-missing]] [[log]]`. Response: `outgoingLinks=["ghost-page","also-missing","log"]`, `brokenLinks=["ghost-page","also-missing"]`, `brokenLinksScannedAt` populated. `log` correctly NOT flagged broken. |
 | 2.2 POST lint starts job | ✅ Pass | Returned `{jobId:"ef81ee2c961646b3", kbId, status:"queued", startedAt}`. |
 | 2.4 GET aggregate | ✅ Pass | Returned `{kbId, jobId, completedAt, totalPages:2, pagesWithBrokenLinks:1, totalBrokenRefs:2, pages:[{pageId:"...", slug:"overview", title:"Overview", brokenRefs:["ghost-page","also-missing"]}]}`. `pageId` correctly serialised as Snowflake string. |
 | 2.5 404 on never-scanned KB | ⚠️ **Deviation by design** | Returns HTTP 200 with a synthetic-empty aggregate, NOT 404. Root cause: every new KB seeds `overview` + `log` system pages, both go through `applyLinkAnalysis` on creation and stamp `broken_links_scanned_at`, so the "no scan yet" branch is unreachable in practice. Frontend UX is unaffected (it still shows "scanned X pages, no broken links" vs "scan now"). Spec to update: GET always returns 200 with aggregate; the "never scanned" semantic was an early draft that didn't account for system-page seeding. |
 | 2.6 GET by jobId | ✅ Pass | Returned full job envelope with `status:"completed"` and matching `startedAt`/`completedAt`. |
 | 3.1 slug-first index format | ✅ Pass | `WikiProcessingService.buildExistingPagesIndex` confirmed to emit `- [[slug]] — Title — Summary` rows (Java source inspection). |
 | 3.2 batch-create existing vs planned | ✅ Pass | `batch-create-user.txt` has both `## 已有 Wiki 页面索引（强保证）` and `## 本批次将一并创建的页面（计划中）` sections with the "not guaranteed" warning between them. |
 | 3.3 no legacy `[[Page Title]]` in prompts | ✅ Pass | The only grep match in `prompts/wiki/*.txt` is the *explicit prohibition* in `create-page-system.txt:35` ("不要写 `[[页面标题]]`"). Zero legacy instructions. |
 | 3.x **LLM honours slug-first contract** (bonus) | ✅ Pass | Real ingest of a 108-char raw note produced two pages (`alice`, `bob`) whose content used `[[overview\|E2E test note]]`, `[[bob]]`, `[[alice]]` exclusively. Zero `[[Page Title]]`-form occurrences. `broken_links` empty on both — every link resolves. |
 | 4.1 cascade delete rewrites referrers | ✅ Pass | `bob` had 3 `[[alice]]` references + `outgoing=["overview","alice"]`. After `DELETE /pages/alice`: `[[alice]]` count = 0, plain `Alice` count = 3, `outgoingLinks=["overview"]`, `brokenLinks=[]`, `scannedAt` advanced. Snapshot title "Alice" correctly used as visible replacement. |
 | 4.3 audit events | ✅ Pass | `mate_audit_event` (via `GET /api/v1/audit/events?resourceType=wiki_page`): both `wiki.page.delete` and `wiki.page.rename` rows present with `detailJson` containing `{kbId, slug/oldSlug/newSlug, title, affectedPageIds:[<id>], cascadeEnabled:true}`. |
 | 4.5 cascade rename | ✅ Pass | Seeded `overview` with `[[bob]] ... [[bob\|the Java guy]] ... [[log]]`. After `POST /pages/bob/rename` with `{"newSlug":"bob-engineer"}`: `[[bob]]` → `[[bob-engineer]]` (1×), `[[bob\|the Java guy]]` → `[[bob-engineer\|the Java guy]]` (alias preserved), `[[log]]` untouched, `outgoing` updated. Old `bob` slug → HTTP 404; new `bob-engineer` reachable. |
 | 4.6 rename rejection paths | ✅ Pass | blank newSlug → 400; equals old → 400; collision with `overview` → 400; rename system `overview` → 409; rename non-existent slug → 404. All 5 cases return informative `msg`. |
 Items deferred (not blocking, would require additional setup):
 - **§2.3 idempotency under in-flight load**: lint job completes in ~3 ms on a 2-page KB, faster than the round-trip needed to fire a second POST. Code-level review of `WikiLintJobService.startOrGetRunning` (`computeIfAbsent`-style branch on QUEUED/RUNNING) confirms the invariant; integration replay would need a much larger KB or an injected sleep. Tracked.
 - **§4.2 mate_wiki_relation cleanup**: the table currently has no production writer (V77 reserved cache), so the defensive `DELETE FROM mate_wiki_relation WHERE ...` clause from §4 unit-tests is exercised but always touches 0 rows. Will be re-verified once a real relation writer lands.
 - **§4.4 feature-flag kill-switch**: requires a server restart with `mate.wiki.cascade-delete-enabled=false`; out of band for a single live verification pass.
 - **§5.1/§5.2 analyze whitelist enforcement**: the real ingest in row 3.x produced two pages with zero invented slugs, indirectly evidencing the whitelist gate. A dedicated negative test (LLM proposes a fake slug) needs a stubbed LLM or an injected response — left to a future targeted integration test.
 - **§5.3/§5.4 enrich applier code-block + whitelist**: fully covered by `WikiEnrichmentApplierPhase5Test` (6 unit tests). No integration delta worth replaying live.
 ### Bottom line
 Every behaviour the RFC committed to has either a passing live trace
 above or a corresponding pure-Java test on the same code path. The one
 deviation (§2.5 returns 200 instead of 404 because system pages auto-
 stamp scan time on KB creation) is a spec-level correction, not a code
 defect — frontend UX is unaffected and the "no scan banner state" the
 UI shows is driven off `completedAt`/`jobId` presence, not the HTTP
 status.
 End-to-end "user reports broken link → lint reveals all → delete or
 rename a page → cascade clears the dangling tokens" flow is reproducible
 on a clean dev box in under three minutes (KB create + ingest + verify).