Migration 9a4e7d1c2b60 (dataset API key binding resource type) is the new
single head after c3f1a9b2e6d4; update the merge-head guard accordingly.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
Legacy knowledge bases (`datasets`) and KnowledgeFS spaces
(`knowledge_fs_control_spaces`) live in different tables, so a dataset API
key binding could only ever name a legacy dataset and the KnowledgeFS service
API never consulted bindings at all: a key scoped to specific knowledge bases
still reached every KnowledgeFS space in the workspace.
- Bindings carry a `resource_type` (`dataset` | `knowledge_fs_space`) and a
nullable `control_space_id` (migration 9a4e7d1c2b60, existing rows stay
`dataset`), with a CHECK that exactly one id column matches the type.
- `dataset_api_key_service` exposes the key scope by kind, validates and
binds KnowledgeFS spaces (tenant-owned, not deleting/deleted), and cleans up
keys scoped only to a space when its deletion is requested. The orphan-key
cleanup is now NULL-safe so a space binding keeps a key alive.
- Legacy dataset routes only honour `dataset` bindings; the KnowledgeFS
authorization service only honours `knowledge_fs_space` bindings and raises
a scope error that the service API maps to 403 (unknown keys stay 401).
- Console key creation accepts `knowledge_space_ids`; list/create responses
return them next to `dataset_ids`.
- The web scope picker lists KnowledgeFS spaces alongside legacy datasets
when KnowledgeFS is enabled and submits the ids by kind; the scope column
counts both.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
Drop the deprecated `POST /knowledge-fs/spaces/<id>/queries` route and the
deprecated buffered `POST .../documents` upload from the service API. Both
only ever failed closed; queries go through admission + the query stream,
documents through the durable source import flow. Regenerate the service
contracts and markdown docs accordingly.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
- retry semantic chunking when the LLM returns an invalid schema
- add corrective schema instructions to retry prompts
- remove capability grant foreign keys from durable deletion records
- preserve capability grant IDs as immutable audit provenance
- add migration and regression coverage
- retry semantic chunking when the LLM returns an invalid schema
- add corrective schema instructions to retry prompts
- remove capability grant foreign keys from durable deletion records
- preserve capability grant IDs as immutable audit provenance
- add migration and regression coverage
Every document-processing failure surfaced as "the document could not be
processed", and every data-source failure as "the source could not be read",
because the root cause was erased at three points: the compilation classifier
collapsed any coded-but-non-retryable error into DOCUMENT_COMPILATION_FAILED,
datasource runtime errors were reduced to per-operation constants, and the
console mapped dozens of codes onto a handful of generic sentences without the
stage or a support reference.
- knowledge-fs: keep the specific public code for any coded error in the
compilation classifier (retry follows the error or the catalog policy), tag
previously uncoded failures (semantic chunking output, embedding dimension,
PDF rendering, lease loss, generation model, unsupported file type) with
public codes, give embedding provider errors their own codes instead of the
parser's, alias datasource runtime codes, and register 17 catalog entries
(model response invalid, parser unsupported type, PDF render, crawl page
not found, provider timeout/unavailable/rejected, content too large, ...).
- api: extend the public error-code enum (including the missing
DOCUMENT_PARSER_TIMEOUT) and regenerate the contracts.
- web: per-code copy that says what happened and what to do, in all 24
locales, plus a "failed while <stage> · reference <id>" line under each
failed task in the background-task drawer and the document task list.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
The KnowledgeFS retrieval node treated the query text as mandatory even though
a query image is a complete input on its own. Text and image are now
alternatives: the node needs at least one of them, never both.
- web: drop the required mark from the query text field; the checklist fails
only when neither variable is bound (new i18n message in every locale).
- api: make `query_variable_selector` optional with a model-level rule that
one of the two selectors must be set, and only map the query variable when
it is bound. At runtime an empty text without images is a configuration
error; an image-only query retrieves with an empty text, merges spaces on
their own scores instead of a text reranker, skips automatic metadata
extraction, and skips the text-based failed-query capture.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
A retrieval test record could be flagged as a bad case any number of times,
piling duplicate open cases onto the quality page. Refuse to file another bad
case while the trace already has one that is open or replaying, and let the
retrieval test page know about it.
- knowledge-fs: `createBadCase` looks up an unresolved case for the trace
inside its transaction and throws `QualityBadCaseAlreadyOpenError`; the
create route answers 409 with `QUALITY_BAD_CASE_ALREADY_OPEN`. Quality trace
list items expose `openBadCaseId`.
- api: pass `open_bad_case_id` through the console trace response;
regenerate the contracts.
- web: show the saved-as-bad-case state instead of the flag action when the
selected trace already has an unresolved case, treat a 409 on flagging the
same way, and refresh the history so the state follows the server.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
Query images attached to a retrieval test vanished as soon as the run's
history record was selected, and were never shown again after a reload,
because nothing persisted them: the gateway dropped `queryImages` when writing
`answer_traces`, the console trace/research responses had no image field, and
the composer cleared its images on every record selection.
- knowledge-fs: add a nullable JSON `query_images` column to `answer_traces`
(migration 0049 for postgres and tidb), persist and read it in the answer
trace repository, and expose `queryImages` on quality trace list items.
- api: enrich trace and research task responses with
`KnowledgeFSQueryImageResponse`, resolving file names and short-lived signed
preview URLs only for files the acting account still owns; regenerate the
console and service contracts.
- web: keep composer images scoped to the selected record like the query
text, remember images per run in this session, fall back to persisted
previews (with a labelled placeholder when the file is gone), show them in
the result panel and as a count on history records, and carry them into
retests.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
Bring the LLM-driven "automatic" metadata filtering mode from the legacy
Knowledge Retrieval node to the KnowledgeFS retrieval node
(knowledge-retrieval-v2).
Backend:
- Accept `automatic` in `metadata_filtering_mode` and add
`metadata_model_config` to the node entity.
- Add `automatic_metadata_filter.py`, which reuses the legacy metadata
filter prompt templates, invokes the configured model, parses the JSON
answer and normalises operators/values against the KnowledgeFS
metadata catalog before building custom metadata conditions.
- Resolve the shared metadata catalog across the selected spaces through
the new `list_metadata_fields` app execution capability (paginated),
fail closed on missing/unavailable models and catalog admission errors,
fail open on extraction errors, and expose the outcome via
`metrics.metadata_filtering` and `llm_usage`.
- Inject the default extractor from the node factory.
- Fix an unescaped brace in the legacy completion prompt template that
broke `.format()`.
Frontend:
- Offer disabled/automatic/manual modes, normalise unknown persisted
modes to disabled, store the automatic model and completion params,
render the model selector through the shared MetadataFilter, and
require a model when saving in automatic mode.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnUM1CdV5LpcxZEQ2vu9Xc
