Commit Graph

13918 Commits

Author SHA1 Message Date
FFXN
06cd47807c feat(knowledge-fs): add content handling for initial source preview and cleanup tasks 2026-09-03 21:01:21 +08:00
FFXN
069be90768 fix (web): show syncing status for initial knowledge sources 2026-09-03 19:03:58 +08:00
FFXN
43d50ce021 feat(knowledge-fs): enhance capability grant admission middleware with deletion handling options 2026-09-03 18:55:37 +08:00
FFXN
51085c6480 Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge 2026-09-03 18:55:23 +08:00
FFXN
ce5d702c46 feat(knowledge-fs): enhance capability grant admission middleware with deletion handling options 2026-09-03 18:54:43 +08:00
Jyong
8569c6be53 test(api): track the typed key binding migration as the alembic head
Migration 9a4e7d1c2b60 (dataset API key binding resource type) is the new
single head after c3f1a9b2e6d4; update the merge-head guard accordingly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 06:07:22 -04:00
Jyong
79116d8c4b feat(knowledge-fs): scope dataset API keys by knowledge base type
Legacy knowledge bases (`datasets`) and KnowledgeFS spaces
(`knowledge_fs_control_spaces`) live in different tables, so a dataset API
key binding could only ever name a legacy dataset and the KnowledgeFS service
API never consulted bindings at all: a key scoped to specific knowledge bases
still reached every KnowledgeFS space in the workspace.

- Bindings carry a `resource_type` (`dataset` | `knowledge_fs_space`) and a
  nullable `control_space_id` (migration 9a4e7d1c2b60, existing rows stay
  `dataset`), with a CHECK that exactly one id column matches the type.
- `dataset_api_key_service` exposes the key scope by kind, validates and
  binds KnowledgeFS spaces (tenant-owned, not deleting/deleted), and cleans up
  keys scoped only to a space when its deletion is requested. The orphan-key
  cleanup is now NULL-safe so a space binding keeps a key alive.
- Legacy dataset routes only honour `dataset` bindings; the KnowledgeFS
  authorization service only honours `knowledge_fs_space` bindings and raises
  a scope error that the service API maps to 403 (unknown keys stay 401).
- Console key creation accepts `knowledge_space_ids`; list/create responses
  return them next to `dataset_ids`.
- The web scope picker lists KnowledgeFS spaces alongside legacy datasets
  when KnowledgeFS is enabled and submits the ids by kind; the scope column
  counts both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 05:59:39 -04:00
Jyong
180ac5ad4d refactor(knowledge-fs): remove deprecated buffered service API routes
Drop the deprecated `POST /knowledge-fs/spaces/<id>/queries` route and the
deprecated buffered `POST .../documents` upload from the service API. Both
only ever failed closed; queries go through admission + the query stream,
documents through the durable source import flow. Regenerate the service
contracts and markdown docs accordingly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 05:59:39 -04:00
Jyong
06a363509e fix(knowledge-fs): show only the shared prefix as the service API endpoint
The API access dialog appended `/queries/admission` to the endpoint it
displays, which is one operation rather than the base every KnowledgeFS
service route shares. Show `<service api>/knowledge-fs/spaces/<id>` instead.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 05:59:39 -04:00
Stephen Zhou
cbfd1f69ee
refactor(knowledge-fs): split translations into dedicated namespace 2026-09-03 17:37:59 +08:00
FFXN
5b7f15f2b1 test(knowledge-fs): add tests for dataset upgrade API with disabled knowledge FS 2026-09-03 17:20:56 +08:00
Stephen Zhou
45d89701c3
Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge 2026-09-03 17:03:32 +08:00
FFXN
25277f66d6 fix(knowledge-fs): harden semantic chunking and deletion cleanup
- retry semantic chunking when the LLM returns an invalid schema
- add corrective schema instructions to retry prompts
- remove capability grant foreign keys from durable deletion records
- preserve capability grant IDs as immutable audit provenance
- add migration and regression coverage
2026-09-03 17:00:17 +08:00
FFXN
de8e91fec2 Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge 2026-09-03 17:00:08 +08:00
FFXN
644a9b96a6 fix(knowledge-fs): harden semantic chunking and deletion cleanup
- retry semantic chunking when the LLM returns an invalid schema
- add corrective schema instructions to retry prompts
- remove capability grant foreign keys from durable deletion records
- preserve capability grant IDs as immutable audit provenance
- add migration and regression coverage
2026-09-03 16:59:57 +08:00
Stephen Zhou
31e52a7eeb
Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-09-03 16:57:39 +08:00
Stephen Zhou
3611a93a3a
Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge
# Conflicts:
#	api/tests/unit_tests/services/test_knowledge_fs_initial_source_reliability.py
2026-09-03 16:57:14 +08:00
Stephen Zhou
ade154b878
Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge 2026-09-03 16:50:05 +08:00
Stephen Zhou
611e7f6d17
test(knowledge-fs): allow canonical crawl duplicates 2026-09-03 16:49:44 +08:00
github-actions[bot]
363753a0c5
chore(i18n): sync translations with en-US (#41733)
Co-authored-by: claude[bot] <41898282+claude[bot]@users.noreply.github.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
2026-09-03 08:46:49 +00:00
Stephen Zhou
e691e0bf1d
Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-09-03 16:33:04 +08:00
Stephen Zhou
42cb6519ea
Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-09-03 16:32:30 +08:00
FFXN
ad66d2c08a fix(knowledge-fs): update crawl function to accept WebsiteCrawlInput type 2026-09-03 16:31:53 +08:00
FFXN
239dedba48 fix(knowledge-fs): enhance website crawl handling with support for exact page fetching 2026-09-03 16:20:42 +08:00
FFXN
f51a759e0c Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge
# Conflicts:
#	api/knowledge-fs-contract.lock.json
2026-09-03 16:20:24 +08:00
FFXN
54335db161 fix(knowledge-fs): enhance website crawl handling with support for exact page fetching 2026-09-03 16:20:01 +08:00
FFXN
6d9d388c73 fix(knowledge-fs): enhance website crawl handling with support for exact page fetching 2026-09-03 16:19:47 +08:00
yyh
e982a3de33
test(web): remove workspace switcher browser test (#41728) 2026-09-03 08:06:17 +00:00
Stephen Zhou
137eb19de9
chore(web): prune unused i18n keys (#41725) 2026-09-03 08:01:55 +00:00
Zhu Lei
612a929907
fix(web): stringify object editor values before Monaco createModel (#41719) 2026-09-03 07:51:41 +00:00
Stephen Zhou
29acaabc6a
chore(web): canonicalize Tailwind classes (#41718) 2026-09-03 07:33:05 +00:00
Jingyi
642ad2fffe
fix(web): align workspace switcher states and semantics (#41713) 2026-09-03 07:18:49 +00:00
Jyong
c8aa2d4f59 feat(knowledge-fs): report specific, actionable background task failures
Every document-processing failure surfaced as "the document could not be
processed", and every data-source failure as "the source could not be read",
because the root cause was erased at three points: the compilation classifier
collapsed any coded-but-non-retryable error into DOCUMENT_COMPILATION_FAILED,
datasource runtime errors were reduced to per-operation constants, and the
console mapped dozens of codes onto a handful of generic sentences without the
stage or a support reference.

- knowledge-fs: keep the specific public code for any coded error in the
  compilation classifier (retry follows the error or the catalog policy), tag
  previously uncoded failures (semantic chunking output, embedding dimension,
  PDF rendering, lease loss, generation model, unsupported file type) with
  public codes, give embedding provider errors their own codes instead of the
  parser's, alias datasource runtime codes, and register 17 catalog entries
  (model response invalid, parser unsupported type, PDF render, crawl page
  not found, provider timeout/unavailable/rejected, content too large, ...).
- api: extend the public error-code enum (including the missing
  DOCUMENT_PARSER_TIMEOUT) and regenerate the contracts.
- web: per-code copy that says what happened and what to do, in all 24
  locales, plus a "failed while <stage> · reference <id>" line under each
  failed task in the background-task drawer and the document task list.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 03:09:29 -04:00
Stephen Zhou
947b654d14
fix(knowledge-fs): allow dismissing finished tasks 2026-09-03 14:37:23 +08:00
Benjamin
8f7cca3ae2
fix(web): guard LLM node model_selector against persisted null (#41707) 2026-09-03 06:20:48 +00:00
Byron.wang
b015aa416f
refactor(api): extract account login service and consolidate adapters (#41184) 2026-09-03 06:20:21 +00:00
非法操作
ba1d5ba172
refactor(api): centralize remote file handling (#41627) 2026-09-03 06:00:13 +00:00
Jyong
57d03faa98 fix(workflow): let the KnowledgeFS retrieval node run on text or image alone
The KnowledgeFS retrieval node treated the query text as mandatory even though
a query image is a complete input on its own. Text and image are now
alternatives: the node needs at least one of them, never both.

- web: drop the required mark from the query text field; the checklist fails
  only when neither variable is bound (new i18n message in every locale).
- api: make `query_variable_selector` optional with a model-level rule that
  one of the two selectors must be set, and only map the query variable when
  it is bound. At runtime an empty text without images is a configuration
  error; an image-only query retrieves with an empty text, merges spaces on
  their own scores instead of a text reranker, skips automatic metadata
  extraction, and skips the text-based failed-query capture.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 01:52:19 -04:00
Jyong
4d71c4a83a fix(knowledge-fs): allow one unresolved bad case per retrieval trace
A retrieval test record could be flagged as a bad case any number of times,
piling duplicate open cases onto the quality page. Refuse to file another bad
case while the trace already has one that is open or replaying, and let the
retrieval test page know about it.

- knowledge-fs: `createBadCase` looks up an unresolved case for the trace
  inside its transaction and throws `QualityBadCaseAlreadyOpenError`; the
  create route answers 409 with `QUALITY_BAD_CASE_ALREADY_OPEN`. Quality trace
  list items expose `openBadCaseId`.
- api: pass `open_bad_case_id` through the console trace response;
  regenerate the contracts.
- web: show the saved-as-bad-case state instead of the flag action when the
  selected trace already has an unresolved case, treat a 409 on flagging the
  same way, and refresh the history so the state follows the server.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 01:40:11 -04:00
zyssyz123
688107c3e5
fix(api): bound suggested question generation latency (#41705) 2026-09-03 05:02:09 +00:00
非法操作
d8c3f7c5ee
refactor(openapi): inject file upload service (#41606) 2026-09-03 04:42:31 +00:00
非法操作
69484a07e1
refactor(console): inject file service (#41603) 2026-09-03 04:42:30 +00:00
非法操作
5f4b1f867d
refactor(web): inject file upload service (#41586) 2026-09-03 04:42:30 +00:00
Stephen Zhou
af7f2b75b3
fix(knowledge-fs): refresh contract subtree lock 2026-09-03 12:03:14 +08:00
Eddy ZHANG
fc7193cf04
refactor(service_api): dep-inject the dataset create payload with @model_validate (#41651) 2026-09-03 03:52:41 +00:00
yyh
59021616d4
refactor(web): migrate legacy form controls to Dify UI (#41682) 2026-09-03 03:50:23 +00:00
Stephen Zhou
8901daa517
test(knowledge-fs): track patched fast-uri version 2026-09-03 11:48:20 +08:00
非法操作
ab0e92b893
refactor(service-api): inject file upload service (#41585) 2026-09-03 03:41:45 +00:00
Stephen Zhou
1279b4008f
fix(knowledge-fs): upgrade vulnerable fast-uri 2026-09-03 11:33:45 +08:00
Wu Tianwei
b0b495d243
feat(web): make markdown form field name length configurable (#41697) 2026-09-03 03:21:10 +00:00