Commit Graph

310 Commits

Author SHA1 Message Date
FFXN
06cd47807c feat(knowledge-fs): add content handling for initial source preview and cleanup tasks 2026-09-03 21:01:21 +08:00
Jyong
79116d8c4b feat(knowledge-fs): scope dataset API keys by knowledge base type
Legacy knowledge bases (`datasets`) and KnowledgeFS spaces
(`knowledge_fs_control_spaces`) live in different tables, so a dataset API
key binding could only ever name a legacy dataset and the KnowledgeFS service
API never consulted bindings at all: a key scoped to specific knowledge bases
still reached every KnowledgeFS space in the workspace.

- Bindings carry a `resource_type` (`dataset` | `knowledge_fs_space`) and a
  nullable `control_space_id` (migration 9a4e7d1c2b60, existing rows stay
  `dataset`), with a CHECK that exactly one id column matches the type.
- `dataset_api_key_service` exposes the key scope by kind, validates and
  binds KnowledgeFS spaces (tenant-owned, not deleting/deleted), and cleans up
  keys scoped only to a space when its deletion is requested. The orphan-key
  cleanup is now NULL-safe so a space binding keeps a key alive.
- Legacy dataset routes only honour `dataset` bindings; the KnowledgeFS
  authorization service only honours `knowledge_fs_space` bindings and raises
  a scope error that the service API maps to 403 (unknown keys stay 401).
- Console key creation accepts `knowledge_space_ids`; list/create responses
  return them next to `dataset_ids`.
- The web scope picker lists KnowledgeFS spaces alongside legacy datasets
  when KnowledgeFS is enabled and submits the ids by kind; the scope column
  counts both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 05:59:39 -04:00
Stephen Zhou
42cb6519ea
Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-09-03 16:32:30 +08:00
Jyong
c8aa2d4f59 feat(knowledge-fs): report specific, actionable background task failures
Every document-processing failure surfaced as "the document could not be
processed", and every data-source failure as "the source could not be read",
because the root cause was erased at three points: the compilation classifier
collapsed any coded-but-non-retryable error into DOCUMENT_COMPILATION_FAILED,
datasource runtime errors were reduced to per-operation constants, and the
console mapped dozens of codes onto a handful of generic sentences without the
stage or a support reference.

- knowledge-fs: keep the specific public code for any coded error in the
  compilation classifier (retry follows the error or the catalog policy), tag
  previously uncoded failures (semantic chunking output, embedding dimension,
  PDF rendering, lease loss, generation model, unsupported file type) with
  public codes, give embedding provider errors their own codes instead of the
  parser's, alias datasource runtime codes, and register 17 catalog entries
  (model response invalid, parser unsupported type, PDF render, crawl page
  not found, provider timeout/unavailable/rejected, content too large, ...).
- api: extend the public error-code enum (including the missing
  DOCUMENT_PARSER_TIMEOUT) and regenerate the contracts.
- web: per-code copy that says what happened and what to do, in all 24
  locales, plus a "failed while <stage> · reference <id>" line under each
  failed task in the background-task drawer and the document task list.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 03:09:29 -04:00
非法操作
ba1d5ba172
refactor(api): centralize remote file handling (#41627) 2026-09-03 06:00:13 +00:00
Jyong
4d71c4a83a fix(knowledge-fs): allow one unresolved bad case per retrieval trace
A retrieval test record could be flagged as a bad case any number of times,
piling duplicate open cases onto the quality page. Refuse to file another bad
case while the trace already has one that is open or replaying, and let the
retrieval test page know about it.

- knowledge-fs: `createBadCase` looks up an unresolved case for the trace
  inside its transaction and throws `QualityBadCaseAlreadyOpenError`; the
  create route answers 409 with `QUALITY_BAD_CASE_ALREADY_OPEN`. Quality trace
  list items expose `openBadCaseId`.
- api: pass `open_bad_case_id` through the console trace response;
  regenerate the contracts.
- web: show the saved-as-bad-case state instead of the flag action when the
  selected trace already has an unresolved case, treat a 409 on flagging the
  same way, and refresh the history so the state follows the server.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 01:40:11 -04:00
Byron.wang
be7d11ca1d
refactor(api): extract forgot password application service (#41111)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-03 01:38:59 +00:00
Stephen Zhou
45bee40955
Merge remote-tracking branch 'origin/main' into deploy/konwledge
# Conflicts:
#	api/controllers/console/datasets/datasets.py
#	api/tests/unit_tests/controllers/console/datasets/test_datasets.py
2026-09-02 21:56:53 +08:00
Jyong
afa771dcac feat(knowledge-fs): keep query images with retrieval test history
Query images attached to a retrieval test vanished as soon as the run's
history record was selected, and were never shown again after a reload,
because nothing persisted them: the gateway dropped `queryImages` when writing
`answer_traces`, the console trace/research responses had no image field, and
the composer cleared its images on every record selection.

- knowledge-fs: add a nullable JSON `query_images` column to `answer_traces`
  (migration 0049 for postgres and tidb), persist and read it in the answer
  trace repository, and expose `queryImages` on quality trace list items.
- api: enrich trace and research task responses with
  `KnowledgeFSQueryImageResponse`, resolving file names and short-lived signed
  preview URLs only for files the acting account still owns; regenerate the
  console and service contracts.
- web: keep composer images scoped to the selected record like the query
  text, remember images per run in this session, fall back to persisted
  previews (with a labelled placeholder when the file is gone), show them in
  the result panel and as a count on history records, and carry them into
  retests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-02 09:17:36 -04:00
YungLe
3cd1fc098c
feat: allow knowledge base API keys to be scoped to a single dataset (#37569)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-02 10:30:42 +00:00
Jyong
049a4c514e fix(knowledge-fs): show task object titles 2026-09-02 01:43:16 -04:00
Stephen Zhou
ecd41aed3f
Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-09-01 09:45:49 +08:00
FFXN
8c472c217b fix(knowledge-fs): add SOURCE_DOCUMENT_COMPILATION_FAILED error code and update DTOs 2026-08-31 18:12:42 +08:00
yyh
78c85ae7c1
fix(agent): add roster publication status contract (#41435)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-28 08:50:30 +00:00
Stephen Zhou
83e827e0b4
fix(knowledge-fs): remove provider sync policy from frontend 2026-08-28 15:55:49 +08:00
Stephen Zhou
9098955a77
Merge remote-tracking branch 'origin/main' into deploy/konwledge
# Conflicts:
#	api/tests/unit_tests/controllers/console/test_knowledge_fs_proxy.py
#	api/tests/unit_tests/services/test_knowledge_fs_proxy.py
#	web/app/components/workflow-app/hooks/use-available-nodes-meta-data.ts
2026-08-28 14:50:39 +08:00
Stephen Zhou
e5dc72812a
fix(knowledge-fs): support source configuration editing 2026-08-27 23:52:18 +08:00
Stephen Zhou
7c3b4aa3c6
fix(knowledge-fs): preserve DOCX image placement 2026-08-27 17:55:02 +08:00
Jyong
42b1be58f3 fix(knowledge-fs): repair document deletion task progress 2026-08-27 04:35:05 -04:00
Jyong
12adc39383 fix(knowledge-fs): preserve spreadsheet image placement 2026-08-27 03:01:57 -04:00
非法操作
bf54dd7814
refactor(api): remove unused workflow archived logs endpoint (#41345) 2026-08-27 03:48:57 +00:00
非法操作
32981687dd
refactor: remove unused workflow run export endpoint (#41335) 2026-08-27 02:38:27 +00:00
Stephen Zhou
30ed0907f3
Merge remote-tracking branch 'origin/main' into deploy/konwledge
# Conflicts:
#	api/controllers/console/app/site.py
#	api/controllers/console/datasets/datasets.py
#	api/core/plugin/backwards_invocation/model.py
#	api/core/rag/rerank/rerank_model.py
#	api/tests/test_containers_integration_tests/controllers/console/auth/test_oauth_server.py
#	api/tests/unit_tests/core/plugin/test_backwards_invocation_model.py
#	api/tests/unit_tests/tasks/test_batch_clean_document_task.py
#	packages/contracts/generated/api/service/orpc.gen.ts
#	web/app/components/datasets/rename-modal/index.tsx
#	web/app/components/header/account-setting/model-provider-page/model-selector/popup-layout.tsx
#	web/features/tag-management/components/tag-search-content.tsx
2026-08-27 10:06:48 +08:00
Byron.wang
30e6069851
refactor(api): thin OAuth controllers and extract application services (#40692) 2026-08-26 07:33:37 +00:00
Wu Tianwei
73dd8739ef
feat: optimize resource access config logic (#41164)
Co-authored-by: fatelei <fatelei@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-26 07:18:57 +00:00
非法操作
c563528a2d
refactor(api): standardize compliance downloads (#41057) 2026-08-26 06:32:13 +00:00
Stephen Zhou
92016aa139
fix(api): align Service API OpenAPI contract (#41248)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-25 10:54:14 +00:00
wangxiaolei
a6fe278662
feat: hide add library skill in agent ui (#41250)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-25 10:26:06 +00:00
wangxiaolei
ddcdbb2d14
feat: add skill enable feature switch (#41246) 2026-08-25 08:42:15 +00:00
非法操作
f360a7bf94
fix: allow OAuth trigger builder verification without credentials (#39199)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-25 05:32:27 +00:00
非法操作
ef92cb7c65
refactor(api): complete account education boundary (#41153)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-25 01:48:12 +00:00
林玮 (Jade Lin)
70a513fc62
fix(api): scope hosted credits by tenant plan (#41131)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-24 06:35:42 +00:00
wangxiaolei
0024a845f7
feat: support agent skill (#39675)
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: 起岚 <155608604+xiaoyaoqilan@users.noreply.github.com>
Co-authored-by: Parman Mohammadalizadeh <prmma23@gmail.com>
Co-authored-by: Escape0707 <tothesong@gmail.com>
Co-authored-by: Xin Zhang <zhangxin@dify.ai>
Co-authored-by: iridescentWen <66297467+iridescentWen@users.noreply.github.com>
Co-authored-by: Bond Zhu <37842169+MRZHUH@users.noreply.github.com>
Co-authored-by: KVOJJJin <jzongcode@gmail.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: 非法操作 <hjlarry@163.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: Chester <superdiao6@gmail.com>
Co-authored-by: WH-2099 <wh2099@pm.me>
Co-authored-by: Jony <619963502@qq.com>
Co-authored-by: Jony <13896935+zyz619963502zyz@users.noreply.github.com>
Co-authored-by: Harsh Kashyap <harsh.kashyap2001@gmail.com>
Co-authored-by: Harsh Kashyap <Harsh23Kashyap@users.noreply.github.com>
Co-authored-by: kenwoodjw <blackxin55+@gmail.com>
Co-authored-by: csurong <csurong1@gmail.com>
Co-authored-by: caosurong <surong.cao@thinkingdata.cn>
Co-authored-by: Taranum Wasu <81034301+Taranum01@users.noreply.github.com>
Co-authored-by: Taranum01 <50813317+Taranum01@users.noreply.github.com>
Co-authored-by: Pranav Agarwal <agarwalpranav0711@gmail.com>
Co-authored-by: Stephen Zhou <hi@hyoban.cc>
Co-authored-by: zl86790 <marshal_li_b@163.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
Co-authored-by: yunlu.wen <yunlu.wen@dify.ai>
Co-authored-by: yyh <yuanyouhuilyz@gmail.com>
2026-08-24 05:29:14 +00:00
Stephen Zhou
0564b75187
Merge remote-tracking branch 'origin/main' into deploy/konwledge
# Conflicts:
#	api/tests/unit_tests/commands/test_generate_swagger_specs.py
#	api/tests/unit_tests/controllers/console/app/test_agent_manage_guard.py
#	api/tests/unit_tests/extensions/test_ext_login.py
#	packages/dify-ui/src/alert-dialog/index.tsx
#	web/app/components/datasets/external-api/external-api-modal/index.tsx
#	web/app/components/datasets/list/dataset-card/components/operations-dropdown.tsx
#	web/features/new-rag/document-list.tsx
#	web/features/new-rag/sources-page.tsx
2026-08-24 11:03:38 +08:00
非法操作
ed45016475
refactor(api): standardize console billing portal errors (#41055)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-24 01:55:48 +00:00
Stephen Zhou
e6fe2cd5d7
Merge remote-tracking branch 'origin/main' into deploy/konwledge
# Conflicts:
#	api/tests/unit_tests/controllers/inner_api/test_auth_wraps.py
#	api/tests/unit_tests/services/test_async_workflow_service.py
#	web/features/new-rag/__tests__/new-knowledge-list.spec.tsx
2026-08-21 10:39:39 +08:00
GGbond
6f70ca8d26
refactor(api): decouple console tag management (#40843)
Co-authored-by: Byron Wang <byron@dify.ai>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-20 12:31:01 +00:00
FFXN
248df5dd25
fix(snippet): support snippet history version export (#41012)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-20 08:52:45 +00:00
非法操作
f97fc01c2f
refactor(api): extract recommended app query service (#40207) 2026-08-20 08:45:47 +00:00
chelsealong
d7b1531431
fix(snippets): implement delete for published snippet workflow versions (#41003)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-20 07:36:17 +00:00
Byron.wang
b4e13ec83e
refactor(api): add account profile application service (#40436)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-08-20 05:03:08 +00:00
Stephen Zhou
4880c5c771
feat(knowledge-fs): migrate add source to async imports 2026-08-20 11:30:43 +08:00
Stephen Zhou
a50ea7d56d
Merge branch 'deploy/konwledge' of https://github.com/langgenius/dify into deploy/konwledge 2026-08-20 10:18:08 +08:00
Stephen Zhou
db8c40b23e
Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-08-20 10:11:26 +08:00
Jyong
0391141936 feat(quality): show replay evidence details 2026-08-19 09:45:41 -04:00
Jyong
66e3ae951f feat(knowledge): add golden question evaluations 2026-08-19 05:54:05 -04:00
Jyong
9553e7710c fix(knowledge-fs): allow optional notes and ignore bad cases 2026-08-19 04:16:42 -04:00
Stephen Zhou
c11f31f5d3
fix(knowledge-fs): recover upgrade jobs in new knowledge list 2026-08-19 16:15:46 +08:00
zyssyz123
414f211cde
fix(auth): harden email code login verification (#40960) 2026-08-19 06:57:47 +00:00
Jyong
39c31ef46c fix(knowledge-fs): clarify golden question evidence editing 2026-08-19 02:40:20 -04:00