Commit Graph

2845 Commits

Author SHA1 Message Date
FFXN
3e5b4640d6 fix(api): fix CI 2026-09-04 22:17:16 +08:00
FFXN
2f2aa585bd Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge 2026-09-04 22:07:17 +08:00
FFXN
b3ae5b8da2 fix(knowledge-fs): retry transient embeddings and recover website source state 2026-09-04 22:06:49 +08:00
Stephen Zhou
57a283d88c
Merge remote-tracking branch 'origin/main' into deploy/konwledge
# Conflicts:
#	web/features/new-rag/__tests__/add-source-page.spec.tsx
#	web/features/new-rag/__tests__/document-detail-page.spec.tsx
#	web/features/new-rag/add-source-page.tsx
#	web/features/new-rag/components/add-source-exit-dialog.tsx
#	web/features/new-rag/crawl-selection-form.tsx
#	web/features/new-rag/document-chunk-tree.tsx
#	web/features/new-rag/document-detail-header.tsx
#	web/features/new-rag/document-detail-page.tsx
#	web/features/new-rag/document-detail-status.tsx
#	web/features/new-rag/document-list.tsx
#	web/features/new-rag/documents-page.tsx
#	web/features/new-rag/documents/__tests__/page.spec.tsx
#	web/features/new-rag/knowledge-space-shell.tsx
#	web/features/new-rag/new-knowledge-list.tsx
#	web/features/new-rag/processing-tasks-drawer.tsx
#	web/features/new-rag/sources-page.tsx
#	web/features/new-rag/sources/create/__tests__/website-crawl-preview.spec.tsx
#	web/features/new-rag/sources/create/website-crawl-preview.tsx
2026-09-04 21:22:37 +08:00
Stephen Zhou
a2740f12a8
Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge 2026-09-04 21:11:18 +08:00
FFXN
8be760ac12 fix(knowledge-fs): preserve website preview bindings and surface import failures 2026-09-04 20:38:05 +08:00
Asuka Minato
08c7a2c5f3
test: migrate application generation sessions and ORM models to SQLite (#40509) 2026-09-04 12:16:31 +00:00
Stephen Zhou
38457f27fb
Merge remote-tracking branch 'origin/main' into deploy/konwledge
# Conflicts:
#	.agents/skills/how-to-write-component/references/data.md
#	.agents/skills/how-to-write-component/references/ownership.md
#	api/controllers/inner_api/__init__.py
#	api/tests/test_containers_integration_tests/services/test_account_service.py
#	api/tests/unit_tests/services/test_account_service.py
#	packages/iconify-collections/custom-public/icons.json
#	packages/iconify-collections/custom-public/info.json
#	web/features/new-rag/__tests__/new-knowledge-list.spec.tsx
#	web/features/new-rag/components/knowledge-space-card.tsx
#	web/features/new-rag/document-chunk-tree.tsx
#	web/features/tag-management/components/tag-search-content.tsx
#	web/features/tag-management/components/tag-selector.tsx
#	web/service/client.spec.ts
2026-09-04 19:42:31 +08:00
Stephen Zhou
6aba69326e
fix(knowledge-fs): show v2 workflow citations 2026-09-04 19:21:59 +08:00
FFXN
9d4c9b7c63 Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge 2026-09-04 18:11:04 +08:00
非法操作
e6c9846726
refactor(api): isolate workflow app log queries (#41509) 2026-09-04 10:06:46 +00:00
FFXN
db44dbe561 fix(knowledge-fs): support namespace preview persistence and capability audits
- declare namespace preview tables and indexes in the runtime schema
- allow capability audit records before a control space exists
- preserve namespace audit details in the sanitized claims summary
- add schema, migration, and audit regression coverage
2026-09-04 17:56:14 +08:00
Jyong
b79b8a879b feat(knowledge-fs): record and count every knowledge space query
Workflow KnowledgeFS retrieval nodes ran through `retrieval-tests`, which
never wrote an AnswerTrace or an overview event, so their queries were
missing from the retrieval history and from the space overview; the history
also only ever showed each member their own traces.

- AnswerTrace gains a `source` (retrieval_test | workflow | service_api |
  agent | mcp; migration 0051_answer_trace_source) derived from the
  Capability v2 caller kind. The retrieval-tests route records one trace per
  run (stages, evidence bundle, profile metadata), returns its id as
  `answerTraceId`, and the workflow node's failed-retrieval capture attaches
  to that trace instead of creating a second record.
- The quality trace list exposes and filters by `source`; traces from other
  caller kinds are visible to any current reader of the space, and counts and
  scores fall back to the evidence embedded in the trace when no bundle row
  exists.
- Overview accounting: retrieval-tests and Research tasks now emit
  `query.requested`, and the Research job state machine emits
  `query.completed` / `query.failed` on terminal stages (wired for both the
  in-process gateway and the durable runtime), so query volume, answer rate
  and outcomes include every caller. Activity details keep `source` and
  `taskKind`.
- Console and service trace routes accept a `source` filter; the retrieval
  test page shows a source badge and an all / retrieval tests / workflow
  filter, with translations for every locale.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-04 05:48:17 -04:00
Pranav Agarwal
067a60e0ac
fix(api): enforce admin and RBAC permissions on model provider model credential GET endpoint (#41599) (#41600)
Co-authored-by: Yunlu Wen <yunlu.wen@dify.ai>
2026-09-04 09:17:41 +00:00
非法操作
b15fb30ff5
refactor(api): extract app statistic queries from console controllers (#41557) 2026-09-04 09:15:51 +00:00
FFXN
c6b2978f6e Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge
# Conflicts:
#	api/knowledge-fs-contract.lock.json
2026-09-04 16:56:11 +08:00
FFXN
40e30d9fa6 feat(knowledge-fs): fix CI. 2026-09-04 16:55:47 +08:00
Asuka Minato
7128456d57
test: migrate core agent runner sessions and ORM models to SQLite (#40508) 2026-09-04 08:50:33 +00:00
Stephen Zhou
876aab708e
fix(knowledge-fs): track workflow retrieval outcomes 2026-09-04 16:50:09 +08:00
bucketbase26
b5ee943a63
fix(trigger): sync plugin relationships on publish (#41756) 2026-09-04 08:33:48 +00:00
eux
7919df7ef5
fix(api): skip invalid markdown image URLs (#41343) 2026-09-04 08:17:33 +00:00
非法操作
f8930aba34
refactor(api): extract console workflow run application service (#41336) 2026-09-04 08:16:49 +00:00
Zhu Lei
949f6c4ad4
fix(api): correct Service API segments has_more pagination (#41776) 2026-09-04 08:00:53 +00:00
Byron.wang
09f301d969
refactor(api): move OpenAPI account access behind application services (#41308) 2026-09-04 07:38:01 +00:00
Jialong(Bruce) Li
95aeb0c362
fix(skills): tolerate __MACOSX metadata folder when importing a Skill package (#41757) 2026-09-04 07:25:01 +00:00
Sourav Rajvi
e0673077a7
fix: close unclosed think tags around agent tool calls (#41764)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-04 07:23:34 +00:00
Zhu Lei
acf18aad13
fix(api): keep segment attachments when attachment_ids omitted (#41777) 2026-09-04 07:21:32 +00:00
Zhu Lei
1942925eb1
fix(api): persist plugin trigger subscription expires_at from provider (#41773) 2026-09-04 07:21:17 +00:00
Zhu Lei
5355edefe3
fix(api): correct dataset and document list has_more pagination (#41784) 2026-09-04 07:19:33 +00:00
非法操作
cea89f784f
refactor: make explore localization request-driven (#41798) 2026-09-04 06:03:46 +00:00
Byron.wang
81abb7322d
ci: parallelize change detection and gate style jobs (#41678)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-04 05:05:30 +00:00
Wu Tianwei
550196e3b8
feat: app deployment v2 (#41444)
Co-authored-by: zhangx1n <zhangxin@dify.ai>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-04 02:01:35 +00:00
非法操作
8e2058a962
refactor: pass notification language explicitly (#41770) 2026-09-04 01:52:30 +00:00
Jyong
8150a8fca7 feat(knowledge-fs): rate limit console writes and every service API route
The legacy dataset console rate limits every mutating, download, and
retrieval route and the dataset service API limits every request, while
KnowledgeFS only covered uploads and query admission. Apply the knowledge
rate limit to all space-scoped POST/PUT/PATCH/DELETE console routes, the
document downloads, and research/query entrypoints (workspace-level source
previews, staged-upload discards, the deprecated buffered query route, and
the internal stream transport stay exempt). Extract the service API check
into `check_knowledge_rate_limit` and call it from the KnowledgeFS profile
helper so all service routes share the legacy per-workspace limit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 11:10:04 -04:00
Jyong
cce0d818cf feat(knowledge-fs): authorize spaces with the legacy dataset RBAC points
KnowledgeFS spaces used their own `knowledge_space_*` RBAC vocabulary, so
workspace roles configured for knowledge bases did not apply to them. Reuse
the legacy `dataset_*` permission points instead, following the mapping the
dataset console already applies:

- read      -> dataset_readonly
- create    -> dataset_create_and_management
- edit / document write -> dataset_edit
- delete    -> dataset_delete
- access config -> dataset_access_config
- query / research / trace detail -> dataset_retrieval_recall

The enterprise `/knowledge-fs/permission-keys/batch` lookup now answers in
that vocabulary and is translated back into the product capabilities the
console exposes as `permission_keys`, so the web stays unchanged; the
historical `knowledge_space_*` keys remain accepted during the transition.
Workspace-level checks send the mapped dataset scene with
`resource_type=dataset`, and the RBAC-disabled fallback grants every dataset
point KnowledgeFS consults.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 11:10:04 -04:00
Keith
e6a38fd793
refactor(models): remove legacy db.session property wrappers on Document (#41721) 2026-09-03 13:34:49 +00:00
FFXN
bbde5fab52 fix(knowledge-fs): update content handling to use KnowledgeFSCrawlImportPagePayload and add cleanup tests 2026-09-03 21:31:00 +08:00
FFXN
06cd47807c feat(knowledge-fs): add content handling for initial source preview and cleanup tasks 2026-09-03 21:01:21 +08:00
Jyong
8569c6be53 test(api): track the typed key binding migration as the alembic head
Migration 9a4e7d1c2b60 (dataset API key binding resource type) is the new
single head after c3f1a9b2e6d4; update the merge-head guard accordingly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 06:07:22 -04:00
Jyong
79116d8c4b feat(knowledge-fs): scope dataset API keys by knowledge base type
Legacy knowledge bases (`datasets`) and KnowledgeFS spaces
(`knowledge_fs_control_spaces`) live in different tables, so a dataset API
key binding could only ever name a legacy dataset and the KnowledgeFS service
API never consulted bindings at all: a key scoped to specific knowledge bases
still reached every KnowledgeFS space in the workspace.

- Bindings carry a `resource_type` (`dataset` | `knowledge_fs_space`) and a
  nullable `control_space_id` (migration 9a4e7d1c2b60, existing rows stay
  `dataset`), with a CHECK that exactly one id column matches the type.
- `dataset_api_key_service` exposes the key scope by kind, validates and
  binds KnowledgeFS spaces (tenant-owned, not deleting/deleted), and cleans up
  keys scoped only to a space when its deletion is requested. The orphan-key
  cleanup is now NULL-safe so a space binding keeps a key alive.
- Legacy dataset routes only honour `dataset` bindings; the KnowledgeFS
  authorization service only honours `knowledge_fs_space` bindings and raises
  a scope error that the service API maps to 403 (unknown keys stay 401).
- Console key creation accepts `knowledge_space_ids`; list/create responses
  return them next to `dataset_ids`.
- The web scope picker lists KnowledgeFS spaces alongside legacy datasets
  when KnowledgeFS is enabled and submits the ids by kind; the scope column
  counts both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 05:59:39 -04:00
Jyong
180ac5ad4d refactor(knowledge-fs): remove deprecated buffered service API routes
Drop the deprecated `POST /knowledge-fs/spaces/<id>/queries` route and the
deprecated buffered `POST .../documents` upload from the service API. Both
only ever failed closed; queries go through admission + the query stream,
documents through the durable source import flow. Regenerate the service
contracts and markdown docs accordingly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-03 05:59:39 -04:00
FFXN
5b7f15f2b1 test(knowledge-fs): add tests for dataset upgrade API with disabled knowledge FS 2026-09-03 17:20:56 +08:00
Stephen Zhou
3611a93a3a
Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge
# Conflicts:
#	api/tests/unit_tests/services/test_knowledge_fs_initial_source_reliability.py
2026-09-03 16:57:14 +08:00
zyssyz123
a702b3bb2b
fix(agent): surface missing tool credentials (#41726) 2026-09-03 08:53:55 +00:00
Stephen Zhou
ade154b878
Merge remote-tracking branch 'origin/deploy/konwledge' into deploy/konwledge 2026-09-03 16:50:05 +08:00
Stephen Zhou
611e7f6d17
test(knowledge-fs): allow canonical crawl duplicates 2026-09-03 16:49:44 +08:00
Byron.wang
692010f0fc
refactor(api): decouple console OAuth flow from legacy account services (#41188)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-09-03 08:47:14 +00:00
Stephen Zhou
42cb6519ea
Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-09-03 16:32:30 +08:00
Byron.wang
b015aa416f
refactor(api): extract account login service and consolidate adapters (#41184) 2026-09-03 06:20:21 +00:00
非法操作
ba1d5ba172
refactor(api): centralize remote file handling (#41627) 2026-09-03 06:00:13 +00:00