opensource/dify
2
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-09-05 08:48:10 +08:00
Code Issues Packages Projects Releases Wiki Activity
13,935 Commits 823 Branches 177 Tags 1.4 GiB
8150a8fca7
Commit Graph

424 Commits

Author SHA1 Message Date
Jyong
8150a8fca7 feat(knowledge-fs): rate limit console writes and every service API route 
The legacy dataset console rate limits every mutating, download, and
retrieval route and the dataset service API limits every request, while
KnowledgeFS only covered uploads and query admission. Apply the knowledge
rate limit to all space-scoped POST/PUT/PATCH/DELETE console routes, the
document downloads, and research/query entrypoints (workspace-level source
previews, staged-upload discards, the deprecated buffered query route, and
the internal stream transport stay exempt). Extract the service API check
into `check_knowledge_rate_limit` and call it from the KnowledgeFS profile
helper so all service routes share the legacy per-workspace limit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
 2026-09-03 11:10:04 -04:00
Jyong
79116d8c4b feat(knowledge-fs): scope dataset API keys by knowledge base type 
Legacy knowledge bases (`datasets`) and KnowledgeFS spaces
(`knowledge_fs_control_spaces`) live in different tables, so a dataset API
key binding could only ever name a legacy dataset and the KnowledgeFS service
API never consulted bindings at all: a key scoped to specific knowledge bases
still reached every KnowledgeFS space in the workspace.

- Bindings carry a `resource_type` (`dataset` | `knowledge_fs_space`) and a
  nullable `control_space_id` (migration 9a4e7d1c2b60, existing rows stay
  `dataset`), with a CHECK that exactly one id column matches the type.
- `dataset_api_key_service` exposes the key scope by kind, validates and
  binds KnowledgeFS spaces (tenant-owned, not deleting/deleted), and cleans up
  keys scoped only to a space when its deletion is requested. The orphan-key
  cleanup is now NULL-safe so a space binding keeps a key alive.
- Legacy dataset routes only honour `dataset` bindings; the KnowledgeFS
  authorization service only honours `knowledge_fs_space` bindings and raises
  a scope error that the service API maps to 403 (unknown keys stay 401).
- Console key creation accepts `knowledge_space_ids`; list/create responses
  return them next to `dataset_ids`.
- The web scope picker lists KnowledgeFS spaces alongside legacy datasets
  when KnowledgeFS is enabled and submits the ids by kind; the scope column
  counts both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
 2026-09-03 05:59:39 -04:00
Jyong
180ac5ad4d refactor(knowledge-fs): remove deprecated buffered service API routes 
Drop the deprecated `POST /knowledge-fs/spaces/<id>/queries` route and the
deprecated buffered `POST .../documents` upload from the service API. Both
only ever failed closed; queries go through admission + the query stream,
documents through the durable source import flow. Regenerate the service
contracts and markdown docs accordingly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
 2026-09-03 05:59:39 -04:00
Stephen Zhou
42cb6519ea
Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-09-03 16:32:30 +08:00
Eddy ZHANG
fc7193cf04
refactor(service_api): dep-inject the dataset create payload with @model_validate (#41651) 2026-09-03 03:52:41 +00:00
非法操作
ab0e92b893
refactor(service-api): inject file upload service (#41585) 2026-09-03 03:41:45 +00:00
Stephen Zhou
45bee40955
Merge remote-tracking branch 'origin/main' into deploy/konwledge 
# Conflicts:
#	api/controllers/console/datasets/datasets.py
#	api/tests/unit_tests/controllers/console/datasets/test_datasets.py
 2026-09-02 21:56:53 +08:00
YungLe
3cd1fc098c
feat: allow knowledge base API keys to be scoped to a single dataset (#37569) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-09-02 10:30:42 +00:00
Eddy ZHANG
a5445f2424
refactor(service_api): dep-inject segment payloads with @model_validate (#41571) 2026-09-02 02:59:30 +00:00
Stephen Zhou
cd7e1be7cb
Merge remote-tracking branch 'origin/main' into deploy/konwledge 
# Conflicts:
#	api/services/feature_service.py
#	api/tests/unit_tests/controllers/console/workspace/test_members.py
#	api/tests/unit_tests/core/workflow/nodes/agent_v2/test_validators.py
#	api/tests/unit_tests/services/test_system_feature_service_knowledge_fs.py
#	pnpm-lock.yaml
 2026-09-02 10:31:10 +08:00
Byron.wang
c4ea406586
refactor(api): decouple system features and web adapters (#40772) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-09-01 11:50:52 +00:00
Eddy ZHANG
6411d6235c
refactor(service_api): dep-inject query params with @model_validate (#41562) 2026-09-01 05:55:50 +00:00
Stephen Zhou
ecd41aed3f
Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-09-01 09:45:49 +08:00
zyssyz123
762019a160
fix(workflow): block manual runs for trigger workflows (#41519) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-08-31 09:30:52 +00:00
Eddy ZHANG
884ece570e
refactor(service_api): dep-inject dataset payloads with @model_validate (#41490) 2026-08-30 15:06:36 +00:00
Eddy ZHANG
8b3c2ee274
refactor(service_api): dep-inject app payloads with @model_validate (#41491) 2026-08-30 12:51:50 +00:00
Eddy ZHANG
37988de678
refactor(service_api): dep-inject payloads with @model_validate (#41374) 2026-08-28 14:47:18 +00:00
Stephen Zhou
30ed0907f3
Merge remote-tracking branch 'origin/main' into deploy/konwledge 
# Conflicts:
#	api/controllers/console/app/site.py
#	api/controllers/console/datasets/datasets.py
#	api/core/plugin/backwards_invocation/model.py
#	api/core/rag/rerank/rerank_model.py
#	api/tests/test_containers_integration_tests/controllers/console/auth/test_oauth_server.py
#	api/tests/unit_tests/core/plugin/test_backwards_invocation_model.py
#	api/tests/unit_tests/tasks/test_batch_clean_document_task.py
#	packages/contracts/generated/api/service/orpc.gen.ts
#	web/app/components/datasets/rename-modal/index.tsx
#	web/app/components/header/account-setting/model-provider-page/model-selector/popup-layout.tsx
#	web/features/tag-management/components/tag-search-content.tsx
 2026-08-27 10:06:48 +08:00
FFXN
18155535cf refactor(knowledge_fs): replace credential service with dataset API key authorization. Remove the API token feature at the space level for knowledgeFs. 2026-08-26 15:40:18 +08:00
Wu Tianwei
73dd8739ef
feat: optimize resource access config logic (#41164) 
Co-authored-by: fatelei <fatelei@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-08-26 07:18:57 +00:00
Stephen Zhou
1c6f0a0388
fix(api): restore Service API OpenAPI semantics (#41274) 2026-08-26 05:05:36 +00:00
Stephen Zhou
92016aa139
fix(api): align Service API OpenAPI contract (#41248) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-08-25 10:54:14 +00:00
AsperforMias
3f7e6cdcb4
fix(tts): propagate and validate actual audio MIME (#41043) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: WH-2099 <wh2099@pm.me>
 2026-08-24 03:38:09 +00:00
Stephen Zhou
db8c40b23e
Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-08-20 10:11:26 +08:00
非法操作
2f6e782215
refactor(api): reuse app definition queries for service API site (#40795) 2026-08-19 07:23:47 +00:00
Stephen Zhou
cc3f901598
Merge remote-tracking branch 'origin/main' into deploy/konwledge 
# Conflicts:
#	api/extensions/ext_celery.py
#	api/tests/unit_tests/controllers/console/auth/test_oauth_server.py
#	lint.config.ts
#	packages/contracts/knowledge-fs-contract.test.mjs
#	vite.config.ts
#	web/features/new-rag/__tests__/document-detail-page.spec.tsx
#	web/features/new-rag/__tests__/knowledge-space-shell.spec.tsx
#	web/features/new-rag/__tests__/processing-task-events.spec.ts
#	web/features/new-rag/create-knowledge-page.tsx
#	web/features/new-rag/document-detail-page.tsx
#	web/features/new-rag/knowledge-space-shell.tsx
 2026-08-17 09:50:10 +08:00
zy
141cbf9832
fix(api): support audio/x-m4a for ASR (#40776) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-08-16 02:47:31 +00:00
非法操作
eec092f93d
refactor(api): reuse app definition queries for service API info (#40476) 2026-08-14 09:38:35 +00:00
非法操作
e0fd3b264a
refactor(api): reuse app definition queries for app parameters (#40465) 2026-08-14 09:38:34 +00:00
非法操作
17c8ff6a33
refactor(api): reuse app definition queries in web and service APIs (#40463) 2026-08-14 09:38:33 +00:00
Jyong
b3a7c7a3e0 feat(knowledge-fs): unify public error handling 2026-08-12 07:00:59 -04:00
Stephen Zhou
574461604d
Merge remote-tracking branch 'origin/main' into deploy/konwledge 
# Conflicts:
#	api/commands/system.py
#	api/controllers/console/init_validate.py
#	api/controllers/console/workspace/account.py
#	api/controllers/console/wraps.py
#	api/controllers/openapi/auth/data.py
#	api/core/hosting_configuration.py
#	api/core/provider_manager.py
#	api/core/rag/retrieval/dataset_retrieval.py
#	api/services/workspace_service.py
#	api/tasks/workflow_cfs_scheduler/entities.py
#	api/tests/test_containers_integration_tests/controllers/console/auth/test_password_reset.py
#	api/tests/test_containers_integration_tests/services/test_feature_service.py
#	api/tests/unit_tests/controllers/console/auth/test_account_activation.py
#	api/tests/unit_tests/controllers/openapi/auth/test_data.py
#	api/tests/unit_tests/controllers/service_api/app/test_hitl_service_api.py
#	api/tests/unit_tests/extensions/otel/test_retrieval_tracing.py
#	api/tests/unit_tests/services/test_feature_service_knowledge_fs.py
#	dev/start-worker
#	packages/contracts/console.ts
#	packages/contracts/package.json
#	packages/iconify-collections/custom-public/icons.json
#	web/app/components/workflow/__tests__/index.spec.tsx
#	web/app/components/workflow/index.tsx
#	web/features/new-rag/document-chunk-tree.tsx
#	web/service/fetch.spec.ts
 2026-08-12 10:33:29 +08:00
WH-2099
ef8544b173
fix(api): bind dataset operations to owners (#40149) 2026-08-11 16:37:14 +00:00
Byron.wang
c1e94f8ee7
refactor(api): consolidate deployment edition handling (#40142) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-08-11 07:39:01 +00:00
Stephen Zhou
fde800e337
Merge remote-tracking branch 'origin/main' into deploy/konwledge 
# Conflicts:
#	.github/dependabot.yml
#	.github/workflows/deploy-dev.yml
#	api/app_factory.py
#	api/core/indexing_runner.py
#	api/services/feature_service.py
#	api/tests/test_containers_integration_tests/controllers/console/auth/test_password_reset.py
#	api/tests/test_containers_integration_tests/services/test_feature_service.py
#	api/tests/unit_tests/controllers/console/auth/test_account_activation.py
#	api/tests/unit_tests/controllers/service_api/app/test_conversation.py
#	api/tests/unit_tests/controllers/web/test_site.py
#	api/tests/unit_tests/core/rag/indexing/test_indexing_runner.py
#	api/tests/unit_tests/core/tools/test_base_tool.py
#	api/tests/unit_tests/extensions/otel/test_retrieval_tracing.py
#	api/tests/unit_tests/extensions/test_ext_login.py
#	api/tests/unit_tests/services/test_feature_service_deployment_edition.py
#	api/tests/unit_tests/services/test_feature_service_knowledge_fs.py
#	api/tests/unit_tests/services/test_feature_service_licensed_seats.py
 2026-08-07 09:55:10 +08:00
非法操作
8170a2a5f3
chore: more upload file size for paid user (#39967) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-08-06 03:23:16 +00:00
Stephen Zhou
5b939067e1
Merge remote-tracking branch 'origin/main' into deploy/konwledge 
# Conflicts:
#	api/tests/unit_tests/services/test_app_dsl_service.py
#	web/features/new-rag/__tests__/new-knowledge-list.spec.tsx
 2026-07-31 14:28:03 +08:00
Parman Mohammadalizadeh
908dc703ec
chore(api): consolidate chained .where() calls into single calls (#39819) 2026-07-30 16:59:50 +00:00
Jyong
a82290fe8f perf(knowledge-fs): reduce capability request overhead 2026-07-30 03:14:51 -04:00
Jyong
34f4e3b145 Update frontend components and application workflows 2026-07-29 06:18:05 -04:00
Jyong
6cfd1abb59 Refactor frontend architecture and simplify implementation 2026-07-29 02:30:27 -04:00
Stephen Zhou
1cde846bcf
fix(dataset): integrate New RAG with KnowledgeFS (#39621) 2026-07-27 12:52:44 +08:00
Jyong
fc7fabe544 Merge branch 'codex/migrate-knowledge-fs' into deploy/konwledge 
# Conflicts:
#	api/tests/unit_tests/services/enterprise/test_rbac_service.py
#	api/tests/unit_tests/services/test_agent_tool_inner_service.py
 2026-07-22 06:16:29 -04:00
Jyong
29858d0db0 Expand application capabilities and refactor core workflows 2026-07-22 05:09:09 -04:00
shakti
6a338654fb
fix: return 400 instead of 500 when file field is missing in audio-to-text endpoints (#39303) (#39322) 2026-07-21 02:19:24 +00:00
wangxiaolei
ef0115d340
chore: when dataset permission is not all team, update rbac config to… (#39227) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-07-20 12:59:29 +00:00
Byron.wang
ab3e4daa95
refactor: explicit DB session propagation across backend paths (#38559) 
Co-authored-by: WH-2099 <wh2099@pm.me>
 2026-07-15 06:48:28 +00:00
非法操作
38aec8b506
chore: chatflow api should check workflow_id and sandbox plan (#38892) 2026-07-15 02:01:53 +00:00
wangxiaolei
c278148c8f
feat: when knowledge permission is all team need update rbac setttings (#38911) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-07-14 09:07:33 +00:00
yyh
ab521c4cfd
refactor(chat): modernize speech-to-text pipeline (#38653) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Yansong Zhang <916125788@qq.com>
 2026-07-13 04:21:29 +00:00