opensource
/
dify
mirror of https://github.com/langgenius/dify.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,583 Commits 383 Branches 165 Tags 562 MiB
Commit Graph

526 Commits

Author SHA1 Message Date
GuanMu 47f480c0dc
fix: unify log format, use placeholders instead of string concatenation (#24544) 2025-08-26 15:45:16 +08:00
huangzhuo1949 98473e9d4f
fix:external dataset weight rerank bug (#24533) 
Co-authored-by: huangzhuo <huangzhuo1@xiaomi.com>
 2025-08-26 14:54:40 +08:00
-LAN- 04954918a5
Merge commit from fork 
* fix(oraclevector): SQL Injection

Signed-off-by: -LAN- <laipz8200@outlook.com>

* fix(oraclevector): Remove bind variables from FETCH FIRST clause

Oracle doesn't support bind variables in the FETCH FIRST clause.
Fixed by using validated integers directly in the SQL string while
maintaining proper input validation to prevent SQL injection.

- Updated search_by_vector method to use validated top_k directly
- Updated search_by_full_text method to use validated top_k directly
- Adjusted parameter numbering for document_ids_filter placeholders

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2025-08-26 13:51:23 +08:00
twwu 85fd97e090 Merge branch 'main' into feat/rag-2 2025-08-25 15:30:18 +08:00
huayaoyue6 23dcb2dc1b
fix(vector): use semantic version comparison for version check (#24409) 2025-08-24 21:04:33 +08:00
-LAN- da9af7b547
[Chore/Refactor] Use centralized naive_utc_now for UTC datetime operations (#24352) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
 2025-08-22 23:53:05 +08:00
Asuka Minato 51cc2bf429
example of next(, None) (#24345) 2025-08-22 18:32:22 +08:00
twwu 8c44151e6f Merge branch 'main' into feat/rag-2 2025-08-22 17:40:34 +08:00
willzhao 5ab6bc283c
[CHORE]: x: T = None to x: Optional[T] = None (#24217) 2025-08-21 21:58:39 +08:00
Yongtao Huang 106ab7f2a8
Fix: safe defaults for BaseModel dict fields (#24098) 
Co-authored-by: Yongtao Huang <99629139+hyongtao-db@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2025-08-21 21:38:55 +08:00
Guangdong Liu 1abf1240b2
refactor: replace try-except blocks with contextlib.suppress for cleaner exception handling (#24284) 2025-08-21 18:18:49 +08:00
8bitpd a183b2affb
fix: rollback when AnalyticDB create zhparser failed (#24260) 
Co-authored-by: xiaozeyu <xiaozeyu.xzy@alibaba-inc.com>
 2025-08-21 15:00:26 +08:00
Amy 738aaee101
fix(api):Fix the issue of empty and not empty operations failing in k… (#24276) 
Co-authored-by: liumin <min.liu@tongdun.net>
 2025-08-21 14:43:08 +08:00
twwu dfd33b3d84 Merge branch 'main' into feat/rag-2 2025-08-21 09:43:51 +08:00
8bitpd 6b1606f4f4
fix: keep idempotent when init AnalyticdbVectorBySql (#24239) 
Co-authored-by: xiaozeyu <xiaozeyu.xzy@alibaba-inc.com>
 2025-08-20 23:22:27 +08:00
yihong 4c1ad40f8e
docs: format all md files (#24195) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-08-20 13:49:11 +08:00
twwu a1666fe058 Merge branch 'main' into feat/rag-2 2025-08-19 14:59:06 +08:00
He Wang 670d479e32
Bump pyobvector to 0.2.15 (#24120) 2025-08-18 17:36:27 +08:00
crazywoola 8288b1dcab
Revert "fix pg_vector extension requires SUPERUSER, but not availabl… (#24108) 2025-08-18 16:46:15 +08:00
Elvis_LEE 16d1289a0a
fix pg_vector extension requires SUPERUSER, but not available on Huawei Cloud RDS (#24093) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2025-08-18 16:29:36 +08:00
Bo Wu 790a6ec203
fix: return empty list instead of raising exception for qdrant search when score_threshold is 1 (#24032) 2025-08-18 12:44:05 +08:00
twwu 386614951f Merge branch 'main' into feat/rag-2 2025-08-18 11:16:18 +08:00
-LAN- e340fccafb
feat: integrate flask-orjson for improved JSON serialization performance (#23935) 2025-08-14 19:50:59 +08:00
jyong ae3addb922 add credential id 2025-08-12 15:43:11 +08:00
jyong 22b3933cc3 Merge branch 'main' into feat/rag-2 
# Conflicts:
#	api/core/workflow/entities/variable_pool.py
 2025-08-12 11:13:04 +08:00
engchina 7566d90dfe
fix issue #23758 (#23764) 
Co-authored-by: root <root@thinkpad-pc.localdomain>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2025-08-12 10:26:13 +08:00
Harry 543f80ad5d refactor: replace get_real_credential_by_id with get_datasource_credentials in multiple services for consistency 2025-08-11 20:04:04 +08:00
twwu fc779d00df Merge branch 'main' into feat/rag-2 2025-08-11 11:15:58 +08:00
yunqiqiliang 14e1c16cf2
Fix ClickZetta stability and reduce logging noise (#23632) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2025-08-08 22:57:47 +08:00
湛露先生 fd536a943a
word extractor cleans. (#20926) 
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
 2025-08-08 09:37:51 +08:00
yunqiqiliang 62772e8871
fix: ensure vector database cleanup on dataset deletion regardless of document presence (affects all 33 vector databases) (#23574) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2025-08-08 09:18:43 +08:00
Qiang Lee e9045a8838
Fix: Apply Metadata Filters Correctly in Full-Text Search Mode for Tencent Cloud Vector Database (#23564) 2025-08-07 05:36:06 -07:00
twwu 1b3860d012 Merge branch 'main' into feat/rag-2 2025-08-07 16:27:20 +08:00
yunqiqiliang e01510e2a6
feat: Add Clickzetta Lakehouse vector database integration (#22551) 
Co-authored-by: Claude <noreply@anthropic.com>
 2025-08-07 14:21:46 +08:00
twwu 40f3524cfe Merge branch 'main' into feat/rag-2 2025-08-06 14:23:51 +08:00
Yongtao Huang 6b8b31ff64
Remove unnecessary issubclass check (#23455) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2025-08-06 13:43:55 +08:00
jyong 2f163bad8f transform document 2025-08-05 18:16:24 +08:00
zxhlyh 201e4cd64d merge main 2025-08-05 10:30:53 +08:00
Yongtao Huang 406c1952b8
Fix version comparison with imported_version (#23326) 
Signed-off-by: Yongtao Huang <yongtaoh2022@gmail.com>
 2025-08-04 10:40:49 +08:00
wanttobeamaster da5c003f97
chore: tablestore full text search support score normalization (#23255) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-08-01 14:14:11 +08:00
jyong 79f30daf03 transform document 2025-08-01 13:59:11 +08:00
jyong 82819af55c transform document 2025-07-31 15:59:30 +08:00
jyong a7a4c8228e Merge branch 'main' into feat/rag-2 
# Conflicts:
#	web/app/components/workflow/hooks/use-workflow.ts
 2025-07-31 10:30:28 +08:00
Aurelius Huang ffddabde43
feat(notion): Notion Database extracts Rows content `in row order` and appends `Row Page URL` (#22646) 
Co-authored-by: Aurelius Huang <cm.huang@aftership.com>
 2025-07-30 21:35:20 +08:00
kenwoodjw 28478cdc41
feat: support metadata condition filter string array (#23111) 
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
 2025-07-30 16:13:45 +08:00
rhochman eee576355b
Fix: Support for Elasticsearch Cloud Connector (#23017) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-30 11:12:16 +08:00
zxhlyh a996c1d90c merge main 2025-07-29 15:45:01 +08:00
jyong e89398f415 add old auth transform 2025-07-29 14:13:50 +08:00
jyong 657e813c7f add old auth transform 2025-07-28 19:29:36 +08:00
jyong 829e6f0d1a add old auth transform 2025-07-28 19:29:07 +08:00
zxhlyh a95cf6f8b0 merge main 2025-07-28 16:00:38 +08:00
Yongtao Huang 1c05491f1c
Chore: remove duplicate TYPE_CHECKING import (#23013) 
Signed-off-by: Yongtao Huang <yongtaoh2022@gmail.com>
 2025-07-28 10:04:45 +08:00
Asuka Minato a189d293f8
make logging not use f-str, change others to f-str (#22882) 2025-07-25 10:32:48 +08:00
twwu bae2af0c85 Merge branch main into feat/rag-2 2025-07-24 17:40:04 +08:00
Asuka Minato ef51678c73
orm filter -> where (#22801) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2025-07-24 00:57:45 +08:00
wanttobeamaster 8278b39f85
fix tablestore full text search bug (#22853) 2025-07-23 19:31:47 +08:00
wanttobeamaster 1c3c40db69
fix: tablestore TypeError when vector is missing (#22843) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-07-23 18:59:16 +08:00
wlleiiwang b4e152f775
FEAT: Tencent Vector search supports backward compatibility with the previous score calculation approach. (#22820) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-07-23 15:38:31 +08:00
Asuka Minato 6d3e198c3c
Mapped column (#22644) 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-23 00:39:59 +08:00
wanttobeamaster a2048fd0f4
fix: tablestore vdb support metadata filter (#22774) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-07-22 16:48:59 +08:00
zxhlyh 3b8d96f45c merge main 2025-07-22 13:52:24 +08:00
issac2e 58d92970a9
Optimize tencent_vector knowledge base deletion error handling with batch processing support (#22726) 
Co-authored-by: liuchen15 <liuchen15@gaotu.cn>
Co-authored-by: crazywoola <427733928@qq.com>
 2025-07-22 08:21:41 +08:00
zxhlyh 4d36e784b7 merge main 2025-07-21 17:45:26 +08:00
uply23333 ab012fe1a2
fix: improve document filtering in full text search(elasticsearch) (#22683) 2025-07-21 15:59:37 +08:00
8bitpd 9251a66a10
fix: update analyticdb vector to do filter by metadata (#22698) 
Co-authored-by: xiaozeyu <xiaozeyu.xzy@alibaba-inc.com>
 2025-07-21 15:03:37 +08:00
twwu 5b2c99e183 Merge branch 'main' into feat/rag-2 2025-07-18 14:03:48 +08:00
znn ed263aed9f
fix text splitter (#22596) 2025-07-18 13:51:58 +08:00
-LAN- 460a825ef1
refactor: decouple Node and NodeData (#22581) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
 2025-07-18 10:08:51 +08:00
helojo e7d80bf7bf
Fix: the pict type picture was not processed in the docx (#19305) 
Co-authored-by: zqgame <zqgame@zqgame.local>
 2025-07-17 22:53:35 +08:00
zxhlyh 01566035e3 merge main 2025-07-17 16:48:43 +08:00
yihong d2933c2bfe
fix: drop dead code phase2 unused class (#22042) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-07-17 09:33:07 +08:00
wanttobeamaster bf7b2c339b
tablestore vector support more method (#22225) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-07-15 09:58:48 +08:00
Jacky Wu 3e96c0c468
fix: close session before doing long latency operation (#22306) 2025-07-14 15:16:10 +08:00
twwu 682b65034c Merge branch 'main' into feat/rag-2 2025-07-14 11:17:42 +08:00
luckylhb90 a371390d6c
optimize: batch embedding and qdrant write_consistency_factor parameter (#21776) 
Co-authored-by: hobo.l <hobo.l@binance.com>
 2025-07-10 10:16:59 +08:00
twwu e2a141b3bb Merge branch 'main' into feat/rag-2 2025-07-10 10:14:12 +08:00
twwu 9e44f2b805 Merge branch 'main' into feat/rag-2 2025-07-09 15:54:57 +08:00
wlleiiwang 89b52471fb
Optimize the memory usage of Tencent Vector Database (#22079) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-07-09 15:53:06 +08:00
jyong b5e4ce6c68 r2 2025-07-09 14:27:49 +08:00
jyong bc1a517a97 r2 2025-07-08 17:16:10 +08:00
baonudesifeizhai 1c7404099d
fix: prevent timeout in file encoding detection for large files (#21453) 
Co-authored-by: crazywoola <427733928@qq.com>
 2025-07-03 17:06:49 +08:00
jyong 832bef053f Merge branch 'main' into feat/r2 
# Conflicts:
#	docker/docker-compose.middleware.yaml
#	web/app/components/workflow-app/components/workflow-main.tsx
#	web/app/components/workflow-app/hooks/index.ts
#	web/app/components/workflow/hooks-store/store.ts
#	web/app/components/workflow/hooks/index.ts
#	web/app/components/workflow/nodes/_base/components/variable/var-reference-picker.tsx
 2025-07-02 18:20:05 +08:00
efrey kong 826bf25abf
Fix: prevent SQL errors when metadata filter Constant value is None or blank (#21803) 2025-07-02 14:43:01 +08:00
Dongyu Li 00f0b569cc
Feat/kb index (#20868) 
Co-authored-by: twwu <twwu@dify.ai>
 2025-06-25 17:52:59 +08:00
Dongyu Li b77081a19e feat(datasource): update datasource icon 2025-06-23 15:57:37 +08:00
twwu af64f29e87 Merge branch 'feat/rag-pipeline' into deploy/rag-dev 2025-06-23 13:59:39 +08:00
Jin 3e7f8bad56
fix: markdown_extractor lost chunks if it starts without a header(#21308) (#21309) 2025-06-21 23:10:00 +08:00
jyong 9620d6bcd8 Merge branch 'feat/r2' into deploy/rag-dev 
# Conflicts:
#	web/i18n/zh-Hans/app.ts
 2025-06-19 13:32:49 +08:00
jyong f7fbded8b9 Merge branch 'main' into feat/r2 2025-06-19 13:32:07 +08:00
Dongyu Li 82d0a70cb4 feat(datasource): change datasource result type to event-stream 2025-06-19 11:10:24 +08:00
LiuBo 17fe62cf91
feat: add support for Matrixone database (#20714) 2025-06-19 10:20:12 +08:00
NeatGuyCoding 9835730278
Translation fix (#21194) 2025-06-19 09:36:56 +08:00
NeatGuyCoding 2eae7503e1
Minor Improvements for File Validation and Configuration Handling #21179 (#21171) 
Co-authored-by: tech <cto@sb>
 2025-06-18 18:33:28 +08:00
jyong 643efc5d85 Merge branch 'feat/r2' into deploy/rag-dev 2025-06-18 17:11:52 +08:00
Dongyu Li 224111081b feat(datasource): change datasource result type to event-stream 2025-06-18 16:04:40 +08:00
jyong 6f67a34349 r2 qa index 2025-06-18 14:37:18 +08:00
Ademílson Tonato 9e73e8b9e8
feat: add search endpoint for Firecrawl Integration (#20521) 
Co-authored-by: crazywoola <427733928@qq.com>
 2025-06-18 14:37:03 +08:00
Rain Wang 47e0f92c0f
Fixes #20748 KnowledgeRetrievalNode return all external documents when reranker disabled even top-k configed (#20762) 2025-06-18 14:35:12 +08:00
zxhlyh e51d308312 Merge branch 'feat/rag-pipeline' into deploy/rag-dev 2025-06-18 13:47:36 +08:00
kazuya-awano 45c89bd6de
feat: add pagenation to notion extractor (#20919) 2025-06-18 11:30:55 +08:00