opensource
/
dify
mirror of https://github.com/langgenius/dify.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,023 Commits 291 Branches 160 Tags 518 MiB
Commit Graph

306 Commits

Author SHA1 Message Date
-LAN- 04954918a5
Merge commit from fork 
* fix(oraclevector): SQL Injection

Signed-off-by: -LAN- <laipz8200@outlook.com>

* fix(oraclevector): Remove bind variables from FETCH FIRST clause

Oracle doesn't support bind variables in the FETCH FIRST clause.
Fixed by using validated integers directly in the SQL string while
maintaining proper input validation to prevent SQL injection.

- Updated search_by_vector method to use validated top_k directly
- Updated search_by_full_text method to use validated top_k directly
- Adjusted parameter numbering for document_ids_filter placeholders

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2025-08-26 13:51:23 +08:00
huayaoyue6 23dcb2dc1b
fix(vector): use semantic version comparison for version check (#24409) 2025-08-24 21:04:33 +08:00
Guangdong Liu 1abf1240b2
refactor: replace try-except blocks with contextlib.suppress for cleaner exception handling (#24284) 2025-08-21 18:18:49 +08:00
8bitpd a183b2affb
fix: rollback when AnalyticDB create zhparser failed (#24260) 
Co-authored-by: xiaozeyu <xiaozeyu.xzy@alibaba-inc.com>
 2025-08-21 15:00:26 +08:00
8bitpd 6b1606f4f4
fix: keep idempotent when init AnalyticdbVectorBySql (#24239) 
Co-authored-by: xiaozeyu <xiaozeyu.xzy@alibaba-inc.com>
 2025-08-20 23:22:27 +08:00
yihong 4c1ad40f8e
docs: format all md files (#24195) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-08-20 13:49:11 +08:00
He Wang 670d479e32
Bump pyobvector to 0.2.15 (#24120) 2025-08-18 17:36:27 +08:00
crazywoola 8288b1dcab
Revert "fix pg_vector extension requires SUPERUSER, but not availabl… (#24108) 2025-08-18 16:46:15 +08:00
Elvis_LEE 16d1289a0a
fix pg_vector extension requires SUPERUSER, but not available on Huawei Cloud RDS (#24093) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2025-08-18 16:29:36 +08:00
Bo Wu 790a6ec203
fix: return empty list instead of raising exception for qdrant search when score_threshold is 1 (#24032) 2025-08-18 12:44:05 +08:00
-LAN- e340fccafb
feat: integrate flask-orjson for improved JSON serialization performance (#23935) 2025-08-14 19:50:59 +08:00
engchina 7566d90dfe
fix issue #23758 (#23764) 
Co-authored-by: root <root@thinkpad-pc.localdomain>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2025-08-12 10:26:13 +08:00
yunqiqiliang 14e1c16cf2
Fix ClickZetta stability and reduce logging noise (#23632) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2025-08-08 22:57:47 +08:00
yunqiqiliang 62772e8871
fix: ensure vector database cleanup on dataset deletion regardless of document presence (affects all 33 vector databases) (#23574) 
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2025-08-08 09:18:43 +08:00
Qiang Lee e9045a8838
Fix: Apply Metadata Filters Correctly in Full-Text Search Mode for Tencent Cloud Vector Database (#23564) 2025-08-07 05:36:06 -07:00
yunqiqiliang e01510e2a6
feat: Add Clickzetta Lakehouse vector database integration (#22551) 
Co-authored-by: Claude <noreply@anthropic.com>
 2025-08-07 14:21:46 +08:00
Yongtao Huang 406c1952b8
Fix version comparison with imported_version (#23326) 
Signed-off-by: Yongtao Huang <yongtaoh2022@gmail.com>
 2025-08-04 10:40:49 +08:00
wanttobeamaster da5c003f97
chore: tablestore full text search support score normalization (#23255) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-08-01 14:14:11 +08:00
rhochman eee576355b
Fix: Support for Elasticsearch Cloud Connector (#23017) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-30 11:12:16 +08:00
Asuka Minato a189d293f8
make logging not use f-str, change others to f-str (#22882) 2025-07-25 10:32:48 +08:00
Asuka Minato ef51678c73
orm filter -> where (#22801) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Claude <noreply@anthropic.com>
 2025-07-24 00:57:45 +08:00
wanttobeamaster 8278b39f85
fix tablestore full text search bug (#22853) 2025-07-23 19:31:47 +08:00
wanttobeamaster 1c3c40db69
fix: tablestore TypeError when vector is missing (#22843) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-07-23 18:59:16 +08:00
wlleiiwang b4e152f775
FEAT: Tencent Vector search supports backward compatibility with the previous score calculation approach. (#22820) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-07-23 15:38:31 +08:00
Asuka Minato 6d3e198c3c
Mapped column (#22644) 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-07-23 00:39:59 +08:00
wanttobeamaster a2048fd0f4
fix: tablestore vdb support metadata filter (#22774) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-07-22 16:48:59 +08:00
issac2e 58d92970a9
Optimize tencent_vector knowledge base deletion error handling with batch processing support (#22726) 
Co-authored-by: liuchen15 <liuchen15@gaotu.cn>
Co-authored-by: crazywoola <427733928@qq.com>
 2025-07-22 08:21:41 +08:00
uply23333 ab012fe1a2
fix: improve document filtering in full text search(elasticsearch) (#22683) 2025-07-21 15:59:37 +08:00
8bitpd 9251a66a10
fix: update analyticdb vector to do filter by metadata (#22698) 
Co-authored-by: xiaozeyu <xiaozeyu.xzy@alibaba-inc.com>
 2025-07-21 15:03:37 +08:00
yihong d2933c2bfe
fix: drop dead code phase2 unused class (#22042) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-07-17 09:33:07 +08:00
wanttobeamaster bf7b2c339b
tablestore vector support more method (#22225) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-07-15 09:58:48 +08:00
Jacky Wu 3e96c0c468
fix: close session before doing long latency operation (#22306) 2025-07-14 15:16:10 +08:00
luckylhb90 a371390d6c
optimize: batch embedding and qdrant write_consistency_factor parameter (#21776) 
Co-authored-by: hobo.l <hobo.l@binance.com>
 2025-07-10 10:16:59 +08:00
wlleiiwang 89b52471fb
Optimize the memory usage of Tencent Vector Database (#22079) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-07-09 15:53:06 +08:00
LiuBo 17fe62cf91
feat: add support for Matrixone database (#20714) 2025-06-19 10:20:12 +08:00
NeatGuyCoding 9835730278
Translation fix (#21194) 2025-06-19 09:36:56 +08:00
kurokobo 4689e8953e
fix: shorten connection timeout to pypi.org for deprecation check for weaviate client (#21131) 2025-06-18 09:25:52 +08:00
Bowen Liang 366ddb05ae
test: run vdb test of oceanbase with docker compose in CI tests (#20945) 2025-06-16 11:05:19 +08:00
yihong 65c7c01d90
fix: clean up two unreachable code (#20773) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-06-07 23:06:46 +08:00
jefferyvvv 37c3283450
fix: opensearch vector search falls back to keyword search (#20723) 
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
 2025-06-06 16:29:15 +08:00
jefferyvvv 4271602cfc
fix: opensearch metadata filtering returns empty (#20701) 
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
Co-authored-by: crazywoola <427733928@qq.com>
 2025-06-06 09:10:01 +08:00
jefferyvvv 138ad6e8b3
fix: opensearch fulltext search with metadata filtering dsl error (#20702) 
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
 2025-06-05 23:09:00 +08:00
kenwoodjw 01d500db14
fix: autocorrect everything in web (#20605) 
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
 2025-06-04 14:12:24 +08:00
zhaobingshuang 3f7aa38d77
fix: #20560 When elasticsearch is used as the vector database, the Retrieval Test fails to filter the data after setting the Score Threshold, and the score of the recalled results is empty (#20561) 2025-06-03 13:24:26 +08:00
Cheney Zhang b4b59148dc
check zilliz cloud of full-text search (#20519) 2025-06-02 18:04:13 +08:00
yihong 5a991295e0
fix: drop some type fixme (#20344) 2025-05-30 14:10:09 +08:00
rouxiaomin 4c4887c5fc
feat(qdrant):add replication_factor when create collection in qdrant (#20133) 
Co-authored-by: 刘敏 <min.liu@tongdun.net>
 2025-05-27 14:46:04 +08:00
He Huang 6f48af2610
Refactor OpenSearch config to separate use_ssl and verify_certs flags (#20075) 
Co-authored-by: he.huang <he.huang1@outlook.com>
Co-authored-by: crazywoola <427733928@qq.com>
 2025-05-22 10:14:38 +08:00
wlleiiwang 7d230acf40
tencent vectordb compatible with version 1.1.3 and below (#20056) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-05-21 20:24:05 +08:00
-LAN- 3196dc2d61
refactor: Use typed SQLAlchemy base model and fix type errors (#19980) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
 2025-05-21 15:38:03 +08:00
k-kayashima b292990075
Fix: Ensure unique index names for pgvector knowledge tables (#19672) 
Co-authored-by: crazywoola <427733928@qq.com>
 2025-05-15 11:43:44 +08:00
Will bfa652f2d0
fix: metadata filtering condition variable unassigned; fix External K… (#19208) 2025-05-07 14:52:09 +08:00
QuantumGhost bd1bbfee4b
Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 2025-04-29 18:04:33 +08:00
Ahmad Zidan 8266815cda
feat: add AWS Managed IAM auth for OpenSearch vector DB (#18963) 2025-04-29 15:10:08 +08:00
Ethan 8b4ea01810
feat: support access milvus with token (#19034) 2025-04-29 14:52:13 +08:00
Jiang 37e2f73909
[Lindorm VDB] Add the QUERY_TIMEOUT parameter to force the search query to fail. (#18613) 
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
 2025-04-25 09:42:58 +08:00
王晓阳 0babdffe3e
feat: support vastbase vector database (#16308) 2025-04-24 18:04:57 +08:00
lauding eb1ce3dd6b
feat: support huawei cloud vector database (#16141) 2025-04-22 13:03:35 +08:00
tmuife 7b6523e54d
Update Oracle db connection library and change connection pool to single connection (#18466) 2025-04-21 17:56:57 +08:00
Rain Wang 83f1aeec1d
Fix ORDER BY (score, id) error in api/core/rag/datasource/vdb/analyticdb/analyticdb_vector_sql.py line 249 (#18252) 2025-04-17 14:15:05 +08:00
Rain Wang e8d98e3d89
Add analyzer_params config for milvus vectordb (#18180) 2025-04-17 10:38:56 +08:00
YANG d119c7d629
ignore errors when creating duplicate indexes (#18069) 
Co-authored-by: 璟义 <yangshangpo.ysp@alibaba-inc.com>
 2025-04-15 15:48:16 +08:00
Jasonfish 1f722cde22
fix(api): Some params were ignored when creating empty Datasets through API (#17932) 2025-04-14 10:24:01 +08:00
Yongtao Huang 5d72003ebb
Remove dead code (#17899) 2025-04-11 20:33:52 +08:00
yihong f04d52c044
fix: autocorrect everything in api (#17859) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-04-11 15:24:39 +08:00
wlleiiwang 9d20561af4
create db if not exists (#17796) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-04-10 18:03:22 +08:00
Jyong 8b3be4224d
revert batch query (#17707) 2025-04-09 20:25:36 +08:00
wlleiiwang f148f1efa2
fix: Check collection exists before drop it. (#17692) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-04-09 19:14:32 +08:00
Jyong abfcd9d3b6
fix segment query index not effect (#17704) 2025-04-09 19:09:08 +08:00
Han f1e4d5ed6c
Fix Performance Issues: (#17083) 
Co-authored-by: Wang Han <wanghan@zhejianglab.org>
 2025-04-09 11:22:53 +08:00
wlleiiwang 42a42a7962
FEAT: support Tencent vectordb to full text search (#16865) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-04-07 09:50:03 +08:00
Jiang fd1e40d22e
Lindorm VDB bugfix (#17357) 
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
 2025-04-02 21:31:59 +08:00
Jiang ff388fe3e6
optimize lindorm vdb add_texts (#17212) 
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
 2025-04-01 11:06:35 +08:00
非法操作 44f911a0a8
chore: docstring not match the function parameter (#17162) 2025-03-31 13:19:15 +08:00
wanttobeamaster 7f70cadacb
feat: support Tablestore vector database (#16601) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-03-27 15:53:33 +08:00
wlleiiwang a743d5dc71
feat: tencent vectordb: use grpc client and set upsert batch size (#16016) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-03-27 12:20:16 +08:00
yourchanges 59a86dabee
fix: fix missing oceanbase config enable_hybrid_search init (#16852) 
Co-authored-by: 李远军 <4842@9ji.com>
 2025-03-26 21:15:54 +08:00
Jyong 2174225259
fix milvus filter search (#16725) 2025-03-25 16:22:43 +08:00
hsiong 6157f57872
feat: Add OceanBase hybrid search features (#16652) 
Co-authored-by: 李远军 <4842@9ji.com>
Co-authored-by: yourchanges <yourchanges@gmail.com>
 2025-03-25 14:32:00 +08:00
kenwoodjw a113356695
fix: pgvector metadata filter (#16688) 
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
 2025-03-25 11:34:33 +08:00
Jiang fc8c765215
Fix/vdb lindorm (#16660) 
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
 2025-03-25 09:19:06 +08:00
Jyong 1be0d26c1f
fix metadata filter not affect in keyword-search and fulltext-search (#16644) 2025-03-24 18:35:16 +08:00
chenhuan0728 770c461a8f
feat： add openGauss PQ acceleration feature (#16432) 
Co-authored-by: chenhuan <huan.chen0728@foxmail>
 2025-03-24 15:16:40 +08:00
Jyong d135677c25
add vdb document id index (#16244) 
Co-authored-by: crazywoola <427733928@qq.com>
 2025-03-20 01:38:15 +08:00
Jyong a8879057c0
fix tidb metadata filter (#16237) 2025-03-19 19:44:56 +08:00
Jyong 81325df368
fix weaviate metadata filter (#16230) 2025-03-19 18:26:53 +08:00
Jyong c3c957bb80
change recreate_collection function to create_collection (#16212) 2025-03-19 17:13:08 +08:00
Jyong abeaea4f79
Support knowledge metadata filter (#15982) 2025-03-18 16:42:19 +08:00
Jyong 33ba7e659b
fix vector db sql injection (#16096) 2025-03-18 15:07:29 +08:00
LittleFish-15 223ab5a38f
feat: support openGauss vector database (#15865) 2025-03-17 19:42:54 +08:00
huangzhuo1949 695a7400a9
fix:delete empty table bug (#15517) 
Co-authored-by: huangzhuo <huangzhuo1@xiaomi.com>
 2025-03-17 10:53:26 +08:00
Jyong 84a866028a
fix document could be None (#15818) 2025-03-14 16:40:01 +08:00
Yuichiro Utsumi 5f9d236d22
Feat: Add pg_bigm for keyword search in pgvector (#13876) 
Signed-off-by: Yuichiro Utsumi <utsumi.yuichiro@fujitsu.com>
 2025-03-13 16:32:34 +08:00
kenwoodjw 087bb60b31
fix: preserve Unicode characters in keyword search queries (#15522) 
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
 2025-03-12 18:34:42 +08:00
engchina 9c1db7dca7
modify oracle lexer name Fixes #15106 (#15108) 
Co-authored-by: engchina <atjapan2015@gmail.com>
 2025-03-06 18:58:51 +08:00
llinvokerl d04f40c274
Fix empty results issue in full-text search with Milvus vector database (#14885) 
Co-authored-by: liusurong.lsr <liusurong.lsr@alibaba-inc.com>
 2025-03-05 12:27:01 +08:00
engchina c8de30f3d9
feat: support oracle oci autonomouse database. Fixes #14792 and Fixes #14628. (#14804) 
Co-authored-by: engchina <atjapan2015@gmail.com>
 2025-03-04 09:22:04 +08:00
Rhys 548f6ef2b6
fix: incorrect score in the chroma vector (#14273) 2025-02-25 09:40:22 +08:00
Bowen Liang dfdd6dfa20
fix: change the config name and fix typo in description of the number of retrieval executors (#13856) 2025-02-19 09:13:36 +08:00
Jyong aa19bb3f30
fix session close issue (#13946) 2025-02-18 19:29:57 +08:00