opensource
/
dify
mirror of https://github.com/langgenius/dify.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,008 Commits 295 Branches 160 Tags 506 MiB
Commit Graph

423 Commits

Author SHA1 Message Date
-LAN- 460a825ef1
refactor: decouple Node and NodeData (#22581) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
 2025-07-18 10:08:51 +08:00
helojo e7d80bf7bf
Fix: the pict type picture was not processed in the docx (#19305) 
Co-authored-by: zqgame <zqgame@zqgame.local>
 2025-07-17 22:53:35 +08:00
yihong d2933c2bfe
fix: drop dead code phase2 unused class (#22042) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-07-17 09:33:07 +08:00
wanttobeamaster bf7b2c339b
tablestore vector support more method (#22225) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-07-15 09:58:48 +08:00
Jacky Wu 3e96c0c468
fix: close session before doing long latency operation (#22306) 2025-07-14 15:16:10 +08:00
luckylhb90 a371390d6c
optimize: batch embedding and qdrant write_consistency_factor parameter (#21776) 
Co-authored-by: hobo.l <hobo.l@binance.com>
 2025-07-10 10:16:59 +08:00
wlleiiwang 89b52471fb
Optimize the memory usage of Tencent Vector Database (#22079) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-07-09 15:53:06 +08:00
baonudesifeizhai 1c7404099d
fix: prevent timeout in file encoding detection for large files (#21453) 
Co-authored-by: crazywoola <427733928@qq.com>
 2025-07-03 17:06:49 +08:00
efrey kong 826bf25abf
Fix: prevent SQL errors when metadata filter Constant value is None or blank (#21803) 2025-07-02 14:43:01 +08:00
Dongyu Li 00f0b569cc
Feat/kb index (#20868) 
Co-authored-by: twwu <twwu@dify.ai>
 2025-06-25 17:52:59 +08:00
Jin 3e7f8bad56
fix: markdown_extractor lost chunks if it starts without a header(#21308) (#21309) 2025-06-21 23:10:00 +08:00
LiuBo 17fe62cf91
feat: add support for Matrixone database (#20714) 2025-06-19 10:20:12 +08:00
NeatGuyCoding 9835730278
Translation fix (#21194) 2025-06-19 09:36:56 +08:00
NeatGuyCoding 2eae7503e1
Minor Improvements for File Validation and Configuration Handling #21179 (#21171) 
Co-authored-by: tech <cto@sb>
 2025-06-18 18:33:28 +08:00
Ademílson Tonato 9e73e8b9e8
feat: add search endpoint for Firecrawl Integration (#20521) 
Co-authored-by: crazywoola <427733928@qq.com>
 2025-06-18 14:37:03 +08:00
Rain Wang 47e0f92c0f
Fixes #20748 KnowledgeRetrievalNode return all external documents when reranker disabled even top-k configed (#20762) 2025-06-18 14:35:12 +08:00
kazuya-awano 45c89bd6de
feat: add pagenation to notion extractor (#20919) 2025-06-18 11:30:55 +08:00
kurokobo 4689e8953e
fix: shorten connection timeout to pypi.org for deprecation check for weaviate client (#21131) 2025-06-18 09:25:52 +08:00
Bowen Liang 366ddb05ae
test: run vdb test of oceanbase with docker compose in CI tests (#20945) 2025-06-16 11:05:19 +08:00
Bowen Liang 0f3d4d0b6e
chore: bump mypy to 1.16 (#20608) 2025-06-11 01:01:33 +08:00
QuantumGhost c439e82038
refactor(api): Decouple `ParameterExtractorNode` from `LLMNode` (#20843) 
- Extract methods used by `ParameterExtractorNode` from `LLMNode` into a separate file.
- Convert `ParameterExtractorNode` into a subclass of `BaseNode`.
- Refactor code referencing the extracted methods to ensure functionality and clarity.
- Fixes the issue that `ParameterExtractorNode` returns error when executed.
- Fix relevant test cases.

Closes #20840.
 2025-06-10 11:47:50 +08:00
yihong 65c7c01d90
fix: clean up two unreachable code (#20773) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-06-07 23:06:46 +08:00
jefferyvvv 37c3283450
fix: opensearch vector search falls back to keyword search (#20723) 
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
 2025-06-06 16:29:15 +08:00
jefferyvvv 4271602cfc
fix: opensearch metadata filtering returns empty (#20701) 
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
Co-authored-by: crazywoola <427733928@qq.com>
 2025-06-06 09:10:01 +08:00
jefferyvvv 138ad6e8b3
fix: opensearch fulltext search with metadata filtering dsl error (#20702) 
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
 2025-06-05 23:09:00 +08:00
kenwoodjw 01d500db14
fix: autocorrect everything in web (#20605) 
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
 2025-06-04 14:12:24 +08:00
zhaobingshuang 3f7aa38d77
fix: #20560 When elasticsearch is used as the vector database, the Retrieval Test fails to filter the data after setting the Score Threshold, and the score of the recalled results is empty (#20561) 2025-06-03 13:24:26 +08:00
Cheney Zhang b4b59148dc
check zilliz cloud of full-text search (#20519) 2025-06-02 18:04:13 +08:00
Dongyu Li 1ea4459d9f
update knowledge base api (#20426) 2025-05-30 14:45:30 +08:00
-LAN- a6ea15e63c
Refactor/message cycle manage and knowledge retrieval (#20460) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
 2025-05-30 14:36:44 +08:00
yihong 5a991295e0
fix: drop some type fixme (#20344) 2025-05-30 14:10:09 +08:00
-LAN- 482e50aae9
Refactor/remove db from cycle manager (#20455) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
 2025-05-30 04:34:13 +08:00
rouxiaomin 4c4887c5fc
feat(qdrant):add replication_factor when create collection in qdrant (#20133) 
Co-authored-by: 刘敏 <min.liu@tongdun.net>
 2025-05-27 14:46:04 +08:00
He Huang 6f48af2610
Refactor OpenSearch config to separate use_ssl and verify_certs flags (#20075) 
Co-authored-by: he.huang <he.huang1@outlook.com>
Co-authored-by: crazywoola <427733928@qq.com>
 2025-05-22 10:14:38 +08:00
wlleiiwang 7d230acf40
tencent vectordb compatible with version 1.1.3 and below (#20056) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-05-21 20:24:05 +08:00
-LAN- 3196dc2d61
refactor: Use typed SQLAlchemy base model and fix type errors (#19980) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
 2025-05-21 15:38:03 +08:00
Darlewo 8cb3b4aef2
fix: multiple retrieve reranking_enabled switch (#19958) 2025-05-20 15:22:03 +08:00
Amir Mohsen Asaran c9ee60e197
Feat(WaterCrawl error handling): add custom exceptions and error handling (#19948) 2025-05-20 10:25:16 +08:00
-LAN- 4977bb21ec
feat(workflow): domain model for workflow node execution (#19430) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2025-05-17 00:56:16 +08:00
k-kayashima b292990075
Fix: Ensure unique index names for pgvector knowledge tables (#19672) 
Co-authored-by: crazywoola <427733928@qq.com>
 2025-05-15 11:43:44 +08:00
非法操作 085bd1aa93
chore: model.query change to db.session.query (#19551) 
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
 2025-05-13 09:13:12 +08:00
非法操作 14cd71ed0a
chore: all model.query replace to db.session.query (#19521) 2025-05-12 15:19:41 +08:00
非法操作 b00f94df64
fix: replace all dataset.Model.query to db.session.query(Model) (#19509) 2025-05-12 13:52:33 +08:00
湛露先生 1119790b02
clean rag word_extractor. (#19397) 
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
 2025-05-09 16:39:16 +08:00
Will bfa652f2d0
fix: metadata filtering condition variable unassigned; fix External K… (#19208) 2025-05-07 14:52:09 +08:00
Hao Cheng 9bcf837f17
fix: use only supported operators in metadata filter system prompts (#19195) 2025-05-03 20:08:08 +08:00
Will a212a63e6a
fix: time type metadata filtering error (#19192) 2025-05-03 20:07:37 +08:00
Bowen Liang 12c96b93d9
immediately return initialed tiktokenizer instance and remove dead code in usage of tiktokenizer (#17957) 2025-04-30 16:07:20 +08:00
QuantumGhost bd1bbfee4b
Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 2025-04-29 18:04:33 +08:00
Ahmad Zidan 8266815cda
feat: add AWS Managed IAM auth for OpenSearch vector DB (#18963) 2025-04-29 15:10:08 +08:00
Ethan 8b4ea01810
feat: support access milvus with token (#19034) 2025-04-29 14:52:13 +08:00
Panpan 83187b30c0
fix: fix rerank model runner usage (#19008) 2025-04-29 14:51:21 +08:00
Wesley b62eb61400
fix depth param issue for WaterCrawl (#18839) 2025-04-27 11:04:56 +08:00
Jiang 37e2f73909
[Lindorm VDB] Add the QUERY_TIMEOUT parameter to force the search query to fail. (#18613) 
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
 2025-04-25 09:42:58 +08:00
王晓阳 0babdffe3e
feat: support vastbase vector database (#16308) 2025-04-24 18:04:57 +08:00
Jyong e2cb7006c4
check metadata_filtering_conditions could be None in auto mode (#18548) 2025-04-22 17:09:33 +08:00
lauding eb1ce3dd6b
feat: support huawei cloud vector database (#16141) 2025-04-22 13:03:35 +08:00
tmuife 7b6523e54d
Update Oracle db connection library and change connection pool to single connection (#18466) 2025-04-21 17:56:57 +08:00
Rain Wang d2e3744ca3
Switching from CONSOLE_API_URL to FILES_URL in word_extractor.py (#18249) 2025-04-18 16:05:48 +08:00
Rain Wang 83f1aeec1d
Fix ORDER BY (score, id) error in api/core/rag/datasource/vdb/analyticdb/analyticdb_vector_sql.py line 249 (#18252) 2025-04-17 14:15:05 +08:00
Rain Wang e8d98e3d89
Add analyzer_params config for milvus vectordb (#18180) 2025-04-17 10:38:56 +08:00
Jyong 95283b4dd3
Feat/change split length method (#18097) 
Co-authored-by: JzoNg <jzongcode@gmail.com>
 2025-04-16 12:28:22 +08:00
YANG d119c7d629
ignore errors when creating duplicate indexes (#18069) 
Co-authored-by: 璟义 <yangshangpo.ysp@alibaba-inc.com>
 2025-04-15 15:48:16 +08:00
Jasonfish 1f722cde22
fix(api): Some params were ignored when creating empty Datasets through API (#17932) 2025-04-14 10:24:01 +08:00
Yongtao Huang 5d72003ebb
Remove dead code (#17899) 2025-04-11 20:33:52 +08:00
briqt 91cfa90503
Fix external knowledge Issues: (#17685) (#17843) 2025-04-11 15:37:27 +08:00
yihong f04d52c044
fix: autocorrect everything in api (#17859) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2025-04-11 15:24:39 +08:00
wlleiiwang 9d20561af4
create db if not exists (#17796) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-04-10 18:03:22 +08:00
Jyong 8b3be4224d
revert batch query (#17707) 2025-04-09 20:25:36 +08:00
wlleiiwang f148f1efa2
fix: Check collection exists before drop it. (#17692) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-04-09 19:14:32 +08:00
Jyong abfcd9d3b6
fix segment query index not effect (#17704) 2025-04-09 19:09:08 +08:00
Han f1e4d5ed6c
Fix Performance Issues: (#17083) 
Co-authored-by: Wang Han <wanghan@zhejianglab.org>
 2025-04-09 11:22:53 +08:00
Steven Li abead647e2
fix: Extract docx file fails when the file contains an invalid link (#17576) 2025-04-08 13:59:33 +08:00
Amir Mohsen Asaran f54905e685
feat: Integrate WaterCrawl.dev as a new knowledge base provider (#16396) 
Co-authored-by: crazywoola <427733928@qq.com>
 2025-04-07 12:43:23 +08:00
wlleiiwang 42a42a7962
FEAT: support Tencent vectordb to full text search (#16865) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-04-07 09:50:03 +08:00
crazywoola 3e698074e7
Fix/17466 cannot create a knowledge base by adding files (#17470) 2025-04-06 00:03:05 +08:00
Panpan fc3f14c0ee
fix: keep image url (#17430) 2025-04-04 15:55:48 +08:00
Perfecto 16c722d1d8
fix: move hardcoded text to language settings (#16990) (#17133) 2025-04-02 22:35:51 +08:00
Jyong 6104b91d3f
add doc support in knowledge base for unstructured (#17352) 2025-04-02 21:35:01 +08:00
Jiang fd1e40d22e
Lindorm VDB bugfix (#17357) 
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
 2025-04-02 21:31:59 +08:00
Jiang ff388fe3e6
optimize lindorm vdb add_texts (#17212) 
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
 2025-04-01 11:06:35 +08:00
非法操作 44f911a0a8
chore: docstring not match the function parameter (#17162) 2025-03-31 13:19:15 +08:00
jiangbo721 a1aa325ce3
Chore/code format and Repair commit_id 3254018d more deleted codes and Fix naming error ambiguity between workflow_run_id and workflow_id (#17075) 
Co-authored-by: 刘江波 <jiangbo721@163.com>
 2025-03-29 14:15:53 +08:00
wanttobeamaster 7f70cadacb
feat: support Tablestore vector database (#16601) 
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
 2025-03-27 15:53:33 +08:00
wlleiiwang a743d5dc71
feat: tencent vectordb: use grpc client and set upsert batch size (#16016) 
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
 2025-03-27 12:20:16 +08:00
Jyong 30792a1e1a
install pandoc (#16825) 2025-03-26 22:34:10 +08:00
yourchanges 59a86dabee
fix: fix missing oceanbase config enable_hybrid_search init (#16852) 
Co-authored-by: 李远军 <4842@9ji.com>
 2025-03-26 21:15:54 +08:00
Jyong 6a857e01f6
fix multiple metadata filter's confusing setting (#16771) 2025-03-26 14:16:21 +08:00
taokuizu 0c2a459c30
fix typo in _process_metadata_filter_func (#16780) 2025-03-26 09:01:41 +08:00
Jyong 2174225259
fix milvus filter search (#16725) 2025-03-25 16:22:43 +08:00
hsiong 6157f57872
feat: Add OceanBase hybrid search features (#16652) 
Co-authored-by: 李远军 <4842@9ji.com>
Co-authored-by: yourchanges <yourchanges@gmail.com>
 2025-03-25 14:32:00 +08:00
kenwoodjw a113356695
fix: pgvector metadata filter (#16688) 
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
 2025-03-25 11:34:33 +08:00
Jiang fc8c765215
Fix/vdb lindorm (#16660) 
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
 2025-03-25 09:19:06 +08:00
Jyong 86a1859d02
Metadata variable value fix (#16665) 2025-03-25 09:07:11 +08:00
Jyong 1be0d26c1f
fix metadata filter not affect in keyword-search and fulltext-search (#16644) 2025-03-24 18:35:16 +08:00
chenhuan0728 770c461a8f
feat： add openGauss PQ acceleration feature (#16432) 
Co-authored-by: chenhuan <huan.chen0728@foxmail>
 2025-03-24 15:16:40 +08:00
Jyong d135677c25
add vdb document id index (#16244) 
Co-authored-by: crazywoola <427733928@qq.com>
 2025-03-20 01:38:15 +08:00
Jyong a8879057c0
fix tidb metadata filter (#16237) 2025-03-19 19:44:56 +08:00
Jyong 81325df368
fix weaviate metadata filter (#16230) 2025-03-19 18:26:53 +08:00
Jyong b8ef3149ef
metadata expect value check error (#16210) 2025-03-19 17:48:01 +08:00