wanttobeamaster
bf7b2c339b
tablestore vector support more method ( #22225 )
...
Co-authored-by: xiaozhiqing.xzq <xiaozhiqing.xzq@alibaba-inc.com>
2025-07-15 09:58:48 +08:00
Jacky Wu
3e96c0c468
fix: close session before doing long latency operation ( #22306 )
2025-07-14 15:16:10 +08:00
twwu
682b65034c
Merge branch 'main' into feat/rag-2
2025-07-14 11:17:42 +08:00
luckylhb90
a371390d6c
optimize: batch embedding and qdrant write_consistency_factor parameter ( #21776 )
...
Co-authored-by: hobo.l <hobo.l@binance.com>
2025-07-10 10:16:59 +08:00
twwu
e2a141b3bb
Merge branch 'main' into feat/rag-2
2025-07-10 10:14:12 +08:00
twwu
9e44f2b805
Merge branch 'main' into feat/rag-2
2025-07-09 15:54:57 +08:00
wlleiiwang
89b52471fb
Optimize the memory usage of Tencent Vector Database ( #22079 )
...
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
2025-07-09 15:53:06 +08:00
jyong
b5e4ce6c68
r2
2025-07-09 14:27:49 +08:00
jyong
bc1a517a97
r2
2025-07-08 17:16:10 +08:00
baonudesifeizhai
1c7404099d
fix: prevent timeout in file encoding detection for large files ( #21453 )
...
Co-authored-by: crazywoola <427733928@qq.com>
2025-07-03 17:06:49 +08:00
jyong
832bef053f
Merge branch 'main' into feat/r2
...
# Conflicts:
# docker/docker-compose.middleware.yaml
# web/app/components/workflow-app/components/workflow-main.tsx
# web/app/components/workflow-app/hooks/index.ts
# web/app/components/workflow/hooks-store/store.ts
# web/app/components/workflow/hooks/index.ts
# web/app/components/workflow/nodes/_base/components/variable/var-reference-picker.tsx
2025-07-02 18:20:05 +08:00
efrey kong
826bf25abf
Fix: prevent SQL errors when metadata filter Constant value is None or blank ( #21803 )
2025-07-02 14:43:01 +08:00
Dongyu Li
00f0b569cc
Feat/kb index ( #20868 )
...
Co-authored-by: twwu <twwu@dify.ai>
2025-06-25 17:52:59 +08:00
Dongyu Li
b77081a19e
feat(datasource): update datasource icon
2025-06-23 15:57:37 +08:00
twwu
af64f29e87
Merge branch 'feat/rag-pipeline' into deploy/rag-dev
2025-06-23 13:59:39 +08:00
Jin
3e7f8bad56
fix: markdown_extractor lost chunks if it starts without a header( #21308 ) ( #21309 )
2025-06-21 23:10:00 +08:00
jyong
9620d6bcd8
Merge branch 'feat/r2' into deploy/rag-dev
...
# Conflicts:
# web/i18n/zh-Hans/app.ts
2025-06-19 13:32:49 +08:00
jyong
f7fbded8b9
Merge branch 'main' into feat/r2
2025-06-19 13:32:07 +08:00
Dongyu Li
82d0a70cb4
feat(datasource): change datasource result type to event-stream
2025-06-19 11:10:24 +08:00
LiuBo
17fe62cf91
feat: add support for Matrixone database ( #20714 )
2025-06-19 10:20:12 +08:00
NeatGuyCoding
9835730278
Translation fix ( #21194 )
2025-06-19 09:36:56 +08:00
NeatGuyCoding
2eae7503e1
Minor Improvements for File Validation and Configuration Handling #21179 ( #21171 )
...
Co-authored-by: tech <cto@sb>
2025-06-18 18:33:28 +08:00
jyong
643efc5d85
Merge branch 'feat/r2' into deploy/rag-dev
2025-06-18 17:11:52 +08:00
Dongyu Li
224111081b
feat(datasource): change datasource result type to event-stream
2025-06-18 16:04:40 +08:00
jyong
6f67a34349
r2 qa index
2025-06-18 14:37:18 +08:00
Ademílson Tonato
9e73e8b9e8
feat: add search endpoint for Firecrawl Integration ( #20521 )
...
Co-authored-by: crazywoola <427733928@qq.com>
2025-06-18 14:37:03 +08:00
Rain Wang
47e0f92c0f
Fixes #20748 KnowledgeRetrievalNode return all external documents when reranker disabled even top-k configed ( #20762 )
2025-06-18 14:35:12 +08:00
zxhlyh
e51d308312
Merge branch 'feat/rag-pipeline' into deploy/rag-dev
2025-06-18 13:47:36 +08:00
kazuya-awano
45c89bd6de
feat: add pagenation to notion extractor ( #20919 )
2025-06-18 11:30:55 +08:00
jyong
f7a4e5d1a6
Merge branch 'main' into feat/r2
2025-06-18 10:57:44 +08:00
kurokobo
4689e8953e
fix: shorten connection timeout to pypi.org for deprecation check for weaviate client ( #21131 )
2025-06-18 09:25:52 +08:00
Dongyu Li
cf66d111ba
feat(datasource): change datasource result type to event-stream
2025-06-17 18:29:02 +08:00
Dongyu Li
739ebf2117
feat(datasource): change datasource result type to event-stream
2025-06-17 18:24:09 +08:00
jyong
b277acc298
Merge branch 'main' into feat/r2
2025-06-16 14:08:02 +08:00
Bowen Liang
366ddb05ae
test: run vdb test of oceanbase with docker compose in CI tests ( #20945 )
2025-06-16 11:05:19 +08:00
jyong
1d8b390584
Merge branch 'main' into feat/r2
...
# Conflicts:
# docker/docker-compose.middleware.yaml
2025-06-11 17:16:27 +08:00
Bowen Liang
0f3d4d0b6e
chore: bump mypy to 1.16 ( #20608 )
2025-06-11 01:01:33 +08:00
jyong
95a24156de
r2
2025-06-10 18:20:32 +08:00
QuantumGhost
c439e82038
refactor(api): Decouple ParameterExtractorNode from LLMNode ( #20843 )
...
- Extract methods used by `ParameterExtractorNode` from `LLMNode` into a separate file.
- Convert `ParameterExtractorNode` into a subclass of `BaseNode`.
- Refactor code referencing the extracted methods to ensure functionality and clarity.
- Fixes the issue that `ParameterExtractorNode` returns error when executed.
- Fix relevant test cases.
Closes #20840 .
2025-06-10 11:47:50 +08:00
yihong
65c7c01d90
fix: clean up two unreachable code ( #20773 )
...
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-06-07 23:06:46 +08:00
jefferyvvv
37c3283450
fix: opensearch vector search falls back to keyword search ( #20723 )
...
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
2025-06-06 16:29:15 +08:00
jefferyvvv
4271602cfc
fix: opensearch metadata filtering returns empty ( #20701 )
...
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
Co-authored-by: crazywoola <427733928@qq.com>
2025-06-06 09:10:01 +08:00
jefferyvvv
138ad6e8b3
fix: opensearch fulltext search with metadata filtering dsl error ( #20702 )
...
Co-authored-by: wenjun.gu <wenjun.gu@envision-energy.com>
2025-06-05 23:09:00 +08:00
kenwoodjw
01d500db14
fix: autocorrect everything in web ( #20605 )
...
Signed-off-by: kenwoodjw <blackxin55+@gmail.com>
2025-06-04 14:12:24 +08:00
jyong
9cdd2cbb27
r2
2025-06-03 19:02:57 +08:00
jyong
309fffd1e4
Merge branch 'main' into feat/r2
...
# Conflicts:
# api/core/repositories/sqlalchemy_workflow_node_execution_repository.py
# api/core/workflow/entities/node_entities.py
# api/core/workflow/enums.py
2025-06-03 18:56:49 +08:00
jyong
4130c50643
r2
2025-06-03 18:32:39 +08:00
jyong
7b7f8ef51d
r2
2025-06-03 18:12:24 +08:00
zhaobingshuang
3f7aa38d77
fix : #20560 When elasticsearch is used as the vector database, the Retrieval Test fails to filter the data after setting the Score Threshold, and the score of the recalled results is empty ( #20561 )
2025-06-03 13:24:26 +08:00
Cheney Zhang
b4b59148dc
check zilliz cloud of full-text search ( #20519 )
2025-06-02 18:04:13 +08:00
Dongyu Li
1ea4459d9f
update knowledge base api ( #20426 )
2025-05-30 14:45:30 +08:00
-LAN-
a6ea15e63c
Refactor/message cycle manage and knowledge retrieval ( #20460 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-05-30 14:36:44 +08:00
yihong
5a991295e0
fix: drop some type fixme ( #20344 )
2025-05-30 14:10:09 +08:00
-LAN-
482e50aae9
Refactor/remove db from cycle manager ( #20455 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-05-30 04:34:13 +08:00
jyong
a025db137d
Merge branch 'main' into feat/r2
2025-05-29 09:54:28 +08:00
jyong
7f59ffe7af
r2
2025-05-28 17:56:04 +08:00
rouxiaomin
4c4887c5fc
feat(qdrant):add replication_factor when create collection in qdrant ( #20133 )
...
Co-authored-by: 刘敏 <min.liu@tongdun.net>
2025-05-27 14:46:04 +08:00
jyong
5fc2bc58a9
r2
2025-05-27 00:01:23 +08:00
jyong
35be8721b9
Merge branch 'main' into feat/r2
2025-05-26 14:50:33 +08:00
He Huang
6f48af2610
Refactor OpenSearch config to separate use_ssl and verify_certs flags ( #20075 )
...
Co-authored-by: he.huang <he.huang1@outlook.com>
Co-authored-by: crazywoola <427733928@qq.com>
2025-05-22 10:14:38 +08:00
wlleiiwang
7d230acf40
tencent vectordb compatible with version 1.1.3 and below ( #20056 )
...
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
2025-05-21 20:24:05 +08:00
-LAN-
3196dc2d61
refactor: Use typed SQLAlchemy base model and fix type errors ( #19980 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-05-21 15:38:03 +08:00
Darlewo
8cb3b4aef2
fix: multiple retrieve reranking_enabled switch ( #19958 )
2025-05-20 15:22:03 +08:00
jyong
82be119fec
Merge branch 'main' into feat/r2
2025-05-20 15:18:52 +08:00
Amir Mohsen Asaran
c9ee60e197
Feat(WaterCrawl error handling): add custom exceptions and error handling ( #19948 )
2025-05-20 10:25:16 +08:00
-LAN-
4977bb21ec
feat(workflow): domain model for workflow node execution ( #19430 )
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-05-17 00:56:16 +08:00
jyong
4ff971c8a3
r2
2025-05-16 11:26:56 +08:00
jyong
360f8a3375
Merge branch 'main' into feat/r2
2025-05-15 15:15:23 +08:00
k-kayashima
b292990075
Fix: Ensure unique index names for pgvector knowledge tables ( #19672 )
...
Co-authored-by: crazywoola <427733928@qq.com>
2025-05-15 11:43:44 +08:00
非法操作
085bd1aa93
chore: model.query change to db.session.query ( #19551 )
...
Co-authored-by: QuantumGhost <obelisk.reg+git@gmail.com>
2025-05-13 09:13:12 +08:00
非法操作
14cd71ed0a
chore: all model.query replace to db.session.query ( #19521 )
2025-05-12 15:19:41 +08:00
非法操作
b00f94df64
fix: replace all dataset.Model.query to db.session.query(Model) ( #19509 )
2025-05-12 13:52:33 +08:00
湛露先生
1119790b02
clean rag word_extractor. ( #19397 )
...
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
2025-05-09 16:39:16 +08:00
jyong
3f1363503b
r2
2025-05-07 16:19:09 +08:00
Will
bfa652f2d0
fix: metadata filtering condition variable unassigned; fix External K… ( #19208 )
2025-05-07 14:52:09 +08:00
jyong
a998022c12
r2
2025-05-06 16:18:34 +08:00
jyong
a25cc4e8af
r2
2025-05-06 13:56:13 +08:00
Hao Cheng
9bcf837f17
fix: use only supported operators in metadata filter system prompts ( #19195 )
2025-05-03 20:08:08 +08:00
Will
a212a63e6a
fix: time type metadata filtering error ( #19192 )
2025-05-03 20:07:37 +08:00
Bowen Liang
12c96b93d9
immediately return initialed tiktokenizer instance and remove dead code in usage of tiktokenizer ( #17957 )
2025-04-30 16:07:20 +08:00
QuantumGhost
bd1bbfee4b
Enhance Code Consistency Across Repository with .editorconfig ( #19023 )
2025-04-29 18:04:33 +08:00
Ahmad Zidan
8266815cda
feat: add AWS Managed IAM auth for OpenSearch vector DB ( #18963 )
2025-04-29 15:10:08 +08:00
Ethan
8b4ea01810
feat: support access milvus with token ( #19034 )
2025-04-29 14:52:13 +08:00
Panpan
83187b30c0
fix: fix rerank model runner usage ( #19008 )
2025-04-29 14:51:21 +08:00
Wesley
b62eb61400
fix depth param issue for WaterCrawl ( #18839 )
2025-04-27 11:04:56 +08:00
Jiang
37e2f73909
[Lindorm VDB] Add the QUERY_TIMEOUT parameter to force the search query to fail. ( #18613 )
...
Co-authored-by: jiangzhijie <jiangzhijie.jzj@alibaba-inc.com>
2025-04-25 09:42:58 +08:00
王晓阳
0babdffe3e
feat: support vastbase vector database ( #16308 )
2025-04-24 18:04:57 +08:00
Jyong
e2cb7006c4
check metadata_filtering_conditions could be None in auto mode ( #18548 )
2025-04-22 17:09:33 +08:00
lauding
eb1ce3dd6b
feat: support huawei cloud vector database ( #16141 )
2025-04-22 13:03:35 +08:00
tmuife
7b6523e54d
Update Oracle db connection library and change connection pool to single connection ( #18466 )
2025-04-21 17:56:57 +08:00
Rain Wang
d2e3744ca3
Switching from CONSOLE_API_URL to FILES_URL in word_extractor.py ( #18249 )
2025-04-18 16:05:48 +08:00
Rain Wang
83f1aeec1d
Fix ORDER BY (score, id) error in api/core/rag/datasource/vdb/analyticdb/analyticdb_vector_sql.py line 249 ( #18252 )
2025-04-17 14:15:05 +08:00
Rain Wang
e8d98e3d89
Add analyzer_params config for milvus vectordb ( #18180 )
2025-04-17 10:38:56 +08:00
Jyong
95283b4dd3
Feat/change split length method ( #18097 )
...
Co-authored-by: JzoNg <jzongcode@gmail.com>
2025-04-16 12:28:22 +08:00
YANG
d119c7d629
ignore errors when creating duplicate indexes ( #18069 )
...
Co-authored-by: 璟义 <yangshangpo.ysp@alibaba-inc.com>
2025-04-15 15:48:16 +08:00
Jasonfish
1f722cde22
fix(api): Some params were ignored when creating empty Datasets through API ( #17932 )
2025-04-14 10:24:01 +08:00
Yongtao Huang
5d72003ebb
Remove dead code ( #17899 )
2025-04-11 20:33:52 +08:00
briqt
91cfa90503
Fix external knowledge Issues: ( #17685 ) ( #17843 )
2025-04-11 15:37:27 +08:00
yihong
f04d52c044
fix: autocorrect everything in api ( #17859 )
...
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
2025-04-11 15:24:39 +08:00
wlleiiwang
9d20561af4
create db if not exists ( #17796 )
...
Co-authored-by: wlleiiwang <wlleiiwang@tencent.com>
2025-04-10 18:03:22 +08:00
