opensource
/
dify
mirror of https://github.com/langgenius/dify.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
7,400 Commits 296 Branches 160 Tags 513 MiB
Commit Graph

259 Commits

Author SHA1 Message Date
yihong 56e15d09a9
feat: mypy for all type check (#10921) 2024-12-24 18:38:51 +08:00
yihong efa8eb379f
fix: memory leak by pypdfium2 close(maybe) #11510 (#11700) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2024-12-17 00:42:01 +08:00
-LAN- 430ca3322b
chore(dependency): bump gunicorn to 23.0 (#11560) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
 2024-12-13 16:16:58 +08:00
-LAN- 8d4bb9b40d
feat: integrate opendal storage (#11508) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
 2024-12-11 14:50:54 +08:00
Warren Chen 376726cf90
[feat] Add AWS Bedrock rerank (#11349) 
Co-authored-by: crazywoola <427733928@qq.com>
 2024-12-05 11:31:43 +08:00
Bowen Liang a67b0d4771
chore(lint): extract ruff configs into .ruff.toml file keeping pyproject.toml clean (#11222) 2024-12-01 12:51:28 +08:00
Bowen Liang 2feb44e2c5
chore(dep): bump flask from 3.0.1 to 3.1.0 and flask-compress to 1.17 (#11195) 2024-11-29 09:28:53 +08:00
yihong b791a80b75
chore: update chromadb version to 0.5.20 (#11038) 
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
 2024-11-25 11:14:04 +08:00
Bowen Liang 6c8e208ef3
chore: bump minimum supported Python version to 3.11 (#10386) 2024-11-24 13:28:46 +08:00
Bowen Liang d3051eed48
chore (dep): bump gevent from v23 to v24 for better support for Python 3.11 and 3.12 (#10387) 2024-11-23 00:07:07 +08:00
-LAN- c5f7d650b5
feat: Allow using file variables directly in the LLM node and support more file types. (#10679) 
Co-authored-by: Joel <iamjoel007@gmail.com>
 2024-11-22 16:30:22 +08:00
Tao Wang 90d6ebc879
Add youtube-transcript-api as tool (#10772) 
Co-authored-by: crazywoola <427733928@qq.com>
 2024-11-18 10:58:16 +08:00
Bowen Liang 51db59622c
chore(lint): cleanup repeated cause exception in logging.exception replaced by helpful message (#10425) 2024-11-15 15:41:40 +08:00
Bowen Liang 365cb4b368
chore(lint): bump ruff from 0.6.9 to 0.7.3 (#10714) 2024-11-15 09:19:41 +08:00
Kalo Chin ad16180b1a
feat(tool): fal ai wizper ASR built-in tool (#10716) 2024-11-15 09:01:07 +08:00
liuhaoran 7c2a9b0744
celery worker log format following LOG_FORMAT env#9404 (#10016) 
Signed-off-by: root <root@localhost.localdomain>
Co-authored-by: root <root@localhost.localdomain>
 2024-11-08 17:12:09 +08:00
Bowen Liang 574c4a264f
chore(lint): Use logging.exception instead of logging.error (#10415) 2024-11-07 21:13:02 +08:00
Bowen Liang d45d90e8ae
chore: lazy import sagemaker (#10342) 2024-11-06 12:45:22 +08:00
Benjamin acb22f0fde
Updates: Add mplfonts library for customizing matplotlib fonts and Va… (#9903) 2024-11-05 10:34:28 +08:00
powerfool 878d13ef42
Added OceanBase as an option for the vector store in Dify (#10010) 2024-10-29 21:10:18 +08:00
roadgoat19 c8ef9223e5
feat: couchbase integration (#6165) 
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: Elliot Scribner <elliot.scribner@couchbase.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: Bowen Liang <bowenliang@apache.org>
 2024-10-29 15:00:23 +08:00
Jyong caf7bc8569
upgrade nltk, unstructured and starlette (#9860) 2024-10-25 17:15:44 +08:00
Jyong 121bb99cc2
downgrade unstructured nltk version (#9726) 2024-10-23 19:02:27 +08:00
Jyong cac1ef7ade
remove ppt import (#9721) 2024-10-23 18:22:30 +08:00
Jyong 3e9d271b52
nltk security issue and upgrade unstructured (#9558) 2024-10-23 16:23:55 +08:00
Zven 8e7a752b2a
feat: add upstash as a new vector database provider (#9644) 2024-10-23 09:16:35 +08:00
Bowen Liang 3f1c84f65a
chore: cleanup ineffective linter rules exclusions (#9580) 2024-10-22 09:18:31 +08:00
Bowen Liang 495cf58014
dep: bump pydantic to 2.9 (#9077) 2024-10-21 23:32:09 +08:00
-LAN- e61752bd3a
feat/enhance the multi-modal support (#8818) 2024-10-21 10:43:49 +08:00
Warren Wong b597a0d31c
fix: Azure OpenAI o1 max_completion_token and get_num_token_from_messages error (#9326) 
Co-authored-by: wwwc <wwwc@outlook.com>
 2024-10-15 16:26:44 +08:00
非法操作 ffc3f33670
chore: remove the copied zhipu_ai sdk (#9270) 2024-10-14 10:53:45 +08:00
ice yao d15ba3939d
Add Volcengine VikingDB as new vector provider (#9287) 2024-10-13 21:26:05 +08:00
Shili Cao 2ec6ffe478
feat:support baidu vector db (#9185) 2024-10-12 23:24:17 +08:00
zhuhao d9773c963f
chore: fix the misclassification of the opensearch-py package (#9266) 2024-10-12 17:37:01 +08:00
zhuhao 7a6970e570
feat: add supabase object storage (#9229) 2024-10-11 22:48:57 +08:00
zhuhao 05c1ef75c4
feat: add allow_llm_to_see_data flag for vanna (#9156) 2024-10-10 17:39:26 +08:00
Jyong 68107fe355
downgrade elastic to 8.14 (#9171) 2024-10-10 16:43:11 +08:00
Bowen Liang b279d19040
chore: avoid star imports usage (#9123) 2024-10-09 21:12:50 +08:00
Bowen Liang 240b66d737
chore: avoid implicit optional in type annotations of method (#8727) 2024-10-09 14:36:43 +08:00
Bowen Liang b360feb4c1
refactor: introduce storage factory and speed up api startup by importing storage client on demand (#9086) 2024-10-09 14:15:27 +08:00
Bowen Liang 896998ef3f
chore: refine python dependency list and check dependencies in order (#9061) 2024-10-08 15:11:45 +08:00
Bowen Liang 7c0b159a81
chore: removing unused imports in tests (#9049) 2024-10-08 11:13:11 +08:00
Bowen Liang b933c9d206
chore: move testing env variables from pyproject.toml to pytest.ini (#9019) 2024-10-08 08:40:29 +08:00
呆萌闷油瓶 f45042aa8e
fix:ddg ratelimit 202 (#9047) 2024-10-07 22:13:41 +08:00
zhuhao 2571b0c4e3
feat: add baidu obs storage (#9024) 2024-10-07 11:09:27 +08:00
Jyong 9d221a5e19
external knowledge api (#8913) 
Co-authored-by: Yi <yxiaoisme@gmail.com>
 2024-09-30 15:38:43 +08:00
zhuhao fb49413a41
feat: add voyage ai as a new model provider (#8747) 2024-09-29 16:55:59 +08:00
Bowen Liang 74f58f29f9
chore: bump ruff to 0.6.8 for fixing violation in SIM910 (#8869) 2024-09-29 00:29:59 +08:00
Bowen Liang a8b837c4a9
dep: bump ElasticSearch from 8.14.x to 8.15.x (#8197) 2024-09-25 22:55:24 +08:00
zhuhao 1ecf70dca0
feat: add mixedbread as a new model provider (#8523) 2024-09-24 11:20:15 +08:00
ice yao d7aada38a1
Add nomic embedding model provider (#8640) 2024-09-23 19:57:21 +08:00
Bowen Liang 86f90fd9ff
chore: skip PLR6201 linter rule (#8666) 2024-09-23 15:28:57 +08:00
CXwudi 97895ec41a
chore: add Gemini newest experimental models (close #7121) (#8621) 2024-09-22 13:38:08 +08:00
ice yao 0665268578
Add Fireworks AI as new model provider (#8428) 2024-09-22 10:13:00 +08:00
呆萌闷油瓶 c8b9bdebfe
feat:use xinference tts stream mode (#8616) 2024-09-22 10:08:35 +08:00
Joe b32a7713e0
feat: update pyproject.toml (#8368) 2024-09-21 23:59:50 +08:00
Nam Vu bf55b1910f
fix: pyproject.toml typo (#8396) 2024-09-14 09:45:49 +08:00
Bowen Liang 5b98acde2f
chore: improve usage of striping prefix or suffix of string with Ruff 0.6.5 (#8392) 2024-09-13 23:34:39 +08:00
Bowen Liang a1104ab97e
chore: refurish python code by applying Pylint linter rules (#8322) 2024-09-13 22:42:08 +08:00
-LAN- 80a322aaa2
chore: update version to 0.8.2 in packaging and docker-compose files (#8352) 2024-09-13 13:45:13 +08:00
takatost e90d3c29ab
feat: add OpenAI o1 series models support (#8328) 2024-09-13 02:15:19 +08:00
ybalbert001 d4985fb3aa
Fix: Support Bedrock cross region inference [#8190](https://github.com/langgenius/dify/issues/8190) (#8317) 2024-09-12 19:15:20 +08:00
Bowen Liang 8815511ccb
chore: apply flake8-pytest-style linter rules (#8307) 2024-09-12 18:09:16 +08:00
Bowen Liang 40fb4d16ef
chore: refurbish Python code by applying refurb linter rules (#8296) 2024-09-12 15:50:49 +08:00
Bowen Liang c69f5b07ba
chore: apply ruff E501 line-too-long linter rule (#8275) 
Co-authored-by: -LAN- <laipz8200@outlook.com>
 2024-09-12 14:00:36 +08:00
Bowen Liang 0f14873255
chore: cleanup ruff flake8-simplify linter rules (#8286) 
Co-authored-by: -LAN- <laipz8200@outlook.com>
 2024-09-12 12:55:45 +08:00
Bowen Liang 781d294f49
chore: cleanup pycodestyle E rules (#8269) 2024-09-11 18:55:00 +08:00
Bowen Liang 292220c596
chore: apply pep8-naming rules for naming convention (#8261) 2024-09-11 16:40:52 +08:00
Bowen Liang d109881410
chore(api/models): apply ruff reformatting (#7600) 
Co-authored-by: -LAN- <laipz8200@outlook.com>
 2024-09-10 17:08:06 +08:00
Bowen Liang 2cf1187b32
chore(api/core): apply ruff reformatting (#7624) 2024-09-10 17:00:20 +08:00
ice yao c8df92d0eb
add volcengine tos storage (#8164) 2024-09-10 09:19:47 +08:00
ybalbert001 954580a4af
feat: support more model types and builtin tools on aws/sagemaker (#8061) 
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
 2024-09-09 10:34:11 +08:00
tmuife 89aede80cc
Add OCI(Oracle Cloud Infrastructure) Generative AI Service as a Model Provider (#7775) 
Co-authored-by: Walter Jin <jinshuhaicc@gmail.com>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: walter from vm <walter.jin@oracle.com>
 2024-09-06 14:15:40 +08:00
zhuhao e0d3cd91c6
support huawei cloud obs storage (#7980) (#7981) 2024-09-06 14:00:47 +08:00
-LAN- 8986be0aab
chore: Update versions to 0.7.3 (#7895) 2024-09-03 09:49:32 +08:00
Bowen Liang 3f467613fc
feat: support configs for code execution request (#7704) 2024-08-27 19:38:33 +08:00
Hélio Lúcio 7b7576ad55
Add Azure AI Studio as provider (#7549) 
Co-authored-by: Hélio Lúcio <canais.hlucio@voegol.com.br>
 2024-08-27 09:52:59 +08:00
-LAN- 13be84e4d4
chore(api/controllers): Apply Ruff Formatter. (#7645) 2024-08-26 15:29:10 +08:00
Bowen Liang 17fd773a30
chore(api/services): apply ruff reformatting (#7599) 
Co-authored-by: -LAN- <laipz8200@outlook.com>
 2024-08-26 13:43:57 +08:00
Bowen Liang 979422cdc6
chore(api/tasks): apply ruff reformatting (#7594) 2024-08-26 13:38:37 +08:00
sino efc136cce5
feat: Introduce Ark SDK v3 and ensure compatibility with models of SDK v2 (#7579) 
Co-authored-by: crazywoola <427733928@qq.com>
 2024-08-24 19:29:45 +08:00
Bowen Liang b035c02f78
chore(api/tests): apply ruff reformat #7590 (#7591) 
Co-authored-by: -LAN- <laipz8200@outlook.com>
 2024-08-23 23:52:25 +08:00
Bowen Liang 2da63654e5
chore(api/configs): apply ruff reformat (#7590) 2024-08-23 23:46:01 +08:00
Bowen Liang 3ace01cfb3
chore: cleanup and rearrange unclassified configs into feature config groups (#7586) 2024-08-23 22:40:07 +08:00
Jyong 4ce47284dc
update nltk version to 3.8.1 (#7544) 2024-08-22 18:17:21 +08:00
Xiao Ley 0c99a3d0c5
fix the issue of the refine_switches at param being invalid in the Novita.AI tool (#7485) 2024-08-21 15:09:05 +08:00
Bowen Liang 68dc6d5bc3
chore: rearrange api python dependencies (#7391) 2024-08-19 14:05:41 +08:00
Bowen Liang bbb6fcc4f0
chore: update ruff from 0.5.x to 0.6.x (#7384) 2024-08-19 09:21:11 +08:00
-LAN- 9414143b5f
chore(api/libs): Apply ruff format. (#7301) 2024-08-15 17:53:12 +08:00
-LAN- 3571292fbf
chore(api): Introduce Ruff Formatter. (#7291) 2024-08-15 12:54:05 +08:00
miendinh f104b930cf
feat: support elasticsearch vector database (#3558) 
Co-authored-by: miendinh <miendinh@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
Co-authored-by: crazywoola <427733928@qq.com>
 2024-08-13 17:36:20 +08:00
Bowen Liang ccb6ddd840
chore: bump Ruff to 0.5.7 (#7174) 2024-08-12 10:24:48 +08:00
ybalbert001 f2cb1fb09f
Fix : Workflow "start" paste url not support s3 pre-signed URL (#6855) 
Co-authored-by: Yuanbo Li <ybalbert@amazon.com>
 2024-08-11 16:45:15 +08:00
Yanyi Liu 5b32f2e0dd
Feat: Add model provider Text Embedding Inference for embedding and rerank (#7132) 2024-08-09 19:12:13 +08:00
非法操作 312d905c9b
chore: update duckduckgo tool (#6983) 2024-08-06 10:16:04 +08:00
takatost 6da14c2d48
security: fix api image security issues (#6971) 2024-08-05 20:21:08 +08:00
Bowen Liang 70283f5b9f
dep: support for Python 3.12 (#6771) 2024-08-02 21:14:36 +08:00
JuHyung Son 2e941bb91c
add new provider Solar (#6884) 2024-08-02 20:48:09 +08:00
Bowen Liang 6aa02f8c63
dep: bump pgvecto-rs client from 0.1.x to 0.2.x (#6891) 2024-08-02 15:51:23 +08:00
Weaxs f6e8e120a1
support xinference tts (#6746) 2024-08-01 11:59:15 +08:00
Bowen Liang daa31b2cb3
chore: remove redundant version pinning for indirect dependencies (#6772) 2024-07-30 08:45:57 +08:00
Bowen Liang b414ea41d6
dep: initial support for Milvus 2.4.x (#6084) 2024-07-29 19:56:45 +08:00
Jyong cf258b7a67
add xlsx support hyperlink extract (#6722) 2024-07-26 19:26:52 +08:00
Jyong e4bb943fe5
Feat/delete single dataset retrival (#6570) 2024-07-24 12:50:11 +08:00
takatost 0fb741f269
fix: downgraded sentry-sdk to 1.44.1 due to claude LLM token returning 0 (#6597) 2024-07-24 04:49:03 +08:00
dependabot[bot] 642723d09e
chore(deps): bump sentry-sdk from 1.39.2 to 2.8.0 in /api (#6517) 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
 2024-07-23 13:48:23 +08:00
Weaxs 4e2fba404d
WebscraperTool bypass cloudflare site by cloudscraper (#6337) 2024-07-17 14:13:57 +08:00
Bowen Liang 7943f7f697
chore: fix legacy API usages of Query.get() by Session.get() in SqlAlchemy 2 (#6340) 2024-07-17 13:54:35 +08:00
Bowen Liang c5d06e7943
dep: bump Pydantic from 2.7 to 2.8 (#6273) 2024-07-16 13:40:58 +08:00
Zhuo Qiu 63e34e5227
feat: support MyScale vector database (#6092) 2024-07-11 15:21:59 +08:00
Bowen Liang f9d00e0498
chore: use poetry for linter tools installation and bump Ruff from 0.4 to 0.5 (#6081) 2024-07-09 23:06:23 +08:00
8bitpd 7c70eb87bc
feat: support AnalyticDB vector store (#5586) 
Co-authored-by: xiaozeyu <xiaozeyu.xzy@alibaba-inc.com>
 2024-07-09 13:32:04 +08:00
chenxu9741 6ef401a9f0
feat:add tts-streaming config and future (#5492) 2024-07-09 11:33:58 +08:00
takatost 68b1d063f7
chore: remove tsne unused code (#6077) 2024-07-08 21:25:19 +08:00
K8sCat 71c50b7e20
feat: add Llama 3 and Mixtral model options to ddgo_ai.yaml (#5979) 
Signed-off-by: K8sCat <k8scat@gmail.com>
 2024-07-05 21:11:15 +08:00
ahasasjeb cc63af8e72
Removed firecrawl-py, fixed and improved firecrawl tool (#5896) 
Co-authored-by: -LAN- <laipz8200@outlook.com>
 2024-07-05 18:04:51 +08:00
longzhihun fdfbbde10d
[seanguo] modify bedrock Claude3 invoke method to converse API (#5768) 
Co-authored-by: Chenhe Gu <guchenhe@gmail.com>
 2024-07-01 04:36:13 +08:00
scotlandowl 73ce945d40
Feat/add json process tool (#5555) 2024-06-28 11:57:32 +08:00
Bowen Liang 2a13ef9ae0
chore: rearrange python dependencies in groups (#5603) 2024-06-27 17:52:54 +08:00
Bowen Liang dcb72e0067
chore: apply flake8-comprehensions Ruff rules to improve collection comprehensions (#5652) 
Co-authored-by: -LAN- <laipz8200@outlook.com>
 2024-06-27 11:21:31 +08:00
Joe 4e2de638af
feat: add ops trace (#5483) 
Co-authored-by: takatost <takatost@gmail.com>
 2024-06-26 17:33:29 +08:00
-LAN- f7234c93af
chore(pyproject.toml): Add Ruff formatter config. (#5627) 2024-06-26 16:30:28 +08:00
tmuife 6a09409ec9
Add Oracle23ai as a vector datasource (#5342) 
Co-authored-by: walter from vm <walter.jin@oracle.com>
 2024-06-22 01:48:07 +08:00
Bowen Liang aed56b1a8f
fix: Revert "feat: initial support for Milvus 2.4.x (#3795)" downgrading to 2.3.x for Linux arm64 installation failure (#5414) 2024-06-20 11:18:05 +08:00
Bowen Liang bb33ffc332
feat: initial support for Milvus 2.4.x (#3795) 2024-06-19 13:55:44 +08:00
Bowen Liang 3cc6093e4b
feat: introduce pydantic-settings for config definition and validation (#5202) 
Co-authored-by: -LAN- <laipz8200@outlook.com>
 2024-06-19 13:41:12 +08:00
baojingyu d160d1ed02
feat: support opensearch approximate k-NN (#5322) 2024-06-19 12:44:33 +08:00
-LAN- 4e3d76a1d1
chore: add `novita_client` to `pyproject.toml` (#5349) 2024-06-18 14:52:20 +08:00
Bowen Liang 7450b9acf3
dep: bump chromadb from 0.5.0 to 0.5.1 (#5345) 2024-06-18 14:05:14 +08:00
Bowen Liang c7d378555a
chore: set build system to Poetry and remove unnecessary settings with package mode disabled (#5263) 2024-06-18 13:27:03 +08:00
quicksand 147a39b984
feat: support tencent cos storage (#5297) 2024-06-17 19:18:52 +08:00
Bowen Liang 7a758a35fe
fix: pin tenacity to 8.3.0 (#5319) 2024-06-17 18:03:42 +08:00
-LAN- 5a99aeb864
fix(core): Reorder `field_validator` and `classmethod` to fit Pydantic V2. (#5257) 2024-06-17 10:04:28 +08:00
takatost d098bdc59b
version to 0.6.11 (#5224) 2024-06-15 02:46:24 +08:00
Jyong ba5f8afaa8
Feat/firecrawl data source (#5232) 
Co-authored-by: Nicolas <nicolascamara29@gmail.com>
Co-authored-by: chenhe <guchenhe@gmail.com>
Co-authored-by: takatost <takatost@gmail.com>
 2024-06-15 02:46:02 +08:00
quicksand 4080f7b8ad
feat: support tencent vector db (#3568) 2024-06-14 19:25:17 +08:00
Bowen Liang 2b04388361
chore: remove bump-pydantic dependency (#5177) 2024-06-14 15:05:17 +08:00
dependabot[bot] 54e02b8147
chore(deps): bump authlib from 1.2.0 to 1.3.1 in /api (#5115) 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: takatost <takatost@gmail.com>
 2024-06-14 03:55:40 +08:00
dependabot[bot] 7189a4c379
chore(deps): bump azure-identity from 1.15.0 to 1.16.1 in /api (#5116) 
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: takatost <takatost@gmail.com>
 2024-06-14 03:24:32 +08:00
Bowen Liang f976740b57
improve: mordernizing validation by migrating pydantic from 1.x to 2.x (#4592) 2024-06-14 01:05:37 +08:00
Bowen Liang cdc08a434f
feat: support Chroma vector store (#5015) 2024-06-13 18:02:18 +08:00
takatost 0c92f81efc
chore: sync pyproject.toml from requirements.txt (#5130) 2024-06-13 00:06:05 +08:00
Bowen Liang 25b0a97851
build: use Poetry as default build system for dependency installation in CI jobs (#5088) 2024-06-12 14:43:03 +08:00
takatost e3544c6ef7
fix: dependency package versions are not synchronized to requirements.txt (#5084) 2024-06-11 22:21:18 +08:00
Matri f62f71a81a
build: initial support for poetry build tool (#4513) 
Co-authored-by: Bowen Liang <bowenliang@apache.org>
 2024-06-11 13:11:28 +08:00
takatost 3579fd1b09
feat: add create tenant command (#4974) 2024-06-06 00:42:00 +08:00
Bowen Liang f32b440c4a
chore: fix indention violations by applying E111 to E117 ruff rules (#4925) 2024-06-05 14:05:15 +08:00
Bowen Liang b234710af9
chore: fix invalid escape sequences by applying W605 rule (#4851) 2024-06-02 10:02:37 +08:00
Bowen Liang 3fda2245a4
improve: extract method for safe loading yaml file and avoid using PyYaml's FullLoader (#4031) 2024-05-24 12:08:12 +08:00
Bowen Liang e8e213ad1e
chore: apply and fix flake8-bugbear lint rules (#4496) 2024-05-20 16:34:13 +08:00