Commit Graph

31 Commits

Author SHA1 Message Date
-LAN- 559ab46ee1
fix: Removes redundant token calculations and updates dependencies
Eliminates unnecessary pre-calculation of token limits and recalculation of max tokens
across multiple app runners, simplifying the logic for prompt handling.

Updates tiktoken library from version 0.8.0 to 0.9.0 for improved tokenization performance.

Increases default token limit in TokenBufferMemory to accommodate larger prompt messages.

These changes streamline the token management process and leverage the latest
improvements in the tiktoken library.

Fixes potential token overflow issues and prepares the system for handling larger
inputs more efficiently.

Relates to internal optimization tasks.

Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-04-28 15:39:12 +08:00
yihong 56e15d09a9
feat: mypy for all type check (#10921) 2024-12-24 18:38:51 +08:00
非法操作 08ac36812b
feat: support LLM process document file (#10966)
Co-authored-by: -LAN- <laipz8200@outlook.com>
2024-11-22 19:32:44 +08:00
-LAN- c5f7d650b5
feat: Allow using file variables directly in the LLM node and support more file types. (#10679)
Co-authored-by: Joel <iamjoel007@gmail.com>
2024-11-22 16:30:22 +08:00
-LAN- 25ca0278dd
refactor(core): Remove extra_config from File. (#10203) 2024-11-08 18:13:24 +08:00
非法操作 5f12616cb9
fix: file type document is not supported (#9618) 2024-10-22 16:33:50 +08:00
-LAN- e61752bd3a
feat/enhance the multi-modal support (#8818) 2024-10-21 10:43:49 +08:00
kurokobo 857055b797
fix: remove the latest message from the user that does not have any answer yet (#9297) 2024-10-13 23:25:50 +08:00
Hash Brown 511ffa4698
fix: follow-up (suggested questions) does not refer to the most recen… (#9122) 2024-10-09 21:10:55 +08:00
Hash Brown 8c51d06222
feat: regenerate in `Chat`, `agent` and `Chatflow` app (#7661) 2024-09-22 03:15:11 +08:00
Bowen Liang a1104ab97e
chore: refurish python code by applying Pylint linter rules (#8322) 2024-09-13 22:42:08 +08:00
Bowen Liang 40fb4d16ef
chore: refurbish Python code by applying refurb linter rules (#8296) 2024-09-12 15:50:49 +08:00
Bowen Liang 2cf1187b32
chore(api/core): apply ruff reformatting (#7624) 2024-09-10 17:00:20 +08:00
Even c013086e64
fix: next suggest question logic problem (#6451)
Co-authored-by: evenyan <yikun.yan@ubtrobot.com>
2024-07-19 20:26:11 +08:00
takatost d522308a29
chore: optimize memory fetch performance (#6039) 2024-07-07 08:54:24 +08:00
takatost 23dc6edb99
chore: optimize memory messages fetch count limit (#6021) 2024-07-06 03:25:38 +08:00
takatost 3929d289e0
feat: set default memory messages limit to infinite (#5002) 2024-06-06 17:39:44 +08:00
takatost d1dbbc1e33
feat: backend model load balancing support (#4927) 2024-06-05 00:13:04 +08:00
takatost 4ad3f2cdc2
fix: image text when retrieve chat histories (#3220) 2024-04-09 15:20:45 +08:00
takatost 7753ba2d37
FEAT: NEW WORKFLOW ENGINE (#3160)
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: nite-knite <nkCoding@gmail.com>
Co-authored-by: jyong <718720800@qq.com>
2024-04-08 18:51:46 +08:00
Vikey Chen 237bb8514e
replace message content type list to string when file_objs is empty .. (#2745) 2024-03-08 18:46:31 +08:00
Bowen Liang 843280f82b
enhancement: introduce Ruff for Python linter for reordering and removing unused imports with automated pre-commit and sytle check (#2366) 2024-02-06 13:21:13 +08:00
Bowen Liang cc9e74123c
improve: introduce isort for linting Python imports (#1983) 2024-01-12 12:34:01 +08:00
takatost d069c668f8
Model Runtime (#1858)
Co-authored-by: StyleZhang <jasonapring2015@outlook.com>
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
Co-authored-by: chenhe <guchenhe@gmail.com>
Co-authored-by: jyong <jyong@dify.ai>
Co-authored-by: Joel <iamjoel007@gmail.com>
Co-authored-by: Yeuoly <admin@srmxy.cn>
2024-01-02 23:42:00 +08:00
Charlie.Wei 6b57e4e0ff
Fix chitchat lost context (#1828)
Co-authored-by: luowei <glpat-EjySCyNjWiLqAED-YmwM>
Co-authored-by: crazywoola <427733928@qq.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2023-12-23 10:05:17 +08:00
takatost 41d0a8b295
feat: [backend] vision support (#1510)
Co-authored-by: Garfield Dai <dai.hai@foxmail.com>
2023-11-13 22:05:46 +08:00
Garfield Dai 42a5b3ec17
feat: advanced prompt backend (#1301)
Co-authored-by: takatost <takatost@gmail.com>
2023-10-12 10:13:10 -05:00
takatost 5fa2161b05
feat: server multi models support (#799) 2023-08-12 00:57:00 +08:00
John Wang 7599f79a17
feat: claude api support (#572) 2023-07-17 00:14:19 +08:00
John Wang 3241e4015b
feat: upgrade langchain (#430)
Co-authored-by: jyong <718720800@qq.com>
2023-06-25 16:49:14 +08:00
John Wang db896255d6 Initial commit 2023-05-15 08:51:32 +08:00