dd34002db2
Merge remote-tracking branch 'upstream/main' into feat/rag-2
2025-09-16 14:59:35 +08:00
bab4975809
chore: add ast-grep rule to convert Optional[T] to T | None ( #25560 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-09-15 13:06:33 +08:00
a13d7987e0
chore: adopt StrEnum and auto() for some string-typed enums ( #25129 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2025-09-12 21:14:26 +08:00
23cd615489
Merge branch 'feat/queue-based-graph-engine' into feat/rag-2
2025-09-08 14:30:43 +08:00
a78339a040
remove bare list, dict, Sequence, None, Any ( #25058 )
...
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
2025-09-06 03:32:23 +08:00
103a9a4e67
fix(graph_engine): add type hint for workers_to_remove
2025-09-05 01:59:11 +08:00
15b3443e9e
fix(debug_logging_layer): remove access for variable pool
2025-09-05 01:52:19 +08:00
b9394d542c
Merge branch 'feat/queue-based-graph-engine' into feat/rag-2
...
# Conflicts:
# api/core/app/apps/advanced_chat/generate_task_pipeline.py
# api/pyproject.toml
# api/uv.lock
# docker/docker-compose-template.yaml
# docker/docker-compose.yaml
# web/package.json
2025-09-04 20:30:08 +08:00
e229510e73
perf: eliminate lock contention in worker pool by removing callbacks
...
Remove worker idle/active callbacks that caused severe lock contention.
Instead, use sampling-based monitoring where worker states are queried
on-demand during scaling decisions. This eliminates the performance
bottleneck caused by workers acquiring locks 10+ times per second.
Changes:
- Remove callback parameters from Worker class
- Add properties to expose worker idle state directly
- Update WorkerPool to query worker states without callbacks
- Maintain scaling functionality with better performance
2025-09-04 19:37:31 +08:00
36048d1526
feat(graph_engine): allow to scale down without lock
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-04 19:32:07 +08:00
ad9eed2551
fix: disable scale for perfermance
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-04 19:11:22 +08:00
4d63bd2083
refactor(graph_engine): rename SimpleWorkerPool to WorkerPool
2025-09-04 17:47:13 +08:00
226f14a20f
feat(graph_engine): implement scale down worker
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-04 15:35:20 +08:00
ac057a2d40
Chore: remove dead code in class Graph ( #22791 )
...
Co-authored-by: Yongtao Huang <99629139+hyongtao-db@users.noreply.github.com>
2025-09-04 10:30:04 +08:00
04bbf540d9
chore: code format
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-04 02:33:53 +08:00
657c27ec75
feat(graph_engine): make runtime state read-only in layer
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-04 02:30:40 +08:00
8332472944
refactor(graph_engine): rename Layer to GraphEngineLayer
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-04 02:11:31 +08:00
fe3f03e50a
feat: add property-based access control to GraphRuntimeState
...
- Replace direct field access with private attributes and property decorators
- Implement deep copy protection for mutable objects (dict, LLMUsage)
- Add helper methods: set_output(), get_output(), update_outputs()
- Add increment_node_run_steps() and add_tokens() convenience methods
- Update loop_node and event_handlers to use new accessor methods
- Add comprehensive unit tests for immutability and validation
- Ensure backward compatibility with existing property access patterns
2025-09-04 02:08:58 +08:00
d4aed3df5c
Merge branch 'feat/queue-based-graph-engine' into feat/rag-2
...
# Conflicts:
# api/core/memory/token_buffer_memory.py
# api/core/rag/extractor/notion_extractor.py
# api/core/repositories/sqlalchemy_workflow_node_execution_repository.py
# api/core/variables/variables.py
# api/core/workflow/graph/graph.py
# api/core/workflow/graph_engine/entities/event.py
# api/services/dataset_service.py
# web/app/components/app-sidebar/index.tsx
# web/app/components/base/tag-management/selector.tsx
# web/app/components/base/toast/index.tsx
# web/app/components/datasets/create/website/index.tsx
# web/app/components/datasets/create/website/jina-reader/base/options-wrap.tsx
# web/app/components/workflow/header/version-history-button.tsx
# web/app/components/workflow/hooks/use-inspect-vars-crud-common.ts
# web/app/components/workflow/hooks/use-workflow-interactions.ts
# web/app/components/workflow/panel/version-history-panel/index.tsx
# web/service/base.ts
2025-09-03 15:01:06 +08:00
7ace7e65e1
fix preview and recommend plugins
2025-09-02 16:44:18 +08:00
1770b93e5b
chore(graph_engine): Add a TODO commment in `_update_response_outputs` in event_handlers
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-02 15:20:03 +08:00
d8ff4aa9ba
feat(graph_engine): Handle NodeRunAgentLogEvent
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-02 15:02:07 +08:00
9f8f21bf87
chore: remove backup files
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-02 15:01:58 +08:00
0b0dc63f29
Merge remote-tracking branch 'origin/main' into feat/queue-based-graph-engine
2025-09-02 11:52:25 +08:00
ffba341258
[CHORE]: remove redundant-cast ( #24807 )
2025-09-01 14:05:32 +08:00
8433cf4437
refactor(graph_engine): Merge event_collector and event_emitter into event_manager
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-01 13:15:58 +08:00
bb5d52539c
refactor(graph_engine): Merge branch_handler into edge_processor
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-01 12:53:06 +08:00
88622f70fb
refactor(graph_engine): Move setup methods into `__init__`
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-01 12:08:03 +08:00
0fdb1b2bc9
refactor(graph_engine): Correct private attributes and private methods naming
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-01 04:37:23 +08:00
a5cb9d2b73
refactor(graph_engine): inline output_registry into response_coordinator
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-01 03:59:53 +08:00
64c1234724
refactor(graph_engine): Merge worker management into one WorkerPool
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-01 03:23:47 +08:00
202fdfcb81
refactor(graph_engine): Remove backward compatibility code
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-01 02:41:16 +08:00
e2f4c9ba8d
refactor(graph_engine): Merge state managers into unified_state_manager
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-09-01 02:08:08 +08:00
82193580de
chore: improve typing
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-30 16:35:57 +08:00
d0dd728e6c
Merge branch 'main' into feat/rag-2
...
# Conflicts:
# api/core/app/entities/queue_entities.py
# api/core/workflow/graph_engine/entities/event.py
2025-08-29 11:29:51 +08:00
1db04aa729
Merge branch 'feat/queue-based-graph-engine' into feat/rag-2
2025-08-28 18:12:49 +08:00
c396788128
chore(graph_engine): add final mark to classes
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-28 15:38:35 +08:00
5c01dd97e8
clean typos words. ( #24667 )
...
Signed-off-by: zhanluxianshen <zhanluxianshen@163.com>
2025-08-28 15:23:59 +08:00
e3a7b1f691
fix: type hints
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-28 05:24:18 +08:00
8aab7f49c3
chore(graph_engine): Use `XXX | None` instead of `Optional[XXX]`
2025-08-28 05:09:33 +08:00
1e12c1cbf2
[autofix.ci] apply automated fixes
2025-08-27 21:00:36 +00:00
affedd6ce4
chore(graph_engine): Use `XXX | None` instead of `Optional[XXX]`
2025-08-28 04:59:49 +08:00
ef21097774
refactor(graph_engine): Remove unnecessary check from SkipPropagator
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-28 04:45:26 +08:00
1d377fe994
refactor(graph_engine): Use _ to mark unused variable in BranchHandler
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-28 04:44:45 +08:00
c82697f267
refactor(graph_engine): Remove `node_id` from SkipPropagator.skip_branch_paths
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-28 04:43:56 +08:00
98b25c0bbc
refactor(graph_engine): Convert attrs to private in error_handler
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-28 04:42:37 +08:00
1cd0792606
chore(graph_events): Improve type hints
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-28 04:41:48 +08:00
7cbf4093f4
chore(graph_engine): Use `TYPE | None` instead of `Optional`
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-28 04:30:50 +08:00
8129ca7c05
chore(graph_engine): Move error_strategy.py to protocols/
...
Signed-off-by: -LAN- <laipz8200@outlook.com>
2025-08-28 04:29:32 +08:00
65617f000d
feat(event_collector): Update to use ReadWriteLock
2025-08-28 03:26:42 +08:00
QuantumGhost