Commit Graph

1019 Commits

Author SHA1 Message Date
Yansong Zhang
e9ee897973 fix: resolve remaining CI failures for style checks and unit tests
- Add model_features property and build_execution_context method to
  AgentAppRunner to fix mypy attr-defined errors
- Export WorkflowComment, WorkflowCommentReply, WorkflowCommentMention
  from models/__init__.py to fix import errors
- Add NestedNodeGraphRequest, NestedNodeGraphResponse,
  NestedNodeParameterSchema to services/workflow/entities.py
- Update test_agent_chat_app_runner: tests for invalid LLM mode and
  invalid strategy now reflect unified AgentAppRunner behavior
  (no longer raises ValueError for these cases)

Made-with: Cursor
2026-04-13 16:07:38 +08:00
Yansong Zhang
b804c7ed47 fix: restore SandboxExpiredRecordsCleanConfig, remove debug logs
- Restore SandboxExpiredRecordsCleanConfig (billing/ops config that
  was mistakenly removed with sandbox execution code)
- Remove [DEBUG-AGENT] logging from app_generate_service.py

Made-with: Cursor
2026-04-13 14:39:48 +08:00
Yansong Zhang
94b3087b98 fix: resolve remaining CI failures
- app_model_config_service.py: add AppMode.AGENT to exhaustive match
- app_service.py: fix possibly unbound default_model_dict variable

Made-with: Cursor
2026-04-13 13:56:08 +08:00
zyssyz123
3e0578a1c6
Merge branch 'main' into feat/new-agent-node 2026-04-13 13:43:47 +08:00
dev-miro26
f4d5e2f43d
refactor(api): improve type safety in MCPToolManageService.execute_auth_actions (#34824) 2026-04-13 05:29:10 +00:00
Yansong Zhang
c03b25a940 merge: resolve conflicts with origin/main
Conflicts resolved:
- workflow_app_runner.py: adopt main's DifyGraphInitContext pattern
- token_buffer_memory.py: adopt main's match/case, add AppMode.AGENT
- app_dsl_service.py: adopt main's match/case, add AppMode.AGENT

Made-with: Cursor
2026-04-13 12:52:56 +08:00
HeYinKazune
a111d56ea3
refactor: use sessionmaker in workflow_tools_manage_service.py (#34896)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-13 03:47:29 +00:00
wdeveloper16
8436470fcb
refactor: replace bare dict with TypedDicts in annotation_service (#34998) 2026-04-13 03:46:33 +00:00
Yansong Zhang
90cce7693f revert: remove all sandbox and skill related code
Remove ~12,900 lines of sandbox/skill code that was ported from
feat/support-agent-sandbox. This reverts to direct tool execution
(the original behavior before sandbox integration).

Removed:
- core/sandbox/ (SandboxBuilder, bash tools, providers, initializers)
- core/skill/ (SkillManager, assembler, entities)
- core/virtual_environment/ (5 provider implementations)
- core/zip_sandbox/ (archive operations)
- core/app_assets/ (asset management)
- core/app_bundle/ (bundle management)
- controllers/cli_api/ (DifyCli callback endpoints)
- services/sandbox/ (provider service)
- services/skill_service, app_asset_service, app_bundle_service
- models/sandbox.py, app_asset.py
- bin/dify-cli-* (3 platform binaries)
- web sandbox-provider-page and service
- SandboxLayer, _resolve_sandbox_context, _invoke_tool_in_sandbox
- CliApiConfig, DIFY_SANDBOX_CONTEXT_KEY
- sandbox-related migrations

Preserved: All Agent V2 core functionality (agent-v2 node, strategy
engine, transparent upgrade, LLM remapping, memory, context, tools
via direct execution).

Made-with: Cursor
2026-04-13 10:42:36 +08:00
-LAN-
8c4ea5c898
fix: external dataset tenant checks for bound knowledge APIs (#34734) 2026-04-13 01:47:57 +00:00
GDS K S
0862fd74b0
refactor: migrate _model_to_insertion_dict return type to TypedDict (#34988) 2026-04-12 09:50:55 +00:00
wdeveloper16
0841b4c663
refactor(api): migrate tools, account, workflow and plugin services to SQLAlchemy 2.0 (#34966)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:45:27 +00:00
wdeveloper16
440602f52a
refactor(services): migrate summary_index_service to SQLAlchemy 2.0 select() API (#34971)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:37:16 +00:00
wdeveloper16
510120410b
refactor(services): migrate trigger_provider_service to SQLAlchemy 2.0 select() API (#34972)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:36:13 +00:00
wdeveloper16
4ef67fef3a
refactor(services): migrate builtin_tools_manage_service to SQLAlchemy 2.0 select() API (#34973)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:25:51 +00:00
wdeveloper16
7bd5e80323
refactor(services): migrate datasource_provider_service to SQLAlchemy 2.0 select() API (#34974) 2026-04-12 01:23:24 +00:00
wdeveloper16
7515eee0a8
refactor(services): migrate dataset_service and clear_free_plan_tenant_expired_logs to SQLAlchemy 2.0 select() API (#34970)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-12 01:21:52 +00:00
Sven Weidenmann
859920a81f
refactor: migrate verify_subscription_credentials return type to TypedDict (#34967) 2026-04-11 16:41:40 +00:00
wangxiaolei
65d66768c1
fix: fix tool output duplicate (#34962) 2026-04-11 15:07:31 +00:00
plind
2dc015b360
fix(api): default parent_mode to paragraph for hierarchical chunking via API (#34635)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-10 10:55:40 +00:00
Yansong Zhang
77c182f738 feat(api): propagate all app features in transparent upgrade
VirtualWorkflowSynthesizer._build_features() now extracts ALL legacy
app features from AppModelConfig into the synthesized workflow.features:

- opening_statement + suggested_questions
- sensitive_word_avoidance (keywords/API moderation)
- more_like_this
- speech_to_text / text_to_speech
- retriever_resource

Previously workflow.features was hardcoded to "{}", losing all these
features during transparent upgrade. Now AdvancedChatAppRunner's
moderation, opening text, and other feature layers work correctly
for transparently upgraded old apps.

Made-with: Cursor
2026-04-10 18:47:18 +08:00
corevibe555
98eedf14dc
refactor(services): replace Union with | syntax in service layer (#34905) 2026-04-10 07:44:47 +00:00
呆萌闷油瓶
cd3ee5bd5d
fix: sqlalchemy.orm.exc.DetachedInstanceError (#34910) 2026-04-10 07:44:22 +00:00
wangxiaolei
bcd738d2e6
fix: fix orm_exc.DetachedInstanceError (#34904)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-10 07:13:59 +00:00
corevibe555
488fcd4f83
refactor(services): replace Union with | syntax in service layer (batch 2) (#34906)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-10 07:05:31 +00:00
-LAN-
b90fe73c96
fix(api): prevent cross-tenant external API use-check disclosure (#34744) 2026-04-10 03:23:32 +00:00
dataCenter430
1117b6e72d
refactor: convert appmode misc if/elif to match/case (#30001) (#34869) 2026-04-10 00:35:12 +00:00
YBoy
2352269ba9
refactor(api): type recommend app database retrieval dicts with TypedDicts (#34873) 2026-04-10 00:32:24 +00:00
aliworksx08
e143dbce50
refactor: migrate session.query to select API in webhook service (#34849) 2026-04-09 14:16:33 +00:00
aliworksx08
8ad131bb3b
refactor: migrate session.query to select API in file service (#34852) 2026-04-09 14:15:59 +00:00
wangxiaolei
41eeb1f2e7
fix: fix sqlalchemy.orm.exc.DetachedInstanceError (#34845) 2026-04-09 10:55:48 +00:00
Yansong Zhang
e50c36526e fix(api): fix transparent upgrade SSE channel mismatch and chat mode routing
- workflow_execute_task: add AppMode.CHAT/AGENT_CHAT/COMPLETION to the
  AdvancedChatAppGenerator routing branch so transparently upgraded old
  apps can execute through the workflow engine.
- app_generate_service: use app_model.mode (not hardcoded AppMode.AGENT)
  for SSE event subscription channel, ensuring the subscriber and
  Celery publisher use the same Redis channel key.

Made-with: Cursor
2026-04-09 17:27:41 +08:00
-LAN-
d1e33ba9ea
refactor(api): reduce Dify GraphInitParams usage (#34825) 2026-04-09 07:59:15 +00:00
Yansong Zhang
e2e16772a1 fix(api): fix DSL import, memory loading, and remaining test coverage
1. DSL Import fix: change self._session.commit() to self._session.flush()
   in app_dsl_service.py _create_or_update_app() to avoid "closed transaction"
   error. DSL import now works: export agent app -> import -> new app created.

2. Memory loading attempt: added _load_memory_messages() to AgentV2Node
   that loads TokenBufferMemory from conversation history. However, chatflow
   engine manages conversations differently from easy-UI (conversation may
   not be in DB at query time, or uses ConversationVariablePersistenceLayer
   instead of Message table). Memory needs further investigation.

Test results:
- Multi-turn memory: Turn 1 OK, Turn 2 LLM doesn't see history (needs deeper fix)
- Service API with API Key: PASSED (answer="Sixteen" for 8+8)
- DSL Import: PASSED (status=completed, new app created)
- Token aggregation: PASSED (node=49, workflow=49)

Known: memory in multi-turn chatflow needs to use graphon's built-in
memory mechanism (MemoryConfig on node + ConversationVariablePersistenceLayer)
rather than direct DB query.

Made-with: Cursor
2026-04-09 14:47:55 +08:00
aliworksx08
4c05316a7b
refactor(api): deduplicate DSL shared entities into dsl_entities.py (#34762)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-09 06:04:18 +00:00
carlos4s
66e588c8ca
refactor(api): use sessionmaker in builtin tools manage service (#34812) 2026-04-09 05:58:38 +00:00
aliworksx08
ee789db443
refactor: migrate session.query to select API in plugin services (#34817)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-09 05:49:59 +00:00
dataCenter430
5f53748d07
refactor: convert ToolProviderType if/elif to match/case (#30001) (#34794) 2026-04-09 05:48:40 +00:00
carlos4s
a76a8876d1
refactor(api): use sessionmaker in datasource provider service (#34811) 2026-04-09 05:43:13 +00:00
carlos4s
be1f4b34f8
refactor(api): use sessionmaker in workflow & RAG pipeline services (#34805) 2026-04-09 05:42:39 +00:00
Yansong Zhang
482a004efe fix(api): fix duplicate answer and completion app upgrade issues
1. Remove StreamChunkEvent from AgentV2Node._run_without_tools():
   The agent-v2 node was yielding StreamChunkEvent during LLM streaming,
   AND the downstream answer node was outputting the same text via
   {{#agent.text#}} variable reference, causing "FourFour" duplication.
   Now text only flows through outputs.text -> answer node (single path).

2. Map inputs to query for completion app transparent upgrade:
   Completion apps send {inputs: {query: "..."}} not {query: "..."}.
   VirtualWorkflowSynthesizer route now extracts query from inputs
   when the top-level query is missing.

Verified:
- Old chat app: "What is 2+2?" -> "Four" (was "FourFour")
- Old completion app: {inputs: {query: "What is 3+3?"}} -> "3 + 3 = 6" (was failing)
- Old agent-chat app: still works

Made-with: Cursor
2026-04-09 12:02:43 +08:00
Yansong Zhang
7052257c8d fix(api): use lazy workflow persistence for transparent upgrade of old apps
VirtualWorkflowSynthesizer.ensure_workflow() creates a real draft
workflow on first call for a legacy app, persisting it to the database.
On subsequent calls, returns the existing draft.

This is needed because AdvancedChatAppGenerator's worker thread looks
up workflows from the database by ID. Instead of hacking the generator
to skip DB lookups, we treat this as a lazy one-time upgrade: the old
app gets a real workflow that can also be edited in the workflow editor.

Verified: old chat app created on main branch ("What is 2+2?" -> "Four")
and old agent-chat app ("Say hello" -> "Hello!") both successfully
execute through the Agent V2 engine with AGENT_V2_TRANSPARENT_UPGRADE=true.

Made-with: Cursor
2026-04-09 11:28:16 +08:00
Yansong Zhang
edfcab6455 fix(api): add AGENT mode to app list filtering
Add AppMode.AGENT branch in get_paginate_apps() so that
filtering apps by mode=agent works correctly.
Discovered during comprehensive E2E testing.

14/14 E2E tests pass covering:
- A: New Agent app full lifecycle (create, draft, configs, publish, run)
- B: Old app creation compat (chat, completion, agent-chat, advanced-chat, workflow)
- C: App listing and filtering (all modes, agent filter)
- D: Workflow editor compat (block configs)
- E: DSL export

Made-with: Cursor
2026-04-09 10:54:05 +08:00
Yansong Zhang
66212e3575 feat(api): implement zero-migration transparent upgrade (Phase 8)
Add two feature-flag-controlled upgrade paths that allow existing apps
and LLM nodes to transparently run through the Agent V2 engine without
any database migration:

1. AGENT_V2_TRANSPARENT_UPGRADE (default: off):
   When enabled, old apps (chat/completion/agent-chat) bypass legacy
   Easy-UI runners. VirtualWorkflowSynthesizer converts AppModelConfig
   to an in-memory Workflow (start -> agent-v2 -> answer) at runtime,
   then executes via AdvancedChatAppGenerator. Falls back to legacy
   path on any synthesis error.

   VirtualWorkflowSynthesizer maps:
   - model JSON -> ModelConfig
   - pre_prompt/chat_prompt_config -> prompt_template
   - agent_mode.tools -> ToolMetadata[]
   - agent_mode.strategy -> agent_strategy
   - dataset_configs -> context
   - file_upload -> vision

2. AGENT_V2_REPLACES_LLM (default: off):
   When enabled, DifyNodeFactory.create_node() transparently remaps
   nodes with type="llm" to type="agent-v2" before class resolution.
   Since AgentV2NodeData is a strict superset of LLMNodeData, the
   mapping is lossless. With tools=[], Agent V2 behaves identically
   to LLM Node.

Both flags default to False for safety. Turn off = instant rollback.
46 existing tests pass. Flask starts successfully.

Made-with: Cursor
2026-04-09 10:30:52 +08:00
Yansong Zhang
44491e427c feat(api): enable all sandbox/skill controller routes and resolve dependencies (P0)
Resolve the full dependency chain to enable all previously disabled controllers:

Enabled routes:
- sandbox_files: sandbox file browser API
- sandbox_providers: sandbox provider management API
- app_asset: app asset management API
- skills: skill extraction API
- CLI API blueprint: DifyCli callback endpoints (/cli/api/*)

Dependencies extracted (64 files, ~8000 lines):
- models/sandbox.py, models/app_asset.py: DB models
- core/zip_sandbox/: zip-based sandbox execution
- core/session/: CLI API session management
- core/memory/: base memory + node token buffer
- core/helper/creators.py: helper utilities
- core/llm_generator/: context models, output models, utils
- core/workflow/nodes/command/: command node type
- core/workflow/nodes/file_upload/: file upload node type
- core/app/entities/: app_asset_entities, app_bundle_entities, llm_generation_entities
- services/: asset_content, skill, workflow_collaboration, workflow_comment
- controllers/console/app/error.py: AppAsset error classes
- core/tools/utils/system_encryption.py

Import fixes:
- dify_graph.enums -> graphon.enums in skill_service.py
- get_signed_file_url_for_plugin -> get_signed_file_url in cli_api.py

All 5 controllers verified: import OK, Flask starts successfully.
46 existing tests still pass.

Made-with: Cursor
2026-04-09 09:36:16 +08:00
dataCenter430
9c4f897b9a
refactor: convert segmentType if/elif to match/case in webhook_service.py (#30001) (#34770) 2026-04-09 00:36:28 +00:00
Jake Armstrong
8f46c9113c
refactor(api): deduplicate ImportMode and ImportStatus enums from rag_pipeline_dsl_service (#34759) 2026-04-08 23:23:04 +00:00
carlos4s
5aa4e23f54
refactor(api): use sessionmaker in end user, retention & cleanup services (#34765) 2026-04-08 23:21:28 +00:00
carlos4s
1d971d3240
refactor(api): use sessionmaker in plugin & trigger services (#34764) 2026-04-08 23:18:26 +00:00
carlos4s
4c70bfa8b8
refactor(api): use sessionmaker in trigger provider service & dataset… (#34774) 2026-04-08 23:13:38 +00:00