Commit Graph

10150 Commits

Author SHA1 Message Date
Yansong Zhang
e2e16772a1 fix(api): fix DSL import, memory loading, and remaining test coverage
1. DSL Import fix: change self._session.commit() to self._session.flush()
   in app_dsl_service.py _create_or_update_app() to avoid "closed transaction"
   error. DSL import now works: export agent app -> import -> new app created.

2. Memory loading attempt: added _load_memory_messages() to AgentV2Node
   that loads TokenBufferMemory from conversation history. However, chatflow
   engine manages conversations differently from easy-UI (conversation may
   not be in DB at query time, or uses ConversationVariablePersistenceLayer
   instead of Message table). Memory needs further investigation.

Test results:
- Multi-turn memory: Turn 1 OK, Turn 2 LLM doesn't see history (needs deeper fix)
- Service API with API Key: PASSED (answer="Sixteen" for 8+8)
- DSL Import: PASSED (status=completed, new app created)
- Token aggregation: PASSED (node=49, workflow=49)

Known: memory in multi-turn chatflow needs to use graphon's built-in
memory mechanism (MemoryConfig on node + ConversationVariablePersistenceLayer)
rather than direct DB query.

Made-with: Cursor
2026-04-09 14:47:55 +08:00
-LAN-
ec56f4e839
fix(docker): restore S3_ADDRESS_STYLE env examples (#34826) 2026-04-09 06:44:28 +00:00
Jonathan Chang
d5ababfed0
refactor(api): deduplicate json serialization in AppModelConfig.from_model_config_dict (#34795) 2026-04-09 06:14:48 +00:00
yyh
8225f98565
fix(web): use nuqs for log conversation url state (#34820) 2026-04-09 06:09:27 +00:00
aliworksx08
4c05316a7b
refactor(api): deduplicate DSL shared entities into dsl_entities.py (#34762)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-09 06:04:18 +00:00
carlos4s
66e588c8ca
refactor(api): use sessionmaker in builtin tools manage service (#34812) 2026-04-09 05:58:38 +00:00
Renzo
9a51c2f56a
refactor: migrate session.query to select API in deal dataset vector index task (#34819) 2026-04-09 05:50:59 +00:00
aliworksx08
ee789db443
refactor: migrate session.query to select API in plugin services (#34817)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-09 05:49:59 +00:00
carlos4s
d360929af1
refactor(api): use sessionmaker in pgvecto_rs VDB service (#34818) 2026-04-09 05:49:03 +00:00
dataCenter430
5f53748d07
refactor: convert ToolProviderType if/elif to match/case (#30001) (#34794) 2026-04-09 05:48:40 +00:00
Renzo
e3cc4b83c8
refactor: migrate session.query to select API in clean dataset task (#34815) 2026-04-09 05:46:36 +00:00
aliworksx08
b5acc8e392
refactor: migrate session.query to select API in core tools (#34814)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-09 05:44:49 +00:00
Renzo
f5ea61e93e
refactor: migrate session.query to select API in document indexing sync task (#34813) 2026-04-09 05:44:13 +00:00
carlos4s
a76a8876d1
refactor(api): use sessionmaker in datasource provider service (#34811) 2026-04-09 05:43:13 +00:00
carlos4s
be1f4b34f8
refactor(api): use sessionmaker in workflow & RAG pipeline services (#34805) 2026-04-09 05:42:39 +00:00
Yansong Zhang
b21a443d56 fix(api): resolve all remaining known issues
1. Fix workflow-level total_tokens=0:
   Call graph_runtime_state.add_tokens(usage.total_tokens) in both
   _run_without_tools and _run_with_tools paths after node execution.
   Previously only graphon's internal ModelInvokeCompletedEvent handler
   called add_tokens, which agent-v2 doesn't emit.

2. Fix Turn 2 SSE empty response:
   Set PUBSUB_REDIS_CHANNEL_TYPE=streams in .env. Redis Streams
   provides durable event delivery (consumers can replay past events),
   solving the pub/sub at-most-once timing issue.

3. Skill -> Agent runtime integration:
   SandboxBuilder.build() now auto-includes SkillInitializer if not
   already present. This ensures sandbox.attrs has the skill bundle
   loaded for downstream consumers (tool execution in sandbox).

4. LegacyResponseAdapter:
   New module at core/app/apps/common/legacy_response_adapter.py.
   Filters workflow-specific SSE events (workflow_started, node_started,
   node_finished, workflow_finished) from the stream, passing through
   only message/message_end/agent_log/error/ping events that old
   clients expect.

46 unit tests pass.

Made-with: Cursor
2026-04-09 12:53:11 +08:00
Yansong Zhang
4f010cd4f5 fix(api): stop emitting StreamChunkEvent from tool path to prevent answer duplication
The EventAdapter was converting every LLMResultChunk from the agent
strategy into StreamChunkEvent. Combined with the answer node's
{{#agent.text#}} variable output, this caused the final answer to
appear twice (e.g., "It is 2026-04-09 04:27:45.It is 2026-04-09 04:27:45.").

Now LLMResultChunk from strategy output is silently consumed (text still
accumulates in AgentResult.text via the strategy). Only AgentLogEvent
(thought/tool_call/round) is forwarded to the pipeline.

Known remaining issues:
- workflow/message level total_tokens=0 (node level is correct at 33)
  because pipeline aggregation doesn't include agent-v2 node tokens
- Turn 2 SSE delivery timing with Redis pubsub (celery executes OK)

Made-with: Cursor
2026-04-09 12:31:49 +08:00
Yansong Zhang
3d4be88d97 fix(api): remove unsupported 'user' param from FC/ReAct invoke_llm calls
FunctionCallStrategy and ReActStrategy were passing user=self.context.user_id
to ModelInstance.invoke_llm() which doesn't accept that parameter.
This caused tool-using agent runs to fail with:
  "ModelInstance.invoke_llm() got an unexpected keyword argument 'user'"

Verified: Agent V2 with current_time tool now works end-to-end:
  ROUND 1: LLM thought -> CALL current_time -> got time
  ROUND 2: LLM generates answer with time info
Made-with: Cursor
2026-04-09 12:18:07 +08:00
Jake Armstrong
c19a822e1b
refactor: deduplicate DefaultRetrievalModelDict TypedDict into retrieval_service.py (#34758) 2026-04-09 04:13:04 +00:00
Yansong Zhang
482a004efe fix(api): fix duplicate answer and completion app upgrade issues
1. Remove StreamChunkEvent from AgentV2Node._run_without_tools():
   The agent-v2 node was yielding StreamChunkEvent during LLM streaming,
   AND the downstream answer node was outputting the same text via
   {{#agent.text#}} variable reference, causing "FourFour" duplication.
   Now text only flows through outputs.text -> answer node (single path).

2. Map inputs to query for completion app transparent upgrade:
   Completion apps send {inputs: {query: "..."}} not {query: "..."}.
   VirtualWorkflowSynthesizer route now extracts query from inputs
   when the top-level query is missing.

Verified:
- Old chat app: "What is 2+2?" -> "Four" (was "FourFour")
- Old completion app: {inputs: {query: "What is 3+3?"}} -> "3 + 3 = 6" (was failing)
- Old agent-chat app: still works

Made-with: Cursor
2026-04-09 12:02:43 +08:00
dataCenter430
8782787a9e
refactor: convert TelemetryCase if/elif to match/case (#3001) (#34797) 2026-04-09 03:40:07 +00:00
lif
4c6b8f9229
test: add e2e scenarios for app creation and sign-out (#34285)
Signed-off-by: majiayu000 <1835304752@qq.com>
2026-04-09 03:31:13 +00:00
Yansong Zhang
7052257c8d fix(api): use lazy workflow persistence for transparent upgrade of old apps
VirtualWorkflowSynthesizer.ensure_workflow() creates a real draft
workflow on first call for a legacy app, persisting it to the database.
On subsequent calls, returns the existing draft.

This is needed because AdvancedChatAppGenerator's worker thread looks
up workflows from the database by ID. Instead of hacking the generator
to skip DB lookups, we treat this as a lazy one-time upgrade: the old
app gets a real workflow that can also be edited in the workflow editor.

Verified: old chat app created on main branch ("What is 2+2?" -> "Four")
and old agent-chat app ("Say hello" -> "Hello!") both successfully
execute through the Agent V2 engine with AGENT_V2_TRANSPARENT_UPGRADE=true.

Made-with: Cursor
2026-04-09 11:28:16 +08:00
dependabot[bot]
51dcf4ce84
chore(deps): bump litellm from 1.82.6 to 1.83.0 in /api (#34544)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-09 03:27:21 +00:00
wangxiaolei
27e484e7f8
feat: redis add retry logic (#34566)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-09 03:08:25 +00:00
Yansong Zhang
edfcab6455 fix(api): add AGENT mode to app list filtering
Add AppMode.AGENT branch in get_paginate_apps() so that
filtering apps by mode=agent works correctly.
Discovered during comprehensive E2E testing.

14/14 E2E tests pass covering:
- A: New Agent app full lifecycle (create, draft, configs, publish, run)
- B: Old app creation compat (chat, completion, agent-chat, advanced-chat, workflow)
- C: App listing and filtering (all modes, agent filter)
- D: Workflow editor compat (block configs)
- E: DSL export

Made-with: Cursor
2026-04-09 10:54:05 +08:00
BrianWang1990
9308287fea
fix: copy button not working on API Server and API Key pages (#34515)
Co-authored-by: Brian Wang <BrianWang1990@users.noreply.github.com>
Co-authored-by: test <test@testdeMac-mini.local>
Co-authored-by: BrianWang1990 <512dabing99@163.com>
Co-authored-by: Stephen Zhou <hi@hyoban.cc>
Co-authored-by: Stephen Zhou <38493346+hyoban@users.noreply.github.com>
2026-04-09 02:49:40 +00:00
Yansong Zhang
66212e3575 feat(api): implement zero-migration transparent upgrade (Phase 8)
Add two feature-flag-controlled upgrade paths that allow existing apps
and LLM nodes to transparently run through the Agent V2 engine without
any database migration:

1. AGENT_V2_TRANSPARENT_UPGRADE (default: off):
   When enabled, old apps (chat/completion/agent-chat) bypass legacy
   Easy-UI runners. VirtualWorkflowSynthesizer converts AppModelConfig
   to an in-memory Workflow (start -> agent-v2 -> answer) at runtime,
   then executes via AdvancedChatAppGenerator. Falls back to legacy
   path on any synthesis error.

   VirtualWorkflowSynthesizer maps:
   - model JSON -> ModelConfig
   - pre_prompt/chat_prompt_config -> prompt_template
   - agent_mode.tools -> ToolMetadata[]
   - agent_mode.strategy -> agent_strategy
   - dataset_configs -> context
   - file_upload -> vision

2. AGENT_V2_REPLACES_LLM (default: off):
   When enabled, DifyNodeFactory.create_node() transparently remaps
   nodes with type="llm" to type="agent-v2" before class resolution.
   Since AgentV2NodeData is a strict superset of LLMNodeData, the
   mapping is lossless. With tools=[], Agent V2 behaves identically
   to LLM Node.

Both flags default to False for safety. Turn off = instant rollback.
46 existing tests pass. Flask starts successfully.

Made-with: Cursor
2026-04-09 10:30:52 +08:00
dataCenter430
7ca5b726a2
refactor: convert ProviderQuota if/elif to match/case (#30001) (#34791) 2026-04-09 02:28:19 +00:00
dataCenter430
0bdd1267fb
refactor: convert appmode plugin if/elif to match/case (#30001) (#34790) 2026-04-09 02:28:03 +00:00
dataCenter430
3ea88dfc7f
refactor: convert appMode controllers if/elif to match/case (#30001) (#34789) 2026-04-09 02:27:19 +00:00
dataCenter430
2275c5b1a3
refactor: convert file-transfer-method-pipeline if/elif to match/case (#30001) (#34788)
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
2026-04-09 01:43:52 +00:00
Yansong Zhang
96374d7f6a refactor(api): replace legacy agent runners with StrategyFactory in AgentChatAppRunner (Phase 4)
Replace the hardcoded FunctionCallAgentRunner / CotChatAgentRunner /
CotCompletionAgentRunner selection in AgentChatAppRunner with the new
AgentAppRunner class that uses StrategyFactory from Phase 1.

Before: AgentChatAppRunner manually selects FC/CoT runner class based on
model features and LLM mode, then instantiates it directly.

After: AgentChatAppRunner instantiates AgentAppRunner (from sandbox branch),
which internally uses StrategyFactory.create_strategy() to auto-select
the right strategy, and uses ToolInvokeHook for proper agent_invoke
with file handling and thought persistence.

This unifies the agent execution engine: both the new Agent V2 workflow
node and the legacy agent-chat app now use the same StrategyFactory
and AgentPattern implementations.

Also fix: command and file_upload nodes use string node_type instead of
BuiltinNodeTypes.COMMAND/FILE_UPLOAD (not in current graphon version).

46 tests pass. Flask starts successfully.

Made-with: Cursor
2026-04-09 09:42:23 +08:00
Yansong Zhang
44491e427c feat(api): enable all sandbox/skill controller routes and resolve dependencies (P0)
Resolve the full dependency chain to enable all previously disabled controllers:

Enabled routes:
- sandbox_files: sandbox file browser API
- sandbox_providers: sandbox provider management API
- app_asset: app asset management API
- skills: skill extraction API
- CLI API blueprint: DifyCli callback endpoints (/cli/api/*)

Dependencies extracted (64 files, ~8000 lines):
- models/sandbox.py, models/app_asset.py: DB models
- core/zip_sandbox/: zip-based sandbox execution
- core/session/: CLI API session management
- core/memory/: base memory + node token buffer
- core/helper/creators.py: helper utilities
- core/llm_generator/: context models, output models, utils
- core/workflow/nodes/command/: command node type
- core/workflow/nodes/file_upload/: file upload node type
- core/app/entities/: app_asset_entities, app_bundle_entities, llm_generation_entities
- services/: asset_content, skill, workflow_collaboration, workflow_comment
- controllers/console/app/error.py: AppAsset error classes
- core/tools/utils/system_encryption.py

Import fixes:
- dify_graph.enums -> graphon.enums in skill_service.py
- get_signed_file_url_for_plugin -> get_signed_file_url in cli_api.py

All 5 controllers verified: import OK, Flask starts successfully.
46 existing tests still pass.

Made-with: Cursor
2026-04-09 09:36:16 +08:00
dataCenter430
1c7cf44af4
refactor: convert SegmentType controllers if/elif to match/case (#30001) (#34784) 2026-04-09 01:11:47 +00:00
dataCenter430
3325392cc5
refactor: convert segmentType workflow if/elif to match/case (#34785) 2026-04-09 00:51:43 +00:00
dataCenter430
fd2843b0fb
refactor: convert file-transfer-method-tools if/elif to match/case (#30001) (#34783) 2026-04-09 00:42:13 +00:00
volcano303
1898a3f8a5
test: migrate recommended_app_service tests to testcontainers (#34751)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-09 00:36:57 +00:00
dataCenter430
9c4f897b9a
refactor: convert segmentType if/elif to match/case in webhook_service.py (#30001) (#34770) 2026-04-09 00:36:28 +00:00
dataCenter430
47b9d48f70
refactor: convert ToolProviderType if/elif to match/case (#30001) (#34768)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
2026-04-09 00:17:22 +00:00
dataCenter430
ce68f2cdc6
refactor: convert webapp auth type if/elif to match/case (#30001) (#34782) 2026-04-09 00:16:44 +00:00
dataCenter430
a8fa552b3a
refactor: convert importStatus if/elif to match/case (#30001) (#34780) 2026-04-09 00:04:47 +00:00
Jake Armstrong
bd257777a0
refactor(api): deduplicate workflow controller schemas into controller_schemas.py (#34755) 2026-04-08 23:49:04 +00:00
dataCenter430
e6715a2dbe
refactor: convert FileTransferMethod if/elif to match/case (#30001) (#34769) 2026-04-08 23:27:10 +00:00
Jake Armstrong
8f46c9113c
refactor(api): deduplicate ImportMode and ImportStatus enums from rag_pipeline_dsl_service (#34759) 2026-04-08 23:23:04 +00:00
carlos4s
5aa4e23f54
refactor(api): use sessionmaker in end user, retention & cleanup services (#34765) 2026-04-08 23:21:28 +00:00
Renzo
5821511114
refactor: migrate session.query to select API in batch clean and disable segments tasks (#34760) 2026-04-08 23:20:25 +00:00
Renzo
d6d9b04c41
refactor: migrate session.query to select API in add document and clean document tasks (#34761)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2026-04-08 23:19:36 +00:00
Renzo
540289e6c6
refactor: migrate session.query to select API in delete segment and regenerate summary tasks (#34763) 2026-04-08 23:19:03 +00:00
carlos4s
1d971d3240
refactor(api): use sessionmaker in plugin & trigger services (#34764) 2026-04-08 23:18:26 +00:00