The frontend and backend implementation for the human input node.
Co-authored-by: twwu <twwu@dify.ai>
Co-authored-by: JzoNg <jzongcode@gmail.com>
Co-authored-by: yyh <92089059+lyzno1@users.noreply.github.com>
Co-authored-by: zhsama <torvalds@linux.do>
feat: hide output tools and improve JSON formatting for structured output
feat: hide output tools and improve JSON formatting for structured output
fix: handle prompt template correctly to extract selectors for step run
fix: emit StreamChunkEvent correctly for sandbox agent
chore: better debug message
fix: incorrect output tool runtime selection
fix: type issues
fix: align parameter list
fix: align parameter list
fix: hide internal builtin providers from tool list
vibe: implement file structured output
vibe: implement file structured output
fix: refix parameter for tool
fix: crash
fix: crash
refactor: remove union types
fix: type check
Merge branch 'feat/structured-output-with-sandbox' into feat/support-agent-sandbox
fix: provide json as text
fix: provide json as text
fix: get AgentResult correctly
fix: provides correct prompts, tools and terminal predicates
fix: provides correct prompts, tools and terminal predicates
fix: circular import
feat: support structured output in sandbox and tool mode
Added support for an optional `download_filename` parameter in the `get_download_url` and `get_download_urls` methods across various storage classes. This allows users to specify a custom filename for downloads, improving user experience by enabling better file naming during downloads. Updated related methods and tests to accommodate this new functionality.
- Modified the SandboxService and related app generators to replace workflow_execution_id with sandbox_id for improved clarity and consistency in sandbox handling.
- Adjusted the AdvancedChatAppGenerator and WorkflowAppGenerator to align with the new parameter naming convention.
- Updated API routes to use app_id instead of sandbox_id for file operations, aligning with user-specific sandbox workspaces.
- Enhanced SandboxFileService and related classes to accommodate app_id in file listing and download functionalities.
- Refactored storage key generation for sandbox archives to include app_id, ensuring proper file organization.
- Adjusted frontend contracts and services to reflect the new app_id parameter in API calls.
- Implemented the `exists` method in `SandboxFileSource` and its subclasses to verify the availability of sandbox sources.
- Updated `SandboxFileService` to utilize the new `exists` method for improved error handling when listing files and downloading files.
- Removed the previous check for storage existence in `archive_source.py` and replaced it with the new method.
The backend part of the human in the loop (HITL) feature and relevant architecture / workflow engine changes.
Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: 盐粒 Yanli <yanli@dify.ai>
Co-authored-by: CrabSAMA <40541269+CrabSAMA@users.noreply.github.com>
Co-authored-by: Stephen Zhou <38493346+hyoban@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: yihong <zouzou0208@gmail.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
- Replaced direct storage references with SilentStorage in various components to enhance fallback mechanisms.
- Updated storage key formats for sandbox archives and files to improve clarity and consistency.
- Refactored related classes and methods to utilize the new SandboxFilePath structure.
- Adjusted unit tests to reflect changes in the StorageTicket model and its serialization methods.
- Removed the storage proxy controller and its associated endpoints for file download and upload.
- Updated the file controller to use the new storage ticket service for generating download and upload URLs.
- Modified the file presign storage to fallback to ticket-based URLs instead of signed proxy URLs.
- Enhanced unit tests to validate the new ticket generation and retrieval logic.
- Removed unused app asset download and upload endpoints, along with sandbox archive and file download endpoints.
- Updated imports in the file controller to reflect the removal of these endpoints.
- Simplified the generator.py file by consolidating the code context field definition.
- Enhanced the storage layer with a unified presign wrapper for better handling of presigned URLs.
- Add BundleManifest with dsl_filename for 100% tree ID restoration
- Implement two-step import flow: prepare (get upload URL) + confirm
- Use sandbox for zip extraction and file upload via presigned URLs
- Store import session in Redis with 1h TTL
- Add SandboxUploadItem for symmetric download/upload API
- Remove legacy source_zip_extractor, inline logic in service
- Update frontend to use new prepare/confirm API flow
- Added a threaded function to handle the deletion of storage files asynchronously after asset removal.
- Updated the asset removal logic to include a call to the new deletion function, improving performance and responsiveness during asset management operations.
- Updated storage wrappers to utilize a new base class, StorageWrapper, for better delegation of methods.
- Introduced SilentStorage to handle read operations gracefully by returning empty values instead of raising exceptions.
- Enhanced CachedPresignStorage to support batch caching of download URLs, improving performance.
- Refactored FilePresignStorage to support both presigned URLs and signed proxy URLs for downloads.
- Updated AppAssetService to utilize the new storage structure, ensuring consistent asset management.
- Introduced CachedPresignStorage to cache presigned download URLs, reducing repeated API calls.
- Updated AppAssetService to utilize CachedPresignStorage for improved performance in asset download URL generation.
- Refactored asset builders and packagers to support the new storage mechanism.
- Removed unused AppAssetsAttrsInitializer to streamline initialization processes.
- Added unit tests for CachedPresignStorage to ensure functionality and reliability.
- Deleted `AppAssetFileResource` class and its associated file upload logic.
- Removed the `create_file` method from `AppAssetService` to streamline asset management.
- Updated `AppAssetBatchUploadResource` for improved readability by condensing method calls.
- Introduced `GetUploadUrlPayload` and `BatchUploadPayload` models for handling file uploads.
- Implemented `AppAssetFileUploadUrlResource` for generating pre-signed upload URLs.
- Added `AppAssetBatchUploadResource` to support batch creation of asset nodes from a tree structure.
- Enhanced `AppAssetService` with methods for obtaining upload URLs and batch creation of assets.
- Removed checksum handling from file creation to streamline the process.
- Updated the `get_system_default_config` method to accept a `provider_type` parameter for more precise querying.
- Improved error handling to raise a ValueError if no system default provider is configured for the specified tenant and provider type.
- Added fallback logic to ensure a system default configuration is returned when available.
- Introduced AppBundleService for managing app bundle publishing and importing, integrating workflow and asset services.
- Added methods for exporting app bundles as ZIP files, including DSL and asset management.
- Implemented source zip extraction and validation to enhance asset import processes.
- Refactored asset packaging to utilize AssetZipPackager for improved performance and organization.
- Enhanced error handling for bundle format and security during import operations.
- Added AttrMap and AttrKey classes for type-safe attribute storage.
- Implemented AppAssetsAttrs and SkillAttrs for managing application and skill attributes.
- Refactored Sandbox and initializers to utilize the new attribute management system, enhancing modularity and clarity in asset handling.
- Moved sandbox-related classes and functions into a dedicated module for better organization.
- Updated the sandbox initialization process to streamline asset management and environment setup.
- Removed deprecated constants and refactored related code to utilize new sandbox entities.
- Enhanced the workflow context to support sandbox integration, allowing for improved state management during execution.
- Adjusted various components to utilize the new sandbox structure, ensuring compatibility across the application.
- Introduced DraftAppAssetsInitializer for handling draft assets.
- Updated SandboxLayer to conditionally set sandbox ID and storage based on workflow version.
- Improved asset initialization logging and error handling.
- Refactored ArchiveSandboxStorage to support exclusion patterns during archiving.
- Modified command and LLM nodes to retrieve sandbox from workflow context, supporting draft workflows.
- Enhanced the SandboxProviderConfigApi to accept an 'activate' argument when saving provider configurations.
- Introduced a new request parser for activating sandbox providers, requiring a 'type' argument.
- Updated the SandboxProviderService to handle the activation state during configuration saving and provider activation.
- Add AppAssetsInitializer to load published app assets into sandbox
- Refactor VMFactory.create() to VMBuilder with builder pattern
- Extract SandboxInitializer base class and DifyCliInitializer
- Simplify SandboxLayer constructor (remove options/environments params)
- Fix circular import in sandbox module by removing eager SandboxBashTool export
- Update SandboxProviderService to return VMBuilder instead of VirtualEnvironment
- Add helpers.py with connection management utilities:
- with_connection: context manager for connection lifecycle
- submit_command: execute command and return CommandFuture
- execute: run command with auto connection, raise on failure
- try_execute: run command with auto connection, return result
- Add CommandExecutionError to exec.py for typed error handling
with access to exit_code, stderr, and full result
- Remove run_command method from VirtualEnvironment base class
(now available as submit_command helper)
- Update all call sites to use new helper functions:
- sandbox/session.py
- sandbox/storage/archive_storage.py
- sandbox/bash/bash_tool.py
- workflow/nodes/command/node.py
- Add comprehensive unit tests for helpers with connection reuse
