- Added sandbox archive upload and download proxy endpoints with signed URL verification.
- Introduced security helpers for generating and verifying signed URLs.
- Updated file-related API routes to include sandbox archive functionality.
- Refactored app asset storage methods to streamline download/upload URL generation.
- Updated the download script to redirect error output to /dev/null, preventing unnecessary error messages from being displayed.
- Added an explicit exit command to ensure the script terminates correctly after execution.
- Updated storage wrappers to utilize a new base class, StorageWrapper, for better delegation of methods.
- Introduced SilentStorage to handle read operations gracefully by returning empty values instead of raising exceptions.
- Enhanced CachedPresignStorage to support batch caching of download URLs, improving performance.
- Refactored FilePresignStorage to support both presigned URLs and signed proxy URLs for downloads.
- Updated AppAssetService to utilize the new storage structure, ensuring consistent asset management.
- Introduced CachedPresignStorage to cache presigned download URLs, reducing repeated API calls.
- Updated AppAssetService to utilize CachedPresignStorage for improved performance in asset download URL generation.
- Refactored asset builders and packagers to support the new storage mechanism.
- Removed unused AppAssetsAttrsInitializer to streamline initialization processes.
- Added unit tests for CachedPresignStorage to ensure functionality and reliability.
- Introduced `GetUploadUrlPayload` and `BatchUploadPayload` models for handling file uploads.
- Implemented `AppAssetFileUploadUrlResource` for generating pre-signed upload URLs.
- Added `AppAssetBatchUploadResource` to support batch creation of asset nodes from a tree structure.
- Enhanced `AppAssetService` with methods for obtaining upload URLs and batch creation of assets.
- Removed checksum handling from file creation to streamline the process.
- Updated the SandboxManager to rename the method for deleting storage to better reflect its purpose.
- Adjusted the WorkflowVariableCollectionApi to utilize the new method name.
- Improved error handling in ArchiveSandboxStorage's delete method to log exceptions during deletion.
- Renamed `build_skill_artifact_set` to `build_skill_bundle` for improved clarity in asset management.
- Updated references in `SkillManager` to reflect the new method name and ensure consistent handling of skill bundles.
- Added `AppAssetsAttrsInitializer` to `SandboxManager` to enhance asset initialization processes.
- Implemented output truncation in `SandboxBashTool` to manage long command outputs effectively.
- Introduced AppBundleService for managing app bundle publishing and importing, integrating workflow and asset services.
- Added methods for exporting app bundles as ZIP files, including DSL and asset management.
- Implemented source zip extraction and validation to enhance asset import processes.
- Refactored asset packaging to utilize AssetZipPackager for improved performance and organization.
- Enhanced error handling for bundle format and security during import operations.
- Replaced SkillArtifactSet with SkillBundle across various components, enhancing the organization of skill dependencies and references.
- Updated SkillManager methods to load and save bundles instead of artifacts, improving clarity in asset management.
- Refactored SkillCompiler to compile skills into bundles, streamlining the dependency resolution process.
- Adjusted DifyCli and SandboxBashSession to utilize ToolDependencies, ensuring consistent handling of tool references.
- Introduced AssetReferences for better management of file dependencies within skill bundles.
- Updated binary files for Dify CLI on various platforms (darwin amd64, darwin arm64, linux amd64, linux arm64).
- Refactored skill compilation in LLMNode to improve clarity and maintainability by explicitly naming parameters and incorporating AppAssets for base path management.
- Minor fix in AppAssetFileTree to remove unnecessary leading slash in path construction.
- Introduced SandboxManager.delete_storage method for improved storage management.
- Refactored skill loading and tool artifact handling in DifyCliInitializer and SandboxBashSession.
- Updated LLMNode to extract and compile tool artifacts, enhancing integration with skills.
- Improved attribute management in AttrMap for better error handling and retrieval methods.
- Added AttrMap and AttrKey classes for type-safe attribute storage.
- Implemented AppAssetsAttrs and SkillAttrs for managing application and skill attributes.
- Refactored Sandbox and initializers to utilize the new attribute management system, enhancing modularity and clarity in asset handling.
- Introduced threading for loading skills and uploading compiled content to improve performance.
- Added data classes for better structure and clarity in handling loaded and compiled skills.
- Refactored the skill compilation process to separate loading and uploading, enhancing maintainability.
- Changed the command timeout duration from 60 seconds to 1 hour for improved session stability.
- Refactored session cleanup logic to utilize the CLI API session object instead of session ID, enhancing clarity and maintainability.
- Added a keepalive thread to maintain the E2B sandbox timeout, preventing premature termination.
- Introduced a stop event to manage the lifecycle of the keepalive thread.
- Refactored the sandbox initialization to include the new keepalive functionality.
- Enhanced logging to capture failures in refreshing the sandbox timeout.
- Moved sandbox-related classes and functions into a dedicated module for better organization.
- Updated the sandbox initialization process to streamline asset management and environment setup.
- Removed deprecated constants and refactored related code to utilize new sandbox entities.
- Enhanced the workflow context to support sandbox integration, allowing for improved state management during execution.
- Adjusted various components to utilize the new sandbox structure, ensuring compatibility across the application.
- Introduced DraftAppAssetsInitializer for handling draft assets.
- Updated SandboxLayer to conditionally set sandbox ID and storage based on workflow version.
- Improved asset initialization logging and error handling.
- Refactored ArchiveSandboxStorage to support exclusion patterns during archiving.
- Modified command and LLM nodes to retrieve sandbox from workflow context, supporting draft workflows.
- Added DIFY_CLI_ROOT constant for the root directory of Dify CLI.
- Updated DIFY_CLI_PATH and DIFY_CLI_CONFIG_PATH to use absolute paths.
- Modified app asset initialization to create directories under DIFY_CLI_ROOT.
- Enhanced Docker and E2B environment file handling to use workspace paths.
- Introduce _app_id attribute to store application ID from system variables
- Add _get_app_id method to retrieve and validate app_id
- Update on_graph_start to log app_id during sandbox initialization
- Integrate ArchiveSandboxStorage for persisting and restoring sandbox files
- Ensure proper error handling for sandbox file operations
- Add AppAssetsInitializer to load published app assets into sandbox
- Refactor VMFactory.create() to VMBuilder with builder pattern
- Extract SandboxInitializer base class and DifyCliInitializer
- Simplify SandboxLayer constructor (remove options/environments params)
- Fix circular import in sandbox module by removing eager SandboxBashTool export
- Update SandboxProviderService to return VMBuilder instead of VirtualEnvironment
- Add helpers.py with connection management utilities:
- with_connection: context manager for connection lifecycle
- submit_command: execute command and return CommandFuture
- execute: run command with auto connection, raise on failure
- try_execute: run command with auto connection, return result
- Add CommandExecutionError to exec.py for typed error handling
with access to exit_code, stderr, and full result
- Remove run_command method from VirtualEnvironment base class
(now available as submit_command helper)
- Update all call sites to use new helper functions:
- sandbox/session.py
- sandbox/storage/archive_storage.py
- sandbox/bash/bash_tool.py
- workflow/nodes/command/node.py
- Add comprehensive unit tests for helpers with connection reuse
The original fix seems correct on its own. However, for chatflows with multiple answer nodes, the `message_replace` command only preserves the output of the last executed answer node.
- Enhanced the SandboxManager to use a sharded locking mechanism for improved concurrency and performance.
- Replaced the global lock with shard-specific locks, allowing for lock-free reads and reducing contention.
- Updated methods for registering, retrieving, unregistering, and counting sandboxes to work with the new sharded structure.
- Improved documentation within the class to clarify the purpose and functionality of the sharding approach.
- Simplified the SandboxLayer initialization by removing unused parameters and consolidating sandbox creation logic.
- Integrated SandboxManager for better lifecycle management of sandboxes during workflow execution.
- Updated error handling to ensure proper initialization and cleanup of sandboxes.
- Enhanced CommandNode to retrieve sandboxes from SandboxManager, improving sandbox availability checks.
- Added unit tests to validate the new sandbox management approach and ensure robust error handling.
- Introduced threading to handle Docker's stdout/stderr streams, improving thread safety and preventing race conditions.
- Replaced buffer-based reading with queue-based reading for stdout and stderr.
- Updated read methods to handle errors and end-of-stream conditions more gracefully.
- Enhanced documentation to reflect changes in the demuxing process.
- Improved error message handling by assigning the stderr output to a variable for better readability.
- Ensured consistent error reporting when a command fails, maintaining clarity in the output.
- Updated error handling to provide detailed stderr output when a command fails.
- Streamlined working directory and command rendering by combining operations into single lines.
- Simplified the command execution logic by removing unnecessary shell invocations.
- Enhanced working directory validation by directly using the `test` command.
- Improved command parsing with `shlex.split` for better handling of raw commands.
- Added checks to verify the existence of the specified working directory before executing commands.
- Updated command execution logic to conditionally change the working directory if provided.
- Included FIXME comments to address future enhancements for native cwd support in VirtualEnvironment.run_command.
- Added FIXME comments to indicate the need for unifying runtime config checking in AdvancedChatAppGenerator and WorkflowAppGenerator.
- Introduced sandbox management in WorkflowService with proper error handling for sandbox release.
- Enhanced runtime feature handling in the workflow execution process.
- Add _DockerDemuxer to properly separate stdout/stderr from multiplexed stream
- Fix binary header garbage in Docker exec output (tty=False 8-byte header)
- Fix LocalVirtualEnvironment.get_command_status() to use os.WEXITSTATUS()
- Update tests to use Transport API instead of raw file descriptors
- Introduced Command node type in workflow with associated UI components and translations.
- Enhanced SandboxLayer to manage sandbox attachment for Command nodes during execution.
- Updated various components and constants to integrate Command node functionality across the workflow.
