dify/web
QuantumGhost 349c3cf7b8
feat(api): Add image multimodal support for LLMNode (#17372)
Enhance `LLMNode` with multimodal capability, introducing support for
image outputs.

This implementation extracts base64-encoded images from LLM responses,
saves them to the storage service, and records the file metadata in the
`ToolFile` table. In conversations, these images are rendered as
markdown-based inline images.
Additionally, the images are included in the LLMNode's output as
file variables, enabling subsequent nodes in the workflow to utilize them.

To integrate file outputs into workflows, adjustments to the frontend code
are necessary.

For multimodal output functionality, updates to related model configurations
are required. Currently, this capability has been applied exclusively to
Google's Gemini models.

Close #15814.

Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
2025-04-30 17:28:02 +08:00
..
.husky chore: merge lint dependency group into dev group of python packages (#18088) 2025-04-15 20:50:06 +08:00
.storybook Introduce Plugins (#13836) 2025-02-17 17:05:13 +08:00
.vscode Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 2025-04-29 18:04:33 +08:00
__mocks__ feat: regenerate history switch navigation (#8749) 2024-10-24 12:09:46 +08:00
app feat(api): Add image multimodal support for LLMNode (#17372) 2025-04-30 17:28:02 +08:00
assets fix: missing files (#2291) 2024-01-30 16:21:40 +08:00
bin Feat/embedding (#553) 2023-07-12 17:27:50 +08:00
config feat: add missing switches (#18619) 2025-04-23 18:02:18 +08:00
context fix(i18n): add functions to retrieve document and pricing page languages (#19142) 2025-04-30 14:58:49 +08:00
docker fixes: fix entrypoint script with missing environment variables (#19039) 2025-04-29 08:57:58 +08:00
hooks fix: check dsl version when create app from explore template (#18872) 2025-04-27 14:00:45 +08:00
i18n feat: support remove first and remove last in variable assigner (#19144) 2025-04-30 15:50:00 +08:00
models feat: Add W&B Weave Tracing Integration (#14262) 2025-04-26 04:28:30 -07:00
public Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 2025-04-29 18:04:33 +08:00
service fix: update reset password token when email code verify success (#18364) 2025-04-18 17:14:51 +08:00
themes Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 2025-04-29 18:04:33 +08:00
types fix: cannot regenerate with image(#15060) (#16611) 2025-04-16 09:56:46 +08:00
utils Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 2025-04-29 18:04:33 +08:00
.dockerignore Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 2025-04-29 18:04:33 +08:00
.env.example fix: clickjacking (#18516) 2025-04-22 16:48:45 +08:00
.gitignore Introduce Plugins (#13836) 2025-02-17 17:05:13 +08:00
Dockerfile chore: bump Nodejs in web image from 20 to 22 LTS (#13341) 2025-04-09 09:40:11 +08:00
README.md fix: bump the minimal node requirement to fix eslint fail (#17938) 2025-04-21 09:51:39 +08:00
eslint.config.mjs chore: eslint add sonar (#17989) 2025-04-14 15:28:20 +08:00
global.d.ts feat: support assistant frontend (#2139) 2024-01-23 19:31:56 +08:00
jest.config.ts feat/TanStack-Form (#18346) 2025-04-18 15:54:22 +08:00
jest.setup.ts feat/TanStack-Form (#18346) 2025-04-18 15:54:22 +08:00
middleware.ts chore: support other webapps embedded in iframe (#18877) 2025-04-27 14:21:27 +08:00
next.config.js Added the missing path of the webpath prefix and the prefix basepath + of static resources to remove the bug of adding more basepath. (#18658) 2025-04-24 17:14:26 +08:00
package.json fix: Chinese input deletes extra character in Safari within Workspaces (#18193) (#19088) 2025-04-29 18:47:35 +08:00
pnpm-lock.yaml fix: Chinese input deletes extra character in Safari within Workspaces (#18193) (#19088) 2025-04-29 18:47:35 +08:00
postcss.config.js Initial commit 2023-05-15 08:51:32 +08:00
tailwind-common-config.ts feat: llm support struct output (#17994) 2025-04-18 16:53:43 +08:00
tailwind.config.js Introduce Plugins (#13836) 2025-02-17 17:05:13 +08:00
tsconfig.json Initial commit 2023-05-15 08:51:32 +08:00
typography.js Chore: frontend infrastructure upgrade (#16420) 2025-03-21 17:41:03 +08:00

README.md

Dify Frontend

This is a Next.js project bootstrapped with create-next-app.

Getting Started

Run by source code

Before starting the web frontend service, please make sure the following environment is ready.

First, install the dependencies:

pnpm install

Then, configure the environment variables. Create a file named .env.local in the current directory and copy the contents from .env.example. Modify the values of these environment variables according to your requirements:

cp .env.example .env.local
# For production release, change this to PRODUCTION
NEXT_PUBLIC_DEPLOY_ENV=DEVELOPMENT
# The deployment edition, SELF_HOSTED
NEXT_PUBLIC_EDITION=SELF_HOSTED
# The base URL of console application, refers to the Console base URL of WEB service if console domain is
# different from api or web app domain.
# example: http://cloud.dify.ai/console/api
NEXT_PUBLIC_API_PREFIX=http://localhost:5001/console/api
# The URL for Web APP, refers to the Web App base URL of WEB service if web app domain is different from
# console or api domain.
# example: http://udify.app/api
NEXT_PUBLIC_PUBLIC_API_PREFIX=http://localhost:5001/api

# SENTRY
NEXT_PUBLIC_SENTRY_DSN=

Finally, run the development server:

pnpm run dev

Open http://localhost:3000 with your browser to see the result.

You can start editing the file under folder app. The page auto-updates as you edit the file.

Deploy

Deploy on server

First, build the app for production:

pnpm run build

Then, start the server:

pnpm run start

If you want to customize the host and port:

pnpm run start --port=3001 --host=0.0.0.0

If you want to customize the number of instances launched by PM2, you can configure PM2_INSTANCES in docker-compose.yaml or Dockerfile.

Storybook

This project uses Storybook for UI component development.

To start the storybook server, run:

pnpm storybook

Open http://localhost:6006 with your browser to see the result.

Lint Code

If your IDE is VSCode, rename web/.vscode/settings.example.json to web/.vscode/settings.json for lint code setting.

Test

We start to use Jest and React Testing Library for Unit Testing.

You can create a test file with a suffix of .spec beside the file that to be tested. For example, if you want to test a file named util.ts. The test file name should be util.spec.ts.

Run test:

pnpm run test

If you are not familiar with writing tests, here is some code to refer to:

Documentation

Visit https://docs.dify.ai/getting-started/readme to view the full documentation.

Community

The Dify community can be found on Discord community, where you can ask questions, voice ideas, and share your projects.