dify/api
Shili Cao 345ac8333c
Add Full-Text & Hybrid Search Support to Baidu Vector DB and Update SDK, Closes #25982 (#25983)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: crazywoola <100913391+crazywoola@users.noreply.github.com>
2025-09-22 10:17:35 +08:00
..
.idea fix nltk averaged_perceptron_tagger download and fix score limit is none (#7582) 2024-08-26 15:14:05 +08:00
.vscode feat/enhance the multi-modal support (#8818) 2024-10-21 10:43:49 +08:00
configs Add Full-Text & Hybrid Search Support to Baidu Vector DB and Update SDK, Closes #25982 (#25983) 2025-09-22 10:17:35 +08:00
constants chore: adopt StrEnum and auto() for some string-typed enums (#25129) 2025-09-12 21:14:26 +08:00
contexts feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
controllers Add Full-Text & Hybrid Search Support to Baidu Vector DB and Update SDK, Closes #25982 (#25983) 2025-09-22 10:17:35 +08:00
core Add Full-Text & Hybrid Search Support to Baidu Vector DB and Update SDK, Closes #25982 (#25983) 2025-09-22 10:17:35 +08:00
docker feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
events feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
extensions chore: improve opendal storage and ensure closing file after reading files in `load_stream` method (#25874) 2025-09-18 14:09:19 +08:00
factories feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
fields feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
libs feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
migrations feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
models feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
repositories feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
schedule chore: add ast-grep rule to convert Optional[T] to T | None (#25560) 2025-09-15 13:06:33 +08:00
services Fix: use data.type instead of type when checking datasource node (#25965) 2025-09-22 10:01:21 +08:00
tasks fix: remote code execution in email endpoints (#25753) 2025-09-18 16:45:34 +08:00
templates Feat/email register refactor (#25369) 2025-09-12 10:24:54 +08:00
tests Add Full-Text & Hybrid Search Support to Baidu Vector DB and Update SDK, Closes #25982 (#25983) 2025-09-22 10:17:35 +08:00
.dockerignore Enhance Code Consistency Across Repository with `.editorconfig` (#19023) 2025-04-29 18:04:33 +08:00
.env.example Add Full-Text & Hybrid Search Support to Baidu Vector DB and Update SDK, Closes #25982 (#25983) 2025-09-22 10:17:35 +08:00
.importlinter feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
.ruff.toml refactor: replace print statements with proper logging (#25773) 2025-09-18 20:35:47 +08:00
Dockerfile chore: update uv to 0.8.9 (#23833) 2025-08-12 23:41:39 +08:00
README.md [Chore/Refactor] Switch from MyPy to Basedpyright for type checking (#25047) 2025-09-03 11:52:26 +08:00
app.py feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
app_factory.py [Chore/Refactor] Improve type checking configuration (#25185) 2025-09-05 08:34:18 +08:00
celery_entrypoint.py Fix: replace stdout prints with debug logging (#25931) 2025-09-18 21:03:20 +08:00
commands.py Fix: replace stdout prints with debug logging (#25931) 2025-09-18 21:03:20 +08:00
dify_app.py refactor: assembling the app features in modular way (#9129) 2024-11-30 23:05:22 +08:00
gunicorn.conf.py feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
pyproject.toml Add Full-Text & Hybrid Search Support to Baidu Vector DB and Update SDK, Closes #25982 (#25983) 2025-09-22 10:17:35 +08:00
pyrightconfig.json feat: knowledge pipeline (#25360) 2025-09-18 12:49:10 +08:00
pytest.ini Refactor/remove db from cycle manager (#20455) 2025-05-30 04:34:13 +08:00
ty.toml chore: apply ty checks on api code with script and ci action (#24653) 2025-09-02 16:05:13 +08:00
uv.lock Add Full-Text & Hybrid Search Support to Baidu Vector DB and Update SDK, Closes #25982 (#25983) 2025-09-22 10:17:35 +08:00

README.md

Dify Backend API

Usage

[!IMPORTANT]

In the v1.3.0 release, poetry has been replaced with uv as the package manager for Dify API backend service.

  1. Start the docker-compose stack

    The backend require some middleware, including PostgreSQL, Redis, and Weaviate, which can be started together using docker-compose.

    cd ../docker
    cp middleware.env.example middleware.env
    # change the profile to other vector database if you are not using weaviate
    docker compose -f docker-compose.middleware.yaml --profile weaviate -p dify up -d
    cd ../api
    
  2. Copy .env.example to .env

    cp .env.example .env
    
  3. Generate a SECRET_KEY in the .env file.

    bash for Linux

    sed -i "/^SECRET_KEY=/c\SECRET_KEY=$(openssl rand -base64 42)" .env
    

    bash for Mac

    secret_key=$(openssl rand -base64 42)
    sed -i '' "/^SECRET_KEY=/c\\
    SECRET_KEY=${secret_key}" .env
    
  4. Create environment.

    Dify API service uses UV to manage dependencies. First, you need to add the uv package manager, if you don't have it already.

    pip install uv
    # Or on macOS
    brew install uv
    
  5. Install dependencies

    uv sync --dev
    
  6. Run migrate

    Before the first launch, migrate the database to the latest version.

    uv run flask db upgrade
    
  7. Start backend

    uv run flask run --host 0.0.0.0 --port=5001 --debug
    
  8. Start Dify web service.

  9. Setup your application by visiting http://localhost:3000.

  10. If you need to handle and debug the async tasks (e.g. dataset importing and documents indexing), please start the worker service.

uv run celery -A app.celery worker -P gevent -c 1 --loglevel INFO -Q dataset,generation,mail,ops_trace,app_deletion,plugin,workflow_storage,conversation

Addition, if you want to debug the celery scheduled tasks, you can use the following command in another terminal:

uv run celery -A app.celery beat

Testing

  1. Install dependencies for both the backend and the test environment

    uv sync --dev
    
  2. Run the tests locally with mocked system environment variables in tool.pytest_env section in pyproject.toml, more can check Claude.md

    uv run pytest                           # Run all tests
    uv run pytest tests/unit_tests/         # Unit tests only
    uv run pytest tests/integration_tests/  # Integration tests
    
    # Code quality
    ../dev/reformat               # Run all formatters and linters
    uv run ruff check --fix ./    # Fix linting issues
    uv run ruff format ./         # Format code
    uv run basedpyright .         # Type checking