mirror of
https://github.com/langgenius/dify.git
synced 2026-08-15 04:59:46 +08:00
chore: remove obsolete development docs (#40098)
This commit is contained in:
parent
4b8a2c7e6c
commit
33b238343b
@ -1,184 +0,0 @@
|
||||
# HITL Standalone Form File Upload Design
|
||||
|
||||
## Context
|
||||
|
||||
HITL standalone forms can be opened directly through a form link and do not require the
|
||||
submitter to sign in through the Web App. After `file` and `file-list` inputs were introduced,
|
||||
this standalone entry point also needed file upload support.
|
||||
|
||||
This entry point has a different identity model from the existing upload paths:
|
||||
|
||||
- Web App upload is backed by Web App authentication and an `EndUser` context.
|
||||
- Service API upload is backed by API key authentication and the `user` parameter.
|
||||
- HITL standalone form submission is link-based and anonymous from the product perspective.
|
||||
The standalone submitter is not necessarily the workflow or chatflow initiator.
|
||||
|
||||
The goal is therefore not to add another general-purpose upload channel. The goal is to
|
||||
provide a constrained, short-lived upload capability that is scoped to one HITL form submission.
|
||||
|
||||
## Goals
|
||||
|
||||
- Support local file upload and remote URL upload from the HITL standalone page.
|
||||
- Keep the standalone page independent from the Web App login flow.
|
||||
- Avoid creating a technical HITL `EndUser`.
|
||||
- Avoid changing the authentication model of existing Web App, Service API, or Console upload endpoints.
|
||||
- Keep upload request parameters aligned with the equivalent Web App upload endpoints where possible.
|
||||
- Invalidate upload capability once the form is submitted, expired, or timed out.
|
||||
- Store uploaded files in a way that remains compatible with the existing workflow resume and file access model.
|
||||
|
||||
## Decision
|
||||
|
||||
HITL standalone upload uses a dedicated upload token that is bound to a form
|
||||
recipient. The token authorizes file upload only while the related form is still valid.
|
||||
|
||||
Files uploaded through the HITL standalone page are recorded under the workflow or
|
||||
chatflow initiator, not under the anonymous standalone submitter and not under a technical HITL `EndUser`.
|
||||
|
||||
This keeps workflow resume aligned with the existing execution model: one initiator owns the
|
||||
workflow run context, and file restoration continues to resolve files through that initiator's
|
||||
access scope. HITL-specific form/token/file relationships remain available for audit and tracing,
|
||||
but they do not become the source of truth for file access control.
|
||||
|
||||
## API Shape
|
||||
|
||||
HITL standalone upload has three endpoint categories:
|
||||
|
||||
| Purpose | HITL endpoint | Aligned Web App endpoint |
|
||||
| --- | --- | --- |
|
||||
| Issue upload token | `POST /api/form/human_input/{form_token}/upload-token` | No direct equivalent |
|
||||
| Upload local file | `POST /api/form/human_input/files/upload` | `POST /api/files/upload` |
|
||||
| Upload remote file | `POST /api/form/human_input/files/remote-upload` | `POST /api/remote-files/upload` |
|
||||
|
||||
Local upload follows the Web App `POST /api/files/upload` parameter shape:
|
||||
|
||||
- `multipart/form-data`
|
||||
- Required `file`
|
||||
|
||||
Remote upload follows the Web App `POST /api/remote-files/upload` parameter shape:
|
||||
|
||||
- `application/json`
|
||||
- Required `url`
|
||||
|
||||
HITL upload endpoints do not accept the Service API `user` parameter. That parameter
|
||||
belongs to the Service API `EndUser` mapping model and does not represent the anonymous
|
||||
standalone form submitter.
|
||||
|
||||
## Upload Token
|
||||
|
||||
The upload token is issued through the form token:
|
||||
|
||||
```http
|
||||
POST /api/form/human_input/{form_token}/upload-token
|
||||
```
|
||||
|
||||
Upload requests carry the token through the `Authorization` header:
|
||||
|
||||
```http
|
||||
Authorization: bearer hitl_upload_{random_value}
|
||||
```
|
||||
|
||||
The `hitl_upload_` prefix only distinguishes this credential from other bearer token types.
|
||||
Security comes from the high-entropy random value, server-side hash storage, and server-side state validation.
|
||||
|
||||
The token is bound to at least:
|
||||
|
||||
- The HITL form.
|
||||
- The form recipient.
|
||||
- The tenant.
|
||||
- The app.
|
||||
|
||||
The token must satisfy these rules:
|
||||
|
||||
- It cannot outlive the form expiration.
|
||||
- It cannot be used after the form is submitted, expired, or timed out.
|
||||
- It is validated through the HITL upload path, not through the existing app token validation chain.
|
||||
|
||||
## Why Authorization Header
|
||||
|
||||
Putting `upload_token` in the request body would avoid additional CORS header configuration, but it has a bad failure mode for file upload. The server often needs to parse the multipart body before it can read a body token, so invalid requests can still consume upload parsing, temporary file, memory, or disk resources.
|
||||
|
||||
Using `Authorization: bearer hitl_upload_{random_value}` keeps authentication before expensive business processing:
|
||||
|
||||
- Invalid local upload requests can be rejected before reading the multipart body.
|
||||
- Invalid remote upload requests can be rejected before any outbound network access.
|
||||
- Bearer credential semantics are explicit and do not mix authentication with business fields.
|
||||
- The token is not exposed through query strings, access logs, referrers, or browser history.
|
||||
|
||||
The tradeoff is that cross-origin deployments must allow the `Authorization` header and accept browser preflight requests. This is a reasonable configuration cost for an earlier and clearer authentication boundary.
|
||||
|
||||
## File Ownership
|
||||
|
||||
The standalone submitter is not a reliable product identity. Assigning files to a technical `EndUser` would also conflict with workflow resume: existing file restoration expects files to be readable through the workflow or chatflow initiator's scope.
|
||||
|
||||
The selected model is:
|
||||
|
||||
- If the original run was started by an `Account`, standalone HITL uploads are stored under that `Account`.
|
||||
- If the original run was started by an `EndUser`, standalone HITL uploads are stored under that `EndUser`.
|
||||
|
||||
This means `UploadFile.created_by_role` and `UploadFile.created_by` continue to be the source of truth for file access control. HITL association records provide auditability but do not grant file access by themselves.
|
||||
|
||||
## Persistence And Audit
|
||||
|
||||
The HITL upload model has two responsibilities:
|
||||
|
||||
- Upload tokens authorize a form recipient to upload files while the form remains valid.
|
||||
- Upload-file association records trace which files were uploaded through which HITL upload token.
|
||||
|
||||
These records are intentionally not tied to an `EndUser`. Their purpose is to preserve the HITL form/token/file relationship for audit and cleanup, not to define a separate file owner identity.
|
||||
|
||||
## Local Upload Boundary
|
||||
|
||||
Local upload should reuse the existing file upload semantics as much as possible:
|
||||
|
||||
- Request parameters stay aligned with Web App local upload.
|
||||
- Existing file size checks, extension restrictions, and document-type handling remain applicable.
|
||||
- Response shape stays aligned with the existing file upload response.
|
||||
- Token validation happens before reading the upload body.
|
||||
|
||||
## Remote Upload Boundary
|
||||
|
||||
Remote upload should reuse the existing remote upload semantics as much as possible:
|
||||
|
||||
- Request parameters stay aligned with Web App remote upload.
|
||||
- Token validation happens before outbound network access.
|
||||
- Remote fetching continues to go through the existing SSRF-safe path.
|
||||
- Remote filename, extension, MIME type, and file size inference stay aligned with existing behavior.
|
||||
- Response shape stays aligned with the existing remote upload response.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### Unauthenticated Standalone Upload Endpoint
|
||||
|
||||
This is the simplest implementation option and does not require Web App login, `EndUser`, or a new token model. It was not selected because it exposes a public file upload surface that can be abused in SaaS or internet-facing deployments. Adding authentication later would also change the endpoint contract after clients have integrated with it.
|
||||
|
||||
### Reuse Web App Login And Upload
|
||||
|
||||
This would maximize reuse of existing Web App upload behavior, but it would bind HITL standalone forms to Web App login, app code, Web App enablement, and enterprise SSO semantics. That coupling is undesirable because HITL forms can be reached from independent channels such as email. It also makes product behavior unclear when the Web App is disabled or the app code is reset.
|
||||
|
||||
### Create `EndUser` From Form Token
|
||||
|
||||
This would satisfy existing upload paths that require an `EndUser` context and would allow form state to limit upload capability. It was not selected because the created identity would be technical rather than a real submitter identity. It would also mix HITL standalone form behavior into the broader `EndUser` model already used by Web App, Service API, triggers, MCP, and other entry points.
|
||||
|
||||
More importantly, files owned by this technical `EndUser` would not naturally be readable through the workflow initiator scope during workflow resume.
|
||||
|
||||
### Technical `EndUser` With File Access Exception
|
||||
|
||||
This would keep the technical `EndUser` as the file owner and add an access-control exception so the workflow initiator can read files uploaded through the same HITL form. It solves the immediate resume problem, but it pushes a HITL-specific rule into the general file access layer. Over time, that makes permission reasoning harder and increases the chance of accidental access expansion.
|
||||
|
||||
Assigning files directly to the workflow or chatflow initiator avoids that bypass and keeps file access governed by the existing owner model.
|
||||
|
||||
## Design Constraints
|
||||
|
||||
- HITL standalone upload does not reuse the Web App login flow.
|
||||
- HITL standalone upload does not create a technical HITL `EndUser`.
|
||||
- Upload token validity is controlled by form state.
|
||||
- File access control continues to use the `UploadFile` owner as the source of truth.
|
||||
- HITL association records provide audit and traceability only.
|
||||
- Workflow resume continues to restore submitted file values through the existing file restoration path.
|
||||
|
||||
## Future Considerations
|
||||
|
||||
- If endpoint parameters change, compare them with the corresponding Web App upload endpoint to avoid unnecessary drift.
|
||||
- If file ownership changes, verify the workflow resume path and the file access model together.
|
||||
- If HITL forms support multiple submissions or reopening, token invalidation semantics must be redefined.
|
||||
- If remote upload policy expands, prefer extending the existing remote upload and SSRF-safe behavior instead of creating a HITL-only network path.
|
||||
@ -1,186 +0,0 @@
|
||||
# EU AI Act Compliance Guide for Dify Deployers
|
||||
|
||||
Dify is an LLMOps platform for building RAG pipelines, agents, and AI workflows. If you deploy Dify in the EU — whether self-hosted or using a cloud provider — the EU AI Act applies to your deployment. This guide covers what the regulation requires and how Dify's architecture maps to those requirements.
|
||||
|
||||
## Is your system in scope?
|
||||
|
||||
The detailed obligations in Articles 12, 13, and 14 only apply to **high-risk AI systems** as defined in Annex III of the EU AI Act. A Dify application is high-risk if it is used for:
|
||||
|
||||
- **Recruitment and HR** — screening candidates, evaluating employee performance, allocating tasks
|
||||
- **Credit scoring and insurance** — assessing creditworthiness or setting premiums
|
||||
- **Law enforcement** — profiling, criminal risk assessment, border control
|
||||
- **Critical infrastructure** — managing energy, water, transport, or telecommunications systems
|
||||
- **Education assessment** — grading students, determining admissions
|
||||
- **Essential public services** — evaluating eligibility for benefits, housing, or emergency services
|
||||
|
||||
Most Dify deployments (customer-facing chatbots, internal knowledge bases, content generation workflows) are **not** high-risk. If your Dify application does not fall into one of the categories above:
|
||||
|
||||
- **Article 50** (end-user transparency) still applies if users interact with your application directly. See the [Article 50 section](#article-50-end-user-transparency) below.
|
||||
- **GDPR** still applies if you process personal data. See the [GDPR section](#gdpr-considerations) below.
|
||||
- The high-risk obligations (Articles 9-15) are less likely to apply, but risk classification is context-dependent. **Do not self-classify without legal review.** Focus on Article 50 (transparency) and GDPR (data protection) as your baseline obligations.
|
||||
|
||||
If you are unsure whether your use case qualifies as high-risk, consult a qualified legal professional before proceeding.
|
||||
|
||||
## Self-hosted vs cloud: different compliance profiles
|
||||
|
||||
| Deployment | Your role | Dify's role | Who handles compliance? |
|
||||
|-----------|----------|-------------|------------------------|
|
||||
| **Self-hosted** | Provider and deployer | Framework provider — obligations under Article 25 apply only if Dify is placed on the market or put into service as part of a complete AI system bearing its name or trademark | You |
|
||||
| **Dify Cloud** | Deployer | Provider and processor | Shared — Dify handles SOC 2 and GDPR for the platform; you handle AI Act obligations for your specific use case |
|
||||
|
||||
Dify Cloud already has SOC 2 Type II and GDPR compliance for the platform itself. But the EU AI Act adds obligations specific to AI systems that SOC 2 does not cover: risk classification, technical documentation, transparency, and human oversight.
|
||||
|
||||
## Supported providers and services
|
||||
|
||||
Dify integrates with a broad range of AI providers and data stores. The following are the key ones relevant to compliance:
|
||||
|
||||
- **AI providers:** HuggingFace (core), plus integrations with OpenAI, Anthropic, Google, and 100+ models via provider plugins
|
||||
- **Model identifiers include:** gpt-4o, gpt-3.5-turbo, claude-3-opus, gemini-2.5-flash, whisper-1, and others
|
||||
- **Vector database connections:** Extensive RAG infrastructure supporting numerous vector stores
|
||||
|
||||
Dify's plugin architecture means actual provider usage depends on your configuration. Document which providers and models are active in your deployment.
|
||||
|
||||
## Data flow diagram
|
||||
|
||||
A typical Dify RAG deployment:
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
USER((User)) -->|query| DIFY[Dify Platform]
|
||||
DIFY -->|prompts| LLM([LLM Provider])
|
||||
LLM -->|responses| DIFY
|
||||
DIFY -->|documents| EMBED([Embedding Model])
|
||||
EMBED -->|vectors| DIFY
|
||||
DIFY -->|store/retrieve| VS[(Vector Store)]
|
||||
DIFY -->|knowledge| KB[(Knowledge Base)]
|
||||
DIFY -->|response| USER
|
||||
|
||||
classDef processor fill:#60a5fa,stroke:#1e40af,color:#000
|
||||
classDef controller fill:#4ade80,stroke:#166534,color:#000
|
||||
classDef app fill:#a78bfa,stroke:#5b21b6,color:#000
|
||||
classDef user fill:#f472b6,stroke:#be185d,color:#000
|
||||
|
||||
class USER user
|
||||
class DIFY app
|
||||
class LLM processor
|
||||
class EMBED processor
|
||||
class VS controller
|
||||
class KB controller
|
||||
```
|
||||
|
||||
**GDPR roles** (providers are typically processors for customer-submitted data, but the exact role depends on each provider's terms of service and processing purpose; deployers should review each provider's DPA):
|
||||
- **Cloud LLM providers (OpenAI, Anthropic, Google)** typically act as processors — requires DPA.
|
||||
- **Cloud embedding services** typically act as processors — requires DPA.
|
||||
- **Self-hosted vector stores (Weaviate, Qdrant, pgvector):** Your organization remains the controller — no third-party transfer.
|
||||
- **Cloud vector stores (Pinecone, Zilliz Cloud)** typically act as processors — requires DPA.
|
||||
- **Knowledge base documents:** Your organization is the controller — stored in your infrastructure.
|
||||
|
||||
## Article 11: Technical documentation
|
||||
|
||||
High-risk systems need Annex IV documentation. For Dify deployments, key sections include:
|
||||
|
||||
| Section | What Dify provides | What you must document |
|
||||
|---------|-------------------|----------------------|
|
||||
| General description | Platform capabilities, supported models | Your specific use case, intended users, deployment context |
|
||||
| Development process | Dify's architecture, plugin system | Your RAG pipeline design, prompt engineering, knowledge base curation |
|
||||
| Monitoring | Dify's built-in logging and analytics | Your monitoring plan, alert thresholds, incident response |
|
||||
| Performance metrics | Dify's evaluation features | Your accuracy benchmarks, quality thresholds, bias testing |
|
||||
| Risk management | — | Risk assessment for your specific use case |
|
||||
|
||||
Some sections can be derived from Dify's architecture and your deployment configuration, as shown in the table above. The remaining sections require your input.
|
||||
|
||||
## Article 12: Record-keeping
|
||||
|
||||
Dify's built-in logging covers several Article 12 requirements:
|
||||
|
||||
| Requirement | Dify Feature | Status |
|
||||
|------------|-------------|--------|
|
||||
| Conversation logs | Full conversation history with timestamps | **Covered** |
|
||||
| Model tracking | Model name recorded per interaction | **Covered** |
|
||||
| Token usage | Token counts per message | **Covered** |
|
||||
| Cost tracking | Cost per conversation (if provider reports it) | **Partial** |
|
||||
| Document retrieval | RAG source documents logged | **Covered** |
|
||||
| User identification | User session tracking | **Covered** |
|
||||
| Error logging | Failed generation logs | **Covered** |
|
||||
| Data retention | Configurable | **Your responsibility** |
|
||||
|
||||
**Retention periods:** The required retention period depends on your role under the Act. Article 18 requires **providers** of high-risk systems to retain logs and technical documentation for **10 years** after market placement. Article 26(6) requires **deployers** to retain logs for at least **6 months**. If you self-host Dify and have substantially modified the system, you may be classified as a provider rather than a deployer. Confirm the applicable retention period with legal counsel.
|
||||
|
||||
## Article 13: Transparency to deployers
|
||||
|
||||
Article 13 requires providers of high-risk AI systems to supply deployers with the information needed to understand and operate the system correctly. This is a **documentation obligation**, not a logging obligation. For Dify deployments, this means the upstream LLM and embedding providers must give you:
|
||||
|
||||
- Instructions for use, including intended purpose and known limitations
|
||||
- Accuracy metrics and performance benchmarks
|
||||
- Known or foreseeable risks and residual risks after mitigation
|
||||
- Technical specifications: input/output formats, training data characteristics, model architecture details
|
||||
|
||||
As a deployer, collect model cards, system documentation, and accuracy reports from each AI provider your Dify application uses. Maintain these as part of your Annex IV technical documentation.
|
||||
|
||||
Dify's platform features provide **supporting evidence** that can inform Article 13 documentation, but they do not satisfy Article 13 on their own:
|
||||
- **Source attribution** — Dify's RAG citation feature shows which documents informed the response, supporting deployer-side auditing
|
||||
- **Model identification** — Dify logs which LLM model generates responses, providing evidence for system documentation
|
||||
- **Conversation logs** — execution history helps compile performance and behavior evidence
|
||||
|
||||
You must independently produce system documentation covering how your specific Dify deployment uses AI, its intended purpose, performance characteristics, and residual risks.
|
||||
|
||||
## Article 50: End-user transparency
|
||||
|
||||
Article 50 requires deployers to inform end users that they are interacting with an AI system. This is a separate obligation from Article 13 and applies even to limited-risk systems.
|
||||
|
||||
For Dify applications serving end users:
|
||||
|
||||
1. **Disclose AI involvement** — tell users they are interacting with an AI system
|
||||
2. **AI-generated content labeling** — identify AI-generated content as such (e.g., clear labeling in the UI)
|
||||
|
||||
Dify's "citation" feature also supports end-user transparency by showing users which knowledge base documents informed the answer.
|
||||
|
||||
> **Note:** Article 50 applies to chatbots and systems interacting directly with natural persons. It has a separate scope from the high-risk designation under Annex III — it applies even to limited-risk systems.
|
||||
|
||||
## Article 14: Human oversight
|
||||
|
||||
Article 14 requires that high-risk AI systems be designed so that natural persons can effectively oversee them. Dify provides **automated technical safeguards** that support human oversight, but they are not a substitute for it:
|
||||
|
||||
| Dify Feature | What It Does | Oversight Role |
|
||||
|-------------|-------------|----------------|
|
||||
| Annotation/feedback system | Human review of AI outputs | **Direct oversight** — humans evaluate and correct AI responses |
|
||||
| Content moderation | Built-in filtering before responses reach users | **Automated safeguard** — reduces harmful outputs but does not replace human judgment on edge cases |
|
||||
| Rate limiting | Controls on API usage | **Automated safeguard** — bounds system behavior, supports overseer's ability to maintain control |
|
||||
| Workflow control | Insert human review steps between AI generation and output | **Oversight enabler** — allows building approval gates into the pipeline |
|
||||
|
||||
These automated controls are necessary building blocks, but Article 14 compliance requires **human oversight procedures** on top of them:
|
||||
- **Escalation procedures** — define what happens when moderation triggers or edge cases arise (who is notified, what action is taken)
|
||||
- **Human review pipeline** — for high-stakes decisions, route AI outputs to a qualified person before they take effect
|
||||
- **Override mechanism** — a human must be able to halt AI responses or override the system's output
|
||||
- **Competence requirements** — the human overseer must understand the system's capabilities, limitations, and the context of its outputs
|
||||
|
||||
### Recommended pattern
|
||||
|
||||
For high-risk use cases (HR, legal, medical), configure your Dify workflow to require human approval before the AI response is delivered to the end user or acted upon.
|
||||
|
||||
## Knowledge base compliance
|
||||
|
||||
Dify's knowledge base feature has specific compliance implications:
|
||||
|
||||
1. **Data provenance:** Document where your knowledge base documents come from. Article 10 requires data governance for training data; knowledge bases are analogous.
|
||||
2. **Update tracking:** When you add, remove, or update documents in the knowledge base, log the change. The AI system's behavior changes with its knowledge base.
|
||||
3. **PII in documents:** If knowledge base documents contain personal data, GDPR applies to the entire RAG pipeline. Implement access controls and consider PII redaction before indexing.
|
||||
4. **Copyright:** Ensure you have the right to use the documents in your knowledge base for AI-assisted generation.
|
||||
|
||||
## GDPR considerations
|
||||
|
||||
1. **Legal basis** (Article 6): Document why AI processing of user queries is necessary
|
||||
2. **Data Processing Agreements** (Article 28): Required for each cloud LLM and embedding provider
|
||||
3. **Data minimization:** Only include necessary context in prompts; avoid sending entire documents when a relevant excerpt suffices
|
||||
4. **Right to erasure:** If a user requests deletion, ensure their conversations are removed from Dify's logs AND any vector store entries derived from their data
|
||||
5. **Cross-border transfers:** Providers based outside the EEA — including US-based providers (OpenAI, Anthropic), and any other non-EEA providers you route to — require Standard Contractual Clauses (SCCs) or equivalent safeguards under Chapter V of the GDPR. Review each provider's transfer mechanism individually.
|
||||
|
||||
## Resources
|
||||
|
||||
- [EU AI Act full text](https://artificialintelligenceact.eu/)
|
||||
- [Dify documentation](https://docs.dify.ai/)
|
||||
- [Dify SOC 2 compliance](https://dify.ai/trust)
|
||||
|
||||
---
|
||||
|
||||
*This is not legal advice. Consult a qualified professional for compliance decisions.*
|
||||
@ -1,187 +0,0 @@
|
||||
# Weaviate Migration Guide: v1.19 → v1.27
|
||||
|
||||
## Overview
|
||||
|
||||
Dify has upgraded from Weaviate v1.19 to v1.27 with the Python client updated from v3.24 to v4.17.
|
||||
|
||||
## What Changed
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
1. **Weaviate Server**: `1.19.0` → `1.27.0`
|
||||
1. **Python Client**: `weaviate-client~=3.24.0` → `weaviate-client==4.17.0`
|
||||
1. **gRPC Required**: Weaviate v1.27 requires gRPC port `50051` (in addition to HTTP port `8080`)
|
||||
1. **Docker Compose**: Added temporary entrypoint overrides for client installation
|
||||
|
||||
### Key Improvements
|
||||
|
||||
- Faster vector operations via gRPC
|
||||
- Improved batch processing
|
||||
- Better error handling
|
||||
|
||||
## Migration Steps
|
||||
|
||||
### For Docker Users
|
||||
|
||||
#### Step 1: Backup Your Data
|
||||
|
||||
```bash
|
||||
cd docker
|
||||
docker compose down
|
||||
sudo cp -r ./volumes/weaviate ./volumes/weaviate_backup_$(date +%Y%m%d)
|
||||
```
|
||||
|
||||
#### Step 2: Update Dify
|
||||
|
||||
```bash
|
||||
git pull origin main
|
||||
docker compose pull
|
||||
```
|
||||
|
||||
#### Step 3: Start Services
|
||||
|
||||
```bash
|
||||
docker compose up -d
|
||||
sleep 30
|
||||
curl http://localhost:8080/v1/meta
|
||||
```
|
||||
|
||||
#### Step 4: Verify Migration
|
||||
|
||||
```bash
|
||||
# Check both ports are accessible
|
||||
curl http://localhost:8080/v1/meta
|
||||
netstat -tulpn | grep 50051
|
||||
|
||||
# Test in Dify UI:
|
||||
# 1. Go to Knowledge Base
|
||||
# 2. Test search functionality
|
||||
# 3. Upload a test document
|
||||
```
|
||||
|
||||
### For Source Installation
|
||||
|
||||
#### Step 1: Update Dependencies
|
||||
|
||||
```bash
|
||||
cd api
|
||||
uv sync --dev
|
||||
uv run python -c "import weaviate; print(weaviate.__version__)"
|
||||
# Should show: 4.17.0
|
||||
```
|
||||
|
||||
#### Step 2: Update Weaviate Server
|
||||
|
||||
```bash
|
||||
cd docker
|
||||
docker compose -f docker-compose.middleware.yaml --profile weaviate up -d weaviate
|
||||
curl http://localhost:8080/v1/meta
|
||||
netstat -tulpn | grep 50051
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Error: "No module named 'weaviate.classes'"
|
||||
|
||||
**Solution**:
|
||||
|
||||
```bash
|
||||
cd api
|
||||
uv sync --reinstall-package weaviate-client
|
||||
uv run python -c "import weaviate; print(weaviate.__version__)"
|
||||
# Should show: 4.17.0
|
||||
```
|
||||
|
||||
### Error: "gRPC health check failed"
|
||||
|
||||
**Solution**:
|
||||
|
||||
```bash
|
||||
# Check Weaviate ports
|
||||
docker ps | grep weaviate
|
||||
# Should show: 0.0.0.0:8080->8080/tcp, 0.0.0.0:50051->50051/tcp
|
||||
|
||||
# If missing gRPC port, add to docker-compose:
|
||||
# ports:
|
||||
# - "8080:8080"
|
||||
# - "50051:50051"
|
||||
```
|
||||
|
||||
### Error: "Weaviate version 1.19.0 is not supported"
|
||||
|
||||
**Solution**:
|
||||
|
||||
```bash
|
||||
# Update Weaviate image in docker-compose
|
||||
# Change: semitechnologies/weaviate:1.19.0
|
||||
# To: semitechnologies/weaviate:1.27.0
|
||||
docker compose down
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
### Data Migration Failed
|
||||
|
||||
**Solution**:
|
||||
|
||||
```bash
|
||||
cd docker
|
||||
docker compose down
|
||||
sudo rm -rf ./volumes/weaviate
|
||||
sudo cp -r ./volumes/weaviate_backup_YYYYMMDD ./volumes/weaviate
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## Rollback Instructions
|
||||
|
||||
```bash
|
||||
# 1. Stop services
|
||||
docker compose down
|
||||
|
||||
# 2. Restore data backup
|
||||
sudo rm -rf ./volumes/weaviate
|
||||
sudo cp -r ./volumes/weaviate_backup_YYYYMMDD ./volumes/weaviate
|
||||
|
||||
# 3. Checkout previous version
|
||||
git checkout <previous-commit>
|
||||
|
||||
# 4. Restart services
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
## Compatibility
|
||||
|
||||
| Component | Old Version | New Version | Compatible |
|
||||
|-----------|-------------|-------------|------------|
|
||||
| Weaviate Server | 1.19.0 | 1.27.0 | ✅ Yes |
|
||||
| weaviate-client | ~3.24.0 | ==4.17.0 | ✅ Yes |
|
||||
| Existing Data | v1.19 format | v1.27 format | ✅ Yes |
|
||||
|
||||
## Testing Checklist
|
||||
|
||||
Before deploying to production:
|
||||
|
||||
- [ ] Backup all Weaviate data
|
||||
- [ ] Test in staging environment
|
||||
- [ ] Verify existing collections are accessible
|
||||
- [ ] Test vector search functionality
|
||||
- [ ] Test document upload and retrieval
|
||||
- [ ] Monitor gRPC connection stability
|
||||
- [ ] Check performance metrics
|
||||
|
||||
## Support
|
||||
|
||||
If you encounter issues:
|
||||
|
||||
1. Check GitHub Issues: https://github.com/langgenius/dify/issues
|
||||
1. Create a bug report with:
|
||||
- Error messages
|
||||
- Docker logs: `docker compose logs weaviate`
|
||||
- Dify version
|
||||
- Migration steps attempted
|
||||
|
||||
## Important Notes
|
||||
|
||||
- **Data Safety**: Existing vector data remains fully compatible
|
||||
- **No Re-indexing**: No need to rebuild vector indexes
|
||||
- **Temporary Workaround**: The entrypoint overrides are temporary until next Dify release
|
||||
- **Performance**: May see improved performance due to gRPC usage
|
||||
@ -1,51 +0,0 @@
|
||||
## library
|
||||
|
||||
- i18next
|
||||
- react-i18next
|
||||
|
||||
## hooks
|
||||
|
||||
- useTranslation
|
||||
- useGetLanguage
|
||||
- useLocale
|
||||
- useRenderI18nObject
|
||||
|
||||
## impl
|
||||
|
||||
- App Boot
|
||||
- app/layout.tsx load i18n and init context
|
||||
- use `<I18nServer/>`
|
||||
- read locale with `getLocaleOnServer` (in node.js)
|
||||
- locale from cookie, or browser request header
|
||||
- only used in client app init and 2 server code(plugin desc, datasets)
|
||||
- use `<I18N/>`
|
||||
- init i18n context
|
||||
- `setLocaleOnClient`
|
||||
- `changeLanguage` (defined in i18n/i18next-config, also init i18n resources (side effects))
|
||||
- is `i18next.changeLanguage`
|
||||
- loads JSON namespaces for the target locale and merges resource bundles (see i18n/i18next-config)
|
||||
- i18n context
|
||||
- `locale` - current locale code (ex `eu-US`, `zh-Hans`)
|
||||
- `i18n` - useless
|
||||
- `setLocaleOnClient` - used by App Boot and user change language
|
||||
|
||||
### load i18n resources
|
||||
|
||||
- client: i18n/i18next-config.ts
|
||||
- ns = camelCase(filename) (app-debug -> appDebug)
|
||||
- keys are flat (dot notation); `keySeparator: false`
|
||||
- ex: `app/components/datasets/create/embedding-process/index.tsx`
|
||||
- `const { t } = useTranslation('datasetSettings')`
|
||||
- `t('form.retrievalSetting.title')`
|
||||
- server: i18n/server.ts
|
||||
- ns = filename (kebab-case) mapped to camelCase namespace
|
||||
- ex: `app/(commonLayout)/datasets/(datasetDetailLayout)/[datasetId]/settings/page.tsx`
|
||||
- `const { t } = await getTranslation(locale, 'dataset-settings')`
|
||||
- `t('form.retrievalSetting.title')`
|
||||
|
||||
## TODO
|
||||
|
||||
- [ ] ts docs for useGetLanguage
|
||||
- [ ] ts docs for useLocale
|
||||
- [ ] client docs for i18n
|
||||
- [ ] server docs for i18n
|
||||
Loading…
Reference in New Issue
Block a user