opensource
/
dify
mirror of https://github.com/langgenius/dify.git
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
8,886 Commits 371 Branches 164 Tags 840 MiB
Commit Graph

3130 Commits

Author SHA1 Message Date
GareArc 80ee2e982e
fix(telemetry): prevent UUID validation error for tenant-prefixed storage IDs 
- get_ops_trace_instance was trying to query App table with storage_id format "tenant-{uuid}"
- This caused psycopg2.errors.InvalidTextRepresentation when app_id is None
- Added early return for tenant-prefixed storage identifiers to skip App lookup
- Enterprise telemetry still works correctly with these storage IDs
 2026-02-05 20:15:11 -08:00
GareArc 5bbc938a0d
fix(telemetry): add prompt generation trace emission for no_variable=false path 
- The no_variable=false code path in generate_rule_config was missing trace emission
- Added timing wrapper and _emit_prompt_generation_trace call to ensure metrics/logs are captured
- Trace now emitted on both success and failure cases for consistency with no_variable=true path
 2026-02-05 20:15:10 -08:00
GareArc 052f50805f
feat(telemetry): add node_execution_id and app_id support to trace metadata 
- Forward kwargs to message_trace to preserve node_execution_id
- Add node_execution_id extraction to all trace methods
- Add app_id parameter to prompt generation API endpoints
- Enable app_id tracing for rule_generate, code_generate, and structured_output operations
 2026-02-05 20:15:10 -08:00
GareArc f5043a8ac8
fix(telemetry): enable metrics and logs for standalone prompt generation 
Remove app_id parameter from three endpoints and update trace manager to use
tenant_id as storage identifier when app_id is unavailable. This allows
standalone prompt generation utilities to emit telemetry.

Changes:
- controllers/console/app/generator.py: Remove app_id=None from 3 endpoints
  (RuleGenerateApi, RuleCodeGenerateApi, RuleStructuredOutputGenerateApi)
- core/ops/ops_trace_manager.py: Use tenant_id fallback in send_to_celery
  - Extract tenant_id from task.kwargs when app_id is None
  - Use 'tenant-{tenant_id}' format as storage identifier
  - Skip traces only if neither app_id nor tenant_id available

The trace metadata still contains the actual tenant_id, so enterprise
telemetry correctly emits metrics and logs grouped by tenant.
 2026-02-05 20:15:10 -08:00
GareArc 22c8d8d772
feat(telemetry): add prompt generation telemetry to Enterprise OTEL 
- Add PromptGenerationTraceInfo trace entity with operation_type field
- Implement telemetry for rule-generate, code-generate, structured-output, instruction-modify operations
- Emit metrics: tokens (total/input/output), duration histogram, requests counter, errors counter
- Emit structured logs with model info and operation context
- Content redaction controlled by ENTERPRISE_INCLUDE_CONTENT env var
- Fix user_id propagation in TraceTask kwargs
- Fix latency calculation when llm_result is None

No spans exported - metrics and logs only for lightweight observability.
 2026-02-05 20:14:49 -08:00
GareArc 8ceb1ed96f
feat(telemetry): add input/output token split to enterprise OTEL traces 
- Add PROMPT_TOKENS and COMPLETION_TOKENS to WorkflowNodeExecutionMetadataKey
- Store prompt/completion tokens in node execution metadata JSON (no schema change)
- Calculate workflow-level token split by summing node executions on-the-fly
- Export gen_ai.usage.input_tokens and output_tokens to enterprise telemetry
- Add semantic convention constants for token attributes
- Maintain backward compatibility (historical data shows null)

BREAKING: None
MIGRATION: None (uses JSON metadata, no schema changes)
 2026-02-05 20:12:30 -08:00
GareArc 701f02f853
feat(telemetry): add invoked_by user tracking to enterprise OTEL 2026-02-05 20:12:29 -08:00
GareArc 3461c3a8ef
feat(enterprise): Add OTEL telemetry with slim traces, metrics, and structured logs 
- Add EnterpriseOtelTrace handler with span emission for workflows and nodes
- Implement minimal-span strategy: slim spans + detailed companion logs
- Add deterministic span/trace IDs for cross-workflow trace correlation
- Add metric collection at 100% accuracy (counters & histograms)
- Add event handlers for app lifecycle and feedback telemetry
- Add cross-workflow trace linking with parent context propagation
- Add OTEL exporter with configurable sampling and privacy controls
- Wire enterprise telemetry into workflow execution pipeline
- Add telemetry configuration in enterprise configs
 2026-02-05 20:12:28 -08:00
Asuka Minato 491fa9923b
refactor: port api/controllers/console/datasets/data_source.py /datasets/metadata.py /service_api/dataset/metadata.py /nodes/agent/agent_node.py api/core/workflow/nodes/datasource/datasource_node.py api/services/dataset_service.py to match case (#31836) 2026-02-02 21:03:16 +09:00
Asuka Minato ce2c41bbf5
refactor: port api/controllers/console/datasets/datasets_document.py api/controllers/service_api/app/annotation.py api/core/app/app_config/easy_ui_based_app/agent/manager.py api/core/app/apps/pipeline/pipeline_generator.py api/core/workflow/nodes/knowledge_retrieval/knowledge_retrieval_node.py to match case (#31832) 2026-02-02 19:07:30 +09:00
FFXN 41177757e6
fix: summary index bug (#31810) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jyong <76649700+JohnJyong@users.noreply.github.com>
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: Yansong Zhang <916125788@qq.com>
Co-authored-by: hj24 <mambahj24@gmail.com>
Co-authored-by: CodingOnStar <hanxujiang@dify.ai>
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-02-02 09:45:17 +08:00
yyh 4f826b4641
refactor(typing): use enum types for workflow status fields (#31792) 2026-02-02 09:41:34 +08:00
Asuka Minato 7828508b30
refactor: remove all reqparser (#29289) 
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Stephen Zhou <38493346+hyoban@users.noreply.github.com>
 2026-02-01 13:43:14 +09:00
盐粒 Yanli b8cb5f5ea2
refactor(typing): Fixup typing A2 - workflow engine & nodes (#31723) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
 2026-01-31 18:00:56 +09:00
盐粒 Yanli 5bc99995fc
fix(api): align graph protocols for response streaming (#31777) 2026-01-31 01:57:36 +09:00
lif 24b280a0ed
fix(i18n): improve Chinese translation of Max Tokens (#31771) 
Signed-off-by: majiayu000 <1835304752@qq.com>
 2026-01-30 20:19:35 +08:00
QuantumGhost 90fe9abab7
revert: revert human input relevant code (#31766) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-01-30 19:18:49 +08:00
QuantumGhost f90fa2b186
fix(api): fix workflow state persistence issue (#31752) 
Ensure workflow pause configuration is correctly set for all entrypoints.
 2026-01-30 17:44:29 +08:00
盐粒 Yanli 5a7dfd15b8
fix: Drain non-stream plugin chunk iterator (#31564) 2026-01-30 16:54:56 +08:00
Asuka Minato 89abea26f9
refactor: rm some dict api/controllers/console/app/generator.py api/core/llm_generator/llm_generator.py (#31709) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-01-30 17:37:20 +09:00
Jax 95d68437d1
fix(redis): Redis Cluster eval errors by adding hash tags to trigger debug keys (#31701) 2026-01-30 16:05:02 +08:00
QuantumGhost 03e3acfc71
feat(api): Human Input Node (backend part) (#31646) 
The backend part of the human in the loop (HITL) feature and relevant architecture / workflow engine changes.

Signed-off-by: yihong0618 <zouzou0208@gmail.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: 盐粒 Yanli <yanli@dify.ai>
Co-authored-by: CrabSAMA <40541269+CrabSAMA@users.noreply.github.com>
Co-authored-by: Stephen Zhou <38493346+hyoban@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: yihong <zouzou0208@gmail.com>
Co-authored-by: Joel <iamjoel007@gmail.com>
 2026-01-30 10:18:49 +08:00
盐粒 Yanli 5bf0251554
chore(typing): reduce ty excludes for A1 (#31721) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-01-30 02:38:57 +08:00
Nie Ronghua ceb6914793
refactor(model): Refactor plugin model schema cache to be process-global to prevent redundant Daemon API calls (#31689) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: -LAN- <laipz8200@outlook.com>
 2026-01-29 14:31:15 +08:00
盐粒 Yanli dbfc47e8b0
fix: SSRF in WordExtractor URL download (credit to @EaEa0001 ) (#31678) 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-29 14:01:21 +08:00
FFXN c2473d85dc
feat: Add summary index for knowledge. (#31625) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jyong <76649700+JohnJyong@users.noreply.github.com>
Co-authored-by: zxhlyh <jasonapring2015@outlook.com>
Co-authored-by: Yansong Zhang <916125788@qq.com>
Co-authored-by: hj24 <mambahj24@gmail.com>
Co-authored-by: CodingOnStar <hanxujiang@dify.ai>
Co-authored-by: CodingOnStar <hanxujiang@dify.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-29 13:47:35 +08:00
eux b48a10d7ec
feat(qdrant): implement full-text search with multi-keyword support (#31658) 2026-01-29 11:12:18 +08:00
fenglin 91532ef429
fix: add list type support for ToolInput constant value in tool node (#31276) 
Co-authored-by: qiaofenglin <qiaofenglin@baidu.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-29 10:49:29 +08:00
-LAN- 24ebe2f5c6
refactor(graph_engine): Add a Config class for graph engine. (#31663) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
 2026-01-28 19:57:55 +08:00
-LAN- 3d414678e3
fix(graph_engine): Cannot run single iteration or loop node (#31470) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
Co-authored-by: Yeuoly <45712896+Yeuoly@users.noreply.github.com>
 2026-01-28 01:05:59 +08:00
-LAN- d76ad15fca
refactor(graph_engine): move observability layer and persistence laye… (#31620) 2026-01-28 00:54:21 +08:00
heyszt eca26a9b9b
feat: Enhances OpenTelemetry node parsers (#30706) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-01-27 15:30:21 +08:00
E.G f6be9cd90d
refactor: replace request.args.get with Pydantic BaseModel validation (#31104) 
Co-authored-by: GlobalStar117 <GlobalStar117@users.noreply.github.com>
Co-authored-by: Asuka Minato <i@asukaminato.eu.org>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-27 10:48:42 +08:00
wangxiaolei e48419937b
feat: chatflow support multimodal (#31293) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-01-27 00:24:48 +08:00
盐粒 Yanli 92011d0a31
refactor: LLM plugin invoke parsing (#31499) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-01-26 14:59:57 +08:00
Asuka Minato b9f1d65d4f
refactor: example of refine dict / Mapping (#31498) 2026-01-26 10:23:38 +08:00
TomoOkuyama 0772d49257
fix(api): fix IRIS hybrid search returning zero results (#31309) 
Co-authored-by: Tomo Okuyama <tomo.okuyama@intersystems.com>
 2026-01-24 10:29:19 +08:00
-LAN- 67eb8c052d
refactor: single-node workflow runner helpers (#31472) 2026-01-24 10:27:44 +08:00
fenglin e8f9d64651
fix(tools): fix ToolInvokeMessage Union type parsing issue (#31450) 
Co-authored-by: qiaofenglin <qiaofenglin@baidu.com>
 2026-01-24 10:18:06 +08:00
-LAN- c575c34ca6
refactor: Move workflow node factory to app workflow (#31385) 
Signed-off-by: -LAN- <laipz8200@outlook.com>
 2026-01-22 18:08:21 +08:00
wangxiaolei a112caf5ec
fix: use thread local isolation the context (#31410) 2026-01-22 18:02:54 +08:00
zejiewang 811e43d0d4
fix: non-auto variable type params of agent node tool are not correctly parsed (#31128) 
Co-authored-by: wangzejie <wangzejie@meicai.cn>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-01-22 14:43:21 +08:00
wangxiaolei 211c57f7b6
fix: remove _try_resolve_user_from_request (#31360) 2026-01-21 21:19:11 +08:00
wangxiaolei 34436fc89c
feat: workflow support register context and read context (#31265) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maries <xh001x@hotmail.com>
 2026-01-21 14:31:47 +08:00
wangxiaolei 121d301a41
refactor: use session factory instead of call db.session directly (#31198) 
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
 2026-01-21 13:43:06 +08:00
wangxiaolei 0f1db88dcb
fix: fix dify-plugin-daemon error message (#31218) 2026-01-19 16:00:44 +08:00
wangxiaolei 3b225c01da
refactor: refactor workflow context (#30607) 2026-01-19 12:18:51 +08:00
Stream de610cbf39
fix: call `get_text_content()` instead of casting to `str` (#31121) 
Signed-off-by: Stream <Stream_2@qq.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-01-16 18:41:00 +09:00
wangxiaolei 2bfc54314e
feat: single run add opentelemetry (#31020) 2026-01-15 11:10:55 +08:00
wangxiaolei 98c88cec34
refactor: delete_endpoint should be idempotent (#30954) 2026-01-15 10:10:10 +08:00