dify/api/core/rag
Jyong 0340784d57 feat(knowledge-fs): add automatic metadata filtering to retrieval node
Bring the LLM-driven "automatic" metadata filtering mode from the legacy
Knowledge Retrieval node to the KnowledgeFS retrieval node
(knowledge-retrieval-v2).

Backend:
- Accept `automatic` in `metadata_filtering_mode` and add
  `metadata_model_config` to the node entity.
- Add `automatic_metadata_filter.py`, which reuses the legacy metadata
  filter prompt templates, invokes the configured model, parses the JSON
  answer and normalises operators/values against the KnowledgeFS
  metadata catalog before building custom metadata conditions.
- Resolve the shared metadata catalog across the selected spaces through
  the new `list_metadata_fields` app execution capability (paginated),
  fail closed on missing/unavailable models and catalog admission errors,
  fail open on extraction errors, and expose the outcome via
  `metrics.metadata_filtering` and `llm_usage`.
- Inject the default extractor from the node factory.
- Fix an unescaped brace in the legacy completion prompt template that
  broke `.format()`.

Frontend:
- Offer disabled/automatic/manual modes, normalise unknown persisted
  modes to disabled, store the automatic model and completion params,
  render the model selector through the shared MetadataFilter, and
  require a model when saving in automatic mode.

Co-Authored-By: Claude Code <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnUM1CdV5LpcxZEQ2vu9Xc
2026-09-02 02:55:12 -04:00
..
cleaner fix(rag): stop the document cleaner from stripping valid characters ï, ¿, ¾ (#39215) 2026-07-27 02:53:33 +00:00
data_post_processor feat: knowledge add more trace (#38959) 2026-07-24 02:27:16 +00:00
datasource Merge remote-tracking branch 'origin/main' into deploy/konwledge 2026-08-07 09:55:10 +08:00
docstore fix: restore upstream behavior after main merge 2026-08-11 10:23:44 +08:00
embedding fix: restore upstream behavior after main merge 2026-08-11 10:23:44 +08:00
entities refactor(api): migrate dataset endpoints to BaseModel (#37957) 2026-07-09 04:22:30 +00:00
extractor fix(api): attach exc_info=True to logger.warning sites that drop the traceback (#41068) 2026-08-21 11:20:42 +00:00
index_processor feat(api): track credit usage contexts (#41181) 2026-08-25 10:21:32 +00:00
models chore: reorg imports (#35308) 2026-04-16 08:50:02 +00:00
pipeline chore: use from __future__ import annotations (#30254) 2026-01-06 23:57:20 +09:00
rerank feat(api): track credit usage contexts (#41181) 2026-08-25 10:21:32 +00:00
retrieval feat(knowledge-fs): add automatic metadata filtering to retrieval node 2026-09-02 02:55:12 -04:00
splitter fix: preserve spaces during recursive text splitting (#39498) 2026-07-27 03:21:26 +00:00
summary_index feat(api): track credit usage contexts (#41181) 2026-08-25 10:21:32 +00:00
__init__.py Feat/dify rag (#2528) 2024-02-22 23:31:57 +08:00