Commit Graph

43 Commits

Author SHA1 Message Date
Stephen Zhou
6aba69326e
fix(knowledge-fs): show v2 workflow citations 2026-09-04 19:21:59 +08:00
Jyong
b79b8a879b feat(knowledge-fs): record and count every knowledge space query
Workflow KnowledgeFS retrieval nodes ran through `retrieval-tests`, which
never wrote an AnswerTrace or an overview event, so their queries were
missing from the retrieval history and from the space overview; the history
also only ever showed each member their own traces.

- AnswerTrace gains a `source` (retrieval_test | workflow | service_api |
  agent | mcp; migration 0051_answer_trace_source) derived from the
  Capability v2 caller kind. The retrieval-tests route records one trace per
  run (stages, evidence bundle, profile metadata), returns its id as
  `answerTraceId`, and the workflow node's failed-retrieval capture attaches
  to that trace instead of creating a second record.
- The quality trace list exposes and filters by `source`; traces from other
  caller kinds are visible to any current reader of the space, and counts and
  scores fall back to the evidence embedded in the trace when no bundle row
  exists.
- Overview accounting: retrieval-tests and Research tasks now emit
  `query.requested`, and the Research job state machine emits
  `query.completed` / `query.failed` on terminal stages (wired for both the
  in-process gateway and the durable runtime), so query volume, answer rate
  and outcomes include every caller. Activity details keep `source` and
  `taskKind`.
- Console and service trace routes accept a `source` filter; the retrieval
  test page shows a source badge and an all / retrieval tests / workflow
  filter, with translations for every locale.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-04 05:48:17 -04:00
FFXN
40e30d9fa6 feat(knowledge-fs): fix CI. 2026-09-04 16:55:47 +08:00
FFXN
ad059f55d4 feat(knowledge-fs): add namespace preview for initial website sources
- create website preview jobs before a knowledge space exists
- persist crawled page content in KnowledgeFS object storage
- import selected pages using preview job and page IDs
- validate preview ownership, expiry, and configuration fingerprint
- clean up preview content after cancellation, expiry, or consumption
- add namespace preview routes, capabilities, migrations, and contracts
- keep online document and drive flows unchanged
2026-09-04 15:52:36 +08:00
Jyong
afa771dcac feat(knowledge-fs): keep query images with retrieval test history
Query images attached to a retrieval test vanished as soon as the run's
history record was selected, and were never shown again after a reload,
because nothing persisted them: the gateway dropped `queryImages` when writing
`answer_traces`, the console trace/research responses had no image field, and
the composer cleared its images on every record selection.

- knowledge-fs: add a nullable JSON `query_images` column to `answer_traces`
  (migration 0049 for postgres and tidb), persist and read it in the answer
  trace repository, and expose `queryImages` on quality trace list items.
- api: enrich trace and research task responses with
  `KnowledgeFSQueryImageResponse`, resolving file names and short-lived signed
  preview URLs only for files the acting account still owns; regenerate the
  console and service contracts.
- web: keep composer images scoped to the selected record like the query
  text, remember images per run in this session, fall back to persisted
  previews (with a labelled placeholder when the file is gone), show them in
  the result panel and as a count on history records, and carry them into
  retests.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015zw5G5SX3HmVfnZof6YWAc
2026-09-02 09:17:36 -04:00
Jyong
4f3a2952d6 perf(knowledge-fs): harden cross-format document ingestion 2026-08-31 07:48:35 -04:00
FFXN
18155535cf refactor(knowledge_fs): replace credential service with dataset API key authorization. Remove the API token feature at the space level for knowledgeFs. 2026-08-26 15:40:18 +08:00
FFXN
013ddcf034 fix(knowledge_fs): accept direct unpublished profile activation responses and validate settings 2026-08-25 16:30:46 +08:00
Stephen Zhou
7c90d80bab
fix(knowledge-fs): allow downloading failed documents
Keep frontend download eligibility aligned with the backend and fetch only the latest failed revision asset.\n\nLinear: https://linear.app/dify/issue/WTA-2162/allow-downloading-documents-when-indexing-fails
2026-08-24 15:35:25 +08:00
Jyong
0391141936 feat(quality): show replay evidence details 2026-08-19 09:45:41 -04:00
Stephen Zhou
beddf6ef3e
fix(knowledge-fs): type replay query parameters 2026-08-19 18:23:57 +08:00
Jyong
66e3ae951f feat(knowledge): add golden question evaluations 2026-08-19 05:54:05 -04:00
Jyong
39c31ef46c fix(knowledge-fs): clarify golden question evidence editing 2026-08-19 02:40:20 -04:00
Jyong
5323b6e4fc Improve application workflows and user experience 2026-08-14 02:27:03 -04:00
FFXN
b9b6498125 fix(knowledge_fs): enhance active profile handling and add migration test 2026-08-13 16:27:47 +08:00
Stephen Zhou
9c5a38abbc
fix(knowledge-fs): unify incomplete model guidance (WTA-1987) 2026-08-13 12:59:27 +08:00
Jyong
b3a7c7a3e0 feat(knowledge-fs): unify public error handling 2026-08-12 07:00:59 -04:00
FFXN
7f0efa20fa feat(knowledge-fs): add bulk logical document deletion support 2026-08-11 19:47:12 +08:00
FFXN
0d4441e2cf feat(knowledge): support KnowledgeFS document downloads
- add single logical document download endpoint
- add batch ZIP download endpoint
- stream files from KnowledgeFS object storage
- validate batch size and deduplicate ZIP filenames
- add download service and facade unit tests
2026-08-11 16:29:23 +08:00
FFXN
26544610ff feat(knowledge_fs): add document availability management with bulk update support 2026-08-11 10:01:33 +08:00
Jyong
1ac8c9e434 feat(knowledge-fs): add durable metadata management 2026-08-10 02:55:09 -04:00
FFXN
cb49d83e9c feat(knowledge_fs): import selected Firecrawl pages during space creation
- include Firecrawl configuration and selected page URLs in space creation
- start a server-side crawl import after KnowledgeFS space provisioning
- require the second crawl to match the complete user selection
- keep the knowledge space valid when website content import fails
- reuse existing background task status and failure UI
- add KnowledgeFS crawl-import API and regression tests
2026-08-07 16:29:25 +08:00
Jyong
1648d8ac6b fix(knowledge-fs): handle initial revisions and published evidence 2026-08-02 14:47:09 -04:00
Jyong
d93902eaff fix(knowledge-fs): validate evidence match response 2026-08-02 11:56:25 -04:00
Jyong
4d5f845b8c Merge codex/golden-question-bulk-import into deploy/konwledge
# Conflicts:
#	api/knowledge-fs-contract.lock.json
#	api/services/knowledge_fs/data_facade.py
#	api/tests/unit_tests/services/test_knowledge_fs_data_facade.py
#	knowledge-fs/scripts/export-capability-v2-operations.test.mjs
#	packages/contracts/generated/api/console/knowledge-fs/orpc.gen.ts
2026-08-02 11:35:09 -04:00
Jyong
c9cdc34abc feat(dataset): add golden question bulk import 2026-08-02 11:29:11 -04:00
Jyong
737f315b53 Merge deploy/konwledge into codex/knowledge-fs-difyctl 2026-08-02 11:24:06 -04:00
Stephen Zhou
abee50b90e
fix(dataset): align New RAG activity filters 2026-07-31 17:00:59 +08:00
Stephen Zhou
e891790ce0
fix(dataset): connect New RAG overview data 2026-07-31 16:23:50 +08:00
Jyong
ce847f2b6f feat(cli): add KnowledgeFS filesystem commands 2026-07-31 04:11:14 -04:00
Stephen Zhou
13778d96a0
fix(dataset): wire New RAG document actions 2026-07-30 21:37:05 +08:00
Jyong
34f4e3b145 Update frontend components and application workflows 2026-07-29 06:18:05 -04:00
Jyong
6cfd1abb59 Refactor frontend architecture and simplify implementation 2026-07-29 02:30:27 -04:00
Stephen Zhou
20a06fdb94
feat(dataset): add New RAG quality control 2026-07-29 10:49:52 +08:00
Jyong
2674190bb4 fix(knowledge-fs): align retrieval profile payload alias 2026-07-28 09:21:40 -04:00
Stephen Zhou
d2380dd951
feat(dataset): add New RAG knowledge settings 2026-07-28 20:56:00 +08:00
Stephen Zhou
ab0b2282ab
fix(dataset): complete New RAG connected source flows 2026-07-28 12:58:18 +08:00
Stephen Zhou
7a20d54847
fix(ci): format KnowledgeFS data facade 2026-07-28 11:47:05 +08:00
Stephen Zhou
e5d4f37469
fix(dataset): restore New RAG document uploads 2026-07-27 21:46:16 +08:00
Stephen Zhou
3de75a3315
fix(dataset): stabilize New RAG KnowledgeFS flows 2026-07-27 16:18:51 +08:00
Stephen Zhou
1cde846bcf
fix(dataset): integrate New RAG with KnowledgeFS (#39621) 2026-07-27 12:52:44 +08:00
Jyong
90b6a7f012 Expand workflow features and update related components 2026-07-23 22:53:41 -04:00
Jyong
29858d0db0 Expand application capabilities and refactor core workflows 2026-07-22 05:09:09 -04:00