opensource/dify
2
0
Fork 0
mirror of https://github.com/langgenius/dify.git synced 2026-09-08 11:04:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into deploy/konwledge

Browse Source 
# Conflicts:
#	.agents/skills/how-to-write-component/references/data.md
#	.agents/skills/how-to-write-component/references/ownership.md
#	api/controllers/inner_api/__init__.py
#	api/tests/test_containers_integration_tests/services/test_account_service.py
#	api/tests/unit_tests/services/test_account_service.py
#	packages/iconify-collections/custom-public/icons.json
#	packages/iconify-collections/custom-public/info.json
#	web/features/new-rag/__tests__/new-knowledge-list.spec.tsx
#	web/features/new-rag/components/knowledge-space-card.tsx
#	web/features/new-rag/document-chunk-tree.tsx
#	web/features/tag-management/components/tag-search-content.tsx
#	web/features/tag-management/components/tag-selector.tsx
#	web/service/client.spec.ts
This commit is contained in:
Stephen Zhou 2026-09-04 19:42:31 +08:00
commit 38457f27fb
No known key found for this signature in database
887 changed files with 50678 additions and 17713 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
.agents/skills/frontend-code-review/references/accessibility-ui.md
View File

@ -2,6 +2,8 @@
Accessibility findings are first-class review findings. Treat broken keyboard access, missing accessible names, focus loss, and unreachable popup content as correctness bugs, not polish.
## Review Evidence
Before finalizing UI or accessibility findings, fetch the latest Web Interface Guidelines as a required baseline:
```text
@ -21,13 +23,26 @@ Flag:
- Clickable `div` or `span` used for actions.
- Router navigation implemented with button or `onClick` when a `Link` / `<a>` is the real semantic element.
- Icon-only buttons without `aria-label` or `aria-labelledby`.
- Icon-only controls without an accessible name; follow the naming rules below and the Dify UI `IconButton` contract.
- Decorative icons missing `aria-hidden="true"`.
- Images without `alt`; use `alt=""` only when truly decorative.
- Heading levels that skip hierarchy in page-level content.
Prefer semantic HTML before ARIA.
## Accessible Names And Descriptions
Read [Accessible names and descriptions] when a change affects labels, ARIA naming, help/error relationships, or hidden text. That document owns the shared implementation and review contract.
Flag violations supported by the final rendered behavior:
- Missing or insufficient names, redundant name overrides, or naming attributes prohibited by the element's role.
- Overrides that omit visible label wording or suppress necessary descendant information.
- Broken or stale label/description references, including relationships lost when responsive content or overlays unmount.
- Descriptions used instead of names, repeated help text, or essential structured content available only as a flattened description.
Inspect the computed name and description in the relevant state. Matching an `aria-label` string to nearby text alone does not prove redundancy.
## Keyboard And Focus
Flag:
@ -56,7 +71,7 @@ Flag:
- Placeholder text used as the only label.
- Password managers accidentally triggered on non-auth fields because autocomplete is missing or wrong.
Prefer visible labels. If visible surrounding text already labels the control, use a visually hidden label or a precise `aria-label`.
Prefer visible labels and associate them through the appropriate field primitive, a native `label`, or `aria-labelledby`. Do not duplicate an existing label with hidden text or `aria-label`; follow [Accessible names and descriptions] when no suitable visible label exists.
## Disabled, Loading, And Async States
@ -107,3 +122,5 @@ Flag:
- Images without dimensions.
- Loading copy using `...` instead of `…`.
- Hardcoded dates, times, numbers, or currency formats instead of `Intl.*`.
[Accessible names and descriptions]: ../../../../packages/dify-ui/docs/accessible-names-and-descriptions.md

4
.agents/skills/frontend-code-review/references/component-architecture.md
View File

@ -25,7 +25,9 @@ Flag:
- A page/tab-level section component becoming the data owner without needing a shared snapshot or shared loading/error/empty UI.
- Feature code promoted to shared only because it appears once or might be reused later.
Accept repeated TanStack Query calls in siblings when each component independently consumes the data. Cache deduplication is not a reason to hoist by itself.
Accept repeated TanStack Query hooks for the same key and input in Client Component siblings under one QueryClient;
shared cache is not a reason to hoist. Separate Server Component QueryClients do not share it, so request-level
deduplication needs an identified request-local cache or verified framework or transport owner.
## Component Boundaries

24
.agents/skills/frontend-code-review/references/data-query-contracts.md
View File

@ -22,10 +22,22 @@ Flag:
- Fake fallback IDs or placeholder inputs used to force a query to run.
- Query results copied into local state for rendering.
- Shared query behavior such as invalidation, stale defaults, or retry rules reimplemented at call sites.
- `prefetchQuery` treated as a hard gate or as returning data/errors to the caller.
- Deprecated imperative reads such as `fetchQuery`, `prefetchQuery`, `ensureQueryData`, or their infinite variants when
the current `query` or `infiniteQuery` contract applies.
Use `useQuery(consoleQuery.xxx.queryOptions(...))` or `useQuery(marketplaceQuery.xxx.queryOptions(...))` directly unless a feature hook performs real orchestration.
For imperative access, treat the choices as independent dimensions:
- `query` or `infiniteQuery` resolves the generated query and returns its data.
- `staleTime` decides whether cached data satisfies this call: `0` treats it as stale, a finite value accepts a freshness
window, `Infinity` accepts it until invalidation, and `'static'` accepts available data even after invalidation.
- `select` projects the resolved value without replacing cached query-function data. An imperative query defaults to no
retries when `retry` is not configured; `enabled` is observer-only, so guard before a conditional call.
- `await` blocks the current flow, `return` transfers the Promise to the caller, and `void` discards the result without
handling rejection. Handle rejection before discarding a potentially rejecting Promise; use `.catch(noop)` only for
intentional silence or feedback owned elsewhere, and preserve rejection for hard gates.
## Mutations
Flag:
@ -44,12 +56,18 @@ Flag:
- Request-time auth, setup, workspace role, or tenant decisions moved into static `next.config redirects()`.
- Dynamic role gates depending on `workspaces.current` implemented as static path redirects.
- Authorization logic depending on soft `prefetchQuery`.
- Authorization logic depending on an imperative query whose rejection is swallowed.
- Removing a client fallback before server API unavailable behavior is defined.
- Global placeholder query contracts introduced to solve a route-local Suspense issue.
- Branding-sensitive UI reading placeholder defaults without checking pending/placeholder state.
- A Server Component rendering or passing an imperative query result that the browser can independently revalidate,
leaving server and client output with different owners.
- A non-blocking Server Component query without pending-query dehydration, Next-compatible error redaction, a
`HydrationBoundary` covering the same-key client consumer, or an explicit Suspense and SSR-content decision.
Separate hard gates from soft prefetches. `fetchQuery` can be a server decision boundary; `prefetchQuery` is cache warmup.
Hard gates await `query` or `infiniteQuery` and preserve rejection; soft prefetches handle failure at the fallback owner.
Treat Server Components as prefetch-and-dehydrate owners by default, rendering returned data only under exclusive server
ownership or a freshness contract that prevents server/client drift.
## Workspace And Tenant

13
.agents/skills/how-to-write-component/references/data.md
View File

@ -20,7 +20,7 @@ Read this document when a component consumes generated contracts, nullable API v
- When a query supplies several sibling owners, shared derived facts, or workflow commands, model it as a graph node with field selectors. Do not keep it in a page and pass its data, query key, observer methods, and status fields back down as a deconstructed query object.
- For missing required input, branch the whole generated input with `skipToken`. Add `enabled` only for an independent execution condition; do not put `skipToken` inside a placeholder payload or coerce IDs to empty strings.
- Return generated `queryOptions()`, `infiniteOptions()`, or `mutationOptions()` directly from TanStack Query atoms. Pass supported options into the generated call instead of spreading into a parallel object.
- Share the exact options between prefetch and render when they represent the same request. Do not extract option helpers merely to reuse input construction.
- For the same logical request, preserve key, input, operation, and result contracts. Share the exact options between prefetch and render only when transport, context, and cache policy are shared; imperative and observer freshness may differ. Do not extract option helpers merely to reuse input construction.
- In Jotai-backed components, consume field-level selectors or named facts rather than the complete query result unless observer methods such as `refetch` or an infinite-scroll field group are part of that owner's contract.
- Keep observer methods inside the component that owns the query surface or behind a named graph command. A parent must not pass raw `refetch`, query keys, or invalidation details merely because a descendant action needs fresh data.
- Treat `refetch()` as an explicit execution command: it can run a query whose observer currently has `enabled: false`. Retry and refresh commands must recheck the permission, identity, or availability condition before including conditionally enabled queries.
@ -39,12 +39,21 @@ Read this document when a component consumes generated contracts, nullable API v
## Prefetch And Hidden Surfaces
- Prefetch expensive secondary content from the trigger or menu-open event when it benefits the visible path. Do not mount hidden subscribers solely to warm the cache.
- `prefetchQuery` is cache warmup, not an authorization or availability gate. Use a hard fetch boundary when the server must decide whether rendering may proceed.
- Use `query` or `infiniteQuery` for imperative access. `staleTime` defines cache acceptance; `select` projects the return
value without replacing cached query-function data. Imperative queries default to no retries when `retry` is not
configured, and observer-only `enabled` does not prevent an imperative call.
- `await` blocks, `return` transfers the Promise, and `void` discards its value without handling rejection. Handle
rejection before discarding a potentially rejecting Promise; use `.catch(noop)` only for intentional silence or
feedback owned elsewhere. Hard server gates await the query and preserve rejection.
## SSR, Authentication, And Workspace
- Static configuration owns path-invariant routing. Request-dependent authentication, setup, role, and tenant decisions belong to SSR or runtime decision boundaries.
- Distinguish soft SSR cache warming from authoritative decisions. Prefetched or placeholder data must not grant access or represent successful availability.
- Treat Server Components as query prefetch-and-dehydrate owners by default. Do not render or pass an imperative query
result when a browser observer can revalidate the same data unless ownership and freshness explicitly prevent drift.
- Non-blocking RSC streaming requires pending-query dehydration without redacting Next.js server errors, a
`HydrationBoundary` around the same-key client consumer, and Suspense when that content must be server-rendered.
- Never reuse tenant-scoped state after switching workspaces. Discard it at the switch boundary or isolate it by workspace identity.
- Do not make product or authorization decisions from bootstrap defaults. Wait for authoritative data, or render an explicit loading or error state.
- Keep loading and Suspense behavior inside the feature that owns the request. Do not add fake global data merely to bypass that boundary.

2
.agents/skills/how-to-write-component/references/ownership.md
View File

@ -19,6 +19,7 @@ Read this document when auditing, adding, moving, splitting, or refactoring Reac
- A page or feature root may wire route identity, providers, layout, navigation, and genuine cross-surface coordination. It must not call a child-specific state or query hook merely to assemble that child's props.
- Keep child contracts at the ownership boundary: stable domain identity, a small immutable snapshot, placement options, or named cross-boundary commands. Do not pass an internal state machine as separate `data`, `pending`, `error`, `retry`, `open`, setter, and callback props when the parent does not use them, and do not hide the same fan-out in a props bag or hook result object.
- Repeated TanStack Query calls in siblings are acceptable when each sibling independently consumes the data; the cache already deduplicates requests.
- Repeated TanStack Query hooks for the same key and input in Client Component siblings under one QueryClient share that cache. Separate Server Component QueryClients do not, so request-level deduplication needs an identified request-local cache or verified framework or transport owner.
- Treat parent input according to what the child boundary does with it:
- If the child only renders or performs a light local decision, pass the snapshot as props. It does not become a new state owner.
- If the child builds queries, dialogs, mutations, derivations, commands, or a reset lifecycle around a stable identity or snapshot, give that boundary a feature-local state file. This does not by itself require a new module or directory.
@ -26,6 +27,7 @@ Read this document when auditing, adding, moving, splitting, or refactoring Reac
- One pass-through layer is acceptable for stable identity and placement. It is not permission to relay workflow state and handlers through an unrelated component.
- Route identity may pass once from a framework route into its feature boundary. If multiple descendants, queries, facts, or commands need it, bridge it into the feature graph and stop passing it as props.
- A query snapshot may cross once as immutable display input. Query keys, observer methods, retry/loading/error groups, and invalidation mechanics belong to the query surface or feature graph and must not be decomposed into props.
- Do not replace prop drilling with one large view-model hook. Move each query, derived value, and handler to the concrete owner that consumes it.
- Keep source selection, defaults, validation, dirty checks, and payload shaping beside the workflow that owns submission.
## Boundaries

2
.github/ISSUE_TEMPLATE/bug_report.yml vendored
View File

@ -1,7 +1,7 @@
name: '🕷️ Bug report'
description: Report errors or unexpected behavior
labels:
- bug
- 🐞 bug
body:
- type: checkboxes
attributes:

2
.github/ISSUE_TEMPLATE/feature_request.yml vendored
View File

@ -1,7 +1,7 @@
name: '⭐ Feature or enhancement request'
description: Propose something new.
labels:
- enhancement
- 💪 enhancement
body:
- type: checkboxes
attributes:

8
.github/actions/setup-web/action.yml vendored
View File

@ -4,13 +4,13 @@ description: Set up Node.js, Vite+, pnpm, and web dependencies
runs:
using: composite
steps:
- name: Setup pnpm
uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
- name: Setup pnpm and Node.js
uses: pnpm/setup@703c52620218391530e48b9e8870d5c0082e1b9b # v2.1.0
with:
run_install: false
install: false
- name: Setup Vite+
uses: voidzero-dev/setup-vp@1b32467adbe183473499fd9d5d372c3ed9641754 # v1.18.0
with:
node-version-file: package.json
node-manager: false
cache: true
run-install: true

30
.github/workflows/api-tests.yml vendored
View File

@ -71,10 +71,10 @@ jobs:
retention-days: 1
api-integration:
name: API Integration Tests
runs-on: depot-ubuntu-24.04
name: API Integration Tests (${{ matrix.python-version }}, shard ${{ matrix.shardIndex }}/${{ matrix.shardTotal }})
runs-on: depot-ubuntu-24.04-4
env:
COVERAGE_FILE: coverage-integration
COVERAGE_FILE: coverage-integration-${{ matrix.python-version }}-${{ matrix.shardIndex }}
STORAGE_TYPE: opendal
OPENDAL_SCHEME: fs
OPENDAL_FS_ROOT: /tmp/dify-storage
@ -82,9 +82,17 @@ jobs:
run:
shell: bash
strategy:
fail-fast: false
matrix:
python-version:
- '3.12'
shardIndex:
- 1
- 2
- 3
- 4
shardTotal:
- 4
steps:
- name: Checkout code
@ -112,6 +120,8 @@ jobs:
-p no:benchmark \
--start-middleware \
-n auto \
--shard-index "${{ matrix.shardIndex }}" \
--shard-total "${{ matrix.shardTotal }}" \
--timeout "${PYTEST_TIMEOUT:-180}" \
api/tests/integration_tests/workflow \
api/tests/integration_tests/tools \
@ -120,8 +130,9 @@ jobs:
- name: Upload integration coverage data
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: api-coverage-integration
path: coverage-integration
name: api-coverage-integration-${{ matrix.python-version }}-${{ matrix.shardIndex }}
path: ${{ env.COVERAGE_FILE }}
if-no-files-found: error
retention-days: 1
api-coverage:
@ -171,12 +182,15 @@ jobs:
echo "" >> "$GITHUB_STEP_SUMMARY"
unit_coverage="$(find coverage-data -type f -name coverage-unit -print -quit)"
integration_coverage="$(find coverage-data -type f -name coverage-integration -print -quit)"
mapfile -t integration_coverage < <(find coverage-data -type f -name 'coverage-integration-*' -print | sort)
: "${unit_coverage:?coverage-unit artifact not found}"
: "${integration_coverage:?coverage-integration artifact not found}"
if [[ "${#integration_coverage[@]}" -ne 4 ]]; then
echo "expected 4 integration coverage artifacts, found ${#integration_coverage[@]}" >&2
exit 1
fi
report_file="$(mktemp)"
uv run --project api coverage combine "$unit_coverage" "$integration_coverage"
uv run --project api coverage combine "$unit_coverage" "${integration_coverage[@]}"
uv run --project api coverage report --show-missing | tee "$report_file"
echo "Summary: \`$(tail -n 1 "$report_file")\`" >> "$GITHUB_STEP_SUMMARY"
{

30
.github/workflows/cli-e2e.yml vendored
View File

@ -87,10 +87,9 @@ jobs:
with:
bun-version: latest
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
- uses: pnpm/setup@703c52620218391530e48b9e8870d5c0082e1b9b # v2.1.0
with:
package_json_field: packageManager
run_install: false
install: false
- name: Install CLI dependencies
working-directory: cli
@ -131,11 +130,6 @@ jobs:
- uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
package_json_field: packageManager
run_install: false
- run: pnpm install --frozen-lockfile
- run: pnpm tree:gen
- name: Run framework + output + error-handling
@ -181,11 +175,6 @@ jobs:
- uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
package_json_field: packageManager
run_install: false
- run: pnpm install --frozen-lockfile
- run: pnpm tree:gen
- name: Run discovery suite
@ -247,11 +236,6 @@ jobs:
- uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
package_json_field: packageManager
run_install: false
- run: pnpm install --frozen-lockfile
- run: pnpm tree:gen
- name: 'Run run/${{ matrix.name }}'
@ -312,11 +296,6 @@ jobs:
- uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
package_json_field: packageManager
run_install: false
- run: pnpm install --frozen-lockfile
- run: pnpm tree:gen
- name: Run auth/login + status + whoami
@ -371,11 +350,6 @@ jobs:
- uses: oven-sh/setup-bun@v2
with:
bun-version: latest
- uses: pnpm/action-setup@0977fd99725f1db4007ccb2928dbb4e90d06cc86 # v6.0.10
with:
package_json_field: packageManager
run_install: false
- run: pnpm install --frozen-lockfile
- run: pnpm tree:gen
- name: Run use / devices / logout / agent (serial)

70
.github/workflows/main-ci.yml vendored
View File

@ -37,8 +37,6 @@ jobs:
# Check which paths were changed to determine which tests to run
check-changes:
name: Check Changed Files
needs: pre_job
if: needs.pre_job.outputs.should_skip != 'true'
runs-on: depot-ubuntu-24.04
outputs:
api-changed: ${{ steps.changes.outputs.api }}
@ -49,6 +47,11 @@ jobs:
migration-changed: ${{ steps.changes.outputs.migration }}
sandbox-runtime-changed: ${{ steps.changes.outputs.sandbox-runtime }}
dify-agent-changed: ${{ steps.changes.outputs.dify-agent }}
python-style-changed: ${{ steps.changes.outputs.python-style }}
dify-agent-style-changed: ${{ steps.changes.outputs.dify-agent-style }}
web-style-changed: ${{ steps.changes.outputs.web-style }}
ts-common-style-changed: ${{ steps.changes.outputs.ts-common-style }}
superlinter-changed: ${{ steps.changes.outputs.superlinter }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dorny/paths-filter@ceb8a2b8f2d89434be7ff52d3de7ec3738c5cc9d # v4.0.3
@ -150,6 +153,58 @@ jobs:
- 'docker/generate_docker_compose'
- 'docker/ssrf_proxy/**'
- 'docker/volumes/sandbox/conf/**'
python-style:
- 'api/**'
- 'scripts/ast_grep_guard.py'
- 'scripts/check_no_new_getattr.py'
- 'scripts/check_no_new_controller_sqlalchemy.py'
- 'scripts/lint_controller_sqlalchemy.py'
- 'scripts/ast_grep_rules/no_new_getattr.yml'
- 'scripts/ast_grep_rules/no_new_controller_sqlalchemy.yml'
- '.github/workflows/style.yml'
- '.github/workflows/main-ci.yml'
dify-agent-style:
- 'dify-agent/**'
- '.github/workflows/style.yml'
- '.github/workflows/main-ci.yml'
web-style:
- 'web/**'
- 'e2e/**'
- 'sdks/nodejs-client/**'
- 'packages/**'
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- 'knip.config.ts'
- 'scripts/check-web-production-unused-after-knip-fix.mjs'
- '.github/workflows/style.yml'
- '.github/actions/setup-web/**'
ts-common-style:
- 'web/**'
- 'cli/**'
- 'e2e/**'
- 'sdks/nodejs-client/**'
- 'packages/**'
- 'package.json'
- 'pnpm-lock.yaml'
- 'pnpm-workspace.yaml'
- 'vite.config.ts'
- 'lint.config.ts'
- 'eslint.config.mjs'
- 'knip.config.ts'
- 'scripts/check-web-production-unused-after-knip-fix.mjs'
- 'oxlint-suppressions.json'
- 'eslint-suppressions.json'
- '.vscode/**'
- '.github/**'
superlinter:
- '**.sh'
- '**.yaml'
- '**.yml'
- '**Dockerfile'
- 'dev/**'
- '.editorconfig'
- '.vite-hooks/**'
# Run tests in parallel while always emitting stable required checks.
api-tests-run:
@ -396,10 +451,19 @@ jobs:
style-check:
name: Style Check
needs: pre_job
if: ${{ always() }}
needs:
- pre_job
- check-changes
uses: ./.github/workflows/style.yml
with:
base-rev: ${{ github.event.pull_request.base.sha || github.event.merge_group.base_sha }}
classification-succeeded: ${{ needs.check-changes.result == 'success' }}
run-python-style: ${{ needs.check-changes.outputs.python-style-changed == 'true' }}
run-dify-agent-style: ${{ needs.check-changes.outputs.dify-agent-style-changed == 'true' }}
run-web-style: ${{ needs.check-changes.outputs.web-style-changed == 'true' }}
run-ts-common-style: ${{ needs.check-changes.outputs.ts-common-style-changed == 'true' }}
run-superlinter: ${{ needs.check-changes.outputs.superlinter-changed == 'true' }}
skip-checks: ${{ needs.pre_job.outputs.should_skip == 'true' }}
vdb-tests-run:

160
.github/workflows/style.yml vendored
View File

@ -6,6 +6,30 @@ on:
base-rev:
required: true
type: string
classification-succeeded:
description: Whether the caller successfully classified the changed files.
required: true
type: boolean
run-python-style:
description: Run Python style checks for the changed files.
required: true
type: boolean
run-dify-agent-style:
description: Run Dify Agent style checks for the changed files.
required: true
type: boolean
run-web-style:
description: Run Web style checks for the changed files.
required: true
type: boolean
run-ts-common-style:
description: Run shared TypeScript style checks for the changed files.
required: true
type: boolean
run-superlinter:
description: Run SuperLinter for the changed files.
required: true
type: boolean
skip-checks:
description: Create the required check runs without repeating previously successful work.
required: false
@ -20,72 +44,72 @@ permissions:
jobs:
python-style:
name: Python Style
if: ${{ !inputs.skip-checks }}
if: ${{ !inputs.skip-checks && (!inputs.classification-succeeded || inputs.run-python-style || inputs.run-dify-agent-style) }}
runs-on: depot-ubuntu-24.04
steps:
- name: Verify changed-file classification
if: ${{ !inputs.classification-succeeded }}
run: |
echo "Changed-file classification failed; refusing to skip Python style checks." >&2
exit 1
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
fetch-depth: 0
- name: Check changed files
id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
api/**
scripts/ast_grep_guard.py
scripts/check_no_new_getattr.py
scripts/check_no_new_controller_sqlalchemy.py
scripts/lint_controller_sqlalchemy.py
scripts/ast_grep_rules/no_new_getattr.yml
scripts/ast_grep_rules/no_new_controller_sqlalchemy.yml
.github/workflows/style.yml
.github/workflows/main-ci.yml
- name: Setup UV and Python
if: steps.changed-files.outputs.any_changed == 'true'
uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
with:
enable-cache: false
python-version: '3.12'
cache-dependency-glob: api/uv.lock
cache-dependency-glob: |
api/uv.lock
dify-agent/uv.lock
- name: Install dependencies
if: steps.changed-files.outputs.any_changed == 'true'
if: inputs.run-python-style
run: uv sync --project api --dev
- name: Run Import Linter
if: steps.changed-files.outputs.any_changed == 'true'
if: inputs.run-python-style
run: uv run --directory api --dev lint-imports
- name: Run Response Contract Linter
if: steps.changed-files.outputs.any_changed == 'true'
if: inputs.run-python-style
run: uv run --project api --dev python api/dev/lint_response_contracts.py --fail-on-mismatch
- name: Run No New Getattr Guard
if: steps.changed-files.outputs.any_changed == 'true'
if: inputs.run-python-style
run: uv run --project api python scripts/check_no_new_getattr.py --base-rev "${{ inputs.base-rev }}"
- name: Run No New Controller SQLAlchemy Guard
if: steps.changed-files.outputs.any_changed == 'true'
if: inputs.run-python-style
run: uv run --project api python scripts/check_no_new_controller_sqlalchemy.py --base-rev "${{ inputs.base-rev }}"
- name: Run Type Checks
if: steps.changed-files.outputs.any_changed == 'true'
if: inputs.run-python-style
env:
PYREFLY_OUTPUT_FORMAT: github
run: make type-check-core
- name: Run Dify Agent Ruff
if: inputs.run-dify-agent-style
run: make -C dify-agent check
- name: Run Dify Agent Type Checks
if: inputs.run-dify-agent-style
run: make -C dify-agent typecheck
- name: Dotenv check
if: steps.changed-files.outputs.any_changed == 'true'
if: inputs.run-python-style
run: uv run --project api dotenv-linter ./api/.env.example ./web/.env.example
web-style:
name: Web Style
if: ${{ !inputs.skip-checks }}
if: ${{ !inputs.skip-checks && (!inputs.classification-succeeded || inputs.run-web-style) }}
runs-on: depot-ubuntu-24.04
defaults:
run:
@ -95,126 +119,84 @@ jobs:
pull-requests: read
steps:
- name: Verify changed-file classification
if: ${{ !inputs.classification-succeeded }}
working-directory: .
run: |
echo "Changed-file classification failed; refusing to skip Web style checks." >&2
exit 1
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Check changed files
id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
web/**
e2e/**
sdks/nodejs-client/**
packages/**
package.json
pnpm-lock.yaml
pnpm-workspace.yaml
knip.config.ts
scripts/check-web-production-unused-after-knip-fix.mjs
.github/workflows/style.yml
.github/actions/setup-web/**
- name: Setup web environment
if: steps.changed-files.outputs.any_changed == 'true'
uses: ./.github/actions/setup-web
- name: Web tsslint
if: steps.changed-files.outputs.any_changed == 'true'
env:
NODE_OPTIONS: --max-old-space-size=4096
run: vp run lint:tss
- name: Web dead code check
if: steps.changed-files.outputs.any_changed == 'true'
working-directory: .
run: vp run knip
- name: Web dead code check production
if: steps.changed-files.outputs.any_changed == 'true'
working-directory: .
run: vp run knip:production
- name: Web production unused declarations check
if: steps.changed-files.outputs.any_changed == 'true'
working-directory: .
run: vp run knip:production-unused-check
ts-common-style:
name: TS Common
if: ${{ !inputs.skip-checks }}
if: ${{ !inputs.skip-checks && (!inputs.classification-succeeded || inputs.run-ts-common-style) }}
runs-on: depot-ubuntu-24.04-4
permissions:
checks: write
pull-requests: read
steps:
- name: Verify changed-file classification
if: ${{ !inputs.classification-succeeded }}
run: |
echo "Changed-file classification failed; refusing to skip TypeScript style checks." >&2
exit 1
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Check changed files
id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
web/**
cli/**
e2e/**
sdks/nodejs-client/**
packages/**
package.json
pnpm-lock.yaml
pnpm-workspace.yaml
vite.config.ts
lint.config.ts
eslint.config.mjs
knip.config.ts
scripts/check-web-production-unused-after-knip-fix.mjs
oxlint-suppressions.json
eslint-suppressions.json
.vscode/**
.github/**
- name: Setup web environment
if: steps.changed-files.outputs.any_changed == 'true'
uses: ./.github/actions/setup-web
- name: Static check
if: steps.changed-files.outputs.any_changed == 'true'
run: pnpm -w check
superlinter:
name: SuperLinter
if: ${{ !inputs.skip-checks }}
if: ${{ !inputs.skip-checks && (!inputs.classification-succeeded || inputs.run-superlinter) }}
runs-on: depot-ubuntu-24.04
steps:
- name: Verify changed-file classification
if: ${{ !inputs.classification-succeeded }}
run: |
echo "Changed-file classification failed; refusing to skip SuperLinter." >&2
exit 1
- name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
persist-credentials: false
- name: Check changed files
id: changed-files
uses: tj-actions/changed-files@9426d40962ed5378910ee2e21d5f8c6fcbf2dd96 # v47.0.6
with:
files: |
**.sh
**.yaml
**.yml
**Dockerfile
dev/**
.editorconfig
.vite-hooks/**
- name: Super-linter
uses: super-linter/super-linter/slim@4ce20838b8ab83717e78138c5b3a1407148e0918 # v8.7.0
if: steps.changed-files.outputs.any_changed == 'true'
env:
BASH_SEVERITY: warning
DEFAULT_BRANCH: origin/main

20
api/.importlinter
View File

@ -203,6 +203,7 @@ forbidden_modules =
name = Account application services and contracts are framework and persistence neutral
type = forbidden
source_modules =
services.account_access_service
services.account_avatar_service
services.account_change_email_ports
services.account_change_email_service
@ -215,11 +216,14 @@ source_modules =
services.account_initialization_service
services.account_integration_service
services.account_login_service
services.account_oauth_service
services.account_password_service
services.account_ports
services.account_profile_service
services.entities.account_access_entities
services.entities.account_entities
services.entities.account_login_entities
services.entities.account_oauth_entities
services.entities.auth_audit_entities
forbidden_modules =
configs
@ -421,6 +425,22 @@ forbidden_modules =
sqlalchemy
werkzeug
[importlinter:contract:file-grant-service-boundary]
name = File grant application service is framework and persistence neutral
type = forbidden
source_modules =
services.file_grant_service
services.entities.file_grant_entities
forbidden_modules =
configs
controllers
extensions
flask
models
repositories
sqlalchemy
werkzeug
[importlinter:contract:account-activation-service-boundary]
name = Account activation application service is framework and persistence neutral
type = forbidden

2
api/README.md
View File

@ -98,7 +98,7 @@ The scripts resolve paths relative to their location, so you can run them from a
uv run pytest tests/integration_tests/ # Integration tests
# Code quality
./dev/reformat # Run all formatters and linters
../dev/reformat # Run all formatters and linters
uv run ruff check --fix ./ # Fix linting issues
uv run ruff format ./ # Format code
uv run pyrefly check # Type checking

36
api/conftest.py
View File

@ -33,6 +33,18 @@ ensure_backend_test_environment(_REPO_ROOT)
def pytest_addoption(parser: pytest.Parser) -> None:
group = parser.getgroup("dify")
group.addoption(
"--shard-index",
type=int,
default=1,
help="One-based index of the test shard to run.",
)
group.addoption(
"--shard-total",
type=int,
default=1,
help="Total number of test shards.",
)
group.addoption(
"--start-middleware",
action="store_true",
@ -58,9 +70,33 @@ def pytest_addoption(parser: pytest.Parser) -> None:
def pytest_configure(config: pytest.Config) -> None:
shard_index = config.getoption("shard_index")
shard_total = config.getoption("shard_total")
if shard_total < 1:
raise pytest.UsageError("--shard-total must be at least 1")
if not 1 <= shard_index <= shard_total:
raise pytest.UsageError("--shard-index must be between 1 and --shard-total")
config.stash[_DIFY_COMPOSE_STACKS_KEY] = []
def pytest_collection_modifyitems(config: pytest.Config, items: list[pytest.Item]) -> None:
"""Select a deterministic, evenly sized slice of collected tests for this shard."""
shard_index = config.getoption("shard_index")
shard_total = config.getoption("shard_total")
if shard_total == 1:
return
selected: list[pytest.Item] = []
deselected: list[pytest.Item] = []
for item_index, item in enumerate(items):
target = selected if item_index % shard_total == shard_index - 1 else deselected
target.append(item)
config.hook.pytest_deselected(items=deselected)
items[:] = selected
def pytest_sessionstart(session: pytest.Session) -> None:
config = session.config
if hasattr(config, "workerinput"):

1
api/constants/languages.py
View File

@ -8,6 +8,7 @@ language_timezone_mapping = {
"de-DE": "Europe/Berlin",
"ja-JP": "Asia/Tokyo",
"ko-KR": "Asia/Seoul",
"lo-LA": "Asia/Vientiane",
"ru-RU": "Europe/Moscow",
"it-IT": "Europe/Rome",
"uk-UA": "Europe/Kyiv",

522
api/controllers/console/app/statistic.py
View File

@ -1,31 +1,22 @@
from datetime import datetime
from decimal import Decimal
import sqlalchemy as sa
from flask import abort
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from werkzeug.exceptions import BadRequest
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
account_initialization_required,
model_validate,
rbac_permission_required,
setup_required,
with_current_user,
)
from core.app.entities.app_invoke_entities import InvokeFrom
from extensions.ext_database import db
from controllers.console.flask_admission import console_account_admission
from controllers.console.wraps import RBACPermission, RBACResourceScope, model_validate
from extensions.ext_application_services import application_services
from fields.base import ResponseModel
from libs.datetime_utils import parse_time_range
from libs.helper import convert_datetime_to_date, dump_response
from libs.login import login_required
from models import AppMode
from models.account import Account
from models.model import App
from libs.helper import dump_response
from libs.login import current_account_with_tenant
from machinery.context import RequestContext
from models.model import App, AppMode
class StatisticTimeRangeQuery(BaseModel):
@ -142,6 +133,20 @@ register_response_schema_models(
)
def _resolve_statistic_time_range(
req_data: StatisticTimeRangeQuery,
) -> tuple[datetime | None, datetime | None, str]:
timezone = current_account_with_tenant().account.timezone
assert timezone is not None
try:
start_date, end_date = parse_time_range(req_data.start, req_data.end, timezone)
except ValueError as error:
raise BadRequest(str(error)) from error
return start_date, end_date, timezone
@console_ns.route("/apps/<uuid:app_id>/statistics/daily-messages")
class DailyMessageStatistic(Resource):
@console_ns.doc("get_daily_message_statistics")
@ -152,52 +157,20 @@ class DailyMessageStatistic(Resource):
"Daily message statistics retrieved successfully",
console_ns.models[DailyMessageStatisticResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_MONITOR,
)
@get_app_model
@model_validate(StatisticTimeRangeQuery)
def get(self, req_data: StatisticTimeRangeQuery, account: Account, app_model: App):
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
COUNT(*) AS message_count
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :invoke_from"""
assert account.timezone is not None
arg_dict: dict[str, object] = {
"tz": account.timezone,
"app_id": app_model.id,
"invoke_from": InvokeFrom.DEBUGGER,
}
try:
start_datetime_utc, end_datetime_utc = parse_time_range(req_data.start, req_data.end, account.timezone)
except ValueError as e:
abort(400, description=str(e))
if start_datetime_utc:
sql_query += " AND created_at >= :start"
arg_dict["start"] = start_datetime_utc
if end_datetime_utc:
sql_query += " AND created_at < :end"
arg_dict["end"] = end_datetime_utc
sql_query += " GROUP BY date ORDER BY date"
response_data = []
with db.engine.begin() as conn:
rs = conn.execute(sa.text(sql_query), arg_dict)
for i in rs:
response_data.append({"date": str(i.date), "message_count": i.message_count})
def get(self, req_data: StatisticTimeRangeQuery, _request_context: RequestContext, app_model: App):
start_date, end_date, timezone = _resolve_statistic_time_range(req_data)
response_data = application_services().app_statistics.get_daily_messages(
app_id=app_model.id,
start_date=start_date,
end_date=end_date,
timezone=timezone,
)
return dump_response(DailyMessageStatisticResponse, {"data": response_data})
@ -212,51 +185,20 @@ class DailyConversationStatistic(Resource):
"Daily conversation statistics retrieved successfully",
console_ns.models[DailyConversationStatisticResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_MONITOR,
)
@get_app_model
@model_validate(StatisticTimeRangeQuery)
def get(self, req_data: StatisticTimeRangeQuery, account: Account, app_model: App):
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
COUNT(DISTINCT conversation_id) AS conversation_count
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :invoke_from"""
assert account.timezone is not None
arg_dict: dict[str, object] = {
"tz": account.timezone,
"app_id": app_model.id,
"invoke_from": InvokeFrom.DEBUGGER,
}
try:
start_datetime_utc, end_datetime_utc = parse_time_range(req_data.start, req_data.end, account.timezone)
except ValueError as e:
abort(400, description=str(e))
if start_datetime_utc:
sql_query += " AND created_at >= :start"
arg_dict["start"] = start_datetime_utc
if end_datetime_utc:
sql_query += " AND created_at < :end"
arg_dict["end"] = end_datetime_utc
sql_query += " GROUP BY date ORDER BY date"
response_data = []
with db.engine.begin() as conn:
rs = conn.execute(sa.text(sql_query), arg_dict)
for i in rs:
response_data.append({"date": str(i.date), "conversation_count": i.conversation_count})
def get(self, req_data: StatisticTimeRangeQuery, _request_context: RequestContext, app_model: App):
start_date, end_date, timezone = _resolve_statistic_time_range(req_data)
response_data = application_services().app_statistics.get_daily_conversations(
app_id=app_model.id,
start_date=start_date,
end_date=end_date,
timezone=timezone,
)
return dump_response(DailyConversationStatisticResponse, {"data": response_data})
@ -271,52 +213,20 @@ class DailyTerminalsStatistic(Resource):
"Daily terminal statistics retrieved successfully",
console_ns.models[DailyTerminalStatisticResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_MONITOR,
)
@get_app_model
@model_validate(StatisticTimeRangeQuery)
def get(self, req_data: StatisticTimeRangeQuery, account: Account, app_model: App):
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
COUNT(DISTINCT messages.from_end_user_id) AS terminal_count
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :invoke_from"""
assert account.timezone is not None
arg_dict: dict[str, object] = {
"tz": account.timezone,
"app_id": app_model.id,
"invoke_from": InvokeFrom.DEBUGGER,
}
try:
start_datetime_utc, end_datetime_utc = parse_time_range(req_data.start, req_data.end, account.timezone)
except ValueError as e:
abort(400, description=str(e))
if start_datetime_utc:
sql_query += " AND created_at >= :start"
arg_dict["start"] = start_datetime_utc
if end_datetime_utc:
sql_query += " AND created_at < :end"
arg_dict["end"] = end_datetime_utc
sql_query += " GROUP BY date ORDER BY date"
response_data = []
with db.engine.begin() as conn:
rs = conn.execute(sa.text(sql_query), arg_dict)
for i in rs:
response_data.append({"date": str(i.date), "terminal_count": i.terminal_count})
def get(self, req_data: StatisticTimeRangeQuery, _request_context: RequestContext, app_model: App):
start_date, end_date, timezone = _resolve_statistic_time_range(req_data)
response_data = application_services().app_statistics.get_daily_terminals(
app_id=app_model.id,
start_date=start_date,
end_date=end_date,
timezone=timezone,
)
return dump_response(DailyTerminalStatisticResponse, {"data": response_data})
@ -331,55 +241,20 @@ class DailyTokenCostStatistic(Resource):
"Daily token cost statistics retrieved successfully",
console_ns.models[DailyTokenCostStatisticResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_MONITOR,
)
@get_app_model
@model_validate(StatisticTimeRangeQuery)
def get(self, req_data: StatisticTimeRangeQuery, account: Account, app_model: App):
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
(SUM(messages.message_tokens) + SUM(messages.answer_tokens)) AS token_count,
SUM(total_price) AS total_price
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :invoke_from"""
assert account.timezone is not None
arg_dict: dict[str, object] = {
"tz": account.timezone,
"app_id": app_model.id,
"invoke_from": InvokeFrom.DEBUGGER,
}
try:
start_datetime_utc, end_datetime_utc = parse_time_range(req_data.start, req_data.end, account.timezone)
except ValueError as e:
abort(400, description=str(e))
if start_datetime_utc:
sql_query += " AND created_at >= :start"
arg_dict["start"] = start_datetime_utc
if end_datetime_utc:
sql_query += " AND created_at < :end"
arg_dict["end"] = end_datetime_utc
sql_query += " GROUP BY date ORDER BY date"
response_data = []
with db.engine.begin() as conn:
rs = conn.execute(sa.text(sql_query), arg_dict)
for i in rs:
response_data.append(
{"date": str(i.date), "token_count": i.token_count, "total_price": i.total_price, "currency": "USD"}
)
def get(self, req_data: StatisticTimeRangeQuery, _request_context: RequestContext, app_model: App):
start_date, end_date, timezone = _resolve_statistic_time_range(req_data)
response_data = application_services().app_statistics.get_daily_token_costs(
app_id=app_model.id,
start_date=start_date,
end_date=end_date,
timezone=timezone,
)
return dump_response(DailyTokenCostStatisticResponse, {"data": response_data})
@ -394,71 +269,20 @@ class AverageSessionInteractionStatistic(Resource):
"Average session interaction statistics retrieved successfully",
console_ns.models[AverageSessionInteractionStatisticResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_MONITOR,
)
@get_app_model(mode=[AppMode.CHAT, AppMode.AGENT_CHAT, AppMode.ADVANCED_CHAT, AppMode.AGENT])
@model_validate(StatisticTimeRangeQuery)
def get(self, req_data: StatisticTimeRangeQuery, account: Account, app_model: App):
converted_created_at = convert_datetime_to_date("c.created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
AVG(subquery.message_count) AS interactions
FROM
(
SELECT
m.conversation_id,
COUNT(m.id) AS message_count
FROM
conversations c
JOIN
messages m
ON c.id = m.conversation_id
WHERE
c.app_id = :app_id
AND m.invoke_from != :invoke_from"""
assert account.timezone is not None
arg_dict: dict[str, object] = {
"tz": account.timezone,
"app_id": app_model.id,
"invoke_from": InvokeFrom.DEBUGGER,
}
try:
start_datetime_utc, end_datetime_utc = parse_time_range(req_data.start, req_data.end, account.timezone)
except ValueError as e:
abort(400, description=str(e))
if start_datetime_utc:
sql_query += " AND c.created_at >= :start"
arg_dict["start"] = start_datetime_utc
if end_datetime_utc:
sql_query += " AND c.created_at < :end"
arg_dict["end"] = end_datetime_utc
sql_query += """
GROUP BY m.conversation_id
) subquery
LEFT JOIN
conversations c
ON c.id = subquery.conversation_id
GROUP BY
date
ORDER BY
date"""
response_data = []
with db.engine.begin() as conn:
rs = conn.execute(sa.text(sql_query), arg_dict)
for i in rs:
response_data.append(
{"date": str(i.date), "interactions": float(i.interactions.quantize(Decimal("0.01")))}
)
def get(self, req_data: StatisticTimeRangeQuery, _request_context: RequestContext, app_model: App):
start_date, end_date, timezone = _resolve_statistic_time_range(req_data)
response_data = application_services().app_statistics.get_average_session_interactions(
app_id=app_model.id,
start_date=start_date,
end_date=end_date,
timezone=timezone,
)
return dump_response(AverageSessionInteractionStatisticResponse, {"data": response_data})
@ -473,61 +297,20 @@ class UserSatisfactionRateStatistic(Resource):
"User satisfaction rate statistics retrieved successfully",
console_ns.models[UserSatisfactionRateStatisticResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_MONITOR,
)
@get_app_model
@model_validate(StatisticTimeRangeQuery)
def get(self, req_data: StatisticTimeRangeQuery, account: Account, app_model: App):
converted_created_at = convert_datetime_to_date("m.created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
COUNT(m.id) AS message_count,
COUNT(mf.id) AS feedback_count
FROM
messages m
LEFT JOIN
message_feedbacks mf
ON mf.message_id=m.id AND mf.rating='like'
WHERE
m.app_id = :app_id
AND m.invoke_from != :invoke_from"""
assert account.timezone is not None
arg_dict: dict[str, object] = {
"tz": account.timezone,
"app_id": app_model.id,
"invoke_from": InvokeFrom.DEBUGGER,
}
try:
start_datetime_utc, end_datetime_utc = parse_time_range(req_data.start, req_data.end, account.timezone)
except ValueError as e:
abort(400, description=str(e))
if start_datetime_utc:
sql_query += " AND m.created_at >= :start"
arg_dict["start"] = start_datetime_utc
if end_datetime_utc:
sql_query += " AND m.created_at < :end"
arg_dict["end"] = end_datetime_utc
sql_query += " GROUP BY date ORDER BY date"
response_data = []
with db.engine.begin() as conn:
rs = conn.execute(sa.text(sql_query), arg_dict)
for i in rs:
response_data.append(
{
"date": str(i.date),
"rate": round((i.feedback_count * 1000 / i.message_count) if i.message_count > 0 else 0, 2),
}
)
def get(self, req_data: StatisticTimeRangeQuery, _request_context: RequestContext, app_model: App):
start_date, end_date, timezone = _resolve_statistic_time_range(req_data)
response_data = application_services().app_statistics.get_user_satisfaction_rates(
app_id=app_model.id,
start_date=start_date,
end_date=end_date,
timezone=timezone,
)
return dump_response(UserSatisfactionRateStatisticResponse, {"data": response_data})
@ -542,52 +325,20 @@ class AverageResponseTimeStatistic(Resource):
"Average response time statistics retrieved successfully",
console_ns.models[AverageResponseTimeStatisticResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_MONITOR,
)
@get_app_model(mode=AppMode.COMPLETION)
@model_validate(StatisticTimeRangeQuery)
def get(self, req_data: StatisticTimeRangeQuery, account: Account, app_model: App):
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
AVG(provider_response_latency) AS latency
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :invoke_from"""
assert account.timezone is not None
arg_dict: dict[str, object] = {
"tz": account.timezone,
"app_id": app_model.id,
"invoke_from": InvokeFrom.DEBUGGER,
}
try:
start_datetime_utc, end_datetime_utc = parse_time_range(req_data.start, req_data.end, account.timezone)
except ValueError as e:
abort(400, description=str(e))
if start_datetime_utc:
sql_query += " AND created_at >= :start"
arg_dict["start"] = start_datetime_utc
if end_datetime_utc:
sql_query += " AND created_at < :end"
arg_dict["end"] = end_datetime_utc
sql_query += " GROUP BY date ORDER BY date"
response_data = []
with db.engine.begin() as conn:
rs = conn.execute(sa.text(sql_query), arg_dict)
for i in rs:
response_data.append({"date": str(i.date), "latency": round(i.latency * 1000, 4)})
def get(self, req_data: StatisticTimeRangeQuery, _request_context: RequestContext, app_model: App):
start_date, end_date, timezone = _resolve_statistic_time_range(req_data)
response_data = application_services().app_statistics.get_average_response_times(
app_id=app_model.id,
start_date=start_date,
end_date=end_date,
timezone=timezone,
)
return dump_response(AverageResponseTimeStatisticResponse, {"data": response_data})
@ -602,54 +353,19 @@ class TokensPerSecondStatistic(Resource):
"Tokens per second statistics retrieved successfully",
console_ns.models[TokensPerSecondStatisticResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_MONITOR)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_MONITOR,
)
@get_app_model
@model_validate(StatisticTimeRangeQuery)
def get(self, req_data: StatisticTimeRangeQuery, account: Account, app_model: App):
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
CASE
WHEN SUM(provider_response_latency) = 0 THEN 0
ELSE (SUM(answer_tokens) / SUM(provider_response_latency))
END as tokens_per_second
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :invoke_from"""
assert account.timezone is not None
arg_dict: dict[str, object] = {
"tz": account.timezone,
"app_id": app_model.id,
"invoke_from": InvokeFrom.DEBUGGER,
}
try:
start_datetime_utc, end_datetime_utc = parse_time_range(req_data.start, req_data.end, account.timezone)
except ValueError as e:
abort(400, description=str(e))
if start_datetime_utc:
sql_query += " AND created_at >= :start"
arg_dict["start"] = start_datetime_utc
if end_datetime_utc:
sql_query += " AND created_at < :end"
arg_dict["end"] = end_datetime_utc
sql_query += " GROUP BY date ORDER BY date"
response_data = []
with db.engine.begin() as conn:
rs = conn.execute(sa.text(sql_query), arg_dict)
for i in rs:
response_data.append({"date": str(i.date), "tps": round(i.tokens_per_second, 4)})
def get(self, req_data: StatisticTimeRangeQuery, _request_context: RequestContext, app_model: App):
start_date, end_date, timezone = _resolve_statistic_time_range(req_data)
response_data = application_services().app_statistics.get_tokens_per_second(
app_id=app_model.id,
start_date=start_date,
end_date=end_date,
timezone=timezone,
)
return dump_response(TokensPerSecondStatisticResponse, {"data": response_data})

62
api/controllers/console/app/workflow_app_log.py
View File

@ -4,29 +4,25 @@ from typing import Any
from dateutil.parser import isoparse
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy.orm import sessionmaker
from controllers.common.schema import query_params_from_model, register_schema_models
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model
from controllers.console.flask_admission import console_account_admission
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
account_initialization_required,
model_validate,
rbac_permission_required,
setup_required,
)
from extensions.ext_database import db
from extensions.ext_application_services import application_services
from fields.base import ResponseModel
from fields.end_user_fields import SimpleEndUser
from fields.member_fields import SimpleAccount
from graphon.enums import WorkflowExecutionStatus
from libs.helper import to_timestamp
from libs.login import login_required
from libs.helper import dump_response, to_timestamp
from machinery.context import RequestContext
from models import App
from models.model import AppMode
from services.workflow_app_service import WorkflowAppService
class WorkflowAppLogQuery(BaseModel):
@ -136,34 +132,32 @@ class WorkflowAppLogApi(Resource):
"Workflow app logs retrieved successfully",
console_ns.models[WorkflowAppLogPaginationResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_LOG_AND_ANNOTATION)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_LOG_AND_ANNOTATION,
)
@get_app_model(mode=[AppMode.WORKFLOW])
@model_validate(WorkflowAppLogQuery)
def get(self, req_data: WorkflowAppLogQuery, app_model: App):
def get(
self,
req_data: WorkflowAppLogQuery,
_request_context: RequestContext,
app_model: App,
):
"""
Get workflow app logs
"""
# get paginate workflow app logs
workflow_app_service = WorkflowAppService()
with sessionmaker(db.engine, expire_on_commit=False).begin() as session:
workflow_app_log_pagination = workflow_app_service.get_paginate_workflow_app_logs(
session=session,
app_model=app_model,
keyword=req_data.keyword,
status=req_data.status,
created_at_before=req_data.created_at__before,
created_at_after=req_data.created_at__after,
page=req_data.page,
limit=req_data.limit,
detail=req_data.detail,
created_by_end_user_session_id=req_data.created_by_end_user_session_id,
created_by_account=req_data.created_by_account,
)
return WorkflowAppLogPaginationResponse.model_validate(
workflow_app_log_pagination, from_attributes=True
).model_dump(mode="json")
result = application_services().workflow_app_logs.list_logs(
tenant_id=app_model.tenant_id,
app_id=app_model.id,
keyword=req_data.keyword,
status=req_data.status,
created_at_before=req_data.created_at__before,
created_at_after=req_data.created_at__after,
page=req_data.page,
limit=req_data.limit,
detail=req_data.detail,
created_by_end_user_session_id=req_data.created_by_end_user_session_id,
created_by_account=req_data.created_by_account,
)
return dump_response(WorkflowAppLogPaginationResponse, result)

297
api/controllers/console/app/workflow_run.py
View File

@ -3,26 +3,19 @@ from uuid import UUID
from flask_restx import Resource
from pydantic import BaseModel, Field, field_validator
from sqlalchemy.orm import sessionmaker
from configs import dify_config
from controllers.common.errors import NotFoundError
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.app.wraps import get_app_model
from controllers.console.flask_admission import console_account_admission
from controllers.console.wraps import (
RBACPermission,
RBACResourceScope,
account_initialization_required,
model_validate,
rbac_permission_required,
setup_required,
with_current_tenant_id,
with_current_user,
)
from core.workflow.human_input_forms import load_form_tokens_by_form_id as _load_form_tokens_by_form_id
from core.workflow.nodes.human_input.pause_reason import HumanInputRequired
from extensions.ext_database import db
from extensions.ext_application_services import application_services
from fields.base import ResponseModel
from fields.workflow_run_fields import (
AdvancedChatWorkflowRunPaginationResponse,
@ -32,14 +25,11 @@ from fields.workflow_run_fields import (
WorkflowRunNodeExecutionResponse,
WorkflowRunPaginationResponse,
)
from graphon.enums import WorkflowExecutionStatus
from libs.custom_inputs import time_duration
from libs.helper import uuid_value
from libs.login import login_required
from models import Account, App, AppMode, WorkflowRunTriggeredFrom
from models.workflow import WorkflowRun
from repositories.factory import DifyAPIRepositoryFactory
from services.workflow_run_service import WorkflowRunListArgs, WorkflowRunService
from libs.helper import dump_response, uuid_value
from machinery.context import RequestContext
from models import App, AppMode, WorkflowRunTriggeredFrom
from services.workflow_run_service import WorkflowRunListArgs
def _build_backstage_input_url(form_token: str | None) -> str | None:
@ -51,10 +41,6 @@ def _build_backstage_input_url(form_token: str | None) -> str | None:
return f"{base_url.rstrip('/')}/form/{form_token}"
# Workflow run status choices for filtering
WORKFLOW_RUN_STATUS_CHOICES = ["running", "succeeded", "failed", "stopped", "partial-succeeded"]
class WorkflowRunListQuery(BaseModel):
last_id: str | None = Field(default=None, description="Last run ID for pagination")
limit: int = Field(default=20, ge=1, le=100, description="Number of items per page (1-100)")
@ -96,6 +82,19 @@ class WorkflowRunCountQuery(BaseModel):
return time_duration(value)
def _workflow_run_list_args(req_data: WorkflowRunListQuery) -> WorkflowRunListArgs:
args: WorkflowRunListArgs = {"limit": req_data.limit}
if req_data.last_id is not None:
args["last_id"] = req_data.last_id
if req_data.status is not None:
args["status"] = req_data.status
return args
def _triggered_from(value: str | None) -> WorkflowRunTriggeredFrom:
return WorkflowRunTriggeredFrom(value) if value else WorkflowRunTriggeredFrom.DEBUGGING
class HumanInputPauseTypeResponse(ResponseModel):
type: Literal["human_input"]
form_id: str
@ -143,37 +142,24 @@ class AdvancedChatAppWorkflowRunListApi(Resource):
"Workflow runs retrieved successfully",
console_ns.models[AdvancedChatWorkflowRunPaginationResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_CREATE_AND_MANAGEMENT,
)
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
@model_validate(WorkflowRunListQuery)
def get(self, req_data: WorkflowRunListQuery, app_model: App):
def get(self, req_data: WorkflowRunListQuery, request_context: RequestContext, app_model: App):
"""
Get advanced chat app workflow run list
"""
args: WorkflowRunListArgs = {"limit": req_data.limit}
if req_data.last_id is not None:
args["last_id"] = req_data.last_id
if req_data.status is not None:
args["status"] = req_data.status
# Default to DEBUGGING if not specified
triggered_from = (
WorkflowRunTriggeredFrom(req_data.triggered_from)
if req_data.triggered_from
else WorkflowRunTriggeredFrom.DEBUGGING
result = application_services().workflow_runs.get_paginate_advanced_chat_workflow_runs(
request_context,
app_id=app_model.id,
args=_workflow_run_list_args(req_data),
triggered_from=_triggered_from(req_data.triggered_from),
)
workflow_run_service = WorkflowRunService()
result = workflow_run_service.get_paginate_advanced_chat_workflow_runs(
app_model=app_model, args=args, triggered_from=triggered_from
)
return AdvancedChatWorkflowRunPaginationResponse.model_validate(result, from_attributes=True).model_dump(
mode="json"
)
return dump_response(AdvancedChatWorkflowRunPaginationResponse, result)
@console_ns.route("/apps/<uuid:app_id>/advanced-chat/workflow-runs/count")
@ -187,34 +173,25 @@ class AdvancedChatAppWorkflowRunCountApi(Resource):
"Workflow runs count retrieved successfully",
console_ns.models[WorkflowRunCountResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_CREATE_AND_MANAGEMENT,
)
@get_app_model(mode=[AppMode.ADVANCED_CHAT])
@model_validate(WorkflowRunCountQuery)
def get(self, req_data: WorkflowRunCountQuery, app_model: App):
def get(self, req_data: WorkflowRunCountQuery, request_context: RequestContext, app_model: App):
"""
Get advanced chat workflow runs count statistics
"""
args = req_data.model_dump(exclude_none=True)
# Default to DEBUGGING if not specified
triggered_from = (
WorkflowRunTriggeredFrom(req_data.triggered_from)
if req_data.triggered_from
else WorkflowRunTriggeredFrom.DEBUGGING
result = application_services().workflow_runs.get_workflow_runs_count(
request_context,
app_id=app_model.id,
status=req_data.status,
time_range=req_data.time_range,
triggered_from=_triggered_from(req_data.triggered_from),
)
workflow_run_service = WorkflowRunService()
result = workflow_run_service.get_workflow_runs_count(
app_model=app_model,
status=args.get("status"),
time_range=args.get("time_range"),
triggered_from=triggered_from,
)
return WorkflowRunCountResponse.model_validate(result).model_dump(mode="json")
return dump_response(WorkflowRunCountResponse, result)
@console_ns.route("/apps/<uuid:app_id>/workflow-runs")
@ -228,35 +205,24 @@ class WorkflowRunListApi(Resource):
"Workflow runs retrieved successfully",
console_ns.models[WorkflowRunPaginationResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_CREATE_AND_MANAGEMENT,
)
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
@model_validate(WorkflowRunListQuery)
def get(self, req_data: WorkflowRunListQuery, app_model: App):
def get(self, req_data: WorkflowRunListQuery, request_context: RequestContext, app_model: App):
"""
Get workflow run list
"""
args: WorkflowRunListArgs = {"limit": req_data.limit}
if req_data.last_id is not None:
args["last_id"] = req_data.last_id
if req_data.status is not None:
args["status"] = req_data.status
# Default to DEBUGGING for workflow if not specified (backward compatibility)
triggered_from = (
WorkflowRunTriggeredFrom(req_data.triggered_from)
if req_data.triggered_from
else WorkflowRunTriggeredFrom.DEBUGGING
result = application_services().workflow_runs.get_paginate_workflow_runs(
request_context,
app_id=app_model.id,
args=_workflow_run_list_args(req_data),
triggered_from=_triggered_from(req_data.triggered_from),
)
workflow_run_service = WorkflowRunService()
result = workflow_run_service.get_paginate_workflow_runs(
app_model=app_model, args=args, triggered_from=triggered_from
)
return WorkflowRunPaginationResponse.model_validate(result, from_attributes=True).model_dump(mode="json")
return dump_response(WorkflowRunPaginationResponse, result)
@console_ns.route("/apps/<uuid:app_id>/workflow-runs/count")
@ -270,34 +236,25 @@ class WorkflowRunCountApi(Resource):
"Workflow runs count retrieved successfully",
console_ns.models[WorkflowRunCountResponse.__name__],
)
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_CREATE_AND_MANAGEMENT,
)
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
@model_validate(WorkflowRunCountQuery)
def get(self, req_data: WorkflowRunCountQuery, app_model: App):
def get(self, req_data: WorkflowRunCountQuery, request_context: RequestContext, app_model: App):
"""
Get workflow runs count statistics
"""
args = req_data.model_dump(exclude_none=True)
# Default to DEBUGGING for workflow if not specified (backward compatibility)
triggered_from = (
WorkflowRunTriggeredFrom(req_data.triggered_from)
if req_data.triggered_from
else WorkflowRunTriggeredFrom.DEBUGGING
result = application_services().workflow_runs.get_workflow_runs_count(
request_context,
app_id=app_model.id,
status=req_data.status,
time_range=req_data.time_range,
triggered_from=_triggered_from(req_data.triggered_from),
)
workflow_run_service = WorkflowRunService()
result = workflow_run_service.get_workflow_runs_count(
app_model=app_model,
status=args.get("status"),
time_range=args.get("time_range"),
triggered_from=triggered_from,
)
return WorkflowRunCountResponse.model_validate(result).model_dump(mode="json")
return dump_response(WorkflowRunCountResponse, result)
@console_ns.route("/apps/<uuid:app_id>/workflow-runs/<uuid:run_id>")
@ -311,23 +268,24 @@ class WorkflowRunDetailApi(Resource):
console_ns.models[WorkflowRunDetailResponse.__name__],
)
@console_ns.response(404, "Workflow run not found")
@setup_required
@login_required
@account_initialization_required
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_CREATE_AND_MANAGEMENT,
)
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
def get(self, app_model: App, run_id: UUID):
def get(self, request_context: RequestContext, app_model: App, run_id: UUID):
"""
Get workflow run detail
"""
run_id_str = str(run_id)
workflow_run_service = WorkflowRunService()
workflow_run = workflow_run_service.get_workflow_run(app_model=app_model, run_id=run_id_str)
workflow_run = application_services().workflow_runs.get_workflow_run(
request_context,
app_id=app_model.id,
run_id=str(run_id),
)
if workflow_run is None:
raise NotFoundError("Workflow run not found")
return WorkflowRunDetailResponse.model_validate(workflow_run, from_attributes=True).model_dump(mode="json")
return dump_response(WorkflowRunDetailResponse, workflow_run)
@console_ns.route("/apps/<uuid:app_id>/workflow-runs/<uuid:run_id>/node-executions")
@ -341,28 +299,22 @@ class WorkflowRunNodeExecutionListApi(Resource):
console_ns.models[WorkflowRunNodeExecutionListResponse.__name__],
)
@console_ns.response(404, "Workflow run not found")
@setup_required
@login_required
@account_initialization_required
@with_current_user
@rbac_permission_required(RBACResourceScope.APP, RBACPermission.APP_CREATE_AND_MANAGEMENT)
@console_account_admission(
rbac_resource_scope=RBACResourceScope.APP,
rbac_permission=RBACPermission.APP_CREATE_AND_MANAGEMENT,
)
@get_app_model(mode=[AppMode.ADVANCED_CHAT, AppMode.WORKFLOW])
def get(self, current_user: Account, app_model: App, run_id: UUID):
def get(self, request_context: RequestContext, app_model: App, run_id: UUID):
"""
Get workflow run node execution list
"""
run_id_str = str(run_id)
workflow_run_service = WorkflowRunService()
node_executions = workflow_run_service.get_workflow_run_node_executions(
app_model=app_model,
run_id=run_id_str,
user=current_user,
node_executions = application_services().workflow_runs.get_workflow_run_node_executions(
request_context,
app_id=app_model.id,
run_id=str(run_id),
)
return WorkflowRunNodeExecutionListResponse.model_validate(
{"data": node_executions}, from_attributes=True
).model_dump(mode="json")
return dump_response(WorkflowRunNodeExecutionListResponse, {"data": node_executions})
@console_ns.route("/workflow/<string:workflow_run_id>/pause-details")
@ -378,11 +330,8 @@ class ConsoleWorkflowPauseDetailsApi(Resource):
console_ns.models[WorkflowPauseDetailsResponse.__name__],
)
@console_ns.response(404, "Workflow run not found")
@setup_required
@login_required
@account_initialization_required
@with_current_tenant_id
def get(self, current_tenant_id: str, workflow_run_id: str):
@console_account_admission()
def get(self, request_context: RequestContext, workflow_run_id: str):
"""
Get workflow pause details.
@ -391,51 +340,31 @@ class ConsoleWorkflowPauseDetailsApi(Resource):
Returns information about why and where the workflow is paused.
"""
# Query WorkflowRun to determine if workflow is suspended
session_maker = sessionmaker(bind=db.engine)
workflow_run_repo = DifyAPIRepositoryFactory.create_api_workflow_run_repository(session_maker=session_maker)
workflow_run = db.session.get(WorkflowRun, workflow_run_id)
if not workflow_run:
details = application_services().workflow_runs.get_pause_details(
request_context,
workflow_run_id=workflow_run_id,
)
if details is None:
raise NotFoundError("Workflow run not found")
if workflow_run.tenant_id != current_tenant_id:
raise NotFoundError("Workflow run not found")
# Check if workflow is suspended
is_paused = workflow_run.status == WorkflowExecutionStatus.PAUSED
if not is_paused:
empty_response = WorkflowPauseDetailsResponse(paused_at=None, paused_nodes=[])
return empty_response.model_dump(mode="json"), 200
pause_entity = workflow_run_repo.get_workflow_pause(workflow_run_id)
pause_reasons = pause_entity.get_pause_reasons() if pause_entity else []
form_tokens_by_form_id = _load_form_tokens_by_form_id(
[reason.form_id for reason in pause_reasons if isinstance(reason, HumanInputRequired)]
return (
dump_response(
WorkflowPauseDetailsResponse,
{
"paused_at": details.paused_at.isoformat() + "Z" if details.paused_at else None,
"paused_nodes": [
{
"node_id": node.node_id,
"node_title": node.node_title,
"pause_type": {
"type": "human_input",
"form_id": node.form_id,
"backstage_input_url": _build_backstage_input_url(node.form_token),
},
}
for node in details.paused_nodes
],
},
),
200,
)
# Build response
paused_at = pause_entity.paused_at if pause_entity else None
paused_nodes: list[PausedNodeResponse] = []
for reason in pause_reasons:
if isinstance(reason, HumanInputRequired):
paused_nodes.append(
PausedNodeResponse(
node_id=reason.node_id,
node_title=reason.node_title,
pause_type=HumanInputPauseTypeResponse(
type="human_input",
form_id=reason.form_id,
backstage_input_url=_build_backstage_input_url(form_tokens_by_form_id.get(reason.form_id)),
),
)
)
else:
raise AssertionError("unimplemented.")
response = WorkflowPauseDetailsResponse(
paused_at=paused_at.isoformat() + "Z" if paused_at else None,
paused_nodes=paused_nodes,
)
return response.model_dump(mode="json"), 200

385
api/controllers/console/auth/oauth.py
View File

@ -1,47 +1,51 @@
import logging
import urllib.parse
import httpx
from flask import current_app, redirect, request
from flask import redirect, request
from flask_restx import Resource
from pydantic import BaseModel, Field
from werkzeug.exceptions import Unauthorized
from werkzeug.wrappers import Response
from configs import dify_config
from constants.languages import languages
from controllers.common.fields import RedirectResponse
from controllers.common.schema import query_params_from_model, register_response_schema_model, register_schema_models
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console.error import AccountInFreezeError, EmailDomainSuspendedError
from enums import DeploymentEdition
from extensions.ext_database import db
from libs.datetime_utils import naive_utc_now
from libs.helper import extract_remote_ip
from controllers.console.wraps import model_validate, setup_required, social_oauth_login_enabled
from extensions.ext_application_services import application_services
from fields.base import ResponseModel
from libs.helper import dump_response, extract_remote_ip
from libs.helper import timezone as validate_timezone_string
from libs.oauth import GitHubOAuth, GoogleOAuth, OAuthUserInfo, decode_oauth_state
from libs.oauth import decode_oauth_state
from libs.token import (
set_access_token_to_cookie,
set_csrf_token_to_cookie,
set_refresh_token_to_cookie,
)
from models import Account, AccountStatus
from services.account_service import AccountService, RegisterService, TenantService
from services.billing_service import BillingService
from services.errors.account import (
AccountNotFoundError,
AccountRegisterError,
SeatsLimitExceededError,
from services.account_errors import (
AccountEmailDomainSuspendedError,
AccountEmailFrozenError,
InvalidOAuthInvitationError,
InvalidOAuthProviderError,
OAuthAccountBannedError,
OAuthAccountNotFoundError,
OAuthIdentityLockUnavailableError,
OAuthInvitationAccountMismatchError,
OAuthProviderAuthorizationError,
OAuthProviderRequestError,
OAuthRegistrationError,
OAuthSeatsLimitExceededError,
OAuthWorkspaceCreationNotAllowedError,
)
from services.errors.account import (
EmailDomainSuspendedError as EmailDomainSuspendedRegistrationError,
from services.entities.account_entities import AccountSessionTokens
from services.entities.account_oauth_entities import (
OAuthAuthorizationRequest,
OAuthCallbackCommand,
OAuthCallbackResult,
OAuthInvitationResult,
)
from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkSpaceNotFoundError
from services.system_feature_service import SystemFeatureService
from .. import console_ns
logger = logging.getLogger(__name__)
class OAuthLoginQuery(BaseModel):
invite_token: str | None = Field(default=None, description="Optional invitation token")
@ -55,31 +59,12 @@ class OAuthCallbackQuery(BaseModel):
state: str | None = Field(default=None, description="OAuth state parameter")
class OAuthErrorResponse(ResponseModel):
error: str = Field(description="OAuth error message")
register_schema_models(console_ns, OAuthLoginQuery, OAuthCallbackQuery)
register_response_schema_model(console_ns, RedirectResponse)
def get_oauth_providers():
with current_app.app_context():
if not dify_config.GITHUB_CLIENT_ID or not dify_config.GITHUB_CLIENT_SECRET:
github_oauth = None
else:
github_oauth = GitHubOAuth(
client_id=dify_config.GITHUB_CLIENT_ID,
client_secret=dify_config.GITHUB_CLIENT_SECRET,
redirect_uri=dify_config.CONSOLE_API_URL + "/console/api/oauth/authorize/github",
)
if not dify_config.GOOGLE_CLIENT_ID or not dify_config.GOOGLE_CLIENT_SECRET:
google_oauth = None
else:
google_oauth = GoogleOAuth(
client_id=dify_config.GOOGLE_CLIENT_ID,
client_secret=dify_config.GOOGLE_CLIENT_SECRET,
redirect_uri=dify_config.CONSOLE_API_URL + "/console/api/oauth/authorize/google",
)
OAUTH_PROVIDERS = {"github": github_oauth, "google": google_oauth}
return OAUTH_PROVIDERS
register_response_schema_models(console_ns, RedirectResponse, OAuthErrorResponse)
def _validated_timezone(value: str | None) -> str | None:
@ -97,22 +82,33 @@ def _validated_language(value: str | None) -> str | None:
return None
def _url_origin(url: str) -> tuple[str, str, int] | None:
parsed_url = urllib.parse.urlsplit(url)
if parsed_url.scheme not in {"http", "https"} or parsed_url.hostname is None:
return None
try:
port = parsed_url.port
except ValueError:
return None
if port is None:
port = 443 if parsed_url.scheme == "https" else 80
return parsed_url.scheme, parsed_url.hostname, port
def _preferred_interface_language() -> str | None:
preferred_lang = request.accept_languages.best_match(languages)
if preferred_lang and preferred_lang in languages:
return preferred_lang
return None
def _get_redirect_target(redirect_url: str | None) -> str:
def _redirect_with_console_session(tokens: AccountSessionTokens, target_url: str) -> Response:
"""Attach application-issued Console session cookies to a redirect response."""
response = redirect(target_url)
set_access_token_to_cookie(request, response, tokens.access_token)
set_refresh_token_to_cookie(request, response, tokens.refresh_token)
set_csrf_token_to_cookie(request, response, tokens.csrf_token)
return response
def _oauth_callback_target(result: OAuthCallbackResult, requested_redirect: str | None) -> str:
if isinstance(result, OAuthInvitationResult):
query = urllib.parse.urlencode({"invite_token": result.invite_token})
return f"{dify_config.CONSOLE_WEB_URL}/signin/invite-settings?{query}"
target_url = _safe_console_redirect_target(requested_redirect)
query_char = "&" if "?" in target_url else "?"
return f"{target_url}{query_char}oauth_new_user={str(result.oauth_new_user).lower()}"
def _safe_console_redirect_target(redirect_url: str | None) -> str:
if not redirect_url:
return dify_config.CONSOLE_WEB_URL
@ -127,28 +123,22 @@ def _get_redirect_target(redirect_url: str | None) -> str:
return dify_config.CONSOLE_WEB_URL
def _preferred_interface_language(language: str | None = None) -> str:
if language:
return language
preferred_lang = request.accept_languages.best_match(languages)
if preferred_lang and preferred_lang in languages:
return preferred_lang
return languages[0]
def _url_origin(url: str) -> tuple[str, str, int] | None:
parsed_url = urllib.parse.urlsplit(url)
if parsed_url.scheme not in {"http", "https"} or parsed_url.hostname is None:
return None
try:
port = parsed_url.port
except ValueError:
return None
if port is None:
port = 443 if parsed_url.scheme == "https" else 80
return parsed_url.scheme, parsed_url.hostname, port
def _redirect_with_console_session(account: Account, target_url: str) -> Response:
"""Create a console session and attach its cookies to a redirect response."""
token_pair = AccountService.login(
account=account,
session=db.session(),
ip_address=extract_remote_ip(request),
)
response = redirect(target_url)
set_access_token_to_cookie(request, response, token_pair.access_token)
set_refresh_token_to_cookie(request, response, token_pair.refresh_token)
set_csrf_token_to_cookie(request, response, token_pair.csrf_token)
return response
def _signin_redirect(message: str, **params: str) -> Response:
query = urllib.parse.urlencode({"message": message, **params})
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?{query}")
@console_ns.route("/oauth/login/<provider>")
@ -158,24 +148,23 @@ class OAuthLogin(Resource):
@console_ns.doc(params={"provider": "OAuth provider name (github/google)"})
@console_ns.doc(params=query_params_from_model(OAuthLoginQuery))
@console_ns.response(302, "Redirect to OAuth authorization URL", console_ns.models[RedirectResponse.__name__])
@console_ns.response(400, "Invalid provider")
def get(self, provider: str):
invite_token = request.args.get("invite_token") or None
timezone = _validated_timezone(request.args.get("timezone") or None)
language = _validated_language(request.args.get("language") or None)
redirect_url = request.args.get("redirect_url") or None
OAUTH_PROVIDERS = get_oauth_providers()
with current_app.app_context():
oauth_provider = OAUTH_PROVIDERS.get(provider)
if not oauth_provider:
return {"error": "Invalid provider"}, 400
auth_url = oauth_provider.get_authorization_url(
invite_token=invite_token,
timezone=timezone,
language=language,
redirect_url=redirect_url,
)
@console_ns.response(400, "Invalid provider", console_ns.models[OAuthErrorResponse.__name__])
@setup_required
@social_oauth_login_enabled
@model_validate(OAuthLoginQuery)
def get(self, req_data: OAuthLoginQuery, provider: str):
try:
auth_url = application_services().accounts.oauth.start_authorization(
provider,
OAuthAuthorizationRequest(
invite_token=req_data.invite_token or None,
timezone=_validated_timezone(req_data.timezone),
language=_validated_language(req_data.language),
redirect_url=req_data.redirect_url or None,
),
)
except InvalidOAuthProviderError:
return dump_response(OAuthErrorResponse, {"error": "Invalid provider"}), 400
return redirect(auth_url)
@ -186,161 +175,53 @@ class OAuthCallback(Resource):
@console_ns.doc(params={"provider": "OAuth provider name (github/google)"})
@console_ns.doc(params=query_params_from_model(OAuthCallbackQuery))
@console_ns.response(302, "Redirect to console with access token", console_ns.models[RedirectResponse.__name__])
@console_ns.response(400, "OAuth process failed")
def get(self, provider: str):
OAUTH_PROVIDERS = get_oauth_providers()
with current_app.app_context():
oauth_provider = OAUTH_PROVIDERS.get(provider)
if not oauth_provider:
return {"error": "Invalid provider"}, 400
code = request.args.get("code")
state = request.args.get("state")
oauth_state = decode_oauth_state(state)
invite_token = oauth_state.get("invite_token")
timezone = _validated_timezone(oauth_state.get("timezone"))
language = _validated_language(oauth_state.get("language"))
redirect_url = oauth_state.get("redirect_url")
if not code:
return {"error": "Authorization code is required"}, 400
@console_ns.response(400, "OAuth process failed", console_ns.models[OAuthErrorResponse.__name__])
@setup_required
@social_oauth_login_enabled
@model_validate(OAuthCallbackQuery)
def get(self, req_data: OAuthCallbackQuery, provider: str):
oauth_state = decode_oauth_state(req_data.state)
try:
token = oauth_provider.get_access_token(code)
user_info = oauth_provider.get_user_info(token)
except httpx.RequestError as e:
error_text = str(e)
if isinstance(e, httpx.HTTPStatusError):
error_text = e.response.text
logger.exception("An error occurred during the OAuth process with %s: %s", provider, error_text)
return {"error": "OAuth process failed"}, 400
except ValueError as e:
logger.warning("OAuth error with %s", provider, exc_info=True)
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={urllib.parse.quote(str(e))}")
if invite_token and RegisterService.is_valid_invite_token(invite_token):
invitation = RegisterService.get_invitation_if_token_valid(
None,
None,
invite_token,
session=db.session(),
result = application_services().accounts.oauth.complete_authorization(
OAuthCallbackCommand(
provider=provider,
code=req_data.code,
invite_token=oauth_state.get("invite_token"),
timezone=_validated_timezone(oauth_state.get("timezone")),
language=_validated_language(oauth_state.get("language")),
browser_language=_preferred_interface_language(),
ip_address=extract_remote_ip(request),
)
)
if not invitation:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Invalid invitation token.")
if invitation["data"]["email"].lower() != user_info.email.lower():
message = "This invitation was sent to another account. Please sign in with the invited account."
query = urllib.parse.urlencode({"message": message, "invite_token": invite_token})
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?{query}")
account = invitation["account"]
if account.status == AccountStatus.BANNED:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Account is banned.")
AccountService.link_account_integrate(provider, user_info.id, account, session=db.session())
target_url = f"{dify_config.CONSOLE_WEB_URL}/signin/invite-settings?invite_token={invite_token}"
return _redirect_with_console_session(account, target_url)
try:
account, oauth_new_user = _generate_account(
provider,
user_info,
timezone=timezone,
language=language,
ip_address=extract_remote_ip(request),
except InvalidOAuthProviderError:
return dump_response(OAuthErrorResponse, {"error": "Invalid provider"}), 400
except (OAuthProviderRequestError, OAuthIdentityLockUnavailableError):
return dump_response(OAuthErrorResponse, {"error": "OAuth process failed"}), 400
except OAuthProviderAuthorizationError as exc:
return _signin_redirect(exc.description)
except InvalidOAuthInvitationError:
return _signin_redirect("Invalid invitation token.")
except OAuthInvitationAccountMismatchError as exc:
return _signin_redirect(
"This invitation was sent to another account. Please sign in with the invited account.",
invite_token=exc.invite_token,
)
except AccountNotFoundError:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Account not found.")
except (WorkSpaceNotFoundError, WorkSpaceNotAllowedCreateError):
return redirect(
f"{dify_config.CONSOLE_WEB_URL}/signin"
"?message=Workspace not found, please contact system admin to invite you to join in a workspace."
except OAuthAccountBannedError:
return _signin_redirect("Account is banned.")
except OAuthAccountNotFoundError:
return _signin_redirect("Account not found.")
except OAuthWorkspaceCreationNotAllowedError:
return _signin_redirect(
"Workspace not found, please contact system admin to invite you to join in a workspace."
)
except SeatsLimitExceededError:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Licensed seats limit exceeded.")
except EmailDomainSuspendedRegistrationError:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={EmailDomainSuspendedError.description}")
except AccountRegisterError as exc:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message={exc.description}")
except OAuthSeatsLimitExceededError:
return _signin_redirect("Licensed seats limit exceeded.")
except AccountEmailDomainSuspendedError:
return _signin_redirect(EmailDomainSuspendedError.description or "")
except AccountEmailFrozenError:
return _signin_redirect(AccountInFreezeError.description or "")
except OAuthRegistrationError as exc:
return _signin_redirect(exc.description)
# Check account status
if account.status == AccountStatus.BANNED:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Account is banned.")
if account.status == AccountStatus.PENDING:
account.status = AccountStatus.ACTIVE
account.initialized_at = naive_utc_now()
db.session.commit()
try:
TenantService.create_owner_tenant_if_not_exist(account, session=db.session())
except Unauthorized:
return redirect(f"{dify_config.CONSOLE_WEB_URL}/signin?message=Workspace not found.")
except WorkSpaceNotAllowedCreateError:
return redirect(
f"{dify_config.CONSOLE_WEB_URL}/signin"
"?message=Workspace not found, please contact system admin to invite you to join in a workspace."
)
target_url = _get_redirect_target(redirect_url)
query_char = "&" if "?" in target_url else "?"
target_url = f"{target_url}{query_char}oauth_new_user={str(oauth_new_user).lower()}"
return _redirect_with_console_session(account, target_url)
def _get_account_by_openid_or_email(provider: str, user_info: OAuthUserInfo) -> Account | None:
account: Account | None = Account.get_by_openid(provider, user_info.id)
if not account:
account = AccountService.get_account_by_email_with_case_fallback(user_info.email, session=db.session())
return account
def _generate_account(
provider: str,
user_info: OAuthUserInfo,
timezone: str | None = None,
language: str | None = None,
ip_address: str | None = None,
) -> tuple[Account, bool]:
# Get account by openid or email.
account = _get_account_by_openid_or_email(provider, user_info)
oauth_new_user = False
if account:
tenants = TenantService.get_join_tenants(account, session=db.session())
if not tenants:
if not SystemFeatureService.is_workspace_creation_allowed():
raise WorkSpaceNotAllowedCreateError()
else:
TenantService.create_owner_tenant(account, session=db.session())
if not account:
normalized_email = user_info.email.lower()
oauth_new_user = True
if not SystemFeatureService.is_registration_allowed():
if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.CLOUD:
freeze_type = BillingService.get_email_freeze_type(normalized_email)
if freeze_type:
if freeze_type == "email_domain_suspended":
raise EmailDomainSuspendedRegistrationError()
raise AccountRegisterError(description=AccountInFreezeError.description or "")
raise AccountRegisterError(description=("Invalid email or password"))
account_name = user_info.name or "Dify"
interface_language = _preferred_interface_language(language)
account = RegisterService.register(
email=normalized_email,
name=account_name,
password=None,
open_id=user_info.id,
provider=provider,
language=interface_language,
timezone=timezone,
ip_address=ip_address,
session=db.session(),
)
# Link account
AccountService.link_account_integrate(provider, user_info.id, account, session=db.session())
return account, oauth_new_user
target_url = _oauth_callback_target(result, oauth_state.get("redirect_url"))
return _redirect_with_console_session(result.tokens, target_url)

31
api/controllers/console/explore/recommended_app.py
View File

@ -7,17 +7,17 @@ from pydantic import BaseModel, Field, computed_field, field_validator
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.explore.error import RecommendedAppNotFoundError
from controllers.console.wraps import account_initialization_required, model_validate, with_current_user
from controllers.console.flask_admission import console_account_admission
from controllers.console.wraps import model_validate
from extensions.ext_application_services import application_services
from fields.base import ResponseModel
from libs.helper import build_icon_url, dump_response
from libs.login import login_required
from models import Account
from machinery.context import RequestContext
from services.recommended_app_query_service import RecommendedAppNotFoundError as RecommendedAppQueryNotFoundError
class RecommendedAppsQuery(BaseModel):
language: str | None = Field(default=None, description="Language code for recommended app localization")
language: str = Field(default="en-US", description="Language code for recommended app localization")
class RecommendedAppInfoResponse(ResponseModel):
@ -97,16 +97,13 @@ register_response_schema_models(
class RecommendedAppListApi(Resource):
@console_ns.doc(params=query_params_from_model(RecommendedAppsQuery))
@console_ns.response(200, "Success", console_ns.models[RecommendedAppListResponse.__name__])
@login_required
@account_initialization_required
@with_current_user
@console_account_admission()
@model_validate(RecommendedAppsQuery)
def get(self, req_data: RecommendedAppsQuery, current_user: Account):
def get(self, req_data: RecommendedAppsQuery, _request_context: RequestContext):
return dump_response(
RecommendedAppListResponse,
application_services().recommended_app_queries.list_recommended(
requested_language=req_data.language,
interface_language=current_user.interface_language,
language=req_data.language,
),
)
@ -115,16 +112,13 @@ class RecommendedAppListApi(Resource):
class LearnDifyAppListApi(Resource):
@console_ns.doc(params=query_params_from_model(RecommendedAppsQuery))
@console_ns.response(200, "Success", console_ns.models[LearnDifyAppListResponse.__name__])
@login_required
@account_initialization_required
@with_current_user
@console_account_admission()
@model_validate(RecommendedAppsQuery)
def get(self, req_data: RecommendedAppsQuery, current_user: Account):
def get(self, req_data: RecommendedAppsQuery, _request_context: RequestContext):
return dump_response(
LearnDifyAppListResponse,
application_services().recommended_app_queries.list_learn_dify(
requested_language=req_data.language,
interface_language=current_user.interface_language,
language=req_data.language,
),
)
@ -133,9 +127,8 @@ class LearnDifyAppListApi(Resource):
class RecommendedAppApi(Resource):
@console_ns.response(200, "Success", console_ns.models[RecommendedAppDetailResponse.__name__])
@console_ns.response(404, "Recommended app not found")
@login_required
@account_initialization_required
def get(self, app_id: UUID):
@console_account_admission()
def get(self, _request_context: RequestContext, app_id: UUID):
try:
result = application_services().recommended_app_queries.get_detail(str(app_id))
except RecommendedAppQueryNotFoundError:

17
api/controllers/console/notification.py
View File

@ -2,7 +2,7 @@ from flask_restx import Resource
from pydantic import BaseModel, Field
from controllers.common.fields import SimpleResultResponse
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.console import console_ns
from controllers.console.flask_admission import console_account_admission
from controllers.console.wraps import model_validate
@ -17,6 +17,10 @@ class DismissNotificationPayload(BaseModel):
notification_id: str = Field(...)
class NotificationQuery(BaseModel):
language: str = Field(default="en-US", description="Notification language")
class NotificationItemResponse(ResponseModel):
notification_id: str | None = None
frequency: str | None = None
@ -32,17 +36,19 @@ class NotificationResponse(ResponseModel):
notifications: list[NotificationItemResponse]
register_schema_models(console_ns, DismissNotificationPayload)
register_schema_models(console_ns, DismissNotificationPayload, NotificationQuery)
register_response_schema_models(console_ns, SimpleResultResponse, NotificationResponse)
@console_ns.route("/notification")
class NotificationApi(Resource):
@console_ns.doc("get_notification")
@console_ns.doc(params=query_params_from_model(NotificationQuery))
@console_ns.doc(
description=(
"Return the active in-product notification for the current user "
"in their interface language (falls back to English if unavailable). "
"in the requested language (defaults to English when omitted). "
"Unavailable translations fall back to English, then the first available content. "
"The notification is NOT marked as seen here; call POST /notification/dismiss "
"when the user explicitly closes the modal."
),
@ -53,8 +59,9 @@ class NotificationApi(Resource):
)
@console_ns.response(200, "Success", console_ns.models[NotificationResponse.__name__])
@console_account_admission(editions=frozenset({DeploymentEdition.CLOUD}))
def get(self, request_context: RequestContext):
result = application_services().notifications.get_active(request_context)
@model_validate(NotificationQuery)
def get(self, query: NotificationQuery, request_context: RequestContext):
result = application_services().notifications.get_active(request_context, query.language)
return dump_response(NotificationResponse, result), 200

10
api/controllers/console/socketio/workflow.py
View File

@ -93,12 +93,10 @@ def handle_collaboration_event(sid, data):
1. mouse_move
2. vars_and_features_update
3. sync_request (ask leader to update graph)
4. app_state_update
5. mcp_server_update
6. workflow_update
7. comments_update
8. node_panel_presence
9. graph_view_state (session reports tab visibility; drives leader election)
4. workflow_update
5. comments_update
6. node_panel_presence
7. graph_view_state (session reports tab visibility; drives leader election)
"""
return collaboration_service.relay_collaboration_event(sid, data)

2
api/controllers/console/workspace/models.py
View File

@ -349,6 +349,8 @@ class ModelProviderModelCredentialApi(Resource):
)
@setup_required
@login_required
@is_admin_or_owner_required
@rbac_permission_required(RBACResourceScope.WORKSPACE, RBACPermission.CREDENTIAL_MANAGE, resource_required=False)
@account_initialization_required
@with_current_user
@with_current_tenant_id

10
api/controllers/console/wraps.py
View File

@ -356,6 +356,16 @@ def email_password_login_enabled[**P, R](view: Callable[P, R]) -> Callable[P, R]
return decorated
def social_oauth_login_enabled[**P, R](view: Callable[P, R]) -> Callable[P, R]:
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs):
if not dify_config.ENABLE_SOCIAL_OAUTH_LOGIN:
abort(403)
return view(*args, **kwargs)
return decorated
def enable_change_email[**P, R](view: Callable[P, R]) -> Callable[P, R]:
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs):

3
api/controllers/files/__init__.py
View File

@ -14,12 +14,13 @@ api = ExternalApi(
files_ns = Namespace("files", description="File operations", path="/")
from . import image_preview, tool_files, upload
from . import appdeploy_files, image_preview, tool_files, upload
api.add_namespace(files_ns)
__all__ = [
"api",
"appdeploy_files",
"bp",
"files_ns",
"image_preview",

401
api/controllers/files/appdeploy_files.py Normal file
View File

@ -0,0 +1,401 @@
"""File endpoints reached with an AppDeploy file grant.
Upload, remote-upload, produce, and resolve authenticate with a Bearer grant;
content authenticates with a per-file token in the query string, because an
``<img src>`` cannot carry a header.
They stay on the public ``files`` blueprint rather than moving under
``inner_api``, whose contract is a fully trusted caller holding the master key.
None of the five meets it: ``content`` is a browser-facing surface, ``upload``
carries an end user's payload, and ``produced`` and ``resolve`` are called by
workers executing third-party plugin code. The one genuinely server-to-server
step, minting the grant, already lives in ``inner_api``.
"""
from __future__ import annotations
from typing import IO
from urllib.parse import quote
from uuid import UUID
from flask import Response, request
from flask_restx import Resource
from pydantic import BaseModel, Field, HttpUrl, ValidationError
from werkzeug.datastructures import FileStorage
import services
from controllers.common.errors import (
BlockedFileExtensionError,
FilenameNotExistsError,
FileTooLargeError,
NoFileUploadedError,
RemoteFileUploadError,
TooManyFilesError,
UnsupportedFileTypeError,
)
from controllers.common.schema import query_params_from_model, register_response_schema_models, register_schema_models
from controllers.files import files_ns
from controllers.files.wraps import FileGrantInvalidError, GrantedFileNotFoundError, file_grant_required
from extensions.ext_application_services import application_services
from fields.base import ResponseModel
from fields.file_fields import FileResponse
from fields.file_grant_fields import ResolvedFileResponse
from libs.exception import BaseHTTPException
from libs.helper import dump_response
from services.entities.file_grant_entities import (
FileContent,
FileGrantClaims,
FileGrantContext,
FileGrantScope,
FileKind,
FileRef,
StoredUpload,
)
from services.errors.file_grant import (
EndUserNotFoundError,
InvalidFileGrantError,
RemoteFileUnavailableError,
TooManyFileRefsError,
)
from services.file_grant_service import MAX_FILE_GRANT_REFS
# Everything outside this whitelist is served as an attachment. Produced files
# carry a plugin-declared MIME type, so SVG and the rest of the XML family must
# never render in the viewer's origin.
INLINE_MIME_TYPES = frozenset({"image/png", "image/jpeg", "image/gif", "image/webp"})
class InvalidFileRequestError(BaseHTTPException):
error_code = "invalid_request"
description = "The file request is malformed."
code = 400
class FileRefPayload(BaseModel):
id: str
kind: FileKind
class RemoteFileUploadPayload(BaseModel):
url: HttpUrl = Field(description="Remote file URL to fetch and store")
class FileResolvePayload(BaseModel):
files: list[FileRefPayload] = Field(default_factory=list, max_length=MAX_FILE_GRANT_REFS)
class FileContentQuery(BaseModel):
token: str = Field(description="Signed content token scoped to this file")
class RemoteFileUploadResponse(FileResponse):
"""Dify's upload shape plus the ``url`` key its remote-upload clients read.
Dify has no service-api remote upload for this endpoint to stand in for, and
the web and console one it does have answers under ``url``. Carrying both
names lets a client of either move over untouched; they hold one URL.
"""
url: str
class ProducedFileResponse(ResponseModel):
id: str
name: str
size: int
mime_type: str | None = None
url: str
internal_url: str
class FileResolveResponse(ResponseModel):
files: list[ResolvedFileResponse]
register_schema_models(files_ns, RemoteFileUploadPayload, FileResolvePayload)
register_response_schema_models(
files_ns, FileResponse, RemoteFileUploadResponse, ProducedFileResponse, FileResolveResponse
)
@files_ns.route("/appdeploy/upload")
class GrantedFileUploadApi(Resource):
"""Store one uploaded file against the grant's end user."""
@file_grant_required(FileGrantScope.UPLOAD)
@files_ns.doc("grant_upload_file")
@files_ns.doc(
responses={
201: "File uploaded",
400: "No file uploaded, the file has no name, or its extension is blocked",
401: "Invalid grant",
403: "Grant lacks the upload scope",
413: "File too large",
415: "Unsupported file type",
}
)
@files_ns.response(201, "File uploaded", files_ns.models[FileResponse.__name__])
def post(self, grant: FileGrantClaims):
upload = _single_upload()
upload_file = _store_upload(
grant,
filename=upload.filename or "",
stream=upload.stream,
mimetype=upload.mimetype,
)
return _granted_file_response(upload_file), 201
@files_ns.route("/appdeploy/remote-upload")
class GrantedRemoteFileUploadApi(Resource):
"""Fetch a remote URL through the SSRF-safe fetcher and store it."""
@file_grant_required(FileGrantScope.UPLOAD)
@files_ns.doc("grant_upload_remote_file")
@files_ns.expect(files_ns.models[RemoteFileUploadPayload.__name__])
@files_ns.doc(
responses={
201: "Remote file uploaded",
400: "Invalid URL, unfetchable remote file, or a blocked extension",
401: "Invalid grant",
403: "Grant lacks the upload scope",
413: "File too large",
415: "Unsupported file type",
}
)
@files_ns.response(201, "Remote file uploaded", files_ns.models[RemoteFileUploadResponse.__name__])
def post(self, grant: FileGrantClaims):
try:
payload = RemoteFileUploadPayload.model_validate(files_ns.payload or {})
except ValidationError as exc:
raise InvalidFileRequestError(str(exc)) from exc
try:
upload_file = application_services().file_grants.store_remote_upload(
context=_grant_context(grant),
url=str(payload.url),
)
except RemoteFileUnavailableError as exc:
raise RemoteFileUploadError(f"Failed to fetch file from {payload.url}") from exc
except EndUserNotFoundError as exc:
raise GrantedFileNotFoundError() from exc
except services.errors.file.FileTooLargeError as exc:
raise FileTooLargeError(exc.description) from exc
except services.errors.file.BlockedFileExtensionError as exc:
raise BlockedFileExtensionError(exc.description) from exc
except services.errors.file.UnsupportedFileTypeError:
raise UnsupportedFileTypeError()
return _remote_granted_file_response(upload_file), 201
@files_ns.route("/appdeploy/produced")
class ProducedFileApi(Resource):
"""Store one file produced by a running workflow node."""
@file_grant_required(FileGrantScope.PRODUCE)
@files_ns.doc("grant_upload_produced_file")
@files_ns.doc(
responses={
201: "Produced file stored",
400: "No file uploaded",
401: "Invalid grant",
403: "Grant lacks the produce scope",
413: "File too large",
}
)
@files_ns.response(201, "Produced file stored", files_ns.models[ProducedFileResponse.__name__])
def post(self, grant: FileGrantClaims):
upload = _single_upload()
try:
tool_file, access = application_services().file_grants.store_produced(
context=_grant_context(grant),
filename=upload.filename,
stream=upload.stream,
mimetype=upload.mimetype,
)
except services.errors.file.FileTooLargeError as exc:
raise FileTooLargeError(exc.description)
except EndUserNotFoundError as exc:
raise GrantedFileNotFoundError() from exc
return ProducedFileResponse(
id=tool_file.id,
name=tool_file.name,
size=tool_file.size,
mime_type=tool_file.mime_type,
url=access.external_url,
internal_url=access.internal_url,
).model_dump(mode="json"), 201
@files_ns.route("/appdeploy/resolve")
class GrantedFileResolveApi(Resource):
"""Re-check ownership and sign fresh URLs at the moment of use."""
@file_grant_required(FileGrantScope.RESOLVE)
@files_ns.doc("grant_resolve_files")
@files_ns.expect(files_ns.models[FileResolvePayload.__name__])
@files_ns.doc(
responses={
200: "Files resolved",
400: "Malformed request",
401: "Invalid grant",
403: "Grant lacks the resolve scope",
}
)
@files_ns.response(200, "Files resolved", files_ns.models[FileResolveResponse.__name__])
def post(self, grant: FileGrantClaims):
try:
payload = FileResolvePayload.model_validate(files_ns.payload or {})
except ValidationError as exc:
raise InvalidFileRequestError(str(exc)) from exc
refs = [FileRef(id=ref.id, kind=ref.kind) for ref in payload.files]
try:
resolved = application_services().file_grants.resolve_file_access(
context=_grant_context(grant),
refs=refs,
)
except EndUserNotFoundError as exc:
raise GrantedFileNotFoundError() from exc
except TooManyFileRefsError as exc:
raise InvalidFileRequestError(str(exc)) from exc
return FileResolveResponse(
files=[ResolvedFileResponse.from_resolved(ref.id, file) for ref, file in zip(refs, resolved, strict=True)]
).model_dump(mode="json")
@files_ns.route("/appdeploy/<uuid:file_id>/content")
class GrantedFileContentApi(Resource):
"""Stream one file's bytes to a holder of its content token."""
@files_ns.doc("grant_file_content")
@files_ns.doc(params=query_params_from_model(FileContentQuery))
@files_ns.doc(
responses={
200: "File stream returned",
401: "Invalid or expired content token",
404: "File not found",
}
)
def get(self, file_id: UUID):
try:
query = FileContentQuery.model_validate(request.args.to_dict(flat=True))
except ValidationError as exc:
raise FileGrantInvalidError() from exc
try:
content = application_services().file_grants.load_content(
token=query.token,
requested_file_id=str(file_id),
)
except InvalidFileGrantError as exc:
raise FileGrantInvalidError() from exc
if content is None:
raise GrantedFileNotFoundError()
return _content_response(content)
def _content_response(content: FileContent) -> Response:
mime_type = _normalized_mime_type(content.mime_type)
inline = mime_type in INLINE_MIME_TYPES
response = Response(
content.stream,
mimetype=mime_type if inline else "application/octet-stream",
direct_passthrough=True,
headers={},
)
response.headers["X-Content-Type-Options"] = "nosniff"
if not inline:
encoded_filename = quote(content.name or "")
response.headers["Content-Disposition"] = f"attachment; filename*=UTF-8''{encoded_filename}"
response.headers["Content-Type"] = "application/octet-stream"
# The sibling preview endpoints advertise `Accept-Ranges` for audio and video.
# Every such type downloads here, so the hint could only ever ride on a
# response no player will seek.
if content.size > 0:
response.headers["Content-Length"] = str(content.size)
return response
def _normalized_mime_type(mime_type: str | None) -> str:
return mime_type.split(";", 1)[0].strip().lower() if mime_type else ""
def _single_upload() -> FileStorage:
if "file" not in request.files:
raise NoFileUploadedError()
if len(request.files) > 1:
raise TooManyFilesError()
upload = request.files["file"]
if not upload.filename:
raise FilenameNotExistsError()
return upload
def _store_upload(
grant: FileGrantClaims,
*,
filename: str,
stream: IO[bytes],
mimetype: str,
) -> StoredUpload:
try:
return application_services().file_grants.store_upload(
context=_grant_context(grant),
filename=filename,
stream=stream,
mimetype=mimetype,
)
except EndUserNotFoundError as exc:
raise GrantedFileNotFoundError() from exc
except services.errors.file.FileTooLargeError as exc:
raise FileTooLargeError(exc.description)
except services.errors.file.BlockedFileExtensionError as exc:
raise BlockedFileExtensionError(exc.description) from exc
except services.errors.file.UnsupportedFileTypeError:
raise UnsupportedFileTypeError()
def _grant_context(grant: FileGrantClaims) -> FileGrantContext:
return FileGrantContext(tenant_id=grant.tenant_id, app_id=grant.app_id, end_user_id=grant.sub)
def _granted_file_response(upload_file: StoredUpload) -> dict[str, object]:
"""Answer an upload exactly as dify's own upload endpoints answer it.
A client moving off ``POST /v1/files/upload`` must not have to read a second
shape, so the same model reads the same ``upload_files`` row: every key dify
leaves null for such a row is null here too. Only the value of ``source_url``
is ours. Dify signs a ``file-preview`` URL there and this channel signs a
content-token URL, which keeps that key's promise of a signed URL that
retrieves the file while keeping the grant its only way in.
"""
signed_url, _ = application_services().file_grants.content_urls(file_id=upload_file.id, kind=FileKind.UPLOAD)
return dump_response(FileResponse, upload_file) | {"source_url": signed_url}
def _remote_granted_file_response(upload_file: StoredUpload) -> dict[str, object]:
"""Answer a remote upload with the upload shape plus dify's ``url`` key.
Reuses the URL already signed for ``source_url`` rather than signing a
second one, so the two keys are one value under the two names dify's two
kinds of client look for.
"""
response = _granted_file_response(upload_file)
return response | {"url": response["source_url"]}
__all__ = [
"GrantedFileContentApi",
"GrantedFileResolveApi",
"GrantedFileUploadApi",
"GrantedRemoteFileUploadApi",
"ProducedFileApi",
]

57
api/controllers/files/wraps.py Normal file
View File

@ -0,0 +1,57 @@
"""Bearer authentication for the AppDeploy file grant endpoints."""
from collections.abc import Callable
from functools import wraps
from flask import request
from extensions.ext_application_services import application_services
from libs.exception import BaseHTTPException
from services.entities.file_grant_entities import FileGrantClaims, FileGrantScope
class FileGrantInvalidError(BaseHTTPException):
error_code = "grant_invalid"
description = "The file grant is missing, malformed, or expired."
code = 401
class FileGrantScopeDeniedError(BaseHTTPException):
error_code = "grant_scope_denied"
description = "The file grant does not carry the required scope."
code = 403
class GrantedFileNotFoundError(BaseHTTPException):
error_code = "file_not_found"
description = "File not found."
code = 404
def file_grant_required[**P, R](scope: FileGrantScope) -> Callable[[Callable[P, R]], Callable[P, R]]:
"""Require a valid file grant carrying ``scope`` and inject its claims."""
def decorator(view: Callable[P, R]) -> Callable[P, R]:
@wraps(view)
def decorated(*args: P.args, **kwargs: P.kwargs) -> R:
kwargs["grant"] = _authenticated_claims(scope)
return view(*args, **kwargs)
return decorated
return decorator
def _authenticated_claims(scope: FileGrantScope) -> FileGrantClaims:
scheme, _, token = request.headers.get("Authorization", "").partition(" ")
if scheme.lower() != "bearer" or not token:
raise FileGrantInvalidError()
claims = application_services().file_grants.decode_grant(token)
if claims is None:
raise FileGrantInvalidError()
if scope not in claims.scopes:
raise FileGrantScopeDeniedError()
return claims

2
api/controllers/inner_api/__init__.py
View File

@ -21,6 +21,7 @@ from .agent import files as _agent_files
from .agent import llm as _agent_llm
from .agent import tools as _agent_tools
from .app import dsl as _app_dsl
from .app import file_grants as _app_file_grants
from .knowledge import retrieval as _knowledge_retrieval
from .knowledge_fs import storage as _knowledge_fs_storage
from .plugin import agent_config as _agent_config
@ -36,6 +37,7 @@ __all__ = [
"_agent_llm",
"_agent_tools",
"_app_dsl",
"_app_file_grants",
"_knowledge_fs_storage",
"_knowledge_retrieval",
"_mail",

203
api/controllers/inner_api/app/file_grants.py Normal file
View File

@ -0,0 +1,203 @@
"""Mint AppDeploy file grants for the enterprise control plane.
This is the only endpoint that asserts an AppDeploy identity: the application
service upserts the subject's ``EndUser`` row, validates the files the caller
claims to reference, and signs a short-lived grant.
"""
from __future__ import annotations
from flask_restx import Resource
from pydantic import BaseModel, Field, ValidationError
from controllers.common.schema import register_response_schema_models, register_schema_models
from controllers.console.wraps import setup_required
from controllers.files.wraps import GrantedFileNotFoundError
from controllers.inner_api import inner_api_ns
from controllers.inner_api.wraps import enterprise_inner_api_only
from extensions.ext_application_services import application_services
from fields.base import ResponseModel
from fields.file_grant_fields import ResolvedFileResponse
from libs.exception import BaseHTTPException
from services.entities.file_grant_entities import FileGrantMintRequest, FileGrantScope, FileKind, FileRef
from services.file_grant_service import (
AppNotFoundError,
EndUserNotFoundError,
TooManyFileRefsError,
)
from services.file_grant_service import (
GrantedFileNotFoundError as ServiceGrantedFileNotFoundError,
)
from services.file_grant_service import (
GrantTtlTooLongError as ServiceGrantTtlTooLongError,
)
from services.file_grant_service import (
InvalidGrantRequestError as ServiceInvalidGrantRequestError,
)
from services.file_grant_service import (
InvalidSubjectError as ServiceInvalidSubjectError,
)
class InvalidGrantRequestError(BaseHTTPException):
error_code = "invalid_request"
description = "The file grant request is malformed."
code = 400
class GrantTtlTooLongError(BaseHTTPException):
error_code = "grant_ttl_too_long"
description = "The requested file grant lifetime exceeds its allowed window."
code = 400
class InvalidSubjectError(BaseHTTPException):
error_code = "invalid_subject"
description = "The subject is empty or contains a NUL byte."
code = 400
class GrantAppNotFoundError(BaseHTTPException):
error_code = "app_not_found"
description = "App not found."
code = 404
class FileGrantFileRef(BaseModel):
id: str
kind: FileKind
class FileGrantMintPayload(BaseModel):
tenant_id: str
app_id: str
subject: str
is_anonymous: bool = False
scopes: list[FileGrantScope]
ttl_seconds: int = Field(gt=0)
# Recorded by the enterprise caller's own audit log; it never enters the
# grant because a rotated key must not strand the files it uploaded.
actor_key_digest: str | None = None
file_ids: list[FileGrantFileRef] = Field(default_factory=list)
optional_file_ids: list[FileGrantFileRef] = Field(default_factory=list)
run_deadline: int | None = None
class FileGrantLimits(ResponseModel):
file_size_limit: int
image_file_size_limit: int
audio_file_size_limit: int
video_file_size_limit: int
workflow_file_upload_limit: int
batch_count_limit: int
class FileGrantFileMetadata(ResponseModel):
id: str
kind: FileKind
name: str
size: int
extension: str
mime_type: str | None = None
class FileGrantMintResponse(ResponseModel):
grant: str
expires_at: int
limits: FileGrantLimits
files: list[FileGrantFileMetadata]
optional_files: list[ResolvedFileResponse]
register_schema_models(inner_api_ns, FileGrantMintPayload)
register_response_schema_models(inner_api_ns, FileGrantMintResponse)
@inner_api_ns.route("/enterprise/file-grants")
class EnterpriseFileGrantApi(Resource):
"""Assert one AppDeploy subject and sign a grant for it."""
@setup_required
@enterprise_inner_api_only
@inner_api_ns.doc("enterprise_mint_file_grant")
@inner_api_ns.expect(inner_api_ns.models[FileGrantMintPayload.__name__])
@inner_api_ns.response(
200,
"File grant minted",
inner_api_ns.models[FileGrantMintResponse.__name__],
)
@inner_api_ns.doc(
responses={
400: "Invalid request, subject, or grant TTL",
404: "App not found, or a required file is not owned by the subject",
}
)
def post(self):
try:
payload = FileGrantMintPayload.model_validate(inner_api_ns.payload or {})
except ValidationError as exc:
raise InvalidGrantRequestError(str(exc)) from exc
try:
result = application_services().file_grants.mint(
FileGrantMintRequest(
tenant_id=payload.tenant_id,
app_id=payload.app_id,
subject=payload.subject,
is_anonymous=payload.is_anonymous,
scopes=tuple(payload.scopes),
ttl_seconds=payload.ttl_seconds,
file_refs=tuple(FileRef(id=ref.id, kind=ref.kind) for ref in payload.file_ids),
optional_file_refs=tuple(FileRef(id=ref.id, kind=ref.kind) for ref in payload.optional_file_ids),
run_deadline=payload.run_deadline,
)
)
except AppNotFoundError as exc:
raise GrantAppNotFoundError() from exc
except ServiceGrantTtlTooLongError as exc:
raise GrantTtlTooLongError() from exc
except ServiceInvalidSubjectError as exc:
raise InvalidSubjectError() from exc
except (ServiceInvalidGrantRequestError, TooManyFileRefsError) as exc:
raise InvalidGrantRequestError(str(exc)) from exc
except (EndUserNotFoundError, ServiceGrantedFileNotFoundError) as exc:
raise GrantedFileNotFoundError() from exc
return FileGrantMintResponse(
grant=result.grant,
expires_at=result.expires_at,
limits=FileGrantLimits(
file_size_limit=result.limits.file_size_limit,
image_file_size_limit=result.limits.image_file_size_limit,
audio_file_size_limit=result.limits.audio_file_size_limit,
video_file_size_limit=result.limits.video_file_size_limit,
workflow_file_upload_limit=result.limits.workflow_file_upload_limit,
batch_count_limit=result.limits.batch_count_limit,
),
files=[
FileGrantFileMetadata(
id=file.id,
kind=file.kind,
name=file.name,
size=file.size,
extension=file.extension,
mime_type=file.mime_type,
)
for file in result.files
],
optional_files=[
ResolvedFileResponse.from_resolved(ref.id, access)
for ref, access in zip(
payload.optional_file_ids,
result.optional_files,
strict=True,
)
],
).model_dump(mode="json")
__all__ = [
"EnterpriseFileGrantApi",
"FileGrantMintPayload",
"FileGrantMintResponse",
]

5
api/controllers/openapi/_contract.py
View File

@ -5,8 +5,9 @@ reference — emitting the Swagger schema AND doing the runtime validation/
serialisation — so the advertised and enforced contracts can't drift. Validation
failures map to a single shape: 422.
They must sit BELOW ``@auth_router.guard`` so auth runs before validation and the
``view.__wrapped__`` unit-test seam unwraps exactly the guard layer.
They must sit below route admission (or a direct ``@auth_router.guard``) so
authentication runs before validation and the ``view.__wrapped__`` unit-test
seam can bypass the outer admission layer.
"""
from __future__ import annotations

156
api/controllers/openapi/account.py
View File

@ -1,12 +1,11 @@
from __future__ import annotations
from datetime import UTC, datetime
from uuid import UUID
from flask_restx import Resource
from sqlalchemy.orm import Session
from werkzeug.exceptions import NotFound
from werkzeug.exceptions import NotFound, Unauthorized
from controllers.common.session import with_session
from controllers.openapi import openapi_ns
from controllers.openapi._contract import accepts, returns
from controllers.openapi._models import (
@ -18,114 +17,77 @@ from controllers.openapi._models import (
SessionRow,
WorkspacePayload,
)
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData
from extensions.ext_redis import redis_client
from libs.oauth_bearer import (
Scope,
TokenType,
get_auth_ctx,
)
from libs.rate_limit import (
LIMIT_ME_PER_ACCOUNT,
enforce,
)
from services.account_service import AccountService, TenantService
from services.oauth_device_flow import (
list_active_sessions,
revoke_oauth_token,
token_belongs_to_subject,
)
from controllers.openapi.flask_admission import openapi_account_admission
from extensions.ext_application_services import application_services
from libs.oauth_bearer import Scope
from libs.rate_limit import LIMIT_ME_PER_ACCOUNT
from machinery.context import AccountRequestContext
from services.account_errors import AccountNotFoundError, AccountSessionNotFoundError
from services.entities.account_access_entities import AccountSessionSnapshot, AccountWorkspaceSnapshot
from services.entities.account_entities import AccountSnapshot
@openapi_ns.route("/account")
class AccountApi(Resource):
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@openapi_account_admission(scope=Scope.FULL, rate_limit=LIMIT_ME_PER_ACCOUNT)
@returns(200, AccountResponse, description="Account info")
@with_session(write=False)
def get(self, session: Session, *, auth_data: AuthData):
enforce(LIMIT_ME_PER_ACCOUNT, key=f"account:{auth_data.account_id}")
account_id_str = str(auth_data.account_id) if auth_data.account_id else None
account = AccountService.get_account_by_id(account_id_str, session=session) if account_id_str else None
memberships = TenantService.get_account_memberships(account_id_str, session=session) if account_id_str else []
default_ws_id = _pick_default_workspace(memberships)
def get(self, request_context: AccountRequestContext):
try:
snapshot = application_services().accounts.access.get(request_context)
except AccountNotFoundError:
raise Unauthorized("account not found") from None
return AccountResponse(
subject_type="account",
subject_email=account.email if account else None,
account=_account_payload(account) if account else None,
workspaces=[_workspace_payload(m) for m in memberships],
default_workspace_id=default_ws_id,
subject_email=snapshot.account.email,
account=_account_payload(snapshot.account),
workspaces=[_workspace_payload(workspace) for workspace in snapshot.workspaces],
default_workspace_id=snapshot.default_workspace_id,
)
@openapi_ns.route("/account/sessions/self")
class AccountSessionsSelfApi(Resource):
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@openapi_account_admission(scope=Scope.FULL)
@returns(200, RevokeResponse, description="Session revoked")
@with_session
def delete(self, session: Session, *, auth_data: AuthData):
revoke_oauth_token(redis_client, str(auth_data.token_id), session=session)
def delete(self, request_context: AccountRequestContext):
application_services().accounts.access.revoke_current_session(request_context)
return RevokeResponse(status="revoked")
@openapi_ns.route("/account/sessions")
class AccountSessionsApi(Resource):
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@openapi_account_admission(scope=Scope.FULL)
@returns(200, SessionListResponse, description="Session list")
@accepts(query=SessionListQuery)
@with_session(write=False)
def get(self, session: Session, *, auth_data: AuthData, query: SessionListQuery):
# SessionListQuery enforces the advertised bounds (extra='forbid', page>=1,
# 1<=limit<=MAX_PAGE_LIMIT) so the server rejects out-of-range paging rather
# than silently coercing (e.g. page=0 -> empty slice).
ctx = get_auth_ctx()
now = datetime.now(UTC)
page = query.page
limit = query.limit
all_rows = list_active_sessions(ctx, now, session=session)
total = len(all_rows)
sliced = all_rows[(page - 1) * limit : page * limit]
items = [
SessionRow(
id=str(r.id),
prefix=r.prefix,
client_id=r.client_id,
device_label=r.device_label,
created_at=_iso(r.created_at),
last_used_at=_iso(r.last_used_at),
expires_at=_iso(r.expires_at),
)
for r in sliced
]
def get(self, request_context: AccountRequestContext, *, query: SessionListQuery):
page = application_services().accounts.access.list_sessions(
request_context,
page=query.page,
limit=query.limit,
)
return SessionListResponse(
page=page,
limit=limit,
total=total,
has_more=page * limit < total,
data=items,
page=page.page,
limit=page.limit,
total=page.total,
has_more=page.has_more,
data=[_session_row(session) for session in page.items],
)
@openapi_ns.route("/account/sessions/<string:session_id>")
class AccountSessionByIdApi(Resource):
@auth_router.guard(scope=Scope.FULL, allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}))
@openapi_account_admission(scope=Scope.FULL)
@returns(200, RevokeResponse, description="Session revoked")
@with_session
def delete(self, session: Session, session_id: str, *, auth_data: AuthData):
ctx = get_auth_ctx()
# 404 (not 403) on cross-subject so the endpoint doesn't leak
# token IDs that belong to other subjects.
if not token_belongs_to_subject(session_id, ctx, session=session):
raise NotFound("session not found")
revoke_oauth_token(redis_client, session_id, session=session)
def delete(self, request_context: AccountRequestContext, session_id: str):
try:
token_id = str(UUID(session_id))
except ValueError:
raise NotFound("session not found") from None
try:
application_services().accounts.access.revoke_session(request_context, token_id=token_id)
except AccountSessionNotFoundError:
# Do not reveal whether a token ID belongs to another account.
raise NotFound("session not found") from None
return RevokeResponse(status="revoked")
@ -137,19 +99,21 @@ def _iso(dt: datetime | None) -> str | None:
return dt.isoformat().replace("+00:00", "Z")
def _pick_default_workspace(memberships) -> str | None:
if not memberships:
return None
for join, tenant in memberships:
if getattr(join, "current", False):
return str(tenant.id)
return str(memberships[0][1].id)
def _session_row(session: AccountSessionSnapshot) -> SessionRow:
return SessionRow(
id=session.id,
prefix=session.prefix,
client_id=session.client_id,
device_label=session.device_label,
created_at=_iso(session.created_at),
last_used_at=_iso(session.last_used_at),
expires_at=_iso(session.expires_at),
)
def _workspace_payload(row) -> WorkspacePayload:
join, tenant = row
return WorkspacePayload(id=str(tenant.id), name=tenant.name, role=getattr(join, "role", ""))
def _workspace_payload(workspace: AccountWorkspaceSnapshot) -> WorkspacePayload:
return WorkspacePayload(id=workspace.id, name=workspace.name, role=workspace.role)
def _account_payload(account) -> AccountPayload:
return AccountPayload(id=str(account.id), email=account.email, name=account.name)
def _account_payload(account: AccountSnapshot) -> AccountPayload:
return AccountPayload(id=account.id, email=account.email, name=account.name)

15
api/controllers/openapi/auth/pipeline.py
View File

@ -119,8 +119,8 @@ class PipelineRouter:
"""Entry point for openapi auth.
`guard()` is the decorator that endpoints attach to. It applies
global gates (edition, token type) then dispatches to the matching
`PipelineRoute` for the token type.
global gates (edition, license, token type) then dispatches to the
matching `PipelineRoute` for the token type.
"""
def __init__(self, routes: dict[TokenType, PipelineRoute]) -> None:
@ -132,6 +132,7 @@ class PipelineRouter:
scope: Scope | None = None,
allowed_token_types: frozenset[TokenType] | None = None,
edition: frozenset[DeploymentEdition] | None = None,
require_valid_enterprise_license: bool = False,
workspace_membership: bool = False,
allowed_roles: frozenset[TenantAccountRole] | None = None,
rbac: RBACRequirement | None = None,
@ -140,6 +141,7 @@ class PipelineRouter:
scope=scope,
allowed_token_types=allowed_token_types,
edition=edition,
require_valid_enterprise_license=require_valid_enterprise_license,
workspace_membership=workspace_membership,
allowed_roles=allowed_roles,
rbac=rbac,
@ -151,6 +153,7 @@ class PipelineRouter:
scope: Scope | None = None,
allowed_token_types: frozenset[TokenType] | None = None,
edition: frozenset[DeploymentEdition] | None = None,
require_valid_enterprise_license: bool = False,
allowed_roles: frozenset[TenantAccountRole] | None = None,
rbac: RBACRequirement | None = None,
) -> Callable:
@ -158,6 +161,7 @@ class PipelineRouter:
scope=scope,
allowed_token_types=allowed_token_types,
edition=edition,
require_valid_enterprise_license=require_valid_enterprise_license,
workspace_membership=True,
allowed_roles=allowed_roles,
rbac=rbac,
@ -169,6 +173,7 @@ class PipelineRouter:
scope: Scope | None,
allowed_token_types: frozenset[TokenType] | None,
edition: frozenset[DeploymentEdition] | None,
require_valid_enterprise_license: bool,
workspace_membership: bool,
allowed_roles: frozenset[TenantAccountRole] | None,
rbac: RBACRequirement | None,
@ -183,6 +188,7 @@ class PipelineRouter:
scope=scope,
allowed_token_types=allowed_token_types,
edition=edition,
require_valid_enterprise_license=require_valid_enterprise_license,
workspace_membership=workspace_membership,
allowed_roles=allowed_roles,
rbac=rbac,
@ -201,6 +207,7 @@ class PipelineRouter:
scope: Scope | None,
allowed_token_types: frozenset[TokenType] | None,
edition: frozenset[DeploymentEdition] | None,
require_valid_enterprise_license: bool,
workspace_membership: bool = False,
allowed_roles: frozenset[TenantAccountRole] | None = None,
rbac: RBACRequirement | None = None,
@ -210,7 +217,9 @@ class PipelineRouter:
raise NotFound()
license_checked = False
if edition is not None and DeploymentEdition.ENTERPRISE in edition:
if dify_config.DEPLOYMENT_EDITION == DeploymentEdition.ENTERPRISE and (
require_valid_enterprise_license or (edition is not None and DeploymentEdition.ENTERPRISE in edition)
):
_check_license()
license_checked = True

88
api/controllers/openapi/flask_admission.py Normal file
View File

@ -0,0 +1,88 @@
"""Flask adapter for account-authenticated OpenAPI admission."""
from __future__ import annotations
from collections.abc import Callable
from functools import wraps
from typing import Concatenate
from flask import Response, request
from werkzeug.exceptions import Unauthorized
from controllers.openapi.auth.composition import auth_router
from controllers.openapi.auth.data import AuthData
from core.logging.context import get_request_id, get_trace_id
from enums import DeploymentEdition
from libs.oauth_bearer import Scope, TokenType
from libs.rate_limit import RateLimit, enforce
from machinery.context import AccountRequestContext
from models.account import Account, AccountStatus
def openapi_account_admission[T, **P, R](
*,
scope: Scope,
editions: frozenset[DeploymentEdition] | None = None,
require_initialized: bool = True,
require_valid_enterprise_license: bool = True,
rate_limit: RateLimit | None = None,
) -> Callable[
[Callable[Concatenate[T, AccountRequestContext, P], R]],
Callable[Concatenate[T, P], R | Response],
]:
"""Authenticate an account bearer and inject framework-neutral identity.
Client-version admission remains attached to the OpenAPI blueprint so it
can also reject requests for removed routes. Edition and Enterprise
license checks are delegated to the shared auth router before the stable
context is constructed.
"""
def decorator(
view: Callable[Concatenate[T, AccountRequestContext, P], R],
) -> Callable[Concatenate[T, P], R | Response]:
@wraps(view)
def inject_request_context(
self: T,
/,
*args: P.args,
**kwargs: P.kwargs,
) -> R:
auth_data = kwargs.pop("auth_data", None)
if not isinstance(auth_data, AuthData):
raise RuntimeError("OpenAPI auth pipeline did not provide valid AuthData")
account = auth_data.caller
if not isinstance(account, Account) or auth_data.account_id is None:
raise Unauthorized("account not found")
if require_initialized and account.status == AccountStatus.UNINITIALIZED:
raise Unauthorized("account not initialized")
account_id = str(auth_data.account_id)
if rate_limit is not None:
enforce(rate_limit, key=f"account:{account_id}")
context = AccountRequestContext(
request_id=get_request_id(),
trace_id=get_trace_id() or request.headers.get("X-Trace-Id"),
account_id=account_id,
access_token_id=str(auth_data.token_id) if auth_data.token_id is not None else None,
)
return view(self, context, *args, **kwargs)
authenticated = auth_router.guard(
scope=scope,
allowed_token_types=frozenset({TokenType.OAUTH_ACCOUNT}),
edition=editions,
require_valid_enterprise_license=require_valid_enterprise_license,
)(inject_request_context)
# Keep one stable test seam: one ``__wrapped__`` skips route admission
# and reaches input parsing/response handling. Client-version admission
# stays blueprint-wide by design.
@wraps(view)
def admitted(self: T, /, *args: P.args, **kwargs: P.kwargs) -> R | Response:
return authenticated(self, *args, **kwargs)
return admitted
return decorator

32
api/controllers/service_api/app/workflow.py
View File

@ -48,6 +48,7 @@ from core.errors.error import (
)
from core.helper.trace_id_helper import get_external_trace_id, get_trace_session_id, omit_trace_session_id_from_payload
from enums import CloudPlan, DeploymentEdition
from extensions.ext_application_services import application_services
from extensions.ext_database import db
from extensions.ext_redis import redis_client
from fields.base import ResponseModel
@ -71,7 +72,6 @@ from services.errors.app import (
TriggerWorkflowServiceModeUnavailableError as TriggerWorkflowServiceModeUnavailableServiceError,
)
from services.errors.llm import InvokeRateLimitError
from services.workflow_app_service import WorkflowAppService
logger = logging.getLogger(__name__)
@ -614,20 +614,16 @@ class WorkflowAppLogApi(Resource):
created_at_before = isoparse(args.created_at__before) if args.created_at__before else None
created_at_after = isoparse(args.created_at__after) if args.created_at__after else None
# get paginate workflow app logs
workflow_app_service = WorkflowAppService()
with sessionmaker(db.engine).begin() as session:
workflow_app_log_pagination = workflow_app_service.get_paginate_workflow_app_logs(
session=session,
app_model=app_model,
keyword=args.keyword,
status=status,
created_at_before=created_at_before,
created_at_after=created_at_after,
page=args.page,
limit=args.limit,
created_by_end_user_session_id=args.created_by_end_user_session_id,
created_by_account=args.created_by_account,
)
return dump_response(WorkflowAppLogPaginationResponse, workflow_app_log_pagination)
result = application_services().workflow_app_logs.list_logs(
tenant_id=app_model.tenant_id,
app_id=app_model.id,
keyword=args.keyword,
status=status,
created_at_before=created_at_before,
created_at_after=created_at_after,
page=args.page,
limit=args.limit,
created_by_end_user_session_id=args.created_by_end_user_session_id,
created_by_account=args.created_by_account,
)
return dump_response(WorkflowAppLogPaginationResponse, result)

7
api/controllers/service_api/dataset/dataset.py
View File

@ -430,10 +430,11 @@ class DatasetListApi(DatasetApiResource):
query_params["tag_ids"] = request.args.getlist("tag_ids")
query = DatasetListQuery.model_validate(query_params)
# provider = request.args.get("provider", default="vendor")
effective_limit = min(query.limit, 100)
datasets, total = DatasetService.get_datasets(
query.page,
query.limit,
effective_limit,
session,
tenant_id,
current_user,
@ -467,8 +468,8 @@ class DatasetListApi(DatasetApiResource):
item["embedding_available"] = True
response = {
"data": data,
"has_more": len(datasets) == query.limit,
"limit": query.limit,
"has_more": query.page * effective_limit < total,
"limit": effective_limit,
"total": total,
"page": query.page,
}

7
api/controllers/service_api/dataset/document.py
View File

@ -1015,8 +1015,9 @@ class DocumentListApi(DatasetApiResource):
query = query.order_by(desc(Document.created_at), desc(Document.position))
effective_limit = min(query_params.limit, 100)
paginated_documents = paginate_query(
query, session=session, page=query_params.page, per_page=query_params.limit, max_per_page=100
query, session=session, page=query_params.page, per_page=effective_limit, max_per_page=100
)
documents = paginated_documents.items
@ -1029,8 +1030,8 @@ class DocumentListApi(DatasetApiResource):
response = {
"data": document_responses(documents, session=session),
"has_more": len(documents) == query_params.limit,
"limit": query_params.limit,
"has_more": query_params.page * effective_limit < paginated_documents.total,
"limit": effective_limit,
"total": paginated_documents.total,
"page": query_params.page,
}

4
api/controllers/service_api/dataset/segment.py
View File

@ -281,7 +281,7 @@ class SegmentApi(DatasetApiResource):
use_defaults_for_malformed_ints=True,
)
page = args.page
limit = args.limit
limit = min(args.limit, 100)
dataset_id_str = str(dataset_id)
dataset = session.scalar(
select(Dataset).where(Dataset.tenant_id == tenant_id, Dataset.id == dataset_id_str).limit(1)
@ -331,7 +331,7 @@ class SegmentApi(DatasetApiResource):
"data": segment_responses_with_summaries(segments, summaries, session=session),
"doc_form": document.doc_form,
"total": total,
"has_more": len(segments) == limit,
"has_more": page * limit < total,
"limit": limit,
"page": page,
}

7
api/controllers/web/app.py
View File

@ -26,6 +26,7 @@ from libs.passport import PassportService
from libs.token import extract_webapp_passport
from models.model import App, EndUser
from services.app_definition_query_service import AppDefinitionNotPublishedError, AppDefinitionUnavailableError
from services.web_passport_service import WebAppAuthType
from services.webapp_access_query_service import (
WebAppAccessAppNotFoundError,
WebAppAccessReferenceRequiredError,
@ -184,12 +185,16 @@ class AppWebAuthPermission(Resource):
if not tk:
raise Unauthorized("Access token is missing.")
decoded = PassportService().verify(tk)
user_id = decoded.get("user_id", "visitor")
except Unauthorized:
raise WebAppAuthRequiredError() from None
except Exception:
logger.exception("Unexpected error during auth verification")
raise
if decoded.get("auth_type") != WebAppAuthType.INTERNAL:
raise WebAppAuthRequiredError()
user_id = decoded.get("user_id")
if not user_id:
raise WebAppAuthRequiredError()
try:
is_allowed = webapp_access.is_user_allowed(user_id=str(user_id), app_id=app_id)

12
api/controllers/web/wraps.py
View File

@ -19,6 +19,7 @@ from models.model import App, EndUser, Site
from services.app_service import AppService
from services.enterprise.enterprise_service import EnterpriseService, WebAppAccessMode, WebAppSettings
from services.system_feature_service import SystemFeatureService
from services.web_passport_gateways import resolve_web_app_auth_type
from services.webapp_auth_service import WebAppAuthService
@ -133,6 +134,14 @@ def _validate_user_accessibility(
if not webapp_settings:
raise WebAppAuthRequiredError("Web app settings not found.")
auth_type = decoded.get("auth_type")
if not auth_type:
raise WebAppAuthRequiredError("Missing auth_type in the token.")
expected_auth_type = resolve_web_app_auth_type(webapp_settings.access_mode)
if auth_type != expected_auth_type:
raise WebAppAuthRequiredError()
if WebAppAuthService.is_app_require_permission_check(
access_mode=webapp_settings.access_mode, session=db.session()
):
@ -140,10 +149,7 @@ def _validate_user_accessibility(
if not EnterpriseService.WebAppAuth.is_user_allowed_to_access_webapp(user_id, app_id):
raise WebAppAuthAccessDeniedError()
auth_type = decoded.get("auth_type")
granted_at = decoded.get("granted_at")
if not auth_type:
raise WebAppAuthAccessDeniedError("Missing auth_type in the token.")
if not granted_at:
raise WebAppAuthAccessDeniedError("Missing granted_at in the token.")
# check if sso has been updated

11
api/core/rag/index_processor/index_processor_base.py
View File

@ -190,7 +190,16 @@ class BaseIndexProcessor(ABC):
upload_file_id_list.append(upload_file_id)
continue
if current_user:
upload_file_id = self._download_image(image.split(" ")[0], current_user)
image_url = image.split(" ")[0]
try:
parsed_url = urlparse(image_url)
except ValueError:
logging.debug("Skipping malformed image reference: %s", image_url)
continue
if parsed_url.scheme not in {"http", "https"} or not parsed_url.netloc:
logging.debug("Skipping non-HTTP image reference: %s", image_url)
continue
upload_file_id = self._download_image(image_url, current_user)
if upload_file_id:
upload_file_id_list.append(upload_file_id)

44
api/core/tools/tool_manager.py
View File

@ -42,7 +42,7 @@ from core.tools.entities.tool_entities import (
ToolProviderType,
emoji_icon_adapter,
)
from core.tools.errors import ToolProviderNotFoundError
from core.tools.errors import ToolProviderCredentialValidationError, ToolProviderNotFoundError
from core.tools.mcp_tool.provider import MCPToolProviderController
from core.tools.mcp_tool.tool import MCPTool
from core.tools.plugin_tool.provider import PluginToolProviderController
@ -233,7 +233,10 @@ class ToolManager:
builtin_provider = None
logger.info("Error getting builtin provider %s:%s", credential_id, e, exc_info=True)
if builtin_provider is None:
raise ToolProviderNotFoundError(f"provider has been deleted: {credential_id}")
raise ToolProviderCredentialValidationError(
f"Tool credential {credential_id} has been deleted. "
"Select or authorize another credential."
)
if builtin_provider is None:
with Session(db.engine) as session:
@ -247,7 +250,10 @@ class ToolManager:
.order_by(BuiltinToolProvider.is_default.desc(), BuiltinToolProvider.created_at.asc())
)
if builtin_provider is None:
raise ToolProviderNotFoundError(f"no default provider for {provider_id}")
raise ToolProviderCredentialValidationError(
f"No workspace credential is configured for tool provider {provider_id}. "
"Authorize the provider or select a credential."
)
else:
builtin_provider = db.session.scalar(
select(BuiltinToolProvider)
@ -259,7 +265,10 @@ class ToolManager:
)
if builtin_provider is None:
raise ToolProviderNotFoundError(f"builtin provider {provider_id} not found")
raise ToolProviderCredentialValidationError(
f"No credential is configured for built-in tool provider {provider_id}. "
"Authorize the provider or select a credential."
)
from core.helper.credential_utils import runtime_check_credential_policy_compliance
@ -294,15 +303,24 @@ class ToolManager:
system_credentials = BuiltinToolManageService.get_oauth_client(tenant_id, provider_id)
oauth_handler = OAuthHandler()
refreshed_credentials = oauth_handler.refresh_credentials(
tenant_id=tenant_id,
user_id=builtin_provider.user_id,
plugin_id=tool_provider.plugin_id,
provider=provider_name,
redirect_uri=redirect_uri,
system_credentials=system_credentials or {},
credentials=decrypted_credentials,
)
try:
refreshed_credentials = oauth_handler.refresh_credentials(
tenant_id=tenant_id,
user_id=builtin_provider.user_id,
plugin_id=tool_provider.plugin_id,
provider=provider_name,
redirect_uri=redirect_uri,
system_credentials=system_credentials or {},
credentials=decrypted_credentials,
)
except Exception as exc:
logger.warning(
"Failed to refresh OAuth credentials for tool provider %s", provider_id, exc_info=True
)
raise ToolProviderCredentialValidationError(
f"OAuth credential for tool provider {provider_id} could not be refreshed. "
"Reauthorize or select another credential."
) from exc
# update the credentials
builtin_provider.encrypted_credentials = json.dumps(
encrypter.encrypt(refreshed_credentials.credentials)

27
api/core/workflow/nodes/agent/message_transformer.py
View File

@ -27,6 +27,7 @@ from services.tools.builtin_tools_manage_service import BuiltinToolManageService
from .events import AgentLogEvent
from .exceptions import AgentNodeError, AgentVariableTypeError, ToolFileNotFoundError
from .think_tags import ThinkStreamState
_file_access_controller = DatabaseFileAccessController()
@ -54,7 +55,7 @@ class AgentMessageTransformer:
conversation_id=conversation_id,
)
text = ""
think_state = ThinkStreamState()
files: list[File] = []
json_list: list[dict | list] = []
@ -125,10 +126,10 @@ class AgentMessageTransformer:
)
elif message.type == ToolInvokeMessage.MessageType.TEXT:
assert isinstance(message.message, ToolInvokeMessage.TextMessage)
text += message.message.text
chunk = think_state.feed_text(message.message.text)
yield StreamChunkEvent(
selector=[node_id, "text"],
chunk=message.message.text,
chunk=chunk,
is_final=False,
)
elif message.type == ToolInvokeMessage.MessageType.JSON:
@ -152,8 +153,9 @@ class AgentMessageTransformer:
linked_file = self._file_from_link_message(message=message, tenant_id=tenant_id)
if linked_file is not None:
files.append(linked_file)
yield from self._close_open_think(think_state=think_state, node_id=node_id)
stream_text = f"{'File' if linked_file is not None else 'Link'}: {message.message.text}\n"
text += stream_text
think_state.feed_text(stream_text)
yield StreamChunkEvent(
selector=[node_id, "text"],
chunk=stream_text,
@ -249,6 +251,8 @@ class AgentMessageTransformer:
else:
agent_logs.append(agent_log)
yield from self._close_open_think(think_state=think_state, node_id=node_id)
yield agent_log
json_output: list[dict[str, Any] | list[Any]] = []
@ -271,6 +275,8 @@ class AgentMessageTransformer:
else:
json_output.append({"data": []})
yield from self._close_open_think(think_state=think_state, node_id=node_id)
yield StreamChunkEvent(
selector=[node_id, "text"],
chunk="",
@ -288,7 +294,7 @@ class AgentMessageTransformer:
node_run_result=NodeRunResult(
status=WorkflowNodeExecutionStatus.SUCCEEDED,
outputs={
"text": text,
"text": think_state.text,
"usage": jsonable_encoder(llm_usage),
"files": ArrayFileSegment(value=files),
"json": json_output,
@ -304,6 +310,17 @@ class AgentMessageTransformer:
)
)
@staticmethod
def _close_open_think(*, think_state: ThinkStreamState, node_id: str) -> Generator[StreamChunkEvent, None, None]:
closed = think_state.close_if_open()
if closed is None:
return
yield StreamChunkEvent(
selector=[node_id, "text"],
chunk=closed,
is_final=False,
)
@staticmethod
def _file_from_link_message(*, message: ToolInvokeMessage, tenant_id: str) -> File | None:
if not isinstance(message.message, ToolInvokeMessage.TextMessage):

109
api/core/workflow/nodes/agent/think_tags.py Normal file
View File

@ -0,0 +1,109 @@
"""Normalize unclosed ``<think>`` tags in agent/workflow text streams.
Reasoning models (GLM, DeepSeek, etc.) wrap chain-of-thought in ``<think>``
tags. A tool call often interrupts generation before ``</think>`` is emitted,
so a later ``<think>`` or the final answer stays nested in the still-open tag
and the UI renders the reply as thinking. See https://github.com/langgenius/dify/issues/41558
"""
from __future__ import annotations
THINK_OPEN = "<think>"
THINK_CLOSE = "</think>"
def has_unclosed_think(text: str) -> bool:
"""Return True when a ``<think>`` is still open in ``text``."""
_, inside, _ = _walk(text, inside=False, content_since_open=False, close_at_end=False)
return inside
def close_unclosed_think_tags(text: str) -> str:
"""Insert missing ``</think>`` before a nested open tag and at end of text."""
repaired, _, _ = _walk(text, inside=False, content_since_open=False, close_at_end=True)
return repaired
def normalize_think_chunk(
chunk: str,
*,
inside: bool,
content_since_open: bool = False,
) -> tuple[str, bool, bool]:
"""Repair nested opens in a stream chunk and return updated open state.
Does not append a trailing ``</think>`` — the caller closes on tool-call
interruptions and at end of stream.
"""
return _walk(chunk, inside=inside, content_since_open=content_since_open, close_at_end=False)
class ThinkStreamState:
"""Track concatenated text and whether a ``<think>`` block is still open."""
def __init__(self) -> None:
self.text = ""
self.inside = False
self.content_since_open = False
def feed_text(self, chunk: str) -> str:
repaired, self.inside, self.content_since_open = normalize_think_chunk(
chunk,
inside=self.inside,
content_since_open=self.content_since_open,
)
self.text += repaired
return repaired
def close_if_open(self) -> str | None:
if not self.inside:
return None
self.inside = False
self.content_since_open = False
self.text += THINK_CLOSE
return THINK_CLOSE
def _walk(
text: str,
*,
inside: bool,
content_since_open: bool,
close_at_end: bool,
) -> tuple[str, bool, bool]:
parts: list[str] = []
i = 0
length = len(text)
open_len = len(THINK_OPEN)
close_len = len(THINK_CLOSE)
while i < length:
if text.startswith(THINK_OPEN, i):
if inside and content_since_open:
parts.append(THINK_CLOSE)
elif inside:
i += open_len
continue
parts.append(THINK_OPEN)
inside = True
content_since_open = False
i += open_len
continue
if text.startswith(THINK_CLOSE, i):
parts.append(THINK_CLOSE)
inside = False
content_since_open = False
i += close_len
continue
ch = text[i]
parts.append(ch)
if inside and not ch.isspace():
content_since_open = True
i += 1
if close_at_end and inside:
parts.append(THINK_CLOSE)
inside = False
content_since_open = False
return "".join(parts), inside, content_since_open

22
api/events/event_handlers/sync_plugin_trigger_when_app_created.py
View File

@ -1,6 +1,6 @@
import logging
from events.app_event import app_draft_workflow_was_synced
from events.app_event import app_draft_workflow_was_synced, app_published_workflow_was_updated
from models.model import App, AppMode
from models.workflow import Workflow
from services.trigger.trigger_service import TriggerService
@ -9,14 +9,26 @@ logger = logging.getLogger(__name__)
@app_draft_workflow_was_synced.connect
def handle(sender, synced_draft_workflow: Workflow, **kwargs):
@app_published_workflow_was_updated.connect
def handle(
sender,
synced_draft_workflow: Workflow | None = None,
published_workflow: Workflow | None = None,
**kwargs,
):
"""
While creating a workflow or updating a workflow, we may need to sync
its plugin trigger relationships in DB.
Sync plugin trigger relationships when a draft changes or is published.
The published workflow must be reconciled as well because production trigger
dispatch relies on these relationships, while debug dispatch does not.
"""
app: App = sender
if app.mode != AppMode.WORKFLOW.value:
# only handle workflow app, chatflow is not supported yet
return
TriggerService.sync_plugin_trigger_relationships(app, synced_draft_workflow)
workflow = published_workflow if published_workflow is not None else synced_draft_workflow
if workflow is None:
return
TriggerService.sync_plugin_trigger_relationships(app, workflow)

148
api/extensions/ext_application_services.py
View File

@ -1,9 +1,11 @@
"""Composition root for application services used by transport adapters."""
import json
import time
from collections.abc import Mapping
from dataclasses import dataclass
from datetime import UTC, datetime
from functools import partial
from typing import cast
from uuid import uuid4
@ -14,35 +16,52 @@ from sqlalchemy.orm import Session, sessionmaker
from configs import dify_config
from constants.dsl_version import CURRENT_APP_DSL_VERSION
from constants.languages import languages
from core.db.session_factory import get_session_maker
from core.helper.ssrf_proxy import ssrf_proxy
from core.schemas.schema_manager import SchemaManager
from core.tools.tool_file_manager import ToolFileManager
from enums import DeploymentEdition, WebAppAccessMode
from extensions.ext_redis import RedisClientWrapper, redis_client
from libs.datetime_utils import naive_utc_now
from extensions.ext_storage import storage
from libs.datetime_utils import naive_utc_now, utc_now
from libs.helper import RateLimiter
from libs.oauth import GitHubOAuth, GoogleOAuth
from libs.oauth_bearer import invalidate_oauth_token_cache
from libs.passport import PassportService
from repositories.account_activation_repository import SQLAlchemyAccountActivationRepository
from repositories.account_integration_repository import SQLAlchemyAccountIntegrationRepository
from repositories.account_oauth_repository import (
AccountServiceOAuthAccountRegistrationGateway,
AccountServiceOAuthSessionGateway,
AccountServiceOAuthWorkspaceGateway,
RegisterServiceOAuthInvitationGateway,
)
from repositories.account_repository import SQLAlchemyAccountRepository
from repositories.app_definition_query_repository import AppDefinitionQueryRepository
from repositories.app_site_command_repository import AppSiteCommandRepository
from repositories.app_statistic_query_repository import AppStatisticQueryRepository
from repositories.data_source_api_key_auth_repository import SQLAlchemyDataSourceApiKeyAuthBindingRepository
from repositories.data_source_oauth_binding_repository import SQLAlchemyDataSourceOAuthBindingRepository
from repositories.explore_banner_query_repository import ExploreBannerQueryRepository
from repositories.factory import DifyAPIRepositoryFactory
from repositories.file_grant_repository import FileGrantRepository
from repositories.installation_state_repository import InstallationStateRepository
from repositories.oauth_access_token_repository import SQLAlchemyOAuthAccessTokenRepository
from repositories.oauth_server_repository import RedisOAuthServerTokenRepository, SQLAlchemyOAuthServerRepository
from repositories.recommended_app_catalog_repository import DatabaseRecommendedAppCatalogRepository
from repositories.sqlalchemy_api_workflow_run_repository import DifyAPISQLAlchemyWorkflowRunRepository
from repositories.step_by_step_tour_repository import SQLAlchemyStepByStepTourStateRepository
from repositories.tag_repository import TagRepository
from repositories.trial_app_query_repository import TrialAppQueryRepository
from repositories.trial_app_usage_repository import TrialAppUsageRepository
from repositories.web_passport_repository import WebPassportRepository
from repositories.webapp_access_query_repository import WebAppAccessQueryRepository
from repositories.workflow_app_log_query_repository import WorkflowAppLogQueryRepository
from repositories.workflow_run_archive_repository import WorkflowRunArchiveBundleQueryRepository
from repositories.workspace_member_query_repository import WorkspaceMemberQueryRepository
from repositories.workspace_query_repository import WorkspaceQueryRepository
from services.account_access_service import AccountAccessService
from services.account_activation_service import AccountActivationService
from services.account_adapters import (
BillingAccountActivationEligibility,
@ -103,11 +122,18 @@ from services.account_login_adapters import (
TurnstileHumanVerificationGateway,
)
from services.account_login_service import ConsoleAuthenticationService
from services.account_oauth_adapters import (
DeploymentOAuthPolicyGateway,
DifyOAuthProviderGateway,
RedisOAuthAccountClaimLock,
)
from services.account_oauth_service import AccountOAuthService, OAuthProviderGateway
from services.account_password_hasher import DefaultAccountPasswordHasher
from services.account_password_service import AccountPasswordService
from services.account_profile_service import AccountProfileService
from services.app_definition_query_service import AppDefinitionQueryService
from services.app_site_service import AppSiteService
from services.app_statistic_query import AppStatisticQuery
from services.auth.data_source_api_key_auth_gateways import (
ProviderApiKeyAuthCredentialValidator,
TenantApiKeyAuthCredentialEncryptor,
@ -118,10 +144,13 @@ from services.billing_service import BillingService
from services.compliance_download_service import ComplianceDownloadService
from services.data_source_oauth_service import DataSourceOAuthService, InvalidDataSourceOAuthProviderError
from services.enterprise.enterprise_service import EnterpriseService
from services.entities.file_grant_entities import FileGrantLimits
from services.errors.enterprise import EnterpriseServiceError
from services.explore_banner_query_service import ExploreBannerQueryService
from services.feature_query_service import FeatureQueryService
from services.feature_service_gateway import FeatureServiceGateway
from services.file_grant_gateways import FileGrantFileGateway, FileGrantRemoteFileGateway, FileGrantTokenGateway
from services.file_grant_service import FileGrantService
from services.file_service import FileService
from services.init_validation_service import InitValidationService
from services.inner_mail_service import InnerMailService
@ -160,6 +189,8 @@ from services.webapp_access_query_service import (
WebAppAccessQueryService,
WebAppAccessUnavailableError,
)
from services.workflow_app_log_query_service import WorkflowAppLogQueryService
from services.workflow_run_service import WorkflowRunService
from services.workflow_statistic_query_service import WorkflowStatisticQueryService
from services.workspace_member_query_service import WorkspaceMemberQueryService
from services.workspace_member_role_resolver import DeploymentWorkspaceMemberRoleResolver
@ -190,6 +221,7 @@ def _is_user_allowed_to_access_webapp(user_id: str, app_id: str) -> bool:
@dataclass(frozen=True, slots=True)
class AccountServices:
access: AccountAccessService
authentication: ConsoleAuthenticationService
avatar: AccountAvatarService
change_email: AccountChangeEmailService
@ -200,6 +232,7 @@ class AccountServices:
forgot_password: AccountForgotPasswordService
initialization: AccountInitializationService
integrations: AccountIntegrationService
oauth: AccountOAuthService
password: AccountPasswordService
profile: AccountProfileService
@ -210,6 +243,7 @@ class ApplicationServices:
account_activation: AccountActivationService
app_definitions: AppDefinitionQueryService
app_sites: AppSiteService
app_statistics: AppStatisticQuery
billing_portal: BillingPortalService
compliance_downloads: ComplianceDownloadService
data_source_api_key_auth: DataSourceApiKeyAuthService
@ -220,6 +254,7 @@ class ApplicationServices:
schema_definitions: SchemaDefinitionService
setup: SetupService
feature_queries: FeatureQueryService
file_grants: FileGrantService
files: FileService
oauth_server: OAuthServerService
init_validation: InitValidationService
@ -230,8 +265,10 @@ class ApplicationServices:
remote_files: RemoteFileService
trial_app_usage: TrialAppUsageRecorder
workflow_run_archives: WorkflowRunArchiveService
workflow_runs: WorkflowRunService
workspace_queries: WorkspaceQueryService
workspace_member_queries: WorkspaceMemberQueryService
workflow_app_logs: WorkflowAppLogQueryService
inner_mail: InnerMailService
web_passport: WebPassportService
tags: TagApplicationService
@ -278,6 +315,86 @@ def _build_oauth_server_service(
)
def _build_file_grant_service(*, database_client: sessionmaker[Session]) -> FileGrantService:
repository = FileGrantRepository(session_factory=database_client)
return FileGrantService(
repository=repository,
files=FileGrantFileGateway(
load_end_user=repository.get_end_user,
subject_exists=repository.subject_exists,
file_service=FileService(session_factory=database_client),
tool_files=ToolFileManager(),
storage=storage,
),
remote_files=FileGrantRemoteFileGateway(),
tokens=FileGrantTokenGateway(
secret_key=dify_config.SECRET_KEY,
external_files_url=dify_config.FILES_URL,
internal_files_url=dify_config.INTERNAL_FILES_URL or dify_config.FILES_URL,
content_token_ttl_seconds=dify_config.FILES_ACCESS_TIMEOUT,
now=lambda: int(time.time()),
),
limits=FileGrantLimits(
file_size_limit=dify_config.UPLOAD_FILE_SIZE_LIMIT,
image_file_size_limit=dify_config.UPLOAD_IMAGE_FILE_SIZE_LIMIT,
audio_file_size_limit=dify_config.UPLOAD_AUDIO_FILE_SIZE_LIMIT,
video_file_size_limit=dify_config.UPLOAD_VIDEO_FILE_SIZE_LIMIT,
workflow_file_upload_limit=dify_config.WORKFLOW_FILE_UPLOAD_LIMIT,
batch_count_limit=dify_config.UPLOAD_FILE_BATCH_LIMIT,
),
now=lambda: int(time.time()),
)
def _build_account_oauth_service(
*,
database_client: sessionmaker[Session],
deployment_edition: DeploymentEdition,
redis: RedisClientWrapper,
accounts: SQLAlchemyAccountRepository,
integrations: SQLAlchemyAccountIntegrationRepository,
memberships: WorkspaceQueryRepository,
) -> AccountOAuthService:
providers: dict[str, OAuthProviderGateway] = {}
if dify_config.GITHUB_CLIENT_ID and dify_config.GITHUB_CLIENT_SECRET:
providers["github"] = DifyOAuthProviderGateway(
provider_name="github",
client=GitHubOAuth(
client_id=dify_config.GITHUB_CLIENT_ID,
client_secret=dify_config.GITHUB_CLIENT_SECRET,
redirect_uri=dify_config.CONSOLE_API_URL + "/console/api/oauth/authorize/github",
),
)
if dify_config.GOOGLE_CLIENT_ID and dify_config.GOOGLE_CLIENT_SECRET:
providers["google"] = DifyOAuthProviderGateway(
provider_name="google",
client=GoogleOAuth(
client_id=dify_config.GOOGLE_CLIENT_ID,
client_secret=dify_config.GOOGLE_CLIENT_SECRET,
redirect_uri=dify_config.CONSOLE_API_URL + "/console/api/oauth/authorize/google",
),
)
policy = DeploymentOAuthPolicyGateway(
billing_enabled=deployment_edition == DeploymentEdition.CLOUD,
)
return AccountOAuthService(
providers=providers,
accounts=accounts,
integrations=integrations,
memberships=memberships,
invitations=RegisterServiceOAuthInvitationGateway(session_factory=database_client),
account_claims=RedisOAuthAccountClaimLock(client=redis),
registration=AccountServiceOAuthAccountRegistrationGateway(session_factory=database_client),
workspaces=AccountServiceOAuthWorkspaceGateway(session_factory=database_client),
sessions=AccountServiceOAuthSessionGateway(session_factory=database_client),
registration_policy=policy,
workspace_policy=policy,
supported_languages=languages,
now=naive_utc_now,
)
def build_application_services(
*,
database_client: sessionmaker[Session],
@ -306,8 +423,19 @@ def build_application_services(
invitation_tokens = RedisInvitationTokenStore(redis=redis)
activation_accounts = SQLAlchemyAccountActivationRepository(session_factory=database_client)
account_provisioning = SQLAlchemyConsoleAuthProvisioningGateway(session_factory=database_client)
workflow_run_repository = DifyAPISQLAlchemyWorkflowRunRepository(session_maker=database_client)
workflow_node_execution_repository = DifyAPIRepositoryFactory.create_api_workflow_node_execution_repository(
session_maker=database_client
)
return ApplicationServices(
accounts=AccountServices(
access=AccountAccessService(
accounts=accounts,
workspaces=workspace_query_repository,
sessions=SQLAlchemyOAuthAccessTokenRepository(session_factory=database_client),
invalidate_token_cache=partial(invalidate_oauth_token_cache, redis),
now=utc_now,
),
authentication=ConsoleAuthenticationService(
accounts=accounts,
workspaces=workspace_query_repository,
@ -426,6 +554,14 @@ def build_application_services(
now=naive_utc_now,
),
integrations=AccountIntegrationService(integrations=integrations),
oauth=_build_account_oauth_service(
database_client=database_client,
deployment_edition=deployment_edition,
redis=redis,
accounts=accounts,
integrations=integrations,
memberships=workspace_query_repository,
),
password=AccountPasswordService(
accounts=accounts,
passwords=passwords,
@ -455,6 +591,7 @@ def build_application_services(
app_sites=AppSiteService(
sites=AppSiteCommandRepository(session_factory=database_client),
),
app_statistics=AppStatisticQueryRepository(session_factory=database_client),
billing_portal=BillingPortalService(
accounts=accounts,
get_subscription=BillingService.get_subscription,
@ -502,6 +639,7 @@ def build_application_services(
features=feature_gateway,
app_dsl_version=CURRENT_APP_DSL_VERSION,
),
file_grants=_build_file_grant_service(database_client=database_client),
files=file_service,
oauth_server=_build_oauth_server_service(database_client=database_client, redis=redis),
init_validation=InitValidationService(
@ -510,7 +648,6 @@ def build_application_services(
expected_password=initialization_password,
),
notifications=NotificationService(
accounts=accounts,
notifications=BillingNotificationGateway(),
),
step_by_step_tour=StepByStepTourService(
@ -537,6 +674,10 @@ def build_application_services(
dispatcher=dispatch_workflow_run_archive_download_task,
sign_download_url=sign_workflow_run_archive_download_url,
),
workflow_runs=WorkflowRunService(
workflow_runs=workflow_run_repository,
node_executions=workflow_node_execution_repository,
),
workspace_queries=WorkspaceQueryService(
workspaces=workspace_query_repository,
plans=DeploymentWorkspacePlanGateway(),
@ -547,6 +688,9 @@ def build_application_services(
),
roles=DeploymentWorkspaceMemberRoleResolver(),
),
workflow_app_logs=WorkflowAppLogQueryService(
logs=WorkflowAppLogQueryRepository(session_factory=database_client),
),
inner_mail=InnerMailService(dispatch=enqueue_inner_mail),
web_passport=WebPassportService(
passports=WebPassportRepository(

49
api/fields/file_grant_fields.py Normal file
View File

@ -0,0 +1,49 @@
"""Response DTOs shared by the AppDeploy file grant surfaces."""
from __future__ import annotations
from fields.base import ResponseModel
from services.entities.file_grant_entities import FileKind, ResolvedFileAccess
class ResolvedFileResponse(ResponseModel):
"""One requested file reference, either resolved or accounted for.
The resolve endpoint and optional mint references answer item by item so one
missing history file cannot fail a whole run. A file that exists but belongs
to another owner is reported exactly like one that never existed.
"""
id: str
ok: bool
kind: FileKind | None = None
name: str | None = None
size: int | None = None
extension: str | None = None
mime_type: str | None = None
url: str | None = None
internal_url: str | None = None
error: str | None = None
@classmethod
def from_resolved(cls, file_id: str, access: ResolvedFileAccess | None) -> ResolvedFileResponse:
"""Describe one resolved reference without performing infrastructure work."""
if access is None:
return cls(id=file_id, ok=False, error="not_found")
file = access.file
return cls(
id=file.id,
ok=True,
kind=file.kind,
name=file.name,
size=file.size,
extension=file.extension,
mime_type=file.mime_type,
url=access.external_url,
internal_url=access.internal_url,
)
__all__ = ["ResolvedFileResponse"]

7
api/libs/datetime_utils.py
View File

@ -17,11 +17,16 @@ class _NowFunction(Protocol):
_now_func: _NowFunction = datetime.datetime.now
def utc_now() -> datetime.datetime:
"""Return a timezone-aware datetime representing the current UTC time."""
return _now_func(datetime.UTC)
def naive_utc_now() -> datetime.datetime:
"""Return a naive datetime object (without timezone information)
representing current UTC time.
"""
return _now_func(datetime.UTC).replace(tzinfo=None)
return utc_now().replace(tzinfo=None)
def ensure_naive_utc(dt: datetime.datetime) -> datetime.datetime:

10
api/libs/oauth_bearer.py
View File

@ -318,6 +318,14 @@ AUDIT_OAUTH_EXPIRED = "oauth.token_expired"
ScopeVariant = Literal["account", "external_sso"]
class _TokenCacheClient(Protocol):
def delete(self, *names: str | bytes) -> object: ...
def invalidate_oauth_token_cache(client: _TokenCacheClient, token_hash: str) -> None:
client.delete(TOKEN_CACHE_KEY_FMT.format(hash=token_hash))
class OAuthAccessTokenResolver:
"""``.for_account()`` / ``.for_external_sso()`` are variant-scoped views
sharing DB + cache plumbing.
@ -385,7 +393,7 @@ class OAuthAccessTokenResolver:
row_id,
extra={"audit": True, "token_id": str(row_id)},
)
self._redis.delete(self._cache_key(token_hash))
invalidate_oauth_token_cache(self._redis, token_hash)
self.cache_set_negative(token_hash)

9
api/machinery/context.py
View File

@ -8,3 +8,12 @@ class RequestContext(NamedTuple):
trace_id: str | None
account_id: str
active_workspace_id: str
class AccountRequestContext(NamedTuple):
"""Stable identity for account-scoped use cases that do not require a workspace."""
request_id: str
trace_id: str | None
account_id: str
access_token_id: str | None = None

11
api/models/account.py
View File

@ -187,15 +187,6 @@ class Account(UserMixin, TypeBase):
def get_status(self) -> AccountStatus:
return self.status
@classmethod
def get_by_openid(cls, provider: str, open_id: str):
account_integrate = db.session.execute(
select(AccountIntegrate).where(AccountIntegrate.provider == provider, AccountIntegrate.open_id == open_id)
).scalar_one_or_none()
if account_integrate:
return db.session.scalar(select(Account).where(Account.id == account_integrate.account_id))
return None
# check current_user.current_tenant.current_role in ['admin', 'owner']
@property
def is_admin_or_owner(self):
@ -312,7 +303,7 @@ class TenantAccountJoin(TypeBase):
)
tenant_id: Mapped[str] = mapped_column(StringUUID)
account_id: Mapped[str] = mapped_column(StringUUID)
current: Mapped[bool] = mapped_column(sa.Boolean, server_default=sa.text("false"), default=False)
current: Mapped[bool] = mapped_column(sa.Boolean, server_default=sa.false(), default=False)
role: Mapped[TenantAccountRole] = mapped_column(
EnumText(TenantAccountRole, length=16), server_default="normal", default=TenantAccountRole.NORMAL
)

4
api/models/agent.py
View File

@ -189,13 +189,13 @@ class Agent(DefaultFieldsMixin, Base):
workflow_node_id: Mapped[str | None] = mapped_column(String(255), nullable=True)
active_config_snapshot_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
active_config_has_model: Mapped[bool] = mapped_column(
sa.Boolean, nullable=False, default=False, server_default=sa.text("false")
sa.Boolean, nullable=False, default=False, server_default=sa.false()
)
active_config_is_published: Mapped[bool] = mapped_column(
sa.Boolean,
nullable=False,
default=False,
server_default=sa.text("false"),
server_default=sa.false(),
comment=(
"Whether the normal shared Agent draft has been published into the active config snapshot. "
"User-scoped debug drafts do not affect this flag."

2
api/models/comment.py
View File

@ -64,7 +64,7 @@ class WorkflowComment(TypeBase):
resolved_at: Mapped[datetime | None] = mapped_column(sa.DateTime, default=None)
resolved_by: Mapped[str | None] = mapped_column(StringUUID, default=None)
resolved: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"), default=False)
resolved: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
# Relationships
replies: Mapped[list[WorkflowCommentReply]] = relationship(
lambda: WorkflowCommentReply, back_populates="comment", cascade="all, delete-orphan", init=False

4
api/models/credential_permission.py
View File

@ -45,9 +45,7 @@ class CredentialPermission(TypeBase):
credential_type: Mapped[str] = mapped_column(String(40), nullable=False)
account_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
has_permission: Mapped[bool] = mapped_column(
sa.Boolean, nullable=False, server_default=sa.text("true"), default=True
)
has_permission: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.true(), default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime, nullable=False, server_default=func.current_timestamp(), init=False
)

48
api/models/dataset.py
View File

@ -202,15 +202,15 @@ class Dataset(Base):
collection_binding_id = mapped_column(StringUUID, nullable=True)
retrieval_model = mapped_column(AdjustedJSON, nullable=True)
summary_index_setting = mapped_column(AdjustedJSON, nullable=True)
built_in_field_enabled = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"))
built_in_field_enabled = mapped_column(sa.Boolean, nullable=False, server_default=sa.false())
icon_info = mapped_column(AdjustedJSON, nullable=True)
runtime_mode = mapped_column(
EnumText(DatasetRuntimeMode, length=255), nullable=True, server_default=sa.text("'general'")
)
pipeline_id = mapped_column(StringUUID, nullable=True)
chunk_structure = mapped_column(sa.String(255), nullable=True)
enable_api = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("true"))
is_multimodal = mapped_column(sa.Boolean, default=False, nullable=False, server_default=sa.text("false"))
enable_api = mapped_column(sa.Boolean, nullable=False, server_default=sa.true())
is_multimodal = mapped_column(sa.Boolean, default=False, nullable=False, server_default=sa.false())
def get_total_documents(self, *, session: Session) -> int:
return self.get_document_count(session=session)
@ -560,7 +560,7 @@ class Document(Base):
completed_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
# pause
is_paused: Mapped[bool | None] = mapped_column(sa.Boolean, nullable=True, server_default=sa.text("false"))
is_paused: Mapped[bool | None] = mapped_column(sa.Boolean, nullable=True, server_default=sa.false())
paused_by = mapped_column(StringUUID, nullable=True)
paused_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
@ -572,10 +572,10 @@ class Document(Base):
indexing_status = mapped_column(
EnumText(IndexingStatus, length=255), nullable=False, server_default=sa.text("'waiting'")
)
enabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("true"))
enabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.true())
disabled_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
disabled_by = mapped_column(StringUUID, nullable=True)
archived: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"))
archived: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false())
archived_reason = mapped_column(String(255), nullable=True)
archived_by = mapped_column(StringUUID, nullable=True)
archived_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True)
@ -588,7 +588,7 @@ class Document(Base):
EnumText(IndexStructureType, length=255), nullable=False, server_default=sa.text("'text_model'")
)
doc_language = mapped_column(String(255), nullable=True)
need_summary: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"))
need_summary: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false())
DATA_SOURCES = ["upload_file", "notion_import", "website_crawl"]
@ -622,10 +622,6 @@ class Document(Base):
return data_source_info_dict
return {}
@property
def data_source_detail_dict(self) -> dict[str, Any]:
return self.get_data_source_detail_dict(session=db.session())
def get_data_source_detail_dict(self, *, session: Session) -> dict[str, Any]:
if self.data_source_info:
if self.data_source_type == "upload_file":
@ -665,10 +661,6 @@ class Document(Base):
return session.get(DatasetProcessRule, self.dataset_process_rule_id)
return None
@property
def dataset(self) -> Dataset | None:
return self.get_dataset(session=db.session())
def get_dataset(self, *, session: Session) -> Dataset | None:
"""Load the owning dataset with the caller-owned database session."""
return session.get(Dataset, self.dataset_id)
@ -694,10 +686,6 @@ class Document(Base):
or 0
)
@property
def uploader(self):
return self.get_uploader(session=db.session())
def get_uploader(self, *, session: Session) -> str | None:
user = session.scalar(select(Account).where(Account.id == self.created_by))
return user.name if user else None
@ -710,10 +698,6 @@ class Document(Base):
def last_update_date(self):
return self.updated_at
@property
def doc_metadata_details(self) -> list[DocMetadataDetailItem] | None:
return self.get_doc_metadata_details(session=db.session())
def get_doc_metadata_details(self, *, session: Session) -> list[DocMetadataDetailItem] | None:
if self.doc_metadata:
document_metadatas = session.scalars(
@ -917,7 +901,7 @@ class DocumentSegment(TypeBase):
# indexing fields
index_node_id: Mapped[str | None] = mapped_column(String(255), nullable=True, default=None)
index_node_hash: Mapped[str | None] = mapped_column(String(255), nullable=True, default=None)
enabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("true"), default=True)
enabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.true(), default=True)
answer: Mapped[str | None] = mapped_column(LongText, nullable=True, default=None)
keywords: Mapped[Any] = mapped_column(sa.JSON, nullable=True, default=None)
disabled_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True, default=None)
@ -1378,7 +1362,7 @@ class TidbAuthBinding(TypeBase):
tenant_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
cluster_id: Mapped[str] = mapped_column(String(255), nullable=False)
cluster_name: Mapped[str] = mapped_column(String(255), nullable=False)
active: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"))
active: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false())
status: Mapped[TidbAuthBindingStatus] = mapped_column(
EnumText(TidbAuthBindingStatus, length=255), nullable=False, server_default=sa.text("'CREATING'")
)
@ -1429,9 +1413,7 @@ class DatasetPermission(TypeBase):
dataset_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
account_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
has_permission: Mapped[bool] = mapped_column(
sa.Boolean, nullable=False, server_default=sa.text("true"), default=True
)
has_permission: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.true(), default=True)
created_at: Mapped[datetime] = mapped_column(
DateTime, nullable=False, server_default=func.current_timestamp(), init=False
)
@ -1547,7 +1529,7 @@ class DatasetAutoDisableLog(TypeBase):
tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
dataset_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
document_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
notified: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"), default=False)
notified: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
created_at: Mapped[datetime] = mapped_column(
DateTime, nullable=False, server_default=sa.func.current_timestamp(), init=False
)
@ -1703,10 +1685,8 @@ class Pipeline(TypeBase):
name: Mapped[str] = mapped_column(sa.String(255), nullable=False)
description: Mapped[str] = mapped_column(LongText, nullable=False, default=sa.text("''"))
workflow_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True, default=None)
is_public: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"), default=False)
is_published: Mapped[bool] = mapped_column(
sa.Boolean, nullable=False, server_default=sa.text("false"), default=False
)
is_public: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
is_published: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
created_by: Mapped[str | None] = mapped_column(StringUUID, nullable=True, default=None)
created_at: Mapped[datetime] = mapped_column(
sa.DateTime, nullable=False, server_default=func.current_timestamp(), init=False
@ -1830,7 +1810,7 @@ class DocumentSegmentSummary(TypeBase):
default=SummaryStatus.GENERATING,
)
error: Mapped[str | None] = mapped_column(LongText, nullable=True, default=None)
enabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("true"), default=True)
enabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.true(), default=True)
disabled_at: Mapped[datetime | None] = mapped_column(DateTime, nullable=True, default=None)
disabled_by: Mapped[str | None] = mapped_column(StringUUID, nullable=True, default=None)
created_at: Mapped[datetime] = mapped_column(

1
api/models/enums.py
View File

@ -208,6 +208,7 @@ class InvokeFrom(StrEnum):
class EndUserType(StrEnum):
"""Persisted type values for the ``end_users.type`` column."""
APP_DEPLOY = "app-deploy"
BROWSER = "browser"
MCP = "mcp"
OPENAPI = "openapi"

40
api/models/model.py
View File

@ -435,9 +435,9 @@ class App(Base):
enable_api: Mapped[bool] = mapped_column(sa.Boolean)
api_rpm: Mapped[int] = mapped_column(sa.Integer, server_default=sa.text("0"))
api_rph: Mapped[int] = mapped_column(sa.Integer, server_default=sa.text("0"))
is_demo: Mapped[bool] = mapped_column(sa.Boolean, server_default=sa.text("false"))
is_public: Mapped[bool] = mapped_column(sa.Boolean, server_default=sa.text("false"))
is_universal: Mapped[bool] = mapped_column(sa.Boolean, server_default=sa.text("false"))
is_demo: Mapped[bool] = mapped_column(sa.Boolean, server_default=sa.false())
is_public: Mapped[bool] = mapped_column(sa.Boolean, server_default=sa.false())
is_universal: Mapped[bool] = mapped_column(sa.Boolean, server_default=sa.false())
tracing = mapped_column(LongText, nullable=True)
max_active_requests: Mapped[int | None]
created_by = mapped_column(StringUUID, nullable=True)
@ -447,7 +447,7 @@ class App(Base):
updated_at: Mapped[datetime] = mapped_column(
sa.DateTime, nullable=False, server_default=func.current_timestamp(), onupdate=func.current_timestamp()
)
use_icon_as_answer_icon: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"))
use_icon_as_answer_icon: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false())
@property
def desc_or_prompt(self) -> str:
@ -989,12 +989,8 @@ class RecommendedApp(TypeBase):
custom_disclaimer: Mapped[str] = mapped_column(LongText, default="")
position: Mapped[int] = mapped_column(sa.Integer, nullable=False, default=0)
is_listed: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, default=True)
is_learn_dify: Mapped[bool] = mapped_column(
sa.Boolean, nullable=False, server_default=sa.text("false"), default=False
)
is_cloud_only: Mapped[bool] = mapped_column(
sa.Boolean, nullable=False, server_default=sa.text("false"), default=False
)
is_learn_dify: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
is_cloud_only: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
install_count: Mapped[int] = mapped_column(sa.Integer, nullable=False, default=0)
language: Mapped[str] = mapped_column(
String(255),
@ -1034,7 +1030,7 @@ class InstalledApp(TypeBase):
app_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
app_owner_tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
position: Mapped[int] = mapped_column(sa.Integer, nullable=False, default=0)
is_pinned: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"), default=False)
is_pinned: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
last_used_at: Mapped[datetime | None] = mapped_column(sa.DateTime, nullable=True, default=None)
created_at: Mapped[datetime] = mapped_column(
sa.DateTime, nullable=False, server_default=func.current_timestamp(), init=False
@ -1239,7 +1235,7 @@ class Conversation(Base):
lambda: MessageAnnotation, backref="conversation", lazy="select", passive_deletes="all"
)
is_deleted: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"))
is_deleted: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false())
@property
def inputs(self) -> dict[str, Any]:
@ -1598,7 +1594,7 @@ class Message(Base):
updated_at: Mapped[datetime] = mapped_column(
sa.DateTime, nullable=False, server_default=func.current_timestamp(), onupdate=func.current_timestamp()
)
agent_based: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"))
agent_based: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false())
workflow_run_id: Mapped[str | None] = mapped_column(StringUUID)
app_mode: Mapped[AppMode | None] = mapped_column(EnumText(AppMode, length=255), nullable=True)
@ -2249,9 +2245,7 @@ class EndUser(Base, UserMixin):
type: Mapped[EndUserType] = mapped_column(EnumText(EndUserType, length=255), nullable=False)
external_user_id = mapped_column(String(255), nullable=True)
name = mapped_column(String(255))
_is_anonymous: Mapped[bool] = mapped_column(
"is_anonymous", sa.Boolean, nullable=False, server_default=sa.text("true")
)
_is_anonymous: Mapped[bool] = mapped_column("is_anonymous", sa.Boolean, nullable=False, server_default=sa.true())
@property
@override
@ -2366,17 +2360,15 @@ class Site(TypeBase):
customize_domain: Mapped[str | None] = mapped_column(String(255), nullable=True, default=None)
chat_color_theme: Mapped[str | None] = mapped_column(String(255), nullable=True, default=None)
prompt_public: Mapped[bool] = mapped_column(
sa.Boolean, nullable=False, server_default=sa.text("false"), default=False
)
prompt_public: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
chat_color_theme_inverted: Mapped[bool] = mapped_column(
sa.Boolean, nullable=False, server_default=sa.text("false"), default=False
sa.Boolean, nullable=False, server_default=sa.false(), default=False
)
show_workflow_steps: Mapped[bool] = mapped_column(
sa.Boolean, nullable=False, server_default=sa.text("true"), default=True
sa.Boolean, nullable=False, server_default=sa.true(), default=True
)
use_icon_as_answer_icon: Mapped[bool] = mapped_column(
sa.Boolean, nullable=False, server_default=sa.text("false"), default=False
sa.Boolean, nullable=False, server_default=sa.false(), default=False
)
custom_disclaimer: Mapped[str] = mapped_column(LongText, nullable=False, default="")
status: Mapped[AppStatus] = mapped_column(
@ -2535,7 +2527,7 @@ class UploadFile(TypeBase):
# 3. Avoid relying on these fields for logic, as their values may not always be accurate.
#
# `used` may indicate whether the file has been utilized by another service.
used: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"))
used: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false())
# The `created_by_role` field indicates whether the file was created by an `Account` or an `EndUser`.
# Its value is derived from the `CreatorUserRole` enumeration.
@ -2846,7 +2838,7 @@ class TraceAppConfig(TypeBase):
onupdate=func.current_timestamp(),
init=False,
)
is_active: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("true"), default=True)
is_active: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.true(), default=True)
@property
def tracing_config_dict(self) -> dict[str, Any]:

2
api/models/oauth.py
View File

@ -45,7 +45,7 @@ class DatasourceProvider(TypeBase):
encrypted_credentials: Mapped[dict[str, Any]] = mapped_column(AdjustedJSON, nullable=False)
user_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True, default=None)
avatar_url: Mapped[str] = mapped_column(LongText, nullable=True, default="default")
is_default: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"), default=False)
is_default: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
expires_at: Mapped[int] = mapped_column(sa.Integer, nullable=False, server_default="-1", default=-1)
visibility: Mapped[PermissionEnum] = mapped_column(
EnumText(PermissionEnum, length=40),

2
api/models/onboarding.py
View File

@ -32,7 +32,7 @@ class AccountStepByStepTourState(TypeBase):
)
account_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
first_workspace_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True, default=None)
skipped: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"), default=False)
skipped: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
completed_task_ids: Mapped[list[str]] = mapped_column(AdjustedJSON, nullable=False, default_factory=list)
manually_enabled_workspace_ids: Mapped[list[str]] = mapped_column(
AdjustedJSON,

2
api/models/snippet.py
View File

@ -47,7 +47,7 @@ class CustomizedSnippet(Base):
workflow_id: Mapped[str | None] = mapped_column(StringUUID, nullable=True)
# State flags
is_published: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"))
is_published: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false())
version: Mapped[int] = mapped_column(sa.Integer, nullable=False, server_default=sa.text("1"))
use_count: Mapped[int] = mapped_column(sa.Integer, nullable=False, server_default=sa.text("0"))

4
api/models/source.py
View File

@ -36,7 +36,7 @@ class DataSourceOauthBinding(TypeBase):
onupdate=func.current_timestamp(),
init=False,
)
disabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=True, server_default=sa.text("false"), default=False)
disabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=True, server_default=sa.false(), default=False)
class DataSourceApiKeyAuthBindingDict(TypedDict):
@ -75,7 +75,7 @@ class DataSourceApiKeyAuthBinding(TypeBase):
onupdate=func.current_timestamp(),
init=False,
)
disabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=True, server_default=sa.text("false"), default=False)
disabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=True, server_default=sa.false(), default=False)
def to_dict(self) -> DataSourceApiKeyAuthBindingDict:
result: DataSourceApiKeyAuthBindingDict = {

4
api/models/tools.py
View File

@ -62,7 +62,7 @@ class ToolOAuthTenantClient(TypeBase):
tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
plugin_id: Mapped[str] = mapped_column(String(255), nullable=False)
provider: Mapped[str] = mapped_column(String(255), nullable=False)
enabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("true"), init=False)
enabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.true(), init=False)
# oauth params of the tool provider
encrypted_oauth_params: Mapped[str] = mapped_column(LongText, nullable=False, init=False)
@ -109,7 +109,7 @@ class BuiltinToolProvider(TypeBase):
onupdate=func.current_timestamp(),
init=False,
)
is_default: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("false"), default=False)
is_default: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.false(), default=False)
# credential type, e.g., "api-key", "oauth2"
credential_type: Mapped[CredentialType] = mapped_column(
EnumText(CredentialType, length=32),

2
api/models/trigger.py
View File

@ -199,7 +199,7 @@ class TriggerOAuthTenantClient(TypeBase):
tenant_id: Mapped[str] = mapped_column(StringUUID, nullable=False)
plugin_id: Mapped[str] = mapped_column(String(255), nullable=False)
provider: Mapped[str] = mapped_column(String(255), nullable=False)
enabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.text("true"), default=True)
enabled: Mapped[bool] = mapped_column(sa.Boolean, nullable=False, server_default=sa.true(), default=True)
# oauth params of the trigger provider
encrypted_oauth_params: Mapped[str] = mapped_column(LongText, nullable=False, default="{}")
created_at: Mapped[datetime] = mapped_column(

30
api/openapi/markdown/console-openapi.md
View File

@ -6570,7 +6570,7 @@ Check if dataset is in use
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| language | query | Language code for recommended app localization | No | string |
| language | query | Language code for recommended app localization | No | string, <br>**Default:** en-US |
#### Responses
@ -6583,7 +6583,7 @@ Check if dataset is in use
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| language | query | Language code for recommended app localization | No | string |
| language | query | Language code for recommended app localization | No | string, <br>**Default:** en-US |
#### Responses
@ -9307,7 +9307,13 @@ Claim a workspace-staged upload. Multipart file bodies remain accepted as a lega
| 302 | Redirect to OAuth callback page |
### [GET] /notification
Return the active in-product notification for the current user in their interface language (falls back to English if unavailable). The notification is NOT marked as seen here; call POST /notification/dismiss when the user explicitly closes the modal.
Return the active in-product notification for the current user in the requested language (defaults to English when omitted). Unavailable translations fall back to English, then the first available content. The notification is NOT marked as seen here; call POST /notification/dismiss when the user explicitly closes the modal.
#### Parameters
| Name | Located in | Description | Required | Schema |
| ---- | ---------- | ----------- | -------- | ------ |
| language | query | Notification language | No | string, <br>**Default:** en-US |
#### Responses
@ -9377,7 +9383,7 @@ Handle OAuth callback and complete login process
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 302 | Redirect to console with access token | **application/json**: [RedirectResponse](#redirectresponse)<br> |
| 400 | OAuth process failed | |
| 400 | OAuth process failed | **application/json**: [OAuthErrorResponse](#oautherrorresponse)<br> |
### [GET] /oauth/data-source/binding/{provider}
Bind OAuth data source with authorization code
@ -9466,7 +9472,7 @@ Initiate OAuth login process
| Code | Description | Schema |
| ---- | ----------- | ------ |
| 302 | Redirect to OAuth authorization URL | **application/json**: [RedirectResponse](#redirectresponse)<br> |
| 400 | Invalid provider | |
| 400 | Invalid provider | **application/json**: [OAuthErrorResponse](#oautherrorresponse)<br> |
### [GET] /oauth/plugin/{provider_id}/datasource/callback
#### Parameters
@ -24403,6 +24409,12 @@ Coarse node-level status used by Inspector to pick a banner.
| title | string | | Yes |
| title_pic_url | string | | Yes |
#### NotificationQuery
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| language | string, <br>**Default:** en-US | Notification language | No |
#### NotificationResponse
| Name | Type | Description | Required |
@ -24514,6 +24526,12 @@ Coarse node-level status used by Inspector to pick a banner.
| ---- | ---- | ----------- | -------- |
| result | string | Operation result | Yes |
#### OAuthErrorResponse
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| error | string | OAuth error message | Yes |
#### OAuthLoginQuery
| Name | Type | Description | Required |
@ -26046,7 +26064,7 @@ Model class for provider quota configuration.
| Name | Type | Description | Required |
| ---- | ---- | ----------- | -------- |
| language | string | Language code for recommended app localization | No |
| language | string, <br>**Default:** en-US | Language code for recommended app localization | No |
#### RedirectResponse

33
api/repositories/account_integration_repository.py
View File

@ -14,6 +14,15 @@ class SQLAlchemyAccountIntegrationRepository(AccountIntegrationRepository):
def __init__(self, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
@override
def find_account_id(self, *, provider: str, open_id: str) -> str | None:
with self._session_factory() as session:
return session.scalar(
select(AccountIntegrate.account_id)
.where(AccountIntegrate.provider == provider, AccountIntegrate.open_id == open_id)
.limit(1)
)
@override
def list_for_account(self, account_id: str) -> list[AccountIntegrationSnapshot]:
with self._session_factory() as session:
@ -23,3 +32,27 @@ class SQLAlchemyAccountIntegrationRepository(AccountIntegrationRepository):
)
).all()
return [AccountIntegrationSnapshot(provider=row.provider, created_at=row.created_at) for row in rows]
@override
def link(self, account_id: str, *, provider: str, open_id: str) -> None:
with self._session_factory.begin() as session:
integration = session.scalar(
select(AccountIntegrate)
.where(
AccountIntegrate.account_id == account_id,
AccountIntegrate.provider == provider,
)
.limit(1)
)
if integration is None:
session.add(
AccountIntegrate(
account_id=account_id,
provider=provider,
open_id=open_id,
encrypted_token="",
)
)
return
integration.open_id = open_id
integration.encrypted_token = ""

128
api/repositories/account_oauth_repository.py Normal file
View File

@ -0,0 +1,128 @@
"""Persistence-backed gateways for Console account OAuth sign-in."""
from typing import override
from sqlalchemy.orm import Session, sessionmaker
from libs.datetime_utils import naive_utc_now
from models.account import Account, AccountStatus
from services.account_errors import (
AccountEmailDomainSuspendedError,
OAuthAccountNotFoundError,
OAuthRegistrationError,
OAuthSeatsLimitExceededError,
OAuthWorkspaceCreationNotAllowedError,
)
from services.account_oauth_service import (
OAuthAccountRegistrationGateway,
OAuthInvitationGateway,
OAuthSessionGateway,
OAuthWorkspaceGateway,
)
from services.account_service import AccountService, RegisterService, TenantService
from services.enterprise.enterprise_service import try_join_default_workspace
from services.entities.account_entities import AccountSessionTokens
from services.entities.account_oauth_entities import (
OAuthAccountRegistration,
OAuthInvitation,
)
from services.errors.account import AccountRegisterError, EmailDomainSuspendedError, SeatsLimitExceededError
from services.errors.workspace import WorkSpaceNotAllowedCreateError, WorkspacesLimitExceededError
class RegisterServiceOAuthInvitationGateway(OAuthInvitationGateway):
def __init__(self, *, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
@override
def resolve(self, invite_token: str) -> OAuthInvitation | None:
with self._session_factory() as session:
invitation = RegisterService.get_invitation_if_token_valid(
None,
None,
invite_token,
session=session,
)
if invitation is None:
return None
account = invitation["account"]
return OAuthInvitation(
account_id=account.id,
account_email=account.email,
account_status=account.status.value,
)
class AccountServiceOAuthAccountRegistrationGateway(OAuthAccountRegistrationGateway):
def __init__(self, *, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
@override
def register(self, registration: OAuthAccountRegistration) -> str:
with self._session_factory() as session:
try:
account = AccountService.create_account(
email=registration.email,
name=registration.name,
interface_language=registration.language,
password=None,
timezone=registration.timezone,
ip_address=registration.ip_address,
check_normalized_email=True,
session=session,
)
account.status = AccountStatus.ACTIVE
account.initialized_at = naive_utc_now()
session.commit()
except EmailDomainSuspendedError as exc:
raise AccountEmailDomainSuspendedError from exc
except SeatsLimitExceededError as exc:
raise OAuthSeatsLimitExceededError from exc
except AccountRegisterError as exc:
raise OAuthRegistrationError(exc.description) from exc
except Exception as exc:
session.rollback()
raise OAuthRegistrationError(f"Registration failed: {exc}") from exc
return account.id
class AccountServiceOAuthWorkspaceGateway(OAuthWorkspaceGateway):
"""Adapt account workspace operations to the OAuth application port."""
def __init__(self, *, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
@override
def create_owner_workspace(self, account_id: str) -> None:
with self._session_factory() as session:
account = session.get(Account, account_id)
if account is None:
raise OAuthAccountNotFoundError
try:
TenantService.create_owner_tenant(account, session=session)
except (WorkSpaceNotAllowedCreateError, WorkspacesLimitExceededError) as exc:
raise OAuthWorkspaceCreationNotAllowedError from exc
@override
def try_join_default_workspace(self, account_id: str) -> None:
try_join_default_workspace(account_id)
class AccountServiceOAuthSessionGateway(OAuthSessionGateway):
"""Adapt Console session issuance to the OAuth application port."""
def __init__(self, *, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
@override
def login(self, account_id: str, *, ip_address: str) -> AccountSessionTokens:
with self._session_factory() as session:
account = session.get(Account, account_id)
if account is None:
raise OAuthAccountNotFoundError
token_pair = AccountService.login(account=account, session=session, ip_address=ip_address)
return AccountSessionTokens(
access_token=token_pair.access_token,
refresh_token=token_pair.refresh_token,
csrf_token=token_pair.csrf_token,
)

10
api/repositories/account_repository.py
View File

@ -1,5 +1,6 @@
"""SQLAlchemy implementation of the account persistence port."""
from datetime import datetime
from typing import override
from sqlalchemy import case, delete, select
@ -45,6 +46,15 @@ class SQLAlchemyAccountRepository(AccountRepository, ConsoleAuthAccountRepositor
account = session.execute(select(Account).where(Account.email == email.lower())).scalar_one_or_none()
return self._to_snapshot(account) if account is not None else None
@override
def activate_pending(self, account_id: str, *, initialized_at: datetime) -> None:
with self._session_factory.begin() as session:
account = session.get(Account, account_id)
if account is None or account.status != AccountStatus.PENDING:
return
account.status = AccountStatus.ACTIVE
account.initialized_at = initialized_at
@override
def get_credentials(self, account_id: str) -> AccountCredentials | None:
with self._session_factory() as session:

3
api/repositories/api_workflow_run_repository.py
View File

@ -41,7 +41,6 @@ from typing import Protocol, TypedDict
from sqlalchemy.orm import Session
from core.repositories.factory import WorkflowExecutionRepository
from core.workflow.nodes.human_input.pause_reason import PauseReason as DifyPauseReason
from graphon.entities.pause_reason import PauseReason as GraphonPauseReason
from graphon.enums import WorkflowType
@ -82,7 +81,7 @@ class WorkflowRunCleanupRef:
created_at: datetime
class APIWorkflowRunRepository(WorkflowExecutionRepository, Protocol):
class APIWorkflowRunRepository(Protocol):
"""
Protocol for service-layer WorkflowRun repository operations.

401
api/repositories/app_statistic_query_repository.py Normal file
View File

@ -0,0 +1,401 @@
"""Database read model for app monitoring statistics."""
from datetime import datetime
from decimal import Decimal
from typing import override
import sqlalchemy as sa
from sqlalchemy.engine import RowMapping
from sqlalchemy.orm import Session, sessionmaker
from core.app.entities.app_invoke_entities import InvokeFrom
from libs.helper import convert_datetime_to_date
from services.app_statistic_query import (
AppStatisticQuery,
AverageResponseTimeStatisticRecord,
AverageSessionInteractionStatisticRecord,
DailyConversationStatisticRecord,
DailyMessageStatisticRecord,
DailyTerminalStatisticRecord,
DailyTokenCostStatisticRecord,
TokensPerSecondStatisticRecord,
UserSatisfactionRateStatisticRecord,
)
def _append_time_range(
sql_query: str,
parameters: dict[str, object],
*,
column: str,
start_date: datetime | None,
end_date: datetime | None,
) -> str:
if start_date is not None:
sql_query += f" AND {column} >= :start_date"
parameters["start_date"] = start_date
if end_date is not None:
sql_query += f" AND {column} < :end_date"
parameters["end_date"] = end_date
return sql_query
class AppStatisticQueryRepository(AppStatisticQuery):
def __init__(self, *, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
def _execute(self, sql_query: str, parameters: dict[str, object]) -> tuple[RowMapping, ...]:
with self._session_factory() as session:
return tuple(session.execute(sa.text(sql_query), parameters).mappings())
@staticmethod
def _parameters(
*,
app_id: str,
timezone: str,
) -> dict[str, object]:
return {
"tz": timezone,
"app_id": app_id,
"excluded_invoke_from": InvokeFrom.DEBUGGER,
}
@override
def get_daily_messages(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> tuple[DailyMessageStatisticRecord, ...]:
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
COUNT(*) AS message_count
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :excluded_invoke_from"""
parameters = self._parameters(
app_id=app_id,
timezone=timezone,
)
sql_query = _append_time_range(
sql_query,
parameters,
column="created_at",
start_date=start_date,
end_date=end_date,
)
sql_query += " GROUP BY date ORDER BY date"
return tuple(
DailyMessageStatisticRecord(date=str(row["date"]), message_count=row["message_count"])
for row in self._execute(sql_query, parameters)
)
@override
def get_daily_conversations(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> tuple[DailyConversationStatisticRecord, ...]:
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
COUNT(DISTINCT conversation_id) AS conversation_count
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :excluded_invoke_from"""
parameters = self._parameters(
app_id=app_id,
timezone=timezone,
)
sql_query = _append_time_range(
sql_query,
parameters,
column="created_at",
start_date=start_date,
end_date=end_date,
)
sql_query += " GROUP BY date ORDER BY date"
return tuple(
DailyConversationStatisticRecord(
date=str(row["date"]),
conversation_count=row["conversation_count"],
)
for row in self._execute(sql_query, parameters)
)
@override
def get_daily_terminals(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> tuple[DailyTerminalStatisticRecord, ...]:
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
COUNT(DISTINCT messages.from_end_user_id) AS terminal_count
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :excluded_invoke_from"""
parameters = self._parameters(
app_id=app_id,
timezone=timezone,
)
sql_query = _append_time_range(
sql_query,
parameters,
column="created_at",
start_date=start_date,
end_date=end_date,
)
sql_query += " GROUP BY date ORDER BY date"
return tuple(
DailyTerminalStatisticRecord(date=str(row["date"]), terminal_count=row["terminal_count"])
for row in self._execute(sql_query, parameters)
)
@override
def get_daily_token_costs(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> tuple[DailyTokenCostStatisticRecord, ...]:
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
(SUM(messages.message_tokens) + SUM(messages.answer_tokens)) AS token_count,
SUM(total_price) AS total_price
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :excluded_invoke_from"""
parameters = self._parameters(
app_id=app_id,
timezone=timezone,
)
sql_query = _append_time_range(
sql_query,
parameters,
column="created_at",
start_date=start_date,
end_date=end_date,
)
sql_query += " GROUP BY date ORDER BY date"
return tuple(
DailyTokenCostStatisticRecord(
date=str(row["date"]),
token_count=int(row["token_count"]) if row["token_count"] is not None else None,
total_price=row["total_price"],
currency="USD",
)
for row in self._execute(sql_query, parameters)
)
@override
def get_average_session_interactions(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> tuple[AverageSessionInteractionStatisticRecord, ...]:
converted_created_at = convert_datetime_to_date("c.created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
AVG(subquery.message_count) AS interactions
FROM
(
SELECT
m.conversation_id,
COUNT(m.id) AS message_count
FROM
conversations c
JOIN
messages m
ON c.id = m.conversation_id
WHERE
c.app_id = :app_id
AND m.invoke_from != :excluded_invoke_from"""
parameters = self._parameters(
app_id=app_id,
timezone=timezone,
)
sql_query = _append_time_range(
sql_query,
parameters,
column="c.created_at",
start_date=start_date,
end_date=end_date,
)
sql_query += """
GROUP BY m.conversation_id
) subquery
LEFT JOIN
conversations c
ON c.id = subquery.conversation_id
GROUP BY
date
ORDER BY
date"""
return tuple(
AverageSessionInteractionStatisticRecord(
date=str(row["date"]),
interactions=float(row["interactions"].quantize(Decimal("0.01"))),
)
for row in self._execute(sql_query, parameters)
)
@override
def get_user_satisfaction_rates(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> tuple[UserSatisfactionRateStatisticRecord, ...]:
converted_created_at = convert_datetime_to_date("m.created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
COUNT(m.id) AS message_count,
COUNT(mf.id) AS feedback_count
FROM
messages m
LEFT JOIN
message_feedbacks mf
ON mf.message_id=m.id AND mf.rating='like'
WHERE
m.app_id = :app_id
AND m.invoke_from != :excluded_invoke_from"""
parameters = self._parameters(
app_id=app_id,
timezone=timezone,
)
sql_query = _append_time_range(
sql_query,
parameters,
column="m.created_at",
start_date=start_date,
end_date=end_date,
)
sql_query += " GROUP BY date ORDER BY date"
return tuple(
UserSatisfactionRateStatisticRecord(
date=str(row["date"]),
rate=float(
round(
row["feedback_count"] * 1000 / row["message_count"] if row["message_count"] > 0 else 0,
2,
)
),
)
for row in self._execute(sql_query, parameters)
)
@override
def get_average_response_times(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> tuple[AverageResponseTimeStatisticRecord, ...]:
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
AVG(provider_response_latency) AS latency
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :excluded_invoke_from"""
parameters = self._parameters(
app_id=app_id,
timezone=timezone,
)
sql_query = _append_time_range(
sql_query,
parameters,
column="created_at",
start_date=start_date,
end_date=end_date,
)
sql_query += " GROUP BY date ORDER BY date"
return tuple(
AverageResponseTimeStatisticRecord(
date=str(row["date"]),
latency=round(row["latency"] * 1000, 4),
)
for row in self._execute(sql_query, parameters)
)
@override
def get_tokens_per_second(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> tuple[TokensPerSecondStatisticRecord, ...]:
converted_created_at = convert_datetime_to_date("created_at")
sql_query = f"""SELECT
{converted_created_at} AS date,
CASE
WHEN SUM(provider_response_latency) = 0 THEN 0
ELSE (SUM(answer_tokens) / SUM(provider_response_latency))
END as tokens_per_second
FROM
messages
WHERE
app_id = :app_id
AND invoke_from != :excluded_invoke_from"""
parameters = self._parameters(
app_id=app_id,
timezone=timezone,
)
sql_query = _append_time_range(
sql_query,
parameters,
column="created_at",
start_date=start_date,
end_date=end_date,
)
sql_query += " GROUP BY date ORDER BY date"
return tuple(
TokensPerSecondStatisticRecord(
date=str(row["date"]),
tps=round(row["tokens_per_second"], 4),
)
for row in self._execute(sql_query, parameters)
)

213
api/repositories/file_grant_repository.py Normal file
View File

@ -0,0 +1,213 @@
from __future__ import annotations
import os
from collections.abc import Sequence
from sqlalchemy import select
from sqlalchemy.orm import Session, sessionmaker
from models.enums import CreatorUserRole, EndUserType
from models.model import App, EndUser, UploadFile
from models.tools import ToolFile
from services.entities.file_grant_entities import (
FileContentRecord,
FileGrantContext,
FileKind,
FileRef,
ResolvedFile,
)
class FileGrantRepository:
def __init__(self, *, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
def get_or_create_subject(
self,
*,
tenant_id: str,
app_id: str,
session_id: str,
external_user_id: str,
is_anonymous: bool,
) -> str | None:
with self._session_factory() as session:
end_user = session.scalar(
self._subject_statement(
tenant_id=tenant_id,
app_id=app_id,
session_id=session_id,
require_app=True,
).limit(1)
)
if end_user is not None:
return end_user.id
with self._session_factory.begin() as session:
app = session.scalar(select(App).where(App.id == app_id, App.tenant_id == tenant_id).with_for_update())
if app is None:
return None
end_user = session.scalar(
self._subject_statement(
tenant_id=tenant_id,
app_id=app_id,
session_id=session_id,
require_app=False,
).limit(1)
)
if end_user is None:
end_user = EndUser(
tenant_id=tenant_id,
app_id=app_id,
type=EndUserType.APP_DEPLOY,
is_anonymous=is_anonymous,
session_id=session_id,
external_user_id=external_user_id,
)
session.add(end_user)
session.flush()
return end_user.id
def subject_exists(self, context: FileGrantContext) -> bool:
with self._session_factory() as session:
return (
session.scalar(
select(EndUser.id)
.where(
EndUser.id == context.end_user_id,
EndUser.tenant_id == context.tenant_id,
EndUser.app_id == context.app_id,
EndUser.type == EndUserType.APP_DEPLOY,
)
.limit(1)
)
is not None
)
def get_end_user(self, context: FileGrantContext) -> EndUser | None:
with self._session_factory(expire_on_commit=False) as session:
return session.scalar(
select(EndUser)
.where(
EndUser.id == context.end_user_id,
EndUser.tenant_id == context.tenant_id,
EndUser.app_id == context.app_id,
EndUser.type == EndUserType.APP_DEPLOY,
)
.limit(1)
)
def resolve_owned_files(
self,
*,
context: FileGrantContext,
refs: Sequence[FileRef],
) -> list[ResolvedFile | None]:
upload_ids = {ref.id for ref in refs if ref.kind == FileKind.UPLOAD}
tool_ids = {ref.id for ref in refs if ref.kind == FileKind.TOOL}
with self._session_factory() as session:
uploads = self._load_uploads(session, context=context, file_ids=upload_ids)
tool_files = self._load_tool_files(session, context=context, file_ids=tool_ids)
return [uploads.get(ref.id) if ref.kind == FileKind.UPLOAD else tool_files.get(ref.id) for ref in refs]
def get_content_record(self, *, file_id: str, kind: FileKind) -> FileContentRecord | None:
with self._session_factory() as session:
match kind:
case FileKind.UPLOAD:
upload_file = session.scalar(select(UploadFile).where(UploadFile.id == file_id).limit(1))
if upload_file is None:
return None
return FileContentRecord(
name=upload_file.name,
size=upload_file.size,
mime_type=upload_file.mime_type,
storage_key=upload_file.key,
)
case FileKind.TOOL:
tool_file = session.scalar(select(ToolFile).where(ToolFile.id == file_id).limit(1))
if tool_file is None:
return None
return FileContentRecord(
name=tool_file.name or "",
size=tool_file.size,
mime_type=tool_file.mimetype,
storage_key=tool_file.file_key,
)
@staticmethod
def _subject_statement(*, tenant_id: str, app_id: str, session_id: str, require_app: bool):
statement = select(EndUser)
if require_app:
statement = statement.join(App, App.id == EndUser.app_id)
predicates = [
EndUser.tenant_id == tenant_id,
EndUser.app_id == app_id,
EndUser.session_id == session_id,
EndUser.type == EndUserType.APP_DEPLOY,
]
if require_app:
predicates.append(App.tenant_id == tenant_id)
return statement.where(*predicates)
@staticmethod
def _load_uploads(
session: Session,
*,
context: FileGrantContext,
file_ids: set[str],
) -> dict[str, ResolvedFile]:
if not file_ids:
return {}
rows = session.scalars(
select(UploadFile).where(
UploadFile.id.in_(file_ids),
UploadFile.tenant_id == context.tenant_id,
UploadFile.created_by_role == CreatorUserRole.END_USER,
UploadFile.created_by == context.end_user_id,
)
).all()
return {
row.id: ResolvedFile(
id=row.id,
kind=FileKind.UPLOAD,
name=row.name,
size=row.size,
extension=row.extension,
mime_type=row.mime_type,
)
for row in rows
}
@staticmethod
def _load_tool_files(
session: Session,
*,
context: FileGrantContext,
file_ids: set[str],
) -> dict[str, ResolvedFile]:
if not file_ids:
return {}
rows = session.scalars(
select(ToolFile).where(
ToolFile.id.in_(file_ids),
ToolFile.tenant_id == context.tenant_id,
ToolFile.user_id == context.end_user_id,
)
).all()
return {
row.id: ResolvedFile(
id=row.id,
kind=FileKind.TOOL,
name=row.name or "",
size=row.size,
extension=os.path.splitext(row.name or "")[1].lstrip(".").lower(),
mime_type=row.mimetype,
)
for row in rows
}
__all__ = ["FileGrantRepository"]

89
api/repositories/oauth_access_token_repository.py Normal file
View File

@ -0,0 +1,89 @@
"""SQLAlchemy persistence for account-scoped OAuth access sessions."""
from __future__ import annotations
from datetime import datetime
from typing import override
from sqlalchemy import func, select, update
from sqlalchemy.orm import Session, sessionmaker
from models.oauth import OAuthAccessToken
from services.account_ports import AccountSessionRepository
from services.entities.account_access_entities import (
AccountSessionRevocation,
AccountSessionSnapshot,
)
class SQLAlchemyOAuthAccessTokenRepository(AccountSessionRepository):
def __init__(self, *, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
@override
def list_active(
self,
*,
account_id: str,
active_at: datetime,
offset: int,
limit: int,
) -> tuple[int, tuple[AccountSessionSnapshot, ...]]:
predicates = (
OAuthAccessToken.account_id == account_id,
OAuthAccessToken.revoked_at.is_(None),
OAuthAccessToken.token_hash.is_not(None),
OAuthAccessToken.expires_at > active_at,
)
with self._session_factory() as session:
total = session.scalar(select(func.count()).select_from(OAuthAccessToken).where(*predicates)) or 0
rows = session.scalars(
select(OAuthAccessToken)
.where(*predicates)
.order_by(OAuthAccessToken.created_at.desc(), OAuthAccessToken.id.desc())
.offset(offset)
.limit(limit)
).all()
return int(total), tuple(self._to_snapshot(row) for row in rows)
@override
def revoke(
self,
*,
account_id: str,
token_id: str,
revoked_at: datetime,
) -> AccountSessionRevocation:
with self._session_factory.begin() as session:
row = session.execute(
select(OAuthAccessToken.account_id, OAuthAccessToken.token_hash)
.where(OAuthAccessToken.id == token_id)
.with_for_update()
).one_or_none()
if row is None or row.account_id != account_id:
return AccountSessionRevocation(owned=False)
token_hash = row.token_hash
if token_hash is not None:
session.execute(
update(OAuthAccessToken)
.where(
OAuthAccessToken.id == token_id,
OAuthAccessToken.account_id == account_id,
OAuthAccessToken.revoked_at.is_(None),
)
.values(revoked_at=revoked_at, token_hash=None)
)
return AccountSessionRevocation(owned=True, token_hash=token_hash)
@staticmethod
def _to_snapshot(row: OAuthAccessToken) -> AccountSessionSnapshot:
return AccountSessionSnapshot(
id=str(row.id),
prefix=row.prefix,
client_id=row.client_id,
device_label=row.device_label,
created_at=row.created_at,
last_used_at=row.last_used_at,
expires_at=row.expires_at,
)

85
api/repositories/sqlalchemy_api_workflow_run_repository.py
View File

@ -22,10 +22,10 @@ Implementation Notes:
import json
import logging
import uuid
from collections.abc import Callable, Sequence
from collections.abc import Callable, Mapping, Sequence
from datetime import datetime
from decimal import Decimal
from typing import Any, cast, override
from typing import Any, NamedTuple, cast, override
import sqlalchemy as sa
from pydantic import ValidationError
@ -33,6 +33,7 @@ from sqlalchemy import and_, delete, func, null, or_, select, tuple_
from sqlalchemy.engine import CursorResult
from sqlalchemy.orm import Session, selectinload, sessionmaker
from core.workflow.human_input_forms import load_form_tokens_by_form_id
from core.workflow.nodes.human_input.entities import FormDefinition
from core.workflow.nodes.human_input.pause_reason import (
HumanInputRequired,
@ -55,6 +56,7 @@ from libs.datetime_utils import naive_utc_now
from libs.helper import convert_datetime_to_date
from libs.infinite_scroll_pagination import InfiniteScrollPagination
from libs.time_parser import get_time_threshold
from models import Message
from models.enums import WorkflowRunTriggeredFrom
from models.human_input import HumanInputForm, HumanInputFormRecipient
from models.workflow import WorkflowAppLog, WorkflowArchiveLog, WorkflowPause, WorkflowPauseReason, WorkflowRun
@ -76,6 +78,18 @@ logger = logging.getLogger(__name__)
_HITL_REASON_TYPES = frozenset({PauseReasonType.LEGACY_HUMAN_INPUT_REQUIRED, PauseReasonType.HITL_REQUIRED})
class WorkflowRunMessageRef(NamedTuple):
message_id: str
conversation_id: str
class WorkflowRunPauseRecord(NamedTuple):
status: WorkflowExecutionStatus
paused_at: datetime | None
reasons: tuple[DifyPauseReason, ...]
form_tokens: Mapping[str, str]
class _WorkflowRunError(Exception):
pass
@ -184,6 +198,31 @@ class DifyAPISQLAlchemyWorkflowRunRepository(APIWorkflowRunRepository):
"""
self._session_maker = session_maker
def get_message_refs(
self,
*,
app_id: str,
workflow_run_ids: Sequence[str],
) -> dict[str, WorkflowRunMessageRef]:
if not workflow_run_ids:
return {}
stmt = select(Message.workflow_run_id, Message.id, Message.conversation_id).where(
Message.app_id == app_id,
Message.workflow_run_id.in_(workflow_run_ids),
)
with self._session_maker() as session:
rows = session.execute(stmt).all()
messages_by_run_id: dict[str, WorkflowRunMessageRef] = {}
for workflow_run_id, message_id, conversation_id in rows:
if workflow_run_id is not None:
messages_by_run_id.setdefault(
workflow_run_id,
WorkflowRunMessageRef(message_id=message_id, conversation_id=conversation_id),
)
return messages_by_run_id
@override
def get_paginated_workflow_runs(
self,
@ -1152,6 +1191,48 @@ class DifyAPISQLAlchemyWorkflowRunRepository(APIWorkflowRunRepository):
pause_reasons=pause_reasons,
)
def get_pause_record(
self,
*,
workspace_id: str,
workflow_run_id: str,
) -> WorkflowRunPauseRecord | None:
stmt = (
select(WorkflowRun)
.options(selectinload(WorkflowRun.pause))
.where(
WorkflowRun.tenant_id == workspace_id,
WorkflowRun.id == workflow_run_id,
)
)
with self._session_maker() as session:
workflow_run = session.scalar(stmt)
if workflow_run is None:
return None
if workflow_run.status != WorkflowExecutionStatus.PAUSED:
return WorkflowRunPauseRecord(
status=workflow_run.status,
paused_at=None,
reasons=(),
form_tokens={},
)
pause_model = workflow_run.pause
if pause_model is None:
reasons: tuple[DifyPauseReason, ...] = ()
else:
reason_models = self._get_reasons_by_pause_id(session, pause_model.id)
reasons = tuple(self._hydrate_pause_reasons(session, reason_models))
form_ids = [reason.form_id for reason in reasons if isinstance(reason, HumanInputRequired)]
form_tokens = load_form_tokens_by_form_id(form_ids, session=session)
return WorkflowRunPauseRecord(
status=workflow_run.status,
paused_at=pause_model.created_at if pause_model is not None else None,
reasons=reasons,
form_tokens=form_tokens,
)
@override
def resume_workflow_pause(
self,

261
api/repositories/workflow_app_log_query_repository.py Normal file
View File

@ -0,0 +1,261 @@
import uuid
from datetime import datetime
from enum import Enum
from typing import override
from sqlalchemy import and_, func, literal, or_, select
from sqlalchemy.engine import RowMapping
from sqlalchemy.orm import Session, aliased, sessionmaker
from graphon.enums import WorkflowExecutionStatus
from libs.helper import escape_like_pattern
from models import Account, EndUser, TenantAccountJoin, WorkflowAppLog, WorkflowRun
from models.enums import CreatorUserRole
from models.trigger import WorkflowTriggerLog
from services.workflow_app_log_query_service import (
WorkflowAppLogAccount,
WorkflowAppLogEndUser,
WorkflowAppLogItem,
WorkflowAppLogPage,
WorkflowAppLogQuery,
WorkflowAppLogRunSummary,
)
class WorkflowAppLogQueryRepository(WorkflowAppLogQuery):
def __init__(self, *, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
@override
def get_paginated(
self,
*,
tenant_id: str,
app_id: str,
keyword: str | None = None,
status: WorkflowExecutionStatus | None = None,
created_at_before: datetime | None = None,
created_at_after: datetime | None = None,
page: int = 1,
limit: int = 20,
detail: bool = False,
created_by_end_user_session_id: str | None = None,
created_by_account: str | None = None,
) -> WorkflowAppLogPage:
with self._session_factory() as session:
workflow_run = aliased(WorkflowRun)
stmt = (
select(
WorkflowAppLog.id.label("log_id"),
WorkflowAppLog.created_from.label("log_created_from"),
WorkflowAppLog.created_by_role.label("log_created_by_role"),
WorkflowAppLog.created_at.label("log_created_at"),
workflow_run.id.label("run_id"),
workflow_run.version.label("run_version"),
workflow_run.status.label("run_status"),
workflow_run.triggered_from.label("run_triggered_from"),
workflow_run.error.label("run_error"),
workflow_run.elapsed_time.label("run_elapsed_time"),
workflow_run.total_tokens.label("run_total_tokens"),
workflow_run.total_steps.label("run_total_steps"),
workflow_run.created_at.label("run_created_at"),
workflow_run.finished_at.label("run_finished_at"),
workflow_run.exceptions_count.label("run_exceptions_count"),
)
.select_from(WorkflowAppLog)
.outerjoin(
workflow_run,
and_(
workflow_run.id == WorkflowAppLog.workflow_run_id,
workflow_run.tenant_id == WorkflowAppLog.tenant_id,
workflow_run.app_id == WorkflowAppLog.app_id,
),
)
.where(
WorkflowAppLog.tenant_id == tenant_id,
WorkflowAppLog.app_id == app_id,
)
)
if detail:
workflow_trigger_log = aliased(WorkflowTriggerLog)
stmt = stmt.outerjoin(
workflow_trigger_log,
and_(
workflow_trigger_log.tenant_id == tenant_id,
workflow_trigger_log.app_id == app_id,
workflow_trigger_log.workflow_run_id == WorkflowAppLog.workflow_run_id,
),
).add_columns(workflow_trigger_log.trigger_metadata.label("trigger_metadata"))
else:
stmt = stmt.add_columns(literal(None).label("trigger_metadata"))
if keyword:
escaped_keyword = escape_like_pattern(keyword[:30])
keyword_like_value = f"%{escaped_keyword}%"
run_creator = aliased(EndUser)
keyword_conditions = (
workflow_run.inputs.ilike(keyword_like_value, escape="\\"),
workflow_run.outputs.ilike(keyword_like_value, escape="\\"),
select(1)
.select_from(run_creator)
.where(
workflow_run.created_by_role == CreatorUserRole.END_USER,
run_creator.id == workflow_run.created_by,
run_creator.session_id.ilike(keyword_like_value, escape="\\"),
)
.exists(),
)
keyword_uuid = self._safe_parse_uuid(keyword)
if keyword_uuid is not None:
stmt = stmt.where(or_(*keyword_conditions, workflow_run.id == keyword_uuid))
else:
stmt = stmt.where(or_(*keyword_conditions))
if status is not None:
stmt = stmt.where(workflow_run.status == status)
if created_at_before is not None:
stmt = stmt.where(WorkflowAppLog.created_at <= created_at_before)
if created_at_after is not None:
stmt = stmt.where(WorkflowAppLog.created_at >= created_at_after)
if created_by_end_user_session_id:
log_creator = aliased(EndUser)
stmt = stmt.where(
WorkflowAppLog.created_by_role == CreatorUserRole.END_USER,
select(1)
.select_from(log_creator)
.where(
log_creator.id == WorkflowAppLog.created_by,
log_creator.session_id == created_by_end_user_session_id,
)
.exists(),
)
if created_by_account:
account_id = session.scalar(
select(Account.id)
.join(TenantAccountJoin, TenantAccountJoin.account_id == Account.id)
.where(
Account.email == created_by_account,
TenantAccountJoin.tenant_id == tenant_id,
)
)
if account_id is None:
raise ValueError(f"Account not found: {created_by_account}")
stmt = stmt.where(
WorkflowAppLog.created_by_role == CreatorUserRole.ACCOUNT,
WorkflowAppLog.created_by == account_id,
)
total = session.scalar(select(func.count()).select_from(stmt.subquery())) or 0
actor_account = aliased(Account)
actor_end_user = aliased(EndUser)
paginated_stmt = (
stmt.outerjoin(
actor_account,
and_(
WorkflowAppLog.created_by_role == CreatorUserRole.ACCOUNT,
actor_account.id == WorkflowAppLog.created_by,
),
)
.outerjoin(
actor_end_user,
and_(
WorkflowAppLog.created_by_role == CreatorUserRole.END_USER,
actor_end_user.id == WorkflowAppLog.created_by,
),
)
.add_columns(
actor_account.id.label("account_id"),
actor_account.name.label("account_name"),
actor_account.email.label("account_email"),
actor_end_user.id.label("end_user_id"),
actor_end_user.type.label("end_user_type"),
actor_end_user.session_id.label("end_user_session_id"),
)
.order_by(WorkflowAppLog.created_at.desc())
.offset((page - 1) * limit)
.limit(limit)
)
rows = session.execute(paginated_stmt).mappings().all()
return WorkflowAppLogPage(
page=page,
limit=limit,
total=total,
has_more=total > page * limit,
data=tuple(self._to_item(row=row, detail=detail) for row in rows),
)
@staticmethod
def _to_item(*, row: RowMapping, detail: bool) -> WorkflowAppLogItem:
run_id = row["run_id"]
workflow_run = (
WorkflowAppLogRunSummary(
id=run_id,
version=row["run_version"],
status=WorkflowAppLogQueryRepository._enum_value(row["run_status"]),
triggered_from=WorkflowAppLogQueryRepository._enum_value(row["run_triggered_from"]),
error=row["run_error"],
elapsed_time=row["run_elapsed_time"],
total_tokens=row["run_total_tokens"],
total_steps=row["run_total_steps"],
created_at=row["run_created_at"],
finished_at=row["run_finished_at"],
exceptions_count=row["run_exceptions_count"],
)
if run_id is not None
else None
)
account_id = row["account_id"]
account = (
WorkflowAppLogAccount(
id=account_id,
name=row["account_name"],
email=row["account_email"],
)
if account_id is not None
else None
)
end_user_id = row["end_user_id"]
end_user = (
WorkflowAppLogEndUser(
id=end_user_id,
type=WorkflowAppLogQueryRepository._enum_value(row["end_user_type"]),
is_anonymous=False,
session_id=row["end_user_session_id"],
)
if end_user_id is not None
else None
)
return WorkflowAppLogItem(
id=row["log_id"],
workflow_run=workflow_run,
details={"trigger_metadata": row["trigger_metadata"]} if detail else None,
created_from=WorkflowAppLogQueryRepository._enum_value(row["log_created_from"]),
created_by_role=WorkflowAppLogQueryRepository._enum_value(row["log_created_by_role"]),
created_by_account=account,
created_by_end_user=end_user,
created_at=row["log_created_at"],
)
@staticmethod
def _enum_value(value: Enum | str | None) -> str:
if value is None:
raise ValueError("Required enum value is missing")
if isinstance(value, str):
return value
return str(value.value)
@staticmethod
def _safe_parse_uuid(value: str) -> uuid.UUID | None:
if len(value) < 32:
return None
try:
return uuid.UUID(value)
except ValueError:
return None

43
api/repositories/workspace_query_repository.py
View File

@ -7,11 +7,17 @@ from sqlalchemy.orm import Session, sessionmaker
from models.account import Tenant, TenantAccountJoin, TenantStatus
from services.account_login_service import ConsoleAuthWorkspaceQuery
from services.account_ports import AccountWorkspaceMembershipQuery
from services.account_ports import AccountWorkspaceMembershipQuery, AccountWorkspaceSnapshotQuery
from services.entities.account_access_entities import AccountWorkspaceSnapshot
from services.workspace_query_service import WorkspaceQuery, WorkspaceRecord
class WorkspaceQueryRepository(WorkspaceQuery, AccountWorkspaceMembershipQuery, ConsoleAuthWorkspaceQuery):
class WorkspaceQueryRepository(
WorkspaceQuery,
AccountWorkspaceMembershipQuery,
AccountWorkspaceSnapshotQuery,
ConsoleAuthWorkspaceQuery,
):
def __init__(self, session_factory: sessionmaker[Session]) -> None:
self._session_factory = session_factory
@ -52,6 +58,35 @@ class WorkspaceQueryRepository(WorkspaceQuery, AccountWorkspaceMembershipQuery,
with self._session_factory() as session:
return tuple(session.scalars(stmt).all())
@override
def list_account_access_workspaces(self, account_id: str) -> tuple[AccountWorkspaceSnapshot, ...]:
"""List every membership for the OpenAPI account identity response.
Unlike the Console workspace picker, the identity response preserves
its existing behavior of including archived memberships.
"""
stmt = (
select(
Tenant.id,
Tenant.name,
TenantAccountJoin.role,
TenantAccountJoin.current,
)
.join(TenantAccountJoin, TenantAccountJoin.tenant_id == Tenant.id)
.where(TenantAccountJoin.account_id == account_id)
.order_by(Tenant.created_at.asc(), Tenant.id.asc())
)
with self._session_factory() as session:
return tuple(
AccountWorkspaceSnapshot(
id=workspace_id,
name=name,
role=role.value,
current=current,
)
for workspace_id, name, role, current in session.execute(stmt).all()
)
@override
def has_active_for_account(self, account_id: str) -> bool:
stmt = (
@ -65,3 +100,7 @@ class WorkspaceQueryRepository(WorkspaceQuery, AccountWorkspaceMembershipQuery,
)
with self._session_factory() as session:
return session.scalar(stmt) is not None
@override
def has_active_membership(self, account_id: str) -> bool:
return self.has_active_for_account(account_id)

77
api/services/account_access_service.py Normal file
View File

@ -0,0 +1,77 @@
"""Application service for account identity and access-session use cases."""
from __future__ import annotations
from collections.abc import Callable
from datetime import datetime
from machinery.context import AccountRequestContext
from services.account_errors import AccountNotFoundError, AccountSessionNotFoundError
from services.account_ports import (
AccountSessionRepository,
AccountSnapshotQuery,
AccountTokenCacheInvalidator,
AccountWorkspaceSnapshotQuery,
)
from services.entities.account_access_entities import AccountAccessSnapshot, AccountSessionPage
class AccountAccessService:
def __init__(
self,
*,
accounts: AccountSnapshotQuery,
workspaces: AccountWorkspaceSnapshotQuery,
sessions: AccountSessionRepository,
invalidate_token_cache: AccountTokenCacheInvalidator,
now: Callable[[], datetime],
) -> None:
self._accounts = accounts
self._workspaces = workspaces
self._sessions = sessions
self._invalidate_token_cache = invalidate_token_cache
self._now = now
def get(self, context: AccountRequestContext) -> AccountAccessSnapshot:
account = self._accounts.get(context.account_id)
if account is None:
raise AccountNotFoundError
workspaces = tuple(self._workspaces.list_account_access_workspaces(context.account_id))
default_workspace_id = next((workspace.id for workspace in workspaces if workspace.current), None)
if default_workspace_id is None and workspaces:
default_workspace_id = workspaces[0].id
return AccountAccessSnapshot(
account=account,
workspaces=workspaces,
default_workspace_id=default_workspace_id,
)
def list_sessions(self, context: AccountRequestContext, *, page: int, limit: int) -> AccountSessionPage:
total, sessions = self._sessions.list_active(
account_id=context.account_id,
active_at=self._now(),
offset=(page - 1) * limit,
limit=limit,
)
return AccountSessionPage(page=page, limit=limit, total=total, items=tuple(sessions))
def revoke_current_session(self, context: AccountRequestContext) -> None:
if context.access_token_id is None:
raise RuntimeError("OpenAPI account admission did not resolve an access token")
self._revoke(context, token_id=context.access_token_id, require_owned=False)
def revoke_session(self, context: AccountRequestContext, *, token_id: str) -> None:
self._revoke(context, token_id=token_id, require_owned=True)
def _revoke(self, context: AccountRequestContext, *, token_id: str, require_owned: bool) -> None:
revocation = self._sessions.revoke(
account_id=context.account_id,
token_id=token_id,
revoked_at=self._now(),
)
if require_owned and not revocation.owned:
raise AccountSessionNotFoundError
if revocation.token_hash is not None:
self._invalidate_token_cache(revocation.token_hash)

60
api/services/account_errors.py
View File

@ -9,6 +9,10 @@ class AccountNotFoundError(AccountApplicationError):
"""The admitted account no longer exists."""
class AccountSessionNotFoundError(AccountApplicationError):
"""The requested access session is not owned by the admitted account."""
class CurrentAccountPasswordIncorrectError(AccountApplicationError):
"""The supplied current password does not match the account credential."""
@ -161,6 +165,62 @@ class EducationDiscountPausedError(AccountApplicationError):
"""Education discount activation is temporarily paused."""
class InvalidOAuthProviderError(AccountApplicationError):
"""The requested Console OAuth provider is unavailable."""
class OAuthProviderRequestError(AccountApplicationError):
"""The remote OAuth provider could not complete the request."""
class OAuthProviderAuthorizationError(AccountApplicationError):
"""The remote OAuth provider rejected the authorization exchange."""
def __init__(self, description: str) -> None:
super().__init__(description)
self.description = description
class OAuthIdentityLockUnavailableError(AccountApplicationError):
"""The OAuth account claim could not be acquired or its lease was lost."""
class InvalidOAuthInvitationError(AccountApplicationError):
"""The OAuth callback references an invitation that can no longer be resolved."""
class OAuthInvitationAccountMismatchError(AccountApplicationError):
"""The OAuth identity does not own the account referenced by the invitation."""
def __init__(self, invite_token: str) -> None:
super().__init__(invite_token)
self.invite_token = invite_token
class OAuthAccountBannedError(AccountApplicationError):
"""The OAuth identity resolves to a banned Console account."""
class OAuthAccountNotFoundError(AccountApplicationError):
"""An account disappeared while the OAuth use case was running."""
class OAuthWorkspaceCreationNotAllowedError(AccountApplicationError):
"""Workspace policy prevents provisioning a workspace for the OAuth account."""
class OAuthSeatsLimitExceededError(AccountApplicationError):
"""Account registration would exceed the licensed seat limit."""
class OAuthRegistrationError(AccountApplicationError):
"""The OAuth account could not be registered."""
def __init__(self, description: str) -> None:
super().__init__(description)
self.description = description
class EducationRateLimitExceededError(AccountApplicationError):
"""Too many education verification or activation requests were made."""

203
api/services/account_oauth_adapters.py Normal file
View File

@ -0,0 +1,203 @@
"""Infrastructure gateways for Console account OAuth sign-in."""
import logging
from collections.abc import Generator
from contextlib import AbstractContextManager, contextmanager
from hashlib import sha256
from threading import Event, Thread
from typing import Protocol, override
import httpx
from redis import RedisError
from redis.exceptions import LockError
from extensions.ext_redis import RedisClientWrapper
from libs.oauth import OAuth
from services.account_errors import (
OAuthIdentityLockUnavailableError,
OAuthProviderAuthorizationError,
OAuthProviderRequestError,
)
from services.account_oauth_service import (
OAuthAccountClaimLease,
OAuthAccountClaimLock,
OAuthProviderGateway,
OAuthRegistrationPolicyGateway,
OAuthWorkspacePolicyGateway,
)
from services.billing_service import BillingService
from services.entities.account_oauth_entities import OAuthAuthorizationRequest, OAuthIdentity
from services.system_feature_service import SystemFeatureService
logger = logging.getLogger(__name__)
_OAUTH_ACCOUNT_CLAIM_LOCK_PREFIX = "oauth:account-claim:"
_OAUTH_ACCOUNT_CLAIM_LOCK_TIMEOUT_SECONDS = 60
_OAUTH_ACCOUNT_CLAIM_LOCK_BLOCKING_TIMEOUT_SECONDS = 10
_OAUTH_ACCOUNT_CLAIM_LOCK_RENEW_INTERVAL_SECONDS = 20
_OAUTH_ACCOUNT_CLAIM_LOCK_HEARTBEAT_JOIN_TIMEOUT_SECONDS = 2
class _RedisLock(Protocol):
def acquire(self) -> bool: ...
def reacquire(self) -> bool: ...
def release(self) -> None: ...
class _RedisOAuthAccountClaimLease(OAuthAccountClaimLease):
def __init__(self, *, locks: tuple[_RedisLock, ...], lost: Event) -> None:
self._locks = locks
self._lost = lost
@override
def ensure_owned(self) -> None:
if self._lost.is_set():
raise OAuthIdentityLockUnavailableError
try:
for lock in self._locks:
lock.reacquire()
except (LockError, RedisError) as exc:
self._lost.set()
raise OAuthIdentityLockUnavailableError from exc
except Exception as exc:
self._lost.set()
raise OAuthIdentityLockUnavailableError from exc
def mark_lost(self) -> None:
self._lost.set()
class DifyOAuthProviderGateway(OAuthProviderGateway):
def __init__(self, *, provider_name: str, client: OAuth) -> None:
self._provider_name = provider_name
self._client = client
@override
def get_authorization_url(self, request: OAuthAuthorizationRequest) -> str:
return self._client.get_authorization_url(
invite_token=request.invite_token,
timezone=request.timezone,
language=request.language,
redirect_url=request.redirect_url,
)
@override
def get_identity(self, code: str) -> OAuthIdentity:
try:
token = self._client.get_access_token(code)
user_info = self._client.get_user_info(token)
except httpx.HTTPError as exc:
error_text = exc.response.text if isinstance(exc, httpx.HTTPStatusError) else str(exc)
logger.exception(
"An error occurred during the OAuth process with %s: %s",
self._provider_name,
error_text,
)
raise OAuthProviderRequestError from exc
except ValueError as exc:
logger.warning("OAuth error with %s", self._provider_name, exc_info=True)
raise OAuthProviderAuthorizationError(str(exc)) from exc
return OAuthIdentity(id=user_info.id, name=user_info.name, email=user_info.email)
class RedisOAuthAccountClaimLock(OAuthAccountClaimLock):
def __init__(self, *, client: RedisClientWrapper) -> None:
self._client = client
@override
def acquire(self, *, provider: str, open_id: str, email: str) -> AbstractContextManager[OAuthAccountClaimLease]:
return self._acquire(
lock_names=(
self._lock_name("identity", provider, open_id),
self._lock_name("email", email),
)
)
@override
def acquire_account(self, account_id: str) -> AbstractContextManager[OAuthAccountClaimLease]:
return self._acquire(lock_names=(self._lock_name("account", account_id),))
@contextmanager
def _acquire(self, *, lock_names: tuple[str, ...]) -> Generator[OAuthAccountClaimLease, None, None]:
sorted_lock_names = sorted(set(lock_names))
locks: list[_RedisLock] = []
try:
for lock_name in sorted_lock_names:
lock = self._client.lock(
lock_name,
timeout=_OAUTH_ACCOUNT_CLAIM_LOCK_TIMEOUT_SECONDS,
blocking_timeout=_OAUTH_ACCOUNT_CLAIM_LOCK_BLOCKING_TIMEOUT_SECONDS,
thread_local=False,
)
if not lock.acquire():
raise OAuthIdentityLockUnavailableError
locks.append(lock)
except (LockError, RedisError) as exc:
self._release(locks)
raise OAuthIdentityLockUnavailableError from exc
except OAuthIdentityLockUnavailableError:
self._release(locks)
raise
stop_heartbeat = Event()
lease = _RedisOAuthAccountClaimLease(locks=tuple(locks), lost=Event())
heartbeat = Thread(
target=self._renew_while_held,
args=(lease, stop_heartbeat),
daemon=True,
name=f"OAuthAccountClaimLock({sha256(''.join(sorted_lock_names).encode()).hexdigest()[:12]})",
)
heartbeat.start()
try:
yield lease
lease.ensure_owned()
finally:
stop_heartbeat.set()
heartbeat.join(timeout=_OAUTH_ACCOUNT_CLAIM_LOCK_HEARTBEAT_JOIN_TIMEOUT_SECONDS)
if heartbeat.is_alive():
logger.warning("OAuth account claim lock heartbeat did not stop before release")
self._release(locks)
@staticmethod
def _lock_name(kind: str, *parts: str) -> str:
digest = sha256("\0".join((kind, *parts)).encode()).hexdigest()
return f"{_OAUTH_ACCOUNT_CLAIM_LOCK_PREFIX}{digest}"
@staticmethod
def _release(locks: list[_RedisLock]) -> None:
for lock in reversed(locks):
try:
lock.release()
except (LockError, RedisError):
logger.warning("Failed to release OAuth account claim lock", exc_info=True)
@staticmethod
def _renew_while_held(lease: _RedisOAuthAccountClaimLease, stop_heartbeat: Event) -> None:
while not stop_heartbeat.wait(_OAUTH_ACCOUNT_CLAIM_LOCK_RENEW_INTERVAL_SECONDS):
try:
lease.ensure_owned()
except OAuthIdentityLockUnavailableError:
lease.mark_lost()
logger.error("OAuth account claim lock ownership was lost; stop renewing", exc_info=True)
return
class DeploymentOAuthPolicyGateway(OAuthRegistrationPolicyGateway, OAuthWorkspacePolicyGateway):
def __init__(self, *, billing_enabled: bool) -> None:
self._billing_enabled = billing_enabled
@override
def is_registration_allowed(self) -> bool:
return SystemFeatureService.is_registration_allowed()
@override
def get_freeze_type(self, email: str) -> str | None:
if not self._billing_enabled:
return None
return BillingService.get_email_freeze_type(email)
@override
def is_creation_allowed(self) -> bool:
return SystemFeatureService.is_workspace_creation_allowed()

283
api/services/account_oauth_service.py Normal file
View File

@ -0,0 +1,283 @@
"""Application service for Console account OAuth sign-in."""
from collections.abc import Callable, Mapping, Sequence
from contextlib import AbstractContextManager
from datetime import datetime
from typing import Protocol
from services.account_email import normalize_email
from services.account_errors import (
AccountEmailDomainSuspendedError,
AccountEmailFrozenError,
InvalidOAuthInvitationError,
InvalidOAuthProviderError,
OAuthAccountBannedError,
OAuthAccountNotFoundError,
OAuthInvitationAccountMismatchError,
OAuthRegistrationError,
OAuthWorkspaceCreationNotAllowedError,
)
from services.account_ports import AccountIntegrationRepository, AccountRepository, AccountWorkspaceMembershipQuery
from services.entities.account_entities import AccountSessionTokens, AccountSnapshot
from services.entities.account_oauth_entities import (
OAuthAccountRegistration,
OAuthAuthorizationRequest,
OAuthCallbackCommand,
OAuthCallbackResult,
OAuthIdentity,
OAuthInvitation,
OAuthInvitationResult,
OAuthSignInResult,
)
_BANNED_ACCOUNT_STATUS = "banned"
_PENDING_ACCOUNT_STATUS = "pending"
class OAuthProviderGateway(Protocol):
def get_authorization_url(self, request: OAuthAuthorizationRequest) -> str: ...
def get_identity(self, code: str) -> OAuthIdentity: ...
class OAuthInvitationGateway(Protocol):
def resolve(self, invite_token: str) -> OAuthInvitation | None: ...
class OAuthAccountClaimLease(Protocol):
def ensure_owned(self) -> None: ...
class OAuthAccountClaimLock(Protocol):
def acquire(self, *, provider: str, open_id: str, email: str) -> AbstractContextManager[OAuthAccountClaimLease]: ...
def acquire_account(self, account_id: str) -> AbstractContextManager[OAuthAccountClaimLease]: ...
class OAuthAccountRegistrationGateway(Protocol):
def register(self, registration: OAuthAccountRegistration) -> str: ...
class OAuthWorkspaceGateway(Protocol):
def create_owner_workspace(self, account_id: str) -> None: ...
def try_join_default_workspace(self, account_id: str) -> None: ...
class OAuthSessionGateway(Protocol):
def login(self, account_id: str, *, ip_address: str) -> AccountSessionTokens: ...
class OAuthRegistrationPolicyGateway(Protocol):
def is_registration_allowed(self) -> bool: ...
def get_freeze_type(self, email: str) -> str | None: ...
class OAuthWorkspacePolicyGateway(Protocol):
def is_creation_allowed(self) -> bool: ...
class AccountOAuthService:
def __init__(
self,
*,
providers: Mapping[str, OAuthProviderGateway],
accounts: AccountRepository,
integrations: AccountIntegrationRepository,
memberships: AccountWorkspaceMembershipQuery,
invitations: OAuthInvitationGateway,
account_claims: OAuthAccountClaimLock,
registration: OAuthAccountRegistrationGateway,
workspaces: OAuthWorkspaceGateway,
sessions: OAuthSessionGateway,
registration_policy: OAuthRegistrationPolicyGateway,
workspace_policy: OAuthWorkspacePolicyGateway,
supported_languages: Sequence[str],
now: Callable[[], datetime],
) -> None:
self._providers = dict(providers)
self._accounts = accounts
self._integrations = integrations
self._memberships = memberships
self._invitations = invitations
self._account_claims = account_claims
self._registration = registration
self._workspaces = workspaces
self._sessions = sessions
self._registration_policy = registration_policy
self._workspace_policy = workspace_policy
self._supported_languages = tuple(supported_languages)
self._now = now
def start_authorization(self, provider: str, request: OAuthAuthorizationRequest) -> str:
return self._provider(provider).get_authorization_url(request)
def complete_authorization(self, command: OAuthCallbackCommand) -> OAuthCallbackResult:
provider = self._provider(command.provider)
identity = provider.get_identity(command.code)
identity_email_key = self._identity_email_key(identity.email)
with self._account_claims.acquire(
provider=command.provider,
open_id=identity.id,
email=identity_email_key,
) as identity_claim:
return self._complete_claimed_authorization(command, identity, identity_claim)
def _complete_claimed_authorization(
self,
command: OAuthCallbackCommand,
identity: OAuthIdentity,
identity_claim: OAuthAccountClaimLease,
) -> OAuthCallbackResult:
if command.invite_token is not None:
return self._complete_invitation(command, identity, identity_claim)
account = self._resolve_account(command.provider, identity)
oauth_new_user = account is None
if account is None:
identity_claim.ensure_owned()
account = self._register_account(command, identity)
identity_claim.ensure_owned()
self._ensure_account_can_login(account)
identity_claim.ensure_owned()
self._integrations.link(account.id, provider=command.provider, open_id=identity.id)
identity_claim.ensure_owned()
with self._account_claims.acquire_account(account.id) as account_claim:
if oauth_new_user:
self._provision_new_account_workspaces(account.id, account_claim)
else:
self._provision_owner_workspace_if_required(account.id, account_claim)
if account.status == _PENDING_ACCOUNT_STATUS:
account_claim.ensure_owned()
self._accounts.activate_pending(account.id, initialized_at=self._now())
account_claim.ensure_owned()
identity_claim.ensure_owned()
tokens = self._sessions.login(account.id, ip_address=command.ip_address)
return OAuthSignInResult(tokens=tokens, oauth_new_user=oauth_new_user)
def _provision_new_account_workspaces(
self,
account_id: str,
account_claim: OAuthAccountClaimLease,
) -> None:
if self._memberships.has_active_membership(account_id):
account_claim.ensure_owned()
self._workspaces.try_join_default_workspace(account_id)
account_claim.ensure_owned()
return
creation_error = OAuthWorkspaceCreationNotAllowedError()
account_claim.ensure_owned()
if self._workspace_policy.is_creation_allowed():
account_claim.ensure_owned()
try:
self._workspaces.create_owner_workspace(account_id)
except OAuthWorkspaceCreationNotAllowedError as exc:
creation_error = exc
else:
account_claim.ensure_owned()
self._workspaces.try_join_default_workspace(account_id)
account_claim.ensure_owned()
return
account_claim.ensure_owned()
self._workspaces.try_join_default_workspace(account_id)
account_claim.ensure_owned()
if self._memberships.has_active_membership(account_id):
return
raise creation_error
def _provision_owner_workspace_if_required(
self,
account_id: str,
account_claim: OAuthAccountClaimLease,
) -> None:
if self._memberships.has_active_membership(account_id):
return
account_claim.ensure_owned()
if not self._workspace_policy.is_creation_allowed():
raise OAuthWorkspaceCreationNotAllowedError
account_claim.ensure_owned()
self._workspaces.create_owner_workspace(account_id)
account_claim.ensure_owned()
def _complete_invitation(
self,
command: OAuthCallbackCommand,
identity: OAuthIdentity,
identity_claim: OAuthAccountClaimLease,
) -> OAuthCallbackResult:
invite_token = command.invite_token
if invite_token is None:
raise AssertionError("invitation completion requires a token")
invitation = self._invitations.resolve(invite_token)
if invitation is None:
raise InvalidOAuthInvitationError
if self._normalize_email(invitation.account_email) != self._normalize_email(identity.email):
raise OAuthInvitationAccountMismatchError(invite_token)
if invitation.account_status == _BANNED_ACCOUNT_STATUS:
raise OAuthAccountBannedError
identity_claim.ensure_owned()
self._integrations.link(invitation.account_id, provider=command.provider, open_id=identity.id)
identity_claim.ensure_owned()
tokens = self._sessions.login(invitation.account_id, ip_address=command.ip_address)
return OAuthInvitationResult(tokens=tokens, invite_token=invite_token)
def _register_account(self, command: OAuthCallbackCommand, identity: OAuthIdentity) -> AccountSnapshot:
normalized_email = self._normalize_email(identity.email)
if not self._registration_policy.is_registration_allowed():
freeze_type = self._registration_policy.get_freeze_type(normalized_email)
if freeze_type == "email_domain_suspended":
raise AccountEmailDomainSuspendedError
if freeze_type:
raise AccountEmailFrozenError
raise OAuthRegistrationError("Invalid email or password")
language = command.language or command.browser_language
if language not in self._supported_languages:
language = self._supported_languages[0]
account_id = self._registration.register(
OAuthAccountRegistration(
email=normalized_email,
name=identity.name or "Dify",
language=language,
timezone=command.timezone,
ip_address=command.ip_address,
)
)
account = self._accounts.get(account_id)
if account is None:
raise OAuthAccountNotFoundError
return account
def _resolve_account(self, provider: str, identity: OAuthIdentity) -> AccountSnapshot | None:
account_id = self._integrations.find_account_id(provider=provider, open_id=identity.id)
if account_id is not None:
account = self._accounts.get(account_id)
if account is not None:
return account
return self._accounts.find_by_email(identity.email)
@staticmethod
def _normalize_email(email: str) -> str:
return email.strip().lower()
@staticmethod
def _identity_email_key(email: str) -> str:
return normalize_email(email.strip())
@staticmethod
def _ensure_account_can_login(account: AccountSnapshot) -> None:
if account.status == _BANNED_ACCOUNT_STATUS:
raise OAuthAccountBannedError
def _provider(self, provider: str) -> OAuthProviderGateway:
gateway = self._providers.get(provider)
if gateway is None:
raise InvalidOAuthProviderError
return gateway

47
api/services/account_ports.py
View File

@ -1,8 +1,14 @@
"""Persistence ports used by account application services."""
"""Ports used by account application services."""
from collections.abc import Sequence
from datetime import datetime
from typing import Protocol
from services.entities.account_access_entities import (
AccountSessionRevocation,
AccountSessionSnapshot,
AccountWorkspaceSnapshot,
)
from services.entities.account_entities import (
AccountCredentials,
AccountDeletionChallenge,
@ -16,11 +22,17 @@ from services.entities.account_entities import (
)
class AccountSnapshotQuery(Protocol):
def get(self, account_id: str) -> AccountSnapshot | None: ...
class AccountRepository(Protocol):
def get(self, account_id: str) -> AccountSnapshot | None: ...
def find_by_email(self, email: str) -> AccountSnapshot | None: ...
def activate_pending(self, account_id: str, *, initialized_at: datetime) -> None: ...
def get_credentials(self, account_id: str) -> AccountCredentials | None: ...
def update_profile(self, account_id: str, changes: AccountProfileChanges) -> AccountSnapshot | None: ...
@ -42,12 +54,45 @@ class AccountRepository(Protocol):
class AccountIntegrationRepository(Protocol):
def find_account_id(self, *, provider: str, open_id: str) -> str | None: ...
def list_for_account(self, account_id: str) -> list[AccountIntegrationSnapshot]: ...
def link(self, account_id: str, *, provider: str, open_id: str) -> None: ...
class AccountWorkspaceMembershipQuery(Protocol):
def list_ids_for_account(self, account_id: str) -> Sequence[str]: ...
def has_active_membership(self, account_id: str) -> bool: ...
class AccountWorkspaceSnapshotQuery(Protocol):
def list_account_access_workspaces(self, account_id: str) -> Sequence[AccountWorkspaceSnapshot]: ...
class AccountSessionRepository(Protocol):
def list_active(
self,
*,
account_id: str,
active_at: datetime,
offset: int,
limit: int,
) -> tuple[int, Sequence[AccountSessionSnapshot]]: ...
def revoke(
self,
*,
account_id: str,
token_id: str,
revoked_at: datetime,
) -> AccountSessionRevocation: ...
class AccountTokenCacheInvalidator(Protocol):
def __call__(self, token_hash: str) -> None: ...
class AccountAvatarFileGateway(Protocol):
def get_owned_signed_url(self, *, account_id: str, upload_file_id: str) -> str | None: ...

74
api/services/account_service.py
View File

@ -72,7 +72,6 @@ from services.errors.account import (
CannotOperateSelfError,
EmailDomainSuspendedError,
InvalidActionError,
LinkAccountIntegrateError,
MemberNotInTenantError,
NoPermissionError,
RefreshTokenNotFoundError,
@ -322,9 +321,9 @@ class AccountService:
@staticmethod
def get_account_by_id(account_id: str, *, session: Session) -> Account | None:
"""Plain ``Account`` getter — no banned check, no tenant rotation,
no ``last_active_at`` write. Use this from read-only identity
endpoints (``/openapi/v1/account``) where ``load_user``'s
side-effects (current-tenant assignment, commit) are unwanted.
no ``last_active_at`` write. Use this from authentication and read
paths where ``load_user``'s current-tenant assignment and commit are
unwanted.
``session`` is injected by the caller so this service stays free
of a Flask-scoped session import.
@ -514,35 +513,6 @@ class AccountService:
return account
@staticmethod
def link_account_integrate(provider: str, open_id: str, account: Account, *, session: Session):
"""Link account integrate"""
try:
# Query whether there is an existing binding record for the same provider
account_integrate: AccountIntegrate | None = session.scalar(
select(AccountIntegrate)
.where(AccountIntegrate.account_id == account.id, AccountIntegrate.provider == provider)
.limit(1)
)
if account_integrate:
# If it exists, update the record
account_integrate.open_id = open_id
account_integrate.encrypted_token = "" # todo
account_integrate.updated_at = naive_utc_now()
else:
# If it does not exist, create a new record
account_integrate = AccountIntegrate(
account_id=account.id, provider=provider, open_id=open_id, encrypted_token=""
)
session.add(account_integrate)
session.commit()
logger.info("Account %s linked %s account %s.", account.id, provider, open_id)
except Exception as e:
logger.exception("Failed to link %s account %s to Account %s", provider, open_id, account.id)
raise LinkAccountIntegrateError("Failed to link account.") from e
@staticmethod
def update_account_email(account: Account, email: str, session: Session) -> Account:
"""Update account email"""
@ -1261,36 +1231,12 @@ class TenantService:
).all()
)
@staticmethod
def get_account_memberships(account_id: str, *, session: Session) -> list[Row[tuple[TenantAccountJoin, Tenant]]]:
"""Return ``(TenantAccountJoin, Tenant)`` rows for every workspace
the account belongs to. Unlike :meth:`get_join_tenants` this keeps
the join row so callers can read ``role``/``current`` alongside the
tenant — used by ``/openapi/v1/account`` to render workspace
membership + pick the default workspace.
``session`` is injected by the caller so this service stays free
of a Flask-scoped session import.
No tenant-status filter: parity with the legacy controller query
(the openapi identity endpoint listed all joined tenants).
"""
return (
session.query(TenantAccountJoin, Tenant)
.join(Tenant, Tenant.id == TenantAccountJoin.tenant_id)
.filter(TenantAccountJoin.account_id == account_id)
.all()
)
@staticmethod
def get_workspaces_for_account(account_id: str, *, session: Session) -> list[Row[tuple[Tenant, TenantAccountJoin]]]:
"""``(Tenant, TenantAccountJoin)`` rows for every workspace the
account belongs to, ordered by ``Tenant.created_at`` ASC — the
canonical ordering for ``/openapi/v1/workspaces``.
Distinct from :meth:`get_account_memberships`: tuple order is
flipped (tenant first) and rows are sorted, so the workspace
listing is stable across requests.
canonical ordering for ``/openapi/v1/workspaces``. Rows keep the
tenant first so callers can serialize the workspace directly.
"""
return list(
session.execute(
@ -1864,8 +1810,6 @@ class RegisterService:
email: str,
name: str,
password: str | None = None,
open_id: str | None = None,
provider: str | None = None,
language: str | None = None,
status: AccountStatus | None = None,
is_setup: bool | None = False,
@ -1894,9 +1838,6 @@ class RegisterService:
account.status = status or AccountStatus.ACTIVE
account.initialized_at = naive_utc_now()
if open_id is not None and provider is not None:
AccountService.link_account_integrate(provider, open_id, account, session=session)
if (
SystemFeatureService.is_workspace_creation_allowed()
and create_workspace_required
@ -2049,11 +1990,6 @@ class RegisterService:
redis_client.setex(cls._get_invitation_token_key(token), expiry_hours * 60 * 60, json.dumps(invitation_data))
return token
@classmethod
def is_valid_invite_token(cls, token: str) -> bool:
data = redis_client.get(cls._get_invitation_token_key(token))
return data is not None
@classmethod
def revoke_token(cls, workspace_id: str | None, email: str | None, token: str):
if workspace_id and email:

122
api/services/app_statistic_query.py Normal file
View File

@ -0,0 +1,122 @@
"""Application boundary for app monitoring statistics."""
from collections.abc import Sequence
from datetime import datetime
from decimal import Decimal
from typing import NamedTuple, Protocol
class DailyMessageStatisticRecord(NamedTuple):
date: str
message_count: int
class DailyConversationStatisticRecord(NamedTuple):
date: str
conversation_count: int
class DailyTerminalStatisticRecord(NamedTuple):
date: str
terminal_count: int
class DailyTokenCostStatisticRecord(NamedTuple):
date: str
token_count: int | None
total_price: Decimal | None
currency: str | None
class AverageSessionInteractionStatisticRecord(NamedTuple):
date: str
interactions: float
class UserSatisfactionRateStatisticRecord(NamedTuple):
date: str
rate: float
class AverageResponseTimeStatisticRecord(NamedTuple):
date: str
latency: float
class TokensPerSecondStatisticRecord(NamedTuple):
date: str
tps: float
class AppStatisticQuery(Protocol):
def get_daily_messages(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> Sequence[DailyMessageStatisticRecord]: ...
def get_daily_conversations(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> Sequence[DailyConversationStatisticRecord]: ...
def get_daily_terminals(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> Sequence[DailyTerminalStatisticRecord]: ...
def get_daily_token_costs(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> Sequence[DailyTokenCostStatisticRecord]: ...
def get_average_session_interactions(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> Sequence[AverageSessionInteractionStatisticRecord]: ...
def get_user_satisfaction_rates(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> Sequence[UserSatisfactionRateStatisticRecord]: ...
def get_average_response_times(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> Sequence[AverageResponseTimeStatisticRecord]: ...
def get_tokens_per_second(
self,
*,
app_id: str,
start_date: datetime | None,
end_date: datetime | None,
timezone: str,
) -> Sequence[TokensPerSecondStatisticRecord]: ...

3
api/services/dataset_service.py
View File

@ -3854,7 +3854,8 @@ class SegmentService:
logger.exception("Failed to regenerate summary for segment %s", segment.id)
# Don't fail the entire update if summary regeneration fails
# update multimodel vector index
VectorService.update_multimodel_vector(segment, args.attachment_ids or [], dataset, session=session)
if args.attachment_ids is not None:
VectorService.update_multimodel_vector(segment, args.attachment_ids, dataset, session=session)
except Exception as e:
logger.exception("update segment index failed")
segment.enabled = False

5
api/services/end_user_service.py
View File

@ -63,6 +63,11 @@ class EndUserService:
EndUser.tenant_id == tenant_id,
EndUser.app_id == app_id,
EndUser.session_id == user_id,
# An AppDeploy row is never a legacy row this could upgrade:
# the type was added after the split, and FileGrantService
# reads its rows by type. Retyping one here would hide it
# from that read and strand the files it owns.
EndUser.type != EndUserType.APP_DEPLOY,
)
.order_by(
# Prioritize records with matching type (0 = match, 1 = no match)

52
api/services/entities/account_access_entities.py Normal file
View File

@ -0,0 +1,52 @@
"""Framework-neutral data contracts for account identity and access sessions."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
from services.entities.account_entities import AccountSnapshot
@dataclass(frozen=True, slots=True)
class AccountWorkspaceSnapshot:
id: str
name: str
role: str
current: bool
@dataclass(frozen=True, slots=True)
class AccountAccessSnapshot:
account: AccountSnapshot
workspaces: tuple[AccountWorkspaceSnapshot, ...]
default_workspace_id: str | None
@dataclass(frozen=True, slots=True)
class AccountSessionSnapshot:
id: str
prefix: str
client_id: str
device_label: str
created_at: datetime | None
last_used_at: datetime | None
expires_at: datetime | None
@dataclass(frozen=True, slots=True)
class AccountSessionPage:
page: int
limit: int
total: int
items: tuple[AccountSessionSnapshot, ...]
@property
def has_more(self) -> bool:
return self.page * self.limit < self.total
@dataclass(frozen=True, slots=True)
class AccountSessionRevocation:
owned: bool
token_hash: str | None = None

2
api/services/entities/account_entities.py
View File

@ -1,4 +1,4 @@
"""Framework-neutral contracts for Console account use cases."""
"""Framework-neutral contracts shared by account use cases."""
from __future__ import annotations

62
api/services/entities/account_oauth_entities.py Normal file
View File

@ -0,0 +1,62 @@
"""Framework-neutral contracts for Console account OAuth sign-in."""
from dataclasses import dataclass
from services.entities.account_entities import AccountSessionTokens as _AccountSessionTokens
@dataclass(frozen=True, slots=True)
class OAuthAuthorizationRequest:
invite_token: str | None = None
timezone: str | None = None
language: str | None = None
redirect_url: str | None = None
@dataclass(frozen=True, slots=True)
class OAuthIdentity:
id: str
name: str
email: str
@dataclass(frozen=True, slots=True)
class OAuthCallbackCommand:
provider: str
code: str
invite_token: str | None
timezone: str | None
language: str | None
browser_language: str | None
ip_address: str
@dataclass(frozen=True, slots=True)
class OAuthInvitation:
account_id: str
account_email: str
account_status: str
@dataclass(frozen=True, slots=True)
class OAuthAccountRegistration:
email: str
name: str
language: str
timezone: str | None
ip_address: str
@dataclass(frozen=True, slots=True)
class OAuthSignInResult:
tokens: _AccountSessionTokens
oauth_new_user: bool
@dataclass(frozen=True, slots=True)
class OAuthInvitationResult:
tokens: _AccountSessionTokens
invite_token: str
type OAuthCallbackResult = OAuthSignInResult | OAuthInvitationResult

139
api/services/entities/file_grant_entities.py Normal file
View File

@ -0,0 +1,139 @@
from __future__ import annotations
from collections.abc import Iterable
from dataclasses import dataclass
from datetime import datetime
from enum import StrEnum
from pydantic import BaseModel
class FileKind(StrEnum):
UPLOAD = "upload"
TOOL = "tool"
class FileGrantScope(StrEnum):
UPLOAD = "upload"
RESOLVE = "resolve"
PRODUCE = "produce"
class FileGrantClaims(BaseModel):
sub: str
tenant_id: str
app_id: str
scopes: list[FileGrantScope]
exp: int
class FileContentClaims(BaseModel):
kind: FileKind
file_id: str
exp: int
@dataclass(frozen=True, slots=True)
class FileGrantContext:
tenant_id: str
app_id: str
end_user_id: str
@dataclass(frozen=True, slots=True)
class FileRef:
id: str
kind: FileKind
@dataclass(frozen=True, slots=True)
class ResolvedFile:
id: str
kind: FileKind
name: str
size: int
extension: str
mime_type: str | None
@dataclass(frozen=True, slots=True)
class ResolvedFileAccess:
file: ResolvedFile
external_url: str
internal_url: str
@dataclass(frozen=True, slots=True)
class FileContentRecord:
name: str
size: int
mime_type: str | None
storage_key: str
@dataclass(frozen=True, slots=True)
class FileContent:
name: str
size: int
mime_type: str | None
stream: Iterable[bytes]
@dataclass(frozen=True, slots=True)
class StoredUpload:
id: str
name: str
size: int
extension: str | None
mime_type: str | None
created_by: str | None
created_at: datetime | None
tenant_id: str | None
source_url: str
@dataclass(frozen=True, slots=True)
class StoredProducedFile:
id: str
name: str
size: int
mime_type: str | None
@dataclass(frozen=True, slots=True)
class RemoteFile:
filename: str
mimetype: str
content: bytes
@dataclass(frozen=True, slots=True)
class FileGrantLimits:
file_size_limit: int
image_file_size_limit: int
audio_file_size_limit: int
video_file_size_limit: int
workflow_file_upload_limit: int
batch_count_limit: int
@dataclass(frozen=True, slots=True)
class FileGrantMintRequest:
tenant_id: str
app_id: str
subject: str
is_anonymous: bool
scopes: tuple[FileGrantScope, ...]
ttl_seconds: int
file_refs: tuple[FileRef, ...]
optional_file_refs: tuple[FileRef, ...]
run_deadline: int | None
@dataclass(frozen=True, slots=True)
class FileGrantMintResult:
grant: str
expires_at: int
limits: FileGrantLimits
files: tuple[ResolvedFile, ...]
optional_files: tuple[ResolvedFileAccess | None, ...]

4
api/services/errors/account.py
View File

@ -38,10 +38,6 @@ class AccountNotLinkTenantError(BaseServiceError):
pass
class LinkAccountIntegrateError(BaseServiceError):
pass
class TenantNotFoundError(BaseServiceError):
pass

34
api/services/errors/file_grant.py Normal file
View File

@ -0,0 +1,34 @@
class AppNotFoundError(Exception):
pass
class EndUserNotFoundError(Exception):
pass
class InvalidGrantRequestError(Exception):
pass
class InvalidFileGrantError(Exception):
pass
class GrantTtlTooLongError(Exception):
pass
class GrantedFileNotFoundError(Exception):
pass
class InvalidSubjectError(Exception):
pass
class RemoteFileUnavailableError(Exception):
pass
class TooManyFileRefsError(Exception):
pass

4
api/services/errors/workspace.py
View File

@ -5,9 +5,5 @@ class WorkSpaceNotAllowedCreateError(BaseServiceError):
pass
class WorkSpaceNotFoundError(BaseServiceError):
pass
class WorkspacesLimitExceededError(BaseServiceError):
pass

3
api/services/explore_banner_query_service.py
View File

@ -7,6 +7,8 @@ from collections.abc import Sequence
from datetime import datetime
from typing import Any, NamedTuple, Protocol
from constants.languages import languages
_DEFAULT_LANGUAGE = "en-US"
@ -37,6 +39,7 @@ class ExploreBannerQueryService:
if not self._enabled:
return ()
language = language if language in languages else _DEFAULT_LANGUAGE
banners = tuple(self._banners.list_enabled(language))
if banners or language == _DEFAULT_LANGUAGE:
return banners

327
api/services/file_grant_gateways.py Normal file
View File

@ -0,0 +1,327 @@
from __future__ import annotations
import mimetypes
import os
import re
import urllib.parse
from collections.abc import Callable, Sequence
from typing import IO, cast
from uuid import uuid4
import httpx
import jwt
from pydantic import ValidationError
from core.file import remote_fetcher
from core.helper import ssrf_proxy
from core.tools.tool_file_manager import ToolFileManager, resolve_extension
from extensions.ext_storage import Storage
from models.model import EndUser
from services.entities.file_grant_entities import (
FileContent,
FileContentClaims,
FileContentRecord,
FileGrantClaims,
FileGrantContext,
FileGrantScope,
FileKind,
RemoteFile,
StoredProducedFile,
StoredUpload,
)
from services.errors.file import FileTooLargeError
from services.errors.file_grant import EndUserNotFoundError
from services.file_service import FileService
FILE_GRANT_AUDIENCE = "dify-files"
FILE_CONTENT_AUDIENCE = "dify-files-content"
_ALGORITHM = "HS256"
class FileGrantTokenGateway:
def __init__(
self,
*,
secret_key: str,
external_files_url: str,
internal_files_url: str,
content_token_ttl_seconds: int,
now: Callable[[], int],
) -> None:
self._secret_key = secret_key
self._external_files_url = external_files_url
self._internal_files_url = internal_files_url
self._content_token_ttl_seconds = content_token_ttl_seconds
self._now = now
def issue_grant(
self,
*,
context: FileGrantContext,
scopes: Sequence[FileGrantScope],
ttl_seconds: int,
) -> tuple[str, int]:
expires_at = self._now() + ttl_seconds
token = jwt.encode(
{
"aud": FILE_GRANT_AUDIENCE,
"sub": context.end_user_id,
"tenant_id": context.tenant_id,
"app_id": context.app_id,
"scopes": [str(scope) for scope in scopes],
"exp": expires_at,
},
self._secret_key,
algorithm=_ALGORITHM,
)
return token, expires_at
def decode_grant(self, token: str) -> FileGrantClaims | None:
payload = self._decode(
token, audience=FILE_GRANT_AUDIENCE, required=["exp", "sub", "tenant_id", "app_id", "scopes"]
)
if payload is None:
return None
try:
return FileGrantClaims.model_validate(payload)
except ValidationError:
return None
def issue_content_urls(self, *, file_id: str, kind: FileKind) -> tuple[str, str]:
external_token = self._issue_content_token(file_id=file_id, kind=kind)
internal_token = self._issue_content_token(file_id=file_id, kind=kind)
path = f"/files/appdeploy/{file_id}/content"
return (
f"{self._external_files_url}{path}?token={external_token}",
f"{self._internal_files_url}{path}?token={internal_token}",
)
def decode_content_token(self, token: str) -> FileContentClaims | None:
payload = self._decode(
token,
audience=FILE_CONTENT_AUDIENCE,
required=["exp", "kind", "file_id"],
)
if payload is None:
return None
try:
return FileContentClaims.model_validate(payload)
except ValidationError:
return None
def _issue_content_token(self, *, file_id: str, kind: FileKind) -> str:
return jwt.encode(
{
"aud": FILE_CONTENT_AUDIENCE,
"kind": str(kind),
"file_id": file_id,
"nonce": os.urandom(8).hex(),
"exp": self._now() + self._content_token_ttl_seconds,
},
self._secret_key,
algorithm=_ALGORITHM,
)
def _decode(self, token: str, *, audience: str, required: list[str]) -> dict[str, object] | None:
try:
return cast(
dict[str, object],
jwt.decode(
token,
self._secret_key,
algorithms=[_ALGORITHM],
audience=audience,
options={"require": ["aud", *required]},
),
)
except jwt.PyJWTError:
return None
class FileGrantFileGateway:
def __init__(
self,
*,
load_end_user: Callable[[FileGrantContext], EndUser | None],
subject_exists: Callable[[FileGrantContext], bool],
file_service: FileService,
tool_files: ToolFileManager,
storage: Storage,
) -> None:
self._load_end_user = load_end_user
self._subject_exists = subject_exists
self._file_service = file_service
self._tool_files = tool_files
self._storage = storage
def store_upload_stream(
self,
*,
context: FileGrantContext,
filename: str,
stream: IO[bytes],
mimetype: str,
) -> StoredUpload:
if not self._subject_exists(context):
raise EndUserNotFoundError(context.end_user_id)
extension = os.path.splitext(filename)[1].lstrip(".").lower()
limit = FileService.file_size_limit(extension=extension)
content = stream.read(limit + 1)
if len(content) > limit:
raise FileTooLargeError(f"File size exceeded. The limit is {limit} bytes.")
return self.store_upload(
context=context,
filename=filename,
content=content,
mimetype=mimetype,
)
def store_upload(
self,
*,
context: FileGrantContext,
filename: str,
content: bytes,
mimetype: str,
source_url: str = "",
) -> StoredUpload:
end_user = self._load_end_user(context)
if end_user is None:
raise EndUserNotFoundError(context.end_user_id)
upload = self._file_service.upload_file(
filename=filename,
content=content,
mimetype=mimetype,
user=end_user,
source_url=source_url,
)
return StoredUpload(
id=upload.id,
name=upload.name,
size=upload.size,
extension=upload.extension,
mime_type=upload.mime_type,
created_by=upload.created_by,
created_at=upload.created_at,
tenant_id=upload.tenant_id,
source_url=upload.source_url,
)
def store_produced(
self,
*,
context: FileGrantContext,
filename: str | None,
stream: IO[bytes],
mimetype: str,
) -> StoredProducedFile:
extension = resolve_extension(filename=filename, mimetype=mimetype).lstrip(".").lower()
limit = FileService.file_size_limit(extension=extension)
content = stream.read(limit + 1)
if len(content) > limit:
raise FileTooLargeError(f"File size exceeded. The limit is {limit} bytes.")
stored = self._tool_files.create_file_by_raw(
user_id=context.end_user_id,
tenant_id=context.tenant_id,
conversation_id=None,
file_binary=content,
mimetype=mimetype,
filename=filename,
)
return StoredProducedFile(
id=stored.id,
name=stored.name or "",
size=stored.size,
mime_type=stored.mimetype,
)
def open_content(self, record: FileContentRecord) -> FileContent:
return FileContent(
name=record.name,
size=record.size,
mime_type=record.mime_type,
stream=self._storage.load(record.storage_key, stream=True),
)
class FileGrantRemoteFileGateway:
def fetch(self, url: str) -> RemoteFile | None:
try:
metadata = remote_fetcher.make_request("HEAD", url=url, follow_redirects=True)
if metadata.status_code != httpx.codes.OK:
metadata.close()
metadata = remote_fetcher.make_request(
"GET",
url=url,
timeout=3,
follow_redirects=True,
stream_response=True,
)
if metadata.status_code != httpx.codes.OK:
metadata.close()
return None
filename, extension, mimetype = self._file_info(metadata)
limit = FileService.file_size_limit(extension=extension)
declared_size = self._declared_size(metadata)
if declared_size is not None and declared_size > limit:
metadata.close()
raise FileTooLargeError(f"File size exceeded. The limit is {limit} bytes.")
if metadata.request.method == "HEAD":
metadata.close()
response = remote_fetcher.make_request(
"GET",
url=url,
timeout=3,
follow_redirects=True,
stream_response=True,
)
if response.status_code != httpx.codes.OK:
response.close()
return None
else:
response = metadata
try:
buffered = ssrf_proxy.buffer_response(response, max_response_bytes=limit)
except ssrf_proxy.ResponseTooLargeError as exc:
raise FileTooLargeError(f"File size exceeded. The limit is {limit} bytes.") from exc
return RemoteFile(filename=filename, mimetype=mimetype, content=buffered.content)
except (httpx.RequestError, ssrf_proxy.UnsupportedResponseEncodingError):
return None
@staticmethod
def _file_info(response: httpx.Response) -> tuple[str, str, str]:
parsed_url = urllib.parse.urlparse(str(response.url))
filename = urllib.parse.unquote(os.path.basename(parsed_url.path))
if not filename:
content_disposition = response.headers.get("Content-Disposition", "")
filename_match = re.search(r'filename="?([^";]+)', content_disposition)
filename = filename_match.group(1) if filename_match else uuid4().hex
extension = os.path.splitext(filename)[1].lstrip(".").lower()
mimetype = (
mimetypes.guess_type(filename)[0]
or response.headers.get("Content-Type", "").split(";", 1)[0].strip()
or "application/octet-stream"
)
return filename, extension, mimetype
@staticmethod
def _declared_size(response: httpx.Response) -> int | None:
value = response.headers.get("Content-Length")
if value is None:
return None
try:
return int(value)
except ValueError:
return None
__all__ = [
"FILE_CONTENT_AUDIENCE",
"FILE_GRANT_AUDIENCE",
"FileGrantFileGateway",
"FileGrantRemoteFileGateway",
"FileGrantTokenGateway",
]

314
api/services/file_grant_service.py Normal file
View File

@ -0,0 +1,314 @@
from __future__ import annotations
import base64
import hashlib
from collections.abc import Callable, Sequence
from typing import IO, Protocol
from services.entities.file_grant_entities import (
FileContent,
FileContentClaims,
FileContentRecord,
FileGrantClaims,
FileGrantContext,
FileGrantLimits,
FileGrantMintRequest,
FileGrantMintResult,
FileGrantScope,
FileKind,
FileRef,
RemoteFile,
ResolvedFile,
ResolvedFileAccess,
StoredProducedFile,
StoredUpload,
)
from services.errors.file_grant import (
AppNotFoundError,
EndUserNotFoundError,
GrantedFileNotFoundError,
GrantTtlTooLongError,
InvalidFileGrantError,
InvalidGrantRequestError,
InvalidSubjectError,
RemoteFileUnavailableError,
TooManyFileRefsError,
)
MAX_SESSION_GRANT_TTL_SECONDS = 7200
MAX_WORKFLOW_EXECUTION_SECONDS = 24 * 60 * 60
RUN_GRANT_EXPIRY_GRACE_SECONDS = 5 * 60
MAX_RUN_GRANT_TTL_SECONDS = MAX_WORKFLOW_EXECUTION_SECONDS + RUN_GRANT_EXPIRY_GRACE_SECONDS
MAX_FILE_GRANT_REFS = 100
class FileGrantRepository(Protocol):
def get_or_create_subject(
self,
*,
tenant_id: str,
app_id: str,
session_id: str,
external_user_id: str,
is_anonymous: bool,
) -> str | None: ...
def subject_exists(self, context: FileGrantContext) -> bool: ...
def resolve_owned_files(
self,
*,
context: FileGrantContext,
refs: Sequence[FileRef],
) -> list[ResolvedFile | None]: ...
def get_content_record(self, *, file_id: str, kind: FileKind) -> FileContentRecord | None: ...
class FileGrantFiles(Protocol):
def store_upload_stream(
self,
*,
context: FileGrantContext,
filename: str,
stream: IO[bytes],
mimetype: str,
) -> StoredUpload: ...
def store_upload(
self,
*,
context: FileGrantContext,
filename: str,
content: bytes,
mimetype: str,
source_url: str = "",
) -> StoredUpload: ...
def store_produced(
self,
*,
context: FileGrantContext,
filename: str | None,
stream: IO[bytes],
mimetype: str,
) -> StoredProducedFile: ...
def open_content(self, record: FileContentRecord) -> FileContent: ...
class FileGrantTokens(Protocol):
def issue_grant(
self,
*,
context: FileGrantContext,
scopes: Sequence[FileGrantScope],
ttl_seconds: int,
) -> tuple[str, int]: ...
def decode_grant(self, token: str) -> FileGrantClaims | None: ...
def issue_content_urls(self, *, file_id: str, kind: FileKind) -> tuple[str, str]: ...
def decode_content_token(self, token: str) -> FileContentClaims | None: ...
class FileGrantRemoteFiles(Protocol):
def fetch(self, url: str) -> RemoteFile | None: ...
class FileGrantService:
def __init__(
self,
*,
repository: FileGrantRepository,
files: FileGrantFiles,
tokens: FileGrantTokens,
remote_files: FileGrantRemoteFiles,
limits: FileGrantLimits,
now: Callable[[], int],
) -> None:
self._repository = repository
self._files = files
self._tokens = tokens
self._remote_files = remote_files
self._limits = limits
self._now = now
@staticmethod
def session_id_for_subject(subject: str) -> str:
digest = hashlib.sha256(subject.encode()).digest()
return base64.urlsafe_b64encode(digest).decode().rstrip("=")
def mint(self, request: FileGrantMintRequest) -> FileGrantMintResult:
self._validate_subject(request.subject)
self._validate_ref_count((*request.file_refs, *request.optional_file_refs))
ttl_seconds = self._effective_ttl_seconds(request)
end_user_id = self._repository.get_or_create_subject(
tenant_id=request.tenant_id,
app_id=request.app_id,
session_id=self.session_id_for_subject(request.subject),
external_user_id=request.subject[:255],
is_anonymous=request.is_anonymous,
)
if end_user_id is None:
raise AppNotFoundError(request.app_id)
context = FileGrantContext(request.tenant_id, request.app_id, end_user_id)
strict_file_count = len(request.file_refs)
resolved_files = self._repository.resolve_owned_files(
context=context,
refs=(*request.file_refs, *request.optional_file_refs),
)
strict_files = resolved_files[:strict_file_count]
if any(file is None for file in strict_files):
raise GrantedFileNotFoundError()
optional_files = resolved_files[strict_file_count:]
grant, expires_at = self._tokens.issue_grant(
context=context,
scopes=request.scopes,
ttl_seconds=ttl_seconds,
)
return FileGrantMintResult(
grant=grant,
expires_at=expires_at,
limits=self._limits,
files=tuple(file for file in strict_files if file is not None),
optional_files=tuple(self._with_access(file) if file is not None else None for file in optional_files),
)
def decode_grant(self, token: str) -> FileGrantClaims | None:
return self._tokens.decode_grant(token)
def store_upload(
self,
*,
context: FileGrantContext,
filename: str,
stream: IO[bytes],
mimetype: str,
) -> StoredUpload:
return self._files.store_upload_stream(
context=context,
filename=filename,
stream=stream,
mimetype=mimetype,
)
def store_remote_upload(self, *, context: FileGrantContext, url: str) -> StoredUpload:
if not self._repository.subject_exists(context):
raise EndUserNotFoundError(context.end_user_id)
remote_file = self._remote_files.fetch(url)
if remote_file is None:
raise RemoteFileUnavailableError(url)
return self._files.store_upload(
context=context,
filename=remote_file.filename,
content=remote_file.content,
mimetype=remote_file.mimetype,
source_url=url,
)
def store_produced(
self,
*,
context: FileGrantContext,
filename: str | None,
stream: IO[bytes],
mimetype: str,
) -> tuple[StoredProducedFile, ResolvedFileAccess]:
if not self._repository.subject_exists(context):
raise EndUserNotFoundError(context.end_user_id)
stored = self._files.store_produced(
context=context,
filename=filename,
stream=stream,
mimetype=mimetype,
)
file = ResolvedFile(stored.id, FileKind.TOOL, stored.name, stored.size, "", stored.mime_type)
return stored, self._with_access(file)
def resolve_files(
self,
*,
context: FileGrantContext,
refs: Sequence[FileRef],
) -> list[ResolvedFile | None]:
self._validate_ref_count(refs)
if not self._repository.subject_exists(context):
raise EndUserNotFoundError(context.end_user_id)
return self._repository.resolve_owned_files(context=context, refs=refs)
def resolve_file_access(
self,
*,
context: FileGrantContext,
refs: Sequence[FileRef],
) -> list[ResolvedFileAccess | None]:
return [
self._with_access(file) if file is not None else None
for file in self.resolve_files(context=context, refs=refs)
]
def load_content(self, *, token: str, requested_file_id: str) -> FileContent | None:
claims = self._tokens.decode_content_token(token)
if claims is None:
raise InvalidFileGrantError()
if claims.file_id != requested_file_id:
return None
record = self._repository.get_content_record(file_id=requested_file_id, kind=claims.kind)
return self._files.open_content(record) if record is not None else None
def content_urls(self, *, file_id: str, kind: FileKind) -> tuple[str, str]:
return self._tokens.issue_content_urls(file_id=file_id, kind=kind)
def _with_access(self, file: ResolvedFile) -> ResolvedFileAccess:
external_url, internal_url = self._tokens.issue_content_urls(file_id=file.id, kind=file.kind)
return ResolvedFileAccess(file=file, external_url=external_url, internal_url=internal_url)
def _effective_ttl_seconds(self, request: FileGrantMintRequest) -> int:
if request.run_deadline is None:
if request.ttl_seconds > MAX_SESSION_GRANT_TTL_SECONDS:
raise GrantTtlTooLongError()
return request.ttl_seconds
now = self._now()
if FileGrantScope.PRODUCE not in request.scopes:
raise InvalidGrantRequestError("A run deadline requires the produce scope.")
if request.run_deadline <= now:
raise InvalidGrantRequestError("The run deadline has expired.")
if request.run_deadline > now + MAX_WORKFLOW_EXECUTION_SECONDS:
raise InvalidGrantRequestError("The run deadline exceeds the workflow execution limit.")
if request.ttl_seconds > MAX_RUN_GRANT_TTL_SECONDS:
raise GrantTtlTooLongError()
return min(request.ttl_seconds, request.run_deadline - now + RUN_GRANT_EXPIRY_GRACE_SECONDS)
@staticmethod
def _validate_subject(subject: str) -> None:
if not subject.strip() or "\x00" in subject:
raise InvalidSubjectError()
@staticmethod
def _validate_ref_count(refs: Sequence[FileRef]) -> None:
if len(refs) > MAX_FILE_GRANT_REFS:
raise TooManyFileRefsError(f"A file grant request may contain at most {MAX_FILE_GRANT_REFS} references.")
__all__ = [
"MAX_FILE_GRANT_REFS",
"MAX_RUN_GRANT_TTL_SECONDS",
"MAX_SESSION_GRANT_TTL_SECONDS",
"MAX_WORKFLOW_EXECUTION_SECONDS",
"RUN_GRANT_EXPIRY_GRACE_SECONDS",
"AppNotFoundError",
"EndUserNotFoundError",
"FileGrantService",
"GrantTtlTooLongError",
"GrantedFileNotFoundError",
"InvalidFileGrantError",
"InvalidGrantRequestError",
"InvalidSubjectError",
"RemoteFileUnavailableError",
"TooManyFileRefsError",
]

25
api/services/file_service.py
View File

@ -131,21 +131,32 @@ class FileService:
file_size: int,
default_file_size_limit: int | None = None,
) -> bool:
return file_size <= FileService.file_size_limit(
extension=extension,
default_file_size_limit=default_file_size_limit,
)
@staticmethod
def file_size_limit(
*,
extension: str,
default_file_size_limit: int | None = None,
) -> int:
"""Return the size an extension is allowed, in bytes."""
if extension in IMAGE_EXTENSIONS:
file_size_limit = dify_config.UPLOAD_IMAGE_FILE_SIZE_LIMIT * 1024 * 1024
file_size_limit = dify_config.UPLOAD_IMAGE_FILE_SIZE_LIMIT
elif extension in VIDEO_EXTENSIONS:
file_size_limit = dify_config.UPLOAD_VIDEO_FILE_SIZE_LIMIT * 1024 * 1024
file_size_limit = dify_config.UPLOAD_VIDEO_FILE_SIZE_LIMIT
elif extension in AUDIO_EXTENSIONS:
file_size_limit = dify_config.UPLOAD_AUDIO_FILE_SIZE_LIMIT * 1024 * 1024
file_size_limit = dify_config.UPLOAD_AUDIO_FILE_SIZE_LIMIT
else:
# Context-specific uploads may override the default limit without changing media-specific limits.
file_size_limit = (
(default_file_size_limit if default_file_size_limit is not None else dify_config.UPLOAD_FILE_SIZE_LIMIT)
* 1024
* 1024
default_file_size_limit if default_file_size_limit is not None else dify_config.UPLOAD_FILE_SIZE_LIMIT
)
return file_size <= file_size_limit
return file_size_limit * 1024 * 1024
def get_file_base64(self, file_id: str) -> str:
with self._session_maker(expire_on_commit=False) as session:

13
api/services/notification_service.py
View File

@ -2,8 +2,8 @@
from typing import Protocol
from constants.languages import languages
from machinery.context import RequestContext
from services.account_ports import AccountRepository
from services.entities.notification_entities import (
AccountNotification,
AccountNotificationBatch,
@ -22,20 +22,15 @@ class NotificationGateway(Protocol):
class NotificationService:
def __init__(self, *, accounts: AccountRepository, notifications: NotificationGateway) -> None:
self._accounts = accounts
def __init__(self, *, notifications: NotificationGateway) -> None:
self._notifications = notifications
def get_active(self, context: RequestContext) -> NotificationResult:
def get_active(self, context: RequestContext, language: str) -> NotificationResult:
batch = self._notifications.get_active(context.account_id)
if not batch.should_show:
return NotificationResult(should_show=False, notifications=())
account = self._accounts.get(context.account_id)
if account is None:
raise RuntimeError("Console account admission resolved an unknown account")
language = account.interface_language or _FALLBACK_LANGUAGE
language = language if language in languages else _FALLBACK_LANGUAGE
notifications = tuple(self._localize(notification, language) for notification in batch.notifications)
return NotificationResult(should_show=bool(notifications), notifications=notifications)

75
api/services/oauth_device_flow.py
View File

@ -10,12 +10,12 @@ import uuid
from dataclasses import asdict, dataclass, field
from datetime import UTC, datetime, timedelta
from enum import StrEnum
from typing import Any, NotRequired, TypedDict
from typing import NotRequired, TypedDict
from sqlalchemy import and_, func, select, update
from sqlalchemy import func, select, update
from sqlalchemy.orm import Session
from libs.oauth_bearer import TOKEN_CACHE_KEY_FMT, AuthContext, SubjectType
from libs.oauth_bearer import SubjectType, invalidate_oauth_token_cache
from models.oauth import OAuthAccessToken
logger = logging.getLogger(__name__)
@ -382,7 +382,7 @@ def mint_oauth_token(
)
if outcome.rotated and outcome.old_hash:
redis_client.delete(TOKEN_CACHE_KEY_FMT.format(hash=outcome.old_hash))
invalidate_oauth_token_cache(redis_client, outcome.old_hash)
return MintResult(token=token, token_id=outcome.token_id, expires_at=expires_at)
@ -487,70 +487,3 @@ def oauth_ttl_days(tenant_id: str | None = None) -> int:
logger.warning("%s=%d above max %d; clamping", _TTL_ENV_VAR, value, MAX_TTL_DAYS)
return MAX_TTL_DAYS
return value
def subject_match_clauses(ctx: AuthContext) -> tuple[Any, ...]:
if ctx.subject_type == SubjectType.ACCOUNT:
return (OAuthAccessToken.account_id == str(ctx.account_id),)
return (
OAuthAccessToken.subject_email == ctx.subject_email,
OAuthAccessToken.subject_issuer == ctx.subject_issuer,
OAuthAccessToken.account_id.is_(None),
)
def list_active_sessions(ctx: AuthContext, now: datetime, *, session: Session) -> list[OAuthAccessToken]:
return list(
session.execute(
select(OAuthAccessToken)
.where(
and_(
*subject_match_clauses(ctx),
OAuthAccessToken.revoked_at.is_(None),
OAuthAccessToken.token_hash.is_not(None),
OAuthAccessToken.expires_at > now,
)
)
.order_by(OAuthAccessToken.created_at.desc())
)
.scalars()
.all()
)
def token_belongs_to_subject(token_id: str, ctx: AuthContext, *, session: Session) -> bool:
row = session.execute(
select(OAuthAccessToken.id).where(
and_(
OAuthAccessToken.id == token_id,
*subject_match_clauses(ctx),
)
)
).first()
return row is not None
def revoke_oauth_token(redis_client: Any, token_id: str, *, session: Session) -> None:
row = (
session.query(OAuthAccessToken.token_hash)
.filter(
OAuthAccessToken.id == token_id,
OAuthAccessToken.revoked_at.is_(None),
)
.one_or_none()
)
pre_revoke_hash = row[0] if row else None
stmt = (
update(OAuthAccessToken)
.where(
OAuthAccessToken.id == token_id,
OAuthAccessToken.revoked_at.is_(None),
)
.values(revoked_at=datetime.now(UTC), token_hash=None)
)
session.execute(stmt)
session.commit()
if pre_revoke_hash:
redis_client.delete(TOKEN_CACHE_KEY_FMT.format(hash=pre_revoke_hash))

20
api/services/recommended_app_query_service.py
View File

@ -5,6 +5,8 @@ from typing import NamedTuple, Protocol
from constants.languages import languages
_DEFAULT_LANGUAGE = "en-US"
class RecommendedAppInfoRecord(NamedTuple):
id: str
@ -116,10 +118,9 @@ class RecommendedAppQueryService:
def list_recommended(
self,
*,
requested_language: str | None,
interface_language: str | None,
language: str,
) -> RecommendedAppListResult:
language = self._resolve_language(requested_language, interface_language)
language = language if language in languages else _DEFAULT_LANGUAGE
page = self._catalog.list_recommended(language)
return RecommendedAppListResult(
@ -130,10 +131,9 @@ class RecommendedAppQueryService:
def list_learn_dify(
self,
*,
requested_language: str | None,
interface_language: str | None,
language: str,
) -> LearnDifyAppListResult:
language = self._resolve_language(requested_language, interface_language)
language = language if language in languages else _DEFAULT_LANGUAGE
page = self._catalog.list_learn_dify(language)
return LearnDifyAppListResult(recommended_apps=self._with_trial_status(page.recommended_apps))
@ -176,11 +176,3 @@ class RecommendedAppQueryService:
)
for app in apps
)
@staticmethod
def _resolve_language(requested_language: str | None, interface_language: str | None) -> str:
if requested_language and requested_language in languages:
return requested_language
if interface_language:
return interface_language
return languages[0]

24
api/services/skill_management_service.py
View File

@ -3747,14 +3747,22 @@ class SkillManagementService:
@staticmethod
def _strip_single_root(paths: list[str]) -> dict[str, str]:
if not paths:
return {}
first_segments = {path.split("/", 1)[0] for path in paths if "/" in path}
root = next(iter(first_segments)) if len(first_segments) == 1 else None
if root is None or f"{root}/{_SKILL_MD}" not in paths or _SKILL_MD in paths:
if not paths or _SKILL_MD in paths:
return {path: path for path in paths}
stripped = {path: path.removeprefix(f"{root}/") for path in paths}
return stripped
# Identify the root by the unique top-level `<root>/SKILL.md`, rather than
# requiring every path to share one first segment: tools like macOS Finder's
# "Compress" add a sibling `__MACOSX/` metadata folder that must not defeat
# stripping of the real skill folder. Entries outside the detected root
# (like `__MACOSX/...`) are dropped rather than passed through, matching
# skill_package_service._normalize_members(ignore_outside_selected_root=True).
skill_md_roots = {
path.split("/", 1)[0] for path in paths if path.count("/") == 1 and path.endswith(f"/{_SKILL_MD}")
}
if len(skill_md_roots) != 1:
return {path: path for path in paths}
root = next(iter(skill_md_roots))
prefix = f"{root}/"
return {path: path.removeprefix(prefix) for path in paths if path == root or path.startswith(prefix)}
def _draft_payload_from_zip(
self,
@ -3775,6 +3783,8 @@ class SkillManagementService:
skill_md_content = ""
for info in infos:
raw_path = normalize_skill_file_path(info.filename.strip("/"))
if raw_path not in path_map:
continue
path = normalize_skill_file_path(path_map[raw_path])
if info.is_dir():
items.append(SkillDraftTreeItemPayload(path=path, kind=SkillFileKind.DIRECTORY))

6
api/services/trigger/trigger_subscription_builder_service.py
View File

@ -267,7 +267,11 @@ class TriggerSubscriptionBuilderService:
credentials=subscription_builder.credentials,
credential_type=credential_type,
credential_expires_at=subscription_builder.credential_expires_at or -1,
expires_at=subscription_builder.expires_at,
expires_at=(
subscription.expires_at
if subscription.expires_at is not None
else (subscription_builder.expires_at or -1)
),
)
# Delete the builder after successful subscription creation

18
api/services/web_passport_gateways.py
View File

@ -10,6 +10,16 @@ from services.enterprise.enterprise_service import PERMISSION_CHECK_MODES, WebAp
from services.web_passport_service import WebAppAuthType, WebPassportUnauthorizedError
def resolve_web_app_auth_type(access_mode: str) -> WebAppAuthType:
if access_mode == WebAppAccessMode.PUBLIC:
return WebAppAuthType.PUBLIC
if access_mode in PERMISSION_CHECK_MODES:
return WebAppAuthType.INTERNAL
if access_mode == WebAppAccessMode.SSO_VERIFIED:
return WebAppAuthType.EXTERNAL
raise ValueError(f"Unsupported web app access mode: {access_mode}")
class DeploymentWebPassportAuthGateway:
def __init__(
self,
@ -25,13 +35,7 @@ class DeploymentWebPassportAuthGateway:
def get_app_auth_type(self, app_id: str) -> WebAppAuthType:
access_mode = self._get_app_access_mode(app_id).access_mode
if access_mode == WebAppAccessMode.PUBLIC:
return WebAppAuthType.PUBLIC
if access_mode in PERMISSION_CHECK_MODES:
return WebAppAuthType.INTERNAL
if access_mode == WebAppAccessMode.SSO_VERIFIED:
return WebAppAuthType.EXTERNAL
raise ValueError(f"Unsupported web app access mode: {access_mode}")
return resolve_web_app_auth_type(access_mode)
class PassportTokenGateway:

Some files were not shown because too many files have changed in this diff Show More