dify/e2e/AGENTS.md

# E2E

This package contains Dify's repository-level Cucumber scenarios with Playwright as the browser layer. This file owns current package architecture, runtime, session and tag semantics, seed, protocol, and cleanup contracts. The repo-local `e2e-cucumber-playwright` skill owns authoring and review methodology; feature-specific facts belong in the nearest feature `AGENTS.md`.

## Commands

Run commands from the repository root. Install dependencies and browsers once with `pnpm install` and `pnpm -C e2e e2e:install`. Run only one local `pnpm -C e2e e2e*` process at a time because runners share ports, auth state, and log paths.

- Existing initialized instance: `pnpm -C e2e e2e`
- Reset, initialize, and run deterministic scenarios: `pnpm -C e2e e2e:full`
- Tagged subset: `pnpm -C e2e e2e -- --tags @smoke`
- Headed debugging: `pnpm -C e2e e2e:headed -- --tags @smoke`
- External runtime preparation and run: `pnpm -C e2e e2e:external:prepare`, then `pnpm -C e2e e2e:external`
- Reset persisted E2E state: `pnpm -C e2e e2e:reset`
- Middleware lifecycle: `pnpm -C e2e e2e:middleware:up` and `pnpm -C e2e e2e:middleware:down`
- Scoped static checks: `vp check e2e`

The runner reuses `web/.next/BUILD_ID` when present. Set `E2E_FORCE_WEB_BUILD=1` to force a frontend rebuild. Use `E2E_BROWSER=webkit` for focused cross-browser runs and `E2E_SLOW_MO=500` with a headed command for local action debugging.

## Runtime Ownership

- `scripts/setup.ts` owns reset, middleware, backend, and frontend startup.
- `scripts/run-cucumber.ts` owns E2E orchestration and Cucumber invocation.
- `support/web-server.ts` owns frontend reuse, readiness, and shutdown.
- `features/support/hooks.ts` owns shared auth bootstrap, scenario lifecycle, and diagnostics.
- `features/support/world.ts` owns `DifyWorld`, the per-scenario behavior `BrowserContext`, and its authenticated setup and cleanup client. Browser and API identities remain separate so unauthenticated and logout journeys cannot invalidate fixture ownership.
- Cross-actor scenarios keep each actor in a separate `BrowserContext` and typed `DifyWorld` state so diagnostics and cleanup cover every actor.
- `features/step-definitions/` contains capability-oriented glue; `common/` is reserved for genuinely cross-capability steps.
- Step definitions that access World state use `async function (this: DifyWorld, ...)`; arrow functions cannot receive Cucumber's bound World instance.

An uninitialized instance is installed and authenticated lazily; an initialized instance signs in and reuses authenticated state. Full runs prove reset and bootstrap during setup rather than through a Gherkin scenario. Cucumber's exit status is the behavior gate, and the runner also requires at least one `testCaseStarted` message so an empty tag selection cannot pass. Do not replace this gate with scenario-count baselines or skipped-scenario allowlists.

## Tags And External Runtime

- Default scenarios use shared authenticated storage state. `@unauthenticated` creates a clean context; `@authenticated` is an intent and selection tag only.
- `@prepared` requires the strict post-merge seed profile.
- `@external-model` and `@external-tool` identify scenarios that call real external runtimes. Deterministic commands exclude these tags; external commands are opt-in.
- `@microphone` uses the checked-in fake audio fixture and an isolated Chromium context.
- `@browser-smoke` runs focused keyboard and navigation coverage in Chromium and WebKit CI lanes.
- Feature-owned services use their own tags. Agent v2 runtime scenarios use `@agent-backend-runtime` and require the explicit runtime-availability step. Set `E2E_START_AGENT_BACKEND=1` to start it locally, or provide `E2E_AGENT_BACKEND_URL` / `AGENT_BACKEND_BASE_URL`.

Do not overload runtime tags to imply unrelated services or silently skip behavior when a required fixture is missing.

## Browser, API, And Contract Boundaries

The action under test belongs to the browser. APIs may prepare fixtures, poll persistence, and clean up; they do not replace the user's `When` action. Prefer a user-observable browser result unless persisted backend state is the contract under test.

For ordinary Console JSON and representable multipart operations, use the scenario- or process-owned generated oRPC client with request and response validation enabled. Call generated operations directly. Do not add handwritten endpoint URLs, duplicate DTOs or schemas, response casts, one-to-one forwarding wrappers, mutable cross-scenario clients, or TanStack Query caching.

Keep helpers only when they own fixture construction, multi-operation orchestration, cleanup registries, invariants, eventual-consistency polling, narrowed test views, or a protocol adapter. SSE, binary downloads, redirect-only flows, external services, and infrastructure readiness may use centralized adapters under their real owner.

Validation failures are contract failures. Trace them to the backend schema owner, update `api/controllers/API_SCHEMA_GUIDE.md` contracts when required, regenerate `@dify/contracts`, and keep the scenario aligned with the product's real state owner. Do not disable validation or add fallback schemas to make E2E pass.

## Seeds, Cleanup, And Diagnostics

- Generate disposable resource names through `support/naming.ts` with an `E2E` prefix.
- Keep deterministic upload material in `fixtures/test-materials/` and resolve it through `support/test-materials.ts`.
- Seed scripts own shared long-lived fixtures; scenarios own disposable resources they create and must register cleanup.
- Use typed `DifyWorld` cleanup fields for known resource types and `registerCleanup(...)` for additional lifecycle owners. Registered callbacks run LIFO after typed cleanup queues.
- Remove child and referencing resources before owners. Attach cleanup failures to the report instead of swallowing them.

Failures produce screenshots and HTML captures under `cucumber-report/artifacts/`; the HTML and Cucumber Messages reports live under `cucumber-report/`. Backend and frontend startup logs live under `.logs/`. Additional CI lanes preserve their own report and log directories.