diff --git a/99 Daily/2026-05-15.md b/99 Daily/2026-05-15.md index 2c20ec2..e86ea65 100644 --- a/99 Daily/2026-05-15.md +++ b/99 Daily/2026-05-15.md @@ -79,3 +79,5 @@ tags: [daily] - 15:42 — session ended | `ai_leed` - 15:44 — session ended | `ai_leed` - 15:53 (6min) — session ended | `Shumiland` +- 15:58 (1min) — session ended | `Shumiland` +- 16:01 — session ended | `Shumiland` diff --git a/raw/jobs-queue__overview.md b/raw/_processed/jobs-queue__overview.md similarity index 100% rename from raw/jobs-queue__overview.md rename to raw/_processed/jobs-queue__overview.md diff --git a/raw/jobs-queue__queues.md b/raw/_processed/jobs-queue__queues.md similarity index 100% rename from raw/jobs-queue__queues.md rename to raw/_processed/jobs-queue__queues.md diff --git a/raw/jobs-queue__quick-start-example.md b/raw/_processed/jobs-queue__quick-start-example.md similarity index 100% rename from raw/jobs-queue__quick-start-example.md rename to raw/_processed/jobs-queue__quick-start-example.md diff --git a/raw/jobs-queue__schedules.md b/raw/_processed/jobs-queue__schedules.md similarity index 100% rename from raw/jobs-queue__schedules.md rename to raw/_processed/jobs-queue__schedules.md diff --git a/raw/jobs-queue__tasks.md b/raw/_processed/jobs-queue__tasks.md similarity index 100% rename from raw/jobs-queue__tasks.md rename to raw/_processed/jobs-queue__tasks.md diff --git a/raw/jobs-queue__workflows.md b/raw/_processed/jobs-queue__workflows.md similarity index 100% rename from raw/jobs-queue__workflows.md rename to raw/_processed/jobs-queue__workflows.md diff --git a/raw/live-preview__client.md b/raw/_processed/live-preview__client.md similarity index 100% rename from raw/live-preview__client.md rename to raw/_processed/live-preview__client.md diff --git a/raw/live-preview__frontend.md b/raw/_processed/live-preview__frontend.md similarity index 100% rename from raw/live-preview__frontend.md rename to raw/_processed/live-preview__frontend.md diff --git a/raw/live-preview__overview.md b/raw/_processed/live-preview__overview.md similarity index 100% rename from raw/live-preview__overview.md rename to raw/_processed/live-preview__overview.md diff --git a/raw/live-preview__server.md b/raw/_processed/live-preview__server.md similarity index 100% rename from raw/live-preview__server.md rename to raw/_processed/live-preview__server.md diff --git a/raw/local-api__access-control.md b/raw/_processed/local-api__access-control.md similarity index 100% rename from raw/local-api__access-control.md rename to raw/_processed/local-api__access-control.md diff --git a/raw/local-api__outside-nextjs.md b/raw/_processed/local-api__outside-nextjs.md similarity index 100% rename from raw/local-api__outside-nextjs.md rename to raw/_processed/local-api__outside-nextjs.md diff --git a/raw/local-api__overview.md b/raw/_processed/local-api__overview.md similarity index 100% rename from raw/local-api__overview.md rename to raw/_processed/local-api__overview.md diff --git a/raw/local-api__server-functions.md b/raw/_processed/local-api__server-functions.md similarity index 100% rename from raw/local-api__server-functions.md rename to raw/_processed/local-api__server-functions.md diff --git a/raw/migration-guide__overview.md b/raw/_processed/migration-guide__overview.md similarity index 100% rename from raw/migration-guide__overview.md rename to raw/_processed/migration-guide__overview.md diff --git a/raw/migration-guide__v3.md b/raw/_processed/migration-guide__v3.md similarity index 100% rename from raw/migration-guide__v3.md rename to raw/_processed/migration-guide__v3.md diff --git a/raw/migration-guide__v4.md b/raw/_processed/migration-guide__v4.md similarity index 100% rename from raw/migration-guide__v4.md rename to raw/_processed/migration-guide__v4.md diff --git a/wiki/_master-index.md b/wiki/_master-index.md index fca0bb7..387c623 100644 --- a/wiki/_master-index.md +++ b/wiki/_master-index.md @@ -35,7 +35,7 @@ This 3-hop pattern works for hundreds of articles without vector search. | [[wiki/reports/_index\|reports/]] | Weekly and monthly summaries — generate: `uv run python scripts/report-generator.py --weekly` | 1 | | [[wiki/infrastructure/_index\|infrastructure/]] | Server inventory: all 10 SSH hosts — optical, optical-dev, optical-prod, baic, librechat, modocmms, box-cli, aimpress, pve | 12 | | [[wiki/testing/_index\|testing/]] | Web app testing: functional, performance, security, UI types; TDD/BDD/Agile methodologies; Selenium/Cypress/Playwright/JMeter/OWASP ZAP tools | 1 | -| [[wiki/payloadcms/_index\|payloadcms/]] | Full Payload CMS reference — getting started, config, database (Postgres/MongoDB/SQLite), all 22 field types, access control, hooks, authentication (cookies, JWT, API keys, custom strategies, token data), admin UI, custom components, Lexical rich text, live preview, versions/drafts, Local/REST/GraphQL APIs, queries, plugins, jobs queue, upload, ecommerce, production deploy, TypeScript, migration guides, i18n, localization, hierarchy | 97 | +| [[wiki/payloadcms/_index\|payloadcms/]] | Full Payload CMS reference — getting started, config, database (Postgres/MongoDB/SQLite), all 22 field types, access control, hooks, authentication (cookies, JWT, API keys, custom strategies, token data), admin UI, custom components, Lexical rich text, live preview, versions/drafts, Local/REST/GraphQL APIs, queries, plugins, jobs queue, upload, ecommerce, production deploy, TypeScript, migration guides, i18n, localization, hierarchy | 111 | | [[wiki/shared-patterns/_index\|shared-patterns/]] | Oliver Agency standard library patterns: httpx, structlog, pydantic-settings, alembic — reuse before writing from scratch | 4 | | [[wiki/mistakes/_index\|mistakes/]] | Anti-patterns extracted from sessions — per-stack running lists (fastapi, react, docker, postgres, general) — injected at session start | 5 | diff --git a/wiki/payloadcms/_index.md b/wiki/payloadcms/_index.md index 1f26a9a..d90c6c0 100644 --- a/wiki/payloadcms/_index.md +++ b/wiki/payloadcms/_index.md @@ -7,8 +7,13 @@ | [[wiki/payloadcms/admin-panel-overview\|Admin Panel Overview]] | Auto-generated Next.js admin UI — structure, config options, auth, routes, i18n, timezones, toasts | raw/admin__overview.md | 2026-05-15 | | [[wiki/payloadcms/admin-preferences\|Admin Preferences]] | Per-user persistent preferences via `usePreferences` hook — built-ins, DB schema, REST/GraphQL, usage pattern | raw/admin__preferences.md | 2026-05-15 | | [[wiki/payloadcms/admin-preview\|Admin Preview & Draft Preview]] | Preview button config, draft preview flow with Next.js (3 steps), conditional preview, PREVIEW_SECRET guard | raw/admin__preview.md | 2026-05-15 | -| [[wiki/payloadcms/jobs-queue\|Jobs Queue]] | Full reference: Tasks, Jobs, Queues, Workflows, Schedules, concurrency controls, worker strategies, gotchas | raw/jobs-queue__*.md | 2026-05-15 | +| [[wiki/payloadcms/jobs-queue\|Jobs Queue — Overview]] | Core concepts (Task/Workflow/Job/Queue), use cases, running/scheduling options (bin script / autoRun / API endpoint), common pitfalls (duplicate scheduling, autoRun on serverless), admin UI visibility | raw/jobs-queue__overview.md | 2026-05-15 | | [[wiki/payloadcms/jobs-queue-jobs\|Jobs Queue — Jobs Detail]] | Queuing from hooks/endpoints/server actions, job options (waitUntil/queue/log), full status schema, access control (jobs.access + overrideAccess), cancellation (cancelByID/cancel/JobCancelledError) | raw/jobs-queue__jobs.md | 2026-05-15 | +| [[wiki/payloadcms/jobs-queue-queues\|Jobs Queue — Queues & Execution]] | 4 execution methods (bin script/autoRun/endpoint/Local API), processing order, queue strategies, Vercel Cron + CRON_SECRET, troubleshooting | raw/jobs-queue__queues.md | 2026-05-15 | +| [[wiki/payloadcms/jobs-queue-quick-start\|Jobs Queue — Quick Start Examples]] | Step-by-step: welcome email (manual queue + afterChange hook), daily scheduled report (schedule + autoRun), one-time waitUntil job, serverless warning | raw/jobs-queue__quick-start-example.md | 2026-05-15 | +| [[wiki/payloadcms/jobs-queue-schedules\|Jobs Queue — Schedules]] | Cron scheduling: schedule vs waitUntil, two-config requirement (schedule + runner), bin scripts vs autoRun vs API endpoint, lifecycle, concurrency hooks, serverless pattern, troubleshooting | raw/jobs-queue__schedules.md | 2026-05-15 | +| [[wiki/payloadcms/jobs-queue-tasks\|Jobs Queue — Tasks]] | Task config reference (slug/handler/inputSchema/outputSchema/retries/schedule), failure handling, idempotency, shouldRestore, nested tasks, handler file paths | raw/jobs-queue__tasks.md | 2026-05-15 | +| [[wiki/payloadcms/jobs-queue-workflows\|Jobs Queue — Workflows]] | Multi-task pipelines with per-task retry recovery: workflow config, stable task IDs, inline tasks, task output access, concurrency controls (exclusive/supersedes), best practices | raw/jobs-queue__workflows.md | 2026-05-15 | | [[wiki/payloadcms/admin\|Admin Panel (full)]] | Complete admin config reference: preview, draft-preview, preferences API, all React hooks | raw/admin__*.md | 2026-05-15 | | [[wiki/payloadcms/custom-components\|Custom Components]] | Root slots, dashboard widgets, edit/list view slots, document views, custom views, providers | raw/custom-components__*.md | 2026-05-15 | @@ -17,6 +22,7 @@ | [[wiki/payloadcms/folders\|Folders — Content Organization]] | Folder grouping for collections (beta): global config, per-collection opt-in, nesting, browse-by-folder view | raw/folders__overview.md | 2026-05-15 | | [[wiki/payloadcms/email\|Email — Adapters, SMTP, Resend, Attachments]] | Full email reference: Nodemailer (SMTP/SendGrid/dev), Resend (serverless), sendEmail API, attachments (Buffer/path/Base64/URL), media collection attachments | raw/email__overview.md | 2026-05-15 | | [[wiki/payloadcms/local-api\|Local API]] | Direct DB access via `payload.*` — CRUD, auth ops, globals, server functions, access control, outside Next.js | raw/local-api__*.md | 2026-05-15 | +| [[wiki/payloadcms/local-api-outside-nextjs\|Local API — Outside Next.js]] | Standalone scripts with `getPayload`, `payload run` CLI, ESM requirement, tsx/swc/bun transpilation options | raw/local-api__outside-nextjs.md | 2026-05-15 | | [[wiki/payloadcms/rest-api\|REST API]] | Auto-generated HTTP endpoints, all query params, auth routes, SDK, custom endpoints, method override | raw/rest-api__overview.md | 2026-05-15 | | [[wiki/payloadcms/queries\|Queries]] | Where operators, and/or logic, depth, pagination, sort, select/populate — full reference with examples | raw/queries__*.md | 2026-05-15 | | [[wiki/payloadcms/ecommerce\|Ecommerce]] | Plugin setup, Stripe payments, cart API, frontend hooks, multi-currency, piecemeal collections | raw/ecommerce__*.md | 2026-05-15 | @@ -62,7 +68,7 @@ | [[wiki/payloadcms/fields-complex\|PayloadCMS — Fields: Complex]] | Complex fields define structure, layout, or relations between documents. Some are data-bearing (Arra | https://payloadcms.com/docs/fields/array | 2026-05-15 | | [[wiki/payloadcms/getting-started\|PayloadCMS — Getting Started]] | Payload is a code-first, open-source Next.js fullstack framework that acts as a CMS, enterprise tool | getting-started__what-is-payload.md | 2026-05-15 | | [[wiki/payloadcms/hooks\|PayloadCMS — Hooks]] | All hook types (Root/Collection/Global/Field), awaited vs fire-and-forget, APIError, performance (avoid read hooks), context caching, jobs offload | raw/hooks__overview.md | 2026-05-15 | -| [[wiki/payloadcms/live-preview\|PayloadCMS — Live Preview]] | - Renders your frontend inside an iframe directly in the Admin Panel | https://payloadcms.com/docs/live-preview/overview | 2026-05-15 | +| [[wiki/payloadcms/live-preview\|Live Preview — Overview]] | iframe-based real-time preview via postMessage: url config (static/dynamic/conditional), breakpoints, server-side vs client-side modes, depth mismatch gotcha | raw/live-preview__overview.md | 2026-05-15 | | [[wiki/payloadcms/rich-text\|PayloadCMS — Rich Text (Lexical)]] | - Rich text is powered by **Lexical** (Meta's framework), exposed via `@payloadcms/richtext-lexical` | https://payloadcms.com/docs/rich-text/overview | 2026-05-15 | | [[wiki/payloadcms/examples-overview\|Examples Overview]] | 9 official starter examples (auth, draft-preview, multi-tenant, tailwind-shadcn, whitelabel…) — `npx create-payload-app --example ` | raw/examples__overview.md | 2026-05-15 | | [[wiki/payloadcms/fields-array\|Array Field]] | Repeating row field — config options, minRows/maxRows, localized, unique gotcha, custom RowLabel, nested arrays | raw/fields__array.md | 2026-05-15 | @@ -99,3 +105,11 @@ | [[wiki/payloadcms/graphql-extending\|GraphQL — Custom Queries and Mutations]] | Add custom GraphQL ops via `graphQL.queries`/`graphQL.mutations` in buildConfig; resolver signature, `depth: 0` gotcha, `buildPaginatedListType`, collection graphQL types | raw/graphql__extending.md | 2026-05-15 | | [[wiki/payloadcms/graphql-schema\|GraphQL — Schema Generation]] | `payload-graphql generate:schema` CLI, PAYLOAD_CONFIG_PATH for non-root configs, `interfaceName` for reusable top-level GraphQL types | raw/graphql__graphql-schema.md | 2026-05-15 | | [[wiki/payloadcms/hierarchy\|Hierarchy — Tree Structure]] | Auto-generated parent-child trees: virtual slug/title paths, opt-in path computation, polymorphic hierarchies, no cascade updates | raw/hierarchy__overview.md | 2026-05-15 | +| [[wiki/payloadcms/live-preview-client\|Live Preview — Client-Side]] | `useLivePreview` hook for React/Vue, base package API (subscribe/unsubscribe/ready/isLivePreviewEvent), full hook implementation, depth mismatch gotcha, CORS/CSRF/CSP troubleshooting | raw/live-preview__client.md | 2026-05-15 | +| [[wiki/payloadcms/live-preview-frontend\|Live Preview — Frontend Integration Guide]] | Decision guide: server-side vs client-side, package summary, when to use each approach | raw/live-preview__frontend.md | 2026-05-15 | +| [[wiki/payloadcms/live-preview-server\|Live Preview — Server-Side]] | `RefreshRouteOnSave` for Next.js App Router (RSC): postMessage→router.refresh() flow, base package API, Autosave tip, CSP gotcha | raw/live-preview__server.md | 2026-05-15 | +| [[wiki/payloadcms/migration-guide-overview\|Migration Guides — Overview]] | Semantic versioning policy, upgrade path table (2→3, 3→4), links to full migration guides | raw/migration-guide__overview.md | 2026-05-15 | +| [[wiki/payloadcms/local-api-access-control\|Local API — Access Control]] | Local API skips access control by default; enforce with `overrideAccess: false` + `user` — when/why, mode table, security gotchas | raw/local-api__access-control.md | 2026-05-15 | +| [[wiki/payloadcms/local-api-server-functions\|Local API — Server Functions]] | `'use server'` bridge between client components and Local API: create/update/upload/auth patterns, built-in login/logout/refresh helpers from `@payloadcms/next/auth`, error handling, access control | raw/local-api__server-functions.md | 2026-05-15 | +| [[wiki/payloadcms/migration-guide-v2-to-v3\|Migration Guide — 2.0 → 3.0]] | Full breaking-change reference: admin replatform to Next.js App Router, bundler removal, secret move, custom component paths, endpoint handlers (Web Request + manual data parsing), cloud storage standalone packages, plugin named exports, reserved field names | raw/migration-guide__v3.md | 2026-05-15 | +| [[wiki/payloadcms/migration-guide-v3-to-v4\|Migration Guide — 3.0 → 4.0]] | All v4 breaking changes: List View Select API default, Globals components.edit rename, forceSelect→select function, hideAPIURL removal, aliased re-export cleanup, useDocumentTitle split, Import/Export hooks | raw/migration-guide__v4.md | 2026-05-15 | diff --git a/wiki/payloadcms/jobs-queue-queues.md b/wiki/payloadcms/jobs-queue-queues.md new file mode 100644 index 0000000..b697a3e --- /dev/null +++ b/wiki/payloadcms/jobs-queue-queues.md @@ -0,0 +1,268 @@ +--- +title: "Jobs Queue — Queues & Execution" +aliases: [jobs-queue-execution, payload-queues, payload-job-runners] +tags: [payloadcms, jobs-queue, queues, cron, workers, autorun, vercel] +sources: [raw/jobs-queue__queues.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +## Overview + +Queues are named channels for grouping jobs. All jobs go into `default` by default. Custom queue names allow different cron schedules (e.g. `nightly` vs `default`). This article covers **how to execute** queued jobs. + +--- + +## Execution Methods + +### 1. Bin Script (Recommended — Dedicated Servers) + +Runs in a **separate process** from Next.js — preferred for production. + +```sh +# Basic +pnpm payload jobs:run + +# Custom queue + limit +pnpm payload jobs:run --queue myQueue --limit 15 + +# Cron schedule (production recommended) +pnpm payload jobs:run --cron "*/5 * * * *" --queue myQueue + +# Also handle scheduled task queuing +pnpm payload jobs:run --cron "*/5 * * * *" --queue myQueue --handle-schedules + +# All queues +pnpm payload jobs:run --all-queues +``` + +**docker-compose.yml pattern:** + +```yaml +services: + nextjs: + command: pnpm start + worker-default: + command: pnpm payload jobs:run --cron "*/5 * * * *" --queue default + worker-nightly: + command: pnpm payload jobs:run --cron "* * * * *" --queue nightly --handle-schedules +``` + +Benefits: independent process, independent scaling, no Next.js overhead. + +--- + +### 2. `autoRun` (Alternative — Dedicated Servers Only) + +Runs within the Next.js process. **Only executes** already-queued jobs — does NOT queue new ones. + +```ts +jobs: { + autoRun: [ + { cron: '*/5 * * * *', queue: 'default', limit: 50 }, + { cron: '* * * * *', queue: 'nightly', limit: 100 }, + ], + shouldAutoRun: async (payload) => { + return process.env.ENABLE_JOB_WORKERS === 'true' + }, +} +``` + +> **Warning:** Do not use `autoRun` on serverless platforms (Vercel, Netlify). Use the Endpoint method instead. + +--- + +### 3. HTTP Endpoint (Serverless — Vercel/Netlify) + +Auto-mounted at `/api/payload-jobs/run`. + +```ts +await fetch('/api/payload-jobs/run?limit=100&queue=nightly', { + method: 'GET', + headers: { Authorization: `Bearer ${token}` }, +}) +``` + +Query params: `limit` (default 10), `queue` (default `'default'`), `allQueues=true`. + +**Vercel Cron (`vercel.json`):** + +```json +{ + "crons": [{ "path": "/api/payload-jobs/run", "schedule": "*/5 * * * *" }] +} +``` + +**Secure with `CRON_SECRET`:** + +```ts +jobs: { + access: { + run: ({ req }) => { + if (req.user) return true + const secret = process.env.CRON_SECRET + if (!secret) return false + return req.headers.get('authorization') === `Bearer ${secret}` + }, + }, +} +``` + +Vercel automatically injects `CRON_SECRET` as the `Authorization` header when triggering the cron. + +--- + +### 4. Local API (Programmatic / Testing) + +```ts +// Run from default queue (limit 10) +await payload.jobs.run() + +// Custom queue + limit +await payload.jobs.run({ queue: 'nightly', limit: 100 }) + +// All queues +await payload.jobs.run({ allQueues: true }) + +// Filtered run +await payload.jobs.run({ where: { 'input.message': { equals: 'secret' } } }) + +// Single job by ID +await payload.jobs.runByID({ id: myJobID }) +``` + +--- + +## Processing Order + +Default: FIFO (`createdAt` ascending). + +```ts +// Global LIFO +jobs: { processingOrder: '-createdAt' } + +// Per-queue +jobs: { + processingOrder: { + default: 'createdAt', // FIFO + queues: { nightly: '-createdAt' }, // LIFO + }, +} + +// Dynamic function +jobs: { + processingOrder: ({ queue }) => + queue === 'myQueue' ? '-createdAt' : 'createdAt', +} + +// Per-job at queue time +await payload.jobs.queue({ + workflow: 'createPost', + processingOrder: '-createdAt', +}) +``` + +--- + +## Common Queue Strategies + +### Priority-Based + +```ts +autoRun: [ + { cron: '* * * * *', queue: 'critical', limit: 100 }, // every minute + { cron: '*/5 * * * *', queue: 'default', limit: 50 }, // every 5 min + { cron: '0 2 * * *', queue: 'batch', limit: 1000 }, // nightly 2 AM +] +``` + +### Feature-Based + +```ts +autoRun: [ + { cron: '*/2 * * * *', queue: 'emails', limit: 100 }, + { cron: '*/10 * * * *', queue: 'images', limit: 50 }, + { cron: '0 * * * *', queue: 'analytics', limit: 1000 }, +] +``` + +### Environment-Based (selective worker activation) + +```ts +shouldAutoRun: async () => process.env.ENABLE_JOB_WORKERS === 'true' +``` + +--- + +## Execution Method Decision Table + +| Method | Best For | Pros | Cons | +|--------|----------|------|------| +| **Bin script** | Dedicated servers (recommended) | Separate process, easy scaling | Requires dedicated server | +| **autoRun** | Dedicated servers (alternative) | Simple setup | Runs in Next.js process | +| **Endpoint** | Serverless (Vercel, Netlify) | Works serverless | Needs external cron trigger | +| **Local API** | Testing, custom scheduling | Full control | Must implement scheduling | + +**Recommendations:** +- Dedicated server → Bin script with `--cron` +- Serverless → Endpoint + Vercel Cron +- Development → Bin script or Local API +- Tests → Local API with `runByID()` + +--- + +## Troubleshooting + +### Jobs not running + +1. **No jobs queued** — `autoRun` executes jobs but doesn't queue them. Add `payload.jobs.queue()` calls or use `schedule` on tasks. +2. **Queue name mismatch** — `schedule: [{ queue: 'reports' }]` must match `autoRun: [{ queue: 'reports' }]`. +3. **Serverless platform** — `autoRun` won't survive request lifecycle on Vercel. Use Endpoint method. +4. **`waitUntil` in the future** — check `payload-jobs` collection for future `waitUntil` values. +5. **HMR in dev** — restart dev server after changing job config. + +### Debug checklist + +```ts +// Verify shouldAutoRun fires +shouldAutoRun: async () => { console.log('shouldAutoRun called'); return true } + +// Make jobs collection visible +jobsCollectionOverrides: ({ defaultJobsCollection }) => ({ + ...defaultJobsCollection, + admin: { ...defaultJobsCollection.admin, hidden: false }, +}) +``` + +In admin, look for: +- `processing: true` (stuck) → worker crashed +- `hasError: true` → check `log` field +- `completedAt: null` → not yet run + +### Jobs running slowly + +- Increase `limit` in `autoRun` +- Decrease cron interval (`* * * * *` = every minute) +- Scale horizontally: multiple servers with `ENABLE_JOB_WORKERS=true` + +--- + +## Key Takeaways + +- `queue` is a named channel — assign at `payload.jobs.queue({ queue: 'nightly' })` +- **Four execution methods**: bin script (recommended), `autoRun`, HTTP endpoint, Local API +- `autoRun` only **runs** jobs — it does NOT queue them; use `schedule` on tasks or manual `payload.jobs.queue()` to queue +- **Serverless**: never use `autoRun`; use `/api/payload-jobs/run` + Vercel Cron + `CRON_SECRET` +- Queue name in `schedule` / `queue()` must exactly match queue name in `autoRun` / bin script +- Bin script = separate process → independent scaling, no Next.js overhead +- Default processing order is FIFO; override globally, per-queue, or per-job with `processingOrder` +- Enable `admin: { hidden: false }` on `payload-jobs` collection for debugging + +--- + +## Related + +- [[wiki/payloadcms/jobs-queue|Jobs Queue — Overview]] +- [[wiki/payloadcms/jobs-queue-jobs|Jobs Queue — Jobs Detail]] +- [[wiki/payloadcms/configuration|Payload Config — Overview]] +- [[wiki/payloadcms/production|Production & Performance]] diff --git a/wiki/payloadcms/jobs-queue-quick-start.md b/wiki/payloadcms/jobs-queue-quick-start.md new file mode 100644 index 0000000..4f3c74d --- /dev/null +++ b/wiki/payloadcms/jobs-queue-quick-start.md @@ -0,0 +1,179 @@ +--- +title: "Jobs Queue — Quick Start Examples" +aliases: [jobs-queue-quick-start, payload-jobs-example, jobs-queue-welcome-email] +tags: [payloadcms, jobs-queue, tasks, scheduling, cron, background-jobs] +sources: [raw/jobs-queue__quick-start-example.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +## Why Use a Job Queue? + +Instead of running slow/risky work inline in hooks: + +- **Non-blocking** — API returns immediately; email/heavy work runs async +- **Resilience** — automatic retries if external service is down +- **Scalability** — job workers can run on separate servers +- **Monitoring** — all jobs stored in DB with status + error logs + +--- + +## Example 1: Welcome Email on User Signup + +### Step 1 — Define the Task (`payload.config.ts`) + +```ts +jobs: { + tasks: [ + { + slug: 'sendWelcomeEmail', + retries: 3, + inputSchema: [ + { name: 'userEmail', type: 'email', required: true }, + { name: 'userName', type: 'text', required: true }, + ], + handler: async ({ input, req }) => { + await req.payload.sendEmail({ + to: input.userEmail, + subject: 'Welcome!', + text: `Hi ${input.userName}, welcome to our platform!`, + }) + return { output: { emailSent: true } } + }, + }, + ], +} +``` + +### Step 2 — Queue the Job (from a Collection hook) + +```ts +// In users collection config +hooks: { + afterChange: [ + async ({ req, doc, operation }) => { + if (operation === 'create') { + await req.payload.jobs.queue({ + task: 'sendWelcomeEmail', + input: { userEmail: doc.email, userName: doc.name }, + }) + } + }, + ], +} +``` + +Job is stored in `payload-jobs` collection immediately; runs async — no API delay. + +### Step 3 — Run Jobs via `autoRun` + +```ts +jobs: { + tasks: [ /* ... */ ], + autoRun: [ + { cron: '*/5 * * * *' }, // check every 5 minutes + ], +} +``` + +--- + +## Example 2: Scheduled Daily Report (No User Trigger) + +### Task with `schedule` Property + +```ts +{ + slug: 'generateDailyReport', + schedule: [ + { + cron: '0 8 * * *', // 8:00 AM daily + queue: 'reports', + }, + ], + handler: async ({ req }) => { + const report = await req.payload.create({ collection: 'reports', data: { /* ... */ } }) + return { output: { reportId: report.id } } + }, +} +``` + +### `autoRun` Must Match the Queue Name + +```ts +autoRun: [ + { + cron: '* * * * *', // check every minute + queue: 'reports', // MUST match schedule.queue + limit: 10, + }, +] +``` + +> **Critical gotcha:** `schedule.queue` and `autoRun.queue` must be identical. Mismatch → jobs queued but never executed. + +### Execution Flow + +1. **8:00 AM** — `schedule` auto-queues a job into `'reports'` queue +2. **Within 1 min** — `autoRun` cron finds the job +3. **Execution** — report generated +4. **Next day** — repeats automatically + +--- + +## One-Time Future Job (`waitUntil`) + +```ts +await payload.jobs.queue({ + task: 'publishPost', + input: { postId: '123' }, + waitUntil: new Date('2024-12-25T15:00:00Z'), +}) +``` + +Different from `schedule` — runs **once** at specified time, not recurring. + +--- + +## Approach Comparison + +| Approach | When to Use | Example | +|----------|-------------|---------| +| **Manual Queuing** | Triggered by user actions / data changes | Welcome emails, payments, notifications | +| **Scheduled (`schedule`)** | Automatic recurring at fixed intervals | Daily reports, weekly cleanups, nightly syncs | +| **`waitUntil`** | One-time job in the future | Publish at 3pm, trial expiry reminder | + +--- + +## Serverless Warning + +`autoRun` does **not** work on Vercel/serverless. Use the [[wiki/payloadcms/jobs-queue-queues|Vercel Cron approach]] with a dedicated API endpoint instead. + +--- + +## Key Takeaways + +- Define tasks in `jobs.tasks[]` with `slug`, `inputSchema`, `retries`, and `handler` +- Queue manually: `req.payload.jobs.queue({ task: 'slug', input: {...} })` +- `autoRun` polls for pending jobs on a cron schedule +- For scheduled recurring tasks: add `schedule` to the task + matching `queue` in `autoRun` +- Queue names in `schedule.queue` and `autoRun.queue` **must match exactly** +- `waitUntil` = one-time future job; `schedule` = recurring +- `autoRun` does not work on serverless (Vercel) — use API endpoint trigger instead + +--- + +## Related Articles + +- [[wiki/payloadcms/jobs-queue|Jobs Queue — Overview]] +- [[wiki/payloadcms/jobs-queue-jobs|Jobs Queue — Jobs Detail]] +- [[wiki/payloadcms/jobs-queue-queues|Jobs Queue — Queues & Execution]] +- [[wiki/payloadcms/hooks|PayloadCMS — Hooks]] +- [[wiki/payloadcms/email|Email — Adapters, SMTP, Resend, Attachments]] + +--- + +## Sources + +- `raw/jobs-queue__quick-start-example.md` +- https://payloadcms.com/docs/jobs-queue/quick-start-example diff --git a/wiki/payloadcms/jobs-queue-schedules.md b/wiki/payloadcms/jobs-queue-schedules.md new file mode 100644 index 0000000..3af67b7 --- /dev/null +++ b/wiki/payloadcms/jobs-queue-schedules.md @@ -0,0 +1,238 @@ +--- +title: "Jobs Queue — Schedules" +aliases: [payload-schedules, jobs-schedule, cron-jobs-payload] +tags: [payloadcms, jobs-queue, cron, scheduling, background-jobs] +sources: [raw/jobs-queue__schedules.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +## Overview + +Payload's `schedule` property on tasks/workflows automatically enqueues jobs on a cron cadence. Scheduling ≠ running — two separate mechanisms must both be configured. + +- **Schedule** → adds jobs to DB queue on cron +- **Runner** → picks up queued jobs and executes handlers + +## When to Use What + +| Approach | Use Case | Example | +|---|---|---| +| `schedule` | Recurring automatic tasks | Daily reports, weekly emails, hourly syncs | +| `waitUntil` | One-time future job | Publish post at 3pm, trial expiry email in 7 days | +| Collection Hook | Triggered by document change | Email on publish, PDF on order create | +| Manual queue | Triggered by user/API action | "Generate Report" button | + +## Critical: Two Configs Required + +Both must be present or scheduled jobs will not work: + +### 1. Schedule Config (on task/workflow) + +```ts +export const SendDigestEmail: TaskConfig<'SendDigestEmail'> = { + slug: 'SendDigestEmail', + schedule: [ + { + cron: '0 0 * * *', // Every day at midnight + queue: 'nightly', + }, + ], + handler: async () => { + await sendDigestToAllUsers() + }, +} +``` + +### 2. Runner Config (must match same queue name) + +```sh +# Recommended: bin script (separate process from Next.js) +pnpm payload jobs:run --cron "* * * * *" --queue nightly --handle-schedules +``` + +Or via `autoRun`: + +```ts +jobs: { + autoRun: [{ cron: '* * * * *', queue: 'nightly' }], +} +``` + +**Checklist before debugging:** +- [ ] Task has `schedule` property? +- [ ] Config has `autoRun` or bin script runner? +- [ ] Both use the **same `queue` name**? +- [ ] Not on serverless? (`autoRun` needs long-running server) + +## Schedule Handling Methods + +### Bin Scripts (Recommended for Dedicated Servers) + +```sh +# Combined: schedule + run +pnpm payload jobs:run --cron "*/5 * * * *" --queue myQueue --handle-schedules + +# Schedule-only (separate processes) +pnpm payload jobs:handle-schedules --cron "*/5 * * * *" --queue myQueue +pnpm payload jobs:handle-schedules --cron "*/5 * * * *" --all-queues +``` + +Benefits: separate process, independent scaling, zero impact on Next.js server. + +### autoRun (Dedicated Server Alternative) + +```ts +jobs: { + autoRun: [ + { + cron: '*/5 * * * *', + queue: 'default', + // disableScheduling: false ← default, set true to decouple + }, + ], +} +``` + +### API Endpoint (Serverless) + +```ts +// Handles both scheduling and running +await fetch('/api/payload-jobs/run?queue=myQueue') + +// Schedule only +await fetch('/api/payload-jobs/handle-schedules?queue=myQueue') + +// Disable scheduling in run endpoint +await fetch('/api/payload-jobs/run?queue=myQueue&disableScheduling=true') +``` + +Configure Vercel Cron to call `GET /api/payload-jobs/run` on your desired interval. + +### Local API (Programmatic) + +```ts +await payload.jobs.handleSchedules() +``` + +## ScheduleConfig Type + +```ts +export type ScheduleConfig = { + cron: string // required; supports 5-field and 6-field (seconds) format + queue: string // required; must match runner queue name + hooks?: { + beforeSchedule?: BeforeScheduleFn + afterSchedule?: AfterScheduleFn + } +} +``` + +## Scheduling Lifecycle + +1. **Cron evaluation** — reads `payload-jobs-stats` global to find which schedules are due +2. **beforeSchedule hook** — default: skip if active/runnable job of same type already exists; customizable +3. **Enqueue Job** — creates job with `waitUntil` set to next scheduled time +4. **afterSchedule hook** — default: updates `payload-jobs-stats` with last scheduled time; customizable + +## Custom Concurrency & Dynamic Input + +```ts +import { countRunnableOrActiveJobsForQueue } from 'payload' + +schedule: [ + { + cron: '* * * * *', + queue: 'reports', + hooks: { + beforeSchedule: async ({ queueable, req }) => { + const count = await countRunnableOrActiveJobsForQueue({ + queue: queueable.scheduleConfig.queue, + req, + taskSlug: queueable.taskConfig?.slug, + onlyScheduled: true, + }) + + return { + shouldSchedule: count < 3, // allow up to 3 simultaneous + input: { text: 'Hi there' }, // dynamic input at schedule time + } + }, + }, + }, +] +``` + +## Common Cron Patterns + +```ts +schedule: [{ cron: '* * * * *', queue: 'frequent' }] // every minute +schedule: [{ cron: '*/5 * * * *', queue: 'default' }] // every 5 min +schedule: [{ cron: '0 * * * *', queue: 'hourly' }] // every hour +schedule: [{ cron: '0 0 * * *', queue: 'nightly' }] // midnight daily +schedule: [{ cron: '30 2 * * *', queue: 'nightly' }] // 2:30 AM daily +schedule: [{ cron: '0 9 * * 1', queue: 'weekly' }] // Mon 9:00 AM +schedule: [{ cron: '0 0 1 * *', queue: 'monthly' }] // 1st of month +schedule: [{ cron: '0 8 * * 1-5', queue: 'weekdays' }] // weekdays 8AM +schedule: [{ cron: '*/30 * * * * *', queue: 'frequent' }] // every 30s (6-field) +``` + +**Cron field order:** + +``` +[second] minute hour day-of-month month day-of-week + 0-59 0-23 1-31 1-12 0-7 (0,7=Sun) +``` + +## Serverless Platforms + +`autoRun` does NOT work on serverless (no long-running process). Instead: + +1. Define `schedule` on tasks as normal +2. Configure Vercel Cron (or similar) to hit `GET /api/payload-jobs/run?queue=` on your desired interval +3. Queued jobs execute during that same request (or separately via another runner) + +## Troubleshooting + +**Jobs not being queued:** +- Check `disableScheduling` is not `true` on `autoRun` +- Validate cron expression at crontab.guru (5-field) or test with seconds (6-field) +- Inspect `payload-jobs-stats` global: `await payload.findGlobal({ slug: 'payload-jobs-stats' })` + +**Jobs queued but not running:** +- See [[wiki/payloadcms/jobs-queue-queues|Jobs Queue — Queues & Execution]] troubleshooting + +**Jobs run at wrong times:** +- Ensure `autoRun` checks queue frequently enough (`* * * * *` = every minute) + +**Duplicate jobs (multiple servers):** + +```ts +// Server 1 — handles schedules +jobs: { shouldAutoRun: () => process.env.HANDLE_SCHEDULES === 'true', autoRun: [/*...*/] } + +// Server 2 — only runs jobs, no scheduling +jobs: { shouldAutoRun: () => true, autoRun: [{ disableScheduling: true }] } +``` + +## Key Takeaways + +- **Scheduling and running are separate concerns** — `schedule` enqueues, runner executes; both must be configured +- **Queue names must match** between `schedule` config and runner (`autoRun`/bin script) +- **Bin scripts are preferred** for dedicated servers — isolated process, no Next.js impact +- **Serverless = no autoRun** — use external cron (Vercel Cron) hitting the API endpoint +- **Default deduplication** — `beforeSchedule` hook skips new job if one already exists; override with custom hook for concurrency +- **Dynamic input** is set at schedule time via `beforeSchedule` return value, not at handler runtime +- Use `waitUntil` for one-time future jobs, not `schedule` (which recurs indefinitely) + +## Sources + +- `raw/jobs-queue__schedules.md` +- https://payloadcms.com/docs/jobs-queue/schedules + +## Related + +- [[wiki/payloadcms/jobs-queue|Jobs Queue — Overview]] +- [[wiki/payloadcms/jobs-queue-queues|Jobs Queue — Queues & Execution]] +- [[wiki/payloadcms/jobs-queue-jobs|Jobs Queue — Jobs Detail]] +- [[wiki/payloadcms/jobs-queue-quick-start|Jobs Queue — Quick Start Examples]] diff --git a/wiki/payloadcms/jobs-queue-tasks.md b/wiki/payloadcms/jobs-queue-tasks.md new file mode 100644 index 0000000..93040d9 --- /dev/null +++ b/wiki/payloadcms/jobs-queue-tasks.md @@ -0,0 +1,203 @@ +--- +title: "Jobs Queue — Tasks" +aliases: [payload-tasks, jobs-queue-tasks] +tags: [payloadcms, jobs-queue, tasks, typescript, scheduling] +sources: [raw/jobs-queue__tasks.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +## Overview + +A **Task** is a strongly-typed function declaration registered in the Payload config that performs isolated business logic. Tasks are the atomic unit of the [[wiki/payloadcms/jobs-queue|Jobs Queue]] — Jobs and Workflows call Tasks. + +Key trait: Tasks support **automatic retries**, making them ideal for non-deterministic work (AI/LLM calls, external API calls, payment processing). + +## Defining Tasks + +Tasks live in `jobs.tasks[]` in `buildConfig`: + +```ts +jobs: { + tasks: [ + { + slug: 'createPost', // unique across tasks AND workflows + retries: 2, + inputSchema: [ + { name: 'title', type: 'text', required: true }, + ], + outputSchema: [ + { name: 'postID', type: 'text', required: true }, + ], + handler: async ({ input, job, req }) => { + const newPost = await req.payload.create({ + collection: 'post', + data: { title: input.title }, + }) + return { output: { postID: newPost.id } } + }, + } as TaskConfig<'createPost'>, + ], +} +``` + +## Task Config Options + +| Option | Description | +|--------|-------------| +| `slug` | Unique identifier — must be unique across tasks and workflows | +| `handler` | Function `async ({ input, job, req, tasks, inlineTask }) => { output: {...} }` or absolute file path | +| `inputSchema` | Payload field definitions — Payload generates TypeScript types from these | +| `outputSchema` | Same — generated output types | +| `retries` | Number of retries on failure; `0` = no retry; `undefined` = inherit from workflow | +| `retries.shouldRestore` | Boolean or function — whether to skip re-running a previously-passed task | +| `onFail` | Callback executed after all retries are exhausted | +| `onSuccess` | Callback executed on successful completion | +| `label` | Human-readable display name | +| `interfaceName` | Override generated TS interface name (default: `Task` + capitalized slug) | +| `concurrency` | Exclusive execution key — requires `jobs.enableConcurrencyControl: true` | +| `schedule` | Auto-queue on cron schedule — see [[wiki/payloadcms/jobs-queue-schedules|Job Schedules]] | + +## Scheduling Tasks + +Add `schedule` to auto-queue the task without calling `payload.jobs.queue()` manually: + +```ts +{ + slug: 'dailyDigest', + schedule: [ + { cron: '0 8 * * *', queue: 'daily' }, + ], + handler: async ({ req }) => { + // ... send emails + return { output: { emailsSent: 42 } } + }, +} +``` + +**Critical:** `schedule` only *queues* jobs. You must also configure a runner with the same queue name: + +```ts +autoRun: [ + { cron: '* * * * *', queue: 'daily', limit: 10 }, +] +``` + +Common cron patterns: +```ts +{ cron: '0 * * * *', queue: 'hourly' } // every hour +{ cron: '0 0 * * *', queue: 'nightly' } // midnight +{ cron: '0 9 * * 1', queue: 'weekly' } // Monday 9AM +{ cron: '*/5 * * * *', queue: 'frequent' } // every 5 min +{ cron: '*/3 * * * * *',queue: 'realtime' } // every 3 sec (6-field syntax) +``` + +## Common Patterns + +### Database Operations +```ts +{ slug: 'updateRelatedPosts', retries: 2, + handler: async ({ input, req }) => { + const posts = await req.payload.find({ collection: 'posts', where: { category: { equals: input.categoryId } } }) + for (const post of posts.docs) { + await req.payload.update({ collection: 'posts', id: post.id, data: { categoryUpdatedAt: new Date().toISOString() } }) + } + return { output: { postsUpdated: posts.docs.length } } + } +} +``` + +### External API Calls +```ts +{ slug: 'syncToThirdParty', retries: 3, + handler: async ({ input, req }) => { + const response = await fetch('https://api.example.com/sync', { method: 'POST', body: JSON.stringify(doc) }) + if (!response.ok) throw new Error(`API error: ${response.statusText}`) + return { output: { synced: true, apiResponse: await response.json() } } + } +} +``` + +## Handling Failures + +- **Throw errors** with descriptive messages — stored in `job.error` and visible in Admin UI +- **Prevent retries** by throwing `JobCancelledError`: + ```ts + throw new JobCancelledError('Job was cancelled') + ``` +- **Inspect failures** after the fact: + ```ts + const job = await payload.findByID({ collection: 'payload-jobs', id: jobId }) + if (job.hasError) { + console.log(job.error) + const failedTasks = job.log?.filter(e => e.state === 'failed') + } + ``` + +## Task Execution Lifecycle + +1. Worker picks up the job from the queue +2. Handler executes with provided `input` +3. Success → output stored, job completes +4. Error thrown → task retried (up to `retries` count) +5. All retries exhausted → task and job fail + +> **Idempotency:** Tasks should produce the same result when run multiple times with the same input, since retries cause repeated execution. + +## Task Restoration (`shouldRestore`) + +By default, if a workflow is re-run and a task previously passed, the task is **skipped** and previous output is reused. + +Override with `retries.shouldRestore`: +- `true` (default) — skip if previously passed +- `false` — always re-run, even if previously passed +- `function({ input }) => boolean` — custom logic (e.g. re-run if date has changed) + +```ts +retries: { + shouldRestore: ({ input }) => new Date(input.someDate) > new Date() ? false : true, +} +``` + +## Nested Tasks + +Tasks can call sub-tasks via `tasks` (registered tasks) and `inlineTask` (anonymous functions): + +```ts +handler: async ({ tasks, inlineTask }) => { + await inlineTask('Sub Task 1', { task: () => ({ output: {} }) }) + await tasks.CreateSimple('Sub Task 2', { input: { message: 'hello' } }) + return { output: {} } +} +``` + +Enable `jobs.addParentToTaskLog: true` for better observability when using nested tasks. + +## Handler File Paths (Advanced) + +Pass an absolute path + named export instead of an inline function to avoid bundling large dependencies in Next.js: + +```ts +handler: path.resolve(dirname, 'src/tasks/createPost.ts') + '#createPostHandler' +``` + +> Requires separate transpilation of handler files and a sophisticated build pipeline. Prefer inline functions unless bundle size is a real concern. + +## Key Takeaways + +- Tasks are the atomic unit: one slug, one handler, strongly-typed input/output +- `retries` enables durable AI/external API workflows — always set for non-deterministic operations +- `schedule` + matching `autoRun` queue = periodic auto-execution without manual queuing +- Throw errors for failures; use `JobCancelledError` to stop all retries immediately +- Tasks are idempotent by default via `shouldRestore` — re-runs skip already-passed tasks +- Handler file paths are an escape hatch for large dependencies; avoid unless necessary +- Enable `addParentToTaskLog: true` when using nested tasks + +## Related + +- [[wiki/payloadcms/jobs-queue|Jobs Queue — Overview]] +- [[wiki/payloadcms/jobs-queue-jobs|Jobs Queue — Jobs Detail]] +- [[wiki/payloadcms/jobs-queue-queues|Jobs Queue — Queues & Execution]] +- [[wiki/payloadcms/jobs-queue-schedules|Jobs Queue — Schedules]] +- [[wiki/payloadcms/jobs-queue-quick-start|Jobs Queue — Quick Start Examples]] +- [[wiki/payloadcms/hooks-collections|Collection Hooks]] (triggering tasks from afterChange) diff --git a/wiki/payloadcms/jobs-queue-workflows.md b/wiki/payloadcms/jobs-queue-workflows.md new file mode 100644 index 0000000..fb8edf4 --- /dev/null +++ b/wiki/payloadcms/jobs-queue-workflows.md @@ -0,0 +1,228 @@ +--- +title: "Jobs Queue — Workflows" +aliases: [payload-workflows, jobs-workflows, workflow-retry] +tags: [payloadcms, jobs-queue, workflows, concurrency, retry] +sources: [raw/jobs-queue__workflows.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +## What is a Workflow? + +A **Workflow** combines multiple Tasks into a sequential pipeline with smart failure recovery. If any task fails mid-workflow, the handler re-runs but **already-completed tasks are skipped** — only the failed task and everything after it retries. + +Use a workflow when you have **2+ dependent tasks** and want per-task retry logic. For a single operation, use a plain [[wiki/payloadcms/jobs-queue-tasks|Task]] instead. + +## Defining a Workflow + +Add to `jobs.workflows[]` in your Payload config: + +```ts +{ + slug: 'onboardUser', + inputSchema: [ + { name: 'userId', type: 'text', required: true }, + ], + handler: async ({ job, tasks }) => { + await tasks.createProfile('step-create-profile', { input: { userId: job.input.userId } }) + await tasks.sendWelcomeEmail('step-send-email', { input: { userId: job.input.userId } }) + await tasks.addToMailingList('step-add-list', { input: { userId: job.input.userId } }) + }, +} +``` + +### Workflow Config Options + +| Option | Description | +|--------|-------------| +| `slug` | Unique name (shared namespace with task slugs) | +| `handler` | Async function or file path string | +| `inputSchema` | Field definitions — generates TypeScript types | +| `interfaceName` | Override generated TS interface name (default: `Workflow` + slug) | +| `label` | Human-friendly display name | +| `queue` | Queue name, defaults to `"default"` | +| `retries` | Workflow-level retry cap; `0` = fail on any task error; `undefined` = inherit per-task retries | +| `concurrency` | Prevent parallel execution of jobs with the same key (see [Concurrency Controls](#concurrency-controls)) | + +### Task IDs Must Be Stable + +Each `tasks.someTask(id, ...)` call needs a **stable, unique ID**. On retry, Payload uses this ID to find cached output. If the ID changes between runs, the task re-executes unnecessarily. + +```ts +// Good — stable descriptive IDs +await tasks.sendEmail('send-welcome-email', { input }) + +// Bad — positional numbers work but are hard to debug +await tasks.sendEmail('1', { input }) +``` + +## Inline Tasks + +Run ad-hoc logic without a pre-declared task using `inlineTask`. Useful for one-off steps. + +```ts +handler: async ({ job, tasks, inlineTask }) => { + await tasks.createPost('1', { input: { title: job.input.title } }) + + const { newPost } = await inlineTask('2', { + task: async ({ req }) => { + const newPost = await req.payload.update({ + collection: 'posts', + id: '2', + data: { title: 'updated!' }, + req, + retries: 3, + }) + return { output: { newPost } } + }, + }) +} +``` + +**Drawback:** inline task data stored on `job.taskStatus.inline['2']` is **untyped**. + +## Failure & Recovery + +``` +First run: + step1 → ✅ profile created + step2 → ❌ email service down + step3 → never reached + +Retry: + step1 → skipped (returns cached output) + step2 → ✅ email service recovered + step3 → ✅ added to list +``` + +The entire `handler` re-runs, but completed tasks return their stored result immediately. + +## Accessing Task Outputs + +```ts +handler: async ({ job, tasks }) => { + // Method 1: capture return value + const result = await tasks.createDocument('create-doc', { + input: { title: 'My Document' }, + }) + const docId = result.output.documentId + + // Method 2: read from job.taskStatus + const docId2 = job.taskStatus.createDocument['create-doc'].output.documentId + + await tasks.updateDocument('update-doc', { + input: { documentId: docId, status: 'published' }, + }) +} +``` + +**Task status shape:** +```ts +job.taskStatus[taskSlug][taskId] = { + input: { /* provided input */ }, + output: { /* returned output */ }, + complete: true, + totalTried: 1, +} +``` + +## Best Practices + +### Keep Tasks Small and Focused +One task = one concern with independent retry logic. A monolithic task is all-or-nothing. + +### Pass IDs, Not Objects +```ts +// ✅ Pass ID — task fetches what it needs +await tasks.processUser('process', { input: { userId: '123' } }) + +// ❌ Avoid passing large objects as input +await tasks.processUser('process', { input: { user: { /* entire object */ } } }) +``` + +### Set Appropriate Retry Counts +- **External APIs** (email, payment): 3–5 retries +- **Database operations**: 1–2 retries +- **Idempotent operations**: Higher retries are safe +- **Non-idempotent** (creates, charges, sends): Lower retries to avoid duplicates + +### Handle Errors with Context +```ts +try { + const result = await fetch('https://api.example.com/data') + if (!result.ok) throw new Error(`API ${result.status}: ${result.statusText}`) + return { output: { success: true } } +} catch (error) { + throw new Error(`Failed to sync data for user ${input.userId}: ${error.message}`) +} +``` + +## Concurrency Controls + +Prevent race conditions when multiple jobs operate on the same resource. + +**Enable first:** +```ts +export default buildConfig({ + jobs: { + enableConcurrencyControl: true, // adds indexed concurrencyKey column — may need migration + }, +}) +``` + +**Then add `concurrency` to the workflow:** +```ts +{ + slug: 'syncDocument', + concurrency: ({ input }) => `sync:${input.documentId}`, + handler: async ({ job, inlineTask }) => { /* ... */ } +} +``` + +### Full Concurrency Config + +```ts +concurrency: { + key: ({ input, queue }) => `sync:${input.documentId}`, + exclusive: true, // only one at a time (default: true) + supersedes: false, // delete older pending jobs (default: false) +} +``` + +### Common Patterns + +| Pattern | Config | Use when | +|---------|--------|----------| +| Sequential, all jobs run | `exclusive: true, supersedes: false` | Processing distinct document versions | +| Latest-wins | `exclusive: true, supersedes: true` | Regenerating embeddings/thumbnails after rapid edits | +| Queue-specific | key includes `queue` param | Same resource OK to process concurrently across queues | + +### How Concurrency Works +1. Key computed from input, stored on job document at queue time +2. Job runner excludes jobs whose key is currently running +3. If two same-key jobs are picked up in the same batch, only the first runs; others go back to `processing: false` +4. All jobs eventually complete — they wait their turn + +### Important Notes +- Concurrency is **global across queues** by default — include queue name in key if you want queue-specific behavior +- Only **pending** jobs are deleted by `supersedes`, not running ones +- Jobs without `concurrency` config run in parallel as normal + +## Key Takeaways + +- Workflow = multi-task pipeline with per-task retry; each task resumes from failure point, not from the beginning +- Task IDs must be stable across handler re-runs — use descriptive strings, not positional numbers +- `inlineTask` is convenient but its output is untyped +- Pass IDs not objects to keep job input small and tasks independently fetchable +- `retries: 0` at workflow level disables all task-level retries and fails immediately on any error +- `enableConcurrencyControl: true` requires a DB migration (adds `concurrencyKey` column) +- `supersedes: true` deletes only **pending** jobs — a running job always completes + +## Related + +- [[wiki/payloadcms/jobs-queue-tasks|Jobs Queue — Tasks]] — individual task config and retry logic +- [[wiki/payloadcms/jobs-queue|Jobs Queue — Overview]] — queue concepts, use cases, admin UI +- [[wiki/payloadcms/jobs-queue-jobs|Jobs Queue — Jobs Detail]] — queuing, status schema, cancellation +- [[wiki/payloadcms/jobs-queue-queues|Jobs Queue — Queues & Execution]] — execution methods and scheduling +- [[wiki/payloadcms/jobs-queue-schedules|Jobs Queue — Schedules]] — cron scheduling for tasks +- [[wiki/payloadcms/database-migrations|Database Migrations]] — required when enabling concurrency controls diff --git a/wiki/payloadcms/jobs-queue.md b/wiki/payloadcms/jobs-queue.md index 4e2649c..e6b0be7 100644 --- a/wiki/payloadcms/jobs-queue.md +++ b/wiki/payloadcms/jobs-queue.md @@ -1,348 +1,117 @@ --- -tags: [payloadcms, tech-patterns] -topic: payloadcms -sources: - - https://payloadcms.com/docs/jobs-queue/overview - - https://payloadcms.com/docs/jobs-queue/tasks - - https://payloadcms.com/docs/jobs-queue/jobs - - https://payloadcms.com/docs/jobs-queue/queues - - https://payloadcms.com/docs/jobs-queue/workflows - - https://payloadcms.com/docs/jobs-queue/schedules - - https://payloadcms.com/docs/jobs-queue/quick-start-example +title: "Jobs Queue — Overview" +aliases: [payload-jobs-queue, jobs-queue-overview, payload-background-jobs] +tags: [payloadcms, jobs-queue, background-tasks, scheduling, workers] +sources: [raw/jobs-queue__overview.md] created: 2026-05-15 +updated: 2026-05-15 --- -# PayloadCMS — Jobs Queue +## What It Is -## Overview +Payload's Jobs Queue offloads long-running, expensive, or future-scheduled work from your main API server to separate compute resources. -Payload's Jobs Queue offloads long-running, expensive, or future-scheduled work from your main API to separate compute resources. Core use cases: +## Core Concepts -- **Non-blocking hooks** — queue slow operations (embeddings, third-party API calls, emails) so the API response returns immediately -- **Scheduled actions** — `waitUntil` defers a job to a future timestamp (scheduled publish, trial-expiry email) -- **Periodic operations** — cron-based recurring tasks (nightly syncs, weekly reports, rebuild triggers) -- **Offloading heavy compute** — run large-dependency code in a separate process, keeping Next.js bundle lean (dynamic imports only in worker) +| Concept | Description | +|---------|-------------| +| **Task** | A specific function with business logic (`handler: async () => {}`) | +| **Workflow** | Ordered grouping of tasks; retryable from the exact failure point | +| **Job** | A single instance of a task or workflow queued for execution | +| **Queue** | Named segment that groups jobs (e.g. `emails`, `nightly`) | -All jobs persist in the `payload-jobs` collection until processed. +## Use Cases -See also: [[wiki/payloadcms/configuration|Configuration]], [[wiki/payloadcms/getting-started|Getting Started]] +**Non-blocking hooks** +- Vector embeddings on document change +- Sending data to third-party APIs +- Triggering emails from customer actions ---- +**Scheduled actions** (via `waitUntil`) +- Publish/unpublish posts at a future time +- Trial reminder emails after X days -## Key Concepts (Tasks, Jobs, Queues, Workflows) +**Periodic sync** (via cron) +- Nightly rebuild of frontend +- Off-peak third-party API sync -| Concept | What it is | -|---------|-----------| -| **Task** | A typed function definition (`slug` + `inputSchema` + `outputSchema` + `handler`). The unit of business logic. | -| **Job** | A runtime instance of one Task or one Workflow. Stored in `payload-jobs`. | -| **Queue** | A named group of jobs. Default queue: `"default"`. Used to segment execution cadence (nightly vs every-5-min). | -| **Workflow** | An ordered sequence of Tasks that can be retried from the point of failure, not from scratch. | +**Offloading heavy compute** +- Open-source embedding models (keep out of Next.js bundle via dynamic import) +- PDF generation, headless browser tasks +- Durable multi-step pipelines -**Flow summary:** -1. Define Tasks/Workflows in config -2. Queue Jobs via `payload.jobs.queue()` or via `schedule` property -3. Jobs sit in DB (`payload-jobs`) until a worker picks them up -4. Worker runs jobs via bin script / `autoRun` / API endpoint / Local API +## How Jobs Flow ---- +``` +Define Task → Queue Job → Stored in DB → Run Job → Task executes → Complete/Retry +``` -## Quick Start +Scheduling is **two-part**: defining a `schedule` in config is passive; you must also actively execute `handleSchedules` to queue jobs. -### Example 1 — Manual trigger (hook → queue → run) +## Running Jobs — 4 Options + +| Option | When to Use | +|--------|-------------| +| `pnpm payload jobs:run --cron "* * * * *"` | Dedicated server (recommended — separate process) | +| `autoRun: [{ cron, queue }]` | Dedicated server (runs within Next.js, handles both scheduling + running) | +| `GET /api/payload-jobs/run?queue=emails` | Serverless (triggered by Vercel Cron, GitHub Actions) | +| `payload.jobs.run({ queue })` | Manual / programmatic | + +## Scheduling Jobs — 3 Options + +| Option | When to Use | +|--------|-------------| +| `pnpm payload jobs:handle-schedules --cron "* * * * *"` | Dedicated server bin script | +| `GET /api/payload-jobs/handle-schedules?queue=emails` | Serverless | +| `autoRun` (default — also calls `handleSchedules`) | Dedicated server only | + +## Key Takeaways + +- **Scheduling ≠ Running** — two separate independent steps; the `queue` name must match between them +- **Jobs persist in DB** between queuing and execution — track status, retries, errors there +- **`autoRun` is dual-purpose** by default: it both queues scheduled jobs AND runs queued jobs; set `disableScheduling: true` to run only +- **Never use `autoRun` on serverless** (Vercel, Lambda) — use API endpoints + external cron instead +- **Avoid duplicate scheduling**: don't use `handle-schedules` bin script AND `autoRun` for the same queue simultaneously +- **Dynamic imports** in Task handlers keep large dependencies out of the main Next.js bundle +- **Admin UI visibility**: `payload-jobs` collection is hidden by default — use `jobsCollectionOverrides` to expose for debugging + +## Common Pitfalls + +```ts +// WRONG: schedule defined but never executed +export const EmailTask: TaskConfig = { + slug: 'sendEmail', + schedule: [{ cron: '0 8 * * *', queue: 'emails' }], // passive only! + handler: async () => { /* ... */ }, +} +// Must add: bin script, API endpoint, OR autoRun +``` + +```ts +// WRONG: duplicate scheduling — both handle schedules for same queue +pnpm payload jobs:handle-schedules --cron "* * * * *" +// + autoRun without disableScheduling: true → duplicate jobs queued +``` + +```ts +// FIX: autoRun run-only mode +autoRun: [{ cron: '* * * * *', queue: 'emails', disableScheduling: true }] +``` + +## Expose Jobs in Admin UI ```ts -// payload.config.ts jobs: { - tasks: [ - { - slug: 'sendWelcomeEmail', - retries: 3, - inputSchema: [ - { name: 'userEmail', type: 'email', required: true }, - { name: 'userName', type: 'text', required: true }, - ], - handler: async ({ input, req }) => { - await req.payload.sendEmail({ to: input.userEmail, subject: 'Welcome!', - text: `Hi ${input.userName}` }) - return { output: { emailSent: true } } - }, - }, - ], - autoRun: [{ cron: '*/5 * * * *' }], -} -``` - -Queue from `afterChange` hook: - -```ts -afterChange: [async ({ req, doc, operation }) => { - if (operation === 'create') { - await req.payload.jobs.queue({ - task: 'sendWelcomeEmail', - input: { userEmail: doc.email, userName: doc.name }, - }) - } -}] -``` - -### Example 2 — Scheduled recurring job - -```ts -tasks: [{ - slug: 'generateDailyReport', - schedule: [{ cron: '0 8 * * *', queue: 'reports' }], - handler: async ({ req }) => { - const report = await generateReport() - return { output: { reportId: report.id } } + jobsCollectionOverrides: ({ defaultJobsCollection }) => { + defaultJobsCollection.admin = { hidden: false } + return defaultJobsCollection }, -}], -autoRun: [{ cron: '* * * * *', queue: 'reports', limit: 10 }], -``` - -> **Critical:** `schedule.queue` and `autoRun.queue` (or bin script `--queue`) **must match exactly**. - ---- - -## Task Definition - -```ts -{ - slug: 'createPost', // unique across tasks + workflows - label: 'Create Post', - retries: 2, // 0 = no retry; undefined = inherit from workflow - inputSchema: [ - { name: 'title', type: 'text', required: true }, - ], - outputSchema: [ - { name: 'postID', type: 'text', required: true }, - ], - handler: async ({ input, job, req }) => { - const post = await req.payload.create({ collection: 'post', data: { title: input.title }, req }) - return { output: { postID: post.id } } - }, - onFail: async () => { /* ... */ }, - onSuccess: async () => { /* ... */ }, - schedule: [{ cron: '0 8 * * *', queue: 'daily' }], // optional - concurrency: ({ input }) => `post:${input.title}`, // optional, requires enableConcurrencyControl -} as TaskConfig<'createPost'> -``` - -**Handler file path (advanced):** Pass an absolute path + named export instead of inline function to keep large deps out of the Next.js bundle: - -```ts -handler: path.resolve(dirname, 'src/tasks/createPost.ts') + '#createPostHandler' -``` - -**Task idempotency:** Tasks should be safe to run multiple times — retries may re-execute them. - -**Prevent retry from inside handler:** -```ts -throw new JobCancelledError('Job was cancelled') -``` - -**Restore behavior** (`retries.shouldRestore`): -- `true` (default) — skip re-run if task previously passed; restore cached output -- `false` — always re-run -- `fn({ input }) => boolean` — custom logic - -**Nested tasks** — call sub-tasks from within a handler: -```ts -handler: async ({ tasks, inlineTask }) => { - await inlineTask('Sub Task 1', { task: () => ({ output: {} }) }) - await tasks.CreateSimple('Sub Task 2', { input: { message: 'hello' } }) - return { output: {} } } ``` ---- +## Related Articles -## Queue Config - -### Running jobs — 4 methods - -| Method | Best for | Command / config | -|--------|----------|-----------------| -| **Bin script** | Dedicated server (recommended) | `pnpm payload jobs:run --cron "*/5 * * * *" --queue myQueue` | -| **autoRun** | Dedicated server (alternative) | `autoRun: [{ cron, queue, limit }]` in config | -| **API endpoint** | Serverless (Vercel, Netlify) | `GET /api/payload-jobs/run?queue=myQueue&limit=100` | -| **Local API** | Testing / programmatic | `payload.jobs.run({ queue, limit })` | - -### `autoRun` config options - -```ts -autoRun: [ - { cron: '*/5 * * * *', queue: 'default', limit: 50 }, - { cron: '* * * * *', queue: 'critical', limit: 100 }, -], -shouldAutoRun: async (payload) => process.env.ENABLE_JOB_WORKERS === 'true', -``` - -- `disableScheduling: true` on an `autoRun` entry → only runs jobs, does not handle schedules -- **Never use `autoRun` on serverless** — requires a long-running server - -### Processing order - -Default: FIFO. Override globally or per-queue: - -```ts -processingOrder: { - default: 'createdAt', // FIFO - queues: { nightly: '-createdAt' }, // LIFO -} -// or as a function: -processingOrder: ({ queue }) => queue === 'nightly' ? '-createdAt' : 'createdAt' -``` - -### Vercel Cron (serverless) - -`vercel.json`: -```json -{ "crons": [{ "path": "/api/payload-jobs/run", "schedule": "*/5 * * * *" }] } -``` - -Secure with `CRON_SECRET` env var — Vercel injects it as `Authorization: Bearer `. - -### Expose jobs collection in Admin UI - -```ts -jobsCollectionOverrides: ({ defaultJobsCollection }) => { - defaultJobsCollection.admin = { ...(defaultJobsCollection.admin ?? {}), hidden: false } - return defaultJobsCollection -}, -``` - ---- - -## Workflows - -Workflows combine multiple tasks; on failure they resume from the failed task, not from the beginning. - -```ts -{ - slug: 'onboardUser', - inputSchema: [{ name: 'userId', type: 'text', required: true }], - retries: 3, // falls back to per-task retries if undefined - handler: async ({ job, tasks }) => { - await tasks.createProfile('step1', { input: { userId: job.input.userId } }) - await tasks.sendEmail('step2', { input: { userId: job.input.userId } }) - await tasks.addToList('step3', { input: { userId: job.input.userId } }) - }, -} as WorkflowConfig<'onboardUser'> -``` - -- Task IDs (`'step1'`) must be **stable** across retries — use descriptive strings, not sequential numbers -- Completed tasks return **cached output** on re-runs — no duplicate side effects -- Access prior task output via `job.taskStatus.createProfile['step1'].output.profileId` - -**Inline tasks** (ad-hoc, no pre-definition): -```ts -await inlineTask('2', { task: async ({ req }) => { /* ... */ return { output: {} } } }) -``` - -### Concurrency controls - -Requires `jobs.enableConcurrencyControl: true` (adds indexed field + may need migration). - -```ts -concurrency: { - key: ({ input }) => `sync:${input.documentId}`, - exclusive: true, // one at a time (default true) - supersedes: false, // delete older pending jobs with same key (default false) -} -``` - -`supersedes: true` — use when only the latest state matters (embeddings regeneration, thumbnail rebuilds). - ---- - -## Schedules / Cron - -Scheduling and execution are **two separate steps**: - -1. `schedule` property on task/workflow → queues a job at the given cron interval -2. A runner (`autoRun`, bin script, API endpoint) → actually executes queued jobs - -Both must use the **same `queue` name**. - -```ts -// ScheduleConfig shape: -{ cron: string, queue: string, hooks?: { beforeSchedule?, afterSchedule? } } -``` - -**Common cron patterns:** -``` -0 8 * * * → daily at 8 AM -0 0 * * * → nightly at midnight -0 * * * * → hourly -*/5 * * * * → every 5 min -0 9 * * 1 → every Monday at 9 AM -*/30 * * * * * → every 30 seconds (6-field with seconds) -``` - -**`waitUntil` vs `schedule`:** -- `schedule` → recurring (repeats automatically) -- `waitUntil` → one-time future job - -```ts -await payload.jobs.queue({ task: 'publishPost', input: { postId: '123' }, - waitUntil: new Date('2024-12-25T15:00:00Z') }) -``` - -**Lifecycle:** cron evaluation → `beforeSchedule` hook (default: skip if active job exists) → enqueue with `waitUntil` → `afterSchedule` hook (updates `payload-jobs-stats` global). - ---- - -## Running the Worker - -### Dedicated server (bin script — recommended) - -```sh -# Run + handle schedules (single command for both): -pnpm payload jobs:run --cron "*/5 * * * *" --queue default --handle-schedules - -# Separate processes (schedule vs run): -pnpm payload jobs:handle-schedules --cron "* * * * *" --queue nightly -pnpm payload jobs:run --cron "* * * * *" --queue nightly -``` - -**docker-compose pattern:** -```yaml -services: - nextjs: { command: pnpm start } - worker: { command: pnpm payload jobs:run --cron "*/5 * * * *" --queue default } - nightly: { command: pnpm payload jobs:run --cron "* * * * *" --queue nightly --handle-schedules } -``` - -### Cancelling jobs - -```ts -await payload.jobs.cancelByID({ id: jobId }) -await payload.jobs.cancel({ where: { workflowSlug: { equals: 'createPost' } } }) -``` - -### Job status fields - -```ts -{ completedAt, hasError, totalTried, processing, taskStatus, log, waitUntil } -``` - ---- - -## Gotchas - -- **Schedule ≠ execution.** Defining `schedule` on a task does nothing without a runner. You need BOTH. -- **Queue name mismatch.** If `schedule.queue: 'reports'` but `autoRun.queue: 'default'` — jobs queue but never run. -- **Duplicate scheduling.** Don't run BOTH `jobs:handle-schedules` bin script AND `autoRun` (with `disableScheduling: false`) for the same queue — causes duplicate jobs. -- **`autoRun` on serverless.** Never. Use API endpoint + Vercel Cron instead. -- **HMR in dev.** Hot Module Reload breaks cron schedules; restart dev server after changing job config. -- **Multi-server scheduling.** Multiple servers running `autoRun` without coordination will queue duplicate scheduled jobs. Use `HANDLE_SCHEDULES` env var to designate one scheduler. -- **Non-idempotent tasks + retries.** Retries re-run the handler — ensure side effects (charges, sends, creates) are safe to repeat or use `retries: 0`. -- **Concurrency requires migration.** `enableConcurrencyControl: true` adds a DB field — run migrations before deploying. - ---- - -## Related - -- [[wiki/payloadcms/configuration|Configuration]] -- [[wiki/payloadcms/getting-started|Getting Started]] -- [[wiki/payloadcms/database|Database & Migrations]] +- [[wiki/payloadcms/jobs-queue-jobs|Jobs Queue — Jobs Detail]] — queuing from hooks, `waitUntil`, job status schema, access control, cancellation +- [[wiki/payloadcms/hooks|PayloadCMS — Hooks]] — how to queue jobs from afterChange/afterOperation hooks +- [[wiki/payloadcms/configuration|Payload Config — Overview]] — `autoRun` and bin script config in `buildConfig` +- [[wiki/payloadcms/production|Production & Performance]] — deployment patterns for dedicated vs serverless workers diff --git a/wiki/payloadcms/live-preview-client.md b/wiki/payloadcms/live-preview-client.md new file mode 100644 index 0000000..31523dd --- /dev/null +++ b/wiki/payloadcms/live-preview-client.md @@ -0,0 +1,177 @@ +--- +title: "Live Preview — Client-Side Integration" +aliases: [live-preview-client, useLivePreview, live-preview-react, live-preview-vue] +tags: [payloadcms, live-preview, react, vue, frontend] +sources: [raw/live-preview__client.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +# Live Preview — Client-Side Integration + +> For apps using **React Server Components** (Next.js App Router), use [[wiki/payloadcms/live-preview|server-side Live Preview]] instead — it's simpler and more performant. Client-side is for Pages Router, SPAs, Vue, Nuxt. + +The Admin Panel emits a `window.postMessage` event on every form change. The client subscribes, merges incoming data with initial data (including relationship population), and updates state. + +## Hook Args (all frameworks) + +| Arg | Required | Description | +|-----|----------|-------------| +| `serverURL` | ✓ | URL of your Payload server | +| `initialData` | | Initial document data; live data merges on top | +| `depth` | | Relationship depth to populate. **Must match your initial fetch depth exactly.** Default: `0` | +| `apiRoute` | | Override if `routes.api` is non-default. Default: `/api` | + +## Hook Return Values + +| Value | Description | +|-------|-------------| +| `data` | Live document data, merged with `initialData` | +| `isLoading` | `true` until first message received from Admin Panel | + +## React + +Install: +```bash +npm install @payloadcms/live-preview-react +``` + +```tsx +'use client' +import { useLivePreview } from '@payloadcms/live-preview-react' +import { Page as PageType } from '@/payload-types' + +export const PageClient: React.FC<{ page: PageType }> = ({ page: initialPage }) => { + const { data, isLoading } = useLivePreview({ + initialData: initialPage, + serverURL: PAYLOAD_SERVER_URL, + depth: 2, // must match initial fetch depth + }) + return

{data.title}

+} +``` + +- Use only in `'use client'` components — **not** in React Server Components +- Pass fetched page data from parent server component as `initialPage` prop + +## Vue 3 / Nuxt 3 + +Install: +```bash +npm install @payloadcms/live-preview-vue +``` + +```vue + + +``` + +## Building Your Own Hook + +Install the base package: +```bash +npm install @payloadcms/live-preview +``` + +### API + +| Function | Description | +|----------|-------------| +| `subscribe({ callback, serverURL, initialData, depth })` | Subscribes to Admin Panel `postMessage` events; calls `callback(mergedData)` on each change | +| `unsubscribe(subscription)` | Cleans up the subscription | +| `ready({ serverURL })` | Signals Admin Panel that frontend is ready to receive messages | +| `isLivePreviewEvent(event)` | Checks if a `MessageEvent` comes from the Admin Panel (debounced form state) | + +### Full React Hook Implementation + +```tsx +import { subscribe, unsubscribe, ready } from '@payloadcms/live-preview' +import { useCallback, useEffect, useState, useRef } from 'react' + +export const useLivePreview = (props: { + depth?: number + initialData: T + serverURL: string +}): { data: T; isLoading: boolean } => { + const { depth = 0, initialData, serverURL } = props + const [data, setData] = useState(initialData) + const [isLoading, setIsLoading] = useState(true) + const hasSentReadyMessage = useRef(false) + + const onChange = useCallback((mergedData) => { + setData(mergedData) + setIsLoading(false) + }, []) + + useEffect(() => { + const subscription = subscribe({ callback: onChange, depth, initialData, serverURL }) + + if (!hasSentReadyMessage.current) { + hasSentReadyMessage.current = true + ready({ serverURL }) + } + + return () => { unsubscribe(subscription) } + }, [serverURL, onChange, depth, initialData]) + + return { data, isLoading } +} +``` + +Key responsibilities of a custom hook: +1. Call `subscribe()` — handles postMessage listening, data merging, relationship population, callback invocation +2. Call `ready()` once — tells Admin Panel the iframe frontend is listening +3. Call `unsubscribe()` on unmount — prevents memory leaks + +## Gotchas + +### Relationships/uploads not populating +- Frontend and Payload on different domains → configure `cors` **and** `csrf` in `payload.config.ts`: + ```ts + cors: ['http://localhost:3001'], + csrf: ['http://localhost:3001'], + ``` +- Applies to different ports and subdomains too + +### Relationships disappear after editing +- `depth` in `useLivePreview` must **exactly match** the `depth` in your initial `payload.find()` call + ```tsx + // Initial fetch: + await payload.find({ collection: 'pages', depth: 1, where: { slug: { equals: 'home' } } }) + // Hook: + useLivePreview({ initialData, serverURL, depth: 1 }) // same depth! + ``` + +### Iframe refuses to connect (CSP) +- Add to your frontend's Content Security Policy: + ``` + frame-ancestors: "self" localhost:* https://your-payload-admin.com; + ``` + +### Conditional / fragile UI +- Guard optional chain access: `data?.relatedPosts?.[0]?.title` — not `data.relatedPosts[0].title` +- Fields removed in Admin will result in `undefined` values; use optional chaining or defaults + +## Key Takeaways + +- **Client-side** = real-time updates on every keystroke via `window.postMessage`; use for Pages Router, SPAs, Vue/Nuxt +- **Server-side** = refreshes on save/autosave; use for Next.js App Router RSC (preferred when possible) +- `depth` in the hook must **exactly match** the initial data fetch depth — mismatch silently drops relationships +- Custom hooks must: subscribe → send `ready()` → unsubscribe on unmount +- Cross-domain setups require both `cors` and `csrf` config in Payload +- CSP `frame-ancestors` must whitelist the Admin Panel domain + +## Related + +- [[wiki/payloadcms/live-preview|Live Preview Overview]] — Admin Panel config, breakpoints, dynamic URL, server-side setup +- [[wiki/payloadcms/queries|Queries & Depth]] — `depth` parameter controls relationship auto-population +- [[wiki/payloadcms/authentication-cookies|Authentication Cookies & CSRF]] — cross-domain cookie/CSRF config +- [[wiki/payloadcms/configuration|Payload Config]] — `cors` option for cross-domain requests diff --git a/wiki/payloadcms/live-preview-frontend.md b/wiki/payloadcms/live-preview-frontend.md new file mode 100644 index 0000000..5c7ded4 --- /dev/null +++ b/wiki/payloadcms/live-preview-frontend.md @@ -0,0 +1,61 @@ +--- +title: "Live Preview — Frontend Integration Guide" +aliases: [live-preview-frontend, payload-live-preview-frontend] +tags: [payloadcms, live-preview, react, next.js, vue, nuxt] +sources: [raw/live-preview__frontend.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +# Live Preview — Frontend Integration Guide + +Entry point for adding Payload Live Preview to your frontend app. Pick an approach based on Server Component support. + +## Decision Tree + +``` +Does your framework support Server Components? +├── Yes (Next.js App Router, etc.) +│ └── → Use Server-Side Live Preview (recommended) +└── No (Pages Router, Vue, Nuxt, SPA) + └── → Use Client-Side Live Preview +``` + +## Server-Side (Recommended) + +- **Package:** `@payloadcms/live-preview-react` +- **How it works:** Admin Panel sends a `postMessage` on save/autosave → frontend calls `router.refresh()` → Server Components re-fetch from Payload with latest draft +- **Simpler setup** — no hook in every page component +- **More performant** — only refreshes on save, not every keystroke +- See [[wiki/payloadcms/live-preview|Live Preview — Full Reference]] for the `RefreshRouteOnSave` pattern + +## Client-Side + +- **Packages:** `@payloadcms/live-preview-react` (React), `@payloadcms/live-preview-vue` (Vue/Nuxt), `@payloadcms/live-preview` (framework-agnostic) +- **How it works:** `useLivePreview` hook subscribes to `postMessage`, merges incoming data into local state on every keystroke +- **Gotcha:** `depth` in the hook must match the depth used in the initial server fetch +- See [[wiki/payloadcms/live-preview-client|Live Preview — Client-Side]] for the full `useLivePreview` API + +## Package Summary + +| Scenario | Package | +|---|---| +| React (any) | `@payloadcms/live-preview-react` | +| Vue 3 / Nuxt 3 | `@payloadcms/live-preview-vue` | +| Custom / framework-agnostic | `@payloadcms/live-preview` | + +## Key Takeaways + +- **Prefer server-side** if you're on Next.js App Router — it's simpler and more performant +- Client-side works for any JS framework via `useLivePreview` hook +- Both approaches rely on `window.postMessage` from the Admin Panel iframe +- Admin Panel config (`admin.livePreview`) is shared — the frontend pick is independent of the Payload config + +## Related + +- [[wiki/payloadcms/live-preview|Live Preview — Full Reference]] — Payload config, breakpoints, dynamic URL, all gotchas +- [[wiki/payloadcms/live-preview-client|Live Preview — Client-Side]] — `useLivePreview` hook API and base package + +## Sources + +- `raw/live-preview__frontend.md` — Payload CMS docs: "Implementing Live Preview in your frontend" diff --git a/wiki/payloadcms/live-preview-server.md b/wiki/payloadcms/live-preview-server.md new file mode 100644 index 0000000..68eba1a --- /dev/null +++ b/wiki/payloadcms/live-preview-server.md @@ -0,0 +1,150 @@ +--- +title: "Live Preview — Server-Side" +aliases: [server-live-preview, RefreshRouteOnSave, server-components-live-preview] +tags: [payloadcms, live-preview, nextjs, react-server-components, drafts] +sources: [raw/live-preview__server.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +## Overview + +Server-side Live Preview is for frameworks that support **React Server Components** (e.g. Next.js App Router). It works differently from [[wiki/payloadcms/live-preview-client|client-side Live Preview]]: instead of streaming form-state diffs, it triggers a full server roundtrip on every document save (draft save, autosave, or publish). + +**Flow:** Admin Panel emits `window.postMessage` → front-end catches it → calls `router.refresh()` → Next.js re-fetches fresh data from the [[wiki/payloadcms/local-api|Local API]]. + +> **Not applicable to:** Next.js Pages Router, React Router, Vue 3, Nuxt.js, Svelte (use [[wiki/payloadcms/live-preview-client|client-side]] instead). + +--- + +## React / Next.js App Router + +### Install + +```bash +npm install @payloadcms/live-preview-react +``` + +### Usage pattern + +Two files: a server page component + a thin `'use client'` wrapper. + +**`page.tsx`** (Server Component): +```tsx +import { RefreshRouteOnSave } from './RefreshRouteOnSave' +import { getPayload } from 'payload' +import config from '../payload.config' + +export default async function Page() { + const payload = await getPayload({ config }) + const page = await payload.findByID({ + collection: 'pages', + id: '123', + draft: true, + }) + + return ( + <> + +

{page.title}

+ + ) +} +``` + +**`RefreshRouteOnSave.tsx`** (Client Component): +```tsx +'use client' +import { RefreshRouteOnSave as PayloadLivePreview } from '@payloadcms/live-preview-react' +import { useRouter } from 'next/navigation.js' + +export const RefreshRouteOnSave = () => { + const router = useRouter() + return ( + router.refresh()} + serverURL={process.env.NEXT_PUBLIC_PAYLOAD_URL} + /> + ) +} +``` + +The `RefreshRouteOnSave` component must be a Client Component because it subscribes to `window.postMessage`. + +--- + +## Building Your Own Refresh Component + +Install the base package (framework-agnostic): + +```bash +npm install @payloadcms/live-preview +``` + +| Export | Purpose | +|--------|---------| +| `ready({ serverURL })` | Tells Admin Panel the front-end is ready to receive messages | +| `isDocumentEvent(event, serverURL)` | Checks if a `MessageEvent` is a document-level event from the Admin Panel | + +### Minimal implementation + +```tsx +import { isDocumentEvent, ready } from '@payloadcms/live-preview' +import { useCallback, useEffect, useRef } from 'react' + +export const RefreshRouteOnSave = ({ refresh, serverURL }) => { + const hasSentReadyMessage = useRef(false) + + const onMessage = useCallback((event) => { + if (isDocumentEvent(event, serverURL)) refresh() + }, [refresh, serverURL]) + + useEffect(() => { + window.addEventListener('message', onMessage) + if (!hasSentReadyMessage.current) { + hasSentReadyMessage.current = true + ready({ serverURL }) + } + return () => window.removeEventListener('message', onMessage) + }, [serverURL, onMessage]) + + return null +} +``` + +--- + +## Key Takeaways + +- **Server-side only for RSC frameworks** — requires `router.refresh()` (Next.js App Router pattern); not for SPA-style routing +- **Roundtrip on save** — less real-time than client-side (`useLivePreview`); updates appear only after a document save event +- **Enable Autosave** — set `versions.drafts.autosave.interval: 375` to make server-side feel responsive +- **Split Client/Server** — `RefreshRouteOnSave` must be `'use client'`; the page component stays a Server Component +- **Base package** — `@payloadcms/live-preview` provides `ready` + `isDocumentEvent` for custom non-React implementations +- **CSP gotcha** — if your front-end sets `Content-Security-Policy`, add `frame-ancestors: "self" localhost:* https://your-site.com` to allow the Admin Panel iframe + +--- + +## Troubleshooting + +| Symptom | Fix | +|---------|-----| +| Updates lag compared to client-side | Enable Autosave with low interval (`375ms`) | +| Iframe refuses to connect | Add Admin Panel domain to CSP `frame-ancestors` directive | + +--- + +## Related + +- [[wiki/payloadcms/live-preview|Live Preview — Overview]] — iframe config, breakpoints, mode selection +- [[wiki/payloadcms/live-preview-client|Live Preview — Client-Side]] — `useLivePreview` hook for SPAs +- [[wiki/payloadcms/live-preview-frontend|Live Preview — Frontend Integration Guide]] — server vs client decision guide +- [[wiki/payloadcms/local-api|Local API]] — what `router.refresh()` hits under the hood +- [[wiki/payloadcms/versions-autosave|Versions & Autosave]] — enable for responsive server-side preview + +--- + +## Sources + +- `raw/live-preview__server.md` +- https://payloadcms.com/docs/live-preview/server diff --git a/wiki/payloadcms/live-preview.md b/wiki/payloadcms/live-preview.md index 04ae2f6..cfb8eba 100644 --- a/wiki/payloadcms/live-preview.md +++ b/wiki/payloadcms/live-preview.md @@ -1,12 +1,15 @@ --- -tags: [payloadcms, tech-patterns] -topic: payloadcms +title: "Live Preview — Overview" +aliases: [live-preview, payload-live-preview, visual-editing] +tags: [payloadcms, tech-patterns, live-preview] sources: + - raw/live-preview__overview.md - https://payloadcms.com/docs/live-preview/overview - https://payloadcms.com/docs/live-preview/server - https://payloadcms.com/docs/live-preview/client - https://payloadcms.com/docs/live-preview/frontend created: 2026-05-15 +updated: 2026-05-15 --- # PayloadCMS — Live Preview @@ -56,9 +59,27 @@ url: ({ data, collectionConfig, locale, req }) => `${data.tenant.url}/${data.slug}${locale ? `?locale=${locale.code}` : ''}` ``` -- Args: `data`, `locale`, `collectionConfig`, `globalConfig`, `req` +**URL function args:** + +| Arg | Description | +|-----|-------------| +| `data` | Document data including unsaved changes | +| `locale` | Current locale being edited (if localization enabled) | +| `collectionConfig` | Collection admin config of the document | +| `globalConfig` | Global admin config of the document | +| `req` | Payload Request object (use for fully-qualified URLs: `${req.protocol}//${req.host}/${data.slug}`) | + - Return `undefined` or `null` to conditionally hide Live Preview (acts as access control) - Return a relative path — Payload auto-injects protocol/host from the browser window +- Useful for unknown preview URLs at build time (e.g. Vercel preview deployments) + +#### Conditional Rendering + +Return `null`/`undefined` to hide the Live Preview button for certain users or documents: + +```ts +url: ({ req }) => (req.user?.role === 'admin' ? '/hello-world' : null) +``` ### Breakpoints @@ -69,9 +90,18 @@ breakpoints: [ ] ``` -- "Responsive" (100% width/height) is always available and is the default -- Toolbar also has manual width/height inputs ("Custom" mode) -- "Open in new window" button lets you freely resize outside the iframe +**Breakpoint options** (* = required): + +| Option | Description | +|--------|-------------| +| `label` * | Label shown in the toolbar dropdown | +| `name` * | Internal identifier | +| `width` * | iframe width in px | +| `height` * | iframe height in px | + +- **"Responsive"** (100% width/height) is always available and is the default — no config needed +- Toolbar has manual width/height inputs that temporarily switch to "Custom" mode +- **"Open in new window"** button closes the iframe and opens a resizable browser window; closing it re-opens the iframe ## Code Examples @@ -177,7 +207,20 @@ For server-side custom: use `ready` + `isDocumentEvent` from the same package to - **RSC + client hook** — do not use `useLivePreview` in Server Components; wrap in a `'use client'` component - **Conditional rendering** — returning `null` from `url` function hides the Live Preview button; use it for role-based access +## Key Takeaways + +- Live Preview renders your frontend in an **iframe** inside the Admin Panel; no save required for client-side mode +- Uses `window.postMessage` for real-time sync between Admin Panel and frontend +- `url` can be a **dynamic function** — use it for multi-tenant, localization, unknown preview URLs, or conditional access +- Return `null`/`undefined` from `url` to hide Live Preview for specific users or documents (role-based access control) +- **Depth mismatch** in `useLivePreview` vs initial fetch is the most common bug — always match them +- Server-side mode (Next.js App Router) = simpler, but fires on save; client-side = fires on every keystroke +- "Responsive" breakpoint is always available; "Open in new window" for free resize + ## Related -- [[wiki/payloadcms/versions|Versions & Drafts]] — autosave + drafts make server-side Live Preview responsive -- [[wiki/payloadcms/configuration|Configuration]] — `admin.livePreview` lives in the root Payload config +- [[wiki/payloadcms/live-preview-client|Live Preview — Client-Side]] — `useLivePreview` hook, Vue, framework-agnostic API +- [[wiki/payloadcms/live-preview-frontend|Live Preview — Frontend Integration Guide]] — server-side vs client-side decision guide +- [[wiki/payloadcms/document-views|Document Views]] — `livePreview` document view key and tab config +- [[wiki/payloadcms/configuration|Payload Config — Overview]] — `admin.livePreview` root config +- [[wiki/payloadcms/localization|Localization]] — `locale` arg in dynamic URL function diff --git a/wiki/payloadcms/local-api-access-control.md b/wiki/payloadcms/local-api-access-control.md new file mode 100644 index 0000000..9c9a711 --- /dev/null +++ b/wiki/payloadcms/local-api-access-control.md @@ -0,0 +1,65 @@ +--- +title: "Local API — Access Control" +aliases: [local-api-access-control, payload-overrideAccess] +tags: [payloadcms, local-api, access-control, security, server-side] +sources: [raw/local-api__access-control.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +## Overview + +Local API operations **skip access control by default**. This is by design — server-side scripts and admin tasks often run without a user context. You must explicitly opt in to access control enforcement when it matters. + +## Default Behaviour: Access Control Skipped + +```ts +// Access control is NOT checked — runs as a privileged operation +const doc = await payload.create({ + collection: 'users', + data: { email: 'test@test.com', password: 'test' }, +}) +``` + +Useful for: +- Seed scripts and migrations +- Admin-only server jobs (cron tasks, background workers) +- Internal service-to-service operations + +## Enforcing Access Control + +Pass `overrideAccess: false` **and** the authenticated `user` object to make the operation respect collection-level access rules. + +```ts +const doc = await payload.create({ + collection: 'users', + overrideAccess: false, // enforce collection access control + user, // authenticated user to check against + data: { email: 'test@test.com', password: 'test' }, +}) +``` + +- If `user` lacks permission → Payload throws a `Forbidden` error +- Works on all CRUD operations: `create`, `find`, `findByID`, `update`, `delete` + +## When to Use Each Mode + +| Scenario | `overrideAccess` | Pass `user`? | +|----------|-----------------|--------------| +| Server scripts, cron jobs, seeds | `true` (default) | No | +| Server actions executing on behalf of a logged-in user | `false` | Yes — `req.user` | +| Webhooks / third-party integrations with trusted tokens | `true` (default) | No | +| Custom API endpoints that mirror REST behaviour | `false` | Yes | + +## Key Takeaways + +- Local API **bypasses access control by default** — safe for trusted server code, dangerous if exposed to untrusted input. +- To enforce permissions: set `overrideAccess: false` AND pass the `user` object. +- Both conditions are required — `overrideAccess: false` without a `user` will treat the request as unauthenticated (all collection-access functions receive `undefined` as user). +- In [[wiki/payloadcms/hooks-collections|Collection Hooks]] that run after user-triggered events, access control is already enforced by the REST/GraphQL layer; using the Local API inside hooks with `overrideAccess: true` lets you escalate privileges intentionally. +- See [[wiki/payloadcms/access-control|Access Control Overview]] for collection/field-level access function signatures. + +## Sources + +- `raw/local-api__access-control.md` +- https://payloadcms.com/docs/local-api/access-control diff --git a/wiki/payloadcms/local-api-outside-nextjs.md b/wiki/payloadcms/local-api-outside-nextjs.md new file mode 100644 index 0000000..02e02cc --- /dev/null +++ b/wiki/payloadcms/local-api-outside-nextjs.md @@ -0,0 +1,81 @@ +--- +title: "Payload Local API — Outside Next.js" +aliases: [payload-standalone, payload-scripts, payload-outside-nextjs] +tags: [payloadcms, local-api, standalone, scripts, esm] +sources: [raw/local-api__outside-nextjs.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +## Overview + +Payload can run entirely outside of Next.js — useful for standalone scripts (seeding, migrations, one-off ops) or in other frontend frameworks (SvelteKit, Remix, Nuxt). + +**Requirement:** Payload is fully ESM. Scripts must be ESM format or use dynamic imports. + +## Running Standalone Scripts + +Use `getPayload({ config })` to get an initialized Payload instance: + +```ts +import { getPayload } from 'payload' +import config from '@payload-config' + +const seed = async () => { + const payload = await getPayload({ config }) + + await payload.create({ + collection: 'users', + data: { email: 'dev@payloadcms.com', password: 'some-password' }, + }) +} + +await seed() +``` + +Execute with: + +```sh +payload run src/seed.ts +``` + +## `payload run` — What It Does + +1. **Loads env vars** exactly like Next.js does — no need for `dotenv` (using `dotenv` directly can cause env var mismatches) +2. **Initializes tsx** — runs TypeScript directly without manually installing `ts-node`/`tsx` + +## Troubleshooting Import Errors + +### Option 1 — SWC mode (faster, but may break some imports) + +```sh +payload run src/seed.ts --use-swc +``` + +Requires `@swc-node/register` installed in the project. + +### Option 2 — Alternative runtime (e.g. Bun) + +```sh +bunx --bun payload run src/seed.ts --disable-transpile +``` + +`--disable-transpile` disables Payload's own transpilation; Bun handles it natively. + +## Key Takeaways + +- `getPayload({ config })` works anywhere Node/Bun runs — not just inside Next.js +- Always use `payload run` for scripts; avoids `dotenv` vs Next.js env loading conflicts +- Default transpiler is **tsx**; switch to `--use-swc` for speed or `--disable-transpile` for Bun +- Payload is fully ESM — `require()` / CommonJS scripts won't work +- The same [[wiki/payloadcms/local-api|Local API]] methods (`create`, `find`, `update`, `delete`) are available in standalone scripts + +## Related + +- [[wiki/payloadcms/local-api|Local API — full reference]] +- [[wiki/payloadcms/queries|Queries — where, depth, pagination]] + +## Sources + +- `raw/local-api__outside-nextjs.md` +- https://payloadcms.com/docs/local-api/outside-nextjs diff --git a/wiki/payloadcms/local-api-server-functions.md b/wiki/payloadcms/local-api-server-functions.md new file mode 100644 index 0000000..4cbf29d --- /dev/null +++ b/wiki/payloadcms/local-api-server-functions.md @@ -0,0 +1,183 @@ +--- +title: "Local API — Server Functions" +aliases: [server-functions, server-actions-payload, payload-server-actions] +tags: [payloadcms, local-api, server-functions, next-js, authentication] +sources: [raw/local-api__server-functions.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +## Overview + +**Server functions** (formerly "server actions") are Next.js-native async functions marked with `'use server'` that run exclusively on the server. They bridge the gap between client components and Payload's [[wiki/payloadcms/local-api|Local API]], which cannot be called directly from client-side code. + +## Why Use Server Functions + +- **Local API is server-only** — client components can't call `payload.*` directly; server functions act as the secure bridge +- **No extra endpoints needed** — avoids creating REST/GraphQL routes just for simple CRUD operations +- **Security** — restricts exposed operations to only what the function explicitly allows +- **Performance** — Next.js optimizes caching, DB queries, and network overhead automatically + +## Basic Pattern + +```ts +// server/actions.ts +'use server' + +import { getPayload } from 'payload' +import config from '@payload-config' + +export async function createPost(data) { + const payload = await getPayload({ config }) + try { + return await payload.create({ collection: 'posts', data }) + } catch (error) { + throw new Error(`Error creating post: ${error.message}`) + } +} +``` + +```ts +// components/PostForm.tsx +'use client' +import { createPost } from '../server/actions' + +// call like any async function from onClick/onSubmit +const newPost = await createPost({ title: 'Hello' }) +``` + +## Common Operations + +### Create Document + +```ts +'use server' +export async function createPost(data) { + const payload = await getPayload({ config }) + return await payload.create({ collection: 'posts', data }) +} +``` + +### Update Document + +```ts +'use server' +export async function updatePost(id, data) { + const payload = await getPayload({ config }) + return await payload.update({ collection: 'posts', id, data }) +} +``` + +### Authenticate User (check session) + +```ts +'use server' +import { headers as getHeaders } from 'next/headers' + +export const authenticateUser = async () => { + const payload = await getPayload({ config }) + const headers = await getHeaders() + const { user } = await payload.auth({ headers }) + return user ? { hello: user.email } : { hello: 'Not authenticated' } +} +``` + +### File Upload + +Pass the file as a second argument and merge it into the data: + +```ts +'use server' +export async function createPostWithUpload(data, upload) { + const payload = await getPayload({ config }) + return await payload.create({ + collection: 'posts', + data: { ...data, media: upload }, + }) +} +``` + +## Built-in Auth Server Functions + +`@payloadcms/next/auth` exports ready-made server functions for auth operations that manage cookies automatically — no need to handle tokens manually. + +| Function | Import | Purpose | +|----------|--------|---------| +| `login` | `@payloadcms/next/auth` | Verify credentials, set auth cookie | +| `logout` | `@payloadcms/next/auth` | Clear auth cookie + sessions | +| `refresh` | `@payloadcms/next/auth` | Refresh token + session | + +**Pattern** — wrap each in a thin `'use server'` helper (config can't be imported in client components): + +```ts +'use server' +import { login } from '@payloadcms/next/auth' +import config from '@payload-config' + +export async function loginAction({ email, password }: { email: string; password: string }) { + return await login({ collection: 'users', config, email, password }) +} +``` + +```ts +'use server' +import { logout } from '@payloadcms/next/auth' +import config from '@payload-config' + +export async function logoutAction() { + return await logout({ allSessions: true, config }) +} +``` + +## Error Handling + +- Always wrap Local API calls in `try/catch` +- Return structured error objects — don't expose raw errors to the frontend +- Log server-side for debugging + +```ts +export async function createPost(data) { + try { + const payload = await getPayload({ config }) + return await payload.create({ collection: 'posts', data }) + } catch (error) { + console.error('Error creating post:', error) + return { error: 'Failed to create post' } + } +} +``` + +## Security + +- **Access control**: Check `user.role` before sensitive operations +- **Avoid leaking data**: Never return passwords, tokens, or sensitive fields +- Use Payload's `UnauthorizedError` for clean rejection: + +```ts +import { UnauthorizedError } from 'payload' + +export async function deletePost(postId, user) { + if (!user || user.role !== 'admin') throw new UnauthorizedError() + const payload = await getPayload({ config }) + return await payload.delete({ collection: 'posts', id: postId }) +} +``` + +## Key Takeaways + +- `'use server'` at the top of the file (or function) marks it as a server function +- `getPayload({ config })` is the entry point — import `config` from `@payload-config` +- All [[wiki/payloadcms/local-api|Local API]] operations (`create`, `update`, `delete`, `find`, `auth`) work inside server functions +- Built-in auth helpers (`login`/`logout`/`refresh`) from `@payloadcms/next/auth` handle cookie complexity +- Server functions replace the need for custom [[wiki/payloadcms/rest-api|REST API]] endpoints for internal operations +- Always return the result — don't just run the operation and discard +- Use `overrideAccess: false` + pass `user` if you want [[wiki/payloadcms/local-api-access-control|access control]] enforced +- For file uploads, pass the `File` object as a separate argument and merge into data + +## Related + +- [[wiki/payloadcms/local-api|Local API — Overview]] +- [[wiki/payloadcms/local-api-access-control|Local API — Access Control]] +- [[wiki/payloadcms/local-api-outside-nextjs|Local API — Outside Next.js]] +- [[wiki/payloadcms/authentication-operations|Authentication — Operations]] +- [[wiki/payloadcms/rest-api|REST API]] diff --git a/wiki/payloadcms/local-api.md b/wiki/payloadcms/local-api.md index 24b2612..3d4d63f 100644 --- a/wiki/payloadcms/local-api.md +++ b/wiki/payloadcms/local-api.md @@ -1,12 +1,15 @@ --- -tags: [payloadcms, tech-patterns] -topic: payloadcms +title: "PayloadCMS — Local API" +aliases: [local-api, payload-local-api, payload-server-api] +tags: [payloadcms, local-api, server-side, nextjs, typescript] sources: + - raw/local-api__overview.md - https://payloadcms.com/docs/local-api/overview - https://payloadcms.com/docs/local-api/outside-nextjs - https://payloadcms.com/docs/local-api/server-functions - https://payloadcms.com/docs/local-api/access-control created: 2026-05-15 +updated: 2026-05-15 --- # PayloadCMS — Local API @@ -241,8 +244,25 @@ const post = await payload.find({ collection: 'posts', req }) - For file uploads in server functions, merge the `File` object into `data` before calling `payload.create()` - `beforeRead`/`afterRead` hooks may not receive full doc when `select` is active — use entity-level `select` config to force fields +## Key Takeaways + +- **No HTTP overhead** — Local API hits the DB directly; ideal for RSC, seeds, hooks, custom route handlers +- **Access control off by default** (`overrideAccess: true`) — always set `overrideAccess: false` + `user` for user-facing logic +- **Two access patterns:** `req.payload` (in hooks/access control) vs `await getPayload({ config })` (everywhere else) +- **Thread `req` through** all operations when using Postgres or MongoDB replica sets (transactions) +- **Full TypeScript inference** — `payload.create({ collection: 'posts', data: { ... } })` returns typed `Post` +- **`pagination: false`** skips count queries — useful for internal data fetching where counts aren't needed +- **`disableErrors: true`** makes `findByID` return `null` instead of throwing — use cautiously in production +- **`context`** passes extra metadata to hooks without polluting document data (e.g. `triggerBeforeChange` flag) +- **Server functions** (`'use server'`) are the bridge for using Local API from client-triggered actions in Next.js +- **`payload run`** CLI handles `.env` automatically for standalone scripts — no need for `dotenv` + ## Related - [[wiki/payloadcms/queries|Queries]] - [[wiki/payloadcms/rest-api|REST API]] -- [[wiki/payloadcms/admin-panel-overview|Admin Panel]] +- [[wiki/payloadcms/hooks|Hooks]] +- [[wiki/payloadcms/authentication-operations|Authentication — Operations]] +- [[wiki/payloadcms/local-api-outside-nextjs|Local API — Outside Next.js]] +- [[wiki/payloadcms/local-api-access-control|Local API — Access Control]] +- [[wiki/payloadcms/database-transactions|Database — Transactions]] diff --git a/wiki/payloadcms/migration-guide-overview.md b/wiki/payloadcms/migration-guide-overview.md new file mode 100644 index 0000000..3452ea4 --- /dev/null +++ b/wiki/payloadcms/migration-guide-overview.md @@ -0,0 +1,37 @@ +--- +title: "Migration Guides — Overview" +aliases: [payload-migration, payload-upgrade, payload-breaking-changes] +tags: [payloadcms, migration, upgrade, versioning] +sources: [raw/migration-guide__overview.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +# Migration Guides — Overview + +Payload follows [semantic versioning](https://semver.org/). Major version bumps may include breaking changes; each upgrade path has a dedicated migration guide. + +## Upgrade Paths + +| From | To | Guide | +|------|----|-------| +| Payload 2.x | 3.0 | [[wiki/payloadcms/migration-troubleshooting\|Migration & Troubleshooting]] — v2→v3 section | +| Payload 3.x | 4.0 | [[wiki/payloadcms/migration-troubleshooting\|Migration & Troubleshooting]] — v3→v4 section | + +## Key Takeaways + +- Payload uses **semantic versioning** — minor/patch releases are safe to upgrade; only major bumps may break. +- Always read the migration guide for the specific major version before upgrading. +- Two active upgrade paths exist: **2→3** and **3→4**. +- See [[wiki/payloadcms/database-migrations\|Database Migrations]] for running DB schema migrations after a Payload upgrade. +- See [[wiki/payloadcms/installation\|Installation]] for version requirements (Node 24+, Next.js 16.2.6+). + +## Related + +- [[wiki/payloadcms/migration-troubleshooting\|Migration & Troubleshooting]] — full codemod steps, dependency dedup fix, common errors +- [[wiki/payloadcms/database-migrations\|Database Migrations]] — `migrate`, `migrate:create`, `migrate:rollback` CLI reference +- [[wiki/payloadcms/getting-started\|Getting Started]] — fresh install baseline + +--- + +*Source: raw/migration-guide__overview.md — https://payloadcms.com/docs/migration-guide/overview* diff --git a/wiki/payloadcms/migration-guide-v2-to-v3.md b/wiki/payloadcms/migration-guide-v2-to-v3.md new file mode 100644 index 0000000..404109e --- /dev/null +++ b/wiki/payloadcms/migration-guide-v2-to-v3.md @@ -0,0 +1,205 @@ +--- +title: "Payload CMS 2.0 → 3.0 Migration Guide" +aliases: [payload-v2-to-v3, payload-migration-v3, payload-upgrade-3] +tags: [payloadcms, migration, nextjs, breaking-changes, upgrade] +sources: [raw/migration-guide__v3.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +# Payload CMS 2.0 → 3.0 Migration Guide + +v3 replatforms the Admin Panel from React Router SPA → **Next.js App Router** with full React Server Component support. Core APIs are unchanged; the blast radius is in the HTTP layer, admin config, and custom components. + +--- + +## Installation + +```bash +npx create-payload-app # scaffold new project to copy auto-generated files from +pnpm i next react react-dom payload @payloadcms/ui @payloadcms/next +pnpm remove express nodemon @payloadcms/bundler-webpack @payloadcms/bundler-vite +``` + +- All `@payloadcms/*` package versions **must be in sync** — no mixed versions. +- Run DB migrations if you have existing data (see [[wiki/payloadcms/database-migrations|Database Migrations]]). +- If uploads use `formatOptions`/`imageSizes`/`resizeOptions`, install `sharp` and add it to `buildConfig({ sharp })`. + +--- + +## Breaking Changes — Config + +| What changed | Before | After | +|---|---|---| +| Bundler | `admin.bundler: webpackBundler()` | Remove entirely — Next.js bundles now | +| Secret | `payload.init({ secret })` | Move to `buildConfig({ secret: process.env.PAYLOAD_SECRET })` | +| Client env vars | `PAYLOAD_PUBLIC_*` | `NEXT_PUBLIC_*` | +| `req` object | extends Express Request (`req.headers['x']`) | extends Web Request (`req.headers.get('x')`) | +| Admin CSS | `admin.css` / `admin.scss` | Edit `(payload)/custom.scss`; or use `admin.components.providers` | +| `admin.indexHTML` | present | removed | +| Collection admin hooks | `collection.admin.hooks.beforeDuplicate` | field-level `hooks.beforeDuplicate` array | +| `upload.staticDir` | relative path | **must be absolute** (`path.resolve(dirname, '../uploads')`) | +| `upload.staticURL` | present | removed; use `generateFileURL` | +| `admin.favicon` | string path | `admin.icons: [{ path, sizes }]` | +| `admin.meta.ogImage` | string | `admin.meta.openGraph.images: []` | +| `admin.livePreview.url` args | `documentInfo` | `collectionConfig` / `globalConfig` | +| Logout/inactivity routes | `admin.logoutRoute`, `admin.inactivityRoute` | `admin.routes.logout`, `admin.routes.inactivity` | +| `custom` property | available on client | **server-only**; use `admin.custom` for client | +| `hooks.afterError` | single function | **array** of functions + new args (`req`, `res`) | +| Public dir | `./src/public` | `./public` (Next.js convention) | +| Localized sub-fields | manual `localized: true` | auto-stripped when parent is localized; use `allowLocalizedWithinLocalized` flag to disable | + +--- + +## Custom Components + +- All Payload UI components moved: `'payload'` → `'@payloadcms/ui'` +- Components now defined as **file paths** (strings), not imported references: + ```diff + - Button: MyComponent + + Button: '/src/components/Logout#MyComponent' + ``` +- All custom components are **server-rendered by default** — add `'use client'` for state/hooks. +- `admin.description` no longer accepts React components; use `admin.components.edit.Description` / `admin.components.elements.Description` / `admin.components.Description`. +- Array `RowLabel` and Collapsible `Label` only accept component **paths** (no plain strings). +- View keys are now **camelCase** (`Edit` → `edit`, `Default` → `default`). +- Custom views use `{ Component: './path/to/View.tsx' }` shape; custom root views go under `edit.root`. +- View Tab `isActive`/`href` no longer receive `match`/`location` (React Router only); use `usePathname()` instead. +- `pillLabel` → `tab.Pill` component path. +- `react-i18n` `Trans` → `@payloadcms/ui` `Translation` component with `elements` map. + +--- + +## Endpoints + +- Handler signature: `(req)` only — no `res`, no `next`; return `Response.json(...)`. +- `req.params` → `req.routeParams`. +- `req.data` / `req.locale` are **not** auto-resolved; call utilities manually: + ```ts + import { addDataAndFileToRequest, addLocalesToRequest } from '@payloadcms/next/utilities' + await addDataAndFileToRequest(req) + await addLocalesToRequest(req) + ``` + +--- + +## React Hooks + +| Hook | Change | +|---|---| +| `useTitle` | Removed; use `const { title } = useDocumentInfo()` | +| `useDocumentInfo` | `collection`/`global` → `collectionSlug`/`globalSlug` | +| `useConfig` | Import from `@payloadcms/ui`; returns `ClientConfig` (not `SanitizedConfig`); destructure as `const { config } = useConfig()` | +| `useCollapsible` | `collapsed` → `isCollapsed`; `withinCollapsible` → `isWithinCollapsible` | +| `useTranslation` | No options; use full `group:key` syntax (`t('general:cancel')`) | + +--- + +## Types + +```diff +- Fields → FormState +- BlockField → BlocksField +- BlockFieldProps → BlocksFieldProps +``` + +--- + +## Email Adapters + +Email extracted into adapters — no longer bundled: + +```ts +import { nodemailerAdapter } from '@payloadcms/email-nodemailer' +// or +import { resendAdapter } from '@payloadcms/email-resend' + +buildConfig({ email: nodemailerAdapter({ ... }) }) +``` + +- Rate-limiting removed (only available with custom Express server). + +--- + +## Plugins + +- All plugins use **named exports** with `Plugin` suffix: + ```diff + - import seo from '@payloadcms/plugin-seo' + + import { seoPlugin } from '@payloadcms/plugin-seo' + ``` + +### Cloud Storage + +`@payloadcms/plugin-cloud-storage` adapters extracted to standalone packages: + +| Service | New package | +|---|---| +| AWS S3 | `@payloadcms/storage-s3` | +| Azure | `@payloadcms/storage-azure` | +| GCS | `@payloadcms/storage-gcs` | +| Vercel Blob | `@payloadcms/storage-vercel-blob` | + +```ts +import { s3Storage } from '@payloadcms/storage-s3' +plugins: [s3Storage({ collections: { media: true }, bucket: process.env.S3_BUCKET, config: { ... } })] +``` + +### Plugin Field Overrides (form-builder, redirects) + +`fields` option changed from array → function receiving `defaultFields`: +```ts +fields: ({ defaultFields }) => [...defaultFields, { name: 'custom', type: 'text' }] +``` + +### Payload Cloud Plugin + +```diff +- @payloadcms/plugin-cloud → payloadCloud() ++ @payloadcms/payload-cloud → payloadCloudPlugin() +``` + +--- + +## Reserved Field Names + +Avoid these in Collections/Globals — Payload will sanitize them: + +- General: `file`, `_id` (Mongo), `__v` (Mongo) +- Internal (avoid `_` prefix generally): `_order`, `_path`, `_uuid`, `_parent_id`, `_locale` +- Auth: `salt`, `hash`, `password`, `email`, `username`, `apiKey`, `resetPasswordToken`, `resetPasswordExpiration` +- Upload: `filename`, `mimetype`, `filesize`, `width`, `height`, `url`, `thumbnailURL`, `focalX`, `focalY`, `sizes` + +--- + +## Key Takeaways + +- **Admin Panel** is now a Next.js App Router app — remove all Webpack/Vite bundler config. +- **`secret`** moves from `payload.init()` → `buildConfig()`. +- **Custom components** are file paths (strings), server-rendered by default; add `'use client'` for hooks. +- **Endpoint handlers** return `Response.json()` — no Express `res` arg; parse `req.data` manually. +- **Cloud storage** plugins are now standalone packages per provider. +- **Plugin imports** are named exports with `Plugin` suffix. +- **React hooks** import from `@payloadcms/ui`, not `'payload'`. +- All `@payloadcms/*` packages must be **version-synced**. + +--- + +## Related + +- [[wiki/payloadcms/migration-guide-overview|Migration Guides — Overview]] +- [[wiki/payloadcms/migration-troubleshooting|Migration & Troubleshooting]] +- [[wiki/payloadcms/installation|Installation]] +- [[wiki/payloadcms/configuration|Payload Config — Overview]] +- [[wiki/payloadcms/custom-components-authoring|Custom Components — Authoring Guide]] +- [[wiki/payloadcms/environment-vars|Environment Variables]] +- [[wiki/payloadcms/plugins|Official Plugins]] +- [[wiki/payloadcms/upload|Upload & Media]] +- [[wiki/payloadcms/email|Email — Adapters]] + +--- + +## Sources + +- `raw/migration-guide__v3.md` +- https://payloadcms.com/docs/migration-guide/v3 diff --git a/wiki/payloadcms/migration-guide-v3-to-v4.md b/wiki/payloadcms/migration-guide-v3-to-v4.md new file mode 100644 index 0000000..24b917b --- /dev/null +++ b/wiki/payloadcms/migration-guide-v3-to-v4.md @@ -0,0 +1,185 @@ +--- +title: "Migration Guide — 3.0 → 4.0" +aliases: [payload-v4-migration, payload-3-to-4] +tags: [payloadcms, migration, breaking-changes, codemod] +sources: [raw/migration-guide__v4.md] +created: 2026-05-15 +updated: 2026-05-15 +--- + +# Migration Guide — Payload 3.0 → 4.0 + +Use the codemod first — it handles most breaking changes automatically: + +```bash +npx @payloadcms/codemod +``` + +The codemod is idempotent and safe on partially-migrated projects. Individual transforms can be run with `--transform `. + +--- + +## Breaking Changes + +### List View Select API — now default + +`admin.enableListViewSelectAPI` has been **removed**. The List View always uses the Select API (query only active columns). Remove the property if you had it set to `true`. + +Codemod: `migrate-list-view-select-api` + +--- + +### Globals: `admin.components.elements` → `admin.components.edit` + +Global configs now use the same `admin.components` shape as Collections. + +```diff +admin: { + components: { +- elements: { +- SaveButton: '/path/to/CustomSaveButton', +- Description: '/path/to/CustomDescription', +- }, ++ Description: '/path/to/CustomDescription', ++ edit: { ++ SaveButton: '/path/to/CustomSaveButton', ++ }, + }, +} +``` + +Slots under `admin.components.edit` for Globals now match Collections: `beforeDocumentControls`, `editMenuItems`, `PreviewButton`, `PublishButton`, `SaveButton`, `SaveDraftButton`, `Status`, `UnpublishButton`. (`Upload` stays Collection-only.) + +Globals also now support custom views via `admin.components.views` with arbitrary keys. + +Codemod: `globals-components-edit` + +--- + +### `forceSelect` → `select` function + +Static `forceSelect` is removed. Each Collection/Global now accepts a top-level `select` function: + +```diff +- forceSelect: { +- title: true, +- slug: true, +- }, ++ select: ({ select }) => (select ? { ...select, title: true, slug: true } : undefined), +``` + +- Returning `undefined` leaves the caller's `select` unchanged. +- The function **replaces** (not deep-merges) the caller's select — spread `select` manually to preserve previous deep-merge behavior. +- Codemod handles nested `forceSelect` by importing `deepMergeSimple` from `payload/shared`. + +Codemod: `migrate-force-select` + +--- + +### `admin.hideAPIURL` removed + +Use `admin.components.views.edit.api.tab.condition` instead: + +```diff +- hideAPIURL: true, ++ components: { ++ views: { ++ edit: { ++ api: { tab: { condition: () => false } }, ++ }, ++ }, ++ }, +``` + +Codemod: `remove-hide-api-url` + +--- + +### Aliased re-exports removed from `@payloadcms/ui` and `@payloadcms/next` + +Import directly from `payload` or `payload/shared`: + +| Symbol | Old source | New source | +|--------|------------|------------| +| `Column`, `ListViewSlots`, `ListViewClientProps` | `@payloadcms/ui` | `payload` | +| `EntityType`, `formatAdminURL`, `mergeListSearchAndWhere` | `@payloadcms/ui/shared` | `payload/shared` | +| `mergeHeaders`, `headersWithCors`, `createPayloadRequest`, `addDataAndFileToRequest`, `sanitizeLocales`, `addLocalesToRequestFromData` | `@payloadcms/next/utilities` | `payload` | + +**Renamed types** (import from `payload`): + +| Old | New | +|-----|-----| +| `ListPreferences` | `CollectionPreferences` | +| `ListComponentClientProps` | `ListViewClientProps` | +| `ListComponentServerProps` | `ListViewServerProps` | + +Codemod: `migrate-aliased-exports` (adds `as` aliases so existing code compiles — rename usages manually to fully commit). + +--- + +### `useDocumentInfo` no longer provides `title` / `setDocumentTitle` + +Document title is split into its own `DocumentTitleContext`. Use `useDocumentTitle` hook: + +```diff +- import { useDocumentInfo } from '@payloadcms/ui' ++ import { useDocumentTitle } from '@payloadcms/ui' + +- const { title, setDocumentTitle } = useDocumentInfo() ++ const { title, setDocumentTitle } = useDocumentTitle() +``` + +If you also use other `useDocumentInfo` properties, keep both calls. + +Codemod: `migrate-document-title-context` (handles mixed destructures, removes unused import, preserves aliases). + +--- + +### Plugin Import/Export: `toCSV` / `fromCSV` → hooks + +`toCSV` and `fromCSV` in `custom['plugin-import-export']` are removed. Use `hooks.beforeExport` / `hooks.beforeImport`: + +```diff +custom: { + 'plugin-import-export': { +- toCSV: ({ value, columnName, row, doc }) => { ... }, +- fromCSV: ({ value, data }) => ..., ++ hooks: { ++ beforeExport: ({ value, columnName, siblingData, data }) => { ... }, ++ beforeImport: ({ value, data }) => ..., ++ }, + }, +} +``` + +Arg mapping: `row` → `siblingData`, `doc` → `data`. New `format` param (`'csv' | 'json'`) available in `beforeExport`. + +Codemod: `migrate-import-export-hooks` (rename `row`/`doc` references manually after migration). + +--- + +## Key Takeaways + +- **Run `npx @payloadcms/codemod` first** — covers ~80% of changes automatically; idempotent and safe to re-run. +- **`forceSelect` is gone** — new `select` function replaces (not merges); spread the caller's `select` to preserve old behavior. +- **Globals admin.components shape unified** — `elements` → `edit`, `Description` hoisted to top level. +- **Title split from DocumentInfoContext** — components that only needed `title` can drop `useDocumentInfo` entirely. +- **Re-exports cleaned up** — import `Column`, `mergeHeaders`, etc. from `payload` / `payload/shared` directly. +- **Import/Export plugin API** — `toCSV`/`fromCSV` replaced with `hooks.beforeExport`/`hooks.beforeImport`; mind renamed args (`row` → `siblingData`, `doc` → `data`). + +--- + +## Related + +- [[wiki/payloadcms/migration-guide-v2-to-v3|Migration Guide — 2.0 → 3.0]] +- [[wiki/payloadcms/migration-guide-overview|Migration Guides — Overview]] +- [[wiki/payloadcms/collection-config|Collection Config]] — `select` function, admin components +- [[wiki/payloadcms/globals-config|Globals Config]] — updated `admin.components` shape +- [[wiki/payloadcms/plugins|Official Plugins]] — Import/Export plugin hooks + +--- + +## Sources + +- `raw/migration-guide__v4.md` +- https://payloadcms.com/docs/migration-guide/v4