vault backup: 2026-05-10 22:37:43

This commit is contained in:
Vadym Samoilenko 2026-05-10 22:37:43 +01:00
parent e92e504dc2
commit 1a5bac866c
11 changed files with 812 additions and 1 deletions

View file

@ -23,7 +23,7 @@ This 3-hop pattern works for hundreds of articles without vector search.
| [[wiki/tech-patterns/_index\|tech-patterns/]] | Recurring tech stacks: FastAPI, React/Vite, Next.js, Azure AD, AI, Box, One2Edit, Redis/Celery, cost-tracker, OMG API | 28 |
| [[wiki/architecture/_index\|architecture/]] | Cross-cutting architectural patterns: Docker Compose, multi-agent AI, GCP timeout, RAG, hotfolder, optical-dev deploy, cost-tracker, new-project checklist, troubleshooting playbooks, ADR log, Cloud Run Jobs | 11 |
| [[wiki/client-knowledge/_index\|client-knowledge/]] | Per-client notes for Ford, H&M, L'Oréal, Barclays, Ferrero, 3M, BAIC | 7 |
| [[wiki/concepts/_index\|concepts/]] | Atomic knowledge extracted from Claude Code sessions | 166 |
| [[wiki/concepts/_index\|concepts/]] | Atomic knowledge extracted from Claude Code sessions | 174 |
| [[wiki/connections/_index\|connections/]] | Cross-cutting insights linking 2+ concepts: FastAPI+Azure AD+Docker trinity, AI→cost-tracker, Apache+Vite basePath, GCP→REST polling, Box+hotfolder, Docker DNS+AdGuard, Celery prefork×faster_whisper memory stacking | 10 |
| [[wiki/qa/_index\|qa/]] | Filed answers to queries (saved with `--file-back`) | 0 |
| [[wiki/homelab/_index\|homelab/]] | Self-hosted infra: Proxmox install, IOMMU/PCI passthrough, hypervisor setup, budget builds, HP Elitedesk G3, Homarr API + Apps + Boards + Certificates + Integrations + Settings + Tasks + AdGuard + Clock + Docker Stats + Docker Integration + Download Client + Firewall + Proxmox Integration + Radarr + Readarr + Sonarr + Bookmarks + Calendar + Icons + App Widget + Weather + GitHub + Nextcloud + qBittorrent + RSS Feed + Speedtest Tracker + System Health Monitoring + System Resources + Services Map + Media Stack | 43 |

View file

@ -216,5 +216,13 @@
| [[wiki/concepts/figma-fig-binary-format\|Figma .fig File — Proprietary Binary Format; Assets Extractable, Layout Data Not]] | `.fig` = ZIP with proprietary binary `canvas.fig` (magic bytes `fig-kiwij`); image assets extractable via `unzip`, but layer/component/text data requires Figma REST API or Desktop | daily/2026-05-10.md | 2026-05-10 |
| [[wiki/concepts/vite-base-url-slash-concatenation]] | `${BASE_URL}file.png` works in dev (BASE_URL="/") but breaks in prod — always use explicit slash or `new URL(file, BASE_URL).href` | daily/2026-05-10.md | 2026-05-10 |
| [[wiki/concepts/adguard-parallel-instances-diagnostic]] | Two AdGuard instances on different IPs — how to identify the real production one vs empty placeholder; fix is router-side DNS update | daily/2026-05-03.md | 2026-05-03 |
| [[wiki/concepts/payload-cms-overrideaccess-bypass]] | `overrideAccess: true` in Payload CMS completely bypasses access control — critical anti-pattern in webhook handlers; unauthenticated lead modification | daily/2026-05-09.md | 2026-05-09 |
| [[wiki/concepts/nextjs-unstable-cache-force-dynamic]] | `unstable_cache` + tag revalidation vs `force-dynamic` — ISR pattern for shared rarely-changing data; 99%+ reduction in external API calls | daily/2026-05-09.md | 2026-05-09 |
| [[wiki/concepts/map-ratelimiter-memory-leak]] | In-memory Map rate limiters grow unbounded — `setInterval` eviction with `.unref()` prevents memory leak and test runner hang | daily/2026-05-09.md | 2026-05-09 |
| [[wiki/concepts/vitest-module-level-env-testing]] | `vi.stubEnv` + `vi.resetModules()` + dynamic `await import()` required when module reads env vars at load time | daily/2026-05-09.md | 2026-05-09 |
| [[wiki/concepts/webhook-replay-attack-prevention]] | Timestamp ±5min validation + callId deduplication (DB or Redis NX) to prevent webhook replay attacks — HMAC alone is not sufficient | daily/2026-05-09.md | 2026-05-09 |
| [[wiki/concepts/secrets-git-history-purge]] | `git-filter-repo --invert-paths --path .env` permanently removes secrets from history; force push required; all clones must be redone | daily/2026-05-09.md | 2026-05-09 |
| [[wiki/concepts/payload-cms-push-dev-prod]] | `push: process.env.NODE_ENV === 'development'` — auto-schema in dev, migration-only in prod; `push: false` default silently breaks fresh DB | daily/2026-05-09.md | 2026-05-09 |
| [[wiki/concepts/cloudflare-proxied-domain-npm-subpath]] | Cloudflare-only domains have no NPM nginx vhost — Custom Locations fail; solution is a separate subdomain with its own NPM Proxy Host | daily/2026-05-09.md | 2026-05-09 |
<!-- Articles added automatically by compile.py -->
<!-- Format: | [[concepts/slug]] | One-line summary | daily/YYYY-MM-DD.md | date | -->

View file

@ -0,0 +1,88 @@
---
name: cloudflare-proxied-domain-npm-subpath
description: Cloudflare-proxied domains without an NPM proxy host have no nginx config — NPM Custom Locations cannot be added; requires a separate subdomain
type: concept
---
# Nginx Proxy Manager — Custom Locations Don't Work for Cloudflare-Only Domains
## The Setup
A typical homelab has two categories of external domains:
1. **NPM-managed domains**: Domain → Cloudflare → NPM → internal service. NPM generates a full nginx vhost config.
2. **Cloudflare-only domains**: Domain → Cloudflare → directly to internal IP (Cloudflare Full/Strict mode, or just DNS). No NPM entry exists.
## The Problem
NPM's "Custom Locations" feature lets you add path-based routing to an existing Proxy Host:
```
nextcloud.ai-impress.com/push/ → http://localhost:7867
```
But this only works if a Proxy Host entry for `nextcloud.ai-impress.com` exists in NPM. If the domain goes directly through Cloudflare to the Nextcloud server, **NPM has no nginx config file for it**. There is no vhost to attach a Custom Location to.
You cannot add a Custom Location for a domain that has no NPM Proxy Host.
## Real Example — Nextcloud Signaling Server
The `nextcloud.ai-impress.com` domain was accessed via Cloudflare without a matching NPM proxy host. Attempting to add `nextcloud.ai-impress.com/push/` as a custom location to route WebSocket traffic to the `strukturag/nextcloud-spreed-signaling` server failed — there was no NPM entry to attach it to.
### Incorrect Approach
```
Existing NPM host: (none for nextcloud.ai-impress.com)
Attempt: Add Custom Location → /push/ → localhost:8083
Result: Error — no proxy host to add location to
```
### Correct Approach
Create a **separate subdomain** with its own NPM Proxy Host entry:
```
NPM Proxy Host #39:
Domain: signal.ai-impress.com
Forward: localhost:8083 (signaling server)
SSL: Let's Encrypt
WebSocket support: enabled
Nextcloud config (config.php):
'signaling_servers' => [
['url' => 'wss://signal.ai-impress.com']
]
```
## General Rule
| Domain routing | NPM Custom Locations available? |
|---------------|----------------------------------|
| Domain → NPM → service (NPM manages SSL) | Yes |
| Domain → Cloudflare → NPM → service | Yes (NPM has the entry) |
| Domain → Cloudflare → direct IP (no NPM) | **No** — NPM has no vhost for this domain |
## HTTP 404 from Signaling Server (Not an Error)
A related diagnostic: the `strukturag/nextcloud-spreed-signaling` server returns HTTP 404 when accessed without a WebSocket upgrade header. This is normal and expected behavior — the server only handles WebSocket connections, not plain HTTP GET requests. A 404 from a curl to the signaling URL does not indicate misconfiguration.
```bash
curl https://signal.ai-impress.com/spreed
# → 404 Not Found (expected — WebSocket upgrade required)
```
Test the WebSocket connection instead:
```bash
wscat -c wss://signal.ai-impress.com/spreed
```
## Container Config Path Mismatch (strukturag/nextcloud-spreed-signaling)
The signaling server container expects its config at `/config/server.conf`, not `/etc/signaling/server.conf`. Mount accordingly:
```yaml
volumes:
- ./signaling/server.conf:/config/server.conf:ro
```
**Source:** daily/2026-05-09.md | 2026-05-09

View file

@ -0,0 +1,94 @@
---
name: map-ratelimiter-memory-leak
description: In-memory Map rate limiters grow unbounded in long-running Node.js servers — need setInterval eviction with .unref()
type: concept
---
# In-Memory Map Rate Limiter — Memory Leak and Eviction Pattern
## The Pattern
A token-bucket rate limiter using a JavaScript `Map` is a common, dependency-free implementation:
```ts
const buckets = new Map<string, { tokens: number; lastRefill: number }>()
export function rateLimit(ip: string): boolean {
const now = Date.now()
let bucket = buckets.get(ip)
if (!bucket) {
bucket = { tokens: MAX_TOKENS, lastRefill: now }
buckets.set(ip, bucket)
}
// refill logic...
return bucket.tokens > 0
}
```
## The Problem
Every unique IP address creates a new `Map` entry. Entries are never deleted. In a long-running server this means:
- Each unique client = permanent memory allocation
- After weeks of production traffic, thousands of stale entries occupy RAM
- In a high-traffic scenario (CDN scraping, bot traffic) this becomes a slow memory leak that eventually causes OOM
The `Map` also holds strong references, preventing GC of the bucket objects.
## The Fix — `setInterval` with `.unref()`
```ts
const REFILL_INTERVAL_MS = 60_000 // 1 minute
// Evict stale buckets that haven't been touched in 2 refill cycles
setInterval(() => {
const cutoff = Date.now() - REFILL_INTERVAL_MS * 2
for (const [ip, bucket] of buckets) {
if (bucket.lastRefill < cutoff) {
buckets.delete(ip)
}
}
}, REFILL_INTERVAL_MS).unref()
// ^^^^^^^^
// .unref() prevents the interval from keeping the process alive
// if this is the only remaining async operation (important for tests)
```
### Why `.unref()`?
`setInterval` holds an event loop reference. If you don't call `.unref()`:
- Test runners (Vitest, Jest) will hang after tests complete
- `process.exit()` won't fire until the interval fires or is cleared
- In serverless/edge environments the process may not terminate cleanly
`.unref()` makes the interval "passive" — it runs if the event loop is still active from other work, but it won't prevent the process from exiting.
## Sizing the Eviction Window
The cutoff of `REFILL_INTERVAL_MS * 2` gives each IP two full refill cycles of inactivity before eviction. A returning client after eviction simply gets a fresh full-token bucket (as if first visit), which is the intended behavior.
For aggressive rate limiting (e.g., blocking brute-force) make the window longer to ensure blocked IPs stay tracked:
```ts
const EVICTION_WINDOW_MS = 24 * 60 * 60 * 1000 // 24 hours
setInterval(() => {
const cutoff = Date.now() - EVICTION_WINDOW_MS
for (const [ip, bucket] of buckets) {
if (bucket.lastRefill < cutoff) buckets.delete(ip)
}
}, 60_000).unref()
```
## Production Recommendation
For production rate limiting under real load, prefer Redis-backed solutions (`ioredis` + sliding window script, or `@upstash/ratelimit`) that:
- Survive server restarts
- Work across multiple instances
- Have built-in expiry via Redis TTL
Use in-memory `Map` for:
- Development / single-instance deployments
- Low-traffic internal tooling
- When Redis is unavailable and you control the instance count
**Source:** daily/2026-05-09.md | 2026-05-09

View file

@ -0,0 +1,86 @@
---
name: nextjs-unstable-cache-force-dynamic
description: unstable_cache with tag-based revalidation vs force-dynamic — ISR pattern for rarely-changing external API data in Next.js 15
type: concept
---
# Next.js 15 — `unstable_cache` vs `force-dynamic` for External API Routes
## The Problem with `force-dynamic`
```ts
// BAD — disables ALL caching, hits DB + external API on every request
export const dynamic = 'force-dynamic'
export async function GET() {
const tariffs = await fetchFromExternalAPI()
// ...
}
```
`export const dynamic = 'force-dynamic'` is a segment-level directive that opts the entire route out of Next.js's Data Cache. Every request triggers:
1. A fresh fetch to the external API
2. A fresh DB query
3. Full server rendering overhead
This is appropriate for user-specific data that changes per-request (e.g., cart contents, auth status). It is **wrong** for data that is the same for all users and changes infrequently (e.g., pricing tariffs, product catalogs).
## The Fix — `unstable_cache` with Tag-Based Revalidation
```ts
import { unstable_cache } from 'next/cache'
// Remove: export const dynamic = 'force-dynamic'
const getCachedTariffs = unstable_cache(
async () => {
const [ezyTariffs, dbTariffs] = await Promise.all([
fetchFromExternalAPI(),
payload.find({ collection: 'tariffs', limit: 1000 }),
])
return mergeTariffs(ezyTariffs, dbTariffs.docs)
},
['tariffs'], // cache key
{
revalidate: 300, // revalidate every 5 minutes (seconds)
tags: ['tariffs'], // tag for on-demand invalidation
}
)
export async function GET() {
const tariffs = await getCachedTariffs()
return Response.json(tariffs)
}
```
### On-Demand Revalidation (e.g., after admin update)
```ts
import { revalidateTag } from 'next/cache'
// Call from a webhook or admin action after tariffs change
revalidateTag('tariffs')
```
## Choosing the Right Strategy
| Data type | Strategy | Why |
|-----------|----------|-----|
| Per-user, changes every request | `force-dynamic` or `no-store` | No shared cache possible |
| Shared, changes rarely (tariffs, config) | `unstable_cache` with `revalidate` | Serve stale, refresh in background |
| Static, never changes at runtime | Default (static generation) | Build-time bake-in |
| Shared, changes after specific events | `unstable_cache` with `tags` | Invalidate precisely |
## Naming Note
`unstable_cache` is stable in practice despite the prefix — Vercel ships production apps on it. The `unstable_` prefix signals it may be renamed in a future API stabilization. The `cache()` function (React 19's `use cache` directive) is the eventual successor.
## Performance Impact
For a tariff endpoint called N times/minute:
- `force-dynamic`: N × (external API latency + DB query)
- `unstable_cache` (5 min TTL): 1 × (external API latency + DB query) per 5 min + N × cache read
At 100 req/min this is a ~99.7% reduction in external API calls.
**Source:** daily/2026-05-09.md | 2026-05-09

View file

@ -0,0 +1,99 @@
---
name: payload-cms-overrideaccess-bypass
description: overrideAccess:true in Payload CMS completely bypasses the access control system — critical anti-pattern in webhook handlers
type: concept
---
# Payload CMS — `overrideAccess: true` Bypasses All Access Control
## What It Does
Passing `overrideAccess: true` to any Payload local API call (`payload.find()`, `payload.update()`, `payload.create()`, etc.) tells Payload to **skip the collection's `access` functions entirely**. No role check, no ownership check, no field-level restriction — the call succeeds as if it came from a superadmin.
```ts
// This bypasses ALL access control — any caller can modify any lead
await payload.update({
collection: 'leads',
id: leadId,
data: { status: 'converted' },
overrideAccess: true, // ← NEVER do this in a public-facing handler
})
```
## Why It Exists
`overrideAccess` is intended for:
- Server-side seed scripts
- Internal background jobs that run with inherent trust
- Admin CLI utilities
It is **not** intended for request handlers that accept untrusted input.
## The Anti-Pattern
Using `overrideAccess: true` inside a webhook route (or any publicly-reachable API route) means:
1. **HMAC validation failure** (or intentional bypass) → attacker can manipulate Payload documents without any identity.
2. Even with HMAC in place — if HMAC is mis-implemented (e.g., empty-string secret fallback) the access control is your last defense. `overrideAccess: true` removes it.
### Real Example (Shumiland project — `src/app/api/binotel/webhook/route.ts`)
```ts
// BAD: overrideAccess:true in a webhook handler
const lead = await payload.find({
collection: 'leads',
where: { phone: { equals: phone } },
overrideAccess: true, // ← attacker can enumerate leads
})
await payload.update({
collection: 'leads',
id: lead.docs[0].id,
data: { callStatus: newStatus },
overrideAccess: true, // ← attacker can overwrite any lead
})
```
## The Fix
Remove `overrideAccess: true` and either:
**Option A — use a system-level user identity (recommended for webhooks):**
```ts
// Create a service account user and use req.user
const systemUser = await getSystemUser() // cached, read once
await payload.update({
collection: 'leads',
id: lead.docs[0].id,
data: { callStatus: newStatus },
user: systemUser, // proper identity context
// no overrideAccess
})
```
**Option B — use `req.payload` with the Payload admin user if running inside a Next.js route handler:**
```ts
// The req object from NextRequest has no Payload user attached
// You need to explicitly provide identity
```
**Minimum mitigation** if full fix isn't feasible: scope `overrideAccess: true` only to the narrowest possible read (by phone + exact match), and verify HMAC rigorously before reaching it.
## Related Vulnerabilities
When combined with other gaps this becomes critical:
| Gap | Effect when combined with overrideAccess |
|-----|------------------------------------------|
| `BINOTEL_HMAC_SECRET ?? ''` empty fallback | HMAC skipped entirely → full unauthenticated write |
| No timestamp validation | Replayed requests can re-trigger status changes indefinitely |
| HMAC logged on failure | Secret leaks to any log aggregation system |
## See Also
- [[wiki/concepts/webhook-replay-attack-prevention]] — timestamp + nonce deduplication
- [[wiki/concepts/secrets-git-history-purge]] — if the HMAC secret was committed
**Source:** daily/2026-05-09.md | 2026-05-09

View file

@ -0,0 +1,100 @@
---
name: payload-cms-push-dev-prod
description: Payload CMS postgresAdapter push:false vs push:true — conditional schema auto-push for dev, migration-only for prod
type: concept
---
# Payload CMS — `push` Config for `postgresAdapter` (Dev vs Prod)
## What `push` Does
In `payload.config.ts`, `postgresAdapter` has a `push` option:
```ts
import { postgresAdapter } from '@payloadcms/db-postgres'
export default buildConfig({
db: postgresAdapter({
pool: { connectionString: process.env.DATABASE_URI },
push: true, // or false
}),
})
```
- **`push: true`** — Payload calls `drizzle-kit push` at startup, auto-syncing the schema to the DB. No migrations needed. Schema changes are applied immediately on every restart.
- **`push: false`** (default) — Payload uses `drizzle-kit migrate` style. Schema is only updated by running explicit migration files in `migrationDir`.
## The Critical Gotcha
**`push` defaults to `false`.**
After wiping the database volume (e.g., during development or staging reset), `push: false` means **tables are never created**. The admin panel returns SQL errors:
```
relation "users" does not exist
relation "payload_preferences" does not exist
```
This is a silent failure — there are no startup errors, just 500s when the admin panel tries to query tables that don't exist.
## The Correct Pattern
```ts
postgresAdapter({
pool: { connectionString: process.env.DATABASE_URI },
// Auto-push in dev (no migration workflow needed)
// Migration-only in prod (controlled, auditable, reversible)
push: process.env.NODE_ENV === 'development',
migrationDir: './migrations',
})
```
### Why Not Always `push: true`?
In production `push: true` means every deployment auto-alters the database schema. This is dangerous because:
- Column renames/drops are immediate and irreversible on the running DB
- No migration history → can't rollback schema changes
- Concurrent deployments can race on schema changes
### Why Not Always `push: false`?
In development `push: false` requires running `npx payload migrate` after every collection change. This adds friction and is easy to forget, leading to confusing "table not found" errors after volume resets.
## Running Migrations in Production
```bash
# Generate a migration from current schema changes
npx payload migrate:create --name add-status-to-leads
# Apply all pending migrations
npx payload migrate
# Check migration status
npx payload migrate:status
```
## Docker Compose Entrypoint Pattern
Run migrations before starting the server in production:
```dockerfile
# Dockerfile
CMD ["sh", "-c", "npx payload migrate && node server.js"]
```
Or in `docker-compose.yml`:
```yaml
services:
web:
command: sh -c "npx payload migrate && node .next/standalone/server.js"
depends_on:
db:
condition: service_healthy
```
## Related Scaffold Issue
When generating a new Payload + Next.js project, the scaffold (as of Payload 3.x) may omit `push: false` in the production config template. Always check `payload.config.ts` before deploying to production.
**Source:** daily/2026-05-09.md | 2026-05-09

View file

@ -0,0 +1,115 @@
---
name: secrets-git-history-purge
description: git-filter-repo to permanently remove committed secrets from git history — secrets are recoverable even in private repos until purged
type: concept
---
# Secrets Committed to Git History — Detection and Permanent Purge
## Why Git History Is Dangerous
`git log`, `git show`, `git blame` all expose history. A secret committed in commit `abc123` and later deleted in commit `def456` is still fully visible via `git show abc123:path/to/.env`.
In private repos this matters because:
- Anyone with repo read access (current or past collaborators, CI tokens) can retrieve it
- GitHub/Bitbucket may have indexed it
- Any clone or fork carries the full history
**Rotation is mandatory immediately; purge removes the attack surface permanently.**
## Detection — Finding the Commits
```bash
# Find commits that introduced .env or secrets files
git log --all --full-history -- .env
git log --all --full-history -- "*.env*"
# Show the actual content of a specific commit's .env
git show abc123:.env
# Search all commits for a specific string (slow on large repos)
git log -p --all | grep -i "POSTGRES_PASSWORD\|SECRET\|API_KEY"
```
## Purge — `git-filter-repo`
`git-filter-repo` is the modern replacement for `git filter-branch` (the old approach is slower, more error-prone, and deprecated by Git maintainers).
```bash
# Install
pip install git-filter-repo
# or: brew install git-filter-repo
# Remove specific files from ALL history
git filter-repo --invert-paths --path .env
git filter-repo --invert-paths --path .env.local
git filter-repo --invert-paths --path config/secrets.yml
# Remove multiple paths in one pass (more efficient)
git filter-repo \
--invert-paths \
--path .env \
--path .env.local \
--path docker-compose.override.yml
```
`--invert-paths` means "remove these paths" (keep everything else).
## After the Purge
```bash
# Verify the file is gone from history
git log --all --full-history -- .env
# Should return nothing
# Force push ALL branches (required — history was rewritten)
git push --force --all
git push --force --tags
# Notify all collaborators: their local clones are now diverged
# They MUST delete and re-clone — they cannot merge/rebase from old history
```
## Mandatory Steps After Any Secret Exposure
1. **Rotate immediately** — change the secret value before purging history
2. **Purge** — remove from git history
3. **Force push** — push rewritten history to remote
4. **Re-clone** — all team members must delete their local clone and re-clone
5. **Audit access logs** — check if the leaked secret was used during the exposure window
6. **Add to `.gitignore`** — prevent re-committing
## Real Example (Shumiland project — commits eaa200f and ae9a7cf)
```
eaa200f — committed .env with POSTGRES_PASSWORD, PAYLOAD_SECRET
ae9a7cf — committed docker-compose.yml with BINOTEL_HMAC_SECRET
```
Even though `.env` was added to `.gitignore` in a later commit, the secrets remain visible in those two commits until `git-filter-repo` is run.
## Prevention
```gitignore
# .gitignore
.env
.env.*
!.env.example
docker-compose.override.yml
```
Pre-commit hook (add to `.pre-commit-config.yaml`):
```yaml
- repo: https://github.com/Yelp/detect-secrets
rev: v1.4.0
hooks:
- id: detect-secrets
```
Or use `git-secrets` from AWS:
```bash
git secrets --install
git secrets --register-aws
```
**Source:** daily/2026-05-09.md | 2026-05-09

View file

@ -0,0 +1,99 @@
---
name: vitest-module-level-env-testing
description: vi.stubEnv + vi.resetModules() + dynamic await import() required when testing modules that read env vars at load time
type: concept
---
# Vitest — Testing Modules That Read Env Vars at Import Time
## The Problem
Many Next.js route files or Node.js modules read `process.env` variables at module load time:
```ts
// src/app/api/binotel/webhook/route.ts
const SECRET = process.env.BINOTEL_HMAC_SECRET ?? ''
// ^^^
// Read ONCE when the module is imported — not on each request
```
A standard Vitest test that sets `process.env.BINOTEL_HMAC_SECRET = ''` after importing the module **does not affect** the `SECRET` constant — it was already captured.
```ts
// BAD — env var set after module is already cached
import { POST } from '../route'
test('rejects empty secret', async () => {
process.env.BINOTEL_HMAC_SECRET = '' // too late — module already imported
const res = await POST(makeRequest())
expect(res.status).toBe(500) // might not behave as expected
})
```
## The Fix — `vi.stubEnv` + `vi.resetModules()` + Dynamic Import
```ts
import { vi, describe, beforeEach, afterEach, it, expect } from 'vitest'
describe('webhook route — empty secret', () => {
beforeEach(() => {
vi.stubEnv('BINOTEL_HMAC_SECRET', '') // 1. Set env var before import
vi.resetModules() // 2. Clear module cache
})
afterEach(() => {
vi.unstubAllEnvs() // 3. Restore env vars after test
vi.resetModules() // 4. Clean slate for next test
})
it('returns 500 when secret is empty', async () => {
// 5. Dynamic import AFTER stubEnv + resetModules
const { POST } = await import('../route')
const res = await POST(makeRequest())
expect(res.status).toBe(500)
})
})
```
### Why This Works
| Step | Effect |
|------|--------|
| `vi.stubEnv('KEY', 'value')` | Sets `process.env.KEY` and records it for later restoration |
| `vi.resetModules()` | Clears Vitest's module registry — next `import()` re-executes the module |
| `await import('../route')` | Module re-executes with new `process.env` values, captures updated constants |
| `vi.unstubAllEnvs()` | Restores `process.env` to original values |
## Testing `timingSafeEqual` Length Mismatch
A related gotcha: `crypto.timingSafeEqual(a, b)` throws a `RangeError` if buffers have different lengths. Most implementations add a length-mismatch fast-path:
```ts
function safeCompare(a: string, b: string): boolean {
if (a.length !== b.length) return false // ← must be tested explicitly
return crypto.timingSafeEqual(Buffer.from(a), Buffer.from(b))
}
```
Test the fast-path separately:
```ts
it('returns false for different-length strings without throwing', () => {
expect(safeCompare('abc', 'abcdef')).toBe(false)
// Would throw RangeError if fast-path is missing
})
```
## ESM Module Testing
If the module under test is pure ESM (has `"type": "module"` and uses `import`/`export` without CommonJS), it cannot be `require()`-d in a test. The dynamic `await import()` pattern works because Vitest runs in ESM mode itself.
If a module uses top-level `await` at import time (e.g., connects to DB), `resetModules()` + re-import will re-run that connection on every test — use `vi.mock` to stub the dependency instead.
## Pattern Summary
```
stubEnv → resetModules → await import() → test → unstubAllEnvs → resetModules
```
**Source:** daily/2026-05-09.md | 2026-05-09

View file

@ -0,0 +1,107 @@
---
name: webhook-replay-attack-prevention
description: Timestamp validation (±5min window) + request deduplication (per-phone nonce/ID) to prevent replay attacks on webhook endpoints
type: concept
---
# Webhook Security — Replay Attack Prevention
## What Is a Replay Attack?
An attacker captures a legitimate, HMAC-signed webhook request and re-sends it later. Even if HMAC validation passes (the signature is valid), the request should be rejected because it was already processed.
## Two-Layer Defense
### Layer 1 — Timestamp Validation (±5 Minute Window)
The webhook sender includes a timestamp in the payload or headers. The receiver rejects requests where the timestamp is more than 5 minutes old:
```ts
const MAX_AGE_MS = 5 * 60 * 1000 // 5 minutes
export async function POST(req: NextRequest) {
const body = await req.json()
const { timestamp, callId, phone, ...data } = body
// 1. Timestamp validation
const age = Date.now() - new Date(timestamp).getTime()
if (Math.abs(age) > MAX_AGE_MS) {
return Response.json({ error: 'Request expired' }, { status: 400 })
}
// 2. HMAC validation (must happen before any DB access)
const signature = req.headers.get('X-Binotel-Signature') ?? ''
if (!verifyHmac(body, signature)) {
return Response.json({ error: 'Invalid signature' }, { status: 401 })
}
// 3. Deduplication (see Layer 2)
}
```
### Layer 2 — Request Deduplication (Nonce / Idempotency Key)
Even within the 5-minute window, an attacker could replay a request several times. Use a nonce or call ID stored in the DB:
```ts
// After timestamp + HMAC validation...
// Check if this exact call was already processed
const existing = await payload.find({
collection: 'processed-webhooks',
where: { callId: { equals: callId } },
limit: 1,
})
if (existing.docs.length > 0) {
return Response.json({ status: 'duplicate' }, { status: 200 })
// Return 200 (not 4xx) to prevent the sender retrying
}
// Process the webhook...
// Record the nonce AFTER successful processing
await payload.create({
collection: 'processed-webhooks',
data: { callId, processedAt: new Date().toISOString() },
})
```
### Alternative — Per-Phone Nonce with TTL
If a separate `processed-webhooks` collection is too heavy, use a Redis SET with TTL:
```ts
const nonce = `webhook:${callId}`
const alreadySeen = await redis.set(nonce, '1', 'EX', 300, 'NX')
// NX = only set if not exists, returns null if already set
if (alreadySeen === null) {
return Response.json({ status: 'duplicate' }, { status: 200 })
}
```
## Vulnerability Without These Measures
| Missing defense | Attack vector |
|----------------|---------------|
| No timestamp check | Attacker stores a signed request, replays it days later |
| No deduplication | Attacker replays a fresh request N times in the valid window |
| Both missing | Any captured request can be replayed indefinitely |
## HMAC Is Necessary But Not Sufficient
HMAC proves the request came from someone with the secret. It does NOT prove the request is being received for the first time. Replay prevention is always layered on top of HMAC, not a replacement.
## Standard Timestamp Header Pattern (Stripe-style)
Many providers (Stripe, GitHub, Binotel) include the timestamp in:
- A dedicated header: `Stripe-Signature: t=1234567890,v1=abc...`
- The JSON payload body
The HMAC is computed over `timestamp + "." + raw_body` so the timestamp is signed along with the payload. This prevents an attacker from modifying the timestamp on a captured request.
## See Also
- [[wiki/concepts/payload-cms-overrideaccess-bypass]] — must fix access control to prevent exploits even if replay prevention is bypassed
**Source:** daily/2026-05-09.md | 2026-05-09

View file

@ -1,6 +1,21 @@
# Build Log
## [2026-05-10T12:00:00+02:00] compile | daily/2026-05-09.md
- Source: daily/2026-05-09.md
- Articles created (8):
- [[wiki/concepts/payload-cms-overrideaccess-bypass]] — `overrideAccess: true` bypasses all Payload access control; critical anti-pattern in webhook handlers
- [[wiki/concepts/nextjs-unstable-cache-force-dynamic]] — `unstable_cache` + tag revalidation vs `force-dynamic`; ISR for shared rarely-changing data
- [[wiki/concepts/map-ratelimiter-memory-leak]] — in-memory Map rate limiters grow unbounded; `setInterval` eviction with `.unref()`
- [[wiki/concepts/vitest-module-level-env-testing]] — `vi.stubEnv` + `vi.resetModules()` + dynamic `await import()` for env-at-load-time modules
- [[wiki/concepts/webhook-replay-attack-prevention]] — timestamp ±5min validation + callId deduplication to prevent replay attacks
- [[wiki/concepts/secrets-git-history-purge]] — `git-filter-repo --invert-paths --path .env` permanent secret removal; force push + re-clone required
- [[wiki/concepts/payload-cms-push-dev-prod]] — conditional `push: NODE_ENV === 'development'`; `push: false` default silently breaks fresh DB
- [[wiki/concepts/cloudflare-proxied-domain-npm-subpath]] — Cloudflare-only domains have no NPM vhost; Custom Locations fail; separate subdomain required
- Articles updated: none
- Index updates: [[wiki/concepts/_index]] (166→174, +8 entries); [[wiki/_master-index]] (concepts 166→174)
- Session coverage: Nextcloud signaling server config path mismatch, NPM Custom Location limitation for Cloudflare-proxied domains, Payload CMS security audit (overrideAccess, secrets in git, HMAC logging, replay attacks), Payload CMS performance (force-dynamic, N+1 queries, O(N×M) map lookup, rate limiter leak), Vitest env-at-load-time testing patterns, docker-compose credential mismatch, payload push:false gotcha
## [2026-05-10T23:00:00+02:00] compile | daily/2026-05-03.md (addendum — Glance widgets + dual AdGuard)
- Source: daily/2026-05-03.md
- Articles created: [[wiki/concepts/adguard-parallel-instances-diagnostic]]