The previous BACKEND_DIR_V2=/opt/social-reporting-v2 hardcode failed on
the production host because we did an in-place cutover: V1 and V2 share
a single checkout at /opt/social-reporting. ./v2/deploy/deploy-v2.sh
exited immediately with "V2 dir not found".
Both deploy-v2.sh and rollback-to-v1.sh now derive their repo root from
the script's own location, so they work whatever the parent directory
is named. setup-v2.sh is left alone — it's the first-time provisioner
and creating /opt/social-reporting-v2 from scratch is still a fine
default for greenfield installs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The V2 container runs as uid 1000 (Dockerfile.v2:39 USER 1000). When the
pipeline tried to mkdir briefs/<brief-id>/ on the production server, it hit
EACCES because the host directory was owned by root (V1 had a similar fix
in deploy.sh that we never ported).
- cutover-in-place.sh: chown -R 1000:1000 briefs/ before docker up.
- deploy-v2.sh: same chown on every redeploy + use --env-file (was missing).
Immediate manual fix on the running server (until next deploy):
sudo chown -R 1000:1000 /opt/social-reporting/briefs
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
enable mp4 download, smart Stage 4 cache, default Apify live in prod
Closes the last gaps so the operator app is end-to-end usable in production.
Server:
- routes/reports.ts: GET /api/reports/:id/dashboard[/<file>] serves files
out of the report's brief outputs/ tree (HTML bundle, dataset_v2.json,
any covers referenced relatively). Auth-gated by team viewer role.
Path-traversal guarded.
- index.ts: two new route patterns (with and without trailing path).
Client:
- routes/reports/detail.tsx: "Open dashboard" is a target=_blank anchor at
/api/reports/:id/dashboard/, "Download" same URL with download attribute.
No more dead SPA-internal link.
Pipeline polish (the four open items from the smoke test):
- stage_10_build.ts: covers are now downscaled via ffmpeg (240px / q=6)
before base64 inlining. Hard ceiling per cover 60 KB; falls back to the
original only if it already fits. Honours V3 brief's ≤3 MB HTML bundle.
- lib/apify_client.ts: post-run cost is re-polled with backoff (0/5/15/30s)
instead of a single read. TIKTOK_COMMENTS reports $0 immediately and
$5+ later — without this the soft cap can't fire on it.
- stage_2_pass1_scrape.ts: shouldDownloadVideos:true (and shouldDownloadCovers:true)
by default so videoMeta.downloadAddr is populated for Stage 4 frame
extraction. Disable with DISABLE_VIDEO_DOWNLOADS=true if the budget is
tight.
- stage_4_pass2_enrich.ts: Stage 5 backfill candidates aren't in the
transcripts/comments cache. New loadOrFetchActor() reads what's cached,
identifies missing ids, fetches just those from Apify, and merges back
into the cache. Backfill no longer drops every candidate.
Production defaults:
- .env.example: APIFY_LIVE_APPROVED=true (commented; operators can flip
to false for dry-runs).
- cutover-in-place.sh: sets APIFY_LIVE_APPROVED=true if not already in .env
after the migration step, so a fresh prod cutover doesn't accidentally
dry-run.
62/62 unit tests pass; tsc + vite build green; bundle 269 kB.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two issues from the first server cutover:
1. SPA loaded white-on-black blank because import.meta.env.VITE_AZURE_TENANT_ID
and VITE_AZURE_CLIENT_ID were undefined at runtime. Vite reads VITE_* at
*build time* and inlines them into the bundle; passing them only as
runtime container env vars is too late.
- Dockerfile.v2: declare ARG VITE_AZURE_TENANT_ID, VITE_AZURE_CLIENT_ID,
VITE_BASE; export as ENV before `npm run build`.
- docker-compose.v2.yml: forward AZURE_TENANT_ID / AZURE_CLIENT_ID /
VITE_BASE through `build.args` so the cutover .env values reach Vite.
2. cutover-in-place.sh stopped V1 with `-p social-listening`, but V1's actual
compose project name was `social-reporting` (parent dir). Old V1 containers
were left running. Now we try both project names AND sweep by container
name pattern (anything matching social-listening or social-reporting-db-1
that isn't a V2 container).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The host port 5437 on the optical-dev server was already allocated by
something (probably an old stopped-but-not-removed Postgres container
or another tracking app). V2 doesn't need a host port for db-v2 in
production — app-v2 reaches it over the docker network at db-v2:5432.
Per CLAUDE.md "always check for ports that are already used":
- docker-compose.v2.yml: remove the unconditional db-v2 host port
binding. Compose's list-merge semantics meant `ports: []` in the prod
override didn't actually clear the base list.
- docker-compose.v2.dev.yml (new): local-dev overlay that re-adds the
host port for psql convenience. Use with `-f base -f dev`. Bound to
127.0.0.1 so the db is never reachable from outside the dev machine.
- cutover-in-place.sh: pre-flight check on APP_V2_PORT (3457) — if
it's held by something other than our own V2 container, abort with
a clear message rather than failing mid-deploy.
Verified locally: `compose -f base -f prod up -d` brings up a stack
with db-v2 having no host port (just internal 5432/tcp), app-v2 on
127.0.0.1:3457, /api/health returns {ok:true}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase A scaffolded the SPA at the bare origin (`/`); production lives behind
Apache at `/social-reports/`. Without these fixes, V2's built assets 404 and
Azure SSO rejects the redirect URI mismatch.
- Vite `base: /social-reports/` (overridable via VITE_BASE for dev).
- BrowserRouter basename = import.meta.env.BASE_URL.
- apiFetch + msal-browser script src + token-exchange URL all prefix BASE.
- MSAL redirectUri now matches V1's Azure-registered URI:
`${origin}/social-reports/login.html`.
- New `<Route path="/login.html">` alias renders the same Login component
so React Router matches the redirect URI when MSAL returns.
Deploy ergonomics (the user wants V1 gone from the server):
- v2/deploy/cutover-in-place.sh: run from /opt/social-reporting; stops V1,
pulls main (v2/ appears, V1 dirs deleted), migrates secrets from V1's
.env into v2/.env, swaps Apache, starts V2. Single command, no clone of
a sibling dir needed.
- setup-v2.sh: PURGE_V1=true flag now cleans /opt/social-reporting and
the V1 docker volume after V2 is healthy.
- rollback-to-v1.sh: re-clones the v1-archive branch when V1 is no longer
on disk (REPO_URL required).
62/62 unit tests still pass; vite build emits assets under /social-reports/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V2 lives entirely under v2/ and is built around three asks the team raised
about V1: per-video assets sometimes drifted onto the wrong trend, hashtag
scrapes returned junk that wasn't filterable per-client, and there was no
multi-user model behind Microsoft SSO.
Highlights:
- Stable TikTok numeric-id key for every per-video asset; URL form drift is
logged loudly to drift_log.jsonl and never silently nulls assets. Stage 5
manifest hard-gates Stage 6 if any selected video is missing any required
asset; --drop-failing auto-backfills from the next-best recipe candidates.
- Per-brief engagement floor (min_likes / min_plays / min_stl_pct), applied
at Apify scrape time and re-validated locally; spend_log.json records
raw_returned vs kept_after_floor per scrape.
- Users + teams + memberships with owner/admin/editor/viewer roles; SSO
upserts a user keyed on Azure oid, auto-creates a personal team, and a
super-admin is bootstrapped via BOOTSTRAP_SUPER_ADMIN_EMAIL on first
sign-in. Phase A integration test: 16/16 pass.
- 10-stage TS pipeline (brief → seed → scrape1 → select → scrape2 →
validate → analyse → insights → trends → qa → build) wired through one
CLI; each stage idempotent + resumable from disk via .state sentinels.
§4.5 rubrics shipped under prompts/ and loaded into Claude calls.
- React 18 + Vite + TS + Tailwind operator SPA: brief intake form,
team management, super-admin user list, help/FAQ ported from V1.
- Separate Docker Compose project (name: social-reporting-v2, port 3457,
Postgres 5437) with deploy/setup-v2.sh, deploy-v2.sh, rollback-to-v1.sh
scripts that take over V1's /social-reports URL and let us roll back.
Verification: 62 unit tests pass (auth/session, ids extractor with full URL
fixture, engagement floor, recipes, manifest, linking-fix, MoM compare).
Live smoke run on a Dove brief: 1400 raw → 253 kept (82% culled) → 21
fully-bundled videos → 25 editorial trends across 8 brief-driven categories,
with drift=0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
