V1's running deployment at /opt/social-reporting on the server stays put
until cutover; V1's source is preserved on the v1-archive branch and via
git history. From this commit forward, all work targets v2/.
The new root README points contributors at v2/ and documents the rollback
path (deploy/rollback-to-v1.sh) for the cutover.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V2 lives entirely under v2/ and is built around three asks the team raised
about V1: per-video assets sometimes drifted onto the wrong trend, hashtag
scrapes returned junk that wasn't filterable per-client, and there was no
multi-user model behind Microsoft SSO.
Highlights:
- Stable TikTok numeric-id key for every per-video asset; URL form drift is
logged loudly to drift_log.jsonl and never silently nulls assets. Stage 5
manifest hard-gates Stage 6 if any selected video is missing any required
asset; --drop-failing auto-backfills from the next-best recipe candidates.
- Per-brief engagement floor (min_likes / min_plays / min_stl_pct), applied
at Apify scrape time and re-validated locally; spend_log.json records
raw_returned vs kept_after_floor per scrape.
- Users + teams + memberships with owner/admin/editor/viewer roles; SSO
upserts a user keyed on Azure oid, auto-creates a personal team, and a
super-admin is bootstrapped via BOOTSTRAP_SUPER_ADMIN_EMAIL on first
sign-in. Phase A integration test: 16/16 pass.
- 10-stage TS pipeline (brief → seed → scrape1 → select → scrape2 →
validate → analyse → insights → trends → qa → build) wired through one
CLI; each stage idempotent + resumable from disk via .state sentinels.
§4.5 rubrics shipped under prompts/ and loaded into Claude calls.
- React 18 + Vite + TS + Tailwind operator SPA: brief intake form,
team management, super-admin user list, help/FAQ ported from V1.
- Separate Docker Compose project (name: social-reporting-v2, port 3457,
Postgres 5437) with deploy/setup-v2.sh, deploy-v2.sh, rollback-to-v1.sh
scripts that take over V1's /social-reports URL and let us roll back.
Verification: 62 unit tests pass (auth/session, ids extractor with full URL
fixture, engagement floor, recipes, manifest, linking-fix, MoM compare).
Live smoke run on a Dove brief: 1400 raw → 253 kept (82% culled) → 21
fully-bundled videos → 25 editorial trends across 8 brief-driven categories,
with drift=0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Replace cp frontend/* with cp -r frontend/. to copy all files reliably
- Add mkdir -p as safety net in deploy.sh
- Add apache2 reload after frontend copy in deploy.sh
- setup.sh now copies entire frontend dir instead of hardcoded filenames
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Optional textarea lets users provide strategic guidance like objectives,
competitive context, and focus areas. Injected into Claude prompts at
stages 2, 4, 6, and 8 so all agents can produce more focused output.
Backward compatible — empty context changes nothing.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add apifyBudget field to ClientBrief (default $10)
- Budget split: 70% discovery (evenly across platforms), 30% enrichment
- Per-platform soft cap prevents one platform hogging the budget
- Budget input field added to both frontend and dashboard forms
- Saved briefs preserve budget setting
- Fix Claude Vision 5MB limit: filter oversized thumbnails before batching
- Fix Docker: ensure node user can write to volume-mounted dirs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix TikTok comments actor input: `videos` → `videoUrls` (wrong field name)
- Fix TikTok transcripts actor input: `videos` → `videoUrls` (wrong field name)
- Allow HTTP URLs for thumbnails (TikTok CDN uses HTTP)
- Add date filtering to profile scrapers (TikTok + Instagram)
- Keep videos with unparseable dates instead of dropping them
- Lower visual language threshold from 5 to 3 thumbnails
- Increase thumbnail timeout from 5s to 10s
- Add logging for failed thumbnail downloads
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Remove Desk Research (Stage 7 skipped, sources removed from report)
2. Fix comments scraping: increase cap to 2000, handle alt field names
3. Dynamic stats bar: hide zero-value stats instead of showing "0 Comments"
4. Prompt improvements: enforce timeliness, comment-based insights, creator spotlight algorithm (2-10 videos, exclude >50% dominance)
5. Date filtering: pass date params to Apify actors (oldestCreateTime, onlyPostsNewerThan, uploadDate) + log filter counts
6. Pullquotes: 3-4 generated editorial dividers between sections
7. Thumbnails: download top 50 coverUrl as base64, store on EnrichedVideo
8. Visual Language section: 5 batches of 10 through Claude Vision, synthesized into 5-6 visual codes with thumbnail cards
9. Sticky navigation bar with anchor links to all sections
10. New types: VisualCode, thumbnailUrl on Video, thumbnailBase64 on EnrichedVideo, pullquotes/visualCodes on ReportJSON
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fixed null reference error when loading JSON files (removed deleted jsonPreview element refs)
- Added Export button to download saved briefs as JSON files
- Renamed "Load & Run" to just "Load" per user feedback
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Backend: GET/POST/DELETE /api/briefs endpoints storing JSON files in briefs/ dir
- Frontend: new Saved Briefs tab with cards showing client details, Load & Run, Delete
- Save Current Brief button on Pipeline tab persists form to server
- Both standalone dashboard and static frontend updated
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Delete individual runs (with confirmation)
- Bulk remove all failed or completed runs
- Download report as HTML file (Content-Disposition: attachment)
- View + Download buttons in history table
- Backend: DELETE /api/runs/:id and DELETE /api/runs?status=failed|completed
- Backend: GET /report/:id/download serves with attachment header
- Updated both frontend/index.html and dashboard/index.html
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
ProxyTimeout is not allowed in <Location> context. Moved to server-level
ProxyTimeout directive already set above.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
EventSource auto-reconnects on connection drop, which re-fires the
'connected' event. The handler was POSTing /run on every reconnect,
causing multiple parallel pipeline runs and runaway Apify costs.
Added pipelineStarted guard so /run only fires on first connect.
Fixed in both frontend/index.html and dashboard/index.html.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- TikTok transcripts/comments actor expects 'videos' not 'videoUrls'
- Wrap all enrichment actor calls in safeRunActor so failures skip
instead of crashing the pipeline
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Promise.all() launched all platform scrapers simultaneously, so multiple
expensive runs started before any costs were tracked. Budget check only
saw totals after each run finished, allowing $7+ overspend on a $5 limit.
Now Stage 3 and Stage 5 run each scraper sequentially so the budget
gate can cut off between calls.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
DB_PORT defaults to 5436, DASHBOARD_PORT defaults to 3456.
Prod override no longer redeclares ports (was causing duplicates).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Static frontend (index.html, login.html, config.js) for Apache serving
- JSON-based auth API endpoints (/api/login, /api/auth, /api/logout)
- Apache config with ProxyPass for /social-reports path
- deploy/setup.sh for Ubuntu + Apache + Docker deployment
- docker-compose.prod.yml binds ports to 127.0.0.1 only
- Configurable API base URL via frontend/config.js
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Cookie-based session auth with login page (DASH_USER/DASH_PASS env vars)
- Serve generated reports via /report/:id route with View Report button
- YouTube iframe and Instagram native embeds in HTML reports
- Supporting videos grid per trend with platform icons
- Logout link in dashboard header
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
8-stage TypeScript pipeline with Apify scraping, Claude AI analysis,
real-time dashboard with SSE, PostgreSQL cost tracking, and Apify budget controls.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
