Commit graph

82 commits

Author SHA1 Message Date
Vadym Samoilenko
b3ace22009 feat(infra): move heavy workers to Cloud Run Jobs
Heavy pipeline tasks (ingest, translate, render, rerender) now dispatch
to a Cloud Run Job (va-worker) instead of local Celery workers. optical-dev
runs only api + lightweight worker (notify/embed) within its 2-CPU budget.

- backend/app/tasks/runner.py — Cloud Run Job entrypoint
- backend/app/services/cloud_run_dispatch.py — replaces .delay() for heavy tasks
- backend/Dockerfile.cloudrun — Cloud Run worker image (ffmpeg included)
- docker-compose.optical-dev.yml — 2-CPU safe overrides, disables heavy workers
- cloudbuild.yaml — builds va-worker image and updates Cloud Run Job
- deploy-dev.sh — uses 3-file compose, builds only api+worker locally
- routes_jobs, routes_admin_production, ingest_and_ai, translate_and_synthesize
  — all dispatch sites updated to use cloud_run_dispatch.dispatch()

USE_CELERY_FALLBACK=true in .env.local to use Celery locally during dev.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:47:10 +01:00
Vadym Samoilenko
4623b89aeb feat(mt-16): JWT org_ids claim + transient user.org_ids in deps
- create_access_token gains optional org_ids: list[str] param; encodes
  {exp, sub, org_ids, v:2} — org_ids is a prefilter hint only, never
  used as authorization source of truth (Redis cache is authoritative)
- Login, MS login, refresh endpoints: fetch memberships and include
  org_ids in issued access tokens via _get_user_org_ids() helper
- routes_invitations.py accept flow: same org_ids population on token
- get_current_user: reads org_ids from payload, attaches as transient
  user.__dict__["org_ids"] — available to OrgScopedQuery for prefilter
- Force logout: rotate JWT_SECRET env var at deployment time (no code
  change needed; all existing tokens immediately invalidated)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:46:39 +01:00
Vadym Samoilenko
54fcf47887 feat(mt-14): gcs_prefix on Job, gcs_path helper, rewrite path sites
- gcs_path(job, *parts) helper in gcs.py: uses job.gcs_prefix if set,
  falls back to job._id (legacy) — backward-compatible for all old jobs
- create_job: sets gcs_prefix=orgs/{org_id}/jobs/{job_id} when
  organization_id is known; legacy jobs without org get null prefix
- Rewrote hardcoded f"{job_id}/{lang}/..." paths in:
  - ingest_and_ai.py (4 upload sites)
  - translate_and_synthesize.py (9 sites via bulk regex)
  - render_accessible_video.py (3 sites: segments, video, captions)
  - rerender_accessible_video.py (3 sites)
- tools/migrate_gcs_org_prefix.py: idempotent operator script —
  preflight checks, copy→verify(count+md5)→mongo update→delete,
  ThreadPoolExecutor(4), resume file, dry-run + rollback modes

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:45:12 +01:00
Vadym Samoilenko
595897e61a feat(w-12): JobBrief model, endpoints, migration + brief→job linkage
- JobBrief model (DRAFT→SUBMITTED→APPROVED→FULFILLED) with 6 CRUD
  endpoints: list, create, get, patch (DRAFT only), submit, approve
- All endpoints use MembershipContext; read=VIEWER, mutate=MANAGER,
  approve=ADMIN for org-scoped access
- create_job accepts brief_id Form field; validates APPROVED brief,
  copies organization_id/project_id/deadline from brief, marks brief
  FULFILLED after job insert
- organization_id now populated from project client_id on job create
  (fixes missing multi-tenant field on new jobs)
- migration_2026-04-29-000001: job_briefs collection + 4 indexes
- Wired briefs router into main.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:38:08 +01:00
Vadym Samoilenko
a945653e73 feat(w-14): bulk failures dashboard + sidebar badge
- GET /admin/production/failures: list failed jobs filtered by step/org
- POST /admin/production/bulk-retry: dispatch retry for up to 50 jobs
  with "auto" (from failure.step) or "from_scratch" strategies
- FailuresList.tsx: accordion-grouped by error type, multi-select,
  bulk retry action, step label, retry count (red >3), updated date
- Sidebar: "Failures" item with live badge for production/admin roles
  (polls useJobs with processing_failed,tts_failed,render_failed)
- New useFailures / useBulkRetry hooks

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:36:30 +01:00
Vadym Samoilenko
264561895e feat(w-13): generic /jobs/{id}/retry endpoint + unified failure UI
- POST /jobs/{job_id}/retry dispatches correct pipeline task based on
  failure.step: ingestion/ai_processing → ingest_and_ai_task,
  translation/tts → translate_and_synthesize_task, render → rerender
- Increments retry_count, writes JOB_RETRY audit log entry
- Adds processing_failed to JobStatus type; JobFailure interface on Job
- Replaces TTS-only retry block with FailureBanner showing step/message/
  retry_count for all failed statuses (processing_failed, tts_failed,
  render_failed); Escalate mailto link for high-retry-count cases
- useRetryJob hook + apiClient.retryJob() call new endpoint

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:33:50 +01:00
Vadym Samoilenko
38038862c9 refactor(mt-15): consolidate authz in routes_jobs and dependencies
list_jobs now uses MembershipContext (Redis-cached, 60s TTL) to build
org-scoped queries instead of per-request memberships.find(). Falls back
to legacy get_accessible_project_ids for users with no memberships.

get_job replaces the role-specific CLIENT/PM access check with
get_job_or_403() which uniformly checks organization_id membership for
all roles (returns 404 not 403 to avoid leaking cross-org job existence).

get_accessible_project_ids in dependencies.py now uses _cached_memberships
from authz.py, eliminating the duplicate uncached DB query.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:26:07 +01:00
Vadym Samoilenko
5209f04318 feat(mt-13): bind glossary handlers to client_id via org membership check
All 8 glossary route handlers now verify the requesting user has org
membership in the target client_id using assert_user_in_org() from
core/authz.py. Read endpoints require VIEWER, mutations require MANAGER,
archive requires ADMIN (org-level). Removed dead _assert_can_read() and
_require_client_staff() helpers. Removed unused require_roles/User/UserRole
imports. Also added get_job_or_403() to authz.py for MT-15.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:24:41 +01:00
Vadym Samoilenko
b2d524e702 fix(mt-12): remove PM/CLIENT legacy bypass in _assert_client_access
The unconditional `if user.role in (CLIENT, PROJECT_MANAGER): return`
allowed any PM to access any client regardless of membership. Removed;
kept pm_client_ids legacy fallback for pre-migration users.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:22:56 +01:00
Vadym Samoilenko
08fcb4daa4 feat(pr6): WS real-time updates, per-cue AD playback, upload guard
W-4: team assignment (linguist/reviewer) stored on job at creation,
     auto-assigned to all language QC states on first GET /language-qc
     (lazy init via auto_assign_defaults)

L-3 WS: broadcast_to_job when reviewer opens VTT for editing;
        QCDetail shows "User X is editing [lang]" banner (auto-clears 5s)

R-5: comment broadcast via broadcast_to_job on add_comment();
     QCDetail invalidates comments query on language_qc_comment WS event

L-15: QCDetail subscribes to language_qc_assigned WS event →
      refetches lang-qc data and shows toast

R-7: VttEditor gets onCuePlay prop; AD editor in QCDetail wires
     handleAdCuePlay → switches to accessible video mode, seeks & plays

T-15: beforeunload warning in NewJob while upload is in progress

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:42:57 +01:00
Vadym Samoilenko
bdfa0f82ab fix(lint): restore baseline lint count — no new errors introduced
QCDetail.tsx: 4 new `any` types replaced with `unknown` + type casts.
backend: ruff auto-fix sorted imports, removed unused imports, updated Optional[X] → X | None in routes_share + share_token model.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:16:35 +01:00
Vadym Samoilenko
1317ee7ca4 feat(t6+t7+t11): native captions track, AD audio sync, CSRF protection
T-6: Add Blob URL native <track> in VideoWithCaptions so browser CC button works in fullscreen.
T-7: Sync hidden <audio> AD playback with video play/pause/seeked events.
T-11: Double Submit Cookie CSRF — _set_auth_cookies issues httponly refresh_token + readable csrf_token; /refresh validates X-CSRF-Token header; frontend reads csrf_token cookie and sends header on all refresh calls.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:08:27 +01:00
Vadym Samoilenko
dc1cfd01dc feat(l3): optimistic locking for VTT edits (ETag / 409 Conflict)
Backend:
- VttContentResponse gets etag field (SHA1 of captions+AD content)
- VttUpdateRequest gets if_match field (optional)
- GET /jobs/{id}/vtt: computes and returns etag
- PATCH /jobs/{id}/vtt: if if_match present, fetches current content, recomputes
  hash, returns 409 Conflict if mismatch

Frontend:
- VttContentResponse type + VttUpdateRequest type updated
- QCDetail stores vttEtag from GET response
- All updateVttMutation calls pass if_match: vttEtag
- 409 responses show specific "Conflict: another user has modified" message

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:01:57 +01:00
Vadym Samoilenko
abf81515a4 feat(pm15): share read-only link for client preview
Backend:
- ShareToken model (share_tokens collection)
- POST /jobs/{id}/share — create token (PM/PROD/ADMIN)
- GET /jobs/{id}/share — list active tokens
- DELETE /jobs/{id}/share/{token_id} — revoke token
- GET /public/share/{token} — unauthenticated preview with signed GCS URLs (6h TTL)
  Returns video, captions, AD for all languages

Frontend:
- ShareView.tsx — public page at /share/:token with language switcher, video player, download tiles
- App.tsx — /share/:token route (no auth wrapper)
- QCDetail.tsx — "↗ Share link" button in header → modal to generate + copy link

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 18:56:44 +01:00
Vadym Samoilenko
f1a9e6ee46 feat(pm7): bulk assign linguist/reviewer to all languages in one click
- POST /jobs/{job_id}/languages/bulk-assign — assigns linguist (required) and
  reviewer (optional) across all or selected languages; supports only_unassigned
  flag and optional deadline
- bulkAssignLanguages() added to API client
- QCDetail: "Assign all languages" button in Languages header; opens modal with
  linguist/reviewer dropdowns, deadline, and skip-already-assigned checkbox

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 18:53:14 +01:00
Vadym Samoilenko
13db347d65 feat(pr3+pr4): deadline field, job clone, reject categories, reviewed-cues gate
PM-1 (deadline):
- Job model: add deadline field (job-level PM deadline)
- POST /jobs: accept deadline as ISO date form param
- JobsList: deadline column with overdue highlight (red + warning icon)
- NewJob: date picker for deadline field
- useMultiUpload: pass deadline to batch job creation

PM-2 (clone job):
- POST /jobs/{id}/clone: creates config copy in 'created' state, no reupload
- useCloneJob hook, Clone button in JobsList actions
- navigate to cloned job on success

R-4 (reject categories):
- LanguageQCState: add reject_category field
- reject_language service: accept optional category (timing/mistranslation/terminology/profanity/length/other)
- RejectLanguageRequest: add category field
- QCDetail reject modal: category pill-selector before free-text notes

R-2 (reviewed-cues tracking):
- LanguageQCState: add reviewed_cues (int) + total_cues (nullable)
- POST /jobs/{id}/languages/{lang}/mark-cue-reviewed endpoint
- QCDetail: progress bar + approve gated at 80% for reviewer (admin bypasses)
- markCueReviewed API client method

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 18:39:05 +01:00
Vadym Samoilenko
c7a6f13b10 feat(workflow): PR-2 workflow blockers — PM/Production dashboards, two-stage QC, role routing
Changes:
- Dashboard: add project_manager case (final review / QC counts / new job widgets)
  and production case (AI pipeline / failures widgets)
- Sidebar: add project_manager to Final Review and Audit Log nav items;
  live badge counts for QC Queue (pending_qc) and Final Review (pending_final_review)
- App.tsx: add project_manager to Final Review and Audit Log RoleGates (W-10, PM-18)
- Login: role-based redirect after login — linguist/reviewer → /qc/queue, others → /
- language_qc._assert_can_approve: enforce two-stage QC; remove linguist self-approve
  fallback; require reviewer assignment + submitted_for_review_at (W-6)
- routes_jobs.complete_job: allow project_manager to complete jobs (W-9)
- notify.py: re-enable email notifications (W-7)
- Fix 400 on cue save: treat empty-string audio_description_vtt/captions_vtt as absent
  both in backend (truthy check) and frontend (|| undefined) — root cause was adVtt
  initialising to '' when job has no AD track

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 18:18:24 +01:00
Vadym Samoilenko
a168af1aa7 feat: two-stage QC (linguist→reviewer), project picker, comments, email notifications, deadlines
- Two-stage QC workflow: linguist edits + submits → reviewer approves/rejects per language.
  New statuses: in_progress, pending_review, in_review. New service functions: submit_for_review,
  open_review, assign_reviewer, reassign_reviewer, add_comment. Linguist and reviewer deadlines.
- Reject now resets language to in_progress so linguist can iterate without full re-assignment.
- QC comment threads per language (append-only), visible to all assignees.
- Email notifications via Mailgun on: assignment, submit-for-review, comment, approve, reject.
  Best-effort (failures do not roll back QC actions). asyncio.gather for parallel fan-out.
- New audit actions: LANGUAGE_QC_REVIEWER_ASSIGN/REASSIGN, LANGUAGE_QC_SUBMIT,
  LANGUAGE_QC_OPEN_REVIEW, LANGUAGE_QC_COMMENT.
- Inline project picker in NewJob: "+ Create new project…" option with name, default
  languages, default linguist, default reviewer. Pre-fills languages on the new job.
- Project model extended with default_languages, default_linguist_id, default_reviewer_id.
- RBAC: CLIENT org-members can now create projects (backend guard relaxed).
- LinguistQueue: role toggle "As linguist / As reviewer" + new status tabs.
- QCDetail: two-slot assignment cards (linguist + reviewer), deadline display, role-aware
  action buttons, comments panel with optimistic insert and 15s refetch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 16:59:40 +01:00
Vadym Samoilenko
be0bffe459 fix: get_terms_page avoids GlossaryTerm validation on partial projection
Projected docs only have _id/source_term/translations; validating against
GlossaryTerm (which requires glossary_id, version_id, source_term_lower)
caused 500 on the terms endpoint. Return plain dicts instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:57:12 +01:00
Vadym Samoilenko
125c69fb1d fix: audit log user/security endpoints return correct shapes
- /audit-logs/user/{id}: now accepts email OR ObjectId, returns bare array
- /audit-logs/security: returns bare array instead of {logs, hours} wrapper
  Both match AuditLogEntry[] that the frontend expects.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:48:00 +01:00
Vadym Samoilenko
ad67089b09 fix: remove duplicate /audit-logs route and align pagination params with frontend
The legacy GET /audit-logs (returning wrong shape) shadowed the proper one.
Removed the duplicate and changed page/size params to skip/limit to match
the AuditLogQuery the frontend sends.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:39:22 +01:00
Vadym Samoilenko
dee4d69b40 fix: raise user list size limit to 500 and guard toLocaleString calls
- routes_admin.py: size query param max raised from 100 → 500 so
  ClientDetail.tsx (size=200) no longer returns 422
- GlossaryDetail.tsx: three .toLocaleString() calls guarded with ?? 0
  to prevent TypeError when term_count is undefined on first render

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:20:38 +01:00
Vadym Samoilenko
a3b300b76a docs: add canonical documentation + audit cleanup
- AGENTS.md: canonical project entry point (Quick Nav, pipeline, constraints)
- docs/: complete docs tree — architecture, API spec, DB schema, infra,
  runbook, requirements, tech stack, principles, reference ADRs, guides,
  tasks backlog, testing strategy
- tests/README.md: test commands, structure, known gaps
- README.md / CLAUDE.md / DEPLOYMENT.md: updated with canonical doc links
- .archive/: backup of pre-documentation-pipeline originals
- backend/uv.lock: uv dependency lockfile
- Delete committed __pycache__ .pyc files (should have been gitignored)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:22:51 +01:00
Vadym Samoilenko
4c6624c3d4 fix: code health sweep — M-01 through M-07
M-01 authz.py: move cache_key above try block to avoid NameError when
     first Redis call returns None
M-02 main.py: re-enable validation middleware (was TEMPORARILY DISABLED)
M-03 routes_auth.py / main.py: replace print() debug lines with
     structured logger calls; logger now module-level in routes_auth.py
M-04 gcs.py: asyncio.get_event_loop() → get_running_loop() (deprecation)
M-05 translate_and_synthesize.py: bind loop vars in closure defaults
     to fix B023 ruff warnings (transcreate/translate_captions/etc.)
M-06 rate_limiting.py: only trust X-Forwarded-For when X-Forwarded-Proto
     is https; use rightmost entry (proxy-appended) not leftmost
M-07 validation.py: extend MongoDB operator blocklist to cover $expr,
     $function, $accumulator, $nin, $gte, $lte, $jsonSchema, $mod

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:18:02 +01:00
Vadym Samoilenko
87ae6571fe perf: use DI connection pool for auth routes, async httpx for MS SSO (H-01, H-02)
- login and microsoft_login routes now use Depends(get_database) instead
  of creating a per-request MongoClient — removes connection-pool churn
  under load
- MicrosoftAuthService._get_openid_config/_get_jwks/validate_token are
  now async, using httpx.AsyncClient instead of blocking requests.get —
  removes ~400ms event-loop block per Microsoft login
- Removed unused AsyncIOMotorClient import from routes_auth.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:13:50 +01:00
Vadym Samoilenko
e81acebc45 security: remove exception detail from /auth/refresh response (C-03)
Replaced the bare except that leaked str(e) (JWT library internals,
claim validation messages) with a generic "Invalid refresh token" detail.
Full traceback is now logged server-side via the structured logger.
Re-raises HTTPException before the generic handler so valid 401s from
inner checks are not double-wrapped.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:11:59 +01:00
Vadym Samoilenko
fa351e4d25 feat: per-client glossary — hybrid exact/vector retrieval + AI injection
Adds full glossary system so Gemini uses client-approved terminology
when generating subtitles and translations (critical for 3M brand names
and product codes across 16 target locales).

Backend:
- lib/locales.py: BCP-47 locale registry, normalises xlsx fr_fr → fr-FR
- models/glossary.py: Glossary / GlossaryVersion / GlossaryTerm + enums
- services/glossary_service.py: xlsx parse (openpyxl), ingest to Mongo,
  hybrid retrieval (Aho-Corasick exact + Atlas Vector Search), prompt block
- services/embedding_service.py: Gemini text-embedding-004, batch 100, retry
- tasks/embed_glossary.py: Celery background task for async embedding
- api/v1/routes_glossaries.py: CRUD endpoints under /clients/{id}/glossaries
- gemini.py: _build_glossary_block(), {GLOSSARY} injection in all 4 call sites
- tts.py / gemini_tts.py: pass full locale codes (no split("-")[0] truncation)
- tasks/translate_and_synthesize.py: glossary lookup + injection per language
- prompts: {GLOSSARY} placeholder in ingestion, targeted, transcreation prompts
- pyproject.toml: +openpyxl, +pyahocorasick

Frontend:
- routes/admin/glossaries/: GlossaryList, GlossaryUpload, GlossaryDetail
- App.tsx: 3 new routes under /admin/clients/:clientId/glossaries
- ClientDetail.tsx: Glossaries card with count + quick links
- types/api.ts: Glossary, GlossaryVersion, GlossaryDetail, GlossaryTerm types
- lib/api.ts: 7 new API methods (upload, list, detail, terms, versions, activate, archive)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 13:03:38 +01:00
Vadym Samoilenko
05f25a1141 feat: per-language QC workflow with linguist assignment
- Job.language_qc dict tracks per-language status (pending/in_review/approved/rejected)
  with full event history; qc_assignments denormalized array enables efficient queue queries
- language_qc service handles assign/reassign/approve/reject/reopen with atomic DB updates,
  audit logging, and auto-advancement to pending_final_review when all languages approved
- Linguists can only edit VTT and trigger re-renders for their assigned language (403 guard)
- return_to_qc resets all language statuses while preserving assignments
- routes_language_qc.py: 7 new endpoints; /me/language-qc-queue for linguist queue
- Startup migration idempotently seeds language_qc for all existing jobs
- Frontend: LanguageQCState types, API methods, LinguistQueue page, QCDetail redesigned
  with per-language status badges, assignment dropdown, inline approve/reject buttons,
  progress bar, and reject modal; My QC Queue sidebar link

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 12:09:40 +01:00
Vadym Samoilenko
bab30e1508 feat: VTT version control — snapshots, diff, restore
Backend:
- VttVersion model (vtt_version.py): immutable snapshot per job/lang/kind/version
- vtt_versioning service: create_version (atomic counter + GCS snapshot),
  list_versions, get_version, restore_version, diff_versions (difflib line-level)
- routes_vtt_versions.py: GET /versions, GET /versions/{v}, GET /versions/diff,
  POST /versions/{v}/restore (PRODUCTION/ADMIN only, overwrites live file + audit log)
- Hook create_version into update_job_vtt_content before each live-file overwrite
- Mongo indexes: unique (job_id, lang, kind, version) + (job_id, created_at)

Frontend:
- VttVersionSummary / VttVersionFull / VttDiffResponse types
- api.ts: listVttVersions, getVttVersion, diffVttVersions, restoreVttVersion
- VersionsTab.tsx: lang/kind switcher, version list with A/B compare buttons,
  inline diff viewer (color-coded +/−), content viewer, restore with confirm dialog
- JobDetail.tsx: new "VTT Versions" tab wired to VersionsTab

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 11:46:21 +01:00
Vadym Samoilenko
09550cfca0 feat: audit log integration sweep + cost tracker URL fix + audit log admin UI
- Fix cost tracker dashboard URL (cost.oliver.agency → optical-dev.oliver.solutions/cost-tracker/analytics)
  in UserList, QCDetail, FinalDetail; centralise into src/lib/costTracker.ts

- Wire audit logging across backend routes (was 1 call site, now covers all key events):
  · routes_auth: LOGIN_SUCCESS/FAILURE for local + MS SSO, LOGOUT
  · routes_files: FILE_UPLOAD on signed URL generation
  · routes_jobs: JOB_CREATE, JOB_APPROVE, JOB_REJECT, JOB_STATUS_CHANGE, JOB_DELETE, VTT_EDIT
  · routes_admin: USER_CREATE, USER_UPDATE, USER_ROLE_CHANGE, USER_DEACTIVATE

- Add Audit Log admin UI page (/admin/audit-log):
  · Three tabs: All Events (paginated, server-side filters), Security Events, User Activity
  · Filters: action group, severity, success/failure, free-text search
  · Click-to-expand row shows IP, request ID, resource, details JSON
  · Wired into App.tsx (RoleGate: production + admin) and sidebar nav

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 11:34:06 +01:00
Vadym Samoilenko
1563714454 feat(saas): Phase 3 — membership-based authz + Mailgun + job.organization_id
authz.py (new):
- MembershipContext — per-request membership dict for the current user
- get_membership_context FastAPI dependency
- require_org_role(min_role) — dependency factory keyed off org_id path param
- require_platform_admin()
- OrgScopedQuery — adds organization_id filter; platform admin passes through
- bump_user_membership_cache — invalidates Redis key on membership writes

dependencies.py:
- get_accessible_project_ids now queries memberships collection first;
  legacy pm_client_ids / team.member_user_ids fallback retained until migration runs
  (four job-route access checks at lines 608/1054/1181/1538 are fixed via this function)

routes_clients.py:
- _assert_pm_or_admin and _assert_client_access are now async and query memberships
- All 10 call sites updated with await + db arg

emailer.py:
- Switched from SendGrid to Mailgun REST API via httpx (already in requirements)
- _send() is now fully async; same public method signatures preserved
- send_completion_email uses _send()

config.py:
- Added mailgun_api_key, mailgun_domain, mailgun_from settings
- sendgrid_api_key kept with empty default for backward compat

migration_2026-04-28-000003:
- Backfills job.organization_id from project.client_id
- Creates (organization_id, status, created_at) sparse index on jobs

routes_organizations.py / routes_invitations.py:
- Call bump_user_membership_cache after every membership write

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:56:42 +01:00
Vadym Samoilenko
00fb1aacc6 feat(saas): Phase 2 — invitation flow, email templates, MS SSO zero-membership
Backend:
- models/invitation.py — Invitation model + create/accept/preview schemas
- routes_invitations.py — org-scoped POST/GET/DELETE + public preview/accept endpoints
  Single-use token via find_one_and_update; sha256(token) stored in DB, plaintext in email URL
- emailer.py — _send() helper; send_invitation_email, send_welcome_email, send_password_reset_email
  send_completion_email refactored to use _send()
- migration_2026-04-28-000002 — creates invitations collection with TTL index (30d audit trail)
- routes_auth.py — new MS SSO users provisioned with zero memberships instead of role=PRODUCTION;
  they land on "no access" page until an admin invites them
- main.py — registers invitations_org_router and invitations_router

Frontend:
- routes/AcceptInvite.tsx — public page at /accept-invite?token=...
  Four states: new user (name+password), existing user (confirm), MS user, already-member
- App.tsx — /accept-invite route outside RequireAuth
- types/api.ts — Invitation, InvitationCreate, InvitationPreview, InvitationAcceptRequest/Response
- lib/api.ts — listInvitations, createInvitation, revokeInvitation, previewInvitation, acceptInvitation
- hooks/useClients.ts — useInvitations, useCreateInvitation, useRevokeInvitation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:52:08 +01:00
Vadym Samoilenko
6f1be645ce feat(saas): Phase 0+1 — Organization/Membership entities and dev branch
Introduces the multi-tenant SaaS foundation alongside the existing
client/team/project model (zero-downtime shim period):

Backend:
- app/models/organization.py — Organization + OrgRole enum (OWNER/ADMIN/MANAGER/MEMBER/VIEWER)
- app/models/membership.py — Membership model with MemberDetail for enriched responses
- app/services/membership_service.py — upsert/remove/list/has_org_role helpers
- app/api/v1/routes_organizations.py — /organizations CRUD + /members sub-resource + /me/memberships
- main.py — registers organizations router
- migrations: create memberships collection (unique index) + backfill from pm_client_ids/team members

Frontend:
- types/api.ts — Organization, OrgRole, Membership, OrganizationCreateRequest types; Client marked @deprecated
- hooks/useClients.ts — useOrganizations, useOrganization, useOrgMembers, useAddOrgMember,
  useUpdateOrgMember, useRemoveOrgMember, useMyMemberships
- lib/api.ts — listOrganizations, getOrganization, createOrganization, updateOrganization,
  listOrgMembers, addOrgMember, updateOrgMember, removeOrgMember, getMyMemberships

Reads fall back to the clients collection during transition; all writes go to organizations.
Existing /clients endpoints and hooks are untouched.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:46:24 +01:00
Vadym Samoilenko
269ab09fa6 fix: serialize Client/Team/Project with id not _id + guard undefined client hooks
Pydantic v2 + FastAPI serializes Field(alias="_id") as _id in JSON,
so client.id was always undefined on the frontend — causing option
values to fall back to text content ("3M") and firing /clients/3M/teams 404s.

- Remove Field(alias="_id") from Client/Team/Project models; id is now a
  plain string field populated explicitly in _client_from_doc etc.
- API now returns id not _id, matching the TypeScript Client interface
- Add clientId !== "undefined" guard to useTeams, usePMs, useProjects

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:18:28 +01:00
Vadym Samoilenko
bbd324e3c7 feat: add Project Manager role + client/team assignment panel in admin user editor
- Add project_manager to all role dropdowns (UserList filter, create modal, UserDetail edit form)
- Add indigo badge color for project_manager in user list table
- Expose pm_client_ids in UserResponse schema and all admin user endpoints
- Add pm_client_ids to frontend User type
- Add UserAssignmentsPanel to UserDetail sidebar: PM users see client toggle list; other roles see client → team membership picker
- Add flexible hooks (useTeamsForClient, useAssignPMAny, useRemovePMAny, useAddTeamMemberAny, useRemoveTeamMemberAny)
- Fix useClient guard against literal "undefined" string causing 404 requests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 15:58:55 +01:00
Vadym Samoilenko
2b721d182b feat: Client → Team → Project isolation system with Project Manager role
Backend:
- New UserRole.PROJECT_MANAGER with pm_client_ids[] on User model
- New models: Client (slug-based), Team (member_user_ids[]), Project (client-scoped)
- Job model gains project_id field
- New GET/POST/PATCH/DELETE /clients, /clients/{id}/teams, /clients/{id}/projects,
  /clients/{id}/pm routes (admin-only client CRUD; PM or admin for teams/projects)
- get_accessible_project_ids() helper: staff→all, PM→their clients' projects,
  CLIENT→projects from teams they belong to (with legacy owner fallback)
- list_jobs, get_job, bulk_download, get_vtt_content, delete_job all use new isolation

Frontend:
- UserRole type gains 'project_manager'
- Job, JobCreateRequest gain project_id field
- Client, Team, Project, PMUser types added
- ApiClient: full client/team/project/PM CRUD methods
- useClients hook with all query/mutation hooks
- Admin pages: ClientList + ClientDetail (teams, members, projects, PM assignment)
- NewJob form: client + project picker (shown when clients exist)
- Sidebar: Clients nav item for admin and project_manager roles
- Routes: /admin/clients and /admin/clients/:clientId behind RoleGate

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 15:11:13 +01:00
Vadym Samoilenko
26bfedd7c7 feat: add cost_tracker_project_id assignment UI to QC and Final Review
- PATCH /jobs/{job_id} endpoint for updating title and cost_tracker_project_id
- cost_tracker_project_id exposed on JobResponse (GET /jobs/{id})
- Inline project ID field in QCDetail and FinalDetail — saved via PATCH
- "AI Cost Dashboard" link in UserList header
- cost_tracker_project_id added to Job type and JobUpdateRequest schema

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 11:40:13 +01:00
Vadym Samoilenko
cf761c4bb6 feat: add linguist role and user management navigation
- Add LINGUIST role to UserRole enum (backend + frontend)
- Grant linguists access to QC Review, Final Review, review notes, and VTT editing
- Add MongoDB migration to update schema validator with linguist role
- Add admin seed: vadymsamoilenko@oliver.agency is promoted to admin on startup
- Add User Management sidebar link for admin users
- Fix Login.tsx role type cast to use UserRole instead of hardcoded union

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 11:46:33 +01:00
Vadym Samoilenko
6f963ff7c4 feat: DCMP compliance, descriptive transcript, new languages, QA bug fixes
- Rewrote VTT translation to two-step (text-only → Gemini → apply to original timestamps) preventing caption timing desync
- Added polling fallback for all processing states and Safari visibilitychange WebSocket reconnect
- Added 11 new TTS languages (cs, da, fi, hu, no, sk, sv, es-419, pt-BR, fr-CA)
- Updated caption/AD prompts to DCMP Captioning Key & Description Key standards (line splitting, ♪ music notation, italic tags, caption positioning, ethics guidelines)
- Added descriptive transcript generation (WCAG 2.1 §1.2.1) combining captions + AD into plain text
- Fixed amix normalize=0 to prevent audio loss in rendered videos
- Fixed AD re-timing double-count when source_ms is None
- Fixed cue block numbering to be 1-based in VttEditor and Timeline Preview

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 11:50:43 +00:00
Vadym Samoilenko
f4ddcce066 fix: resolve QA-reported bugs — MP3/VTT desync, crashes, notifications, and more
**BUG-1 & BUG-2 — Wrong audio plays after re-render / MP3 doesn't match text**
Root cause: audio files were named by index (cue_0.mp3, cue_1.mp3). When a cue
was inserted or deleted, all following indices shifted but old MP3 files kept
their original names, so re-render would play the wrong audio for the wrong cue.
Fix: renamed files to cue_N_CONTENTHASH.mp3 and introduced an ad_cue_manifest
stored in the job document that maps each cue index to its correct GCS URI.
Re-render now reads from the manifest instead of guessing by filename.
Also: editing AD cue text in the VTT editor now automatically queues TTS
regeneration for changed cues — no more silent mismatches.

**BUG-3 — App crash / state desync when uploading VTT or clearing TTS queue**
Fixed handleVttFileUpload to only update local editor state after the server
confirms the save — previously local state was updated first, so a network
error left the UI showing content that wasn't actually saved.
Fixed handleClearRegenerationQueue to only remove items from local state if
the server removal succeeded — previously all items were cleared regardless.

**BUG-4 — AI generates different audio descriptions every time**
Added GenerateContentConfig(temperature=0.2, top_p=0.8, top_k=40) to the
Gemini API call so output is more consistent across runs.

**BUG-5 — On-screen text inconsistently described**
Strengthened the AI prompt rule from a vague suggestion to a mandatory
requirement with an explicit format: "Text on screen reads: [exact text]".
Applied to both gemini_ingestion.md and gemini_ingestion_targeted.md.

**BUG-6 — No notification when re-render finishes**
Added rendering_qc toast notification and a dismissible green banner that
appears in QCDetail when re-render transitions to pending_qc. The banner
auto-dismisses after 10 seconds. Also increased WebSocket reconnect attempts
from 5 to 15 and capped backoff at 60s to prevent falling back to manual refresh.

**BUG-7 — Timeline preview looks accurate but isn't after edits**
Added isStale prop to TimelinePreview. The timeline now shows an amber tint
and "Preview may be outdated" label whenever there are unsaved pause point
changes, pending TTS regenerations, or a new VTT has been uploaded.

**BUG-8 — ElevenLabs API errors break TTS with no fallback**
Added try/except fallback chain in _synthesize_single_cue: if the configured
provider fails, it automatically retries with google, then gemini.

**BUG-9 — Concurrent re-render requests cause race conditions**
Made the PENDING_QC → RENDERING_QC status transition conditional (only
succeeds if the job is still in PENDING_QC). Returns HTTP 409 if a re-render
is already in progress. The completion transition back to PENDING_QC is also
conditional so a cancelled/overridden render doesn't corrupt job state.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 13:23:55 +00:00
Vadym Samoilenko
2245a12829 fix: case-insensitive Microsoft user lookup to prevent duplicate key error
Microsoft can return different email casings for the same user (e.g.
VadymSamoilenko@... vs vadymsamoilenko@...). The previous case-sensitive
find_one would miss the existing user, then fail on insert_one with a
duplicate key error on the _id field (ms-{sub[:20]}).

Fix: look up by _id first (deterministic from Microsoft sub), then fall
back to case-insensitive email regex for local-to-Microsoft migrations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 10:42:58 +00:00
Vadym Samoilenko
c413fcb747 feat: add SDH (Subtitles for Deaf and Hard of Hearing) caption output
SDH captions extend standard VTT with speaker identification labels,
sound effects [PHONE RINGS], music notation ♪, and off-screen indicators.

- Add sdh_vtt flag to RequestedOutputs model and frontend form
- Add sdh_captions_vtt_gcs field to LangOutput model
- Inject SDH generation instructions into both Gemini prompts via
  {SDH_FIELD} and {SDH_GUIDELINES} placeholders when requested
- Upload sdh_captions.vtt to GCS in ingest task
- Pass SDH through video_native translation (Gemini generates it directly)
  and traditional translation (translate source SDH VTT via Gemini)
- Expose sdh_captions_vtt in downloads endpoint and bulk zip export

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 15:02:18 +00:00
Vadym Samoilenko
2e8a8dc287 feat: add brand context, ethics guidelines, and improved AD prompt rules
- Add brand_context field (job model, API, frontend form) so clients can
  list brand names present in their video; Gemini uses these names instead
  of generic descriptors (e.g. "Sellotape" not "sticky tape")
- Add ethical guidelines section to both Gemini prompts covering
  person-first language, consistent race/gender description only when
  plot-relevant, no guessing at unconfirmed identity
- Revamp audio description rules: priority ordering (essential →
  high-priority → time-permitting), pre-teaching placement, no cinematic
  jargon, succinct style replacing the former "20% longer" instruction
- Thread brand_context through full stack: routes → job doc → ingest
  task → translate task → both Gemini prompt templates

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 14:46:09 +00:00
Vadym Samoilenko
c6c7ff51c7 fix: clear stale pause points when AD VTT is re-uploaded
Old pause_points in edit_state always overrode new VTT cue timings
during re-render, making AD VTT upload for timing adjustments
non-functional. Clear pause_points and video_segments on AD VTT
upload so re-render falls back to the new cue start times.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 17:07:55 +00:00
Vadym Samoilenko
222826baa7 fix: propagate ElevenLabs voice fetch errors to frontend
- elevenlabs_voices.py: re-raise exception on first fetch failure
  (empty cache) instead of silently returning empty list
- routes_tts.py: catch get_voices() exception and return available=False
  with the error detail; add optional error field to ProviderVoicesResponse
- VoiceSelector: show actual API error message when available=false

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:27:45 +00:00
Vadym Samoilenko
a22fe5c1bc fix: surface ElevenLabs config errors and add availability flag
- Extract actual error message from blob response in previewVoice so
  users see the real API error instead of generic "Failed to generate preview"
- VoicePreviewButton now reads err.message from thrown Error objects
- Add available: bool field to ProviderVoicesResponse; returns false
  when ELEVENLABS_API_KEY is not configured so the frontend can react
  proactively instead of hitting a 400 on preview
- VoiceSelector shows a descriptive config warning when available=false

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:17:00 +00:00
Vadym Samoilenko
1e177a6d5c feat: add ElevenLabs voice selection to frontend and backend
Add dynamic ElevenLabs voice catalog with provider toggle in the UI,
allowing users to browse ElevenLabs voices, configure stability and
similarity boost settings, and preview/synthesize with ElevenLabs TTS.

Backend:
- New elevenlabs_voices.py service with 1-hour cached API fetching
- TTS routes support ?provider= query param for voices and options
- Preview endpoint routes to ElevenLabs or Gemini based on provider
- stability/similarity_boost params flow through TTS synthesis pipeline
- TTSPreferences model extended with ElevenLabs-specific fields
- Deprecated hardcoded elevenlabs_voices config (now fetched dynamically)

Frontend:
- Provider toggle (Gemini/ElevenLabs) in VoiceSelector
- ElevenLabsSettingsPanel with stability and similarity boost sliders
- VoicePreviewButton supports provider-specific preview parameters
- API client passes provider param to voices, options, and preview endpoints
- New VoiceInfo, ProviderVoicesResponse, ProviderOptionsResponse types

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 13:58:56 +00:00
michael
0c3102b77f feat: add Return to QC action for jobs in resting statuses
Allow production/admin users to move jobs back to pending_qc from
completed, pending_final_review, rejected, qc_feedback, tts_failed,
render_failed, approved_english, and approved_source statuses. Includes
single-job endpoint, bulk endpoint, JobDetail inline form with required
notes, bulk action in JobsList with confirmation modal, and a Review
Notes card on the job overview tab.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-14 13:18:02 -06:00
michael
dab294f18a feat: streamline QC approval to skip translation pipeline
QC approval now transitions jobs directly to pending_final_review
since translation, TTS, and accessible video rendering happen before
QC review. Removes unnecessary translate_and_synthesize_task trigger
on approval.

- Update approve_source() to use PENDING_FINAL_REVIEW status
- Update bulk_approve_jobs() to use PENDING_FINAL_REVIEW status
- Remove translate_and_synthesize_task.delay() calls from both endpoints
- Update JobDetail progress indicator to reflect new flow
- Update CLAUDE.md state machine documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 10:37:37 -06:00
michael
e371dc401a feat: add save button to voice settings panel for TTS regeneration
Add ability to save voice settings changes in QC Review screen without
needing to approve the job. When saved, all TTS segments are regenerated
across all languages with the new voice settings.

Changes:
- Add PUT /jobs/{id}/tts-preferences endpoint to update TTS preferences
- Add UpdateTTSPreferencesRequest schema
- Add updateTTSPreferences API method and useUpdateTTSPreferences hook
- Add Save Voice Settings button with change detection to QCDetail

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 09:05:56 -06:00