Commit graph

86 commits

Author SHA1 Message Date
Vadym Samoilenko
c1948ea198 feat(ux): T-2/PR-7/PR-8 — status color helper, queue stats widget, upload-final-VTT override
T-2: Extract getJobStatusColor() into utils/jobStatusMessages.ts; StatusBadge now uses the
     shared helper (single source of truth for badge colors).

PR-7: GET /admin/production/queue-stats — returns Celery queue depths via Redis LLEN.
      Production dashboard shows a live panel (10s refresh) with per-queue task counts.

PR-8: POST /admin/production/jobs/{id}/upload-final-vtt — Production/Admin can upload a
      hand-crafted VTT to bypass AI, writing to GCS and advancing the job to PENDING_QC.
      Upload modal added to FailuresList with language + type (captions/ad) selectors.

docker-compose.optical-dev.yml: enable USE_CELERY_FALLBACK=true, set worker replicas=1
      for all pipeline workers (ffmpeg/tts/whisper) with WORKER_CONCURRENCY=2 so the full
      pipeline runs on the 2-CPU optical-dev server until Cloud Run VPC Connector is ready.

Fix: remove unused effectiveMs variable in TimelinePreview (TS6133).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 11:12:36 +01:00
Vadym Samoilenko
e4b350cd7d feat(ux): R-8 linguist language warn, PM CC editing, timeline right-click + CC insert
R-8 — Linguist language competence:
- Add User.languages[] BCP-47 field to backend model + UserResponse schema
- Frontend: show amber warning in assign modal when selected linguist has no
  competence listed for the target language

PM VTT editing (FinalDetail):
- PM and ADMIN can now edit captions/AD in the final review stage
- VttEditor becomes read-write with onCueSave wired to updateVttMutation
- Other roles remain read-only

Timeline right-click + add pause:
- Right-click anywhere on the timeline opens a context menu showing the timestamp
- If near a pause point marker: "Edit timing" + "Regenerate TTS" options
- If on empty space: "Add AD cue at Xs" → inserts a new AD cue in the editor
- Pause point markers widened from 1px → 2px (3px on hover) for easier clicking
- Right-click on a pause point marker directly opens the editor

VttEditor insertAtTimeMs prop:
- New prop triggers programmatic insert at a specific video timestamp
- Used by the timeline right-click "Add AD cue here" action

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 10:51:31 +01:00
Vadym Samoilenko
3f557724d3 feat(api): L-18 blocked-on-source, PR-10 promote-to-qc, R-12 reviewed_cues reset
- POST /{job_id}/actions/blocked_on_source (L-18): linguist/reviewer flags a source
  video issue; moves job to QC_FEEDBACK and records blocked_on_source_reason/at/by
- POST /{job_id}/actions/promote_to_qc (PR-10): production/admin manually bypasses
  AI processing for edge-case failures; adds audit history entry
- Reset reviewed_cues to 0 on submit_for_review (R-12) so reviewer must re-acknowledge
  all cues after each linguist resubmit
- Add assert_job_in_user_org + get_user_org_ids to core/dependencies.py (used by
  the new endpoints and the cross-tenant isolation test suite)
- Remove unused ingest_and_ai_task / translate_and_synthesize_task imports

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 10:38:39 +01:00
Vadym Samoilenko
ff372c7322 fix(security): close MT-17/18/19, restore cross-tenant tests, quick wins
Blocks 1–5 of stabilization plan:

SECURITY
- validation.py: restore settings.upload_max_video_bytes (T-14 regression fix)
  and JSON object key validation that was incorrectly removed
- MT-18: add accessible_org_ids filter to list_for_reviewer/list_for_linguist
  so reviewers/linguists only see jobs from their own org in QC queue
- MT-17: add Membership.team_ids[], write to it on invitation acceptance and
  direct team add/remove; migration backfills from Team.member_user_ids
- MT-19: validate all target_team_ids belong to invitation's org_id at creation

TESTS
- Restore test_cross_tenant_isolation.py (was deleted, only .pyc remained)
- Extend with MT-18 reviewer org isolation tests

QUICK WINS
- W-8: remove time.sleep(1) + dead debug block from POST /jobs (task was
  undefined — would have caused NameError → HTTP 500 on every job creation)
- T-13: warn at startup when REDIS_URL configured but connection failed
- T-16: skip language_qc lifespan migration when count=0 (no DB scan on startup)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-30 10:32:23 +01:00
Vadym Samoilenko
b3ace22009 feat(infra): move heavy workers to Cloud Run Jobs
Heavy pipeline tasks (ingest, translate, render, rerender) now dispatch
to a Cloud Run Job (va-worker) instead of local Celery workers. optical-dev
runs only api + lightweight worker (notify/embed) within its 2-CPU budget.

- backend/app/tasks/runner.py — Cloud Run Job entrypoint
- backend/app/services/cloud_run_dispatch.py — replaces .delay() for heavy tasks
- backend/Dockerfile.cloudrun — Cloud Run worker image (ffmpeg included)
- docker-compose.optical-dev.yml — 2-CPU safe overrides, disables heavy workers
- cloudbuild.yaml — builds va-worker image and updates Cloud Run Job
- deploy-dev.sh — uses 3-file compose, builds only api+worker locally
- routes_jobs, routes_admin_production, ingest_and_ai, translate_and_synthesize
  — all dispatch sites updated to use cloud_run_dispatch.dispatch()

USE_CELERY_FALLBACK=true in .env.local to use Celery locally during dev.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 21:47:10 +01:00
Vadym Samoilenko
4623b89aeb feat(mt-16): JWT org_ids claim + transient user.org_ids in deps
- create_access_token gains optional org_ids: list[str] param; encodes
  {exp, sub, org_ids, v:2} — org_ids is a prefilter hint only, never
  used as authorization source of truth (Redis cache is authoritative)
- Login, MS login, refresh endpoints: fetch memberships and include
  org_ids in issued access tokens via _get_user_org_ids() helper
- routes_invitations.py accept flow: same org_ids population on token
- get_current_user: reads org_ids from payload, attaches as transient
  user.__dict__["org_ids"] — available to OrgScopedQuery for prefilter
- Force logout: rotate JWT_SECRET env var at deployment time (no code
  change needed; all existing tokens immediately invalidated)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:46:39 +01:00
Vadym Samoilenko
54fcf47887 feat(mt-14): gcs_prefix on Job, gcs_path helper, rewrite path sites
- gcs_path(job, *parts) helper in gcs.py: uses job.gcs_prefix if set,
  falls back to job._id (legacy) — backward-compatible for all old jobs
- create_job: sets gcs_prefix=orgs/{org_id}/jobs/{job_id} when
  organization_id is known; legacy jobs without org get null prefix
- Rewrote hardcoded f"{job_id}/{lang}/..." paths in:
  - ingest_and_ai.py (4 upload sites)
  - translate_and_synthesize.py (9 sites via bulk regex)
  - render_accessible_video.py (3 sites: segments, video, captions)
  - rerender_accessible_video.py (3 sites)
- tools/migrate_gcs_org_prefix.py: idempotent operator script —
  preflight checks, copy→verify(count+md5)→mongo update→delete,
  ThreadPoolExecutor(4), resume file, dry-run + rollback modes

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:45:12 +01:00
Vadym Samoilenko
595897e61a feat(w-12): JobBrief model, endpoints, migration + brief→job linkage
- JobBrief model (DRAFT→SUBMITTED→APPROVED→FULFILLED) with 6 CRUD
  endpoints: list, create, get, patch (DRAFT only), submit, approve
- All endpoints use MembershipContext; read=VIEWER, mutate=MANAGER,
  approve=ADMIN for org-scoped access
- create_job accepts brief_id Form field; validates APPROVED brief,
  copies organization_id/project_id/deadline from brief, marks brief
  FULFILLED after job insert
- organization_id now populated from project client_id on job create
  (fixes missing multi-tenant field on new jobs)
- migration_2026-04-29-000001: job_briefs collection + 4 indexes
- Wired briefs router into main.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:38:08 +01:00
Vadym Samoilenko
a945653e73 feat(w-14): bulk failures dashboard + sidebar badge
- GET /admin/production/failures: list failed jobs filtered by step/org
- POST /admin/production/bulk-retry: dispatch retry for up to 50 jobs
  with "auto" (from failure.step) or "from_scratch" strategies
- FailuresList.tsx: accordion-grouped by error type, multi-select,
  bulk retry action, step label, retry count (red >3), updated date
- Sidebar: "Failures" item with live badge for production/admin roles
  (polls useJobs with processing_failed,tts_failed,render_failed)
- New useFailures / useBulkRetry hooks

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:36:30 +01:00
Vadym Samoilenko
264561895e feat(w-13): generic /jobs/{id}/retry endpoint + unified failure UI
- POST /jobs/{job_id}/retry dispatches correct pipeline task based on
  failure.step: ingestion/ai_processing → ingest_and_ai_task,
  translation/tts → translate_and_synthesize_task, render → rerender
- Increments retry_count, writes JOB_RETRY audit log entry
- Adds processing_failed to JobStatus type; JobFailure interface on Job
- Replaces TTS-only retry block with FailureBanner showing step/message/
  retry_count for all failed statuses (processing_failed, tts_failed,
  render_failed); Escalate mailto link for high-retry-count cases
- useRetryJob hook + apiClient.retryJob() call new endpoint

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:33:50 +01:00
Vadym Samoilenko
38038862c9 refactor(mt-15): consolidate authz in routes_jobs and dependencies
list_jobs now uses MembershipContext (Redis-cached, 60s TTL) to build
org-scoped queries instead of per-request memberships.find(). Falls back
to legacy get_accessible_project_ids for users with no memberships.

get_job replaces the role-specific CLIENT/PM access check with
get_job_or_403() which uniformly checks organization_id membership for
all roles (returns 404 not 403 to avoid leaking cross-org job existence).

get_accessible_project_ids in dependencies.py now uses _cached_memberships
from authz.py, eliminating the duplicate uncached DB query.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:26:07 +01:00
Vadym Samoilenko
5209f04318 feat(mt-13): bind glossary handlers to client_id via org membership check
All 8 glossary route handlers now verify the requesting user has org
membership in the target client_id using assert_user_in_org() from
core/authz.py. Read endpoints require VIEWER, mutations require MANAGER,
archive requires ADMIN (org-level). Removed dead _assert_can_read() and
_require_client_staff() helpers. Removed unused require_roles/User/UserRole
imports. Also added get_job_or_403() to authz.py for MT-15.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:24:41 +01:00
Vadym Samoilenko
b2d524e702 fix(mt-12): remove PM/CLIENT legacy bypass in _assert_client_access
The unconditional `if user.role in (CLIENT, PROJECT_MANAGER): return`
allowed any PM to access any client regardless of membership. Removed;
kept pm_client_ids legacy fallback for pre-migration users.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:22:56 +01:00
Vadym Samoilenko
08fcb4daa4 feat(pr6): WS real-time updates, per-cue AD playback, upload guard
W-4: team assignment (linguist/reviewer) stored on job at creation,
     auto-assigned to all language QC states on first GET /language-qc
     (lazy init via auto_assign_defaults)

L-3 WS: broadcast_to_job when reviewer opens VTT for editing;
        QCDetail shows "User X is editing [lang]" banner (auto-clears 5s)

R-5: comment broadcast via broadcast_to_job on add_comment();
     QCDetail invalidates comments query on language_qc_comment WS event

L-15: QCDetail subscribes to language_qc_assigned WS event →
      refetches lang-qc data and shows toast

R-7: VttEditor gets onCuePlay prop; AD editor in QCDetail wires
     handleAdCuePlay → switches to accessible video mode, seeks & plays

T-15: beforeunload warning in NewJob while upload is in progress

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:42:57 +01:00
Vadym Samoilenko
bdfa0f82ab fix(lint): restore baseline lint count — no new errors introduced
QCDetail.tsx: 4 new `any` types replaced with `unknown` + type casts.
backend: ruff auto-fix sorted imports, removed unused imports, updated Optional[X] → X | None in routes_share + share_token model.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:16:35 +01:00
Vadym Samoilenko
1317ee7ca4 feat(t6+t7+t11): native captions track, AD audio sync, CSRF protection
T-6: Add Blob URL native <track> in VideoWithCaptions so browser CC button works in fullscreen.
T-7: Sync hidden <audio> AD playback with video play/pause/seeked events.
T-11: Double Submit Cookie CSRF — _set_auth_cookies issues httponly refresh_token + readable csrf_token; /refresh validates X-CSRF-Token header; frontend reads csrf_token cookie and sends header on all refresh calls.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:08:27 +01:00
Vadym Samoilenko
dc1cfd01dc feat(l3): optimistic locking for VTT edits (ETag / 409 Conflict)
Backend:
- VttContentResponse gets etag field (SHA1 of captions+AD content)
- VttUpdateRequest gets if_match field (optional)
- GET /jobs/{id}/vtt: computes and returns etag
- PATCH /jobs/{id}/vtt: if if_match present, fetches current content, recomputes
  hash, returns 409 Conflict if mismatch

Frontend:
- VttContentResponse type + VttUpdateRequest type updated
- QCDetail stores vttEtag from GET response
- All updateVttMutation calls pass if_match: vttEtag
- 409 responses show specific "Conflict: another user has modified" message

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 19:01:57 +01:00
Vadym Samoilenko
abf81515a4 feat(pm15): share read-only link for client preview
Backend:
- ShareToken model (share_tokens collection)
- POST /jobs/{id}/share — create token (PM/PROD/ADMIN)
- GET /jobs/{id}/share — list active tokens
- DELETE /jobs/{id}/share/{token_id} — revoke token
- GET /public/share/{token} — unauthenticated preview with signed GCS URLs (6h TTL)
  Returns video, captions, AD for all languages

Frontend:
- ShareView.tsx — public page at /share/:token with language switcher, video player, download tiles
- App.tsx — /share/:token route (no auth wrapper)
- QCDetail.tsx — "↗ Share link" button in header → modal to generate + copy link

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 18:56:44 +01:00
Vadym Samoilenko
f1a9e6ee46 feat(pm7): bulk assign linguist/reviewer to all languages in one click
- POST /jobs/{job_id}/languages/bulk-assign — assigns linguist (required) and
  reviewer (optional) across all or selected languages; supports only_unassigned
  flag and optional deadline
- bulkAssignLanguages() added to API client
- QCDetail: "Assign all languages" button in Languages header; opens modal with
  linguist/reviewer dropdowns, deadline, and skip-already-assigned checkbox

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 18:53:14 +01:00
Vadym Samoilenko
13db347d65 feat(pr3+pr4): deadline field, job clone, reject categories, reviewed-cues gate
PM-1 (deadline):
- Job model: add deadline field (job-level PM deadline)
- POST /jobs: accept deadline as ISO date form param
- JobsList: deadline column with overdue highlight (red + warning icon)
- NewJob: date picker for deadline field
- useMultiUpload: pass deadline to batch job creation

PM-2 (clone job):
- POST /jobs/{id}/clone: creates config copy in 'created' state, no reupload
- useCloneJob hook, Clone button in JobsList actions
- navigate to cloned job on success

R-4 (reject categories):
- LanguageQCState: add reject_category field
- reject_language service: accept optional category (timing/mistranslation/terminology/profanity/length/other)
- RejectLanguageRequest: add category field
- QCDetail reject modal: category pill-selector before free-text notes

R-2 (reviewed-cues tracking):
- LanguageQCState: add reviewed_cues (int) + total_cues (nullable)
- POST /jobs/{id}/languages/{lang}/mark-cue-reviewed endpoint
- QCDetail: progress bar + approve gated at 80% for reviewer (admin bypasses)
- markCueReviewed API client method

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 18:39:05 +01:00
Vadym Samoilenko
c7a6f13b10 feat(workflow): PR-2 workflow blockers — PM/Production dashboards, two-stage QC, role routing
Changes:
- Dashboard: add project_manager case (final review / QC counts / new job widgets)
  and production case (AI pipeline / failures widgets)
- Sidebar: add project_manager to Final Review and Audit Log nav items;
  live badge counts for QC Queue (pending_qc) and Final Review (pending_final_review)
- App.tsx: add project_manager to Final Review and Audit Log RoleGates (W-10, PM-18)
- Login: role-based redirect after login — linguist/reviewer → /qc/queue, others → /
- language_qc._assert_can_approve: enforce two-stage QC; remove linguist self-approve
  fallback; require reviewer assignment + submitted_for_review_at (W-6)
- routes_jobs.complete_job: allow project_manager to complete jobs (W-9)
- notify.py: re-enable email notifications (W-7)
- Fix 400 on cue save: treat empty-string audio_description_vtt/captions_vtt as absent
  both in backend (truthy check) and frontend (|| undefined) — root cause was adVtt
  initialising to '' when job has no AD track

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 18:18:24 +01:00
Vadym Samoilenko
a168af1aa7 feat: two-stage QC (linguist→reviewer), project picker, comments, email notifications, deadlines
- Two-stage QC workflow: linguist edits + submits → reviewer approves/rejects per language.
  New statuses: in_progress, pending_review, in_review. New service functions: submit_for_review,
  open_review, assign_reviewer, reassign_reviewer, add_comment. Linguist and reviewer deadlines.
- Reject now resets language to in_progress so linguist can iterate without full re-assignment.
- QC comment threads per language (append-only), visible to all assignees.
- Email notifications via Mailgun on: assignment, submit-for-review, comment, approve, reject.
  Best-effort (failures do not roll back QC actions). asyncio.gather for parallel fan-out.
- New audit actions: LANGUAGE_QC_REVIEWER_ASSIGN/REASSIGN, LANGUAGE_QC_SUBMIT,
  LANGUAGE_QC_OPEN_REVIEW, LANGUAGE_QC_COMMENT.
- Inline project picker in NewJob: "+ Create new project…" option with name, default
  languages, default linguist, default reviewer. Pre-fills languages on the new job.
- Project model extended with default_languages, default_linguist_id, default_reviewer_id.
- RBAC: CLIENT org-members can now create projects (backend guard relaxed).
- LinguistQueue: role toggle "As linguist / As reviewer" + new status tabs.
- QCDetail: two-slot assignment cards (linguist + reviewer), deadline display, role-aware
  action buttons, comments panel with optimistic insert and 15s refetch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 16:59:40 +01:00
Vadym Samoilenko
be0bffe459 fix: get_terms_page avoids GlossaryTerm validation on partial projection
Projected docs only have _id/source_term/translations; validating against
GlossaryTerm (which requires glossary_id, version_id, source_term_lower)
caused 500 on the terms endpoint. Return plain dicts instead.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:57:12 +01:00
Vadym Samoilenko
125c69fb1d fix: audit log user/security endpoints return correct shapes
- /audit-logs/user/{id}: now accepts email OR ObjectId, returns bare array
- /audit-logs/security: returns bare array instead of {logs, hours} wrapper
  Both match AuditLogEntry[] that the frontend expects.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:48:00 +01:00
Vadym Samoilenko
ad67089b09 fix: remove duplicate /audit-logs route and align pagination params with frontend
The legacy GET /audit-logs (returning wrong shape) shadowed the proper one.
Removed the duplicate and changed page/size params to skip/limit to match
the AuditLogQuery the frontend sends.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:39:22 +01:00
Vadym Samoilenko
dee4d69b40 fix: raise user list size limit to 500 and guard toLocaleString calls
- routes_admin.py: size query param max raised from 100 → 500 so
  ClientDetail.tsx (size=200) no longer returns 422
- GlossaryDetail.tsx: three .toLocaleString() calls guarded with ?? 0
  to prevent TypeError when term_count is undefined on first render

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:20:38 +01:00
Vadym Samoilenko
a3b300b76a docs: add canonical documentation + audit cleanup
- AGENTS.md: canonical project entry point (Quick Nav, pipeline, constraints)
- docs/: complete docs tree — architecture, API spec, DB schema, infra,
  runbook, requirements, tech stack, principles, reference ADRs, guides,
  tasks backlog, testing strategy
- tests/README.md: test commands, structure, known gaps
- README.md / CLAUDE.md / DEPLOYMENT.md: updated with canonical doc links
- .archive/: backup of pre-documentation-pipeline originals
- backend/uv.lock: uv dependency lockfile
- Delete committed __pycache__ .pyc files (should have been gitignored)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:22:51 +01:00
Vadym Samoilenko
4c6624c3d4 fix: code health sweep — M-01 through M-07
M-01 authz.py: move cache_key above try block to avoid NameError when
     first Redis call returns None
M-02 main.py: re-enable validation middleware (was TEMPORARILY DISABLED)
M-03 routes_auth.py / main.py: replace print() debug lines with
     structured logger calls; logger now module-level in routes_auth.py
M-04 gcs.py: asyncio.get_event_loop() → get_running_loop() (deprecation)
M-05 translate_and_synthesize.py: bind loop vars in closure defaults
     to fix B023 ruff warnings (transcreate/translate_captions/etc.)
M-06 rate_limiting.py: only trust X-Forwarded-For when X-Forwarded-Proto
     is https; use rightmost entry (proxy-appended) not leftmost
M-07 validation.py: extend MongoDB operator blocklist to cover $expr,
     $function, $accumulator, $nin, $gte, $lte, $jsonSchema, $mod

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:18:02 +01:00
Vadym Samoilenko
87ae6571fe perf: use DI connection pool for auth routes, async httpx for MS SSO (H-01, H-02)
- login and microsoft_login routes now use Depends(get_database) instead
  of creating a per-request MongoClient — removes connection-pool churn
  under load
- MicrosoftAuthService._get_openid_config/_get_jwks/validate_token are
  now async, using httpx.AsyncClient instead of blocking requests.get —
  removes ~400ms event-loop block per Microsoft login
- Removed unused AsyncIOMotorClient import from routes_auth.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:13:50 +01:00
Vadym Samoilenko
e81acebc45 security: remove exception detail from /auth/refresh response (C-03)
Replaced the bare except that leaked str(e) (JWT library internals,
claim validation messages) with a generic "Invalid refresh token" detail.
Full traceback is now logged server-side via the structured logger.
Re-raises HTTPException before the generic handler so valid 401s from
inner checks are not double-wrapped.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:11:59 +01:00
Vadym Samoilenko
fa351e4d25 feat: per-client glossary — hybrid exact/vector retrieval + AI injection
Adds full glossary system so Gemini uses client-approved terminology
when generating subtitles and translations (critical for 3M brand names
and product codes across 16 target locales).

Backend:
- lib/locales.py: BCP-47 locale registry, normalises xlsx fr_fr → fr-FR
- models/glossary.py: Glossary / GlossaryVersion / GlossaryTerm + enums
- services/glossary_service.py: xlsx parse (openpyxl), ingest to Mongo,
  hybrid retrieval (Aho-Corasick exact + Atlas Vector Search), prompt block
- services/embedding_service.py: Gemini text-embedding-004, batch 100, retry
- tasks/embed_glossary.py: Celery background task for async embedding
- api/v1/routes_glossaries.py: CRUD endpoints under /clients/{id}/glossaries
- gemini.py: _build_glossary_block(), {GLOSSARY} injection in all 4 call sites
- tts.py / gemini_tts.py: pass full locale codes (no split("-")[0] truncation)
- tasks/translate_and_synthesize.py: glossary lookup + injection per language
- prompts: {GLOSSARY} placeholder in ingestion, targeted, transcreation prompts
- pyproject.toml: +openpyxl, +pyahocorasick

Frontend:
- routes/admin/glossaries/: GlossaryList, GlossaryUpload, GlossaryDetail
- App.tsx: 3 new routes under /admin/clients/:clientId/glossaries
- ClientDetail.tsx: Glossaries card with count + quick links
- types/api.ts: Glossary, GlossaryVersion, GlossaryDetail, GlossaryTerm types
- lib/api.ts: 7 new API methods (upload, list, detail, terms, versions, activate, archive)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 13:03:38 +01:00
Vadym Samoilenko
05f25a1141 feat: per-language QC workflow with linguist assignment
- Job.language_qc dict tracks per-language status (pending/in_review/approved/rejected)
  with full event history; qc_assignments denormalized array enables efficient queue queries
- language_qc service handles assign/reassign/approve/reject/reopen with atomic DB updates,
  audit logging, and auto-advancement to pending_final_review when all languages approved
- Linguists can only edit VTT and trigger re-renders for their assigned language (403 guard)
- return_to_qc resets all language statuses while preserving assignments
- routes_language_qc.py: 7 new endpoints; /me/language-qc-queue for linguist queue
- Startup migration idempotently seeds language_qc for all existing jobs
- Frontend: LanguageQCState types, API methods, LinguistQueue page, QCDetail redesigned
  with per-language status badges, assignment dropdown, inline approve/reject buttons,
  progress bar, and reject modal; My QC Queue sidebar link

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 12:09:40 +01:00
Vadym Samoilenko
bab30e1508 feat: VTT version control — snapshots, diff, restore
Backend:
- VttVersion model (vtt_version.py): immutable snapshot per job/lang/kind/version
- vtt_versioning service: create_version (atomic counter + GCS snapshot),
  list_versions, get_version, restore_version, diff_versions (difflib line-level)
- routes_vtt_versions.py: GET /versions, GET /versions/{v}, GET /versions/diff,
  POST /versions/{v}/restore (PRODUCTION/ADMIN only, overwrites live file + audit log)
- Hook create_version into update_job_vtt_content before each live-file overwrite
- Mongo indexes: unique (job_id, lang, kind, version) + (job_id, created_at)

Frontend:
- VttVersionSummary / VttVersionFull / VttDiffResponse types
- api.ts: listVttVersions, getVttVersion, diffVttVersions, restoreVttVersion
- VersionsTab.tsx: lang/kind switcher, version list with A/B compare buttons,
  inline diff viewer (color-coded +/−), content viewer, restore with confirm dialog
- JobDetail.tsx: new "VTT Versions" tab wired to VersionsTab

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 11:46:21 +01:00
Vadym Samoilenko
09550cfca0 feat: audit log integration sweep + cost tracker URL fix + audit log admin UI
- Fix cost tracker dashboard URL (cost.oliver.agency → optical-dev.oliver.solutions/cost-tracker/analytics)
  in UserList, QCDetail, FinalDetail; centralise into src/lib/costTracker.ts

- Wire audit logging across backend routes (was 1 call site, now covers all key events):
  · routes_auth: LOGIN_SUCCESS/FAILURE for local + MS SSO, LOGOUT
  · routes_files: FILE_UPLOAD on signed URL generation
  · routes_jobs: JOB_CREATE, JOB_APPROVE, JOB_REJECT, JOB_STATUS_CHANGE, JOB_DELETE, VTT_EDIT
  · routes_admin: USER_CREATE, USER_UPDATE, USER_ROLE_CHANGE, USER_DEACTIVATE

- Add Audit Log admin UI page (/admin/audit-log):
  · Three tabs: All Events (paginated, server-side filters), Security Events, User Activity
  · Filters: action group, severity, success/failure, free-text search
  · Click-to-expand row shows IP, request ID, resource, details JSON
  · Wired into App.tsx (RoleGate: production + admin) and sidebar nav

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 11:34:06 +01:00
Vadym Samoilenko
1563714454 feat(saas): Phase 3 — membership-based authz + Mailgun + job.organization_id
authz.py (new):
- MembershipContext — per-request membership dict for the current user
- get_membership_context FastAPI dependency
- require_org_role(min_role) — dependency factory keyed off org_id path param
- require_platform_admin()
- OrgScopedQuery — adds organization_id filter; platform admin passes through
- bump_user_membership_cache — invalidates Redis key on membership writes

dependencies.py:
- get_accessible_project_ids now queries memberships collection first;
  legacy pm_client_ids / team.member_user_ids fallback retained until migration runs
  (four job-route access checks at lines 608/1054/1181/1538 are fixed via this function)

routes_clients.py:
- _assert_pm_or_admin and _assert_client_access are now async and query memberships
- All 10 call sites updated with await + db arg

emailer.py:
- Switched from SendGrid to Mailgun REST API via httpx (already in requirements)
- _send() is now fully async; same public method signatures preserved
- send_completion_email uses _send()

config.py:
- Added mailgun_api_key, mailgun_domain, mailgun_from settings
- sendgrid_api_key kept with empty default for backward compat

migration_2026-04-28-000003:
- Backfills job.organization_id from project.client_id
- Creates (organization_id, status, created_at) sparse index on jobs

routes_organizations.py / routes_invitations.py:
- Call bump_user_membership_cache after every membership write

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:56:42 +01:00
Vadym Samoilenko
00fb1aacc6 feat(saas): Phase 2 — invitation flow, email templates, MS SSO zero-membership
Backend:
- models/invitation.py — Invitation model + create/accept/preview schemas
- routes_invitations.py — org-scoped POST/GET/DELETE + public preview/accept endpoints
  Single-use token via find_one_and_update; sha256(token) stored in DB, plaintext in email URL
- emailer.py — _send() helper; send_invitation_email, send_welcome_email, send_password_reset_email
  send_completion_email refactored to use _send()
- migration_2026-04-28-000002 — creates invitations collection with TTL index (30d audit trail)
- routes_auth.py — new MS SSO users provisioned with zero memberships instead of role=PRODUCTION;
  they land on "no access" page until an admin invites them
- main.py — registers invitations_org_router and invitations_router

Frontend:
- routes/AcceptInvite.tsx — public page at /accept-invite?token=...
  Four states: new user (name+password), existing user (confirm), MS user, already-member
- App.tsx — /accept-invite route outside RequireAuth
- types/api.ts — Invitation, InvitationCreate, InvitationPreview, InvitationAcceptRequest/Response
- lib/api.ts — listInvitations, createInvitation, revokeInvitation, previewInvitation, acceptInvitation
- hooks/useClients.ts — useInvitations, useCreateInvitation, useRevokeInvitation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:52:08 +01:00
Vadym Samoilenko
6f1be645ce feat(saas): Phase 0+1 — Organization/Membership entities and dev branch
Introduces the multi-tenant SaaS foundation alongside the existing
client/team/project model (zero-downtime shim period):

Backend:
- app/models/organization.py — Organization + OrgRole enum (OWNER/ADMIN/MANAGER/MEMBER/VIEWER)
- app/models/membership.py — Membership model with MemberDetail for enriched responses
- app/services/membership_service.py — upsert/remove/list/has_org_role helpers
- app/api/v1/routes_organizations.py — /organizations CRUD + /members sub-resource + /me/memberships
- main.py — registers organizations router
- migrations: create memberships collection (unique index) + backfill from pm_client_ids/team members

Frontend:
- types/api.ts — Organization, OrgRole, Membership, OrganizationCreateRequest types; Client marked @deprecated
- hooks/useClients.ts — useOrganizations, useOrganization, useOrgMembers, useAddOrgMember,
  useUpdateOrgMember, useRemoveOrgMember, useMyMemberships
- lib/api.ts — listOrganizations, getOrganization, createOrganization, updateOrganization,
  listOrgMembers, addOrgMember, updateOrgMember, removeOrgMember, getMyMemberships

Reads fall back to the clients collection during transition; all writes go to organizations.
Existing /clients endpoints and hooks are untouched.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:46:24 +01:00
Vadym Samoilenko
269ab09fa6 fix: serialize Client/Team/Project with id not _id + guard undefined client hooks
Pydantic v2 + FastAPI serializes Field(alias="_id") as _id in JSON,
so client.id was always undefined on the frontend — causing option
values to fall back to text content ("3M") and firing /clients/3M/teams 404s.

- Remove Field(alias="_id") from Client/Team/Project models; id is now a
  plain string field populated explicitly in _client_from_doc etc.
- API now returns id not _id, matching the TypeScript Client interface
- Add clientId !== "undefined" guard to useTeams, usePMs, useProjects

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:18:28 +01:00
Vadym Samoilenko
bbd324e3c7 feat: add Project Manager role + client/team assignment panel in admin user editor
- Add project_manager to all role dropdowns (UserList filter, create modal, UserDetail edit form)
- Add indigo badge color for project_manager in user list table
- Expose pm_client_ids in UserResponse schema and all admin user endpoints
- Add pm_client_ids to frontend User type
- Add UserAssignmentsPanel to UserDetail sidebar: PM users see client toggle list; other roles see client → team membership picker
- Add flexible hooks (useTeamsForClient, useAssignPMAny, useRemovePMAny, useAddTeamMemberAny, useRemoveTeamMemberAny)
- Fix useClient guard against literal "undefined" string causing 404 requests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 15:58:55 +01:00
Vadym Samoilenko
2b721d182b feat: Client → Team → Project isolation system with Project Manager role
Backend:
- New UserRole.PROJECT_MANAGER with pm_client_ids[] on User model
- New models: Client (slug-based), Team (member_user_ids[]), Project (client-scoped)
- Job model gains project_id field
- New GET/POST/PATCH/DELETE /clients, /clients/{id}/teams, /clients/{id}/projects,
  /clients/{id}/pm routes (admin-only client CRUD; PM or admin for teams/projects)
- get_accessible_project_ids() helper: staff→all, PM→their clients' projects,
  CLIENT→projects from teams they belong to (with legacy owner fallback)
- list_jobs, get_job, bulk_download, get_vtt_content, delete_job all use new isolation

Frontend:
- UserRole type gains 'project_manager'
- Job, JobCreateRequest gain project_id field
- Client, Team, Project, PMUser types added
- ApiClient: full client/team/project/PM CRUD methods
- useClients hook with all query/mutation hooks
- Admin pages: ClientList + ClientDetail (teams, members, projects, PM assignment)
- NewJob form: client + project picker (shown when clients exist)
- Sidebar: Clients nav item for admin and project_manager roles
- Routes: /admin/clients and /admin/clients/:clientId behind RoleGate

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 15:11:13 +01:00
Vadym Samoilenko
26bfedd7c7 feat: add cost_tracker_project_id assignment UI to QC and Final Review
- PATCH /jobs/{job_id} endpoint for updating title and cost_tracker_project_id
- cost_tracker_project_id exposed on JobResponse (GET /jobs/{id})
- Inline project ID field in QCDetail and FinalDetail — saved via PATCH
- "AI Cost Dashboard" link in UserList header
- cost_tracker_project_id added to Job type and JobUpdateRequest schema

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 11:40:13 +01:00
Vadym Samoilenko
cf761c4bb6 feat: add linguist role and user management navigation
- Add LINGUIST role to UserRole enum (backend + frontend)
- Grant linguists access to QC Review, Final Review, review notes, and VTT editing
- Add MongoDB migration to update schema validator with linguist role
- Add admin seed: vadymsamoilenko@oliver.agency is promoted to admin on startup
- Add User Management sidebar link for admin users
- Fix Login.tsx role type cast to use UserRole instead of hardcoded union

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 11:46:33 +01:00
Vadym Samoilenko
6f963ff7c4 feat: DCMP compliance, descriptive transcript, new languages, QA bug fixes
- Rewrote VTT translation to two-step (text-only → Gemini → apply to original timestamps) preventing caption timing desync
- Added polling fallback for all processing states and Safari visibilitychange WebSocket reconnect
- Added 11 new TTS languages (cs, da, fi, hu, no, sk, sv, es-419, pt-BR, fr-CA)
- Updated caption/AD prompts to DCMP Captioning Key & Description Key standards (line splitting, ♪ music notation, italic tags, caption positioning, ethics guidelines)
- Added descriptive transcript generation (WCAG 2.1 §1.2.1) combining captions + AD into plain text
- Fixed amix normalize=0 to prevent audio loss in rendered videos
- Fixed AD re-timing double-count when source_ms is None
- Fixed cue block numbering to be 1-based in VttEditor and Timeline Preview

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 11:50:43 +00:00
Vadym Samoilenko
f4ddcce066 fix: resolve QA-reported bugs — MP3/VTT desync, crashes, notifications, and more
**BUG-1 & BUG-2 — Wrong audio plays after re-render / MP3 doesn't match text**
Root cause: audio files were named by index (cue_0.mp3, cue_1.mp3). When a cue
was inserted or deleted, all following indices shifted but old MP3 files kept
their original names, so re-render would play the wrong audio for the wrong cue.
Fix: renamed files to cue_N_CONTENTHASH.mp3 and introduced an ad_cue_manifest
stored in the job document that maps each cue index to its correct GCS URI.
Re-render now reads from the manifest instead of guessing by filename.
Also: editing AD cue text in the VTT editor now automatically queues TTS
regeneration for changed cues — no more silent mismatches.

**BUG-3 — App crash / state desync when uploading VTT or clearing TTS queue**
Fixed handleVttFileUpload to only update local editor state after the server
confirms the save — previously local state was updated first, so a network
error left the UI showing content that wasn't actually saved.
Fixed handleClearRegenerationQueue to only remove items from local state if
the server removal succeeded — previously all items were cleared regardless.

**BUG-4 — AI generates different audio descriptions every time**
Added GenerateContentConfig(temperature=0.2, top_p=0.8, top_k=40) to the
Gemini API call so output is more consistent across runs.

**BUG-5 — On-screen text inconsistently described**
Strengthened the AI prompt rule from a vague suggestion to a mandatory
requirement with an explicit format: "Text on screen reads: [exact text]".
Applied to both gemini_ingestion.md and gemini_ingestion_targeted.md.

**BUG-6 — No notification when re-render finishes**
Added rendering_qc toast notification and a dismissible green banner that
appears in QCDetail when re-render transitions to pending_qc. The banner
auto-dismisses after 10 seconds. Also increased WebSocket reconnect attempts
from 5 to 15 and capped backoff at 60s to prevent falling back to manual refresh.

**BUG-7 — Timeline preview looks accurate but isn't after edits**
Added isStale prop to TimelinePreview. The timeline now shows an amber tint
and "Preview may be outdated" label whenever there are unsaved pause point
changes, pending TTS regenerations, or a new VTT has been uploaded.

**BUG-8 — ElevenLabs API errors break TTS with no fallback**
Added try/except fallback chain in _synthesize_single_cue: if the configured
provider fails, it automatically retries with google, then gemini.

**BUG-9 — Concurrent re-render requests cause race conditions**
Made the PENDING_QC → RENDERING_QC status transition conditional (only
succeeds if the job is still in PENDING_QC). Returns HTTP 409 if a re-render
is already in progress. The completion transition back to PENDING_QC is also
conditional so a cancelled/overridden render doesn't corrupt job state.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 13:23:55 +00:00
Vadym Samoilenko
2245a12829 fix: case-insensitive Microsoft user lookup to prevent duplicate key error
Microsoft can return different email casings for the same user (e.g.
VadymSamoilenko@... vs vadymsamoilenko@...). The previous case-sensitive
find_one would miss the existing user, then fail on insert_one with a
duplicate key error on the _id field (ms-{sub[:20]}).

Fix: look up by _id first (deterministic from Microsoft sub), then fall
back to case-insensitive email regex for local-to-Microsoft migrations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 10:42:58 +00:00
Vadym Samoilenko
c413fcb747 feat: add SDH (Subtitles for Deaf and Hard of Hearing) caption output
SDH captions extend standard VTT with speaker identification labels,
sound effects [PHONE RINGS], music notation ♪, and off-screen indicators.

- Add sdh_vtt flag to RequestedOutputs model and frontend form
- Add sdh_captions_vtt_gcs field to LangOutput model
- Inject SDH generation instructions into both Gemini prompts via
  {SDH_FIELD} and {SDH_GUIDELINES} placeholders when requested
- Upload sdh_captions.vtt to GCS in ingest task
- Pass SDH through video_native translation (Gemini generates it directly)
  and traditional translation (translate source SDH VTT via Gemini)
- Expose sdh_captions_vtt in downloads endpoint and bulk zip export

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 15:02:18 +00:00
Vadym Samoilenko
2e8a8dc287 feat: add brand context, ethics guidelines, and improved AD prompt rules
- Add brand_context field (job model, API, frontend form) so clients can
  list brand names present in their video; Gemini uses these names instead
  of generic descriptors (e.g. "Sellotape" not "sticky tape")
- Add ethical guidelines section to both Gemini prompts covering
  person-first language, consistent race/gender description only when
  plot-relevant, no guessing at unconfirmed identity
- Revamp audio description rules: priority ordering (essential →
  high-priority → time-permitting), pre-teaching placement, no cinematic
  jargon, succinct style replacing the former "20% longer" instruction
- Thread brand_context through full stack: routes → job doc → ingest
  task → translate task → both Gemini prompt templates

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 14:46:09 +00:00
Vadym Samoilenko
c6c7ff51c7 fix: clear stale pause points when AD VTT is re-uploaded
Old pause_points in edit_state always overrode new VTT cue timings
during re-render, making AD VTT upload for timing adjustments
non-functional. Clear pause_points and video_segments on AD VTT
upload so re-render falls back to the new cue start times.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 17:07:55 +00:00
Vadym Samoilenko
222826baa7 fix: propagate ElevenLabs voice fetch errors to frontend
- elevenlabs_voices.py: re-raise exception on first fetch failure
  (empty cache) instead of silently returning empty list
- routes_tts.py: catch get_voices() exception and return available=False
  with the error detail; add optional error field to ProviderVoicesResponse
- VoiceSelector: show actual API error message when available=false

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:27:45 +00:00
Vadym Samoilenko
a22fe5c1bc fix: surface ElevenLabs config errors and add availability flag
- Extract actual error message from blob response in previewVoice so
  users see the real API error instead of generic "Failed to generate preview"
- VoicePreviewButton now reads err.message from thrown Error objects
- Add available: bool field to ProviderVoicesResponse; returns false
  when ELEVENLABS_API_KEY is not configured so the frontend can react
  proactively instead of hitting a 400 on preview
- VoiceSelector shows a descriptive config warning when available=false

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 14:17:00 +00:00