Commit graph

253 commits

Author SHA1 Message Date
Michael Clervi
975c489693 fix: regenerate descriptive transcript on VTT edit + handle empty string VTT fields
Some checks failed
Deploy Backend / Deploy API to Cloud Run (push) Has been cancelled
CI / Backend Lint & Test (push) Has been cancelled
CI / Frontend Lint & Test (push) Has been cancelled
CI / Security Scan (push) Has been cancelled
CI / Dependency Check (push) Has been cancelled
Deploy Backend / Deploy Worker to Cloud Run (push) Has been cancelled
Deploy Backend / Run Smoke Tests (push) Has been cancelled
Deploy Backend / Notify Deployment Status (push) Has been cancelled
CI / Integration Tests (push) Has been cancelled
CI / Build Backend Docker Image (push) Has been cancelled
CI / Build Frontend (push) Has been cancelled
2026-05-01 10:40:48 +00:00
Vadym Samoilenko
e7917cde10 fix: use 'is None' check for Motor collection to avoid NotImplementedError
PyMongo Collection raises NotImplementedError on bool(), so 'if not self.collection'
crashes on every audit log write. Changed to 'if self.collection is None'.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:23:44 +01:00
Vadym Samoilenko
dee4d69b40 fix: raise user list size limit to 500 and guard toLocaleString calls
- routes_admin.py: size query param max raised from 100 → 500 so
  ClientDetail.tsx (size=200) no longer returns 422
- GlossaryDetail.tsx: three .toLocaleString() calls guarded with ?? 0
  to prevent TypeError when term_count is undefined on first render

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:20:38 +01:00
Vadym Samoilenko
e48d63bdbd fix: generate valid ObjectId for audit log entries
default_factory=PyObjectId produced "" (empty string) since
Annotated[str, ...] is a type annotation, not a callable factory.
Replace with lambda: str(ObjectId()) to generate a real unique ID.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 15:06:15 +01:00
Vadym Samoilenko
0d46c1440c fix: remove unused TypeScript imports and variables
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:43:36 +01:00
Vadym Samoilenko
85e1e852ed fix: add --no-root to poetry install in Dockerfiles (Poetry 2.x) 2026-04-29 14:35:28 +01:00
Vadym Samoilenko
5cd2fb2743 fix: regenerate poetry.lock + align whisper Dockerfile poetry version
poetry.lock was out of sync with pyproject.toml (cost-tracker and
glossary deps added since last lock). Regenerated with Poetry 2.1.4.
Also updated Dockerfile.whisper-service from poetry==1.8.2 to 2.1.4
to match the main Dockerfile and avoid format incompatibility.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:32:41 +01:00
Vadym Samoilenko
a3b300b76a docs: add canonical documentation + audit cleanup
- AGENTS.md: canonical project entry point (Quick Nav, pipeline, constraints)
- docs/: complete docs tree — architecture, API spec, DB schema, infra,
  runbook, requirements, tech stack, principles, reference ADRs, guides,
  tasks backlog, testing strategy
- tests/README.md: test commands, structure, known gaps
- README.md / CLAUDE.md / DEPLOYMENT.md: updated with canonical doc links
- .archive/: backup of pre-documentation-pipeline originals
- backend/uv.lock: uv dependency lockfile
- Delete committed __pycache__ .pyc files (should have been gitignored)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:22:51 +01:00
Vadym Samoilenko
fd154e7799 fix: upgrade poetry in Dockerfile from 1.8.2 to 2.1.4
poetry.lock was generated with 2.1.4 — using 1.8.2 caused
incompatible lock file error and failed Docker builds.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:21:42 +01:00
Vadym Samoilenko
743a8597c2 fix: auto-sync poetry.lock during Docker build
Prevents build failures when pyproject.toml changes without a lock regen.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:21:01 +01:00
Vadym Samoilenko
4c6624c3d4 fix: code health sweep — M-01 through M-07
M-01 authz.py: move cache_key above try block to avoid NameError when
     first Redis call returns None
M-02 main.py: re-enable validation middleware (was TEMPORARILY DISABLED)
M-03 routes_auth.py / main.py: replace print() debug lines with
     structured logger calls; logger now module-level in routes_auth.py
M-04 gcs.py: asyncio.get_event_loop() → get_running_loop() (deprecation)
M-05 translate_and_synthesize.py: bind loop vars in closure defaults
     to fix B023 ruff warnings (transcreate/translate_captions/etc.)
M-06 rate_limiting.py: only trust X-Forwarded-For when X-Forwarded-Proto
     is https; use rightmost entry (proxy-appended) not leftmost
M-07 validation.py: extend MongoDB operator blocklist to cover $expr,
     $function, $accumulator, $nin, $gte, $lte, $jsonSchema, $mod

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:18:02 +01:00
Vadym Samoilenko
86ef5a86fb refactor: extract broadcast_status_update to shared _websocket_bridge (H-08)
The function was copy-pasted identically in ingest_and_ai.py and
translate_and_synthesize.py. Extracted to tasks/_websocket_bridge.py
as the single definition; all four task modules now import from there.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:14:57 +01:00
Vadym Samoilenko
87ae6571fe perf: use DI connection pool for auth routes, async httpx for MS SSO (H-01, H-02)
- login and microsoft_login routes now use Depends(get_database) instead
  of creating a per-request MongoClient — removes connection-pool churn
  under load
- MicrosoftAuthService._get_openid_config/_get_jwks/validate_token are
  now async, using httpx.AsyncClient instead of blocking requests.get —
  removes ~400ms event-loop block per Microsoft login
- Removed unused AsyncIOMotorClient import from routes_auth.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:13:50 +01:00
Vadym Samoilenko
c6b19d01f2 security: remove default admin password fallback (C-04)
seed_default_admin now skips creation and logs a warning when
DEFAULT_ADMIN_PASSWORD is unset instead of falling back to the
hardcoded ChangeMe123! value. Existing-admin promotion path is
unaffected. Added DEFAULT_ADMIN_PASSWORD to .env.prod.example.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:12:24 +01:00
Vadym Samoilenko
e81acebc45 security: remove exception detail from /auth/refresh response (C-03)
Replaced the bare except that leaked str(e) (JWT library internals,
claim validation messages) with a generic "Invalid refresh token" detail.
Full traceback is now logged server-side via the structured logger.
Re-raises HTTPException before the generic handler so valid 401s from
inner checks are not double-wrapped.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:11:59 +01:00
Vadym Samoilenko
70f6c6befb security: reject refresh tokens used as access tokens (C-02)
get_current_user and get_current_user_optional now reject any token
whose payload carries type="refresh". Access tokens carry no type field
so the check is asymmetric and safe. Prevents a refresh-cookie value
from being replayed as a Bearer access token.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:11:50 +01:00
Vadym Samoilenko
93cb7527ab security: enforce rate limit on /auth/login (C-01)
Removed /api/v1/auth/login from the rate-limit bypass list in both
rate_limiting.py and main.py. The existing 5-req/5-min limit for the
login endpoint was already configured but never applied.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:11:36 +01:00
Vadym Samoilenko
103b409f78 fix: handle role as str or Enum in audit_logger
user.role stored as plain string in MongoDB — calling .value on it
caused AttributeError on every login, blocking all auth.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 14:09:46 +01:00
Vadym Samoilenko
fa351e4d25 feat: per-client glossary — hybrid exact/vector retrieval + AI injection
Adds full glossary system so Gemini uses client-approved terminology
when generating subtitles and translations (critical for 3M brand names
and product codes across 16 target locales).

Backend:
- lib/locales.py: BCP-47 locale registry, normalises xlsx fr_fr → fr-FR
- models/glossary.py: Glossary / GlossaryVersion / GlossaryTerm + enums
- services/glossary_service.py: xlsx parse (openpyxl), ingest to Mongo,
  hybrid retrieval (Aho-Corasick exact + Atlas Vector Search), prompt block
- services/embedding_service.py: Gemini text-embedding-004, batch 100, retry
- tasks/embed_glossary.py: Celery background task for async embedding
- api/v1/routes_glossaries.py: CRUD endpoints under /clients/{id}/glossaries
- gemini.py: _build_glossary_block(), {GLOSSARY} injection in all 4 call sites
- tts.py / gemini_tts.py: pass full locale codes (no split("-")[0] truncation)
- tasks/translate_and_synthesize.py: glossary lookup + injection per language
- prompts: {GLOSSARY} placeholder in ingestion, targeted, transcreation prompts
- pyproject.toml: +openpyxl, +pyahocorasick

Frontend:
- routes/admin/glossaries/: GlossaryList, GlossaryUpload, GlossaryDetail
- App.tsx: 3 new routes under /admin/clients/:clientId/glossaries
- ClientDetail.tsx: Glossaries card with count + quick links
- types/api.ts: Glossary, GlossaryVersion, GlossaryDetail, GlossaryTerm types
- lib/api.ts: 7 new API methods (upload, list, detail, terms, versions, activate, archive)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 13:03:38 +01:00
Vadym Samoilenko
05f25a1141 feat: per-language QC workflow with linguist assignment
- Job.language_qc dict tracks per-language status (pending/in_review/approved/rejected)
  with full event history; qc_assignments denormalized array enables efficient queue queries
- language_qc service handles assign/reassign/approve/reject/reopen with atomic DB updates,
  audit logging, and auto-advancement to pending_final_review when all languages approved
- Linguists can only edit VTT and trigger re-renders for their assigned language (403 guard)
- return_to_qc resets all language statuses while preserving assignments
- routes_language_qc.py: 7 new endpoints; /me/language-qc-queue for linguist queue
- Startup migration idempotently seeds language_qc for all existing jobs
- Frontend: LanguageQCState types, API methods, LinguistQueue page, QCDetail redesigned
  with per-language status badges, assignment dropdown, inline approve/reject buttons,
  progress bar, and reject modal; My QC Queue sidebar link

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 12:09:40 +01:00
Vadym Samoilenko
bab30e1508 feat: VTT version control — snapshots, diff, restore
Backend:
- VttVersion model (vtt_version.py): immutable snapshot per job/lang/kind/version
- vtt_versioning service: create_version (atomic counter + GCS snapshot),
  list_versions, get_version, restore_version, diff_versions (difflib line-level)
- routes_vtt_versions.py: GET /versions, GET /versions/{v}, GET /versions/diff,
  POST /versions/{v}/restore (PRODUCTION/ADMIN only, overwrites live file + audit log)
- Hook create_version into update_job_vtt_content before each live-file overwrite
- Mongo indexes: unique (job_id, lang, kind, version) + (job_id, created_at)

Frontend:
- VttVersionSummary / VttVersionFull / VttDiffResponse types
- api.ts: listVttVersions, getVttVersion, diffVttVersions, restoreVttVersion
- VersionsTab.tsx: lang/kind switcher, version list with A/B compare buttons,
  inline diff viewer (color-coded +/−), content viewer, restore with confirm dialog
- JobDetail.tsx: new "VTT Versions" tab wired to VersionsTab

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 11:46:21 +01:00
Vadym Samoilenko
02c4240e6d fix: remove cost.oliver.agency links from QCDetail and FinalDetail
Replace inline anchor links with plain text — the correct dashboard URL
is only on the UserList button.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 11:36:03 +01:00
Vadym Samoilenko
848f7697f3 fix: revert cost tracker URL in QCDetail/FinalDetail inline links
Only the main AI Cost Dashboard button in UserList should use the new URL.
The inline helper links inside the Cost Tracker Project ID inputs stay on cost.oliver.agency.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 11:35:36 +01:00
Vadym Samoilenko
09550cfca0 feat: audit log integration sweep + cost tracker URL fix + audit log admin UI
- Fix cost tracker dashboard URL (cost.oliver.agency → optical-dev.oliver.solutions/cost-tracker/analytics)
  in UserList, QCDetail, FinalDetail; centralise into src/lib/costTracker.ts

- Wire audit logging across backend routes (was 1 call site, now covers all key events):
  · routes_auth: LOGIN_SUCCESS/FAILURE for local + MS SSO, LOGOUT
  · routes_files: FILE_UPLOAD on signed URL generation
  · routes_jobs: JOB_CREATE, JOB_APPROVE, JOB_REJECT, JOB_STATUS_CHANGE, JOB_DELETE, VTT_EDIT
  · routes_admin: USER_CREATE, USER_UPDATE, USER_ROLE_CHANGE, USER_DEACTIVATE

- Add Audit Log admin UI page (/admin/audit-log):
  · Three tabs: All Events (paginated, server-side filters), Security Events, User Activity
  · Filters: action group, severity, success/failure, free-text search
  · Click-to-expand row shows IP, request ID, resource, details JSON
  · Wired into App.tsx (RoleGate: production + admin) and sidebar nav

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 11:34:06 +01:00
Vadym Samoilenko
0f15d192cb feat(saas): Phase 4 — org settings IA, sidebar org-switcher, invitation UI
- Add /no-access empty-state page for zero-membership users
- Add /org/:orgSlug/settings/* routes (members, teams, invitations, general)
- OrgSettingsLayout with tab nav (NavLink-based)
- OrgMembersPage: member table with search, role editor, remove action
- OrgInvitationsPage: list with status badges + revoke
- OrgTeamsPage: read-only teams list
- OrgGeneralPage: org info display
- InviteMemberModal: email + role form → POST /organizations/:id/invitations
- Sidebar: org-switcher (single label / multi-org dropdown), currentOrgSlug
  derived from route params or first membership, Settings gear link at bottom

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 17:02:11 +01:00
Vadym Samoilenko
1563714454 feat(saas): Phase 3 — membership-based authz + Mailgun + job.organization_id
authz.py (new):
- MembershipContext — per-request membership dict for the current user
- get_membership_context FastAPI dependency
- require_org_role(min_role) — dependency factory keyed off org_id path param
- require_platform_admin()
- OrgScopedQuery — adds organization_id filter; platform admin passes through
- bump_user_membership_cache — invalidates Redis key on membership writes

dependencies.py:
- get_accessible_project_ids now queries memberships collection first;
  legacy pm_client_ids / team.member_user_ids fallback retained until migration runs
  (four job-route access checks at lines 608/1054/1181/1538 are fixed via this function)

routes_clients.py:
- _assert_pm_or_admin and _assert_client_access are now async and query memberships
- All 10 call sites updated with await + db arg

emailer.py:
- Switched from SendGrid to Mailgun REST API via httpx (already in requirements)
- _send() is now fully async; same public method signatures preserved
- send_completion_email uses _send()

config.py:
- Added mailgun_api_key, mailgun_domain, mailgun_from settings
- sendgrid_api_key kept with empty default for backward compat

migration_2026-04-28-000003:
- Backfills job.organization_id from project.client_id
- Creates (organization_id, status, created_at) sparse index on jobs

routes_organizations.py / routes_invitations.py:
- Call bump_user_membership_cache after every membership write

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:56:42 +01:00
Vadym Samoilenko
00fb1aacc6 feat(saas): Phase 2 — invitation flow, email templates, MS SSO zero-membership
Backend:
- models/invitation.py — Invitation model + create/accept/preview schemas
- routes_invitations.py — org-scoped POST/GET/DELETE + public preview/accept endpoints
  Single-use token via find_one_and_update; sha256(token) stored in DB, plaintext in email URL
- emailer.py — _send() helper; send_invitation_email, send_welcome_email, send_password_reset_email
  send_completion_email refactored to use _send()
- migration_2026-04-28-000002 — creates invitations collection with TTL index (30d audit trail)
- routes_auth.py — new MS SSO users provisioned with zero memberships instead of role=PRODUCTION;
  they land on "no access" page until an admin invites them
- main.py — registers invitations_org_router and invitations_router

Frontend:
- routes/AcceptInvite.tsx — public page at /accept-invite?token=...
  Four states: new user (name+password), existing user (confirm), MS user, already-member
- App.tsx — /accept-invite route outside RequireAuth
- types/api.ts — Invitation, InvitationCreate, InvitationPreview, InvitationAcceptRequest/Response
- lib/api.ts — listInvitations, createInvitation, revokeInvitation, previewInvitation, acceptInvitation
- hooks/useClients.ts — useInvitations, useCreateInvitation, useRevokeInvitation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:52:08 +01:00
Vadym Samoilenko
6f1be645ce feat(saas): Phase 0+1 — Organization/Membership entities and dev branch
Introduces the multi-tenant SaaS foundation alongside the existing
client/team/project model (zero-downtime shim period):

Backend:
- app/models/organization.py — Organization + OrgRole enum (OWNER/ADMIN/MANAGER/MEMBER/VIEWER)
- app/models/membership.py — Membership model with MemberDetail for enriched responses
- app/services/membership_service.py — upsert/remove/list/has_org_role helpers
- app/api/v1/routes_organizations.py — /organizations CRUD + /members sub-resource + /me/memberships
- main.py — registers organizations router
- migrations: create memberships collection (unique index) + backfill from pm_client_ids/team members

Frontend:
- types/api.ts — Organization, OrgRole, Membership, OrganizationCreateRequest types; Client marked @deprecated
- hooks/useClients.ts — useOrganizations, useOrganization, useOrgMembers, useAddOrgMember,
  useUpdateOrgMember, useRemoveOrgMember, useMyMemberships
- lib/api.ts — listOrganizations, getOrganization, createOrganization, updateOrganization,
  listOrgMembers, addOrgMember, updateOrgMember, removeOrgMember, getMyMemberships

Reads fall back to the clients collection during transition; all writes go to organizations.
Existing /clients endpoints and hooks are untouched.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:46:24 +01:00
Vadym Samoilenko
269ab09fa6 fix: serialize Client/Team/Project with id not _id + guard undefined client hooks
Pydantic v2 + FastAPI serializes Field(alias="_id") as _id in JSON,
so client.id was always undefined on the frontend — causing option
values to fall back to text content ("3M") and firing /clients/3M/teams 404s.

- Remove Field(alias="_id") from Client/Team/Project models; id is now a
  plain string field populated explicitly in _client_from_doc etc.
- API now returns id not _id, matching the TypeScript Client interface
- Add clientId !== "undefined" guard to useTeams, usePMs, useProjects

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:18:28 +01:00
Vadym Samoilenko
723bbbc695 fix: add project_manager migration + add migration step to full-deploy.sh
- New migration updates MongoDB users collection validator to accept
  project_manager role and pm_client_ids field
- full-deploy.sh was missing the run_migrations step entirely; added it
  after rebuild_containers so new role/field validators apply on every deploy

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:05:17 +01:00
Vadym Samoilenko
bbd324e3c7 feat: add Project Manager role + client/team assignment panel in admin user editor
- Add project_manager to all role dropdowns (UserList filter, create modal, UserDetail edit form)
- Add indigo badge color for project_manager in user list table
- Expose pm_client_ids in UserResponse schema and all admin user endpoints
- Add pm_client_ids to frontend User type
- Add UserAssignmentsPanel to UserDetail sidebar: PM users see client toggle list; other roles see client → team membership picker
- Add flexible hooks (useTeamsForClient, useAssignPMAny, useRemovePMAny, useAddTeamMemberAny, useRemoveTeamMemberAny)
- Fix useClient guard against literal "undefined" string causing 404 requests

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 15:58:55 +01:00
Vadym Samoilenko
9af1ac3e95 fix: correct useAuthStore import path in ClientDetail
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 15:45:24 +01:00
Vadym Samoilenko
acdbb34b9b fix: cost_tracker uses /usage/record endpoint + correct allow/deny_reason fields; suppress edit-state 404 retries
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 15:11:39 +01:00
Vadym Samoilenko
2b721d182b feat: Client → Team → Project isolation system with Project Manager role
Backend:
- New UserRole.PROJECT_MANAGER with pm_client_ids[] on User model
- New models: Client (slug-based), Team (member_user_ids[]), Project (client-scoped)
- Job model gains project_id field
- New GET/POST/PATCH/DELETE /clients, /clients/{id}/teams, /clients/{id}/projects,
  /clients/{id}/pm routes (admin-only client CRUD; PM or admin for teams/projects)
- get_accessible_project_ids() helper: staff→all, PM→their clients' projects,
  CLIENT→projects from teams they belong to (with legacy owner fallback)
- list_jobs, get_job, bulk_download, get_vtt_content, delete_job all use new isolation

Frontend:
- UserRole type gains 'project_manager'
- Job, JobCreateRequest gain project_id field
- Client, Team, Project, PMUser types added
- ApiClient: full client/team/project/PM CRUD methods
- useClients hook with all query/mutation hooks
- Admin pages: ClientList + ClientDetail (teams, members, projects, PM assignment)
- NewJob form: client + project picker (shown when clients exist)
- Sidebar: Clients nav item for admin and project_manager roles
- Routes: /admin/clients and /admin/clients/:clientId behind RoleGate

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 15:11:13 +01:00
Michael Clervi
f7d4624fc7 fix: correct cost tracker API field names and endpoint path
- preflight: use allow (not allowed), deny_reason, project_external_id
- record: units dict {token_input/token_output/char} instead of flat fields
- record: use /usage/record endpoint path

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 13:42:29 +00:00
Vadym Samoilenko
1aa078db94 fix: suppress downloads API call for non-completed jobs on Downloads page
useJobDownloads was firing for any non-early status (pending_qc, translating,
etc.), hitting the backend /downloads endpoint which returns 400 when no outputs
exist yet. React Query then retried 3+ times.

Downloads page only renders content for completed jobs anyway, so disable the
query for any other status by passing 'created' (which is in EARLY_STATUSES).
JobDetail is unaffected — it uses the hook separately with its own status guard.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 14:07:34 +01:00
Vadym Samoilenko
ea21cace96 feat: replace SDK with direct HTTP integration to centralized cost tracker
- New services/cost_tracker.py: sync httpx preflight()/record() + async wrappers;
  BudgetExceeded exception; no-op when COST_TRACKER_BASE_URL is empty
- Preflight budget check added before ingestion (Gemini), per-language translation
  (video-native + traditional), and per-language TTS dispatch
- _record_gemini_usage and _record_tts_cost now call cost_tracker directly;
  removes broken asyncio.get_event_loop() hack from sync Celery worker
- Fix: _cost_ctx now threaded into extract_accessibility_targeted (video-native path)
- Fix: user_id/cost_project_id now propagated through dispatch_language_tts →
  synthesize_cue_task.s() and the rerender_accessible_video.py re-render path
- Remove oliver-cost-tracker SDK dependency (was commented-out/never installed)
- Drop cost_tracker_outbox_path setting and get_cost_tracker() factory
- Update COST_TRACKER_BASE_URL default to optical-dev.oliver.solutions in
  .env.prod.example, docker-compose.yml, and all Cloud Run service yamls
- Cloud Run yamls use Secret Manager ref (cost-tracker-api-key) for the API key

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 13:36:15 +01:00
Vadym Samoilenko
26bfedd7c7 feat: add cost_tracker_project_id assignment UI to QC and Final Review
- PATCH /jobs/{job_id} endpoint for updating title and cost_tracker_project_id
- cost_tracker_project_id exposed on JobResponse (GET /jobs/{id})
- Inline project ID field in QCDetail and FinalDetail — saved via PATCH
- "AI Cost Dashboard" link in UserList header
- cost_tracker_project_id added to Job type and JobUpdateRequest schema

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 11:40:13 +01:00
Vadym Samoilenko
ae2c474061 feat: integrate oliver-cost-tracker SDK into video-accessibility
Add AI cost tracking to all Gemini and TTS call sites:

- config.py: add COST_TRACKER_* env vars (base_url, api_key, source_app,
  outbox_path, enabled)
- dependencies.py: add get_cost_tracker() factory (lru_cache, graceful
  degradation if SDK not installed)
- models/job.py: add cost_tracker_project_id field for cost attribution
- services/gemini.py:
  - add import time, _record_gemini_usage() helper (reads usage_metadata)
  - add _cost_ctx kwarg to extract_accessibility, extract_accessibility_targeted,
    transcreate_content, translate_vtt, rewrite_tts_cue
  - record usage after every generate_content call via asyncio.create_task()
- tasks/ingest_and_ai.py: pass _cost_ctx (user_id, job_id, project_id) to
  extract_accessibility
- tasks/translate_and_synthesize.py: build _cost_ctx from job_doc and pass
  to transcreate_content + translate_vtt calls
- tasks/tts_synthesis.py: add user_id + cost_project_id kwargs, add
  _record_tts_cost() helper (records len(text) chars to cost tracker)
- pyproject.toml: document SDK install instructions (comment)
- .env.prod.example: add COST_TRACKER_* vars

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 11:30:46 +01:00
Vadym Samoilenko
b5e5ad7e42 fix: deploy.sh pull_code now works with monorepo structure
Was checking backend/.git and frontend/.git (submodule pattern) which
never exists — git pull silently skipped, deploying stale code.
Now pulls root repo first, falls back to submodule pattern.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 11:50:20 +01:00
Vadym Samoilenko
cf761c4bb6 feat: add linguist role and user management navigation
- Add LINGUIST role to UserRole enum (backend + frontend)
- Grant linguists access to QC Review, Final Review, review notes, and VTT editing
- Add MongoDB migration to update schema validator with linguist role
- Add admin seed: vadymsamoilenko@oliver.agency is promoted to admin on startup
- Add User Management sidebar link for admin users
- Fix Login.tsx role type cast to use UserRole instead of hardcoded union

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 11:46:33 +01:00
Vadym Samoilenko
c735ba4bb5 fix: add ElevenLabs API key to production env
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 14:28:09 +00:00
Vadym Samoilenko
8356dbdbfe fix: add charset=utf-8 to VTT content-type to prevent ♪ encoding issues
Without charset specification, browsers/tools interpret text/vtt as
Latin-1, causing UTF-8 multi-byte characters like ♪ (U+266A) to render
as garbled text (♪).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 14:17:16 +00:00
Vadym Samoilenko
084c37d1a7 fix: add SDH captions and descriptive transcript to QC Download Assets panel
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 13:51:34 +00:00
Vadym Samoilenko
83919c19b5 fix: add SDH captions to downloads page
SDH (sdh_captions_vtt) was missing from the Downloads page type labels
and filename extensions map.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 13:47:35 +00:00
Vadym Samoilenko
05a2fe5101 fix: timeline disappears after QC re-render
- Set staleTime=0 on useAccessibleVideoEditState so edit-state is
  immediately refetched when invalidated after render completes
- Force prevJobStatusRef='rendering_qc' at render start so the
  pending_qc completion effect always fires, even when fast renders
  complete before the 10s polling interval catches rendering_qc status

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 12:31:06 +00:00
Vadym Samoilenko
6f963ff7c4 feat: DCMP compliance, descriptive transcript, new languages, QA bug fixes
- Rewrote VTT translation to two-step (text-only → Gemini → apply to original timestamps) preventing caption timing desync
- Added polling fallback for all processing states and Safari visibilitychange WebSocket reconnect
- Added 11 new TTS languages (cs, da, fi, hu, no, sk, sv, es-419, pt-BR, fr-CA)
- Updated caption/AD prompts to DCMP Captioning Key & Description Key standards (line splitting, ♪ music notation, italic tags, caption positioning, ethics guidelines)
- Added descriptive transcript generation (WCAG 2.1 §1.2.1) combining captions + AD into plain text
- Fixed amix normalize=0 to prevent audio loss in rendered videos
- Fixed AD re-timing double-count when source_ms is None
- Fixed cue block numbering to be 1-based in VttEditor and Timeline Preview

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 11:50:43 +00:00
Vadym Samoilenko
f4ddcce066 fix: resolve QA-reported bugs — MP3/VTT desync, crashes, notifications, and more
**BUG-1 & BUG-2 — Wrong audio plays after re-render / MP3 doesn't match text**
Root cause: audio files were named by index (cue_0.mp3, cue_1.mp3). When a cue
was inserted or deleted, all following indices shifted but old MP3 files kept
their original names, so re-render would play the wrong audio for the wrong cue.
Fix: renamed files to cue_N_CONTENTHASH.mp3 and introduced an ad_cue_manifest
stored in the job document that maps each cue index to its correct GCS URI.
Re-render now reads from the manifest instead of guessing by filename.
Also: editing AD cue text in the VTT editor now automatically queues TTS
regeneration for changed cues — no more silent mismatches.

**BUG-3 — App crash / state desync when uploading VTT or clearing TTS queue**
Fixed handleVttFileUpload to only update local editor state after the server
confirms the save — previously local state was updated first, so a network
error left the UI showing content that wasn't actually saved.
Fixed handleClearRegenerationQueue to only remove items from local state if
the server removal succeeded — previously all items were cleared regardless.

**BUG-4 — AI generates different audio descriptions every time**
Added GenerateContentConfig(temperature=0.2, top_p=0.8, top_k=40) to the
Gemini API call so output is more consistent across runs.

**BUG-5 — On-screen text inconsistently described**
Strengthened the AI prompt rule from a vague suggestion to a mandatory
requirement with an explicit format: "Text on screen reads: [exact text]".
Applied to both gemini_ingestion.md and gemini_ingestion_targeted.md.

**BUG-6 — No notification when re-render finishes**
Added rendering_qc toast notification and a dismissible green banner that
appears in QCDetail when re-render transitions to pending_qc. The banner
auto-dismisses after 10 seconds. Also increased WebSocket reconnect attempts
from 5 to 15 and capped backoff at 60s to prevent falling back to manual refresh.

**BUG-7 — Timeline preview looks accurate but isn't after edits**
Added isStale prop to TimelinePreview. The timeline now shows an amber tint
and "Preview may be outdated" label whenever there are unsaved pause point
changes, pending TTS regenerations, or a new VTT has been uploaded.

**BUG-8 — ElevenLabs API errors break TTS with no fallback**
Added try/except fallback chain in _synthesize_single_cue: if the configured
provider fails, it automatically retries with google, then gemini.

**BUG-9 — Concurrent re-render requests cause race conditions**
Made the PENDING_QC → RENDERING_QC status transition conditional (only
succeeds if the job is still in PENDING_QC). Returns HTTP 409 if a re-render
is already in progress. The completion transition back to PENDING_QC is also
conditional so a cancelled/overridden render doesn't corrupt job state.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-24 13:23:55 +00:00
Vadym Samoilenko
2245a12829 fix: case-insensitive Microsoft user lookup to prevent duplicate key error
Microsoft can return different email casings for the same user (e.g.
VadymSamoilenko@... vs vadymsamoilenko@...). The previous case-sensitive
find_one would miss the existing user, then fail on insert_one with a
duplicate key error on the _id field (ms-{sub[:20]}).

Fix: look up by _id first (deterministic from Microsoft sub), then fall
back to case-insensitive email regex for local-to-Microsoft migrations.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 10:42:58 +00:00
Vadym Samoilenko
c413fcb747 feat: add SDH (Subtitles for Deaf and Hard of Hearing) caption output
SDH captions extend standard VTT with speaker identification labels,
sound effects [PHONE RINGS], music notation ♪, and off-screen indicators.

- Add sdh_vtt flag to RequestedOutputs model and frontend form
- Add sdh_captions_vtt_gcs field to LangOutput model
- Inject SDH generation instructions into both Gemini prompts via
  {SDH_FIELD} and {SDH_GUIDELINES} placeholders when requested
- Upload sdh_captions.vtt to GCS in ingest task
- Pass SDH through video_native translation (Gemini generates it directly)
  and traditional translation (translate source SDH VTT via Gemini)
- Expose sdh_captions_vtt in downloads endpoint and bulk zip export

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-18 15:02:18 +00:00