---
name: "Accessible Video Processing Platform"
client: Oliver Internal
status: production-ready
tech: [React, FastAPI, Celery, MongoDB, Redis, Google Cloud (GCS, Gemini), Docker]
local_path: /Users/ai_leed/Documents/Projects/Oliver/video-accessibility
deploy: ./scripts/deploy-dev.sh
url: https://optical-dev.oliver.solutions/video-accessibility/
server: optical-dev
tags: [oliver, video, accessibility, ai, captions, audio-description, tts, celery, mongodb]
created: 2026-04-14
last_commit: 2026-05-14
commits: 412
port: 8000
db: MongoDB Atlas
service: docker-compose
---

## Overview

AI-powered SaaS platform that generates legally-required accessibility assets (closed captions, audio descriptions, SDH captions, descriptive transcripts) from video files. Processes videos through an async Celery pipeline using Gemini 2.5 Pro, routes outputs through human QC workflows with per-language approval, supports 50+ languages with transcreation, and delivers final assets via signed GCS URLs. **Production-ready (100%).**

## Tech Stack

| Layer | Tech |
|-------|------|
| Frontend | React 18 + TypeScript (Vite) |
| Backend | FastAPI (Python 3.11+) + Pydantic |
| Async workers | Celery + Redis |
| Database | MongoDB Atlas + Redis (cache/queue) |
| Storage | Google Cloud Storage (signed V4 URLs, 24h expiry) |
| AI | Gemini 2.5 Pro (transcription), Google Translate, Google TTS / ElevenLabs |
| Infrastructure | Docker Compose, Apache mod_proxy + mod_proxy_wstunnel |
| Auth | JWT (access token in memory) + HttpOnly refresh cookie |

## Architecture

Three-tier monorepo: React SPA → FastAPI → Celery workers → MongoDB / Redis / GCS.

**Job state machine (16 states):**
```
CREATED → INGESTING → AI_PROCESSING → PENDING_QC
  → [QC_FEEDBACK ↔ PENDING_QC]
  → APPROVED_ENGLISH → TRANSLATING → TTS_GENERATING
  → RENDERING_VIDEO → RENDERING_QC → PENDING_FINAL_REVIEW → COMPLETED
  [REJECTED] (terminal from any QC state)
```

## Dev Commands

```bash
./scripts/run-local.sh           # Start all services (Docker)
./scripts/run-local.sh --rebuild # Rebuild after code changes
./scripts/run-local.sh --stop    # Stop all

cd frontend && npm run dev        # Frontend dev server (separate terminal)
cd backend && poetry run pytest   # Backend tests
cd frontend && npm test           # Frontend tests
cd backend && ruff check .        # Lint
cd backend && mypy .              # Type-check (run in Docker or with venv)
```

**Local test credentials:**

| Role | Email | Password |
|------|-------|----------|
| Admin | admin@example.com | admin |
| Reviewer | reviewer@example.com | reviewer |
| Client | client@example.com | client123 |

## Deployment

- **Server:** optical-dev (GCP VM)
- **Deploy path:** `/opt/video-accessibility/`
- **Deploy command:** `./scripts/deploy-dev.sh`
- **Frontend-only:** `./scripts/build-frontend.sh`
- **URL:** https://optical-dev.oliver.solutions/video-accessibility/
- **Service:** Docker Compose

**URL Map:**

| Resource | URL |
|----------|-----|
| Frontend SPA | `https://optical-dev.oliver.solutions/video-accessibility/` |
| Backend API | `https://optical-dev.oliver.solutions/video-accessibility-back/` |
| API health | `https://optical-dev.oliver.solutions/video-accessibility-back/health` |
| API docs | `https://optical-dev.oliver.solutions/video-accessibility-back/docs` |
| WebSocket | `wss://optical-dev.oliver.solutions/video-accessibility-back/api/v1/ws/` |

**Post-deploy verification:**
```bash
curl https://optical-dev.oliver.solutions/video-accessibility-back/health
docker compose ps
```

## Environment Variables

Key env vars (see `.env.prod.example` for complete list):

| Variable | Purpose |
|----------|---------|
| `MONGODB_URL` | MongoDB Atlas connection string |
| `REDIS_URL` | Redis queue URL |
| `GCS_BUCKET_NAME` | GCS bucket (`accessible-video`) |
| `GOOGLE_APPLICATION_CREDENTIALS` | Path to GCP service account JSON |
| `GEMINI_API_KEY` | Google Gemini API key |
| `JWT_SECRET_KEY` | JWT signing secret |
| `MS_SSO_CLIENT_ID/SECRET` | Microsoft OAuth (production) |
| `TTS_WORKER_CONCURRENCY` | Celery concurrency for TTS worker (default 2) |

## Security Notes

- Multi-tenancy: `MembershipContext` + `get_job_or_403` enforced on all 25+ job-action endpoints, review notes, VTT versions, WebSocket, and admin users
- Access tokens stored in memory only (never localStorage)
- Refresh tokens in HttpOnly cookies
- English-first QC: source language must be APPROVED before any target language can be approved
- Source VTT edits invalidate downstream language QC states and create version snapshots
- WebSocket: codes 4001/4003/4004/4403 are terminal (no reconnect); client heartbeat every 20s; server keepalive frame every 20s
- RBAC enforced server-side; all reviewer actions emit audit log entries

## Sessions
### 2026-05-14 – Check and fix the Video Accessibility
**Asked:** Check and fix the Video Accessibility tool's version control, user action logging, and cost tracking functionality.
**Done:** Deployed cost tracker updates to optical-dev server using FastAPI after VPN connection was established.

### 2026-05-14 – Check if Video Accessibility tool has
**Asked:** Check if Video Accessibility tool has version control, user action logging, and cost tracking implemented.
**Done:** Implemented audit logging for 41 write endpoints, added non-raising audit logger, integrated Whisper cost tracking, and merged 11 commits to main.

### 2026-05-14 – Check if Video Accessibility tool has
**Asked:** Check if Video Accessibility tool has version control, user action logs, and cost tracking implemented.
**Done:** Verified FastAPI project tests and ruff checks pass; all changes are additive with no breaking modifications.

### 2026-05-14 – Check if Video Accessibility tool has
**Asked:** Check if Video Accessibility tool has version control, user action logging, and cost tracking implemented.
**Done:** Confirmed version control for VTT is fully implemented with models, services, APIs and auto-snapshots; cost tracker exists at /cost-tracker/ endpoint.

### 2026-05-13 – Fix glossary embedding errors where AI
**Asked:** Fix glossary embedding errors where AI doesn't receive full context and dictionary during translation.
**Done:** Deleted 68,732 orphaned terms from archived glossaries that were blocking embedding tasks, then pushed changes and deployed to server.

### 2026-05-13 – Fix embedding dictionary errors where AI
**Asked:** Fix embedding dictionary errors where AI doesn't receive full context and vocabulary during translation.
**Done:** Deleted 2 unnecessary glossaries via UI to reduce processing load and allow the single active 3M glossary (6a04af1936ef6e4b6c21038e) to complete embedding via Vertex AI without 429 errors.

### 2026-05-13 – Fix embedding dictionary issues with AI
**Asked:** Fix embedding dictionary issues with AI translation context and retry delays in API calls.
**Done:** Parsed retryDelay from API errors, reduced concurrency from 5 to 2, increased max_retries, and pushed changes to git for server deployment.

### 2026-05-13 – Fix dictionary embedding errors where AI
**Asked:** Fix dictionary embedding errors where AI receives incomplete context and glossary during translation.
**Done:** Implemented locale fallback in glossary service and fixed AI ingestion to pass complete brand context and glossary terms.

### 2026-05-13 – Fix dictionary/glossary not working with incorrect
**Asked:** Fix dictionary/glossary not working with incorrect translations due to incomplete context passed to AI.
**Done:** Resolved locale fallback issues in glossary service and AI ingestion to properly match language variants and pass complete glossary context to prompts.

### 2026-05-08 – I need to analyze this transcript,
**Asked:** I need to analyze this transcript, but it appears to be incomplete or unclear. The user is asking in Russian: "what did you get with the hook now? show me" — but there's no context about what hook, what was accomplished, or what files were involved.
**Done:** Since the transcript doesn't contain sufficient information about actual work completed, I cannot generate the requested 3-line log accurately. I would need:

### 2026-05-08 – Asked | Done | Log
**Asked:** Asked | Done | Log
**Done:** ---|---|---|

### 2026-05-08 – Check why AI processing failed due
**Asked:** Check why AI processing failed due to exceeded API quota.
**Done:** Identified security issue with exposed API key in chat history and regenerated it in .env file, then restarted containers.

### 2026-05-08 – Developer reported AI Processing failure due
**Asked:** Developer reported AI Processing failure due to Gemini API quota exhaustion (429 error).
**Done:** Identified missing GCP secret and established SSH connection to optical-dev server via IAP tunnel.

### 2026-05-08 – Debug why failed jobs keep making
**Asked:** Debug why failed jobs keep making API requests and exceeding quota limits.
**Done:** Identified that `EARLY_STATUSES` in `useJob.ts` missing failed state cases, causing continuous polling; also flagged compromised API key needing rotation.

### 2026-05-07 – Migrate sandbox users to production environment
**Asked:** Migrate sandbox users to production environment | Reordered imports in ingest_and_ai.py and added strict=True to zip() in vtt.py, completed type-checking | ingest_and_ai.py, vtt.py
**Done:** —

### 2026-05-07 – Can we migrate all users and
**Asked:** Can we migrate all users and data from sandbox to production environment?
**Done:** Successfully migrated all 12 collections from optical to optical-dev, including 34,366 glossary terms and verified data integrity across all collections.

### 2026-05-07 – Can we migrate users and all
**Asked:** Can we migrate users and all data from the sandbox (optical) to optical-dev environment?
**Done:** Conducted reconnaissance and mapped data differences between optical and optical-dev instances across users, jobs, vtt_versions, audit_logs, clients, and migration_history.

### 2026-05-07 – Migrate all users and data from
**Asked:** Migrate all users and data from sandbox (optical) to optical-dev environment.
**Done:** Identified three migration components (GCS files, MongoDB collections, Redis not needed) and outlined the migration strategy requiring schema validation.

### 2026-05-07 – Developer asked if it's possible to
**Asked:** Developer asked if it's possible to migrate users and all data from sandbox version to production.
**Done:** Assistant reviewed infrastructure and configs, confirmed migration is feasible but requested clarification on sandbox version definition.

### 2026-05-06 – Fixed TypeScript type errors in ShareView
**Asked:** Fixed TypeScript type errors in ShareView decision state comparisons | Added 'submitting' state to type union and fixed conditional rendering logic | .claude/settings.local.json, frontend/src/routes/ShareView.tsx
**Done:** Fixed TypeScript type errors in ShareView decision state comparisons | Added 'submitting' state to type union and fixed conditional rendering logic | .claude/settings.local.json, frontend/src/routes/ShareView.tsx

### 2026-05-06 – Fix TypeScript type error in ShareView.tsx
**Asked:** Fix TypeScript type error in ShareView.tsx where decisionState comparison with 'submitting' has no overlap with 'error' | 'idle' type.
**Done:** Added 'submitting' to conditional logic and corrected operator precedence with parentheses on line 206.

### 2026-05-06 – Fix TypeScript type narrowing errors in
**Asked:** Fix TypeScript type narrowing errors in ShareView.tsx where 'submitting' state is unreachable due to operator precedence.
**Done:** Added 'submitting' state to the conditional logic at line 206 to properly include all decision states in type narrowing.

### 2026-05-06 – Fix security validation error when changing
**Asked:** Fix security validation error when changing user role and update role-based access control.
**Done:** Updated role permissions to grant My QC Queue and Help section access, fixed defaultRoleKey mapping in Help.tsx.

### 2026-05-06 – Asked user to change a user
**Asked:** Asked user to change a user role but received a "Security validation failed" 400 error from a regex false positive.
**Done:** Fixed middleware regex patterns in validation.py by adding word boundaries to prevent matching partial usernames like "Josh Smith" triggering `sh\s+`.

### 2026-05-01 – Install Playwright MSP and create detailed
**Asked:** Install Playwright MSP and create detailed user guides for six roles with screenshots.
**Done:** Set up Playwright MSP, built Help component with role-based tabs, TOC navigation, search, and markdown guides for all roles.

### 2026-05-01 – Set up Playwright for automated browser
**Asked:** Set up Playwright for automated browser testing and screenshot capture to create comprehensive user guides for multiple roles.
**Done:** Configured Playwright MSP with browser automation capabilities and prepared environment templates for screenshot generation across different user roles.

### 2026-05-01 – Install agent-sdk-dev plugin for development
**Asked:** Install agent-sdk-dev plugin for development
**Done:** Successfully installed agent-sdk-dev plugin and verified features/ folder is empty

### 2026-05-01 – Asked for plugin installation and TypeScript
**Asked:** Asked for plugin installation and TypeScript type resolution
**Done:** Installed agent-sdk-dev plugin and identified type inference issue with briefsData.briefs property


### 2026-05-01 — P0/P1/P2 + Bug Fixes → Production-Ready

**P0 Multi-tenancy isolation (MT-3/6/7/8):**
- Added `get_job_or_403` / `assert_job_in_user_org` to all 25+ job-action endpoints in `routes_jobs.py`
- Patched `routes_review_notes.py` and `routes_vtt_versions.py` (load parent job → check org)
- WebSocket: org check → `close(4403)` for foreign-org access; heartbeat lowered 30s→20s; server keepalive frame every 20s
- Admin users endpoint: `OrgScopedQuery.for_user_collection(ctx)` with platform-admin bypass

**P1 English-first workflow:**
- `language_qc.py`: target approval blocked until source language is APPROVED (409)
- `routes_jobs.py` PATCH `/vtt`: source edits reset target-lang QC states + write `vtt_versions` snapshot
- Frontend sticky banner in VttEditor warns about downstream invalidation; "Re-translate" button

**P2 Role UX:**
- `LinguistQueue.tsx`: added deadline sort toggle, `defaultRole` prop
- `ReviewerQueue.tsx`: new component wrapping LinguistQueue with `defaultRole="reviewer"`
- App.tsx: `/qc/reviewer-queue` route + RoleGate; `/briefs/*` gated to PM/admin/production
- Dashboard: PM gets "Overdue" + "Stuck >24h" cards; Production gets "Awaiting Upload" + "Pending QC Handoff"
- Admin UserList: org_id cross-org filter dropdown

**Bug fixes:**
- Bug 1/2A: VTT PATCH now regenerates `descriptive_transcript.txt` (reads unchanged stream from GCS, merges, uploads)
- Bug 2B: Pydantic `field_validator(mode='before')` coerces `""` → `None` for captions/AD fields (CC-only 400 fixed)
- Runbook: WebSocket keepalive troubleshooting section + Apache ProxyTimeout guidance (≥60s)

**Merged:** branch `fix/multi-tenancy-and-english-first` → main (30 files, 2353 insertions)

### 2026-04-29 — TTS Worker + Concurrency + Gemini Models

- Upgraded Gemini models to 3.1-pro-preview + 3.1-pro-tts-preview
- Reverted pro TTS to gemini-2.5-pro-preview-tts (3.1 pro TTS doesn't exist)
- Fixed TTS worker crash loop (8 concurrency × 80MB > 512MB limit) via `TTS_WORKER_CONCURRENCY` env var
- Added audit log integration sweep + cost tracker URL fix + audit log admin UI
- Integrated oliver-cost-tracker SDK

### 2026-04-16

- Added linguist role and user management navigation
- Added reviewer queue UI, brief management, org-level download portal
- Multiple QC workflow improvements, PM email notifications, language expansion

## Key Files

| File | Purpose |
|------|---------|
| `backend/app/api/v1/routes_jobs.py` | Main job endpoints + PATCH /vtt |
| `backend/app/api/v1/routes_language_qc.py` | QC approval/rejection endpoints |
| `backend/app/services/language_qc.py` | Per-language QC state machine |
| `backend/app/core/authz.py` | `MembershipContext`, `get_job_or_403`, `OrgScopedQuery` |
| `backend/app/api/v1/routes_websockets.py` | WS subscribe + org check |
| `backend/app/services/descriptive_transcript.py` | Transcript generation |
| `backend/app/schemas/job.py` | `VttUpdateRequest` + `empty_str_to_none` validator |
| `frontend/src/hooks/useJobStatusWebSocket.ts` | WS hook: terminal close, heartbeat |
| `frontend/src/components/VttEditor/VttEditor.tsx` | Editor + retranslate banner |
| `frontend/src/routes/jobs/LinguistQueue.tsx` | Linguist + reviewer queue |
| `frontend/src/routes/Dashboard.tsx` | Role-specific dashboard widgets |


### 2026-05-01 — Brief form, accessible video captions, role assignment

**Org Settings sidebar fix:**
- `Sidebar.tsx`: platform admins (no org memberships) now see Settings link via fallback to `allOrgs[0].id`
- `useOrganizations` hook now accepts `{ enabled }` option to prevent 403 for non-admin roles

**Brief form fixes:**
- Added `GET /admin/brief-assignees` endpoint (accessible to all auth'd roles) returning admin/PM/production users
- Fixed `GET /admin/list-all-projects` — PMs now see all projects, not just their assigned ones
- Expanded Requested Outputs: added SDH Captions, Descriptive Transcript, Accessible Video (MP4) + method selector
- Briefs now accessible to client role (added to RoleGate in App.tsx)

**Accessible video captions:**
- `QCDetail.tsx`: added `retimedCaptionsVtt` state populated from `vttContent.retimed_captions_vtt`
- Added caption overlay on accessible video player (absolute-positioned, bottom-14)
- Falls back to original `captionsVtt` when retimed version not yet generated (overlay method)

**Admin/PM as linguist/reviewer:**
- `routes_admin.py`: `GET /admin/users` now accepts comma-separated `role` param (e.g. `linguist,admin`)
- `QCDetail.tsx`: assignment dropdowns now fetch `linguist,admin,production` and `reviewer,admin,production`