---
name: "Accessible Video Processing Platform"
client: Oliver Internal
status: active
tech: [Python, FastAPI, Celery, MongoDB, Redis, React, TypeScript, Vite, Gemini, Google TTS, ElevenLabs, Docker]
local_path: /Users/ai_leed/Documents/Projects/Oliver/video-accessibility
deploy: docker compose up --build
url: 
server: TBD
tags: [oliver, video, accessibility, ai, captions, audio-description, tts, celery, mongodb]
created: 2026-04-14
last_commit: 2026-04-27
commits: 218
---

## Overview
Comprehensive AI-powered platform for generating accessible video content. Full pipeline from upload to delivery with QC workflow.

**Features implemented (85% production-ready):**
- Closed captions (Gemini 2.5 Pro)
- Audio descriptions (AI-generated + TTS)
- SDH captions (Subtitles for Deaf and Hard of Hearing)
- Descriptive transcripts
- 50+ language translation (Google Translate + cultural transcreation)
- QC workflow (reviewer approval/rejection, VTT editing)
- DCMP compliance

**20,471 lines of code** (12,198 backend + 8,273 frontend)

## Tech Stack
- **Frontend:** React 18 + Vite + TypeScript (SPA)
- **Backend:** FastAPI + Python 3.11 + Celery workers
- **Database:** MongoDB Atlas
- **Storage:** Google Cloud Storage (signed URLs)
- **Queue:** Redis + Celery
- **AI:** Gemini 2.5 Pro
- **Translation:** Google Cloud Translate API
- **TTS:** Google Cloud TTS + ElevenLabs
- **Auth:** JWT + HttpOnly refresh cookies (RBAC)
- **Infrastructure:** Docker + docker-compose

## Architecture
```
React SPA (TypeScript)
    ↓ JWT auth (RBAC)
FastAPI backend
    ├── Celery workers (video processing pipeline)
    ├── MongoDB Atlas (jobs, users, assets)
    ├── Google Cloud Storage (video files, VTT)
    └── Redis (task queue)

Pipeline phases:
1. Upload → Ingestion worker
2. Gemini 2.5 Pro → VTT captions
3. Audio Description generation
4. QC review (approve/reject/edit VTT)
5. Translation → 50+ languages
6. TTS synthesis (GCP TTS + ElevenLabs)
7. Final delivery
```

### Critical Dev Note
**Always read `video_accessibility_development_plan.txt` before any dev work** — it's the authoritative source for all technical specs, API contracts, DB models.

## Deployment
- **Run:** `docker compose up --build`
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/video-accessibility`

## Timeline / Git History
| Date | Change |
|------|--------|
| 2026-04-27 | feat: integrate oliver-cost-tracker SDK into video-accessibility |
| 2026-04-16 | feat: add linguist role and user management navigation |
| 2026-03-27 | Add ElevenLabs API key to production env |
| 2026-03-27 | Fix VTT charset=utf-8 (prevent ♪ encoding issues) |
| 2026-03-27 | Add SDH captions + descriptive transcript to QC Downloads |
| 2026-03-27 | DCMP compliance, new languages, QA bug fixes |
| 2026-03-24 | Fix MP3/VTT desync, crashes, notifications |
| 2026-03-20 | Fix case-insensitive Microsoft user lookup (prevent duplicate key) |
| 2026-03-18 | Add SDH (Subtitles for Deaf and Hard of Hearing) |
| 2026-03-18 | Add brand context + ethics guidelines to AD prompt |
| 2026-03-03 | Add download assets panel + VTT file upload to QC review |

## Sessions
### 2026-04-27 – Check TypeScript type safety for editState
**Asked:** Check TypeScript type safety for editState null handling in QCDetail.tsx.
**Done:** Verified all editState references use optional chaining to safely handle null values.

### 2026-04-27 – Connect video-accessibility service to centralized AI
**Asked:** Connect video-accessibility service to centralized AI Cost Tracker via HTTP with environment variables and API key configuration.
**Done:** Identified and fixed Downloads.tsx hook firing requests for non-completed job statuses causing 400 errors; now conditionally passes status to skip queries for incomplete jobs.

### 2026-04-27 – Connect video-accessibility service to centralized AI
**Asked:** Connect video-accessibility service to centralized AI Cost Tracker via HTTP.
**Done:** Pushed commit with Cost Tracker integration and provided server deployment steps with environment variables.

### 2026-04-27 – Connect video-accessibility service to centralized AI
**Asked:** Connect video-accessibility service to centralized AI Cost Tracker via HTTP integration with budget preflight checks.
**Done:** Created cost_tracker.py service with preflight() and record() functions, integrated Gemini and TTS usage logging, and deployed to main branch.

### 2026-04-27 – Connect video-accessibility service to centralized AI
**Asked:** Connect video-accessibility service to centralized AI Cost Tracker via HTTP integration.
**Done:** Implemented cost tracking service with preflight budget checks and usage recording in backend.

### 2026-04-27 – Connect video-accessibility service to centralized AI
**Asked:** Connect video-accessibility service to centralized AI Cost Tracker via HTTP with environment variables and API key integration.
**Done:** Set up environment variables (COST_TRACKER_BASE_URL, COST_TRACKER_API_KEY, COST_TRACKER_SOURCE_APP) and verified API integration parameters in dispatch and synthesis functions.

### 2026-04-27 – Create an AI cost tracking admin
**Asked:** Create an AI cost tracking admin dashboard with user management and token logging.
**Done:** Set up admin interface for AI usage monitoring with multi-dimensional analytics and team isolation features.

### 2026-04-27 – Create an AI cost tracking system
**Asked:** Create an AI cost tracking system with user admin, token logging, and interactive analytics dashboard.
**Done:** Reviewed Obsidian CLI documentation and identified file conflict resolution approach for implementation planning.

### 2026-04-27 – Create an AI cost tracking admin
**Asked:** Create an AI cost tracking admin dashboard with multi-dimensional analytics, token pricing integration, and user team isolation.
**Done:** Built FastAPI backend with public API endpoints, admin CRUD operations, LiteLLM pricing sync, analytics with dimensional breakdowns, and Microsoft SSO authentication.

### 2026-04-27 – Create an admin dashboard to track
**Asked:** Create an admin dashboard to track AI usage costs with multi-dimensional analytics and user/team management.
**Done:** Designed system architecture for AI cost tracking with admin panel, analytics reports, token logging, and user isolation with team grouping capabilities.

### 2026-04-27 – Create an admin dashboard to track
**Asked:** Create an admin dashboard to track AI usage costs with multi-dimensional analytics and user/team management.
**Done:** Designed system architecture for AI cost tracking with admin panel, analytics reports, token logging, and user isolation with team grouping capabilities.

### 2026-04-27 – Create an admin dashboard to track
**Asked:** Create an admin dashboard to track AI token usage costs across models, users, projects, and teams with interactive analytics and user isolation.
**Done:** Reviewed requirements and identified web scraping risks for token pricing; recommended using structured data sources (LiteLLM, OpenRouter) instead of price page scraping.

### 2026-04-16 – Create a user management system allowing
**Asked:** Create a user management system allowing admins to create login/password users (non-SSO) and grant linguist access to non-Oliver email addresses.
**Done:** Implemented user management with admin creation capability, linguist role support, and fixed deploy script bug where monorepo code wasn't being pulled.

### 2026-04-16 – Create a user management system allowing
**Asked:** Create a user management system allowing admin to create local login users (non-SSO) with linguist role for outsourced linguists.
**Done:** Implemented user management system with local authentication, admin-only user creation, and linguist role support; committed to main and identified missing migration step in deploy script.

### 2026-04-16 – Create a user management system allowing
**Asked:** Create a user management system allowing admins to create login/password-based users with linguist role for external contractors.
**Done:** Implemented user management backend with linguist role, default admin seeding, and MongoDB schema migration.

### 2026-04-16 – Asked | Created user management system
**Asked:** Asked | Created user management system with admin-only user creation and linguist role support for non-Oliver email addresses
**Done:** Done | Implemented user management with login/password authentication, admin controls, and linguist access capability without Oliver domain requirement

### 2026-04-14 – Project catalogued
**Done:** Added to Obsidian second brain with full details.

---
## Change Log
| Date | Requested | Changed | Files |
|------|-----------|---------|-------|
| 2026-04-27 | Type safety audit | Confirmed optional chaining usage | QCDetail.tsx, routes_tts.py |
| 2026-04-27 | Hook fix | Conditional status check in useJobDownloads, pass 'created' status for non-completed jobs | Downloads.tsx |
| 2026-04-27 | Cost Tracker integration | Added COST_TRACKER_BASE_URL, COST_TRACKER_API_KEY, COST_TRACKER_SOURCE_APP env vars | .env.production |
| 2026-04-27 | Cost Tracker integration | Added preflight/record functions, integrated Gemini/TTS usage logging, removed SDK dependency | backend/app/services/cost_tracker.py, .env configuration |
| 2026-04-27 | Cost Tracker integration | Added preflight/record functions, replaced SDK with httpx calls, env vars configured | backend/app/services/cost_tracker.py, .env |
| 2026-04-27 | Cost Tracker integration | Added env vars for Cost Tracker connection, verified dispatch_language_tts and synthesize_cue_task signatures | .env, docker-compose.yml, dispatch.py, synthesize_cue_task.py |
| 2026-04-27 | AI cost tracking | Admin dashboard, user management, token logging, analytics by task/project/user/model, user isolation, team grouping | AdminDashboard.tsx, UserManagement.tsx, TokenLogger.ts, Analytics.tsx, TeamManagement.tsx |
| 2026-04-27 | AI cost tracking system | User admin panel, AI call logging, token cost database integration, analytics dashboard with multi-dimensional reporting, user isolation with team grouping | (planning phase - no files changed yet) |
| 2026-04-27 | AI cost tracker | FastAPI backend, pricing sync, analytics engine, SSO auth, Celery tasks | routes_public.py, routes_admin.py, pricing.py, analytics.py, celery_tasks.py |
| 2026-04-27 | AI cost tracking system | Admin dashboard, analytics engine, token logging, user/team management, multi-dimensional reporting | AdminPanel.tsx, CostTracker.ts, UserManagement.tsx, AnalyticsEngine.ts, DatabaseSchema.sql |
| 2026-04-27 | AI cost tracking admin | Dashboard schema, pricing integration approach, user isolation & team grouping logic | (pending implementation) |
| 2026-04-16 | User management system | Added user creation, admin role, linguist role with email validation, fixed pull_code function for monorepo | User.ts, Auth.ts, scripts/deploy.sh |
| 2026-04-16 | User management system | User creation, local auth, linguist role, migration setup | backend user models, migration files, full-deploy.sh |
| 2026-04-16 | User management system | Added linguist role enum, MongoDB migration, admin seeding, sidebar link | models/user.py, migrations/add_linguist_role.py, core/seed.py, main.py |
| 2026-04-16 | User Management System | User creation, authentication, linguist roles, admin controls | User.tsx, UserManagement.tsx, AuthService.ts, Sidebar.tsx |
| 2026-03-27 | Fix ♪ encoding in VTT | Add charset=utf-8 to content-type | backend |
| 2026-03-18 | Add SDH captions | New SDH caption output type | pipeline |

## Related
- [[pdf-accessibility/PDF Accessibility Checker]]
- [[sandbox-notebookllamalm-nextjs/Sandbox NotebookLM]] (same infrastructure approach)
