Implements word-level speech analysis using faster-whisper to refine
AD pause points. Gemini's timestamps are snapped to natural speech gaps
(sentence/phrase boundaries) to prevent pauses mid-word.
Key changes:
- Add WhisperService for transcription and gap detection
- Add dedicated Celery task routed to 'whisper' queue
- Integrate refinement into render_accessible_video task
- Cache Whisper transcripts in MongoDB for reuse across languages
- Add dedicated whisper-worker with concurrency=1 to prevent OOM
Configuration:
- Uses faster-whisper 'base' model (multilingual, ~145MB)
- 5-second search window after Gemini's recommended point
- Falls back to original timestamp if no gap found
Infrastructure:
- New Docker stage: whisper-worker
- New Cloud Run service: accessible-video-whisper-worker
- Updated docker-compose.yml with whisper-worker service
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously only the final pending_qc status was broadcast via WebSocket.
Now all intermediate status changes (ingesting, ai_processing) are also
broadcast so the frontend can update in real-time during reprocessing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Created By filter dropdown was empty because client_id was not
being returned by the API. Added client_id to JobResponse schema
and included it in the list_jobs response.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add created_by_name field to JobResponse schema and API
- Batch-fetch user names in list_jobs endpoint for efficiency
- Convert JobsList from card layout to sortable data table
- Add search box (job name, filename, created by user)
- Add user filter dropdown (populated from current jobs)
- Add status filter dropdown (individual statuses from current jobs)
- Add date range filter (All Time, Last 7 Days, Last 30 Days)
- Add sortable columns: Job Name, Created By, Date Created, Status
- Fetch all jobs for full client-side filtering capability
- Add responsive horizontal scroll for mobile
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When jobs with accessible video option enabled enter video rendering
phase, the status now transitions to 'rendering_video' so users can
see why processing is taking longer. This provides better visibility
into the video rendering pipeline.
Changes:
- Added RENDERING_VIDEO status to JobStatus enum
- Updated render_accessible_video task to set new status
- Added status display to StatusBadge, jobStatusMessages
- Included new status in JobsList Translation filter group
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Modified render_accessible_video.py to explicitly pass TMPDIR to
tempfile.TemporaryDirectory() so files are created in shared volume
- Updated docker-compose.yml to run containers as root initially,
chown /shared-tmp to app:app, then switch to app user for celery
- This ensures both worker containers can access the same temp files
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Celery doesn't allow calling result.get() within a task by default to
prevent deadlocks. Use allow_join_result() context manager since we've
already confirmed the task is complete via ready() polling.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a dedicated Celery queue (ffmpeg) with concurrency=1 to serialize
all FFmpeg operations. This prevents CPU spikes when multiple render
tasks run in parallel with multiple languages.
Changes:
- Add ffmpeg_operations.py with run_ffmpeg_command and run_ffprobe_command tasks
- Update VideoRendererService to dispatch ffmpeg commands via the queue
- Add ffmpeg-worker service to docker-compose with --concurrency=1
- Configure main worker to exclude the ffmpeg queue
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add validation for accessible_video_gcs (file exists, size 0.1MB-5GB)
- Add validation for retimed_captions_vtt_gcs when accessible video exists
- Add AD Videos count to asset validation panel
- Include retimed captions in VTT file count
- Remove AI confidence from validation panel and backend checks
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix video event listeners not re-attaching when video element remounts
(add activeTab?.videoUrl to useEffect dependency array)
- Add retimed_captions_vtt to VTT API response for accessible videos
- Use retimed captions for accessible video tab in VideoReviewPlayer
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a comprehensive video review feature to the Final Review page that allows
reviewers to watch videos with caption overlays and add timestamped notes.
Backend:
- New ReviewNote model for MongoDB with job_id, asset_key, timestamp, content
- CRUD API endpoints at /jobs/{job_id}/review-notes
- Owner-only edit/delete permissions (admins can bypass)
- Database indexes for efficient querying
Frontend:
- VideoReviewPlayer component with video player and caption overlay
- NotesSidebar for viewing/adding notes with auto-highlight when video reaches timestamp
- SyncedCaptionList with auto-scroll and click-to-seek
- AssetTabs for switching between languages and accessible videos
- React Query hooks with 30s polling for collaborative updates
Features:
- Notes persist to database and are shared across all reviewers
- Notes highlight for 5 seconds when video playback reaches their timestamp
- Click note to seek video to that position
- Pause video to add note at current timestamp
- Accessible videos use retimed captions when available
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The method field (overlay/pause_insert) is metadata, not a downloadable
file. Including it in the downloads dict caused the frontend to render
a broken download link.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update _get_video_properties() to extract audio sample_rate, channels,
and pix_fmt in addition to video properties
- Add _extract_segment_reencoded() for frame-accurate cuts using
re-encoding instead of stream copy (fixes keyframe-only cut limitation)
- Add _create_freeze_segment_matched() to enforce source audio property
matching (fixes silent pauses caused by sample rate mismatch)
- Update _render_pause_insert_method() to use new methods with uniform
encoding parameters
- Add -video_track_timescale 90000 for consistent timebase across segments
Root causes fixed:
1. -c copy could only cut at keyframes, causing audio dropouts
2. Sample rate mismatch (48kHz source vs 44.1kHz MP3) caused silent
freeze-frame segments when concatenated
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The accessible video render task was being dispatched to the 'render' queue
but no worker was listening to it. Added 'render' to:
- Dockerfile CMD args for worker queue list
- celery_worker.py import and log message
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add new deliverable type that renders video with audio descriptions embedded.
Supports two AI-determined methods:
- Direct Overlay: ducks original audio and overlays AD TTS (for minimal dialogue)
- Pause-Insert: freeze-frame video, insert AD, re-time subtitles (for significant dialogue)
Backend:
- Add Pydantic schemas for Gemini analysis response
- Add Gemini prompt and analyze_accessible_video_placement() method
- Add video_renderer.py service using FFmpeg for both rendering methods
- Add vtt_retimer.py service for pause-insert subtitle adjustment
- Add render_accessible_video.py Celery task
- Modify TTS service to return individual per-cue segments
- Update translate_and_synthesize.py to save segments and trigger rendering
- Update download endpoint to include accessible video outputs
Frontend:
- Add accessible_video_mp4 checkbox to NewJob form
- Update TypeScript types for new deliverable
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Updated Gemini ingestion prompt to explicitly require:
- Detect the spoken language first
- Write ALL outputs (summary, transcript, captions, audio_description) in that language
- Do NOT translate to English - keep everything in the original language
This fixes the issue where German videos would get English audio descriptions.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add model selection (flash vs pro) for quality control
- Add speed slider (0.5x - 2.0x) for pacing adjustment
- Add style presets (neutral, calm, energetic, professional, warm, documentary)
- Add custom style prompt option for advanced customization
- New /tts/options endpoint returns available TTS options
- Voice preview now tests all settings so users hear exact output
- Backward compatible: all new fields have sensible defaults
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The Gemini TTS service uses pydub which requires ffmpeg to convert
audio formats. Previously only the Worker container had ffmpeg.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The import was using a non-existent module path `..deps` instead of
`...core.dependencies`, causing the API container to fail on startup.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Gemini TTS service with 30 voices and 24 languages
- Add TTS API endpoints for voice listing and preview
- Add per-language voice selection in job creation form
- Add voice override at QC approval stage
- Add VoiceSelector and VoicePreviewButton components
- Update TTSPreferences model with provider and voice mapping
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Change model from gemini-2.5-pro to gemini-3-pro-preview
- Upgrade google-genai package from ^1.31.0 to ^1.56.0
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add migration to update jobs collection validator with new statuses
- Update mongodb-init.js for fresh deployments
- Fix deploy.sh to properly run migrations with 'python migrate.py up'
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Upload form now has "English / Different language" radio with optional language hint
- Gemini auto-detects language and saves outputs to outputs.{detected_language}
- QC review dynamically loads/saves VTT for source language
- New APPROVED_SOURCE status for non-English videos (APPROVED_ENGLISH kept for backwards compat)
- Translation pipeline reads from source language and passes source_language to Google Translate
- All existing English jobs continue to work unchanged
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
