video-accessibility/backend/app
michael 05bde8326d feat: add Whisper-based pause point refinement for audio descriptions
Implements word-level speech analysis using faster-whisper to refine
AD pause points. Gemini's timestamps are snapped to natural speech gaps
(sentence/phrase boundaries) to prevent pauses mid-word.

Key changes:
- Add WhisperService for transcription and gap detection
- Add dedicated Celery task routed to 'whisper' queue
- Integrate refinement into render_accessible_video task
- Cache Whisper transcripts in MongoDB for reuse across languages
- Add dedicated whisper-worker with concurrency=1 to prevent OOM

Configuration:
- Uses faster-whisper 'base' model (multilingual, ~145MB)
- 5-second search window after Gemini's recommended point
- Falls back to original timestamp if no gap found

Infrastructure:
- New Docker stage: whisper-worker
- New Cloud Run service: accessible-video-whisper-worker
- Updated docker-compose.yml with whisper-worker service

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 08:27:48 -06:00
..
__pycache__ fixed middleware silent swallowing of exceptions FINALLY 2025-10-08 18:19:59 -05:00
api/v1 fix: include client_id in JobResponse for user filter 2025-12-27 07:28:05 -06:00
core feat: add Whisper-based pause point refinement for audio descriptions 2025-12-27 08:27:48 -06:00
lib initial commit 2025-08-24 16:28:33 -05:00
middleware fixed overly aggressive security algorithm 2025-10-08 17:49:20 -05:00
migrations add approved_source and qc_feedback job statuses to MongoDB schema 2025-12-22 13:12:14 -06:00
models feat: add rendering_video status for accessible video processing 2025-12-27 06:49:46 -06:00
prompts feat: add accessible video (MP4 with embedded audio descriptions) 2025-12-26 11:06:41 -06:00
schemas feat: add Whisper-based pause point refinement for audio descriptions 2025-12-27 08:27:48 -06:00
services feat: add Whisper-based pause point refinement for audio descriptions 2025-12-27 08:27:48 -06:00
tasks feat: add Whisper-based pause point refinement for audio descriptions 2025-12-27 08:27:48 -06:00
telemetry initial commit 2025-08-24 16:28:33 -05:00
main.py feat: add video review with timestamped notes to Final Review 2025-12-26 15:30:00 -06:00