video-accessibility/DEPLOYMENT.md

Accessible Video Platform - Production Deployment Guide

Canonical source moved: See docs/project/runbook.md for the current deployment runbook and docs/project/infrastructure.md for the infrastructure inventory. This file is kept for historical reference.

=<3D> Table of Contents

• Overview
• Server Requirements
• Pre-Deployment Checklist
• Initial Setup
• Deployment Steps
• Post-Deployment Verification
• Maintenance & Operations
• Troubleshooting

---

<<3C> Overview

This guide covers deploying the Accessible Video Processing Platform to a production server at ai-sandbox.oliver.solutions.

Architecture:

• Frontend: React SPA served by Apache at /video-accessibility
• Backend API: Docker container proxied by Apache at /video-accessibility-back
• Worker: Docker container for Celery background processing
• MongoDB: Docker container for database
• Redis: Docker container for queue/cache

Server Details:

• Host: ai-sandbox.oliver.solutions (GCP VM)
• Resources: 32GB RAM, 8 CPU cores
• OS: Linux with Apache 2.4+
• SSL: Wildcard certificate configured

---

=<3D> Server Requirements

Software Requirements

• Docker 20.10+
• Docker Compose 1.29+
• Apache 2.4+ with modules:
  • mod_rewrite
  • mod_proxy
  • mod_proxy_http
  • mod_proxy_wstunnel
  • mod_headers
• Node.js 20+ and npm (for building frontend)
• Git (for code deployment)

Apache Modules Setup

sudo a2enmod rewrite proxy proxy_http proxy_wstunnel headers
sudo systemctl restart apache2

Directory Structure

/opt/accessible-video/
 docker-compose.yml
 docker-compose.prod.yml
 .env.production
 backend/               # Git clone
 frontend/              # Git clone
 secrets/
    gcp-credentials.json
 scripts/
    deploy.sh
    build-frontend.sh
    mongodb-init.js
 volumes/              # Created by Docker
     mongodb-data/
     redis-data/
     logs/

---

 Pre-Deployment Checklist

1. Secrets & Credentials

• Google Cloud credentials JSON file ready
• Generate secure JWT secret (64 characters)

  openssl rand -hex 32
  

• Gemini API key obtained
• (Optional) SendGrid API key for emails
• (Optional) ElevenLabs API key for TTS

2. DNS & SSL

• DNS for ai-sandbox.oliver.solutions configured
• Wildcard SSL certificate installed in Apache
• Certificate includes *.ai-sandbox.oliver.solutions

3. GCP Setup

• GCS bucket accessible-video created
• Service account has Storage Admin role
• Service account JSON key downloaded

4. Firewall & Ports

• Port 443 (HTTPS) open
• Port 80 (HTTP) open (for redirect to HTTPS)
• Port 8000 accessible from localhost only (Docker API)

---

=<3D> Initial Setup

Step 1: Clone Repositories

# Create project directory
sudo mkdir -p /opt/video-accessibility
cd /opt/video-accessibility

# Clone backend
git clone <your-backend-repo-url> backend

# Clone frontend
git clone <your-frontend-repo-url> frontend

# Set ownership
sudo chown -R $USER:$USER /opt/video-accessibility

Step 2: Create Secrets Directory

# Create secrets directory
mkdir -p /opt/video-accessibility/secrets

# Copy GCP credentials
cp /path/to/gcp-credentials.json /opt/video-accessibility/secrets/gcp-credentials.json

# Set restrictive permissions
chmod 600 /opt/video-accessibility/secrets/gcp-credentials.json

Step 3: Configure Environment Variables

# Copy template
cp .env.production.template .env.production

# Edit with secure values
nano .env.production

Required changes in .env.production:

# Generate JWT secret (64 characters)
JWT_SECRET=<64-char-random-string>

# Update with your values
GCP_PROJECT_ID=optical-414516
GCS_BUCKET=accessible-video
GEMINI_API_KEY=<your-gemini-key>

# Optional
SENDGRID_API_KEY=<your-sendgrid-key>
ELEVENLABS_API_KEY=<your-elevenlabs-key>

# Set secure permissions
chmod 600 .env.production

Step 4: Configure Frontend

# Frontend environment is already configured in frontend/.env.production
# Verify the API URL is correct:
cat frontend/.env.production

Should contain:

VITE_API_BASE_URL=https://ai-sandbox.oliver.solutions/video-accessibility-back

Step 5: Configure Apache

# Add configuration to your VirtualHost
sudo nano /etc/apache2/sites-available/ai-sandbox.oliver.solutions-ssl.conf

# Copy configuration from apache-config-snippet.conf
# Add inside the <VirtualHost *:443> block

# Test configuration
sudo apache2ctl configtest

# Restart Apache
sudo systemctl restart apache2

---

=<3D> Deployment Steps

Option A: Automated Deployment (Recommended)

cd /opt/accessible-video

# Make scripts executable
chmod +x scripts/*.sh

# Run full deployment
./scripts/deploy.sh

This script will:

1. Pull latest code
2. Build Docker images
3. Start containers
4. Build and deploy frontend
5. Run database migrations
6. Verify deployment

Option B: Manual Deployment

1. Build and Start Backend Services

cd /opt/video-accessibility

# Load environment variables
export $(cat .env.production | grep -v '^#' | xargs)

# Build images
docker compose -f docker-compose.yml -f docker-compose.prod.yml build

# Start services
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

# Check status
docker compose ps

2. Initialize Database

# Wait for MongoDB to be ready (about 10 seconds)
sleep 10

# Initialize database
docker compose exec mongodb mongosh < scripts/mongodb-init.js

# Run migrations
docker compose exec api python migrate.py

# Create initial admin user
docker compose exec api python create_test_users.py

3. Build and Deploy Frontend

# Run frontend build script
./scripts/build-frontend.sh

Or manually:

cd frontend

# Install dependencies
npm ci --only=production

# Build
npm run build

# Deploy to Apache
sudo mkdir -p /var/www/html/video-accessibility
sudo rm -rf /var/www/html/video-accessibility/*
sudo cp -r dist/* /var/www/html/video-accessibility/
sudo chown -R www-data:www-data /var/www/html/video-accessibility
sudo chmod -R 755 /var/www/html/video-accessibility

cd ..

---

 Post-Deployment Verification

1. Check Docker Containers

cd /opt/accessible-video
docker-compose ps

All containers should show as "healthy" or "Up".

2. Test Backend API

# Health check
curl https://ai-sandbox.oliver.solutions/video-accessibility-back/health

# Should return: {"status":"healthy","version":"1.0.0"}

3. Test Frontend

Visit in browser:

https://ai-sandbox.oliver.solutions/video-accessibility

You should see the login page.

4. Test WebSocket Connection

Check browser console for WebSocket connection (after logging in).

5. Create Test Job

1. Log in as admin user
2. Upload a test video
3. Verify job appears in the list
4. Check worker logs:

   docker-compose logs -f worker
   

---

=' Maintenance & Operations

Viewing Logs

# View all logs
docker compose logs -f

# View specific service
docker compose logs -f api
docker compose logs -f worker

# View last 100 lines
docker compose logs --tail=100 api

Restarting Services

# Restart all
docker compose restart

# Restart specific service
docker compose restart api
docker compose restart worker

Updating Application

# Pull latest code (as your user, not sudo)
cd /opt/video-accessibility
git pull origin main
cd backend && git pull origin main && cd ..
cd frontend && git pull origin main && cd ..

# Run full deployment (requires sudo)
sudo ./scripts/full-deploy.sh

Updating Frontend Only

cd /opt/video-accessibility
./scripts/build-frontend.sh

Database Backup

# Backup MongoDB
docker compose exec mongodb mongodump --out=/data/backup

# Copy backup from container
docker cp accessible-video-mongodb:/data/backup ./mongodb-backup-$(date +%Y%m%d)

Database Restore

# Copy backup to container
docker cp ./mongodb-backup accessible-video-mongodb:/data/restore

# Restore
docker compose exec mongodb mongorestore /data/restore

Monitoring Resource Usage

# Check Docker container resource usage
docker stats

# Check disk usage
docker system df

# Clean up unused images/containers
docker system prune -a

---

= Troubleshooting

Issue: Containers Won't Start

Check logs:

docker compose logs

Common causes:

• MongoDB authentication failure <20> Check MONGODB_ROOT_PASSWORD
• Redis connection failure <20> Check if Redis container is running
• Port 8000 already in use <20> Check for other processes on port 8000

Issue: 502 Bad Gateway on Backend

Diagnosis:

# Check if API container is running
docker-compose ps api

# Check API logs
docker-compose logs api

# Test API directly
curl http://localhost:8000/health

Solutions:

• API container not started <20> docker-compose up -d api
• API crashed <20> Check logs for errors
• Wrong port in Apache config <20> Should proxy to localhost:8000

Issue: Frontend Shows 404

Check:

# Verify files exist
ls -la /var/www/html/video-accessibility/

# Check Apache logs
sudo tail -f /var/log/apache2/ai-sandbox-error.log

Solutions:

• Files not deployed <20> Run ./scripts/build-frontend.sh
• Apache alias incorrect <20> Check apache-config-snippet.conf
• Permissions wrong <20> sudo chown -R www-data:www-data /var/www/html/video-accessibility

Issue: WebSocket Connection Fails

Check:

# Verify mod_proxy_wstunnel is enabled
apache2ctl -M | grep proxy_wstunnel

Solution:

sudo a2enmod proxy_wstunnel
sudo systemctl restart apache2

Issue: Worker Not Processing Jobs

Check:

# View worker logs
docker compose logs -f worker

# Check Redis connection
docker compose exec worker python -c "import redis; r=redis.Redis(host='redis'); print(r.ping())"

# Check Celery status
docker compose exec worker celery -A celery_worker inspect active

Solutions:

• Worker not listening to correct queues <20> Check Dockerfile CMD
• Redis connection issue <20> Check REDIS_URL environment variable
• GCP credentials missing <20> Check /secrets/gcp-credentials.json is mounted

Issue: Upload Fails / GCS Error

Check:

# Test GCS credentials
docker compose exec api python -c "
from google.cloud import storage
client = storage.Client()
print(list(client.list_buckets()))
"

Solutions:

• Credentials file not found <20> Check secrets/gcp-credentials.json exists
• Wrong project ID <20> Check GCP_PROJECT_ID in .env.production
• Permissions issue <20> Service account needs Storage Admin role

Issue: Database Connection Errors

Check:

# Test MongoDB connection
docker-compose exec api python -c "
from pymongo import MongoClient
import os
client = MongoClient(os.environ['MONGODB_URI'])
print(client.server_info())
"

Solutions:

• Wrong credentials <20> Check MONGODB_ROOT_PASSWORD
• MongoDB not started <20> docker-compose up -d mongodb
• Connection string incorrect <20> Should be mongodb://admin:password@mongodb:27017/...

---

=<3D> Support & Additional Resources

• Project Repository: <your-repo-url>
• Issues: Create an issue in the repository
• Docker Documentation: https://docs.docker.com/
• Apache Documentation: https://httpd.apache.org/docs/2.4/

---

=<3D> Quick Reference Commands

# Full deployment
cd /opt/video-accessibility && ./scripts/deploy.sh

# View logs
docker compose logs -f [service]

# Restart service
docker compose restart [service]

# Update frontend only
./scripts/build-frontend.sh

# Check container status
docker compose ps

# View resource usage
docker stats

# Backup database
docker compose exec mongodb mongodump --out=/data/backup

# Access MongoDB shell
docker compose exec mongodb mongosh

# Run migrations
docker compose exec api python migrate.py

---

Last Updated: 2025-01-10 Version: 1.0.0