ai_qc/MIGRATION_GUIDE.md

# Visual AI QC Migration Guide
## Moving from Web Root to /opt/ Structure

This guide provides step-by-step instructions for migrating the Visual AI QC application from `/var/www/html/ai_qc/` to a split frontend/backend structure.

---

## Current vs. New Structure

### Current Structure
```
/var/www/html/ai_qc/
├── api_server.py              # Backend
├── run_api_server.py          # Backend launcher
├── web_ui.html                # Frontend
├── visual_qc_apps/            # Backend
├── profiles/                  # Backend
├── brand_guidelines/          # Backend
├── uploads/                   # Backend data
├── output/                    # Backend data
├── venv/                      # Backend dependencies
└── ... (all other files)
```

### New Structure
```
Frontend: /var/www/html/ai_qc/
└── index.html                 # Renamed from web_ui.html

Backend: /opt/ai_qc/
├── api_server.py
├── run_api_server.py
├── llm_config.py
├── profile_config.py
├── jwt_validator.py
├── auth_middleware.py
├── visual_qc_apps/
├── profiles/
├── brand_guidelines/
├── config/
├── uploads/
├── output/
├── venv/
└── ... (all Python/backend files)
```

---

## Prerequisites

Before starting, ensure you have:
1. SSH access to the server with sudo privileges
2. Backup of current deployment
3. Maintenance window scheduled (estimated 15-30 minutes downtime)
4. Apache restart permissions

---

## Step-by-Step Migration

### Step 0: Preparation (On your server via SSH)

```bash
# Check current service name
sudo systemctl list-units --type=service | grep ai_qc

# Check current Apache configuration
ls -la /etc/apache2/sites-enabled/ | grep ai_qc

# Backup current deployment
cd /var/www/html/ai_qc
sudo tar -czf ~/ai_qc_backup_$(date +%Y%m%d_%H%M%S).tar.gz .
echo "Backup created at: ~/ai_qc_backup_$(date +%Y%m%d_%H%M%S).tar.gz"
```

### Step 1: Stop Current Service

```bash
# Find the service name (likely ai_qc or ai_qc.service)
sudo systemctl list-units --type=service | grep ai_qc

# Stop the service (replace SERVICE_NAME with actual name)
sudo systemctl stop SERVICE_NAME

# Verify it's stopped
ps aux | grep run_api_server.py
# Should return no results (except the grep command itself)
```

### Step 2: Create Backend Directory Structure

```bash
# Create backend directory
sudo mkdir -p /opt/ai_qc

# Copy all files except web_ui.html to backend
cd /var/www/html/ai_qc
sudo rsync -av \
  --exclude='web_ui.html' \
  --exclude='*.pyc' \
  --exclude='__pycache__' \
  --exclude='.git' \
  . /opt/ai_qc/

# Set ownership
sudo chown -R www-data:www-data /opt/ai_qc

# Set permissions
sudo chmod -R 750 /opt/ai_qc
sudo chmod 770 /opt/ai_qc/uploads /opt/ai_qc/output
sudo chmod 770 /opt/ai_qc/uploads-dev /opt/ai_qc/output-dev
sudo chmod 640 /opt/ai_qc/config/*.env
```

### Step 3: Prepare Frontend Directory

```bash
# Create clean frontend directory
sudo mkdir -p /var/www/html/ai_qc_new

# Copy web_ui.html and rename to index.html
sudo cp /var/www/html/ai_qc/web_ui.html /var/www/html/ai_qc_new/index.html

# Set ownership
sudo chown -R www-data:www-data /var/www/html/ai_qc_new
```

### Step 4: Update Backend Configuration

```bash
# Update production environment config with absolute paths
sudo nano /opt/ai_qc/config/production.env

# Ensure these paths are absolute:
# UPLOAD_FOLDER=/opt/ai_qc/uploads
# OUTPUT_FOLDER=/opt/ai_qc/output
# UPLOAD_FOLDER_DEV=/opt/ai_qc/uploads-dev
# OUTPUT_FOLDER_DEV=/opt/ai_qc/output-dev
# BRAND_GUIDELINES_FOLDER=/opt/ai_qc/brand_guidelines
```

### Step 5: Update Frontend API URLs

```bash
# Edit the frontend to use absolute API paths
sudo nano /var/www/html/ai_qc_new/index.html

# Find all fetch() calls and ensure they use /api/ prefix
# Search for: fetch('
# Replace relative URLs with /api/ prefix if needed
# Example: fetch('/start_analysis') → fetch('/api/start_analysis')
```

**Important**: Check if your current `web_ui.html` already uses `/api/` prefix. If it does, no changes needed!

### Step 6: Update Systemd Service

```bash
# Find current service file
sudo find /etc/systemd/system -name "*ai_qc*"

# Create new service file (or update existing)
sudo nano /etc/systemd/system/ai_qc.service
```

Paste this content:
```ini
[Unit]
Description=Visual AI QC Flask Application
After=network.target

[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/opt/ai_qc
Environment="ENVIRONMENT=production"
Environment="PATH=/opt/ai_qc/venv/bin:/usr/local/bin:/usr/bin:/bin"
ExecStart=/opt/ai_qc/venv/bin/python /opt/ai_qc/run_api_server.py --host localhost --port 7183 --workers 2

Restart=always
RestartSec=10

NoNewPrivileges=true
PrivateTmp=true

StandardOutput=journal
StandardError=journal
SyslogIdentifier=ai_qc

[Install]
WantedBy=multi-user.target
```

Save and exit (Ctrl+X, Y, Enter)

```bash
# Reload systemd
sudo systemctl daemon-reload
```

### Step 7: Update Apache Configuration

```bash
# Find current Apache config
ls -la /etc/apache2/sites-available/ | grep ai_qc

# Backup current config
sudo cp /etc/apache2/sites-available/ai_qc.conf /etc/apache2/sites-available/ai_qc.conf.backup

# Edit Apache config
sudo nano /etc/apache2/sites-available/ai_qc.conf
```

Update the configuration to:
```apache
<VirtualHost *:80>
    ServerName your-domain.com

    # Frontend - serve from web root
    DocumentRoot /var/www/html/ai_qc_new

    <Directory /var/www/html/ai_qc_new>
        Options -Indexes +FollowSymLinks
        AllowOverride None
        Require all granted
        DirectoryIndex index.html
    </Directory>

    # Backend API - proxy to Flask
    ProxyPreserveHost On
    ProxyPass /api http://localhost:7183/api
    ProxyPassReverse /api http://localhost:7183/api
    ProxyTimeout 300

    # Serve uploaded files from backend
    Alias /uploads /opt/ai_qc/uploads
    <Directory /opt/ai_qc/uploads>
        Options -Indexes
        Require all granted
    </Directory>

    # Serve output files from backend
    Alias /output /opt/ai_qc/output
    <Directory /opt/ai_qc/output>
        Options -Indexes
        Require all granted
    </Directory>

    # Development folders (optional)
    Alias /uploads-dev /opt/ai_qc/uploads-dev
    <Directory /opt/ai_qc/uploads-dev>
        Options -Indexes
        Require all granted
    </Directory>

    Alias /output-dev /opt/ai_qc/output-dev
    <Directory /opt/ai_qc/output-dev>
        Options -Indexes
        Require all granted
    </Directory>

    ErrorLog ${APACHE_LOG_DIR}/ai_qc_error.log
    CustomLog ${APACHE_LOG_DIR}/ai_qc_access.log combined
</VirtualHost>
```

Save and exit, then:
```bash
# Test Apache configuration
sudo apache2ctl configtest

# Should return "Syntax OK"
```

### Step 8: Install Dependencies (if needed)

```bash
# Activate virtual environment
cd /opt/ai_qc
source venv/bin/activate

# Install waitress if not already installed (for run_api_server.py)
pip install waitress

# Verify all dependencies
pip install -r requirements.txt

deactivate
```

### Step 9: Test Backend Standalone

```bash
# Test if backend starts correctly
cd /opt/ai_qc
sudo -u www-data /opt/ai_qc/venv/bin/python run_api_server.py --host localhost --port 7183 --workers 2

# In another terminal, test the API
curl http://localhost:7183/api/profiles

# Should return profile data
# Press Ctrl+C to stop the test server
```

### Step 10: Start Services

```bash
# Start the backend service
sudo systemctl start ai_qc

# Check status
sudo systemctl status ai_qc

# Check logs
sudo journalctl -u ai_qc -f

# Verify backend is responding
curl http://localhost:7183/api/profiles
```

### Step 11: Reload Apache

```bash
# Reload Apache to apply new configuration
sudo systemctl reload apache2

# Check Apache status
sudo systemctl status apache2

# Check Apache error log
sudo tail -f /var/log/apache2/ai_qc_error.log
```

### Step 12: Test the Application

```bash
# Test frontend is accessible
curl http://your-domain.com/

# Test API through Apache proxy
curl http://your-domain.com/api/profiles

# Test file serving
curl -I http://your-domain.com/uploads/
curl -I http://your-domain.com/output/
```

**Then test in browser:**
1. Open http://your-domain.com/ in browser
2. Sign in with Microsoft authentication
3. Upload a test file
4. Run analysis
5. Verify results display correctly
6. Check saved files list updates

### Step 13: Finalize Migration

Once everything is working:

```bash
# Rename directories
sudo mv /var/www/html/ai_qc /var/www/html/ai_qc_old
sudo mv /var/www/html/ai_qc_new /var/www/html/ai_qc

# Update Apache config DocumentRoot if needed
sudo nano /etc/apache2/sites-available/ai_qc.conf
# Change: DocumentRoot /var/www/html/ai_qc_new
# To:     DocumentRoot /var/www/html/ai_qc

# Reload Apache
sudo systemctl reload apache2

# Test again to ensure everything still works
```

### Step 14: Enable Service on Boot

```bash
# Ensure service starts on boot
sudo systemctl enable ai_qc

# Verify it's enabled
sudo systemctl is-enabled ai_qc
# Should return "enabled"
```

---

## Verification Checklist

After migration, verify:

- [ ] Backend service is running: `sudo systemctl status ai_qc`
- [ ] Backend responds: `curl http://localhost:7183/api/profiles`
- [ ] Apache is running: `sudo systemctl status apache2`
- [ ] Frontend loads: Visit http://your-domain.com/
- [ ] API calls work: Check browser console for errors
- [ ] Authentication works: Sign in with Microsoft
- [ ] File upload works: Upload test file
- [ ] Analysis works: Run QC analysis on test file
- [ ] Results display: Verify results show correctly
- [ ] Saved files work: Check saved files list updates
- [ ] File downloads work: Download generated PDF reports
- [ ] Logs are clean: `sudo journalctl -u ai_qc -n 50 --no-pager`

---

## Rollback Plan

If issues occur:

### Quick Rollback

```bash
# Stop new service
sudo systemctl stop ai_qc

# Restore old Apache config
sudo cp /etc/apache2/sites-available/ai_qc.conf.backup /etc/apache2/sites-available/ai_qc.conf
sudo systemctl reload apache2

# Rename directories back
sudo mv /var/www/html/ai_qc /var/www/html/ai_qc_failed
sudo mv /var/www/html/ai_qc_old /var/www/html/ai_qc

# Restore old service file (if you backed it up)
sudo cp /etc/systemd/system/ai_qc.service.backup /etc/systemd/system/ai_qc.service
sudo systemctl daemon-reload

# Start old service
sudo systemctl start ai_qc

# Verify old version works
curl http://your-domain.com/
```

### Full Restore from Backup

```bash
# Stop all services
sudo systemctl stop ai_qc
sudo systemctl stop apache2

# Remove new directories
sudo rm -rf /opt/ai_qc
sudo rm -rf /var/www/html/ai_qc_new

# Restore from backup
cd /var/www/html/ai_qc
sudo tar -xzf ~/ai_qc_backup_YYYYMMDD_HHMMSS.tar.gz

# Restore old config and restart
sudo systemctl start ai_qc
sudo systemctl start apache2
```

---

## Monitoring After Migration

### Check Logs Regularly

```bash
# Backend logs
sudo journalctl -u ai_qc -f

# Apache logs
sudo tail -f /var/log/apache2/ai_qc_error.log
sudo tail -f /var/log/apache2/ai_qc_access.log

# Check disk space
df -h /opt/ai_qc
```

### Performance Monitoring

```bash
# Check backend process
ps aux | grep run_api_server

# Check memory usage
sudo systemctl status ai_qc

# Check port is listening
sudo ss -tlnp | grep 7183
```

---

## Benefits of New Structure

✅ **Security**: Backend code not accessible from web root
✅ **Separation**: Frontend and backend can be updated independently
✅ **Scalability**: Backend can be moved to separate server if needed
✅ **Standard Practice**: Matches other apps on your server (/opt/veo3, /opt/voice2text)
✅ **Better Permissions**: More granular access control
✅ **Cleaner Structure**: Clear separation of concerns

---

## Need Help?

If you encounter issues:
1. Check logs: `sudo journalctl -u ai_qc -n 100 --no-pager`
2. Check Apache logs: `sudo tail -100 /var/log/apache2/ai_qc_error.log`
3. Test backend directly: `curl http://localhost:7183/api/profiles`
4. Verify permissions: `ls -la /opt/ai_qc`
5. Check service status: `sudo systemctl status ai_qc`

Common issues:
- **Permission denied**: Check file ownership (`sudo chown -R www-data:www-data /opt/ai_qc`)
- **Module not found**: Check virtual environment (`which python` should show venv path)
- **Port already in use**: Check if old service is still running (`ps aux | grep 7183`)
- **API not responding**: Check if service is running (`sudo systemctl status ai_qc`)