Oliver/video-query
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

new features with video job cycles and video lenght changes

Browse source
This commit is contained in:
Manish Tanwar 2025-10-22 14:24:13 +05:30
parent 63245ce66d
commit 005769489e
23 changed files with 4014 additions and 1474 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

48
.claude/settings.local.json Normal file
View file

@ -0,0 +1,48 @@
{
"permissions": {
"allow": [
"Bash(npm start)",
"Bash(npm install)",
"Bash(timeout 10 npm start)",
"Bash(ls:*)",
"Bash(rm:*)",
"Bash(./node_modules/.bin/react-scripts:*)",
"Bash(timeout 15 npm start)",
"Bash(npm run build:*)",
"Bash(python3 run.py:*)",
"Bash(source:*)",
"Bash(python:*)",
"Bash(pkill:*)",
"Bash(true)",
"Bash(sudo apt:*)",
"Bash(sudo apt install:*)",
"Bash(apt list:*)",
"Bash(pip install:*)",
"Bash(whereis:*)",
"Bash(find:*)",
"Bash(snap list:*)",
"Bash(pip show:*)",
"Read(//usr/bin/**)",
"Bash(sudo apt-get update:*)",
"WebSearch",
"WebFetch(domain:ai.google.dev)",
"Bash(git checkout:*)",
"Bash(git add:*)",
"Bash(python3:*)",
"Bash(systemctl status:*)",
"Bash(kill:*)",
"Bash(curl:*)",
"Read(//etc/**)",
"Read(//var/log/**)",
"Bash(netstat:*)",
"Bash(ss:*)",
"Bash(cat:*)",
"Bash(ffmpeg:*)",
"Read(//usr/**)",
"Bash(awk:*)",
"Bash(deactivate)",
"Bash(xargs kill:*)"
],
"deny": []
}
}

3
.gitignore vendored
View file

@ -382,3 +382,6 @@ logs/
# Temporary files
.tmp/
temp/
# Local development configuration (DO NOT COMMIT)
frontend/public/config.local.js

466
CLAUDE.md
View file

@ -2,19 +2,457 @@
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Build/Lint/Test Commands
- Run script: `python video_query.py <video_path> [--prompt "Your custom prompt"]`
- Install dependencies: `pip install -r requirements.txt` (if requirements.txt exists)
- Create venv: `python -m venv venv && source venv/bin/activate`
- Install required packages: `pip install google-generativeai`
## Project Overview
A full-stack video query application using Google Gemini 2.0 Flash Exp AI for video analysis. Features parallel processing, automatic video splitting, Azure AD B2C authentication (optional), chunked file uploads (up to 5GB), and PDF generation with Mermaid diagrams.
**Tech Stack:**
- Backend: Flask 3.1.0, Hypercorn 0.17.3, google-genai 1.45.0, pdfkit, ffmpeg
- Frontend: React 18.2.0, @azure/msal-react 3.0.12, Bootstrap 5.3.2, react-dropzone 14.2.3
## Development Setup Commands
### Backend Setup
```bash
cd backend
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
pip install -r requirements.txt
# Install system dependencies (Ubuntu/Debian)
sudo apt-get install wkhtmltopdf python3-cairo libcairo2-dev ffmpeg
# macOS
brew install cairo wkhtmltopdf ffmpeg
# Create .env file
echo "GOOGLE_API_KEY=your_api_key_here" > .env
# Run development server
python3 run.py
# Server runs on http://0.0.0.0:5010
```
### Frontend Setup
```bash
cd frontend
npm install
# Configure for local development (optional auth disable)
echo "REACT_APP_DISABLE_AUTH=true" > .env
# Start development server
npm start
# Server runs on http://localhost:3000
# Build for production
npm run build
```
### Quick Restart (Development)
```bash
# From project root
./restart.sh
```
## Build/Test Commands
### Backend
- **Run production server**: `cd backend && source venv/bin/activate && python3 run.py`
- **Test API**: `python test_api.py` (in backend directory)
- **Test webhook**: `python test_webhook.py`
- **Manual test**: `python test_webhook_manual.py`
### Frontend
- **Development server**: `npm start` (port 3000)
- **Production build**: `npm run build`
- **Build with deploy script**: `./build.sh`
### Video Processing
- **Standalone script**: `python video_query.py <video_path> [--prompt "Your custom prompt"]`
- **Note**: The standalone script is deprecated. Use the web application for full features.
## Production Deployment
### Backend Deployment (Ubuntu/CentOS)
1. **Install system packages**:
```bash
sudo apt-get update
sudo apt-get install -y wkhtmltopdf python3-cairo python3-pil libcairo2-dev ffmpeg python3-venv
```
2. **Set up virtual environment**:
```bash
cd /path/to/video-query/backend
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```
3. **Configure environment**:
```bash
echo "GOOGLE_API_KEY=your_production_api_key" > .env
```
4. **Set up systemd service**:
```bash
sudo cp video-query.service /etc/systemd/system/
# Edit service file to match your paths
sudo nano /etc/systemd/system/video-query.service
sudo systemctl daemon-reload
sudo systemctl enable video-query
sudo systemctl start video-query
sudo systemctl status video-query
```
5. **Verify service**:
```bash
curl http://localhost:5010/api/health # If health endpoint exists
journalctl -u video-query -f # View logs
```
### Frontend Deployment
1. **Update production configuration** (`frontend/public/config.js`):
```javascript
window.__APP_CONFIG__ = {
"basePath": "/video-query",
"domain": "https://your-domain.com",
"api": {
"videoProcessingEndpoint": "https://your-domain.com/video_query_back/api/process",
"chunkedUploadEndpoint": "https://your-domain.com/video_query_back"
}
};
```
2. **Build for production**:
```bash
cd frontend
npm run build
```
3. **Deploy to web server**:
```bash
sudo cp -r build/* /var/www/html/video-query/
sudo chown -R www-data:www-data /var/www/html/video-query/
```
4. **Configure Apache** (example):
```apache
<VirtualHost *:443>
ServerName your-domain.com
DocumentRoot /var/www/html
# Frontend
Alias /video-query /var/www/html/video-query
<Directory /var/www/html/video-query>
Options -Indexes +FollowSymLinks
AllowOverride All
Require all granted
</Directory>
# Backend proxy
ProxyPass /video_query_back http://localhost:5010
ProxyPassReverse /video_query_back http://localhost:5010
# WebSocket support (if needed)
ProxyPass /video_query_back/ws ws://localhost:5010/ws
ProxyPassReverse /video_query_back/ws ws://localhost:5010/ws
SSLEngine on
SSLCertificateFile /etc/ssl/certs/your-cert.crt
SSLCertificateKeyFile /etc/ssl/private/your-key.key
</VirtualHost>
```
5. **Configure Nginx** (alternative):
```nginx
server {
listen 443 ssl;
server_name your-domain.com;
ssl_certificate /etc/ssl/certs/your-cert.crt;
ssl_certificate_key /etc/ssl/private/your-key.key;
# Frontend
location /video-query {
alias /var/www/html/video-query;
try_files $uri $uri/ /video-query/index.html;
}
# Backend proxy
location /video_query_back {
proxy_pass http://localhost:5010;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Timeouts for long video processing
proxy_read_timeout 3600s;
proxy_connect_timeout 3600s;
proxy_send_timeout 3600s;
}
}
```
6. **Restart web server**:
```bash
# Apache
sudo systemctl restart apache2
# Nginx
sudo systemctl restart nginx
```
### Azure AD B2C Authentication Setup (Optional)
1. **Configure Azure AD B2C tenant** and register application
2. **Update** `frontend/src/auth/authConfig.js` with your tenant details
3. **Set authentication in frontend/.env**:
```bash
# Enable auth for production
REACT_APP_DISABLE_AUTH=false
# Disable auth for local dev
REACT_APP_DISABLE_AUTH=true
```
## Key Architecture Components
### Parallel Processing (App.js:244-268)
- **Max concurrent videos**: 2 (MAX_PARALLEL constant)
- Uses `Promise.allSettled()` for batch processing
- Each video gets an `AbortController` for cancellation
### Rate Limiting (video_processor.py:221-224)
- **Delay**: 2 seconds between API calls
- Uses `threading.Lock` for thread safety
- Prevents Gemini API rate limit errors (5 RPM free tier)
### Hybrid Upload Strategy (video_processor.py:163-203)
- **Files < 10MB**: Base64 inline data (instant)
- **Files >= 10MB**: File Upload API (slower but reliable)
- Threshold: `SIZE_THRESHOLD_MB = 10`
### Video Splitting (video_splitter.py)
- **Chunk duration**: 25 minutes
- **Automatic**: Videos > 25 minutes split automatically
- **Processing**: Chunks processed in parallel on backend
- **Maximum single chunk**: 55 minutes (Gemini limit)
### Queue Management (App.js:63-336)
- **Queue states**: queued, processing, completed, failed, cancelled
- **Operations**: Stop (cancel), Retry, Remove
- **Abort signal**: Support for canceling in-flight requests
## Configuration Files
### Backend Configuration
- **backend/.env**: `GOOGLE_API_KEY=your_key`
- **backend/run.py**: Hypercorn server config (body size limits, timeouts)
### Frontend Configuration
- **frontend/.env**: `REACT_APP_DISABLE_AUTH=true/false`
- **frontend/public/config.js**: Production config (committed)
- **frontend/public/config.local.js**: Local dev override (not committed, in .gitignore)
### Configuration Priority
1. `config.local.js` (local development) - highest priority
2. `config.js` (production) - fallback
## Code Style Guidelines
- Imports: Standard library imports first, followed by third-party imports, then local imports
- Formatting: PEP 8 compliant with 4-space indentation
- Types: Use type hints for function parameters and return values
- Naming: snake_case for variables/functions, PascalCase for classes
- Error handling: Use try/except blocks with specific exception types
- API Keys: Store in environment variables, not hardcoded
- Documentation: Use docstrings for functions and main modules
- Max line length: 100 characters
- Include helpful comments for complex operations
### Python (Backend)
- **Imports**: Standard library → third-party → local imports
- **Formatting**: PEP 8 compliant with 4-space indentation
- **Types**: Use type hints for function parameters and return values
- **Naming**: snake_case for variables/functions, PascalCase for classes
- **Error handling**: Use try/except blocks with specific exception types
- **API Keys**: Store in environment variables, never hardcode
- **Documentation**: Use docstrings for functions and main modules
- **Max line length**: 100 characters
- **Comments**: Include helpful comments for complex operations
### JavaScript/React (Frontend)
- **Formatting**: 2-space indentation for React components
- **Naming**: camelCase for variables/functions, PascalCase for components
- **State management**: Use React hooks (useState, useEffect, useMsal)
- **Error handling**: Try/catch with user-friendly error messages
- **API calls**: Use authApiClient.js with abort signal support
- **Comments**: JSDoc style for complex functions
## Important Implementation Notes
### Authentication
- **Controlled via .env**: Do NOT remove MSAL code
- **Toggle**: Set `REACT_APP_DISABLE_AUTH=true` to disable
- **Components**: AuthProvider.js, authApiClient.js, authConfig.js
- **Session storage**: Tokens stored in sessionStorage for implicit flow
### File Upload
- **Chunked upload**: All files use chunked upload (chunkedUploader.js)
- **Max file size**: 5GB per file
- **Progress tracking**: Real-time progress via callbacks
- **Supported formats**: MP4, AVI, MOV, WMV, MKV, WEBM
### PDF Generation
- **Dependencies**: wkhtmltopdf, cairosvg, pdfkit
- **Mermaid support**: Diagrams converted to PNG then embedded
- **Endpoint**: POST /api/generate-pdf
- **Client**: ResultDisplay.js handles download
### Error Handling
- **Rate limiting**: 400 INVALID_ARGUMENT → check logs, reduce parallel count
- **Large files**: Use hybrid upload strategy (SIZE_THRESHOLD_MB)
- **Abort errors**: Check for `err.code === 'ERR_CANCELED'` or `abortSignal.aborted`
## Troubleshooting
### Backend Issues
```bash
# Check service status
sudo systemctl status video-query
# View logs
journalctl -u video-query -f
# Restart service
sudo systemctl restart video-query
# Check ffmpeg installation
which ffprobe
ffprobe -version
```
### Frontend Issues
```bash
# Clear browser cache
# Chrome/Firefox: Ctrl+Shift+R (force reload)
# Check config loading
# Browser console: window.__APP_CONFIG__
# Verify build
ls -la frontend/build/
# Check Apache/Nginx logs
tail -f /var/log/apache2/error.log
tail -f /var/log/nginx/error.log
```
### Rate Limiting Issues
- **Symptom**: 400 INVALID_ARGUMENT after 3-4 videos
- **Check**: Gemini API rate limits (5 RPM for free tier)
- **Solutions**:
- Increase delay in video_processor.py (line 222)
- Reduce MAX_PARALLEL in App.js (line 245)
- Lower SIZE_THRESHOLD_MB in video_processor.py (line 163)
### CORS Issues
- **Local dev**: Verify config.local.js exists and points to localhost:5010
- **Production**: Check Apache/Nginx proxy configuration
- **Backend**: Verify CORS settings in app.py
## Testing
### Backend Testing
```bash
cd backend
source venv/bin/activate
# Test video processing
python test_api.py
# Test webhook
python test_webhook.py
# Manual API test
curl -X POST http://localhost:5010/api/process \
-H "Content-Type: application/json" \
-d '{"file_path": "/path/to/video.mp4", "filename": "video.mp4", "prompt": "Test prompt"}'
```
### Frontend Testing
```bash
cd frontend
# Development mode
npm start
# Production build test
npm run build
npx serve -s build -l 3000
```
## Log Extraction & Analytics
### Extract User Logs
```bash
# Quick extraction
./quick_extract.sh
# Robust extraction with error handling
./extract_user_logs_robust.sh
# See LOG_EXTRACTION_README.md for details
```
## Useful File Locations
### Backend
- Main app: `backend/app.py`
- Video processor: `backend/video_processor.py` (Gemini API integration)
- Video splitter: `backend/video_splitter.py` (25-min chunks)
- Chunked upload: `backend/chunked_upload.py`
- Authentication: `backend/auth.py`
### Frontend
- Main app: `frontend/src/App.js` (queue management, parallel processing)
- Queue UI: `frontend/src/components/AuthenticatedContent.js`
- Upload: `frontend/src/components/VideoUpload.js`
- Results: `frontend/src/components/ResultDisplay.js`
- Chunked uploader: `frontend/src/utils/chunkedUploader.js`
### Configuration
- Backend env: `backend/.env`
- Frontend env: `frontend/.env`
- Production config: `frontend/public/config.js`
- Local config: `frontend/public/config.local.js`
- Systemd service: `backend/video-query.service`
## Dependencies Management
### Backend Updates
```bash
cd backend
source venv/bin/activate
pip install --upgrade google-genai flask flask-cors pdfkit
pip freeze > requirements.txt
```
### Frontend Updates
```bash
cd frontend
npm update
npm audit fix
```
## Security Considerations
- **API Keys**: Never commit .env files
- **Authentication**: Azure AD B2C tokens in sessionStorage (consider security implications)
- **CORS**: Specific origin allowlisting in production
- **File validation**: Size and type checks in VideoUpload.js
- **Temporary files**: Automatic cleanup in backend
- **Rate limiting**: Built-in to prevent abuse
## Support & Documentation
- **Main docs**: See README.md for comprehensive feature documentation
- **Deployment**: See DEPLOYMENT.md for detailed deployment guide
- **Log extraction**: See LOG_EXTRACTION_README.md for analytics
- **Parallel processing**: See PARALLEL_PROCESSING.md (if exists)
- **CORS fixes**: See CORS_FIX_SUMMARY.md (if exists)

170
CORS_FIX_SUMMARY.md Normal file
View file

@ -0,0 +1,170 @@
# CORS Fix Summary
## Issue
Frontend running on `http://localhost:3000` was blocked by CORS policy when trying to access backend API at `https://brandtechsandbox.oliver.solutions/video_query_back/api/init-upload`
### Error Message
```
Access to XMLHttpRequest at 'https://brandtechsandbox.oliver.solutions/video_query_back/api/init-upload'
from origin 'http://localhost:3000' has been blocked by CORS policy:
Response to preflight request doesn't pass access control check:
No 'Access-Control-Allow-Origin' header is present on the requested resource.
```
---
## Root Cause
The OPTIONS preflight handler in `app.py` (lines 1124-1132) was only returning `https://ai-sandbox.oliver.solutions` as the allowed origin, not `http://localhost:3000`.
---
## Solution Implemented
### File: `backend/app.py`
#### Changed (lines 1123-1143):
```python
# Handle CORS preflight requests for all API routes
@app.route('/api/<path:path>', methods=['OPTIONS'])
def handle_options(path):
# Get the origin from the request
origin = request.headers.get('Origin')
allowed_origins = ['https://ai-sandbox.oliver.solutions', 'http://localhost:3000']
response = jsonify({})
# Allow the origin if it's in our allowed list
if origin in allowed_origins:
response.headers.add('Access-Control-Allow-Origin', origin)
else:
# Default to production origin
response.headers.add('Access-Control-Allow-Origin', 'https://ai-sandbox.oliver.solutions')
response.headers.add('Access-Control-Allow-Headers', 'Content-Type,Authorization,X-Requested-With')
response.headers.add('Access-Control-Allow-Methods', 'GET,POST,OPTIONS')
response.headers.add('Access-Control-Max-Age', '86400') # 24 hours
response.headers.add('Access-Control-Allow-Credentials', 'true')
return response
```
---
## What Changed
### Before:
- Hardcoded origin: `'https://ai-sandbox.oliver.solutions'`
- Did not check request origin
- Always returned same origin regardless of where request came from
### After:
- Dynamic origin checking
- List of allowed origins: `['https://ai-sandbox.oliver.solutions', 'http://localhost:3000']`
- Returns the origin that made the request if it's in the allowed list
- Falls back to production origin if request origin is not allowed
---
## Existing CORS Configuration (Already Correct)
The main CORS configuration on line 41-46 was already correct:
```python
CORS(app, resources={r"/api/*": {
"origins": ["https://ai-sandbox.oliver.solutions", "http://localhost:3000"],
"supports_credentials": True,
"methods": ["GET", "POST", "OPTIONS"],
"allow_headers": ["Content-Type", "X-Requested-With", "Authorization"]
}}, expose_headers=["Content-Disposition", "Authorization"])
```
---
## Testing
### Before Fix:
```
❌ localhost:3000 → backend API: CORS error
✅ production → backend API: Works
```
### After Fix:
```
✅ localhost:3000 → backend API: Should work
✅ production → backend API: Still works
```
---
## How to Test
1. **Start the backend** (if not already running):
```bash
cd backend
python3 run.py
# or
hypercorn run:app
```
2. **Start the frontend** on localhost:3000:
```bash
cd frontend
npm start
```
3. **Test the upload**:
- Open browser to `http://localhost:3000`
- Try uploading a video
- Check browser console for CORS errors
- Should see successful API calls
4. **Check Network Tab**:
- Open browser DevTools → Network tab
- Look for `init-upload` request
- Check Response Headers for:
- `Access-Control-Allow-Origin: http://localhost:3000`
- `Access-Control-Allow-Credentials: true`
---
## Additional Notes
### Why This Fix is Safe:
1. **Localhost is development only** - Won't be accessible in production
2. **Credentials still required** - Auth is still enforced
3. **Limited to /api/* routes** - Doesn't affect other routes
4. **Production origin still allowed** - No impact on deployed version
### If You Need to Add More Origins:
Update the `allowed_origins` list in `app.py` line 1128:
```python
allowed_origins = [
'https://ai-sandbox.oliver.solutions',
'http://localhost:3000',
'http://localhost:3001', # Add more as needed
'https://another-domain.com'
]
```
---
## Files Modified
1. ✅ `backend/app.py` - Updated OPTIONS handler (lines 1123-1143)
---
## Dependencies
- ✅ `flask-cors==5.0.1` - Already in requirements.txt
- ✅ No new dependencies needed
---
## Status
✅ **FIXED and READY FOR TESTING**
The CORS error should now be resolved. Try uploading a video from `http://localhost:3000` and verify it works.
---
Generated: 2025-10-16

272
PARALLEL_PROCESSING.md Normal file
View file

@ -0,0 +1,272 @@
# Parallel Video Chunk Processing
## Overview
The video-query application now supports **parallel processing** of video chunks, significantly reducing processing time for long videos.
## Performance Improvements
### Before (Sequential Processing)
- **2-hour video** (3 chunks of 50 min each)
- Chunk 1: ~3-5 minutes
- Chunk 2: ~3-5 minutes
- Chunk 3: ~3-5 minutes
- **Total: 9-15 minutes**
### After (Parallel Processing)
- **2-hour video** (3 chunks processed simultaneously)
- All 3 chunks: ~3-5 minutes
- **Total: 3-5 minutes**
**Speed Improvement: 3x faster!**
---
## Configuration
### Default Settings
```python
# Default configuration in VideoProcessor
max_parallel_chunks = 4 # Conservative setting for free tier
```
### Custom Configuration
```python
from video_processor import VideoProcessor
# For Free Tier (5 RPM limit)
processor = VideoProcessor(api_key="your_key", max_parallel_chunks=3)
# For Paid Tier (150 RPM limit)
processor = VideoProcessor(api_key="your_key", max_parallel_chunks=10)
```
---
## API Rate Limits
### Gemini API Limits by Tier
| Tier | RPM Limit | Recommended Workers | Max Video Chunks |
|------|-----------|---------------------|------------------|
| **Free** | 5 RPM | 3-4 workers | 3-4 simultaneous |
| **Paid Tier 1** | 150 RPM | 10+ workers | 10+ simultaneous |
| **Paid Tier 2+** | 1000+ RPM | 20+ workers | 20+ simultaneous |
### Important Notes
- Rate limits are **per project**, not per API key
- Concurrent requests are allowed as long as you stay within RPM limits
- The system automatically respects your configured `max_parallel_chunks` setting
---
## Usage
### Enable Parallel Processing (Default)
```python
from video_processor import VideoProcessor
processor = VideoProcessor()
# Parallel processing is enabled by default
result = processor.process_long_video(
video_path="/path/to/long_video.mp4",
prompt="Generate a meeting summary",
user_email="user@example.com"
)
print(f"Processing mode: {result['processing_mode']}") # Output: "parallel"
```
### Disable Parallel Processing (Sequential)
```python
# If you prefer sequential processing
result = processor.process_long_video(
video_path="/path/to/long_video.mp4",
prompt="Generate a meeting summary",
user_email="user@example.com",
use_parallel=False # Disable parallel processing
)
print(f"Processing mode: {result['processing_mode']}") # Output: "sequential"
```
### Auto-Detection Mode
```python
# Use process_video_auto for automatic detection
result = processor.process_video_auto(
video_path="/path/to/video.mp4",
prompt="Generate documentation",
user_email="user@example.com"
)
# Automatically uses parallel processing for long videos
```
---
## How It Works
1. **Video Splitting**
- Long videos are split into 25-minute chunks (configurable)
- Uses FFmpeg for fast, lossless splitting
2. **Parallel Upload & Processing**
- Chunks are uploaded to Gemini API concurrently
- Multiple API calls run simultaneously (up to `max_parallel_chunks`)
- Thread-safe execution using `ThreadPoolExecutor`
3. **Response Combination**
- Responses are collected in correct order
- Intelligently combined based on prompt type (meeting, documentation, etc.)
- For meetings, can optionally synthesize into unified summary
4. **Cleanup**
- Temporary chunk files are automatically deleted
- Handles errors gracefully
---
## Technical Implementation
### Thread Safety
The implementation uses:
- `concurrent.futures.ThreadPoolExecutor` for parallel execution
- `threading.Lock` for rate limiting
- Order-preserving result collection
### Error Handling
- Each chunk is processed independently
- If one chunk fails, the error is logged with specific details
- Failed chunks return error information without crashing entire job
- Results maintain correct order regardless of completion order
---
## Environment Variables
```bash
# Optional: Set max workers via environment variable
export VIDEO_QUERY_MAX_WORKERS=5
# API Key (required)
export GOOGLE_API_KEY="your_gemini_api_key"
```
---
## Logging
Monitor parallel processing with detailed logs:
```
[INFO] Starting parallel processing of 3 chunks with 4 workers
[INFO] [Parallel] Processing chunk 1/3: /tmp/video_chunk_01.mp4
[INFO] [Parallel] Processing chunk 2/3: /tmp/video_chunk_02.mp4
[INFO] [Parallel] Processing chunk 3/3: /tmp/video_chunk_03.mp4
[INFO] [Parallel] Completed chunk 2/3
[INFO] [Parallel] Progress: 1/3 chunks completed
[INFO] [Parallel] Completed chunk 1/3
[INFO] [Parallel] Progress: 2/3 chunks completed
[INFO] [Parallel] Completed chunk 3/3
[INFO] [Parallel] Progress: 3/3 chunks completed
[INFO] [Parallel] All 3 chunks processed
[INFO] Combining responses from all chunks...
```
---
## Troubleshooting
### Rate Limit Errors
**Symptom:** `429 Too Many Requests` errors
**Solution:**
```python
# Reduce max_parallel_chunks
processor = VideoProcessor(max_parallel_chunks=2)
```
### Memory Issues
**Symptom:** Out of memory errors
**Solution:**
```python
# Process fewer chunks in parallel
processor = VideoProcessor(max_parallel_chunks=2)
```
### Slower Performance
**Symptom:** Parallel processing is slower than expected
**Possible Causes:**
- Network bottleneck (upload bandwidth)
- CPU bottleneck (video encoding)
- API rate limiting
**Solution:**
- Check network speed
- Monitor CPU usage
- Verify API tier and limits
---
## Best Practices
1. **Choose Appropriate Worker Count**
- Free tier: 3-4 workers
- Paid tier: 8-10 workers
- Don't exceed your RPM limit
2. **Monitor Resource Usage**
- Check server memory
- Monitor network bandwidth
- Track API usage
3. **Handle Errors Gracefully**
- Implement retry logic
- Log all errors
- Provide fallback to sequential processing
4. **Optimize Chunk Size**
- 25 minutes for most cases
- 50 minutes if API supports it
- Balance between parallelism and chunk size
---
## Future Enhancements
Potential improvements:
- [ ] Adaptive worker count based on API tier detection
- [ ] Exponential backoff for rate limit errors
- [ ] Progress callbacks for real-time updates
- [ ] Configurable chunk duration
- [ ] Support for asyncio (async/await pattern)
---
## Support
For issues or questions:
1. Check logs for detailed error information
2. Verify API key and rate limits
3. Review configuration settings
4. Consult Gemini API documentation
---
## License
Same as the main project license.

246
README.md
View file

@ -1,32 +1,43 @@
# Video Query Tool
A full-stack web application that processes videos using Google's Gemini AI model, allowing users to upload videos and receive AI-generated content based on customizable prompts. The application features Azure AD B2C authentication, chunked file uploads for large videos, PDF generation with Mermaid diagram support, and comprehensive usage tracking.
A full-stack web application that processes videos using Google's Gemini AI model, allowing users to upload multiple videos simultaneously and receive AI-generated content based on customizable prompts. Features parallel processing, automatic video splitting, Azure AD B2C authentication, chunked file uploads, PDF generation with Mermaid diagrams, and comprehensive usage tracking.
## Features
### Core Functionality
- **Video Processing**: Upload and analyze videos using Google Gemini 2.5 Pro AI model
- **Video Processing**: Upload and analyze videos using Google Gemini 2.0 Flash Exp AI model
- **Multiple Processing Modes**:
- Meeting Summary
- Process/Tool Documentation
- Process Documentation with Mermaid Charts
- Custom Prompts
- **Large File Support**: Chunked upload system supporting files up to 5GB
- **Large File Support**: Chunked upload system supporting files up to 5GB per file
- **PDF Generation**: Convert results to PDF with embedded Mermaid diagrams
- **Authentication**: Azure AD B2C integration with both popup and redirect flows
- **Authentication**: Azure AD B2C integration (optional, controlled via .env)
- **Parallel Processing**: Process up to 2 videos simultaneously
- **Multiple File Upload**: Upload and queue multiple videos at once
- **Long Video Support**: Automatic splitting and parallel chunk processing for videos > 25 minutes
### Technical Features
- **Multiple File Queue**: Upload multiple videos, manage queue (Stop, Retry, Remove)
- **Drag & Drop Upload**: Modern file upload interface with progress tracking
- **Real-time Processing**: Live status updates during video analysis
- **Error Handling**: Comprehensive error handling and user feedback
- **Real-time Processing**: Live status updates with parallel processing indicators
- **Queue Management**: Stop, retry, or remove videos from processing queue anytime
- **Automatic Video Splitting**: Videos > 25 minutes automatically split into 25-min chunks
- **Rate Limiting**: Built-in API rate limiting (2-second delay) to prevent quota errors
- **Error Handling**: Comprehensive error handling with retry capability
- **Processing Time Display**: Shows processing duration for each completed video
- **Usage Analytics**: Automated tracking via webhook integration
- **Production Ready**: Systemd service configuration and deployment scripts
## Limitations
- **Video Length**: Gemini AI processes videos up to 55 minutes maximum
- **File Size**: Application supports uploads up to 5GB
- **Video Length**: No limit - videos automatically split into 25-minute chunks
- **Single Chunk Limit**: Individual chunks must be under 55 minutes (handled automatically)
- **File Size**: Application supports uploads up to 5GB per file
- **Supported Formats**: MP4, AVI, MOV, WMV, MKV, WEBM
- **Parallel Processing**: Max 2 videos simultaneously (rate limit protection)
- **API Rate Limits**: Gemini free tier: 5 RPM (built-in 2s delay between calls)
## Project Structure
@ -34,41 +45,50 @@ A full-stack web application that processes videos using Google's Gemini AI mode
video_query/
├── backend/ # Flask/Hypercorn API server
│ ├── app.py # Main Flask application with PDF generation
│ ├── video_processor.py # Gemini API integration and video processing
│ ├── video_processor.py # Gemini API integration, parallel processing, rate limiting
│ ├── video_splitter.py # Video splitting for long videos (25-min chunks)
│ ├── auth.py # Azure AD B2C authentication handlers
│ ├── chunked_upload.py # Chunked file upload Blueprint
│ ├── run.py # Hypercorn production server
│ ├── requirements.txt # Python dependencies
│ ├── .env # Environment variables (GOOGLE_API_KEY)
│ └── test_*.py # API testing utilities
├── frontend/ # React SPA
│ ├── src/
│ │ ├── components/ # React components
│ │ │ ├── VideoUpload.js # Drag & drop file upload
│ │ │ ├── VideoUpload.js # Multi-file drag & drop upload
│ │ │ ├── PromptSelector.js # Mode selection and prompt editing
│ │ │ ├── ResultDisplay.js # Results with PDF generation
│ │ │ ├── AuthenticatedContent.js # Main application interface
│ │ │ ├── AuthenticatedContent.js # Queue management, processed list
│ │ │ └── Login.js # Authentication interface
│ │ ├── auth/ # Authentication utilities
│ │ │ ├── authConfig.js # Azure AD B2C configuration
│ │ │ ├── AuthProvider.js # MSAL React provider
│ │ │ └── authApiClient.js # Authenticated API client
│ │ └── utils/
│ │ └── chunkedUploader.js # Large file upload handler
│ │ ├── chunkedUploader.js # Large file upload handler
│ │ ├── configLoader.js # Dynamic config loading
│ │ └── pathUtils.js # Path utilities
│ ├── public/
│ │ ├── config.js # Production config (committed)
│ │ ├── config.local.js # Local dev config (not committed)
│ │ └── index.html # Loads both configs
│ ├── package.json # Node.js dependencies
│ ├── .env # Frontend environment variables
│ └── build/ # Production build output
├── DEPLOYMENT.md # Production deployment instructions
├── LOG_EXTRACTION_README.md # Usage analytics documentation
├── CLAUDE.md # Development guidelines and build commands
├── restart.sh # Development restart script
├── quick_extract.sh # Log extraction utility
├── extract_user_logs*.sh # Advanced log processing
└── requirements.txt # Root Python dependencies (legacy)
└── extract_user_logs*.sh # Advanced log processing
```
## Dependencies
### Backend Dependencies
- **Flask 3.1.0**: Web framework
- **google-generativeai 0.8.5**: Gemini AI API client
- **google-genai 1.45.0**: Gemini AI SDK (updated API)
- **Hypercorn 0.17.3**: ASGI production server
- **python-jose**: JWT token validation for Azure AD
- **flask-cors 5.0.1**: Cross-origin resource sharing
@ -76,14 +96,15 @@ video_query/
- **cairosvg 2.8.0**: SVG to PNG conversion for diagrams
- **Pillow 11.2.1**: Image processing
- **python-dotenv 1.1.0**: Environment variable management
- **ffmpeg-python**: Video splitting functionality
### Frontend Dependencies
- **React 18.2.0**: UI framework
- **@azure/msal-react 3.0.12**: Microsoft Authentication Library
- **axios 1.6.0**: HTTP client
- **axios 1.6.0**: HTTP client with abort signal support
- **bootstrap 5.3.2**: UI components and styling
- **mermaid 11.6.0**: Diagram generation
- **react-dropzone 14.2.3**: File upload interface
- **react-dropzone 14.2.3**: Multi-file upload interface
- **showdown 2.1.0**: Markdown to HTML conversion
## Setup Instructions
@ -92,40 +113,42 @@ video_query/
- Python 3.8+
- Node.js 16+
- Google Cloud API key with Gemini access
- Azure AD B2C tenant (for authentication)
- Azure AD B2C tenant (optional, for authentication)
- wkhtmltopdf (for PDF generation)
- ffmpeg/ffprobe (for video splitting)
### Backend Setup
1. **Create and activate virtual environment**:
```bash
python -m venv venv
cd backend
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
```
2. **Install dependencies**:
```bash
pip install -r backend/requirements.txt
pip install -r requirements.txt
```
3. **Set up environment variables**:
3. **Set up environment variables** (create `backend/.env`):
```bash
export GOOGLE_API_KEY="your_gemini_api_key_here"
GOOGLE_API_KEY=your_gemini_api_key_here
```
4. **Install system dependencies for PDF generation**:
4. **Install system dependencies**:
```bash
# Ubuntu/Debian:
sudo apt-get install wkhtmltopdf python3-cairo libcairo2-dev
sudo apt-get install wkhtmltopdf python3-cairo libcairo2-dev ffmpeg
# macOS:
brew install cairo wkhtmltopdf
brew install cairo wkhtmltopdf ffmpeg
```
5. **Start development server**:
```bash
cd backend
python run.py --host 0.0.0.0 --port 5010
python3 run.py
# Server runs on http://0.0.0.0:5010
```
### Frontend Setup
@ -136,14 +159,21 @@ video_query/
npm install
```
2. **Configure authentication** (edit `src/auth/authConfig.js`):
- Update Azure AD B2C tenant ID
- Update client ID
- Update redirect URIs
2. **Configure authentication** (optional):
- Edit `frontend/.env`:
```
REACT_APP_DISABLE_AUTH=true # Disable auth for local dev
```
- For production, update `src/auth/authConfig.js` with Azure AD B2C details
3. **Start development server**:
3. **Configure backend URL for local development**:
- File `frontend/public/config.local.js` already configured for localhost:5010
- This file is not committed (in .gitignore)
4. **Start development server**:
```bash
npm start
# Server runs on http://localhost:3000
```
## Production Deployment
@ -153,6 +183,7 @@ video_query/
- Apache/Nginx web server
- Python 3.8+ with virtual environment
- wkhtmltopdf system package
- ffmpeg/ffprobe for video processing
- Node.js for building frontend
### Backend Deployment
@ -160,38 +191,75 @@ video_query/
1. **Install system packages**:
```bash
sudo apt-get update
sudo apt-get install -y wkhtmltopdf python3-cairo python3-pil libcairo2-dev
sudo apt-get install -y wkhtmltopdf python3-cairo python3-pil libcairo2-dev ffmpeg
```
2. **Create production service** (see `DEPLOYMENT.md` for systemd configuration):
2. **Set up virtual environment and install dependencies**:
```bash
cd backend
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```
3. **Create production .env file**:
```bash
echo "GOOGLE_API_KEY=your_production_api_key" > .env
```
4. **Create systemd service** (see `backend/video-query.service`):
```bash
sudo cp backend/video-query.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable video-query
sudo systemctl start video-query
```
### Frontend Deployment
1. **Build for production**:
```bash
cd frontend
PUBLIC_URL=/video_query npm run build
1. **Update production config** (`frontend/public/config.js`):
```javascript
window.__APP_CONFIG__ = {
"basePath": "/video-query",
"domain": "https://your-domain.com",
"api": {
"videoProcessingEndpoint": "https://your-domain.com/video_query_back/api/process",
"chunkedUploadEndpoint": "https://your-domain.com/video_query_back"
}
};
```
2. **Deploy to web server**:
2. **Build for production**:
```bash
cp -r build/* /var/www/html/video-query/
cd frontend
npm run build
```
3. **Deploy to web server**:
```bash
sudo cp -r build/* /var/www/html/video-query/
```
4. **Configure web server** (Apache example):
```apache
<VirtualHost *:443>
DocumentRoot /var/www/html
# Frontend
Alias /video-query /var/www/html/video-query
# Backend proxy
ProxyPass /video_query_back http://localhost:5010
ProxyPassReverse /video_query_back http://localhost:5010
</VirtualHost>
```
## API Reference
### Authentication Endpoints
- **GET /api/auth-test**: Verify authentication status
### Video Processing Endpoints
- **POST /api/process**: Main video processing endpoint
- Accepts both direct uploads and chunked upload references
- Form data: `video` file, `prompt` text
- JSON data: `file_path`, `filename`, `prompt` (for chunked uploads)
- Accepts JSON: `file_path`, `filename`, `prompt` (for chunked uploads)
- Returns: Processing result with content, processing time, chunks info
### Chunked Upload Endpoints
- **POST /api/init-upload**: Initialize chunked upload session
@ -201,44 +269,90 @@ video_query/
### PDF Generation Endpoints
- **POST /api/generate-pdf**: Generate PDF from HTML with Mermaid diagrams
- JSON data: `html`, `textDiagrams`, `svgDiagrams`, `diagramPngs`
- JSON data: `html`, `textDiagrams`, `diagramPngs`, `videoFileName`
## Usage Analytics
The application includes built-in usage tracking that sends data to a webhook endpoint for analytics purposes. This tracks:
- User email addresses
- Processing timestamps
- Prompts used
- Model information
Log extraction utilities are provided in `extract_user_logs*.sh` scripts.
### Authentication Endpoints (if enabled)
- **GET /api/auth-test**: Verify authentication status
## Configuration Files
### Key Configuration Files
- **CLAUDE.md**: Development guidelines and build commands
- **.gitignore**: Comprehensive exclusion patterns
- **backend/requirements.txt**: Production Python dependencies
- **frontend/package.json**: Node.js dependencies and build scripts
### Backend Configuration
- **backend/.env**: Environment variables
```
GOOGLE_API_KEY=your_api_key
```
### Environment Variables
- `GOOGLE_API_KEY`: Required for Gemini API access
- Various Azure AD B2C configuration in frontend auth config
### Frontend Configuration
- **frontend/.env**: React environment variables
```
REACT_APP_DISABLE_AUTH=true # Optional: disable auth for local dev
```
- **frontend/public/config.js**: Production configuration (committed to git)
- **frontend/public/config.local.js**: Local development override (not committed)
### Key Configuration Details
- **Parallel Processing**: Max 2 concurrent videos (App.js:245)
- **Rate Limiting**: 2-second delay between API calls (video_processor.py:224)
- **File Size Threshold**: 10MB for inline vs upload API (video_processor.py:167)
- **Video Chunk Duration**: 25 minutes (video_splitter.py)
## Usage
### Local Development
1. Start backend: `cd backend && source venv/bin/activate && python3 run.py`
2. Start frontend: `cd frontend && npm start`
3. Open: http://localhost:3000
### Processing Videos
1. **Upload**: Drag & drop multiple videos or click to select
2. **Queue**: Videos appear in "Processing Queue" section
3. **Select Prompt**: Choose processing mode or write custom prompt
4. **Process**: Click "Process N Videos" button
5. **Monitor**: Watch real-time progress (2 videos process in parallel)
6. **Manage**: Use Stop (⏸️), Retry (🔄), or Remove (🗑️) buttons
7. **View Results**: Check "Processed Videos" section for completed results
8. **Download**: Click "Download PDF" or "Copy Formatted" for any completed video
### Processing Long Videos
- Videos > 25 minutes automatically split into chunks
- Each chunk processed in parallel (backend handles this)
- Results intelligently combined
- Processing time displayed for transparency
## Development Utilities
- **restart.sh**: Quick development environment restart
- **backend/test_*.py**: API testing and validation scripts
- **backend/run.py**: Production server with optimized settings for large uploads
- **extract_user_logs*.sh**: Usage analytics extraction
## Security Features
- Azure AD B2C integration with JWT validation
- Azure AD B2C integration with JWT validation (optional)
- CORS protection with specific origin allowlisting
- Secure file upload validation
- Temporary file cleanup
- Token expiration handling
- Rate limiting to prevent API abuse
- Abort signal support for cancellation
## Troubleshooting
### Backend Issues
- **400 INVALID_ARGUMENT**: Usually rate limiting - check logs for details
- **File upload errors**: Verify ffmpeg installed (`which ffprobe`)
- **PDF generation fails**: Ensure wkhtmltopdf installed
### Frontend Issues
- **CORS errors**: Check backend CORS settings in app.py
- **Changes not visible**: Clear browser cache (Ctrl+Shift+R)
- **Config not loading**: Verify config.js and config.local.js exist in public/
### Rate Limiting
- Backend: 2-second delay between API calls (automatic)
- Frontend: Max 2 parallel videos
- Free tier: 5 RPM limit enforced by Gemini API
## License
This project is proprietary and confidential.
This project is proprietary and confidential.

44
backend/app.py
View file

@ -39,7 +39,7 @@ from video_processor import VideoProcessor
app = Flask(__name__)
# Enable CORS with permissive settings for large file uploads
CORS(app, resources={r"/api/*": {
"origins": ["https://ai-sandbox.oliver.solutions"],
"origins": ["https://ai-sandbox.oliver.solutions", "http://localhost:3000"],
"supports_credentials": True,
"methods": ["GET", "POST", "OPTIONS"],
"allow_headers": ["Content-Type", "X-Requested-With", "Authorization"]
@ -113,7 +113,8 @@ def process_video():
user_email = request.user.get("email", request.user.get("preferred_username", "anonymous"))
logger.info(f"Processing chunked upload from {file_path} ({filename}) for user: {user_email}")
result = video_processor.process_video(file_path, prompt, user_email)
# Use auto-processing which handles both short and long videos
result = video_processor.process_video_auto(file_path, prompt, user_email)
# Clean up the uploaded file
try:
@ -121,14 +122,19 @@ def process_video():
logger.info(f"Cleaned up temporary file: {file_path}")
except Exception as cleanup_error:
logger.warning(f"Could not remove temporary file {file_path}: {str(cleanup_error)}")
if result['success']:
content_length = len(result['content']) if result['content'] else 0
logger.info(f"Returning successful response with {content_length} characters")
return jsonify({
response_data = {
'success': True,
'content': result['content']
})
}
# Include chunk information if video was processed in chunks
if result.get('chunks_processed', 0) > 1:
response_data['chunks_processed'] = result['chunks_processed']
response_data['total_chunks'] = result.get('total_chunks', result['chunks_processed'])
return jsonify(response_data)
else:
logger.error(f"Processing failed: {result['message']}")
return jsonify({
@ -205,6 +211,10 @@ def process_video():
logger.info(f"Starting video processing for user: {user_email}...")
result = video_processor.process_video(file_path, prompt, user_email)
logger.info(f"Processing result: success={result['success']}")
# Log if it was processed in chunks
if result.get('chunks_processed', 0) > 1:
logger.info(f"Video was processed in {result['chunks_processed']} chunks")
# Clean up the file after processing
try:
@ -212,14 +222,19 @@ def process_video():
logger.info(f"Cleaned up temporary file: {file_path}")
except Exception as cleanup_error:
logger.warning(f"Could not remove temporary file {file_path}: {str(cleanup_error)}")
if result['success']:
content_length = len(result['content']) if result['content'] else 0
logger.info(f"Returning successful response with {content_length} characters")
return jsonify({
response_data = {
'success': True,
'content': result['content']
})
}
# Include chunk information if video was processed in chunks
if result.get('chunks_processed', 0) > 1:
response_data['chunks_processed'] = result['chunks_processed']
response_data['total_chunks'] = result.get('total_chunks', result['chunks_processed'])
return jsonify(response_data)
else:
logger.error(f"Processing failed: {result['message']}")
return jsonify({
@ -1108,8 +1123,19 @@ def generate_pdf():
# Handle CORS preflight requests for all API routes
@app.route('/api/<path:path>', methods=['OPTIONS'])
def handle_options(path):
# Get the origin from the request
origin = request.headers.get('Origin')
allowed_origins = ['https://ai-sandbox.oliver.solutions', 'http://localhost:3000']
response = jsonify({})
response.headers.add('Access-Control-Allow-Origin', 'https://ai-sandbox.oliver.solutions')
# Allow the origin if it's in our allowed list
if origin in allowed_origins:
response.headers.add('Access-Control-Allow-Origin', origin)
else:
# Default to production origin
response.headers.add('Access-Control-Allow-Origin', 'https://ai-sandbox.oliver.solutions')
response.headers.add('Access-Control-Allow-Headers', 'Content-Type,Authorization,X-Requested-With')
response.headers.add('Access-Control-Allow-Methods', 'GET,POST,OPTIONS')
response.headers.add('Access-Control-Max-Age', '86400') # 24 hours

51
backend/chunked_upload.py
View file

@ -4,18 +4,44 @@ import json
from flask import Blueprint, request, jsonify, current_app
from werkzeug.utils import secure_filename
import logging
from auth import require_auth
from auth import lenient_auth
logger = logging.getLogger('video_query')
# Create blueprint for handling chunked uploads
chunked_upload_bp = Blueprint('chunked_upload', __name__)
# CORS preflight handler for all blueprint routes
def handle_cors_preflight():
"""Handle CORS preflight requests"""
origin = request.headers.get('Origin')
allowed_origins = ['https://ai-sandbox.oliver.solutions', 'http://localhost:3000']
response = jsonify({})
# Allow the origin if it's in our allowed list
if origin in allowed_origins:
response.headers.add('Access-Control-Allow-Origin', origin)
else:
# Default to production origin
response.headers.add('Access-Control-Allow-Origin', 'https://ai-sandbox.oliver.solutions')
response.headers.add('Access-Control-Allow-Headers', 'Content-Type,Authorization,X-Requested-With')
response.headers.add('Access-Control-Allow-Methods', 'GET,POST,OPTIONS')
response.headers.add('Access-Control-Max-Age', '86400') # 24 hours
response.headers.add('Access-Control-Allow-Credentials', 'true')
return response
# Track upload sessions
active_uploads = {}
@chunked_upload_bp.route('/api/init-upload', methods=['OPTIONS'])
def init_upload_options():
"""Handle CORS preflight for init-upload"""
return handle_cors_preflight()
@chunked_upload_bp.route('/api/init-upload', methods=['POST'])
@require_auth
@lenient_auth
def init_upload():
"""Initialize a new chunked upload session"""
if not request.is_json:
@ -59,8 +85,13 @@ def init_upload():
"upload_id": upload_id
})
@chunked_upload_bp.route('/api/upload-chunk/<upload_id>', methods=['OPTIONS'])
def upload_chunk_options(upload_id):
"""Handle CORS preflight for upload-chunk"""
return handle_cors_preflight()
@chunked_upload_bp.route('/api/upload-chunk/<upload_id>', methods=['POST'])
@require_auth
@lenient_auth
def upload_chunk(upload_id):
"""Handle a chunk of file data"""
if upload_id not in active_uploads:
@ -102,8 +133,13 @@ def upload_chunk(upload_id):
"complete": upload['complete']
})
@chunked_upload_bp.route('/api/complete-upload/<upload_id>', methods=['OPTIONS'])
def complete_upload_options(upload_id):
"""Handle CORS preflight for complete-upload"""
return handle_cors_preflight()
@chunked_upload_bp.route('/api/complete-upload/<upload_id>', methods=['POST'])
@require_auth
@lenient_auth
def complete_upload(upload_id):
"""Mark an upload as complete and return the file path for processing"""
if upload_id not in active_uploads:
@ -140,8 +176,13 @@ def complete_upload(upload_id):
"size": upload['uploaded_size']
})
@chunked_upload_bp.route('/api/cancel-upload/<upload_id>', methods=['OPTIONS'])
def cancel_upload_options(upload_id):
"""Handle CORS preflight for cancel-upload"""
return handle_cors_preflight()
@chunked_upload_bp.route('/api/cancel-upload/<upload_id>', methods=['POST'])
@require_auth
@lenient_auth
def cancel_upload(upload_id):
"""Cancel an upload and delete the partial file"""
if upload_id not in active_uploads:

664
backend/video_processor.py
View file

@ -6,8 +6,12 @@ import logging
import requests
import json
import datetime
from typing import Dict, Any, Optional
import base64
from typing import Dict, Any, Optional, List, Tuple
from dotenv import load_dotenv
from video_splitter import VideoSplitter
from concurrent.futures import ThreadPoolExecutor, as_completed
import threading
# Load environment variables from .env file
load_dotenv()
@ -28,15 +32,28 @@ class VideoProcessor:
# Maximum video duration in minutes (Gemini limitation)
MAX_VIDEO_DURATION = 55
# Threshold for chunked upload (10MB)
CHUNKED_UPLOAD_THRESHOLD = 10 * 1024 * 1024
# Webhook URL for tracking usage
WEBHOOK_URL = "https://hook.us1.make.celonis.com/8ri1h8b2he4wudp2jku69mgcxumzxf3v"
def __init__(self, api_key: Optional[str] = None):
"""Initialize with API key from environment variable or direct setting"""
# Parallel processing configuration
# Default max workers for parallel chunk processing
# Free tier: 5 RPM (use 3-4 workers to be safe)
# Paid tier: 150 RPM (can use more workers)
DEFAULT_MAX_WORKERS = 4 # Conservative default for free tier
def __init__(self, api_key: Optional[str] = None, max_parallel_chunks: int = None):
"""
Initialize with API key from environment variable or direct setting
Args:
api_key: Google API key for Gemini
max_parallel_chunks: Maximum number of chunks to process in parallel
(default: 4, recommended 3-4 for free tier, up to 10+ for paid tier)
"""
self.api_key = api_key or os.getenv("GOOGLE_API_KEY")
if not self.api_key:
logger.error("API key not provided")
@ -46,6 +63,16 @@ class VideoProcessor:
logger.info("Initializing Gemini API client")
self.client = genai.Client(api_key=self.api_key)
logger.info("Gemini API client initialized successfully")
# Set parallel processing configuration
self.max_parallel_chunks = max_parallel_chunks or self.DEFAULT_MAX_WORKERS
logger.info(f"Parallel processing enabled with max {self.max_parallel_chunks} concurrent chunks")
# Initialize video splitter
self.video_splitter = VideoSplitter()
# Thread lock for rate limiting
self._rate_limit_lock = threading.Lock()
def send_usage_webhook(self, user_email: str, prompt: str) -> None:
"""
@ -92,66 +119,39 @@ class VideoProcessor:
def process_video(self, video_path: str, prompt: str, user_email: str = "anonymous") -> Dict[str, Any]:
"""
Process a video with the given prompt using Gemini API
Args:
video_path: Path to the video file
prompt: Text prompt to use for video analysis
user_email: Email of the user processing the video (for usage tracking)
Returns:
Dictionary with processing result or error
"""
start_time = time.time()
result = {
"success": False,
"message": "",
"content": ""
"content": "",
"processing_time_seconds": 0
}
logger.info(f"Processing video: {video_path}")
logger.info(f"Prompt: {prompt[:100]}..." if len(prompt) > 100 else f"Prompt: {prompt}")
if not os.path.exists(video_path):
error_msg = f"Video file not found at '{video_path}'"
logger.error(error_msg)
result["message"] = error_msg
return result
try:
# Get file size
file_size = os.path.getsize(video_path)
logger.info(f"File size: {file_size / (1024 * 1024):.2f} MB")
# Upload the video file
logger.info("Uploading video to Gemini API...")
# Log the file size in relation to our threshold (for informational purposes only)
if file_size > self.CHUNKED_UPLOAD_THRESHOLD:
logger.info(f"File size exceeds {self.CHUNKED_UPLOAD_THRESHOLD/(1024*1024):.2f} MB threshold")
else:
logger.info(f"File size below {self.CHUNKED_UPLOAD_THRESHOLD/(1024*1024):.2f} MB threshold")
# All uploads use the same method (our chunking happens in the frontend)
# Google API may handle large files internally in their own way
# Note: display_name is not supported in the new google-genai SDK
video_file = self.client.files.upload(file=video_path)
logger.info(f"Upload successful. File URI: {video_file.uri}")
logger.info(f"Initial file state: {video_file.state.name}")
# Wait for processing if needed
processing_wait_count = 0
while video_file.state.name == "PROCESSING":
processing_wait_count += 1
logger.info(f"File is still processing. Wait count: {processing_wait_count}")
time.sleep(2) # Wait for 2 seconds before checking again
video_file = self.client.files.get(name=video_file.name) # Re-fetch file state
logger.info(f"Updated file state: {video_file.state.name}")
if video_file.state.name != "ACTIVE":
error_msg = f"Error: File did not become active. Current state: {video_file.state.name}"
logger.error(error_msg)
result["message"] = error_msg
return result
file_size_mb = file_size / (1024 * 1024)
logger.info(f"File size: {file_size_mb:.2f} MB")
# Determine MIME type for the video
mime_type, _ = mimetypes.guess_type(video_path)
if not mime_type:
@ -159,23 +159,82 @@ class VideoProcessor:
mime_type = "video/mp4" # Fallback
else:
logger.info(f"MIME type: {mime_type}")
# Create the content parts for the prompt
prompt_parts = [
{"text": prompt},
{"file_data": {
"file_uri": video_file.uri,
"mime_type": mime_type
}}
]
# Generate content using the client
# Use different approach based on file size
# Small files (< 10MB): use inline base64 data (faster, no upload wait)
# Large files (>= 10MB): use file upload API (handles larger files)
# Note: Base64 adds ~37% overhead, so 10MB file = ~13.7MB base64
SIZE_THRESHOLD_MB = 10
uploaded_file = None
if file_size_mb < SIZE_THRESHOLD_MB:
# Small file: Use base64 encoding for inline data
logger.info(f"File < {SIZE_THRESHOLD_MB}MB, using inline base64 data")
with open(video_path, "rb") as video_file_obj:
video_data = video_file_obj.read()
video_base64 = base64.b64encode(video_data).decode('utf-8')
logger.info(f"Base64 encoding complete. Size: {len(video_base64)} characters")
# Create the content parts using inline data
prompt_parts = [
{"text": prompt},
{"inline_data": {
"mime_type": mime_type,
"data": video_base64
}}
]
else:
# Large file: Use file upload API
logger.info(f"File >= {SIZE_THRESHOLD_MB}MB, using file upload API")
upload_start = time.time()
uploaded_file = self.client.files.upload(
file=video_path
)
logger.info(f"Upload complete in {time.time() - upload_start:.1f}s. File URI: {uploaded_file.uri}")
logger.info(f"Initial file state: {uploaded_file.state}")
# Wait for file to be processed
while uploaded_file.state == "PROCESSING":
logger.info("File is still processing, waiting...")
time.sleep(2)
uploaded_file = self.client.files.get(name=uploaded_file.name)
logger.info(f"Updated file state: {uploaded_file.state}")
if uploaded_file.state != "ACTIVE":
error_msg = f"File upload failed. State: {uploaded_file.state}"
logger.error(error_msg)
result["message"] = error_msg
return result
logger.info("File is ACTIVE and ready for processing")
# Create content parts using file reference
prompt_parts = [
{"text": prompt},
{"file_data": {
"file_uri": uploaded_file.uri,
"mime_type": mime_type
}}
]
# Rate limiting: Wait to avoid hitting API limits
# Free tier: 5 RPM, so minimum 12 seconds between requests
with self._rate_limit_lock:
time.sleep(2) # 2 second delay between API calls
# Use the client to generate content with the new SDK API
logger.info("Sending prompt to Gemini for processing...")
api_start = time.time()
response = self.client.models.generate_content(
model='gemini-2.5-pro',
model="gemini-2.5-pro",
contents=prompt_parts
)
logger.info("Received response from Gemini")
api_time = time.time() - api_start
logger.info(f"Received response from Gemini (API call took {api_time:.1f}s)")
# Extract the response content
content = ""
@ -197,19 +256,22 @@ class VideoProcessor:
# Set success result
result["success"] = True
result["content"] = content
result["processing_time_seconds"] = round(time.time() - start_time, 2)
logger.info(f"Processed result with {len(content)} characters")
logger.info(f"Total processing time: {result['processing_time_seconds']}s")
# Send usage data to webhook for tracking
self.send_usage_webhook(user_email, prompt)
# Attempt to delete the file from Gemini storage
try:
logger.info(f"Deleting file from Gemini storage: {video_file.name}")
self.client.files.delete(name=video_file.name)
logger.info("File deleted successfully from Gemini storage")
except Exception as del_err:
logger.warning(f"Could not delete file from Gemini storage: {str(del_err)}")
# Clean up uploaded file if it was used
if uploaded_file:
try:
logger.info(f"Deleting uploaded file: {uploaded_file.name}")
self.client.files.delete(name=uploaded_file.name)
logger.info("File deleted successfully from Gemini storage")
except Exception as del_err:
logger.warning(f"Could not delete file from Gemini storage: {str(del_err)}")
return result
except Exception as e:
@ -219,4 +281,470 @@ class VideoProcessor:
logger.error(error_details)
result["message"] = f"Error processing video: {str(e)}"
result["error_details"] = error_details
return result
return result
def combine_chunk_responses(self, responses: List[str], prompt: str,
num_chunks: int) -> str:
"""
Intelligently combine responses from multiple video chunks.
Args:
responses: List of response texts from each chunk
prompt: Original prompt used for processing
num_chunks: Total number of chunks processed
Returns:
Combined response text
"""
logger.info(f"Combining {len(responses)} chunk responses")
# Detect the prompt type to determine combination strategy
prompt_lower = prompt.lower()
is_meeting_summary = "meeting" in prompt_lower or "summary" in prompt_lower
is_documentation = "documentation" in prompt_lower or "process" in prompt_lower
is_with_charts = "mermaid" in prompt_lower or "diagram" in prompt_lower or "chart" in prompt_lower
if is_with_charts:
return self._combine_with_charts(responses, num_chunks)
elif is_meeting_summary:
return self._combine_meeting_summary(responses, num_chunks)
elif is_documentation:
return self._combine_documentation(responses, num_chunks)
else:
return self._combine_generic(responses, num_chunks)
def _combine_generic(self, responses: List[str], num_chunks: int) -> str:
"""Generic combination: simple sequential joining with section headers."""
logger.info("Using generic combination strategy")
combined = []
combined.append(f"# Complete Video Analysis\n")
combined.append(f"*This video was processed in {num_chunks} parts due to its length.*\n")
for i, response in enumerate(responses, 1):
combined.append(f"\n## Part {i} of {num_chunks}\n")
combined.append(response.strip())
return "\n".join(combined)
def _combine_meeting_summary(self, responses: List[str], num_chunks: int) -> str:
"""Combination strategy optimized for meeting summaries."""
logger.info("Using meeting summary combination strategy")
# First, try to synthesize the segments into a unified summary
try:
logger.info("Attempting to synthesize segments into unified meeting summary")
synthesized = self._synthesize_meeting_segments(responses, num_chunks)
if synthesized:
return synthesized
else:
logger.warning("Synthesis failed, falling back to segment concatenation")
except Exception as e:
logger.warning(f"Error during synthesis: {e}, falling back to segment concatenation")
# Fallback: simple concatenation with formatting
combined = []
combined.append(f"# Complete Meeting Recording Summary\n")
combined.append(f"*This recording was analyzed in {num_chunks} segments.*\n")
combined.append(f"\n---\n")
# Combine all discussion points with clear time markers
for i, response in enumerate(responses, 1):
time_range = self._format_time_range(i, num_chunks)
combined.append(f"\n## Segment {i}: {time_range}\n")
combined.append(response.strip())
combined.append(f"\n---\n")
# Add consolidated note
combined.append(f"\n### Notes")
combined.append(f"- Review all segments above for discussion points and action items")
combined.append(f"- Total recording duration: ~{num_chunks * 50} minutes")
combined.append(f"- Recording was split into {num_chunks} segments for analysis")
return "\n".join(combined)
def _synthesize_meeting_segments(self, responses: List[str], num_chunks: int) -> Optional[str]:
"""
Use AI to synthesize multiple segment summaries into one unified meeting summary.
Args:
responses: List of segment summaries
num_chunks: Number of segments
Returns:
Unified meeting summary or None if synthesis fails
"""
try:
# Prepare the segments for synthesis
segments_text = ""
for i, response in enumerate(responses, 1):
time_range = self._format_time_range(i, num_chunks)
segments_text += f"\n\n### Segment {i} ({time_range}):\n{response.strip()}\n"
# Create synthesis prompt
synthesis_prompt = f"""You are analyzing a meeting recording that was split into {num_chunks} segments due to its length. Below are the summaries from each segment. Your task is to create ONE unified, comprehensive meeting summary that integrates all the information.
SEGMENT SUMMARIES:
{segments_text}
Please provide a SINGLE, UNIFIED meeting summary that:
1. Combines all discussion points into one cohesive narrative (not separated by segments)
2. Consolidates all action items into one master list (removing duplicates if any)
3. Identifies main themes and outcomes across the entire meeting
4. Maintains chronological flow where relevant
5. Uses clear sections: Meeting Summary, Discussion Points, Action Items (with owners)
Format the output as a professional meeting summary document. Do not reference the segments in your output - write as if this was analyzed as one continuous meeting."""
logger.info("Sending synthesis request to Gemini")
synthesis_response = self.client.models.generate_content(
model="gemini-2.5-pro",
contents=synthesis_prompt
)
if synthesis_response.parts:
synthesized_content = ""
for part in synthesis_response.parts:
if hasattr(part, 'text'):
synthesized_content += part.text
if synthesized_content:
logger.info("Successfully synthesized unified meeting summary")
# Add header noting this was synthesized
final_output = "# Meeting Summary\n\n"
final_output += f"*Synthesized from {num_chunks}-segment analysis*\n\n"
final_output += "---\n\n"
final_output += synthesized_content
return final_output
logger.warning("No content in synthesis response")
return None
except Exception as e:
logger.error(f"Error during synthesis: {str(e)}")
return None
def _combine_documentation(self, responses: List[str], num_chunks: int) -> str:
"""Combination strategy optimized for process documentation."""
logger.info("Using documentation combination strategy")
combined = []
combined.append(f"# Complete Process Documentation\n")
combined.append(f"*This process was documented from a {num_chunks}-part video recording.*\n")
combined.append(f"\n## Overview\n")
combined.append(f"This documentation covers the complete process shown in the video. "
f"The content has been organized sequentially across all segments.\n")
for i, response in enumerate(responses, 1):
combined.append(f"\n## Section {i}: {self._format_time_range(i, num_chunks)}\n")
combined.append(response.strip())
combined.append(f"\n\n---\n*End of documentation*")
return "\n".join(combined)
def _combine_with_charts(self, responses: List[str], num_chunks: int) -> str:
"""Combination strategy for documentation with Mermaid diagrams."""
logger.info("Using documentation with charts combination strategy")
combined = []
combined.append(f"# Complete Process Documentation with Workflow Diagrams\n")
combined.append(f"*This analysis spans {num_chunks} video segments.*\n")
# First, add all text content
combined.append(f"\n## Overview and Detailed Steps\n")
for i, response in enumerate(responses, 1):
combined.append(f"\n### Part {i}: {self._format_time_range(i, num_chunks)}\n")
# Separate mermaid diagrams from text content
parts = response.split("```mermaid")
text_part = parts[0].strip()
combined.append(text_part)
# Add mermaid diagrams in a dedicated section
if len(parts) > 1:
for j, diagram_part in enumerate(parts[1:], 1):
if "```" in diagram_part:
diagram_code = diagram_part.split("```")[0]
combined.append(f"\n**Workflow Diagram {i}.{j}:**\n")
combined.append(f"```mermaid{diagram_code}```\n")
# Add any remaining text after the diagram
remaining_text = "```".join(diagram_part.split("```")[1:]).strip()
if remaining_text:
combined.append(remaining_text)
combined.append(f"\n\n---\n*Complete documentation generated from {num_chunks}-part video analysis*")
return "\n".join(combined)
def _format_time_range(self, part_num: int, total_parts: int,
chunk_duration: int = 50) -> str:
"""Format time range for a video part."""
start_min = (part_num - 1) * chunk_duration
end_min = part_num * chunk_duration if part_num < total_parts else "End"
if isinstance(end_min, int):
return f"{start_min}-{end_min} minutes"
else:
return f"{start_min}+ minutes"
def _process_single_chunk(self, chunk_info: Tuple[int, str, str, int, str]) -> Tuple[int, Dict[str, Any]]:
"""
Process a single video chunk. Used for parallel processing.
Args:
chunk_info: Tuple of (chunk_index, chunk_path, chunk_prompt, total_chunks, user_email)
Returns:
Tuple of (chunk_index, result_dict)
"""
chunk_index, chunk_path, chunk_prompt, total_chunks, user_email = chunk_info
logger.info(f"[Parallel] Processing chunk {chunk_index + 1}/{total_chunks}: {chunk_path}")
try:
chunk_result = self.process_video(chunk_path, chunk_prompt, user_email)
logger.info(f"[Parallel] Completed chunk {chunk_index + 1}/{total_chunks}")
return (chunk_index, chunk_result)
except Exception as e:
logger.error(f"[Parallel] Error processing chunk {chunk_index + 1}/{total_chunks}: {str(e)}")
return (chunk_index, {
"success": False,
"message": f"Error processing chunk {chunk_index + 1}: {str(e)}",
"content": ""
})
def _process_chunks_parallel(self, chunk_paths: List[str], prompt: str,
user_email: str) -> List[Dict[str, Any]]:
"""
Process multiple video chunks in parallel using ThreadPoolExecutor.
Args:
chunk_paths: List of paths to video chunk files
prompt: Original prompt for video analysis
user_email: User email for tracking
Returns:
List of result dictionaries in order of chunks
"""
num_chunks = len(chunk_paths)
logger.info(f"Starting parallel processing of {num_chunks} chunks with {self.max_parallel_chunks} workers")
# Prepare chunk information for parallel processing
chunk_infos = []
for i, chunk_path in enumerate(chunk_paths):
chunk_prompt = self._create_chunk_prompt(prompt, i + 1, num_chunks)
chunk_infos.append((i, chunk_path, chunk_prompt, num_chunks, user_email))
# Process chunks in parallel
results = [None] * num_chunks # Pre-allocate results list to maintain order
with ThreadPoolExecutor(max_workers=self.max_parallel_chunks) as executor:
# Submit all chunks for processing
future_to_chunk = {
executor.submit(self._process_single_chunk, chunk_info): chunk_info[0]
for chunk_info in chunk_infos
}
# Collect results as they complete
completed = 0
for future in as_completed(future_to_chunk):
chunk_index = future_to_chunk[future]
try:
chunk_index, result = future.result()
results[chunk_index] = result
completed += 1
logger.info(f"[Parallel] Progress: {completed}/{num_chunks} chunks completed")
except Exception as e:
logger.error(f"[Parallel] Unexpected error for chunk {chunk_index + 1}: {str(e)}")
results[chunk_index] = {
"success": False,
"message": f"Unexpected error: {str(e)}",
"content": ""
}
logger.info(f"[Parallel] All {num_chunks} chunks processed")
return results
def process_long_video(self, video_path: str, prompt: str,
user_email: str = "anonymous", use_parallel: bool = True) -> Dict[str, Any]:
"""
Process a long video by splitting it into chunks and combining the results.
Supports both parallel and sequential processing.
Args:
video_path: Path to the video file
prompt: Text prompt to use for video analysis
user_email: Email of the user processing the video (for usage tracking)
use_parallel: If True, process chunks in parallel; if False, process sequentially
Returns:
Dictionary with processing result or error
"""
start_time = time.time()
result = {
"success": False,
"message": "",
"content": "",
"chunks_processed": 0,
"processing_mode": "parallel" if use_parallel else "sequential",
"processing_time_seconds": 0
}
chunk_paths = []
try:
# Check if video needs splitting
num_chunks, duration_minutes = self.video_splitter.get_chunk_info(video_path)
if num_chunks <= 1:
logger.info("Video does not need splitting, processing normally")
return self.process_video(video_path, prompt, user_email)
logger.info(f"Long video detected: {duration_minutes:.2f} minutes, will be split into {num_chunks} chunks")
logger.info(f"Processing mode: {'PARALLEL' if use_parallel else 'SEQUENTIAL'}")
# Split the video
logger.info("Starting video splitting...")
chunk_paths = self.video_splitter.split_video(video_path)
logger.info(f"Video split into {len(chunk_paths)} chunks successfully")
# Process chunks (parallel or sequential)
if use_parallel:
# Parallel processing using ThreadPoolExecutor
chunk_results = self._process_chunks_parallel(chunk_paths, prompt, user_email)
else:
# Sequential processing (original logic)
chunk_results = []
for i, chunk_path in enumerate(chunk_paths, 1):
logger.info(f"[Sequential] Processing chunk {i}/{len(chunk_paths)}: {chunk_path}")
# Modify prompt to indicate this is part of a multi-part video
chunk_prompt = self._create_chunk_prompt(prompt, i, len(chunk_paths))
# Process this chunk
chunk_result = self.process_video(chunk_path, chunk_prompt, user_email)
chunk_results.append(chunk_result)
logger.info(f"[Sequential] Completed chunk {i}/{len(chunk_paths)}")
# Check for failures in any chunk
chunk_responses = []
for i, chunk_result in enumerate(chunk_results, 1):
if not chunk_result["success"]:
error_msg = f"Failed to process chunk {i}/{len(chunk_paths)}: {chunk_result.get('message', 'Unknown error')}"
logger.error(error_msg)
result["message"] = error_msg
result["chunks_processed"] = i - 1
return result
chunk_responses.append(chunk_result["content"])
# Combine all responses
logger.info("Combining responses from all chunks...")
combined_content = self.combine_chunk_responses(
chunk_responses,
prompt,
len(chunk_paths)
)
result["success"] = True
result["content"] = combined_content
result["chunks_processed"] = len(chunk_paths)
result["processing_time_seconds"] = round(time.time() - start_time, 2)
result["message"] = f"Successfully processed video in {len(chunk_paths)} chunks"
logger.info(f"Long video processing completed successfully: {len(chunk_paths)} chunks")
logger.info(f"Total processing time: {result['processing_time_seconds']}s")
return result
except Exception as e:
import traceback
error_details = traceback.format_exc()
logger.error(f"Error processing long video: {str(e)}")
logger.error(error_details)
result["message"] = f"Error processing long video: {str(e)}"
result["error_details"] = error_details
return result
finally:
# Always clean up chunk files
if chunk_paths:
logger.info("Cleaning up temporary chunk files...")
self.video_splitter.cleanup_chunks(chunk_paths)
def _create_chunk_prompt(self, original_prompt: str, chunk_num: int,
total_chunks: int) -> str:
"""
Create a prompt for a video chunk that provides context about its position.
Args:
original_prompt: The original user prompt
chunk_num: Current chunk number (1-indexed)
total_chunks: Total number of chunks
Returns:
Modified prompt for the chunk
"""
# For meeting summaries, modify the prompt to focus on just summarizing what's in this segment
prompt_lower = original_prompt.lower()
is_meeting = "meeting" in prompt_lower
if is_meeting:
# For meetings, ask for a partial summary of this segment only
if chunk_num == 1:
context = f"[SEGMENT {chunk_num} of {total_chunks} - First 50 minutes] "
context += "Provide a summary of the discussion points and any action items covered in THIS segment only. "
context += "Do not try to provide a complete meeting summary - just summarize what happens in this part. "
elif chunk_num == total_chunks:
context = f"[SEGMENT {chunk_num} of {total_chunks} - Final segment] "
context += "Provide a summary of the discussion points and any action items covered in THIS final segment only. "
context += "This continues from previous segments, but only summarize what happens in this part. "
else:
context = f"[SEGMENT {chunk_num} of {total_chunks} - Middle segment] "
context += "Provide a summary of the discussion points and any action items covered in THIS segment only. "
context += "This is a middle portion of a longer recording - only summarize what happens in this part. "
return context + original_prompt
else:
# For other types, use the original approach
context = f"[PART {chunk_num} of {total_chunks}] "
if chunk_num == 1:
context += "This is the first segment of a longer video. "
elif chunk_num == total_chunks:
context += "This is the final segment continuing from previous parts. "
else:
context += "This is a middle segment continuing from previous parts. "
return context + original_prompt
def process_video_auto(self, video_path: str, prompt: str,
user_email: str = "anonymous") -> Dict[str, Any]:
"""
Automatically process a video, handling both short and long videos.
This method detects if the video needs splitting and processes accordingly.
Args:
video_path: Path to the video file
prompt: Text prompt to use for video analysis
user_email: Email of the user processing the video (for usage tracking)
Returns:
Dictionary with processing result or error
"""
logger.info(f"Auto-processing video: {video_path}")
# Check if video needs splitting
if self.video_splitter.needs_splitting(video_path):
logger.info("Video requires splitting, using long video processing")
return self.process_long_video(video_path, prompt, user_email)
else:
logger.info("Video is within single-chunk limit, using standard processing")
return self.process_video(video_path, prompt, user_email)

238
backend/video_splitter.py Normal file
View file

@ -0,0 +1,238 @@
"""
Video Splitter Module
This module provides functionality to detect video duration and split long videos
into smaller chunks for processing with APIs that have duration limitations.
"""
import ffmpeg
import os
import tempfile
import logging
from typing import List, Tuple, Optional
logger = logging.getLogger('video_query')
class VideoSplitter:
"""
Handles video duration detection and splitting operations.
"""
# Default chunk duration in minutes (25 min to avoid API timeouts)
DEFAULT_CHUNK_DURATION = 25
def __init__(self, chunk_duration_minutes: int = DEFAULT_CHUNK_DURATION):
"""
Initialize VideoSplitter with specified chunk duration.
Args:
chunk_duration_minutes: Duration of each chunk in minutes (default: 50)
"""
self.chunk_duration_minutes = chunk_duration_minutes
self.chunk_duration_seconds = chunk_duration_minutes * 60
logger.info(f"VideoSplitter initialized with chunk duration: {chunk_duration_minutes} minutes")
def get_video_duration(self, video_path: str) -> Optional[float]:
"""
Get the duration of a video file in seconds.
Args:
video_path: Path to the video file
Returns:
Duration in seconds, or None if unable to determine
"""
try:
logger.info(f"Detecting duration for video: {video_path}")
# Explicitly set ffprobe command path to avoid PATH issues
probe = ffmpeg.probe(video_path, cmd='/usr/bin/ffprobe')
# Get duration from video stream
video_info = next(
(stream for stream in probe['streams'] if stream['codec_type'] == 'video'),
None
)
if video_info and 'duration' in video_info:
duration = float(video_info['duration'])
elif 'format' in probe and 'duration' in probe['format']:
duration = float(probe['format']['duration'])
else:
logger.error("Could not find duration in video metadata")
return None
logger.info(f"Video duration: {duration:.2f} seconds ({duration/60:.2f} minutes)")
return duration
except ffmpeg.Error as e:
logger.error(f"FFmpeg error while detecting duration: {e.stderr.decode() if e.stderr else str(e)}")
return None
except Exception as e:
logger.error(f"Error detecting video duration: {str(e)}")
return None
def needs_splitting(self, video_path: str) -> bool:
"""
Check if a video needs to be split based on its duration.
Args:
video_path: Path to the video file
Returns:
True if video duration exceeds chunk duration, False otherwise
"""
duration = self.get_video_duration(video_path)
if duration is None:
logger.warning("Could not determine if video needs splitting")
return False
needs_split = duration > self.chunk_duration_seconds
if needs_split:
logger.info(f"Video needs splitting: {duration/60:.2f} min > {self.chunk_duration_minutes} min")
else:
logger.info(f"Video does not need splitting: {duration/60:.2f} min <= {self.chunk_duration_minutes} min")
return needs_split
def split_video(self, video_path: str, output_dir: Optional[str] = None) -> List[str]:
"""
Split a video into multiple chunks based on the configured chunk duration.
Args:
video_path: Path to the video file to split
output_dir: Directory to save chunks (default: system temp directory)
Returns:
List of paths to the generated chunk files
"""
duration = self.get_video_duration(video_path)
if duration is None:
raise ValueError("Could not determine video duration")
# Use temp directory if none specified
if output_dir is None:
output_dir = tempfile.mkdtemp(prefix="video_chunks_")
logger.info(f"Using temporary directory for chunks: {output_dir}")
else:
os.makedirs(output_dir, exist_ok=True)
# Calculate number of chunks needed
num_chunks = int(duration / self.chunk_duration_seconds) + (
1 if duration % self.chunk_duration_seconds > 0 else 0
)
logger.info(f"Splitting video into {num_chunks} chunks")
chunk_paths = []
video_basename = os.path.splitext(os.path.basename(video_path))[0]
video_extension = os.path.splitext(video_path)[1]
for i in range(num_chunks):
start_time = i * self.chunk_duration_seconds
chunk_output = os.path.join(
output_dir,
f"{video_basename}_chunk_{i+1:02d}{video_extension}"
)
logger.info(f"Creating chunk {i+1}/{num_chunks}: start={start_time}s, output={chunk_output}")
try:
# Split the video using ffmpeg
# Using -t to specify duration of this chunk
# Using -c copy for fast processing (no re-encoding)
stream = ffmpeg.input(video_path, ss=start_time, t=self.chunk_duration_seconds)
stream = ffmpeg.output(
stream,
chunk_output,
c='copy', # Copy streams without re-encoding for speed
map='0', # Include all streams from input
avoid_negative_ts='make_zero' # Handle timestamp issues
)
ffmpeg.run(stream, capture_stdout=True, capture_stderr=True, overwrite_output=True)
chunk_paths.append(chunk_output)
logger.info(f"Successfully created chunk {i+1}/{num_chunks}")
except ffmpeg.Error as e:
error_msg = e.stderr.decode() if e.stderr else str(e)
logger.error(f"FFmpeg error creating chunk {i+1}: {error_msg}")
# Clean up any created chunks on error
self.cleanup_chunks(chunk_paths)
raise RuntimeError(f"Failed to create video chunk {i+1}: {error_msg}")
except Exception as e:
logger.error(f"Error creating chunk {i+1}: {str(e)}")
self.cleanup_chunks(chunk_paths)
raise
logger.info(f"Successfully split video into {len(chunk_paths)} chunks")
return chunk_paths
def cleanup_chunks(self, chunk_paths: List[str]) -> None:
"""
Delete temporary chunk files.
Args:
chunk_paths: List of paths to chunk files to delete
"""
if not chunk_paths:
return
logger.info(f"Cleaning up {len(chunk_paths)} chunk files")
for chunk_path in chunk_paths:
try:
if os.path.exists(chunk_path):
os.remove(chunk_path)
logger.debug(f"Deleted chunk: {chunk_path}")
except Exception as e:
logger.warning(f"Could not delete chunk {chunk_path}: {str(e)}")
# Try to remove the temp directory if it's empty
if chunk_paths:
chunk_dir = os.path.dirname(chunk_paths[0])
try:
if os.path.exists(chunk_dir) and not os.listdir(chunk_dir):
os.rmdir(chunk_dir)
logger.debug(f"Deleted temporary directory: {chunk_dir}")
except Exception as e:
logger.warning(f"Could not delete temporary directory {chunk_dir}: {str(e)}")
def get_chunk_info(self, video_path: str) -> Tuple[int, float]:
"""
Get information about how a video would be chunked without actually splitting it.
Args:
video_path: Path to the video file
Returns:
Tuple of (number_of_chunks, total_duration_in_minutes)
"""
duration = self.get_video_duration(video_path)
if duration is None:
return (0, 0.0)
duration_minutes = duration / 60
num_chunks = int(duration / self.chunk_duration_seconds) + (
1 if duration % self.chunk_duration_seconds > 0 else 0
)
return (num_chunks, duration_minutes)
# Convenience functions for direct use
def get_video_duration(video_path: str) -> Optional[float]:
"""Get video duration in seconds."""
splitter = VideoSplitter()
return splitter.get_video_duration(video_path)
def split_video(video_path: str, chunk_duration_minutes: int = 50,
output_dir: Optional[str] = None) -> List[str]:
"""Split a video into chunks."""
splitter = VideoSplitter(chunk_duration_minutes=chunk_duration_minutes)
return splitter.split_video(video_path, output_dir=output_dir)
def cleanup_chunks(chunk_paths: List[str]) -> None:
"""Clean up chunk files."""
splitter = VideoSplitter()
splitter.cleanup_chunks(chunk_paths)

5
frontend/.env
View file

@ -2,4 +2,7 @@
# The app now uses runtime configuration from config.json for base path detection
# Optional: Set this if you want to override the runtime base path detection
# REACT_APP_BASE_PATH_OVERRIDE=/your-custom-path
# REACT_APP_BASE_PATH_OVERRIDE=/your-custom-path
# Temporarily disable Azure AD authentication for testing
REACT_APP_DISABLE_AUTH=true

65
frontend/CONFIG_README.md Normal file
View file

@ -0,0 +1,65 @@
# Frontend Configuration Guide
## Overview
The frontend uses two configuration files to support both local development and production deployment:
1. **`config.js`** - Production configuration (ALWAYS committed to git)
2. **`config.local.js`** - Local development configuration (NEVER committed to git)
## How It Works
The `index.html` loads both files in order:
1. First loads `config.js` (production config)
2. Then loads `config.local.js` (if it exists, overrides production config)
In production, only `config.js` exists, so production settings are used.
In development, `config.local.js` overrides the production settings to point to localhost.
## Local Development Setup
**`config.local.js`** is already created and configured to point to:
- Frontend: `http://localhost:3000`
- Backend: `http://localhost:5010`
This file is listed in `.gitignore` and will NOT be committed to git.
## Production Deployment
**`config.js`** contains production settings:
- Frontend: `https://brandtechsandbox.oliver.solutions/video-query/`
- Backend: `https://brandtechsandbox.oliver.solutions/video_query_back`
When deploying to production:
1. Only `config.js` is deployed (config.local.js is excluded)
2. The application automatically uses production URLs
3. No manual configuration changes needed
## Important Notes
- ✅ **DO commit** `config.js` (production config)
- ❌ **DO NOT commit** `config.local.js` (local dev config)
- The `.gitignore` file ensures `config.local.js` is never accidentally committed
- If you need to modify production URLs, edit `config.js`
- If you need to modify local development URLs, edit `config.local.js`
## Troubleshooting
### "Backend is not reachable" in local development
1. Check that backend is running: `ps aux | grep "python.*run.py"`
2. Verify backend is listening on port 5010: `ss -tulpn | grep 5010`
3. Check `config.local.js` has correct URLs
4. Clear browser cache and hard reload (Ctrl+Shift+R or Cmd+Shift+R)
### Changes to config not taking effect
1. Hard reload the browser (Ctrl+Shift+R or Cmd+Shift+R)
2. Check browser console for config loading messages
3. Verify the correct config file is being loaded
### CORS errors in browser console
This means the frontend and backend URLs don't match. Check:
1. `config.local.js` (or `config.js`) has correct backend URL
2. Backend CORS settings in `backend/app.py` include your frontend URL

1522
frontend/package-lock.json generated
View file

File diff suppressed because it is too large Load diff

14
frontend/public/config.js
View file

@ -1,16 +1,16 @@
window.__APP_CONFIG__ = {
"_comment": "Dynamic base path configuration - set basePath to override auto-detection",
"basePath": "/video_query",
"domain": "https://ai-sandbox.oliver.solutions",
"_comment": "PRODUCTION CONFIG - Dynamic base path configuration - set basePath to override auto-detection",
"basePath": "/video-query",
"domain": "https://brandtechsandbox.oliver.solutions",
"msal": {
"clientId": "9079054c-9620-4757-a256-23413042f1ef",
"authority": "https://login.microsoftonline.com/e519c2e6-bc6d-4fdf-8d9c-923c2f002385",
"redirectUri": "https://ai-sandbox.oliver.solutions/video_query/",
"postLogoutRedirectUri": "https://ai-sandbox.oliver.solutions/video_query/",
"redirectUri": "https://brandtechsandbox.oliver.solutions/video-query/",
"postLogoutRedirectUri": "https://brandtechsandbox.oliver.solutions/video-query/",
"tenantId": "e519c2e6-bc6d-4fdf-8d9c-923c2f002385"
},
"api": {
"videoProcessingEndpoint": "https://ai-sandbox.oliver.solutions/video_query_back/api/process",
"chunkedUploadEndpoint": "https://ai-sandbox.oliver.solutions/video_query_back"
"videoProcessingEndpoint": "https://brandtechsandbox.oliver.solutions/video_query_back/api/process",
"chunkedUploadEndpoint": "https://brandtechsandbox.oliver.solutions/video_query_back"
}
};

14
frontend/public/config.json
View file

@ -1,16 +1,16 @@
{
"_comment": "Dynamic base path configuration - set basePath to override auto-detection",
"basePath": "/video_query",
"domain": "https://ai-sandbox.oliver.solutions",
"_comment": "LOCAL DEVELOPMENT CONFIG - Points to localhost backend on port 5010",
"basePath": "/",
"domain": "http://localhost:3000",
"msal": {
"clientId": "9079054c-9620-4757-a256-23413042f1ef",
"authority": "https://login.microsoftonline.com/e519c2e6-bc6d-4fdf-8d9c-923c2f002385",
"redirectUri": "https://ai-sandbox.oliver.solutions/video_query/",
"postLogoutRedirectUri": "https://ai-sandbox.oliver.solutions/video_query/",
"redirectUri": "http://localhost:3000/",
"postLogoutRedirectUri": "http://localhost:3000/",
"tenantId": "e519c2e6-bc6d-4fdf-8d9c-923c2f002385"
},
"api": {
"videoProcessingEndpoint": "https://ai-sandbox.oliver.solutions/video_query_back/api/process",
"chunkedUploadEndpoint": "https://ai-sandbox.oliver.solutions/video_query_back"
"videoProcessingEndpoint": "http://localhost:5010/api/process",
"chunkedUploadEndpoint": "http://localhost:5010"
}
}

4
frontend/public/index.html
View file

@ -38,7 +38,11 @@
}
})();
</script>
<!-- Load production config first -->
<script src="./config.js"></script>
<!-- For local development, load config.local.js to override production settings -->
<!-- This file is only present in local development, not in production -->
<script src="./config.local.js" onerror="console.log('No local config found, using production config')"></script>
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>

336
frontend/src/App.js
View file

@ -1,4 +1,4 @@
import React, { useState } from 'react';
import React, { useState, useEffect } from 'react';
import { authApiClient } from './auth/authApiClient';
import { AuthenticatedTemplate, UnauthenticatedTemplate, useMsal } from '@azure/msal-react';
import { InteractionStatus } from '@azure/msal-browser';
@ -6,11 +6,22 @@ import AuthenticatedContent from './components/AuthenticatedContent';
import Login from './components/Login';
import ChunkedUploader from './utils/chunkedUploader';
import { loginRequest } from './auth/authConfig';
import { getApiConfig } from './utils/configLoader';
import { loadConfig, getApiConfig } from './utils/configLoader';
function App() {
// MSAL authentication hook
// Component that uses MSAL hooks (only when auth is enabled)
function AppWithAuth() {
const { instance, inProgress, accounts } = useMsal();
return <AppContent instance={instance} inProgress={inProgress} accounts={accounts} authDisabled={false} />;
}
// Component without auth dependencies
function AppWithoutAuth() {
return <AppContent instance={null} inProgress={null} accounts={[]} authDisabled={true} />;
}
// Main app content component that receives auth props
function AppContent({ instance, inProgress, accounts, authDisabled }) {
const [selectedFile, setSelectedFile] = useState(null);
const [fileName, setFileName] = useState('');
const [mode, setMode] = useState('meeting_summary');
@ -21,46 +32,94 @@ function App() {
const [isLoading, setIsLoading] = useState(false);
const [error, setError] = useState('');
const [uploadProgress, setUploadProgress] = useState(0);
const [authChecked, setAuthChecked] = useState(false);
const [chunksProcessed, setChunksProcessed] = useState(0);
const [totalChunks, setTotalChunks] = useState(0);
const [authChecked, setAuthChecked] = useState(authDisabled); // Skip auth check if disabled
const [configLoaded, setConfigLoaded] = useState(false);
// Queue state for multiple files
const [uploadQueue, setUploadQueue] = useState([]);
const [currentFileIndex, setCurrentFileIndex] = useState(-1);
const [isProcessingQueue, setIsProcessingQueue] = useState(false);
const handleVideoSelect = (file) => {
setSelectedFile(file);
setFileName(file.name);
// Load configuration when authentication is disabled
useEffect(() => {
if (authDisabled) {
loadConfig()
.then(() => {
console.log('Configuration loaded successfully');
setConfigLoaded(true);
})
.catch(error => {
console.error('Failed to load configuration:', error);
setError('Failed to load application configuration');
});
} else {
// Configuration is loaded by AuthProvider when auth is enabled
setConfigLoaded(true);
}
}, [authDisabled]);
const handleVideoSelect = (files) => {
// Handle both single file and multiple files
const fileArray = Array.isArray(files) ? files : [files];
// Create queue items
const queueItems = fileArray.map((file, index) => ({
id: Date.now() + index,
file: file,
fileName: file.name,
status: 'queued', // queued, processing, completed, failed, cancelled
progress: 0,
result: null,
error: null,
processingTime: null,
abortController: new AbortController() // For cancellation support
}));
setUploadQueue(prev => [...prev, ...queueItems]);
setError('');
};
const handleProcessVideo = async () => {
// Validation
if (!selectedFile) {
setError('Please select a video file first');
// Process a single file from the queue
const processSingleFile = async (queueItem) => {
const fileId = queueItem.id;
const file = queueItem.file;
const abortSignal = queueItem.abortController.signal;
// Update status to processing
setUploadQueue(prev => prev.map(item =>
item.id === fileId ? { ...item, status: 'processing', progress: 0 } : item
));
const startTime = Date.now();
// Check if already aborted
if (abortSignal.aborted) {
setUploadQueue(prev => prev.map(item =>
item.id === fileId ? { ...item, status: 'cancelled' } : item
));
return;
}
if (!prompt.trim()) {
setError('Please enter a prompt');
return;
}
setError('');
setIsLoading(true);
setResult('');
setUploadProgress(0);
try {
const fileSize = selectedFile.size;
const fileSize = file.size;
const fileSizeMB = fileSize / (1024 * 1024);
let response;
console.log(`Starting upload of ${selectedFile.name} (${fileSizeMB.toFixed(2)} MB)`);
console.log(`Starting upload of ${file.name} (${fileSizeMB.toFixed(2)} MB)`);
// Always use chunked upload regardless of file size
console.log('Using chunked upload for all files');
// Create chunked uploader with runtime config
const apiConfigForUpload = getApiConfig();
const uploader = new ChunkedUploader(selectedFile, (progress) => {
const uploader = new ChunkedUploader(file, (progress) => {
console.log(`Upload progress: ${progress}%`);
setUploadProgress(progress);
// Update queue item progress
setUploadQueue(prev => prev.map(item =>
item.id === fileId ? { ...item, progress } : item
));
}, apiConfigForUpload.chunkedUploadEndpoint);
// Variable to store upload result
@ -98,35 +157,194 @@ function App() {
headers: {
'Content-Type': 'application/json'
},
timeout: 3600000 // 60 minutes timeout
timeout: 3600000, // 60 minutes timeout
signal: abortSignal // Support cancellation
}
);
// Handle success with additional error checking
if (response && response.data && response.data.success) {
console.log('Processing successful, setting result');
setResult(response.data.content);
console.log('Processing successful for', file.name);
const processingTime = ((Date.now() - startTime) / 1000).toFixed(2);
// Update queue item as completed
setUploadQueue(prev => prev.map(item =>
item.id === fileId ? {
...item,
status: 'completed',
progress: 100,
result: response.data.content,
processingTime: `${processingTime}s`
} : item
));
} else {
const errorMessage = response?.data?.message || 'Processing failed';
console.error('Processing failed:', errorMessage);
setError(errorMessage);
// Update queue item as failed
setUploadQueue(prev => prev.map(item =>
item.id === fileId ? {
...item,
status: 'failed',
error: errorMessage
} : item
));
}
} catch (err) {
console.error('Error processing video:', err);
setError(
err.response?.data?.message ||
'Failed to process the video. Please try again or use a shorter video.'
);
} finally {
setIsLoading(false);
// Check if error is due to cancellation
if (err.name === 'CanceledError' || err.code === 'ERR_CANCELED' || abortSignal.aborted) {
console.log('Video processing was cancelled by user');
setUploadQueue(prev => prev.map(item =>
item.id === fileId ? {
...item,
status: 'cancelled',
error: 'Processing cancelled by user'
} : item
));
return;
}
const errorMessage = err.response?.data?.message ||
'Failed to process the video. Please try again or use a shorter video.';
// Update queue item as failed
setUploadQueue(prev => prev.map(item =>
item.id === fileId ? {
...item,
status: 'failed',
error: errorMessage
} : item
));
}
};
// Process all files in queue with parallel processing
const handleProcessQueue = async () => {
// Validation
if (uploadQueue.length === 0) {
setError('Please add video files to the queue first');
return;
}
if (!prompt.trim()) {
setError('Please enter a prompt');
return;
}
if (!configLoaded) {
setError('Configuration not loaded yet. Please wait a moment and try again.');
return;
}
setError('');
setIsProcessingQueue(true);
// Parallel processing with max 2 concurrent videos (safe for API rate limits)
const MAX_PARALLEL = 2;
const queuedItems = uploadQueue.filter(item =>
item.status === 'queued' || item.status === 'cancelled'
);
console.log(`Starting parallel processing of ${queuedItems.length} videos (max ${MAX_PARALLEL} concurrent)`);
// Process in batches
for (let i = 0; i < queuedItems.length; i += MAX_PARALLEL) {
const batch = queuedItems.slice(i, i + MAX_PARALLEL);
console.log(`Processing batch: ${batch.map(item => item.fileName).join(', ')}`);
// Process batch in parallel using Promise.allSettled
// allSettled ensures all promises complete even if some fail
await Promise.allSettled(
batch.map(queueItem => processSingleFile(queueItem))
);
console.log(`Batch completed`);
}
setIsProcessingQueue(false);
setCurrentFileIndex(-1);
console.log('All videos processed');
};
// Cancel/Stop processing for a specific file
const handleCancelProcessing = (fileId) => {
const item = uploadQueue.find(q => q.id === fileId);
if (item && item.abortController) {
console.log(`Cancelling processing for: ${item.fileName}`);
item.abortController.abort();
// Status will be updated in processSingleFile's catch block
}
};
// Retry a cancelled or failed file
const handleRetryProcessing = async (fileId) => {
console.log(`Retrying file: ${fileId}`);
// Reset the file status and create new abort controller
setUploadQueue(prev => prev.map(item =>
item.id === fileId ? {
...item,
status: 'queued',
error: null,
progress: 0,
abortController: new AbortController()
} : item
));
// Start processing this specific file
const item = uploadQueue.find(q => q.id === fileId);
if (item) {
// Create updated item with new abort controller
const updatedItem = {
...item,
status: 'queued',
error: null,
progress: 0,
abortController: new AbortController()
};
await processSingleFile(updatedItem);
}
};
// Remove file from queue completely
const handleRemoveFromQueue = (fileId) => {
const item = uploadQueue.find(q => q.id === fileId);
// If currently processing, cancel it first
if (item && item.status === 'processing' && item.abortController) {
item.abortController.abort();
}
// Remove from queue
setUploadQueue(prev => prev.filter(item => item.id !== fileId));
};
// Clear entire queue
const handleClearQueue = () => {
// Cancel all processing items
uploadQueue.forEach(item => {
if (item.status === 'processing' && item.abortController) {
item.abortController.abort();
}
});
setUploadQueue([]);
setCurrentFileIndex(-1);
setError('');
};
const resetForm = () => {
setSelectedFile(null);
setFileName('');
setResult('');
setError('');
setChunksProcessed(0);
setTotalChunks(0);
setUploadQueue([]);
setCurrentFileIndex(-1);
setIsProcessingQueue(false);
};
// Handle login
@ -150,6 +368,12 @@ function App() {
// Check for token in URL hash (from implicit flow) and handle auth status
React.useEffect(() => {
// Skip auth check if authentication is disabled
if (authDisabled) {
setAuthChecked(true);
return;
}
const checkAuthStatus = async () => {
try {
// Check if we were redirected here due to token expiration
@ -320,7 +544,7 @@ function App() {
};
checkAuthStatus();
}, [instance, inProgress, accounts]);
}, [instance, inProgress, accounts, authDisabled]);
// Check if we have a manually stored token (from implicit flow)
const isManuallyAuthenticated = () => {
@ -351,7 +575,7 @@ function App() {
<div className="row">
<div className="col-12">
{/* Show loading while auth is being checked */}
{!authChecked || inProgress === InteractionStatus.Login ? (
{!authChecked || (!authDisabled && inProgress === InteractionStatus.Login) ? (
<div className="text-center my-5">
<div className="spinner-border text-primary" role="status">
<span className="visually-hidden">Loading...</span>
@ -359,9 +583,9 @@ function App() {
<p className="mt-3">Verifying authentication status...</p>
</div>
) : (
// Check both MSAL auth and our manual token auth
(accounts.length > 0 || isManuallyAuthenticated()) ? (
// User is authenticated - show the application
// If auth is disabled, always show the app. Otherwise check authentication
(authDisabled || accounts.length > 0 || isManuallyAuthenticated()) ? (
// User is authenticated or auth is disabled - show the application
<AuthenticatedContent
selectedFile={selectedFile}
fileName={fileName}
@ -371,11 +595,20 @@ function App() {
isLoading={isLoading}
error={error}
uploadProgress={uploadProgress}
chunksProcessed={chunksProcessed}
totalChunks={totalChunks}
onVideoSelect={handleVideoSelect}
onModeChange={setMode}
onPromptChange={setPrompt}
handleProcessVideo={handleProcessVideo}
handleProcessVideo={handleProcessQueue}
resetForm={resetForm}
uploadQueue={uploadQueue}
currentFileIndex={currentFileIndex}
isProcessingQueue={isProcessingQueue}
onRemoveFromQueue={handleRemoveFromQueue}
onClearQueue={handleClearQueue}
onCancelProcessing={handleCancelProcessing}
onRetryProcessing={handleRetryProcessing}
/>
) : (
// User is not authenticated - show login
@ -388,4 +621,15 @@ function App() {
);
}
// Main App component that decides which version to use
function App() {
const authDisabled = process.env.REACT_APP_DISABLE_AUTH === 'true';
if (authDisabled) {
return <AppWithoutAuth />;
} else {
return <AppWithAuth />;
}
}
export default App;

17
frontend/src/auth/AuthProvider.js
View file

@ -105,9 +105,19 @@ export const AuthProvider = ({ children }) => {
const [isInitialized, setIsInitialized] = useState(false);
const [initError, setInitError] = useState(null);
// Check if authentication is disabled
const authDisabled = process.env.REACT_APP_DISABLE_AUTH === 'true';
// Initialize MSAL on component mount
useEffect(() => {
const initialize = async () => {
// Skip MSAL initialization if auth is disabled
if (authDisabled) {
console.log("AuthProvider: Authentication disabled, skipping MSAL initialization");
setIsInitialized(true);
return;
}
try {
console.log("AuthProvider: Starting initialization...");
msalInstance = await createMsalInstance();
@ -121,7 +131,7 @@ export const AuthProvider = ({ children }) => {
};
initialize();
}, []);
}, [authDisabled]);
// Show loading until MSAL is initialized
if (!isInitialized) {
@ -150,6 +160,11 @@ export const AuthProvider = ({ children }) => {
);
}
// If auth is disabled, render children directly without MSAL provider
if (authDisabled) {
return children;
}
// Only render MSAL provider if instance was created successfully
if (!msalInstance) {
return (

10
frontend/src/auth/authApiClient.js
View file

@ -49,6 +49,11 @@ const redirectToLogin = async () => {
// Add request interceptor to add auth token to all API requests
authApiClient.interceptors.request.use(
async (config) => {
// Skip token injection if authentication is disabled
if (process.env.REACT_APP_DISABLE_AUTH === 'true') {
console.log("API: Authentication disabled, skipping token injection");
return config;
}
// First check for ID token (preferred for backend compatibility)
const idToken = sessionStorage.getItem("idToken");
if (idToken) {
@ -136,6 +141,11 @@ authApiClient.interceptors.response.use(
return response;
},
async (error) => {
// Skip auth error handling if authentication is disabled
if (process.env.REACT_APP_DISABLE_AUTH === 'true') {
return Promise.reject(error);
}
// Check if error is due to an unauthorized request (status 401)
if (error.response && error.response.status === 401) {
console.log("API: Received 401 Unauthorized response, redirecting to login");

335
frontend/src/components/AuthenticatedContent.js
View file

@ -1,100 +1,237 @@
import React from 'react';
import { useMsal } from '@azure/msal-react';
import VideoUpload from './VideoUpload';
import PromptSelector from './PromptSelector';
import ResultDisplay from './ResultDisplay';
const AuthenticatedContent = (props) => {
const { instance } = useMsal();
const activeAccount = instance.getActiveAccount();
const handleLogout = () => {
instance.logoutRedirect();
};
return (
<div>
<div className="d-flex justify-content-between align-items-center mb-4">
<h1>Video Query Tool</h1>
<div className="d-flex align-items-center">
{activeAccount && (
<div className="me-3">
<small>Signed in as: <strong>{activeAccount.name}</strong></small>
</div>
)}
<button
className="btn btn-outline-secondary btn-sm"
onClick={handleLogout}
>
Sign Out
</button>
</div>
</div>
{!props.result && (
<>
<VideoUpload onVideoSelect={props.onVideoSelect} />
{props.selectedFile && (
<div className="mb-3">
<div className="alert alert-success d-flex align-items-center">
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" fill="currentColor" className="bi bi-check-circle-fill me-2" viewBox="0 0 16 16">
<path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zm-3.97-3.03a.75.75 0 0 0-1.08.022L7.477 9.417 5.384 7.323a.75.75 0 0 0-1.06 1.06L6.97 11.03a.75.75 0 0 0 1.079-.02l3.992-4.99a.75.75 0 0 0-.01-1.05z"/>
</svg>
<div>
<strong>Video selected:</strong> {props.fileName}
</div>
</div>
</div>
)}
<PromptSelector
mode={props.mode}
onModeChange={props.onModeChange}
prompt={props.prompt}
onPromptChange={props.onPromptChange}
disabled={props.isLoading}
/>
{props.error && (
<div className="alert alert-danger mb-3">
{props.error}
</div>
)}
<div className="d-grid gap-2 d-md-flex mb-4">
<button
className="btn btn-primary"
onClick={props.handleProcessVideo}
disabled={!props.selectedFile || props.isLoading}
>
{props.isLoading ? (
<>
<span className="spinner-border spinner-border-sm me-2" role="status" aria-hidden="true"></span>
Processing...
</>
) : 'Process Video'}
</button>
</div>
</>
)}
<ResultDisplay
result={props.result}
isLoading={props.isLoading}
uploadProgress={props.uploadProgress}
fileName={props.fileName}
/>
{props.result && (
<div className="mt-4">
<button className="btn btn-secondary" onClick={props.resetForm}>
Process Another Video
</button>
</div>
)}
</div>
);
};
import React from 'react';
import { useMsal } from '@azure/msal-react';
import VideoUpload from './VideoUpload';
import PromptSelector from './PromptSelector';
import ResultDisplay from './ResultDisplay';
const AuthenticatedContent = (props) => {
const { instance } = useMsal();
const activeAccount = instance.getActiveAccount();
const handleLogout = () => {
instance.logoutRedirect();
};
return (
<div>
<div className="d-flex justify-content-between align-items-center mb-4">
<h1>Video Query Tool</h1>
<div className="d-flex align-items-center">
{activeAccount && (
<div className="me-3">
<small>Signed in as: <strong>{activeAccount.name}</strong></small>
</div>
)}
<button
className="btn btn-outline-secondary btn-sm"
onClick={handleLogout}
>
Sign Out
</button>
</div>
</div>
{/* Always show video upload area */}
<VideoUpload onVideoSelect={props.onVideoSelect} />
{/* Active Queue Display - queued, processing, cancelled */}
{props.uploadQueue && props.uploadQueue.filter(item => ['queued', 'processing', 'cancelled'].includes(item.status)).length > 0 && (
<div className="mb-4">
<div className="card">
<div className="card-header d-flex justify-content-between align-items-center">
<h5 className="mb-0">
Processing Queue ({props.uploadQueue.filter(item => ['queued', 'processing', 'cancelled'].includes(item.status)).length} videos)
{props.uploadQueue.filter(item => item.status === 'processing').length > 0 && (
<span className="badge bg-primary ms-2">
{props.uploadQueue.filter(item => item.status === 'processing').length} processing
</span>
)}
</h5>
<button
className="btn btn-sm btn-outline-danger"
onClick={props.onClearQueue}
disabled={props.isProcessingQueue}
>
Clear Queue
</button>
</div>
<div className="card-body p-0">
<div className="list-group list-group-flush">
{props.uploadQueue
.filter(item => ['queued', 'processing', 'cancelled'].includes(item.status))
.map((item, index) => (
<div key={item.id} className="list-group-item">
<div className="d-flex justify-content-between align-items-start">
<div className="flex-grow-1">
<div className="d-flex align-items-center mb-2">
{/* Status Icon */}
{item.status === 'queued' && (
<span className="badge bg-secondary me-2">⏳ Queued</span>
)}
{item.status === 'processing' && (
<span className="badge bg-primary me-2">
<span className="spinner-border spinner-border-sm me-1"></span>
Processing
</span>
)}
{item.status === 'cancelled' && (
<span className="badge bg-warning text-dark me-2">⚠️ Cancelled</span>
)}
<strong>{item.fileName}</strong>
</div>
{/* Progress Bar */}
{item.status === 'processing' && (
<div className="progress mb-2" style={{height: '20px'}}>
<div
className="progress-bar progress-bar-striped progress-bar-animated"
role="progressbar"
style={{width: `${item.progress}%`}}
aria-valuenow={item.progress}
aria-valuemin="0"
aria-valuemax="100"
>
{item.progress}%
</div>
</div>
)}
{/* Error Message */}
{item.error && (
<div className="alert alert-danger alert-sm mb-0 mt-2">
<small>{item.error}</small>
</div>
)}
</div>
{/* Action Buttons */}
<div className="d-flex flex-column gap-2 ms-2">
{/* Stop Button - Only for processing videos */}
{item.status === 'processing' && (
<button
className="btn btn-sm btn-warning"
onClick={() => props.onCancelProcessing(item.id)}
title="Stop processing"
>
⏸️ Stop
</button>
)}
{/* Retry Button - Only for cancelled or failed videos */}
{(item.status === 'cancelled') && (
<button
className="btn btn-sm btn-info"
onClick={() => props.onRetryProcessing(item.id)}
title="Retry processing"
>
🔄 Retry
</button>
)}
{/* Remove Button - Always visible */}
<button
className="btn btn-sm btn-outline-danger"
onClick={() => props.onRemoveFromQueue(item.id)}
title="Remove from queue"
>
🗑️ Remove
</button>
</div>
</div>
</div>
))}
</div>
</div>
</div>
</div>
)}
{/* Prompt Selector and Process Button - Always Visible */}
<PromptSelector
mode={props.mode}
onModeChange={props.onModeChange}
prompt={props.prompt}
onPromptChange={props.onPromptChange}
disabled={props.isProcessingQueue}
/>
{props.error && (
<div className="alert alert-danger mb-3">
{props.error}
</div>
)}
<div className="d-grid gap-2 d-md-flex mb-4">
<button
className="btn btn-primary btn-lg"
onClick={props.handleProcessVideo}
disabled={!props.uploadQueue || props.uploadQueue.filter(item => ['queued', 'cancelled'].includes(item.status)).length === 0 || props.isProcessingQueue}
>
{props.isProcessingQueue ? (
<>
<span className="spinner-border spinner-border-sm me-2" role="status" aria-hidden="true"></span>
Processing Queue...
</>
) : `Process ${props.uploadQueue && props.uploadQueue.filter(item => ['queued', 'cancelled'].includes(item.status)).length > 0 ? props.uploadQueue.filter(item => ['queued', 'cancelled'].includes(item.status)).length : 0} Video${props.uploadQueue && props.uploadQueue.filter(item => ['queued', 'cancelled'].includes(item.status)).length !== 1 ? 's' : ''}`}
</button>
</div>
{/* Processed List - Completed and Failed Videos */}
{props.uploadQueue && props.uploadQueue.some(item => ['completed', 'failed'].includes(item.status)) && (
<div className="mt-4">
<div className="card">
<div className="card-header bg-success text-white">
<h5 className="mb-0">
✅ Processed Videos ({props.uploadQueue.filter(item => ['completed', 'failed'].includes(item.status)).length})
</h5>
</div>
<div className="card-body p-0">
{props.uploadQueue.filter(item => ['completed', 'failed'].includes(item.status)).map(item => (
<div key={item.id} className="border-bottom">
<div className="p-3">
<div className="d-flex justify-content-between align-items-center mb-3">
<div>
<h6 className="mb-1">
{item.fileName}
{item.status === 'completed' && (
<span className="badge bg-success ms-2">✓ Completed</span>
)}
{item.status === 'failed' && (
<span className="badge bg-danger ms-2">✗ Failed</span>
)}
</h6>
{item.processingTime && (
<small className="text-muted">⏱️ Processing time: {item.processingTime}</small>
)}
</div>
<button
className="btn btn-sm btn-outline-danger"
onClick={() => props.onRemoveFromQueue(item.id)}
title="Remove from list"
>
🗑️ Remove
</button>
</div>
{item.status === 'completed' && item.result && (
<ResultDisplay result={item.result} isLoading={false} uploadProgress={100} fileName={item.fileName} />
)}
{item.status === 'failed' && item.error && (
<div className="alert alert-danger mb-0">
<strong>Error:</strong> {item.error}
</div>
)}
</div>
</div>
))}
</div>
</div>
</div>
)}
</div>
);
};
export default AuthenticatedContent;

911
frontend/src/components/ResultDisplay.js
View file

@ -1,405 +1,508 @@
import React, { useRef, useEffect, useState } from 'react';
import showdown from 'showdown';
import mermaid from 'mermaid';
import { getApiConfig } from '../utils/configLoader';
const ResultDisplay = ({ result, isLoading, uploadProgress = 0, fileName = '' }) => {
const resultRef = useRef(null);
const [htmlContent, setHtmlContent] = useState('');
// Initialize mermaid
useEffect(() => {
mermaid.initialize({
startOnLoad: true,
theme: 'default',
securityLevel: 'loose'
});
}, []);
// Convert markdown to HTML using showdown
useEffect(() => {
if (result) {
const converter = new showdown.Converter({
tables: true,
tasklists: true,
strikethrough: true,
ghCodeBlocks: true
});
const html = converter.makeHtml(result);
setHtmlContent(html);
}
}, [result]);
// Render mermaid diagrams after HTML content is set
useEffect(() => {
if (htmlContent && resultRef.current) {
setTimeout(() => {
try {
// Find text containing "graph" or "sequenceDiagram" or "flowchart" outside of code blocks
const textNodes = Array.from(resultRef.current.childNodes)
.filter(node => node.nodeType === Node.TEXT_NODE ||
(node.nodeType === Node.ELEMENT_NODE &&
node.tagName !== 'PRE' &&
node.tagName !== 'CODE'));
// Standard code blocks with mermaid
const mermaidCodeBlocks = resultRef.current.querySelectorAll('pre code.language-mermaid');
// Also try to find any pre/code with mermaid content when class wasn't set correctly
const potentialMermaidBlocks = Array.from(resultRef.current.querySelectorAll('pre'))
.filter(pre => {
const codeEl = pre.querySelector('code');
if (!codeEl) return false;
const content = codeEl.textContent.trim();
return content.startsWith('graph ') ||
content.startsWith('sequenceDiagram') ||
content.startsWith('flowchart ') ||
content.includes('mermaid');
});
// Process all known mermaid code blocks
const processBlock = (element, index) => {
const isPreElement = element.tagName === 'PRE';
const codeEl = isPreElement ? element.querySelector('code') : null;
const mermaidCode = codeEl ? codeEl.textContent : element.textContent;
if (!mermaidCode.trim()) return;
// Create a div to hold the rendered diagram
const diagramDiv = document.createElement('div');
diagramDiv.className = 'mermaid';
diagramDiv.id = `mermaid-diagram-${index}`;
diagramDiv.textContent = mermaidCode;
// Store the original code as an attribute so we can access it later
diagramDiv.setAttribute('data-original-code', mermaidCode);
// Replace the original element with the diagram div
if (isPreElement && element.parentElement) {
element.parentElement.replaceChild(diagramDiv, element);
} else if (element.parentElement) {
element.parentElement.replaceChild(diagramDiv, element);
}
};
// Process standard mermaid blocks
mermaidCodeBlocks.forEach(processBlock);
// Process potential mermaid blocks not correctly marked
potentialMermaidBlocks.forEach((block, index) => {
// Only process if it wasn't already processed as a standard mermaid block
if (!block.querySelector('code.language-mermaid')) {
processBlock(block, mermaidCodeBlocks.length + index);
}
});
// Now run mermaid rendering
mermaid.run();
} catch (error) {
console.error('Error rendering mermaid diagrams:', error);
}
}, 100); // Small delay to ensure DOM is fully updated
}
}, [htmlContent]);
// This function is no longer needed with the new approach
const copyToClipboard = () => {
if (!htmlContent) return;
// Clone the current content with rendered diagrams
const contentToExport = resultRef.current.cloneNode(true);
document.body.appendChild(contentToExport);
// Select the content
const range = document.createRange();
range.selectNode(contentToExport);
window.getSelection().removeAllRanges();
window.getSelection().addRange(range);
// Copy the selected content as HTML
document.execCommand('copy');
// Clean up
window.getSelection().removeAllRanges();
document.body.removeChild(contentToExport);
alert('Formatted content copied to clipboard! You can now paste it into Word or other applications.');
};
const [isPdfLoading, setIsPdfLoading] = useState(false);
const downloadPdf = async () => {
if (!htmlContent) return;
setIsPdfLoading(true);
try {
// Wait for any final rendering
await new Promise(resolve => setTimeout(resolve, 1000));
// Force re-render if necessary and wait
if (resultRef.current && resultRef.current.querySelectorAll('.mermaid:not(:empty) svg').length !== resultRef.current.querySelectorAll('.mermaid').length) {
// console.log("Forcing mermaid.run() for PDF export.");
try {
mermaid.run({ nodes: resultRef.current.querySelectorAll('.mermaid') });
await new Promise(resolve => setTimeout(resolve, 1500)); // Longer wait after explicit run
} catch(e) {
console.error("Error during mermaid.run() for PDF:", e);
}
}
const diagramPngs = {}; // Store base64 PNGs keyed by diagram ID
const textDiagrams = {}; // Still useful for alt text or if PNG fails
// It's crucial to work on the live DOM state that mermaid.js has modified.
// Clone the resultRef.current to avoid altering the displayed content if we assign IDs.
const contentToExport = resultRef.current.cloneNode(true);
const mermaidDivs = contentToExport.querySelectorAll('.mermaid');
// console.log(`Found ${mermaidDivs.length} .mermaid elements for PDF export.`);
// Create an array of promises for all conversions
const conversionPromises = Array.from(mermaidDivs).map(async (div, index) => {
let diagramId = div.id;
if (!diagramId) {
// If a div.mermaid doesn't have an ID, assign one.
diagramId = `mermaid-export-${index}`;
div.id = diagramId; // Assign ID to the div in our cloned content
// console.log(`Assigned dynamic ID ${diagramId} to a .mermaid div for export.`);
}
const originalCode = div.getAttribute('data-original-code') ||
(div.firstChild && div.firstChild.nodeType === Node.TEXT_NODE ? div.firstChild.textContent.trim() : div.textContent.trim());
if (originalCode) {
textDiagrams[diagramId] = originalCode;
} else {
console.warn(`No original code found for .mermaid div with ID: ${diagramId}`);
}
const svgElement = div.querySelector('svg');
if (svgElement) {
try {
const svgString = new XMLSerializer().serializeToString(svgElement);
const svgDataUrl = `data:image/svg+xml;charset=utf-8,${encodeURIComponent(svgString)}`;
const image = new Image();
// Create a promise for each image load and canvas conversion
await new Promise((resolve, reject) => {
image.onload = () => {
const canvas = document.createElement('canvas');
// --- Determine canvas size ---
// Option 1: Use SVG's explicit width/height if they are pixel values
let svgWidth = parseFloat(svgElement.getAttribute('width'));
let svgHeight = parseFloat(svgElement.getAttribute('height'));
// Option 2: If no explicit width/height, use viewBox (more robust)
if (isNaN(svgWidth) || isNaN(svgHeight) || svgWidth <= 0 || svgHeight <= 0) {
const viewBox = svgElement.getAttribute('viewBox');
if (viewBox) {
const parts = viewBox.split(' ');
svgWidth = parseFloat(parts[2]);
svgHeight = parseFloat(parts[3]);
}
}
// Fallback if dimensions still not found
if (isNaN(svgWidth) || isNaN(svgHeight) || svgWidth <= 0 || svgHeight <= 0) {
console.warn(`Could not determine dimensions for SVG ${diagramId}, using fallback.`);
svgWidth = 600; // Default/fallback width
svgHeight = 400; // Default/fallback height
}
// Apply a scale factor for better resolution
const scaleFactor = 2; // 2x resolution
canvas.width = svgWidth * scaleFactor;
canvas.height = svgHeight * scaleFactor;
const ctx = canvas.getContext('2d');
if (!ctx) {
console.error('Could not get 2D context from canvas');
reject(new Error('Canvas 2D context unavailable'));
return;
}
// Fill background with white to ensure opaque PNG
ctx.fillStyle = 'white';
ctx.fillRect(0, 0, canvas.width, canvas.height);
ctx.drawImage(image, 0, 0, canvas.width, canvas.height);
diagramPngs[diagramId] = canvas.toDataURL('image/png');
// console.log(`Converted SVG ${diagramId} to PNG (length: ${diagramPngs[diagramId].length})`);
resolve();
};
image.onerror = (err) => {
console.error(`Error loading SVG into Image object for ${diagramId}:`, err);
reject(new Error(`Image loading error for SVG ${diagramId}`));
};
image.src = svgDataUrl;
});
} catch (e) {
console.error(`Error converting SVG ${diagramId} to PNG:`, e);
// No PNG for this diagram if conversion fails
}
} else {
console.warn(`No SVG element found in .mermaid div ID: ${diagramId}. Original code snippet: ${(originalCode || '').substring(0,30)}`);
}
});
// Wait for all SVG to PNG conversions to complete
await Promise.all(conversionPromises);
// The HTML to send is the innerHTML of our (potentially ID-modified) cloned container.
const htmlToSend = contentToExport.innerHTML;
// Debug logging commented out
// console.log("Sending to backend for PDF generation:", {
// htmlLength: htmlToSend.length,
// numTextDiagrams: Object.keys(textDiagrams).length,
// numDiagramPngs: Object.keys(diagramPngs).length,
// });
// if (Object.keys(diagramPngs).length > 0) {
// console.log("Diagram PNG IDs:", Object.keys(diagramPngs));
// }
// Debug logging commented out
// console.log("HTML CONTENT START -------------------");
// console.log(htmlToSend);
// console.log("HTML CONTENT END ---------------------");
// Make API request to generate PDF
const authApiClient = require('../auth/authApiClient').authApiClient;
const apiConfig = getApiConfig();
const pdfEndpoint = `${apiConfig.chunkedUploadEndpoint}/api/generate-pdf`;
const response = await authApiClient.post(
pdfEndpoint,
{
html: htmlToSend,
textDiagrams: textDiagrams,
diagramPngs: diagramPngs, // Send the base64 PNGs instead of SVGs
videoFileName: fileName // Send the original video filename
},
{
headers: {
'Content-Type': 'application/json'
}
}
);
if (response.data.success) {
// Convert base64 PDF to blob
const pdfData = atob(response.data.pdf);
const pdfBytes = new Uint8Array(pdfData.length);
for (let i = 0; i < pdfData.length; i++) {
pdfBytes[i] = pdfData.charCodeAt(i);
}
const pdfBlob = new Blob([pdfBytes], { type: 'application/pdf' });
const pdfUrl = URL.createObjectURL(pdfBlob);
// Create download link and trigger download
const downloadLink = document.createElement('a');
downloadLink.href = pdfUrl;
downloadLink.download = response.data.filename || 'video_query_result.pdf';
document.body.appendChild(downloadLink);
downloadLink.click();
// Clean up
document.body.removeChild(downloadLink);
setTimeout(() => {
URL.revokeObjectURL(pdfUrl);
}, 100);
} else {
throw new Error(response.data.message || 'PDF generation failed');
}
} catch (error) {
console.error('Error downloading PDF:', error);
alert('Failed to generate PDF. Please try again later.');
} finally {
setIsPdfLoading(false);
}
};
if (isLoading) {
const isUploading = uploadProgress < 100;
return (
<div className="processing-spinner">
<div className="spinner-border text-primary mb-3" role="status">
<span className="visually-hidden">Loading...</span>
</div>
<p>
{isUploading
? `Uploading video: ${uploadProgress}% complete...`
: 'Processing video... This may take several minutes depending on the video length.'}
</p>
<div className="progress w-75 mb-3">
<div
className="progress-bar progress-bar-striped progress-bar-animated"
role="progressbar"
style={{ width: isUploading ? `${uploadProgress}%` : '100%' }}
aria-valuenow={isUploading ? uploadProgress : 100}
aria-valuemin="0"
aria-valuemax="100"
></div>
</div>
{!isUploading && (
<div className="alert alert-info">
<small>
<strong>Note:</strong> Your video has been uploaded successfully and is being processed by Gemini AI.
This may take several minutes for longer videos (up to 55 minutes supported).
</small>
</div>
)}
</div>
);
}
if (!result) {
return null;
}
return (
<div>
<div className="d-flex justify-content-between align-items-center mb-3">
<h3>Result</h3>
<div className="d-flex gap-2">
<button
className="btn btn-primary btn-sm"
onClick={copyToClipboard}
>
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" className="bi bi-clipboard me-1" viewBox="0 0 16 16">
<path d="M4 1.5H3a2 2 0 0 0-2 2V14a2 2 0 0 0 2 2h10a2 2 0 0 0 2-2V3.5a2 2 0 0 0-2-2h-1v1h1a1 1 0 0 1 1 1V14a1 1 0 0 1-1 1H3a1 1 0 0 1-1-1V3.5a1 1 0 0 1 1-1h1v-1z"/>
<path d="M9.5 1a.5.5 0 0 1 .5.5v1a.5.5 0 0 1-.5.5h-3a.5.5 0 0 1-.5-.5v-1a.5.5 0 0 1 .5-.5h3zm-3-1A1.5 1.5 0 0 0 5 1.5v1A1.5 1.5 0 0 0 6.5 4h3A1.5 1.5 0 0 0 11 2.5v-1A1.5 1.5 0 0 0 9.5 0h-3z"/>
</svg>
Copy Formatted
</button>
<button
className="btn btn-danger btn-sm"
onClick={downloadPdf}
disabled={isPdfLoading}
>
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" className="bi bi-file-pdf me-1" viewBox="0 0 16 16">
<path d="M14 14V4.5L9.5 0H4a2 2 0 0 0-2 2v12a2 2 0 0 0 2 2h8a2 2 0 0 0 2-2zM9.5 3A1.5 1.5 0 0 0 11 4.5h2V14a1 1 0 0 1-1 1H4a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1h5.5v2z"/>
<path d="M4.603 14.087a.81.81 0 0 1-.438-.42c-.195-.388-.13-.776.08-1.102.198-.307.526-.568.897-.787a7.68 7.68 0 0 1 1.482-.645 19.697 19.697 0 0 0 1.062-2.227 7.269 7.269 0 0 1-.43-1.295c-.086-.4-.119-.796-.046-1.136.075-.354.274-.672.65-.823.192-.077.4-.12.602-.077a.7.7 0 0 1 .477.365c.088.164.12.356.127.538.007.188-.012.396-.047.614-.084.51-.27 1.134-.52 1.794a10.954 10.954 0 0 0 .98 1.686 5.753 5.753 0 0 1 1.334.05c.364.066.734.195.96.465.12.144.193.32.2.518.007.192-.047.382-.138.563a1.04 1.04 0 0 1-.354.416.856.856 0 0 1-.51.138c-.331-.014-.654-.196-.933-.417a5.712 5.712 0 0 1-.911-.95 11.651 11.651 0 0 0-1.997.406 11.307 11.307 0 0 1-1.02 1.51c-.292.35-.609.656-.927.787a.793.793 0 0 1-.58.029zm1.379-1.901c-.166.076-.32.156-.459.238-.328.194-.541.383-.647.547-.094.145-.096.25-.04.361.01.022.02.036.026.044a.266.266 0 0 0 .035-.012c.137-.056.355-.235.635-.572a8.18 8.18 0 0 0 .45-.606zm1.64-1.33a12.71 12.71 0 0 1 1.01-.193 11.744 11.744 0 0 1-.51-.858 20.801 20.801 0 0 1-.5 1.05zm2.446.45c.15.163.296.3.435.41.24.19.407.253.498.256a.107.107 0 0 0 .07-.015.307.307 0 0 0 .094-.125.436.436 0 0 0 .059-.2.095.095 0 0 0-.026-.063c-.052-.062-.2-.152-.518-.209a3.876 3.876 0 0 0-.612-.053zM8.078 7.8a6.7 6.7 0 0 0 .2-.828c.031-.188.043-.343.038-.465a.613.613 0 0 0-.032-.198.517.517 0 0 0-.145.04c-.087.035-.158.106-.196.283-.04.192-.03.469.046.822.024.111.054.227.09.346z"/>
</svg>
{isPdfLoading ? 'Generating...' : 'Download PDF'}
</button>
</div>
</div>
<div
className="result-container"
ref={resultRef}
dangerouslySetInnerHTML={{ __html: htmlContent }}
/>
<div className="copy-instruction mt-3">
<p className="mb-0"><strong>Tip:</strong> Click "Copy Formatted" to copy the content in a format suitable for pasting into Word or other document editors.</p>
</div>
</div>
);
};
import React, { useRef, useEffect, useState } from 'react';
import showdown from 'showdown';
import mermaid from 'mermaid';
import { getApiConfig } from '../utils/configLoader';
const ResultDisplay = ({ result, isLoading, uploadProgress = 0, fileName = '', chunksProcessed = 0, totalChunks = 0 }) => {
const resultRef = useRef(null);
const [htmlContent, setHtmlContent] = useState('');
const [showChunkInfo, setShowChunkInfo] = useState(true);
// Initialize mermaid
useEffect(() => {
mermaid.initialize({
startOnLoad: true,
theme: 'default',
securityLevel: 'loose'
});
}, []);
// Convert markdown to HTML using showdown
useEffect(() => {
if (result) {
const converter = new showdown.Converter({
tables: true,
tasklists: true,
strikethrough: true,
ghCodeBlocks: true
});
const html = converter.makeHtml(result);
setHtmlContent(html);
}
}, [result]);
// Render mermaid diagrams after HTML content is set
useEffect(() => {
if (htmlContent && resultRef.current) {
setTimeout(() => {
try {
// Find text containing "graph" or "sequenceDiagram" or "flowchart" outside of code blocks
const textNodes = Array.from(resultRef.current.childNodes)
.filter(node => node.nodeType === Node.TEXT_NODE ||
(node.nodeType === Node.ELEMENT_NODE &&
node.tagName !== 'PRE' &&
node.tagName !== 'CODE'));
// Standard code blocks with mermaid
const mermaidCodeBlocks = resultRef.current.querySelectorAll('pre code.language-mermaid');
// Also try to find any pre/code with mermaid content when class wasn't set correctly
const potentialMermaidBlocks = Array.from(resultRef.current.querySelectorAll('pre'))
.filter(pre => {
const codeEl = pre.querySelector('code');
if (!codeEl) return false;
const content = codeEl.textContent.trim();
return content.startsWith('graph ') ||
content.startsWith('sequenceDiagram') ||
content.startsWith('flowchart ') ||
content.includes('mermaid');
});
// Process all known mermaid code blocks
const processBlock = (element, index) => {
const isPreElement = element.tagName === 'PRE';
const codeEl = isPreElement ? element.querySelector('code') : null;
const mermaidCode = codeEl ? codeEl.textContent : element.textContent;
if (!mermaidCode.trim()) return;
// Create a div to hold the rendered diagram
const diagramDiv = document.createElement('div');
diagramDiv.className = 'mermaid';
diagramDiv.id = `mermaid-diagram-${index}`;
diagramDiv.textContent = mermaidCode;
// Store the original code as an attribute so we can access it later
diagramDiv.setAttribute('data-original-code', mermaidCode);
// Replace the original element with the diagram div
if (isPreElement && element.parentElement) {
element.parentElement.replaceChild(diagramDiv, element);
} else if (element.parentElement) {
element.parentElement.replaceChild(diagramDiv, element);
}
};
// Process standard mermaid blocks
mermaidCodeBlocks.forEach(processBlock);
// Process potential mermaid blocks not correctly marked
potentialMermaidBlocks.forEach((block, index) => {
// Only process if it wasn't already processed as a standard mermaid block
if (!block.querySelector('code.language-mermaid')) {
processBlock(block, mermaidCodeBlocks.length + index);
}
});
// Now run mermaid rendering
mermaid.run();
} catch (error) {
console.error('Error rendering mermaid diagrams:', error);
}
}, 100); // Small delay to ensure DOM is fully updated
}
}, [htmlContent]);
// This function is no longer needed with the new approach
const copyToClipboard = () => {
if (!htmlContent) return;
// Clone the current content with rendered diagrams
const contentToExport = resultRef.current.cloneNode(true);
document.body.appendChild(contentToExport);
// Select the content
const range = document.createRange();
range.selectNode(contentToExport);
window.getSelection().removeAllRanges();
window.getSelection().addRange(range);
// Copy the selected content as HTML
document.execCommand('copy');
// Clean up
window.getSelection().removeAllRanges();
document.body.removeChild(contentToExport);
alert('Formatted content copied to clipboard! You can now paste it into Word or other applications.');
};
const [isPdfLoading, setIsPdfLoading] = useState(false);
const downloadPdf = async () => {
if (!htmlContent) return;
setIsPdfLoading(true);
try {
// Wait for any final rendering
await new Promise(resolve => setTimeout(resolve, 1000));
// Force re-render if necessary and wait
if (resultRef.current && resultRef.current.querySelectorAll('.mermaid:not(:empty) svg').length !== resultRef.current.querySelectorAll('.mermaid').length) {
// console.log("Forcing mermaid.run() for PDF export.");
try {
mermaid.run({ nodes: resultRef.current.querySelectorAll('.mermaid') });
await new Promise(resolve => setTimeout(resolve, 1500)); // Longer wait after explicit run
} catch(e) {
console.error("Error during mermaid.run() for PDF:", e);
}
}
const diagramPngs = {}; // Store base64 PNGs keyed by diagram ID
const textDiagrams = {}; // Still useful for alt text or if PNG fails
// It's crucial to work on the live DOM state that mermaid.js has modified.
// Clone the resultRef.current to avoid altering the displayed content if we assign IDs.
const contentToExport = resultRef.current.cloneNode(true);
const mermaidDivs = contentToExport.querySelectorAll('.mermaid');
// console.log(`Found ${mermaidDivs.length} .mermaid elements for PDF export.`);
// Create an array of promises for all conversions
const conversionPromises = Array.from(mermaidDivs).map(async (div, index) => {
let diagramId = div.id;
if (!diagramId) {
// If a div.mermaid doesn't have an ID, assign one.
diagramId = `mermaid-export-${index}`;
div.id = diagramId; // Assign ID to the div in our cloned content
// console.log(`Assigned dynamic ID ${diagramId} to a .mermaid div for export.`);
}
const originalCode = div.getAttribute('data-original-code') ||
(div.firstChild && div.firstChild.nodeType === Node.TEXT_NODE ? div.firstChild.textContent.trim() : div.textContent.trim());
if (originalCode) {
textDiagrams[diagramId] = originalCode;
} else {
console.warn(`No original code found for .mermaid div with ID: ${diagramId}`);
}
const svgElement = div.querySelector('svg');
if (svgElement) {
try {
const svgString = new XMLSerializer().serializeToString(svgElement);
const svgDataUrl = `data:image/svg+xml;charset=utf-8,${encodeURIComponent(svgString)}`;
const image = new Image();
// Create a promise for each image load and canvas conversion
await new Promise((resolve, reject) => {
image.onload = () => {
const canvas = document.createElement('canvas');
// --- Determine canvas size ---
// Option 1: Use SVG's explicit width/height if they are pixel values
let svgWidth = parseFloat(svgElement.getAttribute('width'));
let svgHeight = parseFloat(svgElement.getAttribute('height'));
// Option 2: If no explicit width/height, use viewBox (more robust)
if (isNaN(svgWidth) || isNaN(svgHeight) || svgWidth <= 0 || svgHeight <= 0) {
const viewBox = svgElement.getAttribute('viewBox');
if (viewBox) {
const parts = viewBox.split(' ');
svgWidth = parseFloat(parts[2]);
svgHeight = parseFloat(parts[3]);
}
}
// Fallback if dimensions still not found
if (isNaN(svgWidth) || isNaN(svgHeight) || svgWidth <= 0 || svgHeight <= 0) {
console.warn(`Could not determine dimensions for SVG ${diagramId}, using fallback.`);
svgWidth = 600; // Default/fallback width
svgHeight = 400; // Default/fallback height
}
// Apply a scale factor for better resolution
const scaleFactor = 2; // 2x resolution
canvas.width = svgWidth * scaleFactor;
canvas.height = svgHeight * scaleFactor;
const ctx = canvas.getContext('2d');
if (!ctx) {
console.error('Could not get 2D context from canvas');
reject(new Error('Canvas 2D context unavailable'));
return;
}
// Fill background with white to ensure opaque PNG
ctx.fillStyle = 'white';
ctx.fillRect(0, 0, canvas.width, canvas.height);
ctx.drawImage(image, 0, 0, canvas.width, canvas.height);
diagramPngs[diagramId] = canvas.toDataURL('image/png');
// console.log(`Converted SVG ${diagramId} to PNG (length: ${diagramPngs[diagramId].length})`);
resolve();
};
image.onerror = (err) => {
console.error(`Error loading SVG into Image object for ${diagramId}:`, err);
reject(new Error(`Image loading error for SVG ${diagramId}`));
};
image.src = svgDataUrl;
});
} catch (e) {
console.error(`Error converting SVG ${diagramId} to PNG:`, e);
// No PNG for this diagram if conversion fails
}
} else {
console.warn(`No SVG element found in .mermaid div ID: ${diagramId}. Original code snippet: ${(originalCode || '').substring(0,30)}`);
}
});
// Wait for all SVG to PNG conversions to complete
await Promise.all(conversionPromises);
// The HTML to send is the innerHTML of our (potentially ID-modified) cloned container.
const htmlToSend = contentToExport.innerHTML;
// Debug logging commented out
// console.log("Sending to backend for PDF generation:", {
// htmlLength: htmlToSend.length,
// numTextDiagrams: Object.keys(textDiagrams).length,
// numDiagramPngs: Object.keys(diagramPngs).length,
// });
// if (Object.keys(diagramPngs).length > 0) {
// console.log("Diagram PNG IDs:", Object.keys(diagramPngs));
// }
// Debug logging commented out
// console.log("HTML CONTENT START -------------------");
// console.log(htmlToSend);
// console.log("HTML CONTENT END ---------------------");
// Make API request to generate PDF
const authApiClient = require('../auth/authApiClient').authApiClient;
const apiConfig = getApiConfig();
const pdfEndpoint = `${apiConfig.chunkedUploadEndpoint}/api/generate-pdf`;
const response = await authApiClient.post(
pdfEndpoint,
{
html: htmlToSend,
textDiagrams: textDiagrams,
diagramPngs: diagramPngs, // Send the base64 PNGs instead of SVGs
videoFileName: fileName // Send the original video filename
},
{
headers: {
'Content-Type': 'application/json'
}
}
);
if (response.data.success) {
// Convert base64 PDF to blob
const pdfData = atob(response.data.pdf);
const pdfBytes = new Uint8Array(pdfData.length);
for (let i = 0; i < pdfData.length; i++) {
pdfBytes[i] = pdfData.charCodeAt(i);
}
const pdfBlob = new Blob([pdfBytes], { type: 'application/pdf' });
const pdfUrl = URL.createObjectURL(pdfBlob);
// Create download link and trigger download
const downloadLink = document.createElement('a');
downloadLink.href = pdfUrl;
downloadLink.download = response.data.filename || 'video_query_result.pdf';
document.body.appendChild(downloadLink);
downloadLink.click();
// Clean up
document.body.removeChild(downloadLink);
setTimeout(() => {
URL.revokeObjectURL(pdfUrl);
}, 100);
} else {
throw new Error(response.data.message || 'PDF generation failed');
}
} catch (error) {
console.error('Error downloading PDF:', error);
// Better error message based on error type
let errorMessage = 'Failed to generate PDF. ';
if (error.response) {
// Server returned an error
errorMessage += error.response.data?.message || `Server error: ${error.response.status}`;
} else if (error.request) {
// Request was made but no response received
errorMessage += 'No response from server. The PDF may be too large or the connection timed out.';
} else if (error.message) {
errorMessage += error.message;
} else {
errorMessage += 'Please try again later.';
}
alert(errorMessage);
} finally {
setIsPdfLoading(false);
}
};
if (isLoading) {
const isUploading = uploadProgress < 100;
return (
<div className="processing-spinner">
<div className="spinner-border text-primary mb-3" role="status">
<span className="visually-hidden">Loading...</span>
</div>
<p>
{isUploading
? `Uploading video: ${uploadProgress}% complete...`
: 'Processing video... This may take several minutes depending on the video length.'}
</p>
<div className="progress w-75 mb-3">
<div
className="progress-bar progress-bar-striped progress-bar-animated"
role="progressbar"
style={{ width: isUploading ? `${uploadProgress}%` : '100%' }}
aria-valuenow={isUploading ? uploadProgress : 100}
aria-valuemin="0"
aria-valuemax="100"
></div>
</div>
{!isUploading && (
<div className="alert alert-info">
<small>
<strong>Note:</strong> Your video has been uploaded successfully and is being processed by Gemini AI.
This may take several minutes for longer videos (up to 55 minutes supported).
</small>
</div>
)}
</div>
);
}
if (!result) {
return null;
}
return (
<div>
{/* Chunk Processing Info - Collapsible */}
{totalChunks > 1 && (
<div className="card mb-3 border-info">
<div
className="card-header bg-info bg-opacity-10 d-flex justify-content-between align-items-center"
style={{ cursor: 'pointer' }}
onClick={() => setShowChunkInfo(!showChunkInfo)}
>
<div className="d-flex align-items-center">
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" fill="currentColor" className="bi bi-film me-2 text-info" viewBox="0 0 16 16">
<path d="M0 1a1 1 0 0 1 1-1h14a1 1 0 0 1 1 1v14a1 1 0 0 1-1 1H1a1 1 0 0 1-1-1V1zm4 0v6h8V1H4zm8 8H4v6h8V9zM1 1v2h2V1H1zm2 3H1v2h2V4zM1 7v2h2V7H1zm2 3H1v2h2v-2zm-2 3v2h2v-2H1zM15 1h-2v2h2V1zm-2 3v2h2V4h-2zm2 3h-2v2h2V7zm-2 3v2h2v-2h-2zm2 3h-2v2h2v-2z"/>
</svg>
<strong>Long Video Processing Info</strong>
</div>
<div className="d-flex align-items-center">
<span className="badge bg-info me-2">{chunksProcessed}/{totalChunks} chunks processed</span>
<svg
xmlns="http://www.w3.org/2000/svg"
width="16"
height="16"
fill="currentColor"
className={`bi bi-chevron-${showChunkInfo ? 'up' : 'down'}`}
viewBox="0 0 16 16"
>
{showChunkInfo ? (
<path fillRule="evenodd" d="M7.646 4.646a.5.5 0 0 1 .708 0l6 6a.5.5 0 0 1-.708.708L8 5.707l-5.646 5.647a.5.5 0 0 1-.708-.708l6-6z"/>
) : (
<path fillRule="evenodd" d="M1.646 4.646a.5.5 0 0 1 .708 0L8 10.293l5.646-5.647a.5.5 0 0 1 .708.708l-6 6a.5.5 0 0 1-.708 0l-6-6a.5.5 0 0 1 0-.708z"/>
)}
</svg>
</div>
</div>
{showChunkInfo && (
<div className="card-body">
<div className="row">
<div className="col-md-12">
<p className="mb-2">
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" className="bi bi-info-circle me-2 text-info" viewBox="0 0 16 16">
<path d="M8 15A7 7 0 1 1 8 1a7 7 0 0 1 0 14zm0 1A8 8 0 1 0 8 0a8 8 0 0 0 0 16z"/>
<path d="m8.93 6.588-2.29.287-.082.38.45.083c.294.07.352.176.288.469l-.738 3.468c-.194.897.105 1.319.808 1.319.545 0 1.178-.252 1.465-.598l.088-.416c-.2.176-.492.246-.686.246-.275 0-.375-.193-.304-.533L8.93 6.588zM9 4.5a1 1 0 1 1-2 0 1 1 0 0 1 2 0z"/>
</svg>
<strong>Your video was longer than 25 minutes and was processed in {totalChunks} separate chunks.</strong>
</p>
<div className="progress mb-3" style={{ height: '25px' }}>
<div
className="progress-bar bg-info progress-bar-striped"
role="progressbar"
style={{ width: `${(chunksProcessed / totalChunks) * 100}%` }}
aria-valuenow={chunksProcessed}
aria-valuemin="0"
aria-valuemax={totalChunks}
>
{chunksProcessed}/{totalChunks} chunks
</div>
</div>
<ul className="list-unstyled mb-0">
{[...Array(totalChunks)].map((_, index) => (
<li key={index} className="mb-1">
{index < chunksProcessed ? (
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" className="bi bi-check-circle-fill text-success me-2" viewBox="0 0 16 16">
<path d="M16 8A8 8 0 1 1 0 8a8 8 0 0 1 16 0zm-3.97-3.03a.75.75 0 0 0-1.08.022L7.477 9.417 5.384 7.323a.75.75 0 0 0-1.06 1.06L6.97 11.03a.75.75 0 0 0 1.079-.02l3.992-4.99a.75.75 0 0 0-.01-1.05z"/>
</svg>
) : (
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" className="bi bi-circle text-secondary me-2" viewBox="0 0 16 16">
<path d="M8 15A7 7 0 1 1 8 1a7 7 0 0 1 0 14zm0 1A8 8 0 1 0 8 0a8 8 0 0 0 0 16z"/>
</svg>
)}
<span className={index < chunksProcessed ? 'text-success fw-bold' : 'text-muted'}>
Chunk {index + 1} ({index * 25}-{Math.min((index + 1) * 25, (index + 1) * 25)} minutes)
{index < chunksProcessed && ' ✓ Completed'}
</span>
</li>
))}
</ul>
<div className="alert alert-info mt-3 mb-0" role="alert">
<small>
<strong>Note:</strong> Videos longer than 25 minutes are automatically split into chunks to avoid API timeouts.
Each chunk is processed separately and the results are combined intelligently.
</small>
</div>
</div>
</div>
</div>
)}
</div>
)}
<div className="d-flex justify-content-between align-items-center mb-3">
<h3>Result</h3>
<div className="d-flex gap-2">
<button
className="btn btn-primary btn-sm"
onClick={copyToClipboard}
>
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" className="bi bi-clipboard me-1" viewBox="0 0 16 16">
<path d="M4 1.5H3a2 2 0 0 0-2 2V14a2 2 0 0 0 2 2h10a2 2 0 0 0 2-2V3.5a2 2 0 0 0-2-2h-1v1h1a1 1 0 0 1 1 1V14a1 1 0 0 1-1 1H3a1 1 0 0 1-1-1V3.5a1 1 0 0 1 1-1h1v-1z"/>
<path d="M9.5 1a.5.5 0 0 1 .5.5v1a.5.5 0 0 1-.5.5h-3a.5.5 0 0 1-.5-.5v-1a.5.5 0 0 1 .5-.5h3zm-3-1A1.5 1.5 0 0 0 5 1.5v1A1.5 1.5 0 0 0 6.5 4h3A1.5 1.5 0 0 0 11 2.5v-1A1.5 1.5 0 0 0 9.5 0h-3z"/>
</svg>
Copy Formatted
</button>
<button
className="btn btn-danger btn-sm"
onClick={downloadPdf}
disabled={isPdfLoading}
>
<svg xmlns="http://www.w3.org/2000/svg" width="16" height="16" fill="currentColor" className="bi bi-file-pdf me-1" viewBox="0 0 16 16">
<path d="M14 14V4.5L9.5 0H4a2 2 0 0 0-2 2v12a2 2 0 0 0 2 2h8a2 2 0 0 0 2-2zM9.5 3A1.5 1.5 0 0 0 11 4.5h2V14a1 1 0 0 1-1 1H4a1 1 0 0 1-1-1V2a1 1 0 0 1 1-1h5.5v2z"/>
<path d="M4.603 14.087a.81.81 0 0 1-.438-.42c-.195-.388-.13-.776.08-1.102.198-.307.526-.568.897-.787a7.68 7.68 0 0 1 1.482-.645 19.697 19.697 0 0 0 1.062-2.227 7.269 7.269 0 0 1-.43-1.295c-.086-.4-.119-.796-.046-1.136.075-.354.274-.672.65-.823.192-.077.4-.12.602-.077a.7.7 0 0 1 .477.365c.088.164.12.356.127.538.007.188-.012.396-.047.614-.084.51-.27 1.134-.52 1.794a10.954 10.954 0 0 0 .98 1.686 5.753 5.753 0 0 1 1.334.05c.364.066.734.195.96.465.12.144.193.32.2.518.007.192-.047.382-.138.563a1.04 1.04 0 0 1-.354.416.856.856 0 0 1-.51.138c-.331-.014-.654-.196-.933-.417a5.712 5.712 0 0 1-.911-.95 11.651 11.651 0 0 0-1.997.406 11.307 11.307 0 0 1-1.02 1.51c-.292.35-.609.656-.927.787a.793.793 0 0 1-.58.029zm1.379-1.901c-.166.076-.32.156-.459.238-.328.194-.541.383-.647.547-.094.145-.096.25-.04.361.01.022.02.036.026.044a.266.266 0 0 0 .035-.012c.137-.056.355-.235.635-.572a8.18 8.18 0 0 0 .45-.606zm1.64-1.33a12.71 12.71 0 0 1 1.01-.193 11.744 11.744 0 0 1-.51-.858 20.801 20.801 0 0 1-.5 1.05zm2.446.45c.15.163.296.3.435.41.24.19.407.253.498.256a.107.107 0 0 0 .07-.015.307.307 0 0 0 .094-.125.436.436 0 0 0 .059-.2.095.095 0 0 0-.026-.063c-.052-.062-.2-.152-.518-.209a3.876 3.876 0 0 0-.612-.053zM8.078 7.8a6.7 6.7 0 0 0 .2-.828c.031-.188.043-.343.038-.465a.613.613 0 0 0-.032-.198.517.517 0 0 0-.145.04c-.087.035-.158.106-.196.283-.04.192-.03.469.046.822.024.111.054.227.09.346z"/>
</svg>
{isPdfLoading ? 'Generating...' : 'Download PDF'}
</button>
</div>
</div>
<div
className="result-container"
ref={resultRef}
dangerouslySetInnerHTML={{ __html: htmlContent }}
/>
<div className="copy-instruction mt-3">
<p className="mb-0"><strong>Tip:</strong> Click "Copy Formatted" to copy the content in a format suitable for pasting into Word or other document editors.</p>
</div>
</div>
);
};
export default ResultDisplay;

53
frontend/src/components/VideoUpload.js
View file

@ -5,29 +5,34 @@ const MAX_FILE_SIZE = 5 * 1024 * 1024 * 1024; // 5GB
const VideoUpload = ({ onVideoSelect }) => {
const [error, setError] = useState('');
const onDrop = useCallback((acceptedFiles) => {
setError('');
// Handle the uploaded files
if (acceptedFiles && acceptedFiles.length > 0) {
const file = acceptedFiles[0];
// Check file size
if (file.size > MAX_FILE_SIZE) {
setError(`File is too large. Maximum size is 5GB.`);
return;
}
// Check file type
const validTypes = ['video/mp4', 'video/avi', 'video/quicktime', 'video/x-ms-wmv', 'video/x-matroska', 'video/webm'];
if (!validTypes.includes(file.type)) {
setError('Please upload a valid video file (MP4, AVI, MOV, WMV, MKV, WEBM)');
return;
const validFiles = [];
// Validate each file
for (const file of acceptedFiles) {
// Check file size
if (file.size > MAX_FILE_SIZE) {
setError(`File "${file.name}" is too large. Maximum size is 5GB per file.`);
return;
}
// Check file type
if (!validTypes.includes(file.type)) {
setError(`File "${file.name}" is not a valid video format. Supported: MP4, AVI, MOV, WMV, MKV, WEBM`);
return;
}
validFiles.push(file);
}
// Pass the file to parent component
onVideoSelect(file);
// Pass all valid files to parent component
onVideoSelect(validFiles);
}
}, [onVideoSelect]);
@ -36,7 +41,7 @@ const VideoUpload = ({ onVideoSelect }) => {
accept: {
'video/*': ['.mp4', '.avi', '.mov', '.wmv', '.mkv', '.webm']
},
maxFiles: 1
multiple: true
});
return (
@ -55,18 +60,18 @@ const VideoUpload = ({ onVideoSelect }) => {
</svg>
</div>
<p>
{isDragActive ?
'Drop the video here...' :
'Drag and drop a video file here, or click to select a file'}
{isDragActive ?
'Drop the videos here...' :
'Drag and drop video files here, or click to select files'}
</p>
<p className="text-muted small">Supported formats: MP4, AVI, MOV, WMV, MKV, WEBM (max 5GB)</p>
<p className="text-muted small">Supported formats: MP4, AVI, MOV, WMV, MKV, WEBM (max 5GB per file) • Multiple files supported</p>
</div>
{error && <div className="alert alert-danger">{error}</div>}
<div className="notice">
<strong>Important:</strong> The Gemini AI model can only process videos up to 55 minutes in length.
Longer videos will fail to process.
<strong>Note:</strong> Videos longer than 25 minutes are automatically split into chunks and processed in parallel.
Maximum video length: No limit (automatic splitting enabled).
</div>
</div>
);

BIN
video-query.zip Normal file
View file

Binary file not shown.