diff --git a/README.md b/README.md index 939a573..52e55d8 100644 --- a/README.md +++ b/README.md @@ -1,168 +1,425 @@ -# VEO3 Usage Report System +# AI Tools Usage Report System -Automated reporting system for VEO3 video generation usage analytics with Microsoft Azure AD Single Sign-On. +Automated reporting system for AI tool usage analytics with Microsoft Azure AD Single Sign-On. Track usage across multiple AI tools including VEO3 video generation, text-to-image, text-to-voice, and more. ## Features -- **PHP Web Interface**: Interactive dashboard for viewing usage reports +- **Multi-Tool Support**: Track usage for 6+ AI tools: + - VEO3 Video Generation + - Text to Image (Dalle, SD3, Leo, etc.) + - Text to Voice (ElevenLabs) + - Speech to Speech + - Document Translation + - Video Query Analysis +- **Dynamic Tool Detection**: Automatically adapts to new tool types +- **PHP Web Interface**: Interactive dashboard with tool filtering - **SSO Authentication**: Microsoft Azure AD (MSAL) integration for secure access -- **Python Automation**: Automated daily/weekly/monthly email reports via SMTP (Mailgun) +- **Python Automation**: Automated daily/weekly/monthly email reports via SMTP - **Granular Analytics**: - Last 24 hours activity - Last 7 days trends - Last 30 days overview + - Tool usage breakdown - Top 25 users per period - **Professional Styling**: Montserrat font, custom yellow (#FFC407) branding -## Setup +## Installation -### PHP Web Reports with SSO +### Prerequisites -1. **Install PHP dependencies:** - ```bash - composer install - ``` +- **PHP**: 7.4 or higher +- **Python**: 3.7 or higher +- **Composer**: For PHP dependency management +- **Web Server**: Apache or Nginx (for production) or PHP built-in server (for development) -2. **Configure SSO (see [SSO-SETUP.md](SSO-SETUP.md)):** - ```bash - cp .env.example .env - nano .env # Add Azure AD tenant and client IDs - ``` +### Step 1: Clone the Repository -3. **For local development (disable SSO):** - ```bash - # In .env - SSO_ENABLED=false - ``` +```bash +git clone git@bitbucket.org:zlalani/sandbox-reports.git +cd sandbox-reports +``` -4. **Start PHP development server:** - ```bash - php -S localhost:8000 - ``` +### Step 2: Install PHP Dependencies -5. **Access the interfaces:** - - `http://localhost:8000/webhook_caller.php` - Fetch data from webhook - - `http://localhost:8000/report.php` - View full report +```bash +composer install +``` + +This installs: +- Microsoft Authentication Library (MSAL) +- Firebase JWT for token validation +- GuzzleHTTP for API requests + +### Step 3: Install Python Dependencies + +```bash +pip install -r requirements.txt +``` + +This installs: +- `requests` - For webhook API calls +- `python-dotenv` - For environment variable management +- `schedule` - For automated task scheduling +- `pytz` - For timezone handling + +### Step 4: Configure Environment Variables + +```bash +cp .env.example .env +nano .env # or use your preferred editor +``` + +Required configuration: + +```bash +# Webhook Configuration +WEBHOOK_URL=https://hook.us1.make.celonis.com/your-webhook-id + +# SMTP Settings (for email reports) +SMTP_SERVER=smtp.mailgun.org +SMTP_PORT=587 +SMTP_USER=your-smtp-username +SMTP_PASSWORD=your-smtp-password +SENDER_EMAIL=reports@yourdomain.com + +# Email Recipients (comma-separated) +REPORT_RECIPIENTS=user1@example.com,user2@example.com + +# SSO Configuration (see SSO-SETUP.md for details) +SSO_ENABLED=true +AZURE_TENANT_ID=your-tenant-id +AZURE_CLIENT_ID=your-client-id +REDIRECT_URI=http://localhost:8000/auth.php + +# Session Configuration +SESSION_SECRET=your-random-secret-key-here +SESSION_LIFETIME=86400 +``` + +### Step 5: Set Up SSO (Optional) + +For production deployments with authentication: + +1. Follow the complete guide in [SSO-SETUP.md](SSO-SETUP.md) +2. Register your application in Azure AD +3. Configure redirect URIs +4. Set `SSO_ENABLED=true` in `.env` + +For development without SSO: + +```bash +# In .env +SSO_ENABLED=false +``` + +### Step 6: Set Up Directory Permissions + +```bash +# Create logs directory +mkdir -p logs +chmod 755 logs + +# Ensure PHP can write webhook response +chmod 755 . +``` + +### Step 7: Start the Application + +#### For Development (Local) + +```bash +# Start PHP built-in server +php -S localhost:8000 +``` + +Access the application: +- **Fetch Data**: http://localhost:8000/webhook_caller.php +- **View Report**: http://localhost:8000/report.php + +#### For Production (Apache/Nginx) + +See [DEPLOYMENT.md](DEPLOYMENT.md) for complete production deployment instructions. + +## Usage + +### Web Interface + +1. **Fetch Data from Webhook** + - Navigate to `webhook_caller.php` + - Select date range (defaults to last 12 weeks) + - Click "Fetch Data" + - Data is saved to `webhook_response.json` + +2. **View Reports** + - Navigate to `report.php` + - View comprehensive analytics dashboard + - Filter by specific AI tools using the filter dropdown + - Use "Refresh Data" button to update from webhook ### Python Email Reports -1. Install dependencies: - ```bash - pip install -r requirements.txt - ``` - -2. Configure environment variables: - ```bash - cp .env.example .env - # Edit .env with your SMTP credentials - ``` - -3. Run the report generator: - ```bash - python veo3_report.py - ``` - -## Configuration (.env) +Generate and send email reports manually: ```bash -# Webhook URL -WEBHOOK_URL=https://hook.us1.make.celonis.com/... - -# SMTP Settings (Mailgun SMTP) -SMTP_SERVER=smtp.mailgun.org -SMTP_PORT=587 -SMTP_USER=your-mailgun-smtp-user@your-domain.com -SMTP_PASSWORD=your-smtp-password -SENDER_EMAIL=reports@your-domain.com - -# Recipients (comma-separated) -REPORT_RECIPIENTS=user1@example.com,user2@example.com +python veo3_report.py ``` -## Scheduling Automated Reports +The script will: +1. Fetch data from the webhook +2. Analyze usage for 24h, 7d, and 30d periods +3. Generate HTML report (`email_report.html`) +4. Send email to configured recipients -### Production Server Deployment (Recommended) +### Automated Reporting -For production servers, use the included systemd service instead of cron: +#### Production Server (Systemd Service - Recommended) -1. **Deploy to server:** - ```bash - # Upload files to server - scp veo3_report.py veo3_scheduler.py requirements.txt .env \ - veo3-report.service deploy.sh user@your-server:/tmp/veo3-deploy/ +```bash +# Deploy to server +scp veo3_report.py veo3_scheduler.py requirements.txt .env \ + veo3-report.service deploy.sh user@your-server:/tmp/veo3-deploy/ - # Run deployment script - ssh user@your-server - cd /tmp/veo3-deploy - chmod +x deploy.sh - sudo ./deploy.sh - ``` +# Run deployment script +ssh user@your-server +cd /tmp/veo3-deploy +chmod +x deploy.sh +sudo ./deploy.sh +``` -2. **Service will run automatically at 7:00 PM EST daily** - -**Benefits:** +The service runs daily at 7:00 PM EST and includes: - ✅ Auto-restart on failure - ✅ Starts on server boot -- ✅ Better logging with journalctl -- ✅ No cron dependency -- ✅ Easy status monitoring +- ✅ Logging with journalctl +- ✅ Status monitoring -See **[DEPLOYMENT.md](DEPLOYMENT.md)** for full deployment guide. +See [DEPLOYMENT.md](DEPLOYMENT.md) for full details. -### Alternative: Manual Scheduling - -#### macOS/Linux (cron) +#### Alternative: Cron Job ```bash -# Daily report at 9 AM -0 9 * * * cd /path/to/VEO3-REPORT && /usr/bin/python3 veo3_report.py +# Edit crontab +crontab -e + +# Add line for daily report at 9 AM +0 9 * * * cd /path/to/sandbox-reports && /usr/bin/python3 veo3_report.py ``` -#### Windows (Task Scheduler) +## Configuration -Create a scheduled task that runs: -``` -python C:\path\to\VEO3-REPORT\veo3_report.py +### Tool Costs + +You can adjust estimated costs per tool in both files: + +**Python** (`veo3_report.py`): +```python +TOOL_COSTS = { + 'VEO3': 3.20, + 'TEXT2IMAGE': 0.04, + 'TEXT2VOICE': 0.30, + 'SPEECH2SPEECH': 0.50, + 'DOCUMENT_TRANSLATION': 0.10, + 'VIDEOQUERY': 0.25, +} ``` -## Report Output +**PHP** (`report.php`): +```php +$toolCosts = [ + 'VEO3' => 3.20, + 'TEXT2IMAGE' => 0.04, + 'TEXT2VOICE' => 0.30, + 'SPEECH2SPEECH' => 0.50, + 'DOCUMENT_TRANSLATION' => 0.10, + 'VIDEOQUERY' => 0.25, +]; +``` -The Python script: -- Fetches data from webhook -- Analyzes three time periods (24h, 7d, 30d) -- Generates HTML report saved as `email_report.html` -- Sends email via SMTP to configured recipients +### Tool Display Names -## Files +Customize how tool names appear in reports: -### Web Interface -- `webhook_caller.php` - Web interface for fetching webhook data -- `report.php` - Interactive web dashboard with collapsible sections +**Python** (`veo3_report.py`): +```python +TOOL_NAMES = { + 'VEO3': 'VEO3 Video Generation', + 'TEXT2IMAGE': 'Text to Image', + # ... add more +} +``` -### Python Automation -- `veo3_report.py` - Automated email report generator -- `veo3_scheduler.py` - Scheduler daemon service (runs continuously) -- `requirements.txt` - Python dependencies +**PHP** (`report.php`): +```php +$toolNames = [ + 'VEO3' => 'VEO3 Video Generation', + 'TEXT2IMAGE' => 'Text to Image', + // ... add more +]; +``` -### Deployment -- `deploy.sh` - Automated server deployment script -- `veo3-report.service` - systemd service configuration -- `DEPLOYMENT.md` - Complete deployment guide +## Project Structure -### Configuration -- `.env.example` - Configuration template -- `.env` - Your configuration (not in git) -- `config.php` - SSO and session configuration -- `env_loader.php` - Environment variable loader +``` +sandbox-reports/ +├── README.md # This file +├── .env.example # Environment variable template +├── .env # Your configuration (not in git) +├── .gitignore # Git ignore rules +│ +├── Web Interface +│ ├── report.php # Main dashboard with filtering +│ ├── webhook_caller.php # Data fetching interface +│ ├── config.php # PHP configuration +│ ├── env_loader.php # Environment loader +│ └── webhook_response.json # Cached data (auto-generated) +│ +├── SSO Authentication +│ ├── AuthMiddleware.php # Authentication middleware +│ ├── JWTValidator.php # JWT token validation +│ ├── auth.php # Auth API endpoint +│ ├── composer.json # PHP dependencies +│ └── SSO-SETUP.md # SSO setup guide +│ +├── Python Reports +│ ├── veo3_report.py # Email report generator +│ ├── veo3_scheduler.py # Scheduling daemon +│ ├── requirements.txt # Python dependencies +│ └── email_report.html # Generated report (auto-created) +│ +└── Deployment + ├── deploy.sh # Deployment script + ├── veo3-report.service # Systemd service file + ├── DEPLOYMENT.md # Deployment guide + └── QUICKSTART.md # Quick start guide +``` -### SSO Files -- `AuthMiddleware.php` - Main authentication class -- `JWTValidator.php` - JWT token validator -- `auth.php` - Authentication API endpoint -- `composer.json` - PHP dependencies -- `SSO-SETUP.md` - Complete SSO setup guide +## Features in Detail -### Generated Files -- `webhook_response.json` - Cached webhook data (auto-generated) -- `email_report.html` - Latest email report (auto-generated) -- `logs/` - Application logs (auto-generated) +### Tool Filtering + +The web interface allows filtering by specific AI tools: +- Select "All Tools" to see complete usage +- Select individual tools to focus on specific usage +- Filter applies to all user tables and statistics + +### Dynamic Tool Detection + +The system automatically handles new tool types: +- Unknown tools are displayed with their raw name +- Cost defaults to $0 for undefined tools +- No code changes needed when new tools are added + +### Multi-Period Analytics + +Each report shows three time periods: +- **Last 24 Hours**: Recent activity snapshot +- **Last 7 Days**: Weekly trends +- **Last 30 Days**: Monthly overview + +Each period includes: +- Total requests and costs +- Unique user count +- Top 25 most active users +- Activity distribution + +### Performance Optimization + +- Refresh button fetches only last 12 weeks (not all data) +- Efficient data caching with `webhook_response.json` +- Responsive design for mobile viewing +- Collapsible sections to reduce page load + +## Troubleshooting + +### PHP Issues + +**Problem**: "Class 'Dotenv' not found" +```bash +# Solution: Install composer dependencies +composer install +``` + +**Problem**: "Permission denied" writing webhook_response.json +```bash +# Solution: Fix directory permissions +chmod 755 . +chmod 666 webhook_response.json # if file exists +``` + +### Python Issues + +**Problem**: "ModuleNotFoundError: No module named 'dotenv'" +```bash +# Solution: Install Python dependencies +pip install -r requirements.txt +``` + +**Problem**: Email not sending +```bash +# Solution: Check SMTP credentials in .env +# Test with: +python veo3_report.py +# Check console output for specific error +``` + +### SSO Issues + +**Problem**: Redirect URI mismatch +- Ensure Azure AD app registration matches exactly +- Check REDIRECT_URI in .env +- See [SSO-SETUP.md](SSO-SETUP.md) for complete guide + +### Data Issues + +**Problem**: No data showing in reports +```bash +# Solution: Fetch data from webhook first +# 1. Visit webhook_caller.php +# 2. Click "Fetch Data" +# 3. Verify webhook_response.json was created +``` + +## Development + +### Adding New Tool Types + +The system is designed to be extensible. To add a new tool type: + +1. Add to tool costs configuration (optional): +```python +# Python +TOOL_COSTS['NEW_TOOL'] = 0.50 + +# PHP +$toolCosts['NEW_TOOL'] = 0.50; +``` + +2. Add display name (optional): +```python +# Python +TOOL_NAMES['NEW_TOOL'] = 'New Tool Name' + +# PHP +$toolNames['NEW_TOOL'] = 'New Tool Name'; +``` + +If you don't add the tool, it will still be tracked and displayed with its raw name and $0 cost. + +## Security + +- SSO authentication via Microsoft Azure AD +- Session management with secure cookies +- JWT token validation +- Environment variables for sensitive data +- `.gitignore` prevents committing secrets +- HTTPS recommended for production + +## License + +Internal use only. + +## Support + +For issues or questions, contact your system administrator or refer to: +- [SSO-SETUP.md](SSO-SETUP.md) - SSO configuration +- [DEPLOYMENT.md](DEPLOYMENT.md) - Production deployment +- [QUICKSTART.md](QUICKSTART.md) - Quick start guide diff --git a/report.php b/report.php index 41d159e..06cb4f5 100644 --- a/report.php +++ b/report.php @@ -453,6 +453,49 @@ $recentDaily = array_slice($dailyCounts, -30, 30, true); @keyframes spin { to { transform: rotate(360deg); } } + .filter-container { + background: white; + padding: 20px 30px; + border-radius: 10px; + box-shadow: 0 2px 8px rgba(0,0,0,0.08); + margin-bottom: 20px; + display: flex; + align-items: center; + gap: 15px; + } + .filter-label { + font-weight: 600; + color: #2c3e50; + font-size: 14px; + } + .filter-select { + padding: 10px 15px; + border: 2px solid #ecf0f1; + border-radius: 8px; + font-size: 14px; + font-family: 'Montserrat', Arial, sans-serif; + color: #2c3e50; + background: white; + cursor: pointer; + transition: border-color 0.3s; + min-width: 250px; + } + .filter-select:hover { + border-color: #FFC407; + } + .filter-select:focus { + outline: none; + border-color: #FFC407; + box-shadow: 0 0 0 3px rgba(255,196,7,0.1); + } + .filter-info { + margin-left: auto; + font-size: 13px; + color: #7f8c8d; + padding: 8px 15px; + background: #f8f9fa; + border-radius: 6px; + } @@ -468,6 +511,21 @@ $recentDaily = array_slice($dailyCounts, -30, 30, true); +
+ + +
Showing all tools
+
+

Total Requests

@@ -545,7 +603,7 @@ $recentDaily = array_slice($dailyCounts, -30, 30, true); } } ?> - + % @@ -892,6 +950,52 @@ $recentDaily = array_slice($dailyCounts, -30, 30, true);