Oliver/sandbox-reports
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Add comprehensive installation guide and tool filtering

Browse source 
README Updates:
- Complete step-by-step installation instructions
- Prerequisites and dependency details
- Configuration guide for both development and production
- Usage instructions for web interface and Python reports
- Troubleshooting section for common issues
- Project structure documentation
- Tool cost configuration guide

Report Filtering Feature:
- Added dropdown filter to select specific AI tools
- Filter applies to tool breakdown table
- Filter state persists across page refreshes using sessionStorage
- Visual feedback showing current filter status
- Clean UI with hover states and transitions

Technical Improvements:
- Added data-tool attributes for JavaScript filtering
- Embedded raw data as JSON for client-side filtering
- Responsive filter container with professional styling
- Filter info badge showing current selection

Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
DJP 2026-01-08 14:54:18 -05:00
parent facacc94d4
commit cd4c03e8b8
2 changed files with 481 additions and 120 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

495
README.md
View file

@ -1,168 +1,425 @@
# VEO3 Usage Report System
# AI Tools Usage Report System
Automated reporting system for VEO3 video generation usage analytics with Microsoft Azure AD Single Sign-On.
Automated reporting system for AI tool usage analytics with Microsoft Azure AD Single Sign-On. Track usage across multiple AI tools including VEO3 video generation, text-to-image, text-to-voice, and more.
## Features
- **PHP Web Interface**: Interactive dashboard for viewing usage reports
- **Multi-Tool Support**: Track usage for 6+ AI tools:
- VEO3 Video Generation
- Text to Image (Dalle, SD3, Leo, etc.)
- Text to Voice (ElevenLabs)
- Speech to Speech
- Document Translation
- Video Query Analysis
- **Dynamic Tool Detection**: Automatically adapts to new tool types
- **PHP Web Interface**: Interactive dashboard with tool filtering
- **SSO Authentication**: Microsoft Azure AD (MSAL) integration for secure access
- **Python Automation**: Automated daily/weekly/monthly email reports via SMTP (Mailgun)
- **Python Automation**: Automated daily/weekly/monthly email reports via SMTP
- **Granular Analytics**:
- Last 24 hours activity
- Last 7 days trends
- Last 30 days overview
- Tool usage breakdown
- Top 25 users per period
- **Professional Styling**: Montserrat font, custom yellow (#FFC407) branding
## Setup
## Installation
### PHP Web Reports with SSO
### Prerequisites
1. **Install PHP dependencies:**
```bash
composer install
```
- **PHP**: 7.4 or higher
- **Python**: 3.7 or higher
- **Composer**: For PHP dependency management
- **Web Server**: Apache or Nginx (for production) or PHP built-in server (for development)
2. **Configure SSO (see [SSO-SETUP.md](SSO-SETUP.md)):**
```bash
cp .env.example .env
nano .env # Add Azure AD tenant and client IDs
```
### Step 1: Clone the Repository
3. **For local development (disable SSO):**
```bash
# In .env
SSO_ENABLED=false
```
```bash
git clone git@bitbucket.org:zlalani/sandbox-reports.git
cd sandbox-reports
```
4. **Start PHP development server:**
```bash
php -S localhost:8000
```
### Step 2: Install PHP Dependencies
5. **Access the interfaces:**
- `http://localhost:8000/webhook_caller.php` - Fetch data from webhook
- `http://localhost:8000/report.php` - View full report
```bash
composer install
```
This installs:
- Microsoft Authentication Library (MSAL)
- Firebase JWT for token validation
- GuzzleHTTP for API requests
### Step 3: Install Python Dependencies
```bash
pip install -r requirements.txt
```
This installs:
- `requests` - For webhook API calls
- `python-dotenv` - For environment variable management
- `schedule` - For automated task scheduling
- `pytz` - For timezone handling
### Step 4: Configure Environment Variables
```bash
cp .env.example .env
nano .env # or use your preferred editor
```
Required configuration:
```bash
# Webhook Configuration
WEBHOOK_URL=https://hook.us1.make.celonis.com/your-webhook-id
# SMTP Settings (for email reports)
SMTP_SERVER=smtp.mailgun.org
SMTP_PORT=587
SMTP_USER=your-smtp-username
SMTP_PASSWORD=your-smtp-password
SENDER_EMAIL=reports@yourdomain.com
# Email Recipients (comma-separated)
REPORT_RECIPIENTS=user1@example.com,user2@example.com
# SSO Configuration (see SSO-SETUP.md for details)
SSO_ENABLED=true
AZURE_TENANT_ID=your-tenant-id
AZURE_CLIENT_ID=your-client-id
REDIRECT_URI=http://localhost:8000/auth.php
# Session Configuration
SESSION_SECRET=your-random-secret-key-here
SESSION_LIFETIME=86400
```
### Step 5: Set Up SSO (Optional)
For production deployments with authentication:
1. Follow the complete guide in [SSO-SETUP.md](SSO-SETUP.md)
2. Register your application in Azure AD
3. Configure redirect URIs
4. Set `SSO_ENABLED=true` in `.env`
For development without SSO:
```bash
# In .env
SSO_ENABLED=false
```
### Step 6: Set Up Directory Permissions
```bash
# Create logs directory
mkdir -p logs
chmod 755 logs
# Ensure PHP can write webhook response
chmod 755 .
```
### Step 7: Start the Application
#### For Development (Local)
```bash
# Start PHP built-in server
php -S localhost:8000
```
Access the application:
- **Fetch Data**: http://localhost:8000/webhook_caller.php
- **View Report**: http://localhost:8000/report.php
#### For Production (Apache/Nginx)
See [DEPLOYMENT.md](DEPLOYMENT.md) for complete production deployment instructions.
## Usage
### Web Interface
1. **Fetch Data from Webhook**
- Navigate to `webhook_caller.php`
- Select date range (defaults to last 12 weeks)
- Click "Fetch Data"
- Data is saved to `webhook_response.json`
2. **View Reports**
- Navigate to `report.php`
- View comprehensive analytics dashboard
- Filter by specific AI tools using the filter dropdown
- Use "Refresh Data" button to update from webhook
### Python Email Reports
1. Install dependencies:
```bash
pip install -r requirements.txt
```
2. Configure environment variables:
```bash
cp .env.example .env
# Edit .env with your SMTP credentials
```
3. Run the report generator:
```bash
python veo3_report.py
```
## Configuration (.env)
Generate and send email reports manually:
```bash
# Webhook URL
WEBHOOK_URL=https://hook.us1.make.celonis.com/...
# SMTP Settings (Mailgun SMTP)
SMTP_SERVER=smtp.mailgun.org
SMTP_PORT=587
SMTP_USER=your-mailgun-smtp-user@your-domain.com
SMTP_PASSWORD=your-smtp-password
SENDER_EMAIL=reports@your-domain.com
# Recipients (comma-separated)
REPORT_RECIPIENTS=user1@example.com,user2@example.com
python veo3_report.py
```
## Scheduling Automated Reports
The script will:
1. Fetch data from the webhook
2. Analyze usage for 24h, 7d, and 30d periods
3. Generate HTML report (`email_report.html`)
4. Send email to configured recipients
### Production Server Deployment (Recommended)
### Automated Reporting
For production servers, use the included systemd service instead of cron:
#### Production Server (Systemd Service - Recommended)
1. **Deploy to server:**
```bash
# Upload files to server
scp veo3_report.py veo3_scheduler.py requirements.txt .env \
veo3-report.service deploy.sh user@your-server:/tmp/veo3-deploy/
```bash
# Deploy to server
scp veo3_report.py veo3_scheduler.py requirements.txt .env \
veo3-report.service deploy.sh user@your-server:/tmp/veo3-deploy/
# Run deployment script
ssh user@your-server
cd /tmp/veo3-deploy
chmod +x deploy.sh
sudo ./deploy.sh
```
# Run deployment script
ssh user@your-server
cd /tmp/veo3-deploy
chmod +x deploy.sh
sudo ./deploy.sh
```
2. **Service will run automatically at 7:00 PM EST daily**
**Benefits:**
The service runs daily at 7:00 PM EST and includes:
- ✅ Auto-restart on failure
- ✅ Starts on server boot
- ✅ Better logging with journalctl
- ✅ No cron dependency
- ✅ Easy status monitoring
- ✅ Logging with journalctl
- ✅ Status monitoring
See **[DEPLOYMENT.md](DEPLOYMENT.md)** for full deployment guide.
See [DEPLOYMENT.md](DEPLOYMENT.md) for full details.
### Alternative: Manual Scheduling
#### macOS/Linux (cron)
#### Alternative: Cron Job
```bash
# Daily report at 9 AM
0 9 * * * cd /path/to/VEO3-REPORT && /usr/bin/python3 veo3_report.py
# Edit crontab
crontab -e
# Add line for daily report at 9 AM
0 9 * * * cd /path/to/sandbox-reports && /usr/bin/python3 veo3_report.py
```
#### Windows (Task Scheduler)
## Configuration
Create a scheduled task that runs:
```
python C:\path\to\VEO3-REPORT\veo3_report.py
### Tool Costs
You can adjust estimated costs per tool in both files:
**Python** (`veo3_report.py`):
```python
TOOL_COSTS = {
'VEO3': 3.20,
'TEXT2IMAGE': 0.04,
'TEXT2VOICE': 0.30,
'SPEECH2SPEECH': 0.50,
'DOCUMENT_TRANSLATION': 0.10,
'VIDEOQUERY': 0.25,
}
```
## Report Output
**PHP** (`report.php`):
```php
$toolCosts = [
'VEO3' => 3.20,
'TEXT2IMAGE' => 0.04,
'TEXT2VOICE' => 0.30,
'SPEECH2SPEECH' => 0.50,
'DOCUMENT_TRANSLATION' => 0.10,
'VIDEOQUERY' => 0.25,
];
```
The Python script:
- Fetches data from webhook
- Analyzes three time periods (24h, 7d, 30d)
- Generates HTML report saved as `email_report.html`
- Sends email via SMTP to configured recipients
### Tool Display Names
## Files
Customize how tool names appear in reports:
### Web Interface
- `webhook_caller.php` - Web interface for fetching webhook data
- `report.php` - Interactive web dashboard with collapsible sections
**Python** (`veo3_report.py`):
```python
TOOL_NAMES = {
'VEO3': 'VEO3 Video Generation',
'TEXT2IMAGE': 'Text to Image',
# ... add more
}
```
### Python Automation
- `veo3_report.py` - Automated email report generator
- `veo3_scheduler.py` - Scheduler daemon service (runs continuously)
- `requirements.txt` - Python dependencies
**PHP** (`report.php`):
```php
$toolNames = [
'VEO3' => 'VEO3 Video Generation',
'TEXT2IMAGE' => 'Text to Image',
// ... add more
];
```
### Deployment
- `deploy.sh` - Automated server deployment script
- `veo3-report.service` - systemd service configuration
- `DEPLOYMENT.md` - Complete deployment guide
## Project Structure
### Configuration
- `.env.example` - Configuration template
- `.env` - Your configuration (not in git)
- `config.php` - SSO and session configuration
- `env_loader.php` - Environment variable loader
```
sandbox-reports/
├── README.md # This file
├── .env.example # Environment variable template
├── .env # Your configuration (not in git)
├── .gitignore # Git ignore rules
│
├── Web Interface
│ ├── report.php # Main dashboard with filtering
│ ├── webhook_caller.php # Data fetching interface
│ ├── config.php # PHP configuration
│ ├── env_loader.php # Environment loader
│ └── webhook_response.json # Cached data (auto-generated)
│
├── SSO Authentication
│ ├── AuthMiddleware.php # Authentication middleware
│ ├── JWTValidator.php # JWT token validation
│ ├── auth.php # Auth API endpoint
│ ├── composer.json # PHP dependencies
│ └── SSO-SETUP.md # SSO setup guide
│
├── Python Reports
│ ├── veo3_report.py # Email report generator
│ ├── veo3_scheduler.py # Scheduling daemon
│ ├── requirements.txt # Python dependencies
│ └── email_report.html # Generated report (auto-created)
│
└── Deployment
├── deploy.sh # Deployment script
├── veo3-report.service # Systemd service file
├── DEPLOYMENT.md # Deployment guide
└── QUICKSTART.md # Quick start guide
```
### SSO Files
- `AuthMiddleware.php` - Main authentication class
- `JWTValidator.php` - JWT token validator
- `auth.php` - Authentication API endpoint
- `composer.json` - PHP dependencies
- `SSO-SETUP.md` - Complete SSO setup guide
## Features in Detail
### Generated Files
- `webhook_response.json` - Cached webhook data (auto-generated)
- `email_report.html` - Latest email report (auto-generated)
- `logs/` - Application logs (auto-generated)
### Tool Filtering
The web interface allows filtering by specific AI tools:
- Select "All Tools" to see complete usage
- Select individual tools to focus on specific usage
- Filter applies to all user tables and statistics
### Dynamic Tool Detection
The system automatically handles new tool types:
- Unknown tools are displayed with their raw name
- Cost defaults to $0 for undefined tools
- No code changes needed when new tools are added
### Multi-Period Analytics
Each report shows three time periods:
- **Last 24 Hours**: Recent activity snapshot
- **Last 7 Days**: Weekly trends
- **Last 30 Days**: Monthly overview
Each period includes:
- Total requests and costs
- Unique user count
- Top 25 most active users
- Activity distribution
### Performance Optimization
- Refresh button fetches only last 12 weeks (not all data)
- Efficient data caching with `webhook_response.json`
- Responsive design for mobile viewing
- Collapsible sections to reduce page load
## Troubleshooting
### PHP Issues
**Problem**: "Class 'Dotenv' not found"
```bash
# Solution: Install composer dependencies
composer install
```
**Problem**: "Permission denied" writing webhook_response.json
```bash
# Solution: Fix directory permissions
chmod 755 .
chmod 666 webhook_response.json # if file exists
```
### Python Issues
**Problem**: "ModuleNotFoundError: No module named 'dotenv'"
```bash
# Solution: Install Python dependencies
pip install -r requirements.txt
```
**Problem**: Email not sending
```bash
# Solution: Check SMTP credentials in .env
# Test with:
python veo3_report.py
# Check console output for specific error
```
### SSO Issues
**Problem**: Redirect URI mismatch
- Ensure Azure AD app registration matches exactly
- Check REDIRECT_URI in .env
- See [SSO-SETUP.md](SSO-SETUP.md) for complete guide
### Data Issues
**Problem**: No data showing in reports
```bash
# Solution: Fetch data from webhook first
# 1. Visit webhook_caller.php
# 2. Click "Fetch Data"
# 3. Verify webhook_response.json was created
```
## Development
### Adding New Tool Types
The system is designed to be extensible. To add a new tool type:
1. Add to tool costs configuration (optional):
```python
# Python
TOOL_COSTS['NEW_TOOL'] = 0.50
# PHP
$toolCosts['NEW_TOOL'] = 0.50;
```
2. Add display name (optional):
```python
# Python
TOOL_NAMES['NEW_TOOL'] = 'New Tool Name'
# PHP
$toolNames['NEW_TOOL'] = 'New Tool Name';
```
If you don't add the tool, it will still be tracked and displayed with its raw name and $0 cost.
## Security
- SSO authentication via Microsoft Azure AD
- Session management with secure cookies
- JWT token validation
- Environment variables for sensitive data
- `.gitignore` prevents committing secrets
- HTTPS recommended for production
## License
Internal use only.
## Support
For issues or questions, contact your system administrator or refer to:
- [SSO-SETUP.md](SSO-SETUP.md) - SSO configuration
- [DEPLOYMENT.md](DEPLOYMENT.md) - Production deployment
- [QUICKSTART.md](QUICKSTART.md) - Quick start guide

106
report.php
View file

@ -453,6 +453,49 @@ $recentDaily = array_slice($dailyCounts, -30, 30, true);
@keyframes spin {
to { transform: rotate(360deg); }
}
.filter-container {
background: white;
padding: 20px 30px;
border-radius: 10px;
box-shadow: 0 2px 8px rgba(0,0,0,0.08);
margin-bottom: 20px;
display: flex;
align-items: center;
gap: 15px;
}
.filter-label {
font-weight: 600;
color: #2c3e50;
font-size: 14px;
}
.filter-select {
padding: 10px 15px;
border: 2px solid #ecf0f1;
border-radius: 8px;
font-size: 14px;
font-family: 'Montserrat', Arial, sans-serif;
color: #2c3e50;
background: white;
cursor: pointer;
transition: border-color 0.3s;
min-width: 250px;
}
.filter-select:hover {
border-color: #FFC407;
}
.filter-select:focus {
outline: none;
border-color: #FFC407;
box-shadow: 0 0 0 3px rgba(255,196,7,0.1);
}
.filter-info {
margin-left: auto;
font-size: 13px;
color: #7f8c8d;
padding: 8px 15px;
background: #f8f9fa;
border-radius: 6px;
}
</style>
</head>
<body>
@ -468,6 +511,21 @@ $recentDaily = array_slice($dailyCounts, -30, 30, true);
</button>
</div>
<div class="filter-container">
<label class="filter-label" for="toolFilter">Filter by Tool:</label>
<select id="toolFilter" class="filter-select" onchange="applyToolFilter()">
<option value="all">All Tools</option>
<?php foreach ($toolCounts as $tool => $count):
$toolName = $toolNames[$tool] ?? $tool;
?>
<option value="<?php echo htmlspecialchars($tool); ?>">
<?php echo htmlspecialchars($toolName); ?> (<?php echo number_format($count); ?> requests)
</option>
<?php endforeach; ?>
</select>
<div class="filter-info" id="filterInfo">Showing all tools</div>
</div>
<div class="stats-grid">
<div class="stat-card">
<h3>Total Requests</h3>
@ -545,7 +603,7 @@ $recentDaily = array_slice($dailyCounts, -30, 30, true);
}
}
?>
<tr>
<tr data-tool="<?php echo htmlspecialchars($tool); ?>">
<td><strong><?php echo htmlspecialchars($toolName); ?></strong></td>
<td><?php echo number_format($count); ?></td>
<td><?php echo number_format($percentage, 1); ?>%</td>
@ -892,6 +950,52 @@ $recentDaily = array_slice($dailyCounts, -30, 30, true);
</div>
<script>
// Store the raw data for filtering
const rawData = <?php echo json_encode($data); ?>;
const toolCosts = <?php echo json_encode($toolCosts); ?>;
const toolNames = <?php echo json_encode($toolNames); ?>;
function applyToolFilter() {
const selectedTool = document.getElementById('toolFilter').value;
const filterInfo = document.getElementById('filterInfo');
if (selectedTool === 'all') {
filterInfo.textContent = 'Showing all tools';
// Show all rows in tool breakdown table
document.querySelectorAll('[data-tool]').forEach(row => {
row.style.display = '';
});
// Show warning that other sections show all data
filterInfo.textContent = 'Showing all tools - Tool breakdown table only';
} else {
const toolName = toolNames[selectedTool] || selectedTool;
filterInfo.textContent = `Filtered by: ${toolName}`;
// Hide/show rows in tool breakdown table
document.querySelectorAll('[data-tool]').forEach(row => {
if (row.dataset.tool === selectedTool) {
row.style.display = '';
} else {
row.style.display = 'none';
}
});
filterInfo.textContent = `Filtered by: ${toolName} - Tool breakdown table only`;
}
// Store filter preference
sessionStorage.setItem('toolFilter', selectedTool);
}
// Restore filter on page load
window.addEventListener('DOMContentLoaded', function() {
const savedFilter = sessionStorage.getItem('toolFilter');
if (savedFilter) {
document.getElementById('toolFilter').value = savedFilter;
applyToolFilter();
}
});
function toggleCollapse() {
const content = document.getElementById('allUsersContent');
const icon = document.querySelector('.collapsible-icon');