creative-x-ferrero/scripts/download_reports.py
DJP b20119b383 Add complete mapping system and automated Box.com monitoring service
Major Features:
- Complete Ferrero ↔ CreativeX mapping system with 93 brands
- Automated Box.com folder monitoring service
- Email notifications with score breakdowns
- Database integration for result storage

Mapping System (v2.0.0):
- mappings.json: 93 brand mappings, 44+ channel mappings
- core/mapping_resolver.py: Translates Ferrero codes to CreativeX format
- scripts/validate_mappings.py: Validation tool for brand/channel support
- scripts/generate_brand_mappings.py: Auto-mapping tool
- scripts/download_reports.py: Scorecard PDF download tool
- Updated scripts/upload.py: Integrated mapping validation
- Updated scripts/check_status.py: Added detailed score display with guidelines

Documentation:
- Updated README.md: Complete user guide with mapping system
- Updated STATUS.md: Production-ready status with test results
- MAPPINGS_GUIDE.md: Complete mapping documentation
- MAPPING_IMPLEMENTATION.md: Implementation summary
- BRAND_MAPPINGS_REVIEW.md: Brand mapping validation guide
- PRODUCTION_BRANDS_SUMMARY.md: Production brand catalog
- PRODUCTION_MAPPING_COMPLETE.md: Mapping completion summary

Automation Service (New):
- creativex-automation/: Complete automated Box monitoring service
- Monitors Box Ferrero-In folder (363284027140) for new files
- Automatically uploads to CreativeX
- Polls for completion (30 min intervals)
- Extracts scores and stores in PostgreSQL creativex_scores table
- Sends formatted emails to file uploader + daveporter@oliver.agency
- Moves processed files to Processed subfolder

Service Components:
- automation/box_monitor.py: Box folder monitoring with uploader detection
- automation/upload_processor.py: CreativeX upload integration
- automation/status_poller.py: CreativeX status polling
- automation/result_handler.py: Score extraction and email sending
- automation/orchestrator.py: Service coordination
- automation/processing_queue.py: JSON-based processing queue
- service.py: Main service entry point
- config.py: Service configuration loader
- requirements.txt: All dependencies
- deployment/systemd/: Systemd service unit file
- Updated shared/notifier.py: Added creativex_upload_complete and creativex_upload_failed templates

Testing:
- Supports --dry-run mode for configuration testing
- Supports --scan-once mode for Box folder testing
- Manual run mode for development/testing
- Comprehensive logging with rotation (10MB, 28 backups)

Database Integration:
- Uses existing creativex_scores table (no migrations needed)
- Compatible with existing Ferrero-Opentext workflows
- Stores full CreativeX API responses in JSONB

Email Templates:
- Matches Ferrero-Opentext styling (#9c27b0 purple for CreativeX)
- Includes score, tier, guidelines breakdown, scorecard URL
- Recipients: Box uploader + CC to daveporter@oliver.agency

Deployment:
- Runs locally for dev/testing
- Systemd service for production
- Auto-restart on failure
- Complete documentation in creativex-automation/README.md

Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
2026-01-29 09:51:16 -05:00

225 lines
6.9 KiB
Python
Executable file

#!/usr/bin/env python3
"""
Download PDF reports for completed uploads
Usage:
python download_reports.py --all
python download_reports.py --file filename.mp4
python download_reports.py --request-id 23135
"""
import argparse
import sys
from pathlib import Path
import requests
# Add parent directory to path
sys.path.insert(0, str(Path(__file__).parent.parent))
from config import load_config
from utils.state_manager import StateManager
from utils.logger import setup_logger
class ReportDownloader:
"""Download PDF reports from Creative X"""
def __init__(self, config):
"""Initialize downloader"""
self.config = config
self.state_manager = StateManager(config.state_file)
self.logger = setup_logger(__name__, config.logs_dir, config.log_level)
# Setup reports directory
self.reports_dir = config.project_root / 'data' / 'reports'
self.reports_dir.mkdir(parents=True, exist_ok=True)
def download_pdf_from_scorecard_url(self, scorecard_url: str, filename: str) -> bool:
"""
Download PDF report from scorecard URL
Args:
scorecard_url: Scorecard web URL
filename: Filename to save as
Returns:
bool: Success
"""
# Try different PDF URL patterns
pdf_urls = [
scorecard_url.replace('/scorecards/', '/scorecards/') + '&format=pdf',
scorecard_url.replace('/scorecards/', '/scorecards/') + '.pdf',
scorecard_url.replace('/audit/scorecards/', '/api/v3/scorecards/') + '/pdf',
]
self.logger.info(f"Attempting to download PDF for: {filename}")
for pdf_url in pdf_urls:
try:
self.logger.info(f" Trying: {pdf_url[:80]}...")
response = requests.get(
pdf_url,
headers={'Authorization': f'Bearer {self.config.access_token}'},
timeout=30
)
if response.status_code == 200:
# Check if response is PDF
content_type = response.headers.get('Content-Type', '')
if 'pdf' in content_type.lower() or response.content[:4] == b'%PDF':
# Save PDF
pdf_path = self.reports_dir / f"{Path(filename).stem}.pdf"
with open(pdf_path, 'wb') as f:
f.write(response.content)
self.logger.info(f" ✓ PDF downloaded: {pdf_path}")
print(f"✓ Downloaded: {pdf_path}")
return True
except Exception as e:
self.logger.debug(f" Failed: {e}")
continue
# If all PDF attempts fail, provide the web URL
self.logger.warning(f" Could not download PDF automatically")
print(f"\n📊 View scorecard online:")
print(f" {scorecard_url}")
print(f"\n💡 To download PDF manually:")
print(f" 1. Open the URL above in your browser")
print(f" 2. Look for 'Export' or 'Download PDF' button")
print(f" 3. Or right-click and 'Print to PDF'")
return False
def download_for_upload(self, filename: str) -> bool:
"""
Download report for specific upload
Args:
filename: Upload filename
Returns:
bool: Success
"""
upload = self.state_manager.get_upload(filename)
if not upload:
print(f"✗ Upload not found: {filename}")
return False
if upload['status'] != StateManager.STATUS_COMPLETED:
print(f"✗ Upload not completed yet: {filename} (status: {upload['status']})")
return False
results = upload.get('results', {})
if not results or 'creatives' not in results or not results['creatives']:
print(f"✗ No results available for: {filename}")
return False
creative = results['creatives'][0]
scorecard_url = creative.get('scorecard_url')
if not scorecard_url:
print(f"✗ No scorecard URL found for: {filename}")
return False
return self.download_pdf_from_scorecard_url(scorecard_url, filename)
def download_for_request_id(self, request_id: int) -> bool:
"""
Download report for specific request ID
Args:
request_id: Creative X request ID
Returns:
bool: Success
"""
# Find upload with this request ID
all_uploads = self.state_manager.get_all_uploads()
for upload in all_uploads:
if upload.get('request_id') == request_id:
return self.download_for_upload(upload['filename'])
print(f"✗ No upload found with request_id: {request_id}")
return False
def download_all_completed(self) -> dict:
"""
Download reports for all completed uploads
Returns:
dict: Summary with counts
"""
completed = self.state_manager.get_uploads_by_status(StateManager.STATUS_COMPLETED)
summary = {
'total': len(completed),
'succeeded': 0,
'failed': 0
}
print(f"\n📥 Downloading {summary['total']} reports...\n")
for upload in completed:
filename = upload['filename']
success = self.download_for_upload(filename)
if success:
summary['succeeded'] += 1
else:
summary['failed'] += 1
# Print summary
print("\n" + "=" * 60)
print(f"Downloaded: {summary['succeeded']} / {summary['total']}")
print(f"Failed: {summary['failed']}")
print(f"Reports saved to: {self.reports_dir}")
print("=" * 60)
return summary
def main():
"""CLI entry point"""
parser = argparse.ArgumentParser(
description='Download PDF reports from Creative X'
)
parser.add_argument('--all', action='store_true',
help='Download all completed reports')
parser.add_argument('--file', help='Download report for specific file')
parser.add_argument('--request-id', type=int,
help='Download report for specific request ID')
args = parser.parse_args()
if not any([args.all, args.file, args.request_id]):
parser.print_help()
sys.exit(1)
# Load configuration
try:
config = load_config()
except Exception as e:
print(f"Error loading configuration: {e}")
sys.exit(1)
# Initialize downloader
downloader = ReportDownloader(config)
# Download based on args
if args.all:
downloader.download_all_completed()
elif args.file:
success = downloader.download_for_upload(args.file)
sys.exit(0 if success else 1)
elif args.request_id:
success = downloader.download_for_request_id(args.request_id)
sys.exit(0 if success else 1)
if __name__ == '__main__':
main()