Added: - DEPLOYMENT.md: Complete production deployment guide - Updated README.md: Step-by-step server setup instructions - Production deployment summary with all server details - Service management commands (systemd) - Monitoring and troubleshooting guides - Queue cleanup automation (48 hours) Production Configuration: - Server: /opt/ferrero-creativex/creative-x-ferrero/creativex-automation - Systemd service installed and running - Box folders hardcoded: 363284027140, 363306582612, 363307501826 - Database: localhost:5437/ferrero_tracking - Email: TWIST-UK-SERVER@oliver.agency (verified sender) - API: Production CreativeX with 93 brands Service Features: - Auto-start on boot (systemd) - Auto-restart on failure - Rotating logs (10MB, 28 backups) - Queue cleanup after 48 hours - Full email notifications (upload started, complete, failed) - 3-folder workflow (Ferrero-In → Processing → Processed) Tested and Validated: - Box connection working - Database connection working - File upload to CreativeX working - Email delivery working - File movement between folders working - Queue management working Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com> |
||
|---|---|---|
| core | ||
| creativex-automation | ||
| scripts | ||
| utils | ||
| .env.template | ||
| .gitignore | ||
| BRAND_MAPPINGS_REVIEW.md | ||
| config.py | ||
| data.json | ||
| get_dimensions.py | ||
| MAPPING_IMPLEMENTATION.md | ||
| mappings.json | ||
| MAPPINGS_GUIDE.md | ||
| PRODUCTION_BRANDS_SUMMARY.md | ||
| PRODUCTION_MAPPING_COMPLETE.md | ||
| README.md | ||
| requirements.txt | ||
| STATUS.md | ||
CreativeX API Integration for Ferrero Assets
Automated upload and testing of Ferrero-branded creative assets via the CreativeX Content API. This tool parses Ferrero filename conventions, uploads files to CreativeX for processing, and retrieves creative quality scores with detailed guideline analysis.
Overview
This Python application automates the workflow of:
- Parsing Ferrero filenames to extract metadata (Brand, Market, Channel)
- Mapping Ferrero codes to CreativeX API format
- Uploading creative assets (videos/images) to CreativeX API
- Tracking upload status through processing
- Retrieving creative quality scores and detailed results
Key Features:
- ✅ Robust filename parsing using Ferrero naming convention (NEW format)
- ✅ Complete mapping system between Ferrero and CreativeX naming
- ✅ Support for 2 brands: Nutella and Rafalleo (Raffaello)
- ✅ Support for 44+ social media channels (Facebook, Instagram, YouTube, TikTok, etc.)
- ✅ State persistence with resume capability
- ✅ Comprehensive error handling and retry logic
- ✅ Dry-run mode and validation tools
- ✅ Batch upload support
- ✅ Detailed score reporting with guideline breakdown
- ✅ Scorecard URLs for viewing/downloading reports
Table of Contents
- Installation
- Configuration
- Brand & Channel Support
- Ferrero Filename Format
- Usage
- Workflow
- Examples
- Viewing Results
- Troubleshooting
- Project Structure
Installation
Prerequisites
- Python 3.8 or higher
- pip (Python package manager)
- Access to CreativeX API (staging or production)
- Access to Ferrero naming convention data.json
Step 1: Clone Repository
git clone git@bitbucket.org:zlalani/creative-x-ferrero.git
cd creative-x-ferrero
Step 2: Setup Virtual Environment
# Create virtual environment
python3 -m venv venv
# Activate virtual environment
# On macOS/Linux:
source venv/bin/activate
# On Windows:
venv\Scripts\activate
Step 3: Install Dependencies
pip install -r requirements.txt
Step 4: Configure Environment
# Copy template
cp .env.template .env
# Edit .env file with your credentials
nano .env # or use your preferred editor
Required environment variables in .env:
CREATIVEX_ACCESS_TOKEN- Your CreativeX API tokenCREATIVEX_API_BASE_URL- API endpoint (staging or production)DATA_JSON_PATH- Path to Ferrero naming convention data.json (optional, defaults to ./data.json)
Step 5: Verify Setup
# Check what brands/channels are available
python get_dimensions.py
# Validate mapping system
python scripts/validate_mappings.py --show-supported
# Test with example filenames
python scripts/validate_mappings.py --test-examples
Configuration
Environment Variables (.env)
# CreativeX API Configuration
CREATIVEX_API_BASE_URL=https://staging-api.creativex.com/api/v3
CREATIVEX_ACCESS_TOKEN=your_token_here
# Data Source (optional - defaults to ./data.json)
DATA_JSON_PATH=/path/to/Ferrero-naming-convention/backend/data.json
# API Settings
API_TIMEOUT=30
API_MAX_RETRIES=3
# Upload Settings
MAX_FILE_SIZE_MB=500
# Logging
LOG_LEVEL=INFO
Supported File Formats
- Videos:
.mp4,.mov - Images:
.jpg,.jpeg,.png
Brand & Channel Support
Currently Supported Brands (2)
| Ferrero Code | Creative X Name | ID |
|---|---|---|
| NUT | Nutella | 423 |
| RAF | Rafalleo | 422 |
Note: "Rafalleo" is the Creative X spelling for Raffaello
Supported Channels (44+)
Facebook (15 placements)
- FBS → Facebook Stories
- FBF → Facebook Feed
- FBR → Facebook Reels
- IGR → Instagram Reels (via Facebook API)
- And 11 more...
Instagram (11 placements)
- IGF → Instagram Feed
- IGR → Instagram Reels
- IST → Instagram Stories
- And 8 more...
YouTube/Google (6 formats)
- YTS → YouTube Shorts
- YTB → YouTube Bumper
- YTI → YouTube In-Stream
- And 3 more...
Other Platforms (12)
- TIK → TikTok
- SNA → Snapchat
- PIN → Pinterest
- TWI → Twitter/X
- AMZ → Amazon
- DV3 → DV360
- And more...
See full mapping list:
python scripts/validate_mappings.py --show-supported
For complete mapping documentation, see MAPPINGS_GUIDE.md
Ferrero Filename Format
NEW Format (Current)
[JOB]_[BRAND]_[SUBJECT]_[ASSET]_[DURATION]_[RATIO]_[SPOT]_[COUNTRY]_[LANGUAGE]_[SOCIAL]_[TRACKING]
Example: 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4
Field Positions
| Position | Field | Required | Format | Example |
|---|---|---|---|---|
| 1 | Job Number | Optional | 7-10 digits | 1234567 |
| 2 | Brand Code | Required | 2-5 chars | NUT (Nutella) |
| 3 | Subject Title | Required | 1-15 chars | MOMENT |
| 4 | Asset Type | Required | 3 chars | OLV (Online Video) |
| 5 | Duration | Optional | e.g., 6S, 30S | 6S |
| 6 | Aspect Ratio | Required | e.g., 1x1, 16x9 | 1x1 |
| 7 | Spot Version | Optional | MST or REF | REF |
| 8 | Country Code | Required | 2 chars | GL (Global), DE (Germany) |
| 9 | Language Code | Required | 2-3 chars | en (English) |
| 10 | Social Media | Required | 3 chars | FBS (FB Stories) |
| 11 | Tracking ID | Optional | 6 alphanumeric | abc123 |
Mapping to CreativeX
The system automatically maps Ferrero codes to CreativeX format:
| Ferrero → CreativeX | Example |
|---|---|
| Brand Code → Brand Name | NUT → "Nutella" |
| Country Code → Market Name | DE → "Germany", GL → "Global" |
| Social Code → Channel + Publisher + Placement | FBS → facebook_paid / facebook / facebook_stories |
Usage
1. Validate Mappings
Check if your brand/channel combinations are supported:
# Show all supported brands and channels
python scripts/validate_mappings.py --show-supported
# Test your filename
python scripts/validate_mappings.py --test-file /path/to/your_file.mp4
# Test example filenames
python scripts/validate_mappings.py --test-examples
2. Upload Files
Upload files to CreativeX for analysis:
# Single file
python scripts/upload.py /path/to/file.mp4
# Multiple files
python scripts/upload.py /path/to/file1.mp4 /path/to/file2.mp4
# Directory (all supported files)
python scripts/upload.py --dir /path/to/videos/
# Dry run (validate only, no upload)
python scripts/upload.py --dry-run /path/to/file.mp4
# Skip already uploaded files
python scripts/upload.py --skip-existing /path/to/files/
Options:
--dir PATH- Upload all supported files from directory--dry-run- Validate filenames and mappings without uploading--skip-existing- Skip files already in uploads_state.json
3. Check Status
Monitor upload progress and view results:
# Check all processing uploads
python scripts/check_status.py
# Show summary
python scripts/check_status.py --summary
# Show detailed scores with guideline breakdown
python scripts/check_status.py --detailed
# Check specific file
python scripts/check_status.py --file filename.mp4
# Continuous polling (every 30 min)
python scripts/check_status.py --poll --interval 30
Options:
--summary- Show upload status summary with scores--detailed- Show detailed scores with guideline breakdown--file FILENAME- Check specific file--poll- Continuously poll until complete--interval N- Polling interval in minutes (default: 30)
4. Download Reports
View scorecard URLs and download PDF reports:
# Try to download all reports
python scripts/download_reports.py --all
# Download for specific file
python scripts/download_reports.py --file filename.mp4
# Download by request ID
python scripts/download_reports.py --request-id 23135
Note: If automatic download doesn't work, the script will provide scorecard URLs to download manually from your browser.
Workflow
Complete Workflow Diagram
┌──────────────────┐
│ Upload Script │
└────────┬─────────┘
│
├─► 1. Parse Filename (extract Ferrero codes)
│
├─► 2. Validate Mappings (check brand/channel support)
│
├─► 3. Validate File (exists, format, size)
│
├─► 4. Map to CreativeX Format (brand name, market name, channel)
│
├─► 5. GET /presigned_url (get S3 upload URL)
│
├─► 6. Upload File to S3 (PUT request)
│
├─► 7. POST /preflights (create preflight with mapped metadata)
│
└─► 8. Save State (persist request_id, status)
┌──────────────────┐
│ Status Checker │
└────────┬─────────┘
│
├─► 1. Load State (read uploads_state.json)
│
├─► 2. GET /preflights/{id} (check status)
│
├─► 3. Update State (save current status)
│
└─► 4. Retrieve Results (when status = completed)
⏱️ Processing Time: Typically 1-2 minutes (staging)
May be longer in production
State Management
The tool tracks all uploads in data/uploads_state.json:
Status Values:
pending- Not yet starteduploading- Getting URL or uploading to S3preflight_created- Preflight created, awaiting processingprocessing- Being analyzed by CreativeXcompleted- Analysis complete, results availablefailed- Error occurred
Resume Capability: If the script crashes or is interrupted, it can resume from the saved state without re-uploading files.
Examples
Example 1: Test Single File
# Step 1: Validate filename and mappings
python scripts/validate_mappings.py --test-file 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4
# Step 2: Dry run
python scripts/upload.py --dry-run 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4
# Step 3: Upload
python scripts/upload.py 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4
# Step 4: Check status
python scripts/check_status.py --summary
# Step 5: View detailed scores
python scripts/check_status.py --detailed
# Step 6: Get scorecard URL for PDF
python scripts/download_reports.py --all
Example 2: Batch Upload
# Validate all files first
python scripts/validate_mappings.py --test-file /path/to/files/*.mp4
# Upload all files
python scripts/upload.py --dir /path/to/ferrero_assets/
# Check progress
python scripts/check_status.py --summary
# View detailed results
python scripts/check_status.py --detailed
Example 3: Handle Unsupported Brand
# Try to upload unsupported brand
python scripts/upload.py 1234567_ROC_LUXURY_OLV_6S_1x1_REF_IT_it_IGF_xyz.mp4
# Error:
# ✗ Brand code 'ROC' not supported in Creative X.
# ✗ Supported: NUT, RAF
# Solution: Wait for Creative X to add brand, or use supported brand
Viewing Results
Summary View
python scripts/check_status.py --summary
Output:
Total Uploads: 2
✓ Completed: 2
Recently Completed:
✓ file1.mp4 - Score: 50% (Needs Work)
✓ file2.mp4 - Score: 75% (Good)
Detailed View
python scripts/check_status.py --detailed
Output:
📊 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4
📍 Basic Info:
Request ID: 23135
Brand: Nutella
Market: Germany
Channel: facebook_paid
Placement: facebook_stories
Completed: 2026-01-22T20:08:40.593Z
🔗 Scorecard: https://staging-app.creativex.com/audit/scorecards/20205
📈 Scores:
Ferrero Oliver Creative Quality Score ⭐
Score: 50% | Tier: Needs Work
Guidelines:
❌ Aspect Ratio (50.0%)
✅ Sound On (50.0%)
Download PDF Reports
# Attempt automatic download
python scripts/download_reports.py --all
# If automatic fails, URLs are provided:
📊 View scorecard online:
https://staging-app.creativex.com/audit/scorecards/20205
💡 To download PDF manually:
1. Open the URL above in your browser
2. Look for 'Export' or 'Download PDF' button
3. Or right-click and 'Print to PDF'
Troubleshooting
Common Issues
1. Brand Not Supported
Error: Brand code 'ROC' not supported in Creative X. Supported: NUT, RAF
Solution:
- Only Nutella (NUT) and Rafalleo (RAF) are currently supported
- Contact Creative X team to add more Ferrero brands
- Update
mappings.jsonwhen new brands are added
2. Channel Not Mapped
Error: Social media code 'XXX' not mapped to Creative X channel
Solution:
- Check available channels:
python scripts/validate_mappings.py --show-supported - If channel should exist, check
mappings.jsonfor typos - Add new channel mappings as needed
3. Invalid Filename Format
Error: ValueError: Invalid filename format: too few components
Solution: Ensure filename follows Ferrero naming convention:
[JOB]_[BRAND]_[SUBJECT]_[ASSET]_[DURATION]_[RATIO]_[SPOT]_[COUNTRY]_[LANGUAGE]_[SOCIAL]_[TRACKING]
At minimum, must have:
[BRAND]_[SUBJECT]_[ASSET]_[RATIO]_[COUNTRY]_[LANGUAGE]_[SOCIAL]
4. File Too Large
Error: File too large: 550.25MB (max: 500MB)
Solution: Compress or resize the file. Configure MAX_FILE_SIZE_MB in .env if needed.
5. API Connection Failed
Error: APIError: Connection error: ...
Solution:
- Check internet connection
- Verify API token in .env
- Confirm API base URL is correct
- Check firewall/proxy settings
6. Missing market_name Error
Error: Missing required metadata fields: market_name
Solution: This should be fixed automatically by the mapping system. If it occurs:
- Check that country code is valid in data.json
- Verify
country_namefield exists in parsed metadata - Contact support if issue persists
Viewing Logs
Detailed logs are saved to data/logs/:
# View latest log
ls -lt data/logs/creativex_*.log | head -1 | xargs cat
# Follow log in real-time
tail -f data/logs/creativex_*.log
# Search for errors
grep "ERROR" data/logs/creativex_*.log
State File Issues
If data/uploads_state.json becomes corrupted:
# Backup is created automatically
mv data/uploads_state.json data/uploads_state.json.old
# Tool will create new state file on next run
Project Structure
creative-x-ferrero/
├── config.py # Configuration loader
├── requirements.txt # Python dependencies
├── .env # Environment variables (not in git)
├── .env.template # Template for .env
├── data.json # Ferrero naming convention mappings
├── mappings.json # Ferrero ↔ CreativeX mapping tables
├── get_dimensions.py # Query CreativeX for available brands/channels
│
├── core/
│ ├── filename_parser.py # Parse Ferrero filenames
│ ├── api_client.py # CreativeX API client
│ ├── data_loader.py # Load/query data.json
│ ├── mapping_resolver.py # Resolve Ferrero ↔ CreativeX mappings
│ └── validators.py # Pre-upload validation
│
├── utils/
│ ├── logger.py # Logging configuration
│ ├── file_handler.py # File utilities
│ └── state_manager.py # JSON state persistence
│
├── scripts/
│ ├── upload.py # Main upload script
│ ├── check_status.py # Status checking script
│ ├── validate_mappings.py # Validate filename and mapping support
│ └── download_reports.py # Download PDF reports
│
├── data/
│ ├── uploads_state.json # Persistent upload tracking
│ ├── reports/ # Downloaded PDF reports
│ └── logs/ # Execution logs
│
└── docs/
├── README.md # This file
├── MAPPINGS_GUIDE.md # Complete mapping documentation
├── MAPPING_IMPLEMENTATION.md # Mapping system implementation
└── STATUS.md # Project status document
API Reference
CreativeX API Endpoints Used
- GET /presigned_url - Get S3 upload URL
- POST /preflights - Create preflight request with metadata
- GET /preflights/{id} - Check preflight status and retrieve results
- GET /dimensions - Query available brands, channels, markets
Authentication
Token passed as query parameter:
GET https://staging-api.creativex.com/api/v3/presigned_url?access_token=XXX&filename=file.mp4
Required Preflight Metadata
{
"name": "filename.mp4",
"brand_name": "Nutella",
"market_name": "Germany",
"channel": "facebook_paid",
"publisher": "facebook",
"placement": "facebook_stories",
"language": "en",
"dimensions": "1x1",
"duration": "6",
"creatives": [
{"source_url": "https://s3.../file.mp4"}
]
}
Adding New Brands/Channels
When CreativeX Adds New Brands
-
Query available brands:
python get_dimensions.py | grep -A 20 "brands" -
Add to
mappings.json:{ "brand_mappings": { "ROC": { "creativex_name": "Ferrero Rocher", "creativex_id": 424, "ferrero_name": "ROCHER" } } } -
Test:
python scripts/validate_mappings.py --show-supported
When Adding New Channels
-
Query available channels:
python get_dimensions.py | grep -A 50 "channels" -
Add to
mappings.jsonunder appropriate category:{ "channel_mappings": { "INSTAGRAM_CODES": { "NEW": { "ferrero_name": "IG - New Feature", "creativex_channel": "instagram_paid", "creativex_publisher": "instagram", "creativex_placement": "new_feature" } } } } -
Test:
python scripts/validate_mappings.py --test-examples
For complete mapping documentation, see MAPPINGS_GUIDE.md
Support
For issues or questions:
- Check Troubleshooting section
- Review MAPPINGS_GUIDE.md for mapping details
- Review logs in
data/logs/ - Check state file:
data/uploads_state.json - Contact: Dave Porter
Version History
-
v2.0.0 (2026-01-22) - Mapping System & Score Reporting
- Complete Ferrero ↔ CreativeX mapping system
- Support for 2 brands: Nutella, Rafalleo
- Support for 44+ social media channels
- Detailed score reporting with guideline breakdown
- Scorecard URLs and PDF report download
- Mapping validation tools
-
v1.0.0 (2026-01-09) - Initial Release
- Ferrero filename parsing (NEW format)
- Upload to CreativeX API
- Status checking and polling
- State persistence with resume capability
License
© 2026 Ferrero. All rights reserved.
Quick Start Checklist
- Clone repository
- Create virtual environment (
python3 -m venv venv) - Activate environment (
source venv/bin/activate) - Install dependencies (
pip install -r requirements.txt) - Copy and configure
.envfile with API credentials - Verify setup (
python get_dimensions.py) - Check supported brands/channels (
python scripts/validate_mappings.py --show-supported) - Test with dry run (
python scripts/upload.py --dry-run test_file.mp4) - Upload test file (
python scripts/upload.py test_file.mp4) - View scores (
python scripts/check_status.py --detailed) - Get scorecard URL (
python scripts/download_reports.py --all)
Production Readiness
For Production Use:
-
Update API Credentials
# In .env file: CREATIVEX_API_BASE_URL=https://api.creativex.com/api/v3 CREATIVEX_ACCESS_TOKEN=your_production_token -
Query Production Dimensions
python get_dimensions.py > production_dimensions.txt cat production_dimensions.txt -
Verify Brand/Channel Mappings
python scripts/validate_mappings.py --show-supported -
Test Upload
python scripts/upload.py --dry-run test_file.mp4 python scripts/upload.py test_file.mp4 python scripts/check_status.py --detailed -
Update Mappings if Needed
- If brand names differ, update
mappings.json - If channel structure differs, update channel mappings
- Re-test after changes
- If brand names differ, update
Ready to upload your first file?
source venv/bin/activate
# Check what's supported
python scripts/validate_mappings.py --show-supported
# Test your filename
python scripts/validate_mappings.py --test-file /path/to/your/file.mp4
# Upload
python scripts/upload.py /path/to/your/file.mp4
# View results
python scripts/check_status.py --detailed
Good luck! 🚀