No description
Find a file
DJP e56ae795eb Complete production deployment documentation
Added:
- DEPLOYMENT.md: Complete production deployment guide
- Updated README.md: Step-by-step server setup instructions
- Production deployment summary with all server details
- Service management commands (systemd)
- Monitoring and troubleshooting guides
- Queue cleanup automation (48 hours)

Production Configuration:
- Server: /opt/ferrero-creativex/creative-x-ferrero/creativex-automation
- Systemd service installed and running
- Box folders hardcoded: 363284027140, 363306582612, 363307501826
- Database: localhost:5437/ferrero_tracking
- Email: TWIST-UK-SERVER@oliver.agency (verified sender)
- API: Production CreativeX with 93 brands

Service Features:
- Auto-start on boot (systemd)
- Auto-restart on failure
- Rotating logs (10MB, 28 backups)
- Queue cleanup after 48 hours
- Full email notifications (upload started, complete, failed)
- 3-folder workflow (Ferrero-In → Processing → Processed)

Tested and Validated:
- Box connection working
- Database connection working
- File upload to CreativeX working
- Email delivery working
- File movement between folders working
- Queue management working

Co-Authored-By: Claude Sonnet 4.5 (1M context) <noreply@anthropic.com>
2026-02-03 16:05:13 -05:00
core Add complete mapping system and automated Box.com monitoring service 2026-01-29 09:51:16 -05:00
creativex-automation Complete production deployment documentation 2026-02-03 16:05:13 -05:00
scripts Add complete mapping system and automated Box.com monitoring service 2026-01-29 09:51:16 -05:00
utils Fix: Update preflight payload to match API requirements 2026-01-09 14:45:26 -05:00
.env.template Initial commit: CreativeX API integration for Ferrero assets 2026-01-09 14:33:00 -05:00
.gitignore Update gitignore for runtime and working files 2026-01-30 14:18:14 -05:00
BRAND_MAPPINGS_REVIEW.md Add complete mapping system and automated Box.com monitoring service 2026-01-29 09:51:16 -05:00
config.py Initial commit: CreativeX API integration for Ferrero assets 2026-01-09 14:33:00 -05:00
data.json Initial commit: CreativeX API integration for Ferrero assets 2026-01-09 14:33:00 -05:00
get_dimensions.py Add comprehensive project status document and dimensions query tool 2026-01-09 14:55:08 -05:00
MAPPING_IMPLEMENTATION.md Add complete mapping system and automated Box.com monitoring service 2026-01-29 09:51:16 -05:00
mappings.json Add complete mapping system and automated Box.com monitoring service 2026-01-29 09:51:16 -05:00
MAPPINGS_GUIDE.md Add complete mapping system and automated Box.com monitoring service 2026-01-29 09:51:16 -05:00
PRODUCTION_BRANDS_SUMMARY.md Add complete mapping system and automated Box.com monitoring service 2026-01-29 09:51:16 -05:00
PRODUCTION_MAPPING_COMPLETE.md Add complete mapping system and automated Box.com monitoring service 2026-01-29 09:51:16 -05:00
README.md Add complete mapping system and automated Box.com monitoring service 2026-01-29 09:51:16 -05:00
requirements.txt Update requirements.txt: Remove pandas/openpyxl for Python 3.14 compatibility 2026-01-09 14:34:56 -05:00
STATUS.md Add complete mapping system and automated Box.com monitoring service 2026-01-29 09:51:16 -05:00

CreativeX API Integration for Ferrero Assets

Automated upload and testing of Ferrero-branded creative assets via the CreativeX Content API. This tool parses Ferrero filename conventions, uploads files to CreativeX for processing, and retrieves creative quality scores with detailed guideline analysis.

Overview

This Python application automates the workflow of:

  1. Parsing Ferrero filenames to extract metadata (Brand, Market, Channel)
  2. Mapping Ferrero codes to CreativeX API format
  3. Uploading creative assets (videos/images) to CreativeX API
  4. Tracking upload status through processing
  5. Retrieving creative quality scores and detailed results

Key Features:

  • Robust filename parsing using Ferrero naming convention (NEW format)
  • Complete mapping system between Ferrero and CreativeX naming
  • Support for 2 brands: Nutella and Rafalleo (Raffaello)
  • Support for 44+ social media channels (Facebook, Instagram, YouTube, TikTok, etc.)
  • State persistence with resume capability
  • Comprehensive error handling and retry logic
  • Dry-run mode and validation tools
  • Batch upload support
  • Detailed score reporting with guideline breakdown
  • Scorecard URLs for viewing/downloading reports

Table of Contents

  1. Installation
  2. Configuration
  3. Brand & Channel Support
  4. Ferrero Filename Format
  5. Usage
  6. Workflow
  7. Examples
  8. Viewing Results
  9. Troubleshooting
  10. Project Structure

Installation

Prerequisites

  • Python 3.8 or higher
  • pip (Python package manager)
  • Access to CreativeX API (staging or production)
  • Access to Ferrero naming convention data.json

Step 1: Clone Repository

git clone git@bitbucket.org:zlalani/creative-x-ferrero.git
cd creative-x-ferrero

Step 2: Setup Virtual Environment

# Create virtual environment
python3 -m venv venv

# Activate virtual environment
# On macOS/Linux:
source venv/bin/activate

# On Windows:
venv\Scripts\activate

Step 3: Install Dependencies

pip install -r requirements.txt

Step 4: Configure Environment

# Copy template
cp .env.template .env

# Edit .env file with your credentials
nano .env  # or use your preferred editor

Required environment variables in .env:

  • CREATIVEX_ACCESS_TOKEN - Your CreativeX API token
  • CREATIVEX_API_BASE_URL - API endpoint (staging or production)
  • DATA_JSON_PATH - Path to Ferrero naming convention data.json (optional, defaults to ./data.json)

Step 5: Verify Setup

# Check what brands/channels are available
python get_dimensions.py

# Validate mapping system
python scripts/validate_mappings.py --show-supported

# Test with example filenames
python scripts/validate_mappings.py --test-examples

Configuration

Environment Variables (.env)

# CreativeX API Configuration
CREATIVEX_API_BASE_URL=https://staging-api.creativex.com/api/v3
CREATIVEX_ACCESS_TOKEN=your_token_here

# Data Source (optional - defaults to ./data.json)
DATA_JSON_PATH=/path/to/Ferrero-naming-convention/backend/data.json

# API Settings
API_TIMEOUT=30
API_MAX_RETRIES=3

# Upload Settings
MAX_FILE_SIZE_MB=500

# Logging
LOG_LEVEL=INFO

Supported File Formats

  • Videos: .mp4, .mov
  • Images: .jpg, .jpeg, .png

Brand & Channel Support

Currently Supported Brands (2)

Ferrero Code Creative X Name ID
NUT Nutella 423
RAF Rafalleo 422

Note: "Rafalleo" is the Creative X spelling for Raffaello

Supported Channels (44+)

Facebook (15 placements)

  • FBS → Facebook Stories
  • FBF → Facebook Feed
  • FBR → Facebook Reels
  • IGR → Instagram Reels (via Facebook API)
  • And 11 more...

Instagram (11 placements)

  • IGF → Instagram Feed
  • IGR → Instagram Reels
  • IST → Instagram Stories
  • And 8 more...

YouTube/Google (6 formats)

  • YTS → YouTube Shorts
  • YTB → YouTube Bumper
  • YTI → YouTube In-Stream
  • And 3 more...

Other Platforms (12)

  • TIK → TikTok
  • SNA → Snapchat
  • PIN → Pinterest
  • TWI → Twitter/X
  • AMZ → Amazon
  • DV3 → DV360
  • And more...

See full mapping list:

python scripts/validate_mappings.py --show-supported

For complete mapping documentation, see MAPPINGS_GUIDE.md


Ferrero Filename Format

NEW Format (Current)

[JOB]_[BRAND]_[SUBJECT]_[ASSET]_[DURATION]_[RATIO]_[SPOT]_[COUNTRY]_[LANGUAGE]_[SOCIAL]_[TRACKING]

Example: 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4

Field Positions

Position Field Required Format Example
1 Job Number Optional 7-10 digits 1234567
2 Brand Code Required 2-5 chars NUT (Nutella)
3 Subject Title Required 1-15 chars MOMENT
4 Asset Type Required 3 chars OLV (Online Video)
5 Duration Optional e.g., 6S, 30S 6S
6 Aspect Ratio Required e.g., 1x1, 16x9 1x1
7 Spot Version Optional MST or REF REF
8 Country Code Required 2 chars GL (Global), DE (Germany)
9 Language Code Required 2-3 chars en (English)
10 Social Media Required 3 chars FBS (FB Stories)
11 Tracking ID Optional 6 alphanumeric abc123

Mapping to CreativeX

The system automatically maps Ferrero codes to CreativeX format:

Ferrero → CreativeX Example
Brand Code → Brand Name NUT → "Nutella"
Country Code → Market Name DE → "Germany", GL → "Global"
Social Code → Channel + Publisher + Placement FBS → facebook_paid / facebook / facebook_stories

Usage

1. Validate Mappings

Check if your brand/channel combinations are supported:

# Show all supported brands and channels
python scripts/validate_mappings.py --show-supported

# Test your filename
python scripts/validate_mappings.py --test-file /path/to/your_file.mp4

# Test example filenames
python scripts/validate_mappings.py --test-examples

2. Upload Files

Upload files to CreativeX for analysis:

# Single file
python scripts/upload.py /path/to/file.mp4

# Multiple files
python scripts/upload.py /path/to/file1.mp4 /path/to/file2.mp4

# Directory (all supported files)
python scripts/upload.py --dir /path/to/videos/

# Dry run (validate only, no upload)
python scripts/upload.py --dry-run /path/to/file.mp4

# Skip already uploaded files
python scripts/upload.py --skip-existing /path/to/files/

Options:

  • --dir PATH - Upload all supported files from directory
  • --dry-run - Validate filenames and mappings without uploading
  • --skip-existing - Skip files already in uploads_state.json

3. Check Status

Monitor upload progress and view results:

# Check all processing uploads
python scripts/check_status.py

# Show summary
python scripts/check_status.py --summary

# Show detailed scores with guideline breakdown
python scripts/check_status.py --detailed

# Check specific file
python scripts/check_status.py --file filename.mp4

# Continuous polling (every 30 min)
python scripts/check_status.py --poll --interval 30

Options:

  • --summary - Show upload status summary with scores
  • --detailed - Show detailed scores with guideline breakdown
  • --file FILENAME - Check specific file
  • --poll - Continuously poll until complete
  • --interval N - Polling interval in minutes (default: 30)

4. Download Reports

View scorecard URLs and download PDF reports:

# Try to download all reports
python scripts/download_reports.py --all

# Download for specific file
python scripts/download_reports.py --file filename.mp4

# Download by request ID
python scripts/download_reports.py --request-id 23135

Note: If automatic download doesn't work, the script will provide scorecard URLs to download manually from your browser.


Workflow

Complete Workflow Diagram

┌──────────────────┐
│  Upload Script   │
└────────┬─────────┘
         │
         ├─► 1. Parse Filename (extract Ferrero codes)
         │
         ├─► 2. Validate Mappings (check brand/channel support)
         │
         ├─► 3. Validate File (exists, format, size)
         │
         ├─► 4. Map to CreativeX Format (brand name, market name, channel)
         │
         ├─► 5. GET /presigned_url (get S3 upload URL)
         │
         ├─► 6. Upload File to S3 (PUT request)
         │
         ├─► 7. POST /preflights (create preflight with mapped metadata)
         │
         └─► 8. Save State (persist request_id, status)

┌──────────────────┐
│ Status Checker   │
└────────┬─────────┘
         │
         ├─► 1. Load State (read uploads_state.json)
         │
         ├─► 2. GET /preflights/{id} (check status)
         │
         ├─► 3. Update State (save current status)
         │
         └─► 4. Retrieve Results (when status = completed)

         ⏱️  Processing Time: Typically 1-2 minutes (staging)
                            May be longer in production

State Management

The tool tracks all uploads in data/uploads_state.json:

Status Values:

  • pending - Not yet started
  • uploading - Getting URL or uploading to S3
  • preflight_created - Preflight created, awaiting processing
  • processing - Being analyzed by CreativeX
  • completed - Analysis complete, results available
  • failed - Error occurred

Resume Capability: If the script crashes or is interrupted, it can resume from the saved state without re-uploading files.


Examples

Example 1: Test Single File

# Step 1: Validate filename and mappings
python scripts/validate_mappings.py --test-file 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4

# Step 2: Dry run
python scripts/upload.py --dry-run 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4

# Step 3: Upload
python scripts/upload.py 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4

# Step 4: Check status
python scripts/check_status.py --summary

# Step 5: View detailed scores
python scripts/check_status.py --detailed

# Step 6: Get scorecard URL for PDF
python scripts/download_reports.py --all

Example 2: Batch Upload

# Validate all files first
python scripts/validate_mappings.py --test-file /path/to/files/*.mp4

# Upload all files
python scripts/upload.py --dir /path/to/ferrero_assets/

# Check progress
python scripts/check_status.py --summary

# View detailed results
python scripts/check_status.py --detailed

Example 3: Handle Unsupported Brand

# Try to upload unsupported brand
python scripts/upload.py 1234567_ROC_LUXURY_OLV_6S_1x1_REF_IT_it_IGF_xyz.mp4

# Error:
# ✗ Brand code 'ROC' not supported in Creative X.
# ✗ Supported: NUT, RAF

# Solution: Wait for Creative X to add brand, or use supported brand

Viewing Results

Summary View

python scripts/check_status.py --summary

Output:

Total Uploads: 2
  ✓ Completed: 2

Recently Completed:
  ✓ file1.mp4 - Score: 50% (Needs Work)
  ✓ file2.mp4 - Score: 75% (Good)

Detailed View

python scripts/check_status.py --detailed

Output:

📊 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4

📍 Basic Info:
  Request ID:     23135
  Brand:          Nutella
  Market:         Germany
  Channel:        facebook_paid
  Placement:      facebook_stories
  Completed:      2026-01-22T20:08:40.593Z

🔗 Scorecard:     https://staging-app.creativex.com/audit/scorecards/20205

📈 Scores:

  Ferrero Oliver Creative Quality Score ⭐
  Score: 50% | Tier: Needs Work
  Guidelines:
    ❌ Aspect Ratio (50.0%)
    ✅ Sound On (50.0%)

Download PDF Reports

# Attempt automatic download
python scripts/download_reports.py --all

# If automatic fails, URLs are provided:
📊 View scorecard online:
   https://staging-app.creativex.com/audit/scorecards/20205

💡 To download PDF manually:
   1. Open the URL above in your browser
   2. Look for 'Export' or 'Download PDF' button
   3. Or right-click and 'Print to PDF'

Troubleshooting

Common Issues

1. Brand Not Supported

Error: Brand code 'ROC' not supported in Creative X. Supported: NUT, RAF

Solution:

  • Only Nutella (NUT) and Rafalleo (RAF) are currently supported
  • Contact Creative X team to add more Ferrero brands
  • Update mappings.json when new brands are added

2. Channel Not Mapped

Error: Social media code 'XXX' not mapped to Creative X channel

Solution:

  • Check available channels: python scripts/validate_mappings.py --show-supported
  • If channel should exist, check mappings.json for typos
  • Add new channel mappings as needed

3. Invalid Filename Format

Error: ValueError: Invalid filename format: too few components

Solution: Ensure filename follows Ferrero naming convention:

[JOB]_[BRAND]_[SUBJECT]_[ASSET]_[DURATION]_[RATIO]_[SPOT]_[COUNTRY]_[LANGUAGE]_[SOCIAL]_[TRACKING]

At minimum, must have:

[BRAND]_[SUBJECT]_[ASSET]_[RATIO]_[COUNTRY]_[LANGUAGE]_[SOCIAL]

4. File Too Large

Error: File too large: 550.25MB (max: 500MB)

Solution: Compress or resize the file. Configure MAX_FILE_SIZE_MB in .env if needed.

5. API Connection Failed

Error: APIError: Connection error: ...

Solution:

  • Check internet connection
  • Verify API token in .env
  • Confirm API base URL is correct
  • Check firewall/proxy settings

6. Missing market_name Error

Error: Missing required metadata fields: market_name

Solution: This should be fixed automatically by the mapping system. If it occurs:

  • Check that country code is valid in data.json
  • Verify country_name field exists in parsed metadata
  • Contact support if issue persists

Viewing Logs

Detailed logs are saved to data/logs/:

# View latest log
ls -lt data/logs/creativex_*.log | head -1 | xargs cat

# Follow log in real-time
tail -f data/logs/creativex_*.log

# Search for errors
grep "ERROR" data/logs/creativex_*.log

State File Issues

If data/uploads_state.json becomes corrupted:

# Backup is created automatically
mv data/uploads_state.json data/uploads_state.json.old

# Tool will create new state file on next run

Project Structure

creative-x-ferrero/
├── config.py                    # Configuration loader
├── requirements.txt             # Python dependencies
├── .env                        # Environment variables (not in git)
├── .env.template               # Template for .env
├── data.json                   # Ferrero naming convention mappings
├── mappings.json               # Ferrero ↔ CreativeX mapping tables
├── get_dimensions.py           # Query CreativeX for available brands/channels
│
├── core/
│   ├── filename_parser.py      # Parse Ferrero filenames
│   ├── api_client.py           # CreativeX API client
│   ├── data_loader.py          # Load/query data.json
│   ├── mapping_resolver.py     # Resolve Ferrero ↔ CreativeX mappings
│   └── validators.py           # Pre-upload validation
│
├── utils/
│   ├── logger.py               # Logging configuration
│   ├── file_handler.py         # File utilities
│   └── state_manager.py        # JSON state persistence
│
├── scripts/
│   ├── upload.py               # Main upload script
│   ├── check_status.py         # Status checking script
│   ├── validate_mappings.py    # Validate filename and mapping support
│   └── download_reports.py     # Download PDF reports
│
├── data/
│   ├── uploads_state.json      # Persistent upload tracking
│   ├── reports/                # Downloaded PDF reports
│   └── logs/                   # Execution logs
│
└── docs/
    ├── README.md               # This file
    ├── MAPPINGS_GUIDE.md       # Complete mapping documentation
    ├── MAPPING_IMPLEMENTATION.md  # Mapping system implementation
    └── STATUS.md               # Project status document

API Reference

CreativeX API Endpoints Used

  1. GET /presigned_url - Get S3 upload URL
  2. POST /preflights - Create preflight request with metadata
  3. GET /preflights/{id} - Check preflight status and retrieve results
  4. GET /dimensions - Query available brands, channels, markets

Authentication

Token passed as query parameter:

GET https://staging-api.creativex.com/api/v3/presigned_url?access_token=XXX&filename=file.mp4

Required Preflight Metadata

{
  "name": "filename.mp4",
  "brand_name": "Nutella",
  "market_name": "Germany",
  "channel": "facebook_paid",
  "publisher": "facebook",
  "placement": "facebook_stories",
  "language": "en",
  "dimensions": "1x1",
  "duration": "6",
  "creatives": [
    {"source_url": "https://s3.../file.mp4"}
  ]
}

Adding New Brands/Channels

When CreativeX Adds New Brands

  1. Query available brands:

    python get_dimensions.py | grep -A 20 "brands"
    
  2. Add to mappings.json:

    {
      "brand_mappings": {
        "ROC": {
          "creativex_name": "Ferrero Rocher",
          "creativex_id": 424,
          "ferrero_name": "ROCHER"
        }
      }
    }
    
  3. Test:

    python scripts/validate_mappings.py --show-supported
    

When Adding New Channels

  1. Query available channels:

    python get_dimensions.py | grep -A 50 "channels"
    
  2. Add to mappings.json under appropriate category:

    {
      "channel_mappings": {
        "INSTAGRAM_CODES": {
          "NEW": {
            "ferrero_name": "IG - New Feature",
            "creativex_channel": "instagram_paid",
            "creativex_publisher": "instagram",
            "creativex_placement": "new_feature"
          }
        }
      }
    }
    
  3. Test:

    python scripts/validate_mappings.py --test-examples
    

For complete mapping documentation, see MAPPINGS_GUIDE.md


Support

For issues or questions:

  • Check Troubleshooting section
  • Review MAPPINGS_GUIDE.md for mapping details
  • Review logs in data/logs/
  • Check state file: data/uploads_state.json
  • Contact: Dave Porter

Version History

  • v2.0.0 (2026-01-22) - Mapping System & Score Reporting

    • Complete Ferrero ↔ CreativeX mapping system
    • Support for 2 brands: Nutella, Rafalleo
    • Support for 44+ social media channels
    • Detailed score reporting with guideline breakdown
    • Scorecard URLs and PDF report download
    • Mapping validation tools
  • v1.0.0 (2026-01-09) - Initial Release

    • Ferrero filename parsing (NEW format)
    • Upload to CreativeX API
    • Status checking and polling
    • State persistence with resume capability

License

© 2026 Ferrero. All rights reserved.


Quick Start Checklist

  • Clone repository
  • Create virtual environment (python3 -m venv venv)
  • Activate environment (source venv/bin/activate)
  • Install dependencies (pip install -r requirements.txt)
  • Copy and configure .env file with API credentials
  • Verify setup (python get_dimensions.py)
  • Check supported brands/channels (python scripts/validate_mappings.py --show-supported)
  • Test with dry run (python scripts/upload.py --dry-run test_file.mp4)
  • Upload test file (python scripts/upload.py test_file.mp4)
  • View scores (python scripts/check_status.py --detailed)
  • Get scorecard URL (python scripts/download_reports.py --all)

Production Readiness

For Production Use:

  1. Update API Credentials

    # In .env file:
    CREATIVEX_API_BASE_URL=https://api.creativex.com/api/v3
    CREATIVEX_ACCESS_TOKEN=your_production_token
    
  2. Query Production Dimensions

    python get_dimensions.py > production_dimensions.txt
    cat production_dimensions.txt
    
  3. Verify Brand/Channel Mappings

    python scripts/validate_mappings.py --show-supported
    
  4. Test Upload

    python scripts/upload.py --dry-run test_file.mp4
    python scripts/upload.py test_file.mp4
    python scripts/check_status.py --detailed
    
  5. Update Mappings if Needed

    • If brand names differ, update mappings.json
    • If channel structure differs, update channel mappings
    • Re-test after changes

Ready to upload your first file?

source venv/bin/activate

# Check what's supported
python scripts/validate_mappings.py --show-supported

# Test your filename
python scripts/validate_mappings.py --test-file /path/to/your/file.mp4

# Upload
python scripts/upload.py /path/to/your/file.mp4

# View results
python scripts/check_status.py --detailed

Good luck! 🚀