# CreativeX API Integration for Ferrero Assets

Automated upload and testing of Ferrero-branded creative assets via the CreativeX Content API. This tool parses Ferrero filename conventions, uploads files to CreativeX for processing, and retrieves creative quality scores with detailed guideline analysis.

## Overview

This Python application automates the workflow of:
1. **Parsing** Ferrero filenames to extract metadata (Brand, Market, Channel)
2. **Mapping** Ferrero codes to CreativeX API format
3. **Uploading** creative assets (videos/images) to CreativeX API
4. **Tracking** upload status through processing
5. **Retrieving** creative quality scores and detailed results

**Key Features:**
- ✅ Robust filename parsing using Ferrero naming convention (NEW format)
- ✅ Complete mapping system between Ferrero and CreativeX naming
- ✅ Support for 2 brands: **Nutella** and **Rafalleo** (Raffaello)
- ✅ Support for 44+ social media channels (Facebook, Instagram, YouTube, TikTok, etc.)
- ✅ State persistence with resume capability
- ✅ Comprehensive error handling and retry logic
- ✅ Dry-run mode and validation tools
- ✅ Batch upload support
- ✅ Detailed score reporting with guideline breakdown
- ✅ Scorecard URLs for viewing/downloading reports

---

## Table of Contents

1. [Installation](#installation)
2. [Configuration](#configuration)
3. [Brand & Channel Support](#brand--channel-support)
4. [Ferrero Filename Format](#ferrero-filename-format)
5. [Usage](#usage)
6. [Workflow](#workflow)
7. [Examples](#examples)
8. [Viewing Results](#viewing-results)
9. [Troubleshooting](#troubleshooting)
10. [Project Structure](#project-structure)

---

## Installation

### Prerequisites

- Python 3.8 or higher
- pip (Python package manager)
- Access to CreativeX API (staging or production)
- Access to Ferrero naming convention data.json

### Step 1: Clone Repository

```bash
git clone git@bitbucket.org:zlalani/creative-x-ferrero.git
cd creative-x-ferrero
```

### Step 2: Setup Virtual Environment

```bash
# Create virtual environment
python3 -m venv venv

# Activate virtual environment
# On macOS/Linux:
source venv/bin/activate

# On Windows:
venv\Scripts\activate
```

### Step 3: Install Dependencies

```bash
pip install -r requirements.txt
```

### Step 4: Configure Environment

```bash
# Copy template
cp .env.template .env

# Edit .env file with your credentials
nano .env  # or use your preferred editor
```

Required environment variables in `.env`:
- `CREATIVEX_ACCESS_TOKEN` - Your CreativeX API token
- `CREATIVEX_API_BASE_URL` - API endpoint (staging or production)
- `DATA_JSON_PATH` - Path to Ferrero naming convention data.json (optional, defaults to ./data.json)

### Step 5: Verify Setup

```bash
# Check what brands/channels are available
python get_dimensions.py

# Validate mapping system
python scripts/validate_mappings.py --show-supported

# Test with example filenames
python scripts/validate_mappings.py --test-examples
```

---

## Configuration

### Environment Variables (.env)

```env
# CreativeX API Configuration
CREATIVEX_API_BASE_URL=https://staging-api.creativex.com/api/v3
CREATIVEX_ACCESS_TOKEN=your_token_here

# Data Source (optional - defaults to ./data.json)
DATA_JSON_PATH=/path/to/Ferrero-naming-convention/backend/data.json

# API Settings
API_TIMEOUT=30
API_MAX_RETRIES=3

# Upload Settings
MAX_FILE_SIZE_MB=500

# Logging
LOG_LEVEL=INFO
```

### Supported File Formats

- **Videos:** `.mp4`, `.mov`
- **Images:** `.jpg`, `.jpeg`, `.png`

---

## Brand & Channel Support

### Currently Supported Brands (2)

| Ferrero Code | Creative X Name | ID |
|--------------|----------------|-----|
| **NUT** | Nutella | 423 |
| **RAF** | Rafalleo | 422 |

> **Note:** "Rafalleo" is the Creative X spelling for Raffaello

### Supported Channels (44+)

**Facebook (15 placements)**
- FBS → Facebook Stories
- FBF → Facebook Feed
- FBR → Facebook Reels
- IGR → Instagram Reels (via Facebook API)
- And 11 more...

**Instagram (11 placements)**
- IGF → Instagram Feed
- IGR → Instagram Reels
- IST → Instagram Stories
- And 8 more...

**YouTube/Google (6 formats)**
- YTS → YouTube Shorts
- YTB → YouTube Bumper
- YTI → YouTube In-Stream
- And 3 more...

**Other Platforms (12)**
- TIK → TikTok
- SNA → Snapchat
- PIN → Pinterest
- TWI → Twitter/X
- AMZ → Amazon
- DV3 → DV360
- And more...

**See full mapping list:**
```bash
python scripts/validate_mappings.py --show-supported
```

For complete mapping documentation, see [MAPPINGS_GUIDE.md](MAPPINGS_GUIDE.md)

---

## Ferrero Filename Format

### NEW Format (Current)

```
[JOB]_[BRAND]_[SUBJECT]_[ASSET]_[DURATION]_[RATIO]_[SPOT]_[COUNTRY]_[LANGUAGE]_[SOCIAL]_[TRACKING]
```

**Example:** `1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4`

### Field Positions

| Position | Field | Required | Format | Example |
|----------|-------|----------|--------|---------|
| 1 | Job Number | Optional | 7-10 digits | `1234567` |
| 2 | Brand Code | **Required** | 2-5 chars | `NUT` (Nutella) |
| 3 | Subject Title | **Required** | 1-15 chars | `MOMENT` |
| 4 | Asset Type | **Required** | 3 chars | `OLV` (Online Video) |
| 5 | Duration | Optional | e.g., 6S, 30S | `6S` |
| 6 | Aspect Ratio | **Required** | e.g., 1x1, 16x9 | `1x1` |
| 7 | Spot Version | Optional | MST or REF | `REF` |
| 8 | Country Code | **Required** | 2 chars | `GL` (Global), `DE` (Germany) |
| 9 | Language Code | **Required** | 2-3 chars | `en` (English) |
| 10 | Social Media | **Required** | 3 chars | `FBS` (FB Stories) |
| 11 | Tracking ID | Optional | 6 alphanumeric | `abc123` |

### Mapping to CreativeX

The system automatically maps Ferrero codes to CreativeX format:

| Ferrero → CreativeX | Example |
|---------------------|---------|
| Brand Code → Brand Name | NUT → "Nutella" |
| Country Code → Market Name | DE → "Germany", GL → "Global" |
| Social Code → Channel + Publisher + Placement | FBS → facebook_paid / facebook / facebook_stories |

---

## Usage

### 1. Validate Mappings

Check if your brand/channel combinations are supported:

```bash
# Show all supported brands and channels
python scripts/validate_mappings.py --show-supported

# Test your filename
python scripts/validate_mappings.py --test-file /path/to/your_file.mp4

# Test example filenames
python scripts/validate_mappings.py --test-examples
```

### 2. Upload Files

Upload files to CreativeX for analysis:

```bash
# Single file
python scripts/upload.py /path/to/file.mp4

# Multiple files
python scripts/upload.py /path/to/file1.mp4 /path/to/file2.mp4

# Directory (all supported files)
python scripts/upload.py --dir /path/to/videos/

# Dry run (validate only, no upload)
python scripts/upload.py --dry-run /path/to/file.mp4

# Skip already uploaded files
python scripts/upload.py --skip-existing /path/to/files/
```

**Options:**
- `--dir PATH` - Upload all supported files from directory
- `--dry-run` - Validate filenames and mappings without uploading
- `--skip-existing` - Skip files already in uploads_state.json

### 3. Check Status

Monitor upload progress and view results:

```bash
# Check all processing uploads
python scripts/check_status.py

# Show summary
python scripts/check_status.py --summary

# Show detailed scores with guideline breakdown
python scripts/check_status.py --detailed

# Check specific file
python scripts/check_status.py --file filename.mp4

# Continuous polling (every 30 min)
python scripts/check_status.py --poll --interval 30
```

**Options:**
- `--summary` - Show upload status summary with scores
- `--detailed` - Show detailed scores with guideline breakdown
- `--file FILENAME` - Check specific file
- `--poll` - Continuously poll until complete
- `--interval N` - Polling interval in minutes (default: 30)

### 4. Download Reports

View scorecard URLs and download PDF reports:

```bash
# Try to download all reports
python scripts/download_reports.py --all

# Download for specific file
python scripts/download_reports.py --file filename.mp4

# Download by request ID
python scripts/download_reports.py --request-id 23135
```

**Note:** If automatic download doesn't work, the script will provide scorecard URLs to download manually from your browser.

---

## Workflow

### Complete Workflow Diagram

```
┌──────────────────┐
│  Upload Script   │
└────────┬─────────┘
         │
         ├─► 1. Parse Filename (extract Ferrero codes)
         │
         ├─► 2. Validate Mappings (check brand/channel support)
         │
         ├─► 3. Validate File (exists, format, size)
         │
         ├─► 4. Map to CreativeX Format (brand name, market name, channel)
         │
         ├─► 5. GET /presigned_url (get S3 upload URL)
         │
         ├─► 6. Upload File to S3 (PUT request)
         │
         ├─► 7. POST /preflights (create preflight with mapped metadata)
         │
         └─► 8. Save State (persist request_id, status)

┌──────────────────┐
│ Status Checker   │
└────────┬─────────┘
         │
         ├─► 1. Load State (read uploads_state.json)
         │
         ├─► 2. GET /preflights/{id} (check status)
         │
         ├─► 3. Update State (save current status)
         │
         └─► 4. Retrieve Results (when status = completed)

         ⏱️  Processing Time: Typically 1-2 minutes (staging)
                            May be longer in production
```

### State Management

The tool tracks all uploads in `data/uploads_state.json`:

**Status Values:**
- `pending` - Not yet started
- `uploading` - Getting URL or uploading to S3
- `preflight_created` - Preflight created, awaiting processing
- `processing` - Being analyzed by CreativeX
- `completed` - Analysis complete, results available
- `failed` - Error occurred

**Resume Capability:** If the script crashes or is interrupted, it can resume from the saved state without re-uploading files.

---

## Examples

### Example 1: Test Single File

```bash
# Step 1: Validate filename and mappings
python scripts/validate_mappings.py --test-file 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4

# Step 2: Dry run
python scripts/upload.py --dry-run 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4

# Step 3: Upload
python scripts/upload.py 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4

# Step 4: Check status
python scripts/check_status.py --summary

# Step 5: View detailed scores
python scripts/check_status.py --detailed

# Step 6: Get scorecard URL for PDF
python scripts/download_reports.py --all
```

### Example 2: Batch Upload

```bash
# Validate all files first
python scripts/validate_mappings.py --test-file /path/to/files/*.mp4

# Upload all files
python scripts/upload.py --dir /path/to/ferrero_assets/

# Check progress
python scripts/check_status.py --summary

# View detailed results
python scripts/check_status.py --detailed
```

### Example 3: Handle Unsupported Brand

```bash
# Try to upload unsupported brand
python scripts/upload.py 1234567_ROC_LUXURY_OLV_6S_1x1_REF_IT_it_IGF_xyz.mp4

# Error:
# ✗ Brand code 'ROC' not supported in Creative X.
# ✗ Supported: NUT, RAF

# Solution: Wait for Creative X to add brand, or use supported brand
```

---

## Viewing Results

### Summary View

```bash
python scripts/check_status.py --summary
```

**Output:**
```
Total Uploads: 2
  ✓ Completed: 2

Recently Completed:
  ✓ file1.mp4 - Score: 50% (Needs Work)
  ✓ file2.mp4 - Score: 75% (Good)
```

### Detailed View

```bash
python scripts/check_status.py --detailed
```

**Output:**
```
📊 1234567_NUT_MOMENT_OLV_6S_1x1_REF_GL_en_FBS_abc123.mp4

📍 Basic Info:
  Request ID:     23135
  Brand:          Nutella
  Market:         Germany
  Channel:        facebook_paid
  Placement:      facebook_stories
  Completed:      2026-01-22T20:08:40.593Z

🔗 Scorecard:     https://staging-app.creativex.com/audit/scorecards/20205

📈 Scores:

  Ferrero Oliver Creative Quality Score ⭐
  Score: 50% | Tier: Needs Work
  Guidelines:
    ❌ Aspect Ratio (50.0%)
    ✅ Sound On (50.0%)
```

### Download PDF Reports

```bash
# Attempt automatic download
python scripts/download_reports.py --all

# If automatic fails, URLs are provided:
📊 View scorecard online:
   https://staging-app.creativex.com/audit/scorecards/20205

💡 To download PDF manually:
   1. Open the URL above in your browser
   2. Look for 'Export' or 'Download PDF' button
   3. Or right-click and 'Print to PDF'
```

---

## Troubleshooting

### Common Issues

#### 1. Brand Not Supported

**Error:** `Brand code 'ROC' not supported in Creative X. Supported: NUT, RAF`

**Solution:**
- Only Nutella (NUT) and Rafalleo (RAF) are currently supported
- Contact Creative X team to add more Ferrero brands
- Update `mappings.json` when new brands are added

#### 2. Channel Not Mapped

**Error:** `Social media code 'XXX' not mapped to Creative X channel`

**Solution:**
- Check available channels: `python scripts/validate_mappings.py --show-supported`
- If channel should exist, check `mappings.json` for typos
- Add new channel mappings as needed

#### 3. Invalid Filename Format

**Error:** `ValueError: Invalid filename format: too few components`

**Solution:** Ensure filename follows Ferrero naming convention:
```
[JOB]_[BRAND]_[SUBJECT]_[ASSET]_[DURATION]_[RATIO]_[SPOT]_[COUNTRY]_[LANGUAGE]_[SOCIAL]_[TRACKING]
```

At minimum, must have:
```
[BRAND]_[SUBJECT]_[ASSET]_[RATIO]_[COUNTRY]_[LANGUAGE]_[SOCIAL]
```

#### 4. File Too Large

**Error:** `File too large: 550.25MB (max: 500MB)`

**Solution:** Compress or resize the file. Configure `MAX_FILE_SIZE_MB` in .env if needed.

#### 5. API Connection Failed

**Error:** `APIError: Connection error: ...`

**Solution:**
- Check internet connection
- Verify API token in .env
- Confirm API base URL is correct
- Check firewall/proxy settings

#### 6. Missing market_name Error

**Error:** `Missing required metadata fields: market_name`

**Solution:** This should be fixed automatically by the mapping system. If it occurs:
- Check that country code is valid in data.json
- Verify `country_name` field exists in parsed metadata
- Contact support if issue persists

### Viewing Logs

Detailed logs are saved to `data/logs/`:

```bash
# View latest log
ls -lt data/logs/creativex_*.log | head -1 | xargs cat

# Follow log in real-time
tail -f data/logs/creativex_*.log

# Search for errors
grep "ERROR" data/logs/creativex_*.log
```

### State File Issues

If `data/uploads_state.json` becomes corrupted:

```bash
# Backup is created automatically
mv data/uploads_state.json data/uploads_state.json.old

# Tool will create new state file on next run
```

---

## Project Structure

```
creative-x-ferrero/
├── config.py                    # Configuration loader
├── requirements.txt             # Python dependencies
├── .env                        # Environment variables (not in git)
├── .env.template               # Template for .env
├── data.json                   # Ferrero naming convention mappings
├── mappings.json               # Ferrero ↔ CreativeX mapping tables
├── get_dimensions.py           # Query CreativeX for available brands/channels
│
├── core/
│   ├── filename_parser.py      # Parse Ferrero filenames
│   ├── api_client.py           # CreativeX API client
│   ├── data_loader.py          # Load/query data.json
│   ├── mapping_resolver.py     # Resolve Ferrero ↔ CreativeX mappings
│   └── validators.py           # Pre-upload validation
│
├── utils/
│   ├── logger.py               # Logging configuration
│   ├── file_handler.py         # File utilities
│   └── state_manager.py        # JSON state persistence
│
├── scripts/
│   ├── upload.py               # Main upload script
│   ├── check_status.py         # Status checking script
│   ├── validate_mappings.py    # Validate filename and mapping support
│   └── download_reports.py     # Download PDF reports
│
├── data/
│   ├── uploads_state.json      # Persistent upload tracking
│   ├── reports/                # Downloaded PDF reports
│   └── logs/                   # Execution logs
│
└── docs/
    ├── README.md               # This file
    ├── MAPPINGS_GUIDE.md       # Complete mapping documentation
    ├── MAPPING_IMPLEMENTATION.md  # Mapping system implementation
    └── STATUS.md               # Project status document
```

---

## API Reference

### CreativeX API Endpoints Used

1. **GET /presigned_url** - Get S3 upload URL
2. **POST /preflights** - Create preflight request with metadata
3. **GET /preflights/{id}** - Check preflight status and retrieve results
4. **GET /dimensions** - Query available brands, channels, markets

### Authentication

Token passed as query parameter:
```
GET https://staging-api.creativex.com/api/v3/presigned_url?access_token=XXX&filename=file.mp4
```

### Required Preflight Metadata

```json
{
  "name": "filename.mp4",
  "brand_name": "Nutella",
  "market_name": "Germany",
  "channel": "facebook_paid",
  "publisher": "facebook",
  "placement": "facebook_stories",
  "language": "en",
  "dimensions": "1x1",
  "duration": "6",
  "creatives": [
    {"source_url": "https://s3.../file.mp4"}
  ]
}
```

---

## Adding New Brands/Channels

### When CreativeX Adds New Brands

1. Query available brands:
   ```bash
   python get_dimensions.py | grep -A 20 "brands"
   ```

2. Add to `mappings.json`:
   ```json
   {
     "brand_mappings": {
       "ROC": {
         "creativex_name": "Ferrero Rocher",
         "creativex_id": 424,
         "ferrero_name": "ROCHER"
       }
     }
   }
   ```

3. Test:
   ```bash
   python scripts/validate_mappings.py --show-supported
   ```

### When Adding New Channels

1. Query available channels:
   ```bash
   python get_dimensions.py | grep -A 50 "channels"
   ```

2. Add to `mappings.json` under appropriate category:
   ```json
   {
     "channel_mappings": {
       "INSTAGRAM_CODES": {
         "NEW": {
           "ferrero_name": "IG - New Feature",
           "creativex_channel": "instagram_paid",
           "creativex_publisher": "instagram",
           "creativex_placement": "new_feature"
         }
       }
     }
   }
   ```

3. Test:
   ```bash
   python scripts/validate_mappings.py --test-examples
   ```

For complete mapping documentation, see [MAPPINGS_GUIDE.md](MAPPINGS_GUIDE.md)

---

## Support

For issues or questions:
- Check [Troubleshooting](#troubleshooting) section
- Review [MAPPINGS_GUIDE.md](MAPPINGS_GUIDE.md) for mapping details
- Review logs in `data/logs/`
- Check state file: `data/uploads_state.json`
- Contact: Dave Porter

---

## Version History

- **v2.0.0** (2026-01-22) - Mapping System & Score Reporting
  - Complete Ferrero ↔ CreativeX mapping system
  - Support for 2 brands: Nutella, Rafalleo
  - Support for 44+ social media channels
  - Detailed score reporting with guideline breakdown
  - Scorecard URLs and PDF report download
  - Mapping validation tools

- **v1.0.0** (2026-01-09) - Initial Release
  - Ferrero filename parsing (NEW format)
  - Upload to CreativeX API
  - Status checking and polling
  - State persistence with resume capability

---

## License

© 2026 Ferrero. All rights reserved.

---

## Quick Start Checklist

- [ ] Clone repository
- [ ] Create virtual environment (`python3 -m venv venv`)
- [ ] Activate environment (`source venv/bin/activate`)
- [ ] Install dependencies (`pip install -r requirements.txt`)
- [ ] Copy and configure `.env` file with API credentials
- [ ] Verify setup (`python get_dimensions.py`)
- [ ] Check supported brands/channels (`python scripts/validate_mappings.py --show-supported`)
- [ ] Test with dry run (`python scripts/upload.py --dry-run test_file.mp4`)
- [ ] Upload test file (`python scripts/upload.py test_file.mp4`)
- [ ] View scores (`python scripts/check_status.py --detailed`)
- [ ] Get scorecard URL (`python scripts/download_reports.py --all`)

---

## Production Readiness

### For Production Use:

1. **Update API Credentials**
   ```bash
   # In .env file:
   CREATIVEX_API_BASE_URL=https://api.creativex.com/api/v3
   CREATIVEX_ACCESS_TOKEN=your_production_token
   ```

2. **Query Production Dimensions**
   ```bash
   python get_dimensions.py > production_dimensions.txt
   cat production_dimensions.txt
   ```

3. **Verify Brand/Channel Mappings**
   ```bash
   python scripts/validate_mappings.py --show-supported
   ```

4. **Test Upload**
   ```bash
   python scripts/upload.py --dry-run test_file.mp4
   python scripts/upload.py test_file.mp4
   python scripts/check_status.py --detailed
   ```

5. **Update Mappings if Needed**
   - If brand names differ, update `mappings.json`
   - If channel structure differs, update channel mappings
   - Re-test after changes

---

**Ready to upload your first file?**

```bash
source venv/bin/activate

# Check what's supported
python scripts/validate_mappings.py --show-supported

# Test your filename
python scripts/validate_mappings.py --test-file /path/to/your/file.mp4

# Upload
python scripts/upload.py /path/to/your/file.mp4

# View results
python scripts/check_status.py --detailed
```

Good luck! 🚀