No description
Find a file
DJP b4e004c822 Complete Python automation implementation - All components built
MAJOR MILESTONE: Complete Python automation system created!

Components Implemented:
 Box Client (box_client.py)
   - JWT authentication via boxsdk
   - Upload with tracking ID suffix
   - Download files
   - Campaign folder creation
   - Connection testing

 Database Client (database.py)
   - PostgreSQL connection pooling
   - generate_unique_tracking_id()
   - store_master_asset() with full_metadata JSONB
   - get_master_asset(tracking_id)
   - check_campaign_upload_complete() - ALL-DONE CHECK!
   - store_derivative_asset()
   - Connection testing

 Filename Parser (filename_parser.py)
   - V2 naming convention parser (ported from PHP)
   - parse_filename() - 10 components
   - strip_upload_components() - Remove Job# and Tracking ID
   - Strict validation with detailed errors

 Metadata Extractor MVP (metadata_extractor_mvp.py)
   - Extract 28 MVP fields from master
   - Update fields from V2 filename (Description, Language, State)
   - Add missing fields with defaults
   - Build asset representation for upload

 Notifier (notifier.py)
   - Mailgun email integration
   - Outgoing webhook sender
   - Email templates (success, error, partial, critical)
   - Configurable recipients

Main Scripts:
 A1→A2 Download (a1_to_a2_download.py)
   - Poll DAM every 5 minutes for A1 campaigns
   - Download all master assets
   - Upload to Box with tracking IDs
   - Store in DB with full metadata
   - ALL-DONE CHECK before status update
   - Update A1→A2 only if all assets successful
   - Send webhook with campaign ID/number
   - Email notifications

 A2→A3 Upload (a2_to_a3_upload.py)
   - Flask webhook receiver for Box uploads
   - Signature validation
   - Async task queue processing
   - Parse V2 filenames
   - Load master metadata
   - Extract MVP fields
   - Upload to DAM
   - ALL-DONE CHECK for campaign
   - Update A2→A3 when all assets uploaded
   - Send webhook notifications

 Test Connection Script (test_connection.py)
   - Verify DAM, Box, Database connectivity
   - Quick health check

 README.md
   - Complete setup guide
   - Usage instructions
   - Configuration examples
   - Troubleshooting

Key Features:
- Python 3.6+ compatible (server requirement)
- Virtual environment isolated
- Configuration-driven (YAML files)
- Easy field updates (no code changes)
- Environment switching (staging/production)
- Comprehensive error handling
- Email + webhook notifications
- Retry logic
- All-done checks before status updates
- Campaign webhook notifications

Ready for testing locally with Python 3.10!

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 16:49:14 -04:00
Python-Version Complete Python automation implementation - All components built 2025-10-30 16:49:14 -04:00
src Add video dimensions to upload and disable CREATOR field 2025-10-30 15:50:36 -04:00
.gitignore Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
43984435_n1izyn3l_config.json Implement proper Box JWT authentication with RSA signing 2025-10-28 14:07:52 -04:00
Box-config.json Implement proper Box JWT authentication with RSA signing 2025-10-28 14:07:52 -04:00
campaign_response.json Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
clean_workflow.php Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
config.example.json Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
config.php Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
config_v3.php Fix status updates and enhance workflow with proper error handling 2025-10-23 16:43:09 -04:00
Content Scaling Flow V3.postman_collection.json Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
Content Scaling Flow.postman_collection_Oliver(New).json Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
Content Scaling Flow_Oliver.Postman_Collection.json Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
DAM_LOOKUPDOMAINS_RAW.json Add Upload from Box workflow - Phase 1 Complete 2025-10-29 15:47:30 -04:00
debug_assets.php Add debug tools: direct asset lookup and delete logging 2025-10-29 17:40:03 -04:00
debug_status.php Fix workflow V3: metadata display and download error handling 2025-10-17 15:13:01 -04:00
ECOMMERCE_ALLOWED_FIELDS.md Add Upload from Box workflow - Phase 1 Complete 2025-10-29 15:47:30 -04:00
fetch_lookupdomains.php Add Upload from Box workflow - Phase 1 Complete 2025-10-29 15:47:30 -04:00
index.php Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
PROJECT_STATUS.md Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
PROJECT_STATUS_2025-10-24.md Add comprehensive project status report for session preservation 2025-10-24 12:21:37 -04:00
PROJECT_STATUS_2025-10-28.md Add comprehensive session report for October 28, 2025 2025-10-28 17:49:31 -04:00
PROJECT_STATUS_2025-10-29.md Add PROJECT_STATUS_2025-10-29.md - Complete session documentation 2025-10-29 15:12:46 -04:00
PROJECT_STATUS_2025-10-30.md Add PROJECT_STATUS_2025-10-30.md - Complete session summary 2025-10-30 14:47:02 -04:00
PYTHON_AUTOMATION_PLAN.md Add comprehensive Python automation plan 2025-10-30 16:20:33 -04:00
QUESTIONS_FOR_DAM_IT_TEAM.md Add questions document for DAM IT team re: metadata inheritance 2025-10-30 09:38:04 -04:00
README.md Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
simple_test.php Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
test_asset_detail.php Add download buttons to test page - downloads confirmed working! 2025-10-17 16:35:16 -04:00
test_direct_download.php Add download buttons to test page - downloads confirmed working! 2025-10-17 16:35:16 -04:00
test_filename_parser.php Add Upload from Box workflow - Phase 1 Complete 2025-10-29 15:47:30 -04:00
test_runner.php Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
test_search.php Fix workflow V3: metadata display and download error handling 2025-10-17 15:13:01 -04:00
test_upload.jpg Match AssetUploader to working standalone script approach 2025-10-28 08:48:58 -04:00
test_upload_standalone.php Simplify standalone test - accept token as argument + show token in UI 2025-10-28 08:39:25 -04:00
UPLOAD_FROM_BOX_COMPLETE.md Fix asset name to use clean filename without OMG Job and Tracking ID 2025-10-29 16:36:04 -04:00
UPLOAD_FROM_BOX_PHASE2_COMPLETE.md Add Phase 2 completion documentation 2025-10-29 15:56:46 -04:00
UPLOAD_FROM_BOX_STATUS.md Add Upload from Box workflow - Phase 1 Complete 2025-10-29 15:47:30 -04:00
UPLOAD_TROUBLESHOOTING.md Re-enable master asset fetching + Upload troubleshooting document 2025-10-24 11:24:39 -04:00
workflow.php Initial commit: Ferrero OpenText Content Scaling Application 2025-10-17 14:29:20 -04:00
workflow_v3.php Add video dimensions to upload and disable CREATOR field 2025-10-30 15:50:36 -04:00

Ferrero OpenText Content Scaling Application

Status: FULLY WORKING - Complete workflow from campaign selection to asset download
Date: September 29, 2025

🎯 Overview

A PHP-based web application for Ferrero's OpenText DAM system that automates the content scaling workflow:

  1. Campaign Discovery - Find campaigns ready for content scaling
  2. Asset Identification - Locate Master and Final asset folders
  3. Asset Download - Download source files for processing
  4. Metadata Extraction - Access complete asset metadata

🚀 Quick Start

Prerequisites

  • MAMP or similar PHP server
  • Internet connection to Ferrero OpenText API

Setup

  1. Copy project to MAMP htdocs directory
  2. Ensure Creds.txt file is present with OAuth2 credentials
  3. Navigate to: http://localhost:8888/ferrero-opentext/clean_workflow.php

No Configuration Required

  • OAuth2 authentication auto-configured
  • API endpoints loaded from Postman collection
  • Credentials loaded automatically

📋 Complete Workflow

Step 1: Campaign Selection

  1. Click "Load '3 - Production & execution' Campaigns"
  2. Select desired campaign from the grid (look for campaigns with both Master and Final folders)
  3. Click "Proceed to Step 2"

Step 2: Asset Discovery & Download

  1. Click "Get Campaign Folders" to discover folder structure

  2. Two possible outcomes:

    • Multiple folders found - Campaign has proper Master/Final structure
    • ⚠️ Only Final Assets folder - Campaign needs different organization
  3. Explore folders to find assets:

    • "🔍 Explore All X Folders" - Compare all folders side by side
    • Individual folder exploration - Examine specific folders
  4. Download assets:

    • 📥 Download - Individual file download
    • 📥 Download All X Files - Bulk folder download
    • 📋 Metadata - View complete asset metadata

🔧 Technical Details

Authentication

  • OAuth2 Client Credentials Flow
  • Client ID: otds-OLV
  • Token URL: https://ppr.dam.ferrero.com/otdsws/oauth2/token
  • Automatic token refresh and expiration handling

API Endpoints Used

# Campaign Discovery
GET /v6/search/text?load_type=metadata&search_config_id=18&search_condition_list=...

# Folder Structure
GET /v6/folders/{campaign_id}/children?load_type=metadata&metadata_to_return=INER_NAME_GENERIC&level_of_detail=slim

# Asset Listing  
GET /v6/folders/{folder_id}/children

# Asset Download
GET /v6/assets/{asset_id}/contents

File Organization

downloads/
├── [folder_name]/          # Bulk downloads organized by folder
│   ├── asset1.jpg
│   └── asset2.jpg
└── individual_file.jpg     # Individual downloads

📊 Current Campaign Data

Available Campaigns

  • 12 campaigns in "3 - Production & execution" stage
  • Brands: KINDER SURPRISE, KINDER BUENO, KINDER CHOCOLATE, etc.
  • Markets: DE (Germany), FR (France), GB (UK), DZ (Algeria)

Verified Working Campaign

  • Name: "LOCAL CAMPAIGN"
  • Campaign ID: C000000212
  • Brand: KINDER SOFTY/BUENO EGGS
  • Folder Structure:
    LOCAL CAMPAIGN
    ├── 00. Master Assets (Empty - workflow organization)
    └── 01. Final Assets (Contains 2 JPEG source files)
    

Asset Metadata Available

For each asset, the system provides:

  • 📋 Basic Info: Asset ID, filename, file type, size, creation dates
  • 🖼️ Content Info: Dimensions, checksums, encoding, content URLs
  • 🏷️ Custom Fields: Campaign data, brand info, project details
  • 🔐 Permissions: Download, edit, export rights
  • 🎨 Renditions: Thumbnails, previews, different sizes

🐍 Python Migration Ready

Documented API Patterns

  • Authentication flow with OAuth2 client credentials
  • Request/response structures captured and tested
  • Asset download endpoints verified and working
  • Metadata extraction patterns established

Planned Python Script Structure

# Future implementation
python3 download-script.py --campaign C000000212
python3 download-script.py --campaign C000000212 --folder-type master
python3 download-script.py --list-campaigns --stage "3 - Production & execution"

Required Campaign Structure

For automated workflow, campaigns should have:

  • Master Assets folder with source files for processing
  • Final Assets folder (empty) for processed outputs
  • Proper metadata with campaign and brand information

⚠️ Current Limitations

Campaign Structure Variance

  • Most campaigns only have "01. Final Assets" folders
  • Source assets currently located in Final Assets folders (naming inconsistent)
  • Need properly configured campaigns with assets in Master Assets folders

Workflow Requirements

For optimal automation, need campaigns where:

  • Source files are in "00. Master Assets" folder
  • "01. Final Assets" folder is empty/ready for processed outputs
  • Multiple asset types for comprehensive testing

🔑 File Structure

Main Application

├── clean_workflow.php      # ✅ Main working interface
├── Creds.txt              # OAuth2 credentials (auto-loaded)
├── Content Scaling Flow.postman_collection_Oliver(New).json
├── src/
│   ├── TestRunner.php      # API orchestration
│   ├── CampaignFormatter.php # Response parsing
│   ├── AssetDownloader.php # Asset download functionality
│   ├── MetadataExtractor.php # Metadata parsing and display
│   ├── OAuth2Handler.php   # Authentication
│   └── ApiClient.php       # HTTP client
├── downloads/             # Downloaded assets storage
└── logs/                  # Operation logs

Supporting Files

├── workflow.php           # Alternative interface (more complex)
├── simple_test.php        # Single endpoint testing
├── index.php             # Full testing suite
└── PROJECT_STATUS.md     # Detailed project status

🎯 Usage Examples

Content Scaling Workflow

  1. Campaign Selection:

    • Load production campaigns
    • Identify campaigns with proper folder structure
    • Select campaign for processing
  2. Asset Discovery:

    • Explore Master Assets folder for source files
    • Verify Final Assets folder for output destination
    • Review asset metadata and permissions
  3. Asset Download:

    • Download individual source files for processing
    • Bulk download entire Master Assets folder
    • Organize files locally for content scaling operations
  4. Processing (External):

    • Process downloaded assets (scaling, adaptation, localization)
    • Prepare processed files for upload
  5. Upload (Future):

    • Upload processed assets to Final Assets folder
    • Update metadata as needed

🚀 MAMP Access

  • Main Interface: http://localhost:8888/ferrero-opentext/clean_workflow.php
  • Authentication: Automatic OAuth2 setup
  • Session Management: Maintains workflow state
  • Reset: Click "🔄 Reset Session" to start fresh

📝 Next Development Phase

Python Script Development

  • Command-line interface for automated workflow
  • Campaign-based processing using documented API patterns
  • Bulk operations for multiple campaigns
  • Integration ready once properly structured campaigns are available

Ready for production use with current campaign structure. Optimized for campaigns with proper Master/Final folder organization.