Commit graph

5 commits

Author SHA1 Message Date
DJP
8ee165a0c1 Add A5→A6 rework workflow automation (Python)
Completes all 4 Python automation scripts for Ferrero DAM workflows.

New Features:
- A5→A6 script for rework asset downloads
- Separate Box folder for revisions (349441822875)
- Folder naming with -Revisions suffix
- Smart tracking ID lookup/reuse for existing assets
- Email notifications for success and partial completion

Changes:
1. NEW: scripts/a5_to_a6_download.py
   - Downloads rework assets from campaigns with status A5
   - Uploads to Box Revisions folder with tracking IDs
   - Updates status A5→A6 when all assets succeed
   - No webhook (rework workflow)
   - Logs to logs/a5_to_a6.log

2. EDIT: shared/database.py
   - Added find_or_create_tracking_id() method
   - Searches by opentext_id + local_campaign_id
   - Reuses existing tracking IDs from A1→A2 workflow
   - Prevents duplicate entries for same asset/campaign

3. EDIT: shared/notifier.py
   - Added a5_to_a6_complete email template
   - Added a5_to_a6_partial email template
   - Shows "(Updated existing)" indicator for reused IDs

Tested:
✓ All connections working (DAM, Box, Database)
✓ Script executes correctly
✓ Log file created successfully
✓ Found 2 A5 campaigns in test

All 4 Python workflows now complete:
✓ A1→A2 (Master Assets)
✓ A2→A3 (Upload from Box)
✓ A5→A6 (Rework Assets)
✓ B1→B2 (Global Masters)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-04 13:36:35 -05:00
DJP
eebfc8f189 Add local_campaign_id extraction and storage for complete campaign tracking
Database Schema:
 Added local_campaign_id VARCHAR(50) column
 Stores the immediate campaign the asset belongs to (C000000551)

Enhanced Extraction:
 extract_global_campaign_reference() now returns 3 values:
   - global_master_campaign_id (C000000068)
   - global_master_folder_id (676f2bcde4c7...)
   - local_campaign_id (C000000551)

 Extracts FERRERO.FIELD.CAMPAIGN ID from same collection
 Only sets local_campaign_id if that collection has global reference
 Logs all three IDs when found

A1→A2 Script:
 Passes local_campaign_id to store_master_asset()
 Stores complete campaign relationship

Database Now Stores:
- tracking_id: ABC123 (unique 6-char)
- opentext_id: 0008a50... (DAM asset ID)
- local_campaign_id: C000000551 (immediate campaign)
- global_master_campaign_id: C000000068 (global master)
- global_master_folder_id: 676f2bcde4c7... (global folder)

Example Relationship:
- Asset downloaded from Local Campaign C000000551
- That campaign references Global Master C000000068
- All three IDs stored for complete traceability

Database schema now complete with full campaign relationship tracking!

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-04 10:31:19 -05:00
DJP
c5165db46a Add Global Campaign Reference extraction and storage for A1→A2 workflow
Database Schema:
 Added global_master_campaign_id VARCHAR(50) column
 Added global_master_folder_id VARCHAR(255) column
 Stores relationship between local campaigns and their global masters

Database Module:
 Added extract_global_campaign_reference() method
 Searches inherited_metadata_collections for L7+ - CAMPAIGN containers
 Extracts FERRERO.FIELD.GLOBAL CAMPAIGN REFERENCE field
 Extracts container_id as global_master_folder_id
 Returns dict with both IDs

A1→A2 Script:
 Calls db.extract_global_campaign_reference() for each asset
 Passes global_master_campaign_id to store_master_asset()
 Passes global_master_folder_id to store_master_asset()
 Logs when Global Campaign Reference found

Example Data Stored:
- Local Campaign C000000551 asset
- global_master_campaign_id: C000000068
- global_master_folder_id: 676f2bcde4c7bcf7ef783e97f7495069bf50b6bc

Usage:
This data enables tracking which Global Master a local asset came from.
Can query all local assets for a specific Global Master campaign.
Foundation for future cross-campaign features.

Based on EXTRACTION_GUIDE.md implementation pattern.

Note: B1→B2 workflow NOT updated (those ARE the global masters)

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-04 10:19:50 -05:00
DJP
8b576bb598 Add A2→A3 polling version and fix database to use existing columns
Created a2_to_a3_upload_polling.py:
- Polls Box folder (348526703108) instead of webhook
- Works locally (no need for public URL)
- Single-run mode (process one file and exit)
- Can be run via cron every 5 minutes

Why Polling Instead of Webhook:
- Webhooks require public URL (doesn't work on localhost)
- Polling works everywhere (local and server)
- Same functionality, different trigger mechanism

Database Fix:
- Don't create new columns (dam_asset_id, upload_status)
- Use existing schema: tracking_id, derivative_filename, file_extension, status
- Simplified store_derivative_asset() to use existing columns only
- Database now compatible with existing schema

Test Results - A2→A3 Polling:
 Polls Box folder 348526703108
 Finds V2 files with tracking IDs
 Downloads from Box
 Loads master metadata from PostgreSQL
 Builds 27 MVP fields
 Updates Description, State, Language from filename
 Uploads to DAM successfully (Asset ID: 214924)
 Stores derivative record
 Processes one file and exits

Both Scripts Working:
 A1→A2: Downloads from DAM → Box (folder 348304357505)
 A2→A3: Uploads from Box → DAM (folder 348526703108)

Cron Setup:
*/5 * * * * python scripts/a1_to_a2_download.py
*/5 * * * * python scripts/a2_to_a3_upload_polling.py

Complete automation ready for production!

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 19:21:13 -04:00
DJP
b4e004c822 Complete Python automation implementation - All components built
MAJOR MILESTONE: Complete Python automation system created!

Components Implemented:
 Box Client (box_client.py)
   - JWT authentication via boxsdk
   - Upload with tracking ID suffix
   - Download files
   - Campaign folder creation
   - Connection testing

 Database Client (database.py)
   - PostgreSQL connection pooling
   - generate_unique_tracking_id()
   - store_master_asset() with full_metadata JSONB
   - get_master_asset(tracking_id)
   - check_campaign_upload_complete() - ALL-DONE CHECK!
   - store_derivative_asset()
   - Connection testing

 Filename Parser (filename_parser.py)
   - V2 naming convention parser (ported from PHP)
   - parse_filename() - 10 components
   - strip_upload_components() - Remove Job# and Tracking ID
   - Strict validation with detailed errors

 Metadata Extractor MVP (metadata_extractor_mvp.py)
   - Extract 28 MVP fields from master
   - Update fields from V2 filename (Description, Language, State)
   - Add missing fields with defaults
   - Build asset representation for upload

 Notifier (notifier.py)
   - Mailgun email integration
   - Outgoing webhook sender
   - Email templates (success, error, partial, critical)
   - Configurable recipients

Main Scripts:
 A1→A2 Download (a1_to_a2_download.py)
   - Poll DAM every 5 minutes for A1 campaigns
   - Download all master assets
   - Upload to Box with tracking IDs
   - Store in DB with full metadata
   - ALL-DONE CHECK before status update
   - Update A1→A2 only if all assets successful
   - Send webhook with campaign ID/number
   - Email notifications

 A2→A3 Upload (a2_to_a3_upload.py)
   - Flask webhook receiver for Box uploads
   - Signature validation
   - Async task queue processing
   - Parse V2 filenames
   - Load master metadata
   - Extract MVP fields
   - Upload to DAM
   - ALL-DONE CHECK for campaign
   - Update A2→A3 when all assets uploaded
   - Send webhook notifications

 Test Connection Script (test_connection.py)
   - Verify DAM, Box, Database connectivity
   - Quick health check

 README.md
   - Complete setup guide
   - Usage instructions
   - Configuration examples
   - Troubleshooting

Key Features:
- Python 3.6+ compatible (server requirement)
- Virtual environment isolated
- Configuration-driven (YAML files)
- Easy field updates (no code changes)
- Environment switching (staging/production)
- Comprehensive error handling
- Email + webhook notifications
- Retry logic
- All-done checks before status updates
- Campaign webhook notifications

Ready for testing locally with Python 3.10!

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 16:49:14 -04:00