Commit graph

5 commits

Author SHA1 Message Date
DJP
ec372576e8 Make email notifications verbose with detailed asset lists
Email Template Enhancements:

1. A1→A2 Complete Email - Now Shows:
    Campaign name and number
    Asset count
    List of ALL processed assets with:
      - Asset name
      - Tracking ID
      - Box file ID
      - Box URL (clickable link)

2. A1→A2 Partial Email - Now Shows:
    Campaign details
    Total/successful/failed counts
    List of SUCCESSFUL assets with:
      - Asset name
      - Tracking ID
      - Box URL
    List of FAILED assets with:
      - Asset name
      - Specific error message
    Note about automatic retry

3. A2→A3 File Uploaded Email - Shows:
    Original filename (with Job# and Tracking ID)
    Clean filename (stripped)
    DAM Asset ID
    Tracking ID
    Note that file was deleted from Box

Benefits:
- Know exactly which assets succeeded/failed
- Can click Box URLs to verify files
- Can track specific errors per asset
- Don't need to check logs for details
- Full visibility into automation status

Example Partial Email:
━━━━━━━━━━━━━━━━━━━━━━━
Campaign Partially Processed
Campaign: KSURPRISE LOCAL (C000000123)
Total: 3 | Successful: 1 | Failed: 2

 Successfully Processed (1):
• asset1.mp4 (Tracking ID: ABC123)
  Box URL: https://app.box.com/file/123

 Failed Assets (2):
• asset2.mp4 (Error: Network timeout)
• asset3.mp4 (Error: Invalid metadata)
━━━━━━━━━━━━━━━━━━━━━━━

Also Updated DEPLOYMENT.md:
- Added Key Features section
- Documented log rotation (28 files, 10MB each)
- Documented Box file deletion
- Documented per-file email notifications

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-31 08:27:05 -04:00
DJP
65f2c9c68e Add Box file deletion, email notifications, and log rotation for A2→A3
Major Features Added:

1. Delete Files from Box After Upload
   - After successful DAM upload, delete file from Box
   - Prevents reprocessing same files
   - Keeps Box folder clean
   - Only deletes on success (keeps on failure for retry)

2. Email Notification for Each Upload
   - New template: a2_to_a3_file_uploaded
   - Sends email immediately after each successful upload
   - Includes: filename, clean filename, asset ID, tracking ID
   - Don't wait for "all done" - notify per file
   - Recipients: configured in .env (REPORT_EMAILS)

3. Log Rotation for Both Scripts
   - Uses RotatingFileHandler
   - Max file size: 10MB per log file
   - Backup count: 28 files (approximately 1 month)
   - Auto-rotates when log reaches 10MB
   - Keeps logs/a1_to_a2.log (current)
   - Backups: logs/a1_to_a2.log.1, .2, .3, etc.
   - Automatically deletes logs older than 28 rotations
   - Applied to both A1→A2 and A2→A3 scripts

Flow Changes:
A2→A3 now:
1. Poll Box folder
2. Find V2 files
3. Download from Box
4. Upload to DAM
5. Delete from Box  NEW
6. Send email notification  NEW
7. Store derivative record
8. Exit

Log Management:
- Active logs: ~10MB max
- Rotated backups: 28 files = ~280MB total
- Automatic cleanup (no manual intervention needed)
- 1 week of detailed logs + 3 weeks of backups

Database:
- Added dam_asset_id and upload_status columns to derivative_assets
- Fixed store_derivative_asset() to use existing schema columns

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 19:41:02 -04:00
DJP
e95988e2bc Fix Box folder naming - use campaign ID (C000000078) not asset ID
Issue: Box folders were named with hex asset ID instead of campaign ID
Example Wrong: 7e2f7c97b003f91f8b2a162b9f62ccab51586fa9_Local_adaptation_test_2
Example Correct: C000000078-Local_adaptation_test_2

Fixes:
1. Use campaign_number (C000000078) instead of campaign_id (hex) for Box folder
2. Change separator from underscore to dash (C000000078-Campaign_Name)

The confusion:
- campaign_id variable = DAM asset_id (hex string for API calls)
- campaign_number variable = actual campaign ID (C000000078)

Now Box folders will be named correctly: C000000078-Local_adaptation_test_2

🤖 Generated with Claude Code
2025-10-30 18:19:06 -04:00
DJP
40ebc56585 Change A1→A2 script to single-run mode (process one campaign and exit)
Changed from continuous loop to single-run mode:
- Processes ONLY the first A1 campaign found
- Exits after processing (success=0, failure=1)
- Cron will run every 5 minutes, naturally processing one at a time

Benefits:
 Controlled processing (one campaign at a time)
 Easy to test manually
 Predictable resource usage
 Failed campaigns retry on next run
 Can stop/start easily

How it works:
1. Cron triggers script every 5 minutes
2. Script finds A1 campaigns
3. Processes first one only
4. If success → Updates to A2, exits
5. If failure → Stays A1, exits
6. Next run processes next A1 (or retries failed)

Also fixed requirements.txt to use >= versions for Python 3.10+ compatibility

🤖 Generated with Claude Code
2025-10-30 17:01:45 -04:00
DJP
b4e004c822 Complete Python automation implementation - All components built
MAJOR MILESTONE: Complete Python automation system created!

Components Implemented:
 Box Client (box_client.py)
   - JWT authentication via boxsdk
   - Upload with tracking ID suffix
   - Download files
   - Campaign folder creation
   - Connection testing

 Database Client (database.py)
   - PostgreSQL connection pooling
   - generate_unique_tracking_id()
   - store_master_asset() with full_metadata JSONB
   - get_master_asset(tracking_id)
   - check_campaign_upload_complete() - ALL-DONE CHECK!
   - store_derivative_asset()
   - Connection testing

 Filename Parser (filename_parser.py)
   - V2 naming convention parser (ported from PHP)
   - parse_filename() - 10 components
   - strip_upload_components() - Remove Job# and Tracking ID
   - Strict validation with detailed errors

 Metadata Extractor MVP (metadata_extractor_mvp.py)
   - Extract 28 MVP fields from master
   - Update fields from V2 filename (Description, Language, State)
   - Add missing fields with defaults
   - Build asset representation for upload

 Notifier (notifier.py)
   - Mailgun email integration
   - Outgoing webhook sender
   - Email templates (success, error, partial, critical)
   - Configurable recipients

Main Scripts:
 A1→A2 Download (a1_to_a2_download.py)
   - Poll DAM every 5 minutes for A1 campaigns
   - Download all master assets
   - Upload to Box with tracking IDs
   - Store in DB with full metadata
   - ALL-DONE CHECK before status update
   - Update A1→A2 only if all assets successful
   - Send webhook with campaign ID/number
   - Email notifications

 A2→A3 Upload (a2_to_a3_upload.py)
   - Flask webhook receiver for Box uploads
   - Signature validation
   - Async task queue processing
   - Parse V2 filenames
   - Load master metadata
   - Extract MVP fields
   - Upload to DAM
   - ALL-DONE CHECK for campaign
   - Update A2→A3 when all assets uploaded
   - Send webhook notifications

 Test Connection Script (test_connection.py)
   - Verify DAM, Box, Database connectivity
   - Quick health check

 README.md
   - Complete setup guide
   - Usage instructions
   - Configuration examples
   - Troubleshooting

Key Features:
- Python 3.6+ compatible (server requirement)
- Virtual environment isolated
- Configuration-driven (YAML files)
- Easy field updates (no code changes)
- Environment switching (staging/production)
- Comprehensive error handling
- Email + webhook notifications
- Retry logic
- All-done checks before status updates
- Campaign webhook notifications

Ready for testing locally with Python 3.10!

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-30 16:49:14 -04:00