Folder-only mode (-N suffix files) was sending minimal metadata that DAM
rejected with "unmarshalling parameter" error. Now uses a reference
asset_representation_template.json as the base for all metadata fields,
ensuring the full field structure (column_name, data_type, domain_id, etc.)
the DAM API requires. Also fixes default/forced value handling to use
DomainValue format for domained fields from the template.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Folder-only mode (-N suffix files) was sending simplified metadata that
PROD DAM rejected with "unmarshalling parameter" error. Updated to use
DomainValue format for domained fields, correct asset type field ID
(FERRERO.FIELD.MKTG.ASSET TYPE), asset type code mapping (e.g. SND→sound),
validity dates, and forced values from config.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
A1→A2 now looks up the opentext_id in master_assets for an M-prefixed record
from B1→B2 and stores it as global_master_tracking_id on the local asset record.
This provides traceability from local campaign assets back to their global master
without changing any existing workflow logic or DAM metadata.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously, re-scored assets with a DAM timestamp suffix (e.g. _2026-03-13-05-53-36)
were treated as new files, leaving multiple 'active' records. Now strips the timestamp
and uses LIKE matching so all variants of the same base asset are properly superseded.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mailgun silently drops emails with multiple recipients in the to field.
Send individual API calls per recipient and split comma-separated addresses.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mailgun API is used when MAILGUN_API_KEY and MAILGUN_DOMAIN are set,
with SMTP as fallback for PPR. Also fixes A2→A3 batch subject line
that was rendering Jinja2 syntax literally instead of substituting values.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Was generating FERRERO.TABULAR.FIELD.MARKETING_TAG (underscore) but DAM
expects FERRERO.TABULAR.FIELD.MARKETING.TAG (dot). Added explicit mapping
for tabular field parent table IDs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
VIDEO_POST_PROD_COMPANY and AUDIO_POST_PROD_COMPANY are not domain fields
but were being wrapped with DomainValue, causing unmarshalling errors.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The _build_fields_from_filename method was using {"value": "..."} without
the required {"type": "string", "value": "..."} structure, causing
unmarshalling errors on the DAM API for -N suffix uploads.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fixes unmarshalling error on DAM upload when using -N suffix files. The API
requires the DomainValue structure when domain_value is true.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The filename_updates logic was only updating field['value'] (singular) but for
tabular fields like MAIN_LANGUAGES, the DAM reads from field['values'] (plural
array). This caused the master's original language (e.g. "Global") to persist
instead of the correct language from the filename (e.g. "PL").
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When no CreativeX score is found for a file, the system was sending a
generic placeholder URL (app.creativex.com/preflight/pretests) to the DAM.
Now sends no URL at all, so only files with actual CreativeX scores get a URL.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove PPR-only gates so PROD supports the same MASTERASSETIDS tabular
field and multi-master ID parsing as PPR. DAM deployment scheduled for
Feb 18 — do not push until then.
Changes:
- filename_parser: Remove is_ppr check, allow multi-master ID parsing in PROD
- a2_to_a3: Populate master_opentext_ids for single-master PROD case
- dam_client: Remove PPR-only skip on domain registration
- metadata_extractor_mvp: Update docstrings only
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The new CreativeX API format stores channel/publisher at the top level
of full_extraction_data instead of inside a data.ferrero_mapped_platforms
wrapper. Add fallback mapping so platforms are correctly populated for
DAM uploads.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
YouTube Ads was missing from the DAM-CX mappings CSV, causing empty
Platform > Rating fields for YouTube assets. Also adds a fallback that
derives the CreativeX platform from the filename social media code (e.g.
YTA -> YouTube) when the database has no mapped platforms.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Populates master_opentext_ids for single-master case so uploads use the
tabular FERRERO.MASTERASSETIDS field instead of the ARTESIA.FIELD.ASSET_ID
fallback. Reverts the workaround from 6517a4f now that the field is being
configured in PROD DAM.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CreativeX lookup now falls back to tracking ID search when filename match fails
(handles mismatched naming from CreativeX PDFs). strip_upload_components now
only removes job number and tracking ID, keeping social media codes (YTA, DV3,
etc.) in the clean filename. Updated SOCIAL_MEDIA_CODES from 4 to 39 codes
sourced from the Ferrero naming tool.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PROD DAM rejects FERRERO.MASTERASSETIDS as it only exists in PPR. Remove the
single-master-to-list conversion so PROD uses the existing single-ID field
(master_opentext_id) instead. Will be re-added when client configures the
tabular field in PROD.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The notifier variable was referenced inside process_box_file but never passed
as a parameter, causing NameError for any file hitting the Master Tracking ID
check. Also changed the check from case-insensitive (.upper().startswith('M'))
to case-sensitive (.startswith('M')) to avoid false positives on random tracking
IDs like mviSv5.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The PROD a2_to_a3 script referenced master_opentext_ids without defining it,
causing NameError for all file uploads. Brings in multi-master tracking ID
support from PPR: filename parser handles multiple IDs (PPR) or single ID
(PROD), metadata extractor supports MASTERASSETIDS tabular field.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
A1→A2 now handles re-processing when campaign is reset to A1 after adding new
master assets. Existing assets reuse tracking IDs and skip Box upload, new assets
are processed normally. Also includes PPR domain registration for multiple master
asset IDs in a2_to_a3 and dam_client.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace exposed database credentials and SQL commands in A1 permanently failed notification email with support contact information (optical@oliver.agency).
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Problem:
- Retry logic only triggered for empty folders (total_assets == 0)
- When "Master Assets" folder doesn't exist, error thrown BEFORE retry check
- Exception caught by outer try/except, sent old upload_failed template
- No database tracking, emails sent every 3 minutes indefinitely
Solution:
- Added retry logic to outer exception handler
- Detects folder/assets errors and applies same 3-attempt tracking
- Now handles both: (1) folder doesn't exist, (2) folder is empty
- Database tracking works for both scenarios
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Remove folder creation logic in get_or_create_subfolder_path() since DAM does not allow folder creation via API. When a subfolder doesn't exist, upload to the parent folder instead of attempting to create it (which was causing 120 second timeouts).
This resolves upload failures in PROD environment during A2→A3 workflow.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit includes critical updates for PPR deployment:
1. Environment-Specific Field Mappings:
- Created field_mappings_ppr.yaml with agency code "Oliver"
- Created field_mappings_prod.yaml with agency code "0000221659"
- Updated config_loader.py to auto-detect environment based on DAM URL
- Enables seamless deployment between PPR and PROD environments
2. Metadata Extractor Enhancements:
- Added MetadataTable extraction support for nested fields
- Enables extraction of "Type of Video & Static Right" multi-value field
- Added logic to apply defaults to existing but empty fields
- Fixed agency name display_value handling for domain fields
3. Default Values Added:
- VIDEO_POST_PROD_COMPANY: "Oliver Marketing Ltd"
- AUDIO_POST_PROD_COMPANY: "Oliver Marketing Ltd"
- PROD_COMPANY (Production House): "-"
These changes ensure:
- Correct agency codes per environment (PPR/PROD)
- Proper extraction of nested tabular fields
- Default values for empty production company fields
- Seamless deployment workflow
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Key Changes:
- Updated metadata_extractor_mvp.py to use SIMPLE structure for all tabular fields
- All tabular fields now use direct value objects (no MetadataTableFieldRow wrapper)
- MAIN_LANGUAGES, ASSETCOMPLIANCE, MARKETING_TAG, CREATIVEX all use SIMPLE structure
- Master Asset ID field updated to SIMPLE structure
- Date fields now use type 'string' instead of 'long'
- Matches DAM reference structure from asset_representation.json
Added Files:
- metadata_extractor_mvp_PROD.py: PROD-specific version with same SIMPLE structure
- Backup files for safety
- Analysis and comparison documentation
Environment:
- Tested and working in PPR environment (ppr.dam.ferrero.com)
- All tabular fields match DAM-supplied reference structure
- Successful uploads confirmed
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Critical Fix:
- extract_global_campaign_reference() now accepts campaign_id parameter
- Always sets local_campaign_id to current campaign as fallback
- Prevents NULL local_campaign_id when no Global Campaign Reference exists
Root Cause:
- Assets without Global Campaign Reference had NULL local_campaign_id
- Caused derivatives to be linked to wrong campaigns
- Same asset in multiple campaigns would share tracking IDs incorrectly
Impact:
- Every asset now has proper local_campaign_id
- Derivatives correctly linked to their source campaign
- Fixes issue where C000001177 assets were showing as C000002098
Changes:
- database.py: Added campaign_id parameter with fallback logic
- a1_to_a2_box_uploader.py: Pass campaign_number to function
- a5_to_a6_download.py: Pass campaign_number to function
Reverted master_asset_id changes per user feedback:
- tracking_id already links derivatives to masters
- No need for additional master_asset_id foreign key
- Only storing dam_asset_id for DAM asset tracking
Changes:
- Reverted get_master_asset() to not return database 'id'
- Updated store_derivative_asset() to only INSERT dam_asset_id
- Updated a2_to_a3_upload_polling.py to pass None for master_asset_id
- Removed master_asset_id from INSERT statement
Note: Migration script still needed for dam_asset_id column only
Email Template Fix:
- Fixed subject line syntax error in a2_to_a3_batch_complete template
- Removed Jinja2 control flow ({% if %}) from subject line
- Changed to simple expression-only format
- Fixes 'Failed to send email' error
Database Logging Fix:
- Updated get_master_asset() to return database primary key 'id'
- Updated store_derivative_asset() to actually store master_asset_id and dam_asset_id
- Updated a2_to_a3_upload_polling.py to pass master_asset['id'] instead of None
- Added migration script to add dam_asset_id column to derivative_assets table
- Fixes issue where derivatives weren't being linked to masters in database
- Enables proper lookups and tracking of uploaded derivatives
Impact:
- Email notifications will now send successfully
- Derivatives will be properly logged and linked to master assets
- Other tools can now find uploaded derivatives in database
